Re: poly: Dumbing down AIs (was: Modeling Economic Singularities

From: Robin Hanson <>
Date: Mon Apr 27 1998 - 09:27:01 PDT

Damien S. writes:
>> I don't see an important difference between killing and preventing from
>> living. And you did say what would be in their place: conscious AIs.
>You don't see an important difference betweeen murder and abortion? Or
>between killing an adult and contraception? I'm baffled.
>I see a huge difference. Killing is something done to me, or done to another
>citizen who could protest. If something is prevented from ever gaining
>consciousness, though, I don't see the problem.

As Nick noted, if we're talking about what laws we expect selfish people to
evolve, we should expect laws to distinguish between killing creatures with
power, and those without. Laws are negotiated between those with power, and
can easily fail to proscribe harm against those without power. So we expect laws
against killing regular adults, and against killing the children of regular
adults. But we might not expect laws against adults killing their own children,
or against killing powerless adults, unless other regular adults care enough
about these creatures.

However Nick raised the issue as a question of ethics, not law. The problem
ethically is not whether a rule might threaten you, Damien S., but how we
might would evaluate an outcome "behind a veil of ignorance", not knowing who
we in particular might be. Behind such a veil if I compare three situations:

1) a full life
2) being killed 1 month after being born
3) never having existed

I'd say that the difference in values I place between 1 & 2 is much greater
than between 2 & 3. And if I define

2') being killed 1 month before being born

I'd say the difference between 2 and 2' is even smaller.

Robin Hanson
RWJF Health Policy Scholar, Sch. of Public Health 510-643-1884
140 Warren Hall, UC Berkeley, CA 94720-7360 FAX: 510-643-8614
Received on Mon Apr 27 16:32:49 1998

This archive was generated by hypermail 2.1.8 : Tue Mar 07 2006 - 14:45:30 PST