Re: poly: Dumbing down AIs (was: Modeling Economic Singularities

From: Nick Bostrom <bostrom@ndirect.co.uk>
Date: Fri Apr 24 1998 - 18:07:50 PDT

Robin Hanson wrote:

> Nick B. writes:
> >However, might it be possible to limit the AI in such a way that it
> >doesn have consciousness or count as a person (and hence doesn't have
> >rights) while retaining most of the benefits? If the AI were
> >domain-specific, and only did exactly what it was told, and did not
> >have any ability for self-reflection or long-term planning, would
> >that save us from having to give it person-status?
>
> It might save you under some rulesy rights ethics, but a more
> consequential ethics would call this a shame, perhaps even a crime.

Note that I was't proposing this, only asking about it's technical
feasibility.

> Imagine that the U.S. still had slavery, and instead of abolishing
> slavery, someone proposed that we genetically modify slave babies so
> that they met your criteria.

The relevant analogy seems rather to be if somebody proposed to
genetically modify zygotes, not babies. I don't think it is ethically
wrong to modify human zygotes in such a way that they grow up to be
brainless organ-banks.

> To me, this outcome is worse than slavery.
> The lives of slaves are not as fun as those of free folk, to be sure,
> but they are lives worth living to those slaves; they rarely commit
> suicide, for example. In contrast, the creatures you describe are just
> not there in some important sense. You've in effect killed the slaves,
> but kept their functioning bodies, and called this an improvement.

You haven't killed any slaves; you've prevented them from being born.
Whether this is an improvement depends on what you put in their
place.

> To me it seems that if you switch from a world where a certain
> creature doesn't exist, to a world where that creature exists and finds
> their life worth living, that switch seems an improvment.

I agree. In fact, I'm happy that there are so many people on our
planet for the simple reason that I think human lives tend to be
valuable. Of course, there might be good practical reasons for
limiting population growth, but in general I would say: the more, the
better (even if it would somewhat lower the average quality of life).

_____________________________________________________
Nick Bostrom
Department of Philosophy, Logic and Scientific Method
London School of Economics
n.bostrom@lse.ac.uk
http://www.hedweb.com/nickb
Received on Sat Apr 25 00:13:51 1998

This archive was generated by hypermail 2.1.8 : Tue Mar 07 2006 - 14:45:30 PST