poly: Dumbing down AIs (was: Modeling Economic Singularities)

From: Robin Hanson <hanson@econ.berkeley.edu>
Date: Fri Apr 24 1998 - 09:36:42 PDT

Nick B. writes:
>However, might it be possible to limit the AI in such a way that it
>doesn have consciousness or count as a person (and hence doesn't have
>rights) while retaining most of the benefits? If the AI were
>domain-specific, and only did exactly what it was told, and did not
>have any ability for self-reflection or long-term planning, would
>that save us from having to give it person-status?

It might save you under some rulesy rights ethics, but a more
consequential ethics would call this a shame, perhaps even a crime.

Imagine that the U.S. still had slavery, and instead of abolishing
slavery, someone proposed that we genetically modify slave babies so
that they met your criteria. To me, this outcome is worse than slavery.
The lives of slaves are not as fun as those of free folk, to be sure,
but they are lives worth living to those slaves; they rarely commit
suicide, for example. In contrast, the creatures you describe are just
not there in some important sense. You've in effect killed the slaves,
but kept their functioning bodies, and called this an improvement.

People seem to get themselves tied up in knots thinking about the ethics
of choices which affect what creatures exist. See, for example,
"Critical-Level Utilitarianism and the Population-Ethics Dilemma",
at http://papers.ssrn.com/paper.taf?abstract_id=73511

To me it seems that if you switch from a world where a certain
creature doesn't exist, to a world where that creature exists and finds
their life worth living, that switch seems an improvment. Even if that
new creature is a slave. But others disagree, and for example seem to
prefer a world with the population and current average wealth of the U.S.
to a world where you also add the a population with the size and
average wealth level of today's rural India.

Robin Hanson
hanson@econ.berkeley.edu http://hanson.berkeley.edu/
RWJF Health Policy Scholar, Sch. of Public Health 510-643-1884
140 Warren Hall, UC Berkeley, CA 94720-7360 FAX: 510-643-8614
Received on Fri Apr 24 16:41:41 1998

This archive was generated by hypermail 2.1.8 : Tue Mar 07 2006 - 14:45:30 PST