Re: poly: Controlling an SI?

Nick writes:
>Some notes on various methods for controlling an SI...

You're really talking about controlling a super intelligence
who suddenly became much much smarter than us, and who is the
only such creature around (a scenario I find quite unlikely).
If there are many such creatures, they may control each other.
If they develop slowly, we can experiment with different
approaches and see what works.

>Controlling a superintelligence by trade
>Why pay for something you can get for free? If the superintelligence
>had the power to control all matter on earth, why would it keep such
>irksome inefficiencies as humans and their wasteful fabrics?

Now you've assumed that this intelligence is so "super" that we have
nothing to offer it that it might want. Even for a creature who is
as smart as the 10,000 smartest humans now on Earth, this seems unlikely.

Robin Hanson
hanson@econ.berkeley.edu http://hanson.berkeley.edu/
RWJF Health Policy Scholar, Sch. of Public Health 510-643-1884
140 Warren Hall, UC Berkeley, CA 94720-7360 FAX: 510-643-8614
Received on Fri May 15 16:41:52 1998