Re: poly: Re: singleton self-interest

From: Nick Bostrom <>
Date: Fri May 01 1998 - 17:01:24 PDT

David Brin wrote:

> The argument over whether a singleton can plan for its own dissolution
> misses a point. Most volitional entities preserve their existence because
> to do so has become an imperative via Darwinian selection. (Because each
> one is a descendant of other entities who happened to care about self-
> preservation, the trait got reinforced.)

When we are designing brand new volitional enteties, these enteties
have not evolved via Darwinian selection, so we shouldn't suppose
that they will place a high value on self-preservation. What has
evolved via Darwinian selection, on the other hand, are their
creators. We should expect the robots to place a high value on
whatever their creators place a high value on -- which is the
preservation of the creators rather than the preservation of the

> Singletons must be designed
> from the start to evade this scenario, retaining volitional control over
> dissolution strictly on the basis of standards and goals. Once singletons
> (or robots) are allowed to self-reproduce, Darwin's logic sets in.

I don't know what it would mean for a singleton to self-reproduce (a
singleton is, so to speak, the whole global system rather than a
specific robot or business company). As for robots, I don't see any
reason why they should be made to self-reproduce. It would seem more
efficient to have other robots that specialize in
robot-manufacturing. Even if they do self-reproduce, Darwin's logic
will only lead to evolution if there is a significant chance of
mutation or recombination, neither of which seems very hard to avoid
in machine replication.

> The
> descendants of those who choose to live won't allow themselves to be
> dissolved.

Yes, *if* we wan't the singleton to eventually dissolve (and that's a
big if), then we would presumably want it to disolve in such a way
that its resources are split between its founders (which can have
continued to exist within the singleton) and their

Nick Bostrom
Department of Philosophy, Logic and Scientific Method
London School of Economics
Received on Fri May 1 23:22:19 1998

This archive was generated by hypermail 2.1.8 : Tue Mar 07 2006 - 14:45:30 PST