Re: poly: Modeling Economic Singularities

Robin Hanson writes:
> >Do you have any experimental evidence for *your* model?
>
> I think you're playing rhetorical games.

It may be rhetoric on certain levels, Robin, but it is not a game. I'm
making the point that what you're doing is not based in the methods of
science. I dislike being forced to do it in this manner, but you
aren't giving me much choice.

> Of course I don't have an experiment directly testing a model of
> future world growth rates. I haven't set up a planet in a situation
> like ours and seen what happens past the point where we are.

Well, as an honest experimentalist in another field who, say, claimed
to have a model that predicted earthquakes, one might hold off on
announcing for a couple of years to document one's claim based on
future events.

You could also, conceivably, have tried to apply your model
retrospectively to reasonably accurate data in order to see if the
model at least fits past behavior. Most theorists do this sort of
thing. Einstein noted that General Relativity retrospectively
explained the motion of Mercury, for example, before sending people
off on expensive expeditions to check for apparent star displacement
during eclipses.

This stuff has been done for centuries.

> How could you possibly be uncertain about this point? If you are
> willing to consider something else as a relevant experiment, then
> what exactly would you count as a relevant experiment for a model of
> future world interest rates?

In the long run, ALL theories are not "accepted", even Einstein's
physics. Everything has to be tested over and over again.

However, there are some preliminary tests that could have been done
here. As I noted, you could at the very least have tried to
demonstrate a fit to historical data, and presented verification
criteria based on future data. A statistical model could have given
us, for example, information on how long a period of time of future
run would be required for a particular percentage of certainty that
your model is accurate. You could ahve given such a model. People in
other fields are required to do so -- why are you exempt?

I hate to use vulgarity, but the claim that an economist can't be held
to the same standards as a medical researcher is pure bull. So it
might take five or twenty years to do a study -- big deal! Many
medical studies take twenty years. Five year studies are bloody well
routine. The scientists in question don't sit back and say "well, it
would take too long to determine if our model of heart disease is
correct, so we are going to just publish right now and pretend we are
exempt from the rules of science."

Many scientists are content to solidly demonstrate a few hard facts
even if they take decades to establish, over having oceans of fuzzy
unproven hypotheses.

> I'm also beginning to suspect that you really aren't familiar with the
> practice of science or engineering in any area which makes substantial
> use of mathematical theories.

Is this a discussion based on reason, Robin, or on ad hominems? I have
not made any claims about your personal knowledge, intelligence or
competance -- as such claims are irrelevant. I've simply asked you
for scientific evidence for your claims.

Were I in the mood to do so, I could easily present evidence on my own
background -- but I will not choose to do so. They have no place in
this discussion.

You are correct to note that I am using a few rhetorical devices here
and there in our discussion. This does not make them bad or even
ignorable. Just because I fully expect the answer to the question "do
you have experimental evidence backing up your position" to be "no"
does not mean that you are exempt from presenting such evidence.

I think you're a brilliant guy -- a really brilliant guy, probably
smarter than me. I also think that being brilliant isn't enough to
save one from requring evidence for one's opinions. A stupid man with
ample evidence backing his point beats a smart guy without evidence
any day of the week.

> The whole point of a theory is to let you extrapolate beyond the
> data which gives you confidence in it.

Sure, you are allowed to use a THEORY to exrapolate beyond your
current data. However, what we have here is a *hypothesis* -- a
potential theory. That is to say, you do not yet have a demonstrated
model -- you only have a model for which you have yet to gather
evidence of predictive power.

I understand the scientific method, the sequence is:

Collect data
Produce a hypothesis to try to explain the data which has predictive
power (i.e. is falsifiable based on future experiment).
Conduct experiments to try to falsify the hypothesis.
Revise the hypothesis if necessary.
Repeat forever.

For a few centuries, Newton held sway, but then once the data no
longer fit Newton, we got Einstein's theories. Presumably, a grand
unification of physics will result in further alterations to the rules
we think we know.

At this point, you have a hypothesis. You have not tested said
hypothesis. As such, it is not *yet* fit to be used as a basis for
prediction by people who care to have evidence for the predictions
they make decisions with.

> If you're always going to ignore theory outside the exact context of
> the data which supports it, because "lord knows what sort of small
> changes make a difference", then you might as well not have any
> theory at all. Just graph the data and extrapolate from there.

As I noted, for many years drug manufacturers didn't realize that
tests performed with a racemic mixture would produce highly different
results from those done with a monoisomeric drug. This resulted in
numerous human disasters, including the Thalidimide mess. (The
monoisomeric form of Thalidimide does not produce the results that the
racemic form produces.)

Simply noting that components of your hypothesis have some evidence
for them isn't sufficient. What if you completely ignored an important
factor? That is why people do EXPERIMENTS.

Now, as it turns out, I've spent quite a lot of time with people who
produce hypotheses about the future near-term price trends of various
securities and commodities. These individuals, in general, do not
attempt to risk money on their hypotheses until they have considerable
bodies of data backing them up. They collect time series data going
back a century or more, and carefully check their hypotheses against
historical data and make predictions and see whether or not these
predictions come true BEFORE risking capital on them. In general, even
when small legitimate effects are found, they vanish within a few
years as word of a new trick gets around and the influx of players
arbitrages away the advantage, but without careful experiment and
testing against data, even the few legitimate ideas that have shown up
over the years (like pairs arbitrage) could never have been
distinguished from the reams of plausible but utterly incorrect
hypotheses.

> >The "overall" approach isn't sufficient. If you have a mathematical
> >model, at the very least you have to try plugging in historical data
> >into the stated equations and see if it comes out with a reasonable
> >match. Even that isn't always enough.
>
> You specifically asked for experiments. If you are willing to look at
> attempts tests on historical data, I can point you to vast numbers of
> data analyses supporting the types of equations I used.

The "types of equations" are not a sufficient test. "We've given
people chemicals that were members of the phenylethylamine family
before and they displayed no toxicity, so we're assuming this is
nontoxic" would get you shot if you were a drug researcher. We need to
see your equations, tested against the real world.

Nothing less is sufficient. Anything else is navel gazing.

Perry
Received on Mon May 4 15:18:35 1998