RE: Criticisms and defense of ABM

[Christopher J. Mackie]

Paul Johnson wrote:

> I think your advisors are correct in that the answer to the question
> "does my ABM match up well with reality" is hard to answer and often
> requires somewhat dubious comparisons of predictions against observed
> data, though.

I agree with Paul's assessment (the whole of it, not just the quote here),
and I would add two other points.  First, in social science--or any
discipline where the objects of study are volitional--the element of
volition makes it intrinsically difficult to match theory against empirical
reality: our billiard-balls don't always bounce at the same angles.  Perhaps
consequently, one of the consistent pathologies of social science is the
"Drunkard's Search Syndrome," the tendency to explore where the data are
easiest to obtain and understand, rather than where the questions are most
fundamental.  As one political-economist wit puts it, "I don't think about
bounded rationality because it's too hard...."

I don't see ABM as any worse than other current political science methods on
this dimension; on the contrary, it has some practical advantages.  ABM
allows the exploration and development of models which have at present no
closed-form solutions.  With proper attention to sensitivity analysis, ABM
can be a useful tool for developing theory *and* highlighting the kinds of
data needed to make the critical tests that Paul mentions.  In fact, since
ABMs usually have to make fewer draconian assumptions to achieve
tractability, they may be systematically easier to test empirically than
comparable closed-form models--at least, on the kinds of problems that
interest me.

Much of the objection to ABM and other 'open-formal' theoretical approaches
in my field strikes me as simple chauvinism on the part of closed-form
aficionados.  At least in political science, closed-form models are
notoriously difficult to test empirically, but you routinely find the same
people defending the (typically weak) empirical testing of closed-form
models and attacking the (perhaps equally weak, but not obviously weaker)
ABM approach.  The often-unspoken premise is that only closed-form solutions
actually give you "an answer" to a question.  That perspective strikes me as
parochial, at best.

Second, if one's goal is to influence policy, ABMs may be consistently
preferable to closed-form approaches.  This is true even when closed-form
solutions are available for a given policy problem, and it has to do with
human psychology.  Simulation results appear more 'real' to
non-mathematicians than do proofs--no matter how elegant.  A simulation
outcome that documents beneficial or adverse effects of a policy is going to
seem more credible to a policy community or the lay public than the most
exquisite deductive reasoning, and therefore be more effective as a policy
communication tool.  This is true ceteris paribus, but becomes even more so
to the extent that my first point is correct--that ABMs are easier to match
accurately to empirical data than are closed-form models.  The incorporation
of more 'realistic' assumptions into ABMs makes them even more effective as
policy-influence tools.

FYI, I'll be presenting these arguments, as secondary points, in the context
of an ABM of school choice plans at the Midwest PoliSci meetings next month,
and an ABM of political learning from media and social-network discourse at
the big annual PoliSci meeting on Labor Day.  The papers aren't done yet,
but if anyone's interested, I'll be happy to add you to the distribution
list.  And I would welcome responses--especially challenges--to the core of
the argument now, in anticipation of a (more hostile :-) audience at the
actual presentations.

Best,  --Chris
=======================
Christopher Mackie
Woodrow Wilson School of Public and International Affairs,
Department of Politics, and
Center for the Study of the Brain, Mind, and Behavior
Princeton University
address@hidden





                  ==================================
   Swarm-Modelling is for discussion of Simulation and Modelling techniques
   esp. using Swarm.  For list administration needs (esp. [un]subscribing),
   please send a message to <address@hidden> with "help" in the
   body of the message.
                  ==================================