----- Original Message -----
From: "Paul Charteris" <address@hidden>
To: <address@hidden>
Sent: Tuesday, March 05, 2002 1:55 PM
Subject: Criticisms and defense of ABM
> Dear Swarm Modelling Group
>
> Criticism 1. One of my dissertation objectives is stated in this way:
> "To explore the utility of Agent-Based Models in describing the
> industry-level outcome of breeding decisions made by individual
> enterprises" As a note, these breeding technologies are animal genetic
> technologies that allow farmers to make more accurate genetic selection
> decisions.
>
> Their view - The aim of assessing whether a methodology is appropriate
> to a problem is not a suitable research objective in its own right, i.e.
> the study should not undertake simulation for the sake of determining if
> simulation works.
>
> My view - Sometimes it is a valid objective to examine the methodology
> rather than just the outcomes of the methodology, especially to test if
> the methodology provides new insights into the system or opens up new
> hypotheses not previously testable.
First, always remember that your committee is always right, even when they
are wrong. Given that, assessing whether a model is appropriate to a
particular situation is critical to correct modeling and is excellent
science. Not doing this task can also end up with very poor science. Case
in point from the first chapter of my dissertation:
A researcher decides to demonstrate that ODE based models are still directly
applicable to natural systems. He fits 11 different ODE based models to a
simple experimental predator-prey system. He orders them by the R^2 value
of the fit and decides that the ODE with the best R^2 value represents the
true dynamics of the system. What he didn't do is test the implicit
assumptions of his model against his results. Two important implicit
assumptions in ODEs are 1.) space is not a factor and that all individuals
are identical and, 2.) There are enough individuals in the system that
demographic stochasticity has no measurable effect on the system dynamics.
(Demographic stochasticity is randomness introduced due to populations sizes
changing in discrete units and randomness in birth and death rates.)
Anyway, bottom line, I didn't believe his results. I added demographic
stochasticity to his "best" model and found that while the experimental
system persisted for 12 days before extinction, his model persisted for at
least 120 days. In other words, the model he believed correctly represented
the "natural" system was in fact far to stable. (There is a bunch of other
stuff as well but it is not germane to this point. This research is being
finalized now and will be submitted to Ecology.)
The first task of any modeler is to assess the tools at hand and determine
which are suitable for the job. This is as much science as running the
model and analyzing the results. Note that this must be done because at the
start of the project we don't know the results. (We are actually just
finishing a book chapter on using multiple model classes to analyze a single
system. A whole chapter and I guess we have no research in it. I'll have
to tell my co-authors.)
> Criticism 2 - All agent-based modeling is essentially prediction.
>
> Their view - any outcomes of an agent-based model are a prediction of
> entities or processes in the system. Prediction can be thought of as any
> declaration or estimate regarding the future.
>
> My view - No the aim of agent-based modeling is frequently not a
> declaration or estimate of the future but frequently relates to
> observing the processes that lead to some future outcome. They argue
> that observing these processes is prediction in its own right. I
> counter-argued that we do not make a declaration or estimate of these
> processes at all- rather we simply parameterize the agents in their own
> model world and hope some "goodies" come out of the model. I guess the
> point is subtle, we (the modeler) do not try and anticipate the future -
> we only parameterize the present and observe what possible futures may
> exist. Maybe this is an estimate of the future after all?
If all agent based modeling is prediction then all modeling must also be
prediction. They do the same thing, it's just a matter of the complexity of
the model. This is really a "tastes great - less filling" argument. In
other words, it is just semantics. You can probably argue that all modeling
is in some very large sense prediction. However, at a lower level this is
certainly not true. For example, you might have a natural system under
study where you already have an outcome and believe that you know the
mechanisms that created the outcome. You therefore create a model that
incorporates those mechanisms and then run the model to see whether the
model replicates the results seen in the natural system. If it doesn't, you
reject the hypothesis that the dynamics of the natural system are caused
solely by the mechanisims in the model. Of course, if they do match, you
can't actually claim that your model is corect, it could be that you have a
different set of mechanisims that happen to produce the same results for
that paramenter space. Anyway, I would call this hypothesis testing, NOT
prediction.
Bottom line, I agree with you in both cases. But then again, I'm not the
one signing your dissertation, so my opinion has little real world weight.
You will find that science is very conservative and at times you may just
have to give in to the gray beards and just agree with them and then just go
ahead and do your thing. Thsi really depends on the atmosphere in your
department. I was very lucky at UCSB as I was encouraged to have an opinion
that differed for my committee's. I have also been told that had I been at
certain other places that this would not have been the case.
> Any thoughts or comments on these points of view would be most welcome.
>
> Cheers, Paul Charteris
>
> ---------------------------------------------------------
> The Department of Animal Sciences
> Fort Collins CO 80523-1171
> USA
> Ph. +1 970 491 5785
> Fax. +1 970 491 5326
> Personal homepage: http://ansci.colostate.edu/dep/people/grads/plc/
> ---------------------------------------------------------
>
>
>
> ==================================
> Swarm-Modelling is for discussion of Simulation and Modelling
techniques
> esp. using Swarm. For list administration needs (esp.
[un]subscribing),
> please send a message to <address@hidden> with "help" in the
> body of the message.
> ==================================
==================================
Swarm-Modelling is for discussion of Simulation and Modelling techniques
esp. using Swarm. For list administration needs (esp. [un]subscribing),
please send a message to <address@hidden> with "help" in the
body of the message.
==================================