Re: Swarm and GIS libraries

[Alex Lancaster]

>>>>> "PB" == Paul Box <address@hidden> writes:

PB> My first attempts at bringing GIS data into swarm was using the
PB> most loosely coupled way available.  I simply dumped the ascii
PB> values of a raster coverage into a file, and read the values into
PB> a grid2d and had agents wander around on top of that.  When it
PB> came time to have multiple coverages, I tried making a series of
PB> grid2ds, with the agent consulting each grid as it made its
PB> decision on where to move about.

Have you had a chance to evaluate either Sven's extensions and
replacements to Grid2d or Erik Nelson's Multi2d? I haven't had a
chance to play around with them myself. It would be great to get some
feedback on these packages - if folks on the mailing list have tried
either of them out in the context of their simulations - I'd certainly
love to know! 

You can find them at the usual place on the website:

 http://www.santafe.edu/projects/swarm/users/user-community.html

I'd also like to discover the usefulness or otherwise of them in the
context of this discussion - the code is there - it may be a matter of
or modifying or extending some of this code then we are already one
step ahead.

PB> This approach had two disadvantages.  1- representing a raster
PB> coverage as a series of integer or floating point values increases
PB> by orders of magnitude the size of the coverage (I had a 500k
PB> grass layer turn into a 17Meg ascii file!), and the way swarm is
PB> currently written, all of the values that are in the grid2d have
PB> to go into active memory (or swap). The second disadvantage,
PB> having a series of grid2ds for each layer, is that all my attempts
PB> at having each agent consult each relevant layer at each step
PB> produced very ugly and unsatisfying code.

This is certainly where a multiple occupancy space 2d library would
come in handy!

PB> I then tried the grass2d code that Jim Westervelt released (this
PB> was announced early on in this swarm-gis list).  This had the
PB> advantage of reading grass data layers directly, but the resulting
PB> surface that the agents run around on is still interpreted as a
PB> series of integer or float values on a grid2d; this again becomes
PB> very large and needs to live in the active memory of the computer
PB> running the simulation.  I currently see no easy way to get around
PB> this.

Yes - this does need some thought - I'd imagine we need to think about
how we might buffer or page some of this information - so not all the
integers or floats need to be in memory at all times.

PB> The code requires substantial revisions before it is available for
PB> release.  For the truly eager and curious, I will make what I have
PB> available to those who contact me privately.  The essentials of
PB> what I have been working on are as follows:

Please do contact me directly at <address@hidden>.

PB> --The difficulty of having a grid2d have more than one value per
PB> cell is problematic in this setup.  I have resolved this by having
PB> each cell contain an object: the object can then have variables
PB> representing the value of each data layer that overlays that
PB> object.  This allows for any arbitrary number of data layers
PB> (coverages) to be contained in a single grid2d.

I'd hope that one of the more general multi-2d spaces could do this
for free..

PB> --The objects occupying the grid cells also contain functions for
PB> interpreting the various datalayers they contain.  This allows
PB> each cell to do it's own "overlay" functions (express values that
PB> are recombinations of the various original data layers).  This
PB> places the responsibility for this on the cell instead of the GIS
PB> package.

Interesting. This certainly brings us to the issue of where the best
place is for data manipulation/storage of "space" objects. The space
libraries as currently implemented do little or no
optimisation/compression of data at runtime - which I can foresee being
a major issue with the kinds of data sets GIS people would be
considering using. You don't want to be sloshing huge amounts of data
in memory at runtime. 

This seems to be also an appropriate time to really think hard about
how we could improve the implementation of spatial data in Swarm. Some
of the current thinking around here has been to look at Swarm as a
kind of "server" that could connect with a number of third party
products. So this could mean that the Swarm "server" might (in some
future implementation) be able to connect to a GIS database at the
socket level. The other reason is to allow folks to write simulation
in the language of choice and use something like ILU (Inter-Language
Unification) to be able to "talk" to the Swarm server, which itself
could be implemented in any of a number of language (e.g. Objective C
or Java). These are all only possibilities at the moment.

>From the GIS perspective, it seems to me that we should let any GIS
package do what it does best and not re-implement what it does best in
Swarm. The other issue is whether packages we interface with are
themselves free software. It certainly easier and preferable to write
interfaces to software that has freely accessible source code, rather
than having to reverse-engineer the interface.

PB> --The objects also have functions for exchanging data with
PB> neighboring cells.  This allows the rasters to have CA-like
PB> functions.  I currently have a "nonsense" function that has grass
PB> from one cell encroaching into neighboring cells when the
PB> environmental conditions suffice.  I have every intent of applying
PB> this to real-world data.  This would be a nice tool for having
PB> diffusion processes, such as groundwater or fire, be simulated.

PB> --The agents, when running around on this surface, get placed into
PB> pixel-objects instead of actual grid2d locations.  

Sounds interesting - I'm not quite sure how you do this in terms of
implementation - could you give me an example?

PB> I will discuss this more as requested.  However, there are a couple
PB> of conclusions I have come to while working on this.

PB> 1- The current data model of the GIS is essentially top-down (a
PB> series of layers overlayed on top of one another).  This seems
PB> incompatible with swarm's bottom-up formulation of things.  I find
PB> it much more interesting to model individual cells in the GIS, and
PB> let the cell be responsible for itself.  Many landscape processes
PB> can then come out as emergent processes.

You are right - this is a quite fundamental distinction between the
GIS and Swarm way of "looking at the world" - is there a way of
reconciling the two? Any ideas folks have out there?


  --- Alex

-- 
  Alex Lancaster         |   e-mail: address@hidden
  Swarm Developer        |      web: http://www.santafe.edu/~alex
  Santa Fe Institute     |      tel: +1-(505) 984-8800 (ext 242)
------------------------------------------------------------------

                  ==================================
   Swarm-GIS is for discussion of Geographical Information Systems (GIS)
   and their integration or use with Swarm.  For list administration
   needs (esp. [un]subscribing), please send a message to
   <address@hidden> with "help" in the body of the message.
                  ==================================