Re: Swarm and GIS libraries

[Dr. Paul Box]

> >>>>> "PB" == Paul Box <address@hidden> writes:
> 
> PB> My first attempts at bringing GIS data into swarm was using the
> PB> most loosely coupled way available.   
> 
> Have you had a chance to evaluate either Sven's extensions and
> replacements to Grid2d or Erik Nelson's Multi2d?  

[snip]+

When I made this first attempt, Sven nor Erik had put this code on the website 
yet.  When I resumed working with this idea here at USU, we started off 
thinking 
that either one of these extensions was an option.  However, while throwing the 
idea around, I realized that it would be really cool to have the pixels 
communicate with each other.  I then started trying to adapt CA2d to have the 
ability to have more than one value.  Without delving too deeply inside the 
code 
of either the Multi2d or the CA2d, we ended up with our own design of just 
having a discrete2d with an object on each grid cell.  The object, as a pixel, 
then has an arbitrary number of values, and functions that can be executed at 
each -step which allow it to exchange data or modify itself based on its 
neighbor's charactaristics.  I found it easier to do this than try to reuse 
some 
of the existing code.

As was stated in "the cathedral and the bazaar", a good programmer knows how to 
write, and a great programmer knows how to reuse code.  This probably shows 
that 
my instincts are not yet those of a great programmer.

> 
> PB> --The difficulty of having a grid2d have more than one value per
> PB> cell is problematic in this setup.  I have resolved this by having
> PB> each cell contain an object: the object can then have variables
> PB> representing the value of each data layer that overlays that
> PB> object.  This allows for any arbitrary number of data layers
> PB> (coverages) to be contained in a single grid2d.
> 
> I'd hope that one of the more general multi-2d spaces could do this
> for free..
> 

Again, my design is different enought that I was more comfortable coming up 
with 
a whole new class.  I could, probably, with some forthought, have reused one of 
their classes; my not using them may have been bad habits on my part, but I'm 
satisfied with the way I've taken now.

> PB> --The objects occupying the grid cells also contain functions for
> PB> interpreting the various datalayers they contain.  This allows
> PB> each cell to do it's own "overlay" functions (express values that
> PB> are recombinations of the various original data layers).  This
> PB> places the responsibility for this on the cell instead of the GIS
> PB> package.
> 
> Interesting. This certainly brings us to the issue of where the best
> place is for data manipulation/storage of "space" objects. The space
> libraries as currently implemented do little or no
> optimisation/compression of data at runtime - which I can foresee being
> a major issue with the kinds of data sets GIS people would be
> considering using. You don't want to be sloshing huge amounts of data
> in memory at runtime. 
> 

One reason for having this setup is that  some of the GIS-type functions, 
during 
the course of a simulation, you may only want to do on individual cells rather 
than an area-wide function.  In a GIS, typically you have a couple of layers 
that you wish to combine into a third layer (layer with trees overlayed with 
layer with slope aspect to find trees on north-facing slopes, for example).  In 
a raster GIS, this is typically done using simple math algebra (adding or 
subtracting values of one layer from values in the next layer, or dividing one 
layer by another).  When you do this in a raster GIS, you do the operation on 
the whole layer, unless you very specifically say that you only want the 
operation done on a small section.  This level of specificity in a GIS will 
probably mean your creating several intermediate files with variations of the 
same name, which you may or may not choose to delete once your operation is 
complete.  Significant mental energy must then be spent to keep up with this 
morass of interchange files, and any lapse in your data management will result 
in your line of thought drowning in the details.

By having the pixels each be responsible for their own operation, it is easy to 
have overlay-type operations happen on an as-needed basis (pixel by pixel).  
This would be useful if you had a soil moisture layer, for example, and in the 
course of your simulation you wanted to have a patchy rain shower fall on your 
area, adding moisture to the soil at uneven rates in different areas.   If an 
off-road vehicle were to cross the soil, and the level to which it compacted 
the 
soil were a function of the soil moisture, soil porosity, grain size, and 
weight 
of the vehicle, the pixel would consult itself for the value of the first three 
variables, ask the vehicle for its weight, and update its porosity based on how 
the vehicle compacted it.  The uneven rain shower could make a lot of 
difference 
on how much damage the vehicle actually does crossing the area.  It is possible 
to simulate such a scenario using traditional overlays etc. in a GIS, but the 
idea of having each pixel manage itself over the course of the simulation seems 
(to me) to be a lot cleaner and more attractive.


> This seems to be also an appropriate time to really think hard about
> how we could improve the implementation of spatial data in Swarm. Some
> of the current thinking around here has been to look at Swarm as a
> kind of "server" that could connect with a number of third party
> products. So this could mean that the Swarm "server" might (in some
> future implementation) be able to connect to a GIS database at the
> socket level. The other reason is to allow folks to write simulation
> in the language of choice and use something like ILU (Inter-Language
> Unification) to be able to "talk" to the Swarm server, which itself
> could be implemented in any of a number of language (e.g. Objective C
> or Java). These are all only possibilities at the moment.
> 
> From the GIS perspective, it seems to me that we should let any GIS
> package do what it does best and not re-implement what it does best in
> Swarm. The other issue is whether packages we interface with are
> themselves free software. It certainly easier and preferable to write
> interfaces to software that has freely accessible source code, rather
> than having to reverse-engineer the interface.
> 

I can hardly wait to hear of the better ways than the ones that I have been 
using.  I have certainly made use of Jim Westervelt's functions for UTMtoXY and 
XYtoUTM conversion, and I plan to make use of all the other functions reading a 
grass layer into a grid2d.  

I have been encountering resistance here regarding my decision to use grass, as 
almost everything here is done using Arc/INFO and Imagine.  However, I feel 
that 
it is not worth the effort to try and adapt swarm to a software package that 
keeps its internal details secret.  Arc/INFO, until recently, made it virtually 
impossible to access its data without using its own (extremely crappy) 
scripting 
language.  The latest release has many capabilities of accessing its data 
through a user's own C code, but I have not even looked at how one would use 
that to have swarm communicate with Arc/INFO.

The reality is that most of the GIS world uses Arc/INFO, and an interface from 
swarm to arc would capture a much wider audience than swarm to grass.  If I get 
enough people demanding an arc/swarm interface (and willing to pay for 
development ;-), then I would dive into the task with all the enthusiasm that 
glen has for porting swarm to NT.

 
> PB> --The agents, when running around on this surface, get placed into
> PB> pixel-objects instead of actual grid2d locations.  
> 
> Sounds interesting - I'm not quite sure how you do this in terms of
> implementation - could you give me an example?
> 

Normally, [putObject atX: Y:] will give the cell at X,Y a pointer to the object 
(agent) that is occupying it at that time step.  Since all cells are occupied 
with pixel objects in this implementation, then putObject is replaced with a 
function that calls getObjectAtX:Y: (calls the pixel object), and sets THAT 
object's pointer to the agent that is crossing over that space.  If you imagine 
the grid2d as a floor, covered by a carpet of pixel-objects, then the agent 
stands on top of the pixel-object instead of the floor itself.

 
> PB> 1- The current data model of the GIS is essentially top-down (a
> PB> series of layers overlayed on top of one another).  This seems
> PB> incompatible with swarm's bottom-up formulation of things.  I find
> PB> it much more interesting to model individual cells in the GIS, and
> PB> let the cell be responsible for itself.  Many landscape processes
> PB> can then come out as emergent processes.
> 
> You are right - this is a quite fundamental distinction between the
> GIS and Swarm way of "looking at the world" - is there a way of
> reconciling the two? Any ideas folks have out there?
> 
I figure that modeling the landscape from the pixel up is the best way to do 
this.  I especially look forward to discussing this with people who think the 
idea is off the wall.  I hope that many beers will be downed over this subject!
 

                  ==================================
   Swarm-GIS is for discussion of Geographical Information Systems (GIS)
   and their integration or use with Swarm.  For list administration
   needs (esp. [un]subscribing), please send a message to
   <address@hidden> with "help" in the body of the message.
                  ==================================