Re: [Savannah-dev] [BACKEND] files parsing and related organisation

[Loic Dachary]

Guillaume Morin writes:
 > I've been thinking a lot on files parsing and the scripts organisations
 > that we are going to need.

        The general structure proposed looks very fine to me. In the
light of my previous mail regarding multi-domains, it could be questioned
wether the configuration file should deal only with one domain or many.
However, I don't see a contradiction. A configuration file could be 
multi-domain by construction, as long as it does not force backend
scripts to deal with multi-domains. For instance, your script is 
dealing with multiple domains which is good. If someone wants to 
write a backend script for another kind of installation and does not
care about multi-domain, she will be free to ignore the possibility
of multiple domains.

        It has another advantage : if many Savannah installations want
to use the same backend machine, the various configuration files from
these installations could be merged into a unified configuration file
on the backend machine. Say, for instance, that at some point there is
a savannah.gnu.org and freesoftware.fsf.org on two different machines (one
way or the other it'll happen next year, I guess). Then, fencepost.gnu.org
running your backend scripts could be configured by a file that is a merge
of the files from savannah.gnu.org and freesoftware.fsf.org.

        Does this make sense ?

 > I think that all data files must be XML to ease parsing in any language.
 > The structure that I suggest is <savannah> as root element and some
 > "role" children just as <configuration> or <dump>.

        I'm not sure about mixing dump + configuration. The two have
different purposes and writing a DTD (and when someone has enough fuel
and libraries to handles schema are available, a schema) will be hard
and unatural. Using namespaces seems better to me, if the intent is
to be able to mix the two for some reason (complete dump of the hosting
platform, for instance ?).

        Note that <dump> will eventually contain everything, not just
a small snapshot of the database for specific purposes as we now do.

 > Enforcing this kind of structure allows every part of the project to
 > share the same data files if they have too. For example, that means that 
 > if a parser finds a list dump and some DNS related configuration
 > information in the same file, it will not scream. I really think we
 > should avoid mixing non related data but enforcing this will be surely
 > be helpful in the future.

        Granted.

 > The point of all this is to allow sharing of functions and data by
 > scripts written for different purposes.
 > 
 > For the backend, we should choose a language. Perl seems to be the best
 > language for it since most contributors know it and it can do every job.
 > I will have to provide some Python modules for Mailman. But Perl should
 > be mandatory for every new common API contributions. Special cases must
 > be discussed on this list.

        I also vote for perl although I would not try to prevent
someone from using another language. I keep in mind that various
people will use these scripts for installations on their own
machines. As far as the savannah.gnu.org installation is concerned,
I think we will use perl because everybody here seems to like it.
Including me ;-)

 > Since every data file will share the same structure, I suggest to create
 > modules in Savannah::.
 > 
 > I plan to write :
 > 
 > Savannah::Common     that contains functions used by others modules
 >                                      such as controlling the well-formedness 
 > of a
 >                                      savannah configuration file.
 > 
 > Savannah::List               that contains a list related functions.

        This subdivision makes sense to me and is acceptable for CPAN.

 > To apply that idea to  the first XML dump of this message :
 > 
 > some functions of Savannah::Common will ensure that the file is well
 > formed (the root of element is <savannah>) and will return a pointer to
 > the <configuration> childs in that case.  Some bits of Savannah::List
 > will then do their parsing job on the lists elements...

        The DTD + validation will take care of this but the idea of
Savannah::Common is needed anyway.

 > This is a proposition. I am not very experienced with Perl modules and
 > XML so I could have made design mistakes.

        For a beginner you have a pretty accurate judgement on this ;-)

        Cheers,

-- 
Loic   Dachary         http://www.dachary.org/  address@hidden
24 av Secretan         http://www.senga.org/      address@hidden
75019    Paris         Tel: 33 1 42 45 09 16        address@hidden
        GPG Public Key: http://www.dachary.org/loic/gpg.txt