Re: [GNUnet-developers] New GNUnet publication program

[Christian Grothoff]

Hi!

This is interesting, in part because I recently did a similar split into two 
steps for the new 0.9.x FS publishing API.  Now, what is not yet possible with 
that API is the serialization into a specification file for the end-user 
between 
the two stages, but that might be a possibility. The main problem I see is 
that the new API allows for "files" to be published that are not on disk (the 
caller would just have to specify a function for how to get the data); that 
kind-of conflicts with the ability to serialize to a human-readable file.  But 
by serializing to a human-readable file + archive with embedded files this can 
be supported sufficiently well.   So the idea is clearly worthwhile, I just 
think it should not be done with hand-hacked clients but within the FS 
library.

Also, I think (/hope) you're wrong about IBlocks not being deleted by gnunet-
unindex.  That'd certainly not be by design.  Finally, I wonder if the ability 
to delete KBlocks is really all that important since I suspect users won't 
keep the specification files around.  But maybe I'm wrong about that, and it is 
certainly a reasonable idea, especially if we integrate it with a way to 
trigger unindexing from tools like gnunet-auto-share (where code may control 
the generation and management of those specs).


My 2 cents

Christian

On Tuesday 01 September 2009 09:33:00 Kenneth Almquist wrote:
> I've written a variant of gnunet-insert, which I've named gnunet-pub.
> The key idea is to split the publishing operation into two steps:
> 1)  Determine precisely what to publish, and
> 2)  Actually publish it.
> 
> A given call to gnunet-pub can perform one or both of these steps.
> If you say
> 
>     gnunet-pub -o spec file...
> 
> gnunet-pub will write a publication specification to the file spec.
> You can then say
> 
>     gnunet-pub -i spec -P
> 
> where the -P option tells gnunet-pub to actually publish the files
> to GNUnet.  The command
> 
>     gnunet-pub -P file...
> 
> will perform both operations in one step.
> 
> This has a number of benefits.
> 
> The first is transparency.  The user can create a specification file
> and then see exactly what will be published, before performing the
> publication.  I feel that this is quite important because publishing
> is an irreversible operation.
> 
> The second benefit is that the front end (the part of gnunet-pub
> which generates the specification) can easily be replaced.  There
> is no need to use a programming language that can interface to C;
> any programming language that is capable of writing a text file
> can be used.  And testing is straightforward, because all you have
> to do is look at the text file generated by the front end to see
> if it is correct.
> 
> A third benefit is that, if the user retains the specification file
> and the published data files, the published content can be recreated.
> This makes it simple to re-publish content if the GNUnet server's
> data base ever crashes.
> 
> The ability to recreate published content has other possible uses.
> Currently, you can unindex a file, but the IBlocks and search records
> pointing to the file will still be in your data base.  With a
> spacification file, it would be possible to delete all of the
> published blocks using the DELETE message in the FS client-server
> protocol.
> 
> Conversely, you might want to be sure that the blocks you have
> published are still in the server data base, and haven't been pushed
> out by other content, or deleted due to software bugs or hardware
> glitches.  If the FS client-server protocol were enhanced to include
> a query to determine whether a block was in the data base, you could
> periodically check whether all of the data you published was still in
> the data base.
> 
> 
> To get the code, documentation, and test cases:
> 
> 1)  Go to a directory containing the GNUnet source, then cd to
>     src/applications/fs.
> 
> 2)  Download http://mysite.verizon.net/vzeefhzu/gnunet-pub.tar.bz2
> 
> 3)  Untar the file.  This will create src/applications/fs/gnunet-pub.
> 
> There are probably a number of issues with this code due to the lack
> of a complete design before coding.  On the other hand, I think that
> the set of test cases is large enough to ensure basic usability.  All
> feedback welcome.
> 
>  - Kenneth Almqist
> 
> 
> _________________________________________________________________
> Windows Live: Keep your friends up to date with what you do online.
> http://windowslive.com/Campaign/SocialNetworking?ocid=PID23285::T:WLMTAGL:O
> N:WL:en-US:SI_SB_online:082009
> 
> _______________________________________________
> GNUnet-developers mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/gnunet-developers
> 

-- 
http://grothoff.org/christian/