[potluck dish] crawling the Freenet WoT with sxml and match

[Arne Babenhauserheide]

Hi Guilers,

I’d like to contribute a small dish, I’ve been working on during the
past two weeks.

A first script downloads xml files of anonymous identities in Freenet,
extracts trust given to other identities and downloads these, too. A
second script parses the identities and turns them into a csv file which
social network researchers can use to investigate the effects of
anonymity on social interaction (at least that’s the goal). A third and
fourth script anonymize and deduplicate the results.

What I like best is using match and sxml to extract the relevant
identifiers, though I’m using mutation to separate the match in
the let-recursion and a final cons which saves the filename:


    (define (parse-trust-values filename)
      (let* ((port (open-input-file filename))
             (sxml (non-breaking-sxml-reader port))
             (closed (close-port port))
             (trust '()))
        (let extract-trust ((sxml sxml))
          (match sxml
            (('Trust ('@ ('Value value) ('Identity uri) rest ...))
             (set! trust
               (cons (cons (wot-uri-key uri)
                           (string->number value))
                     trust)))
            ((a b ...)
             (map extract-trust sxml))
            (else '())))
        (cons (wot-file-key filename) trust)))


I attached a network visualization generated from the data. Some
explanation is available from my website:
http://draketo.de/english/freenet/social-graph-snapshot

trust-sone-reingold.png
Description: Sone social graph in Reingold Layout

The code is available in the guile-freenet repository on
https://notabug.org/arnebab/guile-freenet and on Bitbucket
https://bitbucket.org/ArneBab/freenet-guile and from a static clone:
http://draketo.de/proj/guile-freenet/


Happy Hacking!
Arne Babenhauserheide

PS: I wrote this code without wisp to make it possible to run it with
    Guile out-of-the-box. To be suitable for programs I share, I’d need
    either guildhall integrated into Guile (to provide wisp with just a
    single command) or wisp itself in Guile.
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

signature.asc
Description: PGP signature