Re: printf-like output for gnunet-search

[madmurphy]

Hi Martin!

Absolutely nothing is wrong with “the look” of the current output (I have preserved it!). But what the current program cannot do is manipulating it.

The point of having a format field is that it allows to specify a lot of different input very easily and produce a corresponding amount of output. However, what actually triggered it was the fact that some information is not printed at all by the current program (e.g.: the plugin name) and the possibility to do advanced output manipulation for shell scripting in order to do advanced search (and I think that the official “flagship” GNUnet program for searching the network should be able to do that and beyond).

In the future it will be possible to add other ideas easily, but I would keep it the current proposal for a while first (if it is accepted), because further expansions require a bit of good planning – by the way, I have reviewed the code further (only minor things) – you can find it attached to this email…

Also if this code becomes stable enough, it will become part of GNUnet's code base it can be reused in the future by other GNUnet command line utilities that require printf-like capabilities…

--madmurphy

On Sun, Feb 6, 2022 at 3:23 PM Schanzenbach, Martin <mschanzenbach@posteo.de> wrote:

Hi!

If there is a use case for this kind of functionality this lgtm.
Altough, I do wonder what specifically triggered this. Is there anything wrong with the default output?

BR
Martin

> On 5. Feb 2022, at 09:09, madmurphy <madmurphy333@gmail.com> wrote:
>
> Okay, after thinking about it I did not like that the --verbose argument was ignored when a format was specified. But since, as it turns out, the --verbose argument was just a way to print all the metadata, I have added an argument for formatting the metadata too. So now we are even. In the meanwhile I have also renamed the new arguments and the format specifiers.
>
> Again, the help page will explain the new situation:
>
> $ gnunet-search --help
>
> gnunet-search [OPTIONS] KEYWORD
> Search GNUnet for files that were published on GNUnet
> Arguments mandatory for long options are also mandatory for short options.
>   -a, --anonymity=LEVEL      set the desired LEVEL of receiver-anonymity
>   -c, --config=FILENAME      use configuration file FILENAME
>   -F, --dir-printf=FORMAT    write the search results for directories
>                                according to FORMAT, where %f is the
>                                directory's name, %u is the directory's URI, %m
>                                is the directory's mime type (always equal to
>                                `application/gnunet-directory`), %n is the
>                                search result number and %a is the complete
>                                list of all the printable metadata available,
>                                in which each field is displayed according to
>                                the --prop-printf argument; if missing defaults
>                                to the --printf argument; if the latter is
>                                missing too defaults to `#%n:\ngnunet-download
>                                -o "%f" -R %u\n\n`
>   -f, --printf=FORMAT        write the search results according to FORMAT,
>                                where %f is the file's name, %u is the file's
>                                URI, %m is the file's mime type, %n is the
>                                search result number and %a is the complete
>                                list of all the printable metadata available,
>                                in which each field is displayed according to
>                                the --prop-printf argument; if missing defaults
>                                to `#%n:\ngnunet-download -o "%f" %u\n\n`
>   -h, --help                 print this help
>   -L, --log=LOGLEVEL         configure logging to use LOGLEVEL
>   -l, --logfile=FILENAME     configure logging to write logs to FILENAME
>   -N, --results=VALUE        automatically terminate search after VALUE
>                                results are found
>   -n, --no-network           only search the local peer (no P2P network
>                                search)
>   -o, --output=PREFIX        write search results to file starting with PREFIX
>   -p, --prop-printf=FORMAT   when the %a format specifier appears in --printf
>                                or --dir-printf, list each property according
>                                to FORMAT, where %p is the property's content,
>                                %l is the property's length in bytes, %t is the
>                                property type, %i is the property type's unique
>                                identifier and %w is the name of the plugin
>                                that provided the information; if missing
>                                defaults to `\t%t: %p\n`
>   -t, --timeout=DELAY        automatically terminate search after DELAY
>   -V, --verbose              be verbose
>   -v, --version              print the version number
> Report bugs to
> gnunet-developers@gnu.org
> .
> Home page:
> http://www.gnu.org/s/gnunet/
>
> General help using GNU software:
> http://www.gnu.org/gethelp/
> Now, besides the obvious question “Do you like the idea?”, I would like also to ask a few other questions too:
>
>       • What do you think about the fact that I have named the new arguments --printf, --dir-printf and --prop-printf? Do you think that alternative names would be better?
>       • What do you think about the fact that the format specifiers for --printf and --dir-printf use these letters? Do you think that other letters would be more obvious?
>               • %f – the file's name
>               • %u – the file's URI
>               • %m – the file's mime type
>               • %n – the search result number
>               • %a – the complete list of all the printable metadata available
>       • What do you think about the fact that the format specifiers for --prop-printf use these other letters?
>               • %p – the property's content
>               • %l – the property's length in bytes
>               • %t – the property type
>               • %i – the property type's unique identifier
>               • %w – where the information came from
>       • Do you think that the help page is clear enough?
>       • Do you like the idea?
> I really hope that my proposal does not sound intrusive and actually you like this idea as much as I do. Providing printf-like arguments gives infinite possibilities for shell scripting.
>
> Once again, please find attached a patch, or alternatively the single file that I have edited.
>
> P.S. Since it is a standalone program, if you want to play with this patched version without having to recompile the entire GNUnet, you can just launch:
>
> unzip gnunet-search.rev2.zip
> ln -s /usr/include/gnunet/gnunet_fs_service.h gnunet_fs_service.h
> ln -s /usr/include/gnunet/platform.h platform.h
> gcc -lgnunetutil -lgnunetfs -I/usr/include/gnunet -o 'gnunet-search-test' gnunet-search.c
> ./gnunet-search-test -f '%n. %f\n' 'commons'
>
> (if you follow these steps the program will be compiled without libextractor and the %t specifier in --prop-printf will not be available.)
>
> --madmurphy
>
>
> On Fri, Feb 4, 2022 at 6:21 PM madmurphy <madmurphy333@gmail.com> wrote:
> Hi GNUnet folks!
>
> I have edited the gnunet-search utility to accept a --format and a --dir-format parameters and produce a printf-like output. It was pretty easy to do, I only had to edit gnunet/src/fs/gnunet-search.c. The new text printed by gnunet-search --help can explain the two new arguments in detail:
>
> $ gnunet-search --help
>
> gnunet-search [OPTIONS] KEYWORD
> Search GNUnet for files that were published on GNUnet
> Arguments mandatory for long options are also mandatory for short options.
>   -a, --anonymity=LEVEL      set the desired LEVEL of receiver-anonymity
>   -c, --config=FILENAME      use configuration file FILENAME
>   -F, --dir-format=DIRFORMAT write search results for directories according to
>                                DIRFORMAT, where %n is the result number, %f is
>                                the file's name and %u is the file's URI; if
>                                missing defaults to FORMAT; if the latter is
>                                missing too defaults to '#%n:\ngnunet-download
>                                -o "%f" -R %u\n\n'
>   -f, --format=FORMAT        write search results according to FORMAT, where
>                                %n is the result number, %f is the file's name
>                                and %u is the file's URI; if missing defaults
>                                to '#%n:\ngnunet-download -o "%f" %u\n\n')
>   -h, --help                 print this help
>   -L, --log=LOGLEVEL         configure logging to use LOGLEVEL
>   -l, --logfile=FILENAME     configure logging to write logs to FILENAME
>   -N, --results=VALUE        automatically terminate search after VALUE
>                                results are found
>   -n, --no-network           only search the local peer (no P2P network
>                                search)
>   -o, --output=PREFIX        write search results to file starting with PREFIX
>   -t, --timeout=DELAY        automatically terminate search after DELAY
>   -V, --verbose              be verbose
>   -v, --version              print the version number
> Report bugs to
> gnunet-developers@gnu.org
> .
> Home page:
> http://www.gnu.org/s/gnunet/
>
> General help using GNU software:
> http://www.gnu.org/gethelp/
> Basically with this new patch launching
>
> gnunet-search commons
> and launching
>
> gnunet-search --format='#%n:\ngnunet-download -o "%f" %u\n\n' \
>       --dir-format='#%n:\ngnunet-download -o "%f" -R %u\n\n' \
>       commons
>
> are equivalent.
>
> What do you think? I hope you like the idea.
>
> I have done my best to stick to the coding style of the rest of the program. But a good code review is more than welcomed.
>
> Please find attached a patch, or alternatively the single file I have edited.
>
> --madmurphy
>
> <gnunet-search.patch2.zip><gnunet-search.rev2.zip>

gnunet-search.rev4.patch.zip
Description: Zip archive