Re: Does gnunet-download ever contact gnunet-service-fs?

[Cy]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

> So IF you did not delete the result of a
> previous download, it can indeed complete without any network or IPC
> activity.

I made sure to delete the result of a previous download, specifically because 
of that
possibility. I just checked again, just to be sure. No files already 
downloaded. I start
the download. gnunet-service-fs never leaves its select function. The download 
freezes for
exactly one minute, then completes. It reports no error, and says it's 
completed, but only
14% of the data ever downloads.

I put my break point right after GNUNET_NETWORK_socket_select on line 2368 in 
scheduler.c,
so it would break there upon any socket activity, right?

My best guess is that gnunet-download somehow checks... some local cache, or 
that it
somehow detects where the indexed files are and uses those directly. Or it just 
silently
fails, leaving a corrupted half-directory behind.

> Maybe it takes exactly one minute to compute the checksum on the
> directory structure you are downloading?

That would make sense, if the directory were several dozen gigabytes large, but 
it's only
18 megabytes. Plus I've downloaded larger files successfully before with no 
stalling out. 

The freeze only happens in the particular situation that I'm publishing and/or 
downloading
a directory with a bunch of subdirectories, each with files in them. "bunch" 
being a
quantity I'm not quite sure the value of. Somewhere around 40. Also the files 
in the sub-
subdirectories have to be slightly large, around 80 kilobytes. The contents of 
the files
doesn't seem to matter. More deeply nested files don't seem to make the problem 
worse, at
least. This is the test data I've been using:

gnunet-download -o testdata.tar.gz
gnunet://fs/chk/WDTGJQT000JYDB98QNJQ8D6FGTV0Y7C0R5C04Y24V4HH8TA3HSMGMGXT757QYCNH240S80630E
MT01TS6T19T9QMXGMYQ4Y3FJJR5SG.KZ2BWB1QZEYFYMYFK4WHR99MX6MESK1D20RSH5RP32KB1B430A2JRPD866AD
FAT1YKFAG3YAJ3W4D14K6N8AV8T9Q1PHXXNDT6PDEER.35291

It should download with minimal delay and no freezing, since it's just a single 
file. But
if you untar it to the 18 megabyte directory, and try to publish that, then 
that's when
gnunet-download fails for me.

Interestingly when I download like

$ function doit() {
  rm -rf fail fail.gnd
  time gnunet-download -V -V -V -R -o fail.gnd
gnunet://fs/chk/AJRGAG82WAENCXR4QPWQ7HCR3PQPFBK6J8BRBZV2TPAVKX1Z3631B7T27D2TGQQPCA61H7M4QR
8RG92EVDZZCG3FXYR9NH3ZYW15HW8.AQ6D1QZ9EN6A7M1ZWCWFF88ZP3KBPCNDFJAS3NBWETEP27SVQNDSVKFB4S90
28ETXCECTQ2PZP5XCW2EVVVTGPPSW2173GZG2SZR4E8.28249
}

Then

$ doit

I get output like this:

Starting download `fail.gnd'.
Downloading `fail.gnd' at 28249/28249 (0 ms remaining, 93 b/s). Block took 0 ms 
to
download
Starting download `fail/derproz.gnd'.
Downloading `fail/derproz.gnd' at 10423/10423 (0 ms remaining, 5 KiB/s). Block 
took 0 ms
to download
Starting download `fail/derpykdb.gnd'.
Downloading `fail/derpykdb.gnd' at 9118/9118 (0 ms remaining, 3586 b/s). Block 
took 0 ms
to download
Starting download `fail/derph.gnd'.
Downloading `fail/derph.gnd' at 7844/7844 (0 ms remaining, 2526 b/s). Block 
took 0 ms to
download
Starting download `fail/derpvy.gnd'.
...

But when I do the same command with a timestamp like

$ doit |& s6-tai64n | tee dl.log

I get this:

@400000005ec10a9d15c25b47 Starting download `fail.gnd'.
@400000005ec10a9d15ee3154 Starting download `fail/derproz.gnd'.
@400000005ec10a9d15f8c502 Starting download `fail/derpykdb.gnd'.
@400000005ec10a9d16022822 Starting download `fail/derph.gnd'.
@400000005ec10a9d160a9979 Starting download `fail/derpvy.gnd'.
@400000005ec10a9d1611b649 Starting download `fail/derppjg.gnd'.
@400000005ec10a9d1618768b Starting download `fail/derpmpgd.gnd'.
@400000005ec10a9d161ebf79 Starting download `fail/derpdnuvj.gnd'.
@400000005ec10a9d162466ee Starting download `fail/derprrjr.gnd'.
...

But I assume that's just a failure on the part of synchronizing stdout and 
stderr, so
probably not relevant.
-----BEGIN PGP SIGNATURE-----

iHUEAREIAB0WIQQ+4EYCR2lOGPl0mIHk9gahCsfaVgUCXsEN+gAKCRDk9gahCsfa
VkPXAP4qNK2A6xRrBdoBdgMYHIxNEzWNRKx8bEbqWn8gbXYMvwD/bBzvVBqulWj3
QUx6Y53j2lpqXptBD5waRATVnQTREOI=
=hjZ0
-----END PGP SIGNATURE-----