Re: [GNUnet-developers] gnunet-download --recursive

[Christian Grothoff]

Hi zPlus,

First of all, thanks for the report.

I've updated the gnunet-download man page, hopefully clarifying
the use of the -R option.

As for your publish/download report, when I try your directory
it does download properly from the URI

gnunet://fs/chk/SE1XEY4KFSQC8ASJV01X0P5Y83AWC84XPK977KN3B8Q5DFMKBQQNBE4CC4K9ZQD14TV3NG26PSE3AFYCAJNQDR4Q4V4N5GMPREV5K0G.E847ZS636HXQKFBNNN63FRWE9TXNYCP85FNAPS49JY0WFKXB0V12DB21M7AE5X86R66FX6A9K614PQSB6N0W3VDY7SWC5T36351N52G.2174

(but note I did this offline on a train, so the file is not on the
network).

So I cannot reproduce the download failure.  I was using a configuration
with pretty much everything on defaults, using a system that was
_offline_ at the time.


However, I could reproduce the crash on "-n", and I've fixed that issue
as well.


As for the ".gnd" file being mostly zeros, I also cannot reproduce that.
However, if the download is incomplete, it is perfectly normal for the
file to contain zeros in the meantime. So I'd blame that on the download
failure.  Just for your reference, I've included the hexdump of the ".gnd"
file below.


I hope this helps a bit,

Happy hacking!

Christian

$ hexdump my.gnd
0000000 4789 444e 0a0d 0a1a 0000 a200 0080 0200
0000010 0000 0500 0000 1001 da78 6063 6060 8064
0000020 5960 e620 6204 206e 8096 498a 8922 4015
0000030 38c5 9880 8f05 16d8 18a8 103b 81f3 4ac4
0000040 2b52 f44a 720b 3312 18f3 726c 9332 f3d2
0000050 f34a 4b52 8ad2 18ed 2b72 3253 f48b 9019
0000060 0be4 d2f2 8c19 2c0d 802a 5d18 c8d8 acd4
0000070 8802 8591 206d d946 0431 e697 7829 e625
0000080 1829 2a1b 1a18 9959 039a 8291 8191 7621
0000090 89c5 0505 9939 89c9 9925 79f9 10fa 5d31
00000a0 33a0 9352 f24b 2a8b 0019 121b f73a 6e67
00000b0 6e75 7465 2f3a 662f 2f73 6863 2f6b 314a
00000c0 4d32 4645 4546 4334 4848 4b4b 5347 4759
00000d0 3145 5446 5056 5857 5041 3257 3736 3156
00000e0 4335 5845 5a50 3648 5238 3330 5a4d 5434
00000f0 5942 475a 5245 544e 3638 3243 5630 504a
0000100 3447 4248 5947 4e31 4535 4343 5134 4757
0000110 4844 5257 5246 3953 5332 4e34 5243 414b
0000120 3659 4558 2e52 5943 5445 3748 4352 564e
0000130 5033 5333 5242 4e32 5347 5656 4b59 3447
0000140 5054 5644 3435 5750 5a48 5143 4a4a 344a
0000150 4e37 3634 4536 4658 5435 454b 4b37 5232
0000160 424a 4645 5831 5a57 3134 5737 4435 4534
0000170 4d48 3935 3845 394d 544e 4d39 4751 5453
0000180 5958 4135 3246 3636 4d39 4830 2e38 3738
0000190 0038 0000 d802 0080 0200 0000 0600 0000
00001a0 9304 da78 6063 5860 c0c7 c0c0 c0c4 9cc0
00001b0 a407 1939 4020 8802 8119 1258 c62a c40d
00001c0 50d6 3131 6620 8a81 4231 1731 d892 a816
00001d0 4818 1f1f ac54 2a00 01c6 5753 5a92 a251
00001e0 905f 9893 c799 9050 ce97 6460 566a c401
00001f0 480c 36c2 9939 e949 a579 a979 6925 76c5
0000200 190c c989 7ad9 9520 1a68 7333 d313 f553
0000210 48b1 1918 9a18 1aea e818 989a 1829 1819
0000220 195b 589b 9a19 a218 0f98 dc31 218e 34b8
0000230 c14f 312b c14f 50d8 d0c1 cac8 1cd4 8088
0000240 0c5a 10cd 3a2a fc03 79dc a4b9 9e40 f5e4
0000250 70f4 8209 9bf9 e403 6f95 09d5 416f 9821
0000260 ede4 c2ee bff0 ffbd 7ecc df90 3c0b 8b22
0000270 de81 0614 4661 5986 4073 ca01 e25e ebe9
0000280 7eca 499f 4386 418b b4ef 2f52 d350 4f7f
0000290 c717 8a90 6f5b 6e6f 12cc 9409 7b68 9f98
00002a0 d4a2 c115 b625 75a9 af41 e780 32d2 b2b1
00002b0 e62a 4a6d c44f 7874 aa22 c538 3e84 e9c8
00002c0 8b14 4926 3dbd 16e2 35d4 f53b ddee 6c8f
00002d0 44df 73a2 9d3e 65ff 74f3 b3ce 67e7 9f4e
00002e0 bc69 ebcc a3fc 763f 2b23 0393 a830 1519
00002f0 5818 181a 1804 9c80 bb37 ceea fe1c aa7f
0000300 9cf0 cfb7 8d27 eef3 dabb ceef 6fd0 f7de
0000310 73c4 aa99 e59a f619 0dfd 25a7 45fe f44d
0000320 36fc 1187 1aac 2b45 39d5 071c 80c4 cf64
0000330 09e5 33a1 8f3d 92ad df2f 6e6b 6fa4 f278
0000340 c5ea 8977 3de6 f7e6 3e7c 769b 4d0b 73be
0000350 962e 01f1 9a0f 2b01 6e56 bd67 3c5d b26d
0000360 b952 efe8 8d4a ee86 baef 8deb 0db4 de4f
0000370 634e 63a8 be8c f532 bde6 9273 5628 d4d1
0000380 ca23 2926 69a1 a693 cbdd 35be b7ef b9e5
0000390 4745 7e54 9d9f 279a 47b1 3f63 f29a 7903
00003a0 f7d9 ff58 389c d4af 776a ece2 ed56 9ff4
00003b0 2971 efc2 5e9f f5bb 7f7c d471 ab13 7e17
00003c0 b36a 3ed9 be68 eef7 b3a9 ff9d e2b9 29ed
00003d0 38cf ce71 c907 eb71 eb6a 8966 ea5b 3e1b
00003e0 2d9c 2b93 c075 b2e6 e2d0 9bf5 d672 74e8
00003f0 ccf9 d678 df7b 1ea8 ede8 3587 8d9f c21d
0000400 4d9f 4a08 749d 34b6 2fb5 ae62 02b3 13b2
0000410 1569 0a79 7a0c 0dca 13ae d319 4d23 a6c3
0000420 9aa5 346d a93d 59b5 7849 fddb fbb4 1551
0000430 e2d7 0576 cf54 0934 2be3 490a e6a9 3a97
0000440 5eaf 7d39 aabf cd5a 98ad 6d97 d307 e82f
0000450 9863 7654 7ec3 5cf2 37cf 3fa7 4e08 9507
0000460 9e39 7eae eb2e 129c 009a 489a 4e30 6e67
0000470 6e75 7465 2f3a 662f 2f73 6863 2f6b 4157
0000480 5645 4650 465a 4a44 5436 5154 3354 5042
0000490 3139 3243 5732 5a33 374b 5348 3654 4a48
00004a0 4657 4d32 5935 464e 4853 5248 4532 374e
00004b0 3847 4d32 3835 4337 3341 3848 394d 4e59
00004c0 5834 4454 4e56 4134 4851 5a34 4736 444d
00004d0 444a 3750 324b 3653 5143 474e 3857 4244
00004e0 3135 3344 2e47 3030 465a 4b50 4848 4656
00004f0 434a 3453 5a53 4845 5252 4e57 5a44 3441
0000500 5047 4d50 4345 4158 324d 5157 5241 4437
0000510 3230 4733 5252 3738 4d45 4154 4e52 334d
0000520 5a30 4138 5445 5630 3530 5751 5a59 5850
0000530 4335 5258 5644 3942 5144 3650 4452 3044
0000540 4858 4843 3644 3352 4344 3254 2e30 3837
0000550 3530 0038 0000 3701 0080 0200 0000 0c00
0000560 0000 a202 da78 9095 4ecd 3084 8514 663b
0000570 7f12 7126 aead 37ba 1fd0 30a0 ccc4 d9c4
0000580 71a9 f461 ea05 c14c 1866 2840 f332 3e5e
0000590 05a0 6231 6361 b3d2 77e8 bdcf b4f7 0000
00005a0 0002 6898 6add b54d e066 2d97 d847 8f55
00005b0 745d 52ec d4eb cdf0 7a46 7acf 326c 3ec2
00005c0 763b 75a2 b1de c7bc 79cc 23c7 3a6c 8ec2
00005d0 987e 7b12 f285 cb84 e414 ac69 06eb 3008
00005e0 9136 04e3 ab0f 4995 c7a9 5af6 7c7d c893
00005f0 38d7 52ad a2a1 0172 e43e 6ed6 468c dc9f
0000600 58f1 ffa0 8814 e74b f5c4 4799 2cff defb
0000610 00b4 55cf ef0a 0a79 021d 0d09 5f3d 481f
0000620 6131 c083 7898 7616 c22d 305e a909 3a1f
0000630 1af4 70e3 def8 26bb 23c8 e58f 0b7a ed99
0000640 bedb 7045 d133 b0b5 e163 3e00 0611 94df
0000650 43ca ea84 b6ba dcdb d6bb 676e 20c5 247d
0000660 a113 689d 15c3 8547 b3c8 aa52 38ac a8a0
0000670 cd10 2145 965e a142 14aa ca05 5ae5 8555
0000680 d940 fde6 0f30 6e38 efff b10b 8ee1 67ad
0000690 756e 656e 3a74 2f2f 7366 632f 6b68 312f
00006a0 4547 5056 4d41 3650 3358 4d52 3433 3057
00006b0 3341 3342 4538 4b38 5752 3559 4452 4b54
00006c0 5a54 374d 574a 444e 3136 4350 3936 5439
00006d0 504a 4241 4851 4a54 3541 4550 4157 464b
00006e0 5832 4351 364d 4743 435a 4d4b 5241 564a
00006f0 5150 3342 4154 5634 3038 5341 5154 4750
0000700 4d32 4131 5232 4b2e 3951 3235 3356 5341
0000710 304e 3147 5856 5357 375a 3759 3830 4b43
0000720 5758 3735 5947 3143 5750 3356 4d54 465a
0000730 424a 3433 3935 304b 4337 4b53 4246 3134
0000740 5a58 395a 3543 3456 4839 5037 4359 435a
0000750 4a54 4830 3948 5053 5732 4345 434b 3337
0000760 4753 5238 3148 4259 5136 5833 5253 312e
0000770 3231 3132 0037 0000 0401 0080 0200 0000
0000780 0800 0000 c601 da78 8c6d 4ac1 3103 8610
0000790 d8a7 b683 55e0 cdbd c151 eeed daec 1aed
00007a0 50b5 14a4 a2f7 5f54 6220 910c 586d d9bb
00007b0 df52 87cb 7c11 1319 429a 852c 647c 9be6
00007c0 067f 0e00 a000 6163 ae96 6fa5 f539 ccee
00007d0 e472 8ddd 2777 ee41 3bdb 1f57 bffb e9e3
00007e0 b905 ef1f bd0e ae77 dc0e f736 5deb 9b77
00007f0 6233 e26d e2ba 834a cc6c f052 5885 c502
0000800 c104 d6a0 b212 58a4 605b 7e6f d0cf a8ed
0000810 9705 7622 3055 adb8 abd4 add4 e616 19bd
0000820 a6c3 ec1a 5c64 269b 4594 1159 f0d1 891a
0000830 2c46 62bf e529 2eff 966d f0c7 6adc b92c
0000840 8cc6 5290 1c36 87d9 3d76 3ddf 3f79 4e9f
0000850 1e5e 0ce7 5be5 51f4 890b 1a2b 7824 36de
0000860 4b4a 287c 5867 4d3e b867 d3ca 4b8b 6cfc
0000870 a579 17cc 61de f451 0fe0 b3e7 265f
000087e


On 01/26/2016 11:43 AM, zPlus wrote:
> I am trying to publish a directory (see attachment; tarball for
> convenience).
>
> Publishing seems to work properly (I've tried many different
> combinations, except for pseudonyms)...
>
> $ gnunet-publish mydir/
>
> Publishing `[path]/mydir/hack.png' done.
> URI is
>
`gnunet://fs/chk/J12MEFFE4CHHKKGSYGE1FTVPWXAPW267V15CEXPZH68R03MZ4TBYZGERNT86C20VJPG4HBGY1N5ECC4QWGDHWRFRS92S4NCRKAY6XER.CYETH7RCNV3P3SBR2NGSVVYKG4TPDV54PWHZCQJJJ47N466EXF5TKE7K2RJBEF1XWZ417W5D4EHM59E8M9NT9MQGSTXY5AF2669M0H8.878'.
> Publishing `[path]/mydir/gnunet.png' done.
> URI is
>
`gnunet://fs/chk/1GEVPAMP6X3RM34W0A3B38E8KRWY5RDTKTZM7JWND61PC699TJPABQHTJA5PEWAKF2XQCM6CGZCKMARJVPQB3TA4V80ASTQPG2M1A2R.KQ952V3ASN0G1VXWSZ7Y708CKXW57GYC1PWV3TMZFJB3459K07CSKFB41XZZ9C5V49H7PYCZCTJ0HH9SP2WECKC73SG8RH1YB6Q3XSR.112217'.
> Publishing `[path]/mydir/vik.png' done.
> URI is
>
`gnunet://fs/chk/WAEVPFZFDJ6TTQT3BP91C22W3ZK7HST6HJWF2M5YNFSHHR2EN7G82M587CA3H8M9YN4XTDVN4AQH4Z6GMDJDP7K2S6CQNGW8DB51D3G.00ZFPKHHVFJCS4SZEHRRWNDZA4GPPMECXAM2WQAR7D023GRR87EMTARNM30Z8AET0V05QWYZPX5CXRDVB9DQP6RDD0XHCHD6R3DCT20.78058'.
> Publishing `[path]/mydir/' done.
> URI is
>
`gnunet://fs/chk/ZWVC4JZB3CRCGGR6ZMYS55AFTJT8SE33HNEPC6RBJ3YTX8YMS2FB59W63PNZEBCSRBDEX283S9W623A0YHM2A899ZAB89ZDWXQHNZ0R.RXJ5R7888622E97JVSWT8GWS29812NRCRKRWP0GFSMWJAZ5R1TPZS895PGZPZ0TFW1XTJQMYX6CRVVDAK1BNEZBWCAXFX7M5VFMFT4R.66265'.
>
>
> However I have problems downloading it. I've also tried many
> combinations of the command gnunet-download, but all fail. For example,
>
> $ gnunet-download -o "my.gnd" -R [GND URI]
>
> only downloads the .gnd file and hack.png, and then stops with a 0% bar
> to one of the other files.
>
> In addition, if I add a --no-network flag like this
>
> $ gnunet-download -n -o "my.gnd" -R [GND URI]
> $ gnunet-download --no-network -o "my.gnd" -R [GND URI]
>
> I get SEGFAULTs.
>
> I would also like to know if the published ".gnd" file is valid, because
> it contains a huge gap filled with NULL bytes and I don't know if this
> is a feature or a bug.
>
>
> I would really appreciate if somebody could look into this, and try
> publishing the directory. Thank you.
>
>
>
> _______________________________________________
> GNUnet-developers mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/gnunet-developers
>


On 01/24/2016 07:34 PM, zPlus wrote:
> I would like to request an improvement to the description of option "-R,
> --recursive" of the man page for gnunet-download.
> 
> The current description is:
>     download  directories  recursively  (and in parallel); note that the
> URI must belong to a GNUnet directory and that the filename given must
> end with a '/' -- otherwise,  only  the  file  corresponding  to  the
> URI  will  be downloaded.  Note that in addition to using '-R', you must
> also specify a filename ending in '.gnd' so that the code realizes  that
> the  top-level  file  is  a directory (since we have no meta data).
> 
> 
> From the point of view of a new GNUnet user (me), it is not immediately
> clear how I am supposed to choose the filename, if ending with a '/' or
> with a '.gnd'. This is rather confusing. After several attempts with
> various combinations, the only two that work are "dirname/.gnd" (that
> creates a directory with all files plus a ".gnd" inside it), and
> "dirname.gnd" (that creates a directory with all files inside plus a
> "dirname.gnd" outside it). I am still not sure what the correct approach
> is. I believe it will be easier to simply distinguish between
> "filename" (for non-directories files) and "filename/" (for
> directories), and adding -R will simply download recursively.
> 
> Or am I just plain wrong?
> 
> 
> 
> 
> 
> _______________________________________________
> GNUnet-developers mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/gnunet-developers
>

signature.asc
Description: OpenPGP digital signature