Re: [Duplicity-talk] Management of the temporary storage

[Pascal Levasseur]

Hello,

This an update about my Duplicity pending issue.

I have upgraded to 0.6.04, but the problem is still there with the
following Traceback :

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1241, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1234, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1190, in main
    verify(col_stats)
  File "/usr/bin/duplicity", line 660, in verify
    for backup_ropath, current_path in collated:
  File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line
270, in collate2iters
    relem1 = riter1.next()
  File "/usr/lib/python2.5/site-packages/duplicity/patchdir.py", line
492, in integrate_patch_iters
    final_ropath = patch_seq2ropath(normalize_ps(patch_seq))
  File "/usr/lib/python2.5/site-packages/duplicity/patchdir.py", line
472, in patch_seq2ropath
    misc.copyfileobj(current_file, tempfp)
  File "/usr/lib/python2.5/site-packages/duplicity/misc.py", line 170,
in copyfileobj
    outfp.write(buf)

Can anybody give me some help ?

Pascal

Le mercredi 12 août 2009 à 12:15 +0200, Pascal Levasseur a écrit :
> Hello all,
> 
> I have an issue with the duplicity verify command, all other duplicity
> commands seems OK.
> 
> The duplicity verify command process the first volumes perfectly, then
> abort with this message (sorry for the paranoid xxxx) :
> 
> Running 'ncftpget
> -f /home/zoe/tmp/duplicity-EYYkrt-tempdir/mkstemp-SrTlr3-1 -F -t 30 -o
> useCLNT=0,useHELP_SITE=0  -V -C 'ftp.xxxxx.net'
> 'Duplicity/duplicity-full.20090729T153503Z.vol5.difftar.gpg'
> '/home/zoe/tmp/duplicity-EYYkrt-tempdir/mktemp-5PTaAJ-15'' (attempt #1)
> Deleting /home/zoe/tmp/duplicity-EYYkrt-tempdir/mktemp-hqSU5Q-9
> Processed volume 6 of 214
> Running 'ncftpget
> -f /home/zoe/tmp/duplicity-EYYkrt-tempdir/mkstemp-SrTlr3-1 -F -t 30 -o
> useCLNT=0,useHELP_SITE=0  -V -C 'ftp.xxxxx.net'
> 'Duplicity/duplicity-inc.20090801T110014Z.to.20090810T064752Z.vol2.difftar.gpg'
>  '/home/zoe/tmp/duplicity-EYYkrt-tempdir/mktemp-ha9PCR-16'' (attempt #1)
> Traceback (most recent call last):
>   File "/usr/bin/duplicity", line 825, in <module>
>     with_tempdir(main)
>   File "/usr/bin/duplicity", line 818, in with_tempdir
>     fn()
>   File "/usr/bin/duplicity", line 777, in main
>     verify(col_stats)
>   File "/usr/bin/duplicity", line 554, in verify
>     for backup_ropath, current_path in collated:
>   File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line
> 256, in collate2iters
>     relem1 = riter1.next()
>   File "/usr/lib/python2.5/site-packages/duplicity/patchdir.py", line
> 492, in integrate_patch_iters
>     final_ropath = patch_seq2ropath(normalize_ps(patch_seq))
>   File "/usr/lib/python2.5/site-packages/duplicity/patchdir.py", line
> 472, in patch_seq2ropath
>     misc.copyfileobj(current_file, tempfp)
>   File "/usr/lib/python2.5/site-packages/duplicity/misc.py", line 170,
> in copyfileobj
>     outfp.write(buf)
> IOError: [Errno 28] No space left on device
> 
> 
> This error is not transient, it fails every time I execute the command.
> 
> The /tmp volume (350 Mb) shows a steady increase up to 97% of used space
> until duplicity crash, then goes back to a very low value (df /tmp).
> 
> 
> Filesystem           1K-blocks      Used Available Use% Mounted
> on /dev/mapper/loadstone-tmp
>                         376807    344394     12957  97% /tmp
> 
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/mapper/loadstone-tmp
>                         376807     10311    347040   3% /tmp
> 
> I was not able to find in the /tmp hierarchy the file(s) related to this
> increase of used space.
> 
> The duplicity temporary directory (--tempdir) increase up to 60 Mb
>  (du --si /home/zoe/tmp) then is cleared of any files by duplicity after
> duplicity abort.
> 
> 60M   /home/zoe/tmp/duplicity-EYYkrt-tempdir
> 
> 
> My configuration is :
> 
> - Debian Linux 2.6.26-2-62-686 version 5 with luks crypto file system,
> - /tmp is a separate volume with a fixed size. I dont want to increase
> its size, as the actual size seems ok for all the applications except
> Duplicity and I dont want to mess with the crypto file system. 
> - Duplicity version 0.5.18,
> - Backend is ftp with ncfftp 3.2.1,
> - The Duplicity backup size is more than 200 volumes of 25 MBytes.   
> - The shell script used to start the duplicity verify command is :
> 
> FTP_PASSWORD=xxxxxxx
> export FTP_PASSWORD
> #
> PASSPHRASE=xxxxxxx
> export PASSPHRASE
> #
> TMPDIR=/home/zoe/tmp/
> export TMPDIR
> #
> TEMP=/home/zoe/tmp/
> export TEMP
> #
> TMP=/home/zoe/tmp/
> export TMP
> #
> duplicity verify -v5 --num-retries 10 --ftp-passive \
> --tempdir /home/zoe/tmp/
> --exclude-globbing-filelist /home/zoe/duplicity_xxxxx/duplicity_exclude_files 
> \
> ftp://address@hidden/Duplicity/ /home/zoe/Sauvegarde/
> 
> 
> I think I have used all the "by the book" options to redirect temporary
> files outside of /tmp, but it seems that I fail to find the complete
> solution !
> 
> 
> Thanks in advance for your help.
> 
> Pascal Levasseur
> 
> 
> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk