Re: [Bug-ddrescue] Improving ddrescue

[Antonio Diaz Diaz]

Santiago Cassina wrote:
> Hello, I've already rescued entire disk but I connected it again to run
> the tests you asked

Thanks.


> 1) I had to download and compile newest sources because I had 1.17 version

Note that the newest version is not 1.21, but 1.22-pre5, which you can 
find at http://download.savannah.gnu.org/releases/ddrescue/


> 2) When I'm using new version I never see error count log incremented
> (only "non-trimmed" is incremented on screen), as "errors" count is
> never incremented I don't know if the input device is reopening (maybe
> you can include that on screen too, or in a -v -vv verbose mode like
> "input reopens count" sign or something beautiful like all the new
> output of ddrescue)

Good observations. Maybe the current naming is a little confusing. 
Perhaps I should rename 'errsize' to 'bad-sector', and 'errors' to 'bad 
areas'.

As reopening is optional, maybe instead of (or in addition to) an "input 
reopens count" I could add two counters for 'read errors' and 'slow reads'.


> 3) As I have no clue when input device is reopening I often kill and
> recall ddrescue with same parameters to check if there are read rate
> improvements but is hard to tell

Reopenings may happen every few seconds. Usually once for each 64 KiB of 
increase of the 'non-trimmed' size.


> 4) I tried with --min-read-rate 1MB, 10MB but I don't understand what
> ddrescue does when it can't read at that rate (I just see "non-trimmed"
> incremented)

Only read errors increase 'non-trimmed'. Slow reads make ddrescue skip, 
but it leaves the skipped areas marked as non-tried for later passes. 
The copying phase has been improved since version 1.21. Now it does this:

-----------------------------------------------------------------------
   2) (First phase; Copying) Read the non-tried parts of the input 
file, marking the failed blocks as non-trimmed and skipping beyond them. 
Skip also beyond slow areas. The skipped areas are tried later in four 
additional passes (before trimming), reversing the direction after each 
pass until all the rescue domain is tried.

   The second pass delimits the blocks skipped by the first pass. The 
third and fourth passes read the blocks skipped due to slow areas (if 
any) by the first two passes, in the same direction that each block was 
skipped.  Passes 2 to 4 skip the rest of each block after finding the 
first error.  The last pass is a sweeping pass, with skipping disabled. 
The purpose of the multiple passes is to delimit large errors fast, 
recover the most promising areas first, keep the mapfile small, and 
produce good starting points for trimming.

   Only non-tried areas are read in large blocks. Trimming, scraping 
and retrying are done sector by sector. Each sector is tried at most two 
times; the first in this phase as part of a large block read, the second 
in one of the phases below as a single sector read.
-----------------------------------------------------------------------


> As I previously wrote, I don't see "errors" count incremented, I don't
> know if you "forgot" to error++ or what but the thisk has errors for sure

A sector is not considered an error until it has been tried individually 
instead of as part of a large block read. Non-trimmed and non-scraped 
blocks are not considered errors. 'errors' begin incrementing in the 
trimming phase.


> IN GENERAL: I can see some improvements but not sure what ddrescue is
> doing in background
>
> Anyway, if you need I can do some more tests for you and your amazing
> software, I always use it and I want to help you improving it

It would be useful if you could leave ddrescue (1.22-pre5) finish the 
copying phase, especially passes 3 and 4 where it reads the areas 
skipped because of slow reads. You could also record the time taken by 
each pass using the option '--log-events=FILE' and send me the resulting 
FILE.

ddrescue /dev/sdl3 /nfs/storea/test_ddrescue_sdl3 /tmp/test_ddrescue.log 
--reopen-on-error --idirect --force -vv --min-read-rate 1MB 
--log-events=/tmp/events.log


> When very slow rates detected (65536 Bytes or 232768 Bytes) ddrescue
> changed from Pass 1 (forward) to Pass 2 (backwards) as the output
> shows

This is because pass 1 has skipped beyond all the slow areas forwards. 
Pass 2 delimits the rear edge of slow areas, and then passes 3 and 4 
read these skipped areas.


Thanks,
Antonio.