[Bug-wget] [bug #56946] Performance of `-r -A` is poor

[anonymous]

URL:
  <https://savannah.gnu.org/bugs/?56946>

                 Summary: Performance of `-r -A` is poor
                 Project: GNU Wget
            Submitted by: None
            Submitted on: Tue 24 Sep 2019 06:57:18 PM UTC
                Category: None
                Severity: 3 - Normal
                Priority: 5 - Normal
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: Jo Shields
        Originator Email: address@hidden
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 1.19.4
        Operating System: GNU/Linux
         Reproducibility: Every Time
           Fixed Release: None
         Planned Release: None
              Regression: None
           Work Required: None
          Patch Included: None

    _______________________________________________________

Details:

This takes 4 hours:


wget --quiet -nH --cut-dirs=6 -e robots=off -r --no-parent -A
*_6.0.0.334-0xamarin1+ubuntu1604b1_*.deb
https://download.mono-project.com/repo/debian/pool/main/m/mono/


This takes under 20 seconds to do the same thing:


wget --quiet -O-
https://download.mono-project.com/repo/debian/pool/main/m/mono/ | sed
'/.*_6.0.0.334-0xamarin1+ubuntu1604b1_.*.deb/!d;
s#.*href="#https://download.mono-project.com/repo/debian/pool/main/m/mono/#;
s/\.deb.*/.deb/' > downloadlist.txt
wget --quiet --input-file=downloadlist.txt


The performance difference in filtering is surprising.




    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?56946>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/