[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] How do I use relative files on the cmd line with a base?
From: |
Linda Walsh |
Subject: |
Re: [Bug-wget] How do I use relative files on the cmd line with a base? |
Date: |
Fri, 15 Jun 2012 10:50:52 -0700 |
User-agent: |
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.24) Gecko/20100228 Lightning/0.9 Thunderbird/2.0.0.24 Mnenhy/0.7.6.666 |
Cngel GonzC!lez wrote:
On 14/06/12 13:19, Linda Walsh wrote:
I have a list of files in a directory.
I only want each wget to retrieve 10 of the files, so I want to use
parallel to put 10 of the files on wget's command line.
I tried specifying the base with -B, but it seems to ignore it.
how do I specify a base with files on the command line?
I've wanted to do this before ..... where several files I want to fetch
are in 1 dir, and I only want to type the dirname 1 time, so how does
one do this in wget?
Thanks...
linda
I'm not sure if this is a question on splitting a list of urls in groups
of 10, or about not typing the domain for each url (paste command
doesn't work?).
Assuming you are using a unix-like shell, you can use:
wget http://example.org/path/{file1,file2,subfolder/file3,file4}
and that will expand being equivalent to:
wget http://example.org/path/file1 http://example.org/path/file2
http://example.org/path/subfolder/file3 http://example.org/path/file4
0000
well -- the http address is 80 characters long --
some of the base's pathname chars had shell unfriendly
chars in them -- to the point that I *had* to put the base URL into
a file and not on the command line (kept running into quoting problems)...
The filenames were relatively simple (foo.rpm... x.rpm...etc...)..
Try your example with
base="http://download.openproject.org/repositories/devel:/languages:/perl/openproject_Factory/src/?release=alpha1.0&lang=en&encoding=binary&source="
and files = package?arch=x86_64&fmt=rpm,name=package1
package?arch=no_arch&fmt=rpm,name=package2, ...
Expand that in a shell... and it doesn't work so well...
Not to mention, doing 10 lines at a time make the lines expand to
over 1360 chars (because the base is 136 chars by itself
exclusive of wget options and the file names...
So I wanted to put the base in a file and just use -B "$(<base)", and put the
filenames at the end. -- and not use
the shell but use 'parallel', which spawns processes directly, so I don't have
to worry about expansion --
(I could have it do expansion by running it with bash -c "command", but if
expansion is causing a problem
its a bonus not to incur running a command shell with each iteration)....
Many times, I've had a long URL name that I want to set, then just enter
filenames relative to that base.
and with the base being complicated -- I just want to put it in a file...)
I don't see why putting filenames on the command line ignores the -B switch??
I.e. -- what is the purpose served by ignoring a "B" switch the user specifies
on the command
line when they fetch files? Is there a benefit to it being ignored?