Re: [Bug-wget] CNET download links not working with WGET

[Micah Cowan]

If you read the most recent output of wget that you gave (after quoting 
the URL), it _does_ treat the string of characters as a whole URL. The 
server redirects it to a shorter URL. If I enter that same URL into a 
browser, it does the same redirection there, and results in an HTML 
page, just like what wget gets. That page seems to have some JavaScript 
or something that initiates a separate download of something else; I 
suppose that something else is what you wanted. As you may know, wget 
doesn't execute JavaScript code from a webpage, so you'll need to find 
the real URL to the thing you wanted to download, and feed that to wget.

-mjc

On 06/08/2011 09:38 AM, Jeff Givens wrote
> No, it's not working. It downloads part of the URL and creates a file
> named address@hidden
> which is 68 KB. I cannot wget to treat the string of characters as a
> whole URL. Please help, I really need to get this script working and the
> only place to download this file is from CNET.
>
> > So... looks like it works, then. Your command shell isn't complaining
> > about weird command names, wget is clearly requesting the full and
> > correct URL, it follows redirections, and saves using the final
> > redirection URL (the latest sources wouldn't follow that last step -
> > it'd save using the request URI by default).
> >
> > If you dislike the filename, then provided you have a recent enough
> > version of wget you can add the --content-disposition option if the
> > server provides a rename header ("Content-Disposition"); or else use -E
> > to have wget force the file name to end in .html
> >
> > -mjc
> >
> > (05/26/2011 12:19 PM), Jeff Givens wrote:
> > > Hi, I know this is an older topic but thanks for replying. I forgot to
> > > mention I had already what you listed below and this is the output I
> > > get:
> > >
> > > C:\DOWNLOAD>wget "http://dw.com.com/redir?edId=3&siteId=4&oId=3000-8022_
> > > 4-10804572&ontId=8022_4&spi=077d9109e846975d0db9532bd610588f&lop=link&tag=tdw_dl
> > >
> > >
> > > text&ltype=dl_dlnow&pid=11665648&mfgId=6290020&merId=6290020&pguid=HFsQLwoOYJQAA
> > >
> > >
> > > BuImQcAAAGm&destUrl=http%3A%2F%2Fdownload.cnet.com%2F3001-8022_4-10804572.html%3
> > >
> > >
> > > Fspi%3D077d9109e846975d0db9532bd610588f"
> > > --2011-05-02 12:34:20--
> > > http://dw.com.com/redir?edId=3&siteId=4&oId=3000-8022_4
> > > -10804572&ontId=8022_4&spi=077d9109e846975d0db9532bd610588f&lop=link&tag=tdw_dlt
> > >
> > >
> > > ext&ltype=dl_dlnow&pid=11665648&mfgId=6290020&merId=6290020&pguid=HFsQLwoOYJQAAB
> > >
> > >
> > > uImQcAAAGm&destUrl=http%3A%2F%2Fdownload.cnet.com%2F3001-8022_4-10804572.html%3F
> > >
> > >
> > > spi%3D077d9109e846975d0db9532bd610588f
> > > Resolving dw.com.com... 216.239.113.95
> > > Connecting to dw.com.com|216.239.113.95|:80... connected.
> > > HTTP request sent, awaiting response... 302 Found
> > > Location:
> > > http://download.cnet.com/3001-8022_4-10804572.html?spi=077d9109e846975
> > > d0db9532bd610588f [following]
> > > --2011-05-02 12:34:21--
> > > http://download.cnet.com/3001-8022_4-10804572.html?spi=
> > > 077d9109e846975d0db9532bd610588f
> > > Resolving download.cnet.com... 64.30.224.58
> > > Connecting to download.cnet.com|64.30.224.58|:80... connected.
> > > HTTP request sent, awaiting response... 200 OK
> > > Length: unspecified [text/html]
> > > Saving to:
> > > address@hidden'
> > >
> > > [<=> ] 69,240 77.3K/s in 0.9s
> > >
> > > 2011-05-02 12:34:22 (77.3 KB/s) -
> > > address@hidden
> > > d0db9532bd610588f.1' saved [69240]
> > >
> > >
> > > C:\DOWNLOAD>
> > >
> > > Thanks for your help.
> > >
> > > - Jeff