Re: [Bug-wget] Login to sonic.net using post-data over https not working

[Ray Satiro]

> > So first either set a blank session cookie (described  above) or better 
> > just 
>get
> > one:

> > wget --save-cookies=sonic.txt  --keep-session-cookies 
>https://members.sonic.net/
> >
> > Then login:
> > wget  --load-cookies=sonic.txt --save-cookies=sonic.txt 
>--keep-session-cookies
> >  --post-data="user=a&pw=b" https://members.sonic.net/
> >
> 
> Thanks to your help, I am now  logged on. The next step involves
> posting my tunnel IPv4 endpoint address to  a form. However this seems
> to be less straight forward as I anticipated. As  it turns out, I need
> to go through 2 iterations of the same form to get this  done. The
> first one is a simple no visible fields form which leads to the  same
> page a second time arround on which one needs to update the  endpoint
> input field with the new IP. I can pass the first iteration fine,  but
> the second time my new IP is not updated. As you can see cli  the
> specified IP 76.191.203.16 is not reflected back as it should be  in
> the index.html page that is returned but shows the original value  of
> 76.191.203.14. Here is what I'm issuing:
> 

[...]

> $ wget --no-check-certificate  --load-cookies=sonic.txt
> --post-data='enpoint=76.191.203.16&rdns_server=none&change=1&action=step1'
> -S  -dv https://members.sonic.net/connections/ipv6tunnel/

[...]

>                      <form method="post" action="#">
>                              <table>
>                  <tr><td>Your current tunnel endpoint  is:
> </td><td><input type='text'  name='endpoint'
> value='76.191.203.14'/></td></tr>
>                  <tr>
>                     <td>Please enter your  reverse DNS server IP:
> </td><td><input type='text'  name='rdns_server'
> value='none'/>(Optional)
>                          <input type='hidden'  name='change' value='1'/>
>                     </td>
>                  </tr>
>                  <tr>
>                     <td>
>                          [<a
> href='https://wiki.sonic.net/wiki/Secondary_DNS_Service'
> target='_blank'>Help?</a>]
>                     </td>
>                  </tr>
>                  </table>
>                              <br/><br/>
>                             <div align="center">
>                                 <input type="hidden" name="action"  
>value="step1"/>
>                                <input type="submit"  class="button"
> value="View/Request/Change Tunnel"/>
>                              </div>
>                       </form>


Eric the short answer for you is the problem is at least the misspelling of 
endpoint:
enpoint=76.191.203.16

The long answer if that doesn't solve the problem is this. Web admins seem to 
be 
blocking unattended POSTs more and more. They can do this using captcha and/or 
obscuring the actual post location, among other attributes. From what I 
understand this is done mostly to prevent against spam. From your html:

<form method="post" action="#">
Sometimes there is javascript somewhere that will change a form's action only 
if 
human activity is detected, like focus or a mouse click. So something like this 
could be somewhere:
document.someform.action = "url" or
document.getElementById("someform").setAttribute("action", "url")

When I have a situation like yours rather than immediately wade through the js 
I 
first use Fiddler to monitor requests and  replies in my browser and then 
compare them to wget requests and  replies to see what is happening and what is 
different. I then adjust my wget request to match my browser's request. First 
the data and location. Then if you still have problems you might have to change 
'User-Agent' or 'Referer' for example. A request will usually pass successfully 
once all those things match like 99%. In the other 1% you might have to scrape 
some page using libxml or something. Also some things you can't/shouldn't 
misrepresent using wget, like accepting gzip encoding. If you have to have a 
flawless dupe you might have to use a library like libcurl.


I took a look at sonic's js and I don't see anything like what I described, so 
I 
assume that form action is to post to that page. I've CC'd the wget list for 
the 
benefit of anyone searching through it in a similar situation. If you have any 
more questions please contact me directly so we don't clog the list.

Jay