Re: [Lynx-dev] what user agent header?

[Karen Lewellen]

Interesting answer thanks.
I suppose that test will share what header is being used when I remove the 
option.
I find it is wonderful doing as much when I reach one of those 403 
forbidden pages, works almost every time.
I am no technician, so was unclear how to simulate the googlebot  element?
Thanks again,
Karen


On Thu, 3 Nov 2016, Tim Chase wrote:

> On 2016-11-03 19:31, Karen Lewellen wrote:
> > I am wondering what the default  user agent header is, if you ask
> > lynx not to send a user agent header?
>
> If you uncheck the "Send user agent header", it's simply omitted from
> the list of headers.  You can test it at
>
>  http://headers.cloxy.net/request.php
>
> which just echoes back the headers your browser sends.  With my stock
> lynx on Debian, it includes
>
> * User-Agent: Lynx/2.8.9dev.1 libwww-FM/2.14 SSL-MM/1.4.1
>   GNUTLS/3.3.8
>
> But when I go into options and turn off "Send user agent header",
> that header list-item simply isn't present.
>
> Lynx also offers you the ability to send the header but spoof its
> content to pretend to be some other browser.  Few sites check this any
> more, but I've had the occasional site that gives truncated content
> for normal browsers but gives the full content if you pretend to be
> Googlebot (Google's web crawler).
>
> -tim
>
> _______________________________________________
> Lynx-dev mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/lynx-dev