bug#37207: guix.gnu.org returns Last-Modified = Epoch

[Christopher Baines]

Ludovic Courtès <address@hidden> writes:

> Howdy!
>
> Christopher Baines <address@hidden> skribis:
>
>> Ludovic Courtès <address@hidden> writes:
>>
>>> Since the use of the ‘static-web-site’ service, which puts web site
>>> files in the store, nginx returns a ‘Last-Modified’ header that can
>>> trick clients into caching things forever:
>>>
>>> --8<---------------cut here---------------start------------->8---
>>> $ wget --debug -O /dev/null   https://guix.gnu.org/packages.json 2>&1 | 
>>> grep Last
>>> Last-Modified: Thu, 01 Jan 1970 00:00:01 GMT
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> We should tell nginx to do not emit ‘Last-Modified’, or to take the
>>> state from the /srv/guix.gnu.org symlink, if possible.
>>
>> I ended up looking at this again in relation to Repology [1].
>>
>> 1: 
>> https://github.com/repology/repology-updater/issues/218#issuecomment-525905704
>>
>> Going back to that comment, given that the Last-Modified header (and the
>> ETag) is wrong, it's probably sensible to remove them. That might even
>> fix the issue with Repology fetching the packages.json file.
>>
>> Alternatively (or in addition), we could run a really simple Guile web
>> server that just serves the packages.json file with the right
>> Last-Modified value, and have NGinx proxy requests to that server. This
>> would be pretty easy to setup I believe, and would allow providing a
>> correct value.
>>
>> Thoughts?
>
> I think it wouldn’t really help because the Last-Modified issue is
> pervasive.  It shows for instance when accessing the web site: one often
> has to force the browser to reload pages to get the latest version.
>
> So I’m all for one of the solutions that were proposed earlier.
>
> WDYT?

So I think removing the Last-Modified header from the responses will fix
the issue with the Repology fetcher (as it will stop thinking it's
already fetch the file, since it was last modified in 1970), instead it
will just always process the file.

Removing the Last-Modified header, and maybe the ETag as well from
responses should avoid the issue with web browsers using a cached
version of the page when they probably shouldn't.

I realise what I described with using a Guile web server to serve the
packages.json file wouldn't help with other pages (unless they're served
as well, which is a possibility), but that was just an optimisation over
removing the header entirely, as having the Last-Modified header, with a
correct value would help the Repology fetcher cache the file.

Does that make sense? It still seems to me that a small change to the
NGinx config (I think these lines somewhere in the config would do it
[1]) would help with the Repology fetcher issue, and the issue you
describe with web browsers.

1:

add_header Last-Modified "";
if_modified_since off;
etag off;

signature.asc
Description: PGP signature