Re: Proposal for an Emacs User Survey

[Philip K.]

Adrien Brochard <abrochard@gmx.com> writes:

>>> - cleaner UI, not just HTML embedded in the code
>>
>> What would you have in mind? I know that my example is simple, but I
>> prefer that to sites that take forever to load and reload everything all
>> the time, for no apparent reason.
>
> The most obvious reason to me is that user error handling is pretty
> poor. Because there is no JS, we cannot offer front-end validation, that
> means that the backend server is responsible for validating fields
> submitted. If validation does not pass, the page must "reload" for the
> user and it needs to show exactly what went wrong and preserve the user
> input. That's my definition of a site that reloads all the time.

That could be mitigated with a "graceful degradation" approach, since
most people do have javascript activated by default. HTML5 also has a
few attributes that could help, such as pattern or required, depending
on what questions are being asked.

>>> - mysql instead of sqlite, which also implies a mysql instance running
>>
>> SQLite is actually surprisingly resilient, according to [0]:
>>
>>> SQLite works great as the database engine for most low to medium traffic
>>> websites (which is to say, most websites). The amount of web traffic
>>> that SQLite can handle depends on how heavily the website uses its
>>> database. Generally speaking, any site that gets fewer than 100K
>>> hits/day should work fine with SQLite. The 100K hits/day figure is a
>>> conservative estimate, not a hard upper bound. SQLite has been
>>> demonstrated to work with 10 times that amount of traffic.
>>
>> But either way, I used sqlite to avoid setting up a RDBMS.
>>
>> [0] https://www.sqlite.org/whentouse.html
>
> SQLite is resilient but it's dangerous to use it with multiple threads
> like how your go server does. You could use a single channel model to
> write records one at a time but now you have a risk to lose data from
> memory if your service restarts.

I'm familiar with the danger, but it shouldn't be a problem. As pointed
out in [0], one can activate the SQLite mutex per flag. But at the same
time, even sites as popular as Hacker News are alleged to use SQLite as
their backend.

[0] https://github.com/mattn/go-sqlite3/issues/249

>>
>>> - DOS protection, maybe some rate-limiting and IP blocking
>>> - HTTPS, thankfully it's easier now
>>
>> AFAIK these things can usually be handled by a frond-end such as NGINX.
>
> That's true it can, but that means you need to deploy and configure one.
> https://www.nginx.com/blog/rate-limiting-nginx/
> https://nginx.org/en/docs/http/configuring_https_servers.html

Perhaps using Guix could make that easier:
https://guix.gnu.org/manual/devel/en/html_node/Web-Services.html#NGINX?

>>> - monitoring, how do we know the service is running as expected
>>> - logging, and how to store logs for debug
>>
>> If there is any interest, extending the example to support this would be
>> feasible. The "20%" you mention aren't easy, but from what I see it
>> shouldn't be too hard either.
>
> You're absolutely right. None of it is "too hard". But the problem is
> just how much time and effort are we willing to put into it. If the goal
> is to make an open source polling platform that we can re-use in the
> future, then absolutely all of this can be done. But if we treat this as
> a one-off and see how it turns out, the scope seems exaggerated.

Depending on how many people participate, it would be interesting the
repeat the survey year-by-year, so that trends and changes in the
user-base can be recognized.

-- 
        Philip K.