I ended up with the following NGINX configuration...
in /etc/nginx/conf.d/upstream.conf:
upstream sks_secondary {
server 127.0.0.1:11371 weight=5;
server 172.16.20.52:11371 weight=10;
server 172.16.20.53:11371 weight=10;
server 172.16.20.54:11371 weight=10;
}
upstream sks_primary {
server 127.0.0.1:11371;
server 172.16.20.52:11371 backup;
server 172.16.20.53:11371 backup;
server 172.16.20.54:11371 backup;
}
map $arg_op $sks_server {
"stats" sks_primary;
default sks_secondary;
}
in /etc/nginx/site-available/sks-default:
server {
listen 172.16.20.51:80 default_server;
listen 172.16.20.51:11371 default_server;
listen [::]:80 ipv6only=on default_server;
# listen [::]:11371 ipv6only=on default_server;
access_log off;
server_tokens off;
root /var/www/html;
index index.html index.htm;
proxy_buffering on;
proxy_buffer_size 1k;
proxy_buffers 24 4k;
proxy_busy_buffers_size 8k;
proxy_max_temp_file_size 2048m;
proxy_temp_file_write_size 32k;
location /pks/hashquery {
proxy_ignore_client_abort on;
proxy_pass http://$sks_server;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_pass_header Server;
add_header Via "1.1 sks.undergrid.net:$server_port
(nginx)";
}
location /pks/add {
proxy_ignore_client_abort on;
proxy_pass http://$sks_server;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_pass_header Server;
add_header Via "1.1 sks.undergrid.net:$server_port
(nginx)";
client_max_body_size 8m;
}
location /pks {
proxy_cache ugns_sks_cache;
# proxy_cache_background_update on;
proxy_cache_lock on;
proxy_cache_min_uses 3;
proxy_cache_revalidate on;
proxy_cache_use_stale error timeout updating http_500
http_502 http_503 http_504;
proxy_cache_valid any 10m;
proxy_ignore_client_abort on;
proxy_ignore_headers Cache-Control "Expires";
proxy_pass http://$sks_server;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_pass_header Server;
add_header Via "1.1 sks.undergrid.net:$server_port
(nginx)";
add_header X-Proxy-Cache $upstream_cache_status;
}
}
The NGINX configuration appears to be working fine for me... My 3
backend nodes are operating as I expect as well.. The problem I'm
seeing exhibited currently is that my primary node which is
running along with NGINX seems to be writing an awful lot of log
files and not processing them despite having the same DB_CONFIG
file as the other 3 nodes. With each of the log files running
100MB it is quickly filling up the drive on that node and then the
sks and sks-recon processes simply fall over and crash but the
other 3 nodes running behind it keep chugging along just not
getting any further gossip input.
I keep seeing the following log entry popping up only on my
primary node:
add_keys_merge failed: Eventloop.SigAlarm
On 2/25/2019 12:37 PM, Todd Fleisher
wrote:
I didn't have as many locations configured as
you show in your example but it looked like you were
defining the map but I didn't see it being used in any
of your location blocks unless I'm missing something.
Shouldn't you be using $upstream_server in your
proxy_pass configuration?
I think you’re on to something here. I just tried commenting out
the other servers from the upstream sks_servers_primary block and
am still seeing stats queries hitting the commented out servers.
Kristian - could you please double check the
configuration snippets you provided to me last year and see if
something was missing related to this?
-T