[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[taler-grid5k] 40/141: update configurations
From: |
gnunet |
Subject: |
[taler-grid5k] 40/141: update configurations |
Date: |
Thu, 18 Nov 2021 14:49:41 +0100 |
This is an automated email from the git hooks/post-receive script.
marco-boss pushed a commit to branch master
in repository grid5k.
commit 0765e9d47bafde2925ce7771c438049b6a7ea758
Author: Boss Marco <bossm8@bfh.ch>
AuthorDate: Tue Oct 12 01:10:40 2021 +0200
update configurations
---
configs/etc/default/prometheus | 112 +++++++++++++++++++++++
configs/etc/default/prometheus-nginx-exporter | 38 ++++++++
configs/etc/default/prometheus-postgres-exporter | 19 ++++
configs/etc/monitor/loki.yaml | 15 +--
configs/etc/monitor/prometheus.yaml | 20 ++++
configs/etc/nginx/sites-available/proxy | 8 ++
experiment/env | 4 +-
experiment/run.sh | 2 +-
experiment/scripts/database.sh | 3 +
experiment/scripts/monitor.sh | 4 +
10 files changed, 208 insertions(+), 17 deletions(-)
diff --git a/configs/etc/default/prometheus b/configs/etc/default/prometheus
new file mode 100644
index 0000000..b9b82ca
--- /dev/null
+++ b/configs/etc/default/prometheus
@@ -0,0 +1,112 @@
+ARGS="--config.file=/etc/monitor/prometheus.yaml
--web.listen-address=0.0.0.0:8080"
+
+# Prometheus supports the following options:
+# --config.file="/etc/prometheus/prometheus.yml"
+# Prometheus configuration file path.
+# --web.listen-address="0.0.0.0:9090"
+# Address to listen on for UI, API, and telemetry.
+# --web.read-timeout=5m Maximum duration before timing out read of the
+# request, and closing idle connections.
+# --web.max-connections=512 Maximum number of simultaneous connections.
+# --web.external-url=<URL> The URL under which Prometheus is externally
+# reachable (for example, if Prometheus is served
+# via a reverse proxy). Used for generating
+# relative and absolute links back to Prometheus
+# itself. If the URL has a path portion, it will
+# be used to prefix all HTTP endpoints served by
+# Prometheus. If omitted, relevant URL components
+# will be derived automatically.
+# --web.route-prefix=<path> Prefix for the internal routes of web endpoints.
+# Defaults to path of --web.external-url.
+# --web.local-assets="/usr/share/prometheus/web/"
+# Path to static asset/templates directory.
+# --web.user-assets=<path> Path to user asset directory, available at
+# /user.
+# --web.enable-lifecycle Enable shutdown and reload via HTTP request.
+# --web.enable-admin-api Enable API endpoints for admin control actions.
+# --web.console.templates="/etc/prometheus/consoles"
+# Path to the console template directory,
+# available at /consoles.
+# --web.console.libraries="/etc/prometheus/console_libraries"
+# Path to the console library directory.
+# --web.page-title="Prometheus Time Series Collection and Processing Server"
+# Document title of Prometheus instance.
+# --web.cors.origin=".*" Regex for CORS origin. It is fully anchored.
+# Example: 'https?://(domain1|domain2)\.com'
+# --storage.tsdb.path="/var/lib/prometheus/metrics2/"
+# Base path for metrics storage.
+# --storage.tsdb.retention=15d
+# [DEPRECATED] How long to retain samples in
+# storage. This flag has been deprecated, use
+# "storage.tsdb.retention.time" instead
+# --storage.tsdb.retention.time=15d
+# How long to retain samples in storage. When this
+# flag is set it overrides
+# "storage.tsdb.retention".
+# If neither this flag nor "storage.tsdb.retention"
+# nor "storage.tsdb.retention.size" is set, the
+# retention time defaults to 15d.
+# Units Supported: y, w, d, h, m, s, ms.
+# --storage.tsdb.retention.size=
+# [EXPERIMENTAL] Maximum number of bytes that can
+# be stored for blocks. Units supported: KB, MB,
+# GB, TB, PB. This flag is experimental and can be
+# changed in future releases.
+# --storage.tsdb.use-lockfile
+# Create a lockfile in data directory.
+# --storage.tsdb.allow-overlapping-blocks
+# [EXPERIMENTAL] Allow overlapping blocks, which
+# in turn enables vertical compaction and
+# vertical query merge.
+# --storage.tsdb.wal-compression
+# Compress the tsdb WAL.
+# --storage.remote.flush-deadline=<duration>
+# How long to wait flushing sample on shutdown or
+# config reload.
+# --storage.remote.read-sample-limit=5e7
+# Maximum overall number of samples to return via
+# the remote read interface, in a single query. 0
+# means no limit. This limit is ignored for
+# streamed response types.
+# --storage.remote.read-concurrent-limit=10
+# Maximum number of concurrent remote read calls.
+# 0 means no limit.
+# --storage.remote.read-max-bytes-in-frame=1048576
+# Maximum number of bytes in a single frame for
+# streaming remote read response types before
+# marshalling. Note that client might have limit on
+# frame size as well. 1MB as recommended by
+# protobuf by default.
+# --rules.alert.for-outage-tolerance=1h
+# Max time to tolerate prometheus outage for
+# restoring "for" state of alert.
+# --rules.alert.for-grace-period=10m
+# Minimum duration between alert and restored "for"
+# state. This is maintained only for alerts with
+# configured "for" time greater than grace period.
+# --rules.alert.resend-delay=1m
+# Minimum amount of time to wait before resending
+# an alert to Alertmanager.
+# --alertmanager.notification-queue-capacity=10000
+# The capacity of the queue for pending
+# Alertmanager notifications.
+# --alertmanager.timeout=10s
+# Timeout for sending alerts to Alertmanager.
+# --query.lookback-delta=5m The maximum lookback duration for retrieving
+# metrics during expression evaluations and
+# federation.
+# --query.timeout=2m Maximum time a query may take before being
+# aborted.
+# --query.max-concurrency=20
+# Maximum number of queries executed concurrently.
+# --query.max-samples=50000000
+# Maximum number of samples a single query can load
+# into memory. Note that queries will fail if they
+# try to load more samples than this into memory,
+# so this also limits the number of samples a query
+# can return.
+# --log.level=info Only log messages with the given severity or
+# above. One of: [debug, info, warn, error]
+# --log.format=logfmt Output format of log messages. One of: [logfmt,
+# json]
+
diff --git a/configs/etc/default/prometheus-nginx-exporter
b/configs/etc/default/prometheus-nginx-exporter
new file mode 100644
index 0000000..c7cf7df
--- /dev/null
+++ b/configs/etc/default/prometheus-nginx-exporter
@@ -0,0 +1,38 @@
+ARGS="-nginx.scrape-uri http://127.0.0.1:80/stub_status"
+
+# Prometheus-nginx-exporter supports the following options:
+# -nginx.plus
+# Start the exporter for NGINX Plus. By default, the exporter is started
+# for NGINX. The default value can be overwritten by NGINX_PLUS environment
+# variable.
+# -nginx.retries uint
+# A number of retries the exporter will make on start to connect to the
+# NGINX stub_status page/NGINX Plus API before exiting with an error. The
+# default value can be overwritten by NGINX_RETRIES environment variable.
+# -nginx.retry-interval value
+# An interval between retries to connect to the NGINX stub_status
+# page/NGINX Plus API on start. The default value can be overwritten by
+# NGINX_RETRY_INTERVAL environment variable. (default 5s)
+# -nginx.scrape-uri string
+# A URI or unix domain socket path for scraping NGINX or NGINX Plus metrics.
+# For NGINX, the stub_status page must be available through the URI. For
+# NGINX Plus -- the API. The default value can be overwritten by SCRAPE_URI
+# environment variable. (default "http://127.0.0.1:8080/stub_status")
+# -nginx.ssl-verify
+# Perform SSL certificate verification. The default value can be overwritten
+# by SSL_VERIFY environment variable. (default true)
+# -nginx.timeout value
+# A timeout for scraping metrics from NGINX or NGINX Plus. The default value
+# can be overwritten by TIMEOUT environment variable. (default 5s)
+# -prometheus.const-labels value
+# A comma separated list of constant labels that will be used in every
+# metric. Format is label1=value1,label2=value2... The default value can be
+# overwritten by CONST_LABELS environment variable.
+# -web.listen-address string
+# An address or unix domain socket path to listen on for web interface and
+# telemetry. The default value can be overwritten by LISTEN_ADDRESS
+# environment variable. (default ":9113")
+# -web.telemetry-path string
+# A path under which to expose metrics. The default value can be overwritten
+# by TELEMETRY_PATH environment variable. (default "/metrics")
+
diff --git a/configs/etc/default/prometheus-postgres-exporter
b/configs/etc/default/prometheus-postgres-exporter
new file mode 100644
index 0000000..35fea35
--- /dev/null
+++ b/configs/etc/default/prometheus-postgres-exporter
@@ -0,0 +1,19 @@
+# Connection string for the PostgreSQL database. You need to either connect as
+# superuser, or create a user with enough rights, as described in
+# /usr/share/doc/prometheus-postgres-exporter/README.Debian
+
+# DATA_SOURCE_NAME='postgresql://login:password@hostname:port/'
+# DATA_SOURCE_NAME='user=prometheus host=/run/postgresql dbname=postgres'
+DATA_SOURCE_NAME='<DB_URL_HERE>'
+
+# Set the command-line arguments to pass to the server.
+ARGS=''
+
+# Available flags:
+# --web.listen-address=":9187" Address to listen on for web interface and
telemetry.
+# --web.telemetry-path="/metrics"
+# Path under which to expose metrics.
+# --extend.query-path="" Path to custom queries to run.
+# --log.level="info" Only log messages with the given severity or
above. Valid levels: [debug, info, warn, error, fatal]
+# --log.format="logger:stderr" Set the log target and format. Example:
"logger:syslog?appname=bob&local=7" or "logger:stdout?json=true"
+
diff --git a/configs/etc/monitor/loki.yaml b/configs/etc/monitor/loki.yaml
index 3ba496a..affd655 100644
--- a/configs/etc/monitor/loki.yaml
+++ b/configs/etc/monitor/loki.yaml
@@ -1,7 +1,7 @@
auth_enabled: false
server:
- http_listen_port: 8080
+ http_listen_port: 80
grpc_listen_port: 0
ingester:
@@ -47,7 +47,6 @@ compactor:
limits_config:
reject_old_samples: true
reject_old_samples_max_age: 168h
- unordered_writes: true
chunk_store_config:
max_look_back_period: 0s
@@ -55,15 +54,3 @@ chunk_store_config:
table_manager:
retention_deletes_enabled: false
retention_period: 0s
-
- #ruler:
- # storage:
- # type: local
- # local:
- # directory: /tmp/loki/rules
- # rule_path: /tmp/loki/rules-temp
- # alertmanager_url: http://localhost:9093
- # ring:
- # kvstore:
- # store: inmemory
- # enable_api: true
diff --git a/configs/etc/monitor/prometheus.yaml
b/configs/etc/monitor/prometheus.yaml
index e69de29..2a03395 100644
--- a/configs/etc/monitor/prometheus.yaml
+++ b/configs/etc/monitor/prometheus.yaml
@@ -0,0 +1,20 @@
+global:
+ scrape_interval: 5s
+ evaluation_interval: 5s
+
+scrape_configs:
+ # Self
+ - job_name: 'prometheus'
+ static_configs:
+ - targets: ['127.0.0.1:9090']
+
+ # DB
+ - job_name: 'database'
+ static_configs:
+ - targets: ['<DB_URL_HERE>:9187']
+
+ # Exchange Proxy
+ - job_name: 'exchange-proxy'
+ static_configs:
+ - targets: ['<PROXY_URL_HERE>:9113']
+
diff --git a/configs/etc/nginx/sites-available/proxy
b/configs/etc/nginx/sites-available/proxy
index 3d28add..5e96bf2 100644
--- a/configs/etc/nginx/sites-available/proxy
+++ b/configs/etc/nginx/sites-available/proxy
@@ -11,4 +11,12 @@ server {
#proxy_set_header X-Forwarded-Host "example.com";
#proxy_set_header X-Forwarded-Proto "https";
}
+
+ location /stub_status {
+ stub_status;
+ access_log off;
+ allow 127.0.0.1;
+ deny all;
+ }
+
}
diff --git a/experiment/env b/experiment/env
index fe2465a..a0d4715 100755
--- a/experiment/env
+++ b/experiment/env
@@ -16,8 +16,8 @@ BANK_DOMAIN=bank.perf.taler
PROMETHEUS_CNAME=prometheus.perf.taler
LOKI_CNAME=loki.perf.taler
-PROMETHEUS_PORT=http
-LOKI_PORT=http8080
+PROMETHEUS_PORT=http8080
+LOKI_PORT=http
DB_NAME=taler-exchange
DB_USER=taler
diff --git a/experiment/run.sh b/experiment/run.sh
index 1ba6a5c..f4c8c13 100644
--- a/experiment/run.sh
+++ b/experiment/run.sh
@@ -23,7 +23,7 @@ function set_cname() {
nsupdate -k /root/ddns.key -v << EOF
server ${DNS_HOST}
zone ${DNS_ZONE}
-update add $1 IN CNAME $(hostname | cut -d "." -f 1,2 -).$2.proxy.grid5000.fr
+update add $1 CNAME $(hostname | cut -d "." -f 1,2 -).$2.proxy.grid5000.fr
send
EOF
}
diff --git a/experiment/scripts/database.sh b/experiment/scripts/database.sh
index bf351de..f21dafb 100755
--- a/experiment/scripts/database.sh
+++ b/experiment/scripts/database.sh
@@ -7,6 +7,9 @@ cd /tmp
sed -i "s\<DB_URL_HERE>\postgresql:///${DB_NAME}\g" \
/etc/taler/secrets/exchange-db.secret.conf
+sed -i
"s\<DB_URL_HERE>\postgresql://${DB_USER}:${DB_PASSWORD}@localhost/${DB_NAME}\g"
\
+ /etc/default/prometheus-postgres-exporter
+
echo "listen_addresses='*'" >> /etc/postgresql/13/main/postgresql.conf
echo "host all all 172.16.0.0/12 md5" >> /etc/postgresql/13/main/pg_hba.conf
diff --git a/experiment/scripts/monitor.sh b/experiment/scripts/monitor.sh
index 33ce041..e368222 100755
--- a/experiment/scripts/monitor.sh
+++ b/experiment/scripts/monitor.sh
@@ -1,3 +1,7 @@
#!/bin/bash
+sed -i "s/<DB_URL_HERE>/${DATABASE_DOMAIN}/g" /etc/monitor/prometheus.yaml
+
+sed -i "s/<PROXY_URL_HERE>/${PROXY_DOMAIN}/g" /etc/monitor/prometheus.yaml
+
systemctl enable prometheus loki
--
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.
- [taler-grid5k] 57/141: configure nginx logging to syslog, (continued)
- [taler-grid5k] 57/141: configure nginx logging to syslog, gnunet, 2021/11/18
- [taler-grid5k] 47/141: fix promtail, add helper function script, gnunet, 2021/11/18
- [taler-grid5k] 45/141: add rsyslog config for better performance with loki, gnunet, 2021/11/18
- [taler-grid5k] 48/141: fix script imports, add net packages, gnunet, 2021/11/18
- [taler-grid5k] 46/141: mulitple exch processes possible - syslog configuredwq, gnunet, 2021/11/18
- [taler-grid5k] 62/141: fix rsyslog and wallet log messages, gnunet, 2021/11/18
- [taler-grid5k] 19/141: add nameserver to /etc/resolv only if not there yet, gnunet, 2021/11/18
- [taler-grid5k] 41/141: fix prometheus, gnunet, 2021/11/18
- [taler-grid5k] 64/141: update promtail - issues with too many open files when too much labels are assigned, gnunet, 2021/11/18
- [taler-grid5k] 72/141: configure postgres, gnunet, 2021/11/18
- [taler-grid5k] 40/141: update configurations,
gnunet <=
- [taler-grid5k] 32/141: add taler default env, gnunet, 2021/11/18
- [taler-grid5k] 67/141: add merchant, some comments in scripts, gnunet, 2021/11/18
- [taler-grid5k] 54/141: add step to build newest wallet from source, gnunet, 2021/11/18
- [taler-grid5k] 56/141: add debug to every taler service, gnunet, 2021/11/18
- [taler-grid5k] 29/141: run dns on taler.net, gnunet, 2021/11/18
- [taler-grid5k] 53/141: clear dns updated, gnunet, 2021/11/18
- [taler-grid5k] 71/141: remove trhottle, gnunet, 2021/11/18
- [taler-grid5k] 33/141: update path, gnunet, 2021/11/18
- [taler-grid5k] 38/141: update image - add prometheus + loki, gnunet, 2021/11/18
- [taler-grid5k] 42/141: move exporters from db and nginx on monitor host, gnunet, 2021/11/18