Logging

Hosted Graphite now supports log ingestion and visualization through a managed Loki backend.

Contents:

• Add a Loki Data Source

• Install and Configure a Log Collector

• Query Logs on Your Dashboards

Add a Loki Data Source

Reach out to our support channel and we will create a Loki Key and Data Source for your account. Once created, you can locate your Loki Key in Access ⇒ Sharing:

And you can locate your Loki Data Source in Dashboards ⇒ Connections ⇒ Your Connections:

You will be able to use this Data Source in your dashboards to query and visualize logs that are ingested to our Loki backend (7 day log retention).

Install and Configure a Log Collector

MetricFire supports log forwarding using OpenTelemetry Contrib, Promtail, and Alloy. You must install one of these collectors in your server(s) and this guide uses Promtail - which reads log files like /var/log/syslog and forwards them to our Loki endpoint.

Install Promtail (Ubuntu)

AMD64

wget https://github.com/grafana/loki/releases/download/v3.5.3/promtail_3.5.3_amd64.deb

sudo apt-get install -y ./promtail_3.5.3_amd64.deb

ARM64

Configure Promtail

Edit Promtail’s configuration file, generally located at: /etc/promtail/config.yml

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: https://<YOUR-API-KEY>@www.hostedgraphite.com/logs/sink

scrape_configs:
  - job_name: varlogs
    static_configs:
      - targets:
          - localhost
        labels:
          host: <HOST-NAME>
          job: varlogs
          __path__: /var/log/*log

Make sure to include your HG API-KEY and HOST-NAME of the server. You may also modify the job_name, labels, and path values for more specific collection/querying. Save your config file and restart Promtail to forward your logs.

sudo systemctl restart promtail
sudo systemctl status promtail

You may also need to assign permissions to allow Promtail to read your /var/*log files:

sudo usermod -aG adm promtail
sudo systemctl restart promtail

Query Logs on Your Dashboards

Once Promtail is sending logs and the Loki data source is configured, create a new dashboard panel. You can create custom LogQL queries to visualize your log data in meaningful and useful ways. Add a new panel, set the Data Source to Loki - HostedGraphite, and switch the query editor to "Code" mode.

Basic Log Query

View /var/log/* lines forwarded by your server - you can query by job, host, and/or filename:

{job="varlogs", host="<HOST-NAME>", filename="/var/log/syslog"}

Filter by Content

{job="varlogs", host="<HOST-NAME>"} |= "ERROR"

You can filter using:

• |= substring match

• |~ regex

• |= and != content filtering

Loki supports full LogQL for powerful parsing and field extraction.

Graph Log-Based Metrics

Loki allows you to convert logs into metrics using count_over_time() and similar functions. This turns raw log patterns into quantifiable data that you can graph and alert on. Examples:

Failure Counts per Host:

sum by (host) (
  count_over_time({job="varlogs", host="<HOST-NAME>"} |= "failure" [5m])
)

Cron executions 5m Sum:

sum by (host) (
  count_over_time({job="varlogs", host="<HOST-NAME>"} |= "cron" [5m])
)

Compare Logs and Metrics For Enhanced Visibility

By correlating log data with your existing Graphite metrics, you can observe both system behavior and the underlying events that contribute to changes in performance. This combined view helps you identify patterns to diagnose issues faster, and gain context behind metric fluctuations.

The MetricFire Blog has detailed articles around log + metric monitoring for popular services like MySQL, PostgreSQL, Nginx, Elasticsearch, and many more! Reach out to us at [email protected] to enable Loki Logging for your account!