OpenTelemetry

OpenTelemetry is an open-source observability framework for collecting, processing, and exporting data to a monitoring backend - it can also be configured as a collector agent.

Contents

• OpenTelemetry

  • Linux Installation

  • Configuration

  • OpenTelemetry Dashboard

OpenTelemetry

OpenTelemetry makes a great collector agent because it provides a scalable way to collect and process metrics, traces, and logs in a single pipeline. To use OpenTelemetry as a collector agent, install the Contrib version (otelcol-contrib) instead of the Core version, as it includes extended features needed to forward Graphite metrics.

Linux Installation

Download the OpenTelemetry Contrib Collector package and unpack it. See their installation docs for the latest versions and details on installing to different systems. Packages and files are generally installed at /etc/otelcol-contrib/

Ubuntu/Debian (AMD):

wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.118.0/otelcol-contrib_0.118.0_linux_amd64.deb
sudo dpkg -i otelcol-contrib_0.118.0_linux_amd64.deb

RedHat/CentOS (AMD):

wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.119.0/otelcol-contrib_0.119.0_linux_amd64.rpm && \
sudo rpm -ivh otelcol-contrib_0.119.0_linux_amd64.rpm

Configuration

Locate the configuration file at: /etc/otelcol-contrib/config.yaml and open it in your preferred text editor. To use OpenTelemetry Contrib as a collector agent, you must enable the hostmetrics receiver and carbon exporter. Simply replace the config.yaml file, or include the following on your current config file (make sure to add your HG-API-KEY and HOSTNAME):

receivers:
  hostmetrics:
    collection_interval: 30s
    scrapers:
      cpu: {}
      memory: {}
      load: {}
      filesystem: {}
      disk: {}
      paging: {}
      network: {}
      processes: {}
      process: {}
      system: {}

processors:
  batch: {}
  metricstransform:
    transforms:
      - include: ".*"
        match_type: regexp
        action: update
        new_name: "<HG-API-KEY>.opentel.$$0"
        operations:
          - action: add_label
            new_label: host
            new_value: <HOSTNAME>

exporters:
  carbon:
    endpoint: "carbon.hostedgraphite.com:2003"
    timeout: 10s

service:
  pipelines:
    metrics:
      receivers:
        - hostmetrics
      processors:
        - batch
        - metricstransform
      exporters:
        - carbon

• Save your changes and start the otelcol-contrib service: sudo systemctl restart otelcol-contrib. Within two minutes, tagged system metrics will be forwarded to your Hosted Graphite account (with the ‘opentel’ prefix).

• You can also serve the file manually, to help troubleshoot configuration errors: otelcol-contrib --config /etc/otelcol-contrib/config.yaml

OpenTelemetry Dashboard

Our suggested configuration for OpenTelemetry will return Tagged Metrics to the Hosted Graphite backend. We have a handy dashboard in our Dashboard Library that you can generate to quickly visualize your system performance metrics - per host:

OpenTelemetry Agent Dashboard

If you need help or advice configuring OpenTelemetry as an agent, please send a message to: [email protected] and we would be happy to walk you through the process.