# Welcome to Hosted Graphite Hosted Graphite by MetricFire will host your monitoring environment so you don't have to! ### Contents * [Graphite](#graphite) * [Hosted Graphite](#hosted-graphite) * [Why You Should Use Hosted Graphite](#why-you-should-use-hosted-graphite) * [A Simple Example](#a-simple-example) * [Simple but Powerful](#simple-but-powerful) {% embed url="" %} ### [Graphite](#graphite) [Graphite](https://graphite.readthedocs.io/en/latest/) is an open-source graphing system for time-series data. It features a storage backend (Whisper), a query language with built-in functions for data manipulation, and a basic web interface for visualizations. Metrics sent to Graphite's **carbon endpoint** are timestamped, stored, can be graphed, and it is a well-documented and user-friendly tool for monitoring and analyzing time-series data. ### [Hosted Graphite](#hosted-graphite)
Hosted Graphite by [MetricFire](https://www.metricfire.com/) has improved upon standard Graphite and offers additional tools and features that provide our customers with a robust and well-rounded monitoring solution. * We handle the **aggregation**, **storage**, and **backups** of your data. We have built-in features like [Data Views](https://docs.hostedgraphite.com/additional-features/data-views) and [Aggregation Rules](https://docs.hostedgraphite.com/additional-features/aggregation-rules), and offer a list of [API's](https://docs.hostedgraphite.com/api-guides) for programatic management of your monitoring environment. * Send [server metrics](/agents-guide) from your running instances, [custom metrics](/language-guide) from within your application code, and [integration metrics](/add-ons-and-integrations-guide) from other popular services that we integrate with, like Heroku, AWS, Azure, and GCP. * Visualize your data in meaningful ways with customizable [dashboards](/dashboard-and-graphs/primary-dashboards) that look awesome! * Configure [alerts](/alerting-guide) to your metrics and integrate with popular [notification channels](/alerting-guide/notification-channels-api) such as Email, Slack, Pager Duty, Microsoft Teams, OpsGenie, or custom Webhooks. * We have fast and friendly [support](mailto:support%40hostedgraphite.com) and are dedicated to making sure you get the highest possible value from our services. #### [Why You Should Use Hosted Graphite](#why-you-should-use-hosted-graphite) * Hosted Graphite takes away the burden of self-hosting your own monitoring solution, allowing you more time and freedom to work on your most important tasks. You can measure, analyze, and visualize large amounts of data from your applications and infrastructure without the hassles of setting up your own servers, worrying about scaling, storing data, alerting, or maintenance. * You need a monitoring solution in order to make data-driven decisions for your business. Get an instant overview of how your technology stack is performing by monitoring your systems and observing trends in your data. If you're serious about your application, you need to be serious about your monitoring too. * Hosted Graphite eliminates common challenges of self-hosted Graphite monitoring, like aggregation, scaling, long-term storage, and query timeouts. Designed for large datasets, our optimized database efficiently handles high-cardinality data structures for faster and more reliable performance. * Hosted Graphite has many supported APIs, integrations, plugins, and features that can help you get data in and configure visualizations/alerts in a way that is most useful for you. Anytime you have questions about our product and services, our knowledgeable support team will respond quickly and give you useful advice. ### [A Simple Example](#a-simple-example) A common monitoring target is the number of concurrent users in a web application. A possible solution is to have a scheduled background task, which once a minute queries the appropriate resource and sends us a metric with the current number of users. Each time we receive a message to a metric namespace (like conc\_users), we store the associated numeric values with a [UNIX](https://en.wikipedia.org/wiki/Unix_time) timestamp. When you create a new [dashboard](https://docs.hostedgraphite.com/dashboard-and-graphs/primary-dashboards), you can start graphing the metric with just a few clicks:

Example of graphing a metric

#### [Simple but Powerful](#simple-but-powerful) Although it lets you get up and running quickly with simple graphs, our [Hosted Dashboards](https://docs.hostedgraphite.com/dashboard-and-graphs/primary-dashboards) are a very powerful tool. You can build complex graphs from multiple metrics, use [Graphite functions](https://graphite.readthedocs.io/en/latest/functions.html) to manipulate the metric data, and have the ability to fine tune the styling and functionality of the graphs. Here’s another example: this time three metrics are being graphed simultaneously, and some styling has been applied to improve clarity. Check out the [graph panel reference](http://docs.grafana.org/features/panels/graph/) to see some of the many available possibilities.

Graphing 3 related metrics

Next, you can see our [Getting Started](/getting-started) guide to learn how to send data to your Hosted Graphite account, and use that data to build **custom** **dashboards** and **alerts**. If you don't already have a Hosted Graphite account, you can start a 14-day free trial [here](https://www.hostedgraphite.com/accounts/signup/). You can also [schedule a product demo here ](https://calendly.com/metricfire-sales/hostedgraphite-demo)to speak with a knowledgeable member of our team who can provide support and advice on getting up and running quickly. # Getting Started An easy-to-follow outline of how to send data to a Hosted Graphite account, and then use that data towards building custom dashboards and alerts to get insight into your infrastructure. ### **Contents** 1. [Understanding the Graphite Metric Format](#understanding-the-graphite-metric-format) 1. [Valid Characters](#valid-characters) 2. [Understanding Data Ingestion, Aggregation, and Retention](#understanding-time-series-data-ingestion-aggregation-and-retention) 3. [Sending Metrics](#sending-metrics) 1. [YOUR API KEY](#your-api-key) 4. [Use Your Data to Create Custom Dashboards](#use-your-data-to-create-custom-dashboards) 5. [Use Your Data to Create Custom Alerts](#use-your-data-to-create-custom-alerts) 1. [Configure Notification Channels](#configure-notification-channels) 6. [Integrate With Popular Services Like AWS, Azure, GCP, and Heroku](#integrate-with-popular-services-like-aws-azure-gcp-and-heroku) 7. [Sit Back and Relax](#sit-back-and-relax) ## [Understanding the Graphite Metric Format](#understanding-the-graphite-metric-format) A Graphite metric is a **unique** **time-series** collection of name:value data that can be aggregated, graphed, and alerted on. In other words, a metric is a **unique namespace** that you can send data or query data from. **We bill on the total number of unique metrics stored within a Hosted Graphite account.** Understanding the metric data format is important as this is how our hosted carbon endpoint can accept data. Hosted Graphite metrics must use the following format: **metric.name \[value] \[timestamp]** * **metric.name:** This is a period-delimited path, such as 'servers.prod.memory.free'. The periods will turn each path component into a sub-tree, like a 'folder' structure. The graphite project website has some useful [metric naming advice](https://graphite.readthedocs.io/en/latest/feeding-carbon.html#step-1-plan-a-naming-hierarchy). * **value:** This is an integer or floating point number. Hosted Graphite can only ingest numeric values. * **timestamp** (optional): This is a [UNIX](https://en.wikipedia.org/wiki/Unix_time) timestamp which is the number of seconds since Jan 1st 1970 (always UTC, never local time). If no timestamp is provided, the current time will be assumed which is generally “good enough” for most uses. Here’s an example of a metric name and numeric value fixed with an optional UNIX timestamp: **test.app.conc\_users 59 1706296444** This means that the metric will be stored as being measured at the time specified, instead of the arrival time at our server. This is useful for sending historical data or when you need the timestamps to match exactly with the timestamps on your system logs. Here’s an example of how to send a metric via TCP using the [Netcat](https://netcat.sourceforge.net/) utility: `echo "YOUR-API-KEY.test.app.conc_users 59" | nc carbon.hostedgraphite.com 2003` Since this example was not fixed with a UNIX timestamp, the **test.app.conc\_users** metric will be time-stamped upon ingestion. #### [Valid Characters](#valid-characters) Metric names can contain only **alphanumeric** and `*`, `.`, `-`, `_`, `$`, `%`, `#`, `=` characters. Any other characters will prevent your metric from being ingested, and will be reported on your [Invalid Metrics ](https://docs.hostedgraphite.com/account-management/account-diagnostics#invalid-metrics-graph)graph in the Account Diagnostics UI. ### [Understanding Data Ingestion, Aggregation, and Retention](#understanding-time-series-data-ingestion-aggregation-and-retention) Once data hits our carbon endpoint, it travels through our ingestion layers, is timestamped, and aggregated before reaching our storage backend. We aggregate and store data at these resolutions: * 5-second data is stored for 1 day (please [contact us](mailto:support@metricfire.com) to request this resolution) * 30-second data is stored for 3 days * 300-second (5min) data is stored for 6 months * 3600-second (1hr) data is stored for 2 years When querying data in your Dashboards, we map the following time spans to the following resolutions: * 0h-1h query displays 5s resolution data (available upon request) * 1h-10h query displays 30s resolution data * 10h-5d query displays 300s resolution data * 5d-2y query displays 3600s resolution data ### [Sending Metrics](#send-metrics) The most common types of metrics sent to Hosted Graphite include **server** metrics (from an agent), **custom** metrics (from within your application code), and **integration** metrics (from a 3rd party service that we integrate with). * To learn more about collecting **server** metrics via using an agent, please refer to our [Agents Guide](https://docs.hostedgraphite.com/agents-guide) for a list of agents that are easy to configure with Graphite. Our agents can collect performance statistics from running processes within your servers. * For information on sending **custom** metrics without an agent, review our [Language Guide](https://docs.hostedgraphite.com/language-guide). This is a guide for configuring custom metrics into your application code using socket connections. * Find out how to send **integration** metrics from popular PAAS services that we integrate with, such as AWS, Azure, GCP, Heroku, and more. Our [Add-Ons and Integrations Guide](https://docs.hostedgraphite.com/add-ons-and-integrations-guide) will help you get started! Information about sending metrics is also outlined in a handy guide that can be located by clicking on the 'Send Metrics' button on the Hosted Graphite account homepage:

HG Public Endpoints

Sending Metrics

Try our CLI-Demo tool to send a sample of performance metrics from your local machine to your HG account: ``` bash -c "$(curl -s 'https://www.hostedgraphite.com/demos/cli_system_collector/?user=current')" ``` These metrics can be visualized in the Welcome to HG dashboard that is automatically generated when you send test metrics from the CLI Demo:

CLI-Demo Tool

#### [YOUR API KEY](#your-api-key) Every Hosted Graphite account has an associated API Key. This API Key **must** be sent along with your metric data to associate it with your account. In most cases, this means prefixing the key to your metric names, e.g: **YOUR-API-KEY.metric.name** which will ensure that your data is securely sent to our public carbon endpoint. Additionally, if you ever need to rotate your old API Key, let us know and we would be happy to create a new one for you. We can also leave both keys active while you rotate them them throughout your codebase, and retire the old key at your convenience. It is best practice to rotate your API Key every two years or so. You can find more information about how to send metrics in our [Sending Metrics Guide](https://docs.hostedgraphite.com/sending-metrics), including details on supported protocols and Graphite tagging. Please reach out to if you need help sending meaningful metrics to help accomplish your monitoring goals. ### [Use Your Data to Create Custom Dashboards](#use-your-data-to-create-custom-dashboards) You can locate a full list of stored metrics using our [Metrics Search UI](https://www.hostedgraphite.com/app/metrics/), or [Metrics API](https://docs.hostedgraphite.com/api-guides/metrics-api):

Metrics Search UI

You can then navigate to Dashboards => + Add New Dashboard, and use your metrics to create queries in dashboard panels:

Add New Dashboard

Wildcard (\*) queries are supported and are useful in grabbing all series from a specified index in the metric path. You can also apply [Graphite functions](https://graphite.readthedocs.io/en/latest/functions.html) to your queries to manipulate the data in many different ways. Some commonly used functions include keepLastValue(), aliasByNode(), groupByNode(), summarize(), scale(), movingAverage() - and there are **many** more included in Graphite's library of functions.

Graphite Functions Example

Additionally, you can configure some more advanced features to your dashboards such as **Variables** and **Annotations**. Read more about this in our [Dashboard Guide](https://docs.hostedgraphite.com/dashboard-and-graphs). ### [Use Your Data to Create Custom Alerts](#use-your-data-to-create-custom-dashboards) In a similar way to building metric queries in your dashboard panels, you can use these metric queries to create alerts in our Graphite Alerts UI. Simply name the alert, then add a metric query and description:

Create alert

Then you can set the **criteria** to trigger an alert anytime the numeric value of the metric reports data outside of its expected bounds:

Set alert criteria

#### [Configure Notification Channels](#configure-notification-channels) The default notification channel will be the **email** associated with your account, but you can configure additional email addresses to send alerts to your preferred email account. You can also send alerts to **Slack**, **PagerDuty**, **Microsoft Teams**, **OpsGenie**, and **VictorOps** or create custom alerting webhooks. Read about this in our [Alerting Guide](https://docs.hostedgraphite.com/alerting-guide) for more details.

Add notification channel

### [Integrate With Popular Services Like AWS, Azure, GCP, and Heroku](#integrate-with-popular-services-like-aws-azure-gcp-and-heroku) It’s never been easier to pull data from services running on your cloud infrastructure! See our [Add-Ons and Integrations ](https://docs.hostedgraphite.com/add-ons-and-integrations-guide)guide for a full list of 3rd party services that we integrate with - like Sentry, New Relic, GitHub, Papertrail, Sitespeed, and more! We even have pre-built dashboards in our [Dashboard Library](https://www.hostedgraphite.com/app/dashboard-library/) that offer a great starting point for many of our integrations:

Dashboard Library

### [Sit Back and Relax](#sit-back-and-relax) You are now well on your way to having better visibility into your infrastructure which will help you quickly identify issues, optimize your systems and resource allocation, and save your team valuable time. You can now [invite team members](https://docs.hostedgraphite.com/account-management/team-access), configure [additional features](https://docs.hostedgraphite.com/additional-features), and use our [APIs](https://docs.hostedgraphite.com/api-guides) to programmatically manage your monitoring environment. If you have any questions or issues, our team is standing by 24/7 to offer fast and friendly assistance. You can contact us at or through the in-app chat feature below, happy monitoring! If you don't already have a Hosted Graphite account, you can start a 14-day free trial [here](https://www.hostedgraphite.com/accounts/signup/). You can also [schedule a product demo here ](https://calendly.com/metricfire-sales/hostedgraphite-demo)to speak with a knowledgeable member of our team who can provide support and advice on getting up and running quickly. # HG-CLI Our recommended command line tool that's fun and easy-to-use - for installing and configuring agents in your systems. Contents * [Intstallation](#installation) * [Quick Install](#quick-install) * [Homebrew](#homebrew) * [Linux](#installation) * [Windows (PowerShell)](#windows-powershell) * [Golang](#golang) * [HG-CLI Commands](#hg-cli-commands) * [Metrics and Visualizations](#metrics-and-visualizations) Our HG command line tool is an interactive way to install an agent on your system, without ever needing to touch a configuration file! This tool currently supports [Telegraf installation](/agents-guide/telegraf) and [OpenTelemetry](/agents-guide/opentelemetry) with more features on the roadmap, like Telegraf Plugin Configuration. Once you've installed HG-CLI, you can run it as a **TUI** (Terminal User Interface) or in **CLI** (Command Line Interface) mode. See our list of commands below for all available installation options and command examples. See our handy blog article [HERE](https://www.metricfire.com/blog/step-by-step-guide-for-using-the-hg-cli-agent-installation-tool/) for step-by-step details on how to get this up in running in just a couple of minutes. ## [Installation](#installation) You can find all current [releases on GitHub](https://github.com/hostedgraphite/hg-cli/releases). ### [Quick Install](#quick-install) Run this command in your terminal (Linux/MacOS/Windows). ```bash curl -s "https://www.hostedgraphite.com/scripts/hg-cli/installer/" | sudo sh ``` This will install the latest released version's binaries into your /usr/local/bin directory. ### [Homebrew](#homebrew) You can use Homebrew to tap and install the HG-CLI packages for MacOS/Linux. This method supports AMD64/ARM64 architectures and the official GitHub tap can be found [HERE](https://github.com/hostedgraphite/homebrew-hgcli). Add tap: ``` brew tap hostedgraphite/hgcli ``` Install: ``` brew install hg-cli ``` ### [Linux](#linux) Locate the official release assets [HERE](https://github.com/hostedgraphite/hg-cli/releases/tag/v0.0.1-alpha), and below are some example installation commands: **Ubuntu/Debian** {% tabs %} {% tab title="AMD64" %} ``` curl -LO https://github.com/hostedgraphite/hg-cli/releases/download/v0.2.0/hg-cli_0.2.0_linux_amd64.deb sudo dpkg -i hg-cli_0.2.0_linux_amd64.deb ``` {% endtab %} {% tab title="ARM64" %} ``` curl -LO https://github.com/hostedgraphite/hg-cli/releases/download/v0.2.0/hg-cli_0.2.0_linux_arm64.deb sudo dpkg -i hg-cli_0.2.0_linux_arm64.deb ``` {% endtab %} {% endtabs %} **RedHat/CentOS/Fedora** {% tabs %} {% tab title="AMD64" %} ``` curl -LO https://github.com/hostedgraphite/hg-cli/releases/download/v0.2.0/hg-cli_0.2.0_linux_amd64.rpm sudo rpm -i hg-cli_0.2.0_linux_amd64.rpm ``` {% endtab %} {% tab title="ARM64" %} ``` curl -LO https://github.com/hostedgraphite/hg-cli/releases/download/v0.2.0/hg-cli_0.2.0_linux_arm64.rpm sudo rpm -i hg-cli_0.2.0_linux_arm64.rpm ``` {% endtab %} {% endtabs %} ### [Windows (PowerShell)](#windows-powershell) Locate the official release assets [HERE](https://github.com/hostedgraphite/hg-cli/releases/tag/v0.0.1-alpha), and below are some example installation commands: {% tabs %} {% tab title="AMD64" %} ```powershell Invoke-WebRequest -Uri "https://github.com/hostedgraphite/hg-cli/releases/download/v0.2.0/hg-cli_0.2.0_windows_amd64.zip" -OutFile "hg-cli.zip" Expand-Archive -Path "hg-cli.zip" -DestinationPath "C:\Program Files\hg-cli" ``` {% endtab %} {% tab title="ARM64" %} ```powershell Invoke-WebRequest -Uri "https://github.com/hostedgraphite/hg-cli/releases/download/v0.2.0/hg-cli_0.2.0_windows_arm64.zip" -OutFile "hg-cli.zip" Expand-Archive -Path "hg-cli.zip" -DestinationPath "C:\Program Files\hg-cli" ``` {% endtab %} {% endtabs %} \[Optional] Add to system PATH: 
$env:BACKUPPATH = [Environment]::GetEnvironmentVariable("Path", "Machine")
$env:NEWPATH = [Environment]::GetEnvironmentVariable("Path", "Machine")
$env:NEWPATH += ";C:\Program Files\hg-cli"
[Environment]::SetEnvironmentVariable("Path", $env:NEWPATH, "Machine")
Update-SessionEnvironment
### [Golang](#golang) Install using `go install` ``` go install github.com/hostedgraphite/hg-cli ``` ### [Manual (Binaries)](#manual-binaries) To manually install, download a preferred version [releases page](https://github.com/hostedgraphite/hg-cli/releases) (that matches your OS & ARCH), extract and move the binary file into a desired install location. Example: 
curl -L https://github.com/hostedgraphite/hg-cli/releases/download/<version>/hg-cli_<version>_<os>_<arch>.tar.gz
tar -xf hg-cli_<version>_<os>_<arch>.tar.gz
mv hg-cli /install/path (eg. /usr/local/bin)
### [Perform Checksum](#perform-checksum-on-downloaded-file) {% hint style="info" %} Optional Step {% endhint %} Download the checksum.txt file from the releases listed assets. {% tabs %} {% tab title="Linux" %} ``` sha256sum --ignore-missing --check checksums.txt ``` {% endtab %} {% tab title="MacOS" %} ``` shasum -a 256 --ignore-missing --check checksums.txt ``` {% endtab %} {% endtabs %} ## [HG-CLI Commands](#hg-cli-commands) Once HG-CLI is installed, you can run it as a **TUI** application, or in **CLI** mode (for automated installs). Below are all available options and command examples for using this tool.
Run the interactive TUI application (**recommended**): ``` hg-cli tui ``` Run it in CLI mode with the **default** configuration: ``` hg-cli agent install telegraf --api-key ``` CLI mode for a custom install - to specify which plugins you want: ``` hg-cli agent install telegraf --api-key --plugins cpu,disk,mem ``` CLI mode to update an API key in an existing config (telegraf): ``` hg-cli agent update-apikey telegraf --api-key --config ``` To uninstall an agent (telegraf): ``` hg-cli agent uninstall telegraf ``` Available flags for CLI mode: * \--api-key (required) * \--plugins (plugins to be added during installation) * \--config (config path for updating apikey - required) * \--list (list available monitoring agents)

hg-cli TUI Demonstration

### [Metrics and Visualizations](#metrics-and-visualizations) Once you have run the HG-CLI and installed an agent, the terminal output will show you that the installation was successful, which plugins were enabled, where your configuration file is located, and the command needed to start the service (**required** for collecting and forwarding metrics to your HG account).

HG-CLI Successful Output

Once the agent service has been started, you'll be able to locate your system metrics in our Metrics Search UI (with the *telegraf* prefix). Once telegraf.\* or otel.\* metrics have been sent to your account, a Telegraf/OpenTelemetry dashboard will be automatically created and can be located in your HG account (Dashboards ⇒ Telegraf/OpenTelemetry). This auto-dash offers a great starting point and can be customized further!

Telegraf Auto-Dash

OpenTelemetry Auto-Dash

If you need help enabling additional plugins or have feedback on the HG-CLI tool, please send a message to If you don't already have a Hosted Graphite account, you can start a 14-day free trial [here](https://www.hostedgraphite.com/accounts/signup/). You can also [schedule a product demo here ](https://calendly.com/metricfire-sales/hostedgraphite-demo)to speak with a knowledgeable member of our team who can provide support and advice on getting up and running quickly. # Sending Metrics Hosted Graphite makes it easy to send and manage different types of metrics from any source. * [Supported Protocols](/sending-metrics/supported-protocols) * Details and examples on sending metrics to Hosted Graphite, per protocol * [Graphite Tag Support](/sending-metrics/graphite-tag-support) * Details on how to send, locate, and graph tagged Graphite metrics * [Metric Management](/sending-metrics/metric-management) * Information on how to search for, delete, and prune your stored metrics * [Metric APIs](https://docs.hostedgraphite.com/api-guides) * See our Metrics API and Tag API which can be used for listing, searching, and deleting your metrics with several useful parameters. {% embed url="" %} Sending metrics video tutorial {% endembed %} # Supported Protocols In this section, we will outline the most common protocols used to send metrics to Hosted Graphite, along with some useful examples. Contents * [Supported Protocols](/sending-metrics/supported-protocols) * [TCP Connection](/sending-metrics/supported-protocols#tcp-connection) * [UDP Packets](/sending-metrics/supported-protocols#udp-packets) * [HTTP POST](/sending-metrics/supported-protocols#http-post) * [TCP Over TLS](#tcp-connection) * [StatsD](/sending-metrics/supported-protocols#statsd) * [Quick Reference](/sending-metrics/supported-protocols#quick-reference) ### [TCP Connection](#tcp-connection) A simple way to send metrics to us is via a TCP connection. First, **ensure each metric name is prefixed by your API key**, then you securely send your metric(s) using the [*Metric Data Format*](/#metric-data-format). **Host**: carbon.hostedgraphite.com **Port**: 2003 It is not necessary (nor recommended) to create a new connection for each metric. You can put multiple metrics on separate lines. A long-lived connection will be faster because you won’t have to wait to establish a new connection each time. The following simple example shows how to send a single TCP metric using the [Netcat](http://netcat.sourceforge.net/) utility: ``` echo "YOUR-API-KEY.test.testing 1.2" | nc carbon.hostedgraphite.com 2003 ``` For programming language-specific examples of sending metrics via TCP, check out our [*Language Guide*](/language-guide). ### [UDP Packets](#udp-packets) For a similar, but non-blocking alternative to TCP, you can use UDP packets to send the metric(s) to us. **Ensure each metric name is prefixed by your API key** ([*see above*](/#api-key)) to securely send your UDP metric(s) using the [*Metric Data Format*](/#metric-data-format). **Host**: carbon.hostedgraphite.com **Port**: 2003 Here’s a simple example using [netcat](http://netcat.sourceforge.net/) on linux: ``` echo "YOUR-API-KEY.test.udp.metric 1.2" | nc -uw0 carbon.hostedgraphite.com 2003 ``` You can send multiple metrics on separate lines as with TCP, but be aware there is a size limit of 8192 bytes for UDP packets. For programming language-specific examples of sending metrics via UDP, check out our [*Language Guide*](https://docs.hostedgraphite.com/language-guide). ### [HTTP POST](#http-post) POST your metrics to the following URL: **** Your metric data should be provided in the [*Metric Data Format*](/#metric-data-format) as the HTTP POST payload. Don’t prefix your metric names with your [*API Key*](/#api-key) - instead use it as the username for Basic Authentication. Here’s an example using [curl](https://curl.haxx.se/) on linux, which lets you provide the API key in the URL. ``` curl https://YOUR-API-KEY@www.hostedgraphite.com/api/v1/sink --data-binary "test.http.metric 1.2" ``` If your data was accepted, you will receive a **HTTP/1.1 202 Accepted** response, with no content body. If your data wasn’t accepted, you’ll receive a 400 error with more information about why. For programming language-specific examples of sending metrics via HTTP POST, check out our [*Language Guide*](https://docs.hostedgraphite.com/language-guide). ### [TCP Over TLS](#tcp-over-tls) If you want, you can send your metrics over TCP using an encrypted tunnel. First, **ensure each metric name is prefixed by your API key** ([*see above*](/#api-key)), then send your metric(s) using the [*Metric Data Format*](https://docs.hostedgraphite.com/#metric-data-format). **Host**: carbon.hostedgraphite.com **Port**: 20030 The following simple example shows how to send a single metric using [Ncat](https://nmap.org/ncat/) on linux: ``` echo "YOUR-API-KEY.conc_users 59" | ncat --ssl carbon.hostedgraphite.com 20030 ``` Alternatively, the following example using [OpenSSL](https://www.openssl.org/) should work out of the box in OS X: ``` echo "YOUR-API-KEY.conc_users 59" | openssl s_client -connect carbon.hostedgraphite.com:20030 ``` For programming language-specific examples of sending metrics via TLS over TCP, check out our [*Language Guide*](https://docs.hostedgraphite.com/language-guide). ### [StatsD](#statsd) StatsD is commonly used as a pre-aggregation service and sends metrics via UDP by default. First, **ensure each metric name is prefixed by your API key**, then send your metric(s) using the [*Metric Data Format*](/#metric-data-format). **Host**: statsd.hostedgraphite.com **Port**: 8125 The following simple example shows how to send a single counter metric using [Ncat](https://nmap.org/ncat/) on linux: ``` echo "YOUR-API-KEY.test.statsd.metric:1.2|c" | nc -u -w1 statsd.hostedgraphite.com 8125 ``` Find more information about using StatsD and Hosted StatsD in our [*Add-ons and Integrations Guide*](/add-ons-and-integrations-guide). **NOTE**: to use the above StatsD endpoint, you must also enable the Hosted StatsD feature for your account (Add-Ons ⇒ Hosted StatsD ⇒ Enable). ### [Quick Reference](#quick-reference) You can send Graphite messages via: * TCP & UDP connections to **carbon.hostedgraphite.com** port **2003** (Metric names *must* be [*API Key*](/#api-key)-prefixed) * TCP over TLS tunnel to **carbon.hostedgraphite.com** port **20030** (Metric names *must* be [*API Key*](/#api-key)-prefixed) * HTTP POST to **** (Metric names are *not* [*API Key*](/#api-key)-prefixed, HTTP 202 response on success) * StatsD to **carbon.hostedgraphite.com** port **8125** (Metric names *must* be [*API Key*](/#api-key)-prefixed) * Python Pickle **carbon.hostedgraphite.com** port **2004** (discussed in the [*language guide*](/language-guide)) **Tip**: Send your metrics to us in a non-blocking/asynchronous manner. Holding up processing to send metrics will skew the data (if it’s being used for performance analysis), and in the worst case have a negative impact on the responsiveness of your app to users. Make sure your Graphite metric names contain only [Valid Characters](https://docs.hostedgraphite.com/getting-started#understanding-the-graphite-metric-format). If you don't already have a Hosted Graphite account, you can start a 14-day free trial [here](https://www.hostedgraphite.com/accounts/signup/). You can also [schedule a product demo here ](https://calendly.com/metricfire-sales/hostedgraphite-demo)to speak with a knowledgeable member of our team who can provide support and advice on getting up and running quickly. # Graphite Tag Support Hosted Graphite supports tagging, allowing metrics to be identified using flexible tag/value pairs instead of a strict hierarchical naming structure. * [Graphite Tag Support](#graphing-tagged-metrics) * [What are Graphite Tags?](#what-are-graphite-tags) * [Why use Graphite Tags?](#why-use-graphite-tags) * [Sending Tagged Metrics](#sending-tagged-metrics) * [Graphing and Alerting on Tagged Metrics](#graphing-tagged-metrics) * [seriesByTag()](#seriesbytag) * [aliasByTags() and groupByTags()](#aliasbytags-and-groupbytags) * [Managing Tagged Metrics](#managing-tagged-metrics) * [Tag API](#tag-api) ### [What are Graphite Tags?](#what-are-graphite-tags) With the release of the 1.1.x series, Graphite now supports [tagged metrics](https://graphite.readthedocs.io/en/latest/tags.html). Rather than using the traditional dotted hierarchy, metrics can be stored and retrieved based on tags - for example, `host=webserver-` or `region=us-east-1`. So where before you might have a name like **myapp.webserver-0001.us-east-1.responses.500**, you could instead have **myapp.responses;host=webserver-0001;region=us-east-1;code=500** ### [Why use Graphite Tags?](#why-use-graphite-tags) It’s becoming common in monitoring systems to use a tagged (or labeled) metric format. This allows for more flexibility both in naming and retrieving metrics. Tags also allow you to categorize metrics based on various dimensions or attributes enabling you to filter and aggregate data in ways that make sense for your analysis. **Note** that when using tags, each metric is uniquely identified by its **name** and set of **tag/value pairs**. As a result, an existing tagged metric cannot be updated to contain new tags. For example, let’s count 500 responses for *My App* in both formats: **Dotted Format** ``` myapp.*.*.responses.500 ``` * Order matters * Can have many node options which can be unwieldy **Tag Format** ``` seriesByTag("path=myapp.responses","code=500") ``` * Order does not matter * You do not always need every node in the metric name to locate one metric or series * Does not support wildcards, but accepts regular expressions The latter allows us to quickly slice and dice metrics by their tags that you get to specify when creating the metric. We don’t have to remember what’s in each position in the dotted hierarchy, and the tags make the query “self documenting”. Tags are particularly useful in modern cloud environments, where e.g. host or instance may change often, while the total number of metrics being stored and queried remains fairly constant. ### [Sending Tagged Metrics](#sending-tagged-metrics) You can use the same Carbon endpoints you already use to send tagged metrics. All you need to do is include the tags and values in the updated Carbon line protocol format (unix timestamp is optional): ``` my.series;tag1=value1;tag2=value2 metric_value (timestamp) ``` E.g. to get started quickly you can use the [netcat](http://netcat.sourceforge.net/) utility as follows: ``` echo "YOUR-API-KEY.foo.bar;tag1=value1;tag2=value2 1.2" | nc carbon.hostedgraphite.com 2003 ``` Sending metrics in your favorite language is similar. Refer to [*our language guide*](/language-guide). ### [Graphing and Alerting on Tagged Metrics](#graphing-tagged-metrics) You can query metrics by their tag values, or by the metric name but **all queries must be wrapped with a seriesByTag() graphite function.** You can do this in your dashboards by toggling the query box to *free-text-mode* and typing out the full query. Or you could begin by adding the seriesByTag() function and filling the boxes with your search terms using the expressions (see table below) where necessary. You can also search for tagged metrics in the HG Tag Search UI, and use the *Explore* feature to open dashboard panels with pre-formatted queries. #### [seriesByTag()](#seriesbytag) You can use any number of tag expressions to filter through your metrics: ``` seriesByTag("name=myapp.response") seriesByTag("name=myapp.response","code=200") seriesByTag("name=myapp.response","code=200","env=production") seriesByTag("name=myapp.response","code=200","env=production","server=~*") ```

HG Tagged Metrics Example

Other graphite functions also work, such as *sumSeries* in the below example: ``` sumSeries(seriesByTag("name=myapp.response","code=200","env=production")) sumSeries(seriesByTag("name=myapp.response","code!=200","env=production")) ```

Tagged Metric Alert

**For a full list of Graphite functions and how to use them, check out the** [**Graphite Function Docs**](https://graphite.readthedocs.io/en/latest/functions.html)**.** seriesByTag() supports any number of tag expressions to refine the results. If using multiple expressions, only series that match all of them will be returned. Expressions have the following formats: | tag!=spec | tag value does not exactly match the spec | | ----------- | ---------------------------------------------------- | | tag=\~spec | tag value matches the regular expression spec | | tag!=\~spec | tag value does not match the regular expression spec | | tag=spec | tag value exactly matches the spec | **Example:** Find all series where server matches the regular expression `0\.*` and env is not staging ``` seriesByTag("server=~0\.*","env!=staging") ``` #### [aliasByTags() and groupByTags()](#aliasbytags-and-groupbytags) These are the ‘tagged’ equivalent of the aliasByNodes() and groupByNodes() Graphite functions. **aliasByTags** requires at least one tag name to be passed in, below we have three: ``` aliasByTags(seriesByTag("server=~0\.*","env!=staging"),"env","server","code") ``` **groupByTags** requires an aggregation method to be passed in (e.g. min, max, avg, sum, etc.), below we use sum: ``` groupByTags(seriesByTag("server=~0\.*", "env!=staging"),"sum","code") ``` For more information about querying tagged metrics refer to the [Graphite Tag Docs](https://graphite.readthedocs.io/en/latest/tags.html#querying) ### [Managing Tagged Metrics](#managing-tagged-metrics) You can easily locate and delete your tagged metrics from within our Tag Search feature. Just select a tag name, search for a value, and a list of matching metrics will be rendered with the option to view or delete any selected metrics.

Tag Search UI

We also offer a Tag Expiry feature that will automatically delete tagged metrics after a defined period of inactivity:

Tagged Metric Expiry

### [Tag API](#tag-api) Check out our tag API documentation [*here*](/api-guides/tag-api). # Metric Management Managing your metrics in the Hosted Graphite application is as simple as locating them and deleting them. Contents * [Metric Management](/sending-metrics/metric-management) * [Finding Your Metrics](#finding-your-metrics) * [Viewing Metrics](#viewing-metrics) * [Deleting Metrics](#deleting-metrics) ### [Finding Your Metrics](#finding-your-metrics) By navigating to the Metrics menu and entering a search term into the box, you can search for your metrics. The \* operator is used to add a wider search to your metric name. In this example “counters.foo\*” searches for any metrics starting with “counters.foo”. The metric search follows the same format as that of the Graphite composer. Additionally, you can click on the number of ‘matches’ to render a full list of metrics that match your search pattern.

Metrics Search Example

* To find your **tagged metrics**, navigate to the Tag Search feature of the Hosted Graphite application. Read more about sending and graphing tagged metrics in our *Tagged Metrics Guide*. ### [Viewing Metrics](#viewing-metrics) Selecting the “Quick view” button next to a metric name will bring you to a quick Graphite graph of that metric’s usage in the last 1 hour (with avg, max, min, sum data views). Clicking the “View in Grafana” button will take you directly to a dashboard panel where you can dig into the metric in more detail.

Quick View Example

### [Deleting Metrics](#deleting-metrics) If you no longer need a metric, you can select and delete it or delete all metrics matching the search pattern you have entered. As we run a large distributed system and deletes have a lower priority than new data coming in, deletes can take up to 30 minutes to be processed fully. In the interim, you should not send the same metric names.

Deleting a metric

**NOTE: A deleted metric will be recreated if you continue to send data to it.** We do not delete any data until it expires naturally, even if the metric namespace has been deleted. So if you send data to a deleted metric, it will map to and render all of its respective data stored in our backend. Knowing this can be useful because if you want to remove your ‘stale’ metrics, you can simply delete all metrics (\*) and let your active metrics be automatically recreated. **Expiring Stale Metrics** See our [Expiry Rules ](/account-management/account-settings#metric-expiry)to target metric patterns for deletion if they have not received data for a certain period of time. This is useful if you have metrics from temporary VM instances, and allows you to expire them after a period of inactivity to reduce your **total** number of stored metrics. # Language Guide In this language guide we’ll provide examples showing you how to send metrics to us in your preferred language. This guide can be used for sending us custom metrics through your application code.
Metric Libraries * Ruby [hosted\_graphite | gem](https://rubygems.org/gems/hosted_graphite) * Python [graphyte | pip](https://pypi.org/project/graphyte/) * NodeJS [graphite | npm](https://www.npmjs.com/package/graphite) * Java [Dropwizard module](http://www.dropwizard.io/1.1.0/docs/manual/configuration.html#graphite-reporter) * StatsD [clients](https://github.com/statsd/statsd/blob/master/docs/client_implementations.md)
.NET * [Interesting .NET Metrics Libraries](https://docs.hostedgraphite.com/pages/RXCgGyQdaF80Xr4i3pLX#interesting-.net-metrics-libraries) * [Sending a metric via UDP](/language-guide/.net#sending-a-metric-via-udp)
Go * [Sending a metric via TCP](/language-guide/go#sending-a-metric-via-tcp) * [Sending a metric via UDP](/language-guide/go#sending-a-metric-via-udp) * [Sending a metric using HTTP](/language-guide/go#sending-a-metric-using-http)
Java * [Interesting Java Metrics Libraries](/language-guide/java#interesting-java-metrics-libraries) * [Java DNS Behaviour](/language-guide/java#java-dns-behaviour) * [Sending a metric via TCP](/language-guide/java#sending-a-metric-via-tcp) * [Sending a metric via UDP](/language-guide/java#sending-a-metric-via-udp) * [Sending a metric via HTTP POST](/language-guide/java#sending-a-metric-via-http-post) * [Google App Engine HTTP Post](/language-guide/java#google-app-engine-http-post)
Javascript * [Interesting Javascript Metrics Libraries](/language-guide/javascript#interesting-javascript-metrics-libraries)
Node.js * [Sending a metric via TCP](/language-guide/node.js#sending-a-metric-via-tcp) * [Sending a metric via UDP](/language-guide/node.js#sending-a-metric-via-udp) * [Sending a metric via StatsD](https://docs.hostedgraphite.com/language-guide/node.js#sending-a-metric-via-statsd) * [Sending a metric vis HTTP POST](https://docs.hostedgraphite.com/language-guide/node.js#sending-a-metric-via-http-post)
PHP * [Interesting PHP Metrics Libraries](/language-guide/php#interesting-php-metrics-libraries) * [Sending a metric via TCP](/language-guide/php#sending-a-metric-via-tcp) * [Sending a metric via UDP](/language-guide/php#sending-a-metric-via-udp) * [Sending a metric via UDP on Heroku](/language-guide/php#sending-a-metric-via-udp-on-heroku)
Postman * [Sending a metric via HTTP POST](/language-guide/postman#sending-a-metric-via-http-post)
Python 2.x * [Interesting Python Graphite Libraries](/language-guide/python-2.x#interesting-python-graphite-libraries) * [Sending a metric via TCP](/language-guide/python-2.x#sending-a-metric-via-tcp) * [Sending a metric via UDP](/language-guide/python-2.x#sending-a-metric-via-udp) * [Sending a metric using HTTP POST](/language-guide/python-2.x#sending-a-metric-using-http-post) * [Sending a metric via TCP over TLS tunnel](/language-guide/python-2.x#sending-a-metric-via-tcp-over-tls-tunnel)
Python 3.x * [Sending a metric via TCP](/language-guide/python-2.x#sending-a-metric-via-tcp-1) * [Sending a metric via UDP](/language-guide/python-2.x#sending-a-metric-via-udp-1) * [Sending a metric using HTTP POST](/language-guide/python-2.x#sending-a-metric-using-http-post-1) * [Sending a metric via TCP over TLS tunnel](/language-guide/python-2.x#sending-a-metric-via-tcp-over-tls-tunnel-1) * [Sending a metric via StatsD](https://docs.hostedgraphite.com/language-guide/python-2.x#python-3.x)
Python Pickle * [Carbon Relay](/language-guide/python-pickle#carbon-relay) * [Carbon Relay Secured With Stunnel](/language-guide/python-pickle#carbon-relay-secured-with-stunnel)
Ruby * [Installing the Ruby Gem](/language-guide/ruby#installing-the-ruby-gem) * [Using the Gem to send via TCP](/language-guide/ruby#using-the-gem-to-send-via-tcp) * [Using the Gem to send via UDP](/language-guide/ruby#using-the-gem-to-send-via-udp) * [Using the Gem to send via HTTP](/language-guide/ruby#using-the-gem-to-send-via-http) * [Using the Gem to send via StatsD](/language-guide/ruby#using-the-gem-to-send-via-statsd) * [Basic Ruby](/language-guide/ruby#basic-ruby) * [Sending a metric via TCP](/language-guide/ruby#sending-a-metric-via-tcp) * [Sending a metric via UDP](/language-guide/ruby#sending-a-metric-via-udp) * [Sending a metric via HTTP POST](/language-guide/ruby#sending-a-metric-via-http-post) * [Sending a metric via StatsD](https://docs.hostedgraphite.com/language-guide/ruby#sending-a-metric-via-statsd)
Shell * [Sending a metric via TCP](/language-guide/shell#sending-a-metric-via-tcp) * [Sending a metric via UDP](/language-guide/shell#sending-a-metric-via-udp) * [Sending a metric via TCP over TLS](/language-guide/shell#sending-a-metric-via-tcp-over-tls) * [Sending a metric via HTTPs](/language-guide/shell#sending-a-metric-via-https) * [Sending a metric via StatsD](https://docs.hostedgraphite.com/language-guide/shell#sending-a-metric-via-statsd) * [Send multiple metrics with Curl](/language-guide/shell#send-multiple-metrics-with-curl)
Typescript * [Sending a metric via TCP](https://docs.hostedgraphite.com/language-guide/typescript#sending-a-metric-via-tcp) * [Sending a metric via UDP](https://docs.hostedgraphite.com/language-guide/typescript#sending-a-metric-via-udp) * [Sending a metric via HTTP POST](https://docs.hostedgraphite.com/language-guide/typescript#sending-a-metric-via-http-post) * [Sending a metric via StatsD](https://docs.hostedgraphite.com/language-guide/typescript#sending-a-metric-via-statsd)
# Metric Libraries Want to send metrics directly from your application? Here are some simple libraries that you can use to send metrics from code. * Ruby [hosted\_graphite | gem](https://rubygems.org/gems/hosted_graphite) * Python [graphyte | pip](https://pypi.org/project/graphyte/) * NodeJS [graphite | npm](https://www.npmjs.com/package/graphite) * Java [Dropwizard graphite reporter](https://www.dropwizard.io/en/stable/manual/configuration.html#graphite-reporter) * StatsD [clients](https://github.com/statsd/statsd/blob/master/docs/client_implementations.md) # .NET How to send metrics using .NET ### Interesting .NET Metrics Libraries * **A simple .NET library for Graphite and StatsD**\ - A simple .NET library to collects stats and pass them on to Graphite * **A .NET library to send data to Hosted Graphite**\ Client library to send statistics to HostedGraphite.com using .Net, a work-in-progress ### Sending a metric via UDP ``` var endPoint = new IPEndPoint(Dns.GetHostAddresses("carbon.hostedgraphite.com")[0], 2003); var bytes = Encoding.ASCII.GetBytes("YOUR-API-KEY.foo.net-udp 1.2\n"); var sock = new Socket(AddressFamily.InterNetwork,SocketType.Dgram, ProtocolType.Udp) { Blocking = false }; sock.SendTo(bytes, endPoint); ``` Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # Go How to send metrics using GO ### Sending a metric via TCP ```go import ( "fmt" "net" ) ... conn, err := net.Dial("tcp", "YOUR-UID.carbon.hostedgraphite.com:2003") _, err = fmt.Fprintf(conn, "YOUR-API-KEY.foo.go-tcp 1.2\n") // don't forget to handle the error and close connection ``` ### Sending a metric via UDP ```go import ( "fmt" "net" ) ... conn, err := net.Dial("udp", "YOUR-UID.carbon.hostedgraphite.com:2003") _, err = fmt.Fprintf(conn, "YOUR-API-KEY.foo.go-udp 1.2\n") // don't forget to handle the error and close connection ``` ### Sending a metric using HTTP ``` import ( "bytes" "encoding/base64" "net/http" ) ... req, err := http.NewRequest("PUT", "https://www.hostedgraphite.com/api/v1/sink", bytes.NewBufferString("foo.go-http 1.2\n")) base64_apikey := base64.StdEncoding.EncodeToString([]byte("YOUR-API-KEY")) req.Header.Add( "Authorization", "Basic "+ base64_apikey ) client := http.Client{} reply, errdo := client.Do(req) // don't forget to handle the error and close connection ``` Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # Java How to send metrics using Java Contents * [Java](broken://pages/1jTcOzP3JGZ3DkyCL2pr) * [Interesting Java Metrics Libraries](#interesting-java-metrics-libraries) * [Java DNS Behaviour](#java-dns-behaviour) * [Sending a metric via TCP](#sending-a-metric-via-tcp) * [Sending a metric via UDP](#sending-a-metric-via-udp) * [Sending a metric via HTTP POST](#sending-a-metric-via-http-post) * [Google App Engine HTTP Post](#google-app-engine-http-post) ### [Interesting Java Metrics Libraries](#interesting-java-metrics-libraries) * **Coda Hale’s metrics library** - Developed by Yammer to instrument their JVM-based backend services, it’s a really comprehensive Java metrics library and works well with Hosted Graphite. * **Metrics-StatsD for Coda Hale’s metrics library** ### [Java DNS Behaviour](#java-dns-behaviour) Java’s default JVM behaviour includes an optimisation/attempt to prevent DNS cache poisoning whereby it will read DNS entries for a service and then cache them forever. **This is something that will result in problems with Hosted Graphite**. To service our hundreds of customers we run a **lot** of different machines and we swap machines in and out all the time. This means that if you have cached the DNS entry for a machine that no longer exists, you’re going to drop a lot of data. To prevent this, you can set the JVM cache behaviour to only cache entries for 60 seconds: ``` java.security.Security.setProperty("networkaddress.cache.ttl" , "60"); ``` ### [Sending a metric via TCP](#sending-a-metric-via-tcp) ```java import java.io.DataOutputStream; import java.net.Socket; ... Socket conn = new Socket("YOUR-UID.carbon.hostedgraphite.com", 2003); DataOutputStream dos = new DataOutputStream(conn.getOutputStream()); dos.writeBytes("YOUR-API-KEY.foo.java-tcp 1.2\n"); conn.close(); ``` ### [Sending a metric via UDP](#sending-a-metric-via-udp) ```java import java.net.DatagramSocket; import java.net.DatagramPacket; import java.net.InetAddress; ... DatagramSocket sock = new DatagramSocket(); InetAddress addr = InetAddress.getByName("YOUR-UID.carbon.hostedgraphite.com"); byte[] message = "YOUR-API-KEY.foo.java-udp 1.2\n".getBytes(); DatagramPacket packet = new DatagramPacket(message, message.length, addr, 2003); sock.send(packet); sock.close(); ``` ### [Sending a metric via HTTP POST](#sending-a-metric-via-http-post) ```java import java.util.Base64; import javax.net.ssl.HttpsURLConnection; import java.io.*; import java.net.URL; // Assumes the reader will try/catch the appropriate exceptions, and clean up // The connection when they're done with it! String data = "foo.java-http 1.2"; URL url = new URL("https://www.hostedgraphite.com/api/v1/sink"); HttpsURLConnection connection = (HttpsURLConnection)url.openConnection(); String key = "YOUR_API_KEY"; String authHeader = Base64.getEncoder().encodeToString(key.getBytes()); connection.setRequestProperty("Authorization", "Basic " + authHeader); connection.setRequestMethod("POST"); connection.setRequestProperty("Content-Length", String.valueOf(data.getBytes().length)); connection.setUseCaches (false); connection.setDoInput(true); connection.setDoOutput(true); OutputStream wr = connection.getOutputStream(); wr.write(data.getBytes("UTF-8")); wr.flush(); wr.close(); ``` ### [Google App Engine HTTP Post](#google-app-engine-http-post) ```java URLFetchService fetcher= URLFetchServiceFactory.getURLFetchService(); String data = "foo 1.2"; String key = "YOUR_API_KEY"; String authHeaderString = "Basic " + Base64.encodeBase64String(key).getBytes("ISO-8859-1"); HTTPHeader authHeader = new HTTPHeader("Authorization", authHeaderString); HTTPRequest request = new HTTPRequest("https://www.hostedgraphite.com/api/v1/sink", HTTPMethod.POST); request.getFetchOptions().setDeadline(10); request.setHeader(authHeader); request.setPayload(data.getBytes()); HTTPResponse response = fetcher.fetch(request); ``` Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # Javascript How to send metrics using Javascript ### Interesting Javascript Metrics Libraries * **jquery.graphite.js**\ - A JQuery plugin for graphite that allows you to quickly embed graph images. # Node.js How to send custom metrics using Node.js ### Sending a metric via TCP ```javascript var net = require("net"); var socket = net.createConnection(2003, "YOUR-UID.carbon.hostedgraphite.com", function() { socket.write("YOUR-API-KEY.foo.node-tcp 1.2\n"); socket.end(); }); ``` ### Sending a metric via UDP ```javascript // For Nodejs v10 var dgram = require("dgram"); var message = Buffer.from("YOUR-API-KEY.foo.node-udp 1.2\n") var client = dgram.createSocket("udp4"); client.send(message, 0, message.length, 2003, "YOUR-UID.carbon.hostedgraphite.com", function(err, bytes) { client.close(); }); ``` ### Sending a metric via StatsD ```javascript var dgram = require("dgram"); var message = Buffer.from("YOUR-API-KEY.foo.node-statsd:1.2|c\n") var client = dgram.createSocket("udp4"); client.send(message, 0, message.length, 8125, "statsd.hostedgraphite.com", function(err, bytes) { client.close(); }); ``` ### Sending a metric via HTTP POST ```javascript const http = require('http'); const apiKey = 'YOUR-API-KEY'; const metric = 'test.testing-node-http'; const value = 1.2; const metricData = `${metric} ${value}`; const basicAuthHeader = 'Basic ' + Buffer.from(apiKey).toString('base64'); const options = { hostname: 'www.hostedgraphite.com', path: '/api/v1/sink', method: 'POST', headers: { 'Content-Type': 'text/plain', 'Content-Length': Buffer.byteLength(metricData), 'Authorization': basicAuthHeader } }; const req = http.request(options, () => {}); req.write(metricData); req.end(); ``` Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # PHP How to send metrics using PHP ### Interesting PHP Metrics Libraries * **PHP StatsD**\ - A PHP StatsD client available in Composer\ - A simple PHP StatsD client ### Sending a metric via TCP ```php ``` ### Sending a metric via UDP ```php ``` ### Sending a metric via UDP on Heroku ```php ``` Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # Postman How to send metrics using Postman ### Sending a metric via HTTP POST 1. Open Postman. 2. Create a new request, set its type to *POST* and its *URL* to `https://www.hostedgraphite.com/api/v1/sink`
3. Go to the *Authorization* tab. For the authorization, type select *Basic Auth*. In the *username* field, enter your Hosted Graphite *API key*.
4. Go to the *Body* tab. Select *raw*, and enter a datapoint for your metric (you can send multiple datapoints as long as they are on separate lines). You may also include an optional Unix timestamp with your datapoint. A datapoint has the following parameters: ` `
5. Click the *Send* button to send your datapoint. When your datapoint has been sent successfully, you will receive a response with status *202 Accepted*.
Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # Python 2.x How to send metrics using Python ### Interesting Python Graphite Libraries * **A Python module for sending metrics to Graphite over UDP**\ - Charlie wrote this library, and we use it internally. It’s a good simple library for sending metrics over UDP in Python. * **Diamond - Send system metrics to Graphite**\ [*Diamond*](/agents-guide/diamond) - A really useful Python daemon to send system metrics such as CPU usage, load, Disk IO, etc. ### Sending a metric via TCP ``` import socket conn = socket.create_connection(("YOUR-UID.carbon.hostedgraphite.com", 2003)) conn.send("YOUR-API-KEY.foo.py-tcp 1.2\n") conn.close() ``` ### Sending a metric via UDP ``` import socket sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.sendto("YOUR-API-KEY.foo.py-udp 1.2\n", ("YOUR-UID.carbon.hostedgraphite.com", 2003)) ``` ### Sending a metric using HTTP POST ``` import urllib2, base64 url = "https://www.hostedgraphite.com/api/v1/sink" api_key = "YOUR-API-KEY" request = urllib2.Request(url, "foo.py-http 1.2") request.add_header("Authorization", "Basic %s" % base64.encodestring(api_key).strip()) result = urllib2.urlopen(request) ``` ### Sending a metric via TCP over TLS tunnel ``` import socket, ssl s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ssl_sock = ssl.wrap_socket(s) ssl_sock.connect(("YOUR-UID.carbon.hostedgraphite.com", 20030)) ssl_sock.write("YOUR-API-KEY.foo.py-tls 1.2\n") ``` Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. ## Python 3.x ### Sending a metric via TCP ```python import socket conn = socket.create_connection(("YOUR-UID.carbon.hostedgraphite.com", 2003)) conn.send("YOUR-API-KEY.foo.py3-tcp 1\n".encode('utf-8')) conn.close() ``` ### Sending a metric via UDP ```python import socket sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.sendto("YOUR-API-KEY.foo.py3-udp 1\n".encode('utf-8'), ("YOUR-UID.carbon.hostedgraphite.com", 2003)) ``` ### Sending a metric using HTTP POST With the requests module: ```python import requests response = requests.put("http://www.hostedgraphite.com/api/v1/sink", auth = ("YOUR-API-KEY", ""), data = "foo.py3-http 1") ``` Using only stdlib modules: (Contributed by [Waldo](https://github.com/gwaldo/HostedGraphite_Python3), thanks!) ```python import urllib.request from base64 import b64encode url = "https://www.hostedgraphite.com/api/v1/sink" api_key = b'YOUR-API-KEY' metric = "foo 1.2".encode('utf-8') headers = {'Authorization': b'Basic ' + b64encode(api_key)} request = urllib.request.Request(url, metric, headers) result = urllib.request.urlopen(request) ``` ### Sending a metric via TCP over TLS tunnel ```python import socket, ssl s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ssl_sock = ssl.wrap_socket(s) ssl_sock.connect(("YOUR-UID.carbon.hostedgraphite.com", 20030)) ssl_sock.write("YOUR-API-KEY.foo.py3-tls 1.2\n".encode("UTF-8")) ``` ### Sending a metric via StatsD ```python import socket, ssl import socket statsd_host = 'statsd.hostedgraphite.com' statsd_port = 8125 metric_key = 'YOUR-API-KEY.foo.py3-statsd' metric_value = 1.2 metric_message = f"{metric_key}:{metric_value}|c" sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.sendto(metric_message.encode('utf-8'), (statsd_host, statsd_port)) sock.close() ``` When sending StatsD metrics, make sure to specify counters (c), gauges (g), or timers (ms). Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # Python Pickle Intended for slightly more advanced users, the pickle protocol is a more efficient take on the plaintext protocol, and supports sending metrics to Hosted Graphite in batches. Pickled data forms a list of multi-level tuples: ``` [(path, (timestamp, value)), ...] ``` You’ll need to pack your pickled data into a packet containing a simple header: ``` payload = pickle.dumps(listOfMetricTuples, protocol=2) header = struct.pack("!L", len(payload)) message = header + payload ``` Once you’ve formed a list of sufficient size and pickled it, like `message` above, send the data over a TCP socket to `carbon.hostedgraphite.com:2004`. Your API Key is prefixed to the metricname as per regular Python methods: ``` "YOUR-API-KEY.my.metric.name" ``` If you have any trouble or any questions, [contact us](mailto:help%40hostedgraphite.com). ### Carbon Relay If you would prefer to use a carbon-relay daemon to send metric data to Hosted Graphite, just configure it to use `carbon.hostedgraphite.com:2004` as a `DESTINATION` in your `carbon.conf` and `relay-rules.conf` files. `carbon.conf`: ``` [relay] ... RELAY_METHOD = rules DESTINATIONS = carbon.hostedgraphite.com:2004 ``` `relay-rules.conf`: ``` [default] default = true destinations = carbon.hostedgraphite.com:2004 ``` If you’re currently using the carbon-cache daemon and would like to utilize carbon-relay to send your metrics to us and also to your own carbon instance, you will have to send metrics to carbon-relay instead of carbon. To do this, you have to configure carbon-relay to listen on the port that your current carbon-cache is listening. Then you will need to configure carbon-cache to listen on a new port. You can then add your carbon-cache instance as a `DESTINATION` to have carbon-relay forward all metrics to Hosted Graphite AND your local carbon-cache current carbon.conf: ``` [cache] ... LINE_RECEIVER_PORT = 2003 PICKLE_RECEIVER_PORT = 2004 ... [relay] ... RELAY_METHOD = rules DESTINATIONS = carbon.hostedgraphite.com:2004 ``` This should be changed to: `new carbon.conf`: ``` [cache] ... LINE_RECEIVER_PORT = 2013 PICKLE_RECEIVER_PORT = 2014 ... [relay] ... RELAY_METHOD = rules DESTINATIONS = carbon.hostedgraphite.com:2004, 127.0.0.1:2014 LINE_RECEIVER_PORT = 2003 PICKLE_RECEIVER_PORT = 2004 ``` `relay-rules.conf` ``` [default] default = true destinations = carbon.hostedgraphite.com:2004, 127.0.0.1:2014 ``` If you’re doing this and you’d like to avoid prefixing every metric with your API key, please [contact us to discuss alternatives](mailto:help%40hostedgraphite.com), including IP whitelisting, reach out for our live chat support. ### Carbon Relay Secured With Stunnel If you want to encrypt the metric data from your Carbon Relay, you can use Stunnel. Carbon-Relay should be configured to send metrics to Stunnel over localhost To install Stunnel: ``` sudo apt-get install stunnel or sudo yum install stunnel ``` **Configuring Stunnel** Stunnel will need a configuration file. By default, Stunnel will look at `/etc/stunnel/stunnel.conf`. You may need to create this file yourself. Put the following lines inside `/etc/stunnel/stunnel.conf`: ``` foreground = yes [hg-tls] client = yes accept = 20041 connect = 72c4f55b.carbon.hostedgraphite.com:20040 verify = 2 CAfile = /etc/ssl/certs/ca-bundle.crt #CApath = /etc/ssl/certs ``` * `foreground = yes` This shows the degbug logs. You may want to remove this line once Stunnel is configured and running correctly. * `accept = ` Carbon relay must be configured to send metrics to ``. * connect = 72c4f55b.carbon.hostedgraphite.com:20040 Hosted Graphite will accept Stunnel connections on port 20040 * `CAfile / CApath` You only need one of these. You will need to provide a path to your ssl certificates. * `CAfile` expects a certificate file. Some of the default locations of these files are: ``` "/etc/ssl/certs/ca-certificates.crt" // Debian/Ubuntu/Gentoo etc. "/etc/pki/tls/certs/ca-bundle.crt" // Fedora/RHEL "/etc/ssl/ca-bundle.pem" // OpenSUSE "/etc/pki/tls/cacert.pem" // OpenELEC ``` * CApath expects a directory containing certificates named `XXXXXXXX.0` where `XXXXXXXX` is the hash value of the DER encoded subject of the cert. You can read more about Stunnel configuration options [here](https://www.stunnel.org/static/stunnel.html). **Reconfiguring Carbon-Relay** Carbon relay needs to send metrics to Stunnel now. `carbon.conf`: ``` [relay] ... DESTINATIONS = localhost:20041:stunnel ``` `relay-rules.conf`: ``` [default] default = true destinations = localhost:20041:stunnel ``` Now start Stunnel: ``` stunnel ``` **Note:** If your package manager downloaded stunnel4, you may have to start stunnel with: ``` stunnel4 ``` # Ruby How to send metrics using Ruby ### Installing the Ruby Gem A [Hosted Graphite Ruby Gem](https://rubygems.org/gems/hosted_graphite) is available on github, and can be installed directly by typing: ```ruby gem install hosted_graphite ``` ### Using the Gem to send via TCP ```ruby HostedGraphite.protocol = HostedGraphite::TCP HostedGraphite.send_metric('foo.rb-tcp', 1.2) ``` ### Using the Gem to send via UDP ```ruby HostedGraphite.protocol = HostedGraphite::UDP HostedGraphite.send_metric('foo.rb-udp', 1.2) ``` ### Using the Gem to send via HTTP ```ruby HostedGraphite.protocol = HostedGraphite::HTTP HostedGraphite.send_metric('foo.rb-http', 1.2) ``` ### Using the Gem to send via StatsD ```ruby require 'hosted_graphite' require 'statsd-ruby' HostedGraphite.api_key = '' HostedGraphite.protocol = :statsd HostedGraphite.('foo.rb-statsd', 1.2) ``` ### Basic Ruby Here are some ways to send custom metric via TCP, UDP, HTTP, and StatsD directly from your code. ### Sending a metric via TCP ```ruby require 'socket' conn = TCPSocket.new 'YOUR-UID.carbon.hostedgraphite.com', 2003 conn.puts "YOUR-API-KEY.foo 1.2\n" conn.close ``` ### Sending a metric via UDP ```ruby require 'socket' sock = UDPSocket.new sock.send "YOUR-API-KEY.foo 1.2\n", 0, "YOUR-UID.carbon.hostedgraphite.com", 2003 ``` ### Sending a metric via HTTP POST ```ruby require 'net/http' uri = URI("https://www.hostedgraphite.com/api/v1/sink") api_key = "YOUR-API-KEY" req = Net::HTTP::Post.new(uri.request_uri) req.basic_auth api_key, nil req.body = "foo 1.2" res = Net::HTTP.start(uri.host, uri.port) do |http| http.request(req) end ``` ### Sending a metric via StatsD ```ruby require 'socket' statsd_host = 'statsd.hostedgraphite.com' statsd_port = 8125 metric_key = 'YOUR-API-KEY.foo.ruby-statsd' metric_value = 1.2 metric_message = "#{metric_key}:#{metric_value}|" sock = UDPSocket.new sock.send(metric_message, 0, statsd_host, statsd_port) sock.close ``` **NOTE**: when sending standard ruby StatsD metrics, be sure to define counters (c), gauges (g), or timers (ms). Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # Shell How to send metrics using Shell Contents * [Shell](/language-guide/shell) * [Sending a metric via TCP](#sending-a-metric-via-tcp) * [Sending a metric via UDP](#sending-a-metric-via-udp) * [Sending a metric via TCP over TLS](#sending-a-metric-via-tcp-over-tls) * [Sending a metric via HTTPs](#sending-a-metric-via-https) * [Sending a metric via StatsD](#sending-a-metric-via-statsd) * [Sending multiple metrics with Curl](#sending-multiple-metrics-with-curl) ### [Sending a metric via TCP](#sending-a-metric-via-tcp) Using the [netcat](http://netcat.sourceforge.net/) utility ``` echo "YOUR-API-KEY.foo 1.2" | nc carbon.hostedgraphite.com 2003 ``` ### [Sending a metric via UDP](#sending-a-metric-via-udp) Using the [netcat](http://netcat.sourceforge.net/) utility ``` echo "YOUR-API-KEY.foo 1.2" | nc -uw0 carbon.hostedgraphite.com 2003 ``` ### [Sending a metric via TCP over TLS](#sending-a-metric-via-tcp-over-tls) Using the [Ncat](https://nmap.org/ncat/) utility ``` echo "YOUR-API-KEY.con_users 1.3" | ncat --ssl carbon.hostedgraphite.com 20030 ``` Alternatively, the following example using [OpenSSL](https://www.openssl.org/) should work out of the box in OS X: ``` echo "YOUR-API-KEY.con_users 1.3" | openssl s_client -connect carbon.hostedgraphite.com:20030 ``` ### [Sending a metric via HTTPs](#sending-a-metric-via-https) Using the [Curl](https://curl.haxx.se/) utility ``` curl https://YOUR-API-KEY@www.hostedgraphite.com/api/v1/sink --data-binary "foo 1.2" ``` ### [Sending a metric via StatsD](#sending-a-metric-via-statsd) Using the [Ncat](https://nmap.org/ncat/) utility: ``` echo "YOUR-API-KEY.foo.statsd:1.2|c" | nc -u -w1 statsd.hostedgraphite.com 8125 ``` For StatsD, make sure to specify counters (c), gauges (g), or timers (ms). ### [Sending multiple metrics with Curl](#send-multiple-metrics-with-curl) ``` curl https://YOUR-API-KEY@www.hostedgraphite.com/api/v1/sink --data-binary @data.txt ``` where data.txt contains multiple metrics and values on a separate line: Your API key can be found on your [account home](https://www.hostedgraphite.com/accounts/profile/) page. # TypeScript Send custom metrics using TypeScript ### Sending a metric via TCP ```typescript import * as net from 'net'; const socket = net.createConnection({ host: 'carbon.hostedgraphite.com', port: 2003 }, () => { socket.write('YOUR-API-KEY.foo.ts-tcp 1.2\n'); socket.end(); }); ``` ### Sending a metric via UDP ```typescript import * as dgram from 'dgram'; const socket = dgram.createSocket('udp4'); socket.send('YOUR-API-KEY.foo.ts-udp 1.2\n', 2003, 'carbon.hostedgraphite.com', () => { socket.close(); }); ``` ### Sending a metric via HTTP POST ```typescript import * as https from 'https'; const apiKey = 'YOUR-API-KEY'; const metric = 'foo.ts-http'; const value = 1.2; const metricData = `${metric} ${value}`; const basicAuthHeader = 'Basic ' + Buffer.from(apiKey).toString('base64'); https.request({ hostname: 'www.hostedgraphite.com', path: '/api/v1/sink', method: 'POST', headers: { 'Content-Type': 'text/plain', 'Content-Length': metricData.length, 'Authorization': basicAuthHeader } }, () => {}).end(metricData); ``` ### Sending a metric via StatsD ```typescript import * as dgram from 'dgram'; const socket = dgram.createSocket('udp4'); const apiKey = 'YOUR-API-KEY'; const metricName = 'test.testing-typescript-statsd'; const metricValue = 1.2; const message = `${apiKey}.${metricName}:${metricValue}|`; socket.send(message, 8125, 'statsd.hostedgraphite.com', () => { socket.close(); }); ``` As with any TypeScript (.ts) file, compile it to JavaScript using tsc: `tsc filename.ts` and then execute the compiled script using Node.js: `node filename.js`. # Dashboard and Graphs In this guide we will demonstrate how to build and manage custom dashboards, as well as offer different dashboard options that can be used with Hosted Graphite. * [Primary Dashboards](/dashboard-and-graphs/primary-dashboards) * [Dashboard Library](/dashboard-and-graphs/dashboard-library) * [Local Dashboard Integration](/dashboard-and-graphs/local-dashboard-integration) * [Worldmap Panel](/dashboard-and-graphs/worldmap-panel) * [Graphite Dashboard Guide](/dashboard-and-graphs/graphite-dashboard-guide) * [Graphite Graph Menu Reference](/dashboard-and-graphs/graphite-graph-menu-reference) * [Other Dashboard Options](/dashboard-and-graphs/other-dashboard-options) # Primary Dashboards Our hosted dashboard service provides meaningful visualizations for your data and is completely customizable {% embed url="" %} Full-featured, interactive dashboards come standard with all Hosted Graphite [plans](https://www.hostedgraphite.com/signup/)! In addition to the steps below, we also have a comprehensive video tutorial on dashboard creation that can be seen [here](https://www.youtube.com/watch?v=jarOI0iRiD0). ### Create and Query To create a new dashboard, hover over the dashboard icon on the left-side menu and select `+ New Dashboard`. Then click the `+ Add Visualization` button to use the Query UI to search/select metrics from your datasource (default: HostedGraphite):

Create and Query

### Apply Functions Add a [Graphite function](https://graphite.readthedocs.io/en/latest/functions.html) to manipulate queries and represent your data in a way that is most meaningful to you. Below, we are demonstrating how to use **summarize()** to sum the data by 1hr intervals, **exclude()** to omit a metric pattern, and **aliasByNode()** to reformat the metric name:

Apply Graphite Functions

### Customize Using the menu on the right, you can name the panel and modify the display stylings, axis, change the visualization type, add thresholds, units of measurement, etc:

Customize Panel

### Timeframe To zoom in on each graph you can use the time range controls, highlight the section of a graph you wish to zoom in on, or define a custom timeframe. **NOTE**: each timeframe maps to an aggregation resolution (1h-10h ⇒ 30s data, 10h-5d ⇒ 300s, 5d-2y ⇒ 3600s). You can also set an auto-refresh rate to have your graphs update automatically - which is useful if you have a dedicated display for monitoring your graphs:

Timeframes

### Variables Click the 'settings' icon at the top, and click on 'Variables' then 'Add Variable'. Define a name, set the query type to 'Metric Name Query', and query your datasource (default is HostedGraphite) with a wildcard pattern (\*). You will also need to replace the query in your panel with your new variable name, prefixed with the dollar symbol ($). Now you can toggle between metric views that match your variables:

Dashboard Variables

### Annotations Click the settings icon at the top, click 'Annotations' and then 'Add Annotation Query'. You will name your annotation, select your datasource (default: HostedGraphite), choose a color (default: red), and add a Graphite event tag. In most cases, you will need to copy a webhook key from one of our [Add-Ons](https://docs.hostedgraphite.com/add-ons-and-integrations-guide) into the platform you wish to pull event annotations from. In this case, we have copied a webhook key into a Heroku account and are using the 'heroku' tag to display Heroku events as annotations:

Dashboard Annotations

### Organize and Manage To organize your panels in rows, simply click on the `Add` icon, click 'Row', and name the row. You can drag/drop panels into these collapsable rows, and then save the dashboard by clicking the 'floppy disk' icon at the top. Now your newly saved dashboard can be searched, starred, tagged, and added to a playlist:

Manage Dashboards

### Create Unlimited Dashboards We provide a 14-day [free trial](https://www.hostedgraphite.com/accounts/signup/), where you can experiment with our beautiful hosted dashboards - with no installation or setup. [Book a demo](https://calendly.com/metricfire-sales/hostedgraphite-demo) and talk to us directly about how to build custom dashboards that will help you visualize your data in meaningful ways. For more information, check out some of our publications around monitoring Graphite data with our hosted dashboards on the MetricFire [blog](https://www.metricfire.com/blog/grafana-dashboards-from-basic-to-advanced/). # Dashboard Library We supply several open source templates for you so you don’t have to start from scratch for various services. This makes for a great starting point and can be customized further. The **HG Traffic Dashboard** is included with every new account. This dashboard uses **\_hg\_meta** metrics, which are used for tracing traffic and are provided by us at no extra charge. To generate a dashboard from one of our templates, go to the Dashboard Library page, click on and select one of the options. * **Welcome to HG** * **HG Traffic** * **HG Agent** * **Telegraf** * **Kubernetes Overview** * **Heroku** * **Heroku Postgres** * **Heroku Redis** * **Heroku Kafka** * **AWS Overview** * **Azure Overview** * **GCP Overview** * **collectd** * **Sitespeed** * **OpenTelemetry** * **StatsD Traffic** * **DigitalOcean**
* For AWS, automatic dashboards will be created for individual services that you select during your AWS => HG configuration. We include automatic dashboards for CloudFront, DynamoDB, EBS, EC2, Elasticache, ELB, RDS, Redshift, Route53, and SQS services. * For New Relic, automatic dashboards will be created during your NR => HG configuration. We include automatic dashboards for New Relic Server, and New Relic Browser Overview. * The Telegraf and StatsD Traffic dashboards will be automatically created if metrics prefixed with 'telegraf' or 'statsd' are ingested. * If you need a dashboard that is not included in this library, please reach out to us through the in-app chat feature below, and we'll help you build it! # Local Dashboard Integration Dashboards come standard with all Hosted Graphite plans which is accessible through the Hosted Graphite app. If you have a local dashboard installation such as Grafana and wish to use Hosted Graphite as a data source, you can accomplish this in a few easy steps. ### Allow permission to access your Hosted Graphite data with an Access Key In the Hosted Graphite App menu, click on **Access** and select **Access Keys**. Click on **Add Access Key** in the upper right-hand corner. Give your Access Key a name and select **Graphite** (this will give the external application access to the Graphite data).

Create graphite access key

Copy the Access Key URL and read more details about Access Keys [here](/account-management/access-keys). ### Add the Hosted Graphite Data Source to a local Grafana Instance In the Grafana Dashboard menu, navigate to Connections => Data Sources and select **Graphite**. Name your new data source and paste the Access Key in the URL box, (e.g: ), then click **Save & Test:**

Local Grafana configuration

***Note**: Checking the default box will make the Hosted Graphite data source the primary data source when building new panels.* More information about Graphite Data Sources is documented [here](http://docs.grafana.org/features/datasources/graphite/). ### Select your new Data Source and Render Hosted Graphite Metrics Create a new dashboard panel in your local Grafana environment, select your new Hosted Graphite data source, and enter a metric query:

Local Grafana example

# Worldmap Panel The Worldmap Panel is a tile map of the world that can be overlaid with circles representing data points from a metric query. **Geomap Panel** In versions of Grafana **> 8**, Worldmap has been renamed to [Geomap](https://grafana.com/docs/grafana/latest/panels-visualizations/visualizations/geomap/) and requires a few additional parameters to properly render data from your Graphite metrics. This example shows how to use the built-in Gazetteer for US states but you can also lookup coordinates by country, or define coordinates: * Add a **Reduce** transformation to the query: * Mode: Series to Rows * Calculations: Total * Update map layers in panel edit mode: * Location Mode: Lookup * Lookup Field: Field * Gazetteer: USA States

Transform Query

**Worldmap Panel (deprecated)**

Worldmap Example

Data points are linked to locations on the map by matching part of the metric name to a key in the set of ‘Location Data’. Included location data: * Countries (2 letter codes) * Countries (3 letter codes) * US states * Probes (a [*selection of cities*](/dashboard-and-graphs/worldmap-panel#city-keys-included-in-probes)) This works by matching country codes (like US or GB or FR) or US state codes (TX or NY) to a node or a wildcard in a metric namespace. If there is a match in the list of countries or states then a circle will be drawn at the location. The size of the circle depends on the value of the matched metric. Circle size is relative e.g. if you have 3 countries with values 1, 2, and 3 or 100, 200, and 300 then you will get one small circle, one medium circle, and one large circle. ### Query Example Use the **aliasByNode** function to point to the field containing the country code: aliasByNode(worldmap-metric.country.US, 2) ### Map Visual Option Settings
ItemDescription
CenterThis setting configures the default center of the map. There are 5 centers to choose from or you can choose a custom center. For a custom center, there are two fields: latitude and longitude. Examples of values are 37.09024, -95.712891 for the center of the US, or 55.378051, -3.435973 for Great Britain.
Initial Zoom:The initial zoom factor for the map. This is a value between 1 and 18 where 1 is the most zoomed out.
Min Circle Size:This is minimum size for a circle in pixels.
Max Circle Size:This is the maximum size for a circle in pixels. Depending on the zoom level you might want a larger or smaller max circle size to avoid overlapping.
Unit:The unit is shown in the popover when you hover over a circle. There are two fields the singular form and the plural form. E.g. visit/visits or error/errors
Show Legend:Shows/hide the legend on the bottom left that shows the threshold ranges and their associated colours.
### Map Data Options #### **Location Data** There are four location data options for the Worldmap panel:
ItemDescription
countriesThis is a list of all the countries in the world. It works by matching a country code (US, FR, AU) to a node alias in a time series query.
countries_3letterThe same as above, but with 3 letter codes (USA, FRA, AUS).
statesSimilar to countries but for the states in USA e.g. CA for California
probesThis is a selection of cities around the world.
#### **City keys included in probes** ``` taipei london los-angeles new-jersey portland south-carolina sydney charleroi frankfurt new-york san-francisco seattle singapore amsterdam chicago miami paris sao-paulo silicon-valley tokyo atlanta dallas omaha ``` #### **Threshold Options** Thresholds control the colour of the circles. If one value is specified then two colours are used. For example, if the threshold is set to 10 then values under 10 get the first colour and values that are 10 or more get the second colour. The threshold field also accepts 2 or more comma-separated values. For example, if you have 2 values that represent 3 ranges that correspond to the three colours. For example: if the thresholds are 70, and 90 then the first colour represents < 70, the second colour represents between 70 and 90 and the third colour represents > 90. **Geomap Panel** In versions of Grafana > 8, Worldmap has been renamed to [Geomap](https://grafana.com/docs/grafana/latest/panels-visualizations/visualizations/geomap/) and requires a few additional parameters to properly render data from your Graphite metrics. * Add a **Reduce** transformation to the query: * Mode: Series to Rows * Calculations: Total * Update map layers in panel edit mode: * Location Mode: Lookup * Lookup Field: Field * Gazetteer: USA States
# Graphite Dashboard Guide Contents * [Graphite Dashboard Guide](/dashboard-and-graphs/graphite-dashboard-guide) * [Creating a Graph](#creating-a-graph) * [Choosing a Time Range](#choosing-a-time-range) * [Graph Labeling and Styling](#graph-labeling-and-styling) * [Creating a Graphite Dashboard](#creating-a-graphite-dashboard) * [Saving and Loading Dashboards](#saving-and-loading-dashboards) * [Advanced Dashboard Configuration](#advanced-dashboard-configuration) **NOTE**: We have removed Graphite Composer from the HG application in favor of Grafana. If you require Graphite Composer for any reason, please reach out to us and we will activate it in your account. In the following guide, we’ll walk you through creating a simple Graphite graph for a chosen metric and adding that to a dashboard that you can bookmark for quick access. If you haven’t sent any metrics to Hosted Graphite yet, check out our [*Sending Metrics Guide*](/sending-metrics). [*In this example*](/#a-simple-example), we’re using a metric called `conc_users` - the number of users logged into our imaginary application. We have been sending the metric value (such as the one below) once a minute to Hosted Graphite for the last day. ```sh conc_users 1215 ``` ### [Creating a Graph](#creating-a-graph) Let’s set up a graph for our `conc_users` metric. Log into Hosted Graphite, navigate to Dashboards, and select **Graphite Composer**. When that opens, you’ll be presented with a blank composer dialog. On the tree on the left-hand side, expand the ‘Graphite’ folder. This will contain all the metrics sent using the [*API Key*](/#api-key) associated with your account. At the moment, there’s only one entry - our demo `conc_user` metric, so we click on that.

Choosing our conc_user metric

**Tip**: If you’re going to be sending a wide range of metrics to a Hosted Graphite account, this ‘Graphite’ folder could soon have an unmanageable number of entries. Using periods in your [*metric names*](/#metric-data-format) will introduce a folder structure, allowing for logical grouping of your metrics. E.g. `servers.mario.memory.free` will result in `Graphite\servers\mario\memory\free` in the tree above. See the [Graphite Project website](https://graphite.readthedocs.io/en/latest/feeding-carbon.html#step-1-plan-a-naming-hierarchy) for more info. When we click on our conc\_users metric, Graphite will render a graph of a recent selection that metric data:

A graph of the conc_user metric after clicking on the conc_user entry

### [Choosing a Time Range](#choosing-a-time-range) The next thing we do is to choose how much of our data we’re going to sample. We have two options, via the following icons:

‘Select A Date Range‘ and ‘Select Recent Data‘ buttons

* **Select A Date Range** Choose a start date/time and an end date/time. This is useful for debugging a new metric, or for investigating an incident that occurred during a specific time window. * **Select Recent Data** View a rolling window of the most recent metric data, up to a specified number of minutes/hours/days/weeks/months/years. **Tip** - If you make changes but the graph doesn’t update automatically, use the **Update Graph** button to trigger a refresh of the graph.

‘Update Graph‘ Button

In this example, we want a view of the last 12 hours of the conc\_users metric, so click **Select Recent Data** and enter the appropriate values. ### [Graph Labeling and Styling](#graph-labeling-and-styling) There are many options to make your data more presentable. At the bottom of the composer window, you’ll find a group of menus. These contain various functions to let you apply data transformations, labeling, and styling to your graph:

Composer Menus

In this example, we’ll use only a few of the many available options. You can find more information about the menus in the [*graph menu guide*](/dashboard-and-graphs/graphite-graph-menu-reference). Let’s make the following adjustments to our graph: * **Graph Options** -> **Line Mode** -> **Connected Line**Makes our graph a continuous line through our available metric points. * **Graph Options** -> **Graph Title**Sets the title to `Site Visitors`. * **Graph Options** -> **Display** -> **Line Thickness**Set this value to `3` (pixels) for a slightly thicker line than the default. * **Graph Options** -> **X-Axis** -> **TimeZone**Metric timestamps are sent in UTC. The server (in this example) sending the metrics is in Dublin - Set this to `Europe/Dublin` to apply the appropriate offset to the X-axis timestamps. * **Graph Options** -> **Y-Axis** -> **Label**Set the vertical axis label to `Users`. This results in the following:

A improved graph of the conc_user metric

Click on the floppy disk icon in the composer toolbar to save. You’ll be prompted for a name, enter `Site Visitors`, and click OK. Your graph is now saved and can be recalled at any time by browsing **My Graphs** in the left-hand window of the Composer. ### [Creating a Graphite Dashboard](#creating-a-graphite-dashboard) The final step in this tutorial is to create a bookmarkable Graphite dashboard and add your graph for display. Having followed the tutorial above, we will have a new `Site Visitors` graph in our account. Navigate to Dashboards -> Graphite Composer -> Dashboard in your Hosted Graphite account. When you first open the dashboard page, you’ll be presented with a blank dashboard. Go to **Graphs** -> **New Graph** -> **From Saved Graph**

Loading our saved Graph

In the dialog that opens, an entry called `Site Visitors` will appear. Choose it and then click **Select**. Your graph will appear in the dashboard. Let’s make our graph a little bigger. Click on **Graphs** -> **Resize**, and choose the `Custom` option. Enter in a width of `800` and a height of `500` (or whatever suits your display). **Tip** You can further customize the graphs for this dashboard instance by clicking a graph once to focus it, and clicking once more to bring up a configuration menu (This is the same menu we used when creating the graph - See [*Graphite Graph Menu Reference*](/dashboard-and-graphs/graphite-graph-menu-reference) for more info).

Our Final Dashboard!

### [Saving and Loading Dashboards](#saving-and-loading-dashboards) To save your finished dashboard, click **Dashboard** -> **Save As** and enter a name, e.g. `MainSiteDash`. To get a bookmarkable URL (which will directly open this dashboard instance), click on **Share** in the menu. A dialog will appear (you may be asked to save again) containing a URL that you can copy. Finally, you can see all dashboards that you own by clicking **Dashboard** -> **Finder**. A dialog will appear with a list of all the dashboards that you have saved. ### [Advanced Dashboard Configuration](#advanced-dashboard-configuration) **Graph Data** In [*Creating A Dashboard*](#creating-a-graphite-dashboard), we created a simple graph with some basic styling. Graphite offers many more configuration options. You have the ability to show multiple metrics in the same graph, apply transformations to the metric data, or combine metrics before rendering. All of these advanced features are found via the **Graph Data** button: *Graph Data* button Each time a metric is selected (in the [*tree of metrics*](#creating-a-graph)), an entry called a ‘target’ is added to the Graph Data dialog. For example, had we clicked on metric **\servers\smithers\users\_authed** and then metric **\servers\smithers\users\_guest** in the composer, the **Graph Data** dialog would contain two targets as follows (which would each result in a line on the graph):

Graph Data button

#### **Wildcards** Sometimes we want to display groups of common metrics. Rather than manually adding each metric as a target, we can use wildcards to specify multiple metrics as one target. Using the **Edit** button in the Graph Data dialog, we can manually specify the target as in the following examples: * To graph all the metrics under the **smithers** server: Set the target as **servers.smithers.\*** * To graph the **users\_authed** metric on all servers: Set the target as **servers.\*.users\_authed** **Graph Performance** If you have many metrics stored with us, and your graphs are taking a long time to load, chances are that your [*wildcard use*](#wildcards) is the first place you should look. Try to avoid wildcards such as the following: ``` servers.*.*.cpu ``` In this case, every single metric matched by this wildcard will be queried, even if many of those metrics don’t have data for the selected time range. Multiple, but narrower targets in a graph may result in faster load times than one widely defined target. In our example above, perhaps all we care about are our metrics regarding public-facing machines, but our wildcard is also including metrics from our production or staging machines. In this scenario, we replace the single target with two narrower targets, e.g.: ``` servers.production.*.cpu servers.staging.*.cpu ``` **Data Transformations** Now that we’ve defined *which* metrics we’re interested in, we can perform transformations on them. Back in the **Graph Data** dialog, when we select a target (or multiple targets), we can use the **Apply Function** menu to alter or combine our targets in some way. When a function is applied to a target, you’ll notice that the target’s text changes, eg ```sh before: original_target_text after: functionName(original_target_text) or functionName(original_target_text, optional_params..) ``` Eg, if we selected *conc\_users*, and clicked **Apply Function** –> **Transform** –> **Derivative**, the target text would now say The graph should automatically update to show the derivative of our *conc\_users* metric. If not, click the refresh button. If several transformations are applied to a target, the function calls are nested, with the most recent one on the outside. E.g. ```python lastFunction(secondFunction(firstFunction(original_target))) ``` To undo a function you just applied, click the target and use the **Undo Function**. The outermost function will be removed from the target text. You don’t have to use the **Apply Function** menu - you may also select your target, click **Edit**, then manually edit the target text by hand. A quick summary of the **Apply Function** menu is as follows. Some functions are used on multiple targets (either by selecting multiple entries in the dialog, using wildcards, or both). More information about functions can be found on the *Functions* page on the [Graphite Docs Page](https://graphite.readthedocs.io/). * **Combine** Combining the data points from multiple metrics using functions such as **sum()** * **Transform** Create a new set of data points after applying a transformation such as derivation or timeshifting to the input. * **Calculate** Perform running calculations on the data points such as a moving average, or forecasts. * **Filter** Discard, limit or maintain counts/maximums/minimums of the data points. * **Special** Miscellaneous functions, mostly contain routines that control the appearance of that target on the graph - For example setting a legend name or line color. # Graphite Graph Menu Reference The Graphite composer is a very powerful tool, with many options. We’ll document some of the more commonly used visualization options here. **NOTE**: We have removed Graphite Composer from the HG application in favor of Grafana. If you require Graphite Composer for any reason, please reach out to us and we will activate it in your account. See the [Graphite docs](https://graphite.readthedocs.io/en/latest/functions.html) for a full list of functions that you can apply to your metric queries. * **Graph Options** * **Graph Title** Set the title at the top of the graph. * **Display** Control fonts, colors, and other stylings as well as toggling display of items such as grid lines and axes. * **Line Mode** Control how adjacent data points for a metric are connected: > * **Slope Line** - The graph line through adjacent data points results in a sloped path. > * **Staircase Line** - The graph line through adjacent data points results in a ‘stepped’ path. > * **Connected Line** - Creates a continuous sloped path, filling gaps between data points (see note below). > * **Draw Null As Zero** - Assume that a missing metric value for a given time point is the same as that metric equaling 0. *Note* - For ‘Slope Line’ and ‘Staircase Line’ it is assumed that data points are close enough together or that the graph resolution is coarse enough to result in a connected path. Using ‘Connected Line’ is similar to ‘Slope Line’ but renders a continuous path. For those wishing to use ‘Staircase Line’ but are seeing gaps, you can apply the [Keep Last Value](https://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.keepLastValue) function (**Graph Data** -> (choose metric) -> **Apply Function** -> **Special** -> **Keep Last Value**). * **Area Mode** Control how the area under a graph path is filled: > * **None** - No fill. > * **First Only** - The graph of the first metric (if you have several) has a solid fill underneath. > * **Stacked** - All graphs lines are filled underneath, with a different fill color for each metric. The order in which the metrics are added to the graph determines the fill order. * **X-Axis** > * **Time Format** - Apply a date format to the timestamps using [Python’s strftime() format](http://docs.python.org/library/time.html#time.strftime). > * **Timezone** - Apply the appropriate offset to the timestamp values based on the provided timezone, e.g. America/Los\_Angeles or Europe/Dublin. * **Y-Axis** > * **Label** - Apply a label to the vertical axis. > * **Minimum** - Values below this will not be included in the graph. > * **Maximum** - Values above this will not be included in the graph. > * **Minor Lines** - How many of the minor grid lines will be rendered. > * **Unit** - Control unit rendering (e.g., 1K instead of 1000). > * **Side** - Whether to render the Axis on the left or the right side of the graph. * **Graph Data** This menu allows you to add multiple metrics to your graph (or even the same metric multiple times). It also allows you to perform transformations on your metric data before rendering, allowing for comparisons or more complex analyses of your data. There are far too many possibilities to document in this beginner’s guide. (A simple example would be graphing both “Authenticated” vs “Anonymous” users at the same time. Another example could be to add our `conc_users` data twice, but apply a -1 year time shift to the second instance, so we could compare traffic this time last year. The possibilities are endless!). * **Auto Refresh** When enabled, the graph will automatically refresh. # Other Dashboard Options Below are some alternatives to our Hosted Dashboards Using the [*access keys*](/account-management/access-keys) features, you can allow an external dashboard or alerting system to pull data from Hosted Graphite. Here are some other dashboard options: * **Graphene**\ - A nice dashboard running on Rails using D3 to render the graphs. * **Graphiti**\ - Using Sinatra and Redis to replace the Graphite web app. * **Tasseo**\ - We run this natively, but you might want to run Jason Dixon’s Tasseo dashboard yourself for some reason. * **GDash**\ - Another Sinatra app, using Twitter Bootstrap for layout. * **Configuration:** In `dash.yaml`, add ``` :graphite: https://www.hostedgraphite.com/your/accesskey/ ``` # Alerting Guide In this guide, we’ll show you how to use Hosted Graphite alerting to configure custom alerts to your metrics and integrate with our supported notification channels. * [Alerting Overview](/alerting-guide/alerting-overview) * [Alerts API](/alerting-guide/alerts-api) * [Notification Channels API](/alerting-guide/notification-channels-api) * [Scheduled Mutes API](/alerting-guide/scheduled-mutes-api) * [Using Your Own Alerting](/alerting-guide/using-your-own-alerting) # Alerting Overview Alerts allow you to receive a notification when your data does something unexpected, such as go above or below a set threshold, or stop suddenly. Contents * [Alerting Overview](/alerting-guide/alerting-overview) * [Creating An Alert](#creating-an-alert) * [Alert Name and Metric](#alert-name-and-metric) * [Alert Criteria Panel](#alert-criteria-panel) * [Composite Alerts](#composite-alerts) * [Notification Panel](#notification-panel) * [Email](#email) * [PagerDuty](#pagerduty) * [Slack](#slack) * [Microsoft Teams](#microsoft-teams) * [VictorOps](#victorops) * [OpsGenie](#opsgenie) * [Webhook](#webhook) * [Notification JSON Model](#notification-json-model) * [Alert States](#alert-states) * [Managing An Alert](#managing-an-alert) * [Scheduled Mutes](#scheduled-mutes) * [Troubleshooting Your Alerts](#troubleshooting-your-alerts) > **You can create alerts in Hosted Graphite, Hosted Grafana, or both. However, we offer limited support into Grafana Alerting as it operates on a separate alerting engine that we do not manage.** Hosted Graphite's internal alerting system also has a quicker response time because it is triggered from values upon ingestion, rather than upon render. ### [Creating An Alert](#creating-an-alert)

Create an Alert

#### [Alert Name and Metric](#alert-name-and-metric) From within your Hosted Graphite account, click the “Alert” icon to open the alert creation panel. > * **Alert Name** > > This name is used in notifications. It is a reminder of why you added it, so make it clear and descriptive! e.g. “EU Servers CPU usage”. > * **Graphite Alerting Metric** > > This queries the data that is tested against your criteria: > > * **Wlidcard** patterns are accepted, evaluated, and a list of Triggered Metrics are returned > * **Tagged Metrics** can also be used for alerting metrics, e.g. seriesByTag("name=myapp.response","code=400") > * **Alert Info** > > Alert message sent with notifications. Can contain arbitrary strings like description of the alert, resolution steps to follow, or links to documentation. It is **recommended** to check your alerting metric with the “Check Metric Graph” button to confirm it is rendering the data that you expect. When you’re finished, proceed to the Alert Criteria tab. #### [Alert Criteria Panel](#alert-criteria-panel) There are three ways to define the criteria that will result in a notification being sent. * **Below / Above a Threshold** If you just enter one of the above or below values, it will check whichever one you use. This is useful when there’s an upper or lower bound that this data should not cross. You can evaluate the metric to trigger after crossing the threshold **FOR** x minutes, or **EVER**. * **Missing** An alert notification will be sent to you if the metric does not arrive at all for a certain time period. This is useful for detecting when a system goes down entirely. * **Outside of Bounds** An alert notification will be sent if the metric data you’ve selected goes either above the “above” threshold, or below the “below” threshold. This is useful when your data fits inside an expected range, and can be configured to trigger after crossing the threshold FOR x minutes, or EVER. #### [Composite Alerts](#composite-alerts) Composite alerts allow you to combine multiple conditions into a single alert using AND / OR logic, and up to four conditions can be configured. This helps reduce alert fatigue by correlating multiple metric evaluations before triggering a notification. So instead of alerting on one noisy metric in isolation, composite alerts allow you to build higher-confidence alerts around overall service health. For example: * Condition A: CPU usage is elevated * Condition B: request latency is increasing * Condition C: database connection count is high These conditions can then be evaluated together using AND / OR expressions such as: * `A && B` * `A || B || C` * `(A && B) || C` * `A || (B && C)` Each condition also supports the same alert criteria types available in standard alerts (above, below, missing, outside of bounds). Composite alerts still support wildcard metric queries. When wildcard conditions are used, all matching metrics are evaluated and a Triggered Metrics list is sent within the alert notification payload. This additional context makes it easier to identify exactly which services, hosts, or metric groups contributed to the alert state.
**Service-Level Alerting** Composite alerts allow a more **service**-level approach to alerting by combining multiple metric **signals** into a single evaluation. This helps reduce noisy alerts while improving signal quality and incident visibility. For example, a temporary CPU spike on a single host may not represent a real incident. However, elevated CPU combined with increased latency and database saturation may indicate a genuine service degradation event worth notifying on. For a deeper overview of this alerting philosophy and real-world examples from our internal alerting infrastructure, see our case study [HERE](https://www.metricfire.com/blog/a-real-world-graphite-alerting-case-study-reducing-noise-at-metricfire/). #### [Notification Panel](#notification-panel) Defining a notification channel allows you to receive notifications when an alert triggers. Below are the available notification channel types, and create new ones on your [Notification Channel Page](https://www.hostedgraphite.com/app/alerts/notification-channels/). Click the '+ Add Channel' button to configure a new alerting channel, then 'Save' and apply them to any of your Graphite alerts. “If the query fails” lets you control the behavior if the graphite function query fails. This option only appears for alerts that use graphite functions as part of their metrics. Graphite function queries can fail due to timeouts from matching too many metrics, being malformed, or if it returns duplicate metrics due to aliasing. * **Notify me** A notification is sent when the query fails with a description of the reason. * **Ignore** Notifications are ignored but the alert still changes state and the failure is visible in the event history log. "Alerting Notification Interval" lets you control how often you want to be notified of an alert: * **On state change**\ A notification will be sent only when the alert transitions state from healthy to triggered - or vice versa. An alert that continues alerting will not send subsequent notifications. * **Every** A notification will be sent each time the alert triggers and recovers. Subsequent notifications will then be paused for the configured time period. This allows you to stop the ‘flapping’ behavior that would give you lots of notifications in a short period of time. #### [Email](#email) Send one or multiple emails to your team when an alert is triggered.
#### [PagerDuty](#pagerduty) Send your alerts to your centralized PagerDuty incident monitoring and alerting system. Reference the PD documentation to create/locate your service or integration key required to configure this notification channel.
#### [Slack](#slack) Send an immediate notification to one of your Slack channels. This requires a Slack Webhook endpoint for your channel, see the [Slack documentation](https://slack.com/apps/new/A0F7XDUAZ-incoming-webhooks) for details on how to create this.
#### [Microsoft Teams](#microsoft-teams) Send alerts to your chosen Microsoft Teams channel by creating a Microsoft Teams webhook. See their [documentation](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook?tabs=dotnet) for details on creating a Teams webhook.
#### [VictorOps](#victorops) (Now owned by Splunk) Send your alerts into your VictorOps hub to integrate with all your existing monitoring and alerting infrastructure. See the Splunk [documentation](https://help.victorops.com/knowledge-base/escalation-webhooks/) for more details on creating escalation webhooks.
#### [OpsGenie](#opsgenie) Send alerts to this incident management tool allowing your team to respond to incidents, outages, and other events. Reference the OpsGenie [docs](https://support.atlassian.com/opsgenie/docs/integrate-opsgenie-with-hosted-graphite/) for details around api key management.
#### [Webhook](#webhook) Allows you to configure your own webhook that we will notify with real-time information on your defined alerts. See the JSON format below as that is the data reported to all notification channels, including your webhook URL.
#### [Notification JSON Model](#notification-json-model) Each notification will be JSON encoded in the following format: ```json { "name": "The name of the triggered alert.", "criteria": "The defined alert criteria for the alert.", "graph": "PNG of the rendered graph.", "value": "The last evaluated value of the alerting metric.", "metric": "The graphite metric that triggered an alert.", "logical_expression": "The AND / OR expression used to evaluate composite alerts.", "condition_met": { "A": ["Metric query rule for condition A"], "B": ["Metric query rule for condition B"] }, "triggered_metrics": [ "List of metrics contributing to the triggered alert state." ], "status": "The current status of the alert.", "backoff_minutes": false | 123, "info": null | "Info saved with the alert." } ``` **NOTE**: Composite alerts include additional notification payload fields for **logical expressions**, and **conditions met**. When wildcard queries are used, the triggered metrics list contains the metrics that contributed to the alert state. Slack notifications also include a Condition Matrix view which displays the evaluation status of each configured condition, making it easier to understand exactly why the composite alert triggered.
### [Alert States](#alert-states) Your alerts are listed in the [Alert Overview](https://www.hostedgraphite.com/app/alerts/) section of the Hosted Graphite application. We list them in four categories: > * **Healthy Alerts** > > Alerts that are currently running and within acceptable boundaries. > * **Triggered Alerts** > > Alerts that are currently running and outside acceptable boundaries, this alert will have already notified you via the set notification channel. > * **Muted Alerts** > > Alerts which have been silenced manually or by schedule. These alerts will not notify you until they become active again. > * **Inactive Alerts** > > Alerts that use graphite function metrics but have failed due to the query taking taking too long, being malformed or returning duplicate metrics due to aliasing. **NOTE**: Manually updating an alert will reset the state to *Healthy*. ### [Managing An Alert](#managing-an-alert) From the Alert Overview page, you can hover your mouse over an individual alert to see actions related to managing it.

Managing an Alert

* **View an Alert** Click the eye icon to open the overview popup for an alert. This displays an embedded graph and a history log of the last 3 days of data. There is also a link to the dashboard composer allowing you to view more detailed information on the metric being alerted on. From within the dashboard composer view, alert events will be displayed as annotations. You can hover over the base of the annotation to see the details of the alerting event.

View an Alert

* **Edit an Alert** An alert can be edited to change its metric, criteria, or notification channel and changes may take several minutes to take effect. Updating alert criteria will place it back into the ‘Healthy’ list in the Graphite Alerts UI, but does not change the state of the alert. * **Mute an Alert** An alert can be silenced from notifying you for a specified time period. Currently, the available times are 30 mins, 6hrs, 1 day, and 1 week. * **Delete an Alert** An alert can be deleted from your panel here and this action is irreversible. If an alert was built within the Dashboard UI, you will be unable to edit or delete it from within the Hosted Graphite UI. Feel free to contact our [support](mailto:support%40hostedgraphite.com) for advice on managing alerts using the Hosted Graphite [*alert API*](/alerting-guide/alerts-api), or the [Dashboard API](https://grafana.com/docs/grafana/v7.5/http_api/alerting/). ### [Scheduled Mutes](#scheduled-mutes) Defining a scheduled mute allows you to silence alerts on a one-time or recurring basis for scheduled maintenance or downtime. You can see the available scheduled mutes and add new ones in the [Alerts UI](https://www.hostedgraphite.com/app/alerts/scheduled-mutes/). Once a scheduled mute is created, it must be attached to alerts so that they may be silenced by the scheduled mute - this can be done at the alert [create](#creating-an-alert) and [update](/alerting-guide/alerts-api#updating-alerts) endpoints, or the Hosted Graphite [UI](https://www.hostedgraphite.com/app/alerts/scheduled-mutes/). * **One-time** You can silence alerts on a one-time basis by creating a scheduled mute with no repeat days. * **Recurring** By providing a list of days of the week for the scheduled mute to repeat, you can silence alerts on a recurring basis. ### [Troubleshooting Your Alerts](#troubleshooting-your-alerts) Please contact [support](mailto:support%40hostedgraphite.com) if you think you’ve found a bug, or have any questions, concerns, or suggestions. > * **Is your metric arriving?** > > If are not receiving notifications as expected, please check the [Alert Overview](https://www.hostedgraphite.com/app/alerts/) page and select the alert in question. You can use this to check the metric values for the last few hours are as expected. You can also inspect the Alert History for any recent alerting events. > * **Are some events being ignored?** > > We alert on a 30 second resolution. This means the finer data (5s) is averaged and we alert off the 30 second aggregate. > * **Is your alert not triggering as expected?** > > Alerts built in the Grafana UI may not work as expected, a simple fix would be recreating this alert in the Hosted Graphite Alerts UI. > * **Is your alert not resolving as expected?** > > Unless your alert criteria is set to 'data is missing', Graphite alerts will not trigger or resolve from *null* data. If alerting metric reports intermittent data (for example, 1 datapoint every 10min), null values can be reported between each datapoint. Try wrapping your alerting metric in a Graphite function like [transformNull()](https://graphite.readthedocs.io/en/latest/functions.html?highlight=transformNull#graphite.render.functions.transformNull), [keepLastValue()](https://graphite.readthedocs.io/en/latest/functions.html?highlight=keeplast#graphite.render.functions.keepLastValue), or [movingAverage()](https://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.movingAverage). If the alert criteria is set too low (FOR less than 10min), your alert might not resolve as expected. > * **Is your alert not resolving after updating the criteria?** > > An alert’s state is not changed after the criteria is updated. So while your alert might move to the ‘healthy’ list in our UI, it will remain in a triggered state until new data resolves the alert naturally. If you are looking to quickly resolve an alert by updating the criteria, you could simply delete and recreate the alert. > * **Is your alert triggering but not sending Slack notifications?** > > Check the ‘alert description’ field on the alert configuration. If the description contains an invalid character, like a double "quotation", this could malform the json payload of the Slack webhook and cause the request to return an error. Test your webhook with the following command: ``` curl -X POST -H 'Content-type: application/json' --data '{"text":"Hello, World!"}' ``` If you don't already have a Hosted Graphite account, you can start a 14-day free trial [here](https://www.hostedgraphite.com/accounts/signup/). You can also [schedule a product demo here ](https://calendly.com/metricfire-sales/hostedgraphite-demo)to speak with a knowledgeable member of our team who can provide support and advice on getting up and running quickly. # Alerts API Contents * [Alerts API](/api-guides/alerts-apis) * [Creating Alerts](#creating-alerts) * [Composite Alerts](#composite-alerts) * [Searching Alerts](#searching-alerts) * [Obtain Information on a Single Alert](#obtain-information-on-a-single-alert) * [Updating Alerts](#updating-alerts) * [Deleting Alerts](#deleting-alerts) * [Mute an Alert](#mute-an-alert) * [Unmute an Alert](#unmute-an-alert) * [Check if an Alert is Muted](#check-if-an-alert-is-muted) * [Mute Multiple Alerts](#mute-multiple-alerts) * [Unmute Multiple Alerts](#unmute-multiple-alerts) * [Obtain Alert History](#obtain-alert-history) ### [Creating Alerts](#creating-alerts) `POST` `/v2/alerts/` #### Path Parameters | Name | Type | Description | | ------------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | name\* | String | A name that uniquely identifies this alert. | | metric\* | String | The graphite metric query to alert on. | | alert\_criteria\* | String | The criteria for which the alert triggers. | | additional\_criteria | String | Any extra metrics that are to be included when evaluating the state of an alert, these are referred to as **Composite Alerts**. Note that any additional criteria must be referenced in the expression field and a maximum of 3 additional criteria is allowed. Example: `{"f": {"type": "above", "above_value": 5, "metric": "additional.metric.to.alert.on"}` where `f` is also referenced in the expression field. Defaults to `{}`. | | expression | String | A conditional relationship between multiple metrics. It allows for the combining of multiple alert criteria. It’s represented in the format of a boolean expression where each individual alert criteria is denoted as a single letter. The letter `a` is reserved for the criteria defined in `alert_criteria`. Only required when additional alerting criteria are defined. Maximum number of 3 additional criteria allowed (4 including the default alert criteria). `&&` takes higher precedence over `\|\|` Examples: `a && b, a \|\| c`. Defaults to `a`. | | above\_vale | String | The value above which the alert should trigger. Required for alert type `above` and `outside_bounds`. | | below\_value | String | The value below which the alert should trigger. Required for alert type `below` and `outside_bounds`. | | time\_period | String | The time period in minutes for which the threshold needs to be breached. Leave empty for an instantaneous evaluation. Required for alert type `missing`. | | scheduled\_mutes | String | List of scheduled mutes for this alert. Must be a list of scheduled mute IDs. Defaults to an empty list. | | notification\_channels | String | List of notification channels for this alert. This can be a combination of channel names and channel IDs. Defaults to `Email me` if none exists. | | notification\_type | String | The type of notification interval for this alert. Options are `state_change` (`['state_change']` or '`state_change`' is valid) or `['every', time_in_minutes]`. This lets you control how often you want to be notified for an alert. Defaults to `['every', 60]` if empty. | | info | String | Alert message sent with notifications. Can contain an arbitrary string which may contain a description of the alert, steps to follow or references to documentation. | | on\_query\_failure | String | Controls if a notification is delivered if the graphite function query fails. Only valid for Alerts that have graphite function in the metric field. Defaults to notify. Graphite function query can fail due to timeouts from matching too many metrics, being malformed or if it returns duplicate metrics due to aliasing. | {% tabs %} {% tab title="201" %} ```http HTTP/1.1 201 Content-Type: application/json ``` ```json { "id": "", "url": "/v2/alerts/" } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid format {% endtab %} {% tab title="409" %} Conflict Alert with the name already exists. {% endtab %} {% endtabs %} **Curl example:** ``` curl -H "Content-Type: application/json" -X POST -d \ '{"name": "alert1", "metric": "test.metric.1", "alert_criteria": {"type": "above", "above_value": 5, "time_period": 2}}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/" ``` Creates an alert named `alert1` for metric `test.metric.1` that triggers if the value is above 5 for 2 minutes with the (default) **Email me** notification type. **Request format**: ```json { "name": "string", "metric": "string", "alert_criteria": { "type": "above" | "below" | "outside_bounds" | "missing", "above_value": 123, "below_value": 123, "time_period": 123, }, "additional_criteria": { "b": { "type": "above" | "below" | "outside_bounds" | "missing", "above_value": 123, "below_value": 123, "time_period": 123, "metric": "string" } }, "expression": "a && b", "scheduled_mutes": ["scheduled_mute_id1", "scheduled_mute_id2", "scheduled_mute_id3"], "notification_channels": ["channel_id1", "channel_name1", "channel_id2"], "notification_type": "state_change" | ["state_change"] | ["every", 123], "info": "string", "on_query_failure": null | "ignore" | "notify" } ``` **Alert criteria examples:** Create an alert that triggers if the metric is missing for 10 minutes. ```json "alert_criteria": { "type": "missing", "time_period": 10 } ``` Create an alert that triggers if the metric is above 5.6 ever. ```json "alert_criteria": { "type": "above", "above_value": 5.6 } ``` Create an alert that triggers if the metric is below 2.1 or above 5.6 ever. ```json "alert_criteria": { "type": "outside_bounds", "above_value": 5.6, "below_value": 2.1 } ``` Create an alert that triggers if the metric is below 2.1 or above 5.6 for 10 minutes. ```json "alert_criteria": { "type": "outside_bounds", "above_value": 5.6, "below_value": 2.1, "time_period": 10 } ``` *** ### [Composite Alerts](#composite-alerts) Composite alerts can evaluate multiple metrics/values, and are defined in the **additional\_criteria** and **expression** parameters in the request format. Create an alert when one metric is above 80 and another is below 20. ```json "metric": "my.first.metric", "alert_criteria": { "type": "above", "above_value": 80 }, "additional_criteria": { "b": { "type": "below", "below_value": 20, "metric": "my.second.metric" } }, "expression": "a && b", ``` Create an alert when one metric is below 50 or another is below 30. ```json "metric": "my.first.metric", "alert_criteria": { "type": "above", "above_value": 50 }, "additional_criteria": { "c": { "type": "below", "below_value": 30, "metric": "my.second.metric" } }, "expression": "a || c" ``` Create an alert when one metric (A) is below 50 and another (B) is above 60 or if a third (C) is above 120. Note that performing an **&&** on two monitors gets preference over **||** so this alert expression will be evaluated as **(A && B) || C** ```json "metric": "my.first.metric", "alert_criteria": { "type": "below", "below_value": 50 }, "additional_criteria": { "b": { "type": "above", "above_value": 60, "metric": "my.second.metric" }, "c": { "type": "above", "above_value": 120, "metric": "my.third.metric" } }, "expression": "A && B || C" ``` **NOTE**: Composite alerts are fully supported in our Graphite Alerts UI and can be created, updated, and managed directly from the alert configuration interface. Alert notifications include additional composite context such as logical expressions, condition evaluation details, and triggered metrics. For wildcard-based conditions, notifications will also include a list of Triggered Metrics that contributed to the alert state. The alert overview page currently displays the last evaluated metric associated with the alert notification graph. *** ### [Searching Alerts](#searching-alerts) `GET` `/v2/alerts/` Returns a JSON object containing information on alerts matching the search query. Returns all the alerts if the parameter is left empty: `/v2/alerts/?` #### Path Parameters | Name | Type | Description | | ------ | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | name | String | Format: \&name=\\&name=\The alert name to search for. Can specify multiple alert names to search for more than 1. Performs an OR operation so a query such as `&name=&name=` will return info on both alert\_name1 and alert\_name2 | | id | String | Format: \&id=\\&id=\The alert id to search for. Can specify multiple ids to search for more than 1. Performs an OR operation so a query such as `&id=&id=` will return info on both alert\_id1 and alert\_id2. | | search | String | Format: \&search=\A more generic search that will match any alert whose alert name, metric name or notification channel name contains the search string. | | page | String | Format: \&page=1The page number to query. Used if there are more alerts than can be displayed in a single request. (default: 1) | | max | String | Format: \&max=50The maximum number of alerts to display. Limit of 100. (default: 100) | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 201 Content-Type: application/json ``` ```json { "alerts": [ { "name": "Name of the alert", "id": "unique id of the alert", "alert_criteria": { "type": "type of alert", "above_value": 123, "below_value": 123, "time_period": 123 }, "notification_channels": [""], "notification_type": ["state_change", "every", 123], "scheduled_mutes": [""], "currently_triggered_metrics": [], "muted": true, "status": "alerting", "info": "alert info", "on_query_failure": null } ], "next_page": false } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid form data {% endtab %} {% endtabs %} **Curl example:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/?" ``` Returns a list of all alerts ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/?&search=example ``` Returns a list of alerts that match the search pattern "example". Specifying both alert\_name and alert\_id performs an `OR` operation and returns alert that match either criteria. Including a search string performs an `AND` operation with the other criteria. *** ### [Obtain Information on a Single Alert](#obtain-information-on-a-single-alert) `GET` `/v2/alerts//` Returns information on a single alert {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "name": "Name of the alert", "id": "unique id of the alert", "alert_criteria": { "type": "threshold", "above_value": 123, "below_value": 123, "time_period": 123 }, "expression": "a", "notification_channels": ["email", "sms"], "scheduled_mutes": ["mute_id_1"], "currently_triggered_metrics": ["metric_1", "metric_2"], "notification_type": ["state_change"], "muted": false, "status": "alerting", "info": "Alert is active due to threshold breach.", "on_query_failure": "notify" } ``` {% endtab %} {% tab title="404" %} Not Found Alert doesn’t exist {% endtab %} {% endtabs %} **Curl example:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/123-456-7890/" ``` Obtain information on the alert identified by the id 123-456-7890. The alert id can be found in the URL of the alert when viewed from within the Hosted Graphite Alerts UI. **NOTE**: a request must contain the forward slash character after the alert\_id. *** ### [Updating Alerts](#updating-alerts) `PUT` `/v2/alerts//` Update attributes of an alert identified by the alert id {% tabs %} {% tab title="200" %} ```json HTTP/1.1 200 Content-Type: application/json { "name": "string", "metric": "string", "info": "string", "alert_criteria": { "type": "above", "above_value": 123, "below_value": 123, "time_period": 123 }, "scheduled_mutes": ["scheduled_mute_id1", "scheduled_mute_id2", "scheduled_mute_id3"], "notification_channels": ["channel_id1", "channel_id2", "channel_id3"], "notification_type": ["state_change"] } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid form data {% endtab %} {% tab title="404" %} Not Found Alert doesn’t exist. {% endtab %} {% endtabs %} **Curl Example** ``` curl -H "Content-Type: application/json" -X PUT -d \ '{"alert_criteria": {"time_period": 3, "type": "below", "below_value": 6}}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/123-456-789/" ``` Modify the alert identified by the id `123-456-789` to alert if the metric values are below 6 for 3 minutes. Each field is optional, only the specified fields are updated, and requests must contain the forward slash character after the alert\_id. *** ### [Deleting Alerts](#deleting-alerts) `DELETE` `/v2/alerts//` Delete an alert identified by the alert id. {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="400" %} Bad Request Alert doesn't exist {% endtab %} {% endtabs %} **Curl Example** ``` curl -X DELETE "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/123-456-789/" ``` Delete the alert identified by the id `123-456-789`. *** ### [Mute an Alert](#mute-an-alert) `POST` `/v2/alerts//muted/` Mute alerts identified by the alert id. #### Path Parameters | Name | Type | Description | | ------------------------------------------ | ------ | ------------------------ | | duration\* | String | Time to mute in minutes. | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "duration": 60 } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid form data {% endtab %} {% tab title="404" %} Not Found Alert doesn't exist {% endtab %} {% endtabs %} **Curl Example:** ``` curl -X POST -d '{"duration":60}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/123-456-789/muted/" ``` Mute the alert identified by the id `123-456-789` for 60 minutes. *** ### [Unmute an Alert](#unmute-an-alert) `DELETE` `/v2/alerts//muted/` Unmute an alert identified by the alert id. {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="404" %} Not Found Alert doesn't exist {% endtab %} {% endtabs %} **Curl Example:** ``` curl -X DELETE "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/123-456-789/muted/" ``` Unmute the alert identified by the id `123-456-789`\`. *** ### [Check if an Alert is Muted](#check-if-an-alert-is-muted) `GET` `/v2/alerts//muted/` Get the mute status of an alert identified by the alert id. {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "id": "alert_id", "name": "alert_name", "muted": false, "duration": 30, "scheduled_mutes": ["mute_id1", "mute_id2"] } ``` {% endtab %} {% tab title="404" %} Not Found Alert doesn't exist {% endtab %} {% endtabs %} **Curl Example:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/123-456-789/muted/" ``` Obtain information on the mute status of the alert identified by the id `123-456-789`. Also provides a list of the scheduled mutes currently attached to the alert. *** ### [Mute Multiple Aler](#mute-multiple-alerts)[ts](#mute-multiple-alerts) `POST` `/v2/alerts/muted/` #### Path Parameters | Name | Type | Description | | ------------------------------------------ | ------ | ---------------------------------------------------------------------------------------------------------------------------------------- | | ids | String | List of alert ids to mute. | | search | String | String to filter alerts. Every alert with an alert name, metric name, or notification channel name containing this string will be muted. | | duration\* | String | Time to mute in minutes. | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "ids": [ "alert_id1", "alert_id2" ], "duration": 60 } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid request {% endtab %} {% tab title="404" %} Not Found Alert doesn't exist {% endtab %} {% endtabs %} **Curl examples:** ``` curl -H "Content-Type: application/json" -X POST -d '{"ids": ["123-456-789", "321-654-987"], "duration":60}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/muted/" ``` Mute alerts 123-456-789 and 321-654-987 for 60 minutes. ``` curl -H "Content-Type: application/json" -X POST -d '{"search": "search_string", "duration":60}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/muted/" ``` Mute alerts with alert name, metric name or notification channel name containing `search_string` for 60 minutes. If both `ids` and `search` are empty, then **all** alerts will be muted. *** ### [Unmute Multiple Alerts](#unmute-multiple-alerts) `DELETE` `/v2/alerts/muted/` #### Path Parameters | Name | Type | Description | | ------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------ | | ids | String | List of alert ids to unmute. | | search | String | String to filter alerts. Every alert with an alert name, metric name, or notification channel name containing this string will be unmuted. | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "ids": [ "alert_id1", "alert_id2" ] } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid request {% endtab %} {% tab title="404" %} Not Found Alert doesn't exist {% endtab %} {% endtabs %} **Curl examples:** ``` curl -H "Content-Type: application/json" -X DELETE -d '{"ids": ["123-456-789", "321-654-987"]}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/muted/" ``` Unmute alerts 123-456-789 and 321-654-987. ``` curl -H "Content-Type: application/json" -X DELETE -d '{"search": "search_string"}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/muted/" ``` Unmute alerts with alert name, metric name or notification channel name containing search\_string. If both ids and search are empty, then **all** alerts will be unmuted. *** ### [Obtain Alert History](#obtain-alert-history) `GET` `/v2/alerts/history/?` #### Path Parameters | Name | Type | Description | | ------ | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | String | The alert id to search for. Can specify multiple ids to search for more than 1. Returns history of all alerts if emptyFormat: \&alert\_id=\\&alert\_id=\ | | days | String | The number of days to obtain history of.Format: \&days=1 (default: 3) | | status | String | The status that was recorded. Cant be either ‘alerting’ or ‘recovered’. | | value | String | The value at which the alert triggered or recovered. Value will be null if the alert was a missing metric alert. | | time | String | Unix timestamp at which the alert triggered or recovered. | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "alert_id_1": [ { "status": "alerting", "value": 123, "time": "1672531200", "metric": "The metric that triggered this." } ], "alert_id_2": [ { "status": "healthy", "value": null, "time": "1672531200", "metric": "The metric that triggered this." } ] } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid query parameters {% endtab %} {% endtabs %} **Curl example:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/alerts/history/?id=123-456-789 ``` Obtain alert history of the alert identified by the id `123-456-789`. # Notification Channels API Contents * [Notification Channels API](/alerting-guide/notification-channels-api) * [Creating notification channels](#creating-notification-channels) * [Searching notification channels](#searching-notification-channels) * [Obtain information on a single notification](#obtain-information-on-a-single-notification) * [Updating notification channels](#updating-notification-channels) * [Deleting notification channels](#deleting-notification-channels) ### [Creating notification channels](#creating-notification-channels) `POST` `/v2/notifications/` #### Path Parameters | Name | Type | Description | | --------------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------- | | type\* | String | The type of notification channel to create. | | description\* | String | A name for the channel. | | destination\* | String | The destination for the notification. ie the email address for *email*, slack webhook URL for *slack,* etc. | | auto\_resolve | String | Only used by VictorOps and PagerDuty channels. Defaults to *false*. More info here. | {% tabs %} {% tab title="201" %} ```http HTTP/1.1 201 Content-Type: application/json ``` ```json { "id": "id", "url": "/v2/notifications/" } ``` {% endtab %} {% tab title="404" %} Not Found Invalid form data {% endtab %} {% tab title="409" %} Conflict Channel with the description already exists. {% endtab %} {% endtabs %} **Request format**: ```json HTTP/1.1 200 Content-Type: application/json { "type": "email" | "pagerduty" | "slack" | "webhook" | "victorops" | "opsgenie", "description": "Descriptive name for channel", "destination": "Destination key of the channel.", "auto_resolve": True | False } ``` **Curl example** ``` curl -H "Content-Type: application/json" -X POST -d \ '{"type": "email", "description": "A test notification", "destination": "email@example.com"}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/notifications/" ``` *** ### [Searching notification channels](#searching-notification-channels) `GET` `/v2/notifications/?query` Returns a JSON object containing information on notification channels matching the search query. Returns all the notifications if the parameter is left empty. #### Path Parameters | Name | Type | Description | | ------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | type | String |

Format: \&type=\\&type=\

The notification type to search for. Can specify multiple types to search for more than 1.

| | description | String |

Format: \&descriptions=\\&descriptions=\

The descriptions to search for. Can specify multiple descriptions to search for more than 1.

| | id | String |

Format: \&id=\\&id=\

The notification id to search for. Can specify multiple ids to search for more than 1.

| | page | String |

Format: \&page=1

The page number to query. Used if there are more items than can be displayed in a single request (default: 1).

| | max | String |

Format: \&max=50

The maximum number of items to display. Limit of 100 (default: 100).

| | auto\_resolve | String | For use with VictorOps and PagerDuty channels only. | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "notifications": [ { "type": "email", "destination": "example@example.com", "description": "Email notification channel", "id": "notification_channel_1", "auto_resolve": true }, { "type": "slack", "destination": "https://slack-webhook-url.com", "description": "Slack notification channel", "id": "notification_channel_2", "auto_resolve": false } ], "next_page": false } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid {% endtab %} {% endtabs %} **Curl example** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/notifications/?type=email&type=slack" ``` Specifying multiple fields performs an `OR` operation and returns notifications that match either criteria. *** ### [Obtain information on a single notification](#obtain-information-on-a-single-notification) `GET` `/v2/notifications//` Returns a single notification identified by the notification id. #### Path Parameters | Name | Type | Description | | --------------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------- | | type\* | String | The type of notification channel to create. | | description\* | String | A name for the channel. | | destination\* | String | The destination for the notification. ie the email address for *email*, slack webhook URL for *slack,* etc. | | auto\_resolve | String | Only used by VictorOps and PagerDuty channels. Defaults to *false*. | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "type": "email", "description": "Primary email notification channel", "destination": "example@example.com", "id": "notification_channel_1", "auto_resolve": true } ``` {% endtab %} {% tab title="400" %} Bad Request Doesn't exist {% endtab %} {% endtabs %} **Curl example** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/notification/123-456-789/" ``` Obtain information of the notification channel identified by the id `123-456-789`. *** ### [Updating notification channels](#updating-notification-channels) `PUT` `/v2/notifications//` Updates a notification channel identified by the notification id. #### Path Parameters | Name | Type | Description | | ------------- | ------ | ----------------------------------------------------------------------------------------------------------- | | type | String | The type of notification channel to create. | | description | String | A name for the channel. | | destination | String | The destination for the notification. ie the email address for *email*, slack webhook URL for *slack,* etc. | | auto\_resolve | String | Only used by VictorOps and PagerDuty channels. Defaults to *false*. | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "type": "slack", "description": "Slack notification channel", "destination": "https://slack-webhook-url.com", "id": "notification_channel_2", "auto_resolve": false } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid form data {% endtab %} {% tab title="404" %} Not Found Doesn't exist {% endtab %} {% tab title="409" %} Conflict Notification channel with the description already exists. {% endtab %} {% endtabs %} **Curl example** ``` curl -H "Content-Type: application/json" -X PUT \ -d '{"destination": "new_email@example.com"}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/notifications/123-456-789/" ``` Updates the destination of notification `123-456-789`. Each field is optional and only the specified fields are updated. *** ### [Deleting notification channels](#deleting-notification-channels) `DELETE /v2/notifications//` - Deletes a notification channel identified by the notification id. **Curl example** ``` curl -X DELETE "https://YOUR-API-KEY@api.hostedgraphite.com/v2/notifications/123-456-789/" ``` Delete the notification channel identified by the id `123-456-789`. **Status Codes:** > * **200** - OK > * **404** - Notification channel doesn’t exist. # Scheduled Mutes API Contents * [Scheduled Mutes API](/alerting-guide/scheduled-mutes-api) * [Creating scheduled mutes](#creating-scheduled-mutes) * [Searching scheduled mutes](#searching-scheduled-mutes) * [Obtain information on a single scheduled mute](#obtain-information-on-a-single-scheduled-mute) * [Updating scheduled mutes](#updating-scheduled-mutes) * [Deleting scheduled mutes](#deleting-scheduled-mutes) ### [Creating scheduled mutes](#creating-scheduled-mutes) `POST` `/v2/schedules/` #### Path Parameters | Name | Type | Description | | --------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | name\* | String | A name that uniquely identifies this scheduled mute. | | start\_time\* | String | A 24-hour time string at which the scheduled mute will begin. | | duration\* | String | Mute time in minutes. | | timezone | String | The timezone string for this scheduled mute’s start time. Valid values are “UTC-12”...”UTC-1”, “UTC”, “UTC+1”...”UTC+14”. Defaults to “UTC” if not specified. | | repeat | String | The list of days at which the scheduled mute will repeat on. Becomes a one-time scheduled mute if not specified. Valid values are “mon”, “tue”, “wed”, “thu”, “fri”, “sat” and “sun”. | {% tabs %} {% tab title="201" %} ```http HTTP/1.1 201 Content-Type: application/json ``` ```json { "id": "", "url": "/v2/schedules/" } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid {% endtab %} {% tab title="409" %} Conflict Already exists {% endtab %} {% endtabs %} **Request format**: ``` { "name": "string", "start_time": "14:20", "duration": 123, "timezone": "UTC-2", "repeat": ["mon", "tue", "wed", "thu", "fri", "sat", "sun"] } ``` **Curl example** ``` curl -H "Content-Type: application/json" -X POST -d \ '{"name": "Mute on weekends", "start_time": "00:00", "duration": 1440, "timezone": "UTC+2", "repeat": ["sat", "sun"]}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/schedules/" ``` Creates a scheduled mute named “Mute on weekends” that starts at 12 AM in timezone UTC+2, with a mute duration of 24 hours, and repeats every Saturday and Sunday. *** ### [Searching scheduled mutes](#searching-scheduled-mutes) `GET` `/v2/schedules/?` Returns a JSON object containing information on scheduled mutes matching the search query. Returns all scheduled mutes if the parameter is left empty. #### Path Parameters | Name | Type | Description | | ----------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | name | String |

Format: \&name=\\&name=\

The scheduled mute name to search for. Can specify multiple scheduled mute names to search for more than 1. Performs an OR operation so a query such as

\&name=\\&name=\

will return info on both scheduled\_mute\_name1 and scheduled\_mute\_name2.

| | id | String |

Format: \&id=\\&id=\

The scheduled mute ID to search for. Can specify multiple scheduled mute IDs to search for more than 1. Performs an OR operation so a query such as

\&name=\\&name=\

will return info on both scheduled\_mute\_id1 and scheduled\_mute\_id2.

| | duration | String |

Format: \&duration=\\&id=\

Searches for scheduled mutes by their duration. Can specify multiple durations to search for. Performs an OR operation so a query such as

\&duration=30\&duration=10

will return scheduled mutes with a 10-minute duration or a 30-minute duration.

| | start\_time | String |

Format: \&start\_time=\\&start\_time=\

Searches for scheduled mutes by their start time. Can specify multiple start times to search for. Performs an OR operation so a query such as

\&start\_time=12:00\&start\_time=20:00

will return scheduled mutes that start at either 12 PM or 8 PM.

| | timezone | String |

Format: \&timezone=\\&timezone=\

Searches for scheduled mutes by their timezone. Can specify multiple time zones to search for. Performs an OR operation so a query such as

\&timezone=UTC+2\&timezone=UTC-14

will return scheduled mutes that have either UTC+2 or UTC-14 time zones.

| | repeat | String |

Format: \&repeat=\\&repeat=\

Searches for scheduled mutes by their repeat days. Can specify multiple repeat days to search for. Performs an OR operation so a query such as

\&repeat=sun\&repeat=thu

will return scheduled mutes that repeat on either Sunday or Thursday.

| | page | String |

Format: \&page=1

The page number to query. Used if there are more items that can be displayed in a single request (default: 1).

| | max | String |

Format: \&max=50

The maximum number of items to display. Limit of 100 (default: 100).

| {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "scheduled_mutes": [ { "name": "Maintenance Window", "id": "mute_id_1", "duration": 60, "alerts": ["alert_id1", "alert_id2", "alert_id3"], "start_time": "20:20", "timezone": "UTC+4", "repeat": ["mon", "wed", "fri"] }, { "name": "Holiday Quiet Hours", "id": "mute_id_2", "duration": 120, "alerts": ["alert_id4", "alert_id5"], "start_time": "18:00", "timezone": "UTC+1", "repeat": ["sat", "sun"] } ], "next_page": false } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/schedules/?" ``` Specifying multiple fields performs an `OR` operation and returns scheduled mutes that match either criterion. *** ### [Obtain information on a single scheduled mute](#obtain-information-on-a-single-scheduled-mute) `GET` `/v2/schedules//` Returns information on a single scheduled mute identified by ``. {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "name": "Maintenance Window", "id": "mute_id_1", "duration": 123, "alerts": ["alert_id1", "alert_id2", "alert_id3"], "start_time": "20:20", "timezone": "UTC+4", "repeat": ["mon", "wed", "fri"] } ``` {% endtab %} {% tab title="404" %} Not Found Scheduled mute doesn’t exist. {% endtab %} {% endtabs %} **Curl example:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/v2/schedules/123-456-789/" ``` Obtain information on the scheduled mute identified by the ID `123-456-789`. *** ### [Updating scheduled mutes](#updating-scheduled-mutes) `PUT` `/v2/schedules//` Updates the attributes of a scheduled mute identified by ``. #### Path Parameters | Name | Type | Description | | ----------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | name | String | A name that uniquely identifies this scheduled mute. | | start\_time | String | A 24-hour time string at which the scheduled mute will begin | | duration | String | Mute time in minutes. | | timezone | String | The timezone string for this scheduled mute’s start time. Valid values are “UTC-12”...”UTC-1”, “UTC”, “UTC+1”...”UTC+14”. Defaults to “UTC” if not specified. | | repeat | String | The list of days at which the scheduled mute will repeat on. Becomes a one-time scheduled mute if not specified. Valid values are “mon”, “tue”, “wed”, “thu”, “fri”, “sat” and “sun”. | {% tabs %} {% tab title="200" %} ```json { "name": "string", "start_time": "14:20", "duration": 123, "timezone": "UTC-2", "repeat": [ "mon", "tue", "wed", "thu", "fri", "sat", "sun" ] } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid {% endtab %} {% tab title="404" %} Not Found Scheduled mute doesn’t exist. {% endtab %} {% tab title="409" %} Conflict Scheduled mute with the updated name already exists. {% endtab %} {% endtabs %} **Curl example:** ``` curl -H "Content-Type: application/json" -X PUT \ -d '{"start_time": "14:50", "timezone": "UTC+12"}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v2/schedules/123-456-789/" ``` Updates the start time and timezone of the scheduled mute with the id `123-456-789.` Each field is optional and only the specified fields are updated. *** ### [Deleting scheduled mutes](#deleting-scheduled-mutes) `DELETE` `/v2/schedules//` Deletes a scheduled mute identified by ``. {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="404" %} Not Found Scheduled mute doesn’t exist. {% endtab %} {% endtabs %} **Curl example:** ``` curl -X DELETE "https://YOUR-API-KEY@api.hostedgraphite.com/v2/schedules/123-456-789/" ``` Delete the scheduled mute identified by the ID `123-456-789`. # Using Your Own Alerting If you feel you need complex graphite Alerting Overview features that we don’t yet support there are several great open source projects to provide alerting on your metrics. Two well-maintained projects are [Cabot](https://github.com/arachnys/cabot) and [Seyren](https://github.com/scobal/seyren). ### Connecting Hosted Graphite to Open Source Alerting Tools Connecting Hosted Graphite to open source Alerting tools is a simple process. * First, you need to create an [*access key*](/account-management/access-keys). * Right-click the URL of the access key, and copy it. * Use this value as your Graphite URL in your alerting system. ### Using Hosted Graphite with Cabot ``` GRAPHITE_API: Your Access Key URL GRAPHITE_USER: GRAPHITE_PASS: GRAPHITE_FROM: -10min ``` Both **GRAPHITE\_USER** and **GRAPHITE\_PASS** should be left empty, they’re not necessary when using the [Access Key.](/account-management/access-keys) Note **GRAPHITE\_API** should end with a `‘/’` character. ### Using Hosted Graphite with Seyren Set the **GRAPHITE\_URL** to your Access key URL as either a system property or an environment variable when starting Seyren. In the simplest case: ```python export GRAPHITE_URL=Your access Key URL java -jar seyren-1.2.0.jar open http://localhost:8080 ``` # Agents Guide We support a variety of ways to collect and forward system performance metrics from your running instances and servers. Some of these agents also have plugins compatible with other 3rd party services. * [HG-CLI](/hg-cli) (recommended) * An easy to use command line tool for installing and configuring agents * [The Hosted Graphite Agent](/agents-guide/the-hosted-graphite-agent) * Our agent made specifically for Hosted Graphite, with an easy 1-click installation * [Telegraf](/agents-guide/telegraf) * Our **favorite** agent because of it's many input plugins for 3rd party technologies * [K8 Telegraf Daemon](/agents-guide/k8-telegraf-daemon) * A quick and easy way to monitor the performance of your Kubernetes clusters * [OpenTelemetry](/agents-guide/opentelemetry) * A powerful and flexible observability agent that supports Graphite metric exporting * [StatsD](/agents-guide/statsd-agent) * Commonly used for application monitoring and can also be used as a pre-aggregator * [collectd](/agents-guide/collectd-agent) * A well documented agent that also supports a wide range of 3rd party plugins * [Diamond](/agents-guide/diamond) * A reliable agent that we use for our own internal monitoring ### Security Disclaimer All agents we recommend for metric collection are configured to securely transmit data using your unique Hosted Graphite API key. This ensures that each data packet is authenticated and isolated to your specific environment. We prioritize security in our recommendations, selecting agents with proven records in secure data handling. With the recommended Telegraf agent, most of their input plugins also have an **optional** TLS config for additional security measures when connecting to your internal services.

Telegraf TLS Config

# The Hosted Graphite Agent Our agent is a simple way to get your metrics into Hosted Graphite. You install it on your machines, and it collects and forwards the system’s metrics for you. If you have any questions about the agent, [get in touch](mailto:support%40hostedgraphite.com). Contents * [The Hosted Graphite Agent](/agents-guide/the-hosted-graphite-agent) * [Installation](#installation) * [Platforms](#platforms) * [How it Works](#how-it-works) * [Features](#features) * [Dashboards](#dashboards) * [Agent list](#agent-list) * [Base system metrics](#base-system-metrics) * [Configuration](#configuration) * [Optional Configuration](#optional-configuration) * [Metric path prefix](#metric-path-prefix) * [Proxies](#proxies) * [Local Metric Receiver / Forwarder](#local-metric-receiver-forwarder) * [Dedicated environments](#dedicated-environments) * [MongoDB](#mongodb) * [Troubleshooting](#troubleshooting) * [Source Code](#source-code) * [Uninstall](#uninstall) * [References](#references) ### [Installation](#installation) Get started by pulling down our installer script. For Debian-based systems, including Ubuntu: ``` curl -s https://YOUR-API-KEY@www.hostedgraphite.com/agent/installer/deb/ | sudo sh ``` or for RHEL-based systems: ``` curl -s https://YOUR-API-KEY@www.hostedgraphite.com/agent/installer/rpm/ | sudo sh ``` If you would like to set up installation and configuration via Puppet, Chef, or a similar configuration management system, see our [repository install instructions](https://packagecloud.io/hostedgraphite/hg-agent/install). Once your agent is sending metrics, you will find it in your [Agent list](https://www.hostedgraphite.com/app/agent/), along with a link to an auto-generated [*Base Metrics*](/agents-guide/the-hosted-graphite-agent#base-system-metrics) dashboard named **Hosted Graphite Agent**. #### [Platforms](#platforms) We support the following stable OS releases on 64-bit machines: * CentOS 7 forward (covering the RedHat family); * Debian buster forward; * Ubuntu 16.04 forward (LTS). This should cover most installs on RHEL, CentOS, Amazon Linux, and Debian/Ubuntu family platforms. ### [How it Works](#how-it-works) The Hosted Graphite agent is an easy-to-use bundle of the popular metrics collector [Diamond](https://github.com/python-diamond/Diamond) and the [Supervisor](http://supervisord.org/) process manager along with an embedded build of Python and some support scripts. #### [Features](#features) * Faster submission of metric data from your application because it doesn’t have to do a DNS lookup, or connect out across the internet. * More reliable delivery because the agent stores-and-forwards, buffering on disk when it can’t connect. * Easier debugging for the “my metric is missing” problem because our UI can tell you that an agent has stopped responding. * More secure: the agent uses HTTPS and while we offer TLS-wrapped carbon protocols, it’s more effort to use than the plaintext ones. Sending straight to the agent bypasses this and we take care of it for you. * No need to prefix your metrics with an API key for HG’s authentication: it’s handled for you, using the API key in the agent config. * No need to specify new firewall rules for carbon protocols and keep the rules updated - if you can already get HTTPS out of your network, even via a proxy, then it’ll work. For more details on how the agent works on your machines, see [*System Layout*](/agents-guide/the-hosted-graphite-agent/system-layout). The agent’s default metrics are listed and explained in [*Base Metrics*](#base-system-metrics). ### [Dashboards](#dashboards) #### [Agent list](#agent-list) The [Agent list](https://www.hostedgraphite.com/app/agent/) provides an overview of all agents associated with your account. From here, you can view the status of each agent and other information.
The colored circle representing the **State** can take 3 values: * **Green**: The agent is reporting normally at the correct time. * **Yellow**: Agent has not reported in over 20 minutes, or is reporting with the wrong time. * **Red**: The agent has reported errors in recent log messages.
An agent’s **Clock Offset** is the difference between the time reported by the agent and the time recorded on our server. Incorrect time can produce anomalies in metric data so it is advised to keep the agent’s clock accurate. You can delete the record of an agent from this page, but be aware that this will not *uninstall* the agent. To uninstall an agent, [*use your package manager*](#source-code). ### [Base system metrics](#base-system-metrics)

Base system metrics dashboard

This dashboard can be located and uploaded from our [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library). It displays many of the metrics reported by the Hosted Graphite agent but can be customized further. You can find descriptions of how to read the base system metrics dashboard at [*Base Metrics*](#base-system-metrics). ### [Configuration](#configuration) Agent configuration is minimal and done via `/etc/opt/hg-agent/hg-agent.conf.` This is used to generate a more complete configuration for `diamond` in `/var/opt/hg-agent/diamond.conf.` Typically `hg-agent.conf` contains only an `api_key`: You can find the right value for this on your [Hosted Graphite account page](https://www.hostedgraphite.com/), but it should be filled in automatically by the simple installation process above. Note that the agent takes control of `diamond` configuration: if you want more nuanced control, you can run a separate `diamond` instance or whatever metric collection system you wish. You can still take advantage of the agent’s authentication management and local buffering/forwarding facilities by configuring it to send Graphite data to `localhost`. #### [Optional Configuration](#optional-configuration) #### [**Metric path prefix**](#metric-path-prefix) You can supply a custom prefix instead of `hg_agent`, e.g. to specify a host environment: ``` custom_prefix: hg_agent.prodenv ``` though keep in mind that the neat [*Base Metrics*](#base-system-metrics) dashboard will no longer work for you. You can also specify different methods of picking up the hostname, per hostname\_method in [Diamond’s config](https://diamond.readthedocs.io/en/latest/Getting-Started/Configuration/#collector-settings). By default the agent uses `smart`, but e.g. if you’d prefer the FQDN with `.` replaced by `_`: #### [**Proxies**](#proxies) If your system requires a proxy to get `HTTPS` access to the Internet: ``` https_proxy: http://10.10.1.10:1080 ``` #### [**Local Metric Receiver / Forwarder**](#local-metric-receiver-forwarder) If you would like the receiver to listen on something other than the default [Carbon](http://graphite.readthedocs.io/en/latest/feeding-carbon.html) ports (defaults shown): ``` tcp_port: 2003 udp_port: 2003 ``` If you’d like to keep more spool data for buffering locally during network outages (defaults shown): ``` max_spool_count: 10 spool_rotatesize: 10000000 ``` #### [**Dedicated environments**](#dedicated-environments) Large customers with dedicated Hosted Graphite environments may need to specify metric data and metadata endpoints, e.g. for a cluster `tst`: ``` endpoint_url: https://tst.hostedgraphite.com/api/v1/sink heartbeat_url: https://heartbeat-tst.hostedgraphite.com/beat ``` #### [**MongoDB**](#mongodb) If you want the agent to monitor [MongoDB](https://www.mongodb.com/) on a host: ``` mongodb: enabled: True host: localhost port: 27017 ``` You can specify any of the keys from [Diamond’s MongoDB config](https://diamond.readthedocs.io/en/latest/collectors/MongoDBCollector/) here. ### [Troubleshooting](#troubleshooting) You can check whether your `hg-agent` is reporting metadata to Hosted Graphite via the [Agent list](https://www.hostedgraphite.com/app/agent/), and from there you can use the [*Base Metrics*](#base-system-metrics) dashboard to see if metric data is flowing in. If not, you may need to dig a little deeper. Logs are in `/var/log/hg-agent`: you can see the metrics generated by `diamond` in `archive.log`, as well as the various daemon logs `supervisord.log`, `diamond.log`, `periodic.log`, and `forwarder.log` e.g. ``` tail -f /var/log/hg-agent/forwarder.log ``` Process structure and other details are described in [*System Layout*](/agents-guide/the-hosted-graphite-agent/system-layout). You can inspect the agent as a service like any other, e.g. on Ubuntu `trusty`: ``` sudo service hg-agent status ``` If you’d like to look at things from the agent’s `supervisor` point of view: ``` sudo /opt/hg-agent/bin/supervisorctl --config=/etc/opt/hg-agent/supervisor.conf status ``` ### [Source Code](#source-code) The sources used to build the agent can be followed at: * * * and are licensed under an [MIT license](https://github.com/hostedgraphite/hg-agent/blob/master/LICENSE). Versions of Supervisor & Diamond included can be found in the [agent build scripts](https://github.com/hostedgraphite/hg-agent/blob/master/build.sh). ### [Uninstall](#uninstall) Hosted Graphite Agents can be uninstalled using your package manager. For Debian-based systems: ``` sudo apt-get remove hg-agent ``` For RHEL-based systems: ``` sudo yum remove hg-agent ``` ### [References](#references) * [Base Metrics](/agents-guide/the-hosted-graphite-agent/base-metrics) * [System Layout](/agents-guide/the-hosted-graphite-agent/system-layout) # Base Metrics This document describes the base system metrics exported by the Hosted Graphite agent. Contents * [Base Metrics](/agents-guide/the-hosted-graphite-agent/base-metrics) * [Load](#load) * [CPU utilization](#cpu-utilization) * [Load average](#load-average) * [Processes](#processes) * [Memory](#memory) * [Activity](#activity) * [Virtual memory](#virtual-memory) * [Writeback](#writeback) * [Disk](#disk) * [iostat](#iostat) * [Capacity](#capacity) * [Network](#network) * [Interfaces](#interfaces) * [Sockets](#sockets) We focus on the default “base” dashboard and also provide notes on related metrics not displayed there. As Diamond collectors rely heavily on `/proc` data, many of the notes below are from Linux kernel documentation, e.g. [proc.txt](https://www.kernel.org/doc/Documentation/filesystems/proc.txt); We list metric units - percentage, count, bytes, etc. - in brackets after each metric description. If you find anything unclear or incorrect here, please [let us know](mailto:help%40hostedgraphite.com)! ### [Load](#load) #### [CPU utilization](#cpu-utilization)

User & system CPU graphs

These metrics are found under: ```bash hg_agent.hostname.cpu.cpuid.* ``` and represent percentages of time each *`cpuid`* spends in particular states. We display two of the most interesting on the dashboard: * `user`: normal processes executing in user mode (percentage); * `system`: processes executing in kernel mode (percentage). Others you can use in your own graphs or investigations: * `nice`: niced processes executing in user mode (percentage); * `idle`: nothing to do (percentage); * `iowait`: not really reliable - see note in [proc.txt](https://www.kernel.org/doc/Documentation/filesystems/proc.txt); * `irq`: servicing interrupts (percentage); * `softirq`: servicing software interrupts (percentage); * `steal`: executing other virtual hosts (percentage); * `guest`: running a normal virtual guest (percentage); * `guest_nice`: running a niced virtual guest (percentage). #### [Load average](#load-average)

Load average graph

These metrics are found under: ``` hg_agent.hostname.loadavg.* ``` Load average, [roughly speaking](https://prutser.wordpress.com/2012/05/28/understanding-linux-load-average-part-3/), is the average number of tasks waiting with “something to do” over a period of time: * `01`: 1-minute load average (count); * `05`: 5-minute load average (count); * `15`: 15-minute load average (count). Since the interpretation of load average is [affected](http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages) by the number of cores a machine has, you might like to use these “normalized” versions in your own graphs or investigations: * 01\_normalized: 1-minute load average normalized by #cores (count); * 05\_normalized: 5-minute load average normalized by #cores (count); * 15\_normalized: 15-minute load average normalized by #cores (count). #### [Processes](#processes)

Processes graph

These metrics are found under: ``` hg_agent.hostname.loadavg.* ``` These are simple “snapshot” counters of the process numbers. Note that the number running will typically be maxed out at #cores. * processes\_total: total number of processes on the system (count); * processes\_running: number of processes running (count). ### [Memory](#memory) #### [Activity](#activity)

Memory and swap graphs

These metrics are found under: ``` hg_agent.hostname.memory.* ``` In the “memory activity” graph, we display some of the metrics most relevant to physical memory usage: * `MemTotal`: total usable ram, i.e. physical ram minus a few reserved bits and the kernel binary code (bytes); * `MemAvailable`: an estimate of how much memory is available for starting new applications requires kernel 3.14 or later (bytes); * `Active`: memory used recently, usually not reclaimed unless absolutely necessary (bytes); * `Cached`: in-memory cache for files read from the disk, i.e. the pagecache (bytes). And “swap activity” displays: * `SwapTotal`: the total amount of swap space configured (bytes); * `SwapFree`: the amount of swap space available for use (bytes). There are several other metrics available under memory.\*. If you’re digging further, you can find out what they mean in the [docs for /proc/meminfo](https://www.kernel.org/doc/Documentation/filesystems/proc.txt). #### [Virtual memory](#virtual-memory)

vmstat graphs

These metrics are found under: ``` hg_agent.hostname.vmstat.* ``` These are metrics from `/proc/vmstat` and give some insight into the activity of the Linux virtual memory system. Unfortunately, the counters are [a little underdocumented](https://access.redhat.com/solutions/1160343). First, pages in and out: * `pgpgin`: pages brought in from disk (count); * `pgpgout`: pages written out to disk (count). Note that because everything goes through the [page cache](https://en.wikipedia.org/wiki/Page_cache), these are recorded for essentially all pages read from or written to disk, so if you’re doing a lot of IO they’ll be elevated. Next, [swap usage](https://wiki.archlinux.org/index.php/swap) which generally you want to keep low or nonexistent. See [this article](http://www.linuxjournal.com/article/8178) for more information. * `pswpin`: pages brought in from swap space (count); * `pswpout`: pages swapped out into swap space (count). Finally, [page faults](https://en.wikipedia.org/wiki/Page_fault) made by the virtual memory system to page memory into process address spaces: * `pgfault`: [minor](https://en.wikipedia.org/wiki/Page_fault#Minor) page faults (count); * `pgmajfault`: [major](https://en.wikipedia.org/wiki/Page_fault#Major) page faults (count). Note that page faults will stimulate paging in, so you can expect these to correlate. #### [Writeback](#writeback)

Memory writeback graphs

These metrics are found under: ``` hg_agent.hostname.memory.* ``` * `Dirty`: memory waiting to be written back to disk (bytes). When you change disk-backed memory in the page cache, it’s not written to disk immediately, just marked as “dirty”. This graph allows you to see how much is building up & being written back over time. ### [Disk](#disk) #### [iostat](#iostat)

iostat graphs

These metrics are found under: ``` hg_agent.hostname.iostat.* ``` These metrics are per-disk, and are gathered from [/proc/diskstats](https://www.kernel.org/doc/Documentation/iostats.txt). * `iops`: “I/O operations per second”, i.e. `reads` + `writes` (count); * `write_byte_per_second`: bytes written per second (bytes); * `read_byte_per_second`: bytes read per second (bytes); * `util_percentage`: how much of the time the disk is performing I/O operations (percentage). There are many other `iostat` metrics exported per disk; you can browse your metric tree to see which and compare with [/proc/diskstats](https://www.kernel.org/doc/Documentation/iostats.txt) and [the ‘diskusage’ diamond collector](https://github.com/python-diamond/Diamond/blob/master/src/collectors/diskusage/diskusage.py). #### [Capacity](#capacity)

Disk capacity graphs

These metrics are found under: ``` hg_agent.hostname.diskspace.* ``` Again, these metrics are per-disk. * `byte_avail`: available bytes, i.e. space available for use by non-privileged users (bytes). Apart from this useful graphed value, there are also some more available to you: * `byte_free`: available bytes for the superuser (bytes); * `byte_percentfree`: `byte_free` as a percentage of the total (percentage); * `byte_used`: bytes used (bytes); * `inodes_avail`: available inodes for use by non-privileged users (count); * `inodes_free`: available inodes for the superuser (count); * `inodes_percentfree`: `inodes_free` as a percentage of the total (percentage); * `inodes_used`: inodes used (count). ### [Network](#network) #### [Interfaces](#interfaces)

Network interface graphs

These metrics are found under: ``` hg_agent.hostname.network.* ``` These metrics are per-interface. We graph the following: * `tx_packets`, `rx_packets`: packets transmitted, received (count); * `tx_byte`, `rx_byte`: bytes transmitted, received (bytes); * `tx_drop`, `rx_drop`: packets dropped by the driver on transmit, receive (count). There are many other network metrics exported per interface; you can browse your metric tree to see which and compare with [/proc/net/dev](http://www.onlamp.com/pub/a/linux/2000/11/16/LinuxAdmin.html), which is fairly self-explanatory, and [the ‘network’ diamond collector](https://github.com/python-diamond/Diamond/blob/master/src/collectors/network/network.py). #### [Sockets](#sockets)

Socket graphs

These metrics are found under: ``` hg_agent.hostname.sockets.* ``` They’re drawn from /proc/net/sockstat, which is under-documented. * `used`: total number of sockets [in kernel socket lists](http://elixir.free-electrons.com/linux/latest/source/net/socket.c#L169) (count); * `tcp_inuse`: TCP sockets currently in use (count); * `udp_inuse`: UDP sockets currently in use (count). Others you can use in your own graphs or investigations: * `tcp_mem`: the number of [pages](http://blog.tsunanet.net/2011/03/out-of-socket-memory.html) in use for TCP (count); * `udp_mem`: the same for UDP (count); * `tcp_alloc`: number of sockets allocated for TCP (count); * `tcp_orphan`: sockets [not associated to file descriptors](http://blog.tsunanet.net/2011/03/out-of-socket-memory.html) (count); * `tcp_tw`: sockets in `TIME_WAIT`, i.e. waiting after closing to handle packets still in the network (count). # System Layout This document describes how our Hosted Graphite Agent (hg-agent) works on your machine. Contents * [System Layout](/agents-guide/the-hosted-graphite-agent/system-layout) * [Disk](#disk) * [Processes](#processes) * [supervisord](#supervisord) * [diamond](#diamond) * [periodic](#periodic) * [receiver](#receiver) * [forwarder](#forwarder) * [Configuration](#configuration) * [Resource requirements](#resource-requirements) ### [Disk](#disk) The layout on disk is per [FHS guidelines for add-on application software packages](http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICATIONSOFTWAREPACKAGES): ``` /opt/hg-agent/... (static files) package/... (pyinstaller bundle) collectors/... (Diamond collectors) bin/... (symlinks to binaries in package bundle) /etc/opt/hg-agent/... (configuration files) /var/opt/hg-agent/ (spool or other state files) /var/log/hg-agent/ (log files) ``` The agent is distributed as a [PyInstaller](http://pythonhosted.org/PyInstaller/) bundle: this is a self-contained build of the whole agent system (including an embedded Python) which requires only a compatible libc. You typically shouldn’t need to run anything in /opt/hg-agent/bin, since just editing /etc/opt/hg-agent/hg-agent.conf will result in an automatic configuration reload. If something seems to be stuck, restart the service as normal, e.g. for Ubuntu 14.04: ``` $ sudo service hg-agent restart ``` In /var/log/hg-agent, you can see the metrics generated by diamond in archive.log, as well as the various daemon logs supervisord.log, diamond.log, periodic.log, forwarder.log, and receiver.log. ### [Processes](#processes) The agent runs as a process tree: ``` supervisord ─┬──┬─ diamond ─┬─ diamond (collector) │ │ ├─ diamond (collector) │ │ └─ ... │ └─ periodic ├──── metric receiver │ ├─ UDP │ └─ TCP └──── metric forwarder ├─ Spool Reader └─ Forwarder ``` This all runs as the non-privileged user hg-agent. #### [supervisord](#supervisord) A [process manager](http://supervisord.org/) which we use to manage the sub-processes that make up hg-agent. #### [diamond](#diamond) This popular [metric collection daemon](https://github.com/python-diamond/Diamond) does the primary work of hg-agent: gathering system metrics and exporting them to Hosted Graphite. #### [periodic](#periodic) A “sidecar” that manages periodic tasks for hg-agent, including (re)configuration of diamond and sending “heartbeat” metadata to Hosted Graphite. #### [receiver](#receiver) Threads that listen for metrics via both UDP and TCP. On receipt of metrics we perform some validation, and write the data out to a spool file (/var/opt/hg-agent/spool/\*.spool.\), this write happens once a second. By default, these will listen for metrics on localhost:2003. You may configure the ports for the UDP and TCP receivers to listen on by adding the following values to /etc/opt/hg-agent/hg-agent.conf: ``` udp_port: tcp_port: ``` #### [forwarder](#forwarder) Tails spool files in /var/opt/hg-agent/spool and forwards metrics to Hosted Graphite’s HTTP API. Metrics are batched and, by default, a POST request is sent every 0.5 seconds or once the batch size has reached 250. The agent maintains a progress file that tells us how much of each spool file has been forwarded. This allows us to recover in times when there may be some network connectivity issues between the host server and Hosted Graphite. ### [Configuration](#configuration) Agent configuration in /etc/opt/hg-agent/hg-agent.conf is used to generate a more complete configuration for diamond in /var/opt/hg-agent/diamond.conf. ### [Resource requirements](#resource-requirements) On a representative low-power 4-core Ubuntu 14.04 machine, hg-agent currently takes up the following resources: * \~13MiB of disk space for package files; * a further 100MiB (by default: 10 \* 10MB) for metric spool files; * \~0.02 process CPU seconds/s * \~150MiB physical memory (measured as [unique set size](https://psutil.readthedocs.io/en/latest/)) # Telegraf Telegraf is an easy to use monitoring client compatible with many operating systems that has a Graphite output plugin, and many useful input plugins for collecting a wide variety of system metrics. Contents * [Telegraf](/agents-guide/telegraf) * [Linux Systems](#linux-systems) * [Macos](#macos) * [Windows (via Powershell)](#windows-via-powershell) * [Telegraf Input Plugins](#telegraf-input-plugins) {% embed url="" %} Telegraf Video Tutorial {% endembed %} ### [Linux Systems](#linux-systems) 1. Download Telegraf and unzip it (see the [telegraf installation docs](https://docs.influxdata.com/telegraf/v1/install/) for up-to-date versions and installation commands for many operating systems). Packages and files are generally installed at `/etc/telegraf/` **Ubuntu/Debian (AMD)** ``` wget https://dl.influxdata.com/telegraf/releases/telegraf_1.30.0-1_amd64.deb sudo dpkg -i telegraf_1.30.0-1_amd64.deb ``` **RedHat/CentOS** ``` wget https://dl.influxdata.com/telegraf/releases/telegraf-1.30.0-1.x86_64.rpm sudo yum localinstall telegraf-1.30.0-1.x86_64.rpm ``` 2. Locate the configuration file at `/etc/telegraf/telegraf.conf` and open it in your preferred text editor, or a VIM shell. Make the following changes to the **telegraf.conf** file: * uncomment/configure the following lines: `[[outputs.graphite]]` `servers = ["carbon.hostedgraphite.com:2003"]` `prefix = "YOUR_API_KEY.telegraf"` * alternatively, you can download a preconfigured telegraf.conf file from the Agents => Telegraf as mentioned above. Download the preconfigured file, move it to `/etc/telegraf`, and replace the original. 3. Save your changes and start the telegraf service: `sudo service telegraf start` or `sudo systemctl start telegraf`. Within minutes, telegraf will start sending metrics into your Hosted Graphite account (with the ‘telegraf’ prefix). 4. Troubleshoot errors by running telegraf in the root directory of the telegraf.conf file: `telegraf --config /etc/telegraf/telegraf.conf` ### [Macos](#macos) 1. Install Telegraf: `brew install telegraf` 1. Packages are usually installed at: `/usr/local/etc/` (intel) or: `/opt/homebrew/etc/` (silicon) 2. Similar to the Linux steps listed above, locate the configuration file and open it in your preferred text editor. Uncomment/configure the following lines in your **telegraf.conf** file: > `[[outputs.graphite]]` > > `servers = ["carbon.hostedgraphite.com:2003"]` > > `prefix = "YOUR_API_KEY.telegraf"` 3. Restart telegraf: `brew services restart telegraf`. Metrics usually take about 3-5 minutes to appear in your Hosted Graphite account and will include the ‘telegraf’ prefix in the name. 4. Additionally, you can ‘serve’ the telegraf.conf file which can be helpful for troubleshooting any configuration issues: `telegraf --config /etc/telegraf/telegraf.conf` ### [Windows (via Powershell)](#windows-via-powershell) 1. Download Telegraf using the wget command (an alias for Invoke-Webrequest) and unzip it: ``` wget https://dl.influxdata.com/telegraf/releases/telegraf-1.5.2_windows_amd64.zip -O telegraf-1.5.2_windows_amd64.zip Expand-Archive .\telegraf-1.5.2_windows_amd64.zip ``` Note: The link for the most up-to-date version of the config file can be found on the [Telegraf downloads page](https://portal.influxdata.com/downloads). 2. Create a directory in Program Files, move the two telegraf files into it, and change the directory to the same location: ```bash mkdir 'C:\Program Files\Telegraf' mv .\telegraf-1.5.2_windows_amd64\telegraf\*.* 'C:\Program Files\Telegraf' cd 'C:\Program Files\Telegraf' ``` 3. Create a new configuration file that includes the Graphite output-filter. ```bash ./telegraf.exe --output-filter graphite config > telegraf.conf ``` 4. Open the telegraf.conf file using your preferred text editor. Uncomment the section titled **\[\[outputs.graphite]]** and uncomment/update the servers and prefix options: ```bash [[outputs.graphite]] servers = ["carbon.hostedgraphite.com:2003"] prefix = "YOUR_API_KEY.telegraf" ``` 5. Install telegraf to your services, and start it up: ``` ./telegraf.exe -service install -config 'C:\Program Files\telegraf\telegraf.conf' net start telegraf ``` Metrics will now appear in your Hosted Graphite account with the ‘telegraf’ prefix. More options for collecting Windows metrics include using the Windows Performance Counters plugin, which you can read more about in the [win\_perf\_counters docs](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/win_perf_counters/README.md)). Once telegraf is installed, you can download a custom telegraf config in your Hosted Graphite account if you navigate to Agents => Telegraf. Then install telegraf using the steps below, and replace the default config with the custom config at the path that telegraf is installed at.

Custom Telegraf Config

### [Telegraf Input Plugins](#telegraf-input-plugins) We love the Telegraf agent because it offers a wide variety of easy-to-install [input plugins](https://docs.influxdata.com/telegraf/v1.10/plugins/inputs/) that allow you to collect additional metrics from a range of popular services and technologies like: * Database ([MySQL](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/mysql/README.md), [PostgreSQL](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/postgresql/README.md), [MongoDB](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/outputs/mongodb/README.md), [Redis](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/redis/README.md), [Riak](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/riak/README.md), [RethinkDB](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/rethinkdb/README.md)) * Network ([SNMP](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/snmp/README.md), [Cisco Telemetry](https://github.com/influxdata/telegraf/tree/release-1.14/plugins/inputs/cisco_telemetry_gnmi), [Nstat](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/nstat/README.md), [SFlow](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/sflow/README.md), [IPtables](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/iptables/README.md), [Bond](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/bond/README.md), [Ethtool](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/ethtool/README.md), [LeoFS](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/leofs/README.md), [InfiniBand](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/infiniband/README.md), [JTI](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/jti_openconfig_telemetry/README.md), [Ping](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/ping/README.md)) * DNS ([Unbound](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/unbound/README.md), [PowerDNS](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/powerdns/README.md), [NSD](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/nsd/README.md)) * MQTT ([RabbitMQ](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/rabbitmq/README.md), [ActiveMQ](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/activemq/README.md)) * Apache ([Aurora](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/aurora/README.md), [Kafka](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/outputs/kafka/README.md), [Mesos](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/mesos/README.md), [Solr](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/solr/README.md), [Tomcat](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/tomcat/README.md), [CouchDB](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/couchdb/README.md), [Zipkin](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/zipkin/README.md)) * System Applications/Running Processes: ([Procstat](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/procstat/README.md), [Monit](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/monit/README.md)) * Other: ([Kubernetes](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/kubernetes/README.md), [Jenkins](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/jenkins/README.md), [Jolikia](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/jolokia/README.md), [Elasticsearch](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/outputs/elasticsearch/README.md), [Logstash](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/logstash/README.md), [Kibana](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/kibana/README.md), [Nginx](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/nginx/README.md), [HAproxy](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/haproxy/README.md), [uWSGI](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/uwsgi/README.md), [Docker](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/docker/README.md), [GitHub](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/github/README.md), [Mailchimp](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/mailchimp/README.md), [Salesforce](https://github.com/influxdata/telegraf/blob/release-1.28/plugins/inputs/salesforce/README.md), and many more!) If you need help or advice on configuring the telegraf agent to include some input plugins, send us a message at [support@hostedgraphite.com](mailto:support%40hostedgraphite.com) and we would be happy to walk you through the process. # K8 Telegraf Daemon Deploy Telegraf as a daemonset into your Kubernetes environment to collect and forward performance metrics from your clusters * Clone the telegraf-daemonset repository from the MetricFire GitHub: ``` telegraf-daemonset/ ├── kustomization.yaml └── resources/     ├── config.yaml     ├── daemonset.yaml     ├── namespace.yaml     ├── role.yaml     ├── role-binding.yaml     └── service_account.yaml ``` * Add your **HG API KEY** to the resources/config.yaml file * Point your kubectl command line utility to the context of the cluster you wish to monitor: `kubectl config use-context` * Deploy the kustomization.yaml manifest into your cluster: `kubectl apply -k .` * Generate the **Kubernetes Overview** dashboard from our [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library) to visualize performance metrics from your nodes, containers, pods, and more!

Kubernetes Overview dashboard

* See more configuration options in the official GitHub repo [HERE](https://github.com/influxdata/telegraf/blob/release-1.31/plugins/inputs/kubernetes/README.md), and our detailed blog article around deploying the Telegraf daemonset [HERE](https://www.metricfire.com/blog/guide-to-monitoring-kubernetes-using-telegraf-as-a-daemonset/). # OpenTelemetry OpenTelemetry is an open-source observability framework for collecting, processing, and exporting data to a monitoring backend - it can also be configured as a collector agent. Contents * [OpenTelemetry](/agents-guide/opentelemetry) * [Linux Installation](#linux-installation) * [Configuration](#configuration) * [OpenTelemetry Dashboard](#open-telemetry-dashboard) ## [OpenTelemetry](#opentelemetry) [OpenTelemetry](https://opentelemetry.io/docs/) makes a great collector agent because it provides a scalable way to collect and process metrics, traces, and logs in a single pipeline. To use OpenTelemetry as a collector agent, [install the Contrib version](https://opentelemetry.io/docs/collector/installation/) (otelcol-contrib) instead of the Core version, as it includes extended features needed to forward Graphite metrics. ### [Linux Installation](#linux-installation) Download the OpenTelemetry Contrib Collector package and unpack it. See their [installation docs](https://opentelemetry.io/docs/collector/installation/) for the latest versions and details on installing to different systems. Packages and files are generally installed at `/etc/otelcol-contrib/` **Ubuntu/Debian (AMD):** ``` wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.118.0/otelcol-contrib_0.118.0_linux_amd64.deb sudo dpkg -i otelcol-contrib_0.118.0_linux_amd64.deb ``` **RedHat/CentOS (AMD):** ``` wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.119.0/otelcol-contrib_0.119.0_linux_amd64.rpm && \ sudo rpm -ivh otelcol-contrib_0.119.0_linux_amd64.rpm ``` ### [Configuration](#configuration) Locate the configuration file at: `/etc/otelcol-contrib/config.yaml` and open it in your preferred text editor. To use OpenTelemetry Contrib as a collector agent, you must enable the [hostmetrics receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md) and [carbon exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/carbonexporter/README.md). Simply replace the config.yaml file, or include the following on your current config file (make sure to add your HG-API-KEY and HOSTNAME): ```yaml receivers: hostmetrics: collection_interval: 30s scrapers: cpu: {} memory: {} load: {} filesystem: {} disk: {} paging: {} network: {} processes: {} process: {} system: {} processors: batch: {} metricstransform: transforms: - include: ".*" match_type: regexp action: update new_name: ".opentel.$$0" operations: - action: add_label new_label: host new_value: exporters: carbon: endpoint: "carbon.hostedgraphite.com:2003" timeout: 10s service: pipelines: metrics: receivers: - hostmetrics processors: - batch - metricstransform exporters: - carbon ``` * Save your changes and start the otelcol-contrib service: `sudo systemctl restart otelcol-contrib`. Within two minutes, tagged system metrics will be forwarded to your Hosted Graphite account (with the ‘opentel’ prefix). * You can also serve the file manually, to help troubleshoot configuration errors: `otelcol-contrib --config /etc/otelcol-contrib/config.yaml` ### [OpenTelemetry Dashboard](#open-telemetry-dashboard) Our suggested configuration for OpenTelemetry will return [Tagged Metrics](/sending-metrics/graphite-tag-support) to the Hosted Graphite backend. We have a handy dashboard in our [Dashboard Library](/dashboard-and-graphs/dashboard-library) that you can generate to quickly visualize your system performance metrics - per host:

OpenTelemetry Agent Dashboard

If you need help or advice configuring OpenTelemetry as an agent, please send a message to: [support@hostedgraphite.com](mailto:support%40hostedgraphite.com) and we would be happy to walk you through the process. # collectd Agent collectd is a simple monitoring client which can be installed on most Linux systems. To simplify the setup, we provide a configuration file for you to use ### Installing and configuring collectd * Install [collectd](https://collectd.org/documentation.shtml) on your system. It’s usually available via your normal package manager, e.g. `apt-get install collectd` * From the Hosted Graphite Add-ons page, select collectd and download the appropriate configuration file. Version 5.4 works with most versions of collectd * Copy the downloaded configuration file to `/etc/collect/collectd.conf`. If you want to keep the original configuration file, rename it to collectd.conf.old * Configure optional plugins to collect data from many different technologies and running processes on your servers, see their table of plugins [here](https://collectd.org/wiki/index.php/Table_of_Plugins) * Restart collectd so that it uses the new configuration file, and metrics will appear in your Hosted Graphite account, prefixed with `collectd` ### **Generating and viewing the collectd dashboard** Once collectd is up and running, you can automatically generate a dashboard from our dashboard library. Navigate to Dashboards => [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library) and click 'Generate'.
For more detailed information on how to manually configure collectd, see our [collectd doc](/add-ons-and-integrations-guide/collectd-add-ons) in the Add-Ons and Integrations Guide. # StatsD Agent StatsD is a popular tool for aggregating your data into different statistical views. It provides things like counters, timers, and gauges to provide different ways to look at your data. StatsD can also be used to pre-aggregate your data before sending it to Hosted Graphite. You can configure your StatsD instance to send metrics to us by following our [*StatsD docs*](/add-ons-and-integrations-guide/statsd-add-on) in our Add-Ons and Integrations Guide. If you haven’t installed StatsD yet, you might also find the StatsD [installation instructions](https://github.com/statsd/statsd#installation-and-configuration) helpful. ### Hosted StatsD We also provide a Hosted StatsD solution if you don’t have the luxury of running a server yourself, check out the documentation for [*Hosted StatsD*](/add-ons-and-integrations-guide/statsd-add-on#hosted-statsd) to get set up with only a few clicks! Enable Hosted StatsD in your account to forward metrics to our statsd endpoint: statsd.hostedgraphite.com:8125 ### Try our Data Views Like the added counters but not interested in using StatsD? Learn about our [Data Views ](/additional-features/data-views)provided for all metrics. While Data Views aren’t direct equivalents to StatsD types, they provide similar ways to work with your data: * **counters** -> `:sum` (total count), `:sumrate` (per-second rate) * **gauges** -> `:avg`, `:min`, `:max` (aggregations of reported values) * **timers** -> `:avg`, `:min`, `:max`, `:95pct`, etc. (distribution-style views) # Diamond To collect useful system metrics in Hosted Graphite such as System CPU, User CPU, Load Average, and Physical Memory you can use a python daemon called Diamond. Diamond runs on your server, and intermittently sends these useful system metrics to Graphite. Here’s a quick primer on how to configure it. ### Configuring Diamond * **Download Diamond** - Available at [https://github.com/BrightcoveOS/Diamond](https://github.com/python-diamond/Diamond). * **Install Diamond** - Follow the installation instructions at . * **Edit the config file** - Usually located at `/etc/diamond/diamond.conf` > 1. At the top of the config file, add the HostedGraphiteHandler > > > ``` > > # Handlers for published metrics. > > handlers = diamond.handler.hostedgraphite.HostedGraphiteHandler > > ``` > > 2. Under the `[handlers]` `[[HostedGraphiteHandler]]` section, set: > > > ``` > > apikey = YOUR_API_KEY > > timeout = 15 > > batch = 1 > > ``` > > 3. Under the `[[default]]` section, you can change how frequently metrics are reported: > > 4. On our servers we also enable collectors for network metrics and MySQL: > > > ``` > > [[NetworkCollector]] > > enabled = True > > > > [[MySQLCollector]] > > enabled = True > > hosts = username:pass@localhost:3306/mysql, > > master = True > > innodb = True > > ``` * **More collectors** - Here is a [full list of Diamond collectors](https://github.com/BrightcoveOS/Diamond/wiki/Collectors) for various systems e.g. Hadoop, NetApp, Nginx, Postgres, RabbitMQ, Redis, and Zookeeper. * **Start Diamond** - Usually with `/etc/init.d/diamond start` but it may differ depending on your OS. Diamond should now be configured to send data to Hosted Graphite and it will appear under the ‘servers’ entry in your tree of metrics. Once you start receiving Diamond metrics, an automatic Diamond dashboard will be created in your account which offers a great starting point and can be customized further. # Add-Ons and Integrations Guide In this guide learn how to use Hosted Graphite with a wide array of popular services that we support to send metrics from multiple services and visualize and monitor your entire infrastructure. * [AWS CloudWatch](/add-ons-and-integrations-guide/aws-cloudwatch) * [Azure Monitor Metrics](/add-ons-and-integrations-guide/azure-monitor-metrics) * [Circle CI](/add-ons-and-integrations-guide/circle-ci) * [Cloudbees](/add-ons-and-integrations-guide/cloudbees) * [Collectd](/add-ons-and-integrations-guide/collectd-add-ons) * [DigitalOcean](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/digital-ocean) * [GitHub](/add-ons-and-integrations-guide/github) * [GitLab](/add-ons-and-integrations-guide/gitlab) * [Google Cloud Monitoring](/add-ons-and-integrations-guide/gcp-metrics) * [Heroku](/add-ons-and-integrations-guide/heroku) * [Hosted StatsD](/add-ons-and-integrations-guide/hosted-statsd) * [New Relic](/add-ons-and-integrations-guide/new-relic) * [Papertrail](/add-ons-and-integrations-guide/papertrail) * [Pingdom](/add-ons-and-integrations-guide/pingdom) * [Sentry](/add-ons-and-integrations-guide/sentry) * [Sitespeed](/add-ons-and-integrations-guide/sitespeed) * [StatsD](/add-ons-and-integrations-guide/statsd-add-on) # AWS CloudWatch Hosted Graphite provides an Amazon AWS CloudWatch integration available in the Add-Ons page of your account. This add-on syncs the metrics from the specified AWS services/regions into your HG account. To connect to your CloudWatch account, you need to set up **Identity and Access Management (IAM)** access keys in your [AWS Account](https://console.aws.amazon.com/iam/), with the appropriate permissions to allow Hosted Graphite to connect and collect your metrics. Contents * [AWS CloudWatch](/add-ons-and-integrations-guide/aws-cloudwatch) * [Configuring AWS](#configuring-aws) * [IAM Policy](#iam-policy) * [IAM User](#iam-user) * [Configuring Hosted Graphite](#configuring-hosted-graphite) * [Enabling The AWS CloudWatch Add-On](#enabling-the-aws-cloudwatch-add-on) * [Disabling The CloudWatch Add-On](#disabling-the-cloudwatch-add-on) * [Enabling Account Naming](#enabling-account-naming) * [Enable Route53 Metrics](#enable-route53-metrics) * [Disclaimer](#disclaimer) * [CloudWatch Costs](#cloudwatch-costs) * [AWS Overview Dashboard](#aws-overview-dashboard) * [Metric Name Mapping](#metric-name-mapping) * [List of Metric Name Mappings](#list-of-metric-name-mappings) ### [Configuring AWS](#configuring-aws) #### [IAM Policy](#iam-policy) Let’s create a policy that we will later attach to the user. * In the IAM Dashboard, click on Policies (on the left) or Customer Managed Policies in under IAM Resources, then click on [Create Policy](https://console.aws.amazon.com/iam/home#/policies$new?step=edit). * Switch to the JSON tab, replace the existing text with the policy provided below, and click ‘**Review policy**‘. * Give the policy a name (Eg: HG\_policy) and a description (optional). * Save the policy by clicking ‘**Create Policy**‘. ```json { "Statement": [ { "Sid": "PermissionsForMetrics", "Effect": "Allow", "Action": [ "cloudwatch:ListMetrics", "cloudwatch:GetMetricStatistics", "ec2:DescribeInstances", "ec2:DescribeVolumes", "rds:DescribeDBInstances", "route53:ListHealthChecks", "sqs:ListQueues", "elasticache:DescribeCacheClusters", "elasticloadbalancing:DescribeLoadBalancers", "kinesis:ListStreams", "redshift:DescribeClusters", "elasticmapreduce:ListClusters", "elasticmapreduce:DescribeCluster", "cloudfront:ListDistributions" ], "Resource": [ "*" ] }, { "Sid": "PermissionsForTags", "Effect": "Allow", "Action": [ "elasticache:ListTagsForResource", "elasticloadbalancing:DescribeTags", "cloudfront:ListTagsForResource", "route53:ListTagsForResource", "kinesis:ListTagsForStream", "rds:ListTagsForResource", "lambda:ListFunctions", "iam:GetUser" ], "Resource": [ "*" ] } ], "Version": "2012-10-17" } ``` #### [IAM User](#iam-user) With the policy set up, we’ll create a user to attach the policy to. We’ll use the Access Key/Secret Key tokens to give Hosted Graphite permission to import CloudWatch metric data. * In the IAM Console, click on ‘**Users**‘ on the left or under IAM Resources. Then select ‘**Add User**‘. * Give the user a name (Eg: HG\_addon) and ‘**Programmatic access**‘ for the Access type. * Click ‘**Next: Permissions**‘ and click the ‘**Attach existing policies directly**‘ button at the top. * Search for the policy we just created and attach it by clicking the corresponding checkbox, then press ‘**Next: Review**‘. * Click on ‘**Create User**‘ and copy the Access Key and Secret Key into the relevant fields on the add-on set-up page [here](https://www.hostedgraphite.com/app/addons/aws/account/). Optionally, you can download a copy of this account information, but keep it safe as this information can be sensitive.
### [Configuring Hosted Graphite](#configuring-hosted-graphite) #### [Enabling The AWS CloudWatch Add-On](#enabling-the-aws-cloudwatch-add-on) To enable the CloudWatch add-on, go to the [add-ons](https://www.hostedgraphite.com/app/addons/) page in your Hosted Graphite account and choose the option for **Amazon AWS CloudWatch**. You will be presented with the following page, from which you can create, edit or delete AWS configs or create, edit or delete tags:
If you click **AWS Tags** you will be presented with the following screen which allows you to edit the tags. Each tag has a unique name (shown on the left) and one or more values. Values are separated by a comma. If for example you enter “hello, world” for tag “Tag\_Next” then the CloudWatch Add-On will match any tag called “Tag\_Next” that has either “hello” or “world” as its value.
If you click **Cancel** and go back to the previous screen and click **Account Names**, you can see this screen from which you can edit a particular AWS configuration.

HG Cloudwatch UI

* **AWS Access Key** and **AWS Secret Access Key** These are the keys you saved when you [*set up your IAM user*](#configuring-aws). * **AWS Regions** Choose the regions containing the services you wish to monitor. You must choose at least one region. * **AWS Services** Choose the services which will be queried for metrics. * **EC2 Instance Aliasing** Choose this if you would like Hosted Graphite to import your EC2 instance metrics using an instance’s name instead of its ID. For instance aliasing to work, it is required that your instances have a “Name” tag defined whose value will be used in place of the instance id. **Please note** that enabling this feature will create new metrics in your account for each of your EC2 instances that have the “Name” tag. The old instance id based metrics will remain dormant until they expire. **Name tags containing spaces are currently not supported and will be dropped** * **EMR Cluster Aliasing** Select this if you would like Hosted Graphite to import your EMR cluster metrics using a cluster’s name instead of its ID. For cluster aliasing to work, it is required that your clusters have a “Name” tag defined whose value will be used in place of the cluster id. **Please note** that enabling this feature will create new metrics in your account for each of your EMR clusters that have the “Name” tag. The old cluster id based metrics will remain dormant until they expire. **Name tags containing spaces are currently not supported and will be dropped** * **Service tagging for this AWS account** Choose the services that you would like to enable tagged imports for. This will only import metrics from tagged resources for those services. Once you enable this per account, you can type in the tag’s “key”: “value” pairs that you would like to be imported. These key values must also be present in your AWS resources to be imported successfully. For more information on adding these tags go to the [AWS docs](http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/allocation-how.html). When you click on ‘**Save**‘, some basic checks will be performed on your keys if successful, your configuration will be saved. If you have at least one service chosen, the Cloudwatch add-on will be enabled. #### [Disabling The CloudWatch Add-On](#disabling-the-cloudwatch-add-on) Go to the [add-ons](https://www.hostedgraphite.com/app/addons/) page in your Hosted Graphite account, and choose the option for **Amazon AWS CloudWatch**. Click the **Delete** button and the CloudWatch add-on will be disabled for that account. #### [Enabling Account Naming](#enabling-account-naming) If you use multiple AWS Access Keys on your Hosted Graphite account for different AWS accounts, projects or environments, you can keep the metrics separated by assigning **Account Names** to your Access Keys. Under the list of Access Keys on your HG account, there’s a button to access the Account Names interface. There you can assign names to your Access Keys, which will then become part of the metric name for all metrics retrieved using that key. In the example below, the Access Key was assigned the name ‘test’, so metrics retrieved through that access key will follow the naming structure ‘aws.test.\[service]\*’
If you were previously not using an Account Name and have recently added one, your automatically generated AWS Dashboards will no longer map to the correct metrics. You can tick the option on the Account Names screen (above) to generate new dashboards which use the new Account Names. #### [Enable Route53 Metrics](#enable-route53-metrics) Amazon Route53 metrics are only received if you enable US East (N. Virginia) as the current region. These metrics are not available from any other region. ### [Disclaimer](#disclaimer) While we attempt to minimize the number of API calls that may incur Amazon charges, Hosted Graphite disclaims responsibility for potential costs incurred by the use of this add-on. Our add-on performs read-only requests to the CloudWatch API. Should the provided [*AWS Access Keys*](#configuring-aws) grant greater privileges than what our specified [*IAM Policy*](#iam-policy) defines, responsibility for any activity performed using those keys lies with the customer. #### [CloudWatch Costs](#cloudwatch-costs) Amazon AWS offers the first one million API requests at no charge. In excess of that, Amazon will charge $0.01 per 1000 requests. See the [CloudWatch Pricing](https://aws.amazon.com/cloudwatch/pricing/) page for more information on Amazon pricing. We make calls to every 10min fetch your metrics from AWS, but feel free to reach out to [support@hostedgraphite.com](mailto:support%40hostedgraphite.com) if you would like us to configure a custom pull interval for you. We aim to make as few requests as possible to fetch your metrics. If you have many instances, or are monitoring many services, you will likely exceed this boundary. If this is the case, these charges will likely be negligible in comparison to what [Amazon charges](https://aws.amazon.com/cloudwatch/pricing/) just to have monitoring enabled for those services. ### [AWS Overview Dashboard](#aws-overview-dashboard) This can be located and generated from within the [Dashboard Library](/dashboard-and-graphs/dashboard-library), and offers a great starting point for basic AWS monitoring!
This overview dashboard assumes that you have enabled Account Names, which can be configured in Add-Ons => AWS. ### [Metric Name Mapping](#metric-name-mapping) The AWS metrics for each service are mapped to Hosted Graphite metric names as follows: ```bash aws.{service}.{region}.{grouping}.{id}.{metricname} ``` * **service** - A short token representing the service, e.g. *ec2*, or *rds*. * **region** - The AWS region, e.g. *us-east-1*. * **grouping** - A short token representing the grouping (‘dimension’ in AWS speak) for the metric, e.g. **inst** for *InstanceId*. * **id** - The identifier for the service/instance, e.g. *Instance id* or *Name tag* for EC2. * **metricname** - The AWS metric name is directly used, e.g. *CPUUtilization*. | AWS Service | AWS ‘Dimension’ | HG metric name | | --------------------------------- | -------------------- | -------------------------------------------------- | | Elastic Compute Cloud (EC2) | InstanceId | aws.ec2.{region}.inst.{id}.CPUUtilization | | Elastic Block Store (EBS) | VolumeId | aws.ebs.{region}.vol.{id}.VolumeWriteBytes | | Relational Database Service (RDS) | DBInstanceIdentifier | aws.rds.{region}.inst.{id}.CPUUtilization | | Elastic MapReduce (EMR) | JobFlowId | aws.emr.{region}.job\_flow\_id.{id}.CPUUtilization | So, for example, the *CPUUtilization* metric for the ‘*i-abcd1234*‘ EC2 instance in Virginia will be imported as ‘*aws.ec2.us-east-1.inst.i-abcd1234.CPUUtilization*‘. You can find the mapping for each individual AWS service we support below: #### [List of Metric Name Mappings](#list-of-metric-name-mappings) If you are having trouble querying Hosted Graphite metrics that have been imported from your AWS services perhaps you are not entering the metric query correctly. The format of these metrics changes per service and has multiple formats per service. These formats depend on ‘Dimensions’ in the AWS metric or in some cases if the AWS metric came from an AWS ARN (Amazon Resource Name). For further information on AWS Dimensions and AWS ARNs please consult their corresponding AWS documentation. Here is a list of how each individual service’s metrics are formatted in Hosted Graphite:
AWS ServiceAWS ‘Dimensions’HG metric names
API GatewayApiName, Method, Resource, Stageaws.apigateway.{region}.apiname.{ApiName}.method.{Method}.resource.{Resource}.stage.{Stage}.{name}
ApiName, Stageaws.apigateway.{region}.apiname.{ApiName}.stage.{Stage}.{name}
ApiNameaws.apigateway.{region}.apiname.{ApiName}.{name}
Application Elastic Load BalancingLoadBalancer, AvailabilityZone, TargetGroupaws.alb.{region}.inst.{LoadBalancer}.{AvailabilityZone}.{TargetGroup}.{name}
LoadBalanceraws.alb.{region}.inst.{LoadBalancer}.{name}
arn_format: arn:aws:elasticloadbalancing:{region}:{account-id}:loadbalancer/{loadbalancername}
AutoScaling GroupsAutoScalingGroupNameaws.autoscaling.{AutoScalingGroupName}.{name}
CloudFrontDistributionIdaws.cloudfront.distribution_id.{DistributionId}.{name}
CloudWatch SyntheticsCanaryNameaws.cloudwatchsynthetics.{region}.canaryname.{CanaryName}.{name}
Direct ConnectConnectionId, VirtuaInterfaceIdaws.directconnect.{region}.connection_id.{ConnectionId}.virtual_interface_id. {VirtualInterfaceId}.{name}
ConnectionId, OpticalLaneNumberaws.directconnect.{region}.connection_id.{ConnectionId}.optical_lane_number.{OpticalLaneNumber}.{name}
DynamoDBTableName, Operation, GlobalSecondaryIndexNameaws.dynamodb.{region}.table.{TableName}.operation.{Operation}.secondary_index. {GlobalSecondaryIndexName}.{name}
Elastic Block StoreVolumeIdaws.ebs.{region}.vol.{VolumeId}.{name}
Elastic Compute CloudInstanceIdaws.ec2.{region}.inst.{InstanceId}.{name}
EC2 SpotfleetAvailabilityZone, FleetRequestId, InstanceTypeaws.spotfleet.{AvailabilityZone}.{FleetRequestId}.{InstanceType}.{name}
FleetRequestId, InstanceTypeaws.spotfleet.{FleetRequestId}.{InstanceType}.{name}
FleetRequestId, AvailabilityZoneaws.spotfleet.{AvailabilityZone}.{FleetRequestId}.{name}
FleetRequestIdaws.spotfleet.{FleetRequestId}.{name}
ECSClusterName, ServiceNameaws.ecs.{region}.cluster.{ClusterName}.service.{ServiceName}.{name}
ClusterNameaws.ecs.{region}.cluster.{ClusterName}.{name}
ECS Container InsightsClusterNameaws.ecs_containerinsights.{region}.cluster.{ClusterName}.{name}
ClusterName, ServiceNameaws.ecs_containerinsights.{region}.cluster.{ClusterName}.service.{ServiceName}.{name}
ClusterName, TaskDefinitionFamilyaws.ecs_containerinsights.{region}.cluster.{ClusterName}.task_definition_family.{TaskDefinitionFamily}.{name}
Elastic File SystemFileSystemIdaws.efs.{region}.FileSystemId.{FileSystemId}.{name}
Elastic Load BalancingLoadBalancerName, AvailabilityZoneaws.elb.{region}.inst.{LoadBalancerName}.{AvailabilityZone}.{name}
LoadBalancerNameaws.elb.{region}.inst.{LoadBalancerName}.{name}
Elasticsearch ServiceDomainName, NodeId, ClientIdaws.es.{region}.{DomainName}.{NodeId}.{ClientId}.{name}
DomainName, ClientIdaws.es.{region}.{DomainName}.{ClientId}.{name}
ElastiCacheCacheClusterId, CacheNodeIdaws.elasticache.{region}.cache_cluster_id.{CacheClusterId}.cache_node_id.{CacheNodeId}.{name}
CacheClusterIdaws.elasticache.{region}.cache_cluster_id.{CacheClusterId}.{name}
arn_format: arn:aws:elasticache:{region}:{account_id}:cluster:{instance_name}
Elastic MapReduceJobFlowIdaws.emr.{region}.job_flow_id.{JobFlowId}.{name}
Kinesis FirehoseDeliveryStreamNameaws.firehose.{region}.{DeliveryStreamName}.{name}
Kinesis StreamsStreamNameaws.kinesis.{region}.stream_name.{StreamName}.{name}
Lambda FunctionsFunctionNameaws.lambda.{region}.function.{FunctionName}.{name}
LogsLogGroupNameaws.logs.{region}.{LogGroupName}.{name
LogGroupName, DestinationType, FilterNameaws.logs.{region}.{LogGroupName}.destination_type.{DestinationType}.filter_name.{FilterName}.{name}
Resource, Service, Type, Classaws.logs.{region}.resource.{Resource}.service.{Service}.type.{Type}.class.{Class}.{name}
MemoryDBClusterName, NodeNameaws.memorydb.{region}.cluster.{ClusterName}.node.{NodeName}.{name}
Role, ReplicationGroupIdaws.memorydb.{region}.role.{Role}.replication_group_id.{ReplicationGroupId}.{name}
MQBroker, Topicaws.mq.{region}.{Broker}.{Topic}.{name}
Broker, Queueaws.mq.{region}.{Broker}.{Queue}.{name}
Brokeraws.mq.{region}.{Broker}.{name}
Relational Database ServiceDBInstanceIdentifieraws.rds.{region}.inst.{DBInstanceIdentifier}.{name}
arn_format: arn:aws:rds:{region}:{account_id}:db:{instance_name}
RedshiftNodeID, ClusterIdentifieraws.redshift.{region}.cluster.{ClusterIdentifier}.node.{NodeID}.{name}
ClusterIdentifieraws.redshift.{region}.cluster.{ClusterIdentifier}.{name}
NodeIDaws.redshift.{region}.node.{NodeID}.{name}
Route 53HealthCheckIdaws.route53.{region}.health_check_id.{HealthCheckId}.{name}
S3 (Simple Storage Service)BucketName, StorageTypeaws.s3.{region}.bucket.{BucketName}.type.{StorageType}.{name}
Simple E-mail Serviceaws.ses.{region}.{name}
Simple Notification ServiceTopicNameaws.sns.{region}.{TopicName}.{name}
SMSType, Countryaws.sns.{region}.{SMSType}.{Country}.{name}
aws.sns.{region}.{name}
Simple Queue ServiceQueueNameaws.sqs.{region}.queue.{QueueName}.{name}
# Azure Monitor Metrics Hosted Graphite provides an Azure add-on available in the Add-Ons page of your account. This add-on syncs the metrics from the specified Azure resources into your account. Contents * [Configure Azure](#configuring-azure) * [Configuring Hosted Graphite](#configuring-hosted-graphite) * [Enabling Azure Add-0n](#enabling-the-azure-add-on) * [Additional Configs](#additional-configs) * [Tags](#tags) * [Account Naming](#enabling-account-naming) * [Metric Name Mapping](#metric-name-mapping) * [Azure Overview Dashboard](#azure-overview-dashboard) * [Disabling the Azure Add-0n](#disabling-the-azure-add-on) * [Disclaimer](#disclaimer) {% embed url="" %} Azure Video Tutorial {% endembed %} ### [Configuring](#configuring-aws)[ Azure](#configuring-azure) To connect to your Azure account, you need to provide the following information: * [Tenant ID](#tenant-id) * [Subscription ID](#subscription)[s](#subscription) * [Client Info (ID and Secret)](#client-info) * [Role](#role) #### [Tenant ID](#tenant-id) Search for 'Active Directory' in the search bar which should take you to the tenant overview. Here you will be able to find the tenant id.
#### [Subscription](#subscription) Within the subscription service interface, access the IDs by navigating to the "Overview" tab. Ensure you provide all the necessary IDs for the subscriptions from which you intend to retrieve metrics.
#### [Client ](#client-id)[Info](#client-info) To get the Client ID along with the Client Secret, an application will need to be registered, this will act as authentication allowing Hosted Graphite to pull metrics.
Once the App has been created, the Client ID can be obtained from the overview tab. A Client Secret will need to be generated for the app. Navigate to the 'Certificate & Secrets' tab on the right. The Client Secret will be the under the 'value' column.
#### [Role](#role) The last step is to add permissions to the subscription, this will pass down the permission to all groups and resources within the subscription. Back in the subscription overview navigate to 'Access Control (IAM)'. The role required to pull the metrics is 'Reader'.
In the members section make sure to include the App that was created in the previous step. Review and sign the role assignment.
### [Configuring Hosted Graphite](#configuring-hosted-graphite) #### [Enabling The Azure Add-On](#enabling-the-azure-add-on) To enable the Azure add-on, go to the [add-ons](https://www.hostedgraphite.com/app/addons/) page in your Hosted Graphite account and choose the option for **Azure**. You will be presented with the following page; here you will be able to add, edit, and delete any Azure configurations. This page also provides options to add account names and tags.
The main Azure configuration page looks like the following:

HG Azure UI

Before saving make sure to include the resource to monitor by selecting the enable checkbox. Tagging is also a feature that can be enabled. If 'Tags' checkbox is enabled then tagging will only be applied to that specific resource. If you don't want tagging to be enabled for a resource then leave it unselected. More information on the available metrics can be viewed [here](https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/metrics-index). If you require a specific resource that is not listed, please don't hesitate to email us and we'll work swiftly to add it. ### [Additional Configs](#additional-configs) #### [Tags](#tags) If you click **Azure Tags** you will be presented with the following screen which allows you to edit tags. Each tag has a unique name (shown on the left) and one or more values. Values are separated by a comma. **Example**: Entering “Tag\_Next”: “hello, world” will configure the Add-On to only pull metrics from resources that have the tag “Tag\_Next” and at least one of the matching values of "hello" or "world" for that tag. If tagging is not selected for any services, but tags are configured for the Add-On, it will act as if tagging is enabled for all services.
[Enabling Account Naming](#enabling-account-naming) If you use multiple Azure Client IDs on your Hosted Graphite account for different Azure accounts, projects or environments, you can keep the metrics separated by assigning **Account Names** to your Client IDs. Under the list of Client IDs on your HG account, there’s a button to access the Account Names interface. There you can assign names to your Client IDs, which will then become part of the metric name for all metrics retrieved using that ID.
### [Metric Name Mapping](#metric-name-mapping) The Azure metrics for each service are mapped to Hosted Graphite metric names as follows: ```bash azure.{tenant_id}.{subscription_id}.{resource_group}.{namespace}.{resource_name}.{region}.{metricname} ``` If AccountName is enabled then the tenant\_id will be replaced by the account name provided. ### [Azure Overview Dashboard](#azure-overview-dashboard) This can be found and generated from within the [Dashboard Library](/dashboard-and-graphs/dashboard-library), and offers a great starting point for monitoring your Azure services!
This overview dashboard assumes that you have enabled Account Names, which can be configured in Add-Ons => Azure. ### [Disabling The Azure Add-On](#disabling-the-azure-add-on) Go to the [add-ons](https://www.hostedgraphite.com/app/addons/) page in your Hosted Graphite account, and choose the option for **Azure**. Click the **Delete** button and the Azure add-on will be disabled for that account. ### [Disclaimer](#disclaimer) While we attempt to minimize the number of API calls that may incur Azure charges, Hosted Graphite disclaims responsibility for potential costs incurred by the use of this add-on. For more information please review Azure's API [cost](https://azure.microsoft.com/en-us/pricing/details/monitor/) and [rate-limiting ](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling#subscription-and-tenant-limits)information Our add-on performs read-only requests to the Azure Monitor API. Should the provided [Azure Credentials ](#configuring-azure)grant greater privileges than what is specified in [Role](#role), responsibility for any activity performed using those keys lies with the customer. # GCP Metrics Hosted Graphite provides a Google Cloud Platform add-on available in the Add-Ons page of your account. This add-on syncs the metrics from the specified GCP services into your account. Contents * [Configure GCP](#configuring-gcp) * [Create a Service Account Key](#create-a-service-account-key) * [Roles](#roles) * [Enable Monitoring](#enable-monitoring) * [Configuring Hosted Graphite](#configuring-hosted-graphite) * [Enabling GCP Add-On](#enabling-the-gcp-add-on) * [Additional Configs](#additional-configs) * [Tags](#tags) * [Account Naming](#enabling-account-naming) * [Metric Name Mapping](#metric-name-mapping) * [GCP Overview Dashboard](#gcp-overview-dashboard) * [Disabling the GCP Add-0n](#disabling-the-gcp-add-on) * [Disclaimer](#disclaimer) {% embed url="" %} GCPP Video Tutorial {% endembed %} ### [Configuring GCP](#configuring-gcp) To connect to your GCP project, you need to provide the following information: * Service Account Key #### [Create a Service Account Key](#create-a-service-account-key) In the main Google Console page select the 'IAM & Admin' panel. On the side panel navigate to the 'Service Account' link:

GCP IAM Policy

and create a Service account. #### [Roles](#roles) In the creation of the service account there are two roles that need to be added order for Hosted Graphite to pull these metrics into your account: 'monitoring.viewer' and 'computer.viewer' (if monitoring compute services).

monitoring.viewer

compute.viewer

Once the account has been created, in the Service Accounts page click on the account and create a key.

creating a service key

Ensure that the key type is JSON:

key type-json

This will download a json file to you computer, which can then be uploaded to Hosted Graphite giving us permission to pull metrics from your GCP services. #### [Enable Monitoring](#enable-monitoring) The last thing you will need to do in your GCP account is to enable the Stackdriver API, which enables monitoring for your active GCP services:

enable the stackdriver API

### [Configuring Hosted Graphite](#configuring-hosted-graphite) #### [Enabling The GCP Add-On](#enabling-the-azure-add-on) To enable the GCP add-on, go to the [add-ons](https://www.hostedgraphite.com/app/addons/) page in your Hosted Graphite account and choose the option for **GCP**. You will be presented with the following page; here you will be able to add, edit, and delete any GCP configurations. This page also provides options to add account names and tags.

GCP Add-on

The main GCP configuration page looks like the following:

HG GCP UI

Before saving make sure to include the resource to monitor by selecting the enable checkbox along with uploading the service account key. Tagging is also a feature that can be enabled. (Google refers to them as labels but they serve identical purposes). If 'Labels' checkbox is enabled then tagging will only be applied to that specific resource. If you don't want tagging to be enabled for a resource then leave it unselected. More information on the available metrics can be viewed [here](https://cloud.google.com/monitoring/api/metrics_gcp). If you require a specific GCP service that is not listed, please don't hesitate to email us and we'll work swiftly to add it. ### [Additional Configs](#additional-configs) #### [Tags](#tags) If you click **GCP Labels** you will be presented with the following screen which allows you to edit labels. Each label has a unique name (shown on the left) and one or more values. Values are separated by a comma. **Example**: Entering “Tag\_Next”: “hello, world” will configure the Add-On to only pull metrics from resources that have the label “Tag\_Next” and at least one of the matching values of "hello" or "world" for that label. If labels are not selected for any services, but labels are configured for the Add-On, it will act as if labeling is enabled for all services.

GCP Labels

#### [Enabling Account Naming](#enabling-account-naming) If you use multiple GCP auth keys on your Hosted Graphite account for different GCP projects, you can keep the metrics separated by assigning **Account Names** to your Client IDs. Under the list of Client IDs on your HG account, there’s a button to access the Account Names interface. There you can assign names to your Client IDs, which will then become part of the metric name for all metrics retrieved using that ID.

GCP Account Name

### [Metric Name Mapping](#metric-name-mapping) The Google metrics for each service are mapped to Hosted Graphite metric names as follows: ```bash gcp.{client_id}.{service}.{project_id}.{location/zone}.({label1}.{label2}...).{metric_name} ``` If AccountName is enabled then the client\_id will be replaced by the account name provided. The labels are determined by the resource type and the metric, causing some metrics to have more than one label per metric. For more information regarding resource type label please refer to the GCP [documentation](https://cloud.google.com/monitoring/api/resources). ### [GCP Overview Dashboard](#gcp-overview-dashboard) This can be found and generated from within the [Dashboard Library](/dashboard-and-graphs/dashboard-library), and offers a great starting point for monitoring your Google Cloud services!

GCP Overview Dashboard

### [Disabling The GCP Add-On](#disabling-the-gcp-add-on) Go to the [add-ons](https://www.hostedgraphite.com/app/addons/) page in your Hosted Graphite account, and choose the option for **GCP**. Click the **Delete** button and the GCP add-on will be disabled for that account. ### [Disclaimer](#disclaimer) While we attempt to minimize the number of API calls that may incur Google Cloud Platform charges, Hosted Graphite disclaims responsibility for potential costs incurred by the use of this add-on. For more information please review GCP's API [cost](https://console.cloud.google.com/marketplace/product/google/monitoring.googleapis.com) and [rate-limiting](https://cloud.google.com/monitoring/quotas#more-quota) information. Our add-on performs read-only requests to the Google Cloud API. Should the provided [GCP Service Account Key](#create-a-service-account-key) grant greater privileges than what is specified in [Roles](#roles), responsibility for any activity performed using those keys lies with the customer. # Carbon-C-Relay Carbon-C-Relay provides an efficient way to aggregate, filter, and forward metrics to Hosted Graphite, particularly for systems within a private network. Carbon-C-Relay is a lightweight, high-performance proxy that acts as an intermediary between metric sources and Hosted Graphite. It is commonly used for pre-aggregation and securely forwarding metrics from private networks to external destinations. You can locate the latest releases/assets [HERE](https://github.com/grobian/carbon-c-relay/releases/tag/v3.8.1), and see configuration syntax [HERE](https://github.com/grobian/carbon-c-relay?tab=readme-ov-file#configuration-syntax). Below is a basic sample configuration and setup: Installation (Linux): ``` sudo apt install -y carbon-c-relay ``` Modify your configuration file with the following (generally located at: */etc/carbon-c-relay.conf*): ``` rewrite ^([a-zA-Z0-9_\.\-]*) into .\1 ; cluster hg forward carbon.hostedgraphite.com:2003 ; # Forward internal relay system metrics with the carbon.relays prefix match ^carbon\.relays\..* send to hg stop ; match * send to hg stop ; ``` Restart the service and check status: ``` sudo systemctl restart carbon-c-relay sudo systemctl status carbon-c-relay ``` Now, any metrics sent to **localhost:2003** will be collected and forwarded by the Carbon-C-Relay service. Additionally, internal relay performance metrics (carbon.relays.\*) will be automatically forwarded to your Hosted Graphite account.

Carbon-C-Relay Example Dashboard

# Circle CI CircleCI is a continuous integration tool that allows you to rapidly run tests on every change to your code. One useful way of monitoring the changes in your codebase is by adding annotations whenever your tests run. This handy Hosted Graphite webhook you can do just that! ### How to set up a CircleCI webhook CircleCI’s documentation lists [how to edit your CircleCI configuration to add a new webhook url](https://circleci.com/docs/configuration#notify/). To get your webhook URL from Hosted Graphite - Simply click the “Copy CircleCI webhook URL” button on the [add-ons](https://www.hostedgraphite.com/app/addons/) page, and add the URL to your circle configuration.

CircleCI Webhook

To see these events as [annotations](https://docs.hostedgraphite.com/api-guides/annotations-and-events-api#dashboard-annotations) on your graph, just enable them in your dashboard settings, and query by Tag:

CircleCI Annotation

# Cloudbees Use a webhook to import real-time event data from your Cloudbees environment Use our webhook in your [Cloudbees](https://www.cloudbees.com/) (formerly known as Codeship) account to track events from your CI/CD pipelines. These are sent to Hosted Graphite in real-time and can be displayed on your graphs in the form of annotations. First, copy the webhook URL from within the Hosted Graphite Add-Ons UI, and upload it to your Jenkins instance:

Cloudbees Webhook

Then you can configure a new [annotation](https://docs.hostedgraphite.com/api-guides/annotations-and-events-api#dashboard-annotations) in your dashboard, using the cloudbees Tag:
Now you will receive annotation markers on your graphs from your Cloudbees events. This is useful for comparing metric data before and after code deployments or changes in your Jenkins environment. # Collectd Add Ons collectd is a daemon which collects system performance statistics, can be configured to a Graphite output, and then easily forwarded to a Hosted Graphite account. * See the [collectd first steps](https://collectd.org/wiki/index.php/First_steps) for installation. * Once installed, configure the [write\_graphite](https://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_write_graphite) collectd plugin with the following settings: ``` Host "carbon.hostedgraphite.com" Port "2003" Prefix ".collectd." ``` * Your API key can be found on the main page of your Hosted Graphite account. Review the [write\_graphite](https://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_write_graphite) plugin documentation for details of the other write\_graphite plugin settings. ### Downloadable Configuration: You can download a configuration file for collectd, pre-filled with your Hosted Graphite account details, from our Agents page. These configurations are compatible with multiple versions of collectd but we provide v5.2 and v5.4 for the purpose of keeping the config file as close to the default one as possible.

Configure the collectd Agent

By default, a number of [collectd plugins](https://collectd.org/wiki/index.php/Table_of_Plugins) are enabled including CPU, DF (file system usage), disk, entropy, interface, load, memory, processes, rrdtool, swap, users, and as previously mentioned write\_graphite. There are also many other configureable plugins for DNS, IP Tables, Kafka, Logstash, MQTT, nginx, Postgres, Redis, SNMP, and more! Once downloaded, you can place this file in `/etc/collectd/` (or where ever you’ve configured collectd to store its config file). Then restart the collectd daemon: `sudo service collectd restart` or `sudo systemctl restart collectd` Roughly 150 default metrics per machine will arrive within minutes, and will be prefixed with: `collectd.*` ### Collectd Dashboard: There is a collectd dashboard (v5.4) which you can generate using our [Dashboard Library](/dashboard-and-graphs/dashboard-library).

Collectd Dashboard

This dashboard aims to provide an overview of the default plugins provided by collectd, and can be further customized depending on which collectd plugins are enabled in your **collectd.conf** file. # Digital Ocean This integration automatically imports infrastructure metrics from your DigitalOcean account, allowing you to visualize, alert on, and monitor your DO resources. ### Configuration Hosted Graphite automatically imports metrics from your DigitalOcean account resources, including statistics for Apps, Databases, Autoscale Pools, Droplets, and Load Balancers. If you don't already have a [DigitalOcean Personal Access Token](https://docs.digitalocean.com/reference/api/create-personal-access-token/), create one by navigating to API => Personal Access Tokens in the DigitalOcean Control Panel. Then open the DigitalOcean integration in Hosted Graphite, paste your token into the API Token field, and click Save Config. Once the configuration has been saved, Hosted Graphite will begin collecting metrics from your supported DigitalOcean resources. Metrics are pulled from the DigitalOcean API every 10 minutes and are prefixed with: *digitalocean.\**
DigitalOcean Configuration
#### Tag Filtering (Optional) If you only want to import metrics from specific resources, you can configure one or more DigitalOcean tags within the integration. This can help limit the total number of DO metrics stored with us. When tag filtering is enabled, Hosted Graphite will only import metrics from resources that match the configured tags. **NOTE**: DigitalOcean does not expose tags for Apps or Load Balancers, so tag filters are not applied to those resource types. When tag filtering is enabled, Apps and Load Balancers will still be imported, while Droplets, Databases, and Autoscale Pools are filtered by matching tags. ### Visualization We have a pre-made DigitalOcean Overview dashboard that can be found in our [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library). Generate this dashboard with only 1 click, to quickly visualize your DO metrics.
DigitalOcean Overview Dashboard
If you don't already have a Hosted Graphite account, you can start a 14-day free trial [here](https://www.hostedgraphite.com/accounts/signup/). You can also [schedule a product demo here ](https://calendly.com/metricfire-sales/hostedgraphite-demo)to speak with a knowledgeable member of our team who can provide support and advice on getting up and running quickly. # GitHub Hosted Graphite provides a post-receive github hook, which allows you to flag when commits are added to your repos with dashboard annotations. ### How to Send GitHub Commit Data to Hosted Graphite To get GitHub commits flagged in Hosted Graphite follow these instructions: * **Go to the Addons page, and in the Annotation Add-Ons section click on the GitHub card.** * **Copy the webhook URL, including your API key.**

Copy the webhook URL from the GitHub add-on page

* **Browse to your GitHub repo, visit the ‘settings’ section, then ‘webhooks’** * **Under ‘Webhooks’ click the “Add Webhook” button.** * **Enter the URL in the box provided** * **Select ‘application/json’ as the content type, and ‘send me everything’**

Enter your webhook URL into the Github New Webhooks page

* Hit `Save` Your GitHub events will now be sent to the webhook URL and saved on your account as [Annotations](/api-guides/annotations-and-events-api). Use the following tags to search for the annotations you want to display: * **Push Event Tags** – ‘github’, ‘push’, \, \, \ * **PR Event Tags** – ‘github’, ‘pull-request’, \, \, \ * **Deploy Event Tags** – ‘github’, ‘deployment’, \, \, \, \

GH Annotations

# GitLab Use a webhook to import real-time events from your GitLab repositories and deployments Use our webhook in your GitLab environment to report events from your repositories and deployments. These [GitLab events](https://docs.gitlab.com/ee/user/project/integrations/webhook_events.html) are sent to Hosted Graphite in real-time and can be displayed on your graphs in the form of annotations. First, copy the webhook URL from within the Hosted Graphite Add-Ons UI, and paste into your GitLab account (Settings => Webhooks):

GitLab Webhook URL

Then you can configure a new [annotation](https://docs.hostedgraphite.com/api-guides/annotations-and-events-api#dashboard-annotations) in your dashboard, using the Event Tags for each type of event you want to display:

GitLab Annotation Configuration

You will now receive dashboard annotations for each type of event configured, such as a push event. These markers are useful because you can see if any of your metric data has changed before or after an event within your codebase. # Heroku Monitor your Heroku Apps with Hosted Graphite. Contents * [Installing the Hosted Graphite Heroku Add-On](#installing-the-hosted-graphite-heroku-add-on) * [Heroku Auto-Dashboard](#heroku-auto-dashboard) * [Heroku Add-On Metrics](#heroku-add-on-metrics) * [Local Development Configuration](#local-development-configuration) * [Accessing the API Key Within Your App](#accessing-the-api-key-within-your-app) * [Custom Metrics](#custom-metrics) * [HG Heroku Configuration](#hg-heroku-configuration) * [Router Path Metrics](#router-path-metrics) * [Path Aggregates](#path-aggregates) * [Process Aggregates](#process-aggregates) * [Host Aggregates](#host-aggregates) * [Heroku Event Annotations](#heroku-event-annotations) * [Monitoring Heroku Metrics on an Existing HG Account](#monitoring-heroku-metrics-on-an-existing-hg-account) {% embed url="" %} Heroku Video Tutorial {% endembed %} ### [Installing the Hosted Graphite Heroku Add-On](#installing-the-hosted-graphite-heroku-add-on) Hosted Graphite has a [Heroku Add-On](https://elements.heroku.com/addons/hostedgraphite)! Installing the add-on can be done through the UI of your Heroku account in the ‘Resources’ section, or by issuing the following command in your Heroku CLI: > ``` > heroku addons:create hostedgraphite -a > ``` Make sure to replace the variable **\** with the name of your application, as it appears in your Heroku account. Once the add-on is installed, a new application configuration key will be created, called `HOSTEDGRAPHITE_APIKEY`. Confirm the key by issuing this command: > ``` > heroku config:get HOSTEDGRAPHITE_APIKEY -a > ``` You can now navigate to your new account by clicking on the Hosted Graphite add-on link in the Heroku UI, and wait for the metrics to appear.

Access your new HG account

If you create a Hosted Graphite account via the Heroku Add-On, you do not need to configure any log drains. For more documentation around our Heroku Marketplace Add-On, please visit the [Hosted Graphite on Heroku](https://devcenter.heroku.com/articles/hostedgraphite) page. ### [Heroku Auto-Dashboard](#heroku-auto-dashboard) This dashboard will be automatically created when you enable the Heroku Add-On, or configure Log-Drain metrics to your [existing Hosted Graphite account](#monitoring-heroku-metrics-on-an-existing-hg-account).
Using Heroku’s [Log Drain](https://devcenter.heroku.com/articles/log-drains) and [Log-Runtime](https://devcenter.heroku.com/articles/log-runtime-metrics) metrics we retrieve and process Heroku’s syslog information for your Dynos including: * **CPU Load averages** - 1/5/15-minute averages * **Memory and Swap** - Resident memory, disk cache, swap, total memory, and cumulative totals for pages written to/read from disk. * **HTTP metrics** - The number of requests broken down by HTTP method and status codes, data transferred, and connect/service times. * **Process metrics** - Memory stats from your [One-off process](https://devcenter.heroku.com/articles/process-model) as defined in your Procfile (scheduler, release, run, worker, etc). **Troubleshooting**: If you are not receiving the web (dyno) metrics, you need to **enable log-runtime metrics** for your Heroku app. Log into your Heroku CLI and run the following commands: ``` heroku labs:enable log-runtime-metrics -a ``` ``` heroku restart -a ``` ### [Heroku Add-On Metrics](#heroku-add-on-metrics) If you use the Heroku [Postgres add-on](https://elements.heroku.com/addons/heroku-postgresql) at the **standard** or **premium** tier, these metrics will automatically be forwarded to your Hosted Graphite account. They are prefixed with: **heroku.\.heroku-postgres.\.\** and report dyno statistics for cpu load, memory, db\_size, tables, active/waiting-connections, tmp-disk-available/used, read/write operations, and more! We also have a handy Heroku Postgres dashboard that is available in our [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library).

Heroku Postgres Dashboard

If you use the Heroku [Redis add-on](https://devcenter.heroku.com/articles/heroku-redis) at **any** tier, these metrics will be automatically forwarded to your Hosted Graphite account. They are prefixed with: **heroku.\.heroku-redis.\.\** and report dyno statistics for load, memory, reads/writes, connections, and more! We also have a Heroku Redis dashboard that is available in our [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library).

Heroku Redis Dashboard

If you use the Heroku [Kafka add-on](https://devcenter.heroku.com/articles/kafka-on-heroku) at the **Standard** tier or above, these metrics will automatically forward to your Hosted Graphite account. They are prefixed with: **heroku.\.heroku-kafka.\.\.\** and report per-node statistics for load, memory, reads/writes, operations, and more! We also have a Heroku Kafka dashboard that is available in our [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library).

Heroku Kafka Dashboard

If you use the [Heroku Connect](https://devcenter.heroku.com/articles/heroku-connect) add-on at any tier, these metrics will be automatically forwarded to your Hosted Graphite account via Heroku log drains. They are prefixed with: **heroku.\.herokuconnect.\*** and report sync activity between Salesforce and Heroku Postgres, including operation counts, sync durations, and rows processed. These metrics provide visibility into both source (Salesforce queries) and target (database writes) behavior, helping you monitor data flow and identify performance bottlenecks.

Heroku Connect Dashboard Panels

### [**Local Development Configuration**](#local-development-configuration) In your local development environment, it will be necessary to replicate the above configuration. Since the [Foreman tool](https://devcenter.heroku.com/articles/procfile#developing-locally-with-foreman) will look in the `.env` file for development environment variables, use the following command to copy the `HOSTEDGRAPHITE_APIKEY` from the Heroku config (Do not check the file `.env` into source control). > ``` > heroku config -s | grep HOSTEDGRAPHITE_APIKEY >> .env -a > ``` > > ``` > more .env > ``` More information on accessing your API key is detailed in the below section. #### [Accessing the API Key Within Your App](#accessing-the-api-key-within-your-app) Now that the add-on has been set up, the Hosted Graphite [*API Key*](/#api-key) will be available via the environment variable **HOSTEDGRAPHITE\_APIKEY**. See the appropriate language snippet below for an example of how to access this variable in your app. **Ruby** See also: [*Sending Metrics with Ruby*](/language-guide/ruby) ``` apikey = ENV['HOSTEDGRAPHITE_APIKEY'] ``` **Python** See also: [*Sending Metrics with Python*](/language-guide/python-2.x) ``` apikey = os.environ['HOSTEDGRAPHITE_APIKEY'] ``` **Java** See also: [*Sending Metrics with Java*](/language-guide/java) ``` String apikey = System.getenv("HOSTEDGRAPHITE_APIKEY"); ``` **PHP** See also: [*Sending Metrics with PHP*](/language-guide/php) ``` $apikey = getenv('HOSTEDGRAPHITE_APIKEY'); ``` **Node.js** See also: [*Sending Metrics with Node.js*](/language-guide/node.js) ``` var apikey = process.env.HOSTEDGRAPHITE_APIKEY; ``` #### [Custom Metrics](#custom-metrics) You can also send custom application metrics to your Hosted Graphite account. We provide code snippets in our [language guide](https://docs.hostedgraphite.com/language-guide) that can be configured throughout the codebase of your Heroku app to send Graphite metrics through a socket connection. Custom metrics can be tailored to track anything you want in your app like logins, sign ups, or cancellations. If you migrated your Heroku app monitoring from Librato, please note that Hosted Graphite does not support custom metrics via your app's log-drains. See this helpful [video tutorial](https://www.youtube.com/watch?v=Xzr6pFwhDGM) on how to configure a custom StatsD metric to report an event from your Heroku app to your Hosted Graphite account. ### [HG Heroku Configuration](#hg-heroku-configuration) Navigate to Add-Ons ⇒ Heroku to access all of the following configuration options:

HG Heroku Config

#### [Router Path Metrics](#router-path-metrics) If you enable the Path Metrics feature in the Heroku **Config**, we will collect and forward router statistics (connection times, request methods/statuses) **per path** in your Heroku application. This will increase the total number of metrics being stored in the Hosted Graphite backend, but will give you **increased visibility** into activity within your app, and a higher level of alerting functionality. We offer these metrics in a dot.notation.format, or an underscore\_format giving you the option to choose the metric cardinality that works best for you: **heroku.\.\.router.\.\** or **heroku.\.\.router.\.\** If left on the 'default' metric format, you will receive the standard router metrics giving you a good overview of the activity within your Heroku app. The default router metrics are what can be seen on your [Heroku Auto Dashboard](#heroku-auto-dashboard). #### [Path Aggregates](#path-aggregates) If you have Path Metrics enabled, but experience a high cardinality of metrics due to unique identifiers being created in your app's URLs, our Router Aggregates feature will combine these values to reduce your metric overhead. For example, lets say your app generates endpoints with ID placeholders for a customer each time a route is accessed: "GET /api/path/\" This could create a lot of unique heroku router metrics in your account and you might be better of receiving only 1 metric with a combined value. Simply define a path with the variable in curly brackets {placeholder-id} to combine multiple metrics into one! **NOTE**: Path metrics must first be enabled in your HG account, and you can define as many Router Aggregate paths as you like. #### [Process Aggregates](#process-aggregates) Heroku runs your app code in Dynos defined by [One-Off](https://devcenter.heroku.com/articles/process-model) and [Worker](https://devcenter.heroku.com/articles/background-jobs-queueing) process types in your Procfile (you must have [Log-Runtime Metrics](https://devcenter.heroku.com/articles/log-runtime-metrics) enabled on your Heroku app to receive process metrics). This feature rolls up multiple process instance IDs into a single metric stream, which is good for reducing cardinality and giving you a good performance overview. By default, you will receive individual process metrics, but we recommend enabling aggregation per process to significantly decrease your total metric count. **Contact us** to enable Process Aggregation for any specific Heroku process metrics not listed in our Heroku UI (advanced\_scheduler, clock, celeryworker, worker\_pipeline, beat, mail\_syncer, notifications, orchestrator, priority, etc). #### [Host Aggregates](#host-aggregates) Some Heroku apps report router metrics under multiple hostnames, such as a mix of your Heroku default domain (myapp.herokuapp.com), a custom domain ([www.herokuapp.com](http://www.herokuapp.com)), or internal service domains (UID-123.herokuapp.com). These hostname variations will produce multiple metric series even though they represent the same application. If you don't require data for each host/subdomain of your app, enable the Host Aggregation feature to combine all individual hostname data sets into a single 'overview' series of: **heroku.router.\*** ### [Heroku Event Annotations](#heroku-event-annotations) We also provide a convenient webhook that you can upload to your Heroku account and import real-time Heroku events to your Hosted Graphite account:

Heroku Webhook

Just add our HG webhook into your Heroku account, you can configure [annotations](https://docs.hostedgraphite.com/api-guides/annotations-and-events-api#dashboard-annotations) on your dashboards to display events from your Heroku environment around deployments, builds, domains, collaborators, add-ons, and more. The Heroku Auto-Dashboard has the Heroku annotation toggle built in, and you can generate this dashboard from our [Dashboard Library](/dashboard-and-graphs/dashboard-library). ### [Monitoring Heroku Metrics on an Existing HG Account](#monitoring-heroku-metrics-on-an-existing-hg-account) Its easy to forward your log drain metrics to an existing Hosted Graphite account. This is actually a better value because a dedicated HG account allows you to configure multiple heroku apps, get the full 2 year data retention, and premium level support for the same price as the Heroku add-on (depending on how many metrics your app produces).
**First**, enable [Log-Runtime metrics](https://devcenter.heroku.com/articles/log-runtime-metrics) by running the following command in your Heroku CLI: > ``` > heroku labs:enable log-runtime-metrics -a > ``` **Then**, after enabling runtime metrics, you need to restart your Heroku application: > ``` > heroku restart -a > ``` **Finally**, you can just point the Heroku app logs to your existing Hosted Graphite account by running the following command: > ``` > heroku drains:add https://webhooks.hostedgraphite.com/heroku//logs/\?apikey= -a > ``` **NOTE:** make sure include the app-names in the `drains:add` command. You are able to ship metrics from as many Heroku apps as you like, and metrics will appear as events happen within your app (status.\*, method.\*, etc). # Hosted StatsD StatsD is a simple server daemon that collects data from your systems. It happily collects as much data as you want to send it, aggregates it into a few different views, and intermittently flushes it out to your backend service - in this case Hosted Graphite. When you want to create a new metric, you just send it on and StatsD creates it. ### Why use StatsD Let’s take the example that you have a very busy site sending data every time a user does something interesting. If your site activity were to grow linearly, so would the data sent to your measurement system - and soon you might end up in a situation where you’re overwhelming it. One of the useful things StatsD can do is to sample data; by sending only a percentage of the requests you get a lot fewer data points sent while still capturing an accurate view of the system. ### Enabling Hosted StatsD Our Hosted StatsD service is available for all plans, and is used by a variety of customers big and small. We regularly update it with the latest fixes and features from the open-source project. **Note** that the default flush interval for our Hosted StatsD is 10s.

Enable StatsD Ports to Send Metrics to Our Hosted StatsD Endpoint

### Configuring StatsD Metrics Hosted Graphite supports filtering the raw metrics your Hosted StatsD instance produces for some of its metric types such as counters and timers. This allows you to choose only the metric views you actually use, which reduces the metric usage count for your payment plan. To enable/disable the specific metric views you want us to store, click the `Filters` button on the ‘StatsD’ section of the [add-ons page](https://www.hostedgraphite.com/app/addons/).

Hosted StatsD Enabled

Here you will see a list of StatsD metric types and their corresponding metric views. Checking/unchecking these views will enable/disable them respectively.

Hosted StatsD Configureable Views Filters

Hosted Graphite also allows users to configure StatsD settings for `deleteGauges`, `deleteTimers`, `deleteSets`, and `deleteCounters`. StatsD will send a value of 0 for a metric it hasn’t received any data unless these settings have been activated. `deleteGauges` will stop sending values for inactive gauges, `deleteTimers` will stop sending values for inactive timers, and so on. These are enabled by default, and we recommend they remain activated to prevent stale metrics from persisting in your account. For example, if you have [*metric expiry*](/account-management/account-settings#metric-expiry) set, metrics that are being persisted by the normal StatsD behavior will not expire. To enable/disable the specific config values, click the `Config` button on the ‘StatsD’ section of the [add-ons page](https://www.hostedgraphite.com/app/addons/). Here you will see a list of StatsD config options. Checking/unchecking these options will enable/disable them respectively. Click Save to `save` your changes. ### Sending A StatsD Metric Using the [netcat](http://netcat.sourceforge.net/) utility: ```bash echo "YOUR-API-KEY.test.testing:1.2|c" | nc -u -w1 statsd.hostedgraphite.com 8125 ``` Note that the metric name will prefixed with the type of metric (timers, gauges, counters) when it is processed by our system. For example: `counters.foo.bar` or `gauges.bar.foo`. * **timers** “\metric.name:\|ms” * **gauges** “\metric.name:\|g” * **counters** “\metric.name:\|c” ### **Sending a Tagged StatsD Metric** You can add tags to your StatsD metric using the following format: ``` echo "YOUR-API-KEY.test.statsd-tags;tag1=value1;tag2=value2:1.2|c" | nc -u -w1 statsd.hostedgraphite.com 8125 ``` You can then locate these metrics in the **Tag Search** section of our UI. **Pro Tip -** to visualize tagged metrics on a dashboard, you must wrap the metric with the [seriesByTag() ](https://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.seriesByTag)Graphite function:

Visualize Tagged StatsD Metrics

### Data Views Comparisson Learn about our built-in [Data Views](/additional-features/data-views) provided for all metrics sent to Hosted Graphite. While Data Views aren’t direct equivalents to StatsD types, they provide similar ways to work with your data: * **counters** -> `:sum` (total count), `:sumrate` (per-second rate) * **gauges** -> `:avg`, `:min`, `:max` (aggregations of reported values) * **timers** -> `:avg`, `:min`, `:max`, `:95pct`, etc. (distribution-style views) # New Relic Hosted Graphite provides an add-on for New Relic. This service will sync your metrics from New Relic into your Hosted Graphite account. These metrics are aggregated upon ingestion to Hosted Graphite and the naming schemes could differ from the naming schemes in your New Relic account. We currently support the synchronization of metrics from two New Relic services. > * [APM](http://newrelic.com/application-monitoring) > * [Browser](https://newrelic.com/platform/browser-monitoring) To get this service running, browse to our [add-ons](https://www.hostedgraphite.com/app/addons/) page. Here you will find the “New Relic Metric Import” card. Enter your New Relic access key into the textbox provided, then click the “Save” button.

New Relic config

You can also choose which “instances/apps” from New Relic you would like to import, this can be enabled via the “custom” button found on the same card (only after you have saved an API Key). You have two options for importing metrics from New Relic. You can choose to either import “summary” metrics or “all” metrics. Summary metrics are used to build the default dashboards you see for APM/Browser within your New Relic account and also the ones which we use to populate our automatic dashboards. Once enabled, wait a few minutes and your New Relic summary metrics should start appearing in your account under the following metric paths: - “newrelic.apm.\” - “newrelic.browser.\” We will also create some automatic New Relic dashboards in your account, these are named: - New Relic APM - New Relic Browser Overview ### Importing all metrics: When you enable “all” metric imports for New Relic, a greater range of metrics will be pulled for each app/service. These metrics will appear under the path: “newrelic.{apm|browser}.\.all.\” Importing *all* metrics will result in a large increase in your metric count, so we recommend ensuring your account plan is large enough to deal with 1000s of new metrics per New Relic instance/app. If not, please consider upgrading to a larger plan. If you do have “all” importing enabled, it is important to note that for apm services on New Relic, a “browser” instance is created per app, these metrics automatically get pulled in under the newrelic.apm.verbose.\.browser path and therefore will not be duplicated under newrelic.browser.verbose.\ # Papertrail Papertrail, now owned by Solarwinds, is a log management service which allows you to manage logs from apps, servers, and cloud services with the use of alerts and events. All you need to get started is an account in Papertrail and an account on Hosted Graphite. Check out the [Papertrail documentation](https://help.papertrailapp.com/kb/integrations/hostedgraphite/) for more details on how to create alerts in Papertrail that will create and forward Graphite metrics to your HG account. When you create an alert in Papertrail, it will use a keyword to track the specified events and occurrences in your logs. A Graphite metric will be created and forwarded to your HG account and you can use these metrics to create custom dashboards and alerts.

Configure Papertrail to send metrics to Hosted Graphite

# Pingdom If your site goes down, is it reflected on your graphs? With a pingdom webhook add-on, you can now flag your graphs when pingdom fires an alert. ### How to set up a Pingdom webhook The webhook provided by Hosted Graphite can be used in Pingdom both as an alerting endpoint and with integrations. To get Pingdom alert information added to Hosted Graphite as annotations, follow these instructions: * Copy your Pingdom webhook location by clicking the ‘Copy Webhook Location’ button

Pingdom Webhook

#### Adding an Alerting Endpoint * Browse to your Pingdom account, visit the ‘alerting’ section, and click ‘Alerting Endpoints’ * On the top right-hand side, click “Add Alerting Endpoint” * Click ‘Add Contact Method’, select ‘URL / Webhook’, and paste your Hosted Graphite Pingdom webhook location, click ‘Add’ * Save the endpoint You’ve now saved a Hosted Graphite webhook for your alerting endpoint. Assign this alerting endpoint as one of your contact methods for an outage, and then add the Pingdom annotations to your alerts. #### Adding an Integration * Browse your Pingdom account, visit the ‘Integrations’ section, and click ‘Integrations’ * On the top right-hand side, click “Add integration” * Select Type: ‘URL / Webhook’, give your integration a name (e.g. ‘Hosted Graphite’), and paste your Hosted Graphite Pingdom webhook location. Make sure the Active box is ticked, and click ‘Save integration’ * Select one of your checks or create a new one, and scroll to the bottom of the Edit window. Your new integration should appear in the ‘Connect integrations’ list. Select the integration, and click ‘Create Check’/’Modify Check’ Your PIngdom check will now send detailed event information to Hosted Graphite, including all your specified check tags and the following check-type specific information, as searchable annotation tags: * HTTP: full URL (including HTTP(s) prefix) * DNS: expected IP, name server * UDP: sent string, expect response #### To Enable Pingdom Annotations in Your Dashboards * Browse to your Hosted Graphite graph, and enable Annotations * Enable Pingdom Annotations

Pingdom Annotation

Use the tag “Pingdom” to see all Pingdom alerts or the hostname of the Pingdom check to see tagged annotations for that host. If you are using Pingdom Integrations, you can also search by your check tags or any of the specific fields listed above. More information on annotations can be found by following our [annotations](/api-guides/annotations-and-events-api) documentation. # Sentry When your app encounters an error, does it affect your graphs? With a Sentry webhook add-on, you can pinpoint your Sentry events in time to see if they coincide with changes in your metrics. ### How to set up a Sentry webhook The webhook provided by Hosted Graphite can be used in Sentry as an alerting endpoint. To add Sentry alert information to Hosted Graphite as annotations, follow these instructions: * Navigate to the Add-ons section of your Hosted Graphite account and copy your Sentry webhook location by clicking the ‘Copy’ button and follow the mouseover directions

Sentry webhook

#### Adding an Alerting Endpoint (in your Sentry account) * Browse to your Sentry account, visit ‘Project Settings’ section, and click ‘All Integrations’ * Scroll down to ‘WebHooks’ and make sure the box is checked and save changes if necessary. * Browse to the Alerts settings * Scroll down, and add the Hosted Graphite Sentry webhook location to the field and save your changes. * Scroll back up and either add or edit an alert Rule that corresponds to the Events you want to send to Hosted Graphite. Only alerts you send will appear on your graphs, so choose triggers and intervals that make sense for your use case. * Ensure the notification is being sent through WebHooks. (You can send it to other services too, if you’d like.) You’ve now saved a Hosted Graphite webhook for your alerting endpoint and assigned it to an alert. Now, once you’ve triggered some alerts, you can plot them on your graphs. #### To View Sentry Annotations in Your Dashboards * Browse to your Hosted Graphite graph and open the Annotations dialog * Create and name a new annotation, and query on any tag to see your Sentry alerts

Create annotation

**NOTE**: Sentry tags are sent as key:value pairs, but Hosted Graphite annotations are a series of strings. If your events are tagged with (room: "356B"), you’ll be able to query it using room=356B. By default we store the project slug, server name, error level, and the word “sentry” so you can quickly filter annotations for your dashboards. # Sitespeed Integrate Sitespeed.io with Hosted Graphite to gain insights into how your website or web application performs in different environments and under different conditions. [Sitespeed.io](https://www.sitespeed.io/) is an open-source tool that runs on Docker and can be used for analyzing website performance helping you identify bottlenecks, and optimize your web pages. Sitespeed has a Graphite output and can be easily configured to send metrics to your Hosted Graphite account. #### Send Sitespeed Metrics to Hosted Graphite You can run the following Docker command against any website URL that you wish to test: {% code overflow="wrap" %} ``` docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:39.5.0 -n 1 --graphite.host carbon.hostedgraphite.com --graphite.port 2003 --graphite.namespace .sitespeed_io.default.hg ``` {% endcode %} Running this command will send site performance metrics to your Hosted Graphite account and you can easily visualize them by importing various auto-dashboards from the Sitespeed GitHub [repository](https://github.com/sitespeedio/grafana-bootstrap-docker/tree/83dc8cbf572126e3f40b6b7a5053dae1a5701d20/dashboards/graphite). We also have a Sitespeed Performance dashboard that has been optimized for Hosted Graphite, located in our [Dashboard Library](/dashboard-and-graphs/dashboard-library). Event data will also be stored locally after each run, so it is recommended to run the command from within a designated directory.

Sitespeed Performance Dashboard

#### Sending Annotations To send Sitespeed annotations to Hosted Graphite you must add some required [configuration options](https://www.sitespeed.io/documentation/sitespeed.io/configuration/#the-options) to your docker run command, such as: ``` --graphite.webHost www.hostedgraphite.com --graphite.proxyPath /api/v1/graphite/events/ --graphite.httpPort 443 --graphite.auth : --graphite.resultBaseURL https://result.sitespeed.io --graphite.sendAnnotation true ``` Additionally, you can create a file at the root of your sitespeed directory named **config.json** to store all of the required `--graphite` options and run a cleaner command. Here is a template for a **config.json** file with all of the required Graphite options to include annotations, and a parameter that better aligns the annotations with our default aggregation resolution of 30s: ```json { "graphite": { "host": "carbon.hostedgraphite.com", "port": 2003, "namespace": ".sitespeed_io.default.hg", "webHost": "www.hostedgraphite.com", "proxyPath": "/api/v1/graphite/events/", "httpPort": 443, "auth": ":", "sendAnnotation": true, "annotationRetentionMinutes": ".5" }, "resultBaseURL": "https://result.sitespeed.io" } ``` Save the file then run the Docker command using the **--config** option: `docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:39.5.0 --config config.json` You can also run Sitespeed tests against multiple URL's by listing them in a separate file and including it in the docker run command: Create a new file: `urls.txt` ``` https://www..com/ https://www..com/ https://www..com/ ``` Save the file and run the Docker command with the new file name in place of the URL: `docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:39.5.0 urls.txt --config config.json` #### Visualizing Annotations Default annotations are sent by Sitespeed with tags that will match the following dashboard variables, as defined in the Sitespeed Performance dashboard: **$base $path $group $page $browser $connectivity $testname** These tags can be located in the **Go to result** annotation rule in the Page Metrics Auto Dash:

Sitespeed Annotations

These default tags allow you to toggle between domains and visualize event annotations that correspond to each URL that Sitespeed was run against. Generally you only need to query the **$base $group $page $browser $connectivity** variables to see annotations on the Sitespeed Performance dashboard (annotations might not appear as expected with every default variable in place). #### Metric Reduction Options Sitespeed.io generates a large number of metrics by default, which can increase metric usage in Hosted Graphite. If you need to reduce the number of metrics collected, you can disable sets like summary statistics using `--browsertime.statistics false`, which reduces metrics the forwarded, or use the `--metrics.filter` option to include only specific groups (e.g: `--metrics.filter "*-" --metrics.filter "browsertime.pageSummary.statistics.googleWebVitals.*"` to keep only Core Web Vitals). Keep in mind that filtering metrics may cause some panels in the default Grafana dashboards to stop displaying data, so it’s best used when you only need a subset of performance metrics. # StatsD Add-on StatsD is a network daemon that runs on the Node.js platform. It listens for statistics such as counters and timers sent via UDP, performs aggregation of this data, then sends it on to a configurable backend service - such as Hosted Graphite. You can use StatsD to perform such aggregation before sending your metrics to us by following this short guide. * Install StatsD by following the [installation instructions](https://github.com/etsy/statsd#installation-and-configuration). * Once installed, StatsD can then be used with a config file like this: ``` { port: 8125, flushInterval: 10000, graphitePort: 2003, graphiteHost: "carbon.hostedgraphite.com", graphite: { legacyNamespace: false, globalPrefix: "your-api-key-here" } } ``` ### Hosted StatsD Enable our Hosted StatsD if you don’t have the luxury of running a server yourself. Check out the [*Hosted StatsD*](/add-ons-and-integrations-guide/hosted-statsd) documentation. # Statuspage Statuspage provides a centralized hub where businesses can share incident updates, scheduled maintenance, and overall system status with their users and customers. We provide a webhook that you can configure in your Statuspage account. This will forward events to your Hosted Graphite account in the form of annotations, which you can then configure in your dashboards. Simply navigate to Add-Ons => Statuspage in your Hosted Graphite account to copy the webhook:

Statuspage webhook

Then just create an annotation on your dashboard(s) and query by Tag:

Statuspage Annotation

# Account Management Your Hosted Graphite account includes many useful options and settings to help you access and manage your metrics effectively. * [Access Keys](/account-management/access-keys) * [Account Diagnostics](/account-management/account-diagnostics) * [Account Settings](/account-management/account-settings) * [Team Access: Limited Access Groups](/account-management/team-access) * [SAML Authentication](/account-management/saml-authentication) * [Team Access](/account-management/team-access) # Access Keys Access Keys allow you to share read-only access to your graphite data with external applications, scripts, or dashboard plugins. Contents**:** * [What can I use an Access Key for?](#what-can-i-use-an-access-key-for) * [Access Key List](#access-key-list) * [Using an Access Key](#using-an-access-key) * [Create an Access Key](#create-an-access-key) * [Editing an Access Key](#editing-an-access-key) * [Deleting an Access Key](#deleting-an-access-key) * [Troubleshooting](#troubleshooting) ### [What can I use an Access Key for?](#what-can-i-use-an-access-key-for) An [Access Key](https://www.hostedgraphite.com/app/sharing/) can help you use different Graphite plugins using Hosted Graphite Data, such as: * **Tasseo**\ A real-time Graphite dashboard. Also available within Hosted Graphite - follow the Tasseo link at the top, or grab it from the [Tasseo GitHub page](https://github.com/obfuscurity/tasseo) * **Graphene**\ For an alternative real-time Graphite Dashboard, see the [Graphene GitHub page](https://github.com/jondot/graphene) * **Seyren**\ An alerting dashboard for Graphite. See the [Seyren GitHub page](https://github.com/scobal/seyren) * **The Graphite Render API**\ The [*Graphite Render API*](/api-guides/graphite-render-api) allows you to create images of your graph data. It’s a really comprehensive way to render the data collected by Hosted Graphite. * **Local Dashboard Instance**\ Add Hosted Graphite as a [data source](http://docs.grafana.org/features/datasources/graphite/) to your local/self-hosted dashboard installation to render your metrics. You can find a detailed guide [*here*](/dashboard-and-graphs/local-dashboard-integration). ### [Access Key List](#access-key-list)

Your list of access keys, found in Access => Sharing

### [Using an Access Key](#using-an-access-key) The created access key will be listed in the table as shown in the image above. Simply copy and append the key to your Hosted Graphite URL (e.g: ) to use it in your external application, as per its documentation. ### [Create an Access Key](#create-an-access-key) Click the “Add Access Key” button to do the obvious.

Graphite access key

* **Access Key Name**\ Creating an Access Key requires that you name the key. This has no connection to the key itself, it’s just a handy reminder of why you added it. E.g. to connect to an external dashboard like Tasseo, or to an alerting system such as Seyren. * **Optional - Add An IP Whitelist**\ It’s not necessary for you to add a whitelist, but this will add an **extra level of protection** if your key is leaked, and you want to ensure that data is only coming from servers you control. The whitelist accepts comma-separated IP addresses in the standard format (XXX.XXX.XXX.XXX). Only team managers and account owners will see the Access Key list or have the option to create keys. * **Type - Dashboard or Graphite**\ Unless you’re specifically using an Access Key to share a dashboard, the option you want to pick is **Graphite**. This gives external applications access to Graphite’s data. ### [Editing an Access Key](#editing-an-access-key) Clicking the wrench / Edit icon to bring up the Edit Access Key dialog. There is also an option here to Whitelist an IP if needed: ### [Deleting an Access Key](#deleting-an-access-key) Hit the trash can icon to bring up the Delete Access Key dialog and simply hit the “Yes, Delete” button to purge the Access Key from the system. ### [Troubleshooting](#troubleshooting) * **When I visit the Access Key URL I get a ‘403 Forbidden’ error**\ The access key enables access to the read-only parts of Hosted Graphite that allow you to retrieve data, and anything else returns a 403. If you add ‘/render/’ to the end of the created URL, you’ll see a blank graph image showing a red “No Data”. * **I still get a 403 error at the ‘/render/’ URL**\ Remove any whitelisted IP addresses you have added to the access key, and try again. If you still have trouble accessing the URL, [contact us](mailto:support%40hostedgraphite.com). # Account Diagnostics Visualize traffic statistics within your account by navigating to the “Account Diagnostics” submenu from the “Metrics” tab. Contents * [Live Metrics Graph](#live-metrics-graph) * [Datapoint Rates Graph](#datapoint-rates-graph) * [Metrics Created and Deleted Graph](#metrics-created-and-deleted-graph) * [Activity by Protocol](#activity-by-protocol) * [Busy Metrics Being Ratelimited Graph](#busy-metrics-being-ratelimited-graph) * [Invalid Metrics Graph](#invalid-metrics-graph) * [Why are there Account Limits?](#why-are-there-account-limits) ### [Live Metrics Graph](#live-metrics-graph)

Live Metrics

In the graph above, the **orange line** is the *live metric limit* - the number of metrics you can update in a rolling 5-minute period. One metric name might look like: `my.server.cpu.load`. In this example, the limit is set at 1,500,000 meaning that up to 1.5M metric names can be sent concurrently. The **green line** is *the number of live metrics incoming for the account*, on this graph it fluctuates between around 275K and 350K live metrics. When we see more than the limit sent at the same time, some metrics will be dropped. **Note:** Live metrics can also be referred to as 'concurrent' or 'active' metrics. ### [Datapoint Rates Graph](#datapoint-rates-graph)

Datapoint Rates

In the graph above, the **dark** **blue line** is *the number of data points allowed per second* or *the data point rate limit*. In this example it is set at 750,000, allowing the user to send 750K data points per second. The other lines in the graph represent the number of data points per second hitting your account, by protocol. ### [Metrics Created and Deleted Graph](#metrics-created-and-deleted-graph)

Metrics Created and Deleted

The above graph will give you visual insights into recent metrics being created, deleted, and expired. This can be useful for tracking traffic spikes, and monitoring any configured [expiry rules](https://docs.hostedgraphite.com/account-management/account-settings#metric-expiry). ### [Activity by Protocol](#activity-by-protocol)

Activity by Protocol

This card provides a quick overview of your current traffic and the icon in the status column provides information on your last received data. The green icon indicates that we have seen data arrive recently on that interface. A yellow or red icon indicates that no data has arrived for that protocol for at least 5 and 15 minutes respectively. A blue icon indicates that we have never seen any traffic on that interface. ### [**Busy Metrics Being Ratelimited Graph**](#busy-metrics-being-ratelimited-graph)

Busy Metrics

If a user is sending a high volume of datapoints per second to a single metric, we implement per-metric rate limiting rules to protect our backend. These rules are defined differently than the Live Metric ratelimiting rules, and only target individual metrics with a very high rate of Data Points Per Second. You can read more about why these rules are important, and how they work in this informative [blog article](https://www.hostedgraphite.com/blog/per-metric-rate-limiting-how-we-protect-our-backend). ### [**Invalid Metrics Graph**](#invalid-metrics-graph)

Invalid Metrics

If a user sends metrics that do not match the Graphite format, they will be reported as 'invalid' and cannot be ingested. In the above panel you can see the offending metrics, reason for reporting as invalid, protocol, IP sent from, and timestamp of attempted ingestion. To avoid heavy impact on our ingestion servers, the list is refreshed every 5min, there is a limit of 100 metrics, and the invalid metric names are only stored for 24hrs. We also include a related panel in your HG Traffic Dashboard, as well as an alert for Datapoints Dropped in every Hosted Graphite account. **NOTE:** we currently do not track invalid metrics for StatsD. ### [Why are there Account Limits?](#why-are-there-account-limits) TL;DR - As a prevention measure against accidents and malice. It’s possible for a user to run a script that accidentally (or deliberately) updates millions of metrics a second. Sensible limits on what data we process ensure that one customer cannot affect the quality of service for others. Generally, we want customers to be able to send data at a high rate and we can monitor and increase any limits as necessary. Check out this [article](https://www.hostedgraphite.com/blog/per-metric-rate-limiting-how-we-protect-our-backend) for more details on why these limits are put in place. # Account Settings Contents * [Metric Expiry](#metric-expiry) * [Tag Expiry](#tag-expiry) * [Default Timezone](#default-timezone) * [Billing and Invoices](#billing-and-invoices) * [Changing Your Account Email](#changing-your-account-email) * [Receive Billing Emails to a Different Address](#receive-billing-emails-to-a-different-address) ### [Metric Expiry](#metric-expiry) If you use a lot of transient metrics generated by spin-up VMs or on-demand instances, you may want to expire these after a period of inactivity to help manage your **total** number of stored metrics.

Metric Expiry Example

For example, if you set the Expiry Period to 30, any matching metric that hasn’t received data in 30 days will be deleted. The wildcard (`*`) character is greedy, which means it will match **everything** that fits into the given pattern (as opposed to Graphite where it matches only one node level). For prefixes that are subsets of each other, the more specific name will take priority. So for `foo = 1` and `foo.bar = 5`, `foo` and `foo.baz` and `foo.baz.steve` will be set to 1. `foo.bar`, `foo.bar.baz` and so on will be set to 5. **Consideration**: Our Expiry feature is not intended for blocking metric patterns from ingestion, rather cleaning up stale metrics that haven't received data for X days. **NOTE**: You can configure a global expiry rule to target all (`*`) metrics, but if you target a `*specific.metric.pattern*`, the global expiry will be overridden. Additionally, any expired namespaces will be recreated if they are re-sent, and will continue to map to all of their historic data. ### [Tag Expiry](#tag-expiry) Hosted Graphite supports Expiry Rules for tagged metrics. Similar to how metric expiry rules work with a few exceptions listed below:

Tag Expiry Example

* Tags have to be the **exact** match to the tag listed on the series. * EX: `cpu.usage;`**`region`**`=south;`**`server`**`=db-0001` * In order to add an expiry rule to the series above, the tag field will have to be one of the following: **name**, **region**, or **server**. * In order to set the rule for the series name: **cpu.usage** set the tag to **name**. * This will expire all the series with this name regardless of the tags it is identified by. * At the moment only one tag per field is accepted. * The **value** field does accept **wildcard** characters. * For example if you want to expire all series with the prefix cpu then the tag would be **name** and the value would be **cpu.\*** and all these series would expire after the defined period of inactivity: `cpu.usage.total;`**`region`**`=south;`**`server`**`=db-0001` `cpu.usage.sys;`**`region`**`=east;`**`server`**`=db-0002` `cpu.usage.user;`**`region`**`=west;`**`server`**`=db-0001` * The expiration will be processed by the of time listed. * Ex: Suppose you have 2 series: `cpu.usage;`**`region`**`=south;`**`server`**`=db-0001` `diskspace.available;`**`region`**`=south;`**`server`**`=db-0001` * With the following expiry rules: * **tag**: name; **value**=cpu.usage -> **2 days** * **tag**: server; **value**=db-0001 -> **1 day** * This would expire every series with a server of *db-0001* including the ones listed for the first expiry rule. The following day, if the series `diskspace.available` is recreated, then it would expire once more. ### [Default Timezone](#default-timezone) Setting a default timezone within Hosted Graphite ensures that graphs rendered by Graphite are configured to use that timezone.

Select Timezone

Simply pick the timezone of your choice from the dropdown, and the setting will be automatically saved. ### [Billing and Invoices](#billing-and-invoices) You can easily manage your billing preferences and settings with our provider, Stripe.

Billing and Invoices

### [Changing Your Account Email](#changing-your-account-email) You can change the email address associated with your Hosted Graphite account by navigating to the [account settings](/account-management/account-settings) page, clicking on the **Advanced Options** button at the bottom of the page, and selecting **Enter the new email**.

Change Your Account Email

After you fill out the form a confirmation link will be sent to your new email address to verify the change. If an account already exists with the new email, you’ll need to either rename that one first or contact us to delete it. ### [Receive Billing Emails to a Different Address](#receive-billing-emails-to-a-different-address) If you would like to receive a copy of billing invoices to an email address that is different from the one primarily used on the account, please contact [support](mailto:support%40hostedgraphite.com).
# Team Access: Limited Access Groups Limited Access groups are a way to define groups of dashboards, and limit specific team members access to these groups. Contents * [User Management](#user-management) * [Group Management Page](#group-management-page) * [Team Managers / Account Owners](#team-managers-account-owners) * [Read-Only Users](#read-only-users) * [Read/Write Users](#read-write-users) * [Notes](#notes) Limited Access groups are a way to define groups of dashboards, and limit specific team members' access to these groups. By default, when not using this feature, team members can access all the dashboards in the account. Limited Access groups are created and edited on the Limited Access Group Management page and can then be associated with team members on the User Management page. Once a user has been associated with one or more groups, they will only be able to see the dashboards in that group. **NOTE:** **This feature is not enabled by default**, please [contact us](mailto:support%40hostedgraphite.com) and we will enable this for you! ### [User Management](#user-management) Navigating to Access => Teams will show a list of team members who are assigned some Limited Access groups. To assign a Limited Access Group to a team member, click to reveal the drop-down and select from the list.

Limited Access Groups

Clicking the ‘x’ beside a Limited Access group's name will un-assign the group from that team member. Once finished making any changes click the “Save Changes” button. ### [Group Management Page](#group-management-page) This can be found by clicking the Manage Limited Access Groups button and provides an overview for adding, removing, or editing Limited Access groups:

Group Management

Here you can see the Limited Access groups you have created and some information about them. To edit an existing group: click the pen icon on the right-hand side. To delete a group: click the bin icon on the right-hand side. Clicking the `+ New Limited Access Group` button will reveal a pop-up that looks like this:

Add Limited Access Group

Choose a Limited Access Group name and pick the dashboards you want to be accessible for that group (this can be edited later). Click save to keep this group. After this return to the User Management page to associate the created group with team members. ### [Team Managers / Account Owners](#team-managers-account-owners) These both have access to the team management page, they can create new Limited Access groups, and add and remove dashboards to/from a group. Team managers cannot be assigned a Limited Access Group. Team managers always have access to all dashboards. ### [Read-Only Users](#read-only-users) Read-only team members can be assigned to any number of groups. Once assigned to one or more Limited Access groups, a read-only team member will only see dashboards that belong to the groups to which they have been assigned. They cannot save or create new dashboards. ### [Read/Write Users](#read-write-users) Read/write team members can be assigned any number of groups. Once assigned one or more Limited Access groups, a read/write team member will only be able to see dashboards that belong to the groups they have been assigned. They can save and create new dashboards. When a read/write team member creates a new dashboard, all team members assigned to their Limited Access groups will also gain access. Similarly, when they save a dashboard, all team members with permission to view will see those changes. ### [Notes](#notes) A team member who is not assigned to any Limited Access groups will have access to all dashboards. A team member’s role (read-only, read/write) applies within all the Limited Access groups they are assigned. Even with an access key, a team member can not view a dashboard if they are not assigned to a Limited Access Group with access to the dashboard. If you need to share a dashboard but cannot assign a group to the team member, we suggest using the dashboard snapshot-sharing feature. # SAML Authentication Hosted Graphite supports Single Sign On (SSO) via SAML-enabled identity providers. This allows users to login to our service using their existing organization credentials. Contents * [SAML Setup](#saml-setup) * [Azure Active Directory](#azure-active-directory) * [Okta](#okta) * [OneLogin](#onelogin) * [Ping Identity](#ping-identity) * [Salesforce](#salesforce) * [Auth0](#auth0) A SAML integration is set up on the team’s primary Hosted Graphite account, and any subsequent user signups via SAML will be added to this account as team members. **NOTE:** SAML integration is included for all plans but must be enabled upon request. Please reach out to our [support](mailto:support%40hostedgraphite.com) channel and we will enable this for you. Once enabled, you can locate the SAML Setup page from the Access menu. ### [SAML Setup](#saml-setup) Identity providers require SAML account metadata from Hosted Graphite in order to set up a SAML integration. * Entity ID URL: `https://CLUSTER.hostedgraphite.com/metadata/YOUR-USER-ID/` * Assertion Consumer URL: `https://CLUSTER.hostedgraphite.com/complete/saml/YOUR-USER-ID/` * Connection Type: IdP Initiated via *IdP portal* or SP Initiated via `/login/saml/YOUR-USER-ID/` * XML Metadata: Available by accessing the *Entity ID URL* above while logged in. * NameID Format: Email address. * SAML Version: 2.0

SAML Card

Hosted Graphite can integrate with providers that support the SAML 2.0 specification. Example steps for connecting with some supported identity providers are listed below. ### [Azure Active Directory](#azure-active-directory) **Create a Hosted Graphite integration with Azure AD** 1. Log in to the Azure AD portal, select your directory, then go to **Applications** and **Add**. 2. Click **Add an application from the gallery** then search for and select *Hosted Graphite*. 3. Open the Hosted Graphite application integration page, click **Configure single sign-on,** and then select **Azure AD Single Sign-On**. 4. Enter Identifier `https://CLUSTER.hostedgraphite.com/metadata/YOUR-USER-ID/` 5. Enter the Reply URL h`ttps://CLUSTER.hostedgraphite.com/complete/saml/YOUR-USER-ID/` 6. The final configuration screen shows the values required for the next step. Download your certificate from this page. **Adding Azure AD provider details to Hosted Graphite** 1. Navigate to the SAML Setup page to enter details from the configuration screen of the Azure AD App. * In the Entity ID field, enter your *Issuer URL*. * In the SSO Login URL field, enter your *SAML SSO URL*. * In the Certificate text box, enter the contents of the certificate file you downloaded. 2. Select a default user role for new team members. 3. Click **Save**. This information is also available in the [Azure documentation](https://azure.microsoft.com/en-us/documentation/articles/active-directory-saas-hostedgraphite-tutorial/). ### [Okta](#okta) **Create a Hosted Graphite integration with Okta** 1. Login to the Okta portal, navigate to **Admin**, then **Applications**, and click **Create App Integration**. 2. Select the **SAML 2.0** option, name the app (e.g. Hosted Graphite), and upload the HG icon. 3. In **SAML Settings**, enter the **SSO URL** (found in the HG app SAML Setup as: Assertion Consumer Service URL) and **Audience URI** (found in the HG app SAML Setup as: Entity or Issuer ID). 4. Then set the **Name ID format** as: EmailAddress, **Application username** as: Email. 5. In the **Attribute Settings** section, set **Name** as: email, **Name Format** as: Basic, and **Value** as: user.email, and click **Next**. 6. Click on **View SAML Setup Instructions** to display the information required in the following steps. 7. Now you can navigate your Okta directory and assign people or groups to the HG application. Confirm any additional information for each user and click Done when finished. **Adding Okta provider details to Hosted Graphite** 1. Navigate to the **SAML Setup** page within your main Hosted Graphite application. 2. Enter the details from Step 6 into the relevant fields, set the default user permissions, and click Save. 3. Your team users should now be able to access the Hosted Graphite application through their Okta instance. This information is also available in the [Okta documentation](http://saml-doc.okta.com/SAML_Docs/Configure-SAML-2.0-for-Hosted-Graphite.html). ### [OneLogin](#onelogin) **Create a Hosted Graphite integration with OneLogin** 1. Login to the OneLogin portal, go to **Apps** and then **Add Apps**. 2. Search for *Hosted Graphite* and select the SAML enabled app. 3. Click **Save** to add the app to your Company Apps and display additional configuration tabs. 4. In the **Configuration** tab, enter your HG User ID. This can be found on the SAML Setup page. 5. Click **Save**. 6. Go to the **SSO** tab to view the values you’ll copy into your Hosted Graphite account. **Adding OneLogin provider details to Hosted Graphite** 1. Go to the SAML Setup page to enter the details from the **SSO** section of your OneLogin App. * In the Entity ID field, enter your *SAML Issuer URL*. * In the SSO Login URL field, enter your *SAML Endpoint HTTP URL*. * In the Certificate text box, enter your *X.509 Certificate*. 2. Select a default user role for new team members. 3. Click **Save**. ### [Ping Identity](#ping-identity) **Create a Hosted Graphite integration with Ping Identity** 1. Login to the PingOne portal, go to **Applications**, click **Add Application** then **Search Application Catalog**. 2. Search for *Hosted Graphite* and select the SAML app and click **Setup**. 3. Download the Certificate to enter into Hosted Graphite later, and click **Continue to Next Step**. 4. Enter ACS URL `https://CLUSTER.hostedgraphite.com/complete/saml/YOUR-USER-ID/` 5. Enter Entity ID `https://CLUSTER.hostedgraphite.com/metadata/YOUR-USER-ID/` 6. Select **Continue to Next Step** twice and then **Save and Publish**. **Adding Ping Identity provider details to Hosted Graphite** 1. In PingOne, go to **Applications** and select the app you just created. 2. In the Configuration section, **Issuer** is your Entity ID. 3. Your IDP ID used below is the last parameter of the **Initiate Single Sign-on URL**. 4. In `https://sso.connect.pingidentity.com/sso/idp/SSO.saml2?idpid=${idpid} replace ${idpid}` with your IDP ID. This is your SSO Login URL. 5. Open the certificate file downloaded earlier with a text editor. This is your Certificate. 6. Go to the SAML Setup page and enter your Entity ID, SSO Login URL, and Certificate. 7. Select a default user role for new team members. 8. Click **Save**. ### [Salesforce](#salesforce) **Create a Hosted Graphite integration with Salesforce** 1. Login to Salesforce, go to Settings, and search for “Identity Provider”. 2. Set Identity Provider to enabled, and go to **Connected Apps**. 3. Create a new connected app, and enter a Name and Email. 4. Enter Entity ID `https://wwCLUSTERw.hostedgraphite.com/metadata/YOUR-USER-ID/` 5. Enter ACS URL `https://CLUSTER.hostedgraphite.com/complete/saml/YOUR-USER-ID/` 6. Set NameID format to `emailAddress`. 7. Click **Save**. **Adding Salesforce provider details to Hosted Graphite** 1. In Salesforce, go to **Manage Connected Apps** from Settings and open the app you just created. 2. Under SAML Service Provider Settings, *Issuer* is your Entity ID. 3. Under SAML Login Information, *SP-Initiated Redirect Endpoint* is your SSO Login URL. 4. Under SAML Service Provider Settings, click the name of your certificate and then *Download Certificate*. 5. Go to our SAML Setup page and enter your Entity ID, SSO Login URL, and Certificate. 6. Select a default user role for new team members. 7. Click **Save**. ### [Auth0](#auth0) **Create a Hosted Graphite integration with Auth0** 1. Login to Auth0, select Add New Application and Single Page Web App. 2. Go to Addons and select SAML2 Web App. 3. Enter the Application Callback URL `https://CLUSTER.hostedgraphite.com/complete/saml/YOUR-USER-ID/` 4. Replace the Settings JSON field with the content below: ``` { "audience": "https://CLUSTER.hostedgraphite.com/metadata/YOUR-USER-ID/", "nameIdentifierFormat": "urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress", "nameIdentifierProbes": [ "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress" ] } ``` 5. Click **Save**. **Adding Auth0 provider details to Hosted Graphite** 1. Go to the Addons section of the new Auth0 App and select **SAML2 Web App**. 2. Under the Usage tab, *Issuer* is your Entity ID. 3. *Identity Provider Login URL* is your SSO Login URL. 4. *Identity Provider Certificate* is your Certificate. 5. Go to the SAML Setup page and enter your Entity ID, SSO Login URL, URL and Certificate. 6. Select a default user role for new team members. 7. Click **Save**. **Notes** 1. The team’s primary Hosted Graphite account will continue to log in without SAML. This cannot currently be changed. 2. Existing users cannot be signed in via SAML. Please get [in touch with support](mailto:support%40hostedgraphite.com) if you would like that changed. 3. New users **must** signup via the single-sign-on URL provided by the 3rd party provider, and not via the Hosted Graphite user interface. This is especially important for Azure SAML login. # Team Access We offer the ability to share your account with other team members. You have the option of giving them admin access, read/write access, or restricting them to read-only access. Contents * [Inviting a Team User](#inviting-a-team-user) * [Listing Invited Users](#listing-invited-users) * [Security](#security) We offer the ability to share your account with other team members. You have the option of giving them admin access, read/write access, or restricting them to read-only access. Manage team users from within the Access section of the Hosted Graphite menu. ### [Inviting a Team User](#inviting-a-team-user)

Team Access

Click the “Add a User” button to display the following card:

Add Team User

Enter the user’s email address and choose whether they are a “Team Member” with read-only access, “Read and Write” user with the ability to control everything but the team itself, or a “Team Owner” who has all the “Read and Write” permissions plus the ability to invite and remove other invited team members. The user will receive an invitation to join your account. **The invited user must use the email invitation to create an account that is properly attached to your main account.** Once the user accepts your invitation and logs in to Hosted Graphite, they will be automatically placed in the team account with access to Graphs and Dashboards. ### [Listing Invited Users](#listing-invited-users) Once your invited team members have accepted the invite, and created a profile attached to your main account, they will appear as active team users in this list:

Listing Your Team members

It should go without saying that you should only invite someone you trust to view the account. If you’ve accidentally invited someone you no longer trust, you can delete them by clicking the “Remove User” button.

Remove Team User

After pressing the “Yes, Remove” button, the user will be removed from your team. # Additional Features Additional features that Hosted Graphite has to offer. * [Aggregation Rules](/additional-features/aggregation-rules) * Combine many.\*.metrics into a single metric and AVG value * [Data Views](/additional-features/data-views) * A built-in feature that can be used will all of your metrics stored in HG * [Logging](/additional-features/logging) * Forward logs to our Hosted Loki endpoint for storage and visualization # Aggregation Rules Combine many metrics into a single metric. Contents * [Aggregation Rules](/additional-features/aggregation-rules) * [What are aggregation rules?](#what-are-aggregation-rules) * [Examples](#examples) * [Manage Aggregation Rules](#manage-aggregation-rules) ### [What are aggregation rules?](#what-are-aggregation-rules) Aggregation rules offer the ability to automatically combine datapoints from multiple input metrics into a smaller set of output metrics according to defined rules. This is useful if you need to use hundreds or thousands of input metrics in a dashboard or graph query, but still want a fast answer. Aggregation rules allow you to effectively pre-compute the most expensive parts of some queries. Aggregation rules perform the same job as the [carbon\_aggregator](http://graphite.readthedocs.io/en/latest/config-carbon.html?highlight=aggregate#aggregation-rules-conf) service if you’re self-hosting the open-source Graphite tools. Hosted Graphite’s additional [*data views*](/additional-features/data-views) feature is also supported on top of the output metrics from an aggregation rule, so you can get the `:sum` or an arbitrary percentile across all the metrics matching an aggregation rule. **NOTE**: The output metric names will be automatically prefixed with “aggregates.” even if you do not specify it in the “output” field. **Why are aggregates useful?** If you have a render query using a wildcard to match a large number of metrics, the result of the render query can, in some cases, be very slow to return. Using aggregations reduces the number of metrics that need to be retrieved and can reduce the time spent rendering graphs. ### [Examples](#examples) If you have a number of metrics that match the pattern `servers.*.requests.`, and you want to render `servers.*.requests.200`, the overhead in matching all nodes which match the `*` in the second node, can be skipped if you set up an aggregation rule with the following pattern/output combination: ``` { "pattern": "servers.*.requests.", "output": "aggregates.servers.all.requests." } ``` The effect of this config would be as follows: Datapoints coming in for metrics named like this: ``` servers.server1.requests.200 servers.server2.requests.200 ``` Would be copied and combined as soon as the datapoints are received and a new metric is created: * `aggregates.servers.all.requests.200` Datapoints coming in for metrics named like: ``` servers.server1.requests.404 servers.server2.requests.404 ``` Would be copied and combined as soon as the datapoints are received and a new metric is created: * `aggregates.server.all.requests.404` Now, instead of defining a graph requesting servers.\*.requests.200 which could be slow if it matches hundreds or thousands of metrics, you can define a graph requesting `aggregates.servers.all.requests.200`. That means you’ll be querying just one metric, which will be much faster. You can also use the `<>` pattern as described in the carbon-aggregator docs which allows you to match across nodes. For example, the pattern/output config might look like this: ``` { "pattern": "servers.*.<>", "output": "aggregates.servers.all." } ``` This would combine all metrics seen with a starting `servers.*` into the same metric under `aggregates.servers.all.`. So if we saw some metrics which looked like: ``` servers.server1.requests.200 servers.server2.requests.200 servers.server3.requests.200 servers.server4.requests.200 ``` They would be combined into a new metric: * `aggregates.servers.all.requests.200` Similarly, the following: ``` servers.server1.auth.v2.success servers.server2.auth.v2.success servers.server3.auth.v2.success servers.server4.auth.v2.success ``` Would be matched by the same pattern and would combine into the new metric: * `servers.all.auth.v2.success` ### [Manage Aggregation Rules](#manage-aggregation-rules) Aggregation rules can be conveniently created, updated, and deleted through the Hosted Graphite UI. Your rules will be listed in the “Aggregate Rules” section of the Hosted Graphite application:

Aggregate Rules List

Create a new rule:

Create New Aggregate Rule

Once a new rule is made, a single metric is prefixed with `aggregates` will be created and appear in your metrics list. # Data Views Hosted Graphite automatically generates different statistical views on the data you send, providing fast views on your metrics at the most appropriate resolution for viewing on your dashboard. Contents * [Data Views](/additional-features/data-views) * [Example Usage](#example-usage) * [Comparison to StatsD](#comparison-to-statsd) * [Percentile Views](#percentile-views) * [Data Views and Graphite Functions](#data-views-and-graphite-functions) The different views we keep by default for all our graphite metrics are: * **:avg** \ (default view) Average of datapoints received during the timeframe of the current graph resolution. * **:sum** \ Sum of datapoints received during the timeframe of the current graph resolution. This changes depending on the zoom level. Using ":sum" on a graph with datapoints drawn every 30 seconds will show the sum of all datapoints received in each 30-second period. * **:sumrate** \ The sum is divided by the number of seconds between datapoints, giving you a per-second rate.:sum and :sumrate are the most common views to use with Counter metrics. * **:sum30** **:sum60** **:sumN** \ Provides the sum received over a given number of seconds, calculated from the :sumrate. E.g. ":sum30" provides the sum over 30 seconds. This is useful when you want a sum of a metric that is consistent regardless of what data resolution you are viewing. When drawing a graph at a wider timescale there are more datapoints than available space to draw them. Using the ":sumN" syntax lets you see an accurately scaled sum over a given number of seconds. * **:min** \ Minimum value * **:max** \ Maximum value * **:obvs**\ The number of 'observations' or data-points sent to a metric namespace during the timeframe of the current graph resolution (like 30s). * **:obvsrate** \ The observations are divided by the number of seconds in the collection period, giving you a per-second rate. * **:90pct** **:95pct** **:99pct** **etc...** \ Want arbitrary percentile data? Just add the number after the colon followed by ‘pct’. It accepts values from 01 to 99. If you want the 100th percentile you can use ":max". ### [Example Usage](#example-usage) Append a view to the end of your metric to visualize your aggregated data in different ways. A simple example of this would be `data-view-metric:sum` or `test.testing:max`.

Data Views Example

If you are using tagged metrics, you will need to include the special tag `_hg_dataview`. For example, querying `seriesByTag("_hg_dataview=99pct", "host=db-0001", "name=cpu.usage")`will retrieve the *99th percentile* data view for the metric *cpu.usage* on host *db-0001.* For more information please refer to our [documentation](/sending-metrics/graphite-tag-support) on tagged metrics. ### [Comparison To StatsD](#comparison-to-statsd) The Hosted Graphite views aren’t perfectly analogous to StatsD, but for general use, it accomplishes similar results. Using our Data Views feature over StatsD can greatly reduce the number of metrics stored in your HG account since 1 counter produces 2 (count/rate) metrics, and 1 timer will produce 11 unique metrics. * **counters** -> `:sum` (total count), `:sumrate` (per-second rate) * **gauges** -> `:avg`, `:min`, `:max` (aggregations of reported values) * **timers** -> `:avg`, `:min`, `:max`, `:95pct`, etc. (distribution-style views) ### [Percentile Views](#percentile-views) All percentile views are calculated per resolution (5s, 30s, 300s, 3600s) using reservoir (or random) sampling. We keep 10 samples at the 5s resolution, 40 at 30s & 300s resolution, and 100 at 3600s resolution. This is not as accurate as storing a full histogram, but it requires less storage on our end. As long as we keep the sampling in mind, we find it’s a reasonable trade-off for simple metrics. ### [Data Views and Graphite Functions](#data-views-and-graphite-functions) [Graphite Functions](https://graphite.readthedocs.io/en/latest/functions.html) act on one or more data series, transforming them into a new data series. When using a graphite function it is important to note that: * The data view specified in the query is not changed depending on the function. If you use the [sumSeries()](http://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.sumSeries) function, you will be summing the average values of each series unless you also use **:sum** in your query as well. * The graph’s resolution can be changed by functions, but the resolution of the data series cannot. For example, [summarize()](http://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.summarize) allows you to group all the datapoints in a specified timeframe into one. Changing the zoom level of the graph won’t change the interval output by the summarize() function, but it may change the number of data points available to be summarized. # Logging Hosted Graphite now supports log ingestion and visualization through a managed Loki backend. Contents: * [Add a Loki Data Source](#add-a-loki-data-source) * [Install and Configure a Log Collector](#install-and-configure-a-log-collector) * [Query Logs on Your Dashboards](#query-logs-on-your-dashboards) ### [Add a Loki Data Source](#add-a-loki-data-source) **Reach out to our support channel and we will create a Loki Key and Data Source for your account.** Once created, you can locate your Loki Key in Access ⇒ Sharing:
And you can locate your Loki Data Source in Dashboards ⇒ Connections ⇒ Your Connections:
You will be able to use this Data Source in your dashboards to query and visualize logs that are ingested to our Loki backend (7 day log retention). ### [Install and Configure a Log Collector](#install-and-configure-a-log-collector) MetricFire supports log forwarding using OpenTelemetry Contrib, Promtail, and Alloy. You must install one of these collectors in your server(s) and this guide uses [Promtail](https://grafana.com/docs/loki/latest/send-data/promtail/) - which reads log files like **/var/log/syslog** and forwards them to our Loki endpoint. #### Install Promtail (Ubuntu) {% tabs %} {% tab title="AMD64" %} wget sudo apt-get install -y ./promtail\_3.5.3\_amd64.deb {% endtab %} {% tab title="ARM64" %} wget sudo apt-get install -y ./promtail\_3.5.3\_arm64.deb {% endtab %} {% endtabs %} #### Configure Promtail Edit Promtail’s configuration file, generally located at: **/etc/promtail/config.yml** ```yml server: http_listen_port: 9080 grpc_listen_port: 0 positions: filename: /tmp/positions.yaml clients: - url: https://@www.hostedgraphite.com/logs/sink scrape_configs: - job_name: varlogs static_configs: - targets: - localhost labels: host: job: varlogs __path__: /var/log/*log ``` Make sure to include your HG API-KEY and HOST-NAME of the server. You may also modify the job\_name, labels, and path values for more specific collection/querying. Save your config file and restart Promtail to forward your logs. ``` sudo systemctl restart promtail sudo systemctl status promtail ``` You may also need to assign permissions to allow Promtail to read your /var/\*log files: ``` sudo usermod -aG adm promtail sudo systemctl restart promtail ``` ### [Query Logs on Your Dashboards](#query-logs-on-your-dashboards) Once Promtail is sending logs and the Loki data source is configured, create a new dashboard panel. You can create custom [LogQL](https://grafana.com/docs/loki/latest/query/) queries to visualize your log data in meaningful and useful ways. Add a new panel, set the **Data Source to Loki - HostedGraphite,** and switch the query editor to "Code" mode. #### Basic Log Query View **/var/log/\*** lines forwarded by your server - you can query by job, host, and/or filename: ``` {job="varlogs", host="", filename="/var/log/syslog"} ```
#### Filter by Content ``` {job="varlogs", host=""} |= "ERROR" ``` You can filter using: * `|=` substring match * `|~` regex * `|=` and `!=` content filtering Loki supports full LogQL for powerful [parsing and field extraction](https://grafana.com/docs/loki/latest/query/log_queries/). #### Graph Log-Based Metrics Loki allows you to convert logs into metrics using count\_over\_time() and [similar functions](https://grafana.com/docs/loki/latest/query/metric_queries/#log-range-aggregations). This turns raw log patterns into quantifiable data that you can graph and alert on. Examples: **Failure Counts per Host:** ```logql sum by (host) ( count_over_time({job="varlogs", host=""} |= "failure" [5m]) ) ``` **Cron executions 5m Sum:** ```logql sum by (host) ( count_over_time({job="varlogs", host=""} |= "cron" [5m]) ) ```
**Compare Logs and Metrics For Enhanced Visibility** By correlating log data with your existing Graphite metrics, you can observe both system behavior and the underlying events that contribute to changes in performance. This combined view helps you identify patterns to diagnose issues faster, and gain context behind metric fluctuations.
The MetricFire Blog has detailed articles around log + metric monitoring for popular services like [MySQL](https://www.metricfire.com/blog/visualize-logs-alongside-metrics-a-complete-guide-for-monitoring-slow-mysql-queries/), [PostgreSQL](https://www.metricfire.com/blog/visualize-logs-alongside-metrics-complete-observability-for-slow-postgresql-queries/), [Nginx](https://www.metricfire.com/blog/nginx-performance-monitoring-loki-logs-and-telegraf-metrics/), [Elasticsearch](https://www.metricfire.com/blog/visualize-logs-alongside-metrics-complete-observability-elasticsearch-performance/), and many more! Reach out to us at to enable Loki Logging for your account! # API Guides We have developed a variety of APIs for you to easily build your own integrations to Hosted Graphite tools: * [*Metrics API*](/api-guides/metrics-api) \ Search account metrics * [*Tag API*](/api-guides/tag-api) \ Search tagged metrics * [*Graphite Render API*](/api-guides/graphite-render-api) \ Get more out of your metric data * [*Render Variables API*](/api-guides/render-variables-api) \ Build dynamic metric queries * [*Dashboard HTTP API*](/api-guides/dashboard-api) \ Fetch, create, and update your dashboards * [*Annotations and Events API*](/api-guides/annotations-and-events-api)\ Create events with tags * [*Aggregation Rules API*](/api-guides/aggregation-rules-api)\ Consolidate large sets of metrics to improve query retrieval * [*Alerts API*](/api-guides/alerts-apis)\ Manage alerts, notification channels, and define scheduled mute periods * [Team Members API](https://docs.hostedgraphite.com/api-guides/team-members-api)\ List all Team Members and their respective permissions # Metrics API Contents * [Metrics API](/api-guides/metrics-api) * [Searching Metrics](#searching-metrics) * [Busy Metrics](#busy-metrics) * [Invalid Metrics](#invalid-metrics) * [Deleting Metrics](#deleting-metrics) ### [Searching Metrics](#searching-metrics) ## Locate Metrics `GET` `GET /api/v1/metric/search?` #### Path Parameters | Name | Type | Description | | ----------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------- | | pattern\* | | \[required] metric pattern to search for | | only\_updated\_after | String |

To limit search results for metrics that only received data points in the last n hours

| | only\_updated\_before | String | To limit search results for metrics that have not received data points in the last n month | | with\_times | String | To return the first (“created\_at”) and last (“last\_updated”) seen timestamp for that metric | | page | String | Return the nth page of results, for searches that return more than the maximum page size (default maximum size: 100) | {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "results": [ "foo.bar.1", "foo.bar.2" ], "metrics_matched": 2, "total_metrics": 12345, "next_page": 2 } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid request {% endtab %} {% endtabs %} **Curl examples:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/metric/search?pattern=foo.bar.*" ``` Search for metrics that match the pattern `foo.bar.*` ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/metric/search?pattern=*&only_updated_before=30d" ``` Search for all (\*) metrics that have not received data for 30d **Response format for: `&with_times=true`**: ```json HTTP/1.1 200 Content-Type: application/json { "results": ["foo.bar.1", "foo.bar.2"], "metrics_matched": 2, "total_metrics": 12345, "times": { "foo.bar.1": { "created_at": 1234567890, "last_updated": 1234567890 }, "foo.bar.2": { "created_at": 1234567890, "last_updated": 1234567890 } } } ``` *** ### [Busy Metrics](#busy-metrics) ## Locate Busy Metrics `GET` `/api/v1/metric/busy` {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "metric": "busy-metric.1", "pct_ratelimited": 60 } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid request {% endtab %} {% endtabs %} **Curl Example:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/metric/busy" ``` *** ### [Invalid Metrics](#invalid-metrics) ## Locate Invalid Metrics `GET` `/api/v1/metric/invalid` {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "metric": "busy-metric.1", "pct_ratelimited": 60 } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid request {% endtab %} {% endtabs %} **Curl example:** ``` curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/metric/invalid" ``` *** ### [Deleting Metrics](#deleting-metrics) ## Delete Metrics `DELETE` `/api/v1/metric/delete` {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="400" %} Bad Request Invalid request {% endtab %} {% endtabs %} **Curl example:** ``` curl -X DELETE "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/metric/delete" -d '{"pattern":"test.*"}' ``` Delete all metrics that match the pattern `test.*`. Legacy **POST** route: `POST` `/api/v1/metric/delete` {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="400" %} Bad Request Invalid request {% endtab %} {% endtabs %} **Curl example:** ``` curl --http1.1 -d "pattern=foo.bar.*" "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/metric/delete/" ``` Delete all metrics that match the pattern `foo.bar.*`. # Tag API Contents * [Tag API](/api-guides/tag-api) * [Listing All Tags](#listing-all-tags) * [Search Tagged Metrics](#search-tagged-metrics) * [Locate Tagged Metrics and Values](#locate-tagged-metrics-and-values) * [Deleting Tagged Metrics](#deleting-tagged-metrics) * [Monitoring your Tagged Metrics usage](#monitoring-your-tagged-metrics-usage) ### [Listing All Tags](#listing-all-tags) `GET` `GET /api/v1/tags` {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json [ "cluster", "host", "process", "service", "name" ] ``` {% endtab %} {% tab title="400" %} Bad Request {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/tags" ``` *** ### [Search Tagged Metrics](#search-tagged-metrics) `GET` `GET /api/v1/tags/` #### Path Parameters | Name | Type | Description | | ------ | ------ | ------------------------------------------------------------------------------------------------------------------- | | expr | String | To search for the tagged series that match the expression, users should include both tagname and value in the query | | filter | String | Tag value to search for | | limit | String | To limit the number of search results to the given number | {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="400 " %} Bad Request {% endtab %} {% tab title="501" %} Not Implemented {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/tags/host?filter=prod-00*&limit=3" ``` Search for tag name `host` that matches the value `prod`, and only show 3 matching results. ### [Locate Tagged Metrics and Values](#locate-tagged-metrics-and-values) `GET` `/api/v1/tags/findSeries?` #### Path Parameters | Name | Type | Description | | ------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | expr | String |

Format: expr=tagname=tagvalue or expr=tagname1=tagvalue1\&tagname2=tagvalue2
To search for the tagged series that match the expression, users should include both tagname and value in the query.

| | filter | String |

Format: filter=prod-00\*
Tag value to search for.

| | limit | String |

Format: limit=3
To limit the number of search results to the given number.

| {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json [ "0.queue_backlog;cluster=prod;host=prod-egg-0021;process=grind;service=data_process", "2.queue_backlog;cluster=prod;host=prod-egg-0021;process=grind;service=data_process", "0.items;cluster=prod;host=prod-egg-001;process=dump;service=data_process" ] ``` {% endtab %} {% tab title="400 " %} Bad Request {% endtab %} {% tab title="500 " %} Internal Server Error {% endtab %} {% endtabs %} **Curl examples:** ```bash curl -X GET "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/tags/findSeries?expr=cluster=prod&host=egg-0001" ``` Search for tagged metric/series that match the expression `cluster=prod&host=egg-0001` **Response format with ‘`?filter=machine-000*&limit=3`’**: ```json HTTP/1.1 200 Content-Type: application/json {"tag": "host", "values": [{"count": 59, "value": "machine-0001"}, {"count": 57, "value": "machine-0002"}, {"count": 59, "value": "machine-0003"}] } ``` *** ### [Deleting Tagged Metrics](#deleting-tagged-metrics) `POST` `/api/v1/tags/delSeries` {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="400 " %} Bad Request {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X POST "https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/tags/delSeries" --data-urlencode "path=foo.bar;tag1=val1;tag2=val2" ``` Delete all series that match the path *foo.bar* and *tag1=val1* and *tag2=val2*. *** ### [Monitoring your Tagged Metrics usage](#monitoring-your-tagged-metrics-usage) If you have sent tagged metrics then the number of tagged metrics you have sent will be displayed when you hover over the metric count on your [Homepage](https://www.hostedgraphite.com/app/):

Total Tags in Account

You can also locate, search, and delete your tagged metrics from within our Tag Search UI:

Tag Search UI

# Graphite Render API Contents * [Graphite Render API](/api-guides/graphite-render-api) * [Render Endpoint](#render-endpoint) * [Examples](#examples) With the Graphite Render API, you can: * View raw metric data outside of your primary dashboards. * Apply functions to your metric data. * Search for data within a specific or relative time frame. * Retrieve the data in different display formats. * Use Graphite [Access Keys](https://docs.hostedgraphite.com/account-management/access-keys) to integrate metric data into external tools or services. ### [Render Endpoint](#render-endpoint) ## Render endpoint `GET` `/YOUR-UID/ACCESS-KEY/graphite/render?target=` To integrate Hosted Graphite data externally, [Create an Access Key](/account-management/access-keys) to authorize render requests. Copy the key URL and add `/render?target=` #### Path Parameters | Name | Type | Description | | ---------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | target\* | String | This parameter specifies the path to identify one or more metrics, supports wildcards (\*), and can be used with [Data Views](https://docs.hostedgraphite.com/additional-features/data-views). | | from/to | String | These parameters accept relative or absolute times (or both). Relative Time is the length of time from the current time and is always preceded by a minus sign (-). If no time parameters are specified, the default values are `from=now-24h` and `to=now`. | | \&format=png | String | Renders a graph as a 330×250 px png (default option, if the \&format parameter is left unspecified). | | \&format=svg | String | Renders a graph as a 330×250 px svg. | | \&format=raw | String | Line-delimited output. (target, start timestamp, end timestamp, step, data) | | \&format=csv | String | Creates a CSV file to download (metric, date/time, value) | | \&format=json | String | Returns a JSON object. | {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="403" %} Forbidden Invalid {% endtab %} {% endtabs %} The accepted units of time are: | Abbreviation | Unit | | ------------ | --------------- | | s | Seconds | | min | Minutes | | h | Hours | | d | Days | | w | Weeks | | mon | Month (30 Days) | | y | Year (365 Days) | ### [Examples](#examples) Using the **sample** metric `foo.bar`, below are some example queries and outputs: ``` https://www.hostedgraphite.com/YOUR-UID/ACCESS-KEY/graphite/render?target=foo.bar&from=now-30min ```

Default png render

``` https://www.hostedgraphite.com/YOUR-UID/ACCESS-KEY/graphite/render?target=alias(foo.bar,"Uptime")&from=now-30min ```

Render with aliased metric

``` https://www.hostedgraphite.com/YOUR-UID/ACCESS-KEY/graphite/render?target=foo.bar&from=now-30min&format=json ```

Render as json

**Curl Example:** ``` curl -X GET "https://www.hostedgraphite.com/YOUR-UID/ACCESS-KEY/graphite/render?target=foo.bar&from=now-30min&format=json" ``` For more information about building queries and customizing outputs, visit the [Graphite Render Docs](https://graphite.readthedocs.io/en/latest/render_api.html). # Render Variables API Contents * [Render Variables API](/api-guides/render-variables-api) * [Creating a variable](#creating-a-variable) * [Updating a variable](#updating-a-variable) * [Search variables](#search-variables) * [Get variable values](#get-variable-values) * [Delete a variable](#delete-a-variable) * [Graphing with render variables](#graphing-with-render-variables) Render Variables provide versatility when constructing metric queries. Instead of hard-coding full metric names, use variables as placeholders to dynamically build your queries. This is similar, but not the same as the primary dashboard’s built-in template variables. **NOTE**: we advise against using render variables in alert queries. ### [Creating a variable](#creating-a-variable) `POST` `/v1/variables//` Replace \ with the name you want to give the variable. Place your variable values in a JSON file and include it in the POST request. A variable name is required, however the values can be empty. {% tabs %} {% tab title="201" %} Created { "url": "/v1/variables//" } {% endtab %} {% tab title="400" %} Bad Request JSON could be incorrectly formatted or the variable name already exists {% endtab %} {% endtabs %} **Example variable values:** ```json { "values": [ "foobar", "foo", "bar", "baz" ] } ``` **Curl example:** ```bash curl -X POST "https://YOUR-API-KEY@api.hostedgraphite.com/v1/variables//" -d @values.json ``` Make sure to include the forward slash character at the end of the request. *** ### [Updating a variable](#updating-a-variable) `PUT` `/v1/variables//` Overwrites the values for an existing variable {% tabs %} {% tab title="200: OK { "url": "/v1/variables//" }" %} {% endtab %} {% tab title="400: Bad Request JSON could be incorrectly formatted" %} {% endtab %} {% tab title="404: Not Found " %} {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X PUT "https://@api.hostedgraphite.com/v1/variables//" -d @values.json ``` *** ### [Search variables](#search-variables) `GET` `/v1/variables/?query=` A “contains” query that will return a JSON object with the name and path of variables that contain the search term. To list all variables, leave the query empty. #### Path Parameters | Name | Type | Description | | ----- | ------ | ----------- | | query | String | | {% tabs %} {% tab title="200: OK { "variables" : \[ { "name" : "foo", "url" : "/v1/variables/foo/" }, { "name" : "bar", "url" : "/v1/variables/bar/" } ] }" %} {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X GET "https://@api.hostedgraphite.com/v1/variables/?query=" ``` If you have no variables or there are no variables that match your query you will be returned with an empty object: ```json { "variables": [] } ``` *** ### [Get variable values](#get-variable-values) `GET` `/v1/variables//` Returns a JSON object with the values assigned to \ {% tabs %} {% tab title="200: OK { "values": \[ "foobar", "foo", "bar", "baz" ] }" %} {% endtab %} {% tab title="404: Not Found " %} {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X GET "https://@api.hostedgraphite.com/v1/variables//" ``` *** ### [Delete a variable](#delete-a-variable) `DELETE` `/v1/variables//` {% tabs %} {% tab title="200: OK " %} {% endtab %} {% tab title="404: Not Found " %} {% endtab %} {% endtabs %} **Curl example:** ```bash curl -X DELETE "https://@api.hostedgraphite.com/v1/variables//" ``` *** ### [Graphing with render variables](#graphing-with-render-variables) This cannot be done with the simple query builder, you must toggle the edit mode to the free-text mode. Refer to a render variable by including the variable name enclosed in angled brackets in your metric query: **\** **Using dashboard templates within render variables** You can place template variables within render variables by surrounding them with curly brackets: ```bash $foo.<{$foo}_metric>.bar ``` # Dashboard API Contents * [Dashboard HTTP API](/api-guides/dashboard-api) * [Dashboard Snapshot PNG](#dashboard-snapshot-png) * [Create dashboard](#create-and-update-dashboard) * [Get dashboard](#get-dashboard) * [Delete dashboard](#delete-dashboard) * [Search dashboards](#search-dashboards) * [Create a New Folder](#create-a-new-folder) ### [Dashboard Snapshot PNG](#dashboard-rendered-pngs) `GET` `/api/v3/grafana/render/?target=` Returns PNG image of a dashboard panel. To display any data on the graph you must include a ‘target’, you can append multiple targets to the request as follows: `?target=foo.bar.A&target=foo.bar.B` #### Parameters | Name | Type | Description | | | | | | | ---------------------------------------- | ------ | ------------------------------------------------- | ---------------------------- | - | - | - | --------------------- | | target\* | String | graphite query | | | | | | | from | String |

Format: -\m | h | d | w | M | Y
default: -6h

| | to | String |

Format: now-\m | h | d | w | M | Y
default: now

| | width | String |

number measured in pixels
default: 1000

| | | | | | | height | String |

number measured in pixels
default: 500

| | | | | | | style | String |

Format: “light” | “dark ”
default: dark

| | | | | **Curl Example:** ```bash curl -i -X GET \ "https://@api.hostedgraphite.com/api/v3/grafana/render/?target=foo.bar" ``` **Response**: {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: image/png ``` \ {% endtab %} {% tab title="400" %} Bad Request Invalid form data {% endtab %} {% tab title="401" %} Unauthorized {% endtab %} {% endtabs %} **Time Queries:** * `m` = Minutes * `h` = Hours * `d` = Days * `w` = Weeks * `M` = Months * `Y` = Years *** ### **You can interact with your dashboard instance directly through** [**Grafana's HTTP API**](https://grafana.com/docs/grafana/latest/developers/http_api/) To use the Grafana API, **you will need to create a new Grafana Service Account Token** within the Grafana UI (Settings => Service Accounts => Add Service Account with Admin role => Generate Service Account Token).
Create Service Account & Token

Create Service Account & Token

**Additional Considerations**: * Since MetricFire now hosts Grafana v9.5, dashboard JSON templates must now be wrapped in the following object, in order to create/update dashboards programatically ("folderID": 0, maps to the default "General" folder, but any folderID can be specified): ```json { "dashboard": { ...entire dashboard object... }, "folderId": 0, "overwrite": true } ``` * In all below examples `` refers to the URL subdomain of your hosted instance, `uid` refers to the unique identifier of the related resource. Below are examples of requests made using the Grafana Dashboard HTTP API. ### [Create & Update dashboard](#create-dashboard) `POST` `/api/dashboards/db` Creates a new dashboard or updates an existing dashboard **Example:** ```bash curl -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer " \ "https://.hostedgraphite.com/api/dashboards/db" \ -d @dashboard.json ``` {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "slug": "production-overview", "status": "success", "version": 1 } ``` {% endtab %} {% tab title="400" %} Bad Request Invalid JSON {% endtab %} {% tab title="412" %} Conflict Dashboard already exists {% endtab %} {% endtabs %} **Example dashboard.json file:** ```json { "dashboard": { "title": "Minimal Dashboard", "panels": [] }, "folderId": 0, "overwrite": false } ``` *** ### [Get dashboard](#get-dashboard) `GET` `/api/dashboards/uid/:uid` Returns a dashboard given the dashboard's uid | Name | Value | | ------------- | -------------------------------- | | Content-Type | `application/json` | | Authorization | `Bearer ` | **Curl examples:** ```bash curl -X GET \ -H "Authorization: Bearer " \ "https://.hostedgraphite.com/api/dashboards/uid/" ``` **Response**: {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "meta": { "isStarred": false, "slug": "production-overview" }, "model": { "id": null, "title": "Production Overview", "tags": [ "templated" ], "timezone": "browser", "panels": [ { } ] "schemaVersion": 6, "version": 0 }, } ``` {% endtab %} {% tab title="400" %} Bad Request No dashboard specified {% endtab %} {% tab title="404" %} Not Found Dashboard not found {% endtab %} {% endtabs %} *** ### [Delete dashboard](#delete-dashboard) `DELETE` `/api/dashboards/uid/:uid` Deletes a dashboard given the dashboard slug. | Name | Value | | ------------- | -------------------------------- | | Content-Type | `application/json` | | Authorization | `Bearer ` | **Example:** 
curl -X DELETE \
-H "Authorization: Bearer <SERVICE-ACCOUNT-TOKEN>" \
"https://<UID>.hostedgraphite.com/api/dashboards/uid/<DASHBOARD-UID>"
**Response**: {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "title": "Example Dash", "message": "Dashboard Example Dash deleted", "id": 3 } ``` {% endtab %} {% tab title="400 " %} Bad Request No dashboard specified {% endtab %} {% tab title="404" %} Dashboard not found {% endtab %} {% endtabs %} *** ### [Search dashboards](#search-dashboards) `GET` `/api/search` Returns a list of dashboards given a search query. | Name | Value | | ------------- | -------------------------------- | | Content-Type | `application/json` | | Authorization | `Bearer ` | **Example**: ```bash curl -X GET \ -H "Authorization: Bearer " \ "https://.hostedgraphite.com/api/search" ``` **Response**: {% tabs %} {% tab title="200:" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json [ { "id": 1, "title": "Production Overview", "uri": "db/production-overview", "type": "dash-db", "tags": [ "templated", "production" ], "isStarred": true }, { "id": 4, "title": "Production Webservers", "uri": "db/production-webservers", "type": "dash-db", "tags": [ "production" ], "isStarred": true } ... ] ``` {% endtab %} {% tab title="400" %} Bad Request Invalid form data {% endtab %} {% endtabs %} *** ### [Create a New Folder](#create-a-new-folder) `POST` `/api/folders` Creates a new folder for dashboard organization. **Headers** | Name | Value | | ------------- | -------------------------------- | | Content-Type | `application/json` | | Authorization | `Bearer ` | **JSON Schema** | Name | Type | Description | | ------- | ------ | ------------------ | | `title` | string | name of new folder | **Example**: ```sh curl -X POST -H "Content-Type: application/json" \ -H "Authorization: Bearer " \ "https://.hostedgraphite.com/api/folders" -d '{"title": "Test Folder"}' ``` **Response:** {% tabs %} {% tab title="200" %} ```http HTTP/1.1 200 Content-Type: application/json ``` ```json { "id": 70, "uid": "bfc49500-7c2d-4d73-ba43-d8ab677d4836", "title": "Test Folder", "url": "/dashboards/f/bfc49500-7c2d-4d73-ba43-d8ab677d4836/test-folder", "hasAcl": false, "canSave": true, "canEdit": true, "canAdmin": true, "canDelete": true, "createdBy": "Anonymous", "created": "2024-10-15T13:10:22.922696496Z", "updatedBy": "Anonymous", "updated": "2024-10-15T13:10:22.922696646Z", "version": 1 } ``` {% endtab %} {% tab title="400" %} Errors (invalid json, missing or invalid fields, etc {% endtab %} {% tab title="409" %} Folder already exists {% endtab %} {% endtabs %} # Annotations and Events API Contents * [Annotations and Events API](/api-guides/annotations-and-events-api) * [Creating an annotation](#creating-an-annotation-for-the-impatient) * [Delete a single event](#delete-a-single-event) * [Graphite Events](#graphite-events) * [Graphite Composer](#graphite-composer) * [Dashboard Annotations](#dashboard-annotations) Hosted Graphite supports Annotations and Graphite Events. The Annotations feature is designed as a simpler interface to create events with tags. An annotation relates your metrics to intermittent events for example when you deploy code, run some tests (hopefully successfully), or a long-running job completes. ### [Creating an Annotation](#creating-an-annotation-for-the-impatient) `POST` `/api/v1/annotations/events/` Create an annotation with titles, tags, and descriptions that will be time-stamped as it is received. #### Path Parameters | Name | Type | Description | | --------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | title\* | String | A short summary of the annotation. | | description | String | A more in-depth description of the annotation. | | tags | String | A single word or multiple short strings (all case-insensitive) to organize different types of events. *e.g.* ‘deployments’, ‘feature’, ‘test’ etc. | | start\_time | String | Unix timestamp describing when the event occurred. If this is not specified, the time of event creation is used. | | end\_time | String | Optional timestamp describing when the event completed. **Note:** We currently don’t support [region events](http://docs.grafana.org/reference/annotations/#adding-regions-events). As a result, *end\_time* has no actual use in annotations. | {% tabs %} {% tab title="201 " %} Created {% endtab %} {% tab title="400 " %} Bad Request {% endtab %} {% endtabs %} This snippet of code will create an annotation with the tags “deployment” and “feature”, titled “New Super-duper Feature”: ``` curl -X POST https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/annotations/events/ -d \ "&title=New Super-duper Feature\ &tag=deployment\ &tag=feature" ``` There’s no restriction on having annotations with the same time stamps, but the combination of start time and title must be unique. **Authentication**: The API uses basic HTTP authorization. An example using curl: ```bash curl -X POST https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/annotations/events/ ``` or ```bash curl -u 'YOUR-API-KEY:' -X POST https://api.hostedgraphite.com/api/v1/annotations/events/ ``` *** ### [Delete a single event](#delete-a-single-event) ## Delete an annotation `DELETE` `/api/v1/annotations/events/` #### Path Parameters | Name | Type | Description | | ------------------------------------------- | ------ | ----------- | | title\* | String | | | timestamp\* | String | | {% tabs %} {% tab title="200 " %} OK {% endtab %} {% tab title="400 " %} Bad Request {% endtab %} {% endtabs %} **Curl example**: ```bash curl -X DELETE https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/annotations/events/ \ -d '{"title": "New Super-duper Feature", "timestamp": 1431607046}' ``` *** ### [Graphite Events](#graphite-events) Events as per the Graphite project are fully supported. Insertion and rendering of [Graphite Events](https://graphite.readthedocs.io/en/latest/events.html#graphite-events) work the same as the annotations endpoint, though the Graphite events API takes a JSON formatted data structure. Here's an example of creating a single Graphite event: ```bash curl -X POST https://YOUR-API-KEY@api.hostedgraphite.com/api/v1/graphite/events/ \ -d '{"what": "New Super-duper Feature", "when":1431607046, "tags":"deployment feature"}' ``` #### [Graphite Composer](#graphite-composer) **Annotations**: To support the display of Annotations, there is a function **annotations()** available in graphite to query and display the events. There are two options: ```bash annotations("*") ``` This first option with a wildcard queries all available annotations for the time period selected ```bash annotations("restart", "deployment") ``` This version queries all annotations with the tags “restart” and “deployment” for the time period selected **Events**: The graphite events function **events()** is also available: ```bash drawAsInfinite(events("*") drawAsInfinite(events("restart", "deployment") ``` *** ### [Dashboard Annotations](#dashboard-annotations) The dashboard annotations feature works exactly as specified in the [annotation documentation](https://grafana.com/docs/grafana/latest/dashboards/annotations/). It can be used to query by tags (not graphite targets). * To open the annotations panel, click the settings icon in the top bar and select *Annotations:*

Locate annotations in dashboard settings

* Set the datasource to ‘hostedgraphite’ and use the ‘Graphite event tags’ input box to filter by tags (Individual wildcards also work to display all events):

Configure the annotation query with tags

* You can now see annotations rendered on your graph, and can hover over the base to see the tags and description:

Visualize annotations

# Aggregation Rules API Contents * [Aggregation Rules API](/api-guides/aggregation-rules-api) * [Creating Aggregation Rules](#creating-aggregation-rules) * [Listing Aggregations](#listing-aggregations) * [List Aggregation Rules by ID](#list-aggregation-ules-by-id) * [Deleting Aggregates](#deleting-aggregates) * [Editing Your Aggregations](#editing-your-aggregations) Aggregation rules consolidate the averaged values of many metrics into one, allowing for speedy graph rendering and visualization. Managing aggregation rules can be done easily with Hosted Graphite. Learn more about what our aggregation rules do, and how to use our Aggregation Feature [*here*](/additional-features/aggregation-rules). ### [Creating Aggregation Rules](#creating-aggregation-rules) `POST` `/v1/aggregates/` #### Path Parameters | Name | Type | Description | | ----------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------- | | pattern\* | String | Regex or node pattern to match ingested metrics to. | | output\* | String | Metric output, when a metric is matched with the pattern, those metrics will be mapped to this output and appear in your account. | {% tabs %} {% tab title="200" %} OK (The aggregation rule was created successfully. Returns created aggregation’s ID) {% endtab %} {% tab title="400" %} Bad Request Creation failed. Pattern or output is an invalid input (or not provided). Or the aggregation rule creation failed because the pattern is too computationally expensive to match against your metric names. {% endtab %} {% tab title="403" %} Forbidden Creation failed, you have reached the limit for your account {% endtab %} {% tab title="409" %} Conflict Creation failed, aggregation with this pattern & output already exists. {% endtab %} {% tab title="500" %} Internal Server Error Creation failed due to a server error - this shouldn’t happen. {% endtab %} {% endtabs %} **Curl Example**: ```bash curl -H "Content-Type: application/json" -X POST -d \ '{"pattern": "servers.*.requests.", "output": "servers.all.requests."}' \ "https://YOUR-API-KEY@api.hostedgraphite.com/v1/aggregates/" ``` Creates an aggregation with the pattern: `servers.*.requests.` and the `output servers.all.requests.`. For example, metrics that look like `servers.server-0001.requests.404` and `servers.server-0002.requests.404` would appear in your account as `aggregates.servers.all.requests.404`. *** ### [Listing Aggregations](#listing-aggregations) `GET` `/v1/aggregates` {% tabs %} {% tab title="200" %} OK List of your aggregation rules. {% endtab %} {% endtabs %} ## [List Aggregation Rules by ID](#list-aggregation-rules-by-id) `GET` `/v1/aggregates//` Lists your aggregations or individual aggregation if the optional ID is passed. {% tabs %} {% tab title="200" %} OK Aggregation with passed ID if it exists, empty list otherwise. {% endtab %} {% endtabs %} **Curl Example**: ```bash curl "https://YOUR-API-KEY@api.hostedgraphite.com/v1/aggregates//" ``` *** ### [Deleting Aggregates](#deleting-aggregates) `DELETE` `/v1/aggregates//` Delete aggregation rule with passed ID {% tabs %} {% tab title="200" %} OK Aggregation was successfully deleted. {% endtab %} {% tab title="400" %} Bad Request No aggregation ID passed. {% endtab %} {% tab title="412" %} Precondition Failed Aggregation with passed ID does not exist. {% endtab %} {% tab title="500" %} Internal Server Error Delete failed, please try again. {% endtab %} {% endtabs %} **Curl Example**: ```bash curl -X DELETE "https://YOUR-API-KEY@api.hostedgraphite.com/v1/aggregates//" ``` *** ### [Editing Your Aggregations](#editing-your-aggregations) To edit an aggregation rule, you can do this through the Hosted Graphite UI. Additionally, aggregation rules have no state, so it is safe to delete and re-create them. Learn more about our [*Aggregate Rules Feature*](/additional-features/aggregation-rules).

Aggregate Rules UI

# Alerts APIs We have 3 useful API’s to help you manage and define scheduled mute periods. * [Alerts API](/alerting-guide/alerts-api) * [Notification Channels API](/alerting-guide/notification-channels-api) * [Scheduled Mutes API](/alerting-guide/scheduled-mutes-api) # Team Members API The Team Members API allows you to programmatically retrieve the members and permissions associated with your Hosted Graphite team. `GET` `/team/members/` Returns a list of users associated with your team, and their respective permissions. {% tabs %} {% tab title="200" %} Success {% endtab %} {% tab title="500" %} Internal Server Error {% endtab %} {% endtabs %} **Example JSON response:** ```json { "total_users": 2, "users": [ { "user": "admin@example.com", "permission": "Team Manager" }, { "user": "user@example.com", "permission": "Read and Write" } ] } ``` **Curl example:** ```bash curl -X GET "https://@api.hostedgraphite.com/team/members/" ``` Additional routes for this (create, update, delete) are currently on our development roadmap! # FAQ Contents * [General](#general) * [Business](#business) * [Technical](#technical) * [Account Metrics and Limiting](#account-metrics-and-limiting) * [Customization](#customization) * [Troubleshooting](#troubleshooting) * [Support](#support) * [Changelog](/faq/changelog) ### [General](#general) * **What is Graphite?** [Graphite](https://graphite.readthedocs.io/en/latest/) is an open-source, highly scalable, real-time graphing system. The metrics that you send to Graphite’s *carbon* endpoint will be aggregated, timestamped, and can be graphed using the Graphite-Web [dashboard interface](https://graphite.readthedocs.io/en/latest/dashboard.html). Graphite also offers a [Metric API](https://graphite.readthedocs.io/en/latest/metrics_api.html), [Whisper database](https://graphite.readthedocs.io/en/latest/whisper.html), and a library of [functions](https://graphite.readthedocs.io/en/latest/functions.html) that can be used to manipulate the data upon rendering. * **What is Hosted Graphite?** We host Graphite so you don’t have to! We handle and manage your metric collection, storage, alerting, and visualization services so you can focus on more important things. Hosted Graphite provides ways for you to send us data from your infrastructure, and you can plot that data on custom dashboards that look awesome! We give you the tools that you need to send large amounts of data and provide interesting ways for you to get accurate information back out, analyze it, alert on it, and share it. We also provide integrations with other popular services, have a wide range of account/team management options, and offer great support. * **How is Hosted Graphite different from Vanilla Graphite?** There are a lot of differences between HG Graphite and vanilla Graphite, but there are 2 which will affect users the most: > * Data aggregation is automatic. In vanilla Graphite only one datapoint is saved per resolution interval, so if you want to send data more frequently you need to use some kind of pre-aggregation service, like StatsD or a Carbon Aggregator. Hosted Graphite creates Data Views for all metrics we receive, which consolidate all your datapoints like an aggregator would, but also gives you access to 9+ different versions of your aggregated metrics. Visit the *Data Views* page for a full list of all the data views you can access. > * No ‘Last Write Wins’. As mentioned above, in vanilla Graphite only one datapoint per timestamp can be stored. If a new datapoint is sent to the old timestamp, it overwrites the old value. With Hosted Graphite the new datapoint is incorporated into the data views instead. * **But could I just install the open-source version of Graphite?** You could! Let’s say you take some time to figure out the docs and install it somewhere, then do some tests to get data in and make sure it’s giving you the right data back out. Let’s say it only takes a few hours of pretty enjoyable tinkering, because we are optimists! That’s the easy, fun part. Here’s the unfun part: *at some point in the future, you’re going to have to drop some other important task because your monitoring setup stops working the way you want.* > * *Maybe* your company grows, and you need to send way more data than you originally thought and the server you picked can’t handle it. Now you need to reinstall somewhere, or multiple somewheres. > * *Maybe* someone on your team isn’t getting the right answers out, and you need to help them debug it or create new dashboards. > * *Maybe* your company releases the new version of your software, and even with your beefy new setup it swamps your monitoring. Now nobody know what’s happening in production and people are looking at you like you kicked their puppy. > * *Maybe* now you can’t remember exactly what you were working on originally? > * *Maybe* your boss is wondering why you’re spending all this time maintaining the graphs instead of writing that useful new feature the customers are waiting for? > * *Maybe* you didn’t want to be the Graphite monkey, and all these problems need to just go away... **That’s why we’re doing this - let us handle it all for you.** * **What is a Metric?** A metric is a unique time-series collection of name-value data. This data is sent to us as it is sampled (or aggregated with other sampled data) and it becomes available in your account for later graphing. In other words, this means that you send us a numeric value at a certain time, and we make sure that the number gets collected properly with all the other numbers that many other servers could be sent concurrently. **Note:** we charge on the total number of unique metrics in an account, not the amount of data points the metric receives. ### [Business](#business) * **Do you provide an SLA, Security Details, or keep uptime statistics?** Yes. We aim for (and achieve) three nines of service availability. We’re happy to give you more detailed stats upon request, these can be located [here](https://www.metricfire.com/security/) or you can [contact us](mailto:support%40hostedgraphite.com) for more details. * **What are your support options for businesses?** We pride ourselves in offering great support for all plans, including free trials! We encourage anyone using Hosted Graphite to [book a support call](https://calendly.com/metricfire-sales/customer-support) with us anytime. You will have direct access to experienced engineers that are committed to giving you great service, check out some our great [reviews](https://www.g2.com/products/hosted-graphite/reviews) on G2. * **Can you migrate our existing dashboards?** Absolutely, it's easy to import json objects into our Hosted Dashboards! Please [contact us](mailto:support%40hostedgraphite.com) or reach out to our live chat and we’ll talk you through the process. * **We have a lot of metrics, can we get them all hosted?** We have customers sending millions of metrics and we ingest billions of data points per second. If you think you’re going to need a lot of metrics, we can onboard you at a pace that suits your schedule. We offer a [14-day free trial](https://www.hostedgraphite.com/accounts/signup/) and if you need additional metrics or data points per second, just let us know and we will be happy to accommodate your load testing. If you have sales questions or technical queries, you can [book a demo](https://calendly.com/metricfire-sales/hostedgraphite-demo) with us, and we will gladly address all of your questions. * **Who owns my data? Can I get it back or get data backups?** The data is yours and our Enterprise customers can even benefit from regular data backups to Amazon S3. Other accounts can get one-time whisper exports of their data sets, upon request. * **Is my data backed up?** Definitely! We store 3 copies of your data, for each resolution (30s, 300s, 3600s). * **What happens if I am on an annual subscription and need to upgrade (or downgrade) mid-term?** Annual subscriptions are purchased in advance for the following 12-month period but if your usage increases we might need to change your plan. Plans are subject to upgrade mid-subscription in the event that metric usage exceeds the paid-for metric threshold. Any such upgrade will be advised at least 14 days in advance and you will have the option at any time to reduce your metric usage to fit within the plan you are currently on. See our [pricing page](https://www.metricfire.com/pricing/) for more plan details. ### [Technical](#technical) * **Does Hosted Graphite use Whisper files as its back-end?** No, Hosted Graphite has replaced Graphite’s traditional file-based Whisper format with a compatible datastore running on [Riak](https://en.wikipedia.org/wiki/Riak). This customized database allows us to easily scale up to very large amounts of metrics, while also ensuring that there are multiple (3) redundant copies of the data. Our Riak database has been optimized to handle a high cardinality and is able to read, write, and traverse data quickly and efficiently. * **Does sending metrics affect the speed of my application?** Sending metrics to Hosted Graphite will have a negligible impact on performance if implemented correctly. If you need to perform metric processing or aggregation before sending, consider doing this asynchronously to minimize performance impact e.g. in another thread. * **When should I consider using TCP vs UDP?** UDP is a simple way to get started as you can ‘fire and forget’. However, UDP is a less reliable protocol, if you want to be sure that the message was received a TCP Connection is preferred. If you choose TCP, we recommend that you make the connection asynchronously (e.g. in another thread) to prevent network conditions from affecting the performance of your system. The majority of our incoming traffic is TCP. * **Can I send metrics to Hosted Graphite from within a private network?**\ Yes! You can use [Carbon-C-Relay](/add-ons-and-integrations-guide/carbon-c-relay) as a proxy to send metrics from a private network to Hosted Graphite. It acts as an internal relay, collecting metrics locally (localhost:2003) and securely forwarding them over the internet. * **What IP addresses should I whitelist on my firewall to reach the Hosted Graphite carbon endpoint?** The carbon endpoint currently resolves to the following IP addresses:
`195.201.153.19` `195.201.153.20` `144.76.143.215` `176.9.21.64` `136.243.95.166` `136.243.95.165` `94.130.127.162` `46.4.207.207` \ This list is correct as of October 2025 but is subject to change. To retrieve an up-to-date list of IP addresses, run `nslookup carbon.hostedgraphite.com` in a command prompt or terminal session of your choice. * **How long do you keep data?** Keywords: resolution, retention, aggregation, rollup We aggregate and store data at the following resolutions: * 5 second data is stored for 1 day (please [contact us](mailto:support%40hostedgraphite.com) to request this resolution) * 30 second data is stored for 3 days * 300 second (5min) data is stored for 6 months * 3600 second (1hr) data is stored for 2 years Depending on your query, Graphite will serve data from the most appropriate resolution. For example, if you ask for the most recent 1 hour, that data will come from the 30-second resolution. If you ask for 24 hours, it’ll come from the 300-second resolution, because serving 30-second data for that query would produce too much data to display on a graph. However, if you ask for 1 hour of data from 24 hours ago, that’ll go back to the 30-second resolution because of how ‘zoomed in’ you are. In this way, we allow for a fast, meaningful overview of your data as well as a detailed investigation when it becomes necessary.
When querying data in your Primary Dashboards, we map the following time spans to the following resolutions: * 0h-1h query displays 5s resolution data (available upon request) * 1h-10h query displays 30s resolution data * 10h-5d query displays 300s resolution data * 5d-2y query displays 3600s resolution data \ In some cases, customers needing more accurate counts of occurrences can benefit from using pre-aggregation services, such as [StatsD](https://github.com/statsd/statsd#installation-and-configuration) or [carbon-c-relay](https://github.com/grobian/carbon-c-relay). Contact us at if you need some advice on how to configure a pre-aggregation service. * **What version of Graphite do you offer?** We offer Graphite version 1.1.1. In contrast to vanilla Graphite, our backend doesn’t just keep averages - we keep the min, max, avg, sum, and observations of the data sent allowing much more detailed insight into your metrics. We also provide arbitrary percentile data for even more interesting views of your metrics. Read more about our Data Views [here](https://docs.hostedgraphite.com/additional-features/data-views), which in some cases can be a suitable replacement for StatsD. * **Do you support Graphite Tags?** Yes, we support graphite tagging, have our own [Tag API,](/api-guides/tag-api) and have a Tag Search UI feature so you can easily locate and manage your tagged metrics. Refer to our [Graphite Tag Support ](/sending-metrics/graphite-tag-support)doc for more details. ### [Account Metrics and Limiting](#account-metrics-and-limiting) * **What happens if I go over the limits of my plan?** If your metric limits go over the allowed number of metrics for your plan, we’ll send you a friendly reminder to ask you to either delete metrics or upgrade to the next plan. If your account is still over the allotted number of metrics for your plan after 7 days we’ll drop you an email and your plan will be automatically upgraded. Any charges will be prorated. * **What limits are on my account?** See the [Account Diagnostics](/account-management/account-diagnostics) documentation for information on what limits are in place, and why. This information can also be seen in the HG Traffic dashboard, which is automatically created in every account. * **How do you handle bursty traffic?** We know bursty traffic is a normal part of everyone’s operations, and we don’t want to penalize people for it. At the same time, we need to make sure we’re billing everyone fairly. If your account is storing more than 110% of the metrics allowed under your plan for 14 consecutive days, we’ll send you an email and display an in-account message that we will upgrade your account soon. If you don’t reply and if your metric usage remains above 110% of the limit, your account will be automatically upgraded to the smallest plan that your usage fits into. If that’s not suitable for you, you will need to reduce the number of metrics you’re storing and if you don’t take action or contact us within the 14-day notice period, your subscription will automatically upgrade. You can find more information about metric management in our docs *here*. It’s worth noting that we won’t automatically upgrade any customer during the 14-day notice period and that we’re willing to bend the rules if you just need a bit more time to reduce your usage - we know technical changes can take some time - so please talk to us and we’ll be happy to work with you. Please [contact us](mailto:support%40hostedgraphite.com) if you want to chat about this, we’d much rather be your monitoring partner than draconian enforcers of rules! * **I spin up a lot of VMs, is this going to affect my metric limits?** We can set up a metric expiry policy for metrics you know aren’t going to last long. You can set expiration of 7, 14, 21, or 30 days which will automatically delete a metric if it hasn’t been updated within the specified timeframe. See the documentation on [*metric expiry*](https://docs.hostedgraphite.com/account-management/account-settings#metric-expiry) for more information. * **Is there anything I should know when I’m sending data via Java?** Yes. Java can cache DNS entries indefinitely, which causes problems when we need to replace or add servers to our system. Use our [Java caching guide](/language-guide/java#java-dns-behaviour) to ensure your DNS behaves like a good neighbor. * **Are queries to Graphite cached?** Yes, Hosted Graphite uses Graphite’s default cache settings of 60s for metrics, data, and rendered images. If you really need uncached results, add ‘\&noCache=true’ to your query string. * **Is it possible to control the data retention policy for my account?** Not if your account is in our shared environment. Graphite’s usual backend rolls up metrics and then produces an average at a different resolution, we keep unaltered data at the resolution required. * **What client languages do you support?** Graphite is designed to be language agnostic - as long as you have some way to send a data packet over a network via TCP or UDP, integration is very easy. We provide sample codes for the following: > * [*Metric Libraries*](/language-guide/metric-libraries) > * [*.NET*](/language-guide/.net) > * [*Go*](/language-guide/go) > * [*Java*](/language-guide/java) > * [*Javascript*](/language-guide/javascript) > * [*Node.js*](/language-guide/node.js) > * [*PHP*](/language-guide/php) > * [*Python*](/language-guide/python-2.x) > * [*Ruby*](/language-guide/ruby) > * [*Shell*](/language-guide/shell) ### [Customization](#customization) * **Can I host dashboards locally using Hosted Graphite as a data source?**\ Absolutely! Follow our [*guide to alternative Graphite dashboards*](/dashboard-and-graphs/other-dashboard-options) to use an [*access key*](/account-management/access-keys) that will allow you to connect to self-hosted dashboards, like Tasseo, Tattle, Graphiti, Cabot, or Seyren. * **Can I use Hosted Graphite metrics to send alerts to other alerting services?**\ If you create an [*access key*](/account-management/access-keys), you can hook Hosted Graphite up to [Seyren](https://github.com/scobal/seyren) or [Cabot](https://github.com/arachnys/cabot). Both are excellent alerting frameworks that our customers enjoy. There are more details in our [*Alerting Guide*](/alerting-guide). * **Can I get metrics from my Heroku Applications?** \ Yes! We manage a Heroku Add-On and you can get dyno and postgres metrics with the HG Add-On, or with a dedicated Hosted Graphite account. A [*dedicated Hosted Graphite account*](/add-ons-and-integrations-guide/heroku) is generally a better value than the [Heroku Add-on](https://elements.heroku.com/addons/hostedgraphite), as we offer higher data retention and priority support for all plan levels. See our [*Integration Guide*](/add-ons-and-integrations-guide) for a full list of services that we integrate with including Heroku, AWS, GitHub, CircleCI, Pingdom, New Relic, Papertrail, Sentry, and more! * **Is it possible to share my account with colleagues?**\ For sure, refer to our [*Team Access*](/account-management/team-access) docs for full details. You can invite other team members to use your account, and view your graphs. You also have the option of giving them admin, read/write, or read-only access to your account. * **Do you provide hosted StatsD?**\ Yes, Hosted StatsD can be configured with a few clicks! We also provide [*data views*](/additional-features/data-views) which provide much of the functionality of StatsD. See our [*Add-Ons and Integrations Guide*](/add-ons-and-integrations-guide) for more information. * **How can I collect server metrics such as CPU usage, Load, Network, and Disk IO?**\ Here’s a useful guide on how to use one of our supported [*agents*](/agents-guide) to easily collect system metrics and send them to Hosted Graphite. * **Is it possible to use an agent to collect more than just server metrics?**\ Yes, agents such as collectd and telegraf have plugins to monitor many things outside of just server statistics. The Telegraf agent has a wide variety of [input plugins](https://docs.influxdata.com/telegraf/v1.10/plugins/inputs/) that allow you to collect metrics from popular services and technologies like: - Databases, Networks, DNS, MQTT, GCP, System Applications, Apache, Kubernetes, Jenkins, Jolokia, Elasticsearch, Logstash, Kibana, Mailchimp, Nginx, Docker, GitHub, Salesforce, and many more! Find installation instructions and more details about input plugins in our [*Telegraf*](/agents-guide/telegraf) documentation. * **Can I migrate existing whisper file data to Hosted Graphite?** Yes, and we can help do this for you. * **How do I know how many metrics I have / what plan would my existing install fit in?** In your Graphite directory, run the following to get the total number of metrics you have: ``` find . -type f -print | grep ".wsp" -c ``` This will get the amount used in the last 30 days, which is useful as a gauge of what metrics you are actually using: ``` find . -mtime -30 -type f -print | grep ".wsp" -c ``` If you are using [*StatsD*](/add-ons-and-integrations-guide/statsd-add-on), you can divide this number of metrics by the number of additional aggregations that StatsD sends such as mean, upper, lower, etc. StatsD typically adds 8 extra measurements per metric but may be different for your installation. Unless you’re really attached to StatsD, you don’t need these - we already collect these different [*data views*](/additional-features/data-views) for each metric. Please reference our [pricing page](https://www.metricfire.com/pricing/) for more plan and cost details. * **How can I get my data in?** You can use our [language guide](https://docs.hostedgraphite.com/language-guide) to send your original data values with the timestamp of the original measurement. Send via TCP, and limit your request to < 1000 data points per second - otherwise you risk being rate-limited. Alternatively, if you move to one of our [paid plans](https://www.metricfire.com/pricing/), just drop your whisper files into AWS and send us a mail. We’ll do the migration for you! ### [Troubleshooting](#troubleshooting) * **Why don’t I see any metrics in the dashboard composer’s ‘Graphite’ folder?**\ Ensure that your metrics are being sent using [*the correct format*](/#metric-data-format) and using the correct [*API Key*](/#api-key). * **Why did I get an empty graph after adding a metric to my dashboard?**\ If the metric appears in the folder, then it’s likely one (or both) of two things have happened. > * No data for the currently selected time window - [*choose a more suitable time range*](/dashboard-and-graphs/graphite-dashboard-guide#choosing-a-time-range). If you’re trying to test that a new metric was received, a 30 minute time range is a good choice. > * The data is too sparse for the selected time range - try choosing the **Connected Line** graph style or enabling Graphite’s [Keep Last Value](https://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.keepLastValue) function. See the [*Graph Option Menu*](/dashboard-and-graphs/graphite-graph-menu-reference) for more info on this. * **Why can’t I see any graphs on my dashboard?**\ After adding graphs to a dashboard, you must explicitly save that dashboard, giving it a name. Then the dashboard can be recalled either via a menu or via a book-markable URL. See the [*Dashboard Guide*](/dashboard-and-graphs/graphite-dashboard-guide#saving-and-loading-dashboards) for more information. * **How long does it take for metrics to appear in the system?**\ Generally, it will take about 30 seconds for a new metric to travel through our layers of ingestion and become available in the metric tree. During slower times this might take longer, but we try to keep it to around 30 seconds.\ \ The speed of new data points appearing for an existing metric will also depend on the time scale of the graph you’re viewing. For a short graph timescale (e.g. within the last 24 hours) the metric will become available within a few seconds - 15 seconds at the minimum. If you’re looking at a timespan greater than 24 hours, it will take up to 10 minutes simply because each data point on a graph of that scale is represented by a larger amount of time. One data point received in that time interval is not a valid representation of the actual data stream, so data is only displayed when there is a valid amount collected. This is related to our aggregation resolutions that can be referenced above, in the [technical](https://docs.hostedgraphite.com/faq#technical) section. * **What are \_hg\_meta metrics?**\ These metrics are used to monitor account traffic and come standard with every account. Additionally, a dashboard called ‘HG Traffic Dashboard’ is automatically generated to visualize these metrics. They do not count against your metric total and you are not charged for them. ### [Support](#support) * **Where can I get further help?**\ If you have any questions, feel free to mail [support@hostedgraphite.com](mailto:support%40hostedgraphite.com), and one of our team members will get back to you quickly. You can also start a conversation with our team by logging into your HG account and clicking on the chat bubble at the bottom-right of your screen. * Do you have questions that were not answered in our documentation? Feel free to book a demo with us [here](https://calendly.com/metricfire-sales/hostedgraphite-demo), and we will be happy to have a conversation with you and address any questions about our product and services. If you don't already have a Hosted Graphite account, you can start a 14-day free trial [here](https://www.hostedgraphite.com/accounts/signup/). You can also [schedule a product demo here ](https://calendly.com/metricfire-sales/hostedgraphite-demo)to speak with a knowledgeable member of our team who can provide support and advice on getting up and running quickly. # Changelog Monthly updates to Hosted Graphite by MetricFire - customer facing features and services #### July 2026 * Building out a new MetricFire [MCP server integration](https://www.youtube.com/watch?v=7dIbd2rxARs\&t=9s) and UI for token generation/permission handling. #### June 2026 * Deployed an integration for [DigitalOcean](/add-ons-and-integrations-guide/digital-ocean), which pulls resource metrics into Hosted Graphite (Apps, Droplets, Databases, Load Balancers, Autoscale Pools). #### May 2026 * Updated our Graphite Alerts UI to support [Composite Alerts](https://docs.hostedgraphite.com/alerting-guide/alerting-overview#composite-alerts), allowing you to configure alerts with multiple metrics using AND/OR conditional logic. * Updated the [alert notification payload](https://docs.hostedgraphite.com/alerting-guide/alerting-overview#notification-json-model) to include composite conditions met, logical expression used, and a triggered metrics list. #### April 2026 * Upgraded our Heroku integration to pull [Heroku Connect](https://www.metricfire.com/herokuconnect/) metrics, and added a section for these on our Heroku auto-dashboard. #### March 2026 * Redesigned and updated user interfaces for our Heroku, AWS, Azure, and GCP integrations. #### February 2026 * Deployed a [Team Members API](https://docs.hostedgraphite.com/api-guides/team-members-api), that lists all team users and their respective permissions * New and simplified account signup flow using email verification and [reCaptcha v3](https://developers.google.com/recaptcha/docs/v3) validation #### January 2026 * Received our annual [SOC2 compliance](https://www.metricfire.com/security/) certification, upholding best practices for modern security standards. * Upgraded Kubernetes to a newer version in our Hosted Grafana environment, for optimized node performance. * Added a proper [DELETE route](https://docs.hostedgraphite.com/api-guides/metrics-api#deleting-metrics) for our Metrics API, better compatibility for HTTP/2 requests. #### November 2025 * Added a [Process Aggregate](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/heroku#process-aggregates) feature to the Heroku integration, to combine individual process metrics into single data sets. * Updated the default Heroku dashboard to display Heroku Process metrics (scheduler, release, run, worker). #### October 2025 * Added a [Host Aggregate](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/heroku#host-aggregates) feature to our Heroku integration, that will combine values from all host (subdomain) metrics into a single data set. * Updated the internal version of Graphite-Web to include more recent Graphite functions. #### September 2025 * Researched and tested a method for [GPU performance monitoring](https://www.metricfire.com/blog/why-gpu-monitoring-matters-tracking-utilization-power-and-errors-with-dcgm/), using the DCGM Exporter tool. #### August 2025 * Increased/optimized disk storage capacity on all K8 nodes in our Hosted Grafana environment, for faster user performance. #### July 2025 * Deployed a beta version of the new logging integration to production (Hosted Loki), **contact us** for more details. #### June 2025 * Upgraded Kubernetes to a newer version in our Hosted Grafana environment, for optimized node performance. #### May 2025 * Building out a new logging integration using open-source tools. See the MF Logging page for more details, and to register your interest for this new feature: #### April 2025 * Released [V2](https://github.com/hostedgraphite/hg-cli/releases/tag/v0.2.0) of the HG-CLI tool, with a 'Quick Install' method for the OpenTelemetry Collector Agent * **curl -s "" | sudo sh** * Published content around converting syslog messages to Graphite metrics 2 ways, by using Telegraf with [Loki](https://www.metricfire.com/blog/from-logs-to-metrics-building-an-open-source-logs-to-graphite-pipeline/) and [Grok](https://www.metricfire.com/blog/from-logs-to-metrics-part-2-building-an-open-source-logs-to-graphite-pipeline/) #### March 2025 * Released the first stable version of our [HG-CLI Tool](https://github.com/hostedgraphite/hg-cli), for easy agent installation on any OS: * **curl -s "" | sudo sh** * Published content around using OpenTelemetry Receivers to [collect Java application metrics](https://www.metricfire.com/blog/easiest-way-to-monitor-your-java-application-using-opentelemetry/) #### February 2025 * Released a Graphite method for using the [OpenTelemetry Collector Contrib as an Agent](/agents-guide/opentelemetry) * Published content around using OpenTelemetry Receivers to collect metrics from [NGINX](https://www.metricfire.com/blog/easiest-way-to-monitor-nginx-performance-with-opentelemetry/), [Aerospike](https://www.metricfire.com/blog/how-to-monitor-aerospike-with-opentelemetry-and-metricfire/), [Snowflake](https://www.metricfire.com/blog/how-to-monitor-snowflake-with-opentelemetry/), and [OracleDB EX](https://www.metricfire.com/blog/monitor-oracledb-ex-with-opentelemetry-and-metricfire/) #### January 2025 * Pushed an update that enables Annotations to accept non-Latin characters * Updated [HG Statuspage](https://status.hostedgraphite.com/), now displaying more AWS ingestion metrics #### December 2024 * Deployed new Metrics Search UI that includes 'folder' structure and quick view graphing features * Deployed 'guest' CLI Demo that can be used without an active HG account: **bash -c "$(curl -s '')"** * Updated methods for [integrating Apache Spark with Graphite](https://www.metricfire.com/blog/step-by-step-guide-to-monitoring-apache-spark-with-metricfire/) #### November 2024 * Optimized onboarding flow for HG accounts created through our [Heroku Add-On](https://elements.heroku.com/addons/hostedgraphite) * Updated methods for [converting Prometheus data ⇒ Graphite](https://www.metricfire.com/blog/simple-guide-to-converting-prometheus-metrics-to-graphite-using-telegraf/) #### October 2024 * Continuing to decommission the old Grafana v7 environment for all accounts - please reach out to support if you would like a force-redirect rule from v7 to v9 * Added the AWS/NetworkELB service to our Cloudwatch Add-On #### September 2024 * Deployed a new [Heroku Router Aggregates](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/heroku#router-aggregates) feature - to combine values of high-cardinality Heroku Path metrics * Upgraded the AWS/SNS integration to include more metric dimensions #### August 2024 * Added a Kubernetes integration as a [telegraf-daemonset](https://github.com/metricfire/telegraf-daemonset/) which can be easily deployed into your k8 clusters. This will collect/forward metrics that can be visualized on a Kubernetes Overview dashboard located in our Dashboard Library * All accounts now have active Grafana v9.5 instances, and you can reach out to our support to request a different version #### July 2024 * Added a CLI Demo onboarding tool - for sending sample data that van be viewed in the Welcome to HG dashboard in our Dashboard Library: * **bash -c "$(curl -s )"** * Continued Grafana v9 deployments and dashboard migrations - reach out to request this upgrade #### June 2024 * Added a [Tag Expiry](https://docs.hostedgraphite.com/account-management/account-settings#tag-expiry) feature, for managing stale tagged metrics * Continued Grafana v9 deployments and dashboard migrations - reach out to request this upgrade * Optimized the Quick View metric/alert graphs for cleaner and faster data rendering #### May 2024 * Rolled out our NEW beta Grafana environment hosted on kubernetes (default version 9.5.13) * please reach out directly to request this early * Added a Welcome Dashboard to the [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library), for a smoother onboarding experience #### April 2024 * Deployed a HG home page redesign for the top and side nav bars, giving a more modern look and cleaner UI experience * Continued development for hosted stock Grafana in Kubernetes * Updated methods for [SNMP device performance monitoring](https://www.metricfire.com/blog/step-by-step-guide-to-monitoring-your-snmp-devices-with-telegraf/) - using the Telegraf agent #### March 2024 * Began development work for hosted stock Grafana in Kubernetes * Updated methods for [ELK stack performance monitoring](https://www.metricfire.com/blog/monitor-your-elk-stack-with-telegraf-and-metricfire/) - using the Telegraf agent * Tagged Metric backend service cleanup and optimisation #### February 2024 * New Heroku [Router Path Metrics](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/heroku#router-path-metrics) option for higher visibility into connections/methods/statuses per path in a Heroku app * Deployed new GitLab and Cloudbees webhook [integrations](https://docs.hostedgraphite.com/add-ons-and-integrations-guide) * Integrated HG landing page to use chartJS for faster graph loading on home page and Account Diagnostics page * API key on HG landing page now hidden by default, for increased security #### January 2024 * Released the [Heroku ](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/heroku#heroku-add-on-metrics)[Kafka](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/heroku#heroku-add-on-metrics) integration which forwards Heroku log drain metrics from your Heroku Kafka Add-On * [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library) updated with the Heroku Kafka dashboard * Backend services cleanup, consolidation, and optimisation #### December 2023 * Released a [GCP Add-On](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/gcp-metrics) integration. This feature allows the forwarding of metrics from specified GCP services. * Released the [Heroku Redis](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/heroku#heroku-add-on-metrics) integration which forwards metrics from your Heroku Redis Add-On * [Dashboard Library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library) updated with GCP Overview dashboard and Heroku Redis dashboard #### October 2023 * Released the [Microsoft Azure Add-On](https://docs.hostedgraphite.com/add-ons-and-integrations-guide/azure-monitor-metrics), which syncs the metrics from specified Azure resources into your account. * [Dashboard library](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library) updated with AWS/Azure Overview dashboards * [Busy Metric Ratelimited](https://docs.hostedgraphite.com/account-management/account-diagnostics#busy-metrics-being-ratelimited-graph) feature released an viewable in Account Diagnostics * Added AWS ECS Container Insights service to our Cloudwatch Add-On #### September 2023 * Deployed [Dashboard Library v2](https://docs.hostedgraphite.com/dashboard-and-graphs/dashboard-library) and optimized current dashboard list * Added new pre-made dashboards for Telegraf and Sitespeed * [Invalid Metric Tracking](https://docs.hostedgraphite.com/account-management/account-diagnostics#invalid-metrics-graph) feature released and viewable in Account Diagnostics UI #### August 2023 * [MetricFire Homepage](https://www.metricfire.com/) localised into Japanese, Korean and Portuguese * Released the Heroku Postgres metrics integration and pre-made dashboard * Updated our Sitespeed integration for use with annotations * Hosted Graphite Agent updated to python3 * Added the AWS Cloudwatch Logs service to our Cloudwatch Add-On #### July 2023 * Released the Microsoft Teams Alerting Notification Channel * Greater integration with [Sitespeed.io](https://www.sitespeed.io/) Site Performance Insights * Improvements made around annotation issues * A+ [security header](https://securityheaders.com/) score for hostedgraphite.com and metricfire.com domains * Sentry Reconfiguration for Alerting #### June 2023 * Heroku Add-On Maintenance and migration to Heroku API V3 * Sentry reconfiguration for improved internal alerting and response times * Updates to site stability and indexing **May 2023** * Bug fixes resolving alert resolution and metric expiry feature issues * Backend services cleanup and optimisation * Improvements to Graphite Web / Query Functions * Deployed updates providing greater ability for users to adjust their current plan option self-service #### April 2023 * Added AWS MemoryDB Service to our Cloudwatch Add-On * Bug Fixes for Heroku Add-On services * Metric Expiry Updates and Fixes * ‘Limited Access Group’ feature improvements * Hosted Graphite Ruby Gem updated to support tagged StatsD metrics **March 2023** * Completed a full infrastructure overhaul: * upgraded all backend services and dependencies to newer versions of Python and Ubuntu * replaced all internal servers with newer and more efficient models (NVME in most layers for faster read/write speeds) * Released new documentation domain and format - hosted by gitBook