Graphite & Grafana Performance Tips

If you’re using Graphite to monitor your systems, your dashboards are probably only accessible to a few people in your organization and page load performance likely isn’t a top priority. However, if you’re running a publicly accessible dashboard then chart rendering time can become a pain point.

This is an issue that I’ve been grappling with recently and unfortunately there’s not a lot of material online regarding tuning Graphite for maximum chart rendering performance. I still have some digging that I want to do into what’s slowing down the rendering of certain charts, but here are a few things I’ve learned thus far:

1) It’s highly recommended that you run Graphite on a server with solid state disks.

2) If you’re running Graphite on a machine with SSDs and still experience I/O bottlenecks, the common solution is to switch to a federated storage configuration.

3) You should install memcached and configure graphite to use it in /opt/graphite/webapp/graphite/local_settings.py:

MEMCACHE_HOSTS = [‘127.0.0.1:11211']
DEFAULT_CACHE_DURATION = 600 # Cache images and data for 10 minutes

4) Make sure you’re running a version of Graphite that’s newer than 0.9.12 so that it supports the “maxDataPoints” chart option. This will speed up rendering of charts over long time frames; you can read explanations here and here.

5) You can speed up your Grafana site a bit by enabling mod_expires in Apache and adding this to the apache2.conf:

<ifmodule mod_expires.c>
ExpiresActive on
ExpiresByType image/jpg “access plus 1 month”
ExpiresByType image/jpeg “access plus 1 month”
ExpiresByType image/gif “access plus 1 month”
ExpiresByType image/png “access plus 1 month”
ExpiresByType text/css “access plus 1 month”
ExpiresByType application/pdf “access plus 1 month”
ExpiresByType text/x-javascript “access plus 1 month”
ExpiresByType application/x-shockwave-flash “access plus 1 month” ExpiresByType image/x-icon “access plus 1 year”
ExpiresDefault “access plus 1 week”
</ifmodule>

6) Enable and configure mod_deflate in Apache to reduce the amount of data that needs to be sent from the server to the client for each request.

7) Consider using a Content Delivery Network to speed up your page loads. Cloudflare offers a free CDN service that only takes a few minutes to set up.

8) You can install the Firebug and YSlow add-ons for Firefox to get a better idea of what the bottlenecks are from the browser’s perspective.

9) One option that I haven’t explored yet would be swapping out Whisper with a different database such as InfluxDB or OpenTSB. There are a couple posts about it here and here.

In summary, optimizing Graphite chart rendering performance is something that I’m still trying to better understand. If you have any insights to offer in this area, please let me know!