feat(metrics): Add documentation and contrib scripts

Documentation/debugging.md

@@ -0,0 +1,69 @@
+# Debugging etcd
+
+Diagnosing issues in a distributed application is hard.
+etcd will help as much as it can - just enable these debug features using the CLI flag `-trace=*` or the config option `trace=*`.
+
+## Logging
+
+Log verbosity can be increased to the max using either the `-vvv` CLI flag or the `very_very_verbose=true` config option.
+
+The only supported logging mode is to stdout.
+
+## Metrics
+
+etcd itself can generate a set of metrics.
+These metrics represent many different internal data points that can be helpful when debugging etcd servers.
+
+#### Metrics reference
+
+Each individual metric name is prefixed with `etcd.<NAME>`, where \<NAME\> is the configured name of the etcd server.
+
+* `timer.appendentries.handle`: amount of time a peer takes to process an AppendEntriesRequest from the POV of the peer itself
+* `timer.peer.<PEER>.heartbeat`: amount of time a peer heartbeat operation takes from the POV of the leader that initiated that operation for peer \<PEER\>
+* `timer.command.<COMMAND>`: amount of time a given command took to be processed through the local server's raft state machine. This does not include time waiting on locks.
+
+#### Fetching metrics over HTTP
+
+Once tracing has been enabled on a given etcd server, all metric data is available at the server's `/debug/metrics` HTTP endpoint (i.e. `http://127.0.0.1:4001/debug/metrics`).
+Executing a GET HTTP command against the metrics endpoint will yield the current state of all metrics in the etcd server.
+
+#### Sending metrics to Graphite
+
+etcd supports [Graphite's Carbon plaintext protocol](https://graphite.readthedocs.org/en/latest/feeding-carbon.html#the-plaintext-protocol) - a TCP wire protocol designed for shipping metric data to an aggregator.
+To send metrics to a Graphite endpoint using this protocol, use of the `-graphite-host` CLI flag or the `graphite_host` config option (i.e. `graphite_host=172.17.0.19:2003`).
+
+See an [example graphite deploy script](https://github.com/coreos/etcd/contrib/graphite).
+
+#### Generating additional metrics with Collectd
+
+[Collectd](http://collectd.org/documentation.shtml) gathers metrics from the host running etcd.
+While these aren't metrics generated by etcd itself, it can be invaluable to compare etcd's view of the world to that of a separate process running next to etcd.
+
+See an [example collectd deploy script](https://github.com/coreos/etcd/contrib/collectd).
+
+## Profiling
+
+etcd exposes profiling information from the Go pprof package over HTTP.
+The basic browseable interface is served by etcd at the `/debug/pprof` HTTP endpoint (i.e. `http://127.0.0.1:4001/debug/pprof`).
+For more information on using profiling tools, see http://blog.golang.org/profiling-go-programs.
+
+**NOTE**: In the following examples you need to ensure that the `./bin/etcd` is identical to the `./bin/etcd` that you are targetting (same git hash, arch, platform, etc).
+
+#### Heap memory profile
+
+```
+go tool pprof ./bin/etcd http://127.0.0.1:4001/debug/pprof/heap
+```
+
+#### CPU profile
+
+```
+go tool pprof ./bin/etcd http://127.0.0.1:4001/debug/pprof/profile
+```
+
+#### Blocked goroutine profile
+
+```
+go tool pprof ./bin/etcd http://127.0.0.1:4001/debug/pprof/block
+```
+

Documentation/profiling.md

@@ -1,28 +0,0 @@
-## Profiling
-
-etcd exposes profiling information from the Go pprof package over HTTP.
-The basic browseable interface can be found at `http://127.0.0.1:4001/debug/pprof`.
-
-**NOTE**: In the following examples you need to ensure that the `./bin/etcd` is
-identical to the `./bin/etcd` that you are targetting (same git hash, arch,
-platform, etc).
-
-### Heap memory profile
-
-```
-go tool pprof ./bin/etcd http://127.0.0.1:4001/debug/pprof/heap
-```
-
-### CPU profile
-
-```
-go tool pprof ./bin/etcd http://127.0.0.1:4001/debug/pprof/profile
-```
-
-### Blocked goroutine profile
-
-```
-go tool pprof ./bin/etcd http://127.0.0.1:4001/debug/pprof/block
-```
-
-For more information on using the tools see http://blog.golang.org/profiling-go-programs

contrib/collectd/Dockerfile

@@ -0,0 +1,9 @@
+FROM stackbrew/ubuntu:raring
+
+RUN apt-get update && apt-get install -y collectd
+RUN adduser --system --group --no-create-home collectd
+ADD collectd.conf /etc/collectd/collectd.conf.tmpl
+ADD collectd-wrapper /bin/collectd-wrapper
+RUN chown -R collectd:collectd /etc/collectd
+
+CMD ["collectd-wrapper"]

contrib/collectd/README

@@ -0,0 +1,20 @@
+We're going to use Docker to build a chroot env that can be run with systemd-nspawn since I cannot figure out how to run
+a container using docker in the global network namespace.
+
+1. Build the collectd image using docker
+docker build -t collectd .
+
+2. Run the container (since we have to run it to export it...)
+COLLECTD_CONTAINER=`docker run -name collectd-tmp -d collectd`
+
+3. Export then kill the container
+docker export collectd-tmp > /tmp/collectd.tar
+
+4. Kill the temporary container
+docker kill $COLLECTD_CONTAINER
+
+5. Unpack the tar archive
+mkdir -p /tmp/collectd && tar -xvf /tmp/collectd.tar -C /tmp/collectd/
+
+6. Run collectd with systemd-nspawn - replace the COLLECTD_* env vars with your parameters!
+sudo systemd-run --unit collectd systemd-nspawn -D /tmp/collectd /bin/bash -c "COLLECTD_GRAPHITE_HOSTNAME=172.31.13.241 COLLECTD_LOCAL_HOSTNAME=node1 /bin/collectd-wrapper"

contrib/collectd/collectd-wrapper

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+cat /etc/collectd/collectd.conf.tmpl > /etc/collectd/collectd.conf
+
+cat << EOF >> /etc/collectd/collectd.conf
+Hostname "${COLLECTD_LOCAL_HOSTNAME}"
+
+<Plugin write_graphite>
+    <Carbon>
+        Host "${COLLECTD_GRAPHITE_HOSTNAME}"
+        Port "2003"
+    </Carbon>
+</Plugin>
+EOF
+
+collectd -C /etc/collectd/collectd.conf -f

contrib/collectd/collectd.conf

@@ -0,0 +1,898 @@
+# Config file for collectd(1).
+#
+# Some plugins need additional configuration and are disabled by default.
+# Please read collectd.conf(5) for details.
+#
+# You should also read /usr/share/doc/collectd-core/README.Debian.plugins
+# before enabling any more plugins.
+
+#Hostname "localhost"
+#FQDNLookup true
+#BaseDir "/var/lib/collectd"
+#PluginDir "/usr/lib/collectd"
+#TypesDB "/usr/share/collectd/types.db" "/etc/collectd/my_types.db"
+#Interval 10
+#Timeout 2
+#ReadThreads 5
+
+LoadPlugin logfile
+#LoadPlugin syslog
+
+<Plugin logfile>
+	LogLevel "info"
+	File STDOUT
+	Timestamp true
+	PrintSeverity false
+</Plugin>
+
+#<Plugin syslog>
+#	LogLevel info
+#</Plugin>
+
+#LoadPlugin amqp
+#LoadPlugin apache
+#LoadPlugin apcups
+#LoadPlugin ascent
+#LoadPlugin battery
+#LoadPlugin bind
+#LoadPlugin conntrack
+#LoadPlugin contextswitch
+LoadPlugin cpu
+#LoadPlugin cpufreq
+#LoadPlugin csv
+#LoadPlugin curl
+#LoadPlugin curl_json
+#LoadPlugin curl_xml
+#LoadPlugin dbi
+LoadPlugin df
+#LoadPlugin disk
+#LoadPlugin dns
+#LoadPlugin email
+#LoadPlugin entropy
+#LoadPlugin ethstat
+#LoadPlugin exec
+#LoadPlugin filecount
+#LoadPlugin fscache
+#LoadPlugin gmond
+#LoadPlugin hddtemp
+#LoadPlugin interface
+#LoadPlugin ipmi
+#LoadPlugin iptables
+#LoadPlugin ipvs
+#LoadPlugin irq
+#LoadPlugin java
+#LoadPlugin libvirt
+#LoadPlugin load
+#LoadPlugin madwifi
+#LoadPlugin mbmon
+#LoadPlugin md
+#LoadPlugin memcachec
+#LoadPlugin memcached
+LoadPlugin memory
+#LoadPlugin multimeter
+#LoadPlugin mysql
+#LoadPlugin netlink
+#LoadPlugin network
+#LoadPlugin nfs
+#LoadPlugin nginx
+#LoadPlugin notify_desktop
+#LoadPlugin notify_email
+#LoadPlugin ntpd
+#LoadPlugin numa
+#LoadPlugin nut
+#LoadPlugin olsrd
+#LoadPlugin openvpn
+#<LoadPlugin perl>
+#	Globals true
+#</LoadPlugin>
+#LoadPlugin pinba
+#LoadPlugin ping
+#LoadPlugin postgresql
+#LoadPlugin powerdns
+#LoadPlugin processes
+#LoadPlugin protocols
+#<LoadPlugin python>
+#	Globals true
+#</LoadPlugin>
+#LoadPlugin rrdcached
+#LoadPlugin rrdtool
+#LoadPlugin sensors
+#LoadPlugin serial
+#LoadPlugin snmp
+#LoadPlugin swap
+#LoadPlugin table
+#LoadPlugin tail
+LoadPlugin tcpconns
+#LoadPlugin teamspeak2
+#LoadPlugin ted
+#LoadPlugin thermal
+#LoadPlugin tokyotyrant
+#LoadPlugin unixsock
+#LoadPlugin uptime
+#LoadPlugin users
+#LoadPlugin uuid
+#LoadPlugin varnish
+#LoadPlugin vmem
+#LoadPlugin vserver
+#LoadPlugin wireless
+LoadPlugin write_graphite
+#LoadPlugin write_http
+#LoadPlugin write_mongodb
+
+#<Plugin amqp>
+#	<Publish "name">
+#		Host "localhost"
+#		Port "5672"
+#		VHost "/"
+#		User "guest"
+#		Password "guest"
+#		Exchange "amq.fanout"
+#		RoutingKey "collectd"
+#		Persistent false
+#		StoreRates false
+#	</Publish>
+#</Plugin>
+
+#<Plugin apache>
+#	<Instance "foo">
+#		URL "http://localhost/server-status?auto"
+#		User "www-user"
+#		Password "secret"
+#		VerifyPeer false
+#		VerifyHost false
+#		CACert "/etc/ssl/ca.crt"
+#		Server "apache"
+#	</Instance>
+#
+#	<Instance "bar">
+#		URL "http://some.domain.tld/status?auto"
+#		Host "some.domain.tld"
+#		Server "lighttpd"
+#	</Instance>
+#</Plugin>
+
+#<Plugin apcups>
+#	Host "localhost"
+#	Port "3551"
+#</Plugin>
+
+#<Plugin ascent>
+#	URL "http://localhost/ascent/status/"
+#	User "www-user"
+#	Password "secret"
+#	VerifyPeer false
+#	VerifyHost false
+#	CACert "/etc/ssl/ca.crt"
+#</Plugin>
+
+#<Plugin "bind">
+#	URL "http://localhost:8053/"
+#
+#	ParseTime false
+#
+#	OpCodes true
+#	QTypes true
+#	ServerStats true
+#	ZoneMaintStats true
+#	ResolverStats false
+#	MemoryStats true
+#
+#	<View "_default">
+#		QTypes true
+#		ResolverStats true
+#		CacheRRSets true
+#
+#		Zone "127.in-addr.arpa/IN"
+#	</View>
+#</Plugin>
+
+#<Plugin csv>
+#	DataDir "/var/lib/collectd/csv"
+#	StoreRates false
+#</Plugin>
+
+#<Plugin curl>
+#	<Page "stock_quotes">
+#		URL "http://finance.google.com/finance?q=NYSE%3AAMD"
+#		User "foo"
+#		Password "bar"
+#		VerifyPeer false
+#		VerifyHost false
+#		CACert "/etc/ssl/ca.crt"
+#		MeasureResponseTime false
+#		<Match>
+#			Regex "<span +class=\"pr\"[^>]*> *([0-9]*\\.[0-9]+) *</span>"
+#			DSType "GaugeAverage"
+#			Type "stock_value"
+#			Instance "AMD"
+#		</Match>
+#	</Page>
+#</Plugin>
+
+#<Plugin curl_json>
+## See: http://wiki.apache.org/couchdb/Runtime_Statistics
+#  <URL "http://localhost:5984/_stats">
+#    Instance "httpd"
+#    <Key "httpd/requests/count">
+#      Type "http_requests"
+#    </Key>
+#
+#    <Key "httpd_request_methods/*/count">
+#      Type "http_request_methods"
+#    </Key>
+#
+#    <Key "httpd_status_codes/*/count">
+#      Type "http_response_codes"
+#    </Key>
+#  </URL>
+## Database status metrics:
+#  <URL "http://localhost:5984/_all_dbs">
+#    Instance "dbs"
+#    <Key "*/doc_count">
+#      Type "gauge"
+#    </Key>
+#    <Key "*/doc_del_count">
+#      Type "counter"
+#    </Key>
+#    <Key "*/disk_size">
+#      Type "bytes"
+#    </Key>
+#  </URL>
+#</Plugin>
+
+#<Plugin "curl_xml">
+#	<URL "http://localhost/stats.xml">
+#		Host "my_host"
+#		Instance "some_instance"
+#		User "collectd"
+#		Password "thaiNg0I"
+#		VerifyPeer true
+#		VerifyHost true
+#		CACert "/path/to/ca.crt"
+#
+#		<XPath "table[@id=\"magic_level\"]/tr">
+#			Type "magic_level"
+#			InstancePrefix "prefix-"
+#			InstanceFrom "td[1]"
+#			ValuesFrom "td[2]/span[@class=\"level\"]"
+#		</XPath>
+#	</URL>
+#</Plugin>
+
+#<Plugin dbi>
+#	<Query "num_of_customers">
+#		Statement "SELECT 'customers' AS c_key, COUNT(*) AS c_value \
+#				FROM customers_tbl"
+#		MinVersion 40102
+#		MaxVersion 50042
+#		<Result>
+#			Type "gauge"
+#			InstancePrefix "customer"
+#			InstancesFrom "c_key"
+#			ValuesFrom "c_value"
+#		</Result>
+#	</Query>
+#
+#	<Database "customers_db">
+#		Driver "mysql"
+#		DriverOption "host" "localhost"
+#		DriverOption "username" "collectd"
+#		DriverOption "password" "secret"
+#		DriverOption "dbname" "custdb0"
+#		SelectDB "custdb0"
+#		Query "num_of_customers"
+#		Query "..."
+#	</Database>
+#</Plugin>
+
+#<Plugin df>
+#	Device "/dev/sda1"
+#	Device "192.168.0.2:/mnt/nfs"
+#	MountPoint "/home"
+#	FSType "ext3"
+#	IgnoreSelected false
+#	ReportByDevice false
+#	ReportReserved false
+#	ReportInodes false
+#</Plugin>
+
+#<Plugin disk>
+#	Disk "hda"
+#	Disk "/sda[23]/"
+#	IgnoreSelected false
+#</Plugin>
+
+#<Plugin dns>
+#	Interface "eth0"
+#	IgnoreSource "192.168.0.1"
+#	SelectNumericQueryTypes false
+#</Plugin>
+
+#<Plugin email>
+#	SocketFile "/var/run/collectd-email"
+#	SocketGroup "collectd"
+#	SocketPerms "0770"
+#	MaxConns 5
+#</Plugin>
+
+#<Plugin ethstat>
+#	Interface "eth0"
+#	Map "rx_csum_offload_errors" "if_rx_errors" "checksum_offload"
+#	Map "multicast" "if_multicast"
+#	MappedOnly false
+#</Plugin>
+
+#<Plugin exec>
+#	Exec user "/path/to/exec"
+#	Exec "user:group" "/path/to/exec"
+#	NotificationExec user "/path/to/exec"
+#</Plugin>
+
+#<Plugin filecount>
+#	<Directory "/path/to/dir">
+#		Instance "foodir"
+#		Name "*.conf"
+#		MTime "-5m"
+#		Size "+10k"
+#		Recursive true
+#		IncludeHidden false
+#	</Directory>
+#</Plugin>
+
+#<Plugin gmond>
+#	MCReceiveFrom "239.2.11.71" "8649"
+#
+#	<Metric "swap_total">
+#		Type "swap"
+#		TypeInstance "total"
+#		DataSource "value"
+#	</Metric>
+#
+#	<Metric "swap_free">
+#		Type "swap"
+#		TypeInstance "free"
+#		DataSource "value"
+#	</Metric>
+#</Plugin>
+
+#<Plugin hddtemp>
+#	Host "127.0.0.1"
+#	Port 7634
+#</Plugin>
+
+#<Plugin interface>
+#	Interface "eth0"
+#	IgnoreSelected false
+#</Plugin>
+
+#<Plugin ipmi>
+#	Sensor "some_sensor"
+#	Sensor "another_one"
+#	IgnoreSelected false
+#	NotifySensorAdd false
+#	NotifySensorRemove true
+#	NotifySensorNotPresent false
+#</Plugin>
+
+#<Plugin iptables>
+#	Chain "table" "chain"
+#</Plugin>
+
+#<Plugin irq>
+#	Irq 7
+#	Irq 8
+#	Irq 9
+#	IgnoreSelected true
+#</Plugin>
+
+#<Plugin java>
+#	JVMArg "-verbose:jni"
+#	JVMArg "-Djava.class.path=/usr/share/collectd/java/collectd-api.jar"
+#
+#	LoadPlugin "org.collectd.java.GenericJMX"
+#	<Plugin "GenericJMX">
+#		# See /usr/share/doc/collectd/examples/GenericJMX.conf
+#		# for an example config.
+#	</Plugin>
+#</Plugin>
+
+#<Plugin libvirt>
+#	Connection "xen:///"
+#	RefreshInterval 60
+#	Domain "name"
+#	BlockDevice "name:device"
+#	InterfaceDevice "name:device"
+#	IgnoreSelected false
+#	HostnameFormat name
+#	InterfaceFormat name
+#</Plugin>
+
+#<Plugin madwifi>
+#	Interface "wlan0"
+#	IgnoreSelected false
+#	Source "SysFS"
+#	WatchSet "None"
+#	WatchAdd "node_octets"
+#	WatchAdd "node_rssi"
+#	WatchAdd "is_rx_acl"
+#	WatchAdd "is_scan_active"
+#</Plugin>
+
+#<Plugin mbmon>
+#	Host "127.0.0.1"
+#	Port 411
+#</Plugin>
+
+#<Plugin md>
+#	Device "/dev/md0"
+#	IgnoreSelected false
+#</Plugin>
+
+#<Plugin memcachec>
+#	<Page "plugin_instance">
+#		Server "localhost"
+#		Key "page_key"
+#		<Match>
+#			Regex "(\\d+) bytes sent"
+#			ExcludeRegex "<lines to be excluded>"
+#			DSType CounterAdd
+#			Type "ipt_octets"
+#			Instance "type_instance"
+#		</Match>
+#	</Page>
+#</Plugin>
+
+#<Plugin memcached>
+#	Socket "/var/run/memcached.sock"
+# or:
+#	Host "127.0.0.1"
+#	Port "11211"
+#</Plugin>
+
+#<Plugin mysql>
+#	<Database db_name>
+#		Host "database.serv.er"
+#		Port "3306"
+#		User "db_user"
+#		Password "secret"
+#		Database "db_name"
+#		MasterStats true
+#	</Database>
+#
+#	<Database db_name2>
+#		Host "localhost"
+#		Socket "/var/run/mysql/mysqld.sock"
+#		SlaveStats true
+#		SlaveNotifications true
+#	</Database>
+#</Plugin>
+
+#<Plugin netlink>
+#	Interface "All"
+#	VerboseInterface "All"
+#	QDisc "eth0" "pfifo_fast-1:0"
+#	Class "ppp0" "htb-1:10"
+#	Filter "ppp0" "u32-1:0"
+#	IgnoreSelected false
+#</Plugin>
+
+#<Plugin network>
+#	# client setup:
+#	Server "ff18::efc0:4a42" "25826"
+#	<Server "239.192.74.66" "25826">
+#		SecurityLevel Encrypt
+#		Username "user"
+#		Password "secret"
+#		Interface "eth0"
+#	</Server>
+#	TimeToLive "128"
+#
+#	# server setup:
+#	Listen "0.0.0.0" "25826"
+#	<Listen "239.192.74.66" "25826">
+#		SecurityLevel Sign
+#		AuthFile "/etc/collectd/passwd"
+#		Interface "eth0"
+#	</Listen>
+#	MaxPacketSize 1024
+#
+#	# proxy setup (client and server as above):
+#	Forward true
+#
+#	# statistics about the network plugin itself
+#	ReportStats false
+#
+#	# "garbage collection"
+#	CacheFlush 1800
+#</Plugin>
+
+#<Plugin nginx>
+#	URL "http://localhost/status?auto"
+#	User "www-user"
+#	Password "secret"
+#	VerifyPeer false
+#	VerifyHost false
+#	CACert "/etc/ssl/ca.crt"
+#</Plugin>
+
+#<Plugin notify_desktop>
+#	OkayTimeout 1000
+#	WarningTimeout 5000
+#	FailureTimeout 0
+#</Plugin>
+
+#<Plugin notify_email>
+#	SMTPServer "localhost"
+#	SMTPPort 25
+#	SMTPUser "my-username"
+#	SMTPPassword "my-password"
+#	From "collectd@main0server.com"
+#	# <WARNING/FAILURE/OK> on <hostname>.
+#	# Beware! Do not use not more than two placeholders (%)!
+#	Subject "[collectd] %s on %s!"
+#	Recipient "email1@domain1.net"
+#	Recipient "email2@domain2.com"
+#</Plugin>
+
+#<Plugin ntpd>
+#	Host "localhost"
+#	Port 123
+#	ReverseLookups false
+#</Plugin>
+
+#<Plugin nut>
+#	UPS "upsname@hostname:port"
+#</Plugin>
+
+#<Plugin olsrd>
+#	Host "127.0.0.1"
+#	Port "2006"
+#	CollectLinks "Summary"
+#	CollectRoutes "Summary"
+#	CollectTopology "Summary"
+#</Plugin>
+
+#<Plugin openvpn>
+#	StatusFile "/etc/openvpn/openvpn-status.log"
+#	ImprovedNamingSchema false
+#	CollectCompression true
+#	CollectIndividualUsers true
+#	CollectUserCount false
+#</Plugin>
+
+#<Plugin perl>
+#	IncludeDir "/my/include/path"
+#	BaseName "Collectd::Plugins"
+#	EnableDebugger ""
+#	LoadPlugin Monitorus
+#	LoadPlugin OpenVZ
+#
+#	<Plugin foo>
+#		Foo "Bar"
+#		Qux "Baz"
+#	</Plugin>
+#</Plugin>
+
+#<Plugin pinba>
+#	Address "::0"
+#	Port "30002"
+#	<View "name">
+#		Host "host name"
+#		Server "server name"
+#		Script "script name"
+#	<View>
+#</Plugin>
+
+#<Plugin ping>
+#	Host "host.foo.bar"
+#	Host "host.baz.qux"
+#	Interval 1.0
+#	Timeout 0.9
+#	TTL 255
+#	SourceAddress "1.2.3.4"
+#	Device "eth0"
+#	MaxMissed -1
+#</Plugin>
+
+#<Plugin postgresql>
+#	<Query magic>
+#		Statement "SELECT magic FROM wizard WHERE host = $1;"
+#		Param hostname
+#
+#		<Result>
+#			Type gauge
+#			InstancePrefix "magic"
+#			ValuesFrom "magic"
+#		</Result>
+#	</Query>
+#
+#	<Query rt36_tickets>
+#		Statement "SELECT COUNT(type) AS count, type \
+#		                  FROM (SELECT CASE \
+#		                               WHEN resolved = 'epoch' THEN 'open' \
+#		                               ELSE 'resolved' END AS type \
+#		                               FROM tickets) type \
+#		                  GROUP BY type;"
+#
+#		<Result>
+#			Type counter
+#			InstancePrefix "rt36_tickets"
+#			InstancesFrom "type"
+#			ValuesFrom "count"
+#		</Result>
+#	</Query>
+#
+#	<Database foo>
+#		Host "hostname"
+#		Port 5432
+#		User "username"
+#		Password "secret"
+#
+#		SSLMode "prefer"
+#		KRBSrvName "kerberos_service_name"
+#
+#		Query magic
+#	</Database>
+#
+#	<Database bar>
+#		Interval 60
+#		Service "service_name"
+#
+#		Query backend # predefined
+#		Query rt36_tickets
+#	</Database>
+#</Plugin>
+
+#<Plugin powerdns>
+#	<Server "server_name">
+#		Collect "latency"
+#		Collect "udp-answers" "udp-queries"
+#		Socket "/var/run/pdns.controlsocket"
+#	</Server>
+#	<Recursor "recursor_name">
+#		Collect "questions"
+#		Collect "cache-hits" "cache-misses"
+#		Socket "/var/run/pdns_recursor.controlsocket"
+#	</Recursor>
+#	LocalSocket "/opt/collectd/var/run/collectd-powerdns"
+#</Plugin>
+
+#<Plugin processes>
+#	Process "name"
+#	ProcessMatch "foobar" "/usr/bin/perl foobar\\.pl.*"
+#</Plugin>
+
+#<Plugin protocols>
+#	Value "/^Tcp:/"
+#	IgnoreSelected false
+#</Plugin>
+
+#<Plugin python>
+#	ModulePath "/path/to/your/python/modules"
+#	LogTraces true
+#	Interactive true
+#	Import "spam"
+#
+#	<Module spam>
+#		spam "wonderful" "lovely"
+#	</Module>
+#</Plugin>
+
+#<Plugin rrdcached>
+#	DaemonAddress "unix:/var/run/rrdcached.sock"
+#	DataDir "/var/lib/rrdcached/db/collectd"
+#	CreateFiles true
+#	CollectStatistics true
+#</Plugin>
+
+<Plugin rrdtool>
+	DataDir "/var/lib/collectd/rrd"
+#	CacheTimeout 120
+#	CacheFlush 900
+#	WritesPerSecond 30
+#	RandomTimeout 0
+#
+# The following settings are rather advanced
+# and should usually not be touched:
+#	StepSize 10
+#	HeartBeat 20
+#	RRARows 1200
+#	RRATimespan 158112000
+#	XFF 0.1
+</Plugin>
+
+#<Plugin sensors>
+#	SensorConfigFile "/etc/sensors3.conf"
+#	Sensor "it8712-isa-0290/temperature-temp1"
+#	Sensor "it8712-isa-0290/fanspeed-fan3"
+#	Sensor "it8712-isa-0290/voltage-in8"
+#	IgnoreSelected false
+#</Plugin>
+
+# See /usr/share/doc/collectd/examples/snmp-data.conf.gz for a
+# comprehensive sample configuration.
+#<Plugin snmp>
+#	<Data "powerplus_voltge_input">
+#		Type "voltage"
+#		Table false
+#		Instance "input_line1"
+#		Scale 0.1
+#		Values "SNMPv2-SMI::enterprises.6050.5.4.1.1.2.1"
+#	</Data>
+#	<Data "hr_users">
+#		Type "users"
+#		Table false
+#		Instance ""
+#		Shift -1
+#		Values "HOST-RESOURCES-MIB::hrSystemNumUsers.0"
+#	</Data>
+#	<Data "std_traffic">
+#		Type "if_octets"
+#		Table true
+#		InstancePrefix "traffic"
+#		Instance "IF-MIB::ifDescr"
+#		Values "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets"
+#	</Data>
+#
+#	<Host "some.switch.mydomain.org">
+#		Address "192.168.0.2"
+#		Version 1
+#		Community "community_string"
+#		Collect "std_traffic"
+#		Inverval 120
+#	</Host>
+#	<Host "some.server.mydomain.org">
+#		Address "192.168.0.42"
+#		Version 2
+#		Community "another_string"
+#		Collect "std_traffic" "hr_users"
+#	</Host>
+#	<Host "some.ups.mydomain.org">
+#		Address "192.168.0.3"
+#		Version 1
+#		Community "more_communities"
+#		Collect "powerplus_voltge_input"
+#		Interval 300
+#	</Host>
+#</Plugin>
+
+#<Plugin swap>
+#	ReportByDevice false
+#</Plugin>
+
+#<Plugin table>
+#	<Table "/proc/slabinfo">
+#		Instance "slabinfo"
+#		Separator " "
+#		<Result>
+#			Type gauge
+#			InstancePrefix "active_objs"
+#			InstancesFrom 0
+#			ValuesFrom 1
+#		</Result>
+#		<Result>
+#			Type gauge
+#			InstancePrefix "objperslab"
+#			InstancesFrom 0
+#			ValuesFrom 4
+#		</Result>
+#	</Table>
+#</Plugin>
+
+#<Plugin "tail">
+#	<File "/var/log/exim4/mainlog">
+#		Instance "exim"
+#		<Match>
+#			Regex "S=([1-9][0-9]*)"
+#			DSType "CounterAdd"
+#			Type "ipt_bytes"
+#			Instance "total"
+#		</Match>
+#		<Match>
+#			Regex "\\<R=local_user\\>"
+#			ExcludeRegex "\\<R=local_user\\>.*mail_spool defer"
+#			DSType "CounterInc"
+#			Type "counter"
+#			Instance "local_user"
+#		</Match>
+#	</File>
+#</Plugin>
+
+<Plugin tcpconns>
+	LocalPort "4001"
+	LocalPort "7001"
+</Plugin>
+
+#<Plugin teamspeak2>
+#	Host "127.0.0.1"
+#	Port "51234"
+#	Server "8767"
+#</Plugin>
+
+#<Plugin ted>
+#	Device "/dev/ttyUSB0"
+#	Retries 0
+#</Plugin>
+
+#<Plugin thermal>
+#	ForceUseProcfs false
+#	Device "THRM"
+#	IgnoreSelected false
+#</Plugin>
+
+#<Plugin tokyotyrant>
+#	Host "localhost"
+#	Port "1978"
+#</Plugin>
+
+#<Plugin unixsock>
+#	SocketFile "/var/run/collectd-unixsock"
+#	SocketGroup "collectd"
+#	SocketPerms "0660"
+#	DeleteSocket false
+#</Plugin>
+
+#<Plugin uuid>
+#	UUIDFile "/etc/uuid"
+#</Plugin>
+
+#<Plugin varnish>
+#	<Instance>
+#		CollectCache true
+#		CollectBackend true
+#		CollectConnections true
+#		CollectSHM true
+#		CollectESI false
+#		CollectFetch false
+#		CollectHCB false
+#		CollectSMA false
+#		CollectSMS false
+#		CollectSM false
+#		CollectTotals false
+#		CollectWorkers false
+#	</Instance>
+#
+#	<Instance "myinstance">
+#		CollectCache true
+#	</Instance>
+#</Plugin>
+
+#<Plugin vmem>
+#	Verbose false
+#</Plugin>
+
+#<Plugin write_graphite>
+#	<Carbon>
+#		Host "127.0.01"
+#		Port "2003"
+#		Prefix "collectd"
+#		Postfix "collectd"
+#		StoreRates false
+#		AlwaysAppendDS false
+#		EscapeCharacter "_"
+#	</Carbon>
+#</Plugin>
+
+#<Plugin write_http>
+#	<URL "http://example.com/collectd-post">
+#		User "collectd"
+#		Password "secret"
+#		VerifyPeer true
+#		VerifyHost true
+#		CACert "/etc/ssl/ca.crt"
+#		Format "Command"
+#		StoreRates false
+#	</URL>
+#</Plugin>
+
+#<Plugin write_mongodb>
+#	<Node "example">
+#		Host "localhost"
+#		Port "27017"
+#		Timeout 1000
+#		StoreRates false
+#	<Node>
+#</Plugin>
+
+Include "/etc/collectd/filters.conf"
+Include "/etc/collectd/thresholds.conf"

contrib/graphite/Dockerfile

@@ -0,0 +1,31 @@
+from	stackbrew/ubuntu:precise
+
+run	echo 'deb http://us.archive.ubuntu.com/ubuntu/ precise universe' >> /etc/apt/sources.list
+run	apt-get -y update
+
+# Install required packages
+run	apt-get -y install python-cairo python-django python-twisted python-django-tagging python-simplejson python-pysqlite2 python-support python-pip gunicorn supervisor nginx-light
+run	pip install whisper
+run	pip install --install-option="--prefix=/var/lib/graphite" --install-option="--install-lib=/var/lib/graphite/lib" carbon
+run	pip install --install-option="--prefix=/var/lib/graphite" --install-option="--install-lib=/var/lib/graphite/webapp" graphite-web
+
+# Add system service config
+add	./nginx.conf /etc/nginx/nginx.conf
+add	./supervisord.conf /etc/supervisor/conf.d/supervisord.conf
+
+# Add graphite config
+add	./initial_data.json /var/lib/graphite/webapp/graphite/initial_data.json
+add	./local_settings.py /var/lib/graphite/webapp/graphite/local_settings.py
+add	./carbon.conf /var/lib/graphite/conf/carbon.conf
+add	./storage-schemas.conf /var/lib/graphite/conf/storage-schemas.conf
+run	mkdir -p /var/lib/graphite/storage/whisper
+run	touch /var/lib/graphite/storage/graphite.db /var/lib/graphite/storage/index
+run	chown -R www-data /var/lib/graphite/storage
+run	chmod 0775 /var/lib/graphite/storage /var/lib/graphite/storage/whisper
+run	chmod 0664 /var/lib/graphite/storage/graphite.db
+run cd /var/lib/graphite/webapp/graphite && python manage.py syncdb --noinput
+
+expose	:80
+expose	:2003
+
+cmd	["/usr/bin/supervisord"]

contrib/graphite/README

@@ -0,0 +1,7 @@
+Running graphite under Docker is straightforward:
+
+1. Build the graphite image using Docker
+docker build -t graphite .
+
+2. Run a graphite container. Be sure to replace the $IP field with the IP address at which you wish to expose your graphite web service.
+docker run -p $IP:8080:80 -p $IP:2003:2003 -d graphite

contrib/graphite/carbon.conf

@@ -0,0 +1,62 @@
+[cache]
+LOCAL_DATA_DIR = /var/lib/graphite/storage/whisper/
+
+# Specify the user to drop privileges to
+# If this is blank carbon runs as the user that invokes it
+# This user must have write access to the local data directory
+USER = 
+
+# Limit the size of the cache to avoid swapping or becoming CPU bound.
+# Sorts and serving cache queries gets more expensive as the cache grows.
+# Use the value "inf" (infinity) for an unlimited cache size.
+MAX_CACHE_SIZE = inf
+
+# Limits the number of whisper update_many() calls per second, which effectively
+# means the number of write requests sent to the disk. This is intended to
+# prevent over-utilizing the disk and thus starving the rest of the system.
+# When the rate of required updates exceeds this, then carbon's caching will
+# take effect and increase the overall throughput accordingly.
+MAX_UPDATES_PER_SECOND = 1000
+
+# Softly limits the number of whisper files that get created each minute.
+# Setting this value low (like at 50) is a good way to ensure your graphite
+# system will not be adversely impacted when a bunch of new metrics are
+# sent to it. The trade off is that it will take much longer for those metrics'
+# database files to all get created and thus longer until the data becomes usable.
+# Setting this value high (like "inf" for infinity) will cause graphite to create
+# the files quickly but at the risk of slowing I/O down considerably for a while.
+MAX_CREATES_PER_MINUTE = inf
+
+LINE_RECEIVER_INTERFACE = 0.0.0.0
+LINE_RECEIVER_PORT = 2003
+
+#PICKLE_RECEIVER_INTERFACE = 0.0.0.0
+#PICKLE_RECEIVER_PORT = 2004
+
+#CACHE_QUERY_INTERFACE = 0.0.0.0
+#CACHE_QUERY_PORT = 7002
+
+LOG_UPDATES = False
+
+# Enable AMQP if you want to receve metrics using an amqp broker
+# ENABLE_AMQP = False
+
+# Verbose means a line will be logged for every metric received
+# useful for testing
+# AMQP_VERBOSE = False
+
+# AMQP_HOST = localhost
+# AMQP_PORT = 5672
+# AMQP_VHOST = /
+# AMQP_USER = guest
+# AMQP_PASSWORD = guest
+# AMQP_EXCHANGE = graphite
+
+# Patterns for all of the metrics this machine will store. Read more at
+# http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol#Bindings
+#
+# Example: store all sales, linux servers, and utilization metrics
+# BIND_PATTERNS = sales.#, servers.linux.#, #.utilization
+#
+# Example: store everything
+# BIND_PATTERNS = #

contrib/graphite/initial_data.json

@@ -0,0 +1,20 @@
+[
+  {
+    "pk": 1,
+    "model": "auth.user",
+    "fields": {
+      "username": "admin",
+      "first_name": "",
+      "last_name": "",
+      "is_active": true,
+      "is_superuser": true,
+      "is_staff": true,
+      "last_login": "2011-09-20 17:02:14",
+      "groups": [],
+      "user_permissions": [],
+      "password": "sha1$1b11b$edeb0a67a9622f1f2cfeabf9188a711f5ac7d236",
+      "email": "root@example.com",
+      "date_joined": "2011-09-20 17:02:14"
+    }
+  }
+]

contrib/graphite/local_settings.py

@@ -0,0 +1 @@
+TIME_ZONE = 'UTC'

contrib/graphite/nginx.conf

@@ -0,0 +1,69 @@
+daemon off;
+user www-data;
+worker_processes 1;
+pid /var/run/nginx.pid;
+
+events {
+  worker_connections 1024;
+}
+
+http {
+  sendfile on;
+  tcp_nopush on;
+  tcp_nodelay on;
+  keepalive_timeout 65;
+  types_hash_max_size 2048;
+  server_tokens off;
+
+  server_names_hash_bucket_size 32;
+
+  include /etc/nginx/mime.types;
+  default_type application/octet-stream;
+
+  access_log /var/log/nginx/access.log;
+  error_log /var/log/nginx/error.log;
+
+  gzip on;
+  gzip_disable "msie6";
+
+  server {
+    listen 80 default_server;
+    server_name _;
+
+    open_log_file_cache max=1000 inactive=20s min_uses=2 valid=1m;
+
+    location / {
+        proxy_pass                 http://127.0.0.1:8000;
+        proxy_set_header           X-Real-IP   $remote_addr;
+        proxy_set_header           X-Forwarded-For  $proxy_add_x_forwarded_for;
+        proxy_set_header           X-Forwarded-Proto  $scheme;
+        proxy_set_header           X-Forwarded-Server  $host;
+        proxy_set_header           X-Forwarded-Host  $host;
+        proxy_set_header           Host  $host;
+
+        client_max_body_size       10m;
+        client_body_buffer_size    128k;
+
+        proxy_connect_timeout      90;
+        proxy_send_timeout         90;
+        proxy_read_timeout         90;
+
+        proxy_buffer_size          4k;
+        proxy_buffers              4 32k;
+        proxy_busy_buffers_size    64k;
+        proxy_temp_file_write_size 64k;
+    }
+
+    add_header Access-Control-Allow-Origin "*";
+    add_header Access-Control-Allow-Methods "GET, OPTIONS";
+    add_header Access-Control-Allow-Headers "origin, authorization, accept";
+
+    location /content {
+      alias /var/lib/graphite/webapp/content;
+    }
+
+    location /media {
+      alias /usr/share/pyshared/django/contrib/admin/media;
+    }
+  }
+}

contrib/graphite/storage-schemas.conf

@@ -0,0 +1,7 @@
+[carbon]
+pattern = ^carbon\..*
+retentions = 1m:31d,10m:1y,1h:5y
+
+[default]
+pattern = .*
+retentions = 10s:8d,1m:31d,10m:1y,1h:5y

contrib/graphite/supervisord.conf

@@ -0,0 +1,25 @@
+[supervisord]
+nodaemon = true
+environment = GRAPHITE_STORAGE_DIR='/var/lib/graphite/storage',GRAPHITE_CONF_DIR='/var/lib/graphite/conf'
+
+[program:nginx]
+command = /usr/sbin/nginx
+stdout_logfile = /var/log/supervisor/%(program_name)s.log
+stderr_logfile = /var/log/supervisor/%(program_name)s.log
+autorestart = true
+
+[program:carbon-cache]
+user = www-data
+command = /var/lib/graphite/bin/carbon-cache.py --debug start
+stdout_logfile = /var/log/supervisor/%(program_name)s.log
+stderr_logfile = /var/log/supervisor/%(program_name)s.log
+autorestart = true
+
+[program:graphite-webapp]
+user = www-data
+directory = /var/lib/graphite/webapp
+environment = PYTHONPATH='/var/lib/graphite/webapp'
+command = /usr/bin/gunicorn_django -b127.0.0.1:8000 -w2 graphite/settings.py
+stdout_logfile = /var/log/supervisor/%(program_name)s.log
+stderr_logfile = /var/log/supervisor/%(program_name)s.log
+autorestart = true