tensor.sources

tensor.sources.database.postgresql

class tensor.sources.database.postgresql.PostgreSQL(*a, **kw)[source]

Bases: tensor.objects.Source

Reads PostgreSQL metrics

Configuration arguments:

Parameters:
  • host (str.) – Database host
  • port (int.) – Database port
  • user (str.) – Username
  • password (str.) – Password

Metrics:

(service name).(database name).(metrics):
 Metrics from pg_stat_database

tensor.sources.database.elasticsearch

class tensor.sources.database.elasticsearch.ElasticSearch(*a, **kw)[source]

Bases: tensor.objects.Source

Reads elasticsearch metrics

Configuration arguments:

Parameters:
  • url (str.) – Elasticsearch base URL (default: http://localhost:9200)
  • user (str.) – Basic auth username
  • password (str.) – Password

Metrics:

(service name).cluster.status:
 Cluster status (Red=0, Yellow=1, Green=2)
(service name).cluster.nodes:
 Cluster node count
(service name).indices:
 Total indices in cluster
(service name).shards.total:
 Total number of shards
(service name).shards.primary:
 Number of primary shards
(service name).documents.total:
 Total documents
(service name).documents.rate:
 Documents per second
(service name).documents.size:
 Size of document store in bytes

tensor.sources.database.memcache

class tensor.sources.database.memcache.Memcache(*a, **kw)[source]

Bases: tensor.objects.Source

Reads memcache metrics

Configuration arguments:

Parameters:
  • host (str.) – Database host (default localhost)
  • port (int.) – Database port (default 11211)

Metrics:

(service name).(metrics):
 Metrics from memcached

tensor.sources.docker

class tensor.sources.docker.ContainerStats(*a, **kw)[source]

Bases: tensor.objects.Source

Returns stats for Docker containers on this host

Configuration arguments:

Parameters:url (str.) – Docker stats URL

Metrics:

(service name).(container name).mem_limit:
 Maximum memory for container
(service name).(container name).mem_used:
 Memory used by container
(service name).(container name).cpu:
 Percentage of system CPU in use
(service name).(container name).io_read:
 IO reads per second
(service name).(container name).io_write:
 IO writes per second
(service name).(container name).io_sync:
 IO synchronous op/s
(service name).(container name).io_async:
 IO asynchronous op/s
(service name).(container name).io_total:
 Total IOPS

Note. If a MARATHON_APP_ID environment variable exists on the container then container name will be used instead of that.

tensor.sources.haproxy

class tensor.sources.haproxy.HAProxy(*a, **kw)[source]

Bases: tensor.objects.Source

Reads Nginx stub_status

Configuration arguments:

Parameters:
  • url (str.) – URL to fetch stats from
  • user (str.) – Username
  • password (str.) – Password

Metrics:

(service name).(backend|frontend|nodes).(stats):
 Various statistics

tensor.sources.generator

class tensor.sources.generator.Function(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Produces an arbitrary function

Functions can contain the functions sin, cos, sinh, cosh, tan, tanh, asin, acos, atan, asinh, acosh, atanh, log(n, [base|e]), abs

Or the constants e, pi, and variable x

Configuration arguments:

Parameters:
  • dx (float.) – Resolution with time (steps of x)
  • function (string.) – Function to produce

tensor.sources.linux

tensor.sources.linux.basic

class tensor.sources.linux.basic.CPU(*a)[source]

Bases: tensor.objects.Source

Reports system CPU utilisation as a percentage/100

Metrics:

(service name):Percentage CPU utilisation
(service name).(type):
 Percentage CPU utilisation by type
class tensor.sources.linux.basic.DiskFree(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Returns the free space for all mounted filesystems

Configuration arguments:

Parameters:disks (list.) – List of devices to check (optional)

Metrics:

(service name).(device).used:
 Used space (%)
(service name).(device).bytes:
 Used space (kbytes)
(service name).(device).free:
 Free space (kbytes)
class tensor.sources.linux.basic.DiskIO(*a, **kw)[source]

Bases: tensor.objects.Source

Reports disk IO statistics per device

Configuration arguments:

Parameters:devices (list.) – List of devices to check (optional)

Metrics:

(service name).(device name).reads:
 Number of completed reads
(service name).(device name).read_bytes:
 Bytes read per second
(service name).(device name).read_latency:
 Disk read latency
(service name).(device name).writes:
 Number of completed writes
(service name).(device name).write_bytes:
 Bytes written per second
(service name).(device name).write_latency:
 Disk write latency
class tensor.sources.linux.basic.LoadAverage(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Reports system load average for the current host

Metrics:

(service name):Load average
class tensor.sources.linux.basic.Memory(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Reports system memory utilisation as a percentage/100

Metrics:

(service name):Percentage memory utilisation
class tensor.sources.linux.basic.Network(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Returns all network interface statistics

Configuration arguments:

Parameters:interfaces (list.) – List of interfaces to check (optional)

Metrics:

(service name).(device).tx_bytes:
 Bytes transmitted
(service name).(device).tx_packets:
 Packets transmitted
(service name).(device).tx_errors:
 Errors
(service name).(device).rx_bytes:
 Bytes received
(service name).(device).rx_packets:
 Packets received
(service name).(device).rx_errors:
 Errors

tensor.sources.linux.process

class tensor.sources.linux.process.ProcessCount(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Returns the ps count on the system

Metrics:

(service name):Number of processes
class tensor.sources.linux.process.ProcessStats(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Returns memory used by each active parent process

Metrics:

(service name).proc.(process name).cpu:
 Per process CPU usage
(service name).proc.(process name).memory:
 Per process memory use
(service name).proc.(process name).age:
 Per process age
(service name).user.(user name).cpu:
 Per user CPU usage
(service name).user.(user name).memory:
 Per user memory use

tensor.sources.linux.sensors

class tensor.sources.linux.sensors.SMART(*a, **kw)[source]

Bases: tensor.objects.Source

Returns SMART output for all disks

Metrics:

(service name).(disk).(sensor):
 Sensor value
class tensor.sources.linux.sensors.Sensors(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Returns lm-sensors output

NB. This is very untested on different configurations and versions. Please report any issues with the output of the sensors command to help improve it.

Metrics:

(service name).(adapter).(sensor):
 Sensor value

tensor.sources.media

tensor.sources.media.libav

class tensor.sources.media.libav.DarwinRTSP(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Makes avprobe requests of a Darwin RTSP sample stream (sample_100kbit.mp4)

Configuration arguments:

Parameters:destination – Host name or IP address to check

Metrics: :(service name): Time to complete request

You can also override the hostname argument to make it match metrics from that host.

tensor.sources.munin

class tensor.sources.munin.MuninNode(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Connects to munin-node and retrieves all metrics

Configuration arguments:

Parameters:
  • host (str.) – munin-node hostname (probably localhost)
  • port (int.) – munin-node port (probably 4949)

Metrics:

(service name).(plugin name).(keys...):
 A dot separated tree of munin plugin keys
class tensor.sources.munin.MuninProtocol[source]

Bases: twisted.protocols.basic.LineReceiver

MuninProtocol - provides a line receiver protocol for making requests to munin-node

Requests must be made sequentially

tensor.sources.network

class tensor.sources.network.HTTP(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Performs an HTTP request

Configuration arguments:

Parameters:
  • url (str.) – HTTP URL
  • method (str.) – HTTP request method to use (default GET)
  • match (str.) – A text string to match in the document when it is correct
  • useragent (str.) – User-Agent header to use
  • timeout (int.) – Timeout for connection (default 60s)

Metrics:

(service name).latency:
 Time to complete request
class tensor.sources.network.Ping(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Performs an Ping checks against a destination

Configuration arguments:

Parameters:destination (str.) – Host name or IP address to ping

Metrics:

(service name).latency:
 Ping latency
(service name).loss:
 Packet loss

You can also override the hostname argument to make it match metrics from that host.

tensor.sources.nginx

class tensor.sources.nginx.Nginx(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Reads Nginx stub_status

Configuration arguments:

Parameters:stats_url (str.) – URL to fetch stub_status from

Metrics:

(service name).active:
 Active connections at this time
(service name).accepts:
 Accepted connections
(service name).handled:
 Handled connections
(service name).requests:
 Total client requests
(service name).reading:
 Reading requests
(service name).writing:
 Writing responses
(service name).waiting:
 Waiting connections
class tensor.sources.nginx.NginxLog(*a)[source]

Bases: tensor.objects.Source

Tails Nginx log files, parses them and returns log events for outputs which support them.

Configuration arguments:

Parameters:
  • log_format (str.) – Log format passed to parser, same as the config definition (default: combined)
  • file (str.) – Log file
  • max_lines (int.) – Maximum number of log lines to read per interval to prevent overwhelming Tensor when reading large logs (default 2000)
class tensor.sources.nginx.NginxLogMetrics(*a)[source]

Bases: tensor.objects.Source

Tails Nginx log files, parses them and returns metrics for data usage and requests against other fields.

Configuration arguments:

Parameters:
  • log_format (str.) – Log format passed to parser, same as the config definition
  • file (str.) – Log file
  • max_lines (int.) – Maximum number of log lines to read per interval to prevent overwhelming Tensor when reading large logs (default 2000)
  • resolution (int.) – Aggregate bucket resolution in seconds (default 10)
  • history (bool.) – Read the entire file from scratch if we’ve never seen it (default false)

Metrics:

(service name).total_rbytes:
 Bytes total for all requests
(service name).total_requests:
 Total request count
(service name).stats.(code).(requests|rbytes):
 Metrics by status code
(service name).user-agent.(agent).(requests|rbytes):
 Metrics by user agent
(service name).client.(ip).(requests|rbytes):
 Metrics by client IP
(service name).request.(request path).(requests|rbytes):
 Metrics by request path

tensor.sources.python

tensor.sources.python.uwsgi

class tensor.sources.python.uwsgi.Emperor(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Connects to UWSGI Emperor stats and creates useful metrics

Configuration arguments:

Parameters:
  • host (str.) – Hostname (default localhost)
  • port (int.) – Port
class tensor.sources.python.uwsgi.JSONProtocol[source]

Bases: twisted.internet.protocol.Protocol

JSON line protocol

tensor.sources.rabbitmq

class tensor.sources.rabbitmq.Queues(*a, **kw)[source]

Bases: tensor.objects.Source

Returns Queue information for a particular vhost

Configuration arguments:

Parameters:vhost (str.) – Vhost name

Metrics:

(service_name).(queue).ready:
 Ready messages for queue
(service_name).(queue).unack:
 Unacknowledged messages for queue
(service_name).(queue).ready_rate:
 Ready rate of change per second
(service_name).(queue).unack_rate:
 Unacknowledge rate of change per second

tensor.sources.redis

class tensor.sources.redis.Queues(*a, **kw)[source]

Bases: tensor.objects.Source

Query llen from redis-cli

Configuration arguments:

Parameters:
  • queue (str.) – Queue name (defaults to ‘celery’, just because)
  • db (int.) – DB number
  • clipath (str.) – Path to redis-cli (default: /usr/bin/redis-cli)

Metrics:

(service_name):Queue length
(service_name):Queue rate

tensor.sources.riak

class tensor.sources.riak.RiakStats(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Returns GET/PUT rates for a Riak node

Configuration arguments:

Parameters:
  • url (str.) – Riak stats URL
  • useragent (str.) – User-Agent header to use

Metrics:

(service name).latency:
 Time to complete request

tensor.sources.riemann

class tensor.sources.riemann.RiemannTCP(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Provides a listening server which accepts Riemann metrics and proxies them to our queue.

Configuration arguments:

Parameters:port (int.) – Port to listen on (default 5555)
startTimer()[source]

Creates a Riemann TCP server instead of a timer

class tensor.sources.riemann.RiemannTCPServer(source)[source]

Bases: tensor.protocol.riemann.RiemannProtocol

Server implementation of the Riemann protocol

tensor.sources.sflow

class tensor.sources.sflow.sFlow(config, queueBack, tensor)[source]

Bases: tensor.objects.Source

Provides an sFlow server Source

Configuration arguments:

Parameters:
  • port (int.) – UDP port to listen on
  • dnslookup (bool.) – Enable reverse DNS lookup for device IPs (default: True)

Metrics:

Metrics are published using the key patterns (device).(service name).(interface).(in|out)Octets (device).(service name).(interface).ip (device).(service name).(interface).port

startTimer()[source]

Creates a sFlow datagram server

class tensor.sources.sflow.sFlowReceiver(source)[source]

Bases: tensor.protocol.sflow.server.DatagramReceiver

sFlow datagram protocol

tensor.sources.snmp

class tensor.sources.snmp.SNMP(*a, **kw)[source]

Bases: tensor.objects.Source

Connects to an SNMP agent and retrieves OIDs

Configuration arguments:

Parameters:
  • ip (str.) – SNMP agent host (default: 127.0.0.1)
  • port (int.) – SNMP port (default: 161)
  • community (str.) – SNMP read community
class tensor.sources.snmp.SNMPCisco837(*a, **kw)[source]

Bases: tensor.sources.snmp.SNMP

Connects to a Cisco 837 and makes metrics

Configuration arguments:

Parameters:
  • ip (str.) – SNMP agent host (default: 127.0.0.1)
  • port (int.) – SNMP port (default: 161)
  • community (str.) – SNMP read community
class tensor.sources.snmp.SNMPConnection(host, port, community)[source]

Bases: object

A wrapper class for PySNMP functions

Parameters:
  • host (str.) – SNMP agent host
  • port (int.) – SNMP port
  • community (str.) – SNMP read community

(This is not a source and you shouldn’t try to use it as one)