Skip to content

PearsonEducation/StatsPoller

Repository files navigation

StatsPoller

Overview

StatsPoller is an agent-based metric collection & reporting platform. It currently outputs Graphite-formatted metrics & OpenTSDB-formatted metrics. It functions in a similar way to other metric collection agents, such as TCollector, scollector, collectd, etc. StatsPoller is ideally paired with StatsAgg for alerting, and Graphite or OpenTSDB for metric storage, and Grafana for dashboarding.


What are StatsPoller's core features?

  • Metric collectors that retrieve metrics on a user-specified intervals.
  • Built-in metric collectors have few dependencies.
    • Most metric collectors are built into the main program & rely only on Java 1.7+ being installed to run StatsPoller.
    • For Linux metric collectors, most metrics are retrieved from Linux's 'proc' and/or 'sys' filesystems. As a result, dependencies on apps like vmstat, ifconfig, etc are virtually non-existant in StatsPoller.
  • Advanced Java JMX metric collection is a primary focus in StatsPoller. Its JMX metric retrieval capabilities exceed that of most comparable tools.
  • Built-in metric collectors include....
    • StatsPoller Native (A platform-independent OS metric collector that collects metrics from the perspective of the JVM runtime)
    • Linux OS (network, filesystem, disk i/o, cpu, uptime, memory)
    • Java JMX (Cassandra, Tomcat, JBoss/Wildfly, etc)
    • cAdvisor (Google cAdvisor, for docker container monitoring)
    • File Counter (counts/outputs the number of files in a specified folder)
    • Process Counter (counts/outputs the number of processes running that match a particular regex pattern)
    • Apache HTTP Server (Apache HTTP Server server-status metrics)
    • MongoDB (Mongo status, database, and collection metrics)
    • MySQL (MySQL server metrics)
    • PostgreSQL (PostgreSQL server metrics)
    • Database Querier (Ad-hoc queries for business metrics. Supports MySQL or PostgreSQL)
  • Support for user-created metric collectors (plugins).
    • StatsPoller supports running user-created metric collectors. These plugins are executed by StatsPoller & allow StatsPoller to output the metrics that they collect.
  • Bundled 'user-created metric collectors' include...
    • Windows OS (cpu, disk, iis, memory, network -- collected via PerfMon)
    • SQL Server (collected via PerfMon)
  • Advanced/flexible configuration capabilities
    • Most metric collectors that collect metrics from external services (Mongo, MySQL, Java JMX, etc) can be configured to collect from up to 10,000 independent external services.
    • A single deployment of StatsPoller can be configured to monitor many of JVMs (via JMX), MySQL servers, etc

Why release another metrics poller when there are several good ones already out there?

StatsPoller was originally written at a time when there were few peers on the open-source market. Other tools have emerged then, and many of them are very good at what they do. StatsPoller has served Pearson well & is particularly well suited to monitor Java-based applications, so we figured that we'd just give the market another choice.


Installation

StatsPoller currently supports installation via rpm or via manual installation.

Installation prerequisites

  • Java 1.7 (or newer). Oracle Java is preferred. OpenJDK may also work, but some functionality may be disabled.
  • A valid version of Java must be configured as an environment variable for the user that is running StatsPoller.
  • (Linux) Kernel 2.6 or newer
  • (Linux) Several metric collectors depend on the 'proc' & 'sys' filesystems being mounted somewhere (StatsPoller assumes /proc & /sys by default)

rpm install on Linux (for RedHat, CentOS, etc)

  • sudo rpm -ivh statspoller_version_xyz.rpm

rpm update on Linux (for RedHat, CentOS, etc)

  • sudo rpm -Uvh statspoller_version_xyz.rpm

rpm remove on Linux (for RedHat, CentOS, etc)

  • sudo rpm -e statspoller_version_xyz

deb installation on Linux (for Ubuntu, Mint, etc)

  • deb is being worked on & is targeted at a future StatsPoller release

Installation on Windows

  • A Windows installer is planned for a future release.

Stopping/Starting StatsPoller

RedHat, CentOS, etc -- Versions 6.x & older

Start StatsPoller: sudo /sbin/service statspoller start
Stop StatsPoller: sudo /sbin/service statspoller stop
Restart StatsPoller: sudo /sbin/service statspoller restart
View StatsPoller status: sudo /sbin/service statspoller status

RedHat, CentOS, etc -- Versions 7.x & newer

Start StatsPoller: sudo systemctl start statspoller
Stop StatsPoller: sudo systemctl stop statspoller
Restart StatsPoller: sudo systemctl restart statspoller
View StatsPoller status: sudo systemctl status statspoller


Configuration

After installing StatsPoller, one typically only needs to edit a single configuration file. This file is application.properties. If you're just looking to use the basic metric collectors (ex- Linux OS), then you may only need to configure one of the 'output modules' (Graphite or OpenTSDB). A full listing of StatsPoller's available configuration options, documentation, and examples can be found at example_application.properties. A demo configuration file, with most fields filled out, can be found at example_demo_application.properties.


If you feel constrained by putting a large number of configurations into application.properties, then you can put some of them into configuration files of your own creation @ /conf/optional. Put any file with an extension of .properties into /conf/optional & StatsPoller will read its configuration fields. Please note that several fields can only be set in the main application.properties file. Configurations that can be read from /conf/optional/*.properties include: JMX, ApacheHTTP, FileCounter, ProcessCounter, MySQL, MongoDB, PostgreSQL, cAdvisor, Database Querier, and external metric-collectors.


Thanks to...

About

A metrics collection agent that is compatible with Graphite & OpenTSDB

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages