Skip to content

Deploying a monitoring stack

This example illustrates how to use Cloud Native Computing Foundation (CNCF) monitoring tools with a PingDirectory stack.

The following table lists the actions you might want to take and the available tools.

Purpose Tool
  • Ping Identity Software
Collect Metrics
Display Metrics
Generate Load
  • pingidentity/ldap-sdk-tools
  • pingidentity/apache-jmeter


Much of the generic Prometheus work is taken from the vegasbrianc/prometheus repository.

Before you begin

You must:

  • Complete Get Started to set up your DevOps environment and run a test deployment of the products.
  • Pull our pingidentity-getting-started Git repo to ensure you have the latest sources.

About this task

You will:

  • Deploy the stack
  • Watch the load it generates
  • Learn a bit about using the tools
  • Display the metrics
  • Clean up the stack

PingDirectory produces a wide array of metrics. These metrics can be delivered in StatsD format to a location of your choosing using the StatsD monitoring endpoint for PingDirectory.

See the PingDirectory documentation StatsD Monitoring Endpoint for more information.

Deploy Stack

  1. From pingidentity-devops-getting-started/11-docker-compose/10-monitoring-stack, run:

    docker-compose up -d

    Running this command:

    1. Deploys the Ping Identity software.

    2. Pulls metrics from the Ping Identity software into Prometheus-enabled endpoints (such as, StatsD metrics using statsd_exporter, which formats and hosts the metrics).

    3. Pushes Prometheus to scrape the /metrics endpoint on statsd_exporter.

    4. Generates load to have metrics worth looking at, and push the metrics from the client application (JMeter) to InfluxDB.

    5. Deploys a dashboard in Grafana to visualize the metrics from Prometheus and other tools.

  2. Wait for PingDirectory to become healthy.

    For example:

    docker container ls \
      --filter name=pingdirectory_1 \
      --format 'table {{.Names}}\t{{.Status}}'

    When PingDirectory is healthy, you see something like the following:

    NAMES                                 STATUS
    10-monitoring-stack_pingdirectory_1   Up 2 hours (healthy)

About the Configuration

Because the configuration varies in complexity by use case, this topic focuses on functionality with minimal intervention and what parts you might want to edit.

View Configuration

All relevant configurations are located in your local pingidentity-devops-getting-started/11-docker-compose/10-monitoring-stack/configs directory.

  • The PingDirectory configuration looks like this:

        └── pd.profile
            └── dsconfig
                └── 15-prometheus-stats.dsconfig
  • The baseline server profile

    A single file with two dsconfig commands to create the StatsD monitoring endpoint and define where to push the metrics.

    Traditional profile layering is thought of as getting the profiles from multiple Git repos. However, sending a portion of a profile using the mounted /opt/in volume, and getting the rest of the profile information from a Git repo can still be considered layering.

  • StatsD-Exporter

    The configuration file pingdirectory-statsd-mapping.yml defines which metrics to ingest and how to format them for Prometheus. This file is mounted to a location that is referenced from an argument passed to the startup command from the docker-compose.yaml file. * Prometheus

prometheus.yml defines when and where to look for metrics and any relevant alerting files. * InfluxDB

influxdb.conf prepares InfluxDB to receive metrics from JMeter. * cAdvisor

Specifically for Docker Compose, cAdvisor mounts to the actual Docker processes. * alertmanager

This can be used to set thresholds on metrics, and optionally send notifications. An example threshold is defined in configs/prometheus/alert.rules, and referenced in prometheus.yml. Sending notifications is defined in configs/alertmanager/config.yml. * Grafana

Grafana is a data visualizer. In the Grafana configurations, you'll find:

  • The definition of datasources: datasources/datasource.yml.
  • The definitions of dashboards.

Runtime Data

Grafana and Prometheus runtime data is stored in a Docker volume, so if you start and stop the containers, you won't lose your work. However, it's still a good practice when building dashboards in Grafana to export the dashboard and add the JSON file to the dashboards folder.

How Load is Generated

Auto-Generated Load

Traffic is generated in PingDirectory using our ldap-sdk-tools or apache-jmeter images. When PingDirectory is healthy, these tools run as individual services based on the use case being implemented.

You can view the logs of any of these services directly with docker-compose logs -f <service_name>. For example:

docker-compose logs -f searchrate

Generating Load

  • Option 1

    The most common way to generate load is by using the pingidentity/apache-jmeter image. To be effective with this tool, see JMeter usage.

  • Option 2

    To run another test using the ldap-sdk-tools utility, see ldap-sdk-tools.

  • Option 3

    Use tools available on the PingDirectory server:

    1. Shell into the PingDirectory server:

      docker container exec -it 10-monitoring-stack_pingdirectory_1 sh
    2. Run the modrate tool. Enter:

      modrate \
      --hostname localhost --port 1636 --bindDN cn=administrator --bindPassword 2FederateM0re \
      --entryDN "uid=user.[0-4],ou=people,dc=example,dc=com" \
      --useSSL --trustAll \
      --attribute description --valueLength 12 --numThreads 10 --ratePerSecond 20

      modrate runs in the foreground in the container, so be ready to open another terminal if necessary to avoid stopping modrate.

      modrate produces output like the following:

      > modrate \
      >   --hostname localhost --port 1636 --bindDN cn=administrator --bindPassword 2FederateM0re \
      >   --entryDN "uid=user.[0-4],ou=people,dc=example,dc=com" \
      >   --useSSL --trustAll \
      >   --attribute description --valueLength 12 --numThreads 10 --ratePerSecond 20
            Recent       Recent       Recent      Overall      Overall
          Mods/Sec   Avg Dur ms   Errors/Sec     Mods/Sec   Avg Dur ms
      ------------ ------------ ------------ ------------ ------------
            19.998        5.880        0.000       19.998        5.880
            19.998        4.214        0.000       19.998        5.047
            19.999        3.793        0.000       19.998        4.629
            20.001        3.608        0.000       19.999        4.374

      You also can return to the terminal running modrate after you change the modrate parameter settings to see the effect in Grafana.

Display Metrics

Metrics are displayed at these URLs:

Tool Description Connection Details
Grafana Data displayed in dashboards
PingDirectory Raw StatsD data
cAdvisor Container resource metrics URL: http://localhost:8080
node-exporter Raw node metrics URL: http://localhost:9100/metrics
alertmanager Alerts displayed URL: http://localhost:9093/#/alerts
Prometheus Query collected data URL: https://localhost:9090

The Grafana dashboards correspond to the dashboard definitions in configs/grafana/provisioning/dashboards.

In Grafana, go to Dashboards -> Manage. The pre-populated dashboards with your live load results are displayed.

Finishing Up

To bring down the stack and remove the data stored in the Docker volumes, enter:

docker-compose down
docker volume rm 10-monitoring-stack_grafana_data
docker volume rm 10-monitoring-stack_prometheus_data