Skip to content

Deploying an Elasticsearch SIEM Stack

This example deploys a PingFederate, PingAccess, and PingDirectory stack with Elasticsearch infrastructure built in for visualizing traffic and other security or log data. The architecture looks like this:

alt text

  • Threat intel and TOR Endpoints are provided by AlienVault and the TOR Network Endpoint List.

  • Threat feeds are updated on an interval with setting an environment variable in docker-compose.yaml.

Warning: This stack is not intended for production environments.

Before you begin

You must:

  • Complete Get Started to set up your DevOps environment and run a test deployment of the products.

  • For most Linux distributions (local or on a platform), increase the vm.max_map_count setting to support the necessary heap size . Enter:

    sudo sysctl -w vm.max_map_count=262144
    

Your Linux machine needs at least 12 GB of RAM for Docker to run this stack.

  • For Apple MacOS or Microsoft Windows machines, ensure the Docker Resources is set to a minimum 10 GB of RAM. If you don't, the containers will crash.

  • For Amazon Web Services (AWS), use a M5.XL or M5a.XL VPC. 16 GB RAM is required with at least 50 GBb of storage.

Optional

  • If you're using Slack, you can generate a Slack Webhook URL from the Slack Admin for alerting: https://api.slack.com/messaging/webhooks.

Installing setup

  1. From the pingidentity-devops-getting-started directory, pull the repo to ensure that you have current files:

    git pull
    
  2. Go to the pingidentity-devops-getting-started/11-docker-compose/11-siem-stack/ directory.

  3. Create a siem.env file in the 11-siem-stack directory and copy the following entries into the siem.env file:

    COMPOSE_PROJECT_NAME=es
    ELASTIC_VERSION=7.6.1
    
    ELASTIC_PASSWORD=2FederateM0re
    ES_ADMIN_PD_USER_PASS=FederateTheB3st!
    
    PING_IDENTITY_DEVOPS_USER=<your-username>
    PING_IDENTITY_DEVOPS_KEY=<your-key>
    

Deploying the stack

  1. From the pingidentity-devops-getting-started/11-docker-compose/11-siem-stack/ directory, start the stack:

    docker-compose up -d
    

    Monitor the container startup using one of these commands:

    docker-compose ps
    
    docker-compose logs -f
    
  2. (Optional) If you're using Slack, and you've already created your Webhook URL (see the optional prerequisite above), you can run the Slack configuration script to configure slack alerts:

    ./config_slack_alerts
    

    The script prompts for your Webhook URL and Elasticsearch password.

    • The Webhook URL updates the destination for your alerts within Slack.
    • The password is used to push watchers into Elasticsearch.

    You don't need to provide your Webhook URL in the future. If you don't provide it, it simply will not update it.

    You can re-run this script any time. This will update and push new watchers you create from the ./elasticsearch-siem/watchers folder.

Post-Deployment

When PingDirectory is up and healthy:

  • Kibana console:

  • URL: https://localhost:5601/.

  • User name: es_admin or elastic (local user).
  • Password: FederateTheB3st! (the ES_ADMIN_PD_USER_PASS value in the siem.env file you created).

  • Kibana saved objects

You can load the saved objects by going to "Saved Objects" under the Kibana settings and exporting all. The exported file is saved in ./elasticsearch-siem/kibana_config/kib_base.ndjson.

  • Elasticsearch templates for indexes

You can find index mappings and config in the ./elasticsearch-siem/index_templates directory. The scripts will load the template or templates when the cluster state is green.

  • Logstash pipeline

  • TOR Enrichment

  • Threat Intel (Alien Vault Provided)
  • GEO IP Lookup
  • GEO Distance Query (template driven)
  • Data Parsing
  • The Logstash pipeline is stored in the directory structure. It includes parsers for all Ping Identity log sources.

Cleaning Up

There are persistent volumes used for Elasticsearch data and certificates, so you'll also need to clear the volumes when you bring the stack down. Enter:

docker-compose down
docker volume prune

Dashboard Examples

PingFederate Threat Intel Dashboard

alt text

Ping Identity SIEM Dashboard

alt text

PingFederate Dashboard

Audit and System logs are delivered (set to Debug by default). For Log4J, PingFederate sends logs on two different Syslog ports using a custom mapping.

alt text

PingAccess Dashboard

Audit and System logs are delivered (set to Debug by default). For Log4J, PingAccess sends logs on two different Syslog ports using a custom mapping.

alt text

PingDirectory Dashboard

Audit logs are being delivered. There are two containers that produce load. These are disabled by default. You can uncomment these entries in the docker-compose.yaml file to use them:

  • authrate_ok
  • authrate_ko

For Log4J, PingDirectory sends logs on one Syslog port using a custom mapping.

alt text

Included Slack Alerts

You can customized the following alerts through Watchers:

  • User authenticates over 1200km away within a 6-hour period.
  • User authenticates successfully from TOR through PingFederate (potential credential theft).
  • User authenticates successfully from Known Malicious IP through PingFederate (potential credential theft).
  • Account Lockout detected through PingFederate (potential brute force).
  • Likely SAML signature modifications (forced tampering with authentication protocols).

Slack Alert Examples (not all are shown)

The following image shows Low / Medium / High alert examples:

alt text