Skip to content

Latest commit

 

History

History

monitoring

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
docker-image
docker-image
 
 
grafana
grafana
 
 
scripts
scripts
 
 
README.adoc
README.adoc
 
 
docker-compose.yml
docker-compose.yml
 
 

README.adoc

Monitoring

This directory contains the monitoring configuration for XTDB, with the following structure:

  • docker-compose.yml - the docker-compose file for setting up the development monitoring stack.

  • scripts/ - contains helper scripts for use with the monitoring stack.

  • docker-image/ - contains setup (a gradle project, logback and dockerfile) for building the XTDB monitoring docker image.

  • grafana/: contains all of the necessary grafana-otel setup config for scraping/monitoring XTDB

    • grafana/dashboards/: contains the XTDB Cluster Monitoring and XTDB Node Debugging dashboards.

User Guide

For users of XTDB, we assume that you already have a running Grafana instance and Prometheus setup to scrape your XTDB cluster/node (ie, running on Kubernetes).

If the above is true, you can import the dashboards provided in the grafana/dashboards/ directory into your Grafana instance.

  • Navigate to the Dashboards → New → New dashboard.

  • Click Import dashboard.

  • Upload dashboard JSON file → Upload the desired dashboard from the grafana/dashboards/ directory.

    • Select the Prometheus data source.

    • Click Import.

There are two dashboards provided:

  • XTDB Cluster Monitoring - this dashboard provides an overview of the health of your XTDB cluster.

  • XTDB Node Debugging - this dashboard provides a more detailed view of the health of individual XTDB nodes.

Developer Guide

Starting the development monitoring stack

To set up the monitoring stack, we first need to build the xtdb-monitoring docker image.

Under the build/monitoring directory, we have a gradle project & docker image to build a custom xtdb-monitoring image, that we use within the docker-compose. The contents of this directory can be edited however you see fit, and the image can be built with the following helper script:

./scripts/build-xtdb-monitoring-image.sh

Then, to run the monitoring stack itself, from the root of the directory, run:

docker-compose up -d

This will start the following services:

  • Two XTDB nodes (using the standalone image)

  • Grafana OTEL

    • This is setup with a basic prometheus setup to scrape the XTDB node.

Updating the XTDB nodes

If you want to update the XTDB image after making changes, you can re-run ./scripts/build-xtdb-image.sh and then restart the XTDB nodes in the monitoring stack:

docker-compose restart xtdb-node-1
docker-compose restart xtdb-node-2

Working with Grafana

Grafana is exposed on http://localhost:3004 by default - so one can navigate to this URL to access the Grafana UI.

Prometheus is already running and scraping the XTDB node, so we can start working on dashboards.

As we are exporting dashboards for public release, we cannot automatically import them into Grafana. However, you can manually import the dashboards by following the steps below:

  • Navigate to the Dashboards → New → New dashboard.

  • Click Import dashboard.

  • Upload dashboard JSON file → Upload the desired dashboard from the grafana/dashboards/ directory.

    • Select the Prometheus data source.

    • Click Import.

Saving updated Dashboards

Any changes/additions made to either of the dashboards should be exported and committed for Users to import.

To export a dashboard:

  • Click Share.

  • Click on the Export tab.

  • Ensure Export for sharing externally is selected.

  • Click Save to file.

Once exported, replace the relevant file within the grafana/dashboards/ directory, and commit the changes.