Configuring An Open-Source Observability Stack>
– Michael Levan
The process of setting up an open source observability stack generally involves deciding which tools to use, setting up cloud infrastructure, monitoring setup (configuration of metrics/logs/tracing), making sure telemetry data makes its way from the infrastructure to the stack, and alerting/notification setup.
1. Decide which tools to use: Open source observability tools have become very powerful and easy to use.
Popular choices include Prometheus, Elasticsearch, Grafana, Zipkin, Jaeger, and Kibana.
These tools offer a wide range of capabilities for different types of observability, including metric collection, log storage, visualization, alerting, and more.
Depending on the particular requirements of your stack, you may wish to select tools that specialize in a particular type of observability.
2. Set up cloud infrastructure: Once you’ve decided on the tools you wish to use, you’ll need to set up cloud infrastructure to support the stack.
This typically involves provisioning servers and other resources for your stack, including storage for metrics and logs, as well as for alerting and notification purposes.
3. Monitoring setup: The next step is to configure the various tools in your stack to send metrics and logs from your infrastructure to the stack.
You’ll likely need to use monitoring tools such as Prometheus and Grafana for metrics, and ELK (Elasticsearch, Logstash, and Kibana) for logs.
4. Telemetry data flow: Once the monitoring setup is complete, you will need to configure the data flow between your infrastructure and the observability stack.
This usually involves setting up endpoints on your infrastructure to emit metrics and log data, as well as configuring the tools in the stack to receive the data.
5. Alerting and notification setup: Finally, you’ll need to configure your stack to send alerts and notifications when thresholds are breached or other criteria are met.
This will typically involve setting up webhooks with external providers such as PagerDuty, or options offered by the observability tools.
Link: https://packetpushers.net/configuring-an-open-source-observability-stack/