7. Kafka Monitoring UI

The Lenses Kafka Monitoring tools are a set of pre-defined templates, that use

  • A Time Series database (Prometheus)
  • Custom JMX exporters
  • Built in domain intelligence about operating Kafka with confidence in production

7.1. Kafka Cluster Metrics

A 360-degree of the key metrics of your Kafka cluster curated into a single template, that allows to time travel between the past 60 days (by default) of key metrics, and pro-actively receive alerts and notifications when your streaming platform is under pressure, or signals of partial failures appear.

../../_images/kafka-cluster-metrics-overview.png

7.2. Consumer Producer Metrics

All Kafka Consumer or Producer dashboard to include all metrics for Kafka brokers, Zookeeper, Schema Registry, Connect Distributed, REST Proxy, Lenses and any other JVM application that is connected to Lenses Monitoring.

../../_images/kafka-producer-consumer-metrics-UI.png

7.3. Hard Disk Usage Metrics

A dashboard to display the approximate metrics about the size (in bytes) of your topics. It is useful for planning disk capacity and having an overview of each topic’s size. The “Data Stored per Broker” graph can be used to detect storage imbalanaces between brokers.

../../_images/kafka-hard-disk-usage-metrics.png

7.4. Client Application Monitoring

Operational metrics from your JVM-based Kafka applications. You can use it to monitor performance and usage of system resources in order to detect issues early. Full access to how JVM apps and the Garbage Collector behaves, as well as open file descriptors, and other critical aspects of your own applications.

../../_images/kafka-jvm-client-application-monitoring.png

7.5. Messages Per Topic

A simple dashboard to display the messages per topic. It highlight when segment deletion occurs as per the file size based or time based retention policies.

../../_images/kafka-messages-per-topic.png

7.6. Features

The Monitoring Lens from Landoop, adds a set of additional features including:

  • HTTPS / Certificate support
  • Predictive algorithms to alert before a failure occurs
  • Alert integration with Slack, PagerDuty, Email and other integrations
  • LDAP integration
../../_images/kafka-monitoring-alerting.png