Features Overview

Lenses is a Streaming Data Management Platform for Apache Kafka. It delivers the core elements of Kafka along with a web user interface and enterprise capabilities. This leads to the improved user experience for the engineering teams, business users, data scientists, and administrators while using Apache Kafka.

Flexibility View and store any type of data. Manipulate it using batch processing, or interactive SQL
Integration Connect to popular data stores using one of the 25+ open source stream-reactor Kafka connectors
Security Process and control sensitive data with SSL, LDAP and Kerberos support
Scalability Provision SQL Processors and scale them to suit your requirements with Kubernetes and other scalable modes
High Availability Perform real-time operational business tasks with confidence
Monitoring Provides infrastructure and application topology KPIs
Compatibility Leverage your existing IT infrastructure
Auditing Role-based access and auditing for compliance with regulations

Explore data in motion

Lenses provides a rich Web Interface and the associated REST and WebSocket endpoints all working together enabling you to explore Kafka topics in real-time or browse, search and filter historical data with full access to partition/offset/timestamp information.

../../_images/lenses-explore-data-in-motion.png

SQL Processors

Create boundless data queries with SQL processors to aggregate, join and/or transform streams. Via the UI you can visualize, monitor the topologies or even scale out processing. Furthermore, the execution plan viewer allows you to optimize for performance.

../../_images/lenses-topologies.png

Stream Topologies

With Lenses you can now build and operate with confidence complex streaming topologies, such as ETL data pipelines, and stream processing and analytics combining multiple Connectors, Processors and Topics with full data lineage.

Lenses is able to visualize not only the topologies of LSQL processors but is also able to display a global, high level view of your landscape, including topics, connectors and processors that comprise your enterprise pipelines.

../../_images/streaming-topologies.png

Avro Support

Full support for Avro messages, including decimal type (for financial institutions), is available in Lenses SQL Engine. Lenses platform integrates with your schema registry and provides a rich user interface to create, edit and track schemas.

../../_images/lenses-avro-messages.png

Kafka Connect

With Lenses you can manage multiple connect clusters, build streaming ETL data pipelines with ease while monitoring connectors and their tasks. Lenses contains the largest collection of Apache Kafka connectors with Lenses SQL support, for all major data sources and sinks including Cassandra, Elastic, InfluxDB, Azure CosmosDB, MQTT, JMS and more.

../../_images/lenses-kafka-connectors1.png

See how British Gas streams 4+ billion entries every day into Elastic-Search with a simple SQL statement:

# Upsert into elastic with auto create and index suffix
UPSERT INTO index SELECT id, string_field FROM topic PK id AUTOCREATE WITHINDEXSUFFIX=_{YYYY-MM-dd} WITHDOCTYPE=lenses

Consumers Lag

Monitor Kafka consumer lag in real time and manage the consumer offsets. Set up alerts to make sure you can scale or take the appropriate action.

../../_images/lenses-consumers-lag.png

Monitor Services

Monitor the core services and infrastructure via JMX for Kafka Brokers, Zookeeper, Schema Registry and Connect. Prevent and react to potential issues and get insights into your clusters’ performance.

../../_images/lenses-monitoring.png ../../_images/lenses-offline-partitions.png

Alerts

Set up alerting and prevent issues earlier.

../../_images/lenses-alerts.png

Auditing

Track all changes to your Kafka cluster: topic creation, configuration modification, deletion (for Schemas, Connectors, and Processors). A single point of reference for Who did what and when across your data streaming pipeline with LDAP support.

../../_images/lenses-audits.png

Security

Lenses supports basic authentication and LDAP. Additionally, different operator roles are provided to restrict user actions, giving operators a way to provide security on Kafka. Users can manage topic ACLs (Access Control Lists) to restrict readers and writers of data.

../../_images/lenses-acls.png

Hadoop integration

Upgrade your Cloudera CDH cluster with data streaming capabilities using CSD packages, (Cloudera Custom Service Descriptor) to deploy, configure, scale and manage your Apache Kafka clusters. Fully supports Kafka Connect and allows leveraging Cloudera Manager integration for monitoring.

../../_images/screen-csd.png

Lenses SQL Engine

Lenses SQL is a multi-purpose engine, it fully supports Avro and Json payload types and can execute both batch and real-time streaming SQL.

../../_images/lenses-sql-engine.jpg