Learn streaming data with Lenses

Here is a glossary with all the basics in Kafka and data streaming engineering. Want to learn more? Join our Community Slack channel.

BFGIJMNOPQRSTUVWXYZ

A

  • Agent (Lenses Agent)

    A headless application deployed within a Kafka environment that connects back to Lenses HQ. It acts as a proxy for operations, monitoring, and data access.

  • Agentic Engineering

    A paradigm shift in software development where AI agents are orchestrated with engineering rigor to autonomously build, operate, and govern data systems under human oversight.

  • AI Agent

    An autonomous assistant, such as an SRE Agent, that uses tools and context (often via MCP) to perform complex tasks like troubleshooting or pipeline creation.

  • Apache Kafka

    A distributed event store and stream-processing platform used as a real-time data "central nervous system" for modern enterprises.

C

  • CIMD (Client ID Metadata Document)

    A web-based registration model for MCP clients where the identity is a URL hosting a JSON document, allowing for verified, domain-linked security.

  • Consumer Group

    A coordinated group of consumers that ingest data from topics, sharing the workload and tracking progress via offsets.

D

  • Data Lineage

    A visual representation of the data lifecycle, showing how events flow from producers through topics and processors to final sinks.

  • Data Masking

    A governance policy that obscures sensitive PII fields (like emails or phone numbers) in real-time before agents or unauthorized users see them.

  • DCR (Dynamic Client Registration)

    An OAuth flow that allows AI clients to programmatically register themselves with an authorization server at runtime.

E

  • Environment

    A logical grouping in Lenses representing an individual Kafka cluster and its supporting services (e.g., Schema Registry, Connect).

  • Exactly-Once Semantics (EOS)

    A guarantee that data is processed exactly once, even if failures occur, preventing duplicates and data loss.

H

  • HQ (Control Plane)

    The central Lenses portal where users and agents interact with all Kafka environments and manage governance globally.

K

  • K2K Replicator

    A Kubernetes-native, vendor-neutral tool for moving data, schemas, and offsets between any flavor of Kafka.

  • Kafka Cluster

    A set of one or more Kafka brokers working together to provide scalability and fault tolerance.

  • Kafka Connections

    Scoped access paths in Lenses 6.1+ that allow developers to deploy applications without handling or seeing cluster credentials directly.

  • Kafka Connector

    A pre-built component used to integrate Kafka with external systems (e.g., S3, Snowflake) to move data in (Source) or out (Sink).

  • Kafka Consumer

    An application or agent that reads and processes data from Kafka topics.

L

  • Lag

    The gap between the latest message produced to a topic and the last message a consumer has successfully read.