Metadata-Version: 2.0
Name: datadog-kafka-consumer
Version: 1.4.0
Summary: The Kafka Consumer check
Home-page: https://github.com/DataDog/integrations-core
Author: Datadog
Author-email: packages@datadoghq.com
License: BSD
Keywords: datadog agent kafka_consumer check
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Topic :: System :: Monitoring
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Requires-Dist: kafka-python (==1.3.4)
Requires-Dist: kazoo (==2.4.0)
Requires-Dist: datadog-checks-base

# Agent Check: Kafka Consumer

## Overview

This Agent check only collects metrics for message offsets. If you want to collect JMX metrics from the Kafka brokers or Java-based consumers/producers, see the kafka check.

This check fetches the highwater offsets from the Kafka brokers, consumer offsets that are stored in kafka or zookeeper (for old-style consumers), and the calculated consumer lag (which is the difference between the broker offset and the consumer offset).

## Setup
### Installation

The Agent's Kafka consumer check is packaged with the Agent, so simply [install the Agent][101] on your Kafka nodes.

### Configuration

Create a `kafka_consumer.yaml` file using [this sample configuration file][102] as an example. Then [restart the Datadog Agent][108] to start sending metrics to Datadog.

### Validation

[Run the Agent's `status` subcommand][103] and look for `kafka_consumer` under the Checks section.

## Data Collected
### Metrics
See [metadata.csv][104] for a list of metrics provided by this check.

### Events

`consumer_lag`:

The Datadog Agent emits an event when the value of the `consumer_lag` metric goes below 0, tagging it with `topic`,
`partition` and `consumer_group`.

### Service Checks
The Kafka-consumer check does not include any service check at this time.

## Troubleshooting
### Specifying a non existent partition in your kafka_Consumer.yaml file
If you get this error in your info.log:
```
instance - #0 [Error]: ''
```

Specify the specific partition of your environment for your topic in your kafka_Consumer.yaml file:
```
#my_topic [0, 1, 4, 12]
```

## Further Reading

## Further Reading

* [Monitoring Kafka performance metrics][105]
* [Collecting Kafka performance metrics][106]
* [Monitoring Kafka with Datadog][107]


[101]: https://app.datadoghq.com/account/settings#agent
[102]: https://github.com/DataDog/integrations-core/blob/master/kafka_consumer/conf.yaml.example
[103]: https://docs.datadoghq.com/agent/faq/agent-commands/#agent-status-and-information
[104]: https://github.com/DataDog/integrations-core/blob/master/kafka_consumer/metadata.csv
[105]: https://www.datadoghq.com/blog/monitoring-kafka-performance-metrics/
[106]: https://www.datadoghq.com/blog/collecting-kafka-performance-metrics/
[107]: https://www.datadoghq.com/blog/monitor-kafka-with-datadog/
[108]: https://docs.datadoghq.com/agent/faq/agent-commands/#start-stop-restart-the-agent


