Confluent CCDAK Confluent Certified Developer for Apache Kafka Certification Examination Exam Practice Test

For proper delivery handling with acks=all:

Use callback to log or act on success/failure.

Use try/catch to handle synchronous exceptions like serialization errors or network failures.

From Kafka Producer Documentation:

“Errors can be caught either via the returned Future or via the callback interface. For fatal errors, use a try/catch block around the send call.”

Option B is incorrect because send() returns a Future, not RecordMetadata directly.

Option D is invalid — ProducerRecord has no method called status().

SMTs (Single Message Transforms) are lightweight transformations applied to individual messages as they pass through Kafka Connect.

Chaining SMTs: You can apply multiple SMTs in sequence by defining them in order in the connector config.

Masking or modifying fields is a common use case (e.g., redacting sensitive data).

From Kafka Connect Documentation:

“Single Message Transforms (SMTs) are applied to individual messages. You can chain multiple SMTs together.”

SMTs do not perform joins (B is incorrect).

Converters and SMTs are separate concerns (D is incorrect).

With acks=all and min.insync.replicas=2, Kafka requires at least two in-sync replicas to acknowledge a write. If only one broker is alive, the condition fails, and NotEnoughReplicasException is thrown by the producer.

From Kafka Producer Exception Docs:

“NotEnoughReplicasException is thrown when the number of in-sync replicas is insufficient to satisfy acks=all with min.insync.replicas.”

NetworkException is generic and not raised here.

NotCoordinatorException is related to consumer group coordination.

NotLeaderForPartitionException is unrelated unless accessing an unassigned leader.

Unit Testing → MockProducer

Integration Testing → Testcontainers

Performance Testing → Trogdor

Mock Data Generation → Connect Datagen

MockProducer: Simulates a Kafka producer in unit tests (no real broker interaction).

Testcontainers: Spawns Kafka in Docker for real environment testing.

Trogdor: Kafka’s built-in performance load testing framework.

Connect Datagen: Creates sample source records for test and demo purposes.

From Kafka Developer Tools Guide:

“Kafka developers commonly use MockProducer for unit tests, Testcontainers for integration, and Trogdor for performance tests.”

Correct Matches

path.format→Only valid for some Source Connectors

value.converter→Valid for all Source Connectors

input.path→Only valid for some Source Connectors

errors.retry.timeout→Valid for all Source Connectors

tasks.max→Valid for all Source Connectors

path.formatThis is a connector-specific configuration, commonly used by file-based or storage-based source connectors (for example, HDFS, S3, or FilePulse connectors). It isnot part of the Kafka Connect framework itself, so it is only valid for some source connectors.

value.converterThis is aworker-level configurationthat defines how record values are converted (JSON, Avro, Protobuf, etc.). It applies toall connectorsrunning on the worker unless overridden at the connector level.

input.pathThis configuration is specific to file-based source connectors (e.g., FileStreamSourceConnector). It isnot universally supported, so it is only valid for some source connectors.

errors.retry.timeoutThis is part of Kafka Connect’serror handling framework, introduced to support retries and tolerance of transient errors. It is ageneric Connect configurationand valid for all connectors.

tasks.maxThis is amandatory, generic connector configurationthat controls the maximum number of tasks a connector may create. It is valid forall source (and sink) connectors.

According to the Apache Kafka producer documentation, serialization is configured independently for message keys and values. The key and value can use different serializers depending on their data types.

In this scenario, the message key is a string, so the producer must be configured with org.apache.kafka.common.serialization.StringSerializer for the key. The value contains JSON, so a suitable JSON serializer must be configured for the value, such as a custom JSON serializer, a library-based serializer, or a byte-array serializer after converting the JSON to bytes.

Option B correctly reflects this separation by specifying a string serializer for the key and a JSON serializer for the value. Kafka does not automatically infer serializers, and setting a serializer to null is invalid. Additionally, there is no built-in “JSON” serializer for keys, making Option D incorrect.

Therefore, the correct approach—fully aligned with Kafka’s producer configuration model—is to explicitly configure a StringSerializer for the key and a JSON-capable serializer for the value.

=========================

The ConsumerRebalanceListener lets you handle partition assignments and revocations during rebalances. This is critical for managing offsets, stateful processing, or external transactions.

From Kafka Consumer Rebalance Docs:

“Implementing ConsumerRebalanceListener allows your application to take action before and after partitions are reassigned.”

A is true: It lets your app react when partitions assigned to the consumer change.

B, C, and D are unrelated to consumer rebalancing directly.

According to the official Apache Kafka documentation for Kafka Streams, exactly-once semantics (EOS) are controlled using the processing.guarantee configuration. To ensure that each input record is processed once and only once, and that the results are written atomically to the output topic, Kafka Streams provides built-in EOS support.

The correct and recommended configuration is processing.guarantee=exactly_once_v2, which is the default EOS implementation starting from Kafka 3.0. This mode uses Kafka’s transactional APIs to atomically commit consumer offsets and produce output records within a single transaction. As documented, this prevents duplicates even in the presence of failures, retries, or application restarts.

Option A is incorrect because log compaction does not guarantee exactly-once processing; it only retains the latest record per key. Option B is insufficient because enable.idempotence alone does not handle consumer offset commits or state store consistency in Kafka Streams. Option C is invalid because no such configuration exists in Kafka Streams.

Therefore, setting processing.guarantee=exactly_once_v2 is the only fully correct and officially supported solution for achieving EOS in Kafka Streams applications.

This scenario describes vertical scaling, as defined in the Apache Kafka Streams documentation and general distributed systems terminology. Vertical scaling refers to adding more resources (CPU, memory, or disk) to a single machine to increase processing capacity.

In this example, the Kafka Streams application is moved from a server with four CPU threads to one with ten CPU threads, increasing the available compute resources without increasing the number of application instances. Kafka Streams can take advantage of additional CPU threads through its num.stream.threads configuration, allowing more partitions to be processed in parallel within the same application instance.

Horizontal scaling (also known as scaling out) would involve adding more application instances across multiple servers, not upgrading a single server. Option D is therefore incorrect, and Option C is not a recognized scaling term.

Thus, this example clearly illustrates vertical scaling, where performance is improved by increasing the capacity of a single node.

As documented in the official Apache Kafka Connect framework documentation, Kafka Connect workers form a logical cluster by using a Kafka consumer group. The configuration property that identifies this cluster is group.id.

All Kafka Connect workers configured with the same group.id automatically join the same Connect cluster. Kafka’s group coordination protocol is then used to distribute connector and task configurations among the workers, providing fault tolerance, scalability, and automatic rebalancing.

Option A (cluster.id) is a Kafka internal identifier used by brokers and is not configurable for Kafka Connect workers. Option B (worker.id) does not exist as a valid Kafka Connect configuration. Option D (connector.id) identifies individual connectors within a Connect cluster, not the worker cluster itself.

Therefore, the group.id property is the correct and officially documented configuration that determines which Kafka Connect cluster a worker belongs to.

When using Kafka Connect with sink connectors (like S3 Sink), each task is assigned one or more topic partitions. The total number of partitions (6) is evenly distributed across the available tasks (2). Thus, each task will handle 3 partitions.

From Kafka Connect Documentation:

“Kafka Connect divides the topic partitions among available tasks. For example, a topic with six partitions and two tasks results in each task handling three partitions.”

According to the official Apache Kafka Streams documentation, Kafka Streams relies on internal topics (such as changelog topics and repartition topics) to store state, task assignments, and application metadata. These topics are essential for fault tolerance, rebalancing, and state recovery.

When auto-topic creation is disabled, Kafka Streams cannot automatically create the required internal topics unless they already exist. In this situation, the Streams application will fail during startup with a fatal, non-retriable exception, indicating that the necessary internal topics could not be created.

Kafka Streams does not pause or wait for manual topic creation, nor does it operate without storing metadata. Instead, it explicitly requires these topics to exist or be creatable. While administrators can manually pre-create the required internal topics with the correct configuration, failure to do so causes the application to terminate.

Therefore, the correct and documented behavior is that the Kafka Streams application terminates with a non-retriable exception when auto-topic creation is disabled and required internal topics are missing.

In the Kafka Streams API, operations are classified as stateless or stateful based on whether they require maintaining local state stores. According to the official Kafka Streams documentation, stateless operations process each record independently, without storing or accessing prior records.

Filter is a stateless operation because it evaluates each record individually and decides whether to pass it downstream. It does not require any state or historical context.

GroupBy is also considered stateless because it merely repartitions the stream by assigning a new key and forwarding records to downstream processors. While it triggers the creation of an internal repartition topic, the GroupBy operation itself does not maintain a state store.

In contrast, Reduce is a stateful operation because it aggregates records over time and requires maintaining intermediate results in a state store. Similarly, Join operations are stateful because Kafka Streams must buffer and store records from one or both input streams or tables to perform the join within a defined time window.

Thus, the correct stateless operations are Filter and GroupBy, as documented in the Kafka Streams developer guide.

The StickyAssignor tries to minimize partition movement by preserving existing assignments as much as possible while still achieving a balanced assignment. This improves consumer stability and reduces rebalances.

From the Kafka Consumer Assignor Documentation:

“The StickyAssignor attempts to preserve as many existing assignments as possible, which helps minimize partition movement between rebalances.”

RoundRobinAssignor focuses on even distribution, not stability.

RangeAssignor groups partitions by topic and assigns them consecutively, but can lead to imbalances.

PartitionAssignor is an abstract base class, not an assignor used directly.

RetriableException is a subclass of KafkaException, indicating transient issues that might succeed if retried. Examples include:

LeaderNotAvailableException: Happens when metadata is not yet propagated or leader election is in progress.

NotEnoughReplicasException: Happens when the number of in-sync replicas is insufficient. This can be transient if replicas come back online.

From the Apache Kafka Java Client documentation:

"These exceptions (like LeaderNotAvailableException, NotEnoughReplicasException) are transient and the client will retry them automatically depending on retry configuration."

RecordTooLargeException and AuthorizationException are non-retriable as they indicate client-side or permission errors.

Comprehensive and Detailed Explanation From Exact Extract:

✅ C. Consumers can consume both committed and uncommitted transactions.By default, Kafka consumers only read committed messages if they are configured with isolation.level=read_committed. However, if configured as read_uncommitted, they can also consume uncommitted (potentially aborted) transactional messages.

From Kafka Documentation:

"The isolation.level setting controls whether the consumer will read only committed messages or all messages, including uncommitted messages from ongoing or aborted transactions."

✅ D. Information about producers and their transactions is stored in the _transaction_state topic.Kafka uses an internal topic named __transaction_state to maintain metadata about producer transactions. This topic is essential for tracking the transaction lifecycle, fencing, and recovery.

From Kafka Internals:

“Kafka stores the state of active and completed transactions in an internal topic called __transaction_state.”

A 409 Conflict response in Kafka Connect REST API usually means:

The connector name provided already exists (i.e., a connector with that name is already deployed).

The Kafka Connect cluster is temporarily unavailable to accept new tasks due to a rebalance in progress.

From the Confluent Developer documentation and Kafka Connect REST API spec:

“409 – Conflict. The connector name is already in use or the worker group is in the process of rebalancing and cannot complete the request at this time.”

Option A is incorrect because invalid configurations return 400 Bad Request, not 409.

Option B is invalid because Kafka Connect does not have a strict concept of "capacity" that prevents connector creation with a 409.

1st: Place the connector's JAR file in the directory specified by plugin.path

2nd: Restart the Kafka Connect cluster

3rd: Verify using REST API (/connector-plugins)

4th: Configure the connector

5th: (Repeat of 4th, duplicate step)

1st → Place the connector’s JAR file in the directory specified by the plugin.path configuration.

2nd → Restart the Kafka Connect cluster.

3rd → Verify that the connector is installed by listing all available connectors using the Kafka Connect REST API (/connector-plugins).

4th → Configure the connector using a JSON or properties file with the necessary settings.

5th → Configure the connector using a JSON or properties file with the necessary settings.

Kafka Connect requires that custom connectors be placed in the directory defined by plugin.path. After restarting the cluster, you can use the REST API to confirm availability and then deploy the connector configuration.

From Kafka Connect Documentation:

“After placing the JAR in the plugin.path, you must restart the Connect cluster to pick it up. Use the /connector-plugins REST endpoint to verify.”

The duplication of configuration is an error in the question options and should occur only once.

Log compaction ensures that Kafka retains at least the latest value per key in a topic. Compaction happens in the background and removes older records with the same key in inactive log segments, not immediately.

From the Kafka Documentation > Log Compaction:

“Kafka guarantees that the last message for each key will be retained in the log after compaction, even if earlier messages with the same key are deleted.”

So, D is correct. A is incorrect because compaction does not overwrite; it's a background process. B is incorrect—deletion is not immediate. C incorrectly suggests client-side hashmap behavior.

Using assign() gives manual control over partition assignment. In this mode:

B is true: The application chooses partitions directly without involving the group coordinator.

C is true: Since no group coordination occurs, no rebalancing happens if a consumer dies.

From Kafka Consumer API Docs:

“assign() manually assigns a list of partitions to this consumer. No coordination takes place and no rebalancing is triggered.”

A is false: subscribe() and assign() are mutually exclusive.

D is incorrect: Partitions don't need to be equal across topics.

Kafka Connect source connectors use the offset.storage.topic configuration parameter to define where to store offsets that track the connector's position in the source system (e.g., database, file, etc.).

From Kafka Connect Worker Configuration:

“offset.storage.topic specifies the topic in Kafka where the connector stores offsets for each source partition.”

This allows Kafka Connect to resume from the correct position on restart.

The other options (B–D) are not valid Kafka configs.

fetch-rate: Measures how frequently consumers fetch data — critical for monitoring throughput.

incoming-byte-rate: Shows how much data is being written to the broker per second — a key producer-side metric.

From Kafka JMX Metrics Guide:

“fetch-rate, incoming-byte-rate, request-latency are key performance metrics for producers and consumers.”

B, C, E, and F are application-layer business metrics, not Kafka performance metrics.

The official Apache Kafka Streams developer documentation clearly states that TopologyTestDriver is the recommended and supported tool for unit testing Kafka Streams applications without requiring a running Kafka cluster. It is specifically designed to test complex topologies consisting of multiple sources, processors, state stores, sinks, and even sub-topologies.

TopologyTestDriver allows developers to instantiate the Kafka Streams topology directly in memory and feed test records into input topics while reading results from output topics. This makes it ideal for fast, deterministic, and isolated unit tests. As documented, it supports testing stateful and stateless operations, windowed aggregations, joins, and exactly-once behavior at the topology level.

Option A is incorrect because TestProducer and TestConsumer are not official Kafka Streams testing utilities. Option B is invalid because KafkaUnitTestDriver does not exist in Apache Kafka. Option D (MockProducer and MockConsumer) belongs to the Kafka Clients API and is suitable for producer/consumer testing, but it does not support Kafka Streams topologies or state stores.

Therefore, TopologyTestDriver is the only fully supported and officially documented solution for unit testing Kafka Streams applications without a real Kafka cluster.

If the consumer fails to send heartbeats in time (due to being slow under load), it is considered dead, causing a rebalance. Increasing session.timeout.ms gives the consumer more time to process messages and send heartbeats, preventing premature removal.

From Kafka Docs > Heartbeats and Failures:

“If the consumer does not send a heartbeat in session.timeout.ms, it is considered dead and the group coordinator will trigger a rebalance.”

Reducing max.poll.records might slow consumption further. More consumers won’t help since each partition already has a dedicated consumer.

According to the Apache Kafka consumer and scalability documentation, parallelism in message processing is primarily determined by the number of partitions and the number of consumers within a consumer group. Each partition can be assigned to only one consumer at a time. In this scenario, the topic already has four partitions, but only two consumers are processing them, meaning only two partitions are actively consumed in parallel.

Since monitoring indicates that message processing is CPU-bound, the bottleneck is not Kafka I/O but application-side processing. Adding more consumers (up to the number of partitions) allows Kafka to rebalance the partitions and distribute them across more consumer instances, increasing CPU parallelism and reducing lag.

Option A (adding more partitions) can increase parallelism but is more intrusive and may require repartitioning considerations for keys and downstream systems. Option B increases batch size, which can worsen CPU pressure. Option D may reduce per-poll workload but does not increase parallel processing capacity.

Therefore, the correct and recommended action per Kafka documentation is to add more consumers to the consumer group.

=========================

The Apache Kafka monitoring and operations documentation highlights several controller-level metrics that are critical indicators of cluster health.

ActiveControllerCount must always be 1 in a healthy cluster. A value of 0 indicates no active controller, while values greater than 1 indicate a serious split-brain condition. This metric is fundamental for cluster stability.

OfflinePartitionsCount is one of the most important health metrics. Any value greater than 0 means that some partitions have no available leader, making them unavailable for producers and consumers. This directly indicates a degraded or unhealthy cluster.

UncleanLeaderElectionsPerSec tracks the rate of unclean leader elections. Unclean elections can cause data loss, and any non-zero value is a strong signal of broker failures or misconfiguration and should immediately trigger alerts.

The remaining options are not minimum health indicators. LastCommittedRecordOffset is not a standard health metric, and TopicsToDeleteCount only indicates pending administrative operations, not cluster instability.

Therefore, the three minimum and officially recommended broker metrics for monitoring Kafka cluster health are UncleanLeaderElectionsPerSec, ActiveControllerCount, and OfflinePartitionsCount.

According to the official Apache Kafka consumer semantics documentation, at-most-once delivery means that records may be lost but must never be processed more than once. To achieve this guarantee, a consumer must commit offsets before processing records. If the application crashes after committing but before processing, messages are skipped rather than reprocessed, which satisfies at-most-once semantics.

Option B correctly implements this pattern. It disables automatic offset commits (enable.auto.commit=false) and explicitly calls commitAsync() immediately after polling and before processing records. This ensures that offsets are advanced regardless of whether processing succeeds, eliminating the possibility of duplicate processing.

Option C represents at-least-once semantics, because offsets are committed after processing, which can lead to duplicates on failure. Options A and D rely on automatic commits or incorrect ordering and do not provide strict at-most-once guarantees.

The Kafka documentation explicitly states that manual offset management with commits before processing is required for at-most-once delivery.

Therefore, Option B is the best and officially documented configuration and code skeleton to avoid duplicate message processing.