Skip to content

Commit 1373140

Browse files
chakru-rdgluong-datahub
authored andcommitted
fix(doc): kafka setup doc updates (#15304)
1 parent f50e310 commit 1373140

File tree

2 files changed

+14
-11
lines changed

2 files changed

+14
-11
lines changed

docs/deploy/confluent-cloud.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -124,21 +124,23 @@ cp-schema-registry:
124124
enabled: false
125125
```
126126

127-
Next, disable the `kafkaSetupJob` service:
127+
Next, disable the automatic creation of topics by the system update job:
128128

129129
```
130-
kafkaSetupJob:
131-
enabled: false
130+
global:
131+
kafka:
132+
precreateTopics: false
132133
```
133134

134135
Then, update the `kafka` configurations to point to your Confluent Cloud broker and schema registry instance, along with the topics you've created in Step 1:
135136

136137
```
137-
kafka:
138-
bootstrap:
139-
server: pkc-g4ml2.eu-west-2.aws.confluent.cloud:9092
140-
schemaregistry:
141-
url: https://plrm-qwlpp.us-east-2.aws.confluent.cloud
138+
global:
139+
kafka:
140+
bootstrap:
141+
server: pkc-g4ml2.eu-west-2.aws.confluent.cloud:9092
142+
schemaregistry:
143+
url: https://plrm-qwlpp.us-east-2.aws.confluent.cloud
142144
```
143145

144146
Next, you'll want to create 2 new Kubernetes secrets, one for the JaaS configuration which contains the username and password for Confluent,

docs/how/kafka-config.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,10 @@ the System Update container for topic setup:
8787
- (Deprecated) `METADATA_AUDIT_EVENT_NAME`: The name of the metadata audit event topic.
8888
- (Deprecated) `FAILED_METADATA_CHANGE_EVENT_NAME`: The name of the failed metadata change event topic.
8989

90+
#### Topic Setup
91+
92+
- `DATAHUB_PRECREATE_TOPICS`: Defaults to true, set this to false if you intend to create and configure the topics yourself and not have datahub create them.
93+
9094
### MCE Consumer (datahub-mce-consumer)
9195

9296
- `METADATA_CHANGE_PROPOSAL_TOPIC_NAME`: The name of the topic for Metadata Change Proposals emitted by the ingestion framework.
@@ -274,9 +278,6 @@ Examples:
274278

275279
Configurations specified in `topicDefaults` are applied to all topics by merging them with any configs defined per topic, with the per-topic config taking precedence over those specified in `topicDefault`.
276280

277-
If you intend to create and configure the topics yourself and not have datahub create them, the kafka setup process of
278-
datahub-system-update can be turned off by setting env var DATAHUB_PRECREATE_TOPICS to false
279-
280281
## Debugging Kafka
281282

282283
You can install [kafkacat](https://github.com/edenhill/kafkacat) to consume and produce messaged to Kafka topics.

0 commit comments

Comments
 (0)