Skip to content

Commit 3b733a7

Browse files
viragtripathiviragtripathi
committed
Updated sample configs
1 parent dfda9ea commit 3b733a7

File tree

10 files changed

+38
-32
lines changed

10 files changed

+38
-32
lines changed

‎README.md‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -50,27 +50,27 @@ changed-data events from heterogeneous data platforms to [Redis Stack](https://r
5050
</tr>
5151
<tr><td height="20" colspan="2">&nbsp;</td></tr>
5252
<tr>
53-
<td width="50%"><img src="/images/capabilities/Redis Insight.png" style="float: right;" width="500" height="200"/></td>
53+
<td width="50%"><img src="/images/capabilities/Redis Insight.png" style="float: right;" width="500" height="200" alt="Redis Insight"/></td>
5454
<td> <b>Multi-Tenancy | Partitioning | Linear Scalability</b> <br> Redis Connect manages multi-tenant replication pipelines. A pipeline from source to sink is known as a job. Jobs with different source types can be collocated without becoming noisy neighbors. Jobs can be partitioned for linear scalability across one or more cluster nodes.</td>
5555
</tr>
5656
<tr><td bgcolor="#FFFFFF" colspan="2">&nbsp;</td></tr>
5757
<tr>
5858
<td> <b>High-Availability | Recovery</b> <br> Redis Connect jobs update their checkpoint upon each committed changed-data event within a transactional scope. In the event of a node failure or network split, in-flight jobs will fail over to another node and seamlessly begin replication from the last committed checkpoint. Data is not lost, and order is preserved. Redis Connect works in container orchestration environments such as Kubernetes and OpenShift.</td>
59-
<td width="50%"><img src="/images/capabilities/Redis Connect Cluster.png" style="float: right;" width="500" height="200"/></td>
59+
<td width="50%"><img src="/images/capabilities/Redis Connect Cluster.png" style="float: right;" width="500" height="200" alt="Redis Connect Cluster"/></td>
6060
</tr>
6161
<tr><td bgcolor="#FFFFFF" colspan="2">&nbsp;</td></tr>
6262
<tr>
63-
<td width="50%"><img src="/images/capabilities/Redis Connect Custom Transformer.jpg" style="float: none;" width="500" height="150"/></td>
63+
<td width="50%"><img src="/images/capabilities/Redis Connect Custom Transformer.jpg" style="float: none;" width="500" height="150" alt="Redis Connect Custom Transformation"/></td>
6464
<td> <b>Custom Transformations</b> <br> Redis Connect jobs support user-defined business logic. You can create custom workflows that include user-defined stages for proprietary business rules, custom transformations, de-tokenization, and more. You can also extend the supported list of target sinks.</td>
6565
</tr>
6666
<tr><td bgcolor="#FFFFFF" colspan="2">&nbsp;</td></tr>
6767
<tr>
6868
<td> <b>REST API | CLI | Swagger UI</b> <br> Redis Connect is entirely data-driven and relies on Redis Enterprise as its metadata store. You can configure, start, stop, migrate, and restart jobs using the built-in REST API and interactive CLI. Redis Connect also exposes a Swagger UI to simplify endpoint discovery and operational experience.</td>
69-
<td width="50%"><img src="/images/capabilities/Redis Connect Swagger UI.png" style="float: right;" width="500" height="200"/></td>
69+
<td width="50%"><img src="/images/capabilities/Redis Connect Swagger UI.png" style="float: right;" width="500" height="200" alt="Redis Connect Swagger UI"/></td>
7070
</tr>
7171
<tr><td bgcolor="#FFFFFF" colspan="2">&nbsp;</td></tr>
7272
<tr>
73-
<td width="50%"><img src="/images/capabilities/Redis Enterprise ACL.png" style="float: right;" width="500" height="200"/></td>
73+
<td width="50%"><img src="/images/capabilities/Redis Enterprise ACL.png" style="float: right;" width="500" height="200" alt="Redis Enterprise ACL"/></td>
7474
<td> <b>Enterprise-Grade Security</b> <br> Redis Connect jobs are stateless, so changed-data events are always in-transit. Redis Connect benefits from Redis Enterprise’s security, including RBAC, TLS, and more. Credentials, secrets, and trust-store passwords are never stored in Redis Connect; these secrets can be dynamically rotated with minimal disruption to the replication pipeline. Vault integration is supported.</td>
7575
</tr>
7676
</table>
@@ -134,9 +134,9 @@ start: init Redis Connect Instance (Cluster Member)
134134
redis-connect\bin> redisconnect.bat
135135
```
136136

137-
| Prerequisite Configuration :exclamation: |
138-
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
139-
| Update `credentials.file.path` and `redis.connection.url` in `/config/jobmanager.properties`<br/> Example - <a href="/examples/postgres/demo/config/jobmanager.properties">jobmanager.properties</a> |
137+
| Prerequisite Configuration :exclamation: |
138+
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
139+
| Update `credentials.file.path` and `redis.connection.url` in `/config/jobmanager.properties`<br/> Example - <a href="/examples/postgres/demo/config/jobmanager.properties">jobmanager.properties</a> |
140140

141141
### Starting an instance
142142

@@ -148,7 +148,7 @@ Windows:
148148
```cmd
149149
redis-connect\bin> redisconnect.bat start
150150
```
151-
<img src="/images/quick-start/Redis Connect Start Log.png" style="float: right;" width = 700px height = 250px/>
151+
<img src="/images/quick-start/Redis Connect Start Log.png" style="float: right;" width = 700px height = 250px alt="Redis Connect Start Log"/>
152152

153153
### Swagger UI
154154

@@ -161,9 +161,9 @@ The Redis Connect Swagger UI is available on port 8282 by default. If you're run
161161
<br>_For quick start, use a sample job configuration:_ <a href="/examples/db2/demo/config/samples/payloads/cdc-job.json">DB2</a>, <a href="/examples/files/demo/config/samples/payloads/cdc-job.json">Files</a>, <a href="/examples/gemfire/demo/config/samples/payloads/cdc-job.json">Gemfire</a>, <a href="/examples/mongodb/demo/config/samples/payloads/cdc-job.json">MongoDB</a>, <a href="/examples/mysql/demo/config/samples/payloads/cdc-job.json">MySQL</a>, <a href="/examples/oracle/demo/config/samples/payloads/cdc-job.json">Oracle</a>, <a href="/examples/postgres/demo/config/samples/payloads/cdc-job.json">PostgreSQL</a>, <a href="/examples/mssql/demo/config/samples/payloads/cdc-job.json">SQL Server</a> and <a href="/examples/vertica/demo/config/samples/payloads/cdc-job.json">VERTICA</a>
162162
<br><br><img src="/images/quick-start/Redis Connect Save Job Config.png" style="float: right;" width = 700px height = 375px/>
163163

164-
| Prerequisite Configuration :exclamation: |
165-
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
166-
| Update `credentialsFilePath`, `databaseURL`, `database.dbname`, `database.hostname`, `database.port`, `schemaAndTableName`, and `columns` within sample job configuration for source and target, where applicable. |
164+
| Prerequisite Configuration :exclamation: |
165+
|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
166+
| Update `credentialsFilePath`, `databaseURL`, `database.dbname` (`database.names` for SQL Server), `database.hostname`, `database.port` and `columns` within sample job configuration for source and target, where applicable. |
167167

168168
**Start Job -** `/connect/api/vi/job/transition/start/{jobName}/{jobType}`
169169
<br>_For quick start, use '**stream**' as **jobType**_

‎examples/db2/demo/config/samples/payloads/cdc-job.json‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,9 @@
3636
{ "targetColumn": "FirstName", "sourceColumn": "FNAME" },
3737
{ "targetColumn": "HireDate", "sourceColumn": "HIREDATE", "type": "DATE" }
3838
],
39-
"schemaAndTableName": "DB2INST1.EMP",
39+
"initialLoad": {
40+
"partitions": 4
41+
},
4042
"passThroughEnabled": true
4143
}
4244
}

‎examples/mongodb/demo/config/samples/payloads/cdc-job.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,6 @@
4040
"partitions": 4,
4141
"selectQuery": "{ $match: { '_id' : { $gte : '0' } } }"
4242
},
43-
"schemaAndTableName": "sample_airbnb.listingsAndReviews",
4443
"passThroughEnabled" : true
4544
}
4645
}

‎examples/mysql/demo/config/samples/payloads/cdc-job.json‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,9 @@
3737
{ "targetColumn": "empno", "sourceColumn": "empno", "primaryKey": true },
3838
{ "targetColumn": "hiredate", "sourceColumn": "hiredate", "type": "DATE_TIME", "dateFormat": "YYYY-MM-dd'T'HH:mm:ss" }
3939
],
40-
"schemaAndTableName": "RedisConnect.emp",
40+
"initialLoad": {
41+
"partitions": 4
42+
},
4143
"passThroughEnabled": true
4244
}
4345
}

‎examples/oracle/demo/config/samples/payloads/cdc-job.json‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
"index": 1,
1616
"stageName": "REDIS_HASH_SINK",
1717
"checkpointStageIndicator": true,
18-
"metricsEnabled" : true
18+
"metricsEnabled": true
1919
}
2020
]
2121
},
@@ -25,9 +25,9 @@
2525
"databaseURL": "jdbc:oracle:thin:@172.17.0.1:1521/ORCLPDB1",
2626
"databaseType": "ORACLE",
2727
"customConfiguration": {
28-
"database.dbname" : "ORCLCDB",
28+
"database.dbname": "ORCLCDB",
2929
"database.hostname" : "172.17.0.1",
30-
"database.port" : 1521,
30+
"database.port": 1521,
3131
"database.pdb.name": "ORCLPDB1"
3232
}
3333
},
@@ -37,6 +37,9 @@
3737
{ "targetColumn": "EMPNO", "sourceColumn": "EMPNO", "primaryKey": true},
3838
{ "targetColumn": "HIREDATE", "sourceColumn": "HIREDATE", "type": "DATE"}
3939
],
40+
"initialLoad": {
41+
"partitions": 4
42+
},
4043
"passThroughEnabled": true
4144
}
4245
}

‎examples/postgres/demo/config/samples/payloads/cdc-custom-job.json‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,21 +21,20 @@
2121
"index": 2,
2222
"stageName": "REDIS_HASH_SINK",
2323
"checkpointStageIndicator": true,
24-
"metricsEnabled" : true
24+
"metricsEnabled": true
2525
}
2626
]
2727
},
2828
"source": {
29-
"batchSize": 1000,
3029
"database": {
3130
"credentialsFilePath": "/opt/redislabs/redis-connect/config/samples/credentials",
3231
"credentialsRotationEventListenerEnabled" : true,
3332
"databaseURL": "jdbc:postgresql://redisconnect.c7glrswbovia.us-east-1.rds.amazonaws.com:5432/RedisConnect",
3433
"databaseType": "POSTGRES",
3534
"customConfiguration": {
36-
"database.dbname" : "RedisConnect",
37-
"database.hostname" : "redisconnect.c7glrswbovia.us-east-1.rds.amazonaws.com",
38-
"database.port" : 5432,
35+
"database.dbname": "RedisConnect",
36+
"database.hostname": "redisconnect.c7glrswbovia.us-east-1.rds.amazonaws.com",
37+
"database.port": 5432,
3938
"heartbeat.interval.ms": 10000,
4039
"heartbeat.action.query": "INSERT INTO heartbeat (id, ts) VALUES (1, NOW()) ON CONFLICT(id) DO UPDATE SET ts=EXCLUDED.ts;"
4140
}

‎examples/postgres/demo/config/samples/payloads/cdc-job.json‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,20 +15,19 @@
1515
"index": 1,
1616
"stageName": "REDIS_HASH_SINK",
1717
"checkpointStageIndicator": true,
18-
"metricsEnabled" : true
18+
"metricsEnabled": true
1919
}
2020
]
2121
},
2222
"source": {
23-
"batchSize": 1000,
2423
"database": {
2524
"credentialsFilePath": "/opt/redislabs/redis-connect/config/samples/credentials",
2625
"databaseURL": "jdbc:postgresql://127.0.0.1/RedisConnect",
2726
"databaseType": "POSTGRES",
2827
"customConfiguration": {
29-
"database.dbname" : "RedisConnect",
30-
"database.hostname" : "127.0.0.1",
31-
"database.port" : 5432
28+
"database.dbname": "RedisConnect",
29+
"database.hostname": "127.0.0.1",
30+
"database.port": 5432
3231
}
3332
},
3433
"tables": {
@@ -37,6 +36,9 @@
3736
{ "targetColumn": "empno", "sourceColumn": "empno", "primaryKey": true},
3837
{ "targetColumn": "hiredate", "sourceColumn": "hiredate", "type": "DATE"}
3938
],
39+
"initialLoad": {
40+
"partitions": 4
41+
},
4042
"passThroughEnabled": true
4143
}
4244
}

‎examples/splunk/demo/config/samples/payloads/cdc-job.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,6 @@
5656
{ "primaryKey": true, "sourceColumn": "time", "targetColumn": "time", "type": "INTEGER" },
5757
{ "primaryKey": true, "sourceColumn": "sequenceId", "targetColumn": "sequenceId", "type": "LONG", "passThroughEnabled": false }
5858
],
59-
"schemaAndTableName": "hec_collector",
6059
"passThroughEnabled": true,
6160
"prependTableNameToKeysEnabled": false
6261
}

‎examples/vertica/demo/config/samples/payloads/cdc-job.json‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,6 @@
3838
"partitions": 2,
3939
"selectQuery": "SELECT * FROM (SELECT ALIAS.*, ROW_NUMBER() OVER (ORDER BY employee_key) RowIndex FROM public.employee_dimension ALIAS) ALIAS WHERE ALIAS.RowIndex >= ${start} AND ALIAS.RowIndex <= ${end}"
4040
},
41-
"schemaAndTableName": "public.employee_dimension",
4241
"passThroughEnabled": true
4342
},
4443
"public.customer_dimension": {
@@ -49,7 +48,6 @@
4948
"partitions": 2,
5049
"selectQuery": "SELECT * FROM (SELECT ALIAS.*, ROW_NUMBER() OVER (ORDER BY customer_key) RowIndex FROM public.customer_dimension ALIAS) ALIAS WHERE ALIAS.RowIndex >= ${start} AND ALIAS.RowIndex <= ${end}"
5150
},
52-
"schemaAndTableName": "public.customer_dimension",
5351
"passThroughEnabled": true
5452
}
5553
}

‎k8s-docs/redis-connect/postgres-job.json‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,9 @@
3434
{ "targetColumn": "empno", "sourceColumn": "empno", "primaryKey": true},
3535
{ "targetColumn": "hiredate", "sourceColumn": "hiredate", "type": "DATE"}
3636
],
37-
"schemaAndTableName": "public.emp",
37+
"initialLoad": {
38+
"partitions": 4
39+
},
3840
"passThroughEnabled": true
3941
}
4042
}

0 commit comments

Comments
 (0)