Skip to content

Commit 1fe6e5d

Browse files
tarofacenoelcrlflorence-crl
authored
MOLT Replicator draft docs (#20465)
* MOLT Replicator docs; Oct release notes --------- Co-authored-by: Noel Riopel <noel.riopel@cockroachlabs.com> Co-authored-by: Florence Morris <58752716+florence-crl@users.noreply.github.com>
1 parent 647d5ff commit 1fe6e5d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+2514
-1550
lines changed

src/current/_data/redirects.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -299,6 +299,12 @@
299299
- destination: molt/migrate-data-load-and-replication.md
300300
sources: [':version/migrate-from-postgres.md']
301301

302+
- destination: molt/migrate-load-replicate.md
303+
sources: ['molt/migrate-data-load-replicate-only.md']
304+
305+
- destination: molt/migrate-resume-replication.md
306+
sources: ['molt/migrate-replicate-only.md']
307+
302308
- destination: molt/migration-overview.md
303309
sources: [':version/migration-overview.md']
304310

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{{site.data.alerts.callout_info}}
2+
For CockroachDB-to-CockroachDB migrations, contact your account team for guidance.
3+
{{site.data.alerts.end}}

src/current/_includes/molt/fetch-data-load-output.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,16 @@
11
1. Check the output to observe `fetch` progress.
22

3+
{% if page.name == "migrate-load-replicate.md" %}
4+
<section class="filter-content" markdown="1" data-scope="oracle">
5+
The following message shows the appropriate values for the `--backfillFromSCN` and `--scn` replication flags to use when [starting Replicator](#start-replicator):
6+
7+
{% include_cached copy-clipboard.html %}
8+
~~~
9+
replication-only mode should include the following replicator flags: --backfillFromSCN 26685444 --scn 26685786
10+
~~~
11+
</section>
12+
{% endif %}
13+
314
A `starting fetch` message indicates that the task has started:
415

516
<section class="filter-content" markdown="1" data-scope="postgres">
@@ -16,7 +27,7 @@
1627

1728
<section class="filter-content" markdown="1" data-scope="oracle">
1829
~~~ json
19-
{"level":"info","type":"summary","num_tables":3,"cdc_cursor":"2358840","time":"2025-02-10T14:28:11-05:00","message":"starting fetch"}
30+
{"level":"info","type":"summary","num_tables":3,"cdc_cursor":"26685786","time":"2025-02-10T14:28:11-05:00","message":"starting fetch"}
2031
~~~
2132
</section>
2233

@@ -68,7 +79,7 @@
6879
~~~
6980

7081
{% if page.name != "migrate-bulk-load.md" %}
71-
This message includes a `cdc_cursor` value. You must set the `--defaultGTIDSet` replication flag to this value when starting [`replication-only` mode](#replicate-changes-to-cockroachdb):
82+
This message includes a `cdc_cursor` value. You must set the `--defaultGTIDSet` replication flag to this value when [starting Replicator](#start-replicator):
7283

7384
{% include_cached copy-clipboard.html %}
7485
~~~
@@ -81,15 +92,4 @@
8192
~~~ json
8293
{"level":"info","type":"summary","fetch_id":"f5cb422f-4bb4-4bbd-b2ae-08c4d00d1e7c","num_tables":3,"tables":["migration_schema.employees","migration_schema.payments","migration_schema.payments"],"cdc_cursor":"2358840","net_duration_ms":6752.847625,"net_duration":"000h 00m 06s","time":"2024-03-18T12:30:37-04:00","message":"fetch complete"}
8394
~~~
84-
</section>
85-
86-
{% if page.name == "migrate-data-load-replicate-only.md" %}
87-
<section class="filter-content" markdown="1" data-scope="oracle">
88-
The following message shows the appropriate values for the `--backfillFromSCN` and `--scn` replication flags to use when [starting`replication-only` mode](#replicate-changes-to-cockroachdb):
89-
90-
{% include_cached copy-clipboard.html %}
91-
~~~
92-
replication-only mode should include the following replicator flags: --backfillFromSCN 26685444 --scn 26685786
93-
~~~
94-
</section>
95-
{% endif %}
95+
</section>

src/current/_includes/molt/fetch-metrics.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
### Fetch metrics
2+
13
By default, MOLT Fetch exports [Prometheus](https://prometheus.io/) metrics at `http://127.0.0.1:3030/metrics`. You can override the address with `--metrics-listen-addr '{host}:{port}'`, where the endpoint will be `http://{host}:{port}/metrics`.
24

35
Cockroach Labs recommends monitoring the following metrics during data load:

src/current/_includes/molt/fetch-replication-output.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@
1919
DEBUG [Jan 22 13:52:40] upserted rows conflicts=0 duration=7.620208ms proposed=1 target="\"molt\".\"migration_schema\".\"employees\"" upserted=1
2020
~~~
2121

22-
{% if page.name != "migrate-replicate-only.md" %}
22+
{% if page.name != "migrate-resume-replication.md" %}
2323
{{site.data.alerts.callout_success}}
24-
If replication is interrupted, you can [resume replication]({% link molt/migrate-replicate-only.md %}).
24+
If replication is interrupted, you can [resume replication]({% link molt/migrate-resume-replication.md %}).
2525
{{site.data.alerts.end}}
2626
{% endif %}

src/current/_includes/molt/fetch-replicator-flags.md

Lines changed: 0 additions & 92 deletions
This file was deleted.

src/current/_includes/molt/fetch-secure-connection-strings.md

Lines changed: 0 additions & 34 deletions
This file was deleted.

src/current/_includes/molt/fetch-table-filter-userscript.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,8 @@ api.configureSource("defaultdb.migration_schema", {
3434
});
3535
~~~
3636

37-
Pass the userscript to MOLT Fetch with the `--userscript` [replication flag](#replication-flags):
37+
Pass the userscript to MOLT Replicator with the `--userscript` [flag](#replication-flags):
3838

3939
~~~
40-
--replicator-flags "--userscript table_filter.ts"
40+
--userscript table_filter.ts
4141
~~~

0 commit comments

Comments
 (0)