Skip to content

Commit 8f1452f

Browse files
committed
update quickstart
1 parent ac6949b commit 8f1452f

File tree

1 file changed

+57
-3
lines changed

1 file changed

+57
-3
lines changed

docs/integrations/data-ingestion/clickpipes/mongodb/quickstart.md

Lines changed: 57 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -266,11 +266,11 @@ LIMIT 10;
266266

267267
### Refreshable materialized view {#refreshable-materialized-view}
268268

269-
You can also create [Refreshable Materialized Views](https://clickhouse.com/docs/materialized-view/refreshable-materialized-view), which enable you to schedule query execution for deduplicating rows and storing the results in a flattened destination table. With each scheduled refresh, the destination table is replaced with the latest query results.
269+
If your table size is small, you can create [Refreshable Materialized Views](https://clickhouse.com/docs/materialized-view/refreshable-materialized-view), which enable you to schedule query execution for deduplicating rows and storing the results in a flattened destination table. With each scheduled refresh, the destination table is replaced with the latest query results.
270270

271271
The key advantage of this method is that the query using the `FINAL` keyword runs only once during the refresh, eliminating the need for subsequent queries on the destination table to use `FINAL`.
272272

273-
However, a drawback is that the data in the destination table is only as up-to-date as the most recent refresh. For many use cases, refresh intervals ranging from several minutes to a few hours provide a good balance between data freshness and query performance.
273+
A drawback is that the data in the destination table is only as up-to-date as the most recent refresh. For many use cases, refresh intervals ranging from several minutes to a few hours provide a good balance between data freshness and query performance.
274274

275275
```sql
276276
CREATE TABLE flattened_t1 (
@@ -287,7 +287,7 @@ ENGINE = ReplacingMergeTree()
287287
PRIMARY KEY _id
288288
ORDER BY _id;
289289

290-
CREATE MATERIALIZED VIEW mv1 REFRESH EVERY 1 HOUR TO flattened_t1 AS
290+
CREATE MATERIALIZED VIEW rmv REFRESH EVERY 1 HOUR TO flattened_t1 AS
291291
SELECT
292292
CAST(doc._id, 'String') AS _id,
293293
CAST(doc.order_id, 'String') AS order_id,
@@ -313,3 +313,57 @@ GROUP BY customer_id
313313
ORDER BY customer_id DESC
314314
LIMIT 10;
315315
```
316+
317+
### Incremental materialized view {#incremental-materialized-view}
318+
319+
If your table size is very large or if you want to flatten the data in real-time, you can create [Incremental Materialized Views](https://clickhouse.com/docs/materialized-view/incremental-materialized-view). If your table has frequent updates, it's not recommended to use FINAL modifier in your materialized view as every update will trigger a merge. Instead, you can deduplicate the data at query time by building a normal view on top of the materialized view.
320+
321+
```sql
322+
CREATE TABLE flattened_t1 (
323+
`_id` String,
324+
`order_id` String,
325+
`customer_id` Int64,
326+
`status` String,
327+
`total_amount` Decimal(18, 2),
328+
`order_date` DateTime64(3),
329+
`shipping_info` JSON,
330+
`items` Dynamic,
331+
`_peerdb_version` Int64,
332+
`_peerdb_synced_at` DateTime64(9),
333+
`_peerdb_is_deleted` Int8
334+
)
335+
ENGINE = ReplacingMergeTree()
336+
PRIMARY KEY _id
337+
ORDER BY _id;
338+
339+
CREATE MATERIALIZED VIEW imv TO flattened_t1 AS
340+
SELECT
341+
CAST(doc._id, 'String') AS _id,
342+
CAST(doc.order_id, 'String') AS order_id,
343+
CAST(doc.customer_id, 'Int64') AS customer_id,
344+
CAST(doc.status, 'String') AS status,
345+
CAST(doc.total_amount, 'Decimal64(2)') AS total_amount,
346+
CAST(parseDateTime64BestEffortOrNull(doc.order_date, 3), 'DATETIME(3)') AS order_date,
347+
doc.^shipping AS shipping_info,
348+
doc.items,
349+
_peerdb_version,
350+
_peerdb_synced_at,
351+
_peerdb_is_deleted
352+
FROM t1;
353+
354+
CREATE VIEW flattened_t1_final AS
355+
SELECT * FROM flattened_t1 FINAL WHERE _peerdb_is_deleted = 0;
356+
```
357+
358+
You can now query the view `flattened_t1_final` as follows:
359+
360+
```sql
361+
SELECT
362+
customer_id,
363+
sum(total_amount)
364+
FROM flattened_t1_final
365+
AND shipping_info.city = 'Seattle'
366+
GROUP BY customer_id
367+
ORDER BY customer_id DESC
368+
LIMIT 10;
369+
```

0 commit comments

Comments
 (0)