You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/integrations/data-ingestion/clickpipes/mongodb/faq.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,11 +10,11 @@ title: 'ClickPipes for MongoDB FAQ'
10
10
11
11
### Can I query for individual fields in the JSON datatype? {#can-i-query-for-individual-fields-in-the-json-datatype}
12
12
13
-
You can use **dot notation** for direct field access:
13
+
For direct field access, such as {"user_id": 123}, you can use **dot notation**:
14
14
```sql
15
-
SELECTdoc.customer.nameasnameFROM your_table;
15
+
SELECTdoc.user_idasuser_idFROM your_table;
16
16
```
17
-
For nested object fields, use the `^` operator:
17
+
For direct field access of nested object fields, such as {"address": { "city": "San Francisco", "state": "CA" }}, use the `^` operator:
18
18
```sql
19
19
SELECT doc.^address.cityAS city FROM your_table;
20
20
```
@@ -29,14 +29,14 @@ To learn more about working with JSON, see our [Working with JSON guide](./quick
29
29
MongoDB documents are replicated as JSON type in ClickHouse by default, preserving the nested structure. You have several options to flatten this data. If you want to flatten the data to columns, you can use normal views, materialized views, or query-time access.
30
30
31
31
1.**Normal Views**: Use normal views to encapsulate flattening logic.
32
-
2.**Materialized Views**: For smaller datasets, you can use refreshable materialized with FINALto periodically flatten and deduplicate data. For larger datasets, we recommend using incremental materialized views without FINAL to flatten the data in real-time, and then deduplicate data at query time.
32
+
2.**Materialized Views**: For smaller datasets, you can use refreshable materialized with the [`FINAL` modifier](/sql-reference/statements/select/from#final-modifier)to periodically flatten and deduplicate data. For larger datasets, we recommend using incremental materialized views without `FINAL` to flatten the data in real-time, and then deduplicate data at query time.
33
33
3.**Query-time Access**: Instead of flattening, use dot notation to access nested fields directly in queries.
34
34
35
35
For detailed examples, see our [Working with JSON guide](./quickstart).
36
36
37
37
### Can I connect MongoDB databases that don't have a public IP or are in private networks? {#can-i-connect-mongodb-databases-that-dont-have-a-public-ip-or-are-in-private-networks}
38
38
39
-
We support AWS PrivateLink for connecting to MongoDB databases that don't have a public IP or are in private networks.
39
+
We support AWS PrivateLink for connecting to MongoDB databases that don't have a public IP or are in private networks. Azure Private Link and GCP Private Service Connect are currently not supported.
40
40
41
41
### What happens if I delete a database/table from my MongoDB database? {#what-happens-if-i-delete-a-database-table-from-my-mongodb-database}
Copy file name to clipboardExpand all lines: docs/integrations/data-ingestion/clickpipes/mongodb/quickstart.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -72,7 +72,7 @@ The replicated tables use this standard schema:
72
72
73
73
ClickPipes maps MongoDB collections into ClickHouse using the `ReplacingMergeTree` table engine family. With this engine, updates are modeled as inserts with a newer version (`_peerdb_version`) of the document for a given primary key (`_id`), enabling efficient handling of updates, replaces, and deletes as versioned inserts.
74
74
75
-
`ReplacingMergeTree` clears out duplicates asynchronously in the background. To guarantee the absence of duplicates for the same row, use the [`FINAL` modifier](https://clickhouse.com/docs/sql-reference/statements/select/from#final-modifier). For example:
75
+
`ReplacingMergeTree` clears out duplicates asynchronously in the background. To guarantee the absence of duplicates for the same row, use the [`FINAL` modifier](/sql-reference/statements/select/from#final-modifier). For example:
If your table size is small, you can create [Refreshable Materialized Views](https://clickhouse.com/docs/materialized-view/refreshable-materialized-view), which enable you to schedule query execution for deduplicating rows and storing the results in a flattened destination table. With each scheduled refresh, the destination table is replaced with the latest query results.
269
+
You can create [Refreshable Materialized Views](https://clickhouse.com/docs/materialized-view/refreshable-materialized-view), which enable you to schedule query execution for deduplicating rows and storing the results in a flattened destination table. With each scheduled refresh, the destination table is replaced with the latest query results.
270
270
271
271
The key advantage of this method is that the query using the `FINAL` keyword runs only once during the refresh, eliminating the need for subsequent queries on the destination table to use `FINAL`.
If your table size is very large or if you want to flatten the data in real-time, you can create [Incremental Materialized Views](https://clickhouse.com/docs/materialized-view/incremental-materialized-view). If your table has frequent updates, it's not recommended to use FINAL modifier in your materialized view as every update will trigger a merge. Instead, you can deduplicate the data at query time by building a normal view on top of the materialized view.
319
+
If you want to access flattened columns in real-time, you can create [Incremental Materialized Views](https://clickhouse.com/docs/materialized-view/incremental-materialized-view). If your table has frequent updates, it's not recommended to use the `FINAL` modifier in your materialized view as every update will trigger a merge. Instead, you can deduplicate the data at query time by building a normal view on top of the materialized view.
0 commit comments