Skip to content

Commit 33a9ea1

Browse files
committed
add faq about not propagating deletes
1 parent 293713f commit 33a9ea1

File tree

1 file changed

+10
-0
lines changed
  • docs/integrations/data-ingestion/clickpipes/postgres

1 file changed

+10
-0
lines changed

docs/integrations/data-ingestion/clickpipes/postgres/faq.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,3 +311,13 @@ The current workaround would be to map the NUMERIC type to string on ClickHouse.
311311
## I'm seeing errors like `invalid memory alloc request size <XXX>` during replication/slot creation {#postgres-invalid-memalloc-bug}
312312

313313
There was a bug introduced in Postgres patch versions 17.5/16.9/15.13/14.18/13.21 due to which certain workloads can cause an exponential increase in memory usage, leading to a memory allocation request >1GB which Postgres considers invalid. This bug [has been fixed](https://github.com/postgres/postgres/commit/d87d07b7ad3b782cb74566cd771ecdb2823adf6a) and will be in the next Postgres patch series (17.6...). Please check with your Postgres provider when this patch version will be available for upgrade. If an upgrade isn't immediately possible, a resync of the pipe will be needed as it hits the error.
314+
315+
## I need to maintain a complete historical record in ClickHouse, even when the data is deleted from source Postgres database. Can I completely ignore DELETEs and TRUNCATEs operations from Postgres in ClickPipes? {#ignore-delete-truncate}
316+
317+
Yes! Before creating your Postgres ClickPipe, create a publication without DELETEs. For example:
318+
```
319+
CREATE PUBLICATION <pub_name> FOR TABLES IN SCHEMA <schema_name> WITH (publish = 'insert,update');
320+
```
321+
Then when [setting up](https://clickhouse.com/docs/integrations/clickpipes/postgres#configuring-the-replication-settings) your Postgres ClickPipe, make sure this publication name selected.
322+
323+
Note that TRUNCATEs are ignored by ClickPipes and will not be replicated to ClickHouse.

0 commit comments

Comments
 (0)