Skip to content

Commit 8e90d40

Browse files
jzelinskiejzelinskie
authored
Merge pull request #362 from corkrean/data
initial data doc additions and refactor
2 parents 8973419 + 4efae0c commit 8e90d40

File tree

6 files changed

+163
-1
lines changed

6 files changed

+163
-1
lines changed

‎pages/spicedb/_meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
"concepts": "Concepts",
44
"modeling": "Modeling & Integrating",
55
"ops": "Operations",
6+
"data": "Managing Data",
67
"api": "API Reference",
78
"links": "Links"
89
}

‎pages/spicedb/data/_meta.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"writing-relationships": "Writing Relationships",
3+
"data-syncronization": "Data Syncronization",
4+
"bulk-operations": "Bulk Importing Relationships",
5+
"migrations": "Migrations"
6+
}

‎pages/spicedb/ops/bulk-operations.mdx‎ renamed to ‎pages/spicedb/data/bulk-operations.mdx‎

File renamed without changes.

‎pages/spicedb/data/migrations.mdx‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
import { Callout } from 'nextra/components'
2+
3+
# Migrations
4+
5+
## Migrating from SpiceDB to SpiceDB
6+
7+
<Callout type="warning">
8+
Migrating data at the underlying database level is not recommended and impossible in some cases. Using tools like `pg_dump`/`pg_restore` will break SpiceDB MVCC. Additionally, if you are migrating to a SpiceDB with a different datastore type (e.g. Postgres -> CockroachDB), you ***must*** use the SpiceDB APIs (`exportBulk`/`importBulk` or `zed backup`) to backup and restore.
9+
</Callout>
10+
11+
The options provided below enable you to consistently migrate between datastores with minimal downtime.
12+
13+
All options utilize the [Zed CLI tool](https://github.com/authzed/zed?tab=readme-ov-file#zed).
14+
15+
### Write Downtime Migration
16+
17+
A simple migration that will incur write downtime (not read downtime) for the duration between starting the export and finishing the import.
18+
19+
1. Spin up your new SpiceDB.
20+
2. Stop writes to your old SpiceDB.
21+
3. Run `zed backup create <filename>` against the old SpiceDB.
22+
4. Run `zed backup restore <filename>` against the new SpiceDB with the backup file generated by the previous command.
23+
5. Switch reads to the new SpiceDB.
24+
6. Start writing to the new SpiceDB.
25+
26+
### Near Zero Downtime Migration
27+
28+
This option involves more complexity than the above option.
29+
30+
1. Spin up your new SpiceDB.
31+
2. Run `zed backup create <filename>` against your old SpiceDB.
32+
3. Run `zed backup restore <filename>` against the new SpiceDB with the backup file generated by the previous command.
33+
4. Obtain the zed token that points to the backup revision with `zed backup parse-revision <filename>`.
34+
5. Using the [watch API](https://buf.build/authzed/api/docs/main:authzed.api.v1#authzed.api.v1.WatchRequest), consume relationship changes from the old SpiceDB and write them to the new SpiceDB.
35+
You should provide the zed token that you obtained in the previous request as the `optional_start_cursor` in the `WatchRequest`.
36+
You will need to build a small worker to consume changes and write them to the new SpiceDB.
37+
The worker needs to continuously consume changes until you are ready to turn off writes on the old SpiceDB.
38+
6. Stop writes to the old SpiceDB.
39+
7. Wait for the worker to process all changes. (this is when write downtime will occur)
40+
8. Switch reads and writes to the new SpiceDB.

‎pages/spicedb/data/writing-relationships.mdx‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
import { Tabs } from 'nextra/components'
2+
import { Callout } from 'nextra/components'
3+
4+
# Writing relationships
5+
6+
This page will provide some practical recommendations for writing relationships to SpiceDB.
7+
If you are interested in relationships as a concept, check out this [page](/spicedb/concepts/relationships).
8+
9+
## Retries
10+
11+
When making requests to SpiceDB, it's important to implement proper retry logic to handle transient failures. [SpiceDB APIs use gRPC*](/spicedb/getting-started/client-libraries), which can experience various types of temporary failures that can be resolved through retries.
12+
13+
Retries are recommended for all gRPC methods, not just WriteRelationships.
14+
15+
*SpiceDB can also expose an [HTTP API](/spicedb/getting-started/client-libraries#http-clients); however, gRPC is recommended.
16+
17+
### Implementing Retry Policies
18+
19+
You can implement your own retry policies using the gRPC Service Config.
20+
Below, you will find a recommended Retry Policy.
21+
22+
```
23+
"retryPolicy": {
24+
"maxAttempts": 3,
25+
"initialBackoff": "1s",
26+
"maxBackoff": "4s",
27+
"backoffMultiplier": 2,
28+
"retryableStatusCodes": [
29+
'UNAVAILABLE', 'RESOURCE_EXHAUSTED', 'DEADLINE_EXCEEDED', 'ABORTED',
30+
]
31+
}
32+
```
33+
34+
This retry policy configuration provides exponential backoff with the following behavior:
35+
36+
**`maxAttempts: 3`** - Allows for a maximum of 3 total attempts (1 initial request + 2 retries).
37+
This prevents infinite retry loops while giving sufficient opportunity for transient issues to resolve.
38+
39+
**`initialBackoff: "1s"`** - Sets the initial delay to 1 second before the first retry attempt.
40+
This gives the system time to recover from temporary issues.
41+
42+
**`maxBackoff: "4s"`** - Caps the maximum delay between retries at 4 seconds to prevent excessively long waits that could impact user experience.
43+
44+
**`backoffMultiplier: 2`** - Doubles the backoff time with each retry attempt.
45+
Combined with the other settings, this creates a retry pattern of: 1s → 2s → 4s.
46+
47+
**`retryableStatusCodes`** - Only retries on specific gRPC status codes that indicate transient failures:
48+
-`UNAVAILABLE`: SpiceDB is temporarily unavailable
49+
-`RESOURCE_EXHAUSTED`: SpiceDB is overloaded
50+
-`DEADLINE_EXCEEDED`: Request timed out
51+
-`ABORTED`: Operation was aborted, often due to conflicts that may resolve on retry
52+
53+
You can find a python retry example [here](https://github.com/authzed/examples/blob/main/data/retry/main.py).
54+
55+
## Writes: Touch vs Create
56+
57+
A SpiceDB [relationship update](https://buf.build/authzed/api/docs/main:authzed.api.v1#authzed.api.v1.RelationshipUpdate) can use one of three operation types `CREATE`, `TOUCH`, OR `DELETE`.
58+
This section will cover `CREATE` and `TOUCH`.
59+
You can read more about `DELETE` in [the section below](#deleting-relationships).
60+
61+
### Understanding the Operations
62+
63+
**`CREATE`** - Inserts a new relationship.
64+
If the relationship already exists, the operation will fail with an error.
65+
66+
**`TOUCH`** - Upserts a relationship.
67+
If the relationship already exists, it will do nothing.
68+
If it doesn't exist, it will create it.
69+
70+
### Key Differences
71+
72+
| Operation | Behavior on Existing Relationship | Performance | Use Case |
73+
|-----------|-----------------------------------|-------------|----------|
74+
| `CREATE` | Fails with error | Faster (single insert) | Initial relationship creation |
75+
| `TOUCH` | Updates/overwrites | Slower (delete + insert) | Idempotent operations |
76+
77+
### Special Considerations
78+
79+
**Expiring Relationships:** When working with [expiring relationships](/spicedb/concepts/expiring-relationships), always use `TOUCH`.
80+
If a relationship has expired but hasn't been garbage collected yet, using `CREATE` will return an error.
81+
82+
**Error Handling:** When using `CREATE`, be prepared to handle duplicate relationship errors appropriately in your application logic.
83+
84+
## Deleting Relationships
85+
86+
SpiceDB provides two methods for deleting relationships: using the `WriteRelationships` API with the `DELETE` operation or using the `DeleteRelationships` API.
87+
Each approach has different behaviors and use cases.
88+
89+
### WriteRelationships with the `DELETE` Operation
90+
91+
The [`WriteRelationships`](https://buf.build/authzed/api/docs/main:authzed.api.v1#authzed.api.v1.WriteRelationshipsRequest) API supports a `DELETE` operation type that allows you to remove specific relationships as part of a batch of relationship updates.
92+
93+
**`DELETE`** - Removes a relationship.
94+
If the relationship does not exist, the operation will silently succeed (no-op).
95+
96+
#### Characteristics
97+
98+
- **Atomic Operations**: Can be combined with other relationship operations (`CREATE`, `TOUCH`) in a single atomic transaction
99+
- **Granular Control**: Delete specific relationships alongside creating or updating others
100+
- **Silent Failure**: Does not fail if the relationship doesn't exist
101+
- **Batch Limit**: Subject to the same batch size limits as other `WriteRelationships` operations (1,000 updates by default)
102+
103+
### DeleteRelationships API
104+
105+
The [`DeleteRelationships`](https://buf.build/authzed/api/docs/main:authzed.api.v1#authzed.api.v1.DeleteRelationshipsRequest) API is a dedicated method for bulk deletion of relationships based on filters rather than specifying individual relationships.
106+
107+
#### Characteristics
108+
109+
- **Filter-Based**: Delete relationships based on resource type, relation, subject type, or combinations thereof
110+
- **Bulk Operations**: Can delete many relationships matching the filter criteria in a single call
111+
- **Separate Transaction**: Operates independently from `WriteRelationships`
112+
- **Efficient for Mass Deletion**: Optimized for removing large numbers of relationships
113+
114+
## Bulk Import
115+
116+
Check out [Bulk Importing Relationships](./bulk-operations)

‎pages/spicedb/ops/_meta.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
{
22
"observability": "Observability Tooling",
33
"deploying-spicedb-operator": "Deploying the SpiceDB Operator",
4-
"bulk-operations": "Bulk Importing Relationships",
54
"secure-rag-pipelines": "Secure Your RAG Pipelines with Fine Grained Authorization"
65
}

0 commit comments

Comments
 (0)