You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,7 @@ DiffSync is at its most useful when you have multiple sources or sets of data to
15
15
16
16
# Overview of DiffSync
17
17
18
-
DiffSync acts as an intermediate translation layer between all of the data sets you are diffing and/or syncing. In practical terms, this means that to use DiffSync, you will define a set of data models as well as the “adapters” needed to translate between each base data source and the data model. In Python terms, the adapters will be subclasses of the `DiffSync` class, and each data model class will be a subclass of the `DiffSyncModel` class.
18
+
DiffSync acts as an intermediate translation layer between all of the data sets you are diffing and/or syncing. In practical terms, this means that to use DiffSync, you will define a set of data models as well as the “adapters” needed to translate between each base data source and the data model. In Python terms, the adapters will be subclasses of the `Adapter` class, and each data model class will be a subclass of the `DiffSyncModel` class.
Copy file name to clipboardExpand all lines: docs/source/core_engine/03-store.md
+16-12Lines changed: 16 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,32 +2,36 @@
2
2
3
3
By default, `Diffsync` supports a local memory storage. All the loaded models from the adapters will be stored in memory, and become available for the diff calculation and sync process. This default behavior works well when executing all the steps in the same process, having access to the same memory space. However, if you want to scale out the execution of the tasks, running it in different processes or in totally different workers, a more distributed memory support is necessary.
4
4
5
-
The `store` is a class attribute in the `DiffSync` class, but all the store operations in that class are abstracted in the following methods: `get_all_model_names`, `get`, `get_by_uids`, `add`, `update`, `remove`, `get_or_instantiate`, `update_or_instantiate` and `count`.
5
+
The `store` is a class attribute in the `Adapter` class, but all the store operations in that class are abstracted in the following methods: `get_all_model_names`, `get`, `get_by_uids`, `add`, `update`, `remove`, `get_or_instantiate`, `update_or_instantiate` and `count`.
6
6
7
7
## Use the `LocalStore` Backend
8
8
9
9
When you initialize the `Diffsync` Adapter class, there is an optional keyed-argument, `internal_storage_engine`, defaulting to the `LocalStore` class.
10
10
11
11
```python
12
-
>>>from diffsync import DiffSync
13
-
>>> adapter = DiffSync()
14
-
>>>type(adapter.store)
15
-
<class'diffsync.store.local.LocalStore'>
12
+
>>>from diffsync import Adapter
13
+
>>> adapter = Adapter()
14
+
>>>type(adapter.store)
15
+
<
16
+
17
+
class'diffsync.store.local.LocalStore'>
16
18
```
17
19
18
20
## Use the `RedisStore` Backend
19
21
20
22
To get it, you have to install diffsync package with the "redis" extra option: `pip install diffsync[redis]`
21
23
22
-
The `RedisStore` backend, as the name suggests, connects to an external Redis service, to store data loaded by the `DiffSync` tasks. The biggest change is that it requires to initialize the Redis store class, before using it in the `DiffSync` adapter class.
24
+
The `RedisStore` backend, as the name suggests, connects to an external Redis service, to store data loaded by the `Adapter` tasks. The biggest change is that it requires to initialize the Redis store class, before using it in the `Adapter` adapter class.
Copy file name to clipboardExpand all lines: docs/source/getting_started/01-getting-started.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
To be able to properly compare different datasets, DiffSync relies on a shared data model that both systems must use.
2
-
Specifically, each system or dataset must provide a `DiffSync` "adapter" subclass, which in turn represents its dataset as instances of one or more `DiffSyncModel` data model classes.
2
+
Specifically, each system or dataset must provide a `Adapter` "adapter" subclass, which in turn represents its dataset as instances of one or more `DiffSyncModel` data model classes.
3
3
4
4
When comparing two systems, DiffSync detects the intersection between the two systems (which data models they have in common, and which attributes are shared between each pair of data models) and uses this intersection to compare and/or synchronize the data.
5
5
@@ -41,24 +41,24 @@ Currently the relationships between models are very loose by design. Instead of
41
41
42
42
# Define your system adapter with DiffSync
43
43
44
-
A `DiffSync` "adapter" subclass must reference each model available at the top of the object by its modelname and must have a `top_level` attribute defined to indicate how the diff and the synchronization should be done. In the example below, `"site"` is the only top level object so the synchronization engine will only check all known `Site` instances and all children of each Site. In this case, as shown in the code above, `Device`s are children of `Site`s, so this is exactly the intended logic.
44
+
A `Adapter` "adapter" subclass must reference each model available at the top of the object by its modelname and must have a `top_level` attribute defined to indicate how the diff and the synchronization should be done. In the example below, `"site"` is the only top level object so the synchronization engine will only check all known `Site` instances and all children of each Site. In this case, as shown in the code above, `Device`s are children of `Site`s, so this is exactly the intended logic.
45
45
46
46
```python
47
-
from diffsync importDiffSync
47
+
from diffsync importAdapter
48
48
49
-
classBackendA(DiffSync):
50
49
50
+
classBackendA(Adapter):
51
51
site = Site
52
52
device = Device
53
53
54
54
top_level = ["site"]
55
55
```
56
56
57
-
It's up to the implementer to populate the `DiffSync`'s internal cache with the appropriate data. In the example below we are using the `load()` method to populate the cache but it's not mandatory, it could be done differently.
57
+
It's up to the implementer to populate the `Adapter`'s internal cache with the appropriate data. In the example below we are using the `load()` method to populate the cache but it's not mandatory, it could be done differently.
58
58
59
59
## Model Processing Ordering Logic
60
60
61
-
The models will be processed in a specfic order as defined by `top_level` atttribute on the `DiffSync` object and then the `_children` attribute on the `DiffSyncModel`. The processing algorithm is technically a "Preorder Tree Traversal", which means that "a parent node is processed before any of its child nodes is done." This can be described as:
61
+
The models will be processed in a specfic order as defined by `top_level` atttribute on the `Adapter` object and then the `_children` attribute on the `DiffSyncModel`. The processing algorithm is technically a "Preorder Tree Traversal", which means that "a parent node is processed before any of its child nodes is done." This can be described as:
62
62
63
63
- Start with the first element of the first model in `top_level` and process it.
64
64
- If that model has `_children` set on it, for each child of each child model, in order:
@@ -145,7 +145,7 @@ NetworkImporterAdapter
145
145
>>>
146
146
```
147
147
148
-
# Store data in a `DiffSync` object
148
+
# Store data in a `Adapter` object
149
149
150
150
To add a site to the local cache/store, you need to pass a valid `DiffSyncModel` object to the `add()` function.
151
151
@@ -174,7 +174,7 @@ convenient to manage individual records (as in a database) or modify the entire
174
174
## Manage individual records
175
175
176
176
To update individual records in a remote system, you need to extend your `DiffSyncModel` class(es) to define your own `create`, `update` and/or `delete` methods for each model.
177
-
A `DiffSyncModel` instance stores a reference to its parent `DiffSync` adapter instance in case you need to use it to look up other model instances from the `DiffSync`'s cache.
177
+
A `DiffSyncModel` instance stores a reference to its parent `Adapter` adapter instance in case you need to use it to look up other model instances from the `Adapter`'s cache.
0 commit comments