You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+27-6Lines changed: 27 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -67,32 +67,53 @@ For a more detailed example with test data, please refer to the Confluence docum
67
67
68
68
### Index records into local OpenSearch Docker container
69
69
70
-
1. Follow the instructions in either [Running Opensearch locally with Docker](#running-opensearch-locally-with-docker) or [Running Opensearch and OpenSearch Dashboards locally with Docker](#running-opensearch-and-opensearch-dashboards-locally-with-docker).
70
+
This CLI provides a couple of ways to index records into a local Opensearch instance:
71
71
72
-
2. Open a new terminal, and create a new index. Copy the name of the created index printed to the terminal's output.
72
+
1. A step-by-step approach that mirrors the actions in the TIMDEX ETL StepFunction
73
+
2. A single, convenience command that fully reindexes a source
74
+
75
+
For both, first follow the instructions in either [Running Opensearch locally with Docker](#running-opensearch-locally-with-docker) or [Running Opensearch and OpenSearch Dashboards locally with Docker](#running-opensearch-and-opensearch-dashboards-locally-with-docker), and then open a new terminal for the following commands.
76
+
77
+
#### Option 1: Step-by-Step Bulk Index
78
+
79
+
1. Create a new index. Copy the name of the created index printed to the terminal's output.
73
80
74
81
```shell
75
82
pipenv run tim create -s <source-name>
76
83
```
77
84
78
-
3. Copy the index name and promote the index to the alias.
85
+
2. Copy the index name and promote the index to the alias.
79
86
80
87
```shell
81
88
pipenv run tim promote -a <source-name> -i <index-name>
82
89
```
83
90
84
-
4. Bulk index records from a specified directory (e.g., including S3).
91
+
3. Bulk index records from a TIMDEX dataset (can be local or from S3).
85
92
86
93
```shell
87
-
pipenv run tim bulk-index -s <source-name><filepath-to-records>
94
+
pipenv run tim bulk-index \
95
+
--source <source-name> \
96
+
--run-id <run-id> \
97
+
--run-date <YYYY-MM-DD-run-date> \
98
+
<dataset-location>
88
99
```
89
100
90
-
5. After verifying that the bulk-index was successful, clean up your local OpenSearch instance by deleting the index.
101
+
4. After verifying that the bulk-index was successful, clean up your local OpenSearch instance by deleting the index.
91
102
92
103
```shell
93
104
pipenv run tim delete -i <index-name>
94
105
```
95
106
107
+
#### Option 2: Fully Reindex a Source
108
+
109
+
1. Utilize the CLI command `reindex-source`:
110
+
111
+
```shell
112
+
pipenv run tim --verbose reindex-source \
113
+
--source <source-name> \
114
+
<dataset-location>
115
+
```
116
+
96
117
### Running OpenSearch on AWS
97
118
98
119
1. Ensure that you have the correct AWS credentials set for the Dev1 (or desired) account.
0 commit comments