Skip to content

Commit 1962b47

Browse files
author
Julien Ruaux
committed
docs: Updated README
1 parent efcb151 commit 1962b47

File tree

9 files changed

+265
-136
lines changed

9 files changed

+265
-136
lines changed

‎Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
ARG TRINO_VERSION=396
1+
ARG TRINO_VERSION=395
22

33
FROM docker.io/library/maven:3.8.6-openjdk-18 AS builder
44
WORKDIR /root/trino-redisearch

‎README.adoc‎

Lines changed: 190 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -4,65 +4,94 @@
44
:project-name: redis-sql
55
:project-group: com.redis
66
:project-version: 0.2.5
7+
:project-url: https://github.com/{project-owner}/{project-name}
78
:artifact-id: trino-redisearch
8-
:codecov-token: CQFF495IPH
9+
:trino-dir: /usr/lib/trino
10+
:trino-datadir: /var/trino
11+
:trino-version: 395
912

10-
image:https://github.com/{project-owner}/{project-name}/actions/workflows/early-access.yml/badge.svg["Build Status", link="https://github.com/{project-owner}/{project-name}/actions/workflows/early-access.yml"]
13+
image:{project-url}/actions/workflows/early-access.yml/badge.svg["Build Status", link="{project-url}/actions/workflows/early-access.yml"]
1114
image:https://codecov.io/gh/{project-owner}/{project-name}/branch/master/graph/badge.svg?token={codecov-token}["Coverage", link="https://codecov.io/gh/{project-owner}/{project-name}"]
1215

13-
Redis SQL is a SQL interface to https://oss.redislabs.com/redisearch/[RediSearch] implemented as a https://trino.io[Trino] connector.
14-
It allows the use of RediSearch indexes as tables in Trino.
16+
Redis SQL is a https://trino.io[Trino] connector which allows access to RediSearch data from Trino.
17+
This document describes how to set up the RediSearch Connector to run SQL queries against RediSearch.
1518

16-
== Requirements
17-
18-
To connect to RediSearch you need:
19-
20-
. RediSearch 2.0 or higher
21-
. Network access from the Trino coordinator and workers to RediSearch.
19+
NOTE: RediSearch 2.0 or later is required.
2220

2321
== Configuration
24-
25-
To configure the RediSearch connector, create a catalog properties file `etc/catalog/redisearch.properties` with the following contents (replace properties as appropriate):
22+
To configure the RediSearch connector, create a catalog properties file `etc/catalog/redisearch.properties` with the following contents, replacing the properties as appropriate:
2623

2724
[source,properties]
2825
----
2926
connector.name=redisearch
30-
redisearch.uri=redis://user:pass@host:6379
27+
redisearch.uri=redis://localhost:6379
28+
redisearch.default-schema-name=default
3129
----
3230

33-
=== Configuration Properties
31+
[[properties]]
32+
=== Configuration properties
3433

35-
`redisearch.uri`:: A connection string containing the protocol, credential, and host info to connect to Redis.
36-
+
37-
For example, the connection string may use the format `redis://<user>:<pass>@<host>:<port>`.
38-
+
39-
See the https://github.com/lettuce-io/lettuce-core/wiki/Redis-URI-and-connection-details#uri-syntax[Redis URI syntax] for more information.
40-
+
41-
This property is required and has no default. A Redis URI must be provided to connect to RediSearch.
34+
[cols="1,1,1"]
35+
|===
36+
|Property name |Description |Default
4237

43-
`redisearch.username`:: Redis connection username.
38+
|`redisearch.uri`
39+
|A Redis connection string. https://github.com/lettuce-io/lettuce-core/wiki/Redis-URI-and-connection-details#uri-syntax[Redis URI syntax].
40+
|
4441

45-
`redisearch.password`:: Redis connection password.
42+
|`redisearch.default-schema-name`
43+
|The schema that contains all tables defined without a qualifying schema name.
44+
|`default`
4645

47-
`redisearch.insecure`:: Allow insecure connections (e.g. invalid certificates) to Redis when using SSL.
48-
+
49-
This property is optional and the default is `false`.
46+
|`redisearch.username`
47+
|Redis connection username.
48+
|
5049

51-
`redisearch.timeout`:: Redis command timeout in seconds.
52-
+
53-
This property is optional and the default is `60`.
50+
|`redisearch.password`
51+
|Redis connection password.
52+
|
5453

55-
`redisearch.case-insensitive-name-matching`:: Match index names case insensitively.
56-
+
57-
This property is optional and the default is `false`.
54+
|`redisearch.cluster`
55+
|Connect to a Redis Cluster.
56+
|`false`
5857

59-
`redisearch.default-limit`:: Max number of documents returned by FT.SEARCH and FT.AGGREGATE when no limit is present in the SQL query.
60-
+
61-
This property is optional and the default is `10000`.
58+
|`redisearch.case-insensitive-names`
59+
|Match index names case insensitively.
60+
|`false`
6261

63-
`redisearch.cursor-count`:: Number of rows read during each https://redis.io/docs/stack/search/reference/aggregations/#cursor-api[aggregation cursor] fetch.
64-
+
65-
This property is optional and if it is not specified the RediSearch default is used.
62+
|`redisearch.default-limit`
63+
|Max number of documents returned by FT.SEARCH and FT.AGGREGATE when no limit is present in the SQL query.
64+
|`10000`
65+
66+
|`redisearch.cursor-count`
67+
|Number of rows read during each https://redis.io/docs/stack/search/reference/aggregations/#cursor-api[aggregation cursor] fetch.
68+
|`1000`
69+
|===
70+
71+
== TLS security
72+
The RediSearch connector provides additional security options to support Redis servers with TLS mode.
73+
74+
The allowed configuration values are:
75+
76+
[cols="1,1"]
77+
|===
78+
|Property name |Description
79+
80+
|`redisearch.insecure`
81+
|Allow insecure connections (e.g. invalid certificates) when using SSL.
82+
83+
|`redisearch.cacert-path`
84+
|X.509 CA certificate file to verify with.
85+
86+
|`redisearch.key-path`
87+
|PKCS#8 private key file to authenticate with (PEM format).
88+
89+
|`redisearch.key-password`
90+
|Password of the private key file, or null if it's not password-protected.
91+
92+
|`redisearch.cert-path`
93+
|X.509 certificate chain file to authenticate with (PEM format).
94+
|===
6695

6796

6897
== Docker Example
@@ -102,68 +131,163 @@ docker exec -it trino trino --catalog redisearch --schema default
102131
trino:default> select * from beers;
103132
----
104133

134+
== Tableau
135+
136+
Follow these steps to connect Tableau: https://help.tableau.com/current/pro/desktop/en-us/examples_presto.htm
137+
105138
== Install and Run
106139

107-
=== Install Trino
140+
=== Trino
141+
142+
Trino installation instructions are available here: https://trino.io/docs/current/installation.html
108143

109-
https://trino.io/docs/current/installation.html
144+
The following steps deploy a single-node Trino server on Ubuntu.
110145

111-
For macOS: `brew install trino`
146+
Trino requires a 64-bit of Java 17.
147+
It is recommended to https://www.azul.com/downloads/?package=jdk[Azul Zulu] as the JDK.
148+
[source,console]
149+
----
150+
$ java -version
151+
openjdk version "17.0.4.1" 2022-08-12 LTS
152+
OpenJDK Runtime Environment Zulu17.36+17-CA (build 17.0.4.1+1-LTS)
153+
OpenJDK 64-Bit Server VM Zulu17.36+17-CA (build 17.0.4.1+1-LTS, mixed mode, sharing)
154+
----
112155

113-
=== Install the connector
156+
Download the Trino server tarball and unpack it.
157+
[source,console,subs="verbatim,attributes"]
158+
----
159+
wget https://repo1.maven.org/maven2/io/trino/trino-server/{trino-version}/trino-server-{trino-version}.tar.gz
160+
mkdir {trino-dir}
161+
tar xzvf trino-server-{trino-version}.tar.gz --directory {trino-dir} --strip-components 1
162+
----
114163

115-
1. Download latest https://github.com/redis-field-engineering/{project-name}/releases/latest[release].
164+
Trino needs a data directory for storing logs, etc.
165+
It is recommended to create a data directory outside of the installation directory, which allows it to be easily preserved when upgrading Trino.
166+
[source,console,subs="verbatim,attributes"]
167+
----
168+
mkdir -p {trino-datadir}
169+
----
116170

117-
2. Unzip the release without any directory structure under `<trino>/plugin/redisearch`:
118-
+
171+
Create an `etc` directory inside the installation directory to hold configuration files.
119172
[source,console,subs="verbatim,attributes"]
120173
----
121-
unzip -j trino-redisearch-{project-version}.zip -d /opt/trino/plugin/redisearch
174+
mkdir {trino-dir}/etc
122175
----
123176

124-
3. Create a `redisearch.properties` file under `<trino>/etc/catalog` directory:
125-
+
126-
.redisearch.properties
177+
Create a node properties file `etc/node.properties`.
178+
179+
.{trino-dir}/etc/node.properties
180+
[source,properties,subs="verbatim,attributes"]
181+
----
182+
node.environment=production
183+
node.id=ffffffff-ffff-ffff-ffff-ffffffffffff
184+
node.data-dir={trino-datadir}
185+
----
186+
187+
Create a JVM config file `etc/jvm.config`.
188+
189+
.{trino-dir}/etc/jvm.config
190+
[source,properties]
191+
----
192+
-server
193+
-Xmx16G
194+
-XX:InitialRAMPercentage=80
195+
-XX:MaxRAMPercentage=80
196+
-XX:G1HeapRegionSize=32M
197+
-XX:+ExplicitGCInvokesConcurrent
198+
-XX:+ExitOnOutOfMemoryError
199+
-XX:+HeapDumpOnOutOfMemoryError
200+
-XX:-OmitStackTraceInFastThrow
201+
-XX:ReservedCodeCacheSize=512M
202+
-XX:PerMethodRecompilationCutoff=10000
203+
-XX:PerBytecodeRecompilationCutoff=10000
204+
-Djdk.attach.allowAttachSelf=true
205+
-Djdk.nio.maxCachedBufferSize=2000000
206+
-XX:+UnlockDiagnosticVMOptions
207+
-XX:+UseAESCTRIntrinsics
208+
----
209+
210+
Create a config properties file `etc/config.properties`.
211+
.{trino-dir}/etc/config.properties
212+
[source,properties]
213+
----
214+
coordinator=true
215+
node-scheduler.include-coordinator=true
216+
http-server.http.port=8080
217+
discovery.uri=http://localhost:8080
218+
----
219+
220+
Create a log levels file `etc/log.properties`.
221+
.{trino-dir}/etc/log.properties
222+
[source,properties]
223+
----
224+
io.trino=INFO
225+
----
226+
227+
=== Redis SQL
228+
229+
Download latest {project-url}/releases/latest[release] and unzip without any directory structure under `<trino>/plugin/redisearch`.
230+
231+
[source,console,subs="verbatim,attributes"]
232+
----
233+
wget {project-url}/releases/download/v{trino-version}/{artifact-id}-{trino-version}.zip
234+
unzip -j {artifact-id}-{project-version}.zip -d {trino-dir}/plugin/redisearch
235+
----
236+
237+
Create a `redisearch.properties` file under `{trino-dir}/etc/catalog` directory:
238+
127239
[source,properties]
128240
----
129241
connector.name=redisearch
130242
redisearch.uri=redis://localhost:6379
131243
----
132244

245+
Change and/or add <<properties,properties>> as needed.
246+
133247
=== Run
134248

135-
1. Start Trino server:
136-
+
137-
[source,console]
249+
==== Trino Server
250+
251+
Start the Trino server by running:
252+
253+
[source,console,subs="verbatim,attributes"]
138254
----
139-
trino-server start
255+
{trino-dir}/bin/launcher run
140256
----
141257

142-
2. Connect to Trino using the CLI:
143-
+
144-
[source,console]
258+
==== Trino CLI
259+
260+
Download https://repo1.maven.org/maven2/io/trino/trino-cli/{trino-version}/trino-cli-{trino-version}-executable.jar[trino-cli-{trino-version}-executable.jar], rename it to `trino`, make it executable with `chmod +x`, and run it to show the version of the CLI:
261+
262+
[source,console,subs="verbatim,attributes"]
145263
----
146-
trino --catalog redisearch --schema default
264+
wget https://repo1.maven.org/maven2/io/trino/trino-cli/{trino-version}/trino-cli-{trino-version}-executable.jar
265+
mv trino-cli-{trino-version}-executable.jar trino
266+
chmod +x trino
147267
----
148268

149-
3. Run a SQL query:
150-
+
269+
Connect to Trino using the CLI:
270+
151271
[source,console]
152272
----
153-
trino:default> select count(*) from myIdx;
273+
./trino --catalog redisearch --schema default
154274
----
155275

156-
== Tableau
276+
Run a SQL query:
157277

158-
Follow these steps to connect Tableau: https://help.tableau.com/current/pro/desktop/en-us/examples_presto.htm
278+
[source,console]
279+
----
280+
trino:default> select * from mySearchIndex;
281+
----
159282

160283
== Build
161284

162-
Run these commands to build the Trino connector for RediSearch from source:
285+
Run these commands to build the Trino connector for RediSearch from source (requires Java 17+):
163286

164-
[source,console]
287+
[source,console,subs="verbatim,attributes"]
165288
----
166-
git clone https://github.com/redis-field-engineering/redis-sql.git
167-
cd redis-sql
289+
git clone {project-url}.git
290+
cd {project-name}
168291
./gradlew clean build
169292
----
293+

‎docker/etc/catalog/redisearch.properties‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,6 @@
11
connector.name=redisearch
2-
redisearch.uri=redis://host.docker.internal:6379
2+
redisearch.uri=redis://host.docker.internal:6379
3+
redisearch.cluster=false
4+
redisearch.insecure=false
5+
redisearch.case-insensitive-names=false
6+
redisearch.default-limit=10000

‎docker/setup.sh‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,24 @@
22
set -e
33

44
REDISEARCH_ENVS=0
5-
if [ ! -z "${REDISEARCH_URI}" ] ; then
5+
if [ ! -z "${REDISEARCH_URI}" ] || [ ! -z "${REDISEARCH_USERNAME}" ] || [ ! -z "${REDISEARCH_PASSWORD}" ] || [ ! -z "${REDISEARCH_CLUSTER}" ] \
6+
|| [ ! -z "${REDISEARCH_INSECURE}" ] || [ ! -z "${REDISEARCH_CACERT_PATH}" ] || [ ! -z "${REDISEARCH_KEY_PATH}" ] || [ ! -z "${REDISEARCH_KEY_PASSWORD}" ] || [ ! -z "${REDISEARCH_CERT_PATH}" ] \
7+
|| [ ! -z "${REDISEARCH_CASE_INSENSITIVE_NAMES}" ] || [ ! -z "${REDISEARCH_DEFAULT_LIMIT}" ] || [ ! -z "${REDISEARCH_CURSOR_COUNT}" ]; then
68
REDISEARCH_ENVS=1
79
fi
810

911
export REDISEARCH_URI=${REDISEARCH_URI:-redis://docker.for.mac.host.internal:6379}
12+
export REDISEARCH_USERNAME=${REDISEARCH_USERNAME}
13+
export REDISEARCH_PASSWORD=${REDISEARCH_PASSWORD}
14+
export REDISEARCH_CLUSTER=${REDISEARCH_CLUSTER:-false}
15+
export REDISEARCH_INSECURE=${REDISEARCH_INSECURE:-false}
16+
export REDISEARCH_CACERT_PATH=${REDISEARCH_CACERT_PATH}
17+
export REDISEARCH_KEY_PATH=${REDISEARCH_KEY_PATH}
18+
export REDISEARCH_KEY_PASSWORD=${REDISEARCH_KEY_PASSWORD}
19+
export REDISEARCH_CERT_PATH=${REDISEARCH_CERT_PATH}
20+
export REDISEARCH_CASE_INSENSITIVE_NAMES=${REDISEARCH_CASE_INSENSITIVE_NAMES:-false}
21+
export REDISEARCH_CURSOR_COUNT=${REDISEARCH_CURSOR_COUNT:-1000}
22+
export REDISEARCH_DEFAULT_LIMIT=${REDISEARCH_DEFAULT_LIMIT:-10000}
1023

1124
if [ -f /tmp/redisearch.properties.template ] && [ $REDISEARCH_ENVS -eq 1 ]; then
1225
envsubst < /tmp/redisearch.properties.template > /etc/trino/catalog/redisearch.properties

‎docker/template/redisearch.properties.template‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,13 @@
11
connector.name=redisearch
2-
redisearch.uri=${REDISEARCH_URI}
2+
redisearch.uri=${REDISEARCH_USERNAME}
3+
redisearch.username=${REDISEARCH_USERNAME}
4+
redisearch.password=${REDISEARCH_PASSWORD}
5+
redisearch.cluster=${REDISEARCH_CLUSTER}
6+
redisearch.insecure=${REDISEARCH_INSECURE}
7+
redisearch.cacert-path=${REDISEARCH_CACERT_PATH}
8+
redisearch.key-path=${REDISEARCH_KEY_PATH}
9+
redisearch.key-password=${REDISEARCH_KEY_PASSWORD}
10+
redisearch.cert-path=${REDISEARCH_CERT_PATH}
11+
redisearch.case-insensitive-names=${REDISEARCH_CASE_INSENSITIVE_NAMES}
12+
redisearch.cursor-count=${REDISEARCH_CURSOR_COUNT}
13+
redisearch.default-limit=${REDISEARCH_DEFAULT_LIMIT}

‎pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
<parent>
66
<groupId>io.trino</groupId>
77
<artifactId>trino-root</artifactId>
8-
<version>397</version>
8+
<version>395</version>
99
</parent>
1010

1111
<groupId>com.redis</groupId>

0 commit comments

Comments
 (0)