You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# title and description do not need to be added to markdown, start with H2 (##)
5
5
title: Build Vector Search with Couchbase .NET Semantic Kernel Connector and OpenAI
6
6
short_title: Vector Search with Semantic Kernel
7
7
description:
8
-
- Build a semantic search application using Couchbase Hyperscale vector index with Semantic Kernel.
8
+
- Build a semantic search application using Couchbase Vector Search with Semantic Kernel.
9
9
- Learn to use the Couchbase .NET Vector Store Connector for Microsoft Semantic Kernel.
10
10
- Discover how to generate embeddings with OpenAI and store them in Couchbase.
11
11
- Perform vector similarity searches with filtering using SQL++ and ANN_DISTANCE.
12
12
content_type: tutorial
13
13
filter: sdk
14
14
technology:
15
+
- hyperscale vector index
16
+
- composite vector index
15
17
- fts
16
18
- kv
17
19
tags:
@@ -35,43 +37,43 @@ This demo showcases the **Semantic Kernel Couchbase connector** - a .NET library
35
37
The connector supports three index types:
36
38
-**Hyperscale Vector Index** - for pure vector search at scale ← *Used in this demo*
37
39
-**Composite Vector Index** - for vector search with heavy scalar filtering
38
-
-**FTS** (Full-Text Search) - for hybrid text + semantic search
40
+
-**Search Vector Index** (using Search service) - for hybrid text + semantic search
39
41
40
42
This makes the connector ideal for RAG (Retrieval-Augmented Generation) applications, semantic search engines, hybrid search, and recommendation systems.
41
43
42
44
## Prerequisites
43
45
44
-
### 1. Couchbase Server Setup
46
+
### Couchbase Server Setup
45
47
-**Couchbase Server 8.0+**
46
48
- Local installation or Couchbase Cloud/Capella
47
49
- Bucket with proper read/write permissions
48
50
- Query service enabled for SQL++ operations
49
51
50
-
### 2. OpenAI API Access
52
+
### OpenAI API Access
51
53
-**OpenAI API Key** - Get one from: https://platform.openai.com/api-keys
52
54
- Used for generating text embeddings with `text-embedding-3-small` model
53
55
- Ensure you have sufficient API quota for embedding generation
cd couchbase-semantic-kernel-quickstart/CouchbaseVectorSearchDemo
67
69
```
68
70
69
-
### 2. Install Dependencies
71
+
### Install Dependencies
70
72
```bash
71
73
dotnet restore
72
74
```
73
75
74
-
### 3. Configuration Setup
76
+
### Configuration Setup
75
77
76
78
Update `appsettings.Development.json` with your credentials:
77
79
@@ -124,14 +126,14 @@ internal sealed class Glossary
124
126
125
127
## Step-by-Step Tutorial
126
128
127
-
### Step 1: Prepare Couchbase
129
+
### Prepare Couchbase
128
130
129
131
Ensure you have the bucket, scope, and collection ready in Couchbase:
130
132
-**Bucket**: `demo`
131
133
-**Scope**: `semantic-kernel`
132
134
-**Collection**: `glossary`
133
135
134
-
### Step 2: Data Ingestion and Embedding Generation
136
+
### Data Ingestion and Embedding Generation
135
137
136
138
This step demonstrates how the connector works with Semantic Kernel's vector store abstractions:
137
139
@@ -142,13 +144,12 @@ var collection = vectorStore.GetCollection<string, Glossary>(
142
144
"glossary",
143
145
newCouchbaseQueryCollectionOptions
144
146
{
145
-
IndexName="hyperscale_glossary_index", // Hyperscale index name
146
147
SimilarityMetric="cosine"
147
148
}
148
149
);
149
150
```
150
151
151
-
The `CouchbaseQueryCollectionOptions` works with both Hyperscale and Composite indexes - simply specify the appropriate index name. For FTS indexes, use `CouchbaseSearchCollection` with `CouchbaseSearchCollectionOptions` instead.
152
+
The `CouchbaseQueryCollectionOptions` works with both Hyperscale and Composite indexes - simply specify the appropriate index name. For Search Vector indexes, use `CouchbaseSearchCollection` with `CouchbaseSearchCollectionOptions` instead.
152
153
153
154
**Automatic Embedding Generation** - The connector integrates with Semantic Kernel's `IEmbeddingGenerator` interface to automatically generate embeddings from text. When you provide an embedding generator (in this case, OpenAI's `text-embedding-3-small`), the text is automatically converted to vectors:
154
155
@@ -178,9 +179,13 @@ This creates 6 sample glossary entries with technical terms, generates embedding
178
179
}
179
180
```
180
181
181
-
### Step 3: Hyperscale Index Creation
182
+
### Hyperscale Index Creation
182
183
183
-
This demo uses a **Hyperscale Vector Index** - optimized for pure vector searches without heavy scalar filtering. After documents are inserted, the demo creates the Hyperscale index:
184
+
While the application works without creating indexes manually, you can optionally create a vector index for better performance.
185
+
186
+
This demo uses a **Hyperscale Vector Index** - optimized for pure vector searches without heavy scalar filtering.
187
+
188
+
After documents are inserted, the demo creates the Hyperscale index:
184
189
185
190
```sql
186
191
CREATE VECTOR INDEX `hyperscale_glossary_index`
@@ -200,9 +205,9 @@ USING GSI WITH {
200
205
-**Include Fields**: Non-vector fields for faster retrieval
201
206
-**Quantization**: `IVF,SQ8` (Inverted File with 8-bit scalar quantization)
202
207
203
-
> **Note**: [Composite vector indexes](https://docs.couchbase.com/server/current/vector-index/composite-vector-index.html) can be created similarly by adding scalar fields to the index definition. Use Composite indexes when your queries frequently filter on scalar values before vector comparison. For this demo, we use Hyperscale since we're demonstrating pure semantic search capabilities.
208
+
> **Note**: [Composite vector indexes](https://docs.couchbase.com/server/current/vector-index/composite-vector-index.html) can be created similarly by adding scalar fields to the index definition. Use Composite indexes when your queries frequently filter on scalar values before vector comparison. For this demo, we use Hyperscale since we are demonstrating pure semantic search capabilities.
204
209
205
-
### Step 4: Vector Search Operations
210
+
### Vector Search Operations
206
211
207
212
The demo performs two types of searches using the connector's `SearchAsync()` method with the Hyperscale index:
208
213
@@ -228,6 +233,8 @@ ORDER BY _distance ASC
228
233
LIMIT1
229
234
```
230
235
236
+
> **Note**: The distance metric (`'cosine'` in this example) comes from the `SimilarityMetric` property configured when creating the collection:
237
+
231
238
**Expected Result**: Finds "API" entry with high similarity
232
239
233
240
#### Filtered Vector Search
@@ -280,7 +287,7 @@ Couchbase offers three types of vector indexes optimized for different use cases
0 commit comments