From 3e791edddae29c764e7c5e68eea07ade0047822d Mon Sep 17 00:00:00 2001 From: helloJetBase-tech <178346048+marktech0813@users.noreply.github.com> Date: Tue, 4 Nov 2025 03:00:58 +0200 Subject: [PATCH 1/2] Fixing Method about that issues : Getting an error while running this command : docker compose up -d --build #329 Added detailed information at GenerativeAIExamples/RAG/examples/basic_rag/langchain/README.md Summary - The failure is due to a GPU-required service (Milvus GPU) with no GPU available to Docker. - Required driver: NVIDIA Linux driver R550+/R535+ and NVIDIA Container Toolkit (or the Windows WSL2 NVIDIA driver if on WSL). - You can run CPU-only by switching to pgvector or Milvus CPU image and relying on NVIDIA AI Endpoints for models via NVIDIA_API_KEY. --- RAG/examples/basic_rag/langchain/README.md | 29 ++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/RAG/examples/basic_rag/langchain/README.md b/RAG/examples/basic_rag/langchain/README.md index d4e9dd2a4..600b0f4cd 100644 --- a/RAG/examples/basic_rag/langchain/README.md +++ b/RAG/examples/basic_rag/langchain/README.md @@ -5,6 +5,35 @@ # Basic RAG Using LangChain +# What driver is “missing”? +-You need the NVIDIA proprietary GPU driver on the host with CUDA 12.x support, plus the NVIDIA Container Toolkit. +-Recommended driver branch: R550+ (or at least R535+) so it’s compatible with CUDA 12.x images used by Milvus GPU and NIM. +-Also required: nvidia-container-toolkit configured with Docker so the nvidia runtime is available. +If you’re on WSL2/Windows: install the Windows NVIDIA driver with WSL2 GPU support and enable GPU for Docker Desktop/WSL. The error “WSL environment detected but no adapters were found” means no GPU is exposed to WSL/Docker. + +# How to run without a local GPU (CPU-only path) +Option A: Use pgvector (CPU) instead of Milvus GPU and keep LLM/embeddings on NVIDIA AI Endpoints: +1. Set vector DB to pgvector: + Override env: APP_VECTORSTORE_NAME=pgvector +2. Start only the pgvector service via profiles: + export NVIDIA_API_KEY=YOUR_KEY # required for NVIDIA AI Endpoints + export APP_VECTORSTORE_NAME=pgvector + docker compose --profile pgvector up -d --build +This avoids the Milvus GPU container entirely and uses CPU Postgres+pgvector. +Option B: Keep Milvus but switch to CPU image: +1. Edit RAG/examples/local_deploy/docker-compose-vectordb.yaml: + - Change image: milvusdb/milvus:v2.4.15-gpu to milvusdb/milvus:v2.4.15 + - Remove the deploy.resources.reservations.devices block. +2. Then run docker compose up -d --build. +Don’t enable the NIM microservices profiles (local-nim, nemo-retriever) unless you have a GPU—they each reserve GPUs. + +# If you do want local GPU +- Install an NVIDIA driver R550+ (or R535+) on the host; verify nvidia-smi works. +- Install and configure NVIDIA Container Toolkit: + sudo apt-get install -y nvidia-container-toolkit + sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker +- On WSL2, ensure the Windows NVIDIA driver with WSL support is installed and Docker Desktop has GPU enabled. + ## Example Features This example deploys a basic RAG pipeline for chat Q&A and serves inferencing from an NVIDIA API Catalog endpoint. From 862e2e5e8ba7592e55f80c17e9b27888e642d337 Mon Sep 17 00:00:00 2001 From: helloJetBase-tech <178346048+marktech0813@users.noreply.github.com> Date: Tue, 4 Nov 2025 03:01:58 +0200 Subject: [PATCH 2/2] Update README with clearer GPU driver requirements Reformatted the requirements section for clarity and added numbering for better readability. --- RAG/examples/basic_rag/langchain/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/RAG/examples/basic_rag/langchain/README.md b/RAG/examples/basic_rag/langchain/README.md index 600b0f4cd..0688722fb 100644 --- a/RAG/examples/basic_rag/langchain/README.md +++ b/RAG/examples/basic_rag/langchain/README.md @@ -6,9 +6,9 @@ # Basic RAG Using LangChain # What driver is “missing”? --You need the NVIDIA proprietary GPU driver on the host with CUDA 12.x support, plus the NVIDIA Container Toolkit. --Recommended driver branch: R550+ (or at least R535+) so it’s compatible with CUDA 12.x images used by Milvus GPU and NIM. --Also required: nvidia-container-toolkit configured with Docker so the nvidia runtime is available. +1. You need the NVIDIA proprietary GPU driver on the host with CUDA 12.x support, plus the NVIDIA Container Toolkit. +2. Recommended driver branch: R550+ (or at least R535+) so it’s compatible with CUDA 12.x images used by Milvus GPU and NIM. +3. Also required: nvidia-container-toolkit configured with Docker so the nvidia runtime is available. If you’re on WSL2/Windows: install the Windows NVIDIA driver with WSL2 GPU support and enable GPU for Docker Desktop/WSL. The error “WSL environment detected but no adapters were found” means no GPU is exposed to WSL/Docker. # How to run without a local GPU (CPU-only path)