Skip to content

Commit 221f689

Browse files
mogith-pnmogith-pn
authored
Merge pull request #2 from mogith-pn/Added-notebook-example
Added notebook example
2 parents 8c206d3 + f7de8bc commit 221f689

File tree

2 files changed

+361
-1
lines changed

2 files changed

+361
-1
lines changed

‎clarifai_llm_retriever_example.ipynb‎

Lines changed: 361 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,361 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# DSPy-Clarifai lm and retriever example notebook\n",
8+
"\n",
9+
"This notebook will walk you through on the integration of clarifai into DSPy which enables the DSPy users to leverage clarifai capabilities of calling llm models from clarifai platform and to utilize clarifai app as retriever for their vector search use cases."
10+
]
11+
},
12+
{
13+
"cell_type": "markdown",
14+
"metadata": {},
15+
"source": [
16+
"### Setup"
17+
]
18+
},
19+
{
20+
"cell_type": "code",
21+
"execution_count": null,
22+
"metadata": {},
23+
"outputs": [],
24+
"source": [
25+
"!pip install clarifai"
26+
]
27+
},
28+
{
29+
"cell_type": "markdown",
30+
"metadata": {},
31+
"source": [
32+
"Import necessary packages"
33+
]
34+
},
35+
{
36+
"cell_type": "code",
37+
"execution_count": 7,
38+
"metadata": {},
39+
"outputs": [],
40+
"source": [
41+
"import dspy\n",
42+
"from dspy.retrieve.clarifai_rm import ClarifaiRM "
43+
]
44+
},
45+
{
46+
"cell_type": "markdown",
47+
"metadata": {},
48+
"source": [
49+
"#### Initialize clarifai app id, user id and PAT.\n",
50+
"\n",
51+
"You can browse the portal to obtain [MODEL URL](https://clarifai.com/explore/models) for different models in clarifai community."
52+
]
53+
},
54+
{
55+
"cell_type": "code",
56+
"execution_count": 8,
57+
"metadata": {},
58+
"outputs": [],
59+
"source": [
60+
"#for the demo we are going with llama2-70b-chat\n",
61+
"MODEL_URL = \"https://clarifai.com/meta/Llama-2/models/llama2-70b-chat\" \n",
62+
"PAT = CLARIFAI_PAT\n",
63+
"USER_ID = \"YOUR_USER_ID\"\n",
64+
"APP_ID = \"YOUR_APP_ID\""
65+
]
66+
},
67+
{
68+
"cell_type": "markdown",
69+
"metadata": {},
70+
"source": [
71+
"### Data ingestion into clarifai vectordatabase\n",
72+
"\n",
73+
"To use clarifai as retriever all you have to do is ingest the documents into clarifai app that serves as your vectordatabase to retrieve similar documents.\n",
74+
"To simplify the ingestion, we are utilising the clarifaivectordatabase integration for ingestion."
75+
]
76+
},
77+
{
78+
"cell_type": "code",
79+
"execution_count": null,
80+
"metadata": {},
81+
"outputs": [],
82+
"source": [
83+
"#run this block to ingest the documents into clarifai app as chunks.\n",
84+
"# if you encounter any issue, make sure to run `pip install langchain`\n",
85+
"\n",
86+
"from langchain.text_splitter import CharacterTextSplitter\n",
87+
"from langchain.document_loaders import TextLoader\n",
88+
"from langchain.vectorstores import Clarifai as clarifaivectorstore\n",
89+
"\n",
90+
"loader = TextLoader(\"YOUR_TEXT_FILE_PATH\") #replace with your file path\n",
91+
"documents = loader.load()\n",
92+
"text_splitter = CharacterTextSplitter(chunk_size=1024, chunk_overlap=200)\n",
93+
"docs = text_splitter.split_documents(documents)\n",
94+
"\n",
95+
"clarifai_vector_db = clarifaivectorstore.from_documents(\n",
96+
" user_id=USER_ID,\n",
97+
" app_id=APP_ID,\n",
98+
" documents=docs,\n",
99+
" pat=PAT\n",
100+
")"
101+
]
102+
},
103+
{
104+
"cell_type": "markdown",
105+
"metadata": {},
106+
"source": [
107+
"#### Initialize LLM class"
108+
]
109+
},
110+
{
111+
"cell_type": "markdown",
112+
"metadata": {},
113+
"source": [
114+
"Make sure to pass all the model parameters in inference_params field of clarifaiLLM class. "
115+
]
116+
},
117+
{
118+
"cell_type": "code",
119+
"execution_count": 10,
120+
"metadata": {},
121+
"outputs": [],
122+
"source": [
123+
"\n",
124+
"llm=dspy.Clarifai(model=MODEL_URL, api_key=PAT, n=2, inference_params={\"max_tokens\":100,'temperature':0.6})"
125+
]
126+
},
127+
{
128+
"cell_type": "markdown",
129+
"metadata": {},
130+
"source": [
131+
"Initialize Clarifai Retriever model class\n"
132+
]
133+
},
134+
{
135+
"cell_type": "code",
136+
"execution_count": 12,
137+
"metadata": {},
138+
"outputs": [],
139+
"source": [
140+
"\n",
141+
"retriever_model=ClarifaiRM(clarifai_user_id=USER_ID, clarfiai_app_id=APP_ID, clarifai_pat=PAT)"
142+
]
143+
},
144+
{
145+
"cell_type": "markdown",
146+
"metadata": {},
147+
"source": [
148+
"configure dspy with llm and rm models."
149+
]
150+
},
151+
{
152+
"cell_type": "code",
153+
"execution_count": 13,
154+
"metadata": {},
155+
"outputs": [],
156+
"source": [
157+
"dspy.settings.configure(lm=llm, rm=retriever_model)"
158+
]
159+
},
160+
{
161+
"cell_type": "markdown",
162+
"metadata": {},
163+
"source": [
164+
"### Example: dspy.signature and dspy.module with clairfaiLLM"
165+
]
166+
},
167+
{
168+
"cell_type": "code",
169+
"execution_count": 18,
170+
"metadata": {},
171+
"outputs": [
172+
{
173+
"name": "stdout",
174+
"output_type": "stream",
175+
"text": [
176+
"NEGATIVE\n",
177+
"\n",
178+
"\n"
179+
]
180+
}
181+
],
182+
"source": [
183+
"sentence = \"disney again ransacks its archives for a quick-buck sequel .\" # example from the SST-2 dataset.\n",
184+
"\n",
185+
"classify = dspy.Predict('sentence -> sentiment')\n",
186+
"print(classify(sentence=sentence).sentiment)"
187+
]
188+
},
189+
{
190+
"cell_type": "markdown",
191+
"metadata": {},
192+
"source": [
193+
"### Example: Quick glimpse into how our retriever works when a query is passed to the dspy.Retrieve class\n",
194+
"\n",
195+
"Here we have used a guideline manual for procurement of works.\n",
196+
"\n",
197+
"link : https://doe.gov.in/sites/default/files/Manual%20for%20Procurement%20of%20Works_0.pdf"
198+
]
199+
},
200+
{
201+
"cell_type": "code",
202+
"execution_count": 21,
203+
"metadata": {},
204+
"outputs": [],
205+
"source": [
206+
"retrieve = dspy.Retrieve(k=1)\n",
207+
"topK_passages = retrieve(\"what are the stages in planning, sanctioning and execution of public works\").passages"
208+
]
209+
},
210+
{
211+
"cell_type": "code",
212+
"execution_count": 23,
213+
"metadata": {},
214+
"outputs": [
215+
{
216+
"name": "stdout",
217+
"output_type": "stream",
218+
"text": [
219+
"['1.11\\n\\nProcessing of Public Works\\n\\nFollowing are the stages in planning, sanctioning and execution of work.\\ni)\\n\\nPerspective Planning for works;\\n\\nii)\\n\\nPreparation of Preliminary Project Report (PPR) or Rough Cost Estimate;\\n\\niii) Acceptance of necessity and issue of in-Principle Approval;\\niv) Preparation of Detailed Project Report (DPR) or Preliminary Estimate (PE);\\n\\n15']\n"
220+
]
221+
}
222+
],
223+
"source": [
224+
"print(topK_passages)"
225+
]
226+
},
227+
{
228+
"cell_type": "markdown",
229+
"metadata": {},
230+
"source": [
231+
"## RAG dspy module using clarifai as retriever"
232+
]
233+
},
234+
{
235+
"cell_type": "markdown",
236+
"metadata": {},
237+
"source": [
238+
"Generally to construct a module in dspy, you might need to define \n",
239+
"\n",
240+
"Signature: \n",
241+
"explain the input and output fields in an intuitive way with just few words.\n",
242+
"(\"question\"-> \"answer\")\n",
243+
"\n",
244+
"Module:\n",
245+
"Module can be something where you put the signatures into action by defining a certain module which compiles and generate response for you for the given query."
246+
]
247+
},
248+
{
249+
"cell_type": "markdown",
250+
"metadata": {},
251+
"source": [
252+
"Construct a signaturre class, which defines the input fields and output fields needed. \n",
253+
"Also, give docstrings and description in verbose, so that the dspy signature could understand the context and compile best prompt for the usecase."
254+
]
255+
},
256+
{
257+
"cell_type": "code",
258+
"execution_count": 26,
259+
"metadata": {},
260+
"outputs": [],
261+
"source": [
262+
"class GenerateAnswer(dspy.Signature):\n",
263+
" \"\"\"Answer questions with short factoid answers.\"\"\"\n",
264+
"\n",
265+
" context = dspy.InputField(desc=\"may contain relevant facts about \")\n",
266+
" question = dspy.InputField()\n",
267+
" answer = dspy.OutputField(desc=\"often between 1 and 5 words\")"
268+
]
269+
},
270+
{
271+
"cell_type": "markdown",
272+
"metadata": {},
273+
"source": [
274+
"Define the module with the actions needs to be performed, here we are showing a small RAG use case where we are retrieving similar contexts using our retriever class and generating response based on the factual context using one of the DSPy module `ChainOfThought`."
275+
]
276+
},
277+
{
278+
"cell_type": "code",
279+
"execution_count": 27,
280+
"metadata": {},
281+
"outputs": [],
282+
"source": [
283+
"class RAG(dspy.Module):\n",
284+
" def __init__(self, num_passages=3):\n",
285+
" super().__init__()\n",
286+
"\n",
287+
" self.retrieve = dspy.Retrieve(k=num_passages)\n",
288+
" self.generate_answer = dspy.ChainOfThought(GenerateAnswer)\n",
289+
" \n",
290+
" def forward(self, question):\n",
291+
" context = self.retrieve(question).passages\n",
292+
" prediction = self.generate_answer(context=context, question=question)\n",
293+
" return dspy.Prediction(context=context, answer=prediction.answer)"
294+
]
295+
},
296+
{
297+
"cell_type": "markdown",
298+
"metadata": {},
299+
"source": [
300+
"Now we are passing our query and retrieving relevant chunks using clarifai retriever and based on factual evidence, model is able to generate response."
301+
]
302+
},
303+
{
304+
"cell_type": "code",
305+
"execution_count": 28,
306+
"metadata": {},
307+
"outputs": [
308+
{
309+
"name": "stdout",
310+
"output_type": "stream",
311+
"text": [
312+
"Question: Which bid will be termed as L1 ?\n",
313+
"Predicted Answer: The bidder who quotes the lowest price among\n",
314+
"Retrieved Contexts (truncated): ['If L1 bid is not a ‘Class-I local supplier’, 50 (fifty) percent of the order quantity\\nshall be awarded to L1. Thereafter, the lowest bidder among the ‘Class-I local\\nsupplier’ will be invited to match ...', 'If L1 bid is not a ‘Class-I local supplier’, 50 (fifty) percent of the order quantity\\nshall be awarded to L1. Thereafter, the lowest bidder among the ‘Class-I local\\nsupplier’ will be invited to match ...', 'If L1 bid is not a ‘Class-I local supplier’, 50 (fifty) percent of the order quantity\\nshall be awarded to L1. Thereafter, the lowest bidder among the ‘Class-I local\\nsupplier’ will be invited to match ...']\n"
315+
]
316+
}
317+
],
318+
"source": [
319+
"# Ask any question you like to this RAG program.\n",
320+
"my_question = \"Which bid will be termed as L1?\"\n",
321+
"\n",
322+
"# Get the prediction. This contains `pred.context` and `pred.answer`.\n",
323+
"obj= RAG()\n",
324+
"pred=obj(my_question)\n",
325+
"\n",
326+
"# Print the contexts and the answer.\n",
327+
"print(f\"Question: {my_question}\")\n",
328+
"print(f\"Predicted Answer: {pred.answer}\")\n",
329+
"print(f\"Retrieved Contexts (truncated): {[c[:200] + '...' for c in pred.context]}\")"
330+
]
331+
},
332+
{
333+
"cell_type": "code",
334+
"execution_count": null,
335+
"metadata": {},
336+
"outputs": [],
337+
"source": []
338+
}
339+
],
340+
"metadata": {
341+
"kernelspec": {
342+
"display_name": "Python 3",
343+
"language": "python",
344+
"name": "python3"
345+
},
346+
"language_info": {
347+
"codemirror_mode": {
348+
"name": "ipython",
349+
"version": 3
350+
},
351+
"file_extension": ".py",
352+
"mimetype": "text/x-python",
353+
"name": "python",
354+
"nbconvert_exporter": "python",
355+
"pygments_lexer": "ipython3",
356+
"version": "3.10.13"
357+
}
358+
},
359+
"nbformat": 4,
360+
"nbformat_minor": 2
361+
}

‎dspy/retrieve/clarifai_rm.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,6 @@ def forward(
8181
if isinstance(query_or_queries, str)
8282
else query_or_queries
8383
)
84-
self.clarifai_search.top_k = k if k is not None else self.clarifai_search.top_k
8584
passages = []
8685
queries = [q for q in queries if q]
8786

0 commit comments

Comments
 (0)