You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The most powerful features in DSPy revolve around algorithmically optimizing the prompts (or weights) of LMs, especially when you're building programs that use the LMs within a pipeline.
7
8
8
-
## Remote LMs.
9
-
10
-
These models are managed services. You just need to sign up and obtain an API key.
9
+
Let's first make sure you can set up your language model. DSPy support clients for many remote and local LMs.
11
10
12
-
1.`dspy.OpenAI` for GPT-3.5 and GPT-4.
11
+
## Setting up the LM client.
13
12
14
-
2.`dspy.Cohere`
13
+
You can just call the constructor that connects to the LM. Then, use `dspy.configure` to declare this as the default LM.
15
14
16
-
3.`dspy.Anyscale` for hosted Llama2 models.
15
+
For example, to use OpenAI language models, you can do it as follows.
You need to host these models on your own GPU(s). Below, we include pointers for how to do that.
22
+
['Hello! How can I assist you today?']
21
23
22
-
1.`dspy.HFClientTGI`: for HuggingFace models through the Text Generation Inference (TGI) system. [Tutorial: How do I install and launch the TGI server?](/api/hosting_language_models_locally/TGI)
24
+
## Directly calling the LM.
23
25
24
-
2.`dspy.HFClientVLLM`: for HuggingFace models through vLLM. [Tutorial: How do I install and launch the vLLM server?](/api/hosting_language_models_locally/vLLM)
26
+
You can simply call the LM with a string to give it a raw prompt, i.e. a string.
25
27
26
-
3.`dspy.HFModel` (experimental)
28
+
```python
29
+
gpt3_turbo("hello! this is a raw prompt to GPT-3.5")
30
+
```
27
31
28
-
4.`dspy.Ollama` (experimental)
32
+
This is almost never the recommended way to interact with LMs in DSPy, but it is allowed.
29
33
30
-
5.`dspy.ChatModuleClient` (experimental): [How do I install and use MLC?](/api/hosting_language_models_locally/MLC)
34
+
## Using the LM with DSPy signatures.
31
35
32
-
If there are other clients you want added, let us know!
36
+
You can also use the LM via DSPy [signatures] and [modules], which we discuss in more depth in the remaining guides.
33
37
34
-
## Setting up the LM client.
35
-
36
-
You can just call the constructor that connects to the LM. Then, use
37
-
`dspy.configure` to declare this as the default LM.
38
-
39
-
For example, for OpenAI, you can do it as follows.
38
+
```python
39
+
# Define a module (ChainOfThought) and assign it a signature (return an answer, given a question).
response = qa(question="How many floors are in the castle David Gregory inherited?")
@@ -99,14 +100,53 @@ return the same value! That\'s by design.
99
100
To loop and generate one output at a time with the same input, bypass
100
101
the cache by making sure each request is (slightly) unique, as below.
101
102
102
-
```python
103
+
```python
103
104
for idx inrange(5):
104
105
response = qa(question="How many floors are in the castle David Gregory inherited?", config=dict(temperature=0.7+0.0001*idx))
105
-
print(response.answer)
106
+
print(f'{idx+1}.', response.answer)
106
107
```
107
108
108
-
The specific number of floors in David Gregory's inherited castle is not provided here, so further research would be needed to determine the answer.
109
-
It is not possible to determine the exact number of floors in the castle David Gregory inherited without specific information about the castle's layout and history.
110
-
The castle David Gregory inherited has 5 floors.
111
-
We need more information to determine the number of floors in the castle David Gregory inherited.
112
-
The castle David Gregory inherited has a total of 6 floors.
109
+
1. The specific number of floors in David Gregory's inherited castle is not provided here, so further research would be needed to determine the answer.
110
+
2. It is not possible to determine the exact number of floors in the castle David Gregory inherited without specific information about the castle's layout and history.
111
+
3. The castle David Gregory inherited has 5 floors.
112
+
4. We need more information to determine the number of floors in the castle David Gregory inherited.
113
+
5. The castle David Gregory inherited has a total of 6 floors.
114
+
115
+
116
+
## Remote LMs.
117
+
118
+
These models are managed services. You just need to sign up and obtain an API key.
119
+
120
+
1.`dspy.OpenAI` for GPT-3.5 and GPT-4.
121
+
122
+
2.`dspy.Cohere`
123
+
124
+
3.`dspy.Anyscale` for hosted Llama2 models.
125
+
126
+
127
+
### Local LMs.
128
+
129
+
You need to host these models on your own GPU(s). Below, we include pointers for how to do that.
130
+
131
+
1.`dspy.HFClientTGI`: for HuggingFace models through the Text Generation Inference (TGI) system. [Tutorial: How do I install and launch the TGI server?](/api/hosting_language_models_locally/TGI)
132
+
133
+
2.`dspy.HFClientVLLM`: for HuggingFace models through vLLM. [Tutorial: How do I install and launch the vLLM server?](/api/hosting_language_models_locally/vLLM)
134
+
135
+
3.`dspy.HFModel` (experimental)
136
+
137
+
4.`dspy.Ollama` (experimental)
138
+
139
+
5.`dspy.ChatModuleClient` (experimental): [How do I install and use MLC?](/api/hosting_language_models_locally/MLC)
140
+
141
+
If there are other clients you want added, let us know!
Copy file name to clipboardExpand all lines: docs/building-blocks/3-modules.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -96,20 +96,20 @@ True
96
96
97
97
The others are very similar. They mainly change the internal behavior with which your signature is implemented!
98
98
99
-
1.**`dspy.Predict`**:
99
+
1.**`dspy.Predict`**: Basic predictor. Does not modify the signature. Handles the key forms of learning (i.e., storing the instructions and demonstrations and updates to the LM).
100
100
101
-
2.**`dspy.ChainOfThought`**:
101
+
2.**`dspy.ChainOfThought`**: Teaches the LM to think step-by-step before committing to the signature's response.
102
102
103
-
3.**`dspy.ProgramOfThought`**:
103
+
3.**`dspy.ProgramOfThought`**: Teaches the LM to output code, whose execution results will dictate the response.
104
104
105
-
4.**`dspy.ReAct`**:
105
+
4.**`dspy.ReAct`**: An agent that can use tools to implement the given signature.
106
106
107
-
5.**`dspy.MultiChainComparison`**:
107
+
5.**`dspy.MultiChainComparison`**: Can compare multiple outputs from `ChainOfThought` to produce a final prediction.
108
108
109
109
110
110
We also have some function-style modules:
111
111
112
-
6.**`dspy.majority`**:
112
+
6.**`dspy.majority`**: Can do basic voting to return the most popular response from a set of predictions.
DSPy is a machine learning framework, so working in it involves training sets, development sets, and test sets.
8
+
9
+
For each example in your data, we distinguish typically between three types of values: the inputs, the intermediate labels, and the final label. You can use DSPy effectively without any intermediate or final labels, but you will need at least a few example inputs.
10
+
11
+
## How much data do I need and how do I collect data for my task?
12
+
13
+
Concretely, you can use DSPy optimizers usefully with as few as 10 example inputs, but having 50-100 examples (or even better, 300-500 examples) goes a long way.
14
+
15
+
How can you get examples like these? If your task is extremely unusual, please invest in preparing ~10 examples by hand. Often times, depending on your metric below, you just need inputs and not labels, so it's not that hard.
16
+
17
+
However, chances are that your task is not actually that unique. You can almost always find somewhat adjacent datasets on, say, HuggingFace datasets or other forms of data that you can leverage here.
18
+
19
+
If there's data whose licenses are permissive enough, we suggest you use them. Otherwise, you can also start using/deploying/demoing your system and collect some initial data that way.
20
+
21
+
## DSPy `Example` objects
22
+
23
+
The core data type for data in DSPy is `Example`. You will use **Examples** to represent items in your training set and test set.
24
+
25
+
DSPy **Examples** are similar to Python `dict`s but have a few useful utilities. Your DSPy modules will return values of the type `Prediction`, which is a special sub-class of `Example`.
26
+
27
+
When you use DSPy, you will do a lot of evaluation and optimization runs. Your individual datapoints will be of type `Example`:
28
+
29
+
```python
30
+
qa_pair = dspy.Example(question="This is a question?", answer="This is an answer.")
31
+
32
+
print(qa_pair)
33
+
print(qa_pair.question)
34
+
print(qa_pair.answer)
35
+
```
36
+
**Output:**
37
+
```text
38
+
Example({'question': 'This is a question?', 'answer': 'This is an answer.'}) (input_keys=None)
39
+
This is a question?
40
+
This is an answer.
41
+
```
42
+
43
+
Examples can have any field keys and any value types, though usually values are strings.
In traditional ML, there are separated "inputs" and "labels".
59
+
60
+
In DSPy, the `Example` objects have a `with_inputs()` method, which can mark specific fields as inputs. (The rest are just metadata or labels.)
61
+
62
+
```python
63
+
# Single Input.
64
+
print(qa_pair.with_inputs("question"))
65
+
66
+
# Multiple Inputs; be careful about marking your labels as inputs unless you mean it.
67
+
print(qa_pair.with_inputs("question", "answer"))
68
+
```
69
+
70
+
Values can be accessed using the `.`(dot) operator. You can access the value of key `name` in defined object `Example(name="John Doe", job="sleep")` through `object.name`.
71
+
72
+
To access or exclude certain keys, use `inputs()` and `labels()` methods to return new Example objects containing only input or non-input keys, respectively.
73
+
74
+
```python
75
+
article_summary = dspy.Example(article="This is an article.", summary="This is a summary.").with_inputs("article")
76
+
77
+
input_key_only = article_summary.inputs()
78
+
non_input_key_only = article_summary.labels()
79
+
80
+
print("Example object with Input fields only:", input_key_only)
81
+
print("Example object with Non-Input fields only:", non_input_key_only))
82
+
```
83
+
84
+
**Output**
85
+
```
86
+
Example object with Input fields only: Example({'article': 'This is an article.'}) (input_keys=None)
87
+
Example object with Non-Input fields only: Example({'summary': 'This is a summary.'}) (input_keys=None)
0 commit comments