Skip to content

Commit c57abf9

Browse files
committed
Update deploy_langchain_application.rst
1 parent 900ccfd commit c57abf9

File tree

1 file changed

+30
-13
lines changed

1 file changed

+30
-13
lines changed

docs/source/user_guide/large_language_model/deploy_langchain_application.rst

Lines changed: 30 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,49 +2,56 @@
22
Deploy LangChain Application
33
############################
44

5-
Oracle ADS SDK now supports the deployment of LangChain application to OCI data science model deployment and you can easily do so just by writing a couple lines of code.
5+
Oracle ADS supports the deployment of LangChain application to OCI data science model deployment and you can easily do so just by writing a few lines of code.
66

77
.. versionadded:: 2.9.1
88

9+
.. admonition:: Installation
10+
:class: note
11+
12+
It is important to note that for ADS to serialize and deploy the LangChain application, all components used to build the application must be serializable. For more information regarding LLMs model serialization, see `here <https://python.langchain.com/docs/modules/model_io/llms/llm_serialization>`_.
13+
914
Configuration
1015
*************
1116

1217
Ensure that you have created the necessary `policies, authentication, and authorization for model deployments <https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-policies-auth.htm#model_dep_policies_auth>`_.
13-
Here we're using the ``resource_principal`` as auth type and you can configure the policy as below.
18+
For example, the following policy allows the dynamic group to use ``resource_principal`` to create model deployment.
1419

1520
.. code-block:: shell
1621
1722
allow dynamic-group <dynamic-group-name> to manage data-science-model-deployments in compartment <compartment-name>
1823
19-
Create LangChain Application
20-
****************************
24+
LangChain Application
25+
*********************
2126

22-
Create a simple LangChain application that links prompt and cohere model as below. Remember to replace the ``<cohere_api_key>`` with the actual cohere api key.
27+
Following is a simple LangChain application that build with a prompt template and large language model API. Here the ``Cohere`` model is used as an example. You may replace it with any other LangChain compatible LLM, including OCI Generative AI service.
2328

2429
.. code-block:: python3
2530
2631
import os
2732
from langchain.llms import Cohere
2833
from langchain.chains import LLMChain
2934
from langchain.prompts import PromptTemplate
35+
# Remember to replace the ``<cohere_api_key>`` with the actual cohere api key.
3036
os.environ["COHERE_API_KEY"] = "<cohere_api_key>"
3137
3238
cohere = Cohere()
3339
prompt = PromptTemplate.from_template("Tell me a joke about {subject}")
3440
llm_chain = LLMChain(prompt=prompt, llm=cohere, verbose=True)
3541
36-
Now you have a LangChain object ``llm_chain``. Try running it with the prompt ``{"subject": "animals"}`` and it should give you the corresponding result.
42+
Now you have a LangChain object ``llm_chain``. Try running it with the input ``{"subject": "animals"}`` and it should return a joke about animals.
3743

3844
.. code-block:: python3
3945
40-
llm_chain.run({"subject": "animals"})
46+
llm_chain.invoke({"subject": "animals"})
4147
4248
Initialize the ChainDeployment
4349
******************************
4450

45-
Initialize class ``ChainDeployment`` from ADS SDK and pass the LangChain object ``llm_chain`` from previous step as parameter.
46-
The ``artifact_dir`` is an optional parameter which points to the folder where the model artifacts will be put locally.
47-
In this example, we're using a temporary folder generated by ``tempfile``.
51+
ADS provides the ``ChainDeployment`` to handle the deployment of LangChain applications.
52+
You can initialize ``ChainDeployment`` with the LangChain object ``llm_chain`` from previous section as parameter.
53+
The ``artifact_dir`` is an optional parameter which points to the folder where the model artifacts will be put locally.
54+
In this example, we're using a temporary folder generated by ``tempfile.mkdtemp()``.
4855

4956
.. code-block:: python3
5057
@@ -61,8 +68,8 @@ In this example, we're using a temporary folder generated by ``tempfile``.
6168
Prepare the Model Artifacts
6269
***************************
6370

64-
Call ``prepare`` from ``ChainDeployment`` to generate the ``score.py`` and serialize the LangChain application to ``chain.yaml`` file under ``artifact_dir`` folder.
65-
Parameters ``inference_conda_env`` and ``inference_python_version`` are passed to define the conda environment where your LangChain application will be running on OCI cloud.
71+
Call ``prepare()`` from ``ChainDeployment`` to generate the ``score.py`` and serialize the LangChain application to ``chain.yaml`` file under ``artifact_dir`` folder.
72+
Parameters ``inference_conda_env`` and ``inference_python_version`` are passed to define the conda environment where your LangChain application will be running on OCI.
6673
Here we're using ``pytorch21_p39_gpu_v1`` with python 3.9.
6774

6875
.. code-block:: python3
@@ -72,7 +79,7 @@ Here we're using ``pytorch21_p39_gpu_v1`` with python 3.9.
7279
inference_python_version="3.9",
7380
)
7481
75-
Below is the ``chain.yaml`` file that was saved from ``llm_chain`` object. For more information regarding LLMs model serialization, see `here <https://python.langchain.com/docs/modules/model_io/llms/llm_serialization>`_.
82+
Below is the ``chain.yaml`` file that was saved from ``llm_chain`` object.
7683

7784
.. code-block:: YAML
7885
@@ -107,6 +114,16 @@ Below is the ``chain.yaml`` file that was saved from ``llm_chain`` object. For m
107114
tags: null
108115
verbose: true
109116
117+
Verify the Serialized Application
118+
*********************************
119+
120+
Verify the serialized application by calling ``verify()`` to make sure it is working as expected.
121+
There will be error if your application is not fully serializable.
122+
123+
.. code-block:: python3
124+
125+
chain_deployment.verify({"subject": "animals"})
126+
110127
Save Artifacts to OCI Model Catalog
111128
***********************************
112129

0 commit comments

Comments
 (0)