|
4 | 4 | Large Language Model |
5 | 5 | #################### |
6 | 6 |
|
7 | | -Oracle Cloud Infrastructure (OCI) provides fully managed Infrastructure to work with Large Language Model (LLM). You can train LLM at scale with `Data Science Jobs (Jobs) <https://docs.oracle.com/en-us/iaas/data-science/using/jobs-about.htm>`_, and deploy it with `Data Science Model Deployment (Model Deployments) <https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-about.htm>`_. On top of that, you can build and test your LLM applications with LangChain, then deploy it as your own API using the model deployment. |
| 7 | +Oracle Cloud Infrastructure (OCI) provides fully managed infrastructure to work with Large Language Model (LLM). |
8 | 8 |
|
| 9 | +Train and Deploy LLM |
| 10 | +******************** |
| 11 | +You can train LLM at scale with multi-node and multi-GPU using `Data Science Jobs (Jobs) <https://docs.oracle.com/en-us/iaas/data-science/using/jobs-about.htm>`_, and deploy it with `Data Science Model Deployment (Model Deployments) <https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-about.htm>`_. The following blog posts show examples training and deploying Llama2 models: |
9 | 12 |
|
10 | | -Compatibility with LangChain |
11 | | -**************************** |
12 | | -ADS is designed to be compatible with LangChain, enabling developers to incorporate various LangChain components seamlessly into their langchain applications. |
| 13 | +* `Multi-GPU multinode fine-tuning Llama2 on OCI Data Science <https://blogs.oracle.com/ai-and-datascience/post/multi-gpu-multi-node-finetuning-llama2-oci>`_ |
| 14 | +* `Deploy Llama 2 in OCI Data Science <https://blogs.oracle.com/ai-and-datascience/post/llama2-oci-data-science-cloud-platform>`_ |
| 15 | +* `Quantize and deploy Llama 2 70B on cost-effective NVIDIA A10 Tensor Core GPUs in OCI Data Science <https://blogs.oracle.com/ai-and-datascience/post/quantize-deploy-llama2-70b-costeffective-a10s-oci>`_ |
13 | 16 |
|
14 | | -Deployment Requirements |
15 | | ------------------------ |
16 | | -For successful deployment of LangChain components within ADS, it is crucial to ensure that each component used in the chain is serializable. This is because ADS requires all components to be serializable in order to deploy them as a single unit. |
17 | | - |
18 | | -ADS-Supported Components |
19 | | ------------------------- |
20 | | -ADS natively supports serialization of all its components. This ensures that any component developed or integrated within ADS adheres to the serialization standards. |
21 | | - |
22 | | -Additional LangChain Component Support |
23 | | --------------------------------------- |
24 | | -ADS extends its serialization support to two specific components from the LangChain vector store. These components are: |
25 | | - |
26 | | -- ``OpenSearchVectorSearch``: You can connect to the OCI OpenSearch cluster to perform semantic search along with your embedding model. |
27 | | - |
28 | | -- ``FAISS`` (Facebook AI Similarity Search): If you dont have an OCI OpenSearch cluster, you can use FAISS which is a in-memory vector store to perform semantic search along with your embedding model. |
29 | 17 |
|
| 18 | +Integration with LangChain |
| 19 | +************************** |
| 20 | +ADS is designed to work with LangChain, enabling developers to incorporate various LangChain components and models deployed on OCI seamlessly into their applications. Additionally, ADS can package LangChain applications and deploy it as a REST API endpoint using OCI Data Science Model Deployment. |
30 | 21 |
|
31 | 22 |
|
32 | 23 | .. admonition:: Installation |
|
0 commit comments