> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-docs-zh.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Ray serve 集成

> 使用 LangChain Python 与 Ray serve 集成。

[Ray Serve](https://docs.ray.io/en/latest/serve/index.html) 是一个可扩展的模型服务库，用于构建在线推理API。Serve尤其适用于系统组合，使您能够通过Python代码构建包含多个链和业务逻辑的复杂推理服务。

## 本笔记本的目标

本笔记本展示了一个将OpenAI链部署到生产环境的简单示例。您可将其扩展以部署自托管模型，并轻松定义运行模型所需的硬件资源（GPU和CPU），从而高效实现生产部署。更多选项（包括自动扩缩容）请参阅Ray Serve的 [文档](https://docs.ray.io/en/latest/serve/getting_started.html)。

## 配置ray serve

安装ray `pip install ray[serve]`。

## 通用骨架

部署服务的通用骨架如下：

```python theme={null}
# 0: Import ray serve and request from starlette
from ray import serve
from starlette.requests import Request


# 1: Define a Ray Serve deployment.
@serve.deployment
class LLMServe:
    def __init__(self) -> None:
        # All the initialization code goes here
        pass

    async def __call__(self, request: Request) -> str:
        # You can parse the request here
        # and return a response
        return "Hello World"


# 2: Bind the model to deployment
deployment = LLMServe.bind()

# 3: Run the deployment
serve.api.run(deployment)
```

```python theme={null}
# Shutdown the deployment
serve.api.shutdown()
```

## OpenAI链的部署示例（含自定义提示词）

从 [OpenAI API密钥页面](https://platform.openai.com/account/api-keys) 获取OpenAI API密钥。运行以下代码将提示您提供API密钥。

```python theme={null}
from langchain_classic.chains import LLMChain
from langchain_core.prompts import PromptTemplate
from langchain_openai import OpenAI
```

```python theme={null}
from getpass import getpass

OPENAI_API_KEY = getpass()
```

```python theme={null}
@serve.deployment
class DeployLLM:
    def __init__(self):
        # We initialize the LLM, template and the chain here
        llm = OpenAI(openai_api_key=OPENAI_API_KEY)
        template = "Question: {question}\n\nAnswer: Let's think step by step."
        prompt = PromptTemplate.from_template(template)
        self.chain = LLMChain(llm=llm, prompt=prompt)

    def _run_chain(self, text: str):
        return self.chain(text)

    async def __call__(self, request: Request):
        # 1. Parse the request
        text = request.query_params["text"]
        # 2. Run the chain
        resp = self._run_chain(text)
        # 3. Return the response
        return resp["text"]
```

现在可以绑定部署。

```python theme={null}
# Bind the model to deployment
deployment = DeployLLM.bind()
```

当需要运行部署时，可指定端口号和主机地址。

```python theme={null}
# Example port number
PORT_NUMBER = 8282
# Run the deployment
serve.api.run(deployment, port=PORT_NUMBER)
```

服务已部署在端口 `localhost:8282`，此时可通过POST请求获取结果。

```python theme={null}
import requests

text = "What NFL team won the Super Bowl in the year Justin Beiber was born?"
response = requests.post(f"http://localhost:{PORT_NUMBER}/?text={text}")
print(response.content.decode())
```

***

> 官方原文：[src/oss/python/integrations/providers/ray\_serve.mdx](https://github.com/langchain-ai/docs/blob/main/src/oss/python/integrations/providers/ray_serve.mdx) · 非官方翻译，译文由 AI 生成，以英文原文为准。


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.