Tavily 爬取集成

通过LangChain JavaScript与Tavily爬取工具集成。

Tavily 是专为 AI 智能体（LLM）构建的搜索引擎，能够快速提供实时、准确且真实的结果。Tavily 提供一个 Crawl 端点，该端点从起始 URL 执行结构化网页遍历，内置内容提取和智能发现功能。

概述

集成细节

类	包	PY 支持	下载量	版本
TavilyCrawl	@langchain/tavily	✅

工具特性

返回工件	原生异步	返回数据	定价
❌	✅	基础 URL、页面 URL、原始内容、图片、图标	1,000 免费信用额度 / 月

设置

集成位于 @langchain/tavily 包中，您可以通过以下方式安装：

npm

npm install @langchain/tavily @langchain/core

凭证

设置 Tavily API 密钥 并将其设置为名为 TAVILY_API_KEY 的环境变量。

process.env.TAVILY_API_KEY = "YOUR_API_KEY"

设置 LangSmith（最佳观测性，但非必需）也很有帮助：

process.env.LANGSMITH_TRACING="true"
process.env.LANGSMITH_API_KEY="your-api-key"

实例化

该工具在实例化期间接受以下参数：

• maxDepth（可选，数字）：从起始 URL 的最大跳转数。默认为 3。
• maxBreadth（可选，数字）：每层最大爬取页面数。默认为 50。
• limit（可选，数字）：最大总爬取页面数。默认为 100。
• instructions（可选，字符串）：指导爬虫的自然语言指令。
• selectPaths（可选，字符串[]）：仅爬取包含这些路径正则表达式的 URL。
• selectDomains（可选，字符串[]）：仅爬取这些域名正则表达式。
• excludePaths（可选，字符串[]）：跳过包含这些路径正则表达式的 URL。
• excludeDomains（可选，字符串[]）：跳过来自这些域名正则表达式的 URL。
• allowExternal（可选，布尔值）：允许爬虫跟随外部链接。
• extractDepth（可选，字符串）：每个爬取页面的 "basic" 或 "advanced" 提取深度。默认为 "basic"。
• format（可选，字符串）："markdown" 或 "text" 内容格式。默认为 "markdown"。
• includeImages（可选，布尔值）：包含每个页面发现的图片。默认为 false。

有关可用参数的全面概述，请参阅 Tavily Crawl API 文档。

import { TavilyCrawl } from "@langchain/tavily";

const tool = new TavilyCrawl({
  maxDepth: 3,
  maxBreadth: 50,
  // limit: 100,
  // extractDepth: "basic",
  // format: "markdown",
  // includeImages: false,
  // allowExternal: false,
});

调用

使用参数直接调用

Tavily 爬取工具在调用期间接受以下参数：

• url（必需）：开始爬取的基础 URL。
• 以下参数也可以在调用期间设置：instructions、selectPaths、selectDomains、excludePaths、excludeDomains、allowExternal。

注意：可选参数可用于智能体动态设置。如果您在实例化期间设置了一个参数，然后在调用工具时使用不同的值，工具将使用您在调用期间传递的值。

await tool.invoke({ url: "https://docs.tavily.com" });

{
  "base_url": "https://docs.tavily.com",
  "results": [
    {
      "url": "https://docs.tavily.com/",
      "raw_content": "![light logo](https://mintcdn.com/tavilyai/HY1Rnt85q4usR4-R/logo/light.svg?fit=max&auto=format&n=HY1Rnt85q4usR4-R&q=85&s=c5c878011f13d458af0997f3a540eb4f)\n![dark logo](https://mintcdn.com/tavilyai/HY1Rnt85q4usR4-R/logo/dark.svg?fit=max&auto=format&n=HY1Rnt85q4usR4-R&q=85&s=1521677768a1f26b34a9ad86d04c62cc)\n\n##### Welcome\n\n## Documentation Index\n\nFetch the complete documentation index at: <https://docs.tavily.com/llms.txt>\n\nUse this file to discover all available pages before exploring further.\n\n# Build with Tavily\n\nYour journey to state-of-the-art web search starts right here.\n\n## Python SDK\n\n`pip install tavily-python`\n\n## JavaScript SDK\n\n`npm i @tavily/core`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.search(\"Who is Leo Messi?\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.extract(\"https://en.wikipedia.org/wiki/Artificial_intelligence\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.crawl(\"https://docs.tavily.com\", instructions=\"Find all pages on the Python SDK\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.map(\"https://docs.tavily.com\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.research(\"What are the latest developments in AI?\")\n\nprint(response)`\n\n### Developer Resources\n\n## API Credits Overview\n\n## Rate Limits\n\n## Playground",
      "favicon": "https://docs.tavily.com/mintlify-assets/_mintlify/favicons/tavilyai/SXaxSfweEU3ftIlh/_generated/favicon/apple-touch-icon.png"
    },
    {
      "url": "https://docs.tavily.com/documentation/about",
      "raw_content": "Performance\n\nCommitted to speed and efficiency, our API guarantees real-time and trusted information. Our team works hard to improve Tavily’s performance over time.\n\nIntegration-friendly\n\nWe appreciate the essence of adaptability. That’s why integrating our API with your existing setup is a breeze. You can choose our Python library, JavaScript package or a simple API call. You can also use Tavily through any of our supported partners such as LangChain and LlamaIndex.\n\nTransparent & Informative\n\nOur detailed documentation ensures you’re never left in the dark. From setup basics to nuanced features, we’ve got you covered.\n\n## ​\n\nHow does the Search API work? [...] With LLM hallucinations, it’s crucial to optimize for RAG with the right context and information. This is where Tavily comes in, delivering accurate and precise information for your RAG applications.\n\n## ​\n\nGetting started\n\nSign up for Tavily to get your API key. You get 1,000 free API Credits every month. No credit card required.\n\n## Get your free API key You get 1,000 free API Credits every month. No credit card required.\n\nHead to our API Playground to familiarize yourself with our API.To get started with Tavily’s APIs and SDKs using code, head to our Quickstart Guide and follow the steps.\n\nGot questions? Stumbled upon an issue? Simply intrigued? Don’t hesitate! Our support team is always on standby, eager to assist. Join us, dive deep, and redefine your search experience! Contact us! [...] # About - Tavily Docs\n\nSkip to main content\n\nTavily Docs home pageImage 1: light logoImage 2: dark logo\n\nSearch...\n\nCtrl K Ask AI\n\n   Support\n   Get an API key\n   Get an API key\n\nSearch...\n\nNavigation\n\nGetting Started\n\nAbout\n\nHomeIntroductionAPI & SDKsEcosystemExamplesChangelogHelp\n\n   API Playground\n   Community\n   Blog\n\n##### Getting Started\n\n   About\n   Quickstart\n   Credits & Pricing\n   Rate Limits\n\n##### FAQ\n\n   Frequently Asked Questions\n\nOn this page\n\n   Who are we?\n   What is the Tavily Search Engine?\n   Why choose Tavily?\n   How does the Search API work?\n   Getting started\n\nGetting Started\n\n# About\n\nCopy page\n\nWelcome to Tavily!\n\nCopy page\n\nLooking for a step-by-step tutorial to get started in under 5 minutes? Head to our Quickstart guide and start coding!\n\n## ​\n\nWho are we?",
      "favicon": "https://docs.tavily.com/mintlify-assets/_mintlify/favicons/tavilyai/SXaxSfweEU3ftIlh/_generated/favicon/apple-touch-icon.png"
    },
    {
      "url": "https://docs.tavily.com/documentation/api-reference/introduction",
      "raw_content": "# Introduction - Tavily Docs\n\nSkip to main content\n\nTavily Docs home pageImage 1: light logoImage 2: dark logo\n\nSearch...\n\n⌘K Ask AI\n\n   Support\n   Get an API key\n   Get an API key\n\nSearch...\n\nNavigation\n\nAPI Reference\n\nIntroduction\n\nHomeIntroductionAPI & SDKsEcosystemExamplesChangelogHelp\n\n   API Playground\n   Community\n   Blog\n\n##### API Reference\n\n   Introduction\n   POST Search\n   POST Extract\n   POST Crawl\n   POST Map\n   Research  \n   GET Usage\n\n##### Enterprise API Reference\n\n   API Key Generator  \n\n##### Python SDK\n\n   Quickstart\n   SDK Reference\n\n##### JavaScript SDK\n\n   Quickstart\n   SDK Reference\n\n##### Best Practices\n\n   Search\n   Extract\n   Crawl\n   Research\n   API Key Management\n\nOn this page\n\n   Base URL\n   Authentication\n   Endpoints\n   Project Tracking\n\nAPI Reference [...] When using the SDKs, you can specify a project using the `project_id` parameter when instantiating the client, or by setting the `TAVILY_PROJECT` environment variable.\n\nTavily Search Next\n\n⌘I\n\nTavily Docs home pageImage 3: light logoImage 4: dark logo\n\nlinkedinxgithubyoutube\n\nResources\n\nAboutTavily Search CrawlerTavily Status\n\nLegal\n\nSecurity & CompliancePrivacy PolicyWebsite Terms of UsePlatform Terms of UseCookie Notice\n\nlinkedinxgithubyoutube\n\nPowered by This documentation is built and hosted on Mintlify, a developer documentation platform\n\nAssistant\n\nResponses are generated using AI and may contain mistakes. [...] API Reference\n\n# Introduction\n\nCopy page\n\nEasily integrate our APIs with your services.\n\nCopy page\n\n> ## Documentation Index\n> \n> \n> Fetch the complete documentation index at: \n> \n> \n> Use this file to discover all available pages before exploring further.\n\n## ​\n\nBase URL\n\nThe base URL for all requests to the Tavily API is:\n\n```\n\n```\n\n## ​\n\nAuthentication\n\nAll Tavily endpoints are authenticated using API keys. Get your free API key.\n\n```\ncurl -X POST  \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer tvly-YOUR_API_KEY\" \\\n  -d '{\"query\": \"Who is Leo Messi?\"}'\n```\n\n## ​\n\nEndpoints\n\n`/search`Tavily’s powerful web search API.\n\n`/extract`Tavily’s powerful content extraction API.\n\n`/crawl` , `/map`Tavily’s intelligent sitegraph navigation and extraction tools.",
      "favicon": "https://docs.tavily.com/mintlify-assets/_mintlify/favicons/tavilyai/SXaxSfweEU3ftIlh/_generated/favicon/apple-touch-icon.png"
    },
    {
      "url": "https://github.com/tavily-ai",
      "raw_content": "JavaScript1.8k239  \n2.   tavily-pythontavily-python Public \nThe Tavily Python SDK allows for easy interaction with the Tavily API, offering the full range of our search, extract, crawl, map, and research functionalities directly from your Python programs. E…\n\nPython1.2k148  \n3.   skillsskills Public \nPython23321  \n4.   meeting-prep-agentmeeting-prep-agent Public \nPrepare for meetings with company and attendee research notes, integrated with Google Calendar through MCP\n\nPython19334  \n5.   tavily-jstavily-js Public \nOfficial TypeScript/JavaScript library for Tavily.\n\nTypeScript839  \n6.   tavily-chattavily-chat Public \nConversational agent that fuses chat data with live web results through Tavily search, extract, and crawl.\n\nTypeScript7216  \n\n### Repositories\n\nLoading\n\nType\n\nSelect type [...] ### Repositories\n\nLoading\n\nType\n\nSelect type\n\nAll Public Sources Forks Archived Mirrors Templates \n\nLanguage\n\nSelect language\n\nAll JavaScript Jupyter Notebook Python TypeScript \n\nSort\n\nSelect order\n\nLast updated Name Stars \n\nShowing 10 of 33 repositories [...] langchain-tavily Public  ### Uh oh! There was an error while loading. [Please reload this page.]( past year of commit activity Python22MIT 1511 Updated Apr 16, 2026  \n   langchainjs Public  Forked from langchain-ai/langchainjs\n🦜🔗 Build context-aware reasoning applications ### Uh oh! There was an error while loading. [Please reload this page.]( past year of commit activity TypeScript 0 MIT 3,20800 Updated Apr 16, 2026",
      "favicon": "https://github.githubassets.com/favicons/favicon.svg"
    }
  ],
  "response_time": 13.45
}

使用 ToolCall 调用

我们也可以使用模型生成的 ToolCall 调用该工具，此时将返回一个 @[ToolMessage]：

// This is usually generated by a model, but we'll create a tool call directly for demo purposes.
const modelGeneratedToolCall = {
  args: {
    url: "https://docs.tavily.com",
    instructions: "Find API reference pages",
  },
  id: "1",
  name: tool.name,
  type: "tool_call",
};

const toolMsg = await tool.invoke(modelGeneratedToolCall);

console.log(toolMsg.content.slice(0, 400));

在智能体中使用

我们可以通过将其传递给 createAgent 直接在 LangChain 智能体中使用爬取工具。智能体可以在其工具调用中动态设置 url、instructions 和路径/域名过滤器。

// @lc-docs-hide-cell
import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "gpt-5.5",
  temperature: 0,
});

import { TavilyCrawl } from "@langchain/tavily";
import { createAgent } from "langchain";

const tavilyCrawlTool = new TavilyCrawl({
  maxDepth: 1,
  maxBreadth: 10,
  limit: 10,
});

const agent = createAgent({
  model: llm,
  tools: [tavilyCrawlTool],
});

const userInput = "Crawl https://docs.tavily.com and find the API reference for the search endpoint.";

const stream = await agent.streamEvents(
  { messages: [["human", userInput]] },
  { version: "v3" },
);

for await (const snapshot of stream.values) {
  const lastMsg = snapshot.messages[snapshot.messages.length - 1];
  if (lastMsg.tool_calls?.length) {
    console.dir(lastMsg.tool_calls, { depth: null });
  } else if (lastMsg.content) {
    console.log(lastMsg.content);
  }
}

---

API 参考

有关 Tavily Crawl API 功能和配置的详细文档，请前往 API 参考：docs.tavily.com/documentation/api-reference/endpoint/crawl

---

官方原文：src/oss/javascript/integrations/tools/tavily_crawl.mdx · 非官方翻译，译文由 AI 生成，以英文原文为准。