Skip to content

docs: update Firecrawl example #2652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: update Firecrawl example #2652

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 9 additions & 13 deletions docs/guides/examples/firecrawl-url-crawl.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ Here are two examples of how to use Firecrawl with Trigger.dev:
This task crawls a website and returns the `crawlResult` object. You can set the `limit` parameter to control the number of URLs that are crawled.

```ts trigger/firecrawl-url-crawl.ts
import FirecrawlApp from "@mendable/firecrawl-js";
import Firecrawl from "@mendable/firecrawl-js";
import { task } from "@trigger.dev/sdk";

// Initialize the Firecrawl client with your API key
const firecrawlClient = new FirecrawlApp({
const firecrawlClient = new Firecrawl({
apiKey: process.env.FIRECRAWL_API_KEY, // Get this from your Firecrawl dashboard
});

Expand All @@ -34,15 +34,15 @@ export const firecrawlCrawl = task({
const { url } = payload;

// Crawl: scrapes all the URLs of a web page and return content in LLM-ready format
const crawlResult = await firecrawlClient.crawlUrl(url, {
const crawlResult = await firecrawlClient.crawl(url, {
limit: 100, // Limit the number of URLs to crawl
scrapeOptions: {
formats: ["markdown", "html"],
},
});

if (!crawlResult.success) {
throw new Error(`Failed to crawl: ${crawlResult.error}`);
if (crawlResult.status === "failed") {
throw new Error(`Failed to crawl: ${url}`);
}

return {
Expand All @@ -65,11 +65,11 @@ You can test your task by triggering it from the Trigger.dev dashboard.
This task scrapes a single URL and returns the `scrapeResult` object.

```ts trigger/firecrawl-url-scrape.ts
import FirecrawlApp, { ScrapeResponse } from "@mendable/firecrawl-js";
import Firecrawl from "@mendable/firecrawl-js";
import { task } from "@trigger.dev/sdk";

// Initialize the Firecrawl client with your API key
const firecrawlClient = new FirecrawlApp({
const firecrawlClient = new Firecrawl({
apiKey: process.env.FIRECRAWL_API_KEY, // Get this from your Firecrawl dashboard
});

Expand All @@ -79,13 +79,9 @@ export const firecrawlScrape = task({
const { url } = payload;

// Scrape: scrapes a URL and get its content in LLM-ready format (markdown, structured data via LLM Extract, screenshot, html)
const scrapeResult = (await firecrawlClient.scrapeUrl(url, {
const scrapeResult = await firecrawlClient.scrape(url, {
formats: ["markdown", "html"],
})) as ScrapeResponse;

if (!scrapeResult.success) {
throw new Error(`Failed to scrape: ${scrapeResult.error}`);
}
});

return {
data: scrapeResult,
Expand Down