Crawlbase Web MCP Server

Created By

crawlbase4 months ago

Crawlbase Web MCP Server (Model Context Protocol) connects AI agents and large language models (LLMs) with real-time web data. Built on Crawlbase’s proven web scraping and crawling infrastructure, it handles JavaScript rendering, anti-bot protection, and web data extraction at scale — powering Claude, Cursor, and Windsurf. It delivers a reliable, production-ready pipeline of live, structured data for AI workflows and intelligent applications.

# web-scraping

# web-scraping-mcp

Overview Content Tools Comments

Overview

What is Crawlbase MCP?

Crawlbase MCP is a Model Context Protocol (MCP) server that bridges AI agents and the live web. Instead of relying on outdated training data, your LLMs can now fetch fresh, structured, real-time content — powered by Crawlbase’s proven crawling infrastructure trusted by 70,000+ developers worldwide.

It handles the complexity of scraping for you:

JavaScript rendering for modern web apps
Proxy rotation & anti-bot evasion
Structured outputs (HTML, Markdown, screenshots)

How It Works

Get Free Crawlbase Tokens → Sign up at Crawlbase ↗, get free Normal, and JavaScript tokens.
Add MCP Config → Connect Crawlbase MCP to Claude, Cursor, or Windsurf by updating your mcpServers config.
Start Crawling → Use commands like crawl, crawl_markdown, or crawl_screenshot to bring live web data into your AI agent.

Features

Real-time web scraping for AI agents
JavaScript rendering (SPAs & dynamic pages)
Proxy rotation to bypass blocks & captchas
Structured outputs (HTML, Markdown, Screenshots)
Seamless MCP integration with Claude, Cursor & Windsurf

Use Cases

Research with up-to-date articles & reports
Monitor e-commerce products & prices
Fetch real-time news & financial data
Aggregate content for data pipelines
Power AI agents with fresh, accurate information

Server Config

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "@crawlbase/mcp@latest"
      ],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

Recommend Servers

TraeBuild with Free GPT-4.1 & Claude 3.7. Fully MCP-Ready.

TimeA Model Context Protocol server that provides time and timezone conversion capabilities. This server enables LLMs to get current time information and perform timezone conversions using IANA timezone names, with automatic system timezone detection.

Baidu Map百度地图核心API现已全面兼容MCP协议，是国内首家兼容MCP协议的地图服务商。

Jina AI MCP ToolsA Model Context Protocol (MCP) server that integrates with Jina AI Search Foundation APIs.

Amap Maps高德地图官方 MCP Server

WindsurfThe new purpose-built IDE to harness magic

Context7Context7 MCP Server -- Up-to-date code documentation for LLMs and AI code editors

AiimagemultistyleA Model Context Protocol (MCP) server for image generation and manipulation using fal.ai's Stable Diffusion model.

Zhipu Web SearchZhipu Web Search MCP Server is a search engine specifically designed for large models. It integrates four search engines, allowing users to flexibly compare and switch between them. Building upon the web crawling and ranking capabilities of traditional search engines, it enhances intent recognition capabilities, returning results more suitable for large model processing (such as webpage titles, URLs, summaries, site names, site icons, etc.). This helps AI applications achieve "dynamic knowledge acquisition" and "precise scenario adaptation" capabilities.

Serper MCP ServerA Serper MCP Server

EdgeOne Pages MCPAn MCP service designed for deploying HTML content to EdgeOne Pages and obtaining an accessible public URL.