Crawl4ai Middleware

Created By

Frankieli1237 months ago

Overview

what is Crawl4AI Middleware?

Crawl4AI Middleware is a Model Context Protocol (MCP) server designed to integrate web scraping functionalities of Crawl4AI into applications that support MCP, such as Cursor IDE.

how to use Crawl4AI Middleware?

To use Crawl4AI Middleware, install the required dependencies, configure the server settings, and integrate it with Cursor IDE by modifying its configuration file to include the MCP server details.

key features of Crawl4AI Middleware?

Create web scraping tasks and retrieve task IDs.
Fetch scraping results based on task IDs and save them locally.
List saved scraping result files.
Read the content of saved scraping result files.

use cases of Crawl4AI Middleware?

Automating the extraction of data from websites.
Integrating web scraping capabilities into applications for data analysis.
Facilitating research by gathering information from multiple web sources.

FAQ from Crawl4AI Middleware?

What is the default API address for the server?

The default API address is http://192.168.31.12:11235/.

How do I configure the server?

You can configure the server by creating a .env file or setting system environment variables.

Can I customize the output directory for saved results?

Yes, you can specify a custom output directory in the configuration.

Server Config

{
  "mcpServers": {
    "crawl4ai": {
      "command": "python",
      "args": [
        "E:/APP/craw14ai-middleware/crawl4ai_server.py"
      ]
    }
  }
}

Recommend Servers

TraeBuild with Free GPT-4.1 & Claude 3.7. Fully MCP-Ready.

EdgeOne Pages MCPAn MCP service designed for deploying HTML content to EdgeOne Pages and obtaining an accessible public URL.

Context7Context7 MCP Server -- Up-to-date code documentation for LLMs and AI code editors

Tavily Mcp

Playwright McpPlaywright MCP server

AiimagemultistyleA Model Context Protocol (MCP) server for image generation and manipulation using fal.ai's Stable Diffusion model.

Jina AI MCP ToolsA Model Context Protocol (MCP) server that integrates with Jina AI Search Foundation APIs.

CursorThe AI Code Editor

BlenderBlenderMCP connects Blender to Claude AI through the Model Context Protocol (MCP), allowing Claude to directly interact with and control Blender. This integration enables prompt assisted 3D modeling, scene creation, and manipulation.

MiniMax MCPOfficial MiniMax Model Context Protocol (MCP) server that enables interaction with powerful Text to Speech, image generation and video generation APIs.

WindsurfThe new purpose-built IDE to harness magic