Sponsored by Deepsite.site

Crawlio Mcp

Created By
11 days ago
AI-powered website crawling, analysis, and export via MCP. 38 tools for crawl control, browser enrichment capture, WARC/ZIP/single-HTML export, observation timeline, and evidence-backed findings. 4 resources, 4 prompts. Dual-mode: full (38 tools) or code (6 tools). Install: npx crawlio-mcp
Overview

crawlio-mcp

1.3.4 • Public • Published

crawlio-mcp

MCP server for Crawlio — the AI-native macOS website crawler.

Exposes 36 tools and 4 resources over stdio transport, compatible with any MCP client (Claude Code, Cursor, Windsurf, VS Code, Zed, etc.).

Install

npx crawlio-mcp init

Downloads the binary automatically and configures all detected MCP clients.

Homebrew

brew install crawlio-app/tap/crawlio-mcp
crawlio-mcp init

Bundled with Crawlio.app

brew install --cask crawlio-app/tap/crawlio

The desktop app bundles the MCP server — the cask symlinks it to your PATH automatically.

Quick Setup

Run init to auto-detect and configure your AI clients:

npx crawlio-mcp init          # Configure all detected clients
npx crawlio-mcp init --full   # Enable all 36 tools (default: 6 code-mode tools)
npx crawlio-mcp init --portal # HTTP transport + launchd auto-start
npx crawlio-mcp init --dry-run # Preview changes without writing

Manual configuration

Add to your MCP client config (.mcp.json, mcp.json, etc.):

{
  "mcpServers": {
    "crawlio": {
      "command": "npx",
      "args": ["-y", "crawlio-mcp"]
    }
  }
}

Prerequisites

  • macOS 15 (Sequoia) or later
  • Crawlio.app must be running for control tools (read-only tools work offline)

How it works

This npm package is a thin wrapper that locates or downloads the native CrawlioMCP binary and forwards stdio through it. Binary resolution order:

  1. $CRAWLIO_MCP_BINARY environment variable
  2. ~/.crawlio/bin/CrawlioMCP (npm auto-download cache)
  3. /Applications/Crawlio.app/Contents/Helpers/CrawlioMCP
  4. ~/Applications/Crawlio.app/Contents/Helpers/CrawlioMCP
  5. /opt/homebrew/bin/crawlio-mcp (Homebrew)
  6. /usr/local/bin/crawlio-mcp

If no binary is found, it downloads from GitHub Releases on first run.

Tools

36 tools across 8 categories:

CategoryCountTools
Status & Monitoring6 get_crawl_status, get_crawl_logs, get_errors, get_downloads, get_failed_urls, get_site_tree
Control4 start_crawl, stop_crawl, pause_crawl, resume_crawl
Settings & Config3 get_settings, update_settings, recrawl_urls
Projects5 list_projects, save_project, load_project, delete_project, get_project
Export & Extraction5 export_site, get_export_status, extract_site, get_extraction_status, trigger_capture
Composite Analysis2 analyze_page (capture + poll + status in one call), compare_pages (two-site comparison with summary)
Enrichment6 get_enrichment, submit_enrichment_bundle, submit_enrichment_framework, submit_enrichment_network, submit_enrichment_console, submit_enrichment_dom
Observations & Findings4 get_observations, create_finding, get_findings, get_crawled_urls
OCR1extract_text_from_image

Plus 3 HTTP-only endpoints accessible via execute_api: get_health, get_debug_metrics, dump_state.

Resources

URIDescription
crawlio://statusEngine state and progress
crawlio://settingsCurrent crawl settings
crawlio://site-treeDownloaded file tree
crawlio://enrichmentBrowser enrichment data

Server Config

{
  "mcpServers": {
    "crawlio": {
      "command": "npx",
      "args": [
        "-y",
        "crawlio-mcp"
      ]
    }
  }
}
Recommend Servers
TraeBuild with Free GPT-4.1 & Claude 3.7. Fully MCP-Ready.
Playwright McpPlaywright MCP server
Tavily Mcp
WindsurfThe new purpose-built IDE to harness magic
Baidu Map百度地图核心API现已全面兼容MCP协议,是国内首家兼容MCP协议的地图服务商。
CursorThe AI Code Editor
AiimagemultistyleA Model Context Protocol (MCP) server for image generation and manipulation using fal.ai's Stable Diffusion model.
DeepChatYour AI Partner on Desktop
Jina AI MCP ToolsA Model Context Protocol (MCP) server that integrates with Jina AI Search Foundation APIs.
Amap Maps高德地图官方 MCP Server
Y GuiA web-based graphical interface for AI chat interactions with support for multiple AI models and MCP (Model Context Protocol) servers.
RedisA Model Context Protocol server that provides access to Redis databases. This server enables LLMs to interact with Redis key-value stores through a set of standardized tools.
BlenderBlenderMCP connects Blender to Claude AI through the Model Context Protocol (MCP), allowing Claude to directly interact with and control Blender. This integration enables prompt assisted 3D modeling, scene creation, and manipulation.
MiniMax MCPOfficial MiniMax Model Context Protocol (MCP) server that enables interaction with powerful Text to Speech, image generation and video generation APIs.
Serper MCP ServerA Serper MCP Server
Visual Studio Code - Open Source ("Code - OSS")Visual Studio Code
ChatWiseThe second fastest AI chatbot™
Zhipu Web SearchZhipu Web Search MCP Server is a search engine specifically designed for large models. It integrates four search engines, allowing users to flexibly compare and switch between them. Building upon the web crawling and ranking capabilities of traditional search engines, it enhances intent recognition capabilities, returning results more suitable for large model processing (such as webpage titles, URLs, summaries, site names, site icons, etc.). This helps AI applications achieve "dynamic knowledge acquisition" and "precise scenario adaptation" capabilities.
EdgeOne Pages MCPAn MCP service designed for deploying HTML content to EdgeOne Pages and obtaining an accessible public URL.
Howtocook Mcp基于Anduin2017 / HowToCook (程序员在家做饭指南)的mcp server,帮你推荐菜谱、规划膳食,解决“今天吃什么“的世纪难题; Based on Anduin2017/HowToCook (Programmer's Guide to Cooking at Home), MCP Server helps you recommend recipes, plan meals, and solve the century old problem of "what to eat today"
MCP AdvisorMCP Advisor & Installation - Use the right MCP server for your needs