sitemcp

Fetch an entire site and use it as a MCP Server

https://github.com/user-attachments/assets/ebe2d7c6-4ddc-4a37-8e1e-d80fac49d8ae

Demo in Japanese

https://github.com/user-attachments/assets/24288140-be2a-416c-9e7c-c49be056a373

Install

One-off usage (choose one of the followings):

bunx sitemcp
npx sitemcp
pnpx sitemcp

Install globally (choose one of the followings):

bun i -g sitemcp
npm i -g sitemcp
pnpm i -g sitemcp

Usage

sitemcp https://daisyui.com

# or better concurrency
sitemcp https://daisyui.com --concurrency 10

Match specific pages

Use the -m, --match flag to specify the pages you want to fetch:

sitemcp https://vite.dev -m "/blog/**" -m "/guide/**"

The match pattern is tested against the pathname of target pages, powered by micromatch, you can check out all the supported matching features.

Content selector

We use mozilla/readability to extract readable content from the web page, but on some pages it might return irrelevant contents, in this case you can specify a CSS selector so we know where to find the readable content:

sitemcp https://vite.dev --content-selector ".content"

How to configure with MCP Client

You can execute server from your MCP client (e.g. Claude Desktop).

The below example configuration for Claude Desktop

{
  "mcpServers": {
    "daisy-ui": {
      "command": "npx",
      "args": [
        "-y",
        "sitemcp",
        "https://daisyui.com",
        "-m",
        "/components/**",
      ]
    }
  }
}

Tips

• Some site has a lot of pages. It is better to run sitemcp before registering the server to the MCP client. sitemcp caches the pages in ~/.cache/sitemcp by default. You can disable by --no-cache flag.

License

MIT.

Stats