InterroBot MCP Support Announced

Home » Blog »

Introducing MCP search and retrieval for web crawler content.

InterroBot users can now leverage Claude Desktop and Model Context Protocol (MCP) to analyze crawl data, thanks to the open-source mcp-server-webcrawl project.

InterroBot represents one of five crawlers supported out of the gate by mcp-server-webcrawl. The server connects your InterroBot database directly to MCP-capable AI assistants, giving your AI access to search and document retrieval within the crawl. This enables entirely new workflows, where you can direct searches with your AI assistant regarding your website, getting answers that would previously require manual analysis.

This MCP server supersedes older, more generalized solutions, including the example server listed in the InterroBot Options page (v2.9.1). That will be changed with the next release. mcp-server-webcrawl is the recommended MCP integration for InterroBot.

Setup

Configuring Claude Desktop

Setting up Claude with mcp-server-webcrawl requires a few steps, as outlined in the above video. First, install Python and then:

pip install mcp-server-webcrawl

In the crawl configuration, under Settings/Developer, add the following server configuration:

{ 
  "mcpServers": {
    "interrobot": {
      "command": "mcp-server-webcrawl",
      "args": ["--crawler", "interrobot", "--datasrc", 
        "/path/to/InterroBot/interrobot.v2.db"]
    }
  }
}

The path to your InterroBot database can be found in the InterroBot application under the Options page. For Windows users, it's typically in your Documents folder. For macOS users, refer to the Options page, the directory will be a virtualized path under your home directory.

Main Features

Once connected, your AI assistant can:

  • Full-text search your crawled projects, against the InterroBot search index
  • Filter by content type, HTTP status, and more
  • Analyze content for patterns and inconsistencies
  • Extract insights from your website's structure and content
  • Answer complex questions website content that would otherwise require manual analysis

The integration opens up new workflows for content management, SEO analysis, technical monitoring, and comprehensive site audits - all through natural language conversation with your AI assistant.

Availability

mcp-server-webcrawl is completely free and open-source. It supports not only InterroBot databases but also other popular web crawlers like wget, WARC, Katana, and SiteOne.

While Claude Desktop is currently the principle AI client with MCP support, OpenAI has announced that ChatGPT will support MCP in the future, expanding the potential applications of this technology.

InterroBot is a web crawler and developer tool for Windows, macOS, and Android.
Want to learn more? Check out our help section or download the latest build.