Frequently Asked Questions

Home » Help »

Could you explain the various editions, in a nutshell?

There are currently three editions.

Desktop Edition, covers trials, our web store, Microsoft Store, and App Store (macOS) variations. These products include one year of email support, and one year of licensed upgrades. There are some cross platform differences to be mindful of: JavaScript crawling is currently exclusive to Windows OS, and the App Store is a subscription product. All other features are the same.

Community Edition is the free, unsupported version of InterroBot. This edition occurs after a lapsed Desktop Edition trial. It is the same application and installer. There are several limitations of this version, and some features and crawler options are unavailable (outside of the trial period). These include:

  • JavaScript crawling (Windows)
  • request header overrides (User-Agent, Authorization, X-Auth-Token)
  • user-defined norobots to ignore specific paths

Mobile Edition is available exclusively at the Google Play Store and is similar to Desktop Edition with the following differences:

  • does not offer Community Edition at end of trial or lapsed subscription
  • no support for JavaScript crawling
  • no support for Model Context Protocol (MCP)

How to deal with an excess of site-internal norobots?

If you are encountering norobots while crawling the project domain with InterroBot, a workaround is to modify your site's robots.txt to globally allow InterroBot, internal to your site. As robots.txt is read top to bottom, it is best to add at the bottom of the file to reduce potential for other rules overriding the allow.

User-agent: interrobot
Allow: /

Disallow rules with more specific paths than webroot may need specific overrides to allow InterroBot. Details on robots.txt are available at Google Search Central.

When should I use Community Edition?

Community Edition is the version you'll get automatically when your Desktop Edition trial ends, and it includes most core features of InterroBot. It's a great tool to get started with website analysis without any upfront cost. You can upgrade anytime if you require the advanced features only provided with the standard license.

What are the limits of the crawler?

Part of using your own hardware, be it a desktop environment or a mobile device, is that there are inherent hardware limitations. Performance and indexing capabilities will be proportional to the CPU and network capabilities of your computer. A quality solid-state drive (SSD) is key if you're thinking big.

A mid-tier laptop should be able to handle 100,000 total pages (across all projects). Add a high-performance desktop environment, that can be pushed much further.

For Mobile Edition, your bottleneck will almost certainly be storage with large sites consuming proportionally large amounts of disk. A mid-to-upper end phone should be capable of handling 10,000 pages without taking an inordinate amount of disk for an app. You can go further, just watch the storage.

I found a bug?

Great! Let me know. There's a contact form in the client application, it's under the Options page (the gear icon).

InterroBot is a web crawler and developer tool for Windows, macOS, and Android.
Want to learn more? Check out our help section or download the latest build.