Frequently Asked Questions

Home » Help »

Could you explain the various editions, in a nutshell?

There are currently three editions.

Desktop Edition, covers trials, our web store, Microsoft Store, and App Store (macOS) variations. These products include one year of email support, and one year of licensed upgrades. There are some cross platform differences to be mindful of: JavaScript crawling is currently exclusive to Windows OS, and the App Store is a subscription product. All other features are the same.

Personal Edition is the free, unsupported version of InterroBot. This edition occurs after a lapsed Desktop Edition trial. It is the same application and installer. There are several limitations, the most significant of which is the restriction of commercial use.

Some features and crawler options are unavailable (outside of the trial period). These include:

  • JavaScript crawling (Windows)
  • request header overrides (User-Agent, Authorization, X-Auth-Token)
  • user-defined norobots to ignore specific paths

Mobile Edition is available exclusively at the Google Play Store and is similar to Desktop Edition with the following differences:

  • does not offer Personal Edition at end of trial or lapsed subscription
  • no support for JavaScript crawling

How to deal with an excess of site-internal norobots?

If you are encountering norobots while crawling the project domain with InterroBot, a workaround is to modify your site's robots.txt to globally allow InterroBot, internal to your site. As robots.txt is read top to bottom, it is best to add at the bottom of the file to reduce potential for other rules overriding the allow.

User-agent: interrobot
Allow: /

Disallow rules with more specific paths than webroot may need specific overrides to allow InterroBot. Details on robots.txt are available at Google Search Central.

When is it okay to use the Personal Edition?

Personal Edition is intended for non-commercial use. The Desktop Edition software trial will revert to Personal Edition once the trial has ended. Examples of intended use would include personal blogs, hobby sites, open source projects, and so on. If you are working on behalf of a corporation, you likely do not qualify.

What are the limits of the crawler?

Part of using your own hardware, be it a desktop environment or a mobile device, is that there are inherent hardware limitations. Performance and indexing capabilities will be proportional to the CPU and network capabilities of your computer. A quality solid-state drive (SSD) is key if you're thinking big.

A mid-tier laptop should be able to handle 100,000 total pages (across all projects). Add a high-performance desktop environment, that can be pushed much further.

For Mobile Edition, your bottleneck will almost certainly be storage with large sites consuming proportionally large amounts of disk. A mid-to-upper end phone should be capable of handling 10,000 pages without taking an inordinate amount of disk for an app. You can go further, just watch the storage.

I found a bug?

Great! Let me know. There's a contact form in the client application, it's under the Options page (the gear icon).

InterroBot is a web crawler and developer tool for Windows, macOS, and Android.
Want to learn more? Check out our help section or download the latest build.