Frequently Asked Questions

Home » InterroBot Help

Could you explain the various editions, in a nutshell?

There are currently three editions.

Desktop Edition, which covers trials, our web store, and Microsoft Store . This is the most complete edition, having access to all features. These products include one year of email support, and one year of licensed upgrades. At the end of the one year term, the version you have can be used perpetually, but support will terminate.

Personal Edition is the free, unsupported version of InterroBot. This edition occurs after a lapsed Desktop Edition trial. It is the same application and installer. There are several limitations, the most significant of which is the restriction of commercial use.

Some features and crawler options are unavailable (outside of the trial period). These include:

  • JavaScript crawling
  • request header overrides (User-Agent, Authorization, X-Auth-Token)
  • user-defined norobots to ignore specific paths

Mobile Edition is available exclusively at the Google Play Store (with iOS in the pipeline), and is similar to Desktop Edition with the following differences:

  • is a subscription product, and must be renewed at the end of the term
  • doesn't support JavaScript crawling or 3rd party plugins

How to deal with an excess of site-internal norobots?

If you are encountering norobots while crawling the project domain with InterroBot, a workaround is to modify your site's robots.txt to globally allow InterroBot, internal to your site. As robots.txt is read top to bottom, it is best to add at the bottom of the file to reduce potential for other rules overriding the allow.

User-agent: interrobot
Allow: /

Disallow rules with more specific paths than webroot may need specific overrides to allow InterroBot. Details on robots.txt are available at Google Search Central.

When is it okay to use the Personal Edition?

Personal Edition is intended for non-commercial use. The Desktop Edition software trial will revert to Personal Edition once the trial has ended. Examples of intended use would include personal blogs, hobby sites, open source projects, and so on. If you are working on behalf of a corporation, you likely do not qualify.

What are the limits of the crawler?

Part of using your own hardware, be it a desktop environment or a mobile device, is that there are inherent hardware limitations. Performance and indexing capabilities will be proportional to the CPU and network capabilities of your computer. A quality solid-state drive (SSD) is key if you're thinking big.

A mid-tier laptop should be able to handle 100,000 total pages (across all projects). Add a high-performance desktop environment, that can be pushed much further.

For Mobile Edition, your bottleneck will almost certainly be storage with large sites consuming proportionally large amounts of disk. A mid-to-upper end phone should be capable of handling 10,000 pages without taking an inordinate amount of disk for an app. You can go further, just watch the storage.

I found a bug?

Great! Let me know. There's a contact form in the client application, it's under the Options page (the gear icon).

InterroBot is a web crawler and devtool for Windows and Android.
Want to learn more? Check out our help section or download the latest build.