Astra SiteManager 1.0 Retro Review

Home » Web Crawler News

Software: Astra SiteManager 1.03 (build as01-63r)
Publisher: Mercury Interactive
Released: 1996
Price on Release: $495 (≅$1,000 CPI adjusted for 2023)

The mid-90s were heady days in the field of desktop web crawlers, with 1996 representing a breakout year of innovation and competition in the space. Big tech were salivating for anything internet, and the economy was infused with a gravy train of investor cash to support it all. It was a golden age of desktop software, driven by a Windows monoculture.

Judging from what internet chatter remains of the time, three products stood out as category leaders in the desktop web crawler space: InContext WebAnalyzer, NetCarta WebMapper, and Astra SiteManager.

InContext WebAnalyzer dominated mindshare, but it did not matter in the end. NetCarta was acquired by Microsoft in December 1996, rebranded Microsoft WebMapper and released it for free. It was an all too familiar rug pull by the industry giant, an overnight transformation of the market. Xenu's Link Sleuth, a beloved freeware link checker, would also drop in 1997.

By the spring of 2000, the broader tech market started to give out, taking the NASDAQ from a high of $5,048 to a low of $1,114 in 2001. As the investment dried up, so too did any hope of reanimating these crawlers as viable commercial products within a landscape of competent, free alternatives. It was not until the late aughts that commercial crawlers would flourish again, pivoting towards search engine optimization (SEO) workflows.

Astra SiteManager would ship a 2.0 release in 1999, distributing the utility for free before product updates ultimately ceased. Mercury Interactive would later be shaken by allegations of securities fraud, leading to the resignation of three top executives in 2005. The rights to SiteManager were subsumed by Hewlett-Packard in the 2006 acquisition of Mercury Interactive.


InterroBot is a web crawler and devtool for Windows 10/11.
Want to learn more? Check out our help section or download the latest build.