Interesting: "For Missing Web Pages, a Department of Lost and Found"

New York Times (10/21/04) P. E5; Eisenberg, Anne
Under the direction of IBM lab researcher Andrew Flegg, a team of four student interns has developed a working prototype for software that can check links between Web pages and search for the correct pages if the connections are broken or incorrect. The Peridot program uses a page's source code, context, and other data to build "fingerprints" that encapsulate the page's links and unique features, which are stored for later comparison; the software regularly checks the fingerprints to see if there are any discrepancies in a Web site's links, and then ascertains how important such discrepancies are. Flegg says he conceived of the software as a tool to not only address broken links, but also links whose content had become inaccurate or inapplicable. Oxford University student and IBM intern Ben Delo notes that Peridot required algorithms that could measure the degree of change in a link, determine how significant such a change was, and find the optimum techniques for retrieving the missing links to address the challenge of sifting through a huge number of pages in search of replacement connections. Another challenge cited by Delo is dealing with links that have the right URL but the wrong content. "You need a system to help manage this risk" so that a company's reputation will not be tarnished when visitors at its Web site are misdirected, he says. The software is designed so that users can select the pages to be updated automatically as well as the substitutions that require their notification. Although Peridot is not commercially available, it may eventually be employed to monitor the accuracy of internal corporate Web sites, easing the burden of Web administrators; the software could also be offered as a service to clients by ISPs, and could benefit Web site users as well.
Click Here to View Full Article
(Articles published within 7 days can be accessed free of charge on this site. After 7 days, a pay-per-article option is available. First-time visitors will need to register.)