Robots - Musharof Chy
User Agent String
Tool from ASAP Consulting s.r.o. for detailed user agent string analysis using an online form. Incl...
The Web Robots Pages
Information on the robots.txt Robots Exclusion Standard and other articles about writing well-behav...
Search Engine Robots and Other User Agents
John A. Fotheringham presents data in tabular form on the robots sent by search engines and other s...
About Search Indexing Robots and Spiders
Search Tools Consulting explains how the search engine programs called "robots" or "spiders" work, ...
Search Engine IP Addresses
Lists IP addresses of search engine spiders. Can be searched by IP address. Also links to resources...
HTTP User Agent Index
An alphabetical list of user agents and the deployer behind them, compiled by Christoph Rüegg.
Bots vs Browsers
This large database lists user agents in categories and distinguishes between robots and browsers.
List of User-Agents
A searchable database of user-agents with information about their type, purpose and origin.
User-Agents.My-Addr.com
Contains a database of user-agents for crawlers, spiders, browsers; tools for user-agent lookup and...
ACAP - Automated Content Access Protocol
Standard being developed on behalf of content publishers to communicate permissions information mor...