Collects public web pages for preservation through the Internet Archive. Archival capture is different from ordinary search indexing or an interactive AI answer. A capture does not guarantee that…
Harvests web content for the Portuguese web archive Arquivo.pt. The published crawler examples include Heritrix, Brozzler and Browsertrix identifiers. Keep the exact request string when investigating an archive visit.
Web Archiving
Interface
Web crawler / fetcher
License
Operator-managed / not specified
No agents found
Try a different search, category or interface, or reset the filters.
This directory includes open-source tools and operator-managed web identities. A crawler is not necessarily an installable AI agent. Profiles distinguish purpose, source limits, licenses and costs.