Automated Technical Site Crawler

Continuously traverses internal anchor graphs, probes HTTP status codes, flags 404 dead ends, collapses 301→302 redirect loops, and verifies document semantics.