Updated the Googlebot user agent strings to match the latest Chrome versions at the time.
Today, the tool stands as a powerhouse of , JavaScript rendering , and Generative AI optimization . Whether you are auditing a small blog or an enterprise e-commerce site, the Screaming Frog SEO Spider—whether in its 12.6 legacy form or the modern version—is the definitive standard for technical SEO.
Beyond HTML text pages, the tool evaluates the supporting components of a website:
The default setting. Input a starting URL, and the crawler follows internal links hierarchically to map out the entire website architecture. Screaming Frog SEO Spider 12.6
Go to Configuration > System > Storage Mode and select Database Storage Mode to save crawl data directly to your hard drive instead of consuming your RAM.
JavaScript-heavy websites (especially those built on React, Vue, or Angular) have historically been difficult for traditional crawlers. While previous versions included a JavaScript rendered crawl mode, introduces a dedicated Custom JS tab.
The SEO Spider maintains a that keeps advanced SEO accessible to professionals at all levels: Updated the Googlebot user agent strings to match
Version 20.0 introduced custom JavaScript snippets, giving users the flexibility to directly within the crawling process. This is especially useful for:
Allows users to upload a specific set of URLs via a text file, CSV, or manual copy-paste. This is ideal for verifying redirect chains, checking status codes after a migration, or auditing specific landing pages.
Integrating the GSC API overlays organic performance data directly onto crawl paths. Beyond HTML text pages, the tool evaluates the
For enterprise SEOs crawling massive e-commerce sites (Amazon, Zillow, etc.), version 12.6 introduces a "Performance Mode."
Go to Sitemaps > XML Sitemap to automatically generate a clean, search-engine-ready sitemap based on your clean crawl data.
For most technical SEO audits in 2026, recommended configurations include:
