A URL extraction tool built for operational SEO
Software that extracts, organises and analyses a website’s URLs from a sitemap or a crawl, with the ability to export the essential SEO data.
The goal was to quickly produce a usable list of a site’s URLs, for building a redirect plan or auditing a website. This grew into a full sitemap URL extractor and SEO crawler.
The need
A first Python prototype read a sitemap.xml and exported all URLs to CSV. But not every site has a usable sitemap, some don’t generate one, others make it hard to reach or leave it incomplete.
The solution
To work around that, the script became a crawler able to traverse a site directly, so it can extract URLs whether or not a clean sitemap exists, then organise and export the SEO data.
Key features
- URL extraction from sitemap.xml or live crawl
- Organised, analysable URL output
- Export of essential SEO data
- Ideal for audits and redirect mapping
Results
Building a redirect plan or auditing a site’s URL structure becomes a quick, repeatable task, even on sites with no usable sitemap.
Need an SEO audit or custom tooling? See my SEO audit and consulting.
