Find broken external links on your site
Links to other sites rot. A page you linked in 2012 gets deleted, a company shuts its domain, a department moves its pages. Visitors who click land on an error, and a page full of dead links reads as neglected. You cannot spot them by browsing, because the dead link sits on page 40 of a site you rarely open. A crawl can check every one.
How a link check works
- Crawl your own pages and collect every link on them.
- Rank the targets by how many pages link to them and check the most-linked first.
- Ask each target for its status, first with a light HEAD request, then a GET that does not download the body when the server rejects HEAD.
- Follow redirects by hand, a few hops at most, and read the final status.
The Actor does this with checkLinks turned on, and a cap per site (maxLinkChecksPerSite) so you control the cost.
Broken, ok, and the third answer: unverified
A good checker has three verdicts, not two. Broken means the target answered 404 or 410, a server error, or its host no longer exists in DNS. Ok means it answered. Unverified means the check could not tell: the server answered 403 or 429 to an automated request, the request timed out, or the target's robots.txt asks automated visitors to stay out and the checker obeys it. Calling those broken would send you chasing links that work in a browser, so they get their own list for a manual click.
A real run: norvig.com, 15 pages, 56 links checked
| Verdict | Count | Examples |
|---|---|---|
| Ok | 51 | 27 internal, 24 external |
| Broken, external | 2 | csli-publications.stanford.edu/site/0937073954.html answered 404; www.ic.arc.nasa.gov no longer resolves in DNS |
| Broken, internal | 1 | norvig.com/einstein.html answered 404 and is linked from the home page |
| Unverified | 6 | mitpress.mit.edu/sicp/ and a Medium profile answered 403 to the check; a photo-hosting subdomain gave no answer; three Google URLs returned no verdict, as the target's robots rules or a timeout left the check unable to tell |
The run summary said 56 links checked, 3 broken, 6 unverified, and 4 skipped because the target's robots.txt disallows the request. Skipped links are not charged. Each broken row lists the pages that link to it, so the fix is a find-and-replace on named pages. The dead NASA host is the typical case: the link has been wrong for years and nothing on the site would ever have told the owner.
What to do with each verdict
- Broken, external: remove the link, or search for the page's new address and replace it. Archive copies are an option for citations.
- Broken, internal: fix first; this is your own page failing. Restore the page or redirect it.
- Unverified: open each in a browser once. If it loads, leave it; the site just refuses automated visitors.
Cost
Link checks are $0.001 each, on top of $0.008 per page audited. This run: 15 pages ($0.12) plus 56 links ($0.056), $0.176 in all. For a 100-page site with 300 distinct link targets that is $0.80 plus $0.30. Set maxLinkChecksPerSite low for a first pass and raise it when you see the result.
The same run also returned the full SEO audit of those pages at no extra step.
Try it: SEO Site Audit on Apify. $0.008 per page audited, $0.001 per link checked (optional). The Apify free plan includes monthly credit, so a test run costs nothing out of pocket.