Skip to main content

Internal PageRank: finding orphan pages and link weight leaks

Our write-up of a story first reported by Habr SEO

Internal PageRank: finding orphan pages and link weight leaks

On a large site nobody holds the full link structure in their head at once. The home page leads to the catalogue, the blog links to filters, filters link to each other, and somewhere in that tangle are pages that receive almost no internal weight. Intuition works at fifty pages; at five thousand it does not.

The problem has a classical numerical answer: a site-local version of PageRank. Rather than ranking the whole web, it calculates weight using only the internal links of one site. The site is modelled as a directed graph — pages are nodes, links are edges — and a page's weight is the total link weight that flows to it along all paths from other pages.

Formula and normalisation

The structure is the original formula: page A receives weight from every page that links to it, each donor dividing its own weight evenly across all its outgoing internal links. A damping factor of 0.85 is passed forward; the remainder is distributed as a baseline value. Iterations repeat until convergence.

Normalisation makes the numbers readable. The site-wide mean is set to 1.0, so 1.0 is an average page, 3.0 is three times stronger than average, and 0.2 is nearly weightless. A percentage share sits alongside each score, showing what fraction of the site's total weight a page holds.

Details that keep the calculation honest

Dangling pages — those with no outgoing internal links — bleed weight out of the system in a naive implementation. The fix is to redistribute their weight evenly across the site on every iteration.

URL normalisation matters just as much. http://x, https://www.x/ and x/#fragment must collapse to a single node, or the graph grows duplicate nodes that split weight between themselves.

Sitewide navigation blocks — header menus and footers — link to the same sections from every page on the site. Without filtering them out, all weight concentrates in those navigational targets. The solution is to count only links found inside the content area.

Anti-bot protection requires its own handling. When a site responds to a crawler with a JavaScript challenge page, a plain HTTP request cannot pass it. The correct path is to open the site in a browser, complete the check, and pass the session cookies to the crawler as a parameter.

Reading the report

On a hypothetical online store of a thousand pages, the home page typically scores 8–12 (every page links to it via the logo), top-level catalogues score 2–4, product cards score 0.3–0.8 depending on how many places link to them, and a third-level page reachable only through a single footer link scores around 0.1. None of those numbers is inherently bad; the problem is disproportion — a product card meant to convert sitting at 0.2 while a neighbouring filter page sits at 3.0.

Four sections of a report flag the most actionable issues. Orphan pages have zero incoming links and receive no weight at all. Dangling pages collect weight but pass none onward. Pages at depth minus are unreachable by following links from the home page and appear only through a sitemap. Sorting by weight against business priorities is itself a work plan: a commercial landing page ranked 40th by weight is a signal for internal linking work.

Simulating a link before adding it

The most practical part of the approach is that the effect of a new link can be calculated before touching the site. A candidate donor-to-acceptor pair is added as an edge, weights are recalculated, and the delta is read off. For graphs above 2,500 pages, running a full recalculation for every candidate takes too long; the first pass uses an approximation (expected gain ≈ d · W(donor) / (outgoing links + 1)), and the full recalculation runs only for selected pairs.

Two constraints apply regardless. Total site weight is fixed by normalisation — internal linking does not increase it, only redistributes it. A tool that shows only gains and not which pages lose weight is misleading by omission. And absolute scores from two separate crawls are not comparable if the number of discovered pages differs; percentage share and rank order are the right basis for comparison.

TG X Copied

Project sponsors

Companies that keep this analytics open