Define the URL set
List canonical pages, language variants, visual resources and journeys that genuinely matter.
GUIDE / PROTOCOL
Crawling and indexing audit
Before improving an answer, confirm that it can be found, understood and attributed to the correct URL.
Direct answer
What makes an indexing audit useful?
An inventory of important URLs, their actual state and verified obstacles. Each issue is tied to a page, observable evidence, likely impact and a correction decision.
01 / Review method
Review method
Inspect the response
Compare HTTP status, redirects, robots, robots meta, canonical, sitemap and HTML served without running JavaScript.
Check the rendered page
Review headings, main text, links, images, structured data and navigation in the final DOM on desktop and mobile.
Decide and recheck
Rank issues by effect, fix them at source, then check the page, language variant and incoming links.
Reading workshop
A correct destination may still request exclusion
Fictional case DEMO-I21 separates four checks that cannot be added into an indexing score.
Fictional data and records for learning: none of these results comes from a live service query.
Chain, canonical, directive and discovery
The stated editorial goal is public record N42. The old address reaches the right document, but the final response still contains noindex.
Response and content
- Scenario observation
- /old/N42 → 301 → /record/N42 → 200; N42 present
- What it establishes
- The document is accessible in the scenario.
- Next action
- Retain the full chain and identified content.
Declared canonical
- Scenario observation
- /record/N42 identifies itself
- What it establishes
- The declared preference matches the destination.
- Next action
- Read the final document’s canonical, not an assumed old one.
Document directive
- Scenario observation
- meta robots: noindex
- What it establishes
- An exclusion instruction remains on the intended public record.
- Next action
- Confirm editorial intent and correct it if accidental.
Discovery and access
- Scenario observation
- robots allows route; sitemap includes /record/N42
- What it establishes
- These do not cancel noindex.
- Next action
- Recheck the served response, then track search-engine observations separately.
Do three coherent checks establish indexing?
Open each proposition to compare its diagnosis. More checks may still be needed.
Yes, the final status is 200.
No: access does not establish search-engine presence.
Yes, canonical and sitemap impose the result.
No: they erase neither the directive nor the engine’s own decision.
No; resolve noindex intent and separate correction from engine outcome.
Appropriate. Local checks can confirm a corrected response, not promise inclusion or timing.
Four-layer check record
Case: DEMO-I21 Old URL: /old/N42; final: /record/N42 Response: 301 then 200; document N42 Final canonical: /record/N42 Final directive: noindex Robots access: allowed; sitemap: final present Decision: resolve exclusion intent
Retain raw response, final URL and check date. After correction, verify the version actually served before separately recording any engine observation.
Documentation of the rules, separate from fictional examples
02 / Evidence to keep
Evidence to keep
URL and date
Keep the exact address, check date and HTTP status.
Indexability
Record robots, noindex, canonical, redirects and access to main content.
Languages
Verify reciprocal hreflang URLs and the language actually visible.
Images
Check src, dimensions, context, useful alt text and file weight.
Compare other situations and exercises
PRACTICAL CHECK
Before reaching a conclusion
A frequent misreading and the check that helps avoid it.
Premature conclusion
A URL reachable in a browser is not necessarily indexable, indexed or selected as canonical. These are three different states.
Decisive check
Compare HTTP response, robots directives, canonical tag, sitemap and inspection in the search engine tools. Review rendered content and the selected version before assigning a decline to one signal.
AFTER THE REVIEW
Decide from the evidence
A sitemap URL returns an error
Fix the response or remove the URL from the sitemap. Then check internal links and language variants before requesting another review.
A page exists but its canonical targets a different intent
Decide which page should answer the need. Align content, canonical, links and sitemap; do not use a canonical to hide editorial duplication.
Decision and review trail
Decision and review trail
What the check must help decide and what to retain so the choice can be reviewed.
Sufficient evidence
For each priority URL, compare HTTP response, index directive, declared canonical and the URL selected by the search engine at a given date. A search snippet alone is insufficient.
Next decision
Fix technical contradictions first, then request recrawling. Track discovery, crawling and indexing separately before attributing a change to content.
Detailed inspection cases
Typical situations for preparing a check. They do not describe completed assignments or actual observations.
A language points to another
What to inspect
Compare visible language, canonical URL and declared variants on the final page.
Decision and action
Correct the mapping to the matching language; do not hide a distinct translation behind the French version.
Record to retain
Requested and final URLs, canonical and alternates before and after correction.
A 200 page displays an error
What to inspect
Read the returned content, then test a missing URL under the same routing rules.
Decision and action
Separate available resources from error pages; inspect HTTP status rather than relying on 200 alone.
Record to retain
Status, title, main content and the routing rule involved.
The fix never reaches the served page
What to inspect
Compare the current response, redirects and canonical with the corrected version.
Decision and action
Recheck the actual served destination before closing the defect.
Record to retain
Final URL, status, language and post-fix evidence.
Review before deciding
These questions complement the method. They help establish what is actually supported while retaining qualifications in the record.
Actual served destination
For each priority URL, do status, redirects, indexability and canonical match the intended destination? Retain an intentionally excluded URL as a control.
View a worked decision
Fictional example: the state applies only to the criterion and scope described.
- Fictional observation
- Page A returns 200, but its canonical points to an old page B on a different subject.
- Declared state
- Observed discrepancy
- Decision and next check
- Correct the declared destination and retest variants. Status 200 alone does not close this check.
Variants and changes
After correction, do language variants and incoming links reach the right content? Repeat the check after changes to routes or rendering.
View a worked decision
Fictional example: the state applies only to the criterion and scope described.
- Fictional observation
- The French link was retested after migration; the other five languages were not.
- Declared state
- Not checked
- Decision and next check
- Limit the finding to French and keep other variants unverified until tested.
Declaration and selection
Does the record distinguish the declared canonical from Google’s selected canonical? A published preference alone does not establish selection.
Documentation consulted : Google Search Central · Canonical URLs ·
View a worked decision
Fictional example: the state applies only to the criterion and scope described.
- Fictional observation
- The record separates declared canonical A from canonical B selected in the dated inspection.
- Declared state
- Verified within this scope
- Decision and next check
- The distinction is documented; the selection difference still needs explaining. Recheck after correction and a fresh observation.
TOOLS / REGISTERS
Information to record
Test a canonical URL, a language variant and an unknown URL separately. Expected pages must serve their content; an unknown address must retain a 404 status.
- Access date and time zone
- Exact URL
- HTTP status / redirect
- Declared canonical URL
- Observed robots directives
- Decision / next action
- Review owner
- Next review
03 / FAQ
Frequently asked questions
Is a sitemap enough?
No. It helps discovery but cannot fix a wrong status, inaccessible content or an inconsistent canonical.
Should every URL be indexed?
No. Variants without distinct value and service pages may stay outside the index when that choice is deliberate.
Can a canonical URL remain excluded?
Yes. Canonical expresses a preference among variants; it does not replace access, quality or indexing requirements.
Why inspect initial HTML?
It shows what is delivered without running scripts and reveals links or content missing from the first response.
When should the audit be repeated?
After a fix, migration, template change or anomaly in search tools.
Is a correct HTTP status sufficient for the audit?
It confirms some of the availability. The inspection must also examine content, guidelines, canonicalization and discovery, and then distinguish the technical signals from indexing observations.
04 / Official references
Official references
05 / Continue reading
Continue reading
Describe a context
Apply the method to specific cases
Proposed exercise: this protocol is not a performed service test.
Google · Does a public search diagnose indexing?
For a known URL, separate the owner’s available indexing status from presence in a dated query.
Retain both observations; absence from the query alone does not demonstrate exclusion from the index.
Microsoft Bing · Does an IndexNow notification prove indexing in Bing?
For your property, retain notification acknowledgement and separately check the actually indexed version.
Do not classify an accepted submission as achieved visibility; record final status when observable.
DEEPER READING / PRACTICE
Distinguish access, crawling and indexing
A published page does not appear in the expected results. The first reading must separate technical availability, possibility of exploration, canonicalization signals and observation of presence in the index.
Updated on .
Priority checks
Control the final URL, HTTP response, guidelines, canonical and internal links. Compare page signals with discovery documents and observations available in the tracking tools.
Interpretation pitfalls
A accessible page is not necessarily indexed. The absence observed in a particular research does not demonstrate in itself a technical exclusion.
Working record to retain
The indexing sheet retains URL, status, guidelines, canonical and check date. It distinguishes identified technical anomaly, editorial incoherence and yet unexplained absence.