Before anything else
Robots rules, CDN and firewall behaviour, and what the raw HTML contains. This is unglamorous and it is where a surprising share of the total gain sits, because a blocked crawler makes every other improvement irrelevant.
- AI user agents tested individually
- CDN and WAF rules reviewed
- Raw HTML checked without JavaScript
- Blocking decisions made deliberately
