Technical AEO: schema, llms.txt and speed · Samurai Labs
  1. Home
  2. AI Visibility
  3. Technical AEO

Fix what the machine reads first.

Technical AEO is the engineering work that makes a site eligible to be ranked and quoted: one consistent entity record, schema, an AI crawler policy, fast pages and clean crawl paths. We fix it in the code and on the server, then test it again.

These are code problems.

Core Web Vitals, rendering, redirects and schema are fixed in the theme and on the server. Most agencies send a list to someone else. We ship the fix and re-run the test.

Found in the audit

  • 1.96 sserver response
  • 29.5 smobile load, homepage
  • 4.8 MBhomepage weight
  • 0 of 16pages with a meta description
  • 16old URLs returning 404
  • Nonellms.txt or AI crawler policy

Exit measure

  • under 0.5 son cached pages
  • under 3 son every page
  • under 1.5 MBafter the asset diet
  • 16 of 16titled and described
  • 0dead URLs, all redirected
  • Publishedand checked against the firewall

Sample audit, client anonymised. Exit measures are targets from its programme, not results.

Six things we fix.

01

One entity fact set

Name, parent, head office, founding year, what the company does, its products and its proof: written once, then applied to site schema, Business Profile, LinkedIn, Crunchbase and directories. An AI engine building a picture of the company reads all of them.

02

Schema that says something

Organization, Service, Product, Person, FAQ, Event and Breadcrumb markup generated from the fact set and validated on every release, in place of generic plugin defaults.

03

llms.txt and crawler policy

A plain-text map of the site for AI crawlers, and explicit rules for GPTBot, ClaudeBot, PerplexityBot and Google-Extended in robots.txt.

04

Speed

Server caching and compression, an asset diet, font and image fixes. Measured with Lighthouse before and after, on mobile and desktop.

05

Crawl hygiene

Redirects for dropped URLs, dead links removed, template sitemaps cleaned, one H1 per page, and titles and descriptions that carry the words buyers use.

06

Answer‑first templates

One template per page type with a dated fact block, a named expert, a comparison table where one exists and an answer-shaped FAQ. Built once, then generated for every page of that type.

The same files, on this site.

We publish what we set up for clients. Both files below are live here: /robots.txt and /llms.txt.

robots.txtAI crawler policy
# AI answer engines and their retrieval agents:
# explicitly allowed.
User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ClaudeBot
User-agent: PerplexityBot
User-agent: Google-Extended
Allow: /

Sitemap: https://buildwithsamurai.com/sitemap.xml
llms.txtSite map for AI systems
# Samurai Labs

> Samurai Labs is a product studio and
> technology partner based in India...

## AI Visibility Engineering

- [AI Visibility Audit](/ai-visibility-audit.html):
  A fixed-scope, evidence-led audit.
- [Technical AEO](/technical-aeo.html):
  The engineering work behind being cited.

Excerpts, shortened for the page. The robots.txt group lists more agents than shown.

Four weeks, an output to check each week.

The foundation sprint from the sample programme. You do not wait for a report: each week ends with something you can verify yourself.

WeekWorkOutput you can check
Week 1Access, analytics, caching and compression, redirects, dead links, sitemaps.Server response under 0.5 s. 0 dead URLs. Search Console live.
Week 2Titles, descriptions, H1 fix, alt text, SVG logo, llms.txt, AI crawler policy.Every page re-crawled. Lighthouse SEO 95 or above.
Week 3Entity fact set. Business Profile, LinkedIn and data profiles corrected. Missing directories claimed.One consistent record everywhere, with listing confirmations.
Week 4Page templates signed off. Ranking and AI-answer baselines locked.First pages in review. Baseline report.

A report says what to do. We leave you with a site that does it.

Questions, answered.

Technical AEO is the engineering work that makes a site eligible to be ranked by Google and quoted by AI engines: a consistent entity record, structured data, an AI crawler policy, fast pages and clean crawl paths. It is the foundation that content and outreach then build on.

llms.txt is a plain-text file at the root of a site that gives AI systems a short, structured map of what the site is and where its key pages are. It is a proposed convention, not a standard every engine honours, so we treat it as cheap insurance alongside schema and a clear robots.txt, not as a ranking lever on its own.

They read the same robots.txt but identify themselves with their own names, such as GPTBot, ClaudeBot, PerplexityBot and Google-Extended. A site that declares nothing leaves the decision to defaults. We state the policy explicitly, and check that a firewall or bot filter is not blocking them by accident.

Yes. The technical foundation is a self-contained sprint with outputs you can check yourself: a Lighthouse run, a re-crawl, a schema validator pass. Content and earned presence can follow later, with us or with your own team.

The stack you already have. We work in WordPress and Elementor, in custom-built sites, and in the products we build ourselves. Changes are tracked in version control and shipped to your site, not handed over as a list.

Slow site, thin schema, no crawler policy?

(PING US)

Request an audit ↗

Or reach us at contact@buildwithsamurai.com

Request an audit ↗