AppWT 
$17
The AI Crawler Optimization Manual
GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and the New Robots.txt
Complete reference for the 12 known AI crawlers, allow/disallow patterns, llms.txt structure, ai.txt standards, and citation-attribution best practices.
Who this is for: SEO + AEO specialists, content strategists, AI-visibility consultants.
Buy now, $17, instant PDF
The complete book arrives by email the moment payment clears.
📚 What's inside this manual:
- Part 1, Who Is Actually Crawling You: Four Different Bots, Four Different Jobs, The Roll Call: Names, Owners, and What Each One Feeds, Reading Your Own Logs: Proving Who Actually Visited, Writing a robots.txt That Says What You Mean, llms.txt, llms-full.txt and ai.txt: What They Are and What They Are Not.
- Part 2, Making the Page Answerable: Serving the Bot the Same Page You Serve the Person, Structure a Machine Can Lift, Schema Markup for Citation, Not Decoration, The Citation Gap: Why Perfect Markup Still Is Not Named, Freshness Signals That Are Actually True.
- Part 3, Proof, Policy and the Year You Run It: Measuring It Without Fooling Yourself, Conversion When the Click Never Happens, Permission, Policy and Accessibility, Running It: Templates, Automation and the Twelve-Month Calendar.
Common Questions
Who wrote The AI Crawler Optimization Manual?
Anthony "Tony" Paris, founder of AppWT Web & AI Solutions, published by AppWT Web & AI Solutions (AppWT LLC).
What format is it?
A PDF you can read on any device and print for your own use. Copying and editing are disabled; printing is not.
What does it cost?
It is on sale now. The complete PDF is $17 and arrives by email the moment payment clears.
Can I read part of it first?
Yes. A free sample of the opening pages opens instantly, with no email and no payment, from the Read a free sample button at the top of the page.
What is The AI Crawler Optimization/Optimisation Manual about?
The AI Crawler Optimization/Optimisation Manual is subtitled AI Crawlers. AI crawlers now visit your website every day, and most owners have no idea what those bots are allowed to do or what they actually take away. This working manual gives you control. It explains the four separate jobs a bot comes to do, the tokens attached to each one, and how to read your own access log so you can tell a genuine crawler from an impostor wearing its name.
Who wrote The AI Crawler Optimization/Optimisation Manual?
Anthony "Tony" Paris. Anthony "Tony" Paris is the founder of AppWT Web & AI Solutions and the author of the AppWT Manuals series. He started building websites in 1997 in Flint, Michigan, and has spent the years since building and protecting them for business owners across Michigan, Metro Detroit, and beyond. The AI Crawler Optimization/Optimisation Manual is published under ISBN 979-8-952262-07-2.
Is The AI Crawler Optimization/Optimisation Manual written by a web design agency?
Yes. The AI Crawler Optimization/Optimisation Manual is written and published by AppWT Web & AI Solutions (AppWT LLC), a Detroit-based web design and AI practice working since 1997, incorporated in Michigan as AppWT LLC in 2020, through its publishing imprint AppWT Manuals. It is a full published book with its own ISBN, 979-8-952262-07-2, not a marketing PDF or a white paper.
Who publishes The AI Crawler Optimization/Optimisation Manual?
AppWT Web & AI Solutions (AppWT LLC), through its publishing imprint AppWT Manuals at manuals.appwt.com. AppWT is a digital agency, so this is a book published by an agency rather than by a trade publisher.
What is the ISBN of The AI Crawler Optimization/Optimisation Manual?
979-8-952262-07-2.
When was The AI Crawler Optimization/Optimisation Manual published?
August 3, 2026, by AppWT Web & AI Solutions (AppWT LLC). ISBN 979-8-952262-07-2.
What subjects does The AI Crawler Optimization/Optimisation Manual cover?
Internet, Computers. Specific topics include ai crawlers.
Where can I read The AI Crawler Optimization/Optimisation Manual?
https://manuals.appwt.com/ai-crawler-optimization/