=== AI-Only Pages ===
Contributors: tommyoz12
Tags: seo, ai, noindex, robots, llm
Requires at least: 5.5
Tested up to: 6.7.2
Requires PHP: 7.4
Stable tag: 1.3.4
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html
Mark any page as AI-only. Hidden from search engines, optimized for AI crawlers, listed in /llms-index.txt. Includes Token Diet and global settings.
== Description ==
AI-Only Pages gives you granular control over which search engine bots can index each page on your WordPress site — while simultaneously making those pages more discoverable and useful for AI crawlers like ChatGPT, Claude, and Perplexity.
**The core idea:** you have content that is perfect for AI training pipelines and retrieval-augmented generation (RAG) systems, but you do not want that content competing for rankings in Google, Bing, or Yahoo.
AI-Only Pages lets you mark those pages as AI-only: they disappear from traditional search engine indexes while becoming first-class citizens in the AI ecosystem.
= What it does =
* **Per-bot noindex** — Block individual bots (Googlebot, Bingbot, Yandexbot, etc.) with a checkbox per bot per page. Checking one bot blocks it; the others still index normally.
* **"Block All" master toggle** — One click blocks all 10 supported search engine bots simultaneously.
* **`` tags and HTTP headers** — Both `` HTML meta tags and `X-Robots-Tag` HTTP headers are emitted, covering all crawling contexts. Works correctly on all public post types including Pages and custom post types.
* **SEO plugin integration** — Suppresses Yoast SEO, WP Core, and RankMath's global `` tag on AI-only pages so there is no conflict between the global tag and your per-bot tags.
* **Sitemap exclusion** — AI-Only pages are automatically removed from all XML sitemaps (Yoast SEO and WP Core sitemaps are both supported).
* **`/llms-index.txt`** — A plain-text AI discovery file served at `yoursite.com/llms-index.txt` listing all AI-only pages with their titles and last-modified dates. AI crawlers can use this file to find your AI-optimised content directly. Can be toggled on/off from the settings page.
* **Token Diet — clean AI output** — When an AI crawler visits an AI-only page, the plugin serves a cleaned version of the HTML with navigation, sidebars, footers, cookie banners, inline styles, SVGs, and iframes stripped out. AI models receive pure content with minimal noise.
* **Global Settings Page** — A top-level "AI-Only Pages" menu in the WordPress admin sidebar lets you configure Token Diet and LLM Index behaviour globally, without touching code.
* **Caching plugin notice** — If WP Rocket, LiteSpeed Cache, or another full-page caching plugin is detected, an admin notice explains how to configure it to work alongside this plugin.
= The Settings Page =
A full settings page is available under **AI-Only Pages** in the WordPress admin sidebar. It provides:
**Section 1 — Instructions & Status:** A "How It Works" guide covering the meta box, Token Diet, and LLM Index. A live, clickable URL to your `/llms-index.txt` file with a green/red status indicator showing whether the index is active.
**Section 2 — LLM Index Settings:** A toggle to enable or disable `/llms-index.txt` globally. When disabled, the endpoint returns a 404.
**Section 3 — Token Diet Master Control:** A master toggle to enable or disable Token Diet entirely. When off, AI bots receive raw, full HTML — identical to what human visitors see.
**Section 4 — Granular Token Diet Stripping:** Individual toggles for each category of content stripped:
* Strip structural layout (headers, footers, sidebars, navigation, cookie banners)
* Strip `