AI-readiness scanners can flag WordPress sites when robots.txt does not declare AI content preferences or when the homepage lacks useful HTTP Link response headers. In one support case, both checks were failing even though earlier custom code had already attempted to address them.
The developer first confirmed how the site owner wanted AI services to use the content. The approved configuration allowed traditional search indexing, disallowed AI model training, and disallowed AI retrieval for generated answers. The live robots.txt was then updated with the corresponding Content Signals, and an existing WPCode snippet was revised so the homepage returned Link headers pointing to real public resources: the XML sitemap and Privacy Policy. No fake API catalog was added because the site did not actually have a public API catalog or agent-facing API documentation.
Issue background
An AI website-readiness scan reported two issues on a WordPress site:
- No Content Signals were detected in
robots.txt. - The site’s Link response headers did not satisfy the scanner’s agent-discovery recommendation.
The scan suggested examples such as an API catalog or API documentation link, but the site’s existing integrations were internal systems used for forms, chat, and marketing automation. There was no public API catalog to advertise.
The site also already contained two custom snippets related to the recommendations:
- Content Signals code existed, but the directives were not appearing in the live
robots.txt. - An existing Link-header snippet pointed only to the Privacy Policy.
That meant the work was not a greenfield implementation. The existing code needed to be corrected and verified against the live site.
Diagnosis
Before changing the implementation, the developer separated the technical issue from the policy decision.
Content Signals express preferences about how automated systems may use site content, so the team first asked the site owner to confirm the intended policy for:
- AI model training.
- Traditional search-engine indexing.
- AI retrieval and use in generated answers.
The proposed restrictive configuration was approved:
AI model training: Not allowed
Search engine indexing: Allowed
AI retrieval and generated answers: Not allowedThe developer also reviewed the site for any public API catalog or agent-facing service documentation. None existed.
That finding mattered because the scanner’s example:
Link: </.well-known/api-catalog>; rel="api-catalog"would have been inaccurate for this site. Rather than create a placeholder resource simply to satisfy a scan, the developer chose to advertise useful public resources that actually existed.
Resolution steps
The documented implementation was:
- Confirm the site owner’s AI-content preferences. The owner approved a configuration that blocked AI training and AI retrieval while continuing to allow traditional search indexing.
- Update the live robots.txt output. The Content Signals were configured as:
Content-Signal: ai-train=no, search=yes, ai-input=no- Clear cache and verify the live robots.txt. The developer confirmed that the directives were being served publicly after cache clearing.
- Review the existing Link-header implementation. The site already used a WPCode snippet for HTTP Link response headers, but it advertised only the Privacy Policy.
- Do not invent an API catalog. Because the site had no public API catalog or agent-facing API documentation, the developer intentionally did not add an
api-catalogrelation pointing to a nonexistent resource. - Advertise useful resources that actually exist. The WPCode snippet was updated so the homepage Link headers referenced the site’s XML sitemap and Privacy Policy.
- Clear caches again after the header change. This ensured the updated HTTP response headers were not masked by cached output.
- Verify the homepage response headers. The developer confirmed that the new Link headers were being returned on the live homepage.
- Confirm both scan recommendations had been addressed. After verification, the task was closed as complete.
The source task does not document the creation of API documentation, a /.well-known/api-catalog endpoint, or any public API inventory. Those examples appeared only in the scanner’s recommendation and were deliberately not implemented because they did not reflect the actual site.
Final outcome
The WordPress site was updated with live Content Signals that matched the approved AI-content policy:
ai-train=no
search=yes
ai-input=noThe existing WPCode Link-header snippet was also updated so the homepage advertised real public resources—the XML sitemap and Privacy Policy—instead of pointing to a nonexistent API catalog.
After cache clearing, the developer verified both the live robots.txt output and the homepage HTTP Link headers.
The broader lesson is that AI-readiness work should reflect the site’s real capabilities and the owner’s actual content-usage preferences. A scanner may suggest an API catalog as an example, but adding a fake or empty endpoint is not a good substitute for useful, accurate metadata. First define the policy, then expose only the resources that genuinely exist.
If you need help implementing Content Signals, HTTP Link response headers, or other AI-readiness recommendations on WordPress, contact Freshy. Our WordPress team can review the scanner findings, separate policy decisions from technical implementation, and make sure the live output matches what the site actually supports.
