1. Bot Manager Overview

Bot Manager Overview

SEO Repair Kit’s Bot Manager gives you tools for controlling how search-engine crawlers and AI bots interact with your WordPress website.

It brings together two important areas:

  • LLMs.txt for AI content discovery and bot access controls
  • Robots.txt for managing crawler instructions

In this guide

  • How to access Bot Manager
  • Understanding the Bot Manager tabs
  • LLMs.txt
  • AI Bot Access Control
  • Content selection
  • Robots.txt
  • Where to start

How to Access Bot Manager

From your WordPress dashboard, go to:

SEO Repair Kit → Bot Manager

The Bot Manager screen opens with two main tabs:

  • LLMs.txt
  • Robots.txt

LLMs.txt

The LLMs.txt tab provides tools for generating and managing an LLMs.txt file for your website.

The current interface lets you:

  • Control access for listed AI bots
  • Select Post Types
  • Select Taxonomies
  • Set a Posts/Terms Limit
  • Add Additional Content
  • Generate LLMs.txt
  • Review generated LLMs.txt content
  • Preview the file
  • Save the file
  • Delete LLMs.txt

Create and Manage an LLMs.txt File →

AI Bot Access Control

The AI Bot Access Control section lets you choose which listed AI bots can access your LLMs.txt file.

The current interface includes bots/services such as ChatGPT, Claude, Google Bard/Gemini, Perplexity AI, Bing Chat/Copilot, You.com, Character.AI, CCBot, Anthropic AI, DeepSeek, Grok, Qwen AI, Meta Llama, Cohere, Mistral AI, and Hugging Face.

Unchecked bots are handled according to the blocking behavior described in the Bot Manager interface.

Control AI Bot Access →

Content Selection

Bot Manager lets you choose which website content should be included when generating LLMs.txt.

You can select:

  • Post Types
  • Taxonomies
  • Maximum Posts/Terms per type
  • Additional custom content

The exact Post Types and Taxonomies available depend on what is registered on the WordPress website.

Choose Content for LLMs.txt →

Robots.txt

The Robots.txt tab provides an editor for managing your website’s robots.txt content.

From this area, you can:

  • Edit robots.txt
  • Save changes
  • Validate the file
  • Reset it to default
  • Apply an enhanced configuration
  • Delete the custom robots.txt

Manage Your Robots.txt File →

Where Should I Start?

If you are using Bot Manager for the first time, first review the LLMs.txt configuration.

Choose which AI bots you want to allow, select the content you want included, and then generate and review the file before saving it.

After that, review your Robots.txt configuration separately.

Create and Manage an LLMs.txt File →

2. Create and Manage an LLMs.txt File

How to Create and Manage an LLMs.txt File

SEO Repair Kit’s LLMs.txt Generator helps you create an LLMs.txt file containing selected information and links from your WordPress website.

In this guide

  • Open the LLMs.txt Generator
  • Find your LLMs.txt URL
  • Configure the generator
  • Generate LLMs.txt
  • Review generated content
  • Preview the file
  • Save LLMs.txt
  • Delete LLMs.txt

Open the LLMs.txt Generator

Go to:

SEO Repair Kit → Bot Manager → LLMs.txt

At the top of the screen, SEO Repair Kit displays the location where the LLMs.txt file is available.

Find Your LLMs.txt File

The purple information bar displays:

Your llms.txt file is available at:

followed by your website’s LLMs.txt URL.

For example, it follows a structure such as:

https://example.com/llms.txt

Use the URL displayed for your own website rather than copying a URL from another installation.

Configure Your LLMs.txt

Before generating the file, configure:

  • AI Bot Access Control
  • Post Types
  • Taxonomies
  • Posts/Terms Limit
  • Additional Content

These settings determine the access configuration and website content used by the LLMs.txt workflow.

The following guides explain these options in detail:

Control AI Bot Access →

Choose Content for LLMs.txt →

Generate LLMs.txt

After configuring the required options, click:

Generate LLMs.txt

SEO Repair Kit generates content according to the selected configuration.

Review LLMs.txt Content

Scroll to:

LLMs.txt Content

The generated content appears in the large content area.

Review the generated content before saving the file.

Check that the information and links included are appropriate for the website.

Preview LLMs.txt

Click:

Preview

to review the LLMs.txt output before finalizing it.

This is useful for checking the generated content without relying only on the configuration controls above.

Save the LLMs.txt File

After reviewing the generated content, click:

Save LLMs.txt

This saves the current LLMs.txt content for the website.

Save Generator Settings

The generator also provides:

Save Settings

Use this to save the current generator configuration.

This is separate from the Save LLMs.txt action shown below the generated content.

Reset Generator Options

Click:

Reset Options

when you intentionally want to return the generator options to their default configuration.

Important: Review your current AI bot and content-selection settings before resetting them.

Delete LLMs.txt

The bottom of the generator provides:

Delete LLMs.txt

Use this when you intentionally want to remove the generated LLMs.txt file.

Important: Do not use Delete LLMs.txt simply because you want to regenerate the content. Use Generate LLMs.txt and the available save workflow when updating the file.

Control AI Bot Access →

3. Control AI Bot Access

How to Control AI Bot Access

The AI Bot Access Control section lets you choose which listed AI bots can access your LLMs.txt file.

In this guide

  • Find AI Bot Access Control
  • Understand checked and unchecked bots
  • Select individual bots
  • Select or deselect all bots
  • Save your configuration

Open AI Bot Access Control

Go to:

SEO Repair Kit → Bot Manager → LLMs.txt

Find:

AI Bot Access Control

Understand Bot Access

Each listed bot has a checkbox.

The interface explains that unchecked bots are blocked through server-level checks and robots.txt rules.

It also states that bots not listed in this section are allowed by default.

Therefore, review the available bots carefully according to how you want the site configured.

Select Individual AI Bots

Check the box beside a bot you want enabled according to Bot Manager’s access-control workflow.

The current interface lists:

  • ChatGPT (GPTBot)
  • ChatGPT User
  • Claude (Anthropic)
  • Google Bard/Gemini
  • Perplexity AI
  • Bing Chat/Copilot
  • You.com
  • Character.AI
  • CCBot (Common Crawl)
  • Anthropic AI
  • DeepSeek
  • Grok (xAI)
  • Qwen AI (Alibaba)
  • Meta Llama
  • Cohere
  • Mistral AI
  • Hugging Face

Note: Bot/service names and available controls can change between plugin versions. Document the options shown in the current Bot Manager interface.

Deselect All Bots

The section provides:

Deselect All

Use this when you intentionally want to clear all currently selected AI bot options.

Review the note below the bot list before doing so because unchecked bots are subject to the blocking behavior described by the interface.

Save the Configuration

After changing bot access:

  1. Review the selected bots.
  2. Configure the content you want included in LLMs.txt.
  3. Click Save Settings.
  4. Generate LLMs.txt again if the file needs to reflect the updated configuration.
  5. Review and save the generated file.

Choose Content for LLMs.txt →

4. Choose Content for LLMs.txt

How to Choose Content for LLMs.txt

Bot Manager lets you control which WordPress content is considered when generating your LLMs.txt file.

You can configure:

  • Post Types
  • Taxonomies
  • Posts/Terms Limit
  • Additional Content

In this guide

  • Select Post Types
  • Select Taxonomies
  • Set Posts/Terms Limit
  • Add Additional Content
  • Generate the updated file

Select Post Types

Find:

Select Post Types

Check the Post Types you want included in the LLMs.txt file.

Your current WordPress installation displays options including:

  • Posts
  • Pages
  • Media
  • Forums
  • Topics
  • Replies
  • Floating Elements
  • My Templates
  • Structured Data
  • Jobs
  • Courses
  • Lessons
  • Quizzes
  • Assignments
  • Events
  • Reviews
  • Collections

The exact list depends on the Post Types registered on the website.

Select or Deselect All Post Types

Use:

Select / Deselect All

to quickly change all Post Type selections.

For most configurations, select the content types that contain useful public content rather than automatically including every registered technical content type.

Select Taxonomies

Find:

Select Taxonomies

The current interface displays taxonomy options such as:

  • Categories
  • Tags
  • Formats
  • Topic Tags
  • Job Categories
  • Job Types
  • Job Locations
  • Job Tags
  • Course Categories
  • Event Categories

The exact list can differ depending on the website.

Use Select / Deselect All when you need to change all taxonomy selections at once.

Set the Posts/Terms Limit

Find:

Posts/Terms Limit

The current screen shows a value of:

50

The interface describes this as the:

Maximum number of links to include for each post type.

Adjust this value according to how much selected content you want represented in the generated file.

Add Additional Content

Find:

Additional Content

This field lets you add extra text or links that you want included in the LLMs.txt file.

Use this when important information should be included but is not represented by the selected Post Types or Taxonomies.

Generate the Updated File

After configuring the content:

  1. Review selected Post Types.
  2. Review selected Taxonomies.
  3. Check the Posts/Terms Limit.
  4. Add Additional Content if required.
  5. Click Generate LLMs.txt.
  6. Review LLMs.txt Content.
  7. Preview the output.
  8. Click Save LLMs.txt when satisfied.

Manage Your Robots.txt File →

5. Manage Your Robots.txt File

How to Manage Your Robots.txt File

The Robots.txt tab lets you edit and manage the robots.txt instructions used for website crawlers.

Because robots.txt can affect crawler access, review changes carefully before saving them.

In this guide

  • Open Robots.txt Editor
  • Find the robots.txt URL
  • Understand the editor
  • Save robots.txt
  • Validate robots.txt
  • Reset to Default
  • Apply Enhanced
  • Delete Custom Robots.txt

Open the Robots.txt Editor

Go to:

SEO Repair Kit → Bot Manager → Robots.txt

At the top, SEO Repair Kit displays the URL where your robots.txt file is available.

For example:

https://example.com/robots.txt

Always use the URL displayed for your own website.

Understand the Robots.txt Editor

The large editor displays the current robots.txt content.

Your screenshot contains an example configuration similar to:

User-agent: *

Disallow: /wp-admin/

Allow: /wp-admin/admin-ajax.php

Sitemap: https://example.com/wp-sitemap.xml

This controls crawler instructions and provides the sitemap location shown by the current configuration.

Important: Do not copy robots.txt rules from another website without understanding what they do. Incorrect directives can prevent crawlers from accessing important website areas.

Edit Robots.txt

Make the required changes directly in the Robots.txt Editor.

Review every rule before saving.

Pay particular attention to:

  • User-agent
  • Allow
  • Disallow
  • Sitemap

Save Robots.txt

After making changes, click:

Save Robots.txt

This saves the custom robots.txt configuration.

Validate Robots.txt

Click:

Validate

to use the validation workflow provided by Bot Manager.

Use validation after editing the file and before considering the configuration complete.

Reset to Default

Click:

Reset to Default

when you intentionally want to return the robots.txt configuration to the default provided by the feature.

Important: Review any custom crawler rules before resetting them because they may need to be recreated afterward.

Apply Enhanced

The Robots.txt Editor also provides:

Apply Enhanced

Use this action when you intentionally want Bot Manager to apply its enhanced robots.txt configuration.

Because your screenshot does not show the resulting rules after pressing Apply Enhanced, the documentation should not list or promise specific rules that this button will add until that resulting screen is documented.

This keeps the guide accurate to the current UI.

Delete Custom Robots.txt

Click:

Delete Custom Robots.txt

when you intentionally want to remove the custom robots.txt configuration managed through this screen.

Important: Deleting the custom file is different from resetting its content. Use this action only when the custom robots.txt should actually be removed.

Review the Public Robots.txt File

After saving, use the robots.txt URL displayed at the top of Bot Manager to review the accessible file.

Confirm that the saved rules match the intended crawler configuration.

Bot Manager FAQs →

6. Bot Manager FAQs

Bot Manager Frequently Asked Questions

What does Bot Manager do?

Bot Manager provides tools for managing AI content discovery through LLMs.txt and crawler instructions through Robots.txt.

Where can I find Bot Manager?

Go to:

WordPress Admin → SEO Repair Kit → Bot Manager

What are the main Bot Manager tabs?

The current interface contains:

  • LLMs.txt
  • Robots.txt

Where can I find my LLMs.txt file?

Open Bot Manager → LLMs.txt.

The URL is displayed near the top of the LLMs.txt Generator.

Can I choose which AI bots are allowed?

Yes. Use AI Bot Access Control to configure the listed bots.

What happens when I uncheck an AI bot?

According to the current Bot Manager interface, unchecked listed bots are blocked using server-level checks and robots.txt rules.

What happens to bots that are not listed?

The current interface states that bots not listed in AI Bot Access Control are allowed by default.

Can I choose which Posts and Pages appear in LLMs.txt?

Yes. Use Select Post Types.

Why do I see Post Types that are not listed in this documentation?

The available Post Types depend on what is registered by WordPress, your theme, and installed plugins.

Can I include Categories and Tags?

Yes, where available. Use Select Taxonomies.

What is Posts/Terms Limit?

It controls the maximum number of links included for each selected post type according to the current interface.

Can I add custom text or links?

Yes. Use Additional Content.

What is the difference between Generate LLMs.txt and Save LLMs.txt?

Generate LLMs.txt creates/updates the generated content based on the current configuration.

Save LLMs.txt saves the reviewed LLMs.txt content.

Can I preview LLMs.txt before saving?

Yes. Use Preview.

Can I delete LLMs.txt?

Yes. Use Delete LLMs.txt when you intentionally want to remove it.

Can I edit robots.txt?

Yes. Open:

Bot Manager → Robots.txt

Can I validate robots.txt?

Yes. The Robots.txt Editor provides a Validate action.

What does Apply Enhanced do?

It applies Bot Manager’s enhanced robots.txt configuration. The exact resulting rules should be reviewed in the editor after applying it.

7. Bot Manager Troubleshooting

Bot Manager Troubleshooting

Use this guide when LLMs.txt or Robots.txt is not behaving as expected.

In this guide

  • LLMs.txt has not been generated
  • Expected content is missing
  • Unwanted content appears in LLMs.txt
  • AI bot access is incorrect
  • Robots.txt changes are not saved
  • Robots.txt validation reports an issue
  • Custom robots.txt was accidentally reset or deleted

LLMs.txt Has Not Been Generated

Go to:

SEO Repair Kit → Bot Manager → LLMs.txt

Check that:

  1. Required Post Types are selected.
  2. Required Taxonomies are selected.
  3. Posts/Terms Limit is configured.
  4. AI Bot Access settings have been reviewed.
  5. You clicked Generate LLMs.txt.

After generation, review LLMs.txt Content and save the file.

Expected Content Is Missing from LLMs.txt

Check Select Post Types and Select Taxonomies.

Make sure the content type or taxonomy containing the missing information is selected.

Also review the Posts/Terms Limit, as this controls how many links can be included for each post type according to the interface.

Generate the file again after changing the configuration.

Unwanted Content Appears in LLMs.txt

Review:

  • Selected Post Types
  • Selected Taxonomies
  • Additional Content

Remove selections or custom content that should not be included, then regenerate and review the file.

An AI Bot Has the Wrong Access Configuration

Open:

Bot Manager → LLMs.txt → AI Bot Access Control

Check whether the bot is selected.

Remember that the interface states unchecked listed bots are blocked, while bots not present in the list are allowed by default.

Save the updated configuration after making changes.

Robots.txt Changes Are Not Saved

Open:

Bot Manager → Robots.txt

Make the required edit and click:

Save Robots.txt

Then review the public robots.txt URL displayed at the top of the screen.

Robots.txt Validation Reports an Issue

Review the rules in the editor, particularly:

  • User-agent declarations
  • Allow rules
  • Disallow rules
  • Sitemap entry

Correct the configuration and run Validate again.

Avoid adding crawler directives you do not understand simply to remove a validation message.

I Accidentally Reset Robots.txt

If Reset to Default was used, review the current editor content.

Any custom configuration that is still required may need to be recreated and saved.

I Accidentally Deleted Custom Robots.txt

If Delete Custom Robots.txt was used, return to the Robots.txt Editor and review the current state before recreating a custom configuration.

Still Need Help?

If the issue continues, collect:

  • Bot Manager tab involved
  • Screenshot of the current settings
  • Expected behavior
  • Actual behavior
  • WordPress version
  • SEO Repair Kit version

For LLMs.txt issues, also include the relevant content-selection configuration.

For Robots.txt issues, include the relevant rules, but remove any information you do not want to share publicly.

Then contact SEO Repair Kit Support.