How to Edit robots.txt in Ghost (and Check It Works)

You can type the perfect robots.txt rules and still change nothing on a Ghost site—because Ghost already serves a default robots.txt, and Ghost only switches to your version after you ship it inside a theme and activate that updated theme (Ghost: Modifying robots.txt).

This guide shows the workflow Ghost supports: first confirm what your site is serving right now at https://example.com/robots.txt (example format referenced by MarsGeo), then add a robots.txt file to the root of your theme, install the new theme version, and verify the live output actually updated.

Before you touch anything: robots.txt is “a set of instructions for search engines and other web crawlers,” and it controls crawling, not indexing (Ghost; MarsGeo). If you’re trying to keep a URL out of search results, MarsGeo recommends a noindex meta tag instead of relying on robots.txt.

We’ll also run quick checks on the exact text you plan to publish using a robots.txt generator, a robots.txt checker, and an AI crawler checker so you can ship your theme update with fewer surprises.

How Does Ghost Handle Robots.txt by Default?

For ghost robots.txt, the starting point is simple: Ghost already serves a robots.txt file for your site. Ghost’s own guide on modifying robots.txt states that Ghost comes with a default robots.txt file, so you are never starting from a blank slate.

This default matters because it defines what crawlers see at the standard location (your root-level /robots.txt) unless you deliberately override it. If you have ever edited robots.txt in WordPress through a plugin, this will feel different. In Ghost, you do not edit robots.txt through an admin screen. You replace it through your theme.

Ghost’s Default Vs. Theme-Level Robots.txt

Ghost supports one official override path: put a robots.txt file inside your site’s theme. Ghost documents this directly: “To include a custom robots.txt file in Ghost, you can include one within your site’s theme” (Ghost Help Center).

Once you do that, Ghost switches which file it serves. Ghost also documents the behavior after deployment: after you install a new version of the theme and activate it, the theme’s robots.txt is used in place of Ghost’s default robots.txt (modifying robots.txt).

That “install and activate” detail is the reason you should treat a robots.txt change like a theme release. If you add a file locally but never upload and activate the updated theme, your live site keeps serving Ghost’s default robots.txt.

Ghost is also explicit about the skill level involved. Creating a custom robots.txt in Ghost requires being comfortable writing your own robots.txt and installing a new version of your theme (Ghost Help Center). In practice, that means your robots.txt workflow is a theme workflow, and the rest of this guide will follow that supported path for edit robots.txt Ghost changes.

Before You Edit Anything: Find Your Current Robots.txt

Theme changes are the only supported way to edit robots.txt Ghost sites use, so start by capturing what your site serves right now. You confirm the active ghost robots.txt by checking the standard location at your domain root, https://example.com/robots.txt (example format referenced by MarsGeo).

Open a browser in a private window (to reduce caching).
Visit https://yourdomain.com/robots.txt.
Copy the full contents into a note, and save it as “robots.txt (before)”.

This “before” snapshot matters because Ghost comes with a default robots.txt, and Ghost only swaps it when you install and activate a theme that includes its own robots.txt (Ghost: Modifying robots.txt). If you do not record the current output first, you cannot tell whether your later theme update actually took effect.

What You Are Checking When You Load /robots.txt

When you load /robots.txt, you are checking what Ghost is serving to crawlers at the root of your domain. That is the file search engines request, regardless of where you store it locally during theme development. Ghost’s documentation is explicit that the custom file lives in the theme and replaces the default after you install and activate the updated theme (Ghost).

If /robots.txt does not load, stop and resolve that first. The rest of the workflow assumes you can view the current robots.txt at the root URL, then compare it again after you activate a theme that contains a custom robots.txt.

How to Edit Robots.txt in Ghost (The Only Supported Method)

Once you can load /robots.txt, the edit robots.txt Ghost workflow is simple and strict. Ghost does not provide an admin setting for this. Ghost’s supported method is to ship a theme update that contains your own robots.txt, then install and activate that theme so Ghost serves it instead of the default (Ghost: Modifying robots.txt).

Create a file named robots.txt. Use plain text. Write the crawler directives you want Ghost to serve.
Save robots.txt to the root of your theme. Ghost explicitly instructs you to put the file “to the root of the theme” (Ghost Help Center).
Package and install a new version of the theme. Ghost states that creating a custom robots.txt requires being comfortable “installing a new version of your theme” (Ghost: Modifying robots.txt). Treat this as a theme release, even if the only change is this one file.
Activate the newly installed theme. This activation step is what flips your live ghost robots.txt output. Ghost documents that after the new theme version is installed and activated, the theme’s robots.txt is used in place of Ghost’s default (Ghost: Modifying robots.txt).
Verify the live file at /robots.txt. Reload the root URL and confirm the content matches your theme’s file, line for line.

What To Do If Your Change Does Not Show Up

If /robots.txt still shows the old content, focus on the two steps Ghost calls out: you must install the updated theme version, and you must activate it. Ghost only switches from the default robots.txt after that activation (Ghost Help Center).

Which Ghost Plan Do You Need to Install a Custom Theme?

If you want to edit robots.txt Ghost sites serve, you need the ability to install and activate an updated theme. That is the gate: Ghost only uses a custom ghost robots.txt after you ship it inside a theme and activate that theme version.

Ghost documents the plan requirement clearly: on Ghost(Pro), installing a custom theme requires a subscription to the Creator plan or higher (Ghost: Modifying robots.txt). If you are on a lower Ghost(Pro) plan, you cannot use Ghost’s supported method for robots.txt changes, because that method depends on custom theme installation.

What This Means for Robots.txt Changes

Ghost does not provide an admin setting to edit robots.txt directly. Ghost’s supported workflow is theme-based: you add a robots.txt file to the theme root, install the updated theme, then activate it so Ghost serves the theme’s file instead of the default (Ghost documentation).

So, when you plan a robots.txt update, treat it like any other theme change:

Confirm you can install custom themes on your Ghost(Pro) plan (Creator or higher).
Prepare to upload a new theme version that includes robots.txt at the theme root.
Plan a quick verification step after activation by reloading /robots.txt on your live domain.

If you are working on a self-hosted Ghost install, this specific Ghost(Pro) plan requirement does not apply. Ghost’s documentation callout here is explicitly about Ghost(Pro) subscriptions (Modifying robots.txt).

Robots.txt Rules You Should Double-Check for Ghost Sites

When you edit robots.txt Ghost sites serve, you are writing crawler instructions by hand. Ghost defines robots.txt as “a set of instructions for search engines and other web crawlers,” which tells crawlers what they can look at and what they should ignore (Ghost: Modifying robots.txt). That makes small syntax mistakes expensive, because crawlers read the live file at /robots.txt literally.

Use this checklist to review your ghost robots.txt before you package the theme update.

You included at least one User-agent group. MarsGeo’s example starts with User-agent: *, which targets all crawlers (Setting up robots.txt for your ghost powered blog).
Your rules match the paths you actually want crawled. Robots.txt is about what bots can access, so sanity-check every Disallow line against real URLs on your site (Ghost).
You explicitly block the Ghost admin path if that is your intent. MarsGeo calls out a common directive: disallow /ghost/ (the admin area) (MarsGeo’s Ghost robots.txt generator).
You reference a sitemap URL if you want crawlers to discover it via robots.txt. MarsGeo suggests referencing /sitemap.xml (MarsGeo).
If you add a sitemap line, you use a full URL. The Terrible.dev guide states you must specify the full URL to your sitemap file, and shows an example sitemap line (Terrible.dev).

Example Snippet to Sanity-Check Your Ghost Robots.txt

The Terrible.dev post includes a minimal example that is useful as a quick comparison when your file starts getting complicated:

User-agent: *
Sitemap: http://yourdomain/rss

Treat that snippet as a format reference, not a recommendation for your exact setup. What matters is that your final ghost robots.txt uses valid groups and points crawlers at the sitemap URL you intend to publish.

Unexpected Gotcha: Robots.txt Can’t Guarantee Deindexing

A valid ghost robots.txt can be perfectly formatted and still fail at the one thing many people expect: removing URLs from Google. MarsGeo’s Ghost robots.txt generator states this directly: robots.txt controls crawling, not indexing. So when you edit robots.txt Ghost sites serve, treat it as access control for bots, not a delete button for search results.

Search engines can learn a URL from links on other sites, your own internal links, or historical crawl data. If your robots.txt blocks the crawler from fetching the page, the crawler may still keep the URL in its index because it knows the URL exists. It just cannot re-crawl the content to update what it knows.

What to Use When You Need a URL Out of Search Results

If your goal is deindexing, use a noindex meta tag instead of relying on robots.txt. MarsGeo explicitly recommends this approach when you want to keep a page out of results (MarsGeo).

This distinction matters when you ship a new theme to change robots.txt in Ghost. A robots.txt change can stop future crawling of an area (for example, an admin path), but it does not guarantee that previously discovered URLs disappear from search results.

Before you publish your updated theme, sanity-check each rule by asking one question: “Am I trying to stop crawling, or am I trying to stop indexing?” Use robots.txt for the first case. Use noindex for the second case.

Free Tools to Validate Your Ghost Robots.txt Before You Ship It

A robots.txt change can fail in two ways: you ship rules that block the wrong paths, or you ship the right rules but your live ghost robots.txt output never updates. Before you install and activate a theme update, run a quick validation pass on the exact text you plan to publish at https://example.com/robots.txt.

Generate a clean draft (optional). If you are starting from scratch, use Balzac’s Robots.txt Generator to produce a baseline file you can paste into your theme’s robots.txt. Treat the output as a starting point, then edit it to match your site.
Paste your final file and check for obvious mistakes. Run the exact contents through Balzac’s Robots.txt Checker. You are looking for problems that usually come from manual edits, like missing User-agent lines, malformed directives, or rules that do not parse the way you think they do.
Sanity-check AI crawler access. If your goal includes AI search visibility, confirm that your rules do not accidentally block common AI user agents with Balzac’s AI Crawler Checker. This check does not replace your SEO review; it catches accidental lockouts.

What To Verify Before You Activate the Theme

Ghost defines robots.txt as “a set of instructions for search engines and other web crawlers” (Ghost: Modifying robots.txt). Crawlers read it literally, so validate the parts that most often cause self-inflicted crawl blocks:

Your intent matches the mechanism. Robots.txt controls crawling, not indexing. If you are trying to remove a URL from results, you need a noindex meta tag, not a Disallow rule (MarsGeo).
Your paths match real URLs. If you disallow /ghost/, you are blocking the Ghost admin path (a common directive called out by MarsGeo).
Sitemap lines use a full URL. A Ghost-focused guide notes you must specify the full URL to your sitemap file (Terrible.dev).

After you activate the updated theme, re-open /robots.txt and compare it to the text you validated. Ghost only serves your custom file after you install and activate the theme version that includes it (Ghost).

FAQ: Ghost Robots.txt Editing Questions

If /robots.txt changed after you activated your theme update, you already proved the core workflow works. The remaining questions usually come down to where Ghost expects the file, what “activation” changes, and what to do if you do not want to touch theme files while you edit robots.txt Ghost sites serve.

Where does a custom robots.txt live in Ghost? Ghost supports a theme-based override: you include a robots.txt file within your site’s theme, saved at the root of the theme (Ghost: Modifying robots.txt). A Ghost theme directory example from Terrible.dev looks like ~/blogFolder/content/theme/Casper, and the idea is to place robots.txt in that theme folder root (Terrible.dev).

What URL do crawlers actually request? Crawlers fetch robots.txt from the standard root location on your domain, for example https://example.com/robots.txt (MarsGeo). Even though you store the file in your theme, you always verify it at /robots.txt on the live site.

What happens after you install and activate the updated theme? Ghost swaps what it serves. After you install a new version of the theme and activate it, Ghost uses the theme’s robots.txt in place of Ghost’s default robots.txt (Ghost).

Can I edit robots.txt in the Ghost admin? Ghost’s documentation describes the supported method as shipping robots.txt inside the theme, then installing and activating that theme version (Ghost). If you want a UI toggle, that is a WordPress-style pattern, not the Ghost approach.

What If I’m Not Comfortable Editing Theme Files?

Ghost is direct about the requirement: creating a custom ghost robots.txt requires being comfortable writing your own robots.txt and installing a new version of your theme (Ghost). If that is not you, use one of these paths:

Ask your theme developer to add robots.txt at the theme root and deliver a new theme version you can install and activate.
Keep Ghost’s default robots.txt and avoid changes until you have someone who can safely handle a theme update.

Next Steps: Keep Crawlers Happy After Your Update

If you took the time to edit robots.txt Ghost serves, treat the rollout like a theme release, because that is what Ghost requires. Ghost only switches from its default file after you install and activate a theme that includes a robots.txt at the theme root (Ghost: Modifying robots.txt).

Run this post-update checklist while you still have your theme files open and can revert quickly if needed.

Reload your live /robots.txt and compare it to your theme file. Open https://example.com/robots.txt (same root location format referenced by MarsGeo) and confirm it matches your intended text line for line.
Confirm you activated the updated theme version. If the output did not change, focus on the two actions Ghost documents: install the new theme version and activate it so the theme’s robots.txt replaces Ghost’s default (Ghost).
Validate the exact text you are serving. Paste the live file into Balzac’s Robots.txt Checker to catch parsing issues that can come from a small typo.
Recheck any rules that block sensitive paths. If you disallowed /ghost/ or added a sitemap line, verify those directives still reflect what you want crawled and discovered (examples and tips referenced by MarsGeo).
Make sure your intent matches robots.txt behavior. Robots.txt controls crawling, not indexing. If you expected URLs to disappear from search results, you need a noindex meta tag instead of relying on robots.txt (MarsGeo).

If you want one safe habit to keep: keep a dated copy of every ghost robots.txt change alongside your theme release notes. When something breaks later, you can diff the exact file crawlers saw.

New Balzac signups get 3 free articles with no card.