How to set up your sitemap, robots.txt and llms.txt

TeleportHQ writes three files for search engines and AI tools each time you publish: sitemap.xml (a list of your pages), robots.txt (rules for search engine crawlers) and llms.txt (a summary of your site for AI assistants). You don't need to upload anything. To see or change them, click the Project settings cog at the bottom of the left sidebar, then SEO & social → Search engines. The settings show the files your next publish writes, save on their own, and link to the files on your live site.

Before you start#

• The sitemap and robots.txt are available on every TeleportHQ plan.
• LLMS.txt needs a Trial, Pro or Premium workspace. On an expired workspace it shows a PRO label and no llms.txt file is published. See What's the difference between Trial, Pro and Premium?.
• The files exist on your published site, at yoursite.com/sitemap.xml, yoursite.com/robots.txt and yoursite.com/llms.txt. Changes reach them when you publish. See How to publish your site.

Open the settings#

1. In the TeleportHQ editor, at the bottom of the left sidebar, click the Project settings cog.
2. In the Project settings panel, under Site, click SEO & social.
3. Click Search engines. Its settings open next to the list.

The settings save on their own a moment after each change. There is no Save button.

Sitemap#

The TeleportHQ sitemap lists every page search engines should find, so they discover new pages faster.

1. In Search engines, check that Automatically Generate Sitemap is switched on. It's on by default.
2. Below the switch, read the sitemap your next publish writes. While TeleportHQ builds it, the line "Building what your next publish writes…" shows. Click Refresh to build it again after you change your pages.
3. Once your site is published, click Open sitemap.xml on your site to see the file your visitors' search engines read now.

The sitemap lists your pages, and one address per item on details pages, for example one per product. It leaves out pages with No Index turned on, pages with "404" or "not-found" in their name, and admin panel pages. On a multilingual site, it lists each language version.

Sitemaps on a site with a Blog or Help Center#

When your site has a Blog or a Help Center that anyone can read, TeleportHQ splits the sitemap in parts:

• sitemap.xml becomes an index (a list of sitemaps) that names the others.
• sitemap-pages.xml lists your pages, as above. It changes when you publish.
• sitemap-blog.xml and sitemap-help.xml list your Blog posts and Help Center articles. Your site writes them each time a search engine asks, so a new article is listed without publishing again.

In Search engines, the settings then show sitemap.xml and sitemap-pages.xml, and a line such as "sitemap-blog.xml: your site lists its 12 Blog posts there each time a search engine asks."

A Blog or Help Center gets its own sitemap when its article page is open to everyone, isn't set to No Index and doesn't redirect elsewhere. Drafts, and articles kept out of search results or sent elsewhere, aren't listed.

Robots.txt#

The TeleportHQ robots.txt tells search engines which parts of your site they may visit. The default allows everything and points to your sitemap:

User-agent: *
Allow: /

Sitemap: {{domain_url}}/sitemap.xml

When you publish, {{domain_url}} becomes your site's address, for example https://sunny-bakery.teleporthq.site.

To change it:

1. In Search engines, click inside the Robots.txt editor. For more room, click the button at the editor's top right (tooltip Open robots.txt in a larger editor). Your changes there are kept when you close it.
2. Edit the rules. For example, to keep crawlers out of a /drafts folder, add Disallow: /drafts/ under User-agent: *.
3. Publish your site.

LLMS.txt#

The TeleportHQ llms.txt file helps AI assistants like ChatGPT and Claude understand what your site offers. Next to LLMS.txt, pick one of three modes:

• Auto (default): TeleportHQ writes the file from your pages and updates it on every publish. The settings show the file your next publish writes. Once your site is published, click Open llms.txt on your site to read the live one.
• Edit: you write the file yourself. If you haven't written any text yet, picking Edit fills the box with what your next publish writes. Generate puts that text back at any time. Then change the text. The button at the box's top right (tooltip Open llms.txt in a larger editor) gives you more room.
• Off: no llms.txt file is published.

Then publish your site.

llms.txt and llms-full.txt on a site with a Blog or Help Center#

When your site has a Blog or a Help Center that anyone can read, TeleportHQ's llms files list your articles live:

• /llms.txt starts with your pages, then lists your Blog posts and Help Center articles by category, each with its summary. Your site adds new articles each time the file is requested. The settings say so: "Then your site adds the 12 Help Center articles, by topic, each time the file is requested."
• /llms-full.txt holds every listed article in full, in Markdown (plain text with simple formatting), in the same order. AI tools can read your whole Help Center or Blog from this one file.

In Edit mode, your text replaces the whole llms.txt file, so your site stops adding new articles to it. The settings warn you: "Your text replaces the whole file, so your site stops adding new …". /llms-full.txt keeps working. In Off mode, neither file is published.

Check that it worked#

After publishing, click Open sitemap.xml on your site or Open llms.txt on your site in Search engines. Or open your live site and add /sitemap.xml, /robots.txt or /llms.txt to the address, for example https://sunny-bakery.teleporthq.site/robots.txt. You see the file's content. To help Google find your pages sooner, submit the sitemap address in Google Search Console. See How to connect Google Search Console and see your keywords.

Troubleshooting#

"The preview couldn't be built." TeleportHQ couldn't build the files your next publish writes. Click Try again. Publishing still writes the files.

My changes don't show in the file The files are rebuilt when you publish. Publish your site, then open the file again. Blog posts and Help Center articles are the exception: they're added to the sitemap and llms files without publishing.

A new article isn't in the sitemap or llms.txt yet Your site refreshes these lists every few minutes, not on every request. Wait about 15 minutes, then reload the file. If it still shows the old list, reload it once more. Also check that the article is published and isn't set to stay out of search results.

The file shows old content I uploaded before If your project has an uploaded file named robots.txt, sitemap.xml, llms.txt or llms-full.txt, TeleportHQ publishes that file and doesn't write its own. Delete the uploaded file, then publish. See How to rename or delete files (and remove unused ones).

A page is missing from the sitemap Check whether No Index is on for that page in Page SEO Settings. Pages with "404" or "not-found" in their name are also left out.

LLMS.txt shows a PRO label llms.txt needs a Trial, Pro or Premium workspace. See What's the difference between Trial, Pro and Premium?.

Good to know#

• Turning off the sitemap stops TeleportHQ from publishing sitemap.xml. The default robots.txt still points to it, so remove or change the Sitemap: line too.
• The robots.txt editor can't be empty. If you delete everything, the default rules come back.
• An older default. Projects whose robots.txt still holds the earlier default, with a sample Googlebot rule, now get the default above. A robots.txt you wrote yourself stays as you wrote it.
• Staging addresses are always kept out of search engines, whatever these files say. See How to test changes on a staging site before going live.
• Hiding the whole site. The Show this site in search engines switch, above the sitemap settings, asks search engines not to list any address of your site. It doesn't change these files. See How to hide a page from Google, set a canonical URL or redirect a page.
• Site Health audits check these files. A missing robots.txt or sitemap.xml shows up as an issue. See How to check your site's SEO health.