Skip to content

Fix opensaas.sh llms.txt, add sitemaps and robots.txt - #740

Open
vincanger wants to merge 1 commit into
mainfrom
vince/seo-crawler-files
Open

vincanger wants to merge 1 commit into
mainfrom
vince/seo-crawler-files

Conversation

@vincanger

Copy link
Copy Markdown
Collaborator

Description

Fixes a few crawler/agent discoverability defects on opensaas.sh and the docs.

  • opensaas.sh/llms.txt served the app's HTML. The demo app generated public/llms.txt in an npm predeploy hook, so deploys that ran wasp deploy fly deploy directly shipped without it and the SPA fallback answered instead. It's now emitted by a small Vite plugin in vite.config.ts, so it's part of every client build regardless of deploy command. The generator script now just exports generateLlmsTxt(); the generate-llms-txt/predeploy scripts and the .gitignore entry are removed.
  • No sitemap on opensaas.sh. Added public/sitemap.xml to the template (placeholder domain, / and /pricing) and to the demo app (opensaas.sh URLs), and a Sitemap: line to both robots.txt files.
  • No robots.txt on the docs. Added blog/public/robots.txt pointing to the Astro-generated sitemap-index.xml.
  • Docs: new "Robots.txt & Sitemap" section in the SEO guide, and the deploying checklist mentions both files.

I considered a sitemap generator (e.g. vite-plugin-sitemap), but Wasp routes live in main.wasp.ts, so you'd still list public routes by hand, plus carry an extra dependency. A static file is simpler for template users to edit.

Testing

  • Built the demo app client (wasp build + npx vite build, as the Railway/Fly client builds do) and confirmed build/ contains llms.txt (with blog posts), sitemap.xml, and robots.txt with the Sitemap line.
  • ./opensaas-sh/tools/diff.sh regenerated app_diff cleanly; prettier passes.

After merging, opensaas.sh needs a redeploy for the llms.txt/sitemap fix to go live.

Contributor Checklist

  • Update e2e tests: N/A (static files only)
  • Update demo app: app_diff regenerated
  • Update docs: SEO + deploying guides

- Generate opensaas.sh/llms.txt in the Vite build instead of an npm predeploy
  hook, so it's included whichever deploy command is used. It was missing in
  production and the SPA fallback served HTML instead.
- Add sitemap.xml to the template and opensaas.sh, and reference it from
  robots.txt.
- Add robots.txt to the docs, pointing at the Astro-generated sitemap.
- Document robots.txt and sitemap.xml in the SEO and deploying guides.
+ "scripts": {
+ "generate-llms-txt": "node ./scripts/generate-llms-txt.mjs",
+ "predeploy": "npm run generate-llms-txt",
+ "deploy": "REACT_APP_GOOGLE_ANALYTICS_ID=G-H3LSJCK95H wasp deploy fly deploy",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GA doesn't work on opensaas.sh right now (I think the site's code says "Google Analytics ID is missing"). Probably the same cause as the llms.txt bug: only npm run deploy adds the GA ID, and the last deploy didn't use it.

Does it make sense to fix the GA ID the same way (if it's actively being used), or always deploy opensaas.sh with npm run deploy?

I don't think it's urgent, as Plausible still works, but it might be something to clean up.

Disallow: /account
Disallow: /admin

Sitemap: https://your-saas-app.com/sitemap.xml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nit, but if a template user forgets to change this line, search engines get sent to your-saas-app.com. It's in the deploy checklist, so this is probably fine as is.

@viborc

viborc commented Oct 10, 2026

Copy link
Copy Markdown

So, generally this all looks good and covers the llms.txt, sitemap, and robots.txt fixes from my previous list.

I think we just need to redeploy opensaas.sh after the merge, and then opensaas.sh/llms.txt should show a text file, not the app. I left several smaller comments inline, nothing huge there.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants