llms-txt-generator

# LLMs.txt Generator Upgrader

## Goal

Upgrade `llmtxt-generator.py` using current app structure and existing
`apps/web/public/llms.txt`, then regenerate outputs. Only add new or missing
sections and links. Do not remove, rename, or rewrite existing sections unless
the user explicitly asks.

## Discovery: apps structure

1. Scan `apps/` to understand doc surfaces and routing structure.
2. Note any new top-level doc categories or developer resources that are not
   represented in the current `llms.txt`.

## Baseline: current llms.txt

1. Read `apps/web/public/llms.txt`.
2. Extract section headings (`##`) and link lists.
3. Treat this as the canonical baseline: preserve all existing sections and
   links, and avoid duplicates.

## Update the generator (llmtxt-generator.py)

Always edit the generator, not the output files.

- Add only missing sections or links to `CURATED_SECTIONS` or `ENGLISH_EXTRAS`
  so the generator reproduces the current `llms.txt` plus any new additions.
- Keep descriptions short, factual, and consistent with the current style.
- Keep URL patterns canonical (no `utm_` parameters).
- Keep `LOCALES` aligned with `packages/i18n/src/config.ts` and update
  `LANGUAGE_NAMES` for any newly added locale.
- If new routing patterns appear, update `normalize_llms_urls()` accordingly.

## Regeneration

From the repo root:

```
python llmtxt-generator.py
```

Outputs:

- `apps/web/public/llms.txt`
- `apps/web/public/llms-en.txt`

## Output constraints

- Only add new or missing sections/links; never delete existing ones.
- Preserve ordering unless a new section clearly belongs next to related ones.
- Do not fabricate `llms-full.txt`. Follow repo guidance or user instructions.
- For non-English locales, note the generator preserves existing content if the
  file exists; change this behavior only if explicitly requested.

## Verification

- `apps/web/public/llms.txt` contains all prior sections and any new additions.
- Locale files exist for all configured locales.
- Output URLs are canonical and contain no `utm_` parameters.

## Example requests

- "Upgrade llmtxt-generator.py after a new docs section"
- "Add missing llms.txt sections based on apps structure"
- "Add a new locale and regenerate llms files"