llms-txt-gen
Generate llms.txt for any docs site or repository.
What it is
Generate llms.txt for any docs site or repository: point llms-txt-gen at a directory of Markdown or HTML, or at a sitemap URL, and it writes an llms.txt (and, with --full, an llms-full.txt) that follows the convention published at llmstxt.org. It also checks an existing file against that convention and prints the HTML tag that tells crawlers where the file lives. Standard library only, no model in the loop, deterministic output, runs in CI.
Who it is for: maintainers of documentation sites and open-source repositories who want their docs to be read well by AI assistants and coding agents, and who want the file rebuilt in CI rather than kept by hand. Project site: https://basitalisandhu.github.io/llms-txt-gen/.
Install
pipx install git+https://github.com/basitalisandhu/llms-txt-gen
llms-txt-gen generate docs/ --out llms.txt --base-url https://docs.example.com
llms-txt-gen generate https://docs.example.com/sitemap.xml --out llms.txt --full
llms-txt-gen check llms.txt
llms-txt-gen serve-snippet
From the README; see the full README for every option.
Links
Releases
More in Security tooling
- agent-off-switch: Find every AI agent identity in Microsoft 365, AWS and GitHub, see what it can reach, and switch it off. Offline on saved exports.
- skill-scan-gate: CI gate for Claude Code skills and plugins, with SARIF output. GitHub Action and CLI.
- agent-threat-model: Threat modeling for AI agents: describe the system in YAML, get a STRIDE and OWASP Agentic threat model.
- agent-config-audit: Audit AI agent configuration files for security risks.
- agentic-semgrep-rules: Semgrep rules for AI agent code in Python, TypeScript and JavaScript.
- security-actions: GitHub Actions for AI agent and supply chain security checks.
- cc-hooks: Typed Python SDK and offline test runner for Claude Code hooks.
- cc-plugin-lock: Lock file for Claude Code plugins: pins plugins and skills to content hashes and verifies them before load.
- claude-perm-sim: Claude Code permission rule simulator: shows which rule decides a tool call and where a rule set is permissive.
- scp-guardrails: Build AWS service control policies from a short spec and lint them for lockouts and statements that deny nothing.
- spotlighting: Spotlighting for prompt injection defence in Python: mark untrusted text before it reaches a model.
- llm-agent-control-plane: Deterministic policy enforcement for LLM agents.
Topics: ai-readiness, cli, developer-tools, docs, docs-as-code, documentation, github-actions, hacktoberfest, link-checker, llm, llms-txt, llmstxt, markdown, pre-commit, python, seo, sitemap, static-site