# https://www.robotstxt.org/robotstxt.html User-agent: * Allow: / # LLM Crawler specifications # llms.txt can be found at /llms.txt # llms-full.txt can be found at /llms-full.txt