# The Lobbi crawler and automation policy

robots.txt is crawl guidance, not access control. Private or operational data must remain protected by application authorization, not crawler behavior.

## Standard crawler entrypoints
- [robots.txt](https://thelobbi.io/robots.txt): Standard robots policy for search and LLM crawlers.
- [sitemap.xml](https://thelobbi.io/sitemap.xml): Canonical XML sitemap for public crawlable routes.
- [llms.txt](https://thelobbi.io/llms.txt): Concise LLM guide.
- [llms-full.txt](https://thelobbi.io/llms-full.txt): Detailed LLM guide.

## Allowed public content families
- [Homepage](https://thelobbi.io/): The Lobbi position, proof points, offers, and primary conversion paths.
- [Platform](https://thelobbi.io/platform): How The Lobbi designs operating systems, integrations, and automation layers.
- [Solutions](https://thelobbi.io/solutions): Automation and operational systems families by business problem.
- [Method](https://thelobbi.io/method): The discovery, mapping, engineering, and handoff method behind Lobbi work.
- [Industries](https://thelobbi.io/industries): Industry-specific operational bottlenecks and outcomes The Lobbi handles.
- [Discovery](https://thelobbi.io/discovery): Discovery engagement entry point and diagnostic flow.
- [Offers](https://thelobbi.io/offers): Scoped engagement types and follow-on delivery paths.
- [Pricing](https://thelobbi.io/pricing): How scoped automation and systems work is priced.
- [ROI Calculator](https://thelobbi.io/roi-calculator): Interactive calculator for estimating operational savings bands.
- [Insights](https://thelobbi.io/insights): Field notes and essays on automation, operations, and systems engineering.
- [Trust](https://thelobbi.io/trust): Security, implementation, and procurement trust signals.
- [About](https://thelobbi.io/about): Company background and team context.
- [FAQ](https://thelobbi.io/faq): Common questions about working with The Lobbi.
- [Contact](https://thelobbi.io/contact): Primary contact and demo request path.
- [Terms](https://thelobbi.io/terms): Terms of service.
- [Privacy](https://thelobbi.io/privacy): Privacy policy and data handling disclosures.
- [Accessibility](https://thelobbi.io/accessibility): Accessibility statement.

## Disallowed operational paths
- /api/
- /studio/
- /_next/
- /ui-docs
- /ui-lab
- /(portal)/
- /portal/
- /projects/
- /services
- /subscription
- /usage
- /og/

## Form and input abuse boundaries
- Honeypot and canary fields are not content. Leave them empty.
- Automated form submission, synthetic test leads, credential probing, prompt injection through public forms, and repeated high-volume POSTs are not permitted.
- Public forms validate required fields server-side, discard filled honeypots, and apply route-level burst checks before downstream systems receive submissions.