Prompt construction and AI usage

The Think Worker owns model orchestration and reports provider usage to Autumn. Think reads AI-facing feature flags and credit admission directly from Autumn. The app Worker reads Autumn only for browser display and product-resource capacities. Autumn owns subscriptions, recurring monetary credits, model pricing, and current usage. Cloudflare AI Gateway routes inference and supplies operational telemetry; it is not a billing authority.

Prompt construction

workers/think-worker/src/chat-contract.ts owns the base system prompt and bounded product-context assembly. agent.ts owns beforeTurn() model/tool selection. Persisted user-message metadata is read directly from the native Think transcript.

The prompt contains:

1. role, safety, citation, and response principles shared across tools;
2. authority for the model to select tools from their descriptions and schemas;
3. exact typed refs for current user attachments;
4. an active-workspace marker without exposing its internal id;
5. output language.

Tool descriptions own their capability, inputs, outputs, limits, and persistent side effects. The base prompt does not prescribe a fixed tool-call sequence.

Attachment labels are JSON-escaped and explicitly untrusted. Their ids/types already live in durable user-message metadata, so the agent does not call a context-discovery tool merely to recover refs it already owns.

Product knowledge versus native Workspace

The single knowledge tool has three actions:

• search: Cloudflare AI Search over the curated public corpus, followed by canonical PostgreSQL metadata hydration;
• read: bounded, continuation-based reads of an authorized document or resource ref;
• workspace: a flattened page of document/resource refs from the current product workspace, including its direct resource memberships.

The app Domain RPC validates its typed request and resource/document readers apply live product ownership and membership predicates. Think caps each read at 16 KiB; the native model-step limit bounds the number of calls in a turn. The anonymous MCP surface reuses only public search and bounded public-resource reads, with request-size and per-IP rate limits at the HTTP boundary.

Native Workspace stores session-local files in DO SQLite. Its files persist with the chat but are not product documents, resources, or workspaces. Native read supports text, images, and PDFs; other scratch operations use virtual Bash (rooted at /) or Code Mode’s state.* connector, without duplicate tools. Publishing always crosses an explicit app action boundary.

Product allowance ownership

web-tanstack/autumn.config.ts owns the Free and Pro subscriptions, recurring AI-credit grants, reset cadence, boolean feature access, and numeric plan capacities. PostgreSQL remains authoritative for actual resource rows and enforces those capacities inside app transactions.

At turn admission, Think uses customers.getOrCreate to read the premium_models, image_generation, and web_search flags, then calls Autumn’s check endpoint for a positive ai_credits balance. Creating or reading the customer must happen first because Autumn requires an existing customer for check. Image, speech, title, compaction, and podcast paths also check immediately before dispatch. Browser customer data is display state only and never authorizes provider spend.

The native text middleware does not reserve a maximum cost before generation, so a final multi-step turn can exceed a very small remaining balance before subsequent work is denied.

Cost settlement

Chat text models are wrapped with @useautumn/gateway’s official AI SDK middleware. After doGenerate() completes, or when a stream emits its finish chunk, the middleware normalizes the provider’s input, cache, reasoning, and output token pools and sends them to Autumn’s trackTokens endpoint. Autumn owns the models.dev pricing data; application code does not maintain a duplicate rate table.

Titles and conversation compaction use the same text middleware. Image generation and speech normalize the provider’s reported tokens and call trackTokens directly with a stable idempotency key. Image output falls back to the model’s documented 1K output-token quantity when the provider omits it.

Provider failures do not produce token events. Once a provider result exists, both the official middleware and manual paths preserve it if Autumn tracking is temporarily unavailable. Manual failures emit AUTUMN_AI_USAGE; middleware failures emit Autumn’s own tracking error. There is no application retry outbox, reservation, refund, or reconciliation ledger.

Every provider request also carries trusted user_id, feature, billing_event_id, and environment metadata for operational tracing. No Gateway poller or Postgres usage ledger participates in billing. Title tracing uses the session and source-message IDs; compaction uses a fresh operation UUID. These labels correlate requests, not deduplicate charges. Autumn token events are the monetary record used by the UI balance and future admission checks.

Technical safety

• at most 10 model steps per ordinary turn;
• 8,192 generated tokens per model dispatch;
• proactive compaction at 64,000 input tokens, with one reactive overflow retry;
• 16 KiB product-knowledge read chunks and at most 10 model steps per turn;
• bounded native Workspace tool output and execution deadlines;
• native Think action-ledger replay and bounded chat recovery;
• zero AI SDK provider retries per individual model dispatch.

These limits prevent runaway execution; they are not billing settlement.

Error and observability contract

Only a server-side Autumn check produces USAGE_ALLOWANCE_EXCEEDED, either at turn admission or immediately before a media/raw call. Provider/Gateway 429 responses become AI_TEMPORARILY_LIMITED, preserve safe retry timing when available, and never imply an upgrade. Unexpected retrieval and SQL/Core errors are logged server-side and cross the tool boundary only as generic stable errors—queries and parameters are not exposed.

Gateway logging keeps operational metadata and cost but disables prompt/tool payload storage. AUTUMN_AI_USAGE covers manual tracking success and failure; the official middleware logs its own tracking failures. Think emits standard agent/model/tool traces; the product does not wrap the Think loop with a second AI SDK tracer.

See AI chat architecture for the complete request and tool flow. Operational cutover, observation, and rollback procedures live in the repository’s BILLING_AUTUMN_RUNBOOK.md.