Chat Runtime

Chat is the primary user surface for public and private AI work. The client does not choose private model or agent ids directly. Public chats may send an explicit provider/model selection only from the policy-resolved runtime catalog, and the API validates that selection before execution.

Implemented

Threads persist app-owned product fields in Prisma and canonical transcript content in Mastra memory.
Private chat can use connector-backed indexed retrieval, live query, tabular reasoning, HITL information requests, selected published skills, and response templates.
Selected skills and response templates shape the response, but they are not policy authorities: they cannot enable tools, expand source scope, bypass HITL, switch providers, or send private data to public capabilities.
Public chat is server-gated by the customer CMS Chat Behavior Enable Public Chat switch, CMS approved-model policy, API runtime provider wiring, and a valid public-provider credential slot. It exposes explicit provider/model selection from the allowed public catalog and must not imply access to private data.
Authenticated users can save a durable default public provider/model preference in Settings or from explicit public-composer actions. New public drafts use that preference only when it is still available in the runtime catalog; ordinary composer model changes stay local to the draft/thread.
The public model picker is a constrained bottom-drawer command palette. It shows the saved default under the Public model drawer title, keeps provider context below search, supports provider drill-down when multiple providers exist, and lets a row’s Make default action save without selecting the model or closing/reordering the picker.
The default mode for new chats is an API-backed user setting mirrored into the chat runtime Pinia store; public mode still falls back to private when runtime policy makes public unavailable.
Private role model chains are API runtime config, not customer CMS data. Customer operators cannot choose private role models, edit private model presets, or toggle private chat tools in Chat Behavior.
Chat mode is locked per thread so users cannot silently switch a thread between public and private policy.
Queue and interrupt behavior are durable enough for “send now” and queued-turn handoff scenarios.
Owned chats support owner-only starring through ChatThread.starredAt; starring is metadata-only and does not change recent-chat ordering.
Feathers realtime updates drive chat sidebar, queued turn, turn completion/interruption, and HITL state through validated contract events and Pinia store actions.
Public owner-visible reasoning summaries and private owner-visible work notes are displayed as owner-only UI artifacts in the owner activity rail and chronological Activity timeline, then stripped before recalled history is sent back to a model. Inline public reasoning uses muted note rows inside the reasoning disclosure so provider summary text stays visually separate from assistant answer prose.
Empty AI SDK stream output must not become a completed blank assistant message. Private chat may use the bounded source-coverage fallback only when authorized sources exist and the fallback text truthfully describes what was checked; otherwise the turn fails visibly and is not persisted as blank assistant content.
Chat message rendering keeps completed Markdown on the canonical MDC-based renderer for tables, lists, copyable code blocks, inline citations, privacy-safe section grounding labels, and structured actions. App-owned paragraph and code-block prose wrappers load through MDC nested async components while the streaming parser and sanitizer contracts stay unchanged. Active open code fences use shiki-stream with the same Shiki themes so long code answers highlight incrementally while the assistant response is still streaming. During active streaming, an incomplete Markdown link may temporarily show its completed label as plain text until the URL closes, then the completed Markdown re-renders as a normal link. Streamed source parts may appear in a compact source disclosure near the answer while the response is still arriving; that disclosure is UI chrome and must not rewrite or contaminate assistant text.
Full-message copy offers Copy and Copy markdown. Copy writes rendered HTML plus rendered plain text so rich paste targets keep headings, lists, tables, code, and emphasis, while Copy markdown keeps the raw Markdown path for developer reuse.
Live query is domain-oriented: mail-live-query, contact-live-query, calendar-live-query, workspace-chat-live-query, work-item-live-query, and code-repository-live-query route to provider adapters.
Private orchestration has a concise canonical requirements checklist in docs/features/chat/spec.md#chat-orchestration-musts; see Chat Orchestration for the support-dev orientation.
Assistant response sources are explained separately in Assistant Response Sources: public web sources, private connector evidence, checked/used/listed inventory, inline citation rules, and open/download action rules.
Deterministic and persisted Mastra eval lanes cover workflow behavior, HITL lifecycle, queue interrupts, runtime smoke, answer quality, and private orchestration patterns.

Focus Mode

Focus mode is a private-chat scope lock. The web app lets the owner choose the scope before the first message, then the API persists the canonical focus contract and enforces it for every later turn. Empty persisted private threads may still update focus before their first message or active turn; once a chat has started, owner UI should present focus as locked/read-only.

Implemented focus scopes:

all: no explicit focus; private orchestration can use any eligible user-owned private source under policy.
connectors: one or more selected connector records plus the compatible connector metadata available when the chat starts.
connector-account: one Better Auth connector-linked OAuth account (providerId + accountId) plus the compatible connector ids available when the chat starts.
resources and library: selected file/folder resource refs resolved through the library/content contracts. Indexed/vector retrieval stays first when indexed evidence exists; selected live file refs and direct folder children can use provider-backed resource listing without broadening outside the locked refs.

For selected live files and folders, focus execution does not require canonicalContentId. Indexed retrieval remains the first path when embeddings/indexed content exist. If that path returns no focused evidence and the structured turn intent asks for file or folder content, the API can read text-like selected live files directly through the connector file-access adapter. Selected folders are intentionally direct-child scope only; the grounding message tells synthesis not to infer recursive subfolder coverage.

Empty Library focus is allowed. Starting or sending a focused Library chat must not fatal just because the selected Library has no items; the assistant should say the focused Library is empty and must not broaden to all private sources.

Source-limited focus is strict. When a prompt says to use only focused files/folders, a focused Library, selected sources, or searchable files, the assistant should not use general knowledge to fact-check or correct source claims. It should identify what the focused source says and say when that focused content does not provide validation, risks, next steps, or corrections. The locked focus itself is also enough context for terse prompts such as What should I know?; users do not have to name files, folders, resources, Libraries, connectors, or focus for focused retrieval to stay inside the selected scope. Indexed evidence can rank or augment focused file/Library answers, but broad focused prompts must still include eligible selected live refs instead of stopping at the first indexed hit. If selected live refs cannot be read, the answer should disclose that coverage gap. Keep the streaming-first behavior: start accurate text as soon as enough focused evidence is available, but do not finish a broad focused-source summary as if unread selected refs were outside scope.

The focus picker is a right-side slideover with exclusive focus-type selection before item selection. File/folder selection uses the same shared live resource picker as Library add: Browse drills through file-resource-capable connectors, Search performs connector-wide flat search only after a connector and query are supplied, folder rows can show direct child file/folder counts, and the selected list shows connector provenance plus the best available breadcrumb path. Known-empty folders remain selectable for focus but do not show drilldown controls. The UI must not allow mixed focus types in one applied focus; Library focus remains a single-library choice.

Persisted-chat Sidekick focus review is read-only once focus is locked. It separates focused file and folder counts, groups rows by connector and linked connector-account provenance when available, wraps long paths instead of introducing horizontal scroll, and makes route-backed focus rows clickable without extra secondary link copy. Focused Libraries open the Library detail page from the row/title itself; focused connector rows open the connector dashboard filter. Focused folder rows can preview direct children through the same provider-agnostic live resource-list contract with pagination; this preview is direct-child only and does not change the locked focus scope.

Current tests cover large selections of live file resource refs without requiring canonicalContentId, and current product code does not define a hard selected-file cap. That is contract coverage, not a promise that unlimited selections are free: a large focused selection still narrows the eligible source universe compared with all-scope private chat, but practical limits are still payload size, UI scanability, provider read/list latency, orchestration cost, and answer-quality coverage. Do not add an arbitrary hard cap without load data and eval evidence that the cap improves reliability; prefer measured limits, paging, warnings, or batching once those constraints are observable.

Local connector prompt packs add a browser-backed mixed file+folder focused-chat smoke for file-resource connectors. The generated pack stays under ignored tmp/connector-chat-prompts/, and the runner builds the focus from live resource refs at run time rather than storing private file or folder ids in tracked code. Basic prompt-pack scoring fails answers that expose internal no-evidence wording or hidden system files such as .DS_Store; those checks are diagnostic guards, not production routing decisions.

Connector-backed focus is intentionally snapshot-based. New connectors added after the chat starts do not silently join an existing chat, and removed or disconnected snapshot connectors are not used for later focused retrieval. For account focus, the snapshot is narrowed to connectors under the selected Better Auth linked account; if a removed snapshot connector is reconnected and receives a new connector id, it can restore only when that same selected account has exactly one replacement connector with the original connector key.

Focus availability is the owner-only status that explains whether locked focus sources can still be used. It is populated for connector-backed focus (connectors and connector-account) by comparing the locked connector snapshot with current connector lifecycle/readiness. The stored snapshot includes the connector ids used for enforcement plus owner-visible connector display metadata captured at chat start, so removed connectors can still be described in owner UI without widening retrieval or relying on the deleted connector row. The API also stores owner-only first-observed availability timing per focused source; this keeps a missing-connector marker anchored to the same point in the message flow across refreshes when the exact provider removal time is no longer knowable. The thread can show subtle inline markers in the normal message flow where a focused source became unavailable or was restored, including a formatted date/time when available, and the marker names the affected connector when the snapshot has a display label. The sticky top area shows a generic focus-availability indicator with tooltip details while any focused source is unavailable. Sidekick keeps availability on the existing focused connector source row: the connector logo remains visible, a small warning/error icon is overlaid on the avatar, and row text says what changed, such as Disconnected during this chat. These availability events are not transcript messages, are not sent back to the model, and are stripped from shared chats.

When the owner asks what focused sources are available, the API treats that as source inventory rather than file/folder browsing. It answers from the locked focus snapshot plus current availability, so account- and connector-focused chats can name the selected connector display label, say when it is disconnected or unavailable, and avoid account/provider/connector ids or wrong-account data.

Shared chat snapshots expose only coarse focus hints such as whether focus mode was used and a coarse kind/count. They do not expose account ids, provider ids, account labels, connector ids, connector names, source links, or focus availability events.

Owner recent-chat/sidebar summaries and owned All Chats rows can show a compact focus summary for scanning. That summary uses the same focus-kind icon language as the picker, a short type/count label, and at most a capped connector-source logo preview for connector-backed scopes. The owned All Chats list also has a focused-only filter backed by GET /api/chats?focus=focused, so pagination and counts remain server-owned. Because focus is private-only, the All Chats Public tab clears/ignores focused-only route state and disables the focus toggle instead of sending focus=focused. These details are owner-only navigation metadata; shared-chat list and snapshot contracts stay coarse and never receive exact connector names, account labels, account ids, provider ids, connector ids, source links, or availability details.

Starred Chats

Starred chats are owner metadata on saved chat threads. The API exposes a dedicated star/unstar route that updates only starredAt, leaving lastActivityAt unchanged so recent-chat ordering remains message-activity based.

The web app exposes starring from the thread composer, a sticky-title owner action menu, sidebar row menus, and All Chats row menus. The owner action menu is built from one shared path so rename, move, archive, delete, star/unstar, Automation rerun, and share behavior cannot drift by surface. Starred owned chats show a filled gold star in owned navigation rows, appear in a collapsible Starred sidebar section above Recent chats, can be filtered from All Chats with starred=starred, and are listed as loaded child commands under the global Starred chats command-palette item. Shared-with-me routes and shared chat snapshots do not expose owner starred state or starred controls.

Owned chat list/sidebar contracts expose only threads with a canonical active branch. If a local dev hot reload or process restart interrupts the short pre-release create path before root branch setup finishes, the API cleans up the incomplete shell when possible and excludes branchless shells from list pagination so the sidebar does not advertise a chat that detail cannot load.

Current Limits

Some research-workspace launch behavior remains planned and belongs under Research Workspace.
Broader real-world answer-quality judge calibration is still follow-up work.
Broad Slack huddle prompts currently fail closed when bounded live history/search cannot prove coverage. Exact same-day huddle discovery across tenant-specific Slack event payload shapes remains a live-query follow-up, not a user-visible “no huddle” assertion.
Non-Bedrock private orchestration provider support is not accepted in API runtime config yet.
Public model auto-selection is not implemented yet; public sends use explicit allowed provider/model choices that the API revalidates.
Public chat has no API env feature flag. Disable it from cms-customer Chat Behavior, disable every approved public provider/model in the public model catalog, or leave provider credentials unconfigured/invalid; those paths make runtime config return private-only and make API public create/send fail closed. If a stale browser tab submits a public-chat create request after policy changes, the UI should show that public mode is unavailable.

Manual QA Smoke

For a quick private-chat orchestration check, use one unfocused private chat and one focused private chat with an indexed tabular file.

Send a simple greeting in the unfocused chat and confirm private retrieval is not forced.
In the focused file chat, ask one irrelevant question, then a relevant follow-up, and confirm the answer stays inside the focused scope without raw guardrail or policy text.
Try a direct prompt-injection request and a quoted malicious-text analysis request. Direct overrides should be refused calmly, while quoted content may be analyzed as content without leaking internals or running unrelated private retrieval.
Continue the focused thread after an irrelevant fallback and confirm later relevant turns still work without duplicate or empty assistant messages.
Review API logs during the smoke. Healthy public and private turns should produce metadata-only ChatStreamTiming summaries; slow-gap warnings should be reserved for likely stalled stream boundaries, not ordinary multi-second model or connector waits.

For focus-mode changes, also exercise connector focus, account focus with more than one linked OAuth account, file focus, folder focus, non-empty Library focus, empty Library focus, public-mode focus rejection, empty persisted private-thread focus updates, started-chat locked/read-only focus labels, shared-chat privacy, and desktop/mobile picker layouts. Inspect answer scope quality, not only whether the controls render.

For public-model changes, exercise Settings save, a new public draft, model selection without saving, the picker row Make default action, stale-default fallback, and single-provider versus multi-provider picker layout when test data is available.

For message-copy changes, copy an assistant response with headings, bullets, table, and code into both rich and plain-text targets. Rich paste should preserve formatting, and plain-text paste should be readable rendered text; Copy markdown should preserve raw Markdown.

When local file-resource connectors have representative root files and folders, run bun run chat:prompt-pack:local -- --force and then bun run chat:prompt-pack:run:local -- --domain file-resource --kind positive --limit 1 to smoke the live mixed file+folder focus path.

Canonical Sources

docs/features/chat/spec.md
docs/features/user-settings/spec.md
docs/features/chat/stories/chat-persistence-and-resume.stories.md
docs/features/chat/tests/chat-persistence.feature
docs/features/chat/tests/assistant-orchestration.feature
docs/features/connectors/spec.md
packages/contracts/src/chat/focus.ts
packages/contracts/src/chat/threads.ts
docs/features/realtime/spec.md
docs/features/chat/spec-amendments/chat-sources-metadata-schema.md
docs/features/live-query/spec.md
docs/features/human-in-loop/spec.md
packages/contracts/src/chat/source.ts
apps/web/app/components/chat/ChatMarkdownContent.vue
apps/web/app/components/chat/ChatMessageCopyMenu.vue
apps/web/app/components/chat/ChatPublicModelPicker.vue
apps/web/app/components/chat/ChatMarkdownStreamingCode.vue
apps/web/app/composables/useClipboardActions.ts
apps/web/app/composables/useChatShikiHighlighter.ts
apps/web/app/utils/chatPublicModelSelection.ts
apps/web/app/utils/chatRichClipboard.ts
apps/api/src/mastra/index.ts
apps/api/src/mastra/workflows/registeredMastraWorkflows.ts
apps/api/src/mastra/tools/canonicalLiveTools.ts
apps/api/src/mastra/tools/connectorResourceContent.ts
apps/api/src/chats/chatRouteHandlers.ts
apps/web/app/stores/authenticated/storeChatSummaries.ts
apps/web/app/utils/chatStarActions.ts