Speech To Text

Speech to text is a reusable private authoring control in the chat composer and shared Markdown editor. It records up to two minutes, transcribes through Amazon Bedrock Nova Sonic, and inserts text at the current selection without sending or saving.

Privacy Boundary

Audio is private customer content and remains in browser/API memory only for the active request.
Audio and transcript content must not enter logs, traces, analytics, realtime events, or storage.
Only the customer-configured regional Bedrock Runtime endpoint receives audio.
The transcript remains an unsaved local draft until the user explicitly submits or saves it.

Deployment Configuration

Set the API runtime config to schema version 2 and opt in explicitly:

schemaVersion: 2
speechToText:
  enabled: true
  maxConcurrentRequestsPerReplica: 16
  provider: bedrock
  region: us-east-1

Supported regions are us-east-1, us-west-2, eu-north-1, and ap-northeast-1. Scope bedrock:InvokeModel to amazon.nova-2-sonic-v1:0, and allow HTTPS/HTTP2 egress to the selected regional Bedrock Runtime endpoint. No new application secret is required.

Capacity Planning

Different users transcribe concurrently. Redis enforces only one active transcription for the same authenticated user and keeps that invariant across API replicas; it is not a deployment-wide lock. The 90-second crash-expiry lease is renewed every 30 seconds while a healthy request remains active, and each lease operation has a five-second deadline so a coordinator outage cannot pin replica capacity. Each request has its own Nova Sonic session, a 3,840,000-byte audio cap, and a 30-second API upload-read deadline. Audio is replayed at its real-time cadence, with a provider deadline equal to audio duration plus 60 seconds of response grace.

Tune maxConcurrentRequestsPerReplica to pod resources, set api.replicas for the expected number of simultaneous sessions, and request sufficient Bedrock Nova Sonic bidirectional-stream quota for the customer deployment. The app does not queue or automatically retry throttled audio, preventing retry storms; users receive a recoverable, content-free error instead.

Developer Verification

bun test packages/contracts/src/speechToText/contracts.test.ts
bun test --cwd apps/api src/speechToText src/routes/connectorsRoutes.test.ts
bun run --cwd apps/web test -- test/nuxt/SpeechToTextAudio.nuxt.spec.ts test/nuxt/SpeechToTextButton.nuxt.spec.ts test/nuxt/SpeechToTextErrors.nuxt.spec.ts test/nuxt/SpeechToTextProxyRoutes.nuxt.spec.ts test/nuxt/UseSpeechToText.nuxt.spec.ts

Use agent-browser with a non-sensitive fake microphone stream for local UI verification. A real Bedrock smoke test is optional and must never use customer or personal audio.

Current Limits

English (en-US) only.
Maximum recording length is two minutes.

Planned / Not Yet Implemented

Private audio-file upload.
Realtime partial transcription; dictation is record-then-transcribe today.
Multilingual selection.

Canonical Sources

docs/features/speech-to-text/spec.md
docs/features/speech-to-text/tests/private-dictation.feature
apps/api/src/speechToText/
apps/web/app/components/speech-to-text/SpeechToTextButton.vue