feat(ui): redesign voice cockpit with clinical companion drawer, Svelte 5 migration, and Gemini 3.8 voice
- Re-architect VoiceSession into full-bleed studio cockpit with overlay slide-over companion drawer - Eliminate width squishing defect on LiveInsightPanel and add native dark/light theme support - Integrate authentic Google Gemini 3.8 Flash TTS with Fenrir persona and fallback audio pipeline - Implement comprehensive Playwright E2E visual regression suite across Desktop, iPad, and iPhone (21 tests, 45 snapshots) - Migrate core application to Svelte 5 runes (, , ) with reactive stores - Maintain .NET 8 Azure Functions serverless endpoints and clean build hygiene
This commit is contained in:
jpremill committed
1 parent
c82b112624
commit
1feea40684
148 files changed
+11230
-1706
No files matched your search
@@ -0,0 +1,120 @@
|
||||
# AGENTS.md - Cognitive Clarity Workspace Instructions
|
||||
|
||||
Welcome to **Cognitive Clarity** (`clarity-helper`), an adaptive clinical training and roleplay environment engineered to strengthen cognitive flexibility, practice Cognitive Behavioral Therapy (CBT) restructuring techniques, and simulate high-stakes social interactions in real time.
|
||||
|
||||
---
|
||||
|
||||
## 1. System Architecture Overview
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
User([User Voice / Text]) <--> Frontend[React 19 + Vite + Tailwind + Radix UI]
|
||||
|
||||
subgraph "Frontend Real-Time Multimodal Layer"
|
||||
Frontend <--> WebAudio[WebAudio DSP Engine: Live FFT Analysis & Synthesis]
|
||||
Frontend <--> OpenRouterClient[OpenRouter API: Streaming Roleplay & TTS/STT]
|
||||
OpenRouterClient --> Transcripts[Real-Time Transcription Pipeline]
|
||||
Transcripts --> Supervisor[Parallel Turn Supervisor: OpenRouter google/gemini-3.8-flash]
|
||||
Supervisor --> LiveInsights[LiveInsightPanel & SessionReport]
|
||||
end
|
||||
|
||||
subgraph "Backend Serverless Layer (api/)"
|
||||
Frontend -.->|Optional Enterprise Fallback| AzFunc[Azure Functions v4 .NET 8 Isolated Worker]
|
||||
AzFunc --> ChatFunc["/api/chat (ChatProgression)"]
|
||||
AzFunc --> AnalysisFunc["/api/analysis (ClinicalAnalysis)"]
|
||||
AzFunc --> ScenarioFunc["/api/scenario/generate (GenerateScenario)"]
|
||||
AzFunc <--> OpenRouterAPI[OpenRouter API Endpoint /chat/completions]
|
||||
end
|
||||
|
||||
subgraph "Local Persistence & Offline"
|
||||
Frontend <--> LocalStorage[LocalStorage: Custom Scenarios, Settings & Session History]
|
||||
Frontend <--> ServiceWorker[Vite PWA / Workbox Service Worker: sw.ts]
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Core Repository Structure
|
||||
|
||||
```
|
||||
clarity-helper/
|
||||
├── App.tsx # Primary application router, state coordinator, and hero interface
|
||||
├── constants.ts # CBT Glossary (12 distortions, 5 techniques, 2 concepts), default scenarios, schemas
|
||||
├── types.ts # TypeScript domain models (Scenario, AnalysisResult, UserSettings, Message)
|
||||
├── index.html / index.tsx # HTML entry point & React root mount
|
||||
├── index.css # Global styles & Tailwind base directives
|
||||
├── vite.config.ts # Vite 6 config with PWA injectManifest, aliases, and env injection
|
||||
├── sw.ts # Custom Workbox Service Worker with precaching and caching strategies
|
||||
├── api/ # Azure Functions .NET 8 Isolated Worker backend
|
||||
│ ├── Program.cs # Functions host bootstrap & Application Insights telemetry
|
||||
│ ├── api.csproj # .NET 8 project file with HttpClient Factory
|
||||
│ ├── ChatFunction.cs # POST /api/chat - Headless roleplay progression via OpenRouter
|
||||
│ ├── AnalysisFunction.cs # POST /api/analysis - Real-time clinical evaluation via OpenRouter
|
||||
│ └── ScenarioFunction.cs # POST /api/scenario/generate - Structured scenario generator via OpenRouter
|
||||
├── components/ # UI Component Hierarchy
|
||||
│ ├── VoiceSession.tsx # Multimodal voice roleplay controller
|
||||
│ ├── VoiceSession/ # Voice state components (Header, Controls, Visualizer states)
|
||||
│ ├── ChatSession.tsx # Text-based roleplay controller with streaming SSE
|
||||
│ ├── ChatMessage.tsx # Conversation bubble with role styling
|
||||
│ ├── ChatInput.tsx # Text input with submit handling
|
||||
│ ├── LiveInsightPanel.tsx # Real-time turn-by-turn clinical supervision drawer
|
||||
│ ├── SessionReport.tsx # Comprehensive post-session CBT assessment report
|
||||
│ ├── ScenarioGenerator.tsx # AI-powered interactive custom scenario builder
|
||||
│ ├── CBTReference.tsx # Interactive searchable glossary & technique reference
|
||||
│ ├── AudioVisualizer.tsx # Canvas/CSS real-time FFT frequency visualizer
|
||||
│ ├── AudioDeviceSelector.tsx # Input microphone selector modal
|
||||
│ ├── SettingsModal.tsx # Voice profile & OpenRouter API key configuration
|
||||
│ ├── ReloadPrompt.tsx # PWA update notification prompt
|
||||
│ └── ui/ # Radix UI + Tailwind design system primitives
|
||||
├── hooks/
|
||||
│ └── useLiveSession.ts # Low-level WebAudio + OpenRouter voice synthesis hook
|
||||
├── services/
|
||||
│ ├── apiConfig.ts # OpenRouter API key resolution & model defaults
|
||||
│ ├── openRouterService.ts # Streaming chat, structured supervision & speech API client
|
||||
│ ├── sessionStorage.ts # Persistent session history, markdown and JSON report exports
|
||||
│ └── audioUtils.ts # PCM16 / Base64 / AudioBuffer encoding and decoding helpers
|
||||
├── scripts/ # Headless CLI tools and scenario test suites
|
||||
│ ├── validate-scenarios.ts # Scenario schema and distortion coverage validator
|
||||
│ └── simulate-cbt-session.ts # Headless multi-turn roleplay & clinical scoring simulator
|
||||
├── test/ # Automated Node.js / tsx unit test suite
|
||||
│ ├── apiConfig.test.ts # API key resolution & model defaults tests
|
||||
│ ├── openRouterService.test.ts # Streaming, supervisor & scenario generation tests
|
||||
│ └── scenarios.test.ts # Clinical scenario schema & glossary tests
|
||||
└── .agents/ # Specialized agent skills and workflow blueprints
|
||||
└── skills/ # Domain-specific skill runbooks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Engineering Invariants & Coding Guidelines
|
||||
|
||||
### 3.1 Strict Clinical Roleplay Separation
|
||||
- **No Meta-Commentary Leaks**: When the AI character is engaged in roleplay (`ChatSession`, `VoiceSession`, `ChatFunction.cs`), system instructions MUST forbid the model from breaking character, congratulating the user, giving clinical advice, or outputting feedback tokens.
|
||||
- **Parallel Supervision**: Clinical feedback is exclusively computed out-of-band by a dedicated supervisor model instance (`LiveInsightPanel`, `AnalysisFunction.cs`), ensuring realistic emotional friction during the simulation.
|
||||
|
||||
### 3.2 OpenRouter Exclusive API Policy
|
||||
- **Zero Direct Google API Keys**: The codebase must NOT require or use direct Google Gemini API keys. All model queries (`google/gemini-3.8-flash`, `google/gemini-3.7-flash`, `google/gemini-2.5-flash`, `anthropic/claude-3.5-haiku`, `meta-llama/llama-3.3-70b-instruct`) route exclusively through OpenRouter.
|
||||
- **Client Security**: OpenRouter API Keys provided by the user are stored in browser LocalStorage and never sent to external servers or telemetries.
|
||||
|
||||
### 3.3 TypeScript & Build Hygiene
|
||||
- Keep `npm run lint` (`tsc --noEmit`) and `npm test` passing with zero errors.
|
||||
- Ensure all Radix UI component props utilize `React.ComponentPropsWithoutRef<T>` or type unions compatible with React 19.
|
||||
- Maintain full type synchronization between `types.ts` and backend C# DTOs in `api/*.cs`.
|
||||
|
||||
### 3.4 .NET Azure Functions Backend
|
||||
- Target **.NET 8.0** isolated worker model with `Microsoft.Azure.Functions.Worker.Sdk`.
|
||||
- Call OpenRouter API via `HttpClientFactory` with `OPENROUTER_API_KEY`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Verification & Testing Commands
|
||||
|
||||
| Purpose | Command | Notes |
|
||||
|:---|:---|:---|
|
||||
| **Run Unit Tests** | `npm test` | Executes native Node.js / `tsx` unit test suites |
|
||||
| **Typecheck Frontend** | `npm run lint` | Runs `tsc --noEmit` |
|
||||
| **Build Frontend** | `npm run build:nobump` | Bundles Vite SPA + PWA Service Worker to `dist/` |
|
||||
| **Validate Scenarios** | `npm run validate:scenarios` | Validates default & custom scenario data schemas |
|
||||
| **Simulate Session** | `npm run simulate:session` | Executes headless 3-turn CBT simulation with clinical score |
|
||||
| **Build .NET API** | `dotnet build api/api.csproj` | Compiles Azure Functions Isolated Worker assembly |
|
||||
| **Run Dev Server** | `npm run dev` | Starts Vite on `http://localhost:3000` |
|
||||
Reference in new issue
Block a user