# RealFlow: Quick Packet **Real-Time Conversational AI Pipeline** | Last Updated: 2026-01-24 *3-page visual overview: Overview → Architecture → Market Analysis* --- ## Page 1: Overview & Flow ### What is RealFlow? **RealFlow** is THRPY's real-time conversational AI pipeline that processes multiple layers of understanding before responding. It's more than an audio service—it's a complete "stream of consciousness" that thinks before it speaks. ### The RealFlow Pipeline ``` ┌─────────────────────────────────────────────────────────────┐ │ USER SPEAKS │ └──────────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ [STT] Speech-to-Text │ │ Whisper ONNX | 100-300ms │ └──────────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ [PARALLEL INFERENCE] │ │ ┌────────────────────┐ ┌────────────────────┐ │ │ │ Vocal Inference │ │ Text Inference │ │ │ │ Audio → Emotion │ │ Text → Sentiment │ │ │ │ 50-150ms │ │ 100-300ms │ │ │ └────────────────────┘ └────────────────────┘ │ └──────────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ [SAFETY LAYER] │ │ Fast Gate (<20-50ms) + Full Assessment (<50ms) │ └──────────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ [LLM] Generate Response │ │ Streaming | First Token <200ms │ │ ├─ Safe: Normal therapeutic response │ │ └─ Flagged: Restricted/crisis response │ └──────────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ [TTS] Text-to-Speech │ │ Supertonic ONNX | 50-200ms | ~167x real-time │ └──────────────────────┬──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ USER HEARS RESPONSE │ │ (with synchronized word highlighting ±10ms) │ └─────────────────────────────────────────────────────────────┘ ``` ### Key Features | Feature | Description | Performance | |---------|-------------|-------------| | **Multi-Layer Inference** | Vocal + Text analysis in parallel | 2x faster than sequential | | **Safety-First** | Pre-LLM safety check | <50ms (2-6x faster) | | **Real-Time** | End-to-end processing | <2 seconds total | | **Ultra-Fast TTS** | Supertonic ONNX | ~167x real-time (3-16x faster) | | **Perfect Sync** | Word highlighting | ±10ms accuracy (5-10x better) | | **Complete Pipeline** | Integrated system | Single system, not assembled | --- ## Page 2: Architecture & Performance ### Architecture Overview ``` ┌─────────────────────────────────────────────────────────────┐ │ RealFlow Architecture │ └─────────────────────────────────────────────────────────────┘ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ Frontend │◄────►│ Backend │◄────►│ Services │ │ │ │ │ │ │ │ • WebSocket │ │ • Unified │ │ • STT │ │ • Word │ │ Audio │ │ • TTS │ │ Highlight │ │ Service │ │ • Inference │ │ • Audio │ │ • Safety │ │ • Safety │ │ Playback │ │ • LLM │ │ • LLM │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ └─────────────────────┴─────────────────────┘ │ ┌────────▼────────┐ │ Real-Time │ │ Bidirectional │ │ Streaming │ └─────────────────┘ ``` ### Performance Metrics | Metric | RealFlow | Industry Standard | Advantage | |--------|----------|-------------------|-----------| | **STT Latency** | 100-300ms | 500-1000ms | **2-3x faster** | | **Vocal Inference** | 50-150ms | 200-500ms | **2-3x faster** | | **Text Inference** | 100-300ms | 300-800ms | **2-3x faster** | | **Safety Check** | 20-50ms | 100-300ms | **2-6x faster** | | **LLM First Token** | <200ms | 500-2000ms | **2-10x faster** | | **TTS Generation** | 50-200ms | 200-1000ms | **4-20x faster** | | **End-to-End** | 500-2000ms | 2000-5000ms | **2-5x faster** | | **TTS Speed** | ~167x real-time | 10-50x real-time | **3-16x faster** | ### What Makes It Different ``` ┌─────────────────────────────────────────────────────────────┐ │ Key Differentiators │ └─────────────────────────────────────────────────────────────┘ 1. PARALLEL INFERENCE Most: Sequential (audio OR text) RealFlow: Simultaneous (audio + text) → 2x faster + more comprehensive 2. SAFETY-FIRST Most: Post-processing (after response) RealFlow: Pre-LLM (before response) → 4-10x faster + safer 3. ULTRA-FAST TTS Most: 10-50x real-time RealFlow: 167x real-time → 3-16x faster 4. PERFECT SYNC Most: ±50-100ms word highlighting RealFlow: ±10ms accuracy → 5-10x more accurate 5. COMPLETE PIPELINE Most: Multiple services (you assemble) RealFlow: Single integrated system → Lower latency, simpler 6. THERAPEUTIC FOCUS Most: Generic voice AI RealFlow: Purpose-built for therapy → Built-in safety protocols ``` ### Platform Vision ``` ┌─────────────────────────────────────────────────────────────┐ │ Evolution Path │ └─────────────────────────────────────────────────────────────┘ TODAY: Complete Integrated Pipeline • Fixed components • Optimized for therapy • Multi-layer inference TOMORROW: Modular Platform • Swappable components • Plug-and-play customization • Extensible architecture CORE VALUE: Real-time inferencing + Flexibility ``` --- ## Page 3: Market Position & Competitive Analysis ### Key Differentiators ``` ┌─────────────────────────────────────────────────────────────┐ │ What Makes RealFlow Unique │ └─────────────────────────────────────────────────────────────┘ 1. PARALLEL MULTI-LAYER INFERENCE • Vocal + Text inference simultaneously • Industry: Sequential or single-layer • Advantage: 2x faster + more comprehensive 2. SAFETY-FIRST ARCHITECTURE • Pre-LLM safety check (<50ms) • Industry: Post-processing (200-500ms) • Advantage: 4-10x faster + prevents unsafe responses 3. ULTRA-FAST TTS (167x real-time) • Industry: 10-50x real-time • Advantage: 3-16x faster, local processing 4. COMPLETE INTEGRATED PIPELINE • Single system: STT → Inference → Safety → LLM → TTS • Industry: Multiple services (you assemble) • Advantage: Lower latency, simpler integration 5. THERAPEUTIC PURPOSE-BUILT • Built-in safety protocols, crisis detection • Industry: Generic voice AI • Advantage: Regulatory compliance, clinical focus 6. PERFECT SYNCHRONIZATION • Word highlighting ±10ms accuracy • Industry: ±50-100ms • Advantage: 5-10x more accurate visual feedback ``` ### Market Position ``` ┌─────────────────────────────────────────────────────────────┐ │ Competitive Landscape │ └─────────────────────────────────────────────────────────────┘ TIER 1: Basic Voice AI • Google Dialogflow, Amazon Lex • Focus: Chatbot with voice • RealFlow Advantage: Multi-layer inference, safety-first TIER 2: Advanced Voice AI • Deepgram, AssemblyAI, ElevenLabs • Focus: STT/TTS with basic features • RealFlow Advantage: Complete pipeline, parallel inference TIER 3: Enterprise Solutions • Custom builds, assembled platforms • Focus: Enterprise integration • RealFlow Advantage: Integrated, faster, purpose-built TIER 4: Therapeutic AI • Woebot, Wysa, Talkspace AI • Focus: Mental health chatbots • RealFlow Advantage: Voice-first, real-time, multi-layer ┌─────────────────────────────────────────────────────────────┐ │ RealFlow Market Position │ └─────────────────────────────────────────────────────────────┘ POSITIONING: "The only complete real-time voice AI pipeline purpose-built for therapeutic applications" DIFFERENTIATION: Speed (2-20x faster) + Integration + Safety ``` ### Competitors | Competitor | Type | Focus | RealFlow Advantage | |------------|------|-------|-------------------| | **Google Dialogflow** | Basic Voice AI | Chatbot with voice | Multi-layer inference, safety-first | | **Amazon Lex** | Basic Voice AI | Voice bots | Complete pipeline, therapeutic focus | | **Deepgram** | STT Service | Speech-to-text | Complete pipeline, parallel inference | | **AssemblyAI** | STT Service | STT + sentiment | Multi-layer inference, safety-first | | **ElevenLabs** | TTS Service | Text-to-speech | Complete pipeline, ultra-fast TTS | | **Woebot** | Therapeutic AI | Mental health chatbot | Voice-first, real-time, multi-layer | | **Wysa** | Therapeutic AI | Mental health chatbot | Voice-first, real-time, multi-layer | | **Talkspace AI** | Therapeutic AI | Therapy platform | Complete pipeline, safety protocols | ### Target Clients ``` ┌─────────────────────────────────────────────────────────────┐ │ Who Needs RealFlow │ └─────────────────────────────────────────────────────────────┘ PRIMARY CLIENTS: 1. MENTAL HEALTH PLATFORMS • Therapy apps (Talkspace, BetterHelp competitors) • Mental health startups • Telehealth platforms • Need: Voice-first therapy with safety protocols 2. HEALTHCARE PROVIDERS • Hospitals, clinics • Behavioral health centers • Need: HIPAA-compliant voice AI for patient support 3. ENTERPRISE HEALTHCARE • Employee assistance programs (EAPs) • Corporate wellness platforms • Need: Scalable voice AI for employee mental health 4. HEALTH TECH COMPANIES • Digital health startups • Health AI platforms • Need: Real-time voice AI with therapeutic focus SECONDARY CLIENTS: 5. VOICE AI PLATFORM BUILDERS • Companies building voice AI products • Need: Complete pipeline (not just STT/TTS) 6. RESEARCH INSTITUTIONS • Universities, research labs • Need: Real-time inference platform for studies ``` ### Market Size Analysis ``` ┌─────────────────────────────────────────────────────────────┐ │ Market Opportunity │ └─────────────────────────────────────────────────────────────┘ GLOBAL VOICE AI MARKET • Size: $11.2B (2023) → $50.1B (2030) • CAGR: 23.6% • Segment: Conversational AI growing fastest DIGITAL MENTAL HEALTH MARKET • Size: $5.2B (2023) → $17.5B (2030) • CAGR: 19.1% • Segment: AI-powered therapy growing fastest TARGET ADDRESSABLE MARKET (TAM) • Voice AI for Healthcare: ~$2-3B (2024) • Therapeutic Voice AI: ~$500M-1B (2024) • Growing 20-25% annually SERVICEABLE ADDRESSABLE MARKET (SAM) • Mental health platforms: ~$200-300M (2024) • Healthcare providers: ~$100-200M (2024) • Total SAM: ~$300-500M (2024) SERVICEABLE OBTAINABLE MARKET (SOM) • Year 1: ~$5-10M (early adopters) • Year 3: ~$20-50M (market penetration) • Year 5: ~$50-100M (market leader position) KEY DRIVERS: • Rising mental health awareness • Telehealth adoption acceleration • AI voice technology maturity • Regulatory support for digital health • Cost pressure on traditional therapy ``` ### Competitive Advantages Summary ``` ┌─────────────────────────────────────────────────────────────┐ │ Why RealFlow Wins │ └─────────────────────────────────────────────────────────────┘ SPEED: 2-20x faster than competitors → Better user experience, lower costs INTEGRATION: Complete pipeline vs. assembled pieces → Simpler implementation, lower latency SAFETY: Pre-LLM safety vs. post-processing → Critical for therapeutic applications THERAPEUTIC: Purpose-built vs. generic → Regulatory compliance, clinical focus PARALLEL PROCESSING: Multi-layer inference → More comprehensive understanding MARKET POSITION: Only complete therapeutic voice AI pipeline → First-mover advantage in niche market ``` ### Code Location **Main Service:** `chat-api/app/unified_audio_streaming_service.py` **Related Components:** - Safety: `chat-api/app/safety_client.py` - LLM: `chat-api/app/services/chat_service.py` - Inference: `chat-api/app/services/enhanced_emotion_service.py` - Frontend: `frontend/src/hooks/useUnifiedAudioWebSocket.ts` --- **RealFlow: Real-time inferencing that thinks before it speaks.** *For detailed documentation, see `REALTIME_CONVO_FLOW.md`*