Feature Requests

**ISSUE RESOLVED** - Live video streaming with virtual backgrounds and custom audio devices is now working. ## What Was Fixed: ### 1. Playback URL Configuration - **Problem**: Station was missing the `playback_url` field required for HLS streaming - **Solution**: Directly set playback URL from AWS IVS: `https://9bf556a5c7c5.us-east-1.playback.live-video.net/api/video/v1/us-east-1.383425823812.channel.IC7ZjVeAPRi3.m3u8` - **Root Cause**: Database field was empty despite IVS being provisioned ### 2. Video Stream Architecture - **DJ Panel (IVSBroadcastPanel.jsx)**: - Captures raw camera feed via getUserMedia - Applies MediaPipe Selfie Segmentation for virtual backgrounds - Processes video through canvas with blur/custom background options - Sends ONLY processed stream to AWS IVS (no raw feed) - Broadcasts to: `video_ingest_url: 9bf556a5c7c5.global-contribute.live-video.net` - **Listen Page (LiveVideoViewer.jsx)**: - Receives HLS stream from AWS IVS playback URL - Displays processed video with virtual backgrounds intact - Uses HLS.js with ultra-low latency configuration - No autoplay - requires explicit user click to start ### 3. Audio Configuration - **Audio Device Selection**: DJs can select microphone, virtual audio cable (Serato/VirtualDJ), or system audio - **No Monitoring**: Audio level visualization only - no local playback to prevent double audio - **Audio Sources Supported**: - Microphone input - Virtual audio cables (VB-Cable, BlackHole, Loopback) - System audio via getDisplayMedia ### 4. AWS IVS Settings - Channel ARN: `arn:aws:ivs:us-east-1:383425823812:channel/IC7ZjVeAPRi3` - Stream Key: `sk_us-east-1_uUIO08KZ1GRF_...` (secured) - Stage ARN: `arn:aws:ivs:us-east-1:383425823812:stage/tz009O5SNbKi` - Region: `us-east-1` - Low-Latency Mode: ENABLED ### 5. Recent Improvements Added - ✅ Volume controls for listeners (slider + mute button) - ✅ Countdown timer visibility before broadcast starts - ✅ Removed all autoplay - explicit play button required - ✅ Better error logging for stream diagnostics - ✅ Ultra-low latency HLS.js configuration (1-3 second buffer instead of 5-10 seconds) ## Current Issues Being Investigated: ### **High Latency (~1-2 minutes) - INVESTIGATING** - **Problem**: Video stream is 1-2 minutes behind real-time despite low-latency mode enabled - **Impact**: Listeners miss time-sensitive content (contests, shoutouts, song requests) - **Status**: AWS IVS low-latency mode confirmed enabled, investigating HLS.js player configuration - **Recent Changes**: Updated HLS.js to use aggressive ultra-low latency settings: - `maxBufferLength: 3` (down from 10) - `backBufferLength: 1` (down from 5) - `liveSyncDurationCount: 2` - `liveMaxLatencyDurationCount: 5` - `highBufferWatchdogPeriod: 1` - **Next Steps**: Test if new HLS.js settings reduce latency to 3-5 seconds ### Countdown Timer - May not always display properly before video starts - Needs better z-index handling for overlay visibility ### Listener Count Updates - Real-time listener count updates may have delays - SSE connection should be verified for reliable updates ## Technical Stack: - **Frontend**: React, HLS.js (ultra-low latency config), MediaPipe Selfie Segmentation - **Backend**: AWS IVS (Interactive Video Service), AWS IVS Real-Time - **Video Processing**: Canvas API, MediaStream API - **Audio**: Web Audio API, getUserMedia, getDisplayMedia ## Next Steps: 1. **HIGH PRIORITY**: Test new ultra-low latency HLS.js configuration 2. Verify countdown timer displays consistently 3. Test listener count real-time updates 4. Monitor stream quality and stability with aggressive buffering 5. Document DJ workflow for virtual background setup

⚠️ REFERENCE CONFIGURATION - DO NOT MODIFY ⚠️ 🎯 FINAL WORKING SETTINGS FOR VIRTUAL BACKGROUNDS: 📦 Required NPM Package: - @mediapipe/selfie_segmentation@^0.1.1675465747 🔧 MediaPipe Configuration (VideoSegmentationService.js): this.segmentation.setOptions({ modelSelection: 1, // CRITICAL: Landscape model for better background removal selfieMode: false, effect: 'background' }); 🎭 Segmentation Threshold: - Mask threshold: 245 (ultra-aggressive) - Only pixels with maskValue > 245 are considered "person" - Eliminates all raw video background leakage if (maskValue > 245) { // Render person pixel } else { // Render virtual background pixel } 🎬 Frame Processing: - Target FPS: 15 (prevents memory issues) - Canvas capture stream: 30fps (smooth output) - Throttled processing with pendingFrame flag - Cached canvases for memory optimization 🖼️ Compositing Order: 1. Draw virtual background FIRST (fills entire canvas) 2. Get person pixels from video 3. Get segmentation mask from MediaPipe 4. Get background pixels from canvas 5. Composite: if mask > 245, use person pixels, else use background 💾 Memory Optimization: - Reuse _personCanvas, _maskCanvas, _outputData - Prevents canvas recreation on every frame - Proper disposal in cleanup 📝 Key Lessons: - modelSelection: 1 (landscape) is superior to 0 (general) - Higher threshold (245) prevents edge artifacts - Background must be drawn first, THEN masked person - Canvas stretching (drawImage with width/height) prevents gaps ✅ Result: Clean virtual backgrounds with no raw video leakage

Fixed intermittent video streaming display issues caused by race conditions in StationContext data loading.

COMPLETE VIRTUAL BACKGROUND IMPLEMENTATION DOCUMENTATION === PROBLEM === Virtual backgrounds (blur & custom images) not working after 5 months of development === ROOT CAUSE === MediaPipe segmentation mask not properly rendering person over background === TECHNICAL IMPLEMENTATION === 1. VideoSegmentationService.jsx: - Uses @mediapipe/selfie_segmentation for person detection - Creates canvas element to composite video + background - Processes at 60fps with high quality settings - Canvas resolution matches camera input (up to 4K) 2. Segmentation Process: - Load MediaPipe model from CDN - Set modelSelection: 1 (landscape/high quality) - Process each video frame through MediaPipe - Get segmentationMask (0-255 values, 255=person) - Apply mask to extract person pixels - Composite person over background 3. Background Types: - 'none': Raw video passthrough - 'blur': Blurred version of video - 'custom': User-uploaded image (station background/logo) 4. IVSBroadcastPanel.jsx Integration: - Loads station background_url and logo_url as default backgrounds - User can upload additional backgrounds - Optional background rotation (10 second intervals) - Virtual background applies BEFORE broadcast starts === CRITICAL SETTINGS === Camera Constraints: ```javascript video: { width: { ideal: 3840, min: 1920 }, height: { ideal: 2160, min: 1080 }, frameRate: { ideal: 60, min: 30 }, aspectRatio: 16/9 } ``` IVS Broadcast Config: ```javascript config = IVSBroadcastClient.STANDARD_LANDSCAPE; config.maxResolution = { width: 1920, height: 1080 }; config.maxFramerate = 60; config.maxBitrate = 8500; ``` Canvas Settings: ```javascript ctx = canvas.getContext('2d', { willReadFrequently: true, alpha: false }); ctx.imageSmoothingEnabled = true; ctx.imageSmoothingQuality = 'high'; ``` === COMMON ISSUES === 1. MediaPipe CDN Loading: - Files must load from: https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation/ - Check browser console for 404 errors - CORS must be enabled 2. Mask Application: - segmentationMask.data contains uint8 values (0-255) - Must apply to alpha channel of person pixels - Background drawn FIRST, person drawn ON TOP 3. Performance: - Heavy CPU usage with high resolution - May need to reduce resolution on lower-end devices - 60fps target, falls back to 30fps === FILES INVOLVED === - components/djpanel/VideoSegmentationService.jsx - components/djpanel/IVSBroadcastPanel.jsx - entities/RadioStation.json (background_url, logo_url, custom_background_urls) === TESTING CHECKLIST === [] MediaPipe model loads without 404 errors [] Camera preview shows before segmentation applied [] Segmentation mask detected in console logs [] Person visible over background (not black screen) [] Custom background image displays correctly [] Blur effect works [] Quality is acceptable (not grainy/pixelated) [] Broadcast starts with processed stream === IF BROKEN AGAIN === 1. Check browser console for MediaPipe errors 2. Verify segmentation.onResults() is being called 3. Check if segmentationMask exists in results 4. Verify canvas dimensions match video 5. Test with 'none' background first (raw video) 6. Then test 'blur', then 'custom' 7. Check station.background_url and station.logo_url are valid URLs

**Issue:** Buttons with white/light backgrounds (bg-white, bg-gray-50, etc.) are not displaying text in the peach theme color (#FF8B7B) despite multiple CSS override attempts. **Problem Areas:** - Navigation buttons with bg-white class - Radix UI components (tabs, dialogs) with white backgrounds - Any button-like elements with light background colors **Attempted Solutions (Failed):** 1. Multiple iterations of CSS !important rules in globals.css - all failed 2. Increasingly aggressive selectors - no effect 3. Catch-all rules for light backgrounds - didn't work **Root Cause:** CSS was being overridden by component-level styles with higher specificity. **RESOLUTION:** Modified Button component directly (components/ui/button.jsx) to detect white/light backgrounds and force peach text color at the component level. This bypasses the CSS specificity issues entirely. **Fix Applied:** - Created Button component with built-in logic to detect bg-white, bg-gray-, bg-slate- classes - Automatically applies text-[#FF8B7B] when light background detected - Applied at component render level, not CSS level **Status:** RESOLVED - White buttons now display peach text color correctly.