Computer Vision
Bowser
Autonomous pediatric physical therapy robot that coaches exercises in real time using computer vision.
68ms
end-to-end latency
01 — Problem
Kids doing physical therapy exercises need real-time feedback on form and rep count, but most CV pipelines are too slow for live coaching, and coordinating vision, coaching, and robot navigation as separate systems is hard to get right.
02 — Approach
- 01
Real-time perception pipeline: YOLOv8n detects the person, MediaPipe extracts pose landmarks, joint angles are EMA-smoothed for stability.
- 02
Hysteresis-based rep counting — squat counts once knee angle drops below 100° and returns above 150°, with a 0.8s cooldown to prevent false counts from jitter.
- 03
Depth estimation via Intel DPT runs async on a separate thread so it never blocks the main perception loop.
- 04
Four agents defined via Google ADK (patient record retrieval, clinical, pacer, Gemini coaching assistant); only the assistant agent runs through the ADK Runner — the others are invoked as direct function calls in the video loop.
- 05
Autonomous Roomba navigation driven by spatial data from the depth pipeline.
03 — Architecture
04 — Results
68ms
avg end-to-end latency (M3 Pro CPU)
95.7%
precision
84.6%
recall across 52 scripted reps
Stack
Next
Apex→
Real-time F1 pit-wall strategy simulator that runs 10,000 race simulations in under 0.2 seconds.