The challenge
I play tennis and pickleball, and the hardest part of fixing a stroke is seeing it. Lessons are expensive, and phone video alone doesn't tell you whether your knees were bent enough or your contact was out in front.
Court Evolution wanted something players could use between sessions: point a phone at yourself, swing, and get feedback that sounds like a coach rather than a generic score.
The approach
The backend runs a hybrid pass on each clip. OpenCV samples about 40 frames, MediaPipe Pose finds the body in each one, and the code turns 13 landmarks into 21 joint angles and distances: shoulder-hip separation (the X-factor), knee bend, how far the wrist reaches from the shoulder, stance width. Six evenly spaced frames plus that summary then go to Claude Vision with a stroke-specific prompt, so the coaching is grounded in measured numbers instead of guesses from pixels.
If there's no API key or the model's reply doesn't parse, a rule-based scorer takes over using fixed thresholds. The rules know that touch shots are different: for a dink, third-shot drop or volley, less rotation scores higher, because those strokes should stay compact.
- Two sports, eight strokesForehand, backhand, serve and volley for tennis; dink, drive, serve and third-shot drop for pickleball.
- Measured, not guessedKnee bend, X-factor, contact reach and wrist height come from MediaPipe landmarks on every sampled frame.
- Coach-style feedbackEach stroke is compared with a reference model, such as a Sinner forehand or a Sampras serve, and every aspect gets one concrete fix.
- Works without the AIA threshold scorer returns the same JSON shape when the vision model is unavailable, so the app never dead-ends.
From swing to score
- 01RecordUp to 10 s of WebM from the phone camera
- 02SampleOpenCV keeps about 40 frames
- 03MeasureMediaPipe landmarks to 21 angles per frame
- 04CoachClaude Vision, or threshold rules as fallback
