Case Study · Health Tech · Working Prototype

NutriCoach AI

A personalized nutrition and fitness coach that uses multimodal AI and behavioral context to generate plans built around you as a whole person, not just your last meal.

Claude Vision API Multimodal AI Structured Evals Health Tech B2C
Role
PM + Prototype Builder
Timeline
2026
Type
Working Prototype
Stack
Claude API, Cloudflare Workers, GitHub Pages
Problem

Generic plans fail real people.

Most nutrition apps start with what you ate. They log calories, compute macros from a BMI formula, and return a plan that looks the same for everyone at your height and weight. The plan ignores your sleep, your stress, your schedule, and what your body actually looks like.

The insight behind NutriCoach: personalization has to start with the whole person. Combining a photo (body composition context) with goals and lifestyle signals can produce recommendations that actually fit, and plans that fit get followed.

In short: my mandate was to build a health product that used AI in a meaningfully differentiated way, not just as a wrapper around calorie math.

Discovery & Research

What the market gets wrong.

I audited the top 6 nutrition and fitness apps (MyFitnessPal, Noom, Lose It, Calibrate, MacroFactor, Whoop) and ran qualitative interviews across three user archetypes: beginners trying to lose weight, intermediate users building muscle, and active people optimizing performance. Two artifacts shaped the strategy.

App Churn Rate by Input Friction Level
Not actual artifact, generated for portfolio purposes
📊 Competitive Gap Analysis
FeatureTop AppsNutriCoach
Body composition input❌ None✅ Photo + Multimodal AI
Integrated meal + workout❌ Siloed✅ Unified
Lifestyle context⚠️ Basic✅ Sleep, stress, schedule
Manual food logging✅ Required❌ Not needed
Personalization depth⚠️ BMI-based✅ Multimodal AI
Setup time⚠️ 15+ min✅ Under 3 min
Not actual artifact, generated for portfolio purposes

Three consistent failure patterns emerged from the research:

AI Product Design

Thinking through the full stack.

Shipping an AI product requires more than choosing a model and writing a prompt. Before building anything, I worked through the full set of decisions an AI PM needs to own: model selection, data inputs, cost modeling, infrastructure, and eval design.

The product hypothesis was clear: if we reduce input friction (no food logging) while increasing output relevance (photo + lifestyle context), plan completion rates improve. The question was how to build it responsibly.

Model Selection

I chose Claude Sonnet for its multimodal capability (vision and text in a single API call) combined with strong instruction-following for structured output. Compared to GPT-4o, Sonnet offered comparable quality at lower cost for this use case, which matters when modeling unit economics at scale.

Data & Input Pipeline

The input pipeline conditionally constructs a multimodal message (base64-encoded photo + structured lifestyle context) or falls back to text-only if no photo is provided. This mirrors good production AI design: graceful degradation when optional inputs are absent, consistent output format regardless of input mode.

Auditable Personalization

The prompt explicitly instructs the model to assess body composition, muscle development, and posture from the photo and state that assessment at the start of its response before generating the plan. This makes the personalization auditable rather than a black box, a user or reviewer can check whether the stated visual assessment is actually reflected in the plan's calorie target, protein target, and workout intensity. This was verified by testing with multiple different photos under identical text inputs and confirming the stated assessment and resulting plan differed accordingly.

Cost Modeling

A single NutriCoach generation uses approximately 2,500 input tokens plus ~1,600 image tokens, at Claude Sonnet pricing of ~$3/M tokens. Estimated cost per generation: ~$0.02. At 10K MAU with 2 plan generations per month, that's roughly $400/month, well within MVP budget before introducing a paywall.

Infrastructure

An API key can never be exposed client-side. I built a Cloudflare Worker as a serverless proxy that handles CORS, authenticates with Anthropic, and forwards requests. The free tier handles up to 100K requests per day with zero backend infrastructure, appropriate for an MVP and easy to swap for a Node backend at scale.

Evals & Output Quality

I ran 11 test generations against a 5-criterion rubric: structural completeness, caloric accuracy, photo-responsiveness, dietary compliance, and actionability. The test set deliberately varied photo, goal, activity level, dietary restrictions, and input completeness, including edge cases like no photo provided and minimal field completion.

Results: structural completeness, caloric accuracy, actionability, and photo-responsiveness passed consistently across all runs, including the no-photo fallback case, which produced a transparently caveated estimate rather than a silent guess or a failure.

Dietary compliance surfaced a real bug: two of three restriction-based tests failed. A nut-allergy test included almond butter mislabeled as a "seeds-based alternative," a direct violation. A gluten-free test included unspecified "granola" and "whole grain toast" without confirming the compliant version was used. In both cases, the Grocery List section correctly included the right substitution, but the Meal Plan section did not consistently apply it, a cross-section consistency failure, not a misunderstanding of the restriction itself. This was fixed with an explicit instruction requiring the model to label compliance status on ambiguous ingredients and cross-check meal-plan items against grocery-list substitutions before finalizing output.

One additional finding worth noting honestly: when no goal was specified, the model inferred an intended goal directly from the photo's body composition rather than asking for clarification, a deliberate but debatable UX choice worth flagging as a tradeoff, not simply a strength.

UX & Launch

The 3-step intake flow was designed to keep setup under 3 minutes, directly addressing the logging fatigue problem. Output is tabbed across 4 views so users navigate without scrolling a wall of text. The MVP targets fitness-motivated users 25-40, with a growth loop built around plan sharing and a retention hook of weekly plan refresh based on progress check-in.

Strategy & Product Plan

The input pipeline design.

With those findings in mind, I built a strategy around flipping the input model entirely. The core product decisions:

Input Pipeline — How NutriCoach Works
📸
Photo Upload
Base64 encoded
📋
Lifestyle Inputs
Goals, sleep, stress, diet
🔒
CF Worker Proxy
Secure API gateway
👁️
Claude Vision API
Multimodal inference
Section Eval
4-section parser + validation
📊
Tabbed Output
Meal, workout, macros, grocery
Not actual artifact, generated for portfolio purposes
Live Prototype

Try NutriCoach AI

Upload a photo, share your goals, and get your personalized plan in seconds.

🌿
NutriCoach AI
Personalized nutrition and fitness planning
1 · Photo
2 · Goals
3 · Lifestyle
4 · Your Plan
📸
Click to upload or drag and drop
JPG, PNG up to 10MB · Full body or upper body photo of yourself works best. Photos of objects or clothing will not give accurate results.
Building your personalized plan...
Analyzing your photo and lifestyle inputs

Your Personalized Plan

Based on your photo, goals, and lifestyle

✓ Ready
🥗 Meal Plan
💪 Workout Plan
📊 Macros
🛒 Grocery List