Let’s face acne, together.

The brief

Skincare is sold on who you are: your age, your skin type, the quiz you filled in once. SkinCoach wanted to sell on something harder and far more useful: how you are actually progressing. Photograph your face, get a real measurement, and let the routine change when the skin changes.

That means measuring skin from a smartphone photo, at a standard a dermatologist would recognise. The problem is that no such standard exists yet. Dermatoscopic image standards are still under development, and a bathroom at 7am under a warm bulb is not a clinic. We were asked for the brand, experience design and UI, the packaging, the mobile app, the platform architecture and engineering, and the AI: the whole product.

What we built

Four systems that only work together: a smartphone photo turned into a clinical-grade measurement, and then into a changed routine.

Getting the photo right first

Every model downstream is worthless if the input drifts. So the hardest engineering went into the least glamorous part: making a phone camera behave like an instrument.

The app runs a real-time face detection model that predicts key points on the face, uses them to extract the face region with high precision, and delimits the specific regions of interest that later stages measure. Capture is refused, not corrected after the fact, when the face is absent, tilted, too far away, or badly exposed.

Then there is the light. The same skin photographs differently under a bathroom bulb and under afternoon sun, a difference far larger than the change you are trying to detect. Our answer is printed on the product itself: the SkinCoach QR, a colour-calibration target on the tube that the user holds in frame during a check-in. Known colours in the shot mean the image can be corrected back to a common reference, so artificial light and sunlight resolve to the same measurement.

The dataset nobody else had

You cannot buy annotated smartphone skin photography, so we built the corpus: images captured at the same quality we expect in the real world, not clinic-grade stock.

Every image was labelled by three trained dermatology nurses and two dermatologists, working to consensus annotation with a QA pass over the images themselves. To make that tractable we built the annotators their own tool: zoomable images, a severity slider fine enough to capture granularity between the named grades, a binary call on inflammation, a flag-for-review escape hatch, and a way to mark an image as poor quality so bad inputs never became training signal.

More than 30,000 photos to date.

Measuring the skin

Redness quantification is a custom computer-vision algorithm rather than a network. It estimates the relative level of redness, which correlates with irritation, by calculating statistical moments over specific regions of the image in CIELab colour space, the space designed so that numerical distance approximates perceived difference. It returns a continuous number, so a check-in that moves from 15.245 to 18.318 is a real, comparable change rather than a vibe.

Acne severity is a custom neural network architecture designed for SkinCoach. It takes front and side pictures of the face, runs them through a shared CNN backbone, pools and combines them, and outputs a single value from 0 to 12 representing perceived severity, mapping onto the clinical ladder of clear, mild, moderate and severe. It reaches 88% accuracy on clears, milds and moderates.

We also generate heatmaps from the activations of the network’s final layers, so the model’s decision can be inspected rather than trusted. In a product that tells someone their skin is getting worse, being able to see where it looked is not a luxury.

From measurement to routine

The three variables are tracked across every check-in, so the product can answer the question a customer actually has: is this working? Progress monitoring evaluates the condition since the last check-in, better or worse, using redness, severity and inflammation together with the historical sequence of photographs, and the outputs are matched automatically to a product and routine recommendation.

The next step, experimental at the time of writing, is to predict the treatment action directly: titrate up, titrate down, continue, or stop.

Why it worked

We owned the whole chain, from brand and packaging to app, platform and models, so the colour target could be printed on a tube because the pipeline needed it, and the capture UI could refuse a photo because the model deserved a good one. Decisions that would have been someone else’s problem in a hand-off were simply ours.

The result is a skincare product that measures instead of guessing, and a routine that changes when the customer does.