Medical-Grade Skin Condition Detection & Segmentation System
This is a production-grade medical AI system built for a commercial healthcare client — designed to detect a specific skin condition, accurately segment the boundaries of affected skin regions, and count individual affected areas from clinical images. The system was built entirely solo, from initial data preparation and model training through to a working frontend prototype and full cloud deployment. It is currently in active commercial use and has been patented by the client.
96%
Detection · Segmentation · Region Counting
Human-in-the-Loop RLHF
Solo end-to-end — model to cloud
Client
Confidential Commercial Client (Patented)
Year
2026
Role
Solo — end-to-end (ML + Backend + Frontend + Cloud)
Tech
Confidential Commercial Project
This system is patented by the client. Screenshots, client identity, and condition name cannot be disclosed. Technical architecture and methodology are shared with permission.
The Problem
The client was working in dermatology — a domain where accurate visual assessment of skin conditions is clinically critical but inherently difficult.
Several factors compound the challenge: affected skin areas can look visually similar to healthy skin at different stages of a condition. The boundaries of affected regions are irregular and can blend gradually into surrounding tissue — particularly under varying lighting conditions, different image qualities, and across different skin tones.
Manual clinical assessment is slow, subjective, and can vary between practitioners — making it difficult to monitor disease progression accurately over time or at scale.
The Solution
The delivered system gives clinicians a fast, reliable second opinion on assessment:
- Upload a clinical skin image
- The model returns a segmentation overlay showing the exact boundaries of affected skin areas directly on the image
- Individual affected regions are counted and labelled automatically
- Healthy and affected skin areas are classified and visually distinguished
- Where the clinician identifies a missed region, they can flag it — and that feedback feeds directly into future model improvement
The entire pipeline — from raw image in to annotated result out — runs in seconds.
The system is now in active commercial use and has been patented by the client as a proprietary medical AI product.
Technical Approach & Architecture
Model Architecture — UNet++ for Precise Segmentation
Standard object detection models (YOLO, Faster R-CNN) identify where something is, but are not built for pixel-level boundary mapping. For this condition, the requirement was to trace the exact boundary of each affected skin region — not just draw a bounding box around it.
UNet++ was chosen for its nested, dense skip connections, which allow the model to capture fine-grained spatial detail at multiple scales simultaneously. This is critical for irregular skin conditions where boundaries are gradual rather than sharp.
The architecture handles three outputs in a single forward pass:
- Binary classification (healthy skin vs. affected skin)
- Pixel-level segmentation mask (exact boundaries of affected regions)
- Instance counting (number of distinct affected areas in the image)
Data Pipeline & Version Control (DVC)
One of the core engineering decisions was implementing Data Version Control (DVC) into the training pipeline from the beginning.
DVC tracks every version of the training dataset, model weights, and pipeline configuration — the same way Git tracks code. This means:
- When new clinical images are added to the dataset, the model can be retrained automatically with a full audit trail of what data produced which model version
- Any model can be rolled back to an exact previous version if performance regresses
- The entire experiment history is reproducible — same data in, same weights out, every time
Human Feedback Loop — Reinforcement Learning from Human Feedback (RLHF)
No model is perfect at launch. The system was designed with the assumption that clinicians would occasionally identify affected regions the model missed.
Rather than treating missed detections as failures, a human feedback mechanism was integrated directly into the application:
- When a clinician identifies an affected area the model missed, they mark it in the interface
- That feedback is captured and used to fine-tune the model's detection parameters for similar image characteristics (lighting, skin tone, region size, contrast)
- Over time, the model's performance on previously difficult cases improves automatically based on real clinical corrections
Deployment Architecture
The system was deployed as a cloud-hosted web application — accessible from any device with a browser, requiring no local installation by clinical staff.
The backend serves model inference via a REST API, with the frontend providing a clean clinical interface for image upload, result review, and feedback submission.
Technical Visualizations
1. System Architecture Flow
2. Human-in-the-Loop RLHF Cycle
3. Continuous DVC Pipeline
Key Technical Challenges
1. Detecting gradual, irregular boundaries
The boundaries of affected skin regions are gradual, not sharp — they blend into surrounding healthy skin. Getting UNet++ to reliably trace these edges required careful augmentation strategies during training (rotation, colour jitter, brightness variation) to teach the model to generalise across lighting and skin tone variance.
2. Multi-task output from a single model
Detection, segmentation, and region counting are technically three separate tasks. Engineering a single model that performs all three simultaneously — without one task degrading another — required careful loss function balancing during training.
3. Continuous improvement without downtime
The DVC + RLHF architecture needed to allow model updates to roll out to the live system without taking the application offline or losing the existing model's performance baseline. This required a versioned model serving approach where new weights are validated before replacing the production model.
4. Clinical trust
Medical AI is only useful if clinicians trust it. The interface was designed to show confidence levels alongside predictions, and the feedback loop was surfaced prominently — so clinicians felt in control of the system, not replaced by it.
What This Project Demonstrates
This system was built entirely by one engineer — from dataset preparation and model architecture through to production cloud deployment and the human feedback loop. Every layer of the stack was designed, built, and shipped solo.
If you are building a medical AI product, a diagnostic tool, or any system that needs to improve continuously from real-world usage — this is the kind of engineering Webygraphy brings to your project.
Confidentiality Note
This system was built for a commercial client in the medical technology sector. The client has filed a patent on the product — accordingly, the client name, product name, condition name, and application screenshots cannot be shared publicly.
The technical details, architecture decisions, and methodology described on this page are shared with the client's awareness.
If you are evaluating Webygraphy for a similar project and would like to discuss this work in more depth — including specific technical decisions — we are happy to do so under NDA in a private discovery call.
