Skincare Agent

The Innovation

Skincare Agent is a multimodal AI system that combines computer vision with conversational AI to revolutionize personalized skincare recommendations. Users can upload facial images or describe their concerns in text—the AI handles both.

Technical Architecture

Multimodal Analysis Engine

Vision Models

• Utilizes Gemini Vision API for facial skin analysis
• Integrates open-weight vision models for redundancy
• Achieved 80% identification accuracy in testing across various skin conditions
• Detects acne, hyperpigmentation, dryness, aging signs, and texture issues

Conversational Flow

• Designed 8-step interactive questioning system to capture:
• Skin type (oily, dry, combination, sensitive)
• Primary concerns and pain points
• Current skincare routine
• Budget constraints and preferences
• Improved recommendation relevance by 60% through structured dialogue

Knowledge Base & Retrieval

Vector Database (Qdrant)

• Curated collection of skincare products and ingredient profiles
• Semantic search through product descriptions and benefits
• Sub-3-second retrieval speed for instant recommendations
• Continuously updated with new products and research

Recommendation Engine

• Combines visual analysis with user preferences
• Suggests products based on:
• Identified skin issues from images
• User-stated concerns and skin type
• Ingredient compatibility and effectiveness
• Budget and routine complexity

Production Infrastructure

Serverless Architecture (AWS Lambda)

• Minimized operational costs through serverless deployment
• Auto-scaling for traffic spikes
• Pay-per-use model for cost efficiency

Frontend (Vercel)

• Next.js frontend deployed on Vercel
• Instant global CDN distribution
• Seamless image upload and analysis
• Real-time chat interface

User Experience Flow

1. Image Upload / Text Input: User either uploads a facial photo or describes concerns
2. AI Analysis: Computer vision detects skin issues with 80% accuracy
3. Interactive Questionnaire: 8-step flow captures preferences and needs
4. Smart Recommendations: Vector search retrieves optimal products
5. Personalized Routine: AI suggests complete routine with usage instructions

Results & Impact

• 80% accuracy in skin condition identification
• 60% improvement in recommendation relevance
• Sub-3-second product retrieval from knowledge base
• Serverless deployment reduces costs by ~70% vs traditional hosting

The Difference

Unlike generic skincare quizzes, this system:

• ✅ Sees your skin through advanced vision AI
• ✅ Understands your unique concerns through conversation
• ✅ Recommends based on actual science and ingredients
• ✅ Scales cost-effectively through serverless architecture