🍽️ Leftovr

Your AI-Powered Recipe Assistant & Pantry Manager

A sophisticated multi-agent food concierge system that helps you discover delicious recipes based on what you already have in your pantry.

---

📖 Table of Contents

• Features
• Architecture
• Quick Start
• Setup Instructions
• Usage
• Deployment to Streamlit Cloud
• Project Structure
• Database Management
• Development
• Troubleshooting
• License

---

✨ Features

🤖 Multi-Agent AI System

• Executive Chef (Orchestrator): Routes queries and coordinates agent collaboration
• Pantry Agent: Manages inventory with natural language commands via Model Context Protocol (MCP)
• Recipe Knowledge Agent: Performs hybrid search across recipe databases (Milvus/Qdrant)
• Sous Chef: Curates top-3 recommendations and adapts recipes to your pantry

🔍 Intelligent Recipe Search

• Hybrid Search: Combines semantic (vector) and keyword search for optimal results
• Pantry-Based Matching: Finds recipes that maximize use of your available ingredients
• Expiring Item Prioritization: Suggests recipes using ingredients that expire soon
• Smart Filtering: Respects dietary restrictions, allergies, and cuisine preferences

🗄️ Smart Pantry Management

• Natural language inventory updates ("I have 2 chicken breasts and tomatoes")
• Automatic expiration tracking
• Quantity management with delta operations
• Persistent SQLite storage (~/.leftovr/pantry.db)

🎨 User-Friendly Interface

• Beautiful Streamlit web interface
• Real-time chat-based interaction
• Visual recipe cards with metadata (prep time, servings, match percentage)
• Sidebar showing live pantry inventory and preferences

🧠 Contextual Awareness

• Maintains conversation history
• Remembers user preferences across sessions
• Learns dietary restrictions and allergies
• Adapts recipes based on skill level

---

🏗️ Architecture

System Overview

┌─────────────────────────────────────────────────────────────┐
│                    Streamlit Frontend                        │
│                  (streamlit_app.py)                          │
└───────────────────────┬─────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────────┐
│              LangGraph Workflow Orchestrator                 │
│                    (main.py)                                 │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │ Executive    │  │   Pantry     │  │   Recipe     │     │
│  │    Chef      │─▶│   Agent      │  │  Knowledge   │     │
│  │ (Classifier) │  │   (MCP)      │  │   Agent      │     │
│  └──────────────┘  └──────┬───────┘  └──────┬───────┘     │
│                            │                  │              │
│                            ▼                  ▼              │
│                     ┌──────────────┐  ┌──────────────┐     │
│                     │   SQLite     │  │   Milvus/    │     │
│                     │   Pantry     │  │   Qdrant     │     │
│                     │     DB       │  │   Vector DB  │     │
│                     └──────────────┘  └──────────────┘     │
│                                                              │
│  ┌──────────────┐                                           │
│  │  Sous Chef   │                                           │
│  │ (Recommender)│                                           │
│  └──────────────┘                                           │
└─────────────────────────────────────────────────────────────┘
                        │
                        ▼
                  ┌──────────┐
                  │ OpenAI   │
                  │ GPT-4o   │
                  └──────────┘

Agent Responsibilities

Agent	Role	Technologies
Executive Chef	Query classification, routing, general conversation	OpenAI GPT-4o
Pantry Agent	Inventory CRUD operations, expiration tracking	MCP Server, SQLite
Recipe Knowledge	Hybrid search, ingredient matching, recipe retrieval	Milvus/Zilliz, Qdrant, Sentence Transformers
Sous Chef	Recipe ranking, adaptation, customization	OpenAI GPT-4o

---

🚀 Quick Start

Get Leftovr running locally in 5 minutes!

Prerequisites

Requirement	Where to Get	Required?
Python 3.8+	python.org	✅ Required
OpenAI API Key	platform.openai.com	✅ Required
Zilliz Cloud Account	cloud.zilliz.com	⚠️ Recommended for recipe search

Local Installation (5 Steps)

1. Clone the Repository

git clone https://github.com/your-username/leftovr-app.git
cd leftovr-app

2. Create Virtual Environment

# Create virtual environment
python3 -m venv .venv

# Activate it
source .venv/bin/activate  # macOS/Linux
# OR
.venv\Scripts\activate  # Windows

3. Install Dependencies

pip install -r requirements.txt

4. Configure Environment Variables

# Copy example file
cp .env.example .env

# Edit .env with your favorite editor
nano .env
# OR
code .env  # VS Code

Add your credentials:

# Required
OPENAI_API_KEY=sk-proj-your-key-here

# Recommended (for recipe search)
ZILLIZ_CLUSTER_ENDPOINT=https://your-cluster.api.gcp-us-west1.zillizcloud.com
ZILLIZ_TOKEN=your-token-here

5. Launch the App

streamlit run streamlit_app.py

The app will open automatically at http://localhost:8501 🎉

First Time Setup: Ingest Recipes

For recipe search to work, you need to ingest the recipe database:

# Make sure environment variables are set
source .env  # or set them in your terminal

# Ingest recipes to Zilliz Cloud (recommended)
python scripts/ingest_recipes_milvus.py \
  --input assets/full_dataset.csv \
  --outdir data \
  --build-milvus

# This takes ~10-15 minutes for ~13,000 recipes

Alternative: Skip this step and deploy to Streamlit Cloud instead (see Deployment)

Quick Test

Once running, try these commands:

👋 "Hi, what can you do?"
📦 "I have chicken, tomatoes, and pasta"
🔍 "Find me a recipe with chicken"

---

🛠️ Setup Instructions

1. Create Python Virtual Environment

# Create virtual environment
python3 -m venv .venv

# Activate virtual environment
# On macOS/Linux:
source .venv/bin/activate

# On Windows:
.venv\Scripts\activate

2. Install Dependencies

pip install -r requirements.txt

3. Configure Environment Variables

Create a .env file in the project root:

touch .env

Add the following variables:

# Required
OPENAI_API_KEY=your_openai_api_key_here

# Optional - For LangSmith tracing
LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=your_langsmith_api_key
LANGCHAIN_PROJECT=leftovr-app

# Optional - For Milvus/Zilliz Cloud
ZILLIZ_CLUSTER_ENDPOINT=your_zilliz_endpoint
ZILLIZ_TOKEN=your_zilliz_token

4. Set Up Vector Database (Optional but Recommended)

Option A: Zilliz Cloud (Managed Milvus)

1. Sign up at Zilliz Cloud
2. Create a cluster
3. Add credentials to .env:

   ZILLIZ_CLUSTER_ENDPOINT=https://your-cluster.api.gcp-us-west1.zillizcloud.com
   ZILLIZ_TOKEN=your_token_here

Option B: Local Qdrant

Qdrant will be used automatically if Milvus is not configured. No additional setup needed.

5. Ingest Recipe Data

# Ingest recipes into Milvus (preferred)
python scripts/ingest_recipes_milvus.py --input assets/full_dataset.csv --outdir data --build-milvus

# OR ingest into Qdrant (local)
python scripts/ingest_recipes_qdrant.py

---

💡 Usage

Basic Workflow

1. Tell Leftovr What You Have

   "I have 2 chicken breasts, tomatoes, pasta, and garlic"
   

2. Ask for Recipe Suggestions

   "What can I make with these ingredients?"
   "I want something Italian and vegetarian"
   

3. Get Top 3 Recommendations

   • View recipe cards with prep time, servings, and match percentage
   • See ingredient lists and direction previews
   • Select a recipe (1, 2, or 3)

4. Receive Customized Recipe

   • Adapted to your pantry
   • Substitutions suggested for missing ingredients
   • Formatted with step-by-step instructions

Natural Language Commands

Pantry Management

✅ "Add 500g chicken breast, 3 tomatoes, and 1 onion"
✅ "I ate 2 eggs"
✅ "Remove the expired milk"
✅ "Update garlic quantity to 5 cloves"
✅ "Clear my entire pantry"

Recipe Search

✅ "Find recipes with chicken and tomatoes"
✅ "I'm vegetarian and allergic to nuts"
✅ "What can I cook in under 30 minutes?"
✅ "Suggest Italian recipes for beginners"

General Conversation

✅ "How do I dice an onion?"
✅ "What's the difference between sautéing and frying?"
✅ "Tell me about Mediterranean cuisine"

---

🚀 Deployment to Streamlit Cloud

Deploy Leftovr to Streamlit Cloud for free with a cloud vector database!

Prerequisites for Deployment

Before deploying, you'll need:

1. ✅ GitHub Account - Fork/clone this repository
2. ✅ Streamlit Cloud Account - Sign up at share.streamlit.io
3. ✅ OpenAI API Key - Get from platform.openai.com
4. ✅ Zilliz Cloud Account (Recommended) - Sign up at cloud.zilliz.com

Step 1: Set Up Zilliz Cloud (Vector Database)

Zilliz Cloud provides managed Milvus for fast recipe search:

1. Create Account

   • Go to cloud.zilliz.com
   • Sign up (free tier available)

2. Create a Cluster

   - Cluster Name: leftovr-recipes
   - Cloud Provider: AWS/GCP/Azure (any)
   - Region: Choose closest to your users
   - Cluster Type: Starter (free tier) or Standard
   

3. Get Connection Details

   • After cluster is created, click "Connect"
   • Copy the Public Endpoint (looks like: https://in03-xxx.api.gcp-us-west1.zillizcloud.com)
   • Copy the API Key/Token

4. Ingest Recipe Data

   Run this locally before deploying:

   # Set environment variables
   export ZILLIZ_CLUSTER_ENDPOINT="your_endpoint_here"
   export ZILLIZ_TOKEN="your_token_here"
   export OPENAI_API_KEY="your_openai_key"
   
   # Ingest recipes to Zilliz Cloud
   python scripts/ingest_recipes_milvus.py \
     --input assets/full_dataset.csv \
     --outdir data \
     --build-milvus

   This will:

   • ✅ Create embeddings for ~13,000 recipes
   • ✅ Upload to your Zilliz Cloud cluster
   • ✅ Generate metadata files in data/

Step 2: Prepare Your Repository

1. Fork or Push to GitHub

   # If not already on GitHub
   git init
   git add .
   git commit -m "Initial commit"
   git remote add origin https://github.com/your-username/leftovr-app.git
   git push -u origin main

2. Commit Required Files

   Ensure these files are in your repo:

   ✅ streamlit_app.py
   ✅ main.py
   ✅ requirements.txt
   ✅ agents/
   ✅ database/
   ✅ mcp/
   ✅ data/ingredient_index.json
   ✅ data/recipe_metadata.jsonl
   

3. Update requirements.txt

   Make sure requirements.txt includes:

   streamlit>=1.28.0
   openai>=1.0.0
   langchain>=0.1.0
   langgraph>=0.0.20
   pymilvus>=2.3.0
   qdrant-client>=1.7.0
   sentence-transformers>=2.2.0
   python-dotenv>=1.0.0
   inflect>=7.0.0
   pydantic>=2.0.0

Step 3: Deploy to Streamlit Cloud

1. Go to Streamlit Cloud

   • Visit share.streamlit.io
   • Sign in with GitHub

2. Create New App

   • Click "New app"
   • Select your repository: your-username/leftovr-app
   • Branch: main
   • Main file path: streamlit_app.py
   • App URL (optional): leftovr or custom name

3. Configure Secrets

   Click "Advanced settings" → "Secrets"

   Add your secrets in TOML format:

   # Secrets for Leftovr App
   
   # OpenAI API Key (REQUIRED)
   OPENAI_API_KEY = "sk-proj-xxxxxxxxxxxxxxxxxxxxx"
   
   # Zilliz Cloud Credentials (REQUIRED for recipe search)
   ZILLIZ_CLUSTER_ENDPOINT = "https://in03-xxx.api.gcp-us-west1.zillizcloud.com"
   ZILLIZ_TOKEN = "your_zilliz_token_here"
   
   # Optional: LangSmith for debugging
   LANGCHAIN_TRACING_V2 = "true"
   LANGCHAIN_API_KEY = "ls__xxxxxxxxxxxxxxxx"
   LANGCHAIN_PROJECT = "leftovr-production"

4. Deploy!

   • Click "Deploy"
   • Wait 2-3 minutes for initial build
   • Your app will be live at https://your-app-name.streamlit.app 🎉

Step 4: Accessing Secrets in Code

Streamlit Cloud secrets are automatically available via st.secrets:

import streamlit as st
import os

# Access secrets (already implemented in streamlit_app.py)
os.environ["OPENAI_API_KEY"] = st.secrets.get("OPENAI_API_KEY", "")
os.environ["ZILLIZ_CLUSTER_ENDPOINT"] = st.secrets.get("ZILLIZ_CLUSTER_ENDPOINT", "")
os.environ["ZILLIZ_TOKEN"] = st.secrets.get("ZILLIZ_TOKEN", "")

Step 5: Verify Deployment

Once deployed, test these features:

1. ✅ Pantry Management

   "I have 2 chicken breasts, tomatoes, and pasta"
   

2. ✅ Recipe Search

   "Find recipes with chicken"
   

3. ✅ View Inventory

   "What's in my pantry?"
   

Deployment Checklist

• Zilliz Cloud cluster created
• Recipe data ingested to Zilliz
• Repository pushed to GitHub
• Streamlit Cloud app created
• Secrets configured (OPENAI_API_KEY, ZILLIZ credentials)
• App deployed successfully
• Test all features

Production Considerations

Performance

• Vector DB: Use Zilliz Cloud (managed Milvus) for best performance
• Caching: Streamlit automatically caches recipe searches
• Rate Limits: Be aware of OpenAI API rate limits

Cost Management

• Zilliz Cloud: Free tier available (1M vectors, 1 cluster)
• OpenAI: Pay-per-use (GPT-4o: ~$0.01-0.03 per conversation)
• Streamlit Cloud: Free tier includes unlimited public apps

Security

• ✅ Secrets stored securely in Streamlit Cloud
• ✅ Never commit .env or API keys to GitHub
• ✅ Use environment-specific secrets for dev/prod

Monitoring

Enable LangSmith for production monitoring:

# In Streamlit secrets
LANGCHAIN_TRACING_V2 = "true"
LANGCHAIN_API_KEY = "your_langsmith_key"
LANGCHAIN_PROJECT = "leftovr-production"

View traces, errors, and usage at smith.langchain.com

Updating Your Deployment

# Make changes locally
git add .
git commit -m "Update feature X"
git push origin main

# Streamlit Cloud automatically redeploys!

Custom Domain (Optional)

Streamlit Cloud supports custom domains on paid plans:

1. Go to app settings
2. Click "Custom domain"
3. Add your domain (e.g., leftovr.yourdomain.com)
4. Follow DNS configuration instructions

---

📁 Project Structure

leftovr-app/
├── agents/                          # AI Agent implementations
│   ├── executive_chef_agent.py      # Query classification & routing
│   ├── pantry_agent.py              # Inventory management (MCP)
│   ├── recipe_knowledge_agent.py    # Hybrid search engine
│   └── sous_chef_agent.py           # Recommendation & adaptation
│
├── database/                        # Data persistence layer
│   └── pantry_storage.py            # SQLite operations
│
├── mcp/                             # Model Context Protocol
│   └── server.py                    # MCP server for pantry operations
│
├── scripts/                         # Utility scripts
│   ├── ingest_recipes_milvus.py     # Ingest recipes to Milvus
│   ├── ingest_recipes_qdrant.py     # Ingest recipes to Qdrant
│   ├── evaluate_rag.py              # Offline RAG evaluation & metrics
│   ├── validate_pantry.py           # Database validation
│   └── clear_pantry.py              # Reset pantry database
│
├── tests/                           # Test suite
│   ├── test_pantry_agent_comprehensive.py  # Full pantry agent test suite (26 tests)
│   └── test_hybrid_search.py               # Hybrid search tests
│
├── data/                            # Recipe metadata
│   ├── ingredient_index.json        # Ingredient mappings
│   └── recipe_metadata.jsonl        # Recipe database
│
├── assets/                          # Raw datasets
│   └── full_dataset.csv             # Recipe dataset
│
├── qdrant_data/                     # Qdrant vector storage
│
├── main.py                          # LangGraph workflow orchestration
├── streamlit_app.py                 # Streamlit web interface
├── requirements.txt                 # Python dependencies
├── .env                             # Environment variables (gitignored)
├── .env.example                     # Environment template
└── README.md                        # This file

---

🗄️ Database Management

Pantry Database

The pantry inventory is stored in SQLite at ~/.leftovr/pantry.db.

Validate Database

python scripts/validate_pantry.py

Output:

• Current inventory items
• Quantities and units
• Expiration dates
• Items expiring within 3 days

Clear Pantry

python scripts/clear_pantry.py

Manual Database Access

sqlite3 ~/.leftovr/pantry.db
> SELECT * FROM pantry_items;
> .exit

Vector Database

Milvus/Zilliz Cloud

• Collection: recipes
• Embedding Model: sentence-transformers/all-MiniLM-L6-v2
• Dimension: 384

Qdrant (Local)

• Collection: recipes
• Storage: ./qdrant_data/

---

🧪 Development

Running Tests

We have a comprehensive test suite covering all features:

# Run comprehensive pantry agent tests (26 tests)
python tests/test_pantry_agent_comprehensive.py

# Expected output:
# Total Tests: 26
# ✅ Passed: 26
# ❌ Failed: 0
# Success Rate: 100.0%
# 🎉 ALL TESTS PASSED! 🎉

# Test hybrid search
python tests/test_hybrid_search.py

# Validate pantry database
python scripts/validate_pantry.py

What's Tested

The comprehensive test suite covers:

• ✅ Basic Operations (6 tests): Add, remove, update inventory
• ✅ Natural Language (5 tests): Query interpretation
• ✅ Edge Cases (5 tests): "as well", "too", "also", compound names
• ✅ Food Validation (2 tests): Accept food, reject non-food items
• ✅ Multi-Item Operations (2 tests): Batch operations
• ✅ Clarification Flow (1 test): Multi-turn conversations
• ✅ Expiring Items (1 test): Date-based filtering
• ✅ Operations (4 tests): Consumption, deletion, viewing

RAG Evaluation

Evaluate the quality of recipe search and retrieval using scripts/evaluate_rag.py:

# Run evaluation with default settings (20 samples, top-10 results)
python scripts/evaluate_rag.py

# Custom evaluation (50 samples, top-5 results)
python scripts/evaluate_rag.py --sample 50 --k 5

# Save results to custom output file
python scripts/evaluate_rag.py --output my_evaluation.json

What It Measures

The evaluation script uses graded relevance (0-2 scale) instead of binary matching:

• Grade 2 (Highly Relevant): Exact recipe match OR ≥80% ingredient overlap
• Grade 1 (Relevant): 50-80% ingredient overlap
• Grade 0 (Not Relevant): <50% ingredient overlap

Key Metrics

User-Centric Metrics (Most Important):

• Success@K: Percentage of queries where user finds ANY usable recipe in top-K results
  • ✅ Excellent: ≥90%
  • ✓ Good: 70-89%
  • ⚠️ Fair: 50-69%
  • ❌ Poor: <50%
• Best Relevance MRR: Mean Reciprocal Rank of first highly relevant result
• Avg Best Rank: Average position of first grade-2 result

Quality Metrics:

• nDCG (graded): Normalized Discounted Cumulative Gain with 0-2 relevance scale
• Avg Relevance: Mean relevance grade across top-K results (scale: 0.0-2.0)
• Avg Ingredient Match: Percentage of query ingredients used by retrieved recipes
• Precision@K: Proportion of relevant results (grade ≥1) in top-K

Traditional Metrics (For Comparison):

• Exact Match MRR: Mean Reciprocal Rank of the exact original recipe
• Found in Top-K %: Percentage of queries where exact recipe appears in top-K

Sample Output

📊 EVALUATION SUMMARY (V3 - Advanced Metrics)
====================================================================

📌 Traditional Metrics (Exact Match Only):
   Found exact recipe in top-10: 85.0%
   MRR (exact): 0.456
   Avg Rank (exact): 3.2
   Median Rank (exact): 2

✨ User-Centric Metrics (Graded Relevance):
   Success@10: 95.0% ⭐
      → User finds ANY usable recipe in top-10
   Best Relevance MRR: 0.712
   Avg Best Rank: 1.8
      → Position of first highly relevant result

📈 Result Quality:
   nDCG (graded): 0.823
   Avg Relevance: 1.45 / 2.0
   Avg Ingredient Match: 68.5%
   Precision@10 (relevant): 72.0%

💡 Interpretation:
   ✅ EXCELLENT: Users find usable recipes 95% of the time

Why This Matters

Traditional RAG evaluation often measures exact match only (e.g., "did we retrieve the exact recipe?"), but for cooking:

• Multiple recipes can satisfy a query: If you have chicken and tomatoes, many different chicken-tomato recipes work
• Ingredient overlap matters more: A recipe using 8/10 of your ingredients is highly useful even if it's not the "original" recipe
• User success is what counts: Finding ANY good recipe is success, not just finding one specific recipe

The graded relevance approach better reflects real-world usage patterns.

CLI Testing Mode

For development and debugging, you can run the workflow in CLI mode:

python main.py

Note: This runs a hardcoded test conversation. Use the Streamlit interface for production.

Adding Dependencies

# Install new package
pip install package-name

# Update requirements.txt
pip freeze > requirements.txt

LangSmith Tracing

Enable LangSmith for workflow debugging:

LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=your_langsmith_key
LANGCHAIN_PROJECT=leftovr-app

View traces at smith.langchain.com

---

🔧 Troubleshooting

Common Issues

"Recipe search is not available"

Cause: Vector database not set up or recipe data not ingested.

Solution:

python scripts/ingest_recipes_milvus.py --input assets/full_dataset.csv --outdir data --build-milvus

"Could not connect to MCP server"

Cause: MCP server initialization failed.

Solution: Check that mcp/server.py exists and is valid. The app will continue without MCP but pantry operations may be limited.

"OpenAI API Error"

Cause: Invalid or missing API key.

Solution:

1. Verify OPENAI_API_KEY in .env (local) or Streamlit secrets (cloud)
2. Check API key is valid at platform.openai.com
3. Ensure you have credits/billing enabled

Slow Recipe Search

Cause: Using Qdrant (local) instead of Milvus (cloud).

Solution: Set up Zilliz Cloud for faster semantic search:

• Sign up at cloud.zilliz.com
• Add credentials to .env (local) or Streamlit secrets (cloud)
• Re-run ingestion script

Deployment-Specific Issues

Streamlit Cloud: "ModuleNotFoundError"

Cause: Missing dependencies in requirements.txt

Solution:

1. Check requirements.txt includes all packages
2. Push updated requirements.txt to GitHub
3. Reboot app in Streamlit Cloud

Streamlit Cloud: "Secrets not found"

Cause: Secrets not configured in Streamlit Cloud

Solution:

1. Go to app settings → Secrets
2. Add all required secrets in TOML format
3. Reboot app

Streamlit Cloud: App takes too long to load

Cause: Heavy dependencies or large data files

Solution:

• Use @st.cache_data for expensive operations (already implemented)
• Ensure recipe metadata files are committed to repo
• Consider upgrading to Streamlit Cloud paid tier for more resources

"Permission denied" on pantry database

Cause: Multiple instances trying to access SQLite

Solution:

• Streamlit Cloud: Each user gets their own session (should work)
• Local: Close other running instances
• Check file permissions on ~/.leftovr/pantry.db

Zilliz Cloud connection timeout

Cause: Network issues or wrong endpoint

Solution:

1. Verify ZILLIZ_CLUSTER_ENDPOINT is correct
2. Check cluster is running in Zilliz dashboard
3. Verify token hasn't expired
4. Try regenerating token in Zilliz dashboard

---

📚 Additional Resources

• LangGraph Documentation: langchain-ai.github.io/langgraph
• Streamlit Docs: docs.streamlit.io
• Milvus Documentation: milvus.io/docs
• Model Context Protocol: modelcontextprotocol.io

---

📄 License

See LICENSE file for details.

---

🙏 Acknowledgments

• Recipe dataset from [source/attribution]
• Built with LangChain & LangGraph
• Powered by OpenAI GPT-4o

---

Made with ❤️ and 🍽️

Reduce food waste. Cook delicious meals. Enjoy your leftovers.