An AI-powered Retrieval-Augmented Generation (RAG) customer support agent that delivers documentation-grounded answers using multiple LLM providers, conversational memory, and optional voice synthesis.
This project simulates a production-grade technical support system capable of retrieving relevant documentation context, generating structured responses, and optionally delivering concise voice summaries.
Deployed Version (Text-based):
https://hugging-face-voice-customer-support-agent.streamlit.app/
Full Voice Demo (YouTube):
(https://youtu.be/hWOiNcBGIUI)
- Hybrid semantic + keyword retrieval
- Embeddings via
SentenceTransformers (all-MiniLM-L6-v2) - Cosine similarity scoring
- Keyword boosting for procedural queries
- Dynamic context injection into LLM prompt
Supports runtime switching between:
- Groq (LLaMA / Mixtral)
- OpenAI (GPT models)
- Gemini (Flash models)
Enables:
- Provider comparison
- Rate-limit handling
- Model abstraction architecture
- Maintains last N conversation turns
- Injects structured chat history into prompts
- Enables context-aware follow-up responses
- Support-oriented tone
- Structured step-by-step guidance
- Context-restricted generation
- Voice-optimized summarization layer
- ElevenLabs TTS integration
- Concise conversational summary generation
- Production-safe fallback handling
Note: Voice synthesis may be restricted on public cloud deployments due to third-party free-tier API limitations. Full functionality is demonstrated in the video demo.
Frontend
Streamlit
Retrieval Layer
SentenceTransformers (all-MiniLM-L6-v2)
Scikit-learn (Cosine Similarity)
NumPy
LLM Providers
Groq API (LLaMA / Mixtral)
OpenAI API (GPT models)
Google Gemini API
Voice (Optional)
ElevenLabs API (local usage)
Deployment
Streamlit Community Cloud
Pure semantic similarity may miss command-based or procedural queries (e.g., pip install transformers).
To improve retrieval precision, the system combines:
Semantic similarity (embedding-based search)
Keyword boosting (for command-heavy technical queries)
This ensures technical instructions and CLI commands are prioritized correctly.
The application abstracts model providers to:
Avoid provider lock-in
Compare response quality across models
Handle rate limits gracefully
Demonstrate modular LLM architecture
The provider can be switched dynamically at runtime.
Instead of reading the full technical response:
A second LLM prompt generates a concise conversational summary
Commands are referenced rather than read verbatim
The tone remains natural and support-oriented
This improves usability and creates a more human-like interaction model.
The system maintains recent conversation history using Streamlit session state.
This:
Preserves conversational coherence
Enables contextual follow-up questions
Avoids requiring an external vector database
git clone https://github.com/your-username/HuggingFace_voice_customer_support_agent.git
cd HuggingFace_voice_customer_support_agentpython -m venv venvmacOS / Linux
source venv/bin/activateWindows
venv\Scripts\activatepip install -r requirements.txtCreate a .env file in the project root:
GROQ_API_KEY="your_key"
OPENAI_API_KEY="your_key"
GEMINI_API_KEY="your_key"
ELEVENLABS_API_KEY="your_key"streamlit run app.pyPush repository to GitHub
Create a new app on Streamlit Community Cloud
Select the desired branch
Add API keys under Secrets
Deploy
Voice synthesis may be restricted on public cloud deployments due to third-party API limitations.
Replace local documentation file with automated web crawler ingestion
Integrate a persistent vector database (Qdrant / Pinecone)
Add authentication and usage tracking
Integrate browser-based speech recognition
Add retrieval precision evaluation metrics
MIT License