AI Chatbot Build Guide: From Requirements to Production

Building a production AI chatbot takes 8-12 weeks and $40K-$100K for a focused use case with existing documentation. The build has six phases: requirements, architecture, RAG pipeline, guardrails, testing, and deployment. Skip any phase and you ship a demo, not a product.

Phase 1: Requirements (Week 1-2)

Define before writing code:

Use Case Scope

• Who uses it? Customers, employees, or both?
• What questions? List top 50 real questions from support tickets, sales calls, or internal Slack
• What actions? Read-only Q&A, or write actions (create tickets, book meetings, update records)?
• What languages? Single or multilingual?
• What channels? Website widget, Slack, Teams, mobile app?

Success Metrics

Metric	Target
Answer accuracy	> 80%
Autonomous resolution rate	> 60% (for support bots)
User satisfaction (CSAT)	> 4.0/5.0
Escalation rate	< 30%
Response latency (P95)	< 5 seconds

Content Inventory

List every document the bot needs access to:

• Help center articles
• Product documentation
• FAQ pages
• Internal wikis
• Policy documents
• Past support ticket resolutions

Run the RAG knowledge base checklist against this inventory before building.

Phase 2: Architecture (Week 2-3)

Recommended Stack

Layer	Options
Frontend	React widget, custom UI, or platform (Intercom, Zendesk)
Backend	Python/FastAPI or Node.js
LLM	GPT-4o, Claude 3.5 Sonnet (primary), GPT-4o-mini (fallback)
Vector DB	pgvector, Pinecone, or Weaviate
Embedding	OpenAI text-embedding-3-small or Cohere
Hosting	Vercel, AWS, or Railway

Architecture Diagram

User → Chat UI → API Gateway → Chat Orchestrator
                                    ↓
                              Query classifier
                              (simple vs. complex)
                                    ↓
                              RAG retrieval ← Vector store ← Document pipeline
                                    ↓
                              LLM generation (with context + history)
                                    ↓
                              Output validation + guardrails
                                    ↓
                              Response + citations → User
                                    ↓
                              Logging + analytics

Phase 3: RAG Pipeline (Week 3-5)

Document Processing

1. Crawl or export source documents
2. Clean and normalize text
3. Chunk (512 tokens, 50-token overlap)
4. Generate embeddings
5. Store in vector database with metadata

Retrieval Tuning

• Hybrid search (vector + BM25)
• Top-K: 5 chunks (adjust based on eval)
• Re-ranking with cross-encoder
• Metadata filters by document category
• Similarity threshold: reject matches below 0.7 cosine similarity

Response Generation

System prompt structure:

You are [Bot Name], a helpful assistant for [Company].
Answer ONLY using the provided context. If the context does not
contain the answer, say "I don't have information on that" and
offer to connect the user with a human.

Context: {retrieved_chunks}
Conversation history: {last_5_messages}
User question: {query}

Always include source citations in responses.

Phase 4: Guardrails (Week 5-6)

Input Guardrails

• Prompt injection detection (pattern matching + classifier)
• Off-topic rejection (“I can only help with [topic] questions”)
• PII detection in user input (warn or redact)
• Rate limiting per user/session

Output Guardrails

• Hallucination check: verify claims against retrieved context
• Format validation: structured outputs where needed
• Toxicity filter
• Confidence threshold: escalate to human if retrieval score is low
• Max response length

Escalation Paths

Define when the bot hands off to humans:

• User explicitly asks for human
• Bot confidence below threshold (2 consecutive low-confidence answers)
• Sentiment detection: user frustration
• Topic outside scope
• Action requests the bot cannot perform

Phase 5: Testing (Week 6-8)

Run the full LLM evals production checklist:

• 80+ test cases (core, edge, adversarial)
• Accuracy > 80% on core cases
• Hallucination rate < 5%
• Latency P95 < 5 seconds
• Load test: 50 concurrent users

Beta test with 20-50 real users for 2 weeks. Collect every failure and add to test suite.

Phase 6: Deployment and Optimization (Week 8-12)

Launch Checklist

• Production infrastructure provisioned
• Monitoring and alerting configured
• Analytics tracking (questions, resolution, satisfaction)
• Human escalation workflow tested
• Documentation for support team
• Rollback plan documented

Post-Launch Optimization

Week	Action
1-2	Monitor all conversations, fix top 10 failure patterns
3-4	Add missing content to knowledge base, re-index
5-8	Optimize prompts based on conversation data
9-12	Add new capabilities (actions, integrations) based on usage

Compare build approaches in custom AI app vs. wrapper.

Want a production chatbot built for your use case? TopAhead’s AI Software service delivers chatbots with RAG, guardrails, evals, and monitoring.

FAQ

Should we build custom or use Intercom Fin / Zendesk AI? Use platform AI if your docs are in that platform and accuracy requirements are moderate. Build custom for proprietary data, complex workflows, or accuracy above 85%.

How much does a production chatbot cost to run? $500-$3,000/month depending on query volume, model choice, and infrastructure.

Can the chatbot learn from conversations? Yes. Log failed queries, add answers to knowledge base, and re-index. Do not fine-tune on raw conversations without curation.

How do we handle multiple languages? GPT-4o and Claude handle 20+ languages natively. Ensure knowledge base content exists in each supported language.

What about voice/chatbot hybrids? Add voice via Whisper (speech-to-text) and TTS (text-to-speech) layers. Same RAG backend, different input/output interfaces.