Chatbots & Conversational AI

The idea started simple: bring ChatGPT's capabilities into WhatsApp, the messaging platform where over 2 billion people already spend their time. But the execution was anything but simple. WhatsApp's Business API has strict rate limits, message template requirements, and a 24-hour customer-service window that expires if the user doesn't respond. Building a reliable conversational AI system on top of these constraints required careful architectural decisions that most chatbot tutorials never address.

The product needed to handle multiple conversation types: freeform Q&A (like ChatGPT), image generation from text prompts (like DALL-E), document summarization (users sending PDFs or images of text), and command-based workflows (e.g., "/translate hindi" to switch response language). Each message type required different processing pipelines, different response formats, and different rate-limiting strategies. Group chat support added another layer of complexity — the bot needed to distinguish when it was being addressed versus when it should stay silent, and manage context across multiple participants.

Scalability was a hard requirement from day one. The client anticipated rapid growth through word-of-mouth (WhatsApp's sharing mechanics make viral distribution easy), and needed the system to handle 10,000+ concurrent conversations without degradation. The cost model also needed to work: GPT-4 API calls at scale get expensive fast, so we needed smart caching, model routing (using cheaper models for simple queries), and token management to keep per-user costs under a target threshold.

We built the system on a message queue architecture using BullMQ with Redis as the broker. Every incoming WhatsApp message was immediately acknowledged and placed in a processing queue, decoupling the webhook response from the AI inference. This let us handle traffic spikes gracefully — during peak hours, the queue might build up briefly but no messages were ever lost or timed out. Workers pulled from the queue, classified the message intent (text query, image request, document processing, or command), and routed it to the appropriate handler.

For the AI layer, we implemented a tiered model strategy. Simple queries (greetings, factual questions, translations) were routed to GPT-3.5-turbo for speed and cost efficiency, while complex queries (code generation, nuanced analysis, creative writing) were routed to GPT-4. The routing decision was made by a lightweight classifier fine-tuned on 5,000 labeled message examples. For image generation, we integrated with DALL-E 3 and built a prompt enhancement layer that improved user inputs before sending them to the API — turning "a cat" into "a detailed illustration of a domestic cat in warm lighting, digital art style" to get better results.

Conversation context was managed through a sliding window that kept the last 10 messages per user in Redis, with a summarization step that compressed older context into a system prompt when conversations ran long. This kept token usage manageable while maintaining conversational coherence. For group chats, we implemented mention detection and a "relevance scorer" that determined whether the bot should respond to messages it wasn't directly tagged in. Rate limiting was per-user with configurable tiers, and we built an admin dashboard for managing users, monitoring costs, and viewing conversation analytics.