Creating a support agent with LLMs taught us a lot, and these lessons can help anyone working on something similar. Here's what stood out:

1. Chunking Is the Secret Sauce

   Think of chunking as breaking a big puzzle into manageable pieces. It's not just a technical step, it's the foundation for making your support agent reliable. Each chunk needs to feel complete and easy to understand on its own. Include key elements like headers, footers, and tables to maintain context. Maintaining the integrity of paragraphs is particularly important, as this simple step can significantly reduce the likelihood of errors or ambiguous answers.

2. Give Your Agent the Full Picture

   For your agent to offer helpful answers, it needs to see the bigger picture. Instead of using isolated chunks of information, it is valuable to design a chunking strategy that considers how sections of a document relate to one another. For instance, include nearby chunks when fetching content so the agent can "connect the dots" and deliver a response that makes sense.

   We saw this in action when handling a request for the list of modules in a cohort. Our system uses page-based chunking, meaning each chunk corresponds to a page. In this case, the relevant section about the cohort's modules (Figure 2) was spread across two different pages. To provide a complete answer, retrieving just one chunk wasn't enough — we also needed to fetch the previous or next page to capture all the necessary details. This approach ensures the agent delivers a response that includes the full list of modules rather than an incomplete or misleading answer.

   

   

3. Tackle Tricky Document Layouts Head-On

   Not all documents are created equal. Some have two columns, complex tables, or funky layouts that make extracting information tricky. It is important to develop chunking rules tailored to handle these specific formats while maintaining the natural flow of the content. When done right, this ensures that the agent won't get tripped up by the structure and can deliver accurate, easy-to-follow answers to users.

4. Keep Improving, It's a Journey

   Building a support agent is not a one-and-done deal. As document formats, user queries, or data evolve, so too must the chunking strategy. During our experience, we observed, for instance, that retrieving only the most relevant chunk was sometimes insufficient for answering complex queries. To improve response quality, we expanded our retrieval approach to include the top two most relevant chunks, along with preceding or following ones when applicable. This provided better contextual understanding, especially for multi-page documents.

   Additionally, we optimized embedding generation by implementing asynchronous processing, making API calls to OpenAI embeddings more efficient. This reduced latency and indexing time, ensuring faster responses. To further enhance efficiency, we integrated FAISS indexing and pickle-based caching, allowing us to store and quickly reload precomputed embeddings instead of recalculating them each time.

   These refinements — expanding chunk retrieval, improving embedding generation, and optimizing storage — led to more accurate, faster, and context-aware responses, ensuring the support agent remains effective as content and user needs evolve.

   Regular monitoring and analysis are essential to identify areas for improvement and address recurring issues. This ongoing cycle of optimization ensures the support agent remains reliable, adaptive, and responsive to ever-changing demands.

By integrating these practices, teams can develop more accurate, resilient, and user-friendly support agents, transforming WhatsApp into a powerful, context-aware service channel.