Building a Generative AI Q&A Chat Assistant with Amazon Bedrock

Amazon Finance Automation has taken a significant leap forward in addressing the challenges faced by Accounts Payable (AP) and Accounts Receivable (AR) analysts by developing a generative AI-based question-answer chat assistant. This innovative tool leverages Amazon Bedrock to provide rapid, accurate responses to customer queries, thereby greatly reducing the time analysts spend on case resolutions.

Understanding the Need for Automation

AP and AR analysts at Amazon traditionally engaged in lengthy processes to resolve customer queries. They spent considerable time reaching out to subject matter experts (SMEs) and combing through multiple policy documents and standard operating procedures. The process was inefficient and often led to delays.

Empowering Analysts with a Generative AI Solution

The introduction of a large language model (LLM)-based chat assistant, powered by Amazon Bedrock, revolutionizes how these queries are handled. Analysts can quickly access necessary information, making the entire process more efficient. This article explores how the solution was built and its impact on operational efficiency.

Solution Overview

The solution is anchored on a Retrieval Augmented Generation (RAG) pipeline within Amazon Bedrock, which works by:

• Retrieving relevant documents from a knowledge base.
• Generating a response using a large language model.

The solution’s architecture includes the following components:

• Knowledge Base: Amazon OpenSearch Service is used to store and index finance policy documents. Transitioning to Amazon Bedrock Knowledge Bases could enhance the pipeline’s manageability and extensibility.
• Embedding Model: Utilizes the Amazon Titan Multimodal Embeddings G1 model for accuracy in document embedding.
• Generator Model: A foundation model provided by Amazon Bedrock delivers quick and accurate answers.
• Diversity Ranker: Ensures representation of varied results from the vector index.
• Guardrails: Protects against personal identifiable information (PII) leaks and prompt injection attacks using Amazon Bedrock Guardrails.

Evaluation and Improvements

Initially, the chat assistant’s accuracy was at 49%, prompting the development team to undertake an automated performance evaluation approach. By honing several critical components, the team achieved substantial improvements:

Document Semantic Chunking

A new document segmentation strategy utilizing the QUILL Editor, Amazon Titan Text Embeddings, and OpenSearch Service was implemented, raising the accuracy from 49% to 64% by:

• Converting text to a structured HTML document.
• Identifying logical breaks within the document.
• Generating semantic vectors for document segments.

Prompt Engineering

Further improvements involved prompt engineering to address several inefficiencies. Carefully crafted prompts improved the chat assistant’s accuracy from 64% to 76% by:

• Instructing the LLM not to respond when relevant context is absent.
• Encouraging comprehensive responses when needed.
• Integrating a chain-of-thought reasoning approach.

Using Advanced Embeddings

The transition to the Amazon Titan Text Embeddings G1 model resulted in another accuracy boost, from 76% to 86% by enhancing the relevance of context retrieval.

Conclusion

The journey of developing this chat assistant has demonstrated significant advancements in accuracy and efficiency through various strategies, including RAG pipelines and LLMs with Amazon Bedrock. The improvements have culminated in a leap from an initial 49% accuracy to an impressive 86%.

This generative AI Q&A chat assistant exemplifies how effective automation can be in streamlining finance operations processes. The methodologies outlined here serve as a blueprint for developing similar solutions in other areas.

About the Authors

Here are some of the key minds behind Amazon Finance Automation’s innovative solution:

• Soheb Moin: A Software Development Engineer at Amazon, specializes in designing generative AI tools. In his free time, Soheb indulges in travel and chess.
• Nitin Arora: Senior Software Development Manager, brings 19 years of expertise in scalable software solutions. His interests include music and literature.
• Yunfei Bai: Principal Solutions Architect at AWS, designs AI solutions to meet complex business needs. Yunfei enjoys reading and music when not working.
• Kumar Satyen Gaurav: An experienced Software Development Manager, leads teams in big data technologies. He enjoys reading and strategic challenges in chess.
• Mohak Chugh: A Software Development Engineer involved in generative AI and big data projects. Mohak finds joy in music and performing arts.
• Parth Bavishi: Senior Product Manager focusing on AI capabilities. Passionate about mentoring, volleyball, and reading.

Source: https://aws.amazon.com/blogs/machine-learning/how-amazon-finance-automation-built-a-generative-ai-qa-chat-assistant-using-amazon-bedrock/