Developing a Retrieval Augmented Generation (RAG) Chatbot App Using Adaptive Large Language Models (LLM) and LangChain Framework

Abstract

CARA BURGAN, Dept of Computer Sciences, Mathematics, and Engineering, Shepherd University, Shepherdstown, WV, 25443, JOSIAH KOWALSKI, Dept of Computer Sciences, Mathematics, and Engineering, Shepherd University, Shepherdstown, WV, 25443, and WEIDONG LIAO (Faculty Advisor), Dept of Computer Sciences, Mathematics, and Engineering, Shepherd University, Shepherdstown, WV, 25443.  Developing a Retrieval Augmented Generation (RAG) Chatbot App Using Adaptive Large Language Models (LLM) and LangChain Framework 

 RamChat is an AI chatbot designed to assist Shepherd University students in navigating the student handbook. Developed in Python, it utilizes both API-based and local Large Language Models (LLMs) for natural language processing (NLP), alongside a vector store system. Our aim is to create a high-quality chatbot app tailored for student use.

We began by researching existing chatbot platforms and created a vector store with embeddings from OpenAI's text-embedding-3-small model, trained on the Shepherd University handbook. Testing each LLM helped assess answer types and accuracy.

Development involved debugging and optimizing RamChat's code, including replacing OpenAI's davinci-002 model with gemma, a local LLM based on Google's Gemini model. Ollama framework aids in automatic LLM selection based on user prompts.

Our conference presentation will detail RamChat's development, methodology, challenges, and insights. RamChat represents an innovative application of AI to enhance the Shepherd University student experience.