Introduction

For me, language models (LLMs) and artificial intelligence were a mystery. Despite the abundance of information online and the numerous courses promising access to this emerging technology, I felt that something was missing. This year, I decided to participate in the KaggleX competition in the hope of diving into this fascinating world through a program that offers hands-on experience and collaboration in a high-level environment. The competition was everything I expected and more. There were days when I climbed into the top 50, and others when I fell 200 places, facing a true rollercoaster of emotions. I clearly remember the morning I received the acceptance email for the program: it was a mix of euphoria and pride knowing that I would be part of this journey. Currently, I am part of Cohort 4 of KaggleX, where the focus is on developing a chatbot using Gemma_2, Google's advanced language model. This challenge is a unique opportunity to immerse myself in the deep learning of LLMs, explore their potential, and apply it in designing a tool that not only answers questions but also helps manage conflicts in an empathetic and informed manner.

Project: Specialized Chatbot for Law 675 of Colombia

As part of the KaggleX program, I developed a chatbot specialized in Law 675 of Colombia, which regulates horizontal residential properties. This chatbot was designed to provide quick and accurate responses to questions related to the law, offering homeowners and tenants an accessible tool to help them understand and manage their rights and responsibilities within their communities. Law 675 establishes regulations for coexistence in shared residential properties, and my chatbot serves as a virtual legal assistant, guiding users through common topics such as the use of common areas, participation in assemblies, the role of administrators, and rules of coexistence. Through well-founded answers and clear language, the chatbot aims not only to inform but also to reduce internal conflicts and enhance the living experience within these communities. Thanks to LLM technology and Google's Gemma_2 and Gemini models, this chatbot can process complex questions and provide personalized responses based on the user's specific context and current legislation. This allows users to obtain reliable information, providing practical support that facilitates decision-making within the legal framework. This project was not only a challenge in terms of technical learning but also an opportunity to contribute to harmony and well-being in residential communities in Colombia.

The Gemma_2 Model by Google The Gemma_2 model from Google is a compact and powerful tool for developing practical solutions, especially when it comes to analyzing cases and generating responses in well-defined contexts. In my project, I worked with the specific model "gemma_instruct_2b_en" through a notebook on Kaggle (available here: ‘https://www.kaggle.com/code/r3molino/law-675-fine-tunning-ralm/edit’) and used my own dataset, ‘/kaggle/input/training-675-law’, to fine-tune the model with data relevant to Law 675 of Colombia. After the initial results and a technical review with my mentor, I encountered certain significant challenges while using this model:

Short Responses*: The responses generated by the chatbot were too brief to address the complex and detailed context of Law 675. This approach failed to provide the level of guidance needed for users to fully understand their rights and responsibilities.

Token Limit: The prompts required to formulate complete and detailed responses often exceeded Gemma_2's token limit, resulting in incomplete or irrelevant answers in the context of the law.

Integration of Multiple References: To answer complex questions accurately, the model needed to consider several sections of the same law. However, this required additional configuration, as Gemma_2 was not designed to link multiple references automatically within a single context.

These obstacles highlighted the need for a different strategy to develop a robust and reliable chatbot for Law 675. Despite the limitations, working with Gemma_2 allowed me to explore the challenges and potentials of LLMs in legal applications, as well as refine my approach to handling large volumes of textual data and structuring optimized prompts to improve the accuracy and depth of responses.

Graph database

After evaluating the options for training my language model, I decided to use a graph database to create a RAG (Retrieval-Augmented Generation) system. This approach allows for generating the necessary context for the model to provide more precise and relevant responses regarding Law 675. Graph databases offer several advantages in this type of application: they allow for structuring information in a way that reflects the relationships between articles, chapters, and sections of the law, facilitating navigation and retrieval of specific data. This significantly improves the chatbot's accuracy, as it can dynamically access multiple interconnected legal references, generating well-founded and contextualized responses. Additionally, using graphs optimizes response time and allows for scalability that supports complex queries, which is essential for addressing legal topics with depth and clarity.

Example

Imagine a situation where a homeowner, Juan, is in conflict with his neighbor due to the improper use of the common areas of their building. Juan decides to consult the specialized chatbot for Law 675 to seek guidance on how to proceed.

Initial Inquiry: Juan starts the conversation with the chatbot by typing: "What should I do if my neighbor uses the pool outside of designated hours?"

Context Analysis: The chatbot, using the graph database, accesses relevant information about the usage hours of common areas and the regulations established in Law 675. Through the relationships between the articles of the law and the internal regulations of the property, the chatbot understands the context of the inquiry.

Response Generation: The chatbot responds: "According to Article 16 of Law 675, the use of common areas must respect the hours established by the homeowners' assembly. If your neighbor is using the pool outside of these hours, you can contact the property administrator or file a formal complaint with the assembly. I also recommend reviewing the internal regulations of the property, as they may include specific provisions on this matter."

Follow-Up: Juan, satisfied with the answer, decides to ask about the process for filing a formal complaint. The chatbot, again using its graph database, accesses the necessary information and responds with details about the procedure, including deadlines and required documentation.

This example illustrates how the chatbot not only provides specific answers based on Law 675 but also facilitates the understanding of the rights and responsibilities of homeowners. By using a graph database, the system can offer contextualized and well-founded responses, helping users navigate complex situations more effectively.

Rag architecture

The RAG (Retrieval-Augmented Generation) architecture is an approach that combines information retrieval with text generation, making it particularly useful in artificial intelligence applications like chatbots. Instead of relying solely on pre-trained language models to generate responses, RAG incorporates a retrieval component that searches for relevant information in a database or a set of documents before the model generates the final response. This approach enhances the accuracy and relevance of the chatbot's responses by allowing it to access updated and specific information on a given topic. When a user asks a question, the RAG system first identifies relevant text fragments using a search engine. These fragments are then used as context for the text generation model, resulting in more informed and coherent responses. Furthermore, RAG enables chatbots to adapt to different domains and be easily updated by adding or modifying the retrieval database without the need to retrain the entire language model. This not only improves the quality of user interaction but also optimizes the performance and flexibility of the chatbot across various applications.

Gemini flash

In my journey of developing artificial intelligence applications, I have faced various challenges related to the available computational resources. I remember that during my early tests with the GEMMA_2 model with 2B parameters, I quickly hit the limits of the free tier offered on Kaggle. Seeking a more robust solution, I tried the 7B parameter model on the Google Cloud Platform (GCP), but even this model proved insufficient to meet the needs of my Streamlit application. It was then that I learned about Google's announcement regarding the availability of Gemini Flash 1.5, a large language model (LLM) that would be offered for free until the end of the year. This news seemed like the perfect solution to my problems. Connecting my Streamlit application to an API that provided access to the Gemini Flash 1.5 model offered me several key advantages:

Execution Flexibility: By not having to load the heavy model into my local application, I could maintain the agility and performance of my Streamlit solution. Processing Power: Gemini Flash 1.5, with its architecture of 1.5 trillion parameters, provided me with the processing power needed to tackle more complex and demanding tasks. Integration with RAG Architecture: The ability to leverage the Retrieval-Augmented Generation (RAG) architecture of Gemini Flash 1.5 would allow me to enrich user queries with additional information, significantly improving the quality of the responses. By being able to access this state-of-the-art language model through an API, I avoided the computational resource challenges I faced with other models. This allowed me to focus on building a more robust and flexible Streamlit application without compromising performance. In conclusion, my decision to use Gemini Flash 1.5 for free via an API proved to be a smart and visionary strategy. By harnessing the capabilities of this large language model, I was able to enhance my Streamlit application, overcoming the resource limitations I previously encountered. This approach demonstrates how advancements in artificial intelligence, such as the availability of cutting-edge language models, can transform application development and elevate it to new levels of performance and functionality.

After resolving the computational resource challenge by connecting my application to the powerful Gemini Flash 1.5 API, my next step was to create an attractive and functional user interface using Streamlit. Streamlit became the natural choice for me, as it allowed me to quickly build an interactive web application without having to worry about the backend and frontend details. I started by designing the interaction flow of the chatbot, defining how users could submit their queries and receive responses generated by Gemini Flash 1.5. In Streamlit, I created intuitive components like text boxes for users to enter their questions, and buttons that triggered the request process to the API and displayed the responses. Thanks to the flexibility of Streamlit, I could customize the appearance and layout of the interface elements, achieving a clean and modern design that reflected the power of the AI model I had chosen. One of the most satisfying aspects was being able to integrate the RAG architecture of Gemini Flash 1.5 within my Streamlit application. This allowed me to enrich the chatbot's responses with relevant contextual information, significantly improving the quality and usefulness of the interactions. As I developed and tested my application, I was excited to see how users interacted smoothly with the chatbot. Streamlit made it really easy to iterate on the design and functionality, adjusting details until I achieved an exceptional user experience. In retrospect, the decision to leverage Gemini Flash 1.5 through an API, combined with the flexibility and rapid development that Streamlit provided, became the perfect combination for creating my artificial intelligence chatbot. I am truly satisfied with the final result and excited to offer my users an enriching and efficient experience.

Conclusion

The development of my artificial intelligence chatbot based on Law 675 presented various technical challenges that I had to carefully address. Beyond the initial implementation in Streamlit and the integration with the Gemini Flash 1.5 API, I faced significant hurdles in creating the graph database that would support legal knowledge. Building nodes and hierarchizing legal data required a deep understanding of the structure and relationships within Law 675. I explored different architectures, including the Retrieval-Augmented Generation (RAG) approach offered by Gemini Flash 1.5, and compared it with fine-tuning strategies for the model. Each approach presented particular advantages and challenges that I had to carefully evaluate to find the most effective solution. One of the most challenging yet enriching aspects was creating hypothetical questions that would serve as the basis for the chatbot's interactions. This exercise forced me to think beyond the mere application of the law, delving into the field of legal philosophy. I had to reflect on ethical principles, possible interpretations, and hypothetical scenarios that could arise in the application of this regulation. Integrating this philosophical approach with the technical capabilities of Gemini Flash 1.5 and the graph architecture of my database allowed me to develop a truly robust chatbot capable of addressing a wide range of inquiries related to Law 675. Each interaction became an opportunity to explore new perspectives and deepen the understanding of complex legal issues. Ultimately, this project has provided me with valuable experience at the intersection of artificial intelligence, law, and philosophy. I have learned to navigate technical challenges, balance architectural approaches, and cultivate a holistic vision that integrates legal reasoning with critical thinking. I am convinced that this type of AI-based solution can transform how citizens interact with the legal system, democratizing access to information and promoting a greater understanding of our laws and regulations.