Data Systems Artisan

Retrieval Augmented Generation (RAG) has revolutionized how Large Language Models (LLMs) access and utilize external knowledge, moving beyond static training data to deliver more accurate, relevant, and up-to-date responses. While basic RAG setups are powerful, achieving peak performance and addressing complex real-world scenarios often requires a more sophisticated approach. This... [Read More]

In the rapidly evolving world of Large Language Models (LLMs), the ability to run these powerful tools locally on your hardware is likely appealing. Running AI locally grants you unparalleled control, endless customization, and complete data privacy. This guide will walk you through building your personal AI playground, assembling an... [Read More]

The concept of a data lakehouse is gaining serious traction, and for good reason! It combines the flexibility and cost-effectiveness of a data lake with the data management features and ACID transactions of a data warehouse. Let’s dive into building a practical lakehouse setup using some fantastic open-source and cloud-native... [Read More]

Popular data processing platforms offer users the ability to inject an external process into the data processing pipeline. The data flowing through the data pipeline is fed as input to the external process, while the output produced by the process is fed back into the pipeline. The external process runs... [Read More]

In an earlier post, we looked at how to create a Docker image for SciDB. The image built in that post followed the SciDB Community Edition Installation Guide very closely. The image is functional and a good learning resource, but not very efficient. The image uses around 6GB of space... [Read More]