The next era of databases are serverless, adaptive, and collaborative

First, let’s get real. Before we dive into where we believe databases are heading, let’s take a step back and level set on where we are today. If you’re operating an application at any type of scale and have business critical data in it, you’re likely terrified to make changes to your production database. And rightfully so. It’s a time intensive process to backfill, many things can generally just go wrong and there are downstream impacts to your application, non-primary data stores and external services. As a result, you are likely to choose to only make additive changes, never renaming or modifying columns, until the schema becomes a weird puzzle that mirrors the evolution of your company and application.

Take yourself out of reality for just a second. Doesn’t this feel wrong? You shouldn’t be afraid to make real changes to your database in production. You and your team should only be worrying about one thing, building the best and most efficient application for your users.

The story feels as old as time, or at least as old as January 1, 1970. Whether it’s a monolithic application built through the waterfall methodology or a modern app crafted by a multitude of micro-services that deploy 100 times a day, you eventually run into the same problem: history does, in fact, repeat itself.

As the needs of your users grow, so do the capabilities required to support their new use cases. Search experiences require different backends for recommendations, natural language and free text search. Event driven applications require a time series data store for real time analytics and workflows. Machine learning and artificial intelligence models vary by use case. The combination of these different requirements result in applications composed of various data services exposed through a wide range of developer experiences, pricing models, and new domain expertise within the team. As your business scales, this complexity grows exponentially and execution tends to slow down. Things get sticky when there’s a lot of glue.

Databases have been around for a while now. The database as we know it today starts in the 1970’s, over 50 years ago. Since that time we’ve seen the birth of the relational database, the rise of NoSQL, the ever persistent lingua-franca of data (SQL), and the evolution from on-premise to “the cloud”. Over the last few years, serverless technologies have taken off and the database is no exception. Data stores that have been around for decades are in the process of or already providing serverless offerings. We’ve shared our thoughts on serverless databases in this blog and now live in a world where you really do not have to worry about configuration, scaling, upgrades or maintenance of your infrastructure. At the scale most applications operate at, things “just work”, production readiness is assumed, and low latency is an expectation.

Alongside the rise of serverless are a few other emerging technologies. Maturing low code platforms and the explosive innovation surrounding generative AI. Building applications is getting easier and every developer is assumed to be a full-stack engineer. As these new offerings level-up the individual developer, the expectations placed upon them are also increasing. With the current state of the industry, smaller teams are not only being forced, but also empowered to be more impactful. If you’re interested why developer experience is important to your business, this recent blog from GitHub provides a great overview to the problem space and shares how generative AI is changing the landscape. Removing friction and providing larger building blocks will allow junior, senior and non-traditional developers to tackle more complex problems faster.

With this perfect storm of serverless infrastructure, an influx of tools to boost developer efficiency, and more people empowered to build applications, the next generation of builders will look a lot different than they do today. The 99% developers will be more generalists. Meaning, they will not care to learn the intricacies of a new data store or service to provide some additional functionality to their application. Most applications eventually need similar flavors of the same things. The database for tomorrow’s applications is multi-model, configurable, and built for developer efficiency. At Xata, we believe a data platform will fuel the next wave of applications - one service that provides a complete data layer and evolves with your application needs.

Now this won’t be easy. Databases are hard, and hard for good reason. Data is at the heart of every businesses, user experience, and use case. The database is the one thing you don’t mess with. Ideally, it’s battle-tested, resilient, highly available and performant. Which is why it’s so hard to make changes to it in production applications. That being said, there are common data problems that don’t quite yet have standardized solutions. Data replication between services, zero downtime schema migrations, and horizontal sharding are just a few examples. These are the types of technical challenges that must be solved in order to make a data platform the go-to-solution for this next generation of application developers. Databases are hard, but the data experience for modern developers needs to be easy and quickly add value, not a time sink or system you’re terrified to touch.

As the non-functional requirements tied to the data layer become solved problems and the harder parts about databases become easier, tools of choice will be selected based on the overall experience. We’ve seen this mental shift occur in many industries, technologies and consumer applications. Data solutions will be chosen based on what they can integrate with, how much more efficient they’ll make your team and how accessible they are to the broader organization. When availability, scalability, security and performance are simply expectations — an Amazon-style customer obsession towards developer and user experience becomes equally as important.

Taking this a step further, there are data paradigms that will never go away. Primarily, the familiarity and flexibility of a tabular data store (spreadsheet) and the logical way to communicate with your data (SQL). These ways to enter, transform, and collaborate with data are understood by most today and commonly taught in school as tools to make you successful, regardless of future profession. As application builders become more generalists, they will gravitate towards solutions that not only meet their application needs, but are also easily understood by their colleagues. Not only because this is a good user experience, but because it makes the team more effective. There is less of a learning curve when accessing your data feels familiar.

Application builders will need a database that meets all of their application needs, is built for developer efficiency and collaboration with everyone, not just your engineering team. At Xata, we believe this is the way the world is heading and we’re building a solution to meet the needs of tomorrow’s builders. Since our launch last November, we’re seeing that our vision is resonating with our community. Folks are excited to not have to worry about bolting on services to their application and that they do not have to sacrifice great development experience for their data layer. We meet them where they are today or where they will be soon.

Looking ahead, you’ll see further investment in both our data platform, making hard data problems at scale feel magical and exposing this all through a non-negotiable, premium developer experience. To see what’s on deck, you can view our roadmap here. If you’d like to chat more about where we’re going, come find us on Discord or book some time to chat.

If you find the types of problems we’re trying to solve interesting and our vision clicks with you, we’re also hiring.