Launching Hasura DDN: The world's first metadata-driven data access layer

Hasura DDN has been in development for over a year and it’s finally out in the wild!

Of course, there are some major engineering enhancements to the underlying compiler and runtime that powers Hasura. (Yes, we’re a Rust shop now. Yes, it’s even more performant than Hasura v2 which was already the fastest GraphQL API engine in the world.)

But the biggest evolution in Hasura DDN has been to surface a core product capability that was latent in Hasura v1 and v2. And it was this product capability that was driving successful adoption and business value for Hasura users...

To introduce Hasura DDN to you, I’m going to first talk a little bit about the rising importance of metadata especially for real-time systems. Next, I'll share our learnings about the value of metadata-driven data access from Hasura users and customers. And finally, I'll show you how Hasura DDN makes real-time data access easy and governed, with metadata at its core.

Over the last few years, API endpoint creation has become easier than ever before. While this has spurred innovation, the inevitable API sprawl has dramatically increased the future state cost of implementing data governance. The lack of visibility in how data is accessed and operated causes immediate compliance challenges, and longer-term challenges that affect federation/decentralization and collaboration.

This is where metadata comes in.

Seasoned tech leaders and architects recognize that metadata (and metadata registries) can be a solution to these governance challenges. They act as passive inventories or dictionaries – useful, sure, but not nearly enough to tackle the complex governance issues tied to data access. While the analytical world has seen an explosion of data governance tooling...

...there is still a marked absence of standardized tooling and practices when it comes to "data use" and for the real-time ecosystem.

Experienced engineering teams choose Hasura because:

Hasura is a metadata-driven data access layer.

More specifically, Hasura DDN is the world's first "cloud native" metadata-driven data access layer that provides APIs to support transactional and analytical workloads alike.

That's a fancy way of saying, if you give Hasura the metadata (description of resources, methods, relationships and AuthZ policies) then Hasura automatically provisions and delivers a low-latency, high-concurrency API that you can instantly start using to integrate data into apps, microservices, AI assistants/agents.

Developers hate tools that slow down their ability to "ship." A simple governance task like documentation (a part of metadata) often gets thrown aside, because it is extra work that doesn't contribute to being productive while building APIs.

Hasura flips this equation by allowing developers to create their APIs directly via metadata, instead of having to create metadata after building their APIs.

Hasura improved our software delivery times by putting a higher emphasis on data modeling and database design.

Working out permissions on the basis of each individual function gets quite complex. So being able to do that in a central location with Hasura is a killer feature.

We’re seeing that it takes only a tenth of the time it originally took to develop a new page in our application or create a new component after having adopted Hasura.

With Hasura, P95 latency improved from 2.5s to 150ms – a 15x speed improvement.

Hasura's tooling helps speed up the creation of metadata and hence the final API, helping developers ship faster. Creating metadata with Hasura is faster than writing code for API controllers that securely map to underlying domain objects or methods.

Hasura introspects databases and code and automatically derives the metadata from the underlying domain. This metadata can be conveniently edited and a well-documented, standardized, production-grade API is instantly available. This powers a world class developer experience for building APIs and best of breed CI/CD practices, resulting in massive time savings for developers.

For technology leaders, Hasura guarantees that metadata is always maintained and up to date. This is a historic first when building real-time data access layers (data APIs) because metadata doesn’t need to be maintained out of band.

Hasura's metadata captures semantic information and Hasura's API engine captures usage statistics critical for collaboration, API design and evolution, audit, compliance, quality, and security.

This forms the bedrock for executing a governance and collaboration strategy, for an engineering team that prides itself on shipping velocity.

It’s no surprise some of the largest global banks and healthcare companies run Hasura to provide an interface to their data products. (Nine of the Fortune 20 companies use Hasura).

In the age of AI, startups and enterprises alike are accelerating their transition to becoming data companies with business services attached. Being able to monetize data and provide higher quality services is fuel for growth. A metadata-driven approach.

Quoting from this recent article by Goldman Sachs on The Deflation of Software:

If data gravity within SaaS applications persists, then SaaS businesses may persist for a long time – even if their products become increasingly "wrapped" with generative interfaces and the businesses look increasingly like data warehouses.

As an industry, as we start to roll out and embrace AI, it’s becoming apparent that every data and API engineering team needs to start thinking about governance, with flexible and secure data access upfront. The metadata problem is no longer just restricted to regulated industries like healthcare and financial services – or analytical workloads.

Our next major version release, v3 – the Hasura Data Delivery Network (DDN) – fully embraces these ideas.

Being metadata-driven has been a key building block of Hasura since its inception, and with Hasura DDN we bring it to the forefront!

Hasura DDN helps developers shift their focus from a lower-level abstraction of working at an API-endpoint level, to working at a higher level of abstraction of metadata. This gives them the time to focus on improving their data and business logic.

Hasura DDN helps teams and businesses govern how their data is used and to make sound decisions about investing in their data products.

So let's take a look at what makes Hasura DDN's metadata-driven API engine powerful.

• 1. Hasura DDN metadata == API schema metadata + data catalog metadata
• 2. Unified metadata for heterogeneous data workloads - SQL, NoSQL, Events, APIs
• 3. Standardized data access APIs
• 4. Out-of-the-box API portal & documentation
• 5. Granular data usage analytics (who's using what data how much?)

Data access layers that are accessed over APIs need straddle two worlds at the same time:

• 1. A method oriented API world

  and

• 2. A resource oriented database world

Hasura DDN has a metadata format, a concise specification that brings both worlds together.

• API metadata tends to be highly method oriented because each API endpoint is an independent business method.
• On the other hand, data catalog metadata tend to be highly table oriented because each table has a standardized way to access with a query language like SQL.

Hasura DDN allows mapping a domain model to an underlying physical model that can come from any type of data system.

Hasura DDN creates and serves a standardized API based on the metadata provided.

Hasura DDN's console (the control plane UI), helps API and data consumers onboard on to the APIs. It uses metadata to automatically create ERD diagrams, an API testing portal, relationship graphs, and other semantic information.

Click through on the walkthrough of DDN console below to get a feeling for how metadata helps API consumers.

Hasura DDN helps teams understand and measure the value of their data, by getting granular insight into who's using what bits of their data.

Click through on the walkthrough of the DDN console below to see how metadata is annotated and contextualized with actual usage statistics.

Join us for a live session with Ken Stott on September 10, 2024 at 9am PT to hear about how he used Hasura to build a federated GraphQL API on top of data products at a Top 3 U.S. Bank.

Save your seat today!

The fastest way to learn about Hasura DDN is to get started and build something. In just a few minutes you can test Hasura DDN working against data and business logic. Head to the Quickstart in our docs to get going!

To read more about the technology architecture and the relevant specifications:

• 1. Read more about the metadata on Hasura DDN metadata - Modelling a supergraph
• 2. Read more about the architecture pattern at https://supergraph.io
• 3. Check out the standardized composable data API at GraphQL data specification. The much awaited gRPC and JSON:API specifications are coming out soon too! Subscribe to the github issues linked to stay notified.