ClickHouse

Sydney, we're coming to you 🇦🇺 ClickHouse is teaming up with Vercel for AI SYD on Thursday, October 8 at the AWS Sydney office, and the lightning talk lineup is a good one: 🔐 Patrick Collins (CEO, Dam Secure) planted real CVEs across 16 repos to find out which AI model actually catches vulnerabilities. Public benchmarks are saturated, so his team built their own eval suite from open-source commit history. The winning model? A surprise. 📊 Muhammad Ali (Senior Solution Architect, ClickHouse) presents "From Traces to Trust: Shipping Reliable LLM Apps with Langfuse." Building an LLM demo is easy; running one in production is not. Expect tracing for agent pipelines, evals that catch regressions before users do, and real traces from a live app. Plus pizza and networking with Sydney's AI engineering community, from early-stage startups to global enterprises. 📅 Thursday, Oct 8 | 6:00–9:00 PM 📍 AWS Sydney, Level 37, 2/26 Park St 🎟️ Approval-based registration, request your spot: https://buff.ly/XhAT0tf #ClickHouse #AI #Sydney #LLM #Langfuse #Vercel #Meetup

Segment evaluation at Klaviyo used to take over an hour. It now takes a second. They rebuilt the engine on a 192-node ClickHouse cluster, retiring a Python system layered over MySQL and Cassandra. 🔹 ~170,000 companies, billions of events ingested daily 🔹 Tens of billions of segment membership changes computed daily across millions of segments 🔹 Sharding happens twice, first by company ID and then by profile ID 🔹 Around 60% of segments filter on time, which is where most of the tuning went 🔹 An entity-attribute-value materialized view puts property names in the sort key so conditions filter cheaply https://buff.ly/YclzpG6

WalShadow is probably one of the most ambitious projects we’ve taken on. When we decided to build this six months ago, we genuinely weren’t sure how it would pan out. We were going directly after Postgres’s physical write-ahead log, an area backed by decades of deep engineering expertise. Now, seeing the early results with customers, especially Postgres → ClickHouse replication latency in the hundreds of milliseconds, it feels totally worth it. This wouldn’t have been possible without the team’s deep expertise in Postgres replication, the operational lessons we’ve learned onboarding thousands of customers to Postgres CDC, and our experience deep in the Postgres replication and backups, creating and contributing to WAL-G and WAL-RUS I’m really excited for more customers to try it out. And perhaps the most exciting part: this brings ClickHouse much closer to becoming a physical analytics standby for Postgres. 🚀 https://lnkd.in/gnYvjfMt ClickHouse

Is ClickHouse becoming a universal, multimodal DBMS? 🤔 That question came up during the ClickHouse 26.9 release call. Alexey Milovidov had plenty to say, but you’ll have to watch the recording for his answer. Also featured: 🔹 New LIMIT AFTER/UNTIL syntax for navigating ordered data 🔹 Incremental refreshable materialized views, including replication into Iceberg 🔹 PromQL, AI functions, and On-Demand Compute https://buff.ly/wNP29Tk

ClickHouse just released their official Apache Airflow provider! They've been great partners - we use Clickhouse a bunch to power our product, and it turns out they use Airflow to power theirs! My favorite part: > We run the same pattern ourselves. Our internal data warehouse is built on ClickHouse Cloud with Airflow scheduling the insert jobs across 76 DAGs across 40+ data sources, moving around 6 billion rows a day. The entire company relies on it, from leadership reviewing weekly metrics to product, sales, and support teams answering day-to-day questions, and increasingly the agentic workflows we're building on top of our own data. Airflow is the component that keeps it all fed. Thanks Aditya C. Bentsi Leviav and Alex Francoeur for the great writeup, posting the link in the comments.

Big week for Postgres at ClickHouse ! 🚀 We shipped 3 major launches, all centered around our vision of providing developers with one integrated stack of Postgres and ClickHouse: WalShadow - Postgres to ClickHouse replication directly from the physical WAL, with sub-second latency. https://lnkd.in/guN5-tKA ClickHouse Managed Postgres on GCP - one of our most requested items, with an already overwhelming waitlist! https://lnkd.in/gqQJWtaN chDB Postgres extension - fast object storage imports and exports directly using COPY. https://lnkd.in/gJuUtRPs Many months of hard work went into these launches. Huge kudos to the team! 2 of the 3 - WalShadow and the chDB Postgres extension, are fully open source: https://lnkd.in/gnYvjfMt https://lnkd.in/g88_6KZH Lots more coming. Excited for everyone to try what we’re building for Postgres at ClickHouse! 🐘

If you're logging from .NET and want to query those logs with plain SQL, you can send them straight to ClickHouse from Serilog. The Serilog.Sinks.ClickHouse package adds a sink where you define the schema yourself: typed columns for timestamp, level, and message, a JSON column for structured properties, full-text indexes on the message, and your own TTL and ordering. Correlation-ID middleware ties requests together across log lines. The sink batches events before writing, 10,000 per batch by default. https://lnkd.in/ect-4XcG

This month’s featured community member is Rory Shanks from PostHog 🎉 Rory helped shape ClickHouse 26.8 with improvements to mutations, text-index lookups, and JSON subcolumn reads. You can learn more about his work in our September newsletter, alongside the month’s biggest product updates:  🔹 ClickHouse 26.8 and its new streaming API capabilities  🔹 On-Demand Compute for offloading intensive queries  🔹 CostBench’s performance-per-dollar results  🔹 PromQL and the new TimeSeries table engine  🔹 The latest developments in the Postgres ecosystem Featuring contributions from Mohamed Hussain Shahul Hameed, Tom Schreiber, Lionel Palacin, Sai Krishna Srirampur, Kunal Gupta, David Wheeler, Gülçin Yıldırım Jelínek, Kaushik Iska, Cristina Albu, Yashpreet Bathla, James Cunningham, Himanshu Pandey, Andriy Yakovlev, George Larionov, Pete Hampton and M. Lareb Zafar. Thanks to all of you! 🙏

Conference season is officially open! Last week, ClickHouse’s Gülçin Yıldırım Jelínek kicked things off with two talks at two conferences in the Netherlands. At PGDay Lowlands in Utrecht, organized by PostgreSQL Europe, she presented “Analytics Without Leaving Postgres,” a lightning talk about pg_clickhouse and pg_stat_ch, two open-source extensions that connect PostgreSQL and ClickHouse. The following day at Percona Live in Amsterdam, she covered what’s new with monitoring in PostgreSQL 19, including new logging options, WAL and I/O statistics, WAIT FOR, and new system views. In her latest post, Gülçin shares highlights from both conferences and notes from other sessions she attended. https://lnkd.in/eBkH88WG

What a great evening in Amsterdam! 🚀 Together with Adyen, we hosted a fantastic meetup and with around 300 people joining us, the energy was incredible. At Adyen's beautiful headquarters, Iftach Arbel, Senior AI Research Engineer at Adyen, gave us a fascinating look at how Adyen is putting AI theory into practice, using AI to reduce manual work and make people more effective. Next up was ClickHouse CTO & Co-founder Alexey Milovidov, who delivered a masterclass on what modern data platforms need to handle the accelerating pace of data volumes, real-time requirements, and AI - sharing how ClickHouse itself is built to query billions of rows in milliseconds, the kind of performance that's becoming table stakes for AI-driven applications. Two very different perspectives, but one common theme: how technology and the right data infrastructure can help organizations move faster and work smarter. A huge thank you to Iftach, Alexey our two great hosts Kwok He Chu and Tyler Hannan, Adyen and, of course, everyone who joined us and made the evening such a success. 🙌 Already looking forward to the next one! 👀 Kwok He Chu Iftach Arbel Arno van Driel Yury Izrailevsky Daniel Madalitso Phiri Femke Dullaart-Tokkie Jeroen Berckenkamp Willie Engelbrecht