Erik Hasse

My "Monday Morning" Checklist for a High-Performance Week: ✅ Validate the Contracts: Did any upstream source change their schema over the weekend? I’m moving toward Code-First Data Contracts to ensure our pipelines fail "loudly" rather than corrupting the Lakehouse quietly. ✅ Audit the Latency: In a world of real-time expectations, "Batch" is becoming a dirty word. I'm looking at our Kafka-to-Iceberg lag. If we aren't under 60 seconds, we aren't ready for real-time AI. ✅ Check the "Cost-per-Query": Monday is the best day to review the weekend's compute spend. If a specific dashboard is costing more than the value it provides, it's time to refactor the SQL. ✅ Lineage Integrity: Can I trace a data point from the Power BI report all the way back to the raw S3 bucket? If there’s a gap in the lineage, there’s a gap in the trust. The Goal for the Week: Let’s stop building "Features" and start building "Trust." A reliable pipeline is the best gift you can give your business stakeholders. What is the first thing you check in your data stack on a Monday morning? Let’s share some best practices! 👇 #DataEngineering #MondayMotivaion #DataStrategy #AIInfrastructure #BigData #DataLineage #ModernDataStack #TechLeadership #CloudArchitecture #DataQuality

1

Vector search is not a silver bullet for RAG. If your agent relies on similarity alone, you're leaving massive gaps in your retrieval. Vector search is brilliant for semantic queries. But entire categories of questions need exact match, structure, aggregation, graphs, and multimodal retrieval. In my latest video, I walk through 9 real-world cases, showing where vector search breaks and the retrieval engineering patterns that make agents reliable in production with n8n. We also share the hybrid retrieval n8n workflows we use. What you will learn: 🔹 Why similarity is not the same as relevance, and how to detect it 🔹 When to use keyword and pattern matching for IDs and error codes 🔹 Structured lookups and SQL for tabular questions 🔹 Reranking and metadata filtering for recency and specificity 🔹 Aggregation by counting and computing across records 🔹 GraphRAG for global patterns and multi-hop reasoning 🔹 Multimodal RAG to retrieve images and diagrams 🔹 Post-processing with tools for calculations and trend analysis 🔹 Handling false premise questions with verification and evals using DeepEval Generic chat with your data is easy. Production retrieval demands engineering. What retrieval challenges are breaking your RAG agents? Link to the full video in the comments 👇

23

5 Comments

I fully support Ali Ghodsi's message: the future of #AI requires a deep understanding of data and enterprise context. It's great to see our quote from Danfoss featured in the attached blog post as a prime example of this in action: "What we’re seeing at Danfoss is that Genie Code changes the roles inside a data team, supporting our strategic focus on digitalization and AI. Data scientists still provide direction and review, but engineers, analysts, and domain experts can now actively work in notebooks with the assistant and contribute to advanced analytics workflows. It turns data science into a much more collaborative team activity." We are using the Databricks platform and the assistant extensively for our #forecasting activities, and are seeing firsthand how it accelerates our daily work by understanding both the code and the data 😉

22

5 Comments

The image below from Pietro Schirano is a perfect example of what "moving upstream" looks like in 2026. He's using Claude Code to analyze a raw SNP array file with 700k+ SNPs. This isn't just a coding demo; it's a shift in the Polygenic Risk Score (PRS) paradigm. Traditionally, calculating PRS or filtering high-impact variants required specialized bioinformatics pipelines like PGScatalog, PRSice, or PLINK. AI is radically lowering the bioinformatics barrier by: → Natural Language Queries: No more complex Grep/Awk scripts to find specific rsIDs like APOE or BRCA1. → Automated Feature Engineering: The tedious work of matching effect alleles and handling Liftover is becoming a conversational task. → Instant Context: The AI cross-references clinical significance across cardiovascular, cancer, and metabolic risk categories in seconds. In practical terms: analyses that once took a bioinformatician days can now happen in minutes. This means faster hypothesis generation and lower barriers for clinicians to interpret patient genomic data. The Next Frontier: From PRS to Automated Variant Calling 🧬 While we are currently using AI to "search" through arrays, the real game-changer will be AI agents performing end-to-end Variant Calling directly from raw NGS data (BAM/CRAM files). Moving from identifying known SNPs to discovering de novo variants via natural language would truly democratize the entire genomic pipeline. The "Real World" Catch: A lower barrier to analysis doesn't solve for Population Stratification. PRS is highly sensitive to genetic background, and AI agents will need to automate the calibration of population-specific weights to ensure accuracy. Deep clinical domain knowledge remains the only safeguard against misinterpreting these automated signals. Two questions I'm wrestling with: Should population stratification warnings be built directly into AI genomic tools (similar to how statistical software flags convergence issues) or does this create false confidence? For those working with diverse cohorts: are we seeing the same accuracy gaps in AI-assisted interpretation as we do in traditional PRS methods? Especially curious to hear from epidemiologists and clinical researchers navigating this transition. #Bioinformatics #Genomics #AI #DeepVariant #DataScience #PRS #ClaudeCode #HealthTech

24

5 Comments