Caitlin Sadowski

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments

It was great to hear Robert Wong Julie Willem Angie McAllister, Ph.D. and Markus Kunesch on the recent episode of the @Ed-Technical podcast -- the first of a three-part series about how we're applying Google AI to our work in education and learning: https://lnkd.in/e7hQi6a2. This episode delves deep into LearnLM, how we're making Gemini tuned for pedagogy and how we're integrating this technology into our mainstream products to improve learning outcomes, and evolve how people learn on Google products. It explains inspiring new product ideas, and the steps we take across the company to understand and enhance the journey of learning in a responsible and innovative way. #LearnLM #AI #Education #Google

67

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

69

3 Comments

Check out the latest in our "Debuggability" blog series! This blog is authored by Kevin Downey Jason Johl and me and talks about how we built a Debuggability Paved Road at Intuit using Ephemeral Containers and Argo Workflows. We'd love to hear your thoughts on the topic. Also, what tools do you build as-a-platform to help debug service, compute and network issues in Kubernetes at scale? #intuit #kubernetes #debug

23

My third essay for Yule LOGIC is about LLMs and temperature: https://lnkd.in/gqutMN-R

10

1 Comment

When you need to play with your data in Gen Ai, you need RAG and Chunking is one of the main component in the RAG framework while consuming or processing your data. Excellent post below describing how in different ways chunking in RAG can be implemented..

23

1 Comment

Just had a great chat with my UC Berkeley research group about a cool whitepaper on how AI models can predict multiple tokens at once! 🤖✨ It’s amazing to see how far these models have come. If you’re into AI, check out the whitepaper here: https://lnkd.in/gre2JxCs 📄🔍 #Research #AI #Tech #MachineLearning #UCberkeley

7

1. IMO, agent patterns are one of the most interesting ways to extend LLMs. memary is based on leveraging a graph database (neo4j) for long-term memory (to get around the limited memory inherent in even the largest context windows in LLMs) and augmenting it with other tools like web search and Computer Vision. https://lnkd.in/gfFcBqJx Looks interesting. 2. More about how consolidation effects the entire healthcare industry: https://lnkd.in/gXJKqNJB The Change hack/ransomware attack not only affected claims, service, refills, etc. but now CMS & NCQA are delaying the entire HEDIS reporting process (used for Medicare/Medicaid quality ratings) due to the impact. 3. NARA bans ChatGPT use agency-wide. https://lnkd.in/gW8T94JU Federal Government agencies see this as a risk, including their information leaking and being used to train models (which may then output the wrong information or show PII/PHI other proprietary information). 4. Oh great, now I can't even be angry anymore :) https://lnkd.in/gcrN9mjx

1

1 Comment

Hey everyone, I am an investor and advisor to Katanemo.... They are reinventing the infrastructure primitives for GenAI and launched Arch - https://lnkd.in/gCrbbkEV - an intelligent prompt gateway. Give the project a look on Github, and star the project if it resonates

88

3 Comments

👋 Today myself, Benji Vaughan and Mark Downey (with more names coming soon) set out on a new journey. I'm thrilled introduce Conduit AI (conduit-ai.com), a new company on a mission to empower SMEs with the incredible potential of generative AI, especially retrieval augmented generation (RAG) applications. But first, let's briefly explain what RAG is and why it's a game-changer for businesses. RAG is a powerful AI technique that combines the knowledge stored in large language models (LLMs) with the ability to retrieve and incorporate relevant information from your own data sources. This means RAG-powered applications can generate highly accurate, context-aware, and up-to-date responses to a wide range of queries. For SMEs, this translates to unparalleled productivity gains, enhanced customer experiences, and a significant competitive edge. 🚀 Now, a bit about our journey. Starting Conduit AI has been an exhilarating leap of faith, fuelled by our passion for AI and the unwavering support of our loved ones and mentors. We couldn't have done it without their wisdom and encouragement. 🙏 Over the past year, we've been on a mission to deeply understand how SMEs can effectively adopt generative AI. We've collaborated with companies across various industries and consistently found: SMEs are excited about the potential of LLMs to boost their productivity and profitability. 📈 There's a strong interest in RAG applications that can generate valuable insights from both unstructured and structured data. 💡 Many SMEs are concerned about developing LLMs in-house due to expertise gaps, reliability, trust, and privacy issues. 🔒 We've learned that SMEs are eager to become AI-enabled but need a trusted partner to help them develop a long-term AI vision and deliver products that bring this vision to life. That's where we come in! 🤝 At Conduit AI, our goal is to be the channel that helps SMEs transition into an AI-enabled world. We'll work closely with you to: - Develop a tailored AI strategy 🎯 - Design an adoption roadmap 🗺️ - Identify and implement low-risk, high-return AI applications 💪 With our team's 15 years of ML/AI/NLP experience, having built AI/ML tools for companies like Zurich Insurance and OVO, we're well-equipped to create AI strategies and applications that are tailored to your unique vision, team, and culture. 🌟 We can't wait to share more about our growing team and the incredible stories of the clients we'll be working with. If you're curious about how AI can transform your business, let's chat! 💬 Thanks for being a part of our exciting journey! 🙌 #AIConsulting #TechStartup #SMEs #FutureOfWork #RAG #newventure

45

9 Comments

Recently I learned that LLMs show ability to acquire rare linguistic competence (e.g. "a beautiful five days" is acceptable, but "a blue five pencils" not so much), from a tech talk by Prof. Kyle Mahowald from UT Austin (thanks Dr. Steve Kramer for organizing the guest speaker series). A small, high quality dataset BabyLM was sufficient to study the effect of training data on the model's linguistic ability. By surgically removing different types of phases from training data and observing the degradation model's ability, it can be revealed which part of data is responsible for the particular linguistic ability. This is a great example of doing AI research without breaking a bank. As someone not familiar with linguistics, I take linguistic competence to be a linguistic "value function" or "reward function". Can similar emergence of rare competence (not in training data but model learned it anyway) be shown in other domains? Reference: https://lnkd.in/g4cAPUYk

16

1 Comment

If you think compute scaling will plateau, think again. We are in the baby states of inference time compute. VLLMs coupled with computer tools, reward learning and RL can leverage orders of magnitude more inference time compute than what we have seen with foundation models. Here’s a concrete training architecture (replace \pi with a large pre-trained vLLM and add it to the reward learner too, \R is the space of all computer tools)

22

1 Comment

🌟 Exciting News! 🌟 🚀 Thrilled to announce that our paper, "Beyond One-Size-Fits-All: Adapting Counterfactual Explanations to User Objectives," authored by myself, Jason Liartis, and guided by Professor Giorgos Stamou, has been accepted for presentation at the #HCXAI (Human-Centered Explainable AI) workshop of the #ACM #CHI2024 conference! 📝✨ 🔍 Ever wondered the intricacies of AI transparency and user-centric decision-making? 💭 Our research delves into Counterfactual Explanations (CFEs) to address just that! From healthcare to finance, understanding diverse user objectives is 🔑 to effective AI collaboration. But are current explanations hitting the mark? 🤔 🎯 Join us as we unravel the adaptability of CFEs to user goals and domains! 💡Our position paper offers insights into how the desired properties of CFEs may vary and conflict depending on user objectives, setting the ground for a more nuanced examination and utilization of CFEs! 📚 Dive into our position paper to explore this critical topic: https://lnkd.in/de9aS2zg 🔍 Curious to learn more? Check out #HCXAI #CHI2024 for updates! 🔗 Workshop site: https://lnkd.in/d6S5Admp 🔗 Conference site: https://chi2024.acm.org/ 🌐 Artificial Intelligence and Learning Systems Laboratory 🏛 National Technical University of Athens Spread the word and dive into the future of AI explanations with us! 🚀✨

74

New iterative reasoning technique could improve RAG app accuracy by 50% (vs. Naive RAG)! Another solid example of the rapid innovation cycle on RAG research. BTW, you can quickly find out answers to question like "Why did combining RAG with Fine-Tuning and Iterative Reasoning, such as OODA, improve accuracy by 50 percent?" in Epsilla (YC S23)'s "RAG Research Paper" smart search app at: https://lnkd.in/gVgzpbue , with number of research papers included exceeding 360. #genai #rag #smartsearch #innoation

19

I had a blast at The GenAI Collective's Demo Night last week, and I am so proud of Chalk for winning Best Technology. It was a great opportunity to showcase how Chalk makes it simple to power your ML models with generative AI. Generative AI is often most impactful when mixed with structured data. For the demo, I wrote a Gemini prompt to read bank statement memos and return JSON representing the transaction category, status, whether it was a direct deposit, and a cleaned version of the memo. I used Chalk to build features downstream of the Gemini data. These features are stored along with other features obtained from structured data, so they can be used in the same way for model training and serving. Now you don't have to wrangle two different sources of data for your generative AI work and your traditional machine learning datasets - they're all in one place, stored and treated the same way. The full code for the demo is available at the link in the comments!

43

1 Comment

From my conversations, I'm noticing different trends between how AI engineers deploy evaluation models (model as a judge) in production monitoring vs application testing. 💡 Value - Monitoring: provides real time alerts of performance or used to preemptively block problematic outputs from surfacing to the user -Testing: regression testing applications to establish baseline performance before pushing to deployment ⚡ Speed - Monitoring: near real time, requires extremely fast token processing - Testing: batch, async, often processing large test sets, lower token processing speed required 🎯 Accuracy - Monitoring: overindex on false positives, better to flag something as problematic and be wrong than to let something slip between the cracks - Testing: high correlation with human graders, be as accurate and robust as possible 💰 Cost - Monitoring: prefer cheaper models, otherwise the additional inference cost per generation will add up exponentially - Testing: willing to take on higher costs, calculation is accurate testing leads to better performing outputs 🛑 Risk - Monitoring: if your monitoring judge gets something wrong, there is a high consequence because it directly impacts the end user - Testing: if your testing judge gets something wrong, async review gives some potential to remediate the issue At the moment, it seems using model as a judge for testing is a little more popular due to the high cost constraints of constant monitoring, but that is subject to change as more specialized models come out. As the industry matures, I strongly believe we're gonna see specialization between models for monitoring and models for testing. If you're currently using model as a judge techniques, would love to chat more! Let me know what you think. Twitter Handle: https://lnkd.in/eAsQvtiu

26

We are quickly entering the "trough of disillusionment" in the hype cycle for generative AI. While GPTs excel in automating certain tasks (and can even automate away large portions of some jobs) they often fall short in others. Like all new technology, the issue lies in misapplying generative AI for tasks it wasn't designed for. This is a problem compounded by the GPTs themselves suggesting they can handle these tasks (often cheerfully!). This misalignment creates frustration and unmet expectations which we're starting to see more of these days. As the dust settles I firmly believe we'll recognize GPTs as a transformative technology across many industries. However it's crucial to understand their limitations and apply them appropriately. What are your thoughts on the current state of generative AI and its future applications?

2