Wei Liu

Can LLM preference tuning degrade Multi-Agent system performance? In a recent tweet, Andrej Karpathy questioned why all LLMs sound similar, prompting Sebastian Rascha to speculate about the uniformity of alignment datasets. Aligning user preferences is a last-mile step in training large language models. Regardless of the pre-training datasets used, could the preference datasets used during alignment compromise the effectiveness of LLMs? This issue could adversely affect multi-agent systems incorporating multiple LLMs in their design.

7

1 Comment

When *open* isn't *open source* 🙋🏼‍♂️👀 Reflecting on the open-source landscape for Large Language Models, I explore the essentials: permissive licensing, transparency in training data and model architecture & more #LLM #LLMs #AI #GenerativeAI #opensource https://lnkd.in/gMxbN4Nb

35

2 Comments

OpenCoder doesn't get enough love They open-sourced the entire pipeline to create QwenCoder-level code models. This includes: - Large datasets - High-quality models - Eval framework Tons of great lessons and observations in the paper 📝 Paper: arxiv.org/abs/2411.04905 #special #linkedin #Family #Pakistan #skills #islam #government #AI #ML #MachineLearning #tesla #microsoft #openai

5

Today's reading is Memory Layers in Large Language Models 🧠 This research shows how memory layers can take LLM performance to the next level: 🔑 Trainable key-value lookup mechanism: Adds parameters without increasing computational cost, boosting factual accuracy and task performance. 📈 Big results, small cost: Outperforms models with much larger computational budgets and even beats mixture-of-experts models! ⚙️ Scales seamlessly with up to 128 billion memory parameters The authors from Meta highlights how memory layers are a game-changing addition to future AI architectures, offering smarter, faster, and more accurate solutions. You may want to check out the paper at: https://lnkd.in/gzuqUfYe #AI #LLM #MemoryLayers #Innovation #MachineLearning #FactualAI #FutureOfAI #Meta

19

User embeddings are crucial for the downstream personalized models, but larger embedding models come with higher latency, making real-time inference challenging. This often forces a trade-off: Small embedding model for real-time updates. Or, large embedding model with offline batch updates. Meta's recent paper proposes a hybrid approach: combine stale embeddings from a feature store with real-time embeddings from a complicated user model (with a timeout). If real-time calculations exceed the timeout, use stale embeddings; otherwise, use both. Completed real-time embeddings are then flushed to the feature store, improving future requests. Paper link:

28

Guess what, training large language models (LLMs) can truly be an emotional journey! The recent training of LLaMA 3, powered by 16,000 GPUs, faced up to 8 interruptions per day. As data scales and architectures become more complex, organizations now have the flexibility to choose designs that align with their cost constraints and high availability requirements. Key Takeaways: - Compound AI systems and agent-based architectures represent the future of AI, offering greater flexibility and scalability for tackling complex challenges. When your AI model becomes too large for a single node, exploring tensor parallelism or pipeline parallelism becomes essential. - Developing a robust and scalable data pipeline capable of handling massive volumes of data is crucial for the successful deployment and real-time performance of AI models. - Optimizations to accelerate GPU workloads, including distributed training, are crucial for enhancing performance and ensuring efficient use of resources in large-scale AI training. And a bonus highlight: Ion Stoica, founder of Databricks and Ray, is super nice and let me snap a selfie with him! 📸 #AI #ArtificialIntelligence #DataScience #MachineLearning #LLMs #TrainingAI #Innovations

40

4 Comments

[research update] use a Transformer's multi-head attention activations to fetch RAG style, to augment multi faceted problem breakdown for LLMs in ."Multi-Head RAG: Solving Multi-Aspect Problems with LLMs": 1. Enhances LLMs by integrating document retrieval (RAG) into the LLM context. 2. Challenge: Existing RAG solutions struggle with queries needing multiple documents with varied contents due to distant embeddings. 3. Multi-Head RAG (MRAG): Introduces a novel approach using Transformer's multi-head attention activations to fetch diverse documents. 4. Activation Utilization: Different attention heads capture varied data aspects, improving complex query retrieval. 5. Improved Relevance: MRAG shows up to 20% improvement in relevance over standard RAG baselines. 6. Evaluation Methodology: Provides metrics, synthetic datasets, and real-world use cases to demonstrate effectiveness. 7. Seamless Integration: MRAG can be integrated with existing RAG frameworks and benchmarking tools like RAGAS. 8. Multi-Aspect Problem Solving: Designed specifically to handle multi-aspect queries more effectively. 9. Data Stores Compatibility: Compatible with various data store classes. 10. Potential Applications: Benefits scenarios requiring diverse information retrieval, enhancing LLM responses. Experiment The paper introduces MRAG and evaluates it using synthetic datasets and real-world data. It compares MRAG's performance against standard RAG models. The experiment focuses on the model's ability to retrieve relevant documents across various query aspects, demonstrating up to a 20% improvement in relevance over baselines. Use Case MRAG is tested on real-world scenarios like customer support, where queries often require information from multiple documents with diverse contents. The model's capability to retrieve and integrate multi-aspect data enhances the quality of responses.

81

5 Comments

A very useful checklist to follow when you have to put in production Transformer based models

5

Chinese tech talks hit different. I watched a translation of this talk (https://lnkd.in/e9AbSz95) about using java bytecode manipulation to inject sidecar-like behaviour into JVM apps (https://sermant.io/en/) and the speaker likened the alternating trends of moving mesh-like behaviour into client libs, then out of process, then back into the business logic, to the opening lines of a historical novel called "Romance of the Three Kingdoms" "The empire, long divided, must unite; long united, must divide. Thus it has ever been" I did not expect to find a philosophical rabbit hole in a tech talk about service mesh.

8

Interesting read from NVIDIA: "𝐑𝐚𝐧𝐤𝐑𝐀𝐆: 𝐔𝐧𝐢𝐟𝐲𝐢𝐧𝐠 𝐂𝐨𝐧𝐭𝐞𝐱𝐭 𝐑𝐚𝐧𝐤𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥-𝐀𝐮𝐠𝐦𝐞𝐧𝐭𝐞𝐝" RankRAG is a framework designed to instruct and finetune LLMs for context ranking and answer generation in RAG applications. Link: https://lnkd.in/gsEJ5cVK #llm #nvidia #rag #finetune

7

one of the challenges with RAG is evaluating it. check out this code from Amazon Science if you are building a RAG based system Romina Sharifpour, Ph.D. Baichuan Sun

23

5 Comments

https://lnkd.in/gFHevHGC https://lnkd.in/gxnHhYMj https://lnkd.in/gWHYDHrq these are curated sets made from the *unprocessed most recent stack exchange dump* that we were surprised to find had not been optimized, despite the RL focus currently in the ecosystem, RLSTACK contains a 1 million row dataset of DPO style preference pairs made from the highest rated (preferring chosen answers) and lowest rated responses to each question Select stack contains 1.5 million rows of data comprised of the top scoring response, as well as the chosen response for each question - this is a sft formatted subset of the data, and is the version used in Buzz(https://lnkd.in/gQQG_f8u)! and unstacked is an *uncurated* reformatting demonstrating every question along with every response, categorizing the questions by a unique question index, as well as the overall score for each answer, and whether or not it was a chosen answer. currently all three subsets contain QA pairs from the following exchanges: emacs economics english earthscience engineering genai gaming gamedev graphicdesign hardwarerecs homebrew history hermeneutics interpersonal langdev lifehacks languagelearning law literature linguistics martialarts mathematica mathoverflow mechanics money mythology movies math matheducators networkengineering opendata philosophy opensource quantumcomputing reverseengineering parenting physics patents poker puzzling raspberrypi retrocomputing robotics scicomp scifi salesforce serverfault softwareengineering softwarerecs space sustainability superuser security skeptics stats unix ux vi webapps webmasters worldbuilding writers workplace a version including the rest of the exchanges/the less difficult/reasoning heavy pairs will be released in the coming days along with the data from *the other stack exchange dump which seems to have gone unprocessed*

1

Hallucination is Inevitable in LLMs An important new paper out that shows that even with perfect data and an idealized architecture “LLMs will always hallucinate”. I suspect that this will over time be linked to Gödel’s Incompleteness Theorem [1] and Turing’s Halting Problem [2]. Every technology has trade-offs. We love LLMs but they need to be used in the right way. [1] Good pop science depiction in Oppenheimer (Gödel was Einstein’s walking companion) or also see Rebecca Goldstein’s book Incompleteness [2] Turing is depicted in The Imitation Game and also the progenitor of the famed Turing Test.

37

🔮RAG + KG: Powerful Combination for Translation Multi-AI Agents Large language models (LLMs) have shown great promise in machine translation, but they still struggle with contextually dependent terms, such as new or domain-specific words. This leads to inconsistencies and errors that are difficult to address. Existing solutions often depend on manual identification of such terms, which is impractical given the complexity and evolving nature of language. While RetrievalAugmented Generation (RAG) could provide some assistance, its application to translation is limited by issues such as hallucinations from information overload. =================== In this paper, researchers propose CRAT, a novel multi-agent translation framework that: 》 leverages RAG and causality enhanced self-reflection to address these challenges. This framework consists of several specialized agents: ✱ the Unknown Terms Identification agent detects unknown terms within the context, ✱ the Knowledge Graph (KG) Constructor agent extracts relevant internal knowledge about these terms and retrieves bilingual information from external sources, ✱ the Causality enhanced Judge agent validates the accuracy of the information, ✱ Translator agent incorporates the refined information into the final output. =================== This automated process allows for more precise and consistent handling of key terms during translation. Their results show that CRAT significantly improves translation accuracy, particularly in handling context-sensitive terms and emerging vocabulary. --- If your company is looking for an AI consultant to accelerate business growth, or if you are looking to build your career in AI, send me a message and I can help

4