Zhou Q.

The parameters of compiler optimizations interact in complex and non-trivial ways, and finding optimal combinations remains an open research challenge. There are several approaches to discovering effective combinations of these optimization parameters. Recently, Michael Canesche (Cadence Design Systems) and Gaurav Verma (Stony Brook University) have had promising results using auto-tuning with coordinate descent. To visualize this process, imagine that each optimization represents a 'dimension' in a search space. The combination of all optimizations and their respective parameters forms a vector in this space. Now, consider that each point in this space is mapped to a real number representing the performance of a kernel (assuming a fixed input). Coordinate descent is a search technique that assumes the performance curve is convex. Based on this assumption, it aims to find the optimal point by varying one optimization parameter at a time. Explore as a Storm, Exploit as a Raindrop: On the Benefit of Fine-Tuning Kernel Schedulers with Coordinate Descent: https://lnkd.in/dgxxfdYR The Droplet Search Algorithm for Kernel Scheduling: https://lnkd.in/dBfQid9z

76

Beyond the Basics: While SGD and Adam are staples in PyTorch, advanced algorithms like SLSQP and CMA-ES can tackle tough optimization problems. Learn how these gradient-free methods can enhance your models in Benjamin Bodner's latest article. #PyTorch #DataScience

28

I read a paper on ViT-22B by Google Research and here's a summary. Title: Scaling Vision Transformers to 22 Billion Parameters Links: 1. https://lnkd.in/gJa8PsvJ 2. https://lnkd.in/gPiGxKTJ Motivation: Unlike in LLMs, scaling the standard transformer architecture for vision tasks leads to pathological instability like divergent training loss while training the model. The paper demonstrates architectural changes to address such challenges. Architecture Key Points: 1. Parallel layers: Instead of sequential execution as in the standard Transformer, ViT-22B parallelly executes Attention and MLP Blocks, which reduces training time. 2. QK Normalization: To solve the instability of divergent training loss, LayerNorm is applied to the queries and keys before the dot-product attention computation. 3. Omitting biases on QKV projections and LayerNorms: The bias terms were removed from the QKV projections and all LayerNorms are applied without bias and centering. This improves accelerator utilization without quality degradation.

15

An amazing article on using  Low Rank Adaptation (LoRA) and QLoRA (Quantized Low-Rank Adaptation)  for fine-tuning LLMs. https://lnkd.in/gHu7hfAE #lora #qlora #machinelearning #llms

16

The paper "Attention Is All You Need" presents the Transformer model, a novel architecture for sequence transduction based entirely on attention mechanisms. https://lnkd.in/gbe9HSzE Key IDEAS: - Traditional sequence models rely heavily on recurrent networks, limiting parallelization. - The Transformer architecture replaces recurrence with self-attention, improving training times. - Self-attention allows the model to draw connections across the entire input sequence. - The Transformer achieves a BLEU score of 28.4 on the English-to-German translation task. - Processing can occur in parallel, significantly speeding up computation. - Multi-head attention enables the model to focus on different parts of the input simultaneously. - This model can generalize to other tasks beyond translation, such as constituency parsing. - The design simplifies model architecture by removing the need for complex recurrent structures. - Positional encodings are introduced to maintain the order of sequences. - Attention layers improve the model's ability to learn long-range dependencies. - The paper emphasizes the efficiency gains over state-of-the-art models. - Transformers outperform existing methods with considerably less training cost. - The model can be fine-tuned for various language tasks easily. - Experiments showed better performance with increased attention heads up to a point. - Utilization of dropout and label smoothing during training improves performance. - The model architecture allows for easier interpretation of attention distributions. - Attention heads often learn specific tasks, such as syntactic and semantic relations. - The architecture allows adapting to larger input and output sequences efficiently. - Adequate regularization contributes to superior generalization in learning from data. - Ease of implementation on hardware accelerates research and application. Key INSIGHTS: - Replacing recurrence with attention simplifies architecture and allows for efficient model training. - Self-attention is crucial for capturing dependencies across various positions in input sequences. - The Transformer demonstrates that parallel processing can drastically reduce training time and increase efficiency. - The design showcases how innovations in architecture can redefine state-of-the-art results in machine translation. - By employing a consistent embedding across input and output, the model minimizes errors in translation tasks. - Regularization methods like dropout and label smoothing enhance the robustness of predictions. - Understanding various attention heads can provide insights into the inner workings and decision-making of the model.

7

I'm delighted to share that a white paper I contributed to is now live on Cyient's website! Titled "TinyML Optimization Approaches for Deployment on Edge Devices," this paper delves into strategies for optimizing TinyML models to enable efficient deployment on edge devices. It's an insightful case study showcasing practical approaches in machine learning and edge computing. In this study, I played a pivotal role as a developer, overseeing data preprocessing, model conversion, training, and optimization. It was truly rewarding to achieve the perfect balance between accuracy, model size, and inference time. We employed TinyML techniques to compress and tailor various models while preserving peak accuracy and achieving significant reductions in size and inference time. I want to extend my sincere thanks to my colleagues: Srinivas Kudavelly and Amol Gharpure for their invaluable guidance and expertise. Malavika Rajesh and Manshi Mithra, whose collaboration was instrumental in achieving our research goals. A big thank you to Cyient for providing the opportunity to contribute to cutting-edge research in machine learning. Read our findings here: https://lnkd.in/gK-YfRrX This work opens exciting possibilities for enhancing user experiences and enabling edge computing. #TinyML #MachineLearning #EdgeComputing #DeepLearning #Whitepaper #Cyient #TensorflowLite #Optimization #Tensorflow

21

🚀 New Medium Post 🚀 Excited to share my latest blog on Transformers: Self-Attention Mechanism from Scratch using PyTorch! 🤖 In this post, I break down the fundamentals of self-attention and walk through a step-by-step implementation using PyTorch. Whether you're diving into NLP or interested in understanding the mechanics behind state-of-the-art models, this guide will provide you with hands-on experience. Check it out and let me know your thoughts! 🛠️ #Transformers #DeepLearning #PyTorch #NLP #SelfAttention #AI #MachineLearning #ArtificialIntelligence #DataScience Read “Transformers: Self-Attention Mechanism from scratch using PyTorch.“ by Devmallya Karar on Medium: https://lnkd.in/gvxnFA2X

2

Understanding Conversation Summary Memory vs Conversation Summary Buffer Memory in LangChain For detailed sample code, check out the Google Colab notebook(https://lnkd.in/gRsarpNX) In the evolving field of conversational AI, memory mechanisms play a critical role in maintaining coherent and contextually relevant interactions. Two key approaches, Conversation Summary Memory and Conversation Summary Buffer Memory, offer distinct ways of handling conversational context. While Conversation Summary Memory focuses on summarizing entire dialogues, making it ideal for long conversations where historical context is essential, Conversation Summary Buffer Memory retains a rolling window of recent exchanges, suited for short, focused conversations. Understanding the differences between these memory types and their appropriate use cases can significantly improve chatbot performance. For a deeper dive into how these memory systems function and their practical applications using LangChain, you can explore the full pdf article below. #ConversationalAI #AI #LangChain #Chatbots #ArtificialIntelligence #MemoryMechanisms #MachineLearning #NLP #OpenAI #AIChatbot #Technology

90

🚀 **Just Published: Unlocking the Power of Large Language Models with Chunking Techniques!** 🚀 🔍 Ever wondered how large language models (LLMs) handle long documents while keeping context intact? The answer lies in **chunking**! In my latest blog, I explore the key chunking methods that boost LLM performance, covering everything from **simple sentence splits** to **adaptive AI-based chunking**. With each technique, I’ve included **code samples** to help you put these methods into action! 💡 Whether you're working in **Generative AI**, **NLP**, or **text summarization**, this guide will help you unlock new potential with LLMs. 📈 **Curious to know how it works? Check out the full blog!** https://lnkd.in/gniDQYwW --- #GenerativeAI #AI #MachineLearning #LLM #NLP #AIResearch #DataScience #Chunking #TextProcessing #AIEngineering --- For maximum engagement, tag a few companies or groups if possible, like: - **@OpenAI** - **@Google AI** - **@Hugging Face** - **@DeepMind**    Excited to share these insights and can’t wait to hear your thoughts! #llm #openai #genai #chunking #langchain

12

2 Comments

PyTorch Profiler is an interesting tool to pinpoint bottlenecks and improve efficiency. Here's how it can help: 𝗪𝗵𝗮𝘁 𝗜𝘀 𝗣𝘆𝗧𝗼𝗿𝗰𝗵 𝗣𝗿𝗼𝗳𝗶𝗹𝗲𝗿? It’s a performance analyzer for your models. You can track: ✅ CPU and GPU activity ✅ Memory usage ✅ Execution time of operations 𝗪𝗵𝘆 𝗨𝘀𝗲 𝗣𝘆𝗧𝗼𝗿𝗰𝗵 𝗣𝗿𝗼𝗳𝗶𝗹𝗲𝗿? 🔺 𝗙𝗮𝘀𝘁𝗲𝗿 𝗠𝗼𝗱𝗲𝗹 𝗘𝘅𝗲𝗰𝘂𝘁𝗶𝗼𝗻: Optimize performance and reduce processing time.   🔺 𝗥𝗲𝗱𝘂𝗰𝗲𝗱 𝗠𝗲𝗺𝗼𝗿𝘆 𝗨𝘀𝗮𝗴𝗲: Minimize the memory footprint of your model.   🔺 𝗜𝗺𝗽𝗿𝗼𝘃𝗲𝗱 𝗛𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗨𝘁𝗶𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Make better use of available resources like CPU and GPU.   🔺 𝗜𝗱𝗲𝗮𝗹 𝗳𝗼𝗿 𝗟𝗮𝗿𝗴𝗲-𝗦𝗰𝗮𝗹𝗲 𝗠𝗼𝗱𝗲𝗹𝘀: Especially beneficial for handling complex, resource-intensive tasks.   🔺 𝗣𝗲𝗿𝗳𝗲𝗰𝘁 𝗳𝗼𝗿 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗘𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁𝘀: Crucial for deploying efficient, scalable solutions. The code example analyzes the performance of a simple linear model. It tracks CPU and GPU usage, memory consumption, and tensor shapes. By using the "record_function" decorator, specific parts of the code are labeled for easier analysis. The collected data helps identify inefficiencies, even in basic operations, so developers can make their models run more efficiently. #pytorch #llms #ai #ml #python

47

A simple tutorial on the UNET 3+ Implementation using PyTorch - What is UNet 3+? - Deep Supervision - Classification guided Module https://lnkd.in/gtgTFtAF #tensorflow #pytorch #machinelearning #computervision #deeplearning #datascience #unet #segmentation #keras #AI

8

Is GPT-4o good at table recognition? Not quite! OCR looks fine, but tripped by spanning cell table structure! 🤩 However, our UniTable, with only 124M param trained by self-supervision, shows great understanding! 👉 UniTable paper: arxiv.org/abs/2403.04822 👉 UniTable code: https://lnkd.in/e4a7YMxJ #MachineLearning #GPT-4o #Table #Document

85

7 Comments

NaturalInstructions: A meta-dataset of 76 NLP tasks. Before LLMs, each task required a separate model, but now rich datasets like NatInstruct, FLAN, etc. drive the training and evaluation of LLMs. Paper: https://lnkd.in/gDEM_8XU #instruction_tuning #nlp #datasets #llm

15

New post on DebuggerCafe - Fine-Tuning OPT-350M for Extractive Summarization https://lnkd.in/gYCMt9wu In this article, we use the OPT-350M model for extractive summarization of articles while covering the following topics: * Preparation of the summarization dataset in an instruction following format. * Managing the "Beginning of Sequence" and "End of Sequence" tokens for proper start and end points of the samples. * Training and inference to generate summaries of new unseen articles. #DeepLearning #LLMs #NLP #TextSummarization #ExtractiveSummarization #OPT350M

5

Our code for RAG evaluation is now publicly available at Amazon Science website: https://lnkd.in/g-HKUNZZ. This is the implementation of our evaluation mechanism performed by scoring the RAG on an automatically-generated synthetic exam composed of multiple choice questions based on the corpus of documents associated with the task. #OpenSource #OpenSourcing #AmazonScience #ICML2024 #Evaluation #LLMEvaluation

211

6 Comments

LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models. It is a modular open-source framework to streamline the research, training, and evaluation of MoE algorithms. Built upon three core principles: (i) modular design (ii) efficient training (iii) comprehensive evaluation. The paper benchmarks five state-of-the-art MoE algorithms over three different LLMs and 11 datasets under the zero-shot setting. #MachineLearning #GenAI #LLM #MoE #DataScience

53

1 Comment