Hugo Shi

Last week, the United States Congress passed President Trump’s “Big Beautiful Bill.” I’m disappointed it didn’t include a proposed moratorium on U.S. state-level AI regulation. While there is a role for AI regulation, it is when the technology is new and poorly understood that lobbyists are most likely to succeed at pushing through anti-competitive regulations that hamper open-source and other beneficial AI efforts. A moratorium would have bought more time for regulators to figure out the realistic risks and rewards of AI and thereby avoid bad regulatory proposals. Many jurisdictions loosely follow this trajectory: - When new AI technology is still poorly understood, companies can make grandiose statements about its benefits or dangers, and both traditional and social media are ineffective at fact-checking them and tend to parrot what they say. During this initial period, businesses can get away with saying almost anything. - This opens opportunities for hype as well as fear mongering based on exaggerated claims about AI dangers. Some businesses exploit this opportunity to try to get regulators to pass anti-competitive laws that impede open-source and other competitors. - But eventually, smart regulators learn enough about AI to understand its realistic benefits and risks. For example, the U.S. Senate’s bipartisan Insight Forum on AI, which I participated in, heard from many stakeholders and came to support innovation and dismiss ill-founded fears of “AI takeover” and the like. Indeed, the European Union went through this trajectory as well. After the EU AI Act was passed, many regulators realized many of its “protections” are not actually helpful. They relaxed some of the law’s provisions to make it less stifling of innovation than many observers initially had feared. There are AI regulations that would limit harmful applications appropriately, for example, banning non-consensual deepfake porn and preventing misleading marketing. However, many states, which have less resources than the federal government to deeply understand AI, have proposed harmful regulations, especially those that aim to regulate the technology rather than the applications. For example: - California’s SB 1047 purported to impose safety requirements on frontier AI systems, but it placed ambiguous and/or technically infeasible requirements on model creators to prevent harmful downstream uses. This is akin to holding the maker of a hammer liable if someone uses it for harmful purposes. Fortunately, Governor Gavin Newsom quashed SB 1047 with a veto. - New York’s Responsible AI Safety and Education Act, which passed the state legislature in June and awaits Governor Hochul’s signature or veto, also places ambiguous and unreasonable requirements on model builders, purportedly to guard against “critical harms.” It would hamper open source without making anyone meaningfully safer. [Truncated due to length limit. Full text: https://lnkd.in/gfwAbz4N ]

1,003

93 Comments

We just cracked the code on making AI agents actually reliable. For years, only frontier labs like OpenAI and xAI could use reinforcement learning to dramatically improve their models. Everyone else got stuck trying to write complex reward functions that required PhD-level expertise and months of trial and error. That barrier just disappeared. RULER is a universal reward function that makes RL accessible to any developer building AI agents. No labeled data! No reward engineering! No human feedback required! Just plug it in and watch your agents get smarter. We tested it on four production systems our enterprise customers are deploying: email search agents, voice ordering systems, customer support bots, and reasoning classifiers. In every single case, small open-source models trained with RULER outperformed even o3. This changes everything for companies building AI agents. The same technique that made ChatGPT and Claude so capable is now available to everyone. We're seeing 2x improvements in reliability, 5x faster response times, and 80% cost reductions. The best part? It's completely open source. Star our GitHub repo to get started: https://lnkd.in/gBgeKsup The age of unreliable AI agents is over. Welcome to the RL revolution.

1,775

127 Comments

The resurgence of reinforcement learning is great, but it’s missing the big picture. Chasing traditional RL is the wrong direction. In pursuing better policies and reward shaping, it’s easy to miss a simpler tool we already have: language. There is no reason to work in weights and gradients if you can work in natural language. Today we’re releasing our first set of research on Prompt Learning — not prompt tuning, not chain-of-thought. Something new. To our knowledge, no one has yet rigorously researched, characterized, and measured a full implementation of this approach. Instead of adjusting weights, we use MetaPrompting — where English evals and critiques (rather than just scalar metrics like previously done by the industry) drive targeted prompt updates. It’s a promising avenue of research, and the early results show that it works. Writeup in the comments.

173

10 Comments

Some recent, exciting news on compressed tokenizers and how individual tokens modify images in encoder-decoder models. I hope this opens up a new avenue of research with smaller models across frameworks (perhaps JEPA?). https://lnkd.in/evFw8WBu

70

2 Comments

There’s a new RL algorithm in town… and it’s already shaking up the LLM research community. 🤯 Group Sequence Policy Optimization (GSPO) was created by the research team at Qwen and is purpose built for operating at the sequence level rather than the token level. This means it evaluates, clips, and optimizes entire response sequences by their overall likelihood. The important upshot is that it directly aligns the optimization process with how rewards are actually assigned in real world scenarios! Some key reasons to be excited! 1. Better stability. GSPO improves training stability and helps avoids model collapse that can happen with algorithms like GRPO. 2. Scalable RL for mixture of experts (MoE) models. GSPO inherently solves stability challenges teams face when scaling RL for MoE architectures. Large scale RL becomes more practically feasible! 3. Simplifies Infra. Sequence-level optimization is inherently less sensitive to small numerical differences between training and inference systems compared to token-level methods, which should lead to more robust RL pipelines! We can already see signs that this has paid off in big ways for the open-source community. Qwen3 models are doing really well on benchmarks, boasting competitive performance with top models like Gemini-2.5 Pro on GPQA, AIME25, and LiveCodeBench! https://lnkd.in/gCZMQvUj P.S. 👀 If you want to run RL on your own Qwen3 model with GRPO/GSPO, keep an eye on our open-source post-building repo 🧡 : https://lnkd.in/gXZ32qkm #AI #LLM #ReinforcementLearning

81

10 Comments

Recent research by UIUC and Intel Labs has introduced a new jailbreak technique for Large Language Models (LLMs) known as InfoFlood. This method takes advantage of a vulnerability termed "Information Overload," where excessive linguistic complexity can circumvent safety mechanisms without the need for traditional adversarial prefixes or suffixes.   InfoFlood operates through a three-stage process: Linguistic Saturation, Rejection Analysis, and Saturation Refinement. Initially, it reformulates potentially harmful queries into more complex structures. If the first attempt does not succeed, the system analyzes the response to iteratively refine the query until a successful jailbreak is achieved. Empirical validation across four notable LLMs—GPT-4o, GPT-3.5-turbo, Gemini 2.0, and LLaMA 3.1—indicates that InfoFlood significantly surpasses existing methods, achieving success rates up to three times higher on various benchmarks.   The study underscores significant vulnerabilities in current AI safety measures, as widely used defenses, such as OpenAI’s Moderation API, proved ineffective against InfoFlood attacks. This situation raises important concerns regarding the robustness of AI alignment systems and highlights the necessity for more resilient safety interventions. As LLMs become increasingly integrated into diverse applications, addressing these vulnerabilities is crucial for ensuring the responsible deployment of AI technologies and enhancing their safety against emerging adversarial techniques.  Arxiv: https://lnkd.in/eBty6G7z

53

4 Comments