scaling-laws

code for Scaling Laws for Language Transfer Learning

Code for reproducing the experiments on large-scale pre-training and transfer learning for the paper "Effect of large-scale pre-training on full and few-shot transfer learning for natural and medical images" (https://arxiv.org/abs/2106.00116)

Code for CoNLL BabyLM workshop Mini Minds: Exploring Bebeshka and Zlata Baby Models

[NeurIPS 2023] Multi-fidelity hyperparameter optimization with deep power laws that achieves state-of-the-art results across diverse benchmarks.

[ICML 2023] "Data Efficient Neural Scaling Law via Model Reusing" by Peihao Wang, Rameswar Panda, Zhangyang Wang

A method for calculating scaling laws for LLMs from publicly available models

Scaling laws web calculator to get a model's training compute flops, costs and energy utilization.

[NeurIPS'24 Spotlight] Observational Scaling Laws

[NeurIPS 2023] Multi-fidelity hyperparameter optimization with deep power laws that achieves state-of-the-art results across diverse benchmarks.

Official code for the ICLR 2025 paper, "Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining"

🔥🔥🔥 Latest Advances on Large Recommendation Models

This is a repository containing the code, data, and visulizations that I made as a part of my senior capstone project

Presentation on Scaling Laws for Neural Language Models​

A toolkit for scaling law research ⚖

RSRC Calculator is a practical tool designed to evaluate the efficiency of AI models in the post-scaling era: Recursive Self-Referential Compression (RSRC), this tool computes training efficiency metrics by analyzing factors such as training FLOPs, energy consumption, and model architecture details.

🌹[ICML 2024] Selecting Large Language Model to Fine-tune via Rectified Scaling Law

Codebase for the paper "Memorization in Attention-only Transformers", accepted at AISTATS 2025

Training LLMs at scale and analyzing scaling laws

NeurIPS 2026 - Multi-Network Training for Transfer Learning on Temporal Graphs