[tabular] Add DenseLight (LAMA-style) neural net model by a11to1n3 · Pull Request #5834 · autogluon/autogluon

a11to1n3 · 2026-08-10T13:35:39Z

Issue #, if available: Addresses #4505 (NN half)

Description

Kaggle feedback (tilii7 / #4505): AutoGluon NNs are diverse vs trees but not competitive enough with LAMA-based NNs, so users run LightAutoML beside AG. This PR adds LAMA’s strong tabular NN (DenseLight) natively so that path lives in one package.

Architecture is from LightAutoML (Apache-2.0); training, preprocess, bagging, and device handling are AutoGluon-native. There is no lightautoml dependency.

Follows the custom model tutorial contract pointed at in #4505 (AbstractModel / AbstractTorchModel, _fit, _preprocess, defaults, valid_raw_types), then registers in-tree as DENSELIGHT.

Implementation notes

New model key DENSELIGHT (DenseLightModel → AbstractTorchModel).
Vendored DenseLight net: MLP blocks with concat_input (input concatenated into later hidden blocks), LAMA-leaning defaults (hidden_size=[512, 750], quantile numericals, early stopping).
Auxiliary params: valid_raw_types for bool/int/float/category (tutorial-style).
Fit contract: val holdout if needed, early stopping, time_limit, CPU/GPU, save/load via torch base.
Optional extra: pip install "autogluon.tabular[denselight]" (torch only).
Opt-in only — not added to default presets/portfolios yet.
Unit tests: FitHelper.verify_model + auxiliary-params check.

Usage

from autogluon.tabular import TabularPredictor
from autogluon.tabular.models import DenseLightModel

# Registered key
predictor.fit(train_data, hyperparameters={"DENSELIGHT": {}})

# Custom-model tutorial style (class as key)
predictor.fit(train_data, hyperparameters={DenseLightModel: {"n_epochs": 50}})

Out of scope (follow-ups)

Other LAMA algos (NODE, AutoInt, …).
Ensembling / hill climbing ([tabular] Kaggle Feedback #4505 item 2).
Default portfolio inclusion — needs diversity/competitiveness evidence first.

github-actions · 2026-08-12T08:29:36Z

Job PR-5834-192bb26 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-5834/192bb26/index.html

Addresses autogluon#4505 complaint that AG NNs lag LAMA NNs and force a second stack: vendor LightAutoML DenseLight architecture in-tree with an AutoGluon training loop (no lightautoml dependency). Opt-in via hyperparameters key DENSELIGHT / extra [denselight].

Declare valid_raw_types via auxiliary params, document key and class-key usage, and sync memory-estimate defaults with LAMA hidden sizes.

CI test_tabular failed because DenseLightModel was registered in ag_model_registry but missing from EXPECTED_MODEL_* fixtures in test_model_registry.py.

- Rename LAMA references to LightAutoML (codespell false positive LAMA→LLAMA) - Regenerate uv.lock so denselight optional extra is locked

LAMA is LightAutoML, not Llama. Restore LAMA wording and add `lama` to codespell ignore lists so CI does not rewrite it to LLAMA.

a11to1n3 · 2026-09-09T18:54:46Z

Rebased onto current master (includes #5892 Darwin xgboost markers).

The only real conflict was uv.lock. Kept master's lock and re-added the denselight extra (torch-only). Dropped [ci] Write uv.lock exclude-newer in UTC — that timestamp is already on master.

uv lock --check and ruff on the DenseLight sources are green locally.

Innixma · 2026-09-10T19:24:18Z

Will plan to take a look at this at some point, but probably not until we are close to the next release. Thanks for the contrib!

LennartPurucker · 2026-09-16T08:50:28Z

FYI: we investigated DenseLight on the TabArena side before, and it did not look very promising; see autogluon/tabarena#101 for the thread. RealMLP and TabM seem to be more competitive choices.

Maybe you can pick up this issue and see whether DenseLight can be improved on TabArena first, before integrating it into AutoGluon. It could be that the code and implementation I used back then were sub-optimal.

a11to1n3 · 2026-09-16T09:14:36Z

Hi @LennartPurucker, this is really informative. I will have a look at your issue and test DenseLight against TabArena first and we will see.

a11to1n3 · 2026-09-18T07:05:52Z

Hi @LennartPurucker — we followed up on this.

We re-ran TabArena-Lite with a vendored LAMA DenseLight (CatEmbedder + SWA trainer, LAMA tabular_config.yml defaults: [512, 256], 50 epochs, lr=3e-4), not the ordinal-cat MLP from add_denselight_mlp. Protocol: first split of the 51 datasets, official 8×1 bagging, default + 25 HPO. 1326/1326 bags completed, 0 failures, 0% imputed.

Lite Elo:

method	Elo
DenseLight default	974
DenseLight tuned	1124
DenseLight T+E	1140
FastaiMLP T+E	1168
TorchMLP T+E	1247
TabM T+E	1394
RealMLP T+E	1480

Your earlier read still holds: even with the real LAMA stack, DenseLight is not competitive as a TabArena method — well below TabM / RealMLP, and also below AutoGluon’s existing TorchMLP / FastAI on this Lite board.

#4505 was a different question (Kaggle: LAMA NNs helping a blend vs AG’s NNs). This PR is still opt-in and not in default presets.

@Innixma @LennartPurucker — given that, is it still worth keeping DenseLight as a useful extra NN in AutoGluon for ensemble diversity, or should we close / hold this until there’s evidence it actually helps WeightedEnsemble vs TorchMLP/FastAI?

LennartPurucker · 2026-09-18T08:45:34Z

#4505 was a different question (Kaggle: LAMA NNs helping a blend vs AG’s NNs). This PR is still opt-in and not in default presets. [...] or should we close / hold this until there’s evidence it actually helps WeightedEnsemble vs TorchMLP/FastAI?

With TabArena and your local results, you can simulate this as well and see whether blending would ever pick it up. @Innixma might be able to share more information on this, or Claude can find it, hopefully.

On a side note, IMO, we could always add it to TabArena, which would then exist as a tested and benchmarked model that someone could easily pull into AutoGluon as a custom model via the hyperparameters. I usually add models to TabArena regardless of final performance. But IMO, it would be important if we could pip install the standalone model for this. I got the impression that its usage would otherwise always be a bit obscure. Do you know if this would be possible?

a11to1n3 had a problem deploying to integration-tests August 15, 2026 06:40 — with GitHub Actions Error

a11to1n3 had a problem deploying to integration-tests August 15, 2026 09:10 — with GitHub Actions Error

Innixma added this to the 2.0 Release milestone Aug 19, 2026

a11to1n3 force-pushed the tabular/add-denselight-lama-nn-4505 branch from c32816c to 4ae1f85 Compare August 19, 2026 19:34

a11to1n3 had a problem deploying to integration-tests August 19, 2026 21:52 — with GitHub Actions Error

Innixma force-pushed the tabular/add-denselight-lama-nn-4505 branch from 4ae1f85 to b0461f8 Compare August 25, 2026 19:59

Innixma had a problem deploying to integration-tests August 25, 2026 20:07 — with GitHub Actions Error

a11to1n3 added 6 commits September 9, 2026 20:45

[tabular] Align DenseLight with custom-model tutorial contract

f0634a3

Declare valid_raw_types via auxiliary params, document key and class-key usage, and sync memory-estimate defaults with LAMA hidden sizes.

[tabular] Ruff-format DenseLight sources for lint_check

62e245e

[tabular] Register DenseLight in model registry unit tests

6ad9972

CI test_tabular failed because DenseLightModel was registered in ag_model_registry but missing from EXPECTED_MODEL_* fixtures in test_model_registry.py.

[tabular] Fix DenseLight CI: codespell LAMA + refresh uv.lock

398f7e1

- Rename LAMA references to LightAutoML (codespell false positive LAMA→LLAMA) - Regenerate uv.lock so denselight optional extra is locked

[tabular] Keep LAMA acronym; ignore it in codespell

bf171c2

LAMA is LightAutoML, not Llama. Restore LAMA wording and add `lama` to codespell ignore lists so CI does not rewrite it to LLAMA.

a11to1n3 force-pushed the tabular/add-denselight-lama-nn-4505 branch from b0461f8 to bf171c2 Compare September 9, 2026 18:54

a11to1n3 requested a deployment to integration-tests September 9, 2026 18:58 — with GitHub Actions Waiting

Navigation Menu

[tabular] Add DenseLight (LAMA-style) neural net model - #5834

[tabular] Add DenseLight (LAMA-style) neural net model#5834
a11to1n3 wants to merge 6 commits into
autogluon:masterfrom
a11to1n3:tabular/add-denselight-lama-nn-4505

a11to1n3 commented Aug 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

a11to1n3 commented Sep 9, 2026

Uh oh!

Innixma commented Sep 10, 2026

Uh oh!

LennartPurucker commented Sep 16, 2026 •

edited

Loading

Uh oh!

a11to1n3 commented Sep 16, 2026

Uh oh!

a11to1n3 commented Sep 18, 2026

Uh oh!

LennartPurucker commented Sep 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

a11to1n3 commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Implementation notes

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

a11to1n3 commented Sep 9, 2026

Uh oh!

Innixma commented Sep 10, 2026

Uh oh!

LennartPurucker commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

a11to1n3 commented Sep 16, 2026

Uh oh!

a11to1n3 commented Sep 18, 2026

Uh oh!

LennartPurucker commented Sep 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

a11to1n3 commented Aug 10, 2026 •

edited

Loading

LennartPurucker commented Sep 16, 2026 •

edited

Loading