Skip to content
tarazumPublic

About

Experimental lab for evaluating AI models, architectures, agents, memory systems, and inference runtimes on real-world workloads.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

190 Commits

Folders and files

Repository files navigation

mdlslab

Experimental lab for evaluating AI models, architectures, agents, memory systems, forecasting models, and inference runtimes on real-world workloads.

The lab is intentionally heterogeneous. A forecasting foundation model, a local agentic LLM, a decision model, and an external memory system do not need one artificial benchmark. The common unit is a reproducible experiment.

Labs

Lab Focus State
Muse Glimmer Local multimodal agent model, tool use, long tasks, constrained hardware Planned
TimesFM Time-series foundation models and forecasting In progress (private playground)
Jev Decision-oriented model evaluation Upstream verified; experiments pending
Laya Candidate local/open model Source verification
TencentDB Agent Memory Persistent agent memory and retrieval behavior Upstream verified; experiments pending
Continuity (Project C) Behavioral continuity through memory, self-model, feedback and reflection Autonomous arc complete: exploratory + confirmatory CONT-001 done; result owner-accepted 2026-10-04 with caveats; next arc CONT-005 gated on suite v3 redesign
CLI-Anything Agent tooling and CLI-based software control Planned
Colibri Existing public playground External lab
Mamba Existing public playground External lab

Principles

  • Use evaluation methods appropriate to each subject.
  • Preserve environment, configuration, procedure, and evidence.
  • Compare systems only when they solve compatible problems.
  • Treat vendor benchmarks as context, not as lab findings.
  • Record failures and negative results.
  • Keep the repository public-safe by design.
  • Prefer primary upstream sources and pin versions used by experiments.

See Architecture for the repository model, experiment contract, publication policy, and migration strategy.

Shared tooling

  • Agent resource coordination — binding protocol + lock.py for concurrent agents sharing one machine's GPU/downloads; enforcement research tracked there.

Public repository

Do not commit credentials, private datasets, personal conversations, corporate code, proprietary logs, or restricted model artifacts. Experiments requiring private material should stay outside this repository; only sanitized methodology and results may be published here.

Existing playgrounds

Existing public playgrounds are linked from mdlslab rather than immediately migrated. This preserves their history and URLs while the new lab structure proves itself. A future migration should be deliberate, not a prerequisite for using mdlslab.

About

Experimental lab for evaluating AI models, architectures, agents, memory systems, and inference runtimes on real-world workloads.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages