-
A Non-Linear Online Bandit for Set-Compatible Collection
When an item's value depends on context, a linear value head is not a weak learner — it is a provably wrong one.
-
Reflex Bandit: An Agent with Drift Detection for Active Exploitation Regimes
An agent whose sensors are noiseless but whose meaning must be learned. When the world's shape drifts, ensemble disagreement is not enough — the agent has to watch its own prediction error. A tree-ensemble contextual bandit with two-timescale memory and an EWMA drift detector, in a regime where value-function RL is the wrong tool.
-
MoB Queen: Mixture of Bandits with a Global Density Map and a Queen Move
A stateless agent with a noisy local sensor explores well by leaping — but on sparse worlds it circles in the half of the map it has already seen. The fix is a queen move: eight directional jumps aimed at the argmax of a predicted density map, scored by the same Q-function. The mixture is now three models, each winning in its own regime.
-
Active Perception II: Adding a Visited Map to a GRU DRQN
The baseline GRU DRQN assumed its 48-unit hidden state could carry 'where the agent has already been' across a 400-cell grid. It could not. A 400-bit visited bitmap added as a third input tensor lifts the matched-pair ratio by 19%. The richer variant with three map channels lost the gain back — and that result is the load-bearing finding of this work.
-
Active Perception I: A GRU DRQN for Discovery Under a Noisy Sensor
The moment the value of the next action depends on what was just done, you need recurrence. That is the entire argument for a GRU Deep Recurrent Q-Network: a learned memory of the recent trajectory, trained with the standard DQN stabilisers.
-
Deep Q-Networks
DQN is the algorithm that taught a single neural network to beat human players across 49 Atari games from raw pixels. This is the preface: what Q-learning is, why a network breaks it, and the three tricks that put it back together.
-
MoB: Mixture of Bandits for Sequential Exploring Problems with Stateless Agents and Limited Input
A stateless agent with a noisy local sensor is a blind, deaf, one-legged minesweeper — fine when the world is uniform, systematically wrong when the world is clustered. The fix is not a second expert, but a learned gate that knows when to disable the planner's long leap.
-
The Sequential Recommender: Fitted Q-Iteration
The moment an action changes what the next decision can see or do, the bandit is myopic and the problem becomes sequential reinforcement learning.
-
Building the Bandit Recommender
The practical companion to the bandits essay. We state the recommendation problem the way a product team would, use the fishing simulator as a running analogy, dig into the hashing trick that makes the engine streamable, and compare ε-greedy against LinUCB on a real workload. With a brief survey of the rest of the bandit family.
-
Contextual Bandits and Their Ethical Use Cases
A contextual bandit is a machine-learning algorithm that makes billions of dollars a year and is roughly one-tenth as hyped as GPT. The TikTok doomscroll, the ad that follows you around for a week, the song Spotify queues next, Addictive Games — all of these are bandit-family algorithms.
-
The Essence of Context Bandits
What a contextual bandit actually optimises, in one diagram and a handful of equations. With live math and a flow of the explore-exploit loop.
-
Primer to Gradient Descent
Gradient descent is the most used algorithm in machine learning, and the idea behind it is almost embarrassingly simple. This is the preface: without it, you won't understand 90% of ML.
-
Quartiles, Ridge, and Lasso
Three fundamentals that look unrelated but tell one story: how we describe data, and how we keep a model from believing it too much.
-
Evaluation Metrics
Five metrics, from R² to ROC/AUC, on interactive graphs.