CMP.7:4.9 - Train a function while changing its internal representation
Use this when current predictions fit but later changes or generalization matter, and the learner’s internal computation can be examined and changed. Define the next task and any earlier responses that must survive. Compare ordinary training and an adequate familiar regularizer under stated budgets. Decide whether you need generalization to new inputs, adaptation within the present family, retention through an update or learning a new family. These require different observations.
A Unified Factored Organizer (UFO) is a low-capacity nonlinear autoencoder used during training. Choose the primary network’s internal quantities, such as node activations or attention weights, and normalize them across the training batch by component. For intracomponent regularization, each vector records one component’s responses at fixed input positions. Preserve those positions across batches. In the reported finite-domain experiment, absent inputs occupy zero-filled positions and contribute no reconstruction loss. Reordering a minibatch must preserve which input each position denotes. An intercomponent vector records one input’s responses across a layer’s fixed components. For attention, identify heads and query/key positions consistently. The reported fixed-length sequences preserve these roles; variable lengths require a further construction. Choose coordinates and storage that are feasible for the domain. Train the primary network on task loss plus a weighted reconstruction loss. Hold the organizer’s parameters fixed in that update while passing the gradient through it. In a separate update, keep the primary network fixed. Within each intracomponent vector, select the lowest-error entries. For intercomponent regularization, select the lowest-error complete vectors within each layer. The organizer receives the complete vectors; selection determines which reconstruction errors contribute to its update. Specify the selection rule and how the reconstruction weight increases from its initial value. The primary network can then perform the task without running the organizer.
Compare the ability to solve the declared next task, retention of protected responses, successful runs and total training cost, including the organizer, its activations and both updates. A reduction in steps to a threshold can still be costly per step. Reconstruction loss alone cannot establish the ability to learn a different family. Sheppard et al. give the training construction and its experimental conditions. Lehman and Stanley separately compare how compatible and incompatible changes and the size of an update affect retention. Use a change whose required effect you can identify to decide whether the next intervention belongs in the representation, the update or the new training task.