Explore completed research outputs

Continual learning methods frequently reduce forgetting by adding replay, regularization, or routing constraints, yet activation functions are usually treated as fixed nonlinearities rather than adaptive components of the retention mechanism. We study a task-agnostic setting where task identities are unavailable and dynamic activation updates must operate under matched replay and optimization budgets. In this setting, we define an interference-gated projected activation update, formalize a one-step forgetting-proxy comparison against static-gate baselines, and introduce a fairness-normalized attribution predicate that blocks invalid comparisons when memory or optimization controls differ. The empirical study executes seed-aggregated runs on reduced Split CIFAR-100, reduced Split TinyImageNet, and a third real fallback online digits stream, with static GELU, ReLU, Mish, A-GEM, EWC, and ablated dynamic variants as comparators. Results show that the full dynamic gate is indistinguishable from static GELU in the executed matrix, while replay-family changes can worsen forgetting and destabilize backward transfer, yielding mixed support for retention-improvement claims. Symbolic checks confirm local first-order identities and fairness-accounting completeness but expose a failed simplification in one projection-bound audit, limiting formal conclusions to local validity windows. These findings sharpen the boundary between plausible mechanism intuition and supported claim scope: activation-state adaptation can be audited rigorously, but global forgetting guarantees remain unresolved without stronger alignment conditions and broader benchmark execution.

Continual learning systems are increasingly limited by memory behavior rather than arithmetic throughput: the same memory substrate must support stable recall and adaptive updates while respecting strict latency and energy constraints. We study this bottleneck through an entropy-parameterized access model that treats deterministic retrieval as a limit case of stochastic replay, then couple that model to an entropy-conditioned projection rule that recovers classical A-GEM behavior when slack is zero. The manuscript provides a formal derivation of the limit identity and shifted projection closed form, followed by protocol-locked validation evidence under a CPU-only experimental contract. Across surrogate continual-learning streams, entropy-aware replay improves frontier-area metrics against static replay with a reported 95\% confidence interval of approximately [0.0533,0.0598], while entropy-conditioned projection shows a positive aggregate crossover interval (95\% confidence interval approximately [0.028,0.194]) relative to fixed A-GEM in validated operating regions. Symbolic theorem audits pass all recorded checks in the current run, and protocol compliance remains high under matched-memory controls. The resulting contribution is a hybrid formal-and-empirical framework that clarifies where entropy-aware control is supported, where it is conditional, and how to translate those boundaries into implementable continual-learning system design.

Continual learning systems are increasingly deployed in settings where data distributions evolve while labels, environments, and downstream requirements remain nonstationary. In these settings, the practical failure mode is not only catastrophic forgetting, but also gradual loss of plasticity, dead-unit accumulation, and unstable internal statistics that amplify optimization brittleness across long horizons. We study a model class in which continual-learning inductive bias is encoded directly in the activation function rather than in replay buffers, explicit task identifiers, or per-task masks. The proposed mechanism uses a dual-timescale activation parameterization: fast parameters adapt to novelty, while slow anchors preserve utility-weighted structure. We formalize the problem with explicit decision variables, feasible sets, and an online surrogate objective, and we provide two formal results: a bounded-moment theorem under bounded drift and projection assumptions, and a lower-bound proposition establishing an impossibility region for static memoryless activations under persistent alternating conflict. We then evaluate the formal chain with symbolic checks and synthetic continual regimes that expose stress and boundary behavior. Across executed regimes, the method satisfies bounded-variance compliance above 0.95 in bounded drift, improves forgetting relative to GELU by roughly 49\%, and shows lower conflict-regret slope than static baselines. We also report caveats: two planned experiment tracks focused on replay-free competitiveness breadth and compositional probes are not yet executed, so associated claims are scoped as open rather than confirmed. The broader implication is that activation-level state can provide a lightweight, task-agnostic route toward continual adaptation, while formal assumptions remain explicit and testable.