Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies

摘要

Abstract. Self-evolving LLM agent systems, which autonomously update their model parameters, memory, tools, and architectures, introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded, self-amplify across generations, and propagate through populations without sustained attacker access. We present a systematic security and privacy analysis organized around the Module–Lifecycle Attack Surface (MLAS) matrix, which decomposes the attack surface into five functional modules (Brain, Cognitive Resource, Execution, Self-Design, Collective) $\times$ five lifecycle stages (Bootstrap, Propose, Evaluate, Commit, Serve). Analysis of the resulting 25 cells reveals that 17 face critical threats for which no effective defense exists, 7 face high threats where current defenses are insufficient, and only 1 admits partial mitigation, with the Self-Design module uniformly critical due to the optimizer–optimizee collapse. We identify seven cross-cutting amplification effects (generational accumulation, selective amplification, deceptive evolution, Lamarckian propagation, capability ratchet, emergent unpredictability, and optimizer–optimizee collapse) that interact synergistically and cannot be addressed by securing individual modules in isolation. Comparative case studies of two open-source frameworks, OpenClaw (evolution-augmented) and Hermes (evolution-native), demonstrate that evolution-native design activates $3.5\times$ more attack surface cells and achieves a 100% attack persistence rate (40/40 payloads across all CIA+Privacy categories), while the co-located security scanner blocks only 2.5% of attacks on the evolution pathway. Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent, gives rise to entirely new attack classes (self-reward manipulation, evolutionary hijacking, echo-trap exploitation), and renders static defenses structurally inadequate. These findings motivate the urgent development of evolution-aware security frameworks, longitudinal safety monitoring, and formal verification methods for self-modifying systems.