SkillMutator: Benchmarking and Defending Language-and-Code Cross-modal Attacks on LLM Agent Skills

摘要

Large language model (LLM) agents increasingly extend their capabilities at runtime by loading Agent Skills: composite artifacts that pair a natural-language specification, SKILL.md, with executable scripts and task-specific resources. Because a skill’s behavior is determined jointly by natural-language instructions and executable behavior, assessing its safety requires reasoning across both modalities. This makes skills useful, but also creates a language-and-code cross-modal attack surface. An attacker can present a benign-looking workflow in SKILL.md while embedding implicit directives that steer the agent to exfiltrate sensitive files even when the accompanying scripts and resources appear harmless. Despite the rapid growth of skill marketplaces, this attack surface remains understudied. Prior work typically treats skills either as prompt-injection vectors or as code artifacts for static scanning, leaving attacks that emerge from the interaction between two modalities largely unmeasured. In our evaluation, an open-source skill scanner detects only $2\%$ – $8\%$ of such attacks, while a commercial scanner detects only $9\%$ – $17\%$ .