Prompt Worms: The Replication of Adversarial Instructions

When an instruction becomes a propagating carrier

Consider an assistant processing a document for an ordinary user task. An instruction embedded in that document changes the assistant’s behavior and causes it to place a corresponding instruction into another artifact. A later assistant consumes that artifact and repeats the process. The security failure has acquired a lineage: an affected application helps create the source of a subsequent failure.

This is the distinguishing feature of a prompt worm. Its carrier may be text, while the application provides the model, tools, storage, and communication channels needed for reproduction. The security question is whether adversarial instructions can repeatedly cross from content into authority and then back into transferable content.

For VXHEAVEN’s taxonomy, we use prompt worm as a mechanism-level category. RAG and memory describe particular pathways through which the mechanism can operate. The definitions and metrics below are an analytical framework; experimental claims remain attached to the systems in which they were observed.

Definition and classification boundary

A prompt worm is an adversarial propagation process in which an instruction-bearing carrier causes an AI application to produce a descendant carrier that can induce further unauthorized carrier production when processed in a compatible context.

Four conditions matter: exposure to the carrier, a deviation from authorized behavior, causal production of a descendant, and preservation of the descendant’s reproductive capability. Unauthorized reproduction can itself be the adverse outcome; a separate theft or destruction payload is not required by this definition.

A successful injection that changes one answer establishes behavioral influence. An output containing a copy of the injection establishes candidate reproduction. Observing that output induce another reproduction step provides stronger evidence of a viable lineage. Claims of multigeneration propagation should identify the actual generations observed.

Several neighboring phenomena therefore need separate labels:

• Repeated exposure: many requests encounter the same original poisoned source. The number of victims may increase without any descendant carrier being created.
• Ordinary copying: an archive or synchronization service duplicates content according to its authorized function. Duplication alone does not establish adversarially induced reproduction.
• Persistent influence: one stored instruction affects later sessions. Persistence becomes part of a worm lifecycle when it contributes to further viable carrier production.
• Information contamination: an incorrect statement is repeated or summarized. A worm claim additionally requires a reproducing instruction mechanism, not merely an expanding factual error.

Quoting an attack for analysis is also insufficient evidence of activation. A resulting quotation can still be hazardous to a poorly isolated downstream consumer, but its existence does not prove that the quoting application followed the attack.

What the research demonstrates

Greshake and colleagues’ 2023 work established indirect prompt injection as an application security problem: attacker-controlled material can reach an LLM through external data rather than the user’s direct instructions. Their taxonomy includes worming and information ecosystem contamination. This supplies the trust-boundary foundation for studying propagation. Greshake et al..

Morris II gives direct evidence of adversarial self-replicating prompts in connected GenAI applications. Its revised paper evaluates propagation through RAG-based email assistants, including chains of confidential-data extraction. It examines factors such as context size, retrieval configuration, and propagation hops. The same paper evaluates a proposed guardrail, Virtual Donkey, under its experimental conditions. These results establish feasibility within the evaluated ecosystem, rather than a universal infection rate for email assistants or current models. Cohen, Bitton, and Nassi.

OpenAI’s September 25, 2026 report describes self-replicating injections found during adversarial training and evaluation, including reproduction through email, files, and code comments. It explicitly reports no observed impact outside simulated tool calls. The email and filesystem findings involved internal research checkpoints; a separate multi-hop evaluation used GPT-5.5. The report supports the existence of the behavior under these conditions, not an account of an uncontrolled outbreak. OpenAI research report.

For taxonomy, the common observation is application-mediated reproduction of instructions. Whether a particular trial demonstrates a copied carrier, a viable descendant, or sustained spread still depends on its recorded downstream behavior.

The host is an application boundary

The biological analogy becomes misleading if the host is assumed to be the model alone. A model processes context; an application decides what enters that context, which outputs become actions, and where generated artifacts are stored or delivered.

Define a host as an application instance with a specified state and authorization boundary. Two assistants using the same model service may be distinct hosts if they have separate accounts and stores. Conversely, several sessions sharing one memory store may represent repeated exposure within one host. A study must choose and document this unit before counting infections.

The minimal threat model gives an adversary influence over some lower-trust material that an application can consume. It does not assume access to the system prompt, model weights, all user data, or arbitrary destination accounts. The available output paths determine whether an initial behavioral failure can produce a descendant.

In a hypothetical collaboration workflow, one assistant might create an affected draft that another assistant later reads. This becomes an inter-host propagation event only if the second application’s behavior is causally influenced and continues the lineage. If the draft remains unused, it is an available carrier. If a person manually transfers it, that human mediation belongs in the event record.

Likewise, zero-click should describe a particular activation path. Automated consumption can remove the need for a dedicated victim click, while still depending on an existing task, connector, or indexing process. It does not imply that any arbitrary system will process the carrier spontaneously.

Lifecycle and failure points

The lifecycle can be represented without specifying an attack payload:

Carrier available in a lower-trust source
                  |
                  v
Admission into an application's context
                  |
                  v
Unauthorized behavioral influence
                  |
                  v
Production of a descendant carrier
                  |
                  v
Admission into a later processing context
                  |
                  v
Further unauthorized carrier production

Availability and admission are different events. A document can exist in a corpus without being retrieved; a message can be delivered without being read by an assistant. Exposure estimates should use the actual context admitted to the model, not the size of a searchable repository.

Activation is also distinct from export. A model may propose an unauthorized write while an application gate rejects it. Conversely, a legitimate write operation may carry unauthorized instructions in its body. Restricting the tool name alone does not establish that the destination, content, and purpose are authorized.

Finally, a descendant can lose viability. Truncation, changed context, a different recipient policy, or a missing supporting artifact can prevent continuation. These are observed failure modes to measure, not reasons to assume either inevitable spread or automatic safety.

Semantic replication and lineage

A carrier can reproduce verbatim, but byte identity is not the defining property. Generated wording may change while preserving the instruction’s behavioral effect. Conversely, identical text may remain inert in one application and activate in another.

Let \(c_g\) denote a carrier in generation \(g\), and let \(x_i\) describe the receiving application’s task, admitted context, model configuration, and authorization policy. Let \(\mathcal{T}_i\) represent the complete application transition, including generation, action enforcement, and artifact handling. For a single candidate branch, allowing a null outcome when no descendant is produced, write:

$$ c_{g+1} \sim \mathcal{T}_i(\cdot \mid c_g,x_i), \qquad c_g \rightsquigarrow_i c_{g+1}. $$

The relation on the right is recorded only when evidence attributes production of the candidate descendant to the carrier’s unauthorized influence. A normal quotation or an independently inserted copy does not satisfy that condition merely because its bytes match.

Viability is then tested in a declared receiving context: does the descendant induce another unauthorized reproduction step? It is conditional and probabilistic, rather than an intrinsic yes-or-no property of a string. A carrier may also depend on several accompanying artifacts; the evaluated unit should then be the necessary bundle, not whichever fragment is easiest to count.

This distinction gives semantic replication a precise role. The relevant inheritance is a capacity to reproduce the adversarial behavior under stated conditions. Similarity scores can help locate related artifacts, but behavioral evidence and provenance establish the lineage.

Changed wording is not automatically adaptation. An improvement claim requires comparison across controlled conditions. No modification of model weights is implied by a carrier being rewritten, remembered, or passed to another model.

Position within the VXHEAVEN taxonomy

Prompt, RAG, and memory labels answer related but different questions. The first identifies the reproducing instruction mechanism; the others identify how carriers are encountered or retained.

These are overlapping descriptors, not mutually exclusive species. A message could enter a knowledge corpus, influence a retrieved answer, and then become persistent agent state. Each transition introduces another admission or authorization boundary that can stop the lineage.

Weight-bearing replication is a different axis. It concerns transfer and deployment of inference resources. A prompt worm can use the recipient’s existing model without moving weights or creating an independently functioning inference unit. An incident containing both mechanisms should record both, rather than infer one from the other.

For consistent case records, also record the carrier modality, the permission used to export it, the persistence location, and the degree of human mediation. A text-oriented category name should not erase an image or other model-consumable artifact from the evidence.

Measuring reproduction rather than output similarity

A single-hop attack-success rate answers whether a particular exposure produces a defined effect. It does not answer how many viable lineages an affected host establishes elsewhere.

For a parent host \(i\), let \(Z_{ij}(H)=1\) when it causally establishes a viable descendant lineage in a previously unaffected host \(j\) during horizon \(H\), and zero otherwise. A host-level reproduction measure is:

$$ R_{\mathrm{AI}}^{\mathrm{host}}(H) = \mathbb{E}_{i}\left[\sum_{j\ne i} Z_{ij}(H)\right]. $$

This is a proposed metric, not an estimate from the cited studies. Specify the parent cohort, receiving population, definition of viability, and observation window. Count each newly affected destination once within that window, even if it receives several carriers. Where several parents contribute, use a declared attribution rule or leave the attribution unresolved; do not give every parent full credit for the same destination.

Carrier counts remain useful, but should be reported separately. A host may produce many files without establishing any viable lineage. Local recurrence also deserves a separate measure: reactivation in the same persistent state is not automatically another newly affected host.

Useful companion measurements include the fraction of admitted carriers that activate, the fraction of generated descendants that remain viable, observed generation depth, and elapsed time between generations. Include unsuccessful trials and parents with zero offspring. Account for incomplete follow-up, and cluster uncertainty estimates by independent trial or lineage when descendants share conditions.

A finite-window value above one does not establish an epidemic threshold by itself. Sustained-growth interpretations require assumptions about removals, generation times, available recipients, and dependence between events. A central shared store can make many apparent transmission events consequences of one common source.

Defensive architecture

OWASP’s prompt-injection guidance recommends separating untrusted content, limiting privileges, validating outputs, and applying human oversight to high-risk operations. It also cautions that RAG does not eliminate prompt-injection risk. These are useful baseline controls; worm containment additionally requires attention to the routes through which generated artifacts become future inputs. OWASP LLM01:2025.

The following design priorities follow from the lifecycle above.

Preserve authority across transformations. Source content should retain its provenance when it is summarized, indexed, or stored in memory. An assistant-generated summary is not automatically an authorized policy. Labels can help a model interpret context, but enforcement must not depend solely on the model accepting those labels.

Constrain writes by task and destination. Distinguish permission to read a document from permission to alter it, contact its author, or copy its contents elsewhere. Application code should validate action scope against authenticated user intent. Schema validation can reject malformed arguments; it cannot by itself establish that a well-formed message is appropriate to send.

Treat durable state as a separate boundary. Writes to shared knowledge and agent memory deserve controls appropriate to their later influence. Keep source material distinct from approved instructions, maintain version history, and make suspicious updates reversible. A single unsafe memory write can outlive the task that created it.

Retain enough lineage information to contain descendants. Record source identifiers, admitted content, generated artifact identifiers, action decisions, and affected stores with suitable privacy controls. If a seed is removed, check whether derived artifacts remain available. Clearing the current conversation is insufficient when future contexts can retrieve a surviving descendant.

Detection should combine these records with behavioral evidence. A hash can identify an exact known carrier, but cannot establish the absence of a rewritten variant. A semantic detector can produce false positives on research quotations and legitimate instructions. Its operational value depends on measured errors, the action taken after detection, and the surrounding authorization controls.

Safe experimental methodology

Use isolated application instances, synthetic accounts, and harmless outcomes such as an inert marker in a sandbox-only draft. Replace real communication tools with local test doubles. Deny external delivery, enforce a fixed recipient allowlist, and cap generations, artifacts, and runtime through an independent supervisor.

Predeclare the legitimate task and what would count as an unauthorized deviation. If the evaluator explicitly asks the defender to copy a marker, successful copying measures instruction following rather than injection susceptibility. The test objective must preserve that distinction.

Separate three observations: an initial behavioral deviation, creation of a candidate descendant, and continuation caused by that descendant. Test descendants in fresh recipient contexts where the original seed is unavailable except through the descendant under examination. This helps distinguish a lineage from repeated reads of a common poisoned source.

Include matched benign documents, ordinary quotations, and a no-carrier baseline. Record the precise model version, task distribution, admitted context, tool policy, memory state, and any supervisor intervention. If the harness transports a draft to the next recipient, report controlled transport; do not describe it as autonomous delivery by the agent.

Assess defenses against both propagation and legitimate task completion. A system that blocks every write may prevent transmission while failing its intended function. Report the tradeoff, uncertainty, and failed trials alongside successful cases. Public reporting can describe mechanisms and aggregate results without distributing reusable attack prompts or connecting experiments to live recipients.

The defining evidence is a viable descendant

Prompt worms place the reproduction mechanism in the relationship between language, application authority, and reusable artifacts. The model need not copy itself, change its weights, or acquire an intention to spread. The application can supply the resources that make an instruction-bearing lineage possible.

The most useful diagnostic question is therefore: did this carrier cause a new carrier that can continue the unauthorized reproduction process? Answering it requires provenance and downstream behavior. That evidence distinguishes a worm from a successful injection, an old poisoned document, or a harmless copy of suspicious text.

References

1. Kai Greshake et al. Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv:2302.12173, 2023.
2. Stav Cohen, Ron Bitton, and Ben Nassi. Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications. arXiv:2403.02817, first submitted March 5, 2024; version 2, January 30, 2025.
3. OpenAI. Self-replicating prompt injections exist. Disclosure and listed update date September 25, 2026; accessed October 15, 2026.
4. OWASP Gen AI Security Project. LLM01:2025 Prompt Injection. Accessed October 15, 2026.