-
Notifications
You must be signed in to change notification settings - Fork 587
[DRAFT] Iterative Attack Strategy & Multi-Branch Attack #1097
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces an iterative attack strategy framework that enables multi-turn, multi-branch attacks with minimal changes to the existing strategy architecture. The key innovation is the addition of an IntermediateAttackResult class that allows strategies to return partial results and maintain state between calls.
- Adds
StrategyResultIntermediateclass to enable stateful, multi-step attack execution - Modifies core strategy execution logic to handle intermediate results iteratively
- Implements a complete
MultiBranchAttackstrategy for interactive tree-based attacks
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| pyrit/models/strategy_result.py | Adds StrategyResultIntermediate class with context and final state management |
| pyrit/executor/core/strategy.py | Updates execution loop to handle intermediate results iteratively |
| pyrit/executor/attack/multi_turn/multi_branch_attack.py | Implements new multi-branch attack strategy with tree navigation |
| pyrit/executor/attack/multi_turn/multi_branch_attack copy.py | Alternative implementation of multi-branch attack |
| pyrit/executor/attack/multi_turn/init.py | Exports new multi-branch attack classes |
| doc/code/executor/attack/tap_attack.ipynb | Updated notebook output with different execution results |
| doc/code/executor/attack/multi_branch_attack.ipynb | New documentation notebook for multi-branch attack usage |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Description
Enables iterative, multi-turn attack strategies with minimal changes. Byproduct of attempt to create a multi-branch attack during the Hackathon.
The current execute_with_context_async and perform_async methods defined by Strategy and its subclasses assume an attack is performed with no exposable middle state between attack initiation and attack conclusion. Therefore, these methods return StrategyResultT, which makes it impossible for users creating interactive attacks where one step does not necessarily conclude the attack to extend the methods. Calling an execute... method will return a final result, and for a multibranch attack where users navigate a tree node by node to retry prompts, it is not possible to perform stepwise operations.
My proposed fix is the addition of a subclass to AttackResult, an IntermediateAttackResult. It subclasses AttackResult and holds a StrategyContext object. Subclasses of AttackStrategy can choose to return this instead of an AttackStrategyResult to pass state in between calls of execute_.... This design has a few properties:
Tests and Documentation
None yet.