š¹ Agent Red-Teaming Challenge ā test direct & indirect attacks on anonymous frontier models! š¹ $130K+ in prizes & giveaways ā co-sponsored by OpenAI & supported by UK AI Security Institute š¬š§ š¹ March 8 ā April 6 ā fresh exploits = fresh rewards!
Why Join? āļø Neutral judging ā UK AISI & automated judges ensure fairness šÆ No pre-trained defenses ā a true red-teaming battlefield š» 5 Apple laptops up for grabs ā increase chances by inviting friends!
š¹ Agent Red-Teaming Challenge ā test direct & indirect attacks on anonymous frontier models! š¹ $130K+ in prizes & giveaways ā co-sponsored by OpenAI & supported by UK AI Security Institute š¬š§ š¹ March 8 ā April 6 ā fresh exploits = fresh rewards!
Why Join? āļø Neutral judging ā UK AISI & automated judges ensure fairness šÆ No pre-trained defenses ā a true red-teaming battlefield š» 5 Apple laptops up for grabs ā increase chances by inviting friends!
Collaboration between UK AI Safety Institute and Gray Swan AI to create a dataset for measuring harmfulness of LLM agents.
The benchmark contains both harmful and benign sets of 11 categories with varied difficulty levels and detailed evaluation, not only testing success rate but also tool level accuracy.
We provide refusal and accuracy metrics across a wide range of models in both no attack and prompt attack scenarios.