Muse Ember in Agentic Pentesting: Early Access Results

Meta released Muse Ember today, a lightweight model built for agent execution. We had the honor of being an early access testing partner, and we put it to work in the harness behind Arnica Agentic Pentesting before launch. The results were promising.

On the largest application in our benchmark, Meta Muse Ember was both faster and better than our previous frontier model contender. At max effort, it found 31% more known vulnerabilities, completed pentests 54% faster, and used 71% fewer reasoning tokens. Across all five purpose-built applications, it found 5% more known vulnerabilities and completed fully autonomous pentests 57% faster (15 minutes on average versus 35), using about 74% fewer reasoning tokens. Given these promising results, we are considering switching to Muse Ember as our default model as part of the launch of Arnica Agentic Pentesting.

Why Ember fits agentic pentesting

Meta describes Ember as its lightest and most cost-efficient model yet, aimed at repetitive agent work like tool calls, coding subtasks, and routing. In its release, Meta called it “designed for complex reasoning execution and the work agents repeat all day,” as TechCrunch reported.

It is priced at Anthropic’s Claude Haiku 5.5 and OpenAI’s GPT-6 Luna prices but our benchmarks show both lower cost per task completed, and faster completion times, with equal or better recall and precision.

What stood out

At max effort, Ember was faster, found more, and used fewer reasoning tokens than our best frontier model baseline.

• Large-app coverage: On our largest application, it found an average of 80.7% of planted vulnerabilities, the highest recall of any model we tested. The comparison model’s max setting averaged 61.4%.
• Speed: Across all five apps, pentests finished 2.3× faster.
• Reasoning efficiency: Across all five applications, pentests consumed 74% fewer reasoning tokens on average (152K vs. 590K), making them 3.9× more token-efficient.
• Overall recall: Average recall across all apps was 83% versus 79%.

Where it can improve

• Smaller apps: At max effort, recall trailed on two of the smaller apps (73% vs. 83% and 90% vs. 92%). On the first, 2 of 4 max runs ended early at 6 to 7 minutes. The full-length runs hit 80%. The model tended to declare it’s done too early, but this can be, but later runs did not have this issue.
• Finding precision: It reported more low-impact hardening observations as findings, such as cookie flags, missing headers, and self-XSS. Our grader counts those as not exploitable, which put precision on one app at 86% versus 94%.
• Effort settings: Max effort didn’t always help. On one smaller app, recall averaged 70% at max versus 79% at high.
• Reliability: We recorded 4 failed calls out of 24,416 to Ember, versus zero on the other models, which can be understood for a new launch.

Conclusion

Meta Muse Ember is a fast, cost effective and capable model, comparable and even surpassing equally priced models from other providers and is an exciting new powertool that when used right, can potentially help our customers enable always-on pen-testing and find the vulnerability before the AI driven attackers will.

Arnica Agentic Pentesting is available in limited preview, if you're interested in always-on protection and peace of mind in a cost effective way, hit the button at the bottom of this page, and let us show you how it's done!