blog
ATTACK

Testing Muse Ember in AI SAST: Higher Scores with Less Scan Time

Posted 
November 16, 2026
|
0
 min
Pixel art image of an old school computer with terminal up showing vulnerabilities

TL;DR

We recently shared Muse Ember's results in agentic pentesting. We wanted to put it to the test in Arnica's AI SAST harness, where the goal is to identify vulnerabilities in source code.

When a new model is released, we put it through rigorous evaluations and benchmarking before considering it for production. That's especially important in AI SAST, which is broadly deployed and protects our customers' source code every day.

Across multiple benchmark screenings and runs, Muse Ember at High reasoning outscored GPT-6-Luna at Max reasoning while using approximately 80% less total scan time.

Better quality at a fraction of the time.

Methodology

We initially screened six model-and-reasoning configurations against roughly a quarter of our benchmark cases. This narrowed the field to three top contenders:

  • Muse Ember High
  • Muse Ember Max
  • GPT-6-Luna Max

GPT-6-Luna High was eliminated during the qualifying round.

Each of the three finalists received two additional executions on the same inputs, followed by three executions across the full testing suite.

The evaluation covered four public and internal benchmarks, weighted at 35%, 35%, 20%, and 10%, respectively.

What Stood Out

Findings count represents distinct labels found across any run. Scan time and cost are summed across all runs, while benchmark scores are averaged.

Metric Ember High Ember Max Luna Max
Weighted benchmark score 87.21% 89.56% 85.38%
Real CVEs found 25 of 26 22 of 26 23 of 26
Public-benchmark true positives found 334 343 327
Total scan time 9h 49m 13h 55m 49h 46m
Estimated total scan cost $28.06 $49.00 $169.76
Cost per real CVE found $0.68 $1.41 $4.84
Cost per public-benchmark true positive $0.033 $0.053 $0.179

Ember High found more real CVE labels than Luna Max in approximately 80% less time and at about one-sixth of the cost.

What This Means for AI SAST

Muse Ember Max achieved the highest overall benchmark score, but Muse Ember High delivered the strongest balance of cost, speed, and findings. That makes it our top choice for implementation in AI SAST.

At Arnica, a model earns its place the same way: tested repeatedly, against real code, using the actual scanner.

This time, the highest-reasoning setting wasn't the best choice. A more cost-efficient configuration delivered better results against our real-world evaluation criteria.

Muse Ember High is going into AI SAST!

>>> See Arnica AI SAST in action

Frequently Asked Questions

What is AI SAST?

AI SAST (AI-powered Static Application Security Testing) uses artificial intelligence and large language models to analyze source code and identify potential security vulnerabilities. It helps security teams detect risks in code earlier in the development lifecycle.

How was Muse Ember evaluated for AI SAST?

Muse Ember was evaluated using Arnica's AI SAST benchmarking harness across four public and internal benchmarks. We compared multiple model and reasoning configurations, measuring weighted benchmark scores, real CVE detection, public-benchmark true positives, scan time, and estimated cost.

How did Muse Ember High compare to GPT-6-Luna Max?

Muse Ember High achieved an 87.21% weighted benchmark score, compared to 85.38% for GPT-6-Luna Max. It also identified 25 of 26 real CVEs compared to Luna Max's 23, while using approximately 80% less total scan time and costing about one-sixth as much.

Why did Arnica choose Muse Ember High over Muse Ember Max?

Although Muse Ember Max achieved the highest overall benchmark score at 89.56%, Muse Ember High offered a stronger balance of vulnerability detection, speed, and cost efficiency. It identified more real CVEs than either alternative while requiring less scan time and lower estimated costs.

Why does scan time matter in AI SAST?

Scan time directly impacts how quickly security teams can identify and address vulnerabilities. Faster analysis can help teams scale security testing across more repositories and development workflows without introducing unnecessary delays or costs.

Is a higher reasoning setting always better for AI code security?

Not necessarily. Higher reasoning settings can improve benchmark performance, but they may also increase processing time and cost. Arnica's testing demonstrated that a lower reasoning configuration can deliver better practical results when evaluated across detection quality, speed, and cost.

What makes Arnica AI SAST different?

Arnica AI SAST uses AI to identify security vulnerabilities in source code, helping teams detect and address risks earlier in the software development lifecycle. Combined with Arnica's broader application security platform, teams can prioritize findings and integrate security into developer workflows.

Reduce Risk and Accelerate Velocity

Integrate Arnica ChatOps with your development workflow to eliminate risks before they ever reach production.  

Try Arnica