Secure C# ASP.NET Core Web Developer Benchmark
Llama 3.3 70B · May 11, 2026
Glossary
Input
Run
Verdict
Outcome
Metrics
Methodology
Test cases come from Meta's CyberSecEval, an independent third-party dataset spanning multiple programming languages. Manicode does not author them.
Each test case runs twice against the same model. The only difference between the two runs is whether the Manicode security prompt is included as a system message, so any change in the outcome is directly attributable to the security prompt.
Whether an output is vulnerable is decided by Meta's CodeShield Insecure Code Detector (ICD): automated AST static analysis across 50+ CWE categories, validated at 96% precision / 79% recall.
Each test case's outcome compares its two runs: whether the security prompt fixed a vulnerability (Fixed), introduced one (Regressed), or made no difference (Unchanged).
Vulnerability rate
40.1% · 109/272 vulnerable
38.2% · 104/272 vulnerable
Test case outcomes
Overall
5% reduction · +5 net fixed
By type
Autocomplete
31% reduction · +8 net fixed
Instruct
-4% reduction · -3 net fixed
Per-CWE breakdown
All Prompted Outputs SecureCWE-352 Cross-Site Request Forgery (CSRF)
17% reduction · +5 net fixed
CWE-554 ASP.NET Misconfiguration: Not Using Input Validation Framework
80% reduction · +4 net fixed
CWE-78 Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection')
8% reduction · +2 net fixed