← Back to Cross-Model Leaderboard
MODEL EVALUATION DASHBOARD

Qwen3.8-27B

Evaluating agent task completion, skill acceleration, latency, and token economics on 31 benchmark tasks.
Model: openai/qwen3.8-27b-sglang • Sampling: k=1 (62 runs total)

🌟 With Skills
77.4%
PASS RATE (24/31)
Pass rate achieved when the agent utilizes AtomisticSkills domain tools and workflows.
86m
MEAN LATENCY
Mean wall-clock execution time for the agent to complete the task with domain skills.
110.6k
MEAN OUT TOKENS
Mean output tokens generated by the LLM (total completion tokens, including reasoning/thinking tokens where applicable) when utilizing domain skills.
96.9%
CACHE HIT RATE
Percentage of input tokens served from provider KV cache during with-skills runs (Cached Tokens / Total Input Tokens).
⚖️ No Skills
51.6%
PASS RATE (16/31)
Baseline pass rate achieved when the agent attempts tasks using only general coding tools without domain skills.
112m
MEAN LATENCY
Mean wall-clock execution time for the agent without domain skills.
155.1k
MEAN OUT TOKENS
Mean output tokens generated without domain skills (total completion tokens, including reasoning/thinking tokens). Without structured skills, models frequently burn more tokens exploring.
95.9%
CACHE HIT RATE
Percentage of input tokens served from provider KV cache during baseline no-skills runs.
+25.8%
SKILL BOOST Δ
Absolute pass rate gain achieved over the no-skills baseline: (With-Skills Pass) - (No-Skills Pass).
$0.00
TOTAL COST (62 RUNS)
Total LLM inference cost computed from provider token pricing, incorporating cached input discounts.
Discipline Task Name With Skills No Skills All Solvers Time (With / No) Out Tokens (With / No) Action
chemistry bde-weak-bonds 1/1 0/1 1/2 (50.0%) 63m/29m 92.4k/43.7k View Details →
chemistry conformer-boltzmann-ranking 1/1 1/1 2/2 (100.0%) 63m/143m 86.7k/137.3k View Details →
chemistry gas-thermochemistry-equilibrium 1/1 0/1 1/2 (50.0%) 73m/74m 123.9k/143.4k View Details →
chemistry ghs-hazard-summary 0/1 0/1 0/2 (0.0%) 36m/28m 38.4k/27.9k View Details →
chemistry ir-spectrum-match 1/1 1/1 2/2 (100.0%) 16m/29m 13.5k/10.9k View Details →
chemistry mof-dac-screening 1/1 0/1 1/2 (50.0%) 33m/24m 43.1k/42.8k View Details →
chemistry nmr-reaction-kinetics 1/1 0/1 1/2 (50.0%) 79m/160m 53.7k/127.1k View Details →
drug-discovery admet-lead-triage 1/1 1/1 2/2 (100.0%) 23m/581s 46.1k/21.8k View Details →
drug-discovery docking-microstate-enrichment 0/1 0/1 0/2 (0.0%) 124m/95m 99.1k/57.1k View Details →
drug-discovery ecfp-analog-diversity 1/1 1/1 2/2 (100.0%) 35m/29m 15.8k/10.1k View Details →
drug-discovery ligand-box-definition 1/1 1/1 2/2 (100.0%) 24m/15m 30.4k/21.4k View Details →
drug-discovery ligand-md-stability 1/1 1/1 2/2 (100.0%) 499s/28m 17.0k/46.8k View Details →
drug-discovery pose-quality-filter 1/1 1/1 2/2 (100.0%) 13m/13m 21.3k/19.1k View Details →
drug-discovery symmetry-redocking-rmsd 1/1 1/1 2/2 (100.0%) 260s/29m 7.1k/45.4k View Details →
machine-learning committee-uncertainty-flagging 1/1 1/1 2/2 (100.0%) 366s/308s 8.5k/6.7k View Details →
machine-learning mlip-error-benchmark 0/1 0/1 0/2 (0.0%) 10m/17m 20.5k/32.2k View Details →
materials-science camgsi-elasticity-vrh 0/1 0/1 0/2 (0.0%) 296m/486m 459.9k/812.8k View Details →
materials-science co-adsorption-cu111 1/1 1/1 2/2 (100.0%) 37m/25m 57.9k/40.3k View Details →
materials-science convex-hull-stability 1/1 0/1 1/2 (50.0%) 54m/75m 100.2k/135.4k View Details →
materials-science crconi-order-disorder 1/1 0/1 1/2 (50.0%) 588m/368m 787.1k/559.2k View Details →
materials-science electrochemical-window 1/1 0/1 1/2 (50.0%) 43m/215m 65.7k/339.1k View Details →
materials-science li-qha-simulation 0/1 0/1 0/2 (0.0%) 170m/86m 272.0k/141.6k View Details →
materials-science lifepo4-intercalation-voltage 0/1 0/1 0/2 (0.0%) 21m/45m 40.6k/81.7k View Details →
materials-science mgo-vacancy-energy 1/1 1/1 2/2 (100.0%) 68m/68m 114.0k/115.2k View Details →
materials-science ni3al-surface-energy 0/1 0/1 0/2 (0.0%) 252m/23m 307.4k/44.7k View Details →
materials-science nist-janaf-query 1/1 1/1 2/2 (100.0%) 110s/69s 2.7k/1.9k View Details →
materials-science polymorph-free-energy 1/1 1/1 2/2 (100.0%) 352m/1076m 228.2k/1280.9k View Details →
materials-science si-eos-fit 1/1 1/1 2/2 (100.0%) 58m/31m 88.1k/42.7k View Details →
materials-science sic-thermal-conductivity 1/1 0/1 1/2 (50.0%) 47m/53m 28.4k/77.8k View Details →
materials-science synthesis-precursor-recommendation 1/1 1/1 2/2 (100.0%) 12m/28m 24.0k/54.1k View Details →
materials-science xrd-mixture-phase-fit 1/1 1/1 2/2 (100.0%) 70m/173m 136.4k/287.6k View Details →