Meta's latest open release beats every prior open model on HumanEval and achieves near-GPT-4 on MBPP.
Llama 4 Scout, Meta's smaller release in the 4-series, scored 89.2% on HumanEval — the highest of any open-weight model — and came within two points of GPT-4 on MBPP.
Notably it lags the closed flagships on real-world SWE-bench, where post-training and tool integration dominate.
Scout's 89.2% on HumanEval is the highest of any open-weight model to date, and within two points of GPT-4 on MBPP — closing a gap that had been steady for two model generations.
No. On SWE-bench Verified, where post-training and tool integration dominate the score, Scout lags every leading closed model. The gap suggests HumanEval/MBPP are no longer differentiating benchmarks at the frontier.
No. Scout is Meta's smaller release in the 4-series, optimised for accessible local inference. Larger Llama 4 variants exist with different speed/quality tradeoffs.