AI-assisted summary, reviewed by NullTX editors.
In a recent development, webAI's open-weight TwIL-8B posts 98.2 on MATH-500 and indicates it tops Gemma 4 31B. The card's own footnotes show why the comparison deserves a re-run.
A model small enough to run from a roughly 5 GiB file on a CPU now claims a 98.2 on MATH-500. The most revealing line in its model card isn't a score, though. It's a footnote saying the competitor numbers were never re-run.
Industry observers note that webAI released TwIL-8B as an open-weight model on November 4, and its Hugging Face model card leads with ambitious numbers. The 8.8B-parameter reasoning model is post-trained from IBM's Granite 4.2 8B using full-parameter supervised fine-tuning, followed by GRPO/DAPO reinforcement learning. A final length-aware RL stage produces the "balanced" variant, which is the default.
Sources indicate that the card reports 98.2 on MATH-500 and 88.3 on AIME 2025. It claims TwIL-8B posts the highest MATH-500 and LiveCodeBench scores in its comparison group, which includes Gemma 4 31B at about 3.5 times its size. That claim is worth testing, because small models that beat larger ones are exactly what open-weight people want to hear.
On MATH-500, the card lists TwIL-8B at 98.2, Qwen3.5-9B at 96.6 and Gemma 4 31B at 87.9. On AIME 2025, TwIL-8B scores 88.3 against 87.1 for Gemma 4 31B. On LiveCodeBench, TwIL-8B scores 81.6 against 80.0, though that figure carries a dagger marking it as measured on the verbose variant, not the default balanced one.
The card is also more candid than most. It says TwIL-8B is not a knowledge model. GPQA Diamond comes in at 70.2, trailing Gemma 4 31B by 14.1 points, and the model also trails on MMLU-Pro and HLE. There is no safety or preference tuning beyond what Granite carries, and the license is non-commercial. That honesty makes the claims easier to weigh.
According to recent reports, here is where the headline gets complicated. TwIL-8B's scores come from webAI's own internal high-effort pass@1 runs. The card states that competitor scores are published figures and were not re-run.
In a recent development, gemma 4 31B's 87.9 on MATH-500 comes from Red Hat's evaluation of an FP8 build, using the lighteval harness, not from Google. The card itself notes that this figure is unusually low and may reflect answer extraction. In other words, the 10.3-point gap that anchors the headline compares one team's own harness against a third party's number that the card suspects may be understated.
None of this makes TwIL-8B's result wrong. It means the margin isn't established until both models run under the same harness, prompt and sampling settings. For reference, Google's Gemma 4 model card is the source for several of the other Gemma figures in the table.
It has been revealed that read the whole table and the picture is more balanced than the headline suggests. In the card's head-to-head summary, TwIL-8B beats Gemma 4 31B on 3 of 14 shared benchmarks and loses 11. It leads on MATH-500, LiveCodeBench and AIME 2025, but trails on AIME 2026 (88.3 against 89.2), HMMT Feb 2026 (68.2 against 77.2), GPQA Diamond, MMLU-Pro and IFEval.
The AIME 2025 lead is the shakiest. The card indicates AIME and HMMT sets have 30 to 33 problems, so one problem is worth about 3 points, and gaps of that size sit within noise. A 1.2-point margin is well under one problem, so the result is best read as a tie.
Against the closest same-size rival, Qwen3.5-9B, the card demonstrates the models roughly level on contest math, with TwIL-8B using fewer tokens: about 19.5k per AIME 2026 problem against 31.2k. That efficiency claim may turn out to be the more durable one.
Industry observers note that the card makes replication unusually practical. The Q4_K_M balanced GGUF weighs 4.98 GiB and runs through llama.cpp, with a command given in the card. Larger Q8_0 and F16 builds exist only for the verbose variant. The model also needs a massive generation budget, since a short limit truncates its reasoning and scores far worse. The card suggests settings around 32,768 newly introduced tokens and a 40,960-token context.
Anyone planning a check should expect three catches. First, the card says all its scores were measured on bf16 weights, not the GGUF builds, so a CPU run tests a slightly different artifact, and the card expects small deviations, most likely at Q4_K_M. Second, MATH-500 is 500 problems and each answer can run to tens of thousands of tokens, so CPU testing will be slow. Third, the decoding setup matters, since the card says none of its numbers use low-effort or thinking-off modes.
For a fair comparison, the same prompt, answer extraction and sampling must be applied to Gemma 4 31B too, ideally at full precision.
In a recent development, the card also lists contamination caveats. Its training pools were decontaminated against AIME, HMMT Feb 2025 and 2026, MATH-500, GPQA, MMLU-Pro, HLE, IFEval, IFBench and LiveCodeBench using 13-gram matching. HMMT Nov 2025 and the HumanEval, MBPP and CRUXEval suites weren't in the decontamination set. The MATH-500 and AIME results fall inside the protected group, which counts in the model's favor.
According to recent reports, overall, TwIL-8B looks like a genuinely strong small model for contest math, code and instruction following, and the card is open about its weak spots. The claim to beat Gemma 4 31B is narrower than the headline suggests: a clear lead on one benchmark, a likely tie on another and a loss on most of the rest.
Industry observers note that the useful next step is a same-harness run of both models, with the bf16 weights as a control for the GGUF build. Until independent results appear, the 98.2 is best quoted as webAI's reported figure, and the size-beating framing is better described as unproven than wrong.
Recent findings suggest that disclosure: This is not trading or investment advice. Always do your research before buying any cryptocurrency or investing in any services. Follow us on XΒ @nulltxnews.
Nanbeige4.3-33B matches Qwen3.8-27B at 20 on the Artificial Analysis Intelligence Index but uses 158K output tokens per task against 110K. Here is what that costs locally.Two models just posted the same score. One needed 158,000 output tokens per task to get there. The other needed 110,000.Same Score, Different BillNanbeige4.3-33B, a dense 33-billion-parametβ¦.
One email a week with our new explainers and the news behind them.
More explainers and analysis from NullTX.
