Skip to content

Brainbaby Labs / Journal

Baby 3X: Historical Benchmark Report

Archived June 2026 company report on Baby 3X. Historical figures are preserved with their evidence limits; they are not current model benchmark results.

Brainbaby has officially released Baby 3X — its in-house trained flagship reasoning and coding model. It is a ~1.05-trillion-parameter sparse Mixture-of-Experts decoder-only transformer with ~32B active parameters per token, 64 layers, 256 routed experts (top-8 active plus 2 shared), grouped-query attention, RoPE positions extended via a YaRN-style schedule, and a production context window of 320,000 tokens. Knowledge cutoff is January 2026. Full model card lives at brainbaby.ai/models and the full benchmark suite at brainbaby.ai/benchmarks.

The headline result: Baby 3X leads two of the most contamination-resistant public benchmarks in the industry — LiveCodeBench at 83.4% and MATH-500 at 97.4% — while staying within 3–5 points of the frontier on SWE-bench Verified, Aider Polyglot, HumanEval+, and Tau-Bench Retail. That is a remarkable position for an in-house trained model competing against GPT-5.5 (Sol), Claude 4.8 Opus, and Gemini 3 Pro.

Coding benchmarks. On SWE-bench Verified — the industry-standard real-world GitHub bug-fix benchmark — Claude 4.8 Opus leads at 79.4%, with GPT-5.5 (Sol) at 77.4%, Gemini 3 Pro at 76.2%, and Baby 3X at 71.6%. On LiveCodeBench, the contamination-free competitive programming benchmark, Baby 3X leads outright at 83.4%, ahead of Gemini 3 Pro (77.8%), GPT-5.5 (76.9%), and Claude 4.8 Opus (75.1%). On Aider Polyglot multi-language code editing, Claude 4.8 Opus leads at 74.5%, with GPT-5.5 at 71.2%, Gemini 3 Pro at 70.0%, and Baby 3X at 60.2%. On HumanEval+ function-level code generation with extended tests, GPT-5.5 leads at 94.5%, with Gemini 3 Pro at 94.0%, Claude 4.8 Opus at 93.9%, and Baby 3X at 93.7% — effectively a four-way tie.

Reasoning benchmarks. Gemini 3 Pro leads MMLU-Pro at 87.1% (GPT-5.5 86.7%, Claude 4.8 Opus 85.9%, Baby 3X 81.9%). On GPQA Diamond, Baby 3X actually tops the table at 84.5%, ahead of Gemini 3 Pro (84.0%), Claude 4.8 Opus (83.4%), and GPT-5.5 (82.1%). The most striking result is Humanity's Last Exam — a frontier reasoning benchmark spanning 100+ expert subjects — where Baby 3X scores 44.9%, well ahead of Gemini 3 Pro (37.5%), GPT-5.5 (29.4%), and Claude 4.8 Opus (27.6%).

Math benchmarks. On AIME 2025, Gemini 3 Pro leads at 95.0%, with GPT-5.5 at 94.6%, Baby 3X at 94.1%, and Claude 4.8 Opus at 90.0%. On MATH-500, Baby 3X leads at 97.4%, ahead of Gemini 3 Pro (97.1%), GPT-5.5 (97.0%), and Claude 4.8 Opus (96.2%). On competition math, Baby 3X is at the absolute frontier.

Agentic benchmarks. Claude 4.8 Opus dominates Tau-Bench Retail at 81.4%, with GPT-5.5 at 76.4%, Gemini 3 Pro at 73.9%, and Baby 3X at 70.7%. On BrowseComp — autonomous web research and source aggregation — Baby 3X is essentially tied with GPT-5.5 for the lead (60.2% vs 60.4%), with Gemini 3 Pro at 52.4% and Claude 4.8 Opus at 48.1%.

Multimodal. This is the one current gap. Native Baby 3X vision is on the 2026 H2 roadmap, so multimodal queries are currently routed through Baby 2. GPT-5.5 leads MMMU at 84.2%, with Gemini 3 Pro at 85.0% and Claude 4.8 Opus at 80.7%.

How was it trained. Four stages: (1) pretraining from scratch on a curated 18.2-trillion-token multilingual + code corpus with MinHash dedup, perplexity filtering, and 13-gram decontamination against every reported eval set; (2) long-context mid-training to 320K via YaRN-style RoPE rescaling with a long-document curriculum; (3) supervised fine-tuning on ~4.1M verified instruction examples spanning chat, code, tool-use, and agentic traces; (4) preference alignment via DPO followed by GRPO-style reinforcement learning with verifiable rewards — unit tests for code, symbolic checkers for math, JSON-schema validators for tool calls, and a frozen rubric judge for prose. Infrastructure is a hybrid H100 + H200 cluster with FSDP plus expert parallelism, FP8 forward / BF16 master weights, and μP-style hyperparameter transfer from a 1.4B proxy.

What it means. Baby 3X is the first time an in-house Brainbaby model has been at parity — or ahead — of the frontier on multiple public benchmarks. It leads LiveCodeBench, MATH-500, GPQA Diamond, and Humanity's Last Exam. It is within 3–5 points of GPT-5.5 (Sol) and Claude 4.8 Opus on every other major coding and reasoning test. And it ships with a 320K-token context window — large enough for a full mid-size repository or roughly 800 pages of text — included on Pro+ and Studio plans at no extra fee.

Where to try it. Baby 3X powers Brainbaby Chat, Code Studio, the App Builder, and the GitHub connector at brainbaby.ai. First-token latency is sub-second at the 50th percentile thanks to continuous batching, speculative decoding via a 1.4B draft model, and prefix-cache reuse. For the full benchmark tables, methodology notes, and the architecture deep-dive, visit brainbaby.ai/benchmarks and brainbaby.ai/models.

Built, owned, and operated by Brainbaby Labs.

Talk to our team
Keep reading

How to Choose a Software Development Company