GVS5H: five Qwen3.8-27B agents score 92.4% on LiveCodeBench Hard, but is it a fair match for Fable 5?

A research paper and a small open-source replication are circulating under the same claim: five copies of the open-weight Qwen3.8-27B model, coordinating through files on disk, solve hard competitive-programming problems about as well as Claude Fable 5 on its own. The numbers in the paper check out as arithmetic, but the headline compares a harness plus a model with a bare model, on one benchmark, with results the authors report themselves. This article sets out what was measured, what an independent rebuild found, and what the test does not show.

Key points

• The paper "GVS5H" (arXiv:2608.26480, submitted 27 August 2026, revised 21 September) describes a training-free method in which fresh instances of one model split a coding problem and coordinate through a shared file system with a ledger.
• The authors test eleven open and closed-weight models on the 100 most recent hard LiveCodeBench problems and report gains of up to 25.6 percentage points.
• Locally served Qwen3.8-27B rises from 66.8% to 92.4%; the abstract also reports orchestrated Qwen3.8 Flash Next at 93.0% pass@1 against 90.4% for Fable 5, at about 9% of the cost.
• A third-party rebuild called Memdoor scored a mean of 9.8 out of 10 on the first ten Hard problems over five passes, against 9.6 for the authors' own harness and 8.0 for Fable 5 alone, but it tested only ten problems.
• The abstract says some models were "unchanged or worse", and the contest problems may predate the models' training data, so contamination cannot be ruled out.

What happened

The GVS5H paper, by Victor Gao, Vida Khosrowshahi, Ali Khosrowshahi, Xihao Sun, Juhyun Lee and Simon (Sang Won) Lee, proposes what it calls zero-shot self-orchestration. There is no fine-tuning and no special model. A manager starts several fresh instances of the same model, each with an empty context, and they share a working directory that holds a plan, notes and the current solution. The paper calls this the ledger. Transcript analysis, the authors say, attributes the gain to two things: breaking the problem into steps, and keeping context in files rather than in one ever-growing conversation.

The authors evaluated eleven models, open and closed, on the 100 most recent hard problems of LiveCodeBench, a benchmark of programming-contest tasks. The abstract reports improvements of up to 25.6 percentage points "on pinned backends". Two figures drive the discussion. The first is Qwen3.8-27B, served locally, which goes from 66.8% as a single call to 92.4% with the harness (66.8 plus 25.6 is 92.4, so this is the maximum lift the abstract mentions). The second is Qwen3.8 Flash Next, which the abstract reports at 93.0% pass@1 when orchestrated, against 90.4% for Claude Fable 5, at about 9% of the cost. The code in the project's repository is under the MIT licence, and the paper, figures and run data are under CC BY 4.0.

The claim spread through Hacker News and social media in shortened form: "five 27B models beat Fable 5". That summary mixes the two results. The 93.0% figure belongs to Flash Next; the 27B model reached 92.4%, which is above the 90.4% reported for Fable 5, but the comparison is between an orchestrated open model and a single call to a closed one.

Background

LiveCodeBench collects problems from programming-contest sites and tags them by date so that a model can be tested on problems published after its training data ends. That design is the benchmark's main defence against memorisation. The "Hard" tier is where simple prompting fails most often, which is why a jump from 66.8% to 92.4% is striking.

Qwen3.8-27B itself is an open-weight model from Alibaba's Qwen team, listed on Hugging Face under the Apache-2.0 licence with a native context of 262,144 tokens, extensible to 1,000,000 with YaRN scaling. Its model card reports 90.3 on LiveCodeBench v6 for the model alone, against 88.8 for Opus 4.6 Max, which are Alibaba's own figures. It can be served with vLLM or SGLang, both of which expose an OpenAI-compatible API, which is what the harness needs.

Multi-agent scaffolds are not new. Most of them chain roles such as planner, coder and reviewer inside one conversation. The GVS5H difference is that every worker starts clean and communicates only through files, which keeps each context short and makes every step inspectable afterwards.

The numbers

Qwen3.8 Flash Next + GVS5H93.0

Qwen3.8-27B + GVS5H92.4

Claude Fable 5 (single call)90.4

Qwen3.8-27B (single call)66.8

LiveCodeBench Hard pass rate (author-reported, 100 recent hard problems) (%)

	%
Qwen3.8 Flash Next + GVS5H	93.0
Qwen3.8-27B + GVS5H	92.4
Claude Fable 5 (single call)	90.4
Qwen3.8-27B (single call)	66.8

Source: GVS5H paper abstract (arXiv:2608.26480) and project README

The independent rebuild is smaller. Memdoor, a workflow tool by a different developer, reimplemented the loop with five roles (a planner, two ideators, a solver and a manager, plus a repair worker). It follows the authors' prompts, temperatures, a 128K reply cap and up to ten rounds. It ran Qwen3.8-27B through the OpenRouter hosting service, not the authors' own FP8 vLLM setup, and a test script decides success, not a model. Its results on the first ten Hard problems, five passes each:

Pass	Memdoor workflow	Authors' harness	Fable 5 alone
1	10 of 10	9	7
2	10 of 10	10	8
3	10 of 10	10	10
4	10 of 10	9	8
5	9 of 10	10	7
Mean	9.8	9.6	8.0

The Memdoor author reports that 49 of 50 runs solved their problem, that the 50 runs cost $35.79 for 15.3 million output tokens ($38.77 including 16 attempts that were cut off, stopped or paused), that a run took a median of 40 minutes of model time (longest four hours) and a median of $0.39. The Fable 5 column was taken from the authors' files, not rerun in that harness.

How it works

1. A manager reads the problem and writes a plan into the shared directory.
2. Workers, each with a fresh context, read the plan and notes, do one job (propose an approach, write a solution, test it, review it) and write the result back.
3. A ledger records what is done, what failed and what to try next, so the next worker does not repeat work.
4. A script runs the sample tests; the loop stops when they pass or the round limit is reached.

Because the state lives in files, the cost is mostly extra tokens: plan, notes and status files are read and rewritten on each pass. DevelopersDigest's write-up notes that it found no token counts, latency or per-problem cost beyond the 9% figure, and Memdoor's median of 40 minutes per problem shows the time cost is real.

What it means

For developers using open models. The cleanest result is the internal control: the same Qwen3.8-27B model goes from 66.8% to 92.4% with the harness. If that holds up, a 27B open-weight model served locally can recover much of the gap on hard algorithmic problems by spending more tokens and time. Treat it as a strong, reproducible pattern to try, not a guarantee for your codebase.

For the "open beats closed" story. The headline comparison is not controlled. Orchestrated Qwen is compared with single-call Fable 5, so it measures the scaffold as much as the model. DevelopersDigest also points out that the 93.0% figure is for Flash Next, not the 27B model in the title. A fair test would run the same harness around both models; the paper's own framing that some models were "unchanged or worse" suggests the gain depends on the model.

For benchmark watchers. LiveCodeBench Hard is one contest-style suite. It says little about work in a real repository: reading unfamiliar code, handling dependencies, long-running changes.

What we don't know yet

• Whether Fable 5 plus the same harness would score higher still. The reported comparison does not include it.
• How the final answer is chosen from several attempts, and whether pass@1 counts only the selected answer. The pages we could read do not say.
• Contamination. Memdoor's author notes its problems date from March and April 2025, so the model may have seen them in training. We do not know the dates of the paper's 100 problems.
• The cost. The 9% figure has no published breakdown on the pages we read; the full paper may have one.
• Which models got worse, since the abstract only says some did.
• The authors' table of all eleven models. The repository shows results as images, which we could not read, so we did not verify per-model numbers beyond those in the abstract.

What to watch next

• Replications on private or post-cutoff problems, and on repository-level tasks such as fixing real issues.
• Runs of the same harness around other open and closed models, with token and time costs per solved problem.
• Whether tools such as Memdoor and coding assistants adopt the file-ledger pattern.

Sources

• GVS5H: Zero-Shot Self-Orchestration with Ledger-Based Control for Improved LLM Coding Performance (arXiv:2608.26480)
• GVS5H repository and README
• Memdoor team-solve workflow and benchmark notes
• Qwen3.8-27B model card on Hugging Face
• DevelopersDigest: GVS5H, can Qwen3.8-27B match Fable 5 on coding?