Symbols or Pixels?

Abstract

Can rendering a visual reasoning solution as video justify its inference cost? We compare released VBVR-Pro video models with similarly sized zero-shot Qwen language models on 475 paired VBVR-Pro-Bench samples. With a layout prompt, a text answer requires 480× fewer forward FLOPs at 5B and 1,026× fewer at 27B than a generated video. Under a strict video success threshold, neither pair shows a significant solve-rate difference. Under a lenient threshold that also counts near-miss videos, the video models lead by 9.7 and 13.3 percentage points, with only the 27B difference statistically resolved. The methods solve different samples: at 27B, only 89 of 475 are solved by both, while an oracle choosing either answer would solve 68%. Exploratory splits suggest an advantage for text on non-pixel answers and for video on single-agent paths. These are comparisons of systems, not controlled tests of representation: the language models are zero-shot and graded on final answers, whereas the video models were fine-tuned on VBVR-Pro and receive process-aware scores.

1 Introduction

A video generator can answer a visual reasoning problem by rendering its solution. VBVR-Pro trains this behavior on procedural tasks and scores the resulting trajectory [2, 3]. Many of the same tasks admit a compact answer: a choice, a coordinate, an ordering, or a path. Producing that answer in text avoids generating dozens of frames, but may lose the spatial precision that visual generation supplies.

We ask two questions on the same benchmark samples: how much inference does each approach require, and where does each solve more tasks? We pair the released 5B and 27B-total VBVR-Pro video checkpoints with zero-shot Qwen models of similar size. The text models receive the same first frame and task prompt, then give a final answer. The video models generate a trajectory. Both are graded against the benchmark, with differences in training and scoring stated alongside the results.

The main result is a large cost gap without a clear strict-threshold accuracy gap. The accuracy picture changes when near-miss videos receive credit, and per-sample outcomes reveal complementary strengths. The distinction matters because a final text answer and a generated trajectory can express different kinds of success: one can identify the right object or path without rendering it, while the other can move toward the right state without fully reaching it. Our paired scores expose that difference rather than treating either threshold as a universal definition of reasoning.

2 Related work

Video models can solve some visual tasks by generating their solutions [1]. VBVR and VBVR-Pro turn this idea into procedural training and evaluation suites [2, 3]; other benchmarks examine mazes, puzzles, and games [4, 5]. Process-aware scoring changes how a generated solution is judged [6], which matters when comparing it with a final text answer.

Prior work suggests that representation changes the tasks a model can solve. Visual reasoning can help with paths and geometry [7, 8, 9], while symbolic or programmatic methods can help with discrete tasks [4, 10]. Those studies do not establish the accuracy–cost trade-off for paired text and video answers on VBVR-Pro-Bench; this note measures it directly.

3 Experimental setup

3.1 Benchmark and grading

VBVR-Pro-Bench has 100 tasks with five samples each, split evenly between tasks seen and unseen during VBVR-Pro training. We retain 95 tasks (475 samples) for which both answer types can be graded meaningfully. Three tasks lack a consistent final answer; two more fail a ground-truth-video or trivial-video scorer check.¹ For the remaining tasks, we froze the text answer formats and comparators before grading and checked them against known answers.²

The video models receive a score from 0 to 1 for the whole trajectory.³ We count scores of at least 0.9 as strict successes and also report a 0.7 threshold that includes near misses. Text answers are graded as correct or incorrect against the final answer. All solve rates are task-weighted. Confidence intervals resample tasks;⁴ differences are paired by sample and reported as text minus video in percentage points. Every reported number is computed from per-sample records: the frozen answer specifications, the video v2 scores, the raw text answers and grades, and the outcome categories used in the figures.

3.2 Models and inference

G5 and G27 are the released VBVR-Pro Wan2.2 checkpoints, fine-tuned on VBVR-Pro-SFT. We pair them with zero-shot Qwen3.5-4B and Qwen3.6-27B, and include Qwen3.5-9B as a reference (Table 1). Each text model sees the task prompt and the first frame at the 512² resolution used for video generation. The primary text prompt first describes a structured scene layout, then answers; direct answering is a second condition. Each video model generates an 81-frame solution.⁵ Text decoding is greedy, with a 2,048-token answer limit.⁶

Our video evaluation follows the published VBVR-Pro protocol (Table 2). G5 reproduces its published mean v2 score, and G27 scores slightly below its model card. These mean scores check the reproduction only; the paired results use solve rates on the 95-task set.

3.3 Forward FLOPs

We measure inference cost as the forward FLOPs needed to produce one answer, estimated from model shapes and the measured text token counts.⁷ For video, the DiT processes 5,376 latent tokens at 5B or 21,504 at 27B for 50 denoising steps with two passes per step. For text, the estimate includes the vision encoder and the language-model passes over prompt and generated tokens. It omits the video text encoder and VAE, so the video cost is conservative. FLOPs count arithmetic, not time: text decoding is memory-bound, so the wall-clock gap on real hardware is smaller than the FLOPs gap.

4 Results

4.1 Text needs far fewer FLOPs per answer

The compute gap is large at both scales (Figure 1, Table 3). With the layout prompt, a text answer takes 1.2×10¹³ forward FLOPs at 4B and 8.8×10¹³ at 27B, versus 5.8×10¹⁵ and 9.0×10¹⁶ to generate one video: 480× and 1,026× fewer FLOPs. The video model repeatedly updates a full latent video; the text model processes the first frame and then emits a much shorter answer. This difference persists after accounting for accuracy: at the strict threshold, the layout text runs use 597× and 1,147× fewer FLOPs per solved answer. Direct prompts are shorter still, with ratios of 482× and 1,293× per answer. These are compute ratios rather than speed ratios; text decoding is memory-bound, so the wall-clock gap on real hardware is smaller.

4.2 Accuracy depends on what counts as success

With the layout prompt and a strict video threshold (v2 ≥ 0.9), neither matched pair has a significant solve-rate difference (Table 3). Text solves 27.8% versus video's 22.3% at 5B, and 45.9% versus 41.0% at 27B. The paired differences are +5.5 and +4.8 percentage points, with confidence intervals crossing zero.

At v2 ≥ 0.7, near-miss videos count as successes. Video then leads by 9.7 points at 5B and 13.3 at 27B; only the 27B difference has a confidence interval excluding zero. That 27B gap is concentrated in tasks seen during video fine-tuning (27.2 points in video's favor in-domain, versus a 0.4-point text advantage out-of-domain). The threshold therefore changes the substantive conclusion, especially on familiar tasks: a near-complete trajectory earns credit that a text answer cannot receive.

The layout prompt is a useful way to inspect symbolic reasoning, but it is not a privileged text baseline. Direct answering reaches 33.1% at 4B and 50.3% at 27B under the strict threshold, compared with 27.8% and 45.9% with layouts. Only the 4B direct comparison against G5 has a paired confidence interval entirely above zero (+10.7 points). This sensitivity to prompting limits claims about an inherent advantage of either representation.

Figures 2–5Pair5B: G5 vs Qwen3.5-4B27B: G27 vs Qwen3.6-27BPromptLayoutDirectVideo thresholdv2 ≥ 0.9v2 ≥ 0.7

4.3 Answer type reveals different strengths

The aggregate rates hide a difference by answer type (Figure 2). On non-pixel answers such as labels, counts, orderings, and grid cells, text leads under the strict threshold by 20.8 points at 5B and 18.8 at 27B; both confidence intervals exclude zero. The video model has the clearer advantage on single-agent paths at 27B (29.1 points under the strict threshold) and on pixel-position answers under the lenient threshold (26.7 points at 27B). Its strict-threshold pixel-position advantage is uncertain.

These patterns suggest that the output representation matters, but they do not isolate its effect. Nine of the 11 path tasks are in-domain for the video model. The text models also receive a 512² first frame to match video generation; in earlier runs, higher-resolution input substantially improved their pixel-position answers. The subgroup analysis is exploratory, so the splits are evidence for a follow-up experiment rather than a causal verdict about symbols and pixels.

Text better (CI above 0)Video better (CI below 0)Not significant (CI includes 0)

Loading interactive figure…

4.4 The approaches solve different samples

At 27B under the strict threshold, both models solve only 89 of 475 samples. Text alone solves 129 and video alone 106 (Figure 3). If an oracle could always choose the successful output, their combined solve rate would be 68.2%, versus 45.9% for the better individual model. At 5B the corresponding rates are 43.8% and 27.8%. These are upper bounds, not measured routing methods.

The division is concrete: on non-pixel answers, text has 85 unique successes versus video's 38; on pixel-position answers, video has 68 versus text's 44. A combined solver might use a symbolic answer when the task has a discrete target and a visual generator when localization or trajectory is needed. The oracle gap shows room for such a system, but this study does not establish that a router can identify the right method in advance.

Both solveText onlyVideo onlyNeither

Loading interactive figure…

4.5 Failures point to different bottlenecks

With the layout prompt, 25% of Qwen3.6-27B answers exhaust the 2,048-token budget before giving a final answer; 29% are parsed but wrong (Figure 4). Shorter direct prompting reduces, but does not eliminate, truncation. G27's video scores include 86 of 475 samples (18%) between 0.7 and 0.9. That near-miss band helps explain why the threshold changes the accuracy comparison: the video scorer awards partial credit for a trajectory, while the text grader sees only the final answer.

CorrectIncorrectUnparsableTruncated at 2,048 tokensVideo v2 scores

Loading interactive figure…

4.6 Inspecting paired outputs

Figure 5 places generated videos beside the text model's layout, reasoning, and final answer for the same samples. It lets readers inspect whether a reported win reflects the intended solution, a localization error, or a score near the video threshold.

video bettertietext better

Loading interactive figure…

ExamplePreviousNext

Input

Task prompt (shared by both models)

Reference answer (512-pixel frame coordinates)

Ground-truth solution video

Video reasoning

Text reasoning

Qwen3.6-27BQwen3.5-4B

Parsed answer

Model output

Prompt sent to the language model

5 Limitations

• Training and scoring differ. The video models were fine-tuned on VBVR-Pro-SFT; the text models were evaluated zero-shot. Video receives a trajectory score with partial credit, while text receives a binary final-answer grade. The two thresholds show sensitivity to this difference but cannot remove it.
• Training overlap is possible. Among 235 in-domain benchmark samples checked, 168 have an identical counterpart in VBVR-Pro-SFT. G5's mean score on these duplicates is no higher than on other samples of the same tasks (difference −0.007), though this check does not rule out overlap effects.
• The 27B video reproduction is slightly low. G27 scores 1.8 points below its model card overall and 3.5 points below on in-domain tasks. This could shift the in-domain comparison by a few points (Table 2).
• Some subgroup results are exploratory. The answer-kind split was defined after the zero-shot runs, and the single-agent path group contains only 11 tasks. Subgroup confidence intervals are not adjusted for multiple comparisons.
• Input and decoding choices affect text. Both methods use a 512² first frame. In earlier 97-task text runs, Qwen3.6-27B solved 56.0% of pixel-answer tasks at 1024² versus 38.9% at 512², pooled over prompts. A single greedy run with a 2,048-token limit counts unfinished answers as failures; 25% of its layout answers hit that limit.
• Cost is measured in FLOPs, not time. Video FLOPs follow from the model shapes and text FLOPs from the measured token counts; the estimate omits the video text encoder and VAE. Because text decoding is memory-bound, wall-clock ratios are smaller than the FLOPs ratios. Pretraining compute is not matched and is not disclosed precisely enough to compare here.

6 Conclusion

On these paired visual reasoning tasks, text produces answers with far fewer forward FLOPs than video, with no statistically resolved solve-rate difference at the strict video threshold. Crediting near-miss videos changes the result, especially on tasks seen during video fine-tuning. The per-sample split is more informative than a single aggregate score: non-pixel answers favor text under the strict threshold, single-agent paths favor video at 27B, and pixel-position answers favor video under the lenient threshold. A stronger test of representation would fine-tune the language model on the same data and use a common final-state grader; the complementary successes also make a text-guided visual solver worth testing.

Citations

@misc{li2026symbols,
  author       = {Li, Brian},
  title        = {Symbols or Pixels? Zero-Shot Language Reasoning versus VBVR-Trained Video Generation on Visual Reasoning Tasks},
  year         = {2026},
  howpublished = {LMMS Lab Notes},
  url          = {https://www.lmms-lab.com/notes/symbols-or-pixels/}
}

References

1. Thaddäus Wiedemer, Yuxuan Li, Paul Vicol, et al. Video models are zero-shot learners and reasoners. arXiv:2509.20328, 2025. Paper
2. Maijunxian Wang, Ruisi Wang, Juyi Lin, et al. A Very Big Video Reasoning Suite. arXiv:2602.20159, 2026. Paper
3. Junxiang Xu, Ruisi Wang, Fanyi Pu, et al. VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning. arXiv:2608.26105, 2026. Paper
4. Cheng Yang, Haiyuan Wan, Yiran Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025. Paper
5. Zefan Cai, Haoyi Qiu, Tianyi Ma, et al. MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation. arXiv:2512.14691, 2025. Paper
6. Yifan Li, Yukai Gu, Yingqian Min, et al. Beyond the Last Frame: Process-aware Evaluation for Generative Video Reasoning. arXiv:2512.24952, 2025. Paper
7. Yi Xu, Chengzu Li, Han Zhou, et al. Visual Planning: Let's Think Only with Images. arXiv:2505.11409, 2025. Paper
8. Chengzu Li, Zanyi Wang, Jiaang Li, et al. Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoning. arXiv:2601.21037, 2026. Paper
9. Jialong Wu, Xiaoying Zhang, Hongyi Yuan, et al. Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models. arXiv:2601.19834, 2026. Paper
10. Wen-Ding Li, Keya Hu, Carter Larsen, et al. Combining Induction and Transduction for Abstract Reasoning. arXiv:2411.02272, 2024. Paper