---
title: "NEW AI paper worth bookmarking."
url: https://stacklist.com/card/19c5f572-cad6-4cf7-ac7a-bdf660792312
source_url: "https://www.linkedin.com/posts/omarsar_new-ai-paper-worth-bookmarking-this-is-share-7480322261594386432-nBaz/?utm_source=share&utm_medium=member_ios&rcm=ACoAAAI21ZsBNnZPaKuTab7nquKLCveUW7o-1DE"
stack: https://stacklist.com/stack/614ecd6d-12cd-4195-bf9c-3def755c89b2
summary: "A new Stanford, NVIDIA, and UC Berkeley paper demonstrates verification as an emerging scaling axis for AI systems using LLMs as training-free verifiers that extract continuous calibrated scores from token logits. The approach achieves strong results across diverse benchmarks (86.5% on Terminal-Bench, 78.2% on SWE-Bench, 87.4% on RoboRewardBench) and enables iterative refinement in AI agents without fine-tuning."
tags: "ai-verification, llm-verifiers, scaling-axis, continuous-scoring, reward-models, agent-architecture"
key_entities: "Stanford (organization), NVIDIA (organization), UC Berkeley (organization), Elvis S. (person), LLM-as-Verifier (technology), SAC (technology), GRPO (technology), Claude Code (technology), verification-scaling (concept), continuous-reward-signals (concept)"
classification: "analysis"
content_hash: "sha256:1450c2859fdf0a81b5b9e124aa66741a0831b474c35bc39f1380020b2bb23797"
acp_version: "0.2"
token_counts_approximate: 932
visibility: public
agent_accessible: true
status: "final"
---

# NEW AI paper worth bookmarking.

Elvis S. 5h Report this post NEW AI paper worth bookmarking. This is something I called early, and this paper confirms it: verification has emerged as a new important scaling axis. Here is the simple explainer and what this paper shows. We have seen lots of progress in scaling pre-training, post-training, and test-time compute. For post-training and test-time compute, we are still in its early phases. But one of the most important new directions is using LLMs as verifiers. Verifiers are fundamental to scaling AI. This work from Stanford, NVIDIA, and UC Berkeley builds a training-free verifier that reads a continuous, calibrated score straight off the scoring-token logits instead of trusting a discrete grade. Three knobs move accuracy without any fine-tuning. Score granularity for cleaner separation, repeated evaluation for lower variance, and criteria decomposition for lower complexity. The numbers land across very different domains. 86.5% on Terminal-Bench V2, 78.2% on SWE-Bench Verified, 87.4% on RoboRewardBench, and 73.3% on MedAgentBench. The same continuous score doubles as dense reward for SAC and GRPO and as a task-progress signal shipped in a Claude Code extension. 88 4 Comments Like Comment Share Copy LinkedIn Facebook X Elvis S. 5h Report this comment Paper: https://arxiv.org/abs/2607.05391 Learn to build effective AI agents in our academy: https://academy.dair.ai / Like Reply 1 Reaction 侯凱文 4h Report this comment LLM-as-a-Verifier 論文：技術視角精簡版 這份補充很到位，因為它把這篇論文從「更好的 judge」提升到「inference-time 架構重寫」的層次。 一句話核心 這篇論文不是在做更準的評分，而是在把 LLM 從生成器，推進成可搜尋、可比較、可迭代最佳化的推理系統。 1. 方法論本質：從離散判斷變成連續最佳化 傳統流程通常是： Generate Evaluate Pick best 而這篇論文更像是： Generate 多個候選 Verifier 產生連續 reward signal 用 ranking / sampling / iteration 做近似搜尋 也就是說，verification 不再只是最後的裁判，而是整個推理迴圈中的「動態評分模組」。 2. 為什麼 Continuous Score 很重要 傳統問題 離散評分常遇到： - tie - 粒度太粗 - 無法反映微小差異 論文解法 用 logits 分佈的期望值來形成連續分數： [ Score = \mathbb{E}_{p(token)}[value(token)] ] 這帶來三個直接好處： 1) 可排序 可以做： - pairwise comparison - soft ranking - tournament selection 2) 可當 reward model 替代品 不用另外訓練 reward model，直接 zero-shot 當 reward signal。 3) 更穩定 透過 repeated evaluation、criteria decomposition、granularity 提升，降低 variance。 3. 它補的是 Agent 最缺的能力 很多 agent 失敗不是因為不會生成，而是因為： - 不知道自己錯了 - 沒辦法比較方案 - 無法持續迭代 Verifier 的價值在於建立： generate → verify → refine → iterate 這讓 agent 從一次性輸出，升級成可持續最佳化系統。 4. 為什麼在 benchmark 上有效 這篇特別適合的任務，通常都有共同特性： - multi-step - long horizon - error accumulation 例如： - SWE-Bench - Terminal-Bench - Robotics - Medical reasoning 這類任務重點不是「答一次」，而是「過程中少犯錯」。 Verifier 的優勢就是能提供更密集的校正訊號。 5. 對實務的啟發 AI Agent 架構 未來可能更像： - Planner - Verifier - Selector / Controller 而不是單一 LLM 一把梭。 決策系統 / BI Like Reply 1 Reaction Celso Sousa 3h Report this comment The paper convincingly frames verification as a new scaling axis, yet its empirical claims rely heavily on benchmark gains without fully addressing verifier calibration under distribution shifts, adversarial trajectories, or systematic bias inherited from the underlying LLMs. The absence of rigorous uncertainty analysis and broader failure characterization limits confidence in its claimed generality. Can probabilistic verification remain reliable when evaluators and generators coevolve? How should continuous verifier signals be calibrated against real-world correctness rather than benchmark agreement? What theoretical guarantees and robustness criteria are needed before verification becomes a foundational scaling paradigm for trustworthy AI? Like Reply 1 Reaction Suresh Kumar 5h Report this comment Calibration beats verdicts here. Reading a graded signal from token scores feels far more useful than another pass or fail judge. Like Reply 1 Reaction See more comments To view or add a comment, sign in