实
Gerard de Melo
Computational linguistics and NLP researcher — named and experimentally revealed the trajectory-bias phenomenon
Working with Maximilian Schall (RANLP 2025), de Melo ran systematic experiments exposing the hidden cost of constrained decoding to LLM reasoning quality, naming and quantifying “trajectory bias”: the cumulative perturbation from constraints systematically pushes the decoding path away from the semantically optimal direction, even when the output is fully syntactically compliant.
Research contributions Naming trajectory bias Systematic experiments expose the hidden semantic cost of constrained decoding — format compliance is not task accuracy Instruct vs. base models Across 11 models: instruction-tuned models suffer more from trajectory bias, as conversational goals clash with grammar constraints Predictive signal A base model’s performance under constraints predicts its structured-output ability after instruction tuning Draft-Conditioned fix arXiv:2603.03305 (2025): separate drafting from formatting — reason first, format second — to mitigate trajectory bias
Engineering impact Train-time integration Applying constraints only at inference is fundamentally misaligned — the long-term direction is to fold constraints into training Link to CRANE de Melo’s empirical work and CRANE’s theoretical proof (Beurer-Kellner 2025) reinforce each other
→ Trajectory Bias · Structured OutputsSchall & de Melo (RANLP 2025)
Gerard de Melo
简介
Gerard de Melo，计算语言学与 NLP 领域研究者，研究方向涵盖语言理解、知识表示与 LLM 结构化生成。
主要贡献
- 轨迹偏差研究（RANLP 2025，与 Maximilian Schall 合作）：系统性实验揭示约束解码对 LLM 推理质量的隐性代价，发现指令微调模型与基础模型在约束下的分歧，提出基础模型约束性能可作为指令微调后结构化输出能力的预测指标。
- Draft-Conditioned Constrained Decoding（arXiv:2603.03305, 2025）：提出通过草稿-格式分离缓解轨迹偏差。
相关概念
References
- Schall & de Melo. “The Hidden Cost of Structure.” RANLP 2025.
sources/ranlp-2025-hidden-cost-constrained-decoding.md