Qwen 3.8 follows GPT-5.5 Pro reasoning prefills
0 points
• unknown time
• Article
Link
Reasoning prefill 实验研究了强制开源大语言模型采用来自专有 teacher model 的初始推理步骤会产生什么影响。研究人员将 teacher model 推理过程的前 1% 插入目标模型的推理通道，从而观察目标模型后续输出是否向 teacher 的风格和内容靠拢。本次迭代中以 GPT-5.5 Pro 作为 teacher，分析指标为目标模型响应中对 teacher 可见答案的召回率。
研究在 45 个不同任务上对四个模型进行了评估，任务涵盖 STEM 学科、非 STEM 主题与合成谜题。结果显示 Qwen3.8 A95B 在提供 prefill 后表现出显著提升，与 teacher 的对齐度提高了 18.18 个百分点；在 STEM 类别中这一提升尤为明显，达到了 26.99 个百分点，表明推理预填能有效将模型输出锚定到 teacher 的认知路径上。
其他模型的结果则较为温和或各有差异：DeepSeek V4 Flash 的对齐度略有下降，Inkling 略有改善；Kimi K3 本身就与 GPT-5.5 Pro 保持最高的基线重合度，无论是否加入 prefill 都得分较高，加入 prefill 仅带来 4.54 个百分点的边际增长。这些发现暗示 Kimi K3 相较其他被测模型，可能在训练数据或风格上本就更接近 GPT 系列。
Qwen3.8 A95B 在本次实验中的突出表现提示其可能使用了由 GPT-5.5 Pro 或极为相似模型生成的数据进行训练；这一猜测也得到了其在私有合成谜题上的表现支持——尽管题目具有新颖性，该模型仍明显倾向于镜像 teacher 的推理路径。总体而言，本实验揭示了主导性的 teacher model 对各类开源模型行为与输出模式的潜在影响，说明推理层面的痕迹能够有效塑造模型响应。
The reasoning prefill experiment explores the impact of forcing open-source large language models to adopt the initial reasoning steps generated by a proprietary teacher model. By inserting the first 1% of a teacher model's reasoning process into the target model's own reasoning channel, researchers can observe whether the target model's subsequent output shifts toward the style and content of the teacher. In this iteration, GPT-5.5 Pro served as the teacher, and the analysis measured the recall of the teacher's visible answers within the target models' responses.
The study evaluated four specific models across 45 diverse tasks, encompassing STEM subjects, non-STEM topics, and synthetic puzzles. The results highlight a striking performance jump for the Qwen3.8 A95B model, which showed an 18.18 percentage-point increase in alignment with the teacher model when provided with the prefill. This effect was notably pronounced in STEM categories, where the model demonstrated a 26.99 percentage-point shift, suggesting that the reasoning prefill effectively anchored the model's output to the teacher's cognitive path.
Other models tested showed more modest or varied results. While DeepSeek V4 Flash experienced a slight decrease in alignment, Inkling saw a minor improvement. Kimi K3 exhibited the highest baseline overlap with GPT-5.5 Pro, maintaining high scores regardless of the prefill, though the addition of the prefill only yielded a marginal gain of 4.54 percentage points. These findings suggest that Kimi K3 may already share a closer training lineage or stylistic affinity with the GPT family compared to the other tested models.
The significant success of Qwen3.8 A95B in this experiment implies that it may have been trained on data generated by GPT-5.5 Pro or a very similar model. This hypothesis is bolstered by the model's performance on private synthetic puzzles, where it showed a clear tendency to mirror the teacher's reasoning despite the novelty of the tasks. Overall, the experiment provides a window into the potential influence of dominant teacher models on the behaviors and output patterns of various open models, illustrating how reasoning artifacts can effectively shape model responses.
0 comments • Comments Link
- 我个人观察到 AI 的推理轨迹往往表现出一种"讨好但焦虑"的人格：面对自相矛盾的提示或错误时，模型偶尔会陷入混乱或作出不合逻辑的回应。
- 关于模型蒸馏的研究发现，有方法可以从专有的前沿模型中提取推理轨迹，并将这些轨迹作为预训练数据，用以提升开源模型或下游模型的逻辑一致性和输出质量。
- 有证据表明，像 Qwen 这样的模型可能使用了来自前沿模型的推理轨迹作为训练数据，这可以从这些模型在受到特定专有"思维前缀"引导时所展现的推理模式之间的统计相关性看出端倪。
- 偶尔观察到的"穴居人式"或高度缩略的推理痕迹，通常是特定提示配置、系统指令和高强度推理设置的副产品，而不是模型本身的普遍特征。
- 推理轨迹（作为内部逻辑草稿）与最终输出是不同的：模型通常分别针对不同目标对这两部分进行训练和优化，这可能导致语气或风格上的不一致。
- 通过对前沿模型输出进行训练以实现蒸馏，被一些人视为技术进步的必然阶段，这与历史上新进入者借鉴并迭代早期创新者技术的发展模式一致。
- 关于前沿实验室是否有道德权利来反对蒸馏，各方仍然存在争论，部分原因在于这些实验室自己的基础模型也是在未征得原作者同意的情况下、基于大量互联网数据训练出来的。
- 用户非常重视能够在本地部署高性能模型的能力（即便这些模型是从专有来源蒸馏而来），他们更看重可及性和自主性，而非依赖基于云的封闭前沿系统换来的边际性能提升。
- 一些观察者将对蒸馏的依赖解读为某些实验室采取"快速跟进"战略的证据，认为这表明它们在前沿创新方面可能严重落后。
- 技术持续充当"省力工具"的角色，使用户能够绕过繁重工作；这种动态正被 AI 代理日益模仿——它们通过寻找满足用户请求的更高效路径来最小化不必要的工作量。
本次讨论聚焦于 AI 透明度、模型蒸馏机制以及当前人工智能"军备竞赛"的地缘政治影响。尽管研究人员已经开发出恢复和分析内部推理轨迹的技术，这在机器学习时代引发了关于知识产权的激烈争论。许多参与者对看到专有"护城河"被开放获取模型蚕食感到欣慰，并指出行业历史上广泛进行数据抓取的做法，使其难以在道德上自居。最终，这次讨论突显出追求前沿规模创新与用户对高性能、可自主部署且不依赖集中式专有生态系统的实际需求之间存在明显分歧。 • Personal observations of AI reasoning traces reveal models that frequently exhibit a "pleasing but anxious" persona, occasionally spiraling into confusion or illogical behavior when confronted with self-contradictory prompts or errors.
• Research into model distillation has uncovered methods to extract reasoning traces from proprietary frontier models, using these traces as pre-training data to improve the logical consistency and output quality of open-source or downstream models.
• Evidence suggests that models like Qwen might be trained using reasoning traces from frontier models, as indicated by statistical correlations in reasoning patterns that emerge when these models are primed with specific proprietary thought prefixes.
• The "caveman-style" or highly abbreviated reasoning traces sometimes observed in AI are a byproduct of specific prompt configurations, system instructions, and high-intensity reasoning settings, rather than a universal characteristic of the models themselves.
• There is a distinction between reasoning traces, which function as internal scratchpads for logic, and final outputs; models are often trained to optimize these components for different objectives, which can lead to inconsistencies in tone.
• The act of distilling frontier models by training on their outputs is viewed by some as an inevitable stage in technological progress, mirroring historical patterns where new entrants adopt and iterate upon the techniques of early innovators.
• Debates persist regarding the moral authority of frontier labs to object to distillation, given that their own foundational models were trained on vast swaths of internet data without original creator consent.
• The ability to self-host high-performing models—even if distilled from proprietary sources—is highly valued by users who prioritize accessibility and sovereignty over the marginal performance gains of closed, cloud-based frontier systems.
• Some observers interpret the reliance on distillation as evidence that certain laboratories are operating on a "fast-follow" strategy, potentially trailing frontier innovation by a significant margin.
• Technology continues to function as a tool for "laziness," enabling users to bypass laborious tasks, a dynamic that AI agents are increasingly mimicking by seeking efficient paths to satisfy user requests while minimizing unnecessary work.
The conversation centers on the intersection of AI transparency, the mechanics of model distillation, and the geopolitical implications of the current "arms race" in artificial intelligence. While researchers have developed techniques to recover and analyze internal reasoning traces, this has led to a contentious discourse regarding intellectual property in the era of machine learning. Many participants express a sense of satisfaction in seeing proprietary "moats" eroded by open-access models, arguing that the industry's own history of broad data scraping negates its ability to claim moral superiority. Ultimately, the discussion highlights a clear divide between the pursuit of frontier-scale innovation and the practical, user-driven demand for performant, self-hostable models that do not rely on centralized, proprietary ecosystems.