After 7 months, Glasswing overwelms maintainers with underwhelming findings
457 points
• 1 day ago
• Article
Link
在 Anthropic 的 Project Glasswing 启动后的七个月里，这项计划在软件安全领域真正带来的影响仍遭到质疑。尽管项目一开始就宣称能够识别数千个高严重性与关键严重性漏洞，但实际效果远没有达到预期。追踪数据显示，归因于 Anthropic 的漏洞仅占今年已发布 CVE 总量的 0.5%，而其中真正可能在实际环境中被利用的证据少之又少。
项目内部的"账本"难以继续推进，暴露出发现与修复之间的差距越来越大。自 9 月以来，Anthropic 报告的发现数量几乎增加了 12,000 项，但已确认的修复数量却基本停滞不前。此外，被标记为重复或被撤回的发现数量，明显超过了实际完成的修复数量，这也让人对项目所声称的高真阳性率产生了疑虑。
鉴于人工分诊无法有效扩展，Anthropic 近期调整了策略，推出了自动化的 OSS Scanner 。该工具把验证压力从 Anthropic 内部团队转移给开源维护者：维护者现在需要处理未经核实、由模型生成的报告。 Anthropic 也承认这些发现是在没有人工审查的情况下生成的，这引发了对低质量甚至错误提交风险的担忧。
对这套新扫描器进行的一次实际测试以 curl 项目为对象，进一步凸显了维护者的困境。在 curl 处理的 34 份报告中，超过一半被判定为无效，只有少数被确认属于低严重性问题。这种结果与 Anthropic 所宣称的高成功率形成了鲜明反差，也表明项目产出的严重性或准确性往往达不到最初承诺。
归根结底，尽管 AI 驱动的漏洞发现具备潜力，但 Project Glasswing 的相关数据反映出目前的落地效果未能兑现其宣传叙事。让开源维护者承担大量潜在误报的处理工作，不仅没有减轻负担，反而给原本希望帮助的社区增加了额外工作。随着项目主张与维护者所观察到的现实之间的偏差持续存在，这项倡议的整体有效性与必要性仍将受到持续质疑。
Seven months into Anthropic's Project Glasswing, the initiative continues to face skepticism regarding its actual impact on software security. While the project launched with ambitious claims about identifying thousands of high- and critical-severity vulnerabilities, the reality has proven to be less significant. Tracking data indicates that vulnerabilities attributable to Anthropic account for a mere 0.5% of all CVEs published this year, with an extremely small fraction showing evidence of being exploited in the wild.
The project's internal ledger has struggled to maintain momentum, revealing a growing disconnect between discovery and resolution. Since September, the number of findings reported by Anthropic has surged by nearly 12,000, yet the number of confirmed fixes has remained largely stagnant. Furthermore, the volume of findings marked as duplicates or withdrawn has significantly outpaced the number of actual fixes, casting doubt on the project's reported high true-positive rates.
Recognizing that manual triage was not scaling, Anthropic recently shifted its approach by launching an automated OSS Scanner. This tool effectively offloads the burden of validation from Anthropic's internal teams to open source maintainers, who must now process unverified, model-generated reports. Anthropic acknowledges that these findings are produced without human review, raising concerns about the potential for low-quality or incorrect submissions.
A practical test of this new scanner, involving the curl project, highlighted the challenges maintainers face. Out of 34 reports processed by curl, more than half were determined to be invalid, while only a small minority were confirmed as low-severity issues. This experience stands in stark contrast to the high success rates claimed by Anthropic and suggests that the project's output often lacks the severity or accuracy initially promised.
Ultimately, while AI-driven vulnerability discovery holds promise as a tool, the data from Project Glasswing suggests that the current execution is falling short of its marketing narrative. The reliance on open source maintainers to handle a high volume of potential false positives has created an additional workload for the very communities the project aimed to assist. As the discrepancies between the project's claims and the reality observed by maintainers persist, the overall effectiveness and necessity of the initiative remain subject to ongoing critique.
217 comments • Comments Link
- AI 模型在识别漏洞方面确实展现出能力，尽管它们的有效性在很大程度上取决于代码库质量、安全规范的完整度以及人工监督的到位程度。
- AI 生成的漏洞报告数量非常庞大，但真正被证实、且可被利用的问题却少得多，两者之间存在明显脱节，给开源维护者带来巨大摩擦和"噪音"。
- 目前若不加甄别地呈现潜在缺陷，会让人工维护者背负不可持续的分诊负担，他们往往把这些贡献视为低价值的"垃圾信息"，因此需要转向更好的筛选机制以及自动化的补丁建议。
- 安全专家对这些发现的影响看法不一：有人认为这是一条迈向主动防御的必要路径；也有人认为对 CVE 数据集的关注是一种营销驱动的误导，掩盖了私有、面向企业层级的安全威胁的真实存在。
- "网络末日"的叙事常被批评为夸大其词，但普遍观点是：高级攻击者很可能已经借助 AI 能力以更精准的方式利用那些未被大型开源项目充分审查的私有、非公开软件栈。
- 人们对这类漏洞发现项目背后的动机也普遍存在较大疑虑，批评者认为它们常常是商业安全优先级产品的"亏损引流品"，或者是为了提升 AI 实验室的可见度与影响力而进行的营销内容。
- 验证 AI 发现结果的难度，折射出其他科研领域的共同问题：AI 系统会产出大量"发现"，但这些结果需要投入巨量的专业人工来核验，并从噪音中筛出来。
- 开发高质量的安全代码依然高度依赖人工，因为 AI 模型虽然擅长找出潜在错误，但在提出稳健、不破坏现有功能且在架构上站得住脚的修复方案时，往往会难以做到不引入新的复杂性。
- 虽然漏洞的公开披露数量依旧不多，但这可能更多反映了人工安全团队的资源受限，而非底层 AI 洞察本身的不足；有报道指出，当内部专有安全团队使用类似工具时，成功率更高。
- AI 在网络安全中的最终价值不在于取代人的专业能力，而在于在明确的威胁模型框架下成为"倍增器"；前提是 human-in-the-loop 流程的设计能够有效管理信号与噪音的比例。
讨论凸显了"AI 驱动的漏洞发现"带来的理论前景，与维护软件安全的现实挑战之间的紧张关系。虽然 AI 模型确实能识别出真实的安全缺陷，但它们也会产生大量淹没维护者的发现，许多维护者认为这些低质量报告数量不仅无助，反而会阻碍安全工作。批评者指出，围绕这些工具的营销叙事往往忽视了分诊所需的高昂人工成本，以及缺乏可落地、质量可靠的补丁问题。总体而言，人们普遍认为：尽管 AI 在知识工作中属于合法的阶段性变革，但它在网络安全中的当前应用仍处于"很多猴子"的成熟期，需要与人类专家工作流更好地融合，才能真正产生实质影响。 • AI models demonstrate a genuine capability for identifying vulnerabilities, though effectiveness varies significantly depending on the codebase, the quality of the security specifications, and the thoroughness of human oversight.
• There is a marked disconnect between the high volume of AI-generated vulnerability reports and the actual number of confirmed, exploitable issues, leading to significant friction and "noise" for open-source maintainers.
• The current methodology of indiscriminately surfacing potential bugs places an unsustainable triage burden on human maintainers, who often view these contributions as low-value or "slop," necessitating a shift toward better filtering and automated patch suggestions.
• Security experts are divided on the impact of these findings: some view them as a necessary step toward proactive defense, while others argue the focus on CVE datasets is a marketing-driven misrepresentation that obscures the reality of private, enterprise-level security threats.
• The "cyber-apocalypse" narrative is frequently criticized as exaggeration, yet there is a consensus that advanced attackers likely already use AI-class capabilities to surgically exploit private, non-public software stacks that lack the scrutiny of major open-source projects.
• Significant skepticism exists regarding the motivations behind these vulnerability discovery projects, with critics suggesting they often serve as loss leaders for commercial security prioritization products or as marketing content to bolster the perceived influence of AI labs.
• The difficulty of verifying AI findings mirrors challenges in other scientific domains, where AI systems generate massive lists of "discoveries" that require overwhelming amounts of expert human labor to validate and separate from noise.
• Developing high-quality security code remains a human-intensive task, as AI models often excel at identifying potential errors but frequently struggle to propose robust, non-breaking, and architecturally sound fixes without introducing new complexities.
• While public disclosures of vulnerabilities remain low, this may reflect the resource constraints of human teams rather than the quality of the underlying AI insights, as internal proprietary security teams reportedly see higher success rates with similar tooling.
• The ultimate utility of AI in cybersecurity lies not in replacing human expertise but in acting as a force multiplier for well-defined threat models, provided that the human-in-the-loop workflow is designed to manage the signal-to-noise ratio effectively.
The discussion highlights a tension between the theoretical promise of AI-driven vulnerability discovery and the practical reality of maintaining software security. While AI models can identify genuine security flaws, they also generate a flood of findings that overwhelm human maintainers, many of whom feel the volume of low-quality reports hinders rather than helps security efforts. Critics argue that the marketing narratives surrounding these tools often ignore the high labor cost of triage and the lack of actionable, high-quality patches. Ultimately, there is a shared sentiment that while AI is a legitimate step change in knowledge work, its current application in cybersecurity is maturing through a phase of "lots of monkeys" intelligence, necessitating better integration with human expert workflows to become truly impactful.