We are retiring our bug bounty program
360 points
• 4 days ago
• Article
Link
Turso 正在终止运行近一年的漏洞赏金计划。该计划曾向能复现导致数据损坏的漏洞的提交者支付 1000 美元奖励，但大量低质量、 AI 生成的提交（被称为"slop"）使得该计划不可持续。尽管计划识别出少数优秀的贡献者，经济激励却也吸引了大量自动化、无意义的拉取请求，耗费维护者大量时间来审核和驳回。
该计划最初是为了在重写 SQLite 时增强外界对 Turso 可靠性的信心。团队虽然采用了确定性模拟、模糊测试以及与 SQLite 的差分测试等严格方法，但也承认自动化测试有其局限。赏金的初衷是发现这些工具遗漏的边缘情况，起初确实有效——吸引了一些技术过硬的人，他们不仅发现了真实问题，还为测试基础设施的改进做出贡献。
然而，先进的 AI 工具改变了局面。 1000 美元的奖励成为一些人利用 LLMs 大规模生成漏洞报告的目标，不问真假就大量提交。例如有的 PR 故意破坏数据库头部便宣称发现"bug"，或者误解并发写入等核心特性而提交错误报告。这类提交通常夹杂乱码和虚假断言，维护者要花数小时去处理，而提交者只需几分钟即可生成。
Turso 曾引入担保机制以自动关闭疑似机器人的提交，但这只促使机器人继续开新的问题，要求人工复核。生成 slop 的成本极低，而人工评估的成本很高，这种不对称使赏金计划难以为继。尽管 Turso 重视开放贡献模式，团队最终认为在开放体系中任何金钱激励都会被 AI 利用，不得不终止该计划，以维护维护者的精力和项目的完整性。
Turso is retiring its bug bounty program, which offered $1,000 for demonstrating bugs that lead to data corruption, after nearly a year of operation. The decision stems from an overwhelming influx of low-quality, AI-generated submissions, referred to as "slop," that have made the program unsustainable. While the initiative successfully identified a handful of exceptional contributors, the financial incentive attracted a flood of automated, nonsensical pull requests that consume significant maintainer time to review and dismiss.
The program was originally launched to bolster confidence in Turso's reliability as it rewrites SQLite, a system renowned for its stability. Despite employing rigorous testing methods like deterministic simulation, fuzzing, and differential testing against SQLite, the team acknowledged that automated tests have limitations. The bounty was designed to uncover edge cases their tools missed, and it initially worked well, attracting skilled individuals who found genuine issues and even contributed to improving the testing infrastructure itself.
However, the rise of advanced AI tools changed the landscape dramatically. The $1,000 reward became a target for users leveraging LLMs to generate bug reports at scale, regardless of validity. Examples include PRs that manually corrupt database headers and claim discovery of a "bug," or reports that misunderstand core features like concurrent writes. These submissions, often filled with garbled text and false claims, require hours of maintainer effort to address, while costing the submitters mere minutes to produce.
In response, Turso implemented a vouching system to auto-close suspected bot submissions, but this only led to bots opening new issues demanding manual review. The asymmetry between the low cost of generating slop and the high cost of human evaluation made the bounty untenable. Although Turso values its open contribution model, the team concluded that any financial incentive in an open system is now exploited by AI, forcing them to retire the program to preserve maintainer bandwidth and project integrity.
285 comments • Comments Link
• 软件开发真正的瓶颈不是写代码，而是阅读和理解代码。 AI 生成的代码加剧了这一问题，因为它产出大量没人能完全理解的代码，包括 AI 自己。在我们能对 AI 的输出负责之前，不能把理解能力完全托付给它。
• "战术龙卷风"指的是那些能迅速产出代码但留下破坏性后果的开发者。管理层常因他们的速度而表扬他们，但对必须维护这些代码的工程师来说，他们并不是英雄。这类人往往在公司待很久，有时成为最后留下的工程师，而主张可维护性的人则选择离开。
• 在许多组织里，战术龙卷风往往没有任何惩罚，反而经常得到晋升和加薪。企业优先考虑快速上线以兑现销售承诺，因此无论代码质量如何，能快速交付的人都会被奖励。久而久之，许多工程师放弃了清理烂代码，只专注于把事情做完。
• 这种情况在管理层施压评审人员尽快合并代码时尤为严重，常以"如果我们为明天编码，我们永远也干不完"为由驳回关于可维护性的担忧。评审者最终学会放行这些开发者，接受以后有人来收拾烂摊子的现实。权力关系也会影响决策：资深或有影响力的开发者让别人很难反对。
• AI 可能成为终极的战术龙卷风，乐于添加数百行代码，用条件语句堆砌来修复问题，结果是交织混乱、难以理解的代码和臃肿的测试套件。虽然眼前的错误可能被修掉，测试覆盖率看似提高，但代码库的长期可维护性却在受损。
• 为了遏制 AI 生成的垃圾代码，有人提出对每次提交收取名义性费用（例如 5–10 美元），如果提交确有问题则退还。这样的金钱摩擦能过滤掉靠 AI "喷洒式"提交的恶意行为者，同时不会吓退认真的研究者。对被拒绝提交给予有质量的人工反馈，也会成为工程师宝贵的学习机会。
• 对漏洞赏金提交收费也可以推广到其他领域，例如对客户支持提交收取（例如 100 美元），以保证人工分类与升级。这能降低琐碎问题的噪音，确保真正的问题得到处理。但费用设计必须平衡，避免排斥低收入地区的用户或激励虚假的行为声明。
• 大型 PR 本身并非天生有害，只要组织得当且确为必要的基础性变更。然而，如果它们被用来规避适当的审查或引入不必要的重构，就会成问题。情境很重要：可信工程师在绿地项目上的大规模 PR，和中级工程师在无充分理由下重构成熟产品，是两回事。
• AI 生成代码与人类审查能力之间的不对称正变得越来越严重。 AI 能远快于人类生成和验证代码，但解决之道不一定是更多 AI 。相反，需要更好的流程：更小的迭代 PR 、更严格的审查标准，以维护代码质量。
• 一些组织在尝试限制 AI 生成的提交，比如要求工作量证明或先用 AI 预筛选 PR，但这些方案并不完美且实施成本可能很高。最终，问题可能需要系统性改变，比如对公共仓库采取"只读不动"策略，或建立类似行会的信任网络来审核贡献者。
• AI 生成的垃圾代码正在侵蚀开源协作——过去任何人都能贡献改进的模式正受到冲击。维护者被低质量 PR 和错误报告淹没，不得不关闭或限制贡献渠道。这种转变破坏了开源的社区精神，可能导致更多封闭、仅限邀请的社区出现。
• 尽管面临挑战，一些人认为 AI 将继续存在，必须学会适应。目前正在探索的解决办法包括对提交收费、使用 AI 过滤垃圾代码、建立基于声誉的信任网络；也有人主张完全关闭开放贡献模式，认为在 AI 噪音横行的时代，维持开放协作的成本超过收益。
讨论显示，在软件开发中快速采用 AI 与其带来的实际问题之间存在深刻张力。 AI 能加速代码生成，但常以可维护性、理解和信任为代价。无论是人类还是 AI 的"战术龙卷风"，因速度被赞扬的同时也留下持久的损害，需要别人来收拾残局。社区正尝试各种应对方式——从金钱摩擦到贡献管理的系统性变革——但尚未达成共识。可以确定的是，开放、低摩擦的协作时代可能正在结束，取而代之的是更多门控和基于信任的模式。 • The real bottleneck in software development isn't writing code, but reading and understanding it. AI-generated code exacerbates this by producing massive amounts of code that no one fully understands, including the AI itself. Until we can hold AI accountable for its output, we cannot delegate comprehension to it.
• "Tactical tornadoes" are prolific programmers who pump out code rapidly but leave a wake of destruction. They are often celebrated by management for their speed, but they are rarely heroes to the engineers who must maintain their code. These individuals often stay at companies for decades, sometimes becoming the last engineers standing, while those who advocate for maintainability move on.
• In many organizations, tactical tornadoes face zero consequences and often receive promotions and raises. Businesses prioritize pumping out features to meet sales commitments, and engineers who deliver quickly are rewarded regardless of code quality. Over time, many engineers give up trying to clean up poor code and focus solely on getting work done.
• The situation arises when management pressures reviewers to merge code quickly, dismissing concerns about maintainability with arguments like "if we coded for tomorrow, we'd never get anything done." Reviewers eventually learn to let these developers proceed, accepting that they will have to clean up the mess later. Power dynamics also play a role, as senior or influential developers can make it difficult for others to push back.
• AI can be the ultimate tactical tornado, happily adding hundreds of lines of code wrapped in conditional statements to fix bugs. This results in interleaved, hard-to-understand code and bloated test suites. While the immediate bug may be fixed and test coverage may increase, the long-term maintainability of the codebase suffers.
• To combat AI-generated slop in bug bounty programs, one proposal is to charge a nominal fee (e.g., $5-$10) per submission, which is refunded if the bug is legitimate. This monetary friction would filter out bad actors using AI bots for spray-and-pray tactics while ensuring serious researchers are not deterred. Skilled human feedback on rejected submissions would also serve as a valuable learning experience for engineers.
• Charging for bug bounty submissions could extend to other areas like customer support, where a fee (e.g., $100) would guarantee human triage and escalation. This would reduce noise from trivial issues and ensure genuine problems are addressed. However, the fee must be balanced to avoid excluding users in lower-income regions or incentivizing false claims of intended behavior.
• Large PRs are not inherently problematic if they are well-organized and necessary for fundamental changes. However, they become problematic when they are used to bypass proper review processes or when they introduce unnecessary refactoring. Context matters: a massive PR from a trusted engineer on a greenfield project is different from an intermediate engineer refactoring a mature product without justification.
• The asymmetry between AI's ability to generate code and humans' ability to review it is a growing problem. While AI can produce code orders of magnitude faster than humans can verify, the solution isn't necessarily more AI. Instead, better processes, such as smaller iterative PRs and stricter review standards, are needed to maintain code quality.
• Some organizations are exploring ways to gatekeep AI-generated submissions, such as requiring proof-of-work or using AI to pre-screen PRs. However, these solutions are not foolproof and can be costly to implement. Ultimately, the problem may require systemic changes, such as moving to a "look but don't touch" model for public repos or establishing trust networks like guilds to vet contributors.
• The influx of AI-generated slop is ruining open source collaboration, which traditionally allowed anyone to contribute improvements. Now, maintainers are inundated with low-effort PRs and bug reports, forcing them to shut down or restrict contribution channels. This shift undermines the communal spirit of open source and may lead to more closed, invite-only communities.
• Despite the challenges, some argue that AI is here to stay and that adaptation is necessary. Solutions like charging for submissions, using AI to filter slop, and establishing reputation-based trust networks are being explored. However, others advocate for shutting down open contribution models entirely, arguing that the costs of maintaining them outweigh the benefits in an era of AI-generated noise.
The discussion reveals a deep tension between the rapid adoption of AI in software development and the practical challenges it introduces. While AI can accelerate code generation, it often does so at the cost of maintainability, comprehension, and trust. Tactical tornadoes, whether human or AI, are celebrated for their speed but leave lasting damage that others must clean up. The community is grappling with solutions, from monetary friction to systemic changes in how contributions are managed, but there is no consensus on the best path forward. What is clear is that the era of open, low-friction collaboration may be ending, replaced by more gated and trust-based models.