An open letter to Dario: if you mean it, open the weights
306 points
• 1 day ago
• Article
Link
Jake Gold 向 Anthropic 的 CEO Dario Amodei 发信，回应他最近关于放慢前沿人工智能模型发展的呼吁。 Gold 在肯定 Amodei 的诚意以及其对第三方评估者承诺的同时，认为目前的监管提案方向不对，会无意中导致监管俘获。他主张：如果 Anthropic 真心想为人类利益放慢行业步伐，就应倡导更激进、更有效的政策——要求任何公开发布的 AI 模型都必须开源并完整公开模型权重。
信中强调，传统监管框架（比如按计算量设门槛或依赖复杂的行业协调）往往有利于既有企业。因为这些规则通常由领先实验室参与起草，自然设置了小公司难以逾越的高准入门槛。随着监管条款日益叠加与复杂化，大公司凭借更强的法律和合规能力维持市场地位，把所谓的安全措施变成了扼杀竞争的保护壕沟，而非真正减缓技术进步。
Gold 建议通过法律强制：所有面向公众的模型必须公开权重。这会从根本上改变 AI 开发的经济前提。现在对大规模、计算密集型训练的投资，依赖于模型权重保持专有以保护未来收益。若对公开发布的模型取消这种保护，高成本、激烈竞争的训练动力就会被削弱，从而在不需要政府决定哪些实验室可以继续开发的情况下，放慢整个行业的步伐。
最后，Gold 呼吁 Amodei 发扬其一贯的原则性领导力，回顾他以往为安全与伦理愿意作出的职业和经济牺牲。鉴于 Anthropic 以公共利益公司（Public Benefit Corporation）身份运营，Gold 认为 Amodei 有独特的地位去推动这一政策变革。如果他支持一项要求公开发布时必须公开权重的法律，就等于把负责任的发展置于短期利润之上，证明他对放缓前沿发展的承诺不是空谈，而是愿意为更大利益付出的真实牺牲。
Jake Gold addresses Dario Amodei, the CEO of Anthropic, in response to his recent call for pacing the development of frontier artificial intelligence models. While acknowledging Amodei's sincerity and his commitment to third-party evaluators, Gold argues that current regulatory proposals miss the mark by inadvertently fostering a system of regulatory capture. He posits that if Anthropic is genuinely committed to slowing down the industry for the benefit of humanity, they should advocate for a more radical and effective policy: a mandate requiring any publicly released AI model to be open-sourced with its weights fully disclosed.
The letter emphasizes that traditional regulatory frameworks, such as compute thresholds or complex industry coordination, primarily serve to entrench established companies. Because these rules are typically drafted with the help of the dominant frontier labs, they naturally create high barriers to entry that smaller competitors cannot overcome. As regulations become increasingly layered and intricate, the largest firms use their superior legal and compliance resources to maintain their market position, effectively turning safety measures into a protective moat that stifles competition rather than slowing technological progression.
Gold suggests that shifting the legal landscape to enforce open weights for all public models would fundamentally change the underlying economic assumptions of AI development. Currently, funding for massive, compute-heavy training runs relies on the expectation that model weights remain proprietary, thereby protecting future profits. By stripping away that protection for any model released to the public, the incentive for hyper-competitive, high-cost training would naturally diminish. This approach would slow the pace of progress across the entire industry without requiring government officials to make arbitrary decisions about which labs are permitted to continue their work.
Finally, the author appeals to Amodei's history of principled leadership, noting his previous willingness to make career and financial sacrifices in service of safety and ethics. Given that Anthropic operates as a Public Benefit Corporation, Gold argues that Amodei is uniquely positioned to advocate for this policy change. By championing a law that demands open weights for public releases, Amodei would be choosing the mission of responsible development over short-term profit models, proving that his commitment to pacing the frontier is not just rhetoric, but a genuine sacrifice for the greater good.
98 comments • Comments Link
- 要求向公众出售的任何 AI model 必须强制采用 open weights 的提案，被批评者认为前后矛盾且可能适得其反。该提案假定了一个可能并不存在的全球协作程度，同时低估了开发者将业务迁往监管更宽松司法辖区的便捷性。
- 主要担忧之一是，这类法律可能导致 frontier labs 完全停止公开发布模型，转而走向仅面向企业的封闭模式，从而进一步集中权力，使面向公众的 open-access AI 停滞不前。
- 怀疑者认为，强制透明化未必会减慢发展速度，反而可能促使更多转向专有的黑盒系统，这类系统会在内部侵蚀经济活力。
- 关于 open weights 会摧毁估值、导致 frontier labs 资金短缺的经济论点面临现实挑战：这些 labs 可以转向 B2B 、仅限内部使用或 compute-as-a-service 等商业模式，可能完全规避"public"的定义。
- 有人认为这场辩论被用作监管捕获或市场营销的工具：厂商借安全之名掩护，借机为自家技术建立护城河、抵御竞争。
- 以 US 为中心的监管有效性受到质疑：global labs 和外国竞争对手可能通过继续开发封闭系统获得优势，而 US labs 则被迫披露知识产权。
- 有观点认为，当前的"AI safety"叙事正被用来游说设立准入门槛，类似于历史上行业为维护 closed-source 企业利益而试图扼杀 Linux 等 open-source 替代品的做法。
- 许多参与者强调，AI 进步的根本驱动力是地缘政治，即全球大国之间的高风险竞争，因此单一国家的国内政策难以抑制全球总体进展。
- 有人主张，如果某个 lab 真的认为其 frontier models 对社会构成根本性危险，唯一合乎道德的做法就是完全停止相关研究，而不是试图通过强制开放或协调来减轻风险。
- 这场讨论反映了对行业领袖动机的深刻怀疑：公众普遍认为他们更多受市场地位和权力驱动，而非出于对人类生存的真正担忧。
围绕 frontier AI models 的监管争论，实质上是在理想化的透明呼声与全球经济竞争的务实现实之间的博弈。有人主张通过市场杠杆和立法放慢发展步伐，但也有人认为这些做法天真，可能导致权力进一步集中或把必要研究逼入地下，适得其反。普遍观点是，当前的竞赛本质上是一场地缘政治对抗，这削弱了任何单一国家监管框架的效力。归根结底，这场辩论凸显了对主要参与者的深度不信任：许多观察者认为，推动 AI 发展的仍是企业利益，而非真正的安全考量。 • The proposal to mandate open weights for any AI model sold to the public is viewed by critics as incoherent and potentially counterproductive. It assumes a degree of global cooperation that likely does not exist and underestimates the ease with which developers could simply move operations to more permissive jurisdictions.
• A significant concern is that such a law would result in frontier labs ceasing public releases entirely. This would force them to move to a private, enterprise-only model, leading to further concentration of power and a complete stagnation of open-access AI for the general public.
• Skeptics argue that forcing transparency would not slow down development, but rather accelerate the transition to proprietary, black-box systems that eat the economy from the inside.
• The economic argument—that open weights would destroy valuations and thus starve frontier labs of capital—is challenged by the reality that labs could pivot to B2B, internal-only, or "compute-as-a-service" models, potentially avoiding the "public" designation entirely.
• Some view the debate as a form of regulatory capture or marketing, where incumbents use safety concerns as a thin veil to build moats around their technology and protect against competition.
• The effectiveness of US-centric regulation is questioned, as global labs and foreign competitors would likely gain an advantage by continuing to develop closed systems while US labs are forced to disclose their intellectual property.
• There is a belief that the current "AI safety" narrative is being used to lobby for barriers to entry, echoing historical industry attempts to stifle open-source alternatives like Linux in favor of closed-source corporate control.
• Many participants emphasize that the underlying drivers of AI advancement are geopolitical, involving a high-stakes race between major global powers, making unilateral domestic policies ineffective in curbing total global progress.
• Some suggest that if a lab truly believed their frontier models were fundamentally dangerous to society, the only moral path would be a full cessation of research, rather than attempting to mitigate risks through forced openness or coordination.
• The discussion reflects deep-seated skepticism toward the motives of industry leaders, who are often seen as driven by market dominance and power rather than a genuine concern for humanity's survival.
The discourse surrounding the regulation of frontier AI models is defined by a tension between idealistic calls for transparency and the pragmatic realities of global economic competition. While some propose using market leverage and legislative mandates to slow the pace of development, others contend that such measures are naive, likely to backfire by centralizing power further or driving essential research underground. There is a broad consensus that the current race is fundamentally a geopolitical struggle, which limits the efficacy of any single-nation regulatory framework. Ultimately, the debate highlights profound distrust toward the major players, with many observers concluding that corporate interests, rather than genuine safety considerations, continue to dictate the trajectory of AI development.