More questions about whether researchers can trust OpenAI with unpublished math
866 points
• 4 days ago
• Article
Link
数学界最近爆发了一场关于 OpenAI 可能滥用未发表研究成果的争议。事件由数学家 Andreas Thom 引发：他发现自己与 ChatGPT 就 expander matching problem 及相关工作进行的私人讨论，可能影响了随后由 AI 生成的研究结果，因此提出了质疑。 Thom 对 OpenAI 缺乏透明度表示强烈不信，并称此前在试图确认其对话是否被纳入训练数据或被模型推理过程访问时，遭到公司代表的敷衍性和断然否认。
问题的核心在于研究者无法信任 AI 平台去处理敏感且处于早期阶段的数学构想。批评者指出，即便用户选择退出数据训练，其专有想法仍可能通过多种途径暴露，例如模型与用户的交互日志、点赞 / 点踩等反馈回路，或模型从此前未选择退出的对话中合成见解的能力。人们普遍担心这些 AI 系统可能实质性地"抢先使用"未发表的工作，从而破坏数学研究所依赖的信任与协作精神。
技术层面的不确定性主要源于大型语言模型的黑箱特性。一些讨论者认为，OpenAI 自身可能也无法明确追溯模型输出的来源，因为神经网络的可解释性仍处于起步阶段。但也有人反驳，认为从技术上应当可以审计训练数据，以查明某些具体的数学输入是否被摄取，而这不应被模型复杂的推理过程所掩盖。
讨论还指出了用户在保护知识产权时面临的法律与程序性陷阱。即便启用了退出选项，人们也注意到退出之前的对话仍保留在系统中，后端的模型激活模式可能仍然捕捉到用户研究的要点。这促使越来越多的人主张使用本地私有的 LLMs，而不是依赖那些以不透明数据政策运作的中心化、企业拥有的服务。
这一事件已促使学术界发出更广泛的呼吁，要求采取措施应对相关风险。一些观察者已开始组织起来，制定保护数学研究免受数字挪用的策略。此事清晰地提醒我们：生成式 AI 的便利性，与研究人员在成果公开前维护所有权和保密性的基本需求之间存在着深刻的张力。
A significant controversy has emerged within the mathematics community regarding the potential misuse of unpublished research by OpenAI. The conversation was ignited by mathematician Andreas Thom, who raised concerns after discovering that his own private discussions with ChatGPT about the expander matching problem and related work might have influenced subsequent AI-generated results. Thom expressed deep skepticism toward OpenAI's lack of transparency, noting that previous attempts to clarify whether his conversations were included in training data or accessible to the model's reasoning process received dismissive and categorical denials from company representatives.
The core of the issue lies in the inability of researchers to trust AI platforms with sensitive, early-stage mathematical concepts. Critics argue that even if a user opts out of data training, their proprietary ideas remain vulnerable through various mechanisms, such as model-user interaction logs, feedback loops like thumbs-up or thumbs-down ratings, and the potential for the model to synthesize insights from previous, non-opted-out conversations. There is a prevailing fear that these AI systems may effectively scoop unpublished work, damaging the communal trust and collaborative spirit essential to the advancement of mathematics.
Technical explanations for this uncertainty center on the black-box nature of large language models. Some participants in the discussion suggested that OpenAI itself may not be able to definitively track the provenance of a model's output, as interpretability in neural networks is still a nascent field. Others, however, disagreed, maintaining that it should be technically possible to audit training data to see if specific mathematical inputs were ingested, regardless of the model's complex reasoning processes.
The discussion also highlighted the legal and procedural traps users face when trying to protect their intellectual property. Even with an active opt-out setting, users have noted that conversations occurring prior to the opt-out remain in the system, and that backend model activations might still capture the essence of a user's research. This has led to growing advocacy for utilizing local, private LLMs rather than relying on centralized, corporate-owned services that operate with opaque data practices.
Ultimately, the situation has prompted a broader call for action within the academic community to address these risks. Some observers are already organizing to map out strategies for protecting mathematical research from digital appropriation. The incident serves as a stark reminder of the tension between the utility of generative AI and the fundamental need for researchers to maintain ownership and confidentiality of their work before it is ready for public release.
817 comments • Comments Link
• OpenAI 在攻克复杂数学问题上的最新进展被一些人视为真正的突破，但另一些人则认为这些成果在很大程度上依赖于"抢先"（scooping）那些使用这些模型的人类研究者在渐进式努力中取得的成果。
• 围绕 OpenAI 的模型是凭借超人的"直觉"解决问题，还是通过基于数学语料与像 Lean 这样的经人类定义的验证工具进行系统性暴力搜索（即"摘取低垂果实"）来解决问题，存在激烈争论。
• 许多用户深感担忧：基于云的 AI 工具会造成一种结构性脆弱——私有、专有的知识产权被纳入训练集，使得 AI 公司能够利用并最终超越其自身用户。
• AI 公司提供的"退出（opt-out）"机制被一些评论者视为不可靠或具有误导性，批评者指出设置可能在更新时被重置，且像点赞 / 点踩这样的反馈机制可能会绕过对数据使用的限制。
• 怀疑者认为大型语言模型本质上是依赖于人类创造物并由此繁荣的"抄袭机器"，而支持者则主张所有发现本质上都是对先前工作的综合，这使得当前的 AI 辅助流程成为一种自然且被加速的研究演化。
• 一个反复出现的主题是学术归属中感知到的伦理透明度缺失。批评者认为，尽管 AI 模型可能给出最终证明，但人类数学家提供的具体指导和直觉常被忽视，这助长了被剥削和欺骗的感受。
• 这场讨论反映了对学术界和人类智能认知的更广泛转变。随着 AI 代理自动化高层次的综合与问题解决，衡量何为有意义的人类贡献的"目标门柱"正在迅速移动。
• 观察者指出，为了在上市（IPO）前拿出 AI 突破并博取关注，公司之间的竞争压力推动了激进的"抢先"行为，这往往把公关和展示力量置于协作性的学术规范或公平引用之上。
• 一些参与者主张回归本地或私有的模型托管以保护知识产权，并警告说，依赖集中化云服务供应商会在平台商业目标与用户对数据主权的需求之间造成不可调和的利益冲突。
• 关于数学家是否成为技术进步的"受害者"（类似其他领域的变革），或者这种情形是否代表着一种威胁科学长期可持续性的、特殊的掠夺性寻租行为，意见分歧很大。
这场争论的核心是在于：一方面是 AI 加速解决长期科学难题的潜力，另一方面是对部署这些模型的机构信任度下降的紧张关系。尽管普遍承认 AI 在数学领域是强大的搜索与综合工具，但在公司应被视作中立基础设施提供者，还是作为为了竞争优势而利用私人用户数据的积极参与者上，人们存在重大分歧。辩论触及学术归属的本质、数据提取的伦理，以及当前 AI 发展路径是否滋生了寻租行为——这种行为有使那些为模型能力提供劳动力的研究者被边缘化的风险。 • OpenAI's recent advancements in solving complex mathematical problems are viewed by some as genuine breakthroughs, yet others argue these successes are heavily reliant on "sniping" incremental progress made by human researchers using these models.
• There is a significant debate regarding whether OpenAI models are solving problems via superhuman "intuition" or through systematic brute-force search ("plucking low-hanging fruit") enabled by the mathematical corpus and human-defined validation tools like Lean.
• Many users express deep concern that using cloud-based AI tools creates a structural vulnerability where private, proprietary intellectual property is ingested into training sets, effectively allowing AI companies to capitalize on and eventually outperform their own users.
• The "opt-out" mechanisms provided by AI companies are described by several commenters as unreliable or misleading, noting that toggle settings can be reset during updates and that feedback mechanisms (like thumbs-up/down) may bypass data-usage restrictions.
• Skeptics argue that large language models act as "plagiarism machines" that thrive on abstracting human creativity, while proponents contend that all discovery is inherently a synthesis of prior work, making the current AI-assisted process a natural, albeit accelerated, evolution of research.
• A recurring theme is the perceived lack of ethical transparency in academic attribution; critics argue that while the AI model produces the final proof, the specific guidance and intuition provided by human mathematicians are ignored, fueling feelings of exploitation and fraud.
• The discussion reflects a broader shift in the perception of academia and human intelligence, where many feel that the "goalposts" for what constitutes meaningful human contribution are moving rapidly as AI agents automate high-level synthesis and problem-solving.
• Observers suggest that the competitive pressure for companies to secure AI breakthroughs ahead of IPOs drives aggressive "scooping" behavior, prioritizing PR and demonstrations of power over collaborative academic norms or fair citation practices.
• Some participants argue for a return to local, private hosting of models to protect intellectual property, warning that reliance on centralized cloud providers creates an inevitable conflict of interest between the platform's business goals and the user's need for data sovereignty.
• There is intense disagreement regarding whether mathematicians are "victims" of technological progress, similar to past shifts in other fields, or if this represents a uniquely predatory form of rent-seeking that threatens the long-term sustainability of scientific inquiry.
The discussion centers on the tension between the accelerating potential of AI to solve long-standing scientific challenges and the erosion of trust in the entities deploying these models. While there is a consensus that AI serves as a powerful search and synthesis tool for mathematics, there is profound disagreement over whether companies are acting as neutral infrastructure providers or as active participants that exploit private user data for competitive advantage. The debate touches on fundamental questions regarding the nature of academic attribution, the ethics of data extraction, and whether the current trajectory of AI development incentivizes a "rent-seeking" behavior that risks alienating the very researchers whose labor sustains the models' capabilities.