How Our Rust-to-Zig Rewrite Is Going
508 points
• 1 day ago
• Article
Link
Roc 编程语言背后的团队达成了一个重要里程碑：在将编译器从 Rust 重写为 Zig 的过程中已实现功能对等。历时十八个月、编写了约 30 万行代码，这次迁移标志着项目发展的重大转折。尽管与其他项目的快速重写相比工作量更大，但重写决定源自对基础架构进行根本性调整以提升性能和语言特性的需求。团队正期待在今年晚些时候发布官方 0.1.0 。
转向 Zig 的主要原因是需要更细粒度的内存控制以及更快、更稳定的编译。虽然 Rust 的 borrow checker 在安全性上很出色，但为优化诸如 polymorphic defunctionalization 这样的性能关键任务，Roc 编译器不得不大量使用 unsafe 代码块和自定义内存分配器。 Zig 在生态上将这些内存模式视为常态，并且通过增量编译（incremental compilation）等手段，其构建时间已显示出远超此前 Rust 配置的潜力。
此次迁移带来的核心创新之一是 zero-parse deserialization 。借助以无指针数组形式组织的特定数据结构，编译器能够将缓存的构建结果直接加载到内存，基本上以磁盘 I/O 的速度运行。该方法技术上复杂，但绕过了传统解析开销，从而实现极快的增量重建。项目还利用 Zig 简化了 cross-compilation，以最少的配置为包括 WebAssembly 在内的多个平台生成一致的 static binaries 。
在内存安全方面，团队的实际感受与预期一致。尽管 Rust 常因其安全性受到赞誉，编译器开发的性质却包含一些 borrow checker 无法完全消除的固有风险，尤其是涉及生成的 machine code 的正确性。通过在 safety-checking 模式下使用 Zig，团队得以维持较高的稳定性。迁移并未引发内存相关问题的激增，这表明对于这个高度专业化的项目而言，与 Zig 设计理念在架构层面的契合，弥补了放弃 Rust 编译时内存保证的代价。
这次转型并非没有取舍，开发者确实怀念 Rust 的一些便利，例如测试中自动处理的 drop 逻辑、 private struct fields 以及在发布时更成熟的向后兼容规范。但相比获得的收益，这些被视为可以接受的代价。 Zig 的"减法"设计避开了像 macros 这样复杂的特性，转而依赖 comptime 和对 data layout 的直接操控，这与团队目标高度契合。该决定让他们重新掌控构建流水线，并采用更高效、以 allocator 为中心的编程风格，更好地满足现代编译器的特殊需求。
The team behind the Roc programming language has reached a significant milestone by achieving feature parity in their compiler rewrite from Rust to Zig. After eighteen months and 300,000 lines of code, this transition marks a major shift in the project's development. While this effort was substantial compared to the swift rewrites seen in other projects, the decision to rebuild was motivated by the need for fundamental architectural changes to improve performance and language features. The team is now looking toward their official 0.1.0 release later this year.
The primary driver for moving to Zig was the need for granular memory control and faster, more reliable compilation. While Rust provides excellent safety guarantees through its borrow checker, the Roc compiler requires heavy use of unsafe blocks and custom memory allocators to optimize its performance-critical tasks, such as polymorphic defunctionalization. Zig offered an ecosystem where these memory patterns are standard, alongside build times that, through techniques like incremental compilation, have already shown the potential to be vastly faster than the previous Rust configuration.
A central innovation enabled by this transition is zero-parse deserialization. By utilizing specific data structures organized as arrays without pointers, the compiler can load cached build results directly into memory, essentially operating at the speed of disk I/O. This approach, while technically complex, bypasses traditional parsing overhead, allowing for extremely fast incremental rebuilds. The project has also leveraged Zig to simplify cross-compilation, producing consistent static binaries for various platforms, including WebAssembly, with minimal configuration.
Regarding memory safety, the team found that their practical experience aligned with their expectations. While Rust is often lauded for its safety, the nature of compiler development involves inherent risks that the borrow checker cannot entirely eliminate, particularly concerning the correctness of machine code output. By using Zig with its safety-checking modes, the team successfully maintained a high standard of stability. The shift did not result in a surge of memory-related issues, suggesting that for this specific, highly specialized project, the architectural alignment with Zig's design philosophy outweighed the loss of Rust's compile-time memory guarantees.
The transition has not been without its trade-offs, as the developers do miss certain Rust conveniences, such as the automatic handling of drop logic in tests, private struct fields, and a more mature standard for backward compatibility during releases. However, these are viewed as acceptable costs given the benefits. The subtractive nature of Zig, which avoids complex features like macros in favor of comptime and direct data layout manipulation, has resonated well with the team's goals. This move has allowed them to reclaim control over their build pipeline and adopt a more efficient, allocator-centric programming style that fits the unique demands of a modern compiler.
269 comments • Comments Link
• 关于编译器本质上需要内存不安全代码的说法备受质疑——生成机器码本质上是写入数据的操作，并不必然要求使用不安全操作。
• 虽然在热替换（hot-swapping）或性能关键的运行时内部组件中可能会使用 unsafe 块，但编译器的核心逻辑（如解析和代码生成）在很大程度上仍可保持高层抽象并保持安全。
• 必须区分编译器自身的内存安全性与其生成的二进制的安全性。一个内存安全的编译器即便生成了含漏洞的代码，仍然有价值，因为它能隔离并减少错误的攻击面。
• "不安全"（unsafe）一词常被误解为等同于"低级"（low-level），由此产生的误导让人以为任何系统级工作都必须放弃语言层面的内存保护。
• Zig 的 ReleaseSafe 模式对某些错误提供运行时检查，但它并不能像 Rust 的借用检查那样，提供全面的时序内存安全，例如无法完全防止所有的 use-after-free 情形。
• 编译器性能主要由算法效率决定，而不是语言本身的"低级"特性。历史上有带垃圾回收的高级语言成功用于编译器开发，这挑战了为性能必须使用系统语言的观点。
• 从一种稳定成熟的语言迁移到尚未到达 1.0 的语言风险很大，因为破坏性变更和不稳定的生态可能破坏长期维护性和生产可靠性。
• 快速增量构建（如 Zig 展示的那样）能显著提升开发者生产力，尽管 Rust 也在积极改进构建性能和工件管理。
• Rust 在大量使用泛型和复杂类型系统时，可能在大型项目中带来显著的编译时开销，这促使一些人在特定领域（如 Web 后端）考虑更精简或更简单的替代方案。
• 行业内正经历语言采纳的周期，团队会根据当前项目的瓶颈在编译速度、内存控制与安全保证之间权衡，从而重新评估并可能迁移现有工具链。
核心争论在于：使用 Rust 这样的内存安全高级语言与使用 Zig 这样的底层手动管理语言之间的权衡，能否通过最终的性能与安全成果来证明其合理性。虽然各方普遍认同编译器性能至关重要，但关于内存安全是否是现代工具的根本瓶颈或必要保障存在明显分歧。许多观察者指出，编译器的架构与算法设计仍然是决定速度的主要因素，这表明语言选择往往次于实现者的能力。最终，这场讨论凸显了追求"无畏安全"与满足快速迭代及构建性能的现实需求之间的持久张力。 • The assertion that compiler development inherently requires memory-unsafe code is widely contested, as emitting machine code is a data-writing task that does not fundamentally necessitate unsafe operations.
• While compilers may utilize unsafe blocks for specific needs like hot-swapping or performance-critical runtime internals, core compilation logic, including parsing and code generation, remains largely abstract and safe.
• Distinctions between the safety of a compiler and the safety of the binary it produces are critical. A memory-safe compiler is beneficial even if it generates code that could contain vulnerabilities, as it isolates and reduces the surface area for errors.
• The use of the term "unsafe" is often conflated with "low-level," leading to the misconception that any system-level work requires disabling language-level memory protections.
• Zig's "ReleaseSafe" mode provides runtime checks for certain errors, but it does not offer comprehensive temporal memory safety, such as protection against all use-after-free scenarios, unlike the borrow-checking guarantees found in Rust.
• Compiler performance is primarily driven by algorithmic efficiency rather than the choice of a low-level language. Historically, high-level languages with garbage collection have been effectively used for compilers, challenging the necessity of systems languages for performance goals.
• Transitioning from a stable, established language to a pre-1.0 language involves significant risk, as breaking changes and unstable ecosystems can disrupt long-term maintenance and production reliability.
• Rapid incremental builds, such as those demonstrated by Zig, represent a significant advantage for developer productivity, though Rust is actively working on improving build performance and artifact management.
• Rust's extensive use of generics and complex type systems can cause significant compile-time overhead in large projects, leading some developers to consider more streamlined or simpler alternatives for specific domains like web backends.
• The industry is seeing a cycle of language adoption where teams prioritize different trade-offs—such as compilation speed, memory control, or safety guarantees—often leading to re-evaluations of existing tools and migrations based on current project bottlenecks.
The debate centers on whether the trade-offs involved in using a memory-safe, high-level language like Rust versus a lower-level, manually managed language like Zig are justified by the resulting performance and safety outcomes. While participants generally agree that compiler performance is vital, there is a clear disagreement regarding whether memory safety is an inherent bottleneck or a necessary safeguard in modern tooling. Many observers note that compiler architecture and algorithmic design remain the dominant factors in speed, suggesting that language choice is often secondary to the skill of the implementer. Ultimately, the discussion highlights a persistent tension between the desire for "fearless" safety and the practical demands of fast iteration and build performance.