Towards a harness that can do anything
210 points
• 2 days ago
• Article
Link
为了真正突破 chat pane 的局限，LLM harness 应当直观、透明、精简且具备弹性。其首要目标是将以 tokens 计量的认知负荷降到最低——确保 agent 不会在本应轻松应付的任务上浪费有限的上下文。确定性至关重要：虽然 LLM 应能自主设定目标，但其推演过程应由明确且模块化的步骤构成。通过保持 core prompt 简洁，并允许 agent 在运行时加载所需技能，开发者就能避免随着上下文接近上限而常见的性能衰退。
成功的 harness 会借助 agent 已有的编程和系统运维知识。鉴于 LLMs 在这些领域被大量训练，为它们提供熟悉的环境（比如标准的基于文件的结构）可避免在新奇或复杂的系统上浪费 tokens 。该方法把工具当作 binaries 、把数据当作 text streams 、把运行环境当作 file system，从而让 agent 在后台执行日志记录、数据清洗等复杂任务，同时保持界面轻量。
所提出的 Ambiance 架构借鉴 Unix 的理念：编写专注于单一职责、表现优异的模块化 tools，确保它们通过 text streams 协作，并在失败时能够明显报错。通过采用 Filesystem Hierarchy Standard，harness 按 LLM 能立即识别的方式组织 data：从 logs 和 system binaries，到用户工作区和 documentation，所有内容都映射到特定的 directories 。这样，开发者和 agent 都可以用 grep 、 find 等标准命令来审计、搜索和管理任务，几乎把 operating system 变成了 agent 的自然栖息地。
Ambiance 不采用强制 agent 按固定间隔检查更新的 heartbeat 方式，而是用了一个 event-driven kernel 。该 kernel 作为 LLM 与外界之间的中间层，监控 file changes，仅在必要时才唤起 agent 。此策略避免了持续 polling 的低效，同时确保不会漏掉任何 notification 或外部状态变更。把安全和协调等繁重工作下放给 kernel，系统因此能够保持高效与响应性。
在这个环境中，agent 以不同的 specialized users 身份运行：root user 负责系统级任务与修复，librarian 负责追踪 agent 的表现并维护 system history，primary interface user 负责对外交互。各类 agents 通过 event bus 持续通信，构成一个有凝聚力、自愈且透明的系统。最终，利用 model 已有的关于 files 与 systems 运作的知识，该 harness 便成为一个强大且可适应的框架，使 LLM 能真正体现其 agency 。
To effectively move beyond the constraints of a chat pane, an LLM harness must be intuitive, transparent, lean, and resilient. The primary goal is to minimize cognitive load, which is measured in tokens, by ensuring the agent does not waste its limited context on tasks it should naturally excel at. Determinism is key; while the LLM should choose its own goals, the deliberation process requires well-defined, modular steps. By keeping the core prompt small and allowing the agent to load necessary skills at runtime, developers can prevent the performance degradation that typically occurs as context limits are reached.
A successful harness leverages the agent's a priori coding and systems administration knowledge. Because LLMs are trained extensively on these topics, providing them with a familiar environment—such as a standard file-based structure—prevents the waste of tokens on novel or convoluted systems. This approach treats tools as binaries, data as text streams, and environments as a file system, allowing the agent to perform complex tasks like logging and sanitization in the background while keeping the interface lightweight.
The proposed architecture, known as Ambiance, draws inspiration from the Unix philosophy: write modular tools that do one thing well, ensure they work together via text streams, and design them to fail loudly. By adopting a Filesystem Hierarchy Standard, the harness organizes data in a way that is immediately recognizable to an LLM. Everything from logs and system binaries to user workspaces and documentation is mapped to specific directories. This setup enables both the developer and the agent to audit, search, and manage tasks using standard commands like grep or find, effectively turning the operating system itself into the agent's natural habitat.
Instead of relying on the standard heartbeat method, which forces agents to check for updates at fixed intervals, Ambiance uses an event-driven kernel. This kernel acts as a middle layer between the LLM and the outside world, watching for file changes and invoking the agent only when necessary. This strategy avoids the inefficiency of constant polling while ensuring that no notification or external state change is missed. By offloading the heavy lifting of safety and coordination to this kernel, the system remains reactive and efficient.
Within this environment, the agent operates through specialized users. A root user manages system-level tasks and repairs, a librarian tracks agent performance and maintains system history, and a primary interface user handles external interactions. These agents communicate continuously via an event bus, creating a cohesive, self-healing, and transparent system. Ultimately, by utilizing the model's existing knowledge of how files and systems function, the harness becomes a powerful, adaptable structure that allows the LLM to operate with genuine agency.
103 comments • Comments Link
• 有效的 agentic workflows 优先采用确定性的脚手架，仅在更大、以代码驱动的执行循环中将 LLM 用于处理非确定性边缘情况。
• "Unix philosophy"——小而专、一事做到极致——非常适合 agent 开发，在这种模式下 LLMs 成为连接离散、可靠可执行文件的粘合剂。
• 从一开始对脚本进行过度泛化往往适得其反；"Write Everything Twice"的做法——只有在出现第三个用例时才开始抽象——通常能产生更有弹性的自动化。
• 相比依赖 tests 、 linter hooks 和严格外部结构来限制 agent 故障模式的"harness"或"skeleton"，"committing the prompt"的当前趋势被视为脆弱。
• Behavior-Driven Development (BDD) 和 Gherkin-style tests 对 agents 十分有用，因为它们提供了清晰、可读的规范，可在转为自动化测试前由 AI 手工验证。
• 开发者越来越偏向元过程，让 agents 生成并维护构成系统"exoskeleton"的脚本与工具，从而把"自动化"这件事进一步自动化。
• 将 agents 视为人类协作者更为有效：为它们提供定义明确的 context 、 clear intent 和 robust tooling，往往比依赖无结构的"魔法"prompt-based agents 更可靠。
• 对"agent harnesses"的激增保持健康怀疑，许多人认为它们可能用过度复杂的方法去解决那些 event-driven scripts 或 standard terminal utilities 已能胜任的问题。
• "agentic harness"的定义存在争议；有人倾向用"scaffold"或"skeleton"来强调严格控制，另一些人则把它看作必须由确定性过程掌握的通信通道，以确保可衡量的可靠性。
• 在 prompts 与 code 间流畅迁移逻辑，可以在确定性代码的速度与成本效率以及系统快速扩展所需的灵活性之间找到平衡。
讨论达成的共识是：可靠的 AI 自动化应把 agentic behavior 锚定在确定性、以代码为驱动的 workflows 中，尽量减少对"vibe-based" prompt chains 的依赖。参与者普遍认为，最稳健的系统应把 LLMs 当作专用工具而非主控者，采用由 OS 或 shell 管理编排与状态、由 AI 执行特定且定义明确任务的架构。尽管社区对构建"agent harnesses"充满热情，但这种热情被对简洁性、可审计性以及重用现有 Unix-like primitives 的实际偏好所制约，而非推动创建专有且不透明的框架。总体上，工程师倾向于把 agents 当作能力强但容易出错的同事，需通过严格的 behavioral gates 与 coded tests 来保证其输出的可预测性与可维护性。 • Effective agentic workflows prioritize deterministic scaffolding, relegating the LLM to handle only non-deterministic edge cases within a larger, code-driven execution loop.
• The "Unix philosophy" of small, specialized programs that do one thing well is highly applicable to agent development, where LLMs function as the glue connecting discrete, reliable executables.
• Over-engineering scripts from the start is counterproductive; a "Write Everything Twice" approach, where generalization occurs only when a third use-case emerges, often yields more resilient automation.
• The current trend of "committing the prompt" is seen as a fragile approach compared to building a "harness" or "skeleton" that constrains agent failure modes through tests, linter hooks, and rigid external structures.
• Behavior-Driven Development (BDD) and Gherkin-style tests are valuable for agents, as they provide clear, human-readable specifications that can be validated manually by an AI before being codified into automated tests.
• Developers are increasingly shifting toward meta-processes where agents are tasked with generating and maintaining the scripts and tools that form the "exoskeleton" of the system, effectively automating the automation.
• Treating agents like human collaborators—providing them with well-defined context, clear intent, and robust tooling—outperforms reliance on "magic" prompt-based agents that lack structure.
• There is a healthy skepticism toward the proliferation of "agent harnesses," with many viewing them as potentially over-complicated attempts to solve problems that simple, event-driven scripts or standard terminal utilities already address.
• Defining the "agentic harness" is a point of contention; while some prefer "scaffold" or "skeleton" to emphasize rigid control, others view it as a communication channel that must be owned by a deterministic process to ensure measurable reliability.
• Moving logic between prompts and code in a fluid manner allows for a balance between the speed and cost-efficiency of deterministic code and the flexibility required for rapid system extension.
The discussion reflects a consensus that the future of reliable AI automation lies in minimizing dependence on "vibe-based" prompt chains by anchoring agentic behavior in deterministic, code-driven workflows. Participants emphasize that the most robust systems treat LLMs as specialized tools rather than primary controllers, preferring an architecture where the OS or shell manages the orchestration and state, while the AI performs specific, well-defined tasks. While there is significant enthusiasm for creating "agent harnesses," this is tempered by a practical preference for simplicity, auditability, and the reuse of existing Unix-like primitives over the creation of proprietary, opaque frameworks. Ultimately, the community leans toward a model where engineers treat agents as highly capable but error-prone coworkers, requiring strict behavioral gates and coded tests to ensure their output remains predictable and maintainable.