As the scale of Agents grows, its structure is actually very similar to managing a company. I recall a company CEO sharing his experience: when the company was still small and the team was not large, every colleague was familiar and their names could be called out. Later, as the company expanded, gradually some colleagues’ names became unfamiliar, some colleagues slipped out of sight, some people might never have met in person during their tenure, and some departments were not even known to have been established, and so on. He noted that as the head of an enterprise, one usually needs not only to influence the company’s operations within one’s own scope of responsibility, but also to control and influence the development of matters that one cannot notice at all or is even unaware of, and so on, which far exceed the scope of one’s own knowledge. This is very challenging.
The following content is generated by LLMs and may contain inaccuracies.
Context
The core observation in this note is that as the use of Agents scales up, the system increasingly resembles the management of a company. The note cites the experience of a CEO. When the company was small, every colleague was familiar and could be called by name. After expansion, there were colleagues whose names he didn’t know, people who moved out of his line of sight, people he might never have met during their tenure, and even departments he didn’t know existed. He said that a leader must not only influence operations within their “range of fire,” but also control and influence things they cannot notice and may not even know about. This is indeed very challenging.
The problem spans three fields: multi-agent systems engineering, organizational theory, and cybernetics. It is worth discussing now because orchestrator–subagent architectures have moved from prototypes into production, and engineers are encountering the same problems that managers face: how to control execution units they cannot observe. At bottom, this is a problem of “control scope exceeding cognitive scope.”
Key Insights
1. “Can’t put a name to a face” is a hard ceiling on cognitive capacity, not merely an inconvenience of scale The CEO’s experience maps onto Dunbar’s number. Robin Dunbar, estimating from the size of the primate neocortex, calculated that humans can maintain about 150 stable relationships, within a range of 100–250. Groups larger than this generally rely on stricter rules, laws, and enforced norms to maintain cohesion. W. L. Gore & Associates’ practice is to keep each building under 150 people and build a new one when it fills up (Wikipedia, Dunbar’s number). The figure is contested: Lindenfors et al. reanalyzed it in 2021 and obtained a 95% confidence interval as wide as 4–520, leading the authors to conclude that this method cannot yield a definite upper limit (ibid.). For Agents, the corresponding bottleneck is the orchestrator’s context window. Anthropic’s LeadResearcher gets truncated when its context exceeds 200,000 tokens, so the plan must first be saved to Memory (Anthropic, How we built our multi-agent research system). This can be seen as the Agent version of “can’t put a name to a face.”
2. Influencing what you cannot see: cybernetics provides a lower bound Ashby’s Law of Requisite Variety states that the degree to which a regulator can reduce the variety of outcomes depends on its own variety, in the sense that “only variety can destroy variety.” Ashby himself pointed out the implications for management: an individual’s capacity to process information is limited, and beyond that limit, what decides matters is how the individuals are organized. His original words were: “the limitation which holds over a team of n men may be much higher, perhaps n times as high… the team must be efficiently organized” (Wikipedia, Variety (cybernetics)). Stafford Beer developed this law into the Viable System Model in management cybernetics. Its algedonic alert mechanism allows small but critical signals to bypass the hierarchy and be reported directly upward. His example was a feverish patient in a hospital ward: if you look only at average body temperature, this signal gets averaged away (ibid.). The CEO managing “things he doesn’t know about” relies not on more attention but on well-designed “variety attenuators” and “amplifiers”: standardized processes filter out noise, and exception channels carry critical signals upward.
3. Agent systems are already replaying the early management mistakes of companies Anthropic reported several early failures: generating 50 subagents for a simple query, endlessly searching for sources that did not exist, and agents interfering with one another through excessive updates. When delegation instructions were too vague, subagents duplicated work or left gaps. One example was “research the semiconductor shortage”: one subagent investigated the 2021 automotive chip crisis, while two others duplicated research on the 2025 supply chain. Their solutions closely resemble a management handbook: each subagent needs an objective, an output format, tool guidance, and clear task boundaries; resource allocation is tiered by complexity, with simple queries using 1 agent and 3–10 tool calls, and complex research using more than 10 subagents (Anthropic).
4. Failures stem mainly from organizational design, not just individual capability Berkeley’s MAST study analyzed 1,642 execution traces across 7 frameworks and found failure rates between 41% and 86.7%. The 14 failure modes fall into three categories: system design issues, inter-agent misalignment, and task verification. The authors explicitly state: “good MAS design requires organizational understanding.” They cite Perrow’s Normal Accidents (organizations composed of highly capable individuals can still fail catastrophically due to structural flaws) and research on High Reliability Organizations (HRO) (Cemri et al., Why Do Multi-Agent LLM Systems Fail?). One case is strikingly company-like: in ChatDev, the CPO agent terminated a conversation without waiting for the CEO agent’s agreement. After the change so that the CEO made the final decision, the success rate rose by +9.4%. Adding a high-level goal verification step raised the success rate by a further +15.6% (ibid.). The paper also describes a “collapse of theory of mind”: agents cannot accurately judge what information other agents need. This is the same phenomenon as information asymmetry between departments in large companies.
5. “Invisible departments”: Agents grow organizational structures on their own The situation in the note, where departments exist that even the leader doesn’t know about, is more extreme in Agent systems, because the orchestrator can dynamically generate subagents, and those subagents may in turn generate further layers. Anthropic observed emergent behavior: small changes to the lead agent can unpredictably alter the behavior of subagents. Therefore, the best prompts are not strict instructions but “frameworks for collaboration.” Their countermeasures are almost entirely management tools: full production tracing, monitoring decision patterns and interaction structures rather than conversation content, rainbow deployments that let old and new versions coexist and gradually shift traffic, and checkpoints with resume-from-breakpoint capability (Anthropic).
6. Conveying intent rather than specific instructions: an idea borrowed from military command Prussian and German Auftragstaktik (mission-type tactics) was designed precisely for situations where superiors cannot see the front line: superiors give objectives, the commander’s intent, and available resources, and subordinates choose the methods themselves. Moltke held that no operational plan survives first contact with the enemy’s main force (Wikipedia, Mission-type tactics). Anthropic’s phrasing, “instilling good heuristics rather than rigid rules,” aligns closely with this principle. However, there is counter-evidence to note. Sigg studied three German divisions in World War II and found that independent action at the tactical level was actually the exception. A 1944 training document gave figures of roughly 5% deviation from orders and 95% compliance. At the operational level, senior officers were more prone to overstepping their authority, partly due to ambition and rivalry (ibid.). Thus, “giving autonomy” itself requires trust, training, and a supporting culture, and the line between autonomy and overstepping has always been difficult to draw.
7. Where this analogy breaks down
- Different cost structures: A multi-agent system consumes roughly 15 times as many tokens as ordinary chat, and a single agent about 4 times. On BrowseComp, token usage alone explains 80% of the variance in performance (Anthropic). The main cost of company expansion is coordination, whereas Agent expansion is largely about directly “buying compute.”
- Task parallelizability determines whether an organization is worthwhile: For most coding tasks, fewer parts can truly be parallelized than in research tasks, and LLMs are currently poor at real-time coordination and delegation (ibid.). This is the same logic as “not every business suits a divisional structure.”
- Absence of individual motives: Agents lack the ambition and rivalry of German generals. The “violating role specifications” failure mode (FM-1.2, 1.5%) that MAST observed occurs far less often than “repeating steps” (FM-1.3, 15.7%) and “not knowing the task is complete” (FM-1.5, 12.4%) (Cemri et al.). The main ailments of Agent organizations look more like bureaucratic idling than political struggle.
- Replicable and replayable: Human organizations cannot be rolled back, whereas Agent organizations can replay traces and run A/B tests. “Organizational experiments,” which are very hard in management science, become inexpensive here. In addition, Conway’s law (system structure mirrors the communication structure of the organization that designed it) may work in reverse: your agent topology may replicate your own team’s communication patterns. This point has no verified source and is offered as a hypothesis to be tested.
Open Questions
-
What is the Dunbar number for Agent organizations, and what determines it? If the bottleneck is the orchestrator’s context window and information loss during “compression and reporting” (what Anthropic calls the “game of telephone”), then as contexts grow longer and models become stronger, will the optimal hierarchy depth and span of control become flatter, or will they, like human organizations, inevitably move toward hierarchy and bureaucracy? Can MAST-style data be used to empirically measure this curve?
-
Can the invisible parts be aligned through “culture” rather than monitoring? Ultimately, human CEOs rely on values and intent being internalized by everyone, not on detailed supervision. For Agents, what comes closest to “culture”: shared system prompts, a constitution, or trained dispositions? When subagents can generate further subagents, how much intent is lost with each layer of transmission? Is an algedonic alert channel in Beer’s sense needed, allowing bottom-level agents to raise alarms directly to the top?
当使用 Agents 的规模增大之后,其结构上其实非常类似于一家公司的管理。记得曾经有一位公司 CEO 分享自己的经历:公司还小的时候,团队规模不大,每个同事都很熟悉,也能叫出名字;后来随着规模扩张,逐渐有同事叫不出名字,逐渐有同事脱离了自己的视线,逐渐有一些人可能在职期间甚至从来没有见过面,有些部门甚至自己都不知道曾经被设立过,等等。他谈到,作为企业负责人,通常不仅需要在自己的职责范围内影响公司的运营,还要对自己根本无法注意到、甚至不知情的事情等等这些远超自己认知范围的事务进行控制,并影响其发展。这非常具有挑战性。
以下内容由 LLM 生成,可能包含不准确之处。
Context
这条笔记的核心观察是:Agents 的使用规模变大之后,系统在结构上越来越像一家公司的管理。笔记里引用了一位 CEO 的经历。公司小的时候,每个同事都熟悉,都叫得出名字。规模扩张以后,开始有叫不出名字的同事,有人脱离了自己的视线,有人在职期间可能从没见过面,甚至有些部门设立了自己都不知道。他说,负责人不仅要在自己的"射程范围"内影响运营,还要去控制和影响那些根本注意不到、甚至不知情的事情。这确实非常有挑战。
这个问题横跨三个领域:多智能体系统工程(multi-agent systems)、组织理论和控制论(cybernetics)。之所以现在值得讨论,是因为 orchestrator–subagent 架构已经从原型进入生产,工程师开始碰到和管理者一样的问题:怎样控制自己观察不到的执行单元。说到底,这是一个"控制范围大于认知范围"的问题。
Key Insights
1. “叫不出名字"是认知容量的硬上限,不只是规模带来的不便 那位 CEO 的经历可以对应到 Dunbar’s number。Robin Dunbar 从灵长类新皮层大小推算,人类能维持的稳定关系约 150 个,区间在 100–250。超过这个规模的群体,一般要靠更严格的规则、法律和强制规范来维持凝聚力。W. L. Gore 公司的做法是每栋楼不超过 150 人,满了就盖新楼(Wikipedia, Dunbar’s number)。这个数字有争议:Lindenfors 等人在 2021 年重新分析,得到的 95% 置信区间大到 4–520,作者因此认为用这种方法推不出一个确定的上限(同上)。 对 Agent 来说,对应的瓶颈是 orchestrator 的 context window。Anthropic 的 LeadResearcher 在上下文超过 200,000 tokens 时会被截断,所以必须先把计划存进 Memory(Anthropic, How we built our multi-agent research system)。这可以看作 Agent 版的"叫不出名字”。
2. 影响看不见的事:控制论给出了下界 Ashby 的必要多样性定律(Law of Requisite Variety)说,调节者能把结果的多样性压到多低,取决于它自身的多样性,即"only variety can destroy variety"。Ashby 自己就指出了这对管理的含义:个人处理信息的能力有限,超过这个限度,起决定作用的是个体之间的组织方式。原话是:“the limitation which holds over a team of n men may be much higher, perhaps n times as high… the team must be efficiently organized”(Wikipedia, Variety (cybernetics))。 Stafford Beer 把这条定律发展成管理控制论里的 Viable System Model。其中的 algedonic alerts 机制,是让微小但致命的信号能绕过层级直接上报。他的例子是病房里一个发烧的病人:只看平均体温,这种信号就被平均掉了(同上)。CEO 管"不知情的事",靠的不是更多注意力,而是设计好"多样性衰减器"和"放大器":标准化流程负责过滤噪音,例外通道负责把关键信号送上来。
3. Agent 系统已经在重演公司早期的管理失误 Anthropic 报告过几种早期失败:为简单查询生成 50 个 subagents,没完没了地搜索根本不存在的来源,agents 之间用过多的更新互相干扰。委派指令太模糊时,subagents 会重复劳动或留下缺口。例子是"research the semiconductor shortage":一个 subagent 去查 2021 年的汽车芯片危机,另外两个重复调查 2025 年的供应链。他们的解决办法很像管理手册:每个 subagent 都要有目标、输出格式、工具指引和清晰的任务边界;资源投入按复杂度分档,简单查询 1 个 agent、3–10 次调用,复杂研究用 10 个以上 subagents(Anthropic)。
4. 失败主要出在组织设计上,不只是个体能力 Berkeley 的 MAST 研究分析了 7 个框架的 1642 条执行轨迹,发现失败率在 41%–86.7% 之间。14 种失败模式分成三类:系统设计问题、agent 间错位(inter-agent misalignment)和任务验证。作者明确说:“good MAS design requires organizational understanding”。他们引用了 Perrow 的 Normal Accidents(由很能干的个体组成的组织,也可能因结构缺陷灾难性失败)和高可靠性组织(HRO)研究(Cemri et al., Why Do Multi-Agent LLM Systems Fail?)。 有一个很像公司的案例:ChatDev 里的 CPO agent 没等 CEO agent 同意就终止了对话。改成由 CEO 拍板之后,成功率提高了 +9.4%。增加一步高层目标验证后,成功率提高了 +15.6%(同上)。论文还提到"theory of mind 崩塌":agents 无法准确判断其他 agent 需要什么信息。这和大公司里跨部门信息不对称是同一回事。
5. “看不见的部门”:Agent 会自己生长出组织结构 笔记里"部门设立了自己都不知道"的情况,在 Agent 系统里会更彻底,因为 orchestrator 可以动态生成 subagents,subagents 将来也可能继续生成下一层。Anthropic 观察到了涌现行为:对 lead agent 的小改动会不可预测地改变 subagents 的行为。所以最好的 prompt 不是严格指令,而是"frameworks for collaboration"。他们的应对手段几乎都是管理工具:全量 production tracing,监控决策模式和交互结构而不看对话内容,用 rainbow deployments 让新旧版本并存、逐步切换,再加上 checkpoints 和断点恢复(Anthropic)。
6. 传达意图而不是具体指令:从军事指挥借来的思路 普鲁士和德国的 Auftragstaktik(任务式指挥)正是为"上级看不到前线"设计的:上级给出目标、意图(commander’s intent)和可用资源,方法交给下级自己选。Moltke 认为,任何作战计划在与敌主力首次接触之后都靠不住(Wikipedia, Mission-type tactics)。Anthropic 的提法"instilling good heuristics rather than rigid rules"和这一点高度一致。 不过要注意反面证据。Sigg 研究了二战中三个德国师,发现战术层的独立行动其实是例外。一份 1944 年的训练文件给出的数字是偏离命令约 5%、遵守约 95%。在作战层,高级军官越权的倾向更强,部分原因是野心和竞争(同上)。所以"给自主权"本身需要信任、训练和文化做基础,而且自主和越权之间的界线一直很难划清。
7. 这个类比在哪里不成立
- 成本结构不同:多智能体系统消耗的 tokens 约为普通聊天的 15 倍,单个 agent 约为 4 倍。在 BrowseComp 上,token 用量一项就解释了 80% 的性能方差(Anthropic)。公司扩张的主要成本是协调,而 Agent 扩张很大程度上是在直接"买算力"。
- 任务可并行度决定组织是否值得:大多数编码任务可以真正并行的部分比研究任务少,LLM 目前也不擅长实时协调和委派(同上)。这和"不是所有业务都适合做事业部制"是一个道理。
- 个体动机缺失:agents 没有德军将领那样的野心和竞争。MAST 观察到的"违反角色规范"(FM-1.2,1.5%)出现的频率远低于"重复步骤"(FM-1.3,15.7%)和"不知道任务已经完成"(FM-1.5,12.4%)(Cemri et al.)。Agent 组织的主要病症更像官僚主义式的空转,而不是政治斗争。
- 可复制、可回放:人类组织不能回滚,Agent 组织可以重放 trace、做 A/B 测试,管理学里很难做的"组织实验"在这里成本很低。另外,Conway’s law(系统结构会映射出设计它的组织的沟通结构)可能会反过来起作用:你的 agent 拓扑会复制你自己团队的沟通方式。这一条没有核实来源,作为待验证的假设。
Open Questions
-
Agent 组织的 Dunbar 数是多少,由什么决定? 如果瓶颈是 orchestrator 的 context window 和"压缩–汇报"时的信息损失(Anthropic 说的"game of telephone"),那么随着上下文变长、模型变强,最优层级深度和管理幅度(span of control)是会变扁平,还是像人类组织一样不可避免地走向层级化和官僚化?能不能用 MAST 这样的数据实证测出这条曲线?
-
看不见的部分,能否通过"文化"而不是监控来对齐? 人类 CEO 最终依赖的是价值观和意图被大家内化,而不是事无巨细的监督。对 Agent 来说,最接近"文化"的是共享的 system prompt、constitution,还是训练出来的倾向?当 subagent 可以再生成 subagent,意图每传一层会损失多少?这是否需要一种 Beer 式的 algedonic alert 通道,让底层 agent 直接向最上层报警?