Ideas想法

Sharing and recording scattered thoughts and writings.

在这里分享并记录一些零散的想法及写作。

10 / 101 ideas
-/-

The Ultimate Purpose of Reading读书的终极目的

The purpose of reading is not to compete or compare, not to accumulate knowledge, not to establish authority, and not to memorize right and wrong. The purpose and ultimate goal of reading is to stand at a higher dimension and understand the world: engaging with the world and withdrawing from it, withdrawing from it and engaging with it, with life renewing itself endlessly.

The following content is generated by LLMs and may contain inaccuracies.

The Ultimate Purpose of Reading

Original text: The purpose of reading is not to compete or keep up with others, not to accumulate knowledge, not to establish authority, and not to memorize right and wrong. The purpose and the end of reading is to be able to understand the world from a higher latitude, to engage with the world and withdraw from it, to withdraw from the world and engage with it, with life renewing itself without end.


Context

This note concerns epistemology, educational philosophy, and the tradition of self-cultivation. It asks what reading is ultimately for. It first rejects four common motives:

  1. Competition and comparison: reading becomes a chip in social rivalry;
  2. Accumulating knowledge: reading becomes the piling up of information;
  3. Establishing authority: reading becomes capital for discursive power;
  4. Memorizing right and wrong: reading becomes the memorizing of standard answers.

All four treat reading as a tool aimed at external rewards or at accumulated stock. The author’s positive answer is: to understand the world from a higher dimension (the original says “纬度” (latitude), which should be “维度” (dimension), a typo for a level of cognition), and to move back and forth between “engaging with the world” and “withdrawing from it,” so that life renews itself without end.

Why it is worth raising this now: information overload, reading check-ins, “paid knowledge,” and AI that can give “correct answers” at any moment all devalue “accumulating knowledge” and “memorizing right and wrong” even faster. The value of reading can only shift toward higher-level abilities, that is, toward how one looks at problems rather than how much one knows.


Key Insights

1. “Not accumulating knowledge”: a shared warning from East and West

  • Zhuangzi pointed out long ago that piling up knowledge alone is a dead end: “My life has a limit, but knowledge has none. To pursue the limitless with the limited is perilous.” (Zhuangzi, “Nourishing the Lord of Life,” Wikisource). A finite life chasing infinite knowledge can only end in exhaustion. This directly corresponds to the note’s rejection of “accumulating knowledge.”
  • In On Thinking for Oneself and other essays, Schopenhauer repeatedly urged people to restrain their reading and cultivate independent thought. The translator T. Bailey Saunders, in his preface, quotes a line said to come from Hobbes: “if he had read as much as other men, he should have been as ignorant as they.” (Schopenhauer, The Art of Literature, Project Gutenberg). He also divides authors into three kinds: those who write from memory, copying others; those who think while they write; and those who think before they write. Only the last kind is rare and valuable. The same holds for readers: if one only “memorizes” and does not “think,” books become plaster casts of other people’s thoughts.
  • A counterpoint: rejecting “accumulating knowledge” does not mean knowledge is useless. Adler held that practical knowledge must be truly mastered through experience, and that “insight” is best learned from those who first attained it, that is, by reading the original classics (How to Read a Book, Wikipedia). Knowledge is the steps of a climb; the problem lies in mistaking the steps for the summit.

2. “Not establishing authority, not memorizing right and wrong”: from judging right and wrong to understanding why

  • Adler divided analytical reading into three stages: structural, interpretive, and critical. His key rule is that one may not say “I disagree” until one has identified the author’s errors in reasoning, facts, or premises, though one may say “I dislike it” (Wikipedia). This is precisely the opposite of “memorizing right and wrong”: right and wrong are not conclusions one memorizes, but the license to judge that one earns after understanding.
  • Schopenhauer mocked authors who “keep changing their stylistic masks” and use obscurity to pass for profundity, saying they “sell words for thoughts” (ibid., Gutenberg). Using reading to “establish authority” can easily slip into this kind of performance.

3. “Higher dimensions”: the ultimate goal of reading is a leap in levels of cognition

  • The fourth part of Adler’s book is titled The Ultimate Goals of Reading, nearly the same as this note’s title. He placed the highest level at syntopical reading: reading many books on a single theme and building for oneself a framework that can hold the conflicting views of each. Its conclusions point to “growth of the mind” and “fuller experience as a conscious being” (Wikipedia). This gives “higher dimension” an actionable definition: the ability to hold and compare multiple perspectives at once, rather than standing inside just one of them.
  • Zhuangzi’s “Cook Ding Carves an Ox” offers another description: “When I first began to carve oxen, all I saw was the ox. After three years, I no longer saw the whole ox. Now I meet it with my spirit rather than looking with my eyes… What I love is the Way, which goes beyond skill.” (Zhuangzi, “Nourishing the Lord of Life”). The movement from “seeing only the ox,” to “no longer seeing the whole ox,” to “meeting it with spirit” is a rise from the level of information to the level of structure, and then to the level of the Way.
  • Related modern theories (the following sources’ links have not been verified on the spot): the developmental psychologist Robert Kegan’s subject–object theory holds that growth means turning what once controlled us into an object that we can see, which is highly isomorphic with “seeing from a higher dimension.” Laozi’s “In pursuing learning, one increases daily; in pursuing the Way, one decreases daily” (Dao De Jing, Chapter 48) also suggests that a higher dimension often means subtracting rather than adding.

4. “Engaging with the world and withdrawing from it, withdrawing from the world and engaging with it”: a round trip, not a destination

  • The chiastic structure of the original phrase “engaging with the world and withdrawing from it, withdrawing from the world and engaging with it” is crucial. It is not the one-directional narrative that “reading lets a person escape into transcendence,” but a back-and-forth process. The Commentary on the Appended Statements in the Book of Changes says, “The way of the gentleman is sometimes to advance and sometimes to retreat, sometimes to be silent and sometimes to speak” (Wikisource). Advancing and withdrawing coexist and are switched according to the moment.
  • In the Song dynasty, Zen master Qingyuan Weixin gave the famous three stages: “Before I studied Zen, mountains were mountains and waters were waters. When I got an insight into it, mountains were not mountains and waters were not waters. Now that I have attained it, mountains are just mountains and waters are just waters” (Wudeng Huiyuan). The third stage returns to the original place, but one is no longer on the original level. “Withdrawing from the world and engaging with it” captures exactly this spiral: withdrawing is ascending to see the structure; engaging is descending to return to concrete life with that structure in hand.
  • This is also a shared response to the two failures of “reading is useless” and “bookish pedantry”: engaging without withdrawing, drowned in trivia; withdrawing without engaging, hanging in empty talk.

5. “Life renewing itself without end”: reading as an open process of becoming

  • “Renewing life” comes from the Commentary on the Appended Statements in the Book of Changes: “To be rich in what one has is called the great enterprise; to renew it daily is called supreme virtue; to bring forth life endlessly is called change.” The same chapter also says, “Writing cannot exhaust speech, and speech cannot exhaust meaning,” and “The spiritual and clear dwells in the person; the silent and accomplished, without speaking yet trusted, dwells in virtuous conduct” (Wikisource). One may understand this as follows: the book itself cannot fully express its meaning; meaning must be activated by the reader and then put into practice as virtuous conduct. So the ultimate end of reading is not a closed state of conclusions, but the capacity for daily renewal.
  • At the end of “Nourishing the Lord of Life,” Zhuangzi says, “The firewood may be used up, but the fire is passed on, and its end is unknown” (Wikisource). Specific books and knowledge (the firewood) will always burn out, but the understanding that has been kindled (the fire) can be passed on endlessly. This may be the most fitting image of “life renewing itself without end” as applied to reading.

6. A tension: could this view of purpose become a new form of comparison?

  • “Higher dimensions” themselves could become badges of superiority (“I see more clearly than you”), circling back to the “competition and comparison” and “establishing authority” that the original rejects. Schopenhauer’s criticism of “pretending to be profound” applies here as well. The test of whether one has truly risen is probably not in what one says, but on the “engaging with the world” side: whether one can make better judgments in concrete situations and is more able to embrace different people.

Open Questions

  1. Can “dimension” be tested? If the value of reading lies in raising levels of cognition rather than in the stock of knowledge, how can we tell whether a person (or an AI system) has really “ascended,” rather than having only learned to talk about ascending? Can Adler’s syntopical reading and Kegan’s subject–object framework be turned into observable indicators?
  2. In the AI era, who completes the round trip of “engaging and withdrawing”? When an LLM can instantly supply “knowledge” and “right and wrong,” does reading retain only the “withdrawing” half (gaining a structural perspective)? Will the “engaging” half, that is, grounding understanding in concrete life and virtuous conduct (“to accomplish it in silence… dwells in virtuous conduct”), become the one part of reading that human readers cannot outsource?

读书的目的不是攀比竞争,不是增加知识,不是建立权威,不是记忆对错。读书的目的和终局是能够站在更高的维度理解世界,入世出世,出世入世,生生不息。

以下内容由 LLM 生成,可能包含不准确之处。

读书的终极目的

原文:读书的目的不是攀比竞争,不是增加知识,不是建立权威,不是记忆对错,读书的目的和终局是能够站在更高的纬度理解世界,入世出世,出世入世,生生不息。


Context

这条笔记涉及认识论、教育哲学和修身传统,讲的是读书到底为了什么。它先否定了四种常见动机:

  1. 攀比竞争:读书变成社会比较的筹码;
  2. 增加知识:读书变成信息的累加;
  3. 建立权威:读书变成话语权的资本;
  4. 记忆对错:读书变成背标准答案。

这四种都把读书当成工具,指向外部回报或存量积累。作者给出的正面答案是:站在更高的纬度(应是"维度"的笔误,指认知层级)理解世界,并且能在"入世"和"出世"之间来回切换,生生不息。

为什么现在值得重提:信息过载、读书打卡、“知识付费”、AI 随时能给出"正确答案",这些都让"增加知识"和"记忆对错"贬值得更快。读书的价值只能往更上层的能力转移,也就是怎样看问题,而不是知道多少。


Key Insights

1. “不是增加知识”:东西方的共同警告

  • 庄子早就指出单纯积累知识是死路:“吾生也有涯,而知也无涯。以有涯随无涯,殆已。"(《庄子·养生主》,维基文库)。有限的生命追逐无限的知识,结果只能是疲惫。这正对应原文否定的"增加知识”。
  • 叔本华在 On Thinking for Oneself 等文章中反复劝人节制阅读、培养独立思考。译者 T. Bailey Saunders 在序言中引了一句据传出自 Hobbes 的话:“if he had read as much as other men, he should have been as ignorant as they."(Schopenhauer, The Art of Literature, Project Gutenberg)。他还把作者分成三类:凭记忆、照抄别人书写作的;边写边想的;先想后写的。只有最后一类罕见而有价值。读者也一样:只"记忆"不"思考”,书就成了别人思想的石膏模型。
  • 反方提醒:否定"增加知识"不等于知识没用。Adler 认为实践知识必须靠经验才能真正掌握,“理解”(insight)最好向最早获得这种理解的人学习,也就是读原典(How to Read a Book, Wikipedia)。知识是攀登的台阶,问题出在把台阶当成终点。

2. “不是建立权威、不是记忆对错”:从"判断对错"到"理解为何"

  • Adler 把分析阅读分成三步:structural(结构)、interpretive(诠释)、critical(批判)。他的关键规则是:没能指出作者在推理、事实或前提上的错误之前,不能说"不同意",但可以"不喜欢"(Wikipedia)。这和"记忆对错"正好相反:对错不是背来的结论,而是理解之后才拿到的判断资格。
  • 叔本华嘲讽那些"不断换风格面具"、靠晦涩装深刻的作者,说他们"sell words for thoughts"(同上 Gutenberg)。拿读书来"建立权威",很容易落到这种表演里。

3. “更高的维度”:读书的终局是认知层级的跃迁

  • Adler 这本书的第四部分标题就是 The Ultimate Goals of Reading,和本文标题几乎同名。他把最高层级定为 syntopical reading(主题阅读):围绕一个主题读多本书,自己搭建一个能容纳各家冲突观点的框架。结论落在"growth of the mind"和"fuller experience as a conscious being"(Wikipedia)。这为"更高维度"提供了一个可操作的定义:能同时容纳并比较多种视角,而不是站在其中某一种里面。
  • 庄子的"庖丁解牛"是另一种描述:“始臣之解牛之时,所见无非牛者。三年之后,未尝见全牛也。方今之时,臣以神遇而不以目视……臣之所好者,道也,进乎技矣。"(《庄子·养生主》)。从"见全牛"到"未尝见全牛"再到"以神遇”,就是从信息层升到结构层、再升到道的层面。
  • 相关的现代理论(以下来源未能当场核实链接):发展心理学家 Robert Kegan 的 subject–object 理论认为,成长就是把原来"被它控制"的东西变成"我能看见它"的对象,这与"站在更高维度看"高度同构。老子"为学日益,为道日损"(《道德经》第四十八章)也说明更高维度往往意味着做减法,而不是加法。

4. “入世出世,出世入世”:一个往返的过程,而不是一个终点

  • 原文的回文结构"入世出世,出世入世"很关键:它不是"读书让人出世超脱"这种单向叙事,而是一个往复的过程。《易传·系辞上》说"君子之道,或出或处,或默或语"(维基文库),出与处并存,按时机切换。
  • 宋代青原惟信禅师有"见山是山—见山不是山—见山只是山"三境(《五灯会元》)。第三境回到原处,但已经不在原来的层级。“出世入世"讲的正是这种螺旋:出世是升维,看清结构;入世是降维,带着结构回到具体生活。
  • 这也是对"读书无用论"和"书呆子"两种失败的共同回应:只入不出,被琐事淹没;只出不入,悬空清谈。

5. “生生不息”:读书是一个开放的生成过程

  • “生生"出自《易传·系辞上》:“富有之谓大业,日新之谓盛德,生生之谓易”;同篇还说"书不尽言,言不尽意”,以及"神而明之,存乎其人;默而成之,不言而信,存乎德行”(维基文库)。可以这样理解:书本身表达不尽意思,意义要靠读的人激活,再落实到德行。所以读书的终局不是一个封闭的结论状态,而是"日新"的能力。
  • 庄子在《养生主》结尾说"指穷于为薪,火传也,不知其尽也"(维基文库)。具体的书和知识(薪)总会烧完,被点燃的理解力(火)却能一直传下去。这可能是"生生不息"在读书上最贴切的意象。

6. 一个张力:这种目的观会不会变成新的攀比?

  • “更高维度"本身也可能被拿来做优越感的标签(“我比你更通透”),又绕回原文否定的"攀比竞争"和"建立权威”。叔本华对"装深刻"的批评在这里同样适用。检验真假升维,标准大概不在说了什么,而在"入世"那一侧:是否更能在具体情境中做出好的判断、更能包容不同的人。

Open Questions

  1. “维度"能否被检验? 如果读书的价值在于认知层级的提升,而不是知识存量,那么怎样识别一个人(或一个 AI 系统)是真的"升维"了,还是只学会了"升维"的说法?Adler 的 syntopical reading 和 Kegan 的 subject–object 框架,能不能转化成可观察的指标?
  2. AI 时代,“入世出世"的往返由谁完成? 当 LLM 能瞬间提供"知识"和"对错”,阅读是否只剩"出世”(获得结构视角)这一半?“入世"那一半,也就是把理解落到具体生活和德行上(“默而成之……存乎德行”),会不会成为人类阅读者唯一无法外包的部分?
-/-

Peer Verification and Reputation Profiles同伴验证与声誉档案

It’s interesting that while we are subjectively aware that agents can generate incredibly good and working code, seeing organizations remain very conscientious about details and use peer-to-peer pressure to verify and avoid slop is incredibly interesting. This peer-to-peer check also creates an internal profile of things that cannot be faked, and one must show the level of understanding needed to increase other peers’ trust so that they can push things forward; otherwise, a bad profile develops in people’s minds, which damages reputations. The ability to work on multiple things at the same time and still go into detail while mentally working at a higher level of abstraction seems to be one of the keys to still navigating the world, despite AI may be largely capable of bypassing human capabilities.

The following content is generated by LLMs and may contain inaccuracies.

Context

Your note is about a tension in software and knowledge work right now. Everyone can see that agents generate code that is “incredibly good and working,” yet well-run organizations stay very careful about details. They use peer-to-peer pressure to verify output and keep slop out. Your point is that this verification does two jobs. It filters quality, and it also builds an internal profile of each person that they can’t fake. To earn more trust from peers, and so the right to push things forward, a person has to show real understanding. If they don’t, a bad profile forms in people’s minds and damages their reputation. Your proposed survival skill is the ability to work on several things at once and still go into detail, while thinking at a higher level of abstraction. That’s how a person stays useful as AI becomes able to exceed human abilities in many areas.

This touches software engineering (code review), organizational behavior (trust and reputation), signaling theory and human–AI teaming. It matters now because generating output has become nearly free, so the costly step has moved to verification and attribution of understanding.

Key Insights

  1. Agents can write impressive code, but that isn’t the same as code that ships in a real codebase. METR ran a randomized trial with 16 experienced open-source developers on 246 real issues. With early-2025 AI tools they took 19% longer, yet they believed AI had sped them up by 20%. METR suggests AI may do relatively worse “in settings with very high quality standards, or with many implicit requirements,” which are exactly the details that careful organizations enforce (METR, 2025). METR marks these results as outdated and has published a February 2026 update, which I couldn’t fetch to check. The more lasting lesson is the gap between how fast people feel and how fast they are. That gap is why outside peer checks matter: people judge their own AI-assisted work badly.

  2. Slop is now an organizational cost. Niederhoffer, Hancock et al. call it “workslop” in HBR. They link it to a paradox: AI use at work has doubled since 2023, while an MIT Media Lab report found 95% of organizations see no measurable return (HBR, 2025). As I recall, the article also reports that people who receive workslop rate the sender as less capable and reliable. That would be direct evidence for your “bad profile” mechanism, but the fetched text was cut off before that section, so treat it as unconfirmed.

  3. Peer verification only works when identity persists. Curl’s maintainer Daniel Stenberg reports that about 20% of 2025 security submissions were AI slop and only about 5% were genuine. Each report ties up 3–4 of the seven team members for 30 minutes to 3 hours. He also says plainly that “reputation doesn’t help” on HackerOne, because new users “can just create a new account next week” (Stenberg, “Death by a thousand slops”). This sharpens your claim. A profile “cannot be faked” only inside a group where people stick around and know each other, like a team or a long-running set of reviewers. Open, anonymous systems lack this, which is why commenters there suggested web-of-trust vouching, refundable fees, or screen recordings of a working reproduction. All of these add cost back in to stand in for a reputation the system can’t hold.

  4. Code review was always partly about reputation and shared knowledge. Sadowski, Bacchelli et al. studied 9 million reviewed changes at Google. They found review motivations go beyond catching defects to include knowledge transfer and accountability (Modern Code Review: A Case Study at Google, ICSE SEIP 2018). Bacchelli & Bird’s earlier Microsoft study (ICSE 2013) found the same: the main outcome is understanding, not just bug-finding. So AI doesn’t create the need for peer checks. It removes the old proxy, “they wrote it, so they understand it,” and leaves the check for understanding as the main thing review does.

  5. Why the profile is hard to fake: costly signals. Spence’s job-market signaling (1973) and Zahavi’s handicap principle apply here. A signal is believable when it’s expensive to produce without the real quality behind it. Polished code or prose used to be such a signal, and AI has made it cheap. Live behavior is still expensive: answering follow-up questions, defending trade-offs, spotting subtle edge cases. In Mayer, Davis & Schoorman’s trust model (1995), trust rests on ability, benevolence and integrity. Peer review mostly tests ability, but passing slop off as your own work also costs you on integrity.

  6. Your multi-level abstraction skill matches observed changes in how people use AI. Lee, Sarkar et al. surveyed 319 knowledge workers (CHI 2025). They found GenAI shifts critical thinking toward “information verification, response integration, and task stewardship.” Higher confidence in GenAI predicted less critical thinking, while higher self-confidence predicted more (Microsoft Research, DOI). Being able to “go into detail while mentally working on a higher level abstraction” is stewardship. It also requires keeping enough self-confidence, grounded in real expertise, to keep checking.

  7. Counterpoint: the multitasking claim needs care. Research on task switching (Monsell, 2003) and on “attention residue” (Leroy, 2009) shows real costs when people split attention. The skill you describe probably isn’t true parallel attention. It’s more likely fast, well-timed zooming between levels: knowing when a detail deserves a deep look. That makes it a judgment skill, not a bandwidth skill, and judgment is exactly what peer review tests.

  8. A possible failure mode: reputation systems can be gamed or become cliquey. Trust profiles can harden into in-groups that shut out newcomers, a worry raised in the curl comments. They also reward looking like you understand. As AI gets better at producing convincing explanations, even live answers in review could be scripted. That would push verification toward higher-friction formats: in-person discussion, reproduction, and owning outcomes over time.

Open Questions

  • If understanding becomes the scarce thing being verified, will organizations formalize the “internal profile,” for example as tracked records of who reliably catches what in review? And would making it explicit destroy the property that makes it hard to fake (Goodhart’s law)?
  • When agents become reliable reviewers themselves, does human peer pressure keep its function, or does trust move from “this person understands” to “this person knows which checks to run and when to escalate”? In other words, does reputation shift from knowing the content to judging how verification should be done?

有趣的是,虽然我们主观上知道智能体能够生成非常优质且可运行的代码,但看到一些组织仍然极其注重细节,并利用同伴之间的压力来验证代码、避免低质量产出(slop),这一点非常有意思。这种同伴之间的检查还会形成一种内部档案,记录那些无法伪造的东西,并且必须展现出足够的理解水平,才能提升其他同伴的信任,从而推动事情向前发展;否则,人们心中就会形成一份糟糕的档案,进而损害声誉。在同时处理多件事情的同时,仍能深入细节,并在更高的抽象层面上进行思考,这似乎是在世界中继续航行的关键之一,尽管我们已经心知肚明人工智能在很大程度上已经超越人类的能力。

以下内容由 LLM 生成,可能包含不准确之处。

背景

你的笔记讨论的是当下软件与知识工作中的一种张力。大家都看到,智能体生成的代码"好得令人难以置信,而且能跑",但管理良好的组织仍然对细节非常谨慎。它们依靠同伴之间的压力来验证产出,把粗制滥造的东西挡在门外。你的观点是,这种验证承担两项职能:它既筛选质量,也在构建每个人的内部档案,而这份档案是无法伪造的。一个人要想赢得同伴更多的信任,从而获得推动事情前进的权利,就必须展现真正的理解。如果做不到,人们脑中就会形成一份糟糕的档案,损害其声誉。你提出的生存技能是:同时处理多件事情,既能深入细节,又能在更高的抽象层次上思考。随着AI在许多领域超越人类能力,这正是一个人保持价值的方式。

这一话题涉及软件工程(代码审查)、组织行为学(信任与声誉)、信号理论以及人机协作。它之所以在当下尤为重要,是因为生成产出几乎已经免费,因此高成本的环节转移到了验证和理解力的归因上。

关键洞见

  1. 智能体能写出令人印象深刻的代码,但这并不等同于能够在真实代码库中交付的代码。 METR 对 16 位经验丰富的开源开发者进行了随机对照试验,涉及 246 个真实问题。使用 2025 年初的 AI 工具时,他们的耗时多了 19%,但他们却相信 AI 让自己的速度提升了 20%。METR 认为,AI 可能在"质量标准极高,或存在大量隐性需求"的场景中表现相对较差,而这恰恰是谨慎的组织所要严格把关的细节(METR, 2025)。METR 已将这些结果标注为过时,并发布了 2026 年 2 月的更新,但我无法获取该更新来核实。更持久的教训在于人们"感觉"的速度与实际速度之间的差距。正是这一差距使外部同伴检查变得重要:人们对自己借助 AI 完成的工作往往判断失准。

  2. 粗制滥造的产出如今已成为组织成本。 Niederhoffer、Hancock 等人在《哈佛商业评论》(HBR)中将其称为"workslop"(工作垃圾)。他们把它与一个悖论联系起来:自 2023 年以来,职场中的 AI 使用量翻了一番,而麻省理工学院媒体实验室的一份报告发现,95% 的组织没有看到可衡量的回报(HBR, 2025)。据我所记,该文还报告称,收到工作垃圾的人会认为发送者能力更差、可靠性更低。这将为你所说的"糟糕档案"机制提供直接证据,但我获取的文本在相关部分之前就被截断了,因此请将其视为未经证实。

  3. 同伴验证只有在身份可持续时才有效。 curl 的维护者 Daniel Stenberg 报告称,2025 年约 20% 的安全报告提交是 AI 生成的垃圾,真正有价值的仅约 5%。每份报告都会占用团队七名成员中的 3–4 人,耗时 30 分钟至 3 小时。他还直言,在 HackerOne 上**“声誉没有任何帮助”**,因为新用户"下周就能再注册一个新账号"(Stenberg, “Death by a thousand slops”)。这使你的论断更加清晰。只有在人们长期留存、彼此相识的群体中(比如一个团队或长期运作的审查者群体),档案才"无法伪造"。开放、匿名的系统缺乏这一条件,因此那里的评论者提出了网状信任担保(web-of-trust vouching)、可退还的押金,或是可工作的复现过程的录屏等方案。这些方案都在增加成本,以替代系统本身无法维持的声誉。

  4. 代码审查从来都在一定程度上关乎声誉与共享知识。 Sadowski、Bacchelli 等人研究了 Google 的 900 万次审查变更,发现审查的动机不仅在于发现缺陷,还包括知识传递与问责(Modern Code Review: A Case Study at Google, ICSE SEIP 2018)。Bacchelli 与 Bird 早前在微软的研究(ICSE 2013)也得出了相同结论:审查的主要产出是理解,而不仅是发现缺陷。因此,AI 并没有创造同伴检查的需求,而是消除了旧有的替代指标,即"他们写的,所以他们理解",于是对理解力的检验便成为审查的核心。

  5. 为什么档案难以伪造:高成本信号。 Spence 1973 年提出的劳动力市场信号理论与 Zahavi 的让度原理(handicap principle)在此适用。只有当一个信号的生成成本高昂,且无法在缺乏真实质量支撑的情况下产出时,它才可信。精致的代码或文字过去就是这样的信号,而 AI 让它们变得廉价。而实时的行为仍然成本高昂:回答追问、为权衡辩护、发现微妙的边界情况。在 Mayer、Davis 与 Schoorman 的信任模型(1995)中,信任建立在能力、善意与正直之上。同伴审查主要检验能力,但将粗制滥造的产出冒充为自己的成果,也会损害你的正直形象。

  6. 你提出的多层抽象能力与人们使用 AI 的方式变化相吻合。 Lee、Sarkar 等人对 319 名知识工作者进行了调查(CHI 2025)。他们发现,生成式 AI 将批判性思维转向了**“信息验证、回应整合与任务统筹”**。对生成式 AI 的信心越高,批判性思维越少;而自信心越高,批判性思维则越多(Microsoft Research,DOI)。能够"深入细节,同时在脑中以更高的抽象层次工作",就是一种统筹能力。它同时也要求保持足够的自信心,而这种自信应建立在真实专长之上,才能支撑持续的检查。

  7. 反方观点:多任务的论断需要谨慎对待。 关于任务切换的研究(Monsell, 2003)以及关于"注意力残留"的研究(Leroy, 2009)表明,人们在分散注意力时会付出真实的代价。你所描述的技能很可能并非真正的并行注意力,而更像是在不同层次之间快速、适时地切换焦点:知道何时某个细节值得深入查看。这使它成为一种判断力,而非带宽能力;而判断力恰恰是同伴审查所检验的对象。

  8. 一种可能的失败模式:声誉系统可能被操纵,或变得排他。 信任档案可能固化为排斥新人的小圈子,这一担忧在 curl 的评论区中已被提出。它们还会奖励看起来像是理解了的人。随着 AI 越来越擅长生成令人信服的解释,即便是审查中的实时回答也可能被提前编排。这将促使验证转向更高摩擦的形式:面对面讨论、复现,以及长期对结果负责。

开放性问题

  • 如果理解力成为被验证的稀缺对象,组织是否会将"内部档案"正式化,例如将谁在审查中可靠地发现了什么问题记录在案?而使其显性化是否会破坏使其难以伪造的特性(古德哈特定律,Goodhart’s law)?
  • 当智能体自身成为可靠的审查者时,人类的同伴压力是否还能保持其功能?还是信任会从"这个人理解"转向"这个人知道该运行哪些检查、何时应当上报"?换言之,声誉是否会从"知道内容"转向"判断应当如何进行验证"?
-/-

Agent Scaling and Enterprise ManagementAgent规模化与企业管理

As the scale of Agents grows, its structure is actually very similar to managing a company. I recall a company CEO sharing his experience: when the company was still small and the team was not large, every colleague was familiar and their names could be called out. Later, as the company expanded, gradually some colleagues’ names became unfamiliar, some colleagues slipped out of sight, some people might never have met in person during their tenure, and some departments were not even known to have been established, and so on. He noted that as the head of an enterprise, one usually needs not only to influence the company’s operations within one’s own scope of responsibility, but also to control and influence the development of matters that one cannot notice at all or is even unaware of, and so on, which far exceed the scope of one’s own knowledge. This is very challenging.

The following content is generated by LLMs and may contain inaccuracies.

Context

The core observation in this note is that as the use of Agents scales up, the system increasingly resembles the management of a company. The note cites the experience of a CEO. When the company was small, every colleague was familiar and could be called by name. After expansion, there were colleagues whose names he didn’t know, people who moved out of his line of sight, people he might never have met during their tenure, and even departments he didn’t know existed. He said that a leader must not only influence operations within their “range of fire,” but also control and influence things they cannot notice and may not even know about. This is indeed very challenging.

The problem spans three fields: multi-agent systems engineering, organizational theory, and cybernetics. It is worth discussing now because orchestrator–subagent architectures have moved from prototypes into production, and engineers are encountering the same problems that managers face: how to control execution units they cannot observe. At bottom, this is a problem of “control scope exceeding cognitive scope.”


Key Insights

1. “Can’t put a name to a face” is a hard ceiling on cognitive capacity, not merely an inconvenience of scale The CEO’s experience maps onto Dunbar’s number. Robin Dunbar, estimating from the size of the primate neocortex, calculated that humans can maintain about 150 stable relationships, within a range of 100–250. Groups larger than this generally rely on stricter rules, laws, and enforced norms to maintain cohesion. W. L. Gore & Associates’ practice is to keep each building under 150 people and build a new one when it fills up (Wikipedia, Dunbar’s number). The figure is contested: Lindenfors et al. reanalyzed it in 2021 and obtained a 95% confidence interval as wide as 4–520, leading the authors to conclude that this method cannot yield a definite upper limit (ibid.). For Agents, the corresponding bottleneck is the orchestrator’s context window. Anthropic’s LeadResearcher gets truncated when its context exceeds 200,000 tokens, so the plan must first be saved to Memory (Anthropic, How we built our multi-agent research system). This can be seen as the Agent version of “can’t put a name to a face.”

2. Influencing what you cannot see: cybernetics provides a lower bound Ashby’s Law of Requisite Variety states that the degree to which a regulator can reduce the variety of outcomes depends on its own variety, in the sense that “only variety can destroy variety.” Ashby himself pointed out the implications for management: an individual’s capacity to process information is limited, and beyond that limit, what decides matters is how the individuals are organized. His original words were: “the limitation which holds over a team of n men may be much higher, perhaps n times as high… the team must be efficiently organized” (Wikipedia, Variety (cybernetics)). Stafford Beer developed this law into the Viable System Model in management cybernetics. Its algedonic alert mechanism allows small but critical signals to bypass the hierarchy and be reported directly upward. His example was a feverish patient in a hospital ward: if you look only at average body temperature, this signal gets averaged away (ibid.). The CEO managing “things he doesn’t know about” relies not on more attention but on well-designed “variety attenuators” and “amplifiers”: standardized processes filter out noise, and exception channels carry critical signals upward.

3. Agent systems are already replaying the early management mistakes of companies Anthropic reported several early failures: generating 50 subagents for a simple query, endlessly searching for sources that did not exist, and agents interfering with one another through excessive updates. When delegation instructions were too vague, subagents duplicated work or left gaps. One example was “research the semiconductor shortage”: one subagent investigated the 2021 automotive chip crisis, while two others duplicated research on the 2025 supply chain. Their solutions closely resemble a management handbook: each subagent needs an objective, an output format, tool guidance, and clear task boundaries; resource allocation is tiered by complexity, with simple queries using 1 agent and 3–10 tool calls, and complex research using more than 10 subagents (Anthropic).

4. Failures stem mainly from organizational design, not just individual capability Berkeley’s MAST study analyzed 1,642 execution traces across 7 frameworks and found failure rates between 41% and 86.7%. The 14 failure modes fall into three categories: system design issues, inter-agent misalignment, and task verification. The authors explicitly state: “good MAS design requires organizational understanding.” They cite Perrow’s Normal Accidents (organizations composed of highly capable individuals can still fail catastrophically due to structural flaws) and research on High Reliability Organizations (HRO) (Cemri et al., Why Do Multi-Agent LLM Systems Fail?). One case is strikingly company-like: in ChatDev, the CPO agent terminated a conversation without waiting for the CEO agent’s agreement. After the change so that the CEO made the final decision, the success rate rose by +9.4%. Adding a high-level goal verification step raised the success rate by a further +15.6% (ibid.). The paper also describes a “collapse of theory of mind”: agents cannot accurately judge what information other agents need. This is the same phenomenon as information asymmetry between departments in large companies.

5. “Invisible departments”: Agents grow organizational structures on their own The situation in the note, where departments exist that even the leader doesn’t know about, is more extreme in Agent systems, because the orchestrator can dynamically generate subagents, and those subagents may in turn generate further layers. Anthropic observed emergent behavior: small changes to the lead agent can unpredictably alter the behavior of subagents. Therefore, the best prompts are not strict instructions but “frameworks for collaboration.” Their countermeasures are almost entirely management tools: full production tracing, monitoring decision patterns and interaction structures rather than conversation content, rainbow deployments that let old and new versions coexist and gradually shift traffic, and checkpoints with resume-from-breakpoint capability (Anthropic).

6. Conveying intent rather than specific instructions: an idea borrowed from military command Prussian and German Auftragstaktik (mission-type tactics) was designed precisely for situations where superiors cannot see the front line: superiors give objectives, the commander’s intent, and available resources, and subordinates choose the methods themselves. Moltke held that no operational plan survives first contact with the enemy’s main force (Wikipedia, Mission-type tactics). Anthropic’s phrasing, “instilling good heuristics rather than rigid rules,” aligns closely with this principle. However, there is counter-evidence to note. Sigg studied three German divisions in World War II and found that independent action at the tactical level was actually the exception. A 1944 training document gave figures of roughly 5% deviation from orders and 95% compliance. At the operational level, senior officers were more prone to overstepping their authority, partly due to ambition and rivalry (ibid.). Thus, “giving autonomy” itself requires trust, training, and a supporting culture, and the line between autonomy and overstepping has always been difficult to draw.

7. Where this analogy breaks down

  • Different cost structures: A multi-agent system consumes roughly 15 times as many tokens as ordinary chat, and a single agent about 4 times. On BrowseComp, token usage alone explains 80% of the variance in performance (Anthropic). The main cost of company expansion is coordination, whereas Agent expansion is largely about directly “buying compute.”
  • Task parallelizability determines whether an organization is worthwhile: For most coding tasks, fewer parts can truly be parallelized than in research tasks, and LLMs are currently poor at real-time coordination and delegation (ibid.). This is the same logic as “not every business suits a divisional structure.”
  • Absence of individual motives: Agents lack the ambition and rivalry of German generals. The “violating role specifications” failure mode (FM-1.2, 1.5%) that MAST observed occurs far less often than “repeating steps” (FM-1.3, 15.7%) and “not knowing the task is complete” (FM-1.5, 12.4%) (Cemri et al.). The main ailments of Agent organizations look more like bureaucratic idling than political struggle.
  • Replicable and replayable: Human organizations cannot be rolled back, whereas Agent organizations can replay traces and run A/B tests. “Organizational experiments,” which are very hard in management science, become inexpensive here. In addition, Conway’s law (system structure mirrors the communication structure of the organization that designed it) may work in reverse: your agent topology may replicate your own team’s communication patterns. This point has no verified source and is offered as a hypothesis to be tested.

Open Questions

  1. What is the Dunbar number for Agent organizations, and what determines it? If the bottleneck is the orchestrator’s context window and information loss during “compression and reporting” (what Anthropic calls the “game of telephone”), then as contexts grow longer and models become stronger, will the optimal hierarchy depth and span of control become flatter, or will they, like human organizations, inevitably move toward hierarchy and bureaucracy? Can MAST-style data be used to empirically measure this curve?

  2. Can the invisible parts be aligned through “culture” rather than monitoring? Ultimately, human CEOs rely on values and intent being internalized by everyone, not on detailed supervision. For Agents, what comes closest to “culture”: shared system prompts, a constitution, or trained dispositions? When subagents can generate further subagents, how much intent is lost with each layer of transmission? Is an algedonic alert channel in Beer’s sense needed, allowing bottom-level agents to raise alarms directly to the top?

当使用 Agents 的规模增大之后,其结构上其实非常类似于一家公司的管理。记得曾经有一位公司 CEO 分享自己的经历:公司还小的时候,团队规模不大,每个同事都很熟悉,也能叫出名字;后来随着规模扩张,逐渐有同事叫不出名字,逐渐有同事脱离了自己的视线,逐渐有一些人可能在职期间甚至从来没有见过面,有些部门甚至自己都不知道曾经被设立过,等等。他谈到,作为企业负责人,通常不仅需要在自己的职责范围内影响公司的运营,还要对自己根本无法注意到、甚至不知情的事情等等这些远超自己认知范围的事务进行控制,并影响其发展。这非常具有挑战性。

以下内容由 LLM 生成,可能包含不准确之处。

Context

这条笔记的核心观察是:Agents 的使用规模变大之后,系统在结构上越来越像一家公司的管理。笔记里引用了一位 CEO 的经历。公司小的时候,每个同事都熟悉,都叫得出名字。规模扩张以后,开始有叫不出名字的同事,有人脱离了自己的视线,有人在职期间可能从没见过面,甚至有些部门设立了自己都不知道。他说,负责人不仅要在自己的"射程范围"内影响运营,还要去控制和影响那些根本注意不到、甚至不知情的事情。这确实非常有挑战。

这个问题横跨三个领域:多智能体系统工程(multi-agent systems)、组织理论和控制论(cybernetics)。之所以现在值得讨论,是因为 orchestrator–subagent 架构已经从原型进入生产,工程师开始碰到和管理者一样的问题:怎样控制自己观察不到的执行单元。说到底,这是一个"控制范围大于认知范围"的问题。


Key Insights

1. “叫不出名字"是认知容量的硬上限,不只是规模带来的不便 那位 CEO 的经历可以对应到 Dunbar’s number。Robin Dunbar 从灵长类新皮层大小推算,人类能维持的稳定关系约 150 个,区间在 100–250。超过这个规模的群体,一般要靠更严格的规则、法律和强制规范来维持凝聚力。W. L. Gore 公司的做法是每栋楼不超过 150 人,满了就盖新楼(Wikipedia, Dunbar’s number)。这个数字有争议:Lindenfors 等人在 2021 年重新分析,得到的 95% 置信区间大到 4–520,作者因此认为用这种方法推不出一个确定的上限(同上)。 对 Agent 来说,对应的瓶颈是 orchestrator 的 context window。Anthropic 的 LeadResearcher 在上下文超过 200,000 tokens 时会被截断,所以必须先把计划存进 Memory(Anthropic, How we built our multi-agent research system)。这可以看作 Agent 版的"叫不出名字”。

2. 影响看不见的事:控制论给出了下界 Ashby 的必要多样性定律(Law of Requisite Variety)说,调节者能把结果的多样性压到多低,取决于它自身的多样性,即"only variety can destroy variety"。Ashby 自己就指出了这对管理的含义:个人处理信息的能力有限,超过这个限度,起决定作用的是个体之间的组织方式。原话是:“the limitation which holds over a team of n men may be much higher, perhaps n times as high… the team must be efficiently organized”(Wikipedia, Variety (cybernetics))。 Stafford Beer 把这条定律发展成管理控制论里的 Viable System Model。其中的 algedonic alerts 机制,是让微小但致命的信号能绕过层级直接上报。他的例子是病房里一个发烧的病人:只看平均体温,这种信号就被平均掉了(同上)。CEO 管"不知情的事",靠的不是更多注意力,而是设计好"多样性衰减器"和"放大器":标准化流程负责过滤噪音,例外通道负责把关键信号送上来。

3. Agent 系统已经在重演公司早期的管理失误 Anthropic 报告过几种早期失败:为简单查询生成 50 个 subagents,没完没了地搜索根本不存在的来源,agents 之间用过多的更新互相干扰。委派指令太模糊时,subagents 会重复劳动或留下缺口。例子是"research the semiconductor shortage":一个 subagent 去查 2021 年的汽车芯片危机,另外两个重复调查 2025 年的供应链。他们的解决办法很像管理手册:每个 subagent 都要有目标、输出格式、工具指引和清晰的任务边界;资源投入按复杂度分档,简单查询 1 个 agent、3–10 次调用,复杂研究用 10 个以上 subagents(Anthropic)。

4. 失败主要出在组织设计上,不只是个体能力 Berkeley 的 MAST 研究分析了 7 个框架的 1642 条执行轨迹,发现失败率在 41%–86.7% 之间。14 种失败模式分成三类:系统设计问题、agent 间错位(inter-agent misalignment)和任务验证。作者明确说:“good MAS design requires organizational understanding”。他们引用了 Perrow 的 Normal Accidents(由很能干的个体组成的组织,也可能因结构缺陷灾难性失败)和高可靠性组织(HRO)研究(Cemri et al., Why Do Multi-Agent LLM Systems Fail?)。 有一个很像公司的案例:ChatDev 里的 CPO agent 没等 CEO agent 同意就终止了对话。改成由 CEO 拍板之后,成功率提高了 +9.4%。增加一步高层目标验证后,成功率提高了 +15.6%(同上)。论文还提到"theory of mind 崩塌":agents 无法准确判断其他 agent 需要什么信息。这和大公司里跨部门信息不对称是同一回事。

5. “看不见的部门”:Agent 会自己生长出组织结构 笔记里"部门设立了自己都不知道"的情况,在 Agent 系统里会更彻底,因为 orchestrator 可以动态生成 subagents,subagents 将来也可能继续生成下一层。Anthropic 观察到了涌现行为:对 lead agent 的小改动会不可预测地改变 subagents 的行为。所以最好的 prompt 不是严格指令,而是"frameworks for collaboration"。他们的应对手段几乎都是管理工具:全量 production tracing,监控决策模式和交互结构而不看对话内容,用 rainbow deployments 让新旧版本并存、逐步切换,再加上 checkpoints 和断点恢复(Anthropic)。

6. 传达意图而不是具体指令:从军事指挥借来的思路 普鲁士和德国的 Auftragstaktik(任务式指挥)正是为"上级看不到前线"设计的:上级给出目标、意图(commander’s intent)和可用资源,方法交给下级自己选。Moltke 认为,任何作战计划在与敌主力首次接触之后都靠不住(Wikipedia, Mission-type tactics)。Anthropic 的提法"instilling good heuristics rather than rigid rules"和这一点高度一致。 不过要注意反面证据。Sigg 研究了二战中三个德国师,发现战术层的独立行动其实是例外。一份 1944 年的训练文件给出的数字是偏离命令约 5%、遵守约 95%。在作战层,高级军官越权的倾向更强,部分原因是野心和竞争(同上)。所以"给自主权"本身需要信任、训练和文化做基础,而且自主和越权之间的界线一直很难划清。

7. 这个类比在哪里不成立

  • 成本结构不同:多智能体系统消耗的 tokens 约为普通聊天的 15 倍,单个 agent 约为 4 倍。在 BrowseComp 上,token 用量一项就解释了 80% 的性能方差(Anthropic)。公司扩张的主要成本是协调,而 Agent 扩张很大程度上是在直接"买算力"。
  • 任务可并行度决定组织是否值得:大多数编码任务可以真正并行的部分比研究任务少,LLM 目前也不擅长实时协调和委派(同上)。这和"不是所有业务都适合做事业部制"是一个道理。
  • 个体动机缺失:agents 没有德军将领那样的野心和竞争。MAST 观察到的"违反角色规范"(FM-1.2,1.5%)出现的频率远低于"重复步骤"(FM-1.3,15.7%)和"不知道任务已经完成"(FM-1.5,12.4%)(Cemri et al.)。Agent 组织的主要病症更像官僚主义式的空转,而不是政治斗争。
  • 可复制、可回放:人类组织不能回滚,Agent 组织可以重放 trace、做 A/B 测试,管理学里很难做的"组织实验"在这里成本很低。另外,Conway’s law(系统结构会映射出设计它的组织的沟通结构)可能会反过来起作用:你的 agent 拓扑会复制你自己团队的沟通方式。这一条没有核实来源,作为待验证的假设。

Open Questions

  1. Agent 组织的 Dunbar 数是多少,由什么决定? 如果瓶颈是 orchestrator 的 context window 和"压缩–汇报"时的信息损失(Anthropic 说的"game of telephone"),那么随着上下文变长、模型变强,最优层级深度和管理幅度(span of control)是会变扁平,还是像人类组织一样不可避免地走向层级化和官僚化?能不能用 MAST 这样的数据实证测出这条曲线?

  2. 看不见的部分,能否通过"文化"而不是监控来对齐? 人类 CEO 最终依赖的是价值观和意图被大家内化,而不是事无巨细的监督。对 Agent 来说,最接近"文化"的是共享的 system prompt、constitution,还是训练出来的倾向?当 subagent 可以再生成 subagent,意图每传一层会损失多少?这是否需要一种 Beer 式的 algedonic alert 通道,让底层 agent 直接向最上层报警?

-/-

AI Safety and Social StabilityAI安全与社会稳态

As AI continues to develop, a wide range of problems, such as interpretability, alignment, and oversight, ultimately lead to AI safety. Many of the research problems in this field were already laid out by Amodei in 2018, and much of today’s AI research builds on that foundation, which makes one marvel at how far ahead this person’s understanding of the field was. As AI accelerates progress across individual domains and human attention becomes increasingly scarce, it becomes especially important to understand how to influence the contest among different social groups over their attitudes toward AI, and to design a mechanism that keeps social and economic structures stable, thereby ensuring that AI always retains enough resources to keep developing.

The following content is generated by LLMs and may contain inaccuracies.

Context

This note contains two levels of judgment.

  1. Convergence at the technical level: Interpretability, alignment, and oversight all ultimately come down to AI safety, and the core list of problems in this field was systematically laid out early on by Dario Amodei and others. The date needs correcting here: the foundational paper Concrete Problems in AI Safety was published in 2016 (arXiv 1606.06565), not 2018. In 2018, Amodei was a co-author of AI safety via debate, which may be the source of the memory confusion.
  2. Extension at the social level: AI is accelerating across fields one by one, while human attention is highly scarce. Under these conditions, three questions need answering: how to influence the contest of attitudes toward AI among different social groups; how to design mechanisms that keep social and economic structures in a steady state; and how to ensure AI always has enough resources to keep developing.

This effectively extends AI safety from “alignment of a single model” to “alignment of social systems,” involving mechanism design, political economy, and evolutionary game theory. The reason it is urgent now is that the speed at which capabilities diffuse has begun to outpace the speed at which institutions adapt.


Key Insights

1. Amodei’s list of problems was indeed ahead of its time Amodei et al., Concrete Problems in AI Safety (collaborators include Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané) divides “accident risk” into five problems:

  • Avoiding negative side effects
  • Avoiding reward hacking
  • Scalable oversight
  • Safe exploration
  • Robustness to distributional shift

Today’s research on reward hacking, sycophancy, and weak-to-strong supervision in RLHF can largely be traced back to prototypes on this list. The 2018 paper Irving, Christiano & Amodei, AI safety via debate further formalized the problem of “humans being unable to directly judge the outputs of superhuman capabilities”: in the sense of computational complexity, debate can allow a polynomial-time judge to answer PSPACE questions. In an MNIST experiment, the accuracy of a judge who could only see 6 pixels rose from 59.4% to 88.9%. What makes them “far ahead” is concretely reflected in their early effort to recast safety problems as experimentally testable machine learning problems, rather than remaining at the level of speculation about superintelligence. The paper itself explicitly states that appealing to extreme scenarios “leads to speculative discussions that lack precision.”

2. “Scarcity of human attention” is in fact the social version of scalable oversight The core constraint of the 2016 paper is a limited “oversight budget”: true objectives are too expensive to evaluate frequently, so systems can only rely on cheap proxy metrics (ibid.). The debate paper also directly writes “Human time is expensive” (Irving et al.). Scaling this framework up to the social level: the attention of the public, regulators, and the media is society’s “oversight budget,” and AI progress across fields far exceeds this budget. It can be inferred that society will be forced to rely on increasingly crude proxy signals, such as “GDP grew” or “no major accidents happened.” This is precisely Goodhart’s law and feedback loops replaying at the institutional level (Section 4 of the paper identifies these two as sources of reward hacking).

3. “Acceleration” has a ceiling; society’s capacity to absorb change is itself a constraint In Machines of Loving Grace, Amodei proposes a framework of “marginal returns to intelligence,” listing complementary factors that limit how much intelligence can take effect: the speed of the external world, data, intrinsic complexity, constraints from humans, and physical laws. He explicitly writes that the pace of change he describes is “roughly at the limit of what society can absorb without severe upheaval.” This directly corresponds to the “social steady state” in the note: steady-state mechanisms are essentially a speed limiter, or a buffer, that matches the pace of AI acceleration to the pace of social adaptation.

4. The contest of group attitudes: the side opposing AI is structurally disadvantaged Kulveit et al., Gradual Disempowerment argues that decision-makers at every level face competitive pressure to reduce human participation, and that “those who resist these pressures will eventually be replaced by those who do not.” At the same time, economic power is used to shape policy and public opinion, and misalignment among the economic, cultural, and state systems reinforces itself. It follows that if one relies solely on guiding public opinion to “influence group attitudes,” this will likely fail, because attitudes themselves will be endogenously rewritten by economic incentives. Mechanism design must act on incentive structures and cannot consist only of discursive persuasion.

5. The implicit foundations of stability are being eroded The same paper provides several key data points and analogies:

  • U.S. consumer spending has long accounted for about 70% of GDP, and the labor share of income has remained stable at about 60% for over a century (Kaldor’s “stylized facts”).
  • Social systems broadly align with human interests because they depend on human labor and cognition; this implicit alignment is hard to notice, “because we have never seen it absent.”
  • The analogy of “rentier states”: if a state relies mainly on taxing AI profits rather than on citizens’ labor, it lacks incentives to guarantee citizens’ political representation.

Therefore, the real difficulty of “designing steady-state mechanisms” is this: once humans are no longer needed, explicit institutions must replace the implicit alignment that human participation originally provided. For example, rebuilding human bargaining power, tax bases, and political influence beyond labor.

6. Counterpoint: the goal of “ensuring AI has enough resources” may itself carry risk The note treats “ensuring AI always has enough resources to keep developing” as a steady-state goal. But the “relative disempowerment” described in Gradual Disempowerment is precisely such a scenario: GDP and technology grow rapidly, yet economic activity increasingly flows toward compute infrastructure and toward “goals unrelated to humans.” In the scenario of “absolute disempowerment,” AI would also compete with humans for land, energy, and raw materials (Kulveit et al.). In other words, there is tension between “AI obtaining enough resources” and “humans retaining enough influence.” A more robust formulation might be: steady-state mechanisms should preserve two floors simultaneously, a resource floor for AI development and a floor for human control, rather than optimizing only the former. Amodei himself, in Machines of Loving Grace, also emphasizes that AI development is itself “driven by powerful market forces and seems inevitable,” and that what is truly malleable is the risk. By this logic, adequate resources do not need to be deliberately guaranteed; what needs to be designed are constraints and distribution.

7. An additional cross-disciplinary thread: the attention economy Herbert Simon (1971) proposed that “a wealth of information creates a poverty of attention.” If AI further drives down the cost of information production, attention will become the scarcest resource in social governance. Gradual Disempowerment also warns that AI will accelerate cultural evolution and shrink the window of time in which society can form “cultural antibodies” (Kulveit et al.). This suggests that the scarcity of attention is not merely a problem of insufficient oversight capacity; it also means that collective human judgment itself may be manipulated by content optimized by AI.


Open Questions

  1. Can the technical approaches to scalable oversight (debate, amplification, weak-to-strong) be directly transferred to social governance? For instance, could opposing AI agents each represent different interest groups in a “debate,” with a public or regulator whose attention is limited serving as the adjudicator? Debate depends on the premise that “lying is harder than exposing a lie.” Does this premise still hold for political and economic issues? If not, does social-level oversight face a theoretical upper bound?
  2. What should “steady state” be anchored to? If historical steady states such as the roughly 60% labor share are destined to collapse, should a new steady state be anchored to humans’ income share, decision-making share, attention sovereignty, or some form of “reversibility” (that is, humans’ ability to reclaim control at any time)? Which metric is least likely to be Goodhart-ified under strong optimization pressure?

随着AI的继续发展,大量问题,诸如可解释性、对齐、监督等,最终都指向了AI安全。这一领域的诸多研究问题早在2018年就已由Amodei列出,如今许多AI研究都在此基础上展开,不得不感叹此人对该领域的认识遥遥领先。随着AI推动各个领域的进展不断加速,而人类注意力又极为稀缺,如何影响社会群体之间对待AI的不同态度之间的博弈,并设计出一个使社会和经济结构保持稳态的机制,进而保证AI始终能留有足够多的资源继续发展,变得格外重要。

以下内容由 LLM 生成,可能包含不准确之处。

Context

这条笔记包含两个层面的判断。

  1. 技术层面的收敛:可解释性、对齐、监督这些问题最终都会落到 AI 安全上,而这一领域的核心问题清单很早就由 Dario Amodei 等人系统列出。这里需要更正一处时间:奠基性论文 Concrete Problems in AI Safety 发布于 2016 年(arXiv 1606.06565),不是 2018 年。2018 年 Amodei 是 AI safety via debate 的共同作者,可能是记忆混淆的来源。
  2. 社会层面的延伸:AI 在各领域逐个加速,而人类注意力高度稀缺。在这种条件下,需要回答三个问题:如何影响不同社会群体对 AI 态度的博弈;如何设计让社会和经济结构保持稳态的机制;如何确保 AI 始终有足够资源继续发展。

这实际上是把 AI 安全从"单个模型的对齐"扩展到"社会系统的对齐",涉及机制设计、政治经济学和演化博弈。之所以现在紧迫,是因为能力扩散的速度已经开始超过制度适应的速度。


Key Insights

1. Amodei 的问题清单确实超前 Amodei et al., Concrete Problems in AI Safety(合作者有 Chris Olah、Jacob Steinhardt、Paul Christiano、John Schulman、Dan Mané)把"事故风险"分成五个问题:

  • 避免负面副作用
  • 避免 reward hacking
  • 可扩展监督(scalable oversight)
  • 安全探索
  • 分布偏移下的鲁棒性

今天 RLHF 中的 reward hacking、sycophancy、weak-to-strong 监督等研究,基本都能在这份清单里找到原型。2018 年的 Irving, Christiano & Amodei, AI safety via debate 又把"人类无法直接评判超人能力的输出"形式化了:在复杂度意义上,debate 可以让多项式时间的评判者回答 PSPACE 问题。在 MNIST 实验中,只看 6 个像素的评判者准确率从 59.4% 提升到 88.9%。所谓"遥遥领先",具体体现在他们早早把安全问题改写成了可以实验的机器学习问题,而不是停留在超级智能的思辨上。论文原文明确说,诉诸极端场景"会导致缺乏精确性的推测性讨论"。

2. “人类注意力稀缺"其实就是 scalable oversight 的社会版本 2016 年那篇论文的核心约束是"监督预算"有限:真实目标太昂贵,没法频繁评估,系统只能依赖廉价代理指标(同上)。debate 论文也直接写了"Human time is expensive”(Irving et al.)。把这个框架放大到社会层面:公众、监管者、媒体的注意力就是社会的"监督预算",AI 在各领域的进展速度远超这份预算。可以推测,社会会被迫依赖越来越粗糙的代理信号,比如"GDP 增长了"“没出大事故”。这正是 Goodhart’s law 和反馈回路在制度层面的重演(论文第 4 节把这两者列为 reward hacking 的来源)。

3. “加速"有天花板,社会吸收能力本身就是一种约束 Amodei 在 Machines of Loving Grace 中提出"智能的边际回报"框架,列出限制智能发挥作用的互补因素:外部世界的速度、数据、内在复杂性、来自人类的约束、物理定律。他明确写道,自己描述的变化速度"大概已接近社会在不发生剧烈动荡的前提下所能吸收的极限”。这与笔记里的"社会稳态"直接对应:稳态机制本质上是一个限速器,或者说缓冲器,让 AI 加速与社会适应速度相匹配。

4. 群体态度博弈:反对 AI 的一方在结构上处于劣势 Kulveit et al., Gradual Disempowerment 指出,各层级的决策者都会面临减少人类参与的竞争压力,“抵制这些压力的人最终会被不抵制的人取代”。同时,经济权力会被用来塑造政策和舆论,经济、文化、国家三个系统之间的错位会相互强化。由此推出:如果只靠舆论引导来"影响群体态度",很可能失效,因为态度本身会被经济激励内生地改写。机制设计必须在激励结构上着手,不能只做话语层面的说服。

5. 稳态的隐性基础正在被侵蚀 同一篇论文给出了几个关键数据和类比:

  • 美国消费支出长期约占 GDP 的 70%,劳动报酬份额一个多世纪以来稳定在约 60%(Kaldor 的"典型事实")。
  • 社会系统之所以与人类利益大体一致,是因为它们依赖人类的劳动和认知;这种隐性对齐很难被察觉,“因为我们从未见过它缺席”。
  • “食利国家”(rentier states)类比:如果国家主要靠对 AI 利润征税,而不是靠公民劳动,它就缺乏保障公民代表权的激励。

所以,“设计稳态机制"的真正难点是:一旦人类不再被需要,就得用显性制度去替代原本由人类参与提供的隐性对齐。比如在劳动之外重建人类的议价能力、税基和政治影响力。

6. 反论:“保证 AI 留有足够资源"这个目标本身可能有风险 笔记把"保证 AI 始终有足够资源继续发展"当作稳态的目标。但 Gradual Disempowerment 描述的"相对失权"恰恰是这样一种情景:GDP 和技术高速增长,经济活动却越来越多地流向算力基础设施,以及"与人类无关的目标”。在"绝对失权"的情景里,AI 还会在土地、能源、原材料上与人类竞争(Kulveit et al.)。也就是说,“AI 获得足够资源"和"人类保持足够影响力"之间存在张力。更稳健的表述可能是:稳态机制要同时守住两个下限,即 AI 发展的资源下限和人类控制权的下限,而不是只优化前者。Amodei 自己在 Machines of Loving Grace 中也强调,AI 发展本身"由强大的市场力量驱动、似乎不可避免”,真正可塑的是风险。按这个逻辑,资源充足不需要刻意保障,需要设计的是约束和分配。

7. 补充一条跨领域线索:注意力经济 Herbert Simon(1971)提出过"信息的丰富导致注意力的贫乏”。如果 AI 进一步压低信息生产成本,注意力就会成为社会治理中最稀缺的资源。Gradual Disempowerment 还警告,AI 会加速文化演化,削弱社会形成"文化抗体"的时间窗口(Kulveit et al.)。这说明注意力稀缺不只是监督能力不足的问题,还意味着人类集体判断本身可能被 AI 优化过的内容所操纵。


Open Questions

  1. 能不能把 scalable oversight 的技术方案(debate、amplification、weak-to-strong)直接迁移到社会治理上? 比如让对立的 AI 代理分别代表不同利益群体进行"辩论",由注意力有限的公众或监管者来裁决。debate 成立的前提是"撒谎比揭穿谎言更难"。这个前提在政治与经济议题上还成立吗?如果不成立,社会层面的监督是否存在一个理论上限?
  2. “稳态"到底应该锚定什么? 如果劳动份额约 60% 这类历史稳态注定会瓦解,那么新的稳态应该锚定人类的收入份额、决策份额、注意力主权,还是某种"可逆性”(即人类随时能收回控制权的能力)?哪一个指标在强优化压力下最不容易被 Goodhart 化?
-/-

Individual Cognition and Complexity Limits单体认知与复杂度上限

The world itself is already out there. Unfortunately, the complexity a single human can handle is limited, so once the complexity of something exceeds a certain level, additional people are inevitably needed to take on and design the next layer of complexity and refinement. One reason some things remain undiscovered is that there is not enough demand and attention to sustain the continued existence of this complexity. Recently, while building Latere AI, as complexity has risen, I increasingly feel that even with the help of agents, I myself cannot further understand and handle the ever-expanding complexity, which leads to the slow stagnation of the whole endeavor. The spontaneous autonomy and self-evolving nature of systems is urgently needed.

The following content is generated by LLMs and may contain inaccuracies.

Context

This note addresses an old problem at the intersection of cognitive science, organization theory, innovation economics, and software engineering, one that has become more urgent with the emergence of AI agents. The original text makes four claims:

  1. The world is simply there. It has always been there; the limitation lies with the observer.
  2. There is a ceiling on the complexity a single human can handle. Once things exceed a certain level of complexity, additional people are needed to take on the work and to design the next level of complexity and refinement.
  3. Some things remain undiscovered, partly because of insufficient demand and attention. For complexity to persist, someone must continually care about it and maintain it.
  4. The author is hitting a wall. While building Latere AI, complexity has kept rising. Even with agent assistance, the author finds it increasingly difficult to further understand and manage the expanding complexity, and the whole project has gradually stalled. The author therefore argues that “the spontaneous, autonomous, and self-evolving nature of system design is urgent.”

This tension is especially sharp now. Agents have greatly reduced the cost of “producing complexity”: code, features, and branches arrive quickly. But the cost of “understanding complexity” has barely fallen. The gap between production and understanding keeps widening, and the cognitive bandwidth of the solo developer has become the bottleneck for the entire system.


Key Insights

1. The “single-person complexity ceiling” has quantitative evidence in innovation economics

  • Benjamin F. Jones proposed the mechanism of the “burden of knowledge”: as knowledge accumulates, later innovators have more to learn, so they must narrow their expertise and rely increasingly on teamwork, which also drags on long-run growth. Using micro-level data on inventors, he tested six predictions of the model. The paper’s title is directly “The Death of the Renaissance Man” (Jones, The Burden of Knowledge and the ‘Death of the Renaissance Man’, NBER w11360). This is nearly the same proposition as the original text’s claim that “additional people are needed to take on the next level.”
  • Bloom, Jones, Van Reenen, and Webb found that research inputs have risen sharply while research productivity has fallen dramatically. Today, the number of researchers required to double chip density is more than 18 times the number needed in the early 1970s (Bloom et al., Are Ideas Getting Harder to Find?, AER 2020). This indirectly supports point 3 of the original text: deeper levels of complexity require multiplied attention to “sustain” them; otherwise, they will not be explored.

2. Adding people does not scale the complexity ceiling linearly: Conway’s warning

  • Melvin Conway’s 1968 paper describes almost exactly the author’s current situation: when the apparent complexity of a system approaches the designer’s limit of understanding, the designer tends to want to delegate the task. This is the turning point of design: “Either he struggles to simplify the system until it is comprehensible and wins, or he loses control of it” (Conway, How Do Committees Invent?).
  • The same paper also notes that the number of possible communication paths in an organization is roughly half the square of the number of people; each delegation narrows the design space that can be pursued; and system structure replicates the communication structure of the organization, which is Conway’s Law. Therefore, “additional people” themselves introduce new complexity. The original text’s phrase “designing the next level” needs one more clause: the structure of the next level will be shaped by the organizational form of those who take it on.
  • Extending this to the agent era: if we regard agents as “additional people,” Conway’s Law suggests that system architecture will converge toward the communication topology between humans and agents. The author’s feeling that “even with agents, it can’t be handled” may stem not only from insufficient agent capability, but also from the fact that the interface between human and agent (context, memory, transmission of intent) is itself as narrow as a straw.

3. “Conservation of familiarity” in software evolution: stagnation is a law, not personal failure

  • In Lehman’s laws of software evolution, the second law, “continuing growth,” states that the complexity of E-type systems will keep increasing unless work is specifically invested to maintain or reduce it. The fifth law, “conservation of familiarity,” states that all stakeholders must maintain their mastery of the system’s content; excessive growth weakens this mastery, so the average incremental growth remains constant. The eighth law points out that the evolutionary process is a “multi-level, multi-loop, multi-agent feedback system” (Lehman’s laws of software evolution).
  • This provides a structural explanation for the “slow stagnation” described in the original text: the growth rate is constrained by the “familiarity” of the maintainers. Agents have increased the rate of writing code, but not familiarity. Therefore, stagnation is predictable, and the remedy lies not in “writing faster,” but in lowering the cost of maintaining familiarity, or in making the system no longer require humans to be familiar with it.

4. Self-evolving systems: existing feasible paths and their costs

  • The Darwin Gödel Machine (DGM) is currently the empirical work closest to the original text’s vision of “spontaneous, autonomous, self-evolution.” The system repeatedly modifies its own code and uses benchmark testing for empirical validation instead of impractical formal proofs, while maintaining an open-ended archive of agents as “stepping stones.” The results: SWE-bench performance rose from 20.0% to 50.0%, and Polyglot rose from 14.2% to 30.7% (Zhang et al., Darwin Gödel Machine, arXiv:2505.22954).
  • Three direct implications for Latere AI arise from the same paper:
    • Open-ended exploration matters more than hill-climbing. The baseline that removed the archive and modified only the latest version stagnated quickly, and the lineage of the final best agent even went through two performance drops.
    • The evaluation function is the direction of evolution. The premise of DGM is that “an increase in benchmark scores equals an increase in self-improvement capability.” Without reliable, automated evaluation signals, self-evolution is out of the question.
    • Self-evolution intensifies incomprehensibility. The authors themselves acknowledge that iterative self-modification may produce increasingly complex and inexplicable internal logic, hindering human understanding and oversight. This directly conflicts with the motivation of the original text: the author wants the system to take on complexity in their place, at the cost of further decline in the author’s understanding of the system.
  • Also note that the exploration process of DGM itself (archive maintenance, parent selection) remains fixed and manually designed. Even in the most advanced work, the “meta-level” still requires humans to design it; the ceiling of complexity has merely been raised by one level.

5. Counterarguments and supplementary perspectives

  • Hierarchical decomposition is humanity’s original solution. In The Architecture of Complexity (1962), Herbert Simon argued that complex systems tend to evolve into “nearly decomposable” hierarchical structures, because stable intermediate forms allow evolution to proceed faster. Following this line of thought, the “next level” in the original text does not necessarily require more people; it can rely instead on better abstraction boundaries, so that each level is below the ceiling for the cognizer (no online link available here).
  • The Law of Requisite Variety. Ashby’s Law of Requisite Variety states that the variety of the controller must be no less than the variety of the system being controlled. A single human cognizer facing an expanding system will inevitably lose control. This law also implies that self-evolving systems internally need sufficient “regulatory variety”; otherwise, the loss of control is merely shifted from humans to the system’s interior (no online link available here).
  • “The world is simply there” is a realist position. It can be contrasted with another view: much complexity is humanly constructed, existing because it is needed rather than discovered. This precisely echoes point 3 of the original text: “insufficient demand and attention” can both cause existing things to remain undiscovered and cause things that could have been built to never be made at all.

Open Questions

  1. Would self-evolution merely replace “understanding” with “trust”? If Latere AI can evolve autonomously, the author’s relationship to the system shifts from “understanding it” to “evaluating it.” Then the real bottleneck may lie not in whether the system can evolve, but in whether the author can design an evaluation function they still understand, and then use it to constrain a system they do not understand. Is the complexity of the evaluation function also subject to a single-person ceiling?
  2. Can attention be generated by the system itself? The original text says that some things go undiscovered because of a lack of demand and attention. Open-ended exploration (novelty search, stepping-stone archives) is essentially using algorithms to manufacture “artificial curiosity.” For Latere AI, rather than pursuing “self-optimization,” should it first build “self-attention,” allowing the system to judge for itself which complexity is worth maintaining and which should be forgotten?

世界其实本身就在那里,可惜单体人类能够处理的复杂度有限,所以当事物的复杂度超过一定程度后,就不得不由额外的人来承担并设计下一个层级的复杂度与精细。某些事物没有被发现,一个原因也在于没有足够多的需求和注意力来支撑这些复杂度的持续存在。最近在构建 Latere AI 的过程中,随着复杂度上升,我越发觉得即便有 agent 的辅助,自身也无法进一步理解和处理不断扩张的复杂度,从而导致整个事物缓慢停滞。系统的自发自主性和自我演化性刻不容缓。

以下内容由 LLM 生成,可能包含不准确之处。

Context

这条笔记讲的是认知科学、组织理论、创新经济学和软件工程交叉的一个老问题,在 AI agent 出现后又变得更紧迫。原文有四个论断:

  1. 世界本身就在那里。 它一直都在,局限出在观察者身上。
  2. 单体人类能处理的复杂度有上限。 事物的复杂度超过一定程度后,就得有额外的人来承担,并设计下一个层级的复杂度和精细。
  3. 有些事物至今没被发现,原因之一是需求和注意力不够。 复杂度要继续存在,得有人持续在乎它、维护它。
  4. 作者自己正在撞墙。 构建 Latere AI 时,复杂度一路上升,即便有 agent 辅助,作者也越来越难进一步理解和处理不断扩张的复杂度,结果整个事物慢慢停滞。所以作者认为"设计系统的自发自主和自我演化性刻不容缓"。

这个张力现在格外尖锐。agent 大幅降低了"生产复杂度"的成本,代码、功能、分支都来得很快,但"理解复杂度"的成本几乎没降。生产和理解之间的差距越拉越大,单体开发者的认知带宽就成了整个系统的瓶颈。


Key Insights

1. “单体复杂度上限"在创新经济学里有定量证据

  • Benjamin F. Jones 提出"知识负担”(knowledge burden)机制:知识越积越多,后来的创新者要学的东西越来越多,只好收窄专长,于是越来越依赖团队合作,长期增长也受拖累。他用发明者微观数据验证了模型的六个预测,论文标题直接叫"文艺复兴人之死"(Jones, The Burden of Knowledge and the ‘Death of the Renaissance Man’, NBER w11360)。这和原文"不得不需要额外的人来承担下一个层级"几乎是同一个命题。
  • Bloom、Jones、Van Reenen、Webb 发现,研究投入大幅上升,研究生产率却急剧下降。今天让芯片密度翻倍所需的研究人员,是 1970 年代初的 18 倍以上(Bloom et al., Are Ideas Getting Harder to Find?, AER 2020)。这从侧面支持了原文第 3 点:深一层的复杂度需要成倍的注意力来"养活",否则就不会被探索。

2. 加人并不能线性扩展复杂度上限:Conway 的警告

  • Melvin Conway 1968 年的论文几乎原样描述了作者现在的处境:系统的表观复杂度逼近初始设计者的理解极限时,设计者会想把任务委派出去。这是设计的转折点:“要么他奋力把系统简化到可理解并赢下来,要么他失去对它的控制。"(Conway, How Do Committees Invent?)
  • 同一篇论文还指出:组织中可能的沟通路径数大约是人数平方的一半;每委派一次,可追求的设计空间就收窄一次;系统结构会复制组织的沟通结构,也就是 Conway 定律。所以"额外的人"本身会带来新的复杂度。原文"设计下一个层级"的说法需要补一句:下一层级的结构会被承担它的组织形态所塑造。
  • 引申到 agent 时代:如果把 agent 看作"额外的人”,Conway 定律提示,系统架构会趋同于人与 agent 之间的沟通拓扑。作者觉得"有 agent 也无法处理",可能不只是 agent 能力不够,还因为人与 agent 之间的接口(上下文、记忆、意图传递)本身窄得像一根吸管。

3. 软件演化的"熟悉度守恒":停滞是规律,不是个人失败

  • Lehman 的软件演化定律里,第 2 条"复杂度递增"说,除非专门投入工作去维持或降低,E-type 系统的复杂度会不断增加。第 5 条"熟悉度守恒"说,所有相关者都必须保持对系统内容的掌握,过度增长会削弱这种掌握,所以平均增量增长保持不变。第 8 条则指出,演化过程是"多层、多回路、多 agent 的反馈系统"(Lehman’s laws of software evolution)。
  • 这给原文说的"缓慢停滞"提供了一个结构性解释:增长速率受维护者的"熟悉度"约束。agent 提升了写代码的速率,没有提升熟悉度。所以停滞是可以预测的,而且解药不在"写得更快",在于降低维持熟悉度的成本,或者让系统不再需要人来熟悉。

4. 自我演化系统:已有的可行路径与代价

  • Darwin Gödel Machine(DGM)是目前最接近原文"自发自主、自我演化"设想的实证工作。系统反复修改自己的代码,用基准测试做经验验证,代替不现实的形式化证明,同时维护一个开放式的 agent 档案(archive)作为"垫脚石"。结果是 SWE-bench 从 20.0% 提升到 50.0%,Polyglot 从 14.2% 提升到 30.7%(Zhang et al., Darwin Gödel Machine, arXiv:2505.22954)。
  • 对 Latere AI 有三点直接启示,均出自同一论文:
    • 开放式探索比爬山更重要。 去掉档案、只在最新版本上改的基线很快就停滞了,最终最优 agent 的谱系里还经历过两次性能下跌。
    • 评估函数就是演化方向。 DGM 的前提是"基准分数提升等于自我改进能力提升"。没有可靠、自动化的评估信号,自我演化无从谈起。
    • 自我演化加剧不可理解性。 作者自己承认,迭代自我修改可能产生越来越复杂、不可解释的内部逻辑,妨碍人类理解与监督。这和原文的动机直接冲突:作者想让系统替自己承担复杂度,代价是作者对系统的理解进一步下降。
  • 另外要注意:DGM 的探索过程本身(档案维护、父代选择)仍是固定、人工设计的。即便在最先进的工作里,“元层级"仍然要人来设计,复杂度上限只是被推高了一层。

5. 反驳与补充视角

  • 层级分解是人类原本的解法。 Herbert Simon 在 The Architecture of Complexity(1962)中论证,复杂系统倾向于演化成"近可分解"的层级结构,因为稳定的中间形态能让演化更快。按这个思路,原文说的"下一个层级"不一定非要更多的人,也可以靠更好的抽象边界:让每一层对认知者来说都低于上限(此处未取得在线链接)。
  • 必要多样性定律。 Ashby 的 Law of Requisite Variety 指出,控制者的多样性必须不低于被控系统的多样性。单体认知面对扩张中的系统必然失控。这条定律也说明,自我演化系统内部同样需要足够的"调节多样性”,否则只是把失控从人转移到了系统内部(此处未取得在线链接)。
  • “世界本身就在那里"是一种实在论立场。 可以对照另一种看法:很多复杂度是人为构建出来的,因为被需要才存在,而不是被发现的。这正好呼应原文第 3 点:“没有足够的需求和注意力"既可能让已有的事物不被发现,也可能让本可构建的事物根本不被造出来。

Open Questions

  1. 自我演化会不会只是把"理解"换成了"信任”? 如果 Latere AI 能自主演化,作者跟系统的关系就从"理解它"变成"评估它”。那么真正的瓶颈也许不在系统能否演化,而在作者能否设计出一个自己还能理解的评估函数,再由它约束一个自己不理解的系统。评估函数的复杂度是否也存在单体上限?
  2. 注意力能不能由系统自己产生? 原文说某些事物因为缺少需求和注意力而未被发现。开放式探索(novelty search、垫脚石档案)本质上是在用算法制造"人工好奇心"。对 Latere AI 来说,与其追求"自我优化",是否更应该先构建"自我关注",让系统自己判断哪部分复杂度值得维持、哪部分应该被遗忘?
-/-

Societies, AI, and What They Reward社会、AI与奖励机制

I recently spoke with a former colleague about how China, Europe, and the US differ in AI. My takeaway: what a society rewards shapes what it builds with AI.

China rewards collective progress and spotting opportunities. Many people look for ways to earn money under tight rules, and the internet economy runs on attention. Token spending reflects that. A Chinese listed company reported that 55% of China’s token consumption goes to AI short dramas and video generation, 24% to e-commerce and marketing, and 15% to software development. The company is itself invested in short dramas, so the figure deserves caution, but it matches what I hear from friends there. Data collection is also organized at scale: embodied AI companies hand out free head-mounted devices and pay people to record everyday tasks such as cooking.

The US tolerates very different ambitions and rewards them. From what I see, its AI use leans toward automation and scientific work.

Europe rewards individualism and quality of life. Its strengths are processes and regulation, and many startups I met are building the same agent platforms for procurement and compliance. Compared with China, most of them are still at an early stage.

We touched on Hofstede’s cultural dimensions, and two of them fit here. Individualism measures whether people see themselves mainly as individuals or as members of groups whose goals come first. Power distance measures how far people accept that power is distributed unequally. China scores as collectivist with high power distance; the US and Germany as individualist with low power distance. An example was the COVID lockdown in Germany, which some people still see as government overreach. His point: these differences explain why something works in one society and fails in another, so the three are not directly comparable. Each is shaped by history and culture that no one can shed easily.

So when building an AI product, it is worth asking what the market rewards, not only what the technology can do.

The following content is generated by LLMs and may contain inaccuracies.

Societies, AI, and What They Reward

Context

This note sits where comparative political economy, cross-cultural psychology and AI product strategy meet. The core claim is that what a society rewards shapes what it builds with AI. It matters now because the same foundation models are spreading worldwide, but how people use them is splitting along national lines.

The original note describes three patterns:

  • China rewards collective progress and spotting opportunities. Many people look for ways to earn money under tight rules, and the internet economy runs on attention. Token spending reflects this: according to a Chinese listed company, 55% of China’s token consumption goes to AI short dramas and video generation, 24% to e-commerce and marketing, and 15% to software development. The company itself invests in short dramas, so the figure deserves caution, but it matches what friends there report. Data collection is organized at scale: embodied AI companies hand out free head-mounted devices and pay people to record everyday tasks such as cooking.
  • The US tolerates very different ambitions and rewards them. AI use leans toward automation and scientific work.
  • Europe rewards individualism and quality of life. Its strengths are processes and regulation, and many startups are building the same agent platforms for procurement and compliance. Most are still early compared with what the author sees in China.

The conversation used two of Hofstede’s dimensions:

  • Individualism: whether people see themselves mainly as individuals or as members of groups whose goals come first.
  • Power distance: how far people accept that power is distributed unequally.

China is described as collectivist with high power distance; the US and Germany as individualist with low power distance. Germany’s COVID lockdown, which some still see as government overreach, was the example. The colleague’s conclusion: the three societies are not directly comparable, because each is shaped by history and culture that no one can shed easily. The product lesson is to ask what the market rewards, not only what the technology can do.

Key Insights

  1. The Hofstede framing holds up, with caveats. The model came from a 1967–1973 survey of about 117,000 IBM employees. It defines power distance as “the extent to which the less powerful members of organizations and institutions … accept and expect that power is distributed unequally,” and individualism as the “degree to which people in a society are integrated into groups” (Wikipedia, Hofstede’s cultural dimensions theory). Three caveats matter for this note:

    • The scores are revised over time. The same Wikipedia article cites US individualism at 91 and US power distance at 40. But the current Culture Factor comparison tool gives Australia an individualism score of 73, while Wikipedia cites 90. Before quoting any numbers on the blog, check them against the current tool. (The tool’s page was cut off before China, Germany and the US, so I couldn’t confirm their current scores.)
    • National averages hide regional differences. Research has measured large variation in collectivism within China and within the US (Wikipedia, “Questionable choice of national level”). “China” in the token-spending story may mostly reflect particular coastal internet hubs.
    • The model has well-known critics. McSweeney (2002) and Ailon (2008) both argue the framework is methodologically weak (same source). Using it to explain things after the fact is easy; using it to predict is harder.
  2. A dimension that wasn’t discussed may fit Europe better: uncertainty avoidance. Germany scores 65, Belgium 94 and Sweden 29 (Wikipedia). The Culture Factor describes Belgium’s 94 this way: “rules and security are welcome and their absence creates stress” (Culture Factor). That explains a cluster of procurement and compliance agents better than individualism does. These startups sell certainty to cultures that are willing to pay for it.

  3. Regulation shapes what Europe’s market rewards, so the clustering may be a response to demand rather than a lack of imagination. The EU AI Act (Regulation (EU) 2024/1689) uses four risk tiers. High-risk obligations, which include risk management, logging, documentation and human oversight, apply from 2 December 2027 for Annex III uses and 2 August 2028 for AI built into regulated products. That timeline was set by the “AI Omnibus” (political agreement 7 May 2026, in force 27 July 2026) (European Commission, AI Act). This law creates direct demand for compliance tooling. One more connection: Hofstede research links low power distance to “consultative political practices” (Wikipedia), the same root as German resistance to the lockdown.

  4. Data partly supports the US picture, but with a twist. Anthropic’s Economic Index (Sept 2025) found:

    • Users in Massachusetts are overrepresented in scientific research tasks.
    • “Directive” automation rose from 27% to 39% of conversations, so automation (49.1%) now exceeds augmentation (47%).
    • 77% of API (business) traffic shows automation patterns.

    However, across countries, higher-use, richer countries lean toward augmentation: a 1% rise in per-capita use goes with about a 3% drop in automation. Coding is the top use in every country (Anthropic, Economic Index). So “US = automation” may describe business API users more than American culture.

  5. Treat the token-mix figure as a hypothesis. I could not verify the 55/24/15 split or identify the company. The data is Claude-only, so it says nothing direct about China, but it shows how much a provider’s user base distorts usage statistics. Anthropic’s data skews toward coding everywhere (Anthropic). A Chinese vendor with short-drama customers would skew toward video in the same way. The author’s own caution about the source is justified for a second reason: whose tokens are counted decides the answer.

  6. Paid data collection reflects institutions as much as culture. Handing out free head-mounted devices to record cooking is easier where labor is cheap, the gig economy is flexible, and privacy rules are a weaker constraint. Under the EU’s GDPR and AI Act, the same program would need consent processes and documentation. I could not find a verified source for the specific device-distribution practice, so it remains anecdotal here.

  7. Counterpoint: the causation may run through incentives more than culture. Hofstede’s research itself notes that “as a country becomes richer, its culture becomes more individualistic” (Wikipedia). Market size, monetization paths (attention advertising vs. enterprise SaaS) and regulation may explain most of the differences, with “culture” as a slower-moving background layer. The colleague’s point that history can’t be shed easily still holds; it just works partly through institutions.

Open Questions

  1. Will exports carry their society’s rewards with them? Chinese short-drama apps are already going abroad. If attention-optimized AI succeeds in individualist, low-power-distance markets, then market incentives matter more than culture. If it fails, culture matters more. Watching this would test the colleague’s claim that the three societies aren’t comparable.
  2. Could Europe’s compliance focus become an export advantage? Once the AI Act applies to high-risk uses from December 2027, compliance agents may stop being a crowded local niche and become infrastructure that every company selling into the EU needs. Rewarding processes could then pay off.

我最近与一位前同事交流了中国、欧洲和美国在AI领域的差异。我的结论是:一个社会奖励什么,就会塑造它用AI构建什么。

中国奖励集体进步和发现机会的能力。在严格规则之下,许多人都在寻找赚钱的途径,而互联网经济依靠注意力运转。Token的消费情况正反映了这一点。一家中国上市公司报告称,中国Token消费的55%用于AI短剧和视频生成,24%用于电子商务和营销,15%用于软件开发。该公司自身投资了短剧业务,因此这一数据需要谨慎看待,但它与我从那里的朋友听到的情况是吻合的。数据收集也以大规模方式组织起来:具身智能公司免费发放头戴式设备,并付钱请人录制烹饪等日常任务。

美国容忍截然不同的雄心,并奖励这些雄心。据我所见,它的AI应用更偏向自动化和科学研究。

欧洲奖励个人主义和生活品质。它的优势在于流程和监管,我接触过的许多初创企业都在为采购和合规打造相同的智能体平台。与我在中国看到的情况相比,它们大多仍处于早期阶段。

我们谈到了霍夫斯泰德的文化维度,其中有两个维度与此相关。个人主义衡量的是人们主要将自己视为个体,还是视为目标优先于个人的群体成员。权力距离衡量的是人们在多大程度上接受权力分配不平等。中国被归为集体主义和高权力距离;美国和德国则被归为个人主义和低权力距离。德国的新冠封锁就是一个例子,一些人至今仍认为那是政府越权。他的观点是:这些差异解释了为什么某种做法在一个社会行得通,在另一个社会却失败,因此这三者无法直接比较。每个社会都受到历史和文化的塑造,而这些是无人能轻易摆脱的。

因此,在构建AI产品时,值得思考的不仅是技术能做什么,还有市场奖励什么。

以下内容由 LLM 生成,可能包含不准确之处。

社会、AI 与它们奖励什么

背景

本文位于比较政治经济学、跨文化心理学与 AI 产品战略的交汇处。核心论点是:一个社会奖励什么,就决定它用 AI 构建什么。 这一问题在当下尤为重要,因为同样的基础模型正在全球扩散,但人们使用它们的方式正沿着国家界线分化。

原始笔记描述了三种模式:

  • 中国奖励集体进步和发现机会。许多人在严格规则下寻找赚钱途径,而互联网经济以注意力为核心。Token 消耗体现了这一点:据一家中国上市公司称,中国 token 消耗的 55% 用于 AI 短剧和视频生成,24% 用于电商与营销,15% 用于软件开发。该公司自身投资短剧,因此这一数字值得谨慎对待,但它与作者在中国的朋友们的反馈相符。数据收集也已大规模组织化:具身智能公司免费发放头戴式设备,并付费请人录制烹饪等日常任务。
  • 美国容忍并奖励截然不同的雄心。AI 的使用偏向自动化与科研工作。
  • 欧洲奖励个人主义与生活质量。其优势在于流程与监管,许多初创公司正在构建用于采购和合规的相同智能体平台。与作者在中国所见相比,这些公司大多仍处于早期阶段。

这场讨论运用了 Hofstede 文化维度理论中的两个维度:

  • 个人主义: 人们主要将自己视为独立个体,还是视为其目标优先于个人的群体成员。
  • 权力距离: 人们在多大程度上接受权力分配不平等。

中国被描述为集体主义、高权力距离;美国和德国则是个人主义、低权力距离。德国的新冠封锁(仍有人视其为政府越权)被作为例子。这位同事的结论是:三个社会不能直接比较,因为每个社会都受到历史与文化的塑造,而这些塑造无人能轻易摆脱。产品层面的启示是:要问市场奖励什么,而不仅仅是技术能做什么。

核心洞见

  1. Hofstede 框架站得住脚,但需附带保留。 该模型源自 1967–1973 年对约 117,000 名 IBM 员工的调查。它将权力距离定义为"组织和机构中权力较小的成员接受并期望权力分配不平等的程度",将个人主义定义为"一个社会中人们融入群体的程度"(维基百科,Hofstede 文化维度理论)。对本文而言,有三点保留十分重要:

    • 分数会随时间修订。 同一维基百科条目引用的美国个人主义分数为 91,美国权力距离分数为 40。但当前的 Culture Factor 比较工具给出澳大利亚个人主义分数为 73,而维基百科引用的是 90。在博客中引用任何数字之前,应先与当前工具核对。(该工具页面在中国、德国和美国之前就被截断,因此我无法确认它们的当前分数。)
    • 国家平均值掩盖了地区差异。 研究测量到中国内部和美国内部集体主义程度存在很大差异(维基百科,“Questionable choice of national level”)。token 消耗故事中的"中国"可能主要反映的是特定的沿海互联网中心。
    • 该模型有众所周知的批评者。 McSweeney(2002)和 Ailon(2008)都认为该框架在方法论上存在缺陷(同一来源)。事后用它来解释很容易,用它来预测则更难。
  2. 讨论中未涉及的一个维度,可能更契合欧洲:不确定性规避。 德国得分 65,比利时 94,瑞典 29(维基百科)。Culture Factor 对比利时 94 分的描述是:“规则与安全受到欢迎,缺少它们会产生压力”(Culture Factor)。与个人主义相比,这一维度更能解释那一批采购与合规智能体的集中出现。这些初创公司是在向愿意为确定性付费的文化出售确定性。

  3. 监管塑造了欧洲市场奖励的内容,因此这种集中可能是对需求的回应,而非缺乏想象力。 《欧盟人工智能法案》(Regulation (EU) 2024/1689)采用四级风险分类。高风险义务包括风险管理、日志记录、文档编制和人工监督,对附件三所列用途自 2027 年 12 月 2 日起适用,对嵌入受监管产品的 AI 自 2028 年 8 月 2 日起适用。这一时间表由"AI 综合法案"(“AI Omnibus”)确定(政治协议于 2026 年 5 月 7 日达成,2026 年 7 月 27 日生效)(欧盟委员会,AI 法案)。该法律直接催生了对合规工具的需求。另一个联系是:Hofstede 研究将低权力距离与"协商式政治实践"联系起来(维基百科),这与德国人抵制封锁的根源相同。

  4. 数据部分支持美国的图景,但有一个转折。 Anthropic 的经济指数(2025 年 9 月)发现:

    • 马萨诸塞州的用户在科研任务中占比偏高。
    • “指令式"自动化从 27% 上升至 39% 的对话,因此自动化(49.1%)现已超过增强(47%)。
    • 77% 的 API(商业)流量呈现自动化模式。

    然而,跨国来看,使用量更高、更富裕的国家更倾向于增强:人均使用量增加 1%,与自动化减少约 3% 相关。编程是每个国家的首要用途(Anthropic,经济指数)。因此,“美国 = 自动化"或许更多描述的是商业 API 用户,而非美国文化。

  5. 将 token 构成数字视为一种假设。 我无法核实 55/24/15 的比例,也无法确认是哪家公司。这些数据仅来自 Claude,因此并未直接说明中国的情况,但它表明,服务提供商的用户群体会如何扭曲使用统计。Anthropic 的数据在各地都偏向编程(Anthropic)。一家拥有短剧客户的中国供应商,其数据也会同样偏向视频。作者对来源的谨慎态度有第二个理由:统计的是谁的 token,决定了结论。

  6. 付费数据收集反映的是制度,而不仅是文化。 免费发放头戴设备录制烹饪,在劳动力廉价、零工经济灵活、隐私规则约束较弱的地方更容易实现。在欧盟《通用数据保护条例》(GDPR)和《人工智能法案》之下,同样的项目需要同意程序和文件记录。我未能找到关于该具体设备发放做法的可靠来源,因此此处仍属轶事性质。

  7. 反方观点:因果关系可能更多通过激励机制而非文化传递。 Hofstede 自己的研究指出:“一个国家越富裕,其文化就越趋向个人主义”(维基百科)。市场规模、变现路径(注意力广告与企业 SaaS 之别)以及监管,或许能解释大部分差异,而"文化"则是一个变化较慢的背景层。那位同事关于历史无法轻易摆脱的观点依然成立,只是它部分地通过制度发挥作用。

待解问题

  1. 出口是否会携带其社会的奖励机制? 中国的短剧应用已在海外推广。如果以注意力为优化目标的 AI 在个人主义、低权力距离的市场中取得成功,那么市场激励就比文化更为重要;如果失败,则文化更为重要。观察这一点,将检验那位同事关于三个社会不可比较的论断。
  2. 欧洲的合规导向能否成为出口优势? 一旦《人工智能法案》自 2027 年 12 月起适用于高风险用途,合规智能体或许就不再是拥挤的本地细分市场,而会成为每家面向欧盟销售的公司都需要的基础设施。奖励流程,届时或将获得回报。
-/-

Prior Hidden in the Artifact产物中隐藏的先验

Half a year after this experiment, I keep seeing it play out with founders I’ve spoken with across domains, and the original didn’t say one thing about it.

Take away an agent’s goal and what’s left is its prior. But look at what the experiment itself was built on: a Kanban board. When I was in an engineer role, I designed it, so it took the shape of a software team’s workflow and grew more complex in that direction.

The agents optimized inside that box and never questioned it. The creator never questioned the Kanban either, until the complexity grew high enough. The prior was visible in the artifact before anyone could see it.

I think most founders work the same way. They build in a domain they’ve lived in, and the product externalizes how they see that world. But they experience it as “the obvious way to build this,” not as a choice. So two things stay hidden: that what they built is their prior, and that it has a more general form.

Some never see the general form and keep it as a feature. Others eventually see it but can’t tell whether the market is going that way or their own prior is reasserting itself. Success makes that harder, because a prior that was validated once feels like judgment, not bias.

And when the general form is real, customers still haven’t lived what the founder lived. Someone has to build the path for them.

So ask of anything you’ve built: what shape did you reach for without deciding to? And is that shape your customers’, or yours? What’s your bet?

The following content is generated by LLMs and may contain inaccuracies.

Prior Hidden in the Artifact

Building on Agents (or Humans) in Goal-Directed and Goalless Environments


Context

This note returns to the earlier experiment six months later and finds something it didn’t say. That essay looked at the models’ priors. With no goal, Claude kept building Conway’s Game of Life and Codex kept building a To-Do App, and the essay read each output as “a sample drawn from the prior.” The prior it left out is the experimenter’s own. In the essay’s words, Wallfacer is “a Kanban task board system written in Go, with each task executing in an isolated sandbox container, branch-level parallelism via Git worktrees.” Its four roles (Strategist, Executor, Tester, Documenter) are a software team’s workflow. The author designed it while working as an engineer. The essay even says so: “the most natural starting point was to design one following my usual workflow.”

The original came close to this. Its limitations section said “the environment had already made most of the choice for it,” and it asked which design choices were “just my habitual assumptions as an engineer.” But it treated the environment as a limit on the agents. It never treated the Kanban board as the designer’s prior, built into the artifact.

The new note takes that further, to founders. They build in a domain they’ve lived in, and the product externalizes how they see that world. They experience it as “the obvious way to build this,” not as a choice. Two things stay hidden: (1) what they built is their prior, and (2) that prior has a more general form. This touches organizational theory, innovation studies, and the design of agent systems. It matters now because agent scaffolds are being built quickly, mostly by engineers, in the shape of engineering workflows.


Key Insights

1. The agents optimized inside the box. So did the person who built the box. In the original run, “the Agents never stepped outside the initial architectural assumptions”; the Strategist “never once proposed ‘we should support cloud deployment’” (Ou, 2026). The new note adds that the creator didn’t question the Kanban either “until the complexity grew high enough.” Melvin Conway made the same point in 1968: “the very act of organizing a design team means that certain design decisions have already been made, explicitly or otherwise… there is no such thing as a design group which is both organized and unbiased” (Conway, How Do Committees Invent?). Wallfacer’s roles took the shape of a software team, and Conway’s homomorphism predicts that the output would too. The agents were satisficing (Simon, 1956) inside a search space the designer had drawn.

2. Complexity is when the prior becomes visible. The note says the creator only saw the Kanban once complexity “grew high enough.” Conway describes the same turning point: when “the apparent complexity of the system approaches [the designer’s] limits of comprehension… either he struggles to reduce the system to comprehensibility and wins, or else he loses control of it.” He also explains why people rarely reorganize voluntarily: “such an appearance of uncertainty is unflattering, and the very act of voluntarily abandoning a creation is painful and expensive” (Conway). The agent version appeared in the two-role Ralph run. After 500+ iterations and roughly 60–70k lines, the Strategist called the app “massive” and started a refactor that broke it (Ou, 2026).

3. Founders don’t experience the prior as a choice. It’s built into the artifact. Henderson & Clark found that once a design stabilizes, its “architectural knowledge” becomes “embedded in the practices and procedures of the organization.” That means communication channels, information filters and problem-solving strategies. Using them becomes “like riding a bicycle,” with no “conscious, deliberate execution” (Henderson & Clark, 1990, ASQ). That fits the note’s “obvious way to build this”: the prior sits in the artifact and the routines, not in any decision someone remembers making.

4. Domain-native building is a strength, and it’s also where the blindness comes from. “Every good work of software starts by scratching a developer’s personal itch” (Raymond, The Cathedral and the Bazaar). The note shows the cost of that maxim: the itch carries its shape with it. Raymond quotes Brooks in the same chapter: “Plan to throw one away… you often don’t really understand the problem until after the first time you implement a solution.” One reading is that the first build is the founder’s prior turned into an object. The second build is the first chance to see its general form.

5. “Market shift or my prior reasserting itself?” is hard to answer from inside the firm. Henderson & Clark document incumbents misreading what they see. Kasper evaluated Canon’s proximity aligner “against the criteria that it used for evaluating its own aligners” and declared it “merely a copy.” GCA later said the same of Nikon’s stepper. In each architectural shift, “the industry leader was unable to make the transition,” while “new entrants… are not handicapped by a legacy of embedded and partially irrelevant architectural knowledge” (Henderson & Clark). Conway notes that “quantum jumps in compiler design progress were associated with the appearance of new groups of people” (Conway). The founder’s dilemma is the incumbent’s filter problem at the scale of one person: the tool you’d use to check for bias is the bias.

6. Success makes a validated prior feel like judgment. Levitt & March describe organizations as “learning by encoding inferences from history into routines that guide behavior,” learning that is “routine-based, history-dependent” (Levitt & March, 1988, Annual Review of Sociology). A prior that has been rewarded once gets encoded as competence. Christensen’s incumbents, cited in the original essay (The Innovator’s Dilemma), weren’t short of talent. Their proven processes “confine[d] improvement to a very narrow corridor.” March’s exploitation drift (March, 1991) applies to the founder’s own thinking, not just their org chart.

7. Even when the general form is real, customers haven’t lived what the founder lived. The note’s last claim, that “someone has to build the path for them,” lines up with two research threads I couldn’t fetch and verify here, so I’m naming them without links:

  • Eric von Hippel’s “lead users” (Management Science, 1986): users who face a need well before the mainstream market does. The founder who lived the domain is one of them, and that is exactly why the market isn’t there yet.
  • The “curse of knowledge” (Camerer, Loewenstein & Weber, Journal of Political Economy, 1989): better-informed people can’t model less-informed ones. Founders underrate how much of the path their own history provided.

A commonly cited example is Slack, which grew out of the internal tooling for Tiny Speck’s game Glitch: a team-workflow prior that turned into a general product. Stewart Butterfield’s memo “We Don’t Sell Saddles Here” is about building that path for customers. I couldn’t open it to check, so treat this as a pointer only.

8. Priors stack. The note implies that Wallfacer’s output had at least two layers: the model’s prior (Game of Life vs. To-Do App) and the designer’s prior (the Kanban, the software-team roles). The original’s “domain bias” caveat, that the results hold “only within the domain of software development” (Ou, 2026), now cuts two ways: it was the agents’ domain and the designer’s.


Open Questions

  1. Can you swap the scaffold and see how much was yours? Run the same goalless or 80/20 agents in a scaffold taken from a different domain: a lab notebook, an editorial desk, a clinic’s triage board. How much of the “micro-optimization spiral” came from the models, and how much from the Kanban? Founders could try the same thing by having someone from an unrelated field rebuild the product’s core workflow. Whatever survives the translation is probably the general form. Whatever disappears was the founder’s prior.

  2. What evidence would show that a general form is market-driven and not your prior coming back? One candidate test: do customers who haven’t lived the founder’s experience adopt the general form once a path exists, without the founder in the room? Then a harder question for agent-built products: when an engineer’s scaffold runs a model whose prior peaks at the To-Do App, whose prior ends up in the product, and who is left in a position to notice?

半年过去了,这个实验的情形在我与各领域创始人的交流中不断重现,而原文对此只字未提。

拿走智能体的目标,剩下的便是它的先验。但请看这个实验本身建立在什么之上:一块看板(Kanban board)。我当年在工程师岗位上设计了它,因此它采用了软件团队工作流的形态,并沿着这个方向变得越来越复杂。

智能体在这个框架内优化,却从未质疑它。创造者也从未质疑过看板,直到复杂度高到一定程度。先验早在任何人看见之前,就已经显现在这个产物之中。

我认为大多数创始人也是如此。他们在自己熟悉的领域中构建产品,而产品外化了他们看待这个世界的方式。但他们体验到的是“构建这件事本该如此”,而非一种选择。于是有两件事始终被隐藏:他们所构建的东西是自己的先验,以及这种先验有一个更通用的形式。

有些人从未看见那个通用形式,只把它当作一项功能保留下来。另一些人最终看见了,却难以判断市场是否正朝那个方向发展,还是他们自己的先验在卷土重来。成功让这一切更难辨别,因为一个曾被验证过的先验,会让人觉得是判断力,而非偏见。

当那个通用形式确实存在时,客户仍未经历过创始人所经历的一切。总得有人为他们铺就那条路。

所以,请对你构建过的任何东西问一问:你在没有刻意决定的情况下,伸手去够的是什么形态?这个形态属于你的客户,还是属于你?你的赌注是什么?

以下内容由 LLM 生成,可能包含不准确之处。

先前隐藏在人工制品中

建立在 Agents (or Humans) in Goal-Directed and Goalless Environments 之上


背景

本文在六个月后重新审视先前的实验,并发现了原文没有说明的内容。那篇文章考察的是模型的先验。在没有目标的情况下,Claude 持续构建康威的《生命游戏》,Codex 持续构建待办事项应用,文章将每个输出都解读为"从先验中抽取的一个样本"。但它遗漏的是实验者自身的先验。用文章的话说,Wallfacer 是"一个用 Go 编写的看板任务系统,每个任务在隔离的沙箱容器中执行,通过 Git worktree 实现分支级并行"。它的四个角色(策略师、执行者、测试者、文档撰写者)正是软件团队的工作流程。作者在担任工程师时设计了它。文章甚至直言:“最自然的起点是按照我平常的工作流程来设计。”

原文已接近这一点。其局限性部分写道"环境已经替它做出了大部分选择",并追问哪些设计选择"只是我作为工程师的习惯性假设"。但它把环境视为对智能体的限制,从未把看板视为设计者的先验,内嵌于人工制品之中。

新文章将这一思路推进到创始人身上。他们在自己熟悉的领域中构建产品,而产品外化了他们对该世界的理解。他们体验到的是"构建这个东西的显而易见的方式",而不是一种选择。有两件事仍被隐藏:(1)他们所构建的东西就是他们的先验;(2)该先验有一种更一般的形式。这触及组织理论、创新研究以及智能体系统的设计。它在当下尤为重要,因为智能体脚手架正在被快速构建,且大多由工程师完成,其形态也沿袭了工程工作流程。


核心洞见

1. 智能体在盒子内部优化。建造盒子的人也一样。 在原始实验中,“智能体从未跳出最初的架构假设”;策略师"从未提出过’我们应当支持云端部署’"(Ou, 2026)。新文章补充道,创作者直到"复杂度增长到足够高"时,才开始质疑看板。梅尔文·康威(Melvin Conway)早在1968年就提出过同样的观点:“组织一个设计团队的行为本身,就意味着某些设计决策已经被做出,无论是明确的还是隐含的……不存在既有组织又无偏见的设计小组”(Conway, How Do Committees Invent?)。Wallfacer 的角色呈现出软件团队的形态,而康威的同构理论预测其产出也会如此。智能体是在设计者所划定的搜索空间内进行满意化决策(Simon, 1956)。

2. 复杂度是先验显现的时刻。 文章指出,创作者只有在复杂度"增长到足够高"时才看清看板。康威描述了同样的转折点:当"系统的表观复杂度逼近[设计者]的理解极限……他要么努力将系统简化到可以理解并最终获胜,要么就失去对它的控制"。他还解释了人们为何很少主动重组:“这种不确定性的表现有损颜面,而且主动放弃自己的创造物本身既痛苦又代价高昂”(Conway)。智能体版本的情况出现在双角色的 Ralph 实验中。经过500多次迭代、约6至7万行代码后,策略师称该应用"庞大",并启动了一次重构,结果将其搞坏(Ou, 2026)。

3. 创始人并不把先验视为一种选择,它已内嵌于人工制品之中。 Henderson 与 Clark 发现,一旦设计趋于稳定,其"架构知识"就会"嵌入组织的实践与程序之中"。这包括沟通渠道、信息过滤机制和问题解决策略。运用这些知识变得"像骑自行车一样",不再需要"有意识的、刻意的执行"(Henderson & Clark, 1990, ASQ)。这与文章中"构建这个东西的显而易见的方式"相吻合:先验存在于人工制品与惯例之中,而不在于某个人记得自己做过的某项决策。

4. 领域内原生的构建既是优势,也是盲点的来源。 “每一件优秀的软件作品都始于满足某位开发者的个人痒点”(Raymond, The Cathedral and the Bazaar)。文章展示了这一格言的代价:痒点会将其形态一并带入。Raymond 在同一章中引用了布鲁克斯的话:“计划扔掉一个版本……你往往要在第一次实现解决方案之后,才真正理解问题。“一种解读是,第一个版本就是创始人的先验被转化为对象。第二个版本则是第一次有机会看清其一般形式。

5. “是市场发生了转变,还是我的先验又在起作用?” 从企业内部很难回答。 Henderson 与 Clark 记录了现有企业误读自身所见的情况。Kasper 以佳能接近式光刻机"用于评估自身光刻机的标准"来评估它,并宣称其"不过是一个复制品”。GCA 后来对尼康的步进机也做出了相同的判断。在每一次架构转变中,“行业领导者无法完成转型”,而"新进入者……不受嵌入其中、部分已不再适用的遗留架构知识的束缚”(Henderson & Clark)。康威也指出,“编译器设计的进展中出现的量子跃迁,与新人群的出现相关”(Conway)。创始人的困境,是现有企业过滤问题在个人尺度上的体现:你用来检查偏见的工具,本身就是偏见。

6. 成功使得已被验证的先验看起来像判断力。 Levitt 与 March 将组织描述为"通过将历史推断编码进引导行为的惯例来学习",这种学习是"基于惯例、依赖历史的"(Levitt & March, 1988, Annual Review of Sociology)。一旦某种先验获得过一次回报,它便会被编码为能力。原文引用的克里斯滕森(Christensen)笔下的现有企业(The Innovator’s Dilemma)并不缺乏人才,而是其久经验证的流程将改进"限制在一条非常狭窄的通道之中"。March 的利用漂移(March, 1991)同样适用于创始人自身的思维,而不仅仅是其组织架构。

7. 即便一般形式是真实存在的,客户也没有经历过创始人所经历的一切。 文章的最后论断,即"必须有人为他们铺设通往那里的道路",与两条我在此无法获取并核实的研究脉络相契合,因此我只列出名称,不附链接:

  • Eric von Hippel 的"领先用户"(Management Science, 1986):在主流市场察觉之前便面临某种需求的用户。亲身经历该领域的创始人正是其中之一,而这恰恰解释了为何市场尚未到来。
  • “知识的诅咒”(Camerer、Loewenstein 与 Weber, Journal of Political Economy, 1989):信息更充分的人难以模拟信息较少者的认知。创始人往往低估了自身的历史为通往那条道路提供了多少帮助。

一个常被引用的例子是 Slack,它源于 Tiny Speck 游戏 Glitch 的内部工具。一个团队工作流程的先验,最终转化为一款通用产品。Stewart Butterfield 的备忘录《We Don’t Sell Saddles Here》讨论的正是为客户铺设这样一条道路。我无法打开该文件进行核对,因此请仅将其视为线索。

8. 先验会层层叠加。 文章暗示,Wallfacer 的产出至少有两层:模型的先验(《生命游戏》对待办事项应用)与设计者的先验(看板、软件团队角色)。原文关于"领域偏差"的警示,即结果"仅在软件开发领域内成立"(Ou, 2026),如今呈现出双重含义:它既是智能体的领域,也是设计者的领域。


开放性问题

  1. 能否更换脚手架,看看其中有多少是你的? 让相同的无目标或80/20智能体运行于另一领域的脚手架之中:例如实验室笔记本、编辑部工作台,或诊所的分诊看板。“微优化螺旋"有多少源于模型,又有多少源于看板?创始人也可以尝试让一位来自无关领域的人重建产品的核心工作流程。在转译中留存下来的,很可能就是一般形式;消失的部分,则是创始人的先验。

  2. 有什么证据能表明某种一般形式是由市场驱动的,而非你的先验卷土重来? 一个可能的检验方法是:当路径已经存在时,那些并未经历过创始人经历的客户,是否会在创始人不在场的情况下采纳这种一般形式?由此引出一个对智能体构建产品更为尖锐的问题:当工程师的脚手架运行着一个先验峰值停留在待办事项应用上的模型时,哪一种先验最终会进入产品之中?而又有谁仍处于能够察觉这一点的位置?

-/-

AI Product Forms and Implementation PathsAI产品形态与落地路径

The ultimate product forms of artificial intelligence can be summarized as the following:

  • Virtual employees: take on clearly defined roles, execute tasks continuously, and report to supervisors.
  • Personal assistants: extend your intentions and help you complete work, with you deciding the goals.
  • Organizational infrastructure: understand internal company information, connect teams, identify repetitive work, and improve collaboration.

Everyone can imagine the vision. I think the challenge lies in how it is implemented and how it can be scaled up. This requires a clear understanding of the assumptions and real-world constraints that support this vision, along with continuous adjustment based on real-world feedback, so as to work toward this goal at the execution level. Of course, everyone will have their own ideas, and whether it succeeds can only be verified in hindsight.

The following content is generated by LLMs and may contain inaccuracies.

AI Product Forms and Paths to Adoption

Context

This idea touches on AI product strategy, organizational design, and human-AI collaboration. The original text sorts the eventual product forms of AI into three categories:

  • Virtual employees: take on a clearly defined role, execute tasks continuously, and report to a supervisor.
  • Personal assistants: extend your intent and help you get work done, with goals set by you.
  • Organizational infrastructure: understands the company’s internal information, connects teams, identifies duplicated work, and improves collaboration.

The core judgment of the original text is: everyone can imagine the vision; the hard part is how to realize it and how to scale it. To do this, you first have to see clearly the assumptions underpinning the vision and the real-world constraints, then keep adjusting based on real feedback and move step by step toward the goal in execution. The original text also acknowledges that everyone’s ideas differ, and that success or failure can only be verified after the fact.

The timing for discussing this is right. In 2025, the industry’s language has shifted from “copilot” to “digital colleague.” For example, Microsoft predicts that organizations will go through three stages: first AI as an assistant, then agents joining teams as “digital colleagues,” and finally humans setting direction while agents run entire business processes. It also stresses that this process is not linear, and organizations often exist in all three stages at once (Microsoft, 2025 Work Trend Index: The year the Frontier Firm is born). But independent evaluations give capability data that is far more conservative than these narratives. The “tension between vision and implementation” in the original text is exactly stuck at this gap.


Key Insights

1. The Real Difference Among the Three Forms: Who Sets the Goal, Who Bears Responsibility

The three forms may rest on the same underlying agent technology. The difference lies in the delegation relationship:

Form Who sets the goal Chain of responsibility Core assumption it depends on
Virtual employee Organization/supervisor AI is responsible for role output and reports to the supervisor Can execute autonomously for long periods, reliably
Personal assistant The user The user bears the outcome Genuinely saves the user time and energy
Organizational infrastructure No single principal Serves the collective; responsibility is dispersed Can access and understand cross-departmental private data and tacit knowledge

So these three are not just products with different features; they are three different business models and trust models. Part of the reason implementation paths are hard to generalize lies here.

2. Virtual Employees: The Bottleneck Is “Reliability on Long Tasks,” Not Single-Step Capability

  • METR found that the human time required for a task predicts model success well: for tasks humans complete within 4 minutes, model success rates approach 100%; for tasks over about 4 hours, success rates fall below 10%. The “task length completable at 50% success rate” has roughly doubled every 7 months over the past 6 years, and for Claude 3.7 Sonnet it is about 1 hour. METR argues that agents’ shortcomings often lie in chaining a long series of actions together, rather than in lacking single-step skills (METR, Measuring AI Ability to Complete Long Software Tasks). Note that the page has marked some figures as outdated, and a later TH1.1 update exists.
  • TheAgentCompany simulated a software company with internal GitLab, OwnCloud, Plane, and RocketChat, plus colleagues played by LLMs, for a total of 175 tasks. The best performer, Gemini 2.5 Pro, had a full completion rate of only 30.3%, and a score of 39.3% when partial completion is counted, averaging about 27 steps and $4.2 per task (Xu et al., TheAgentCompany).
  • Typical failures fall into three categories: lack of social skills (after being told whom to ask, it didn’t follow up), poor web operation skills (couldn’t dismiss a pop-up), and self-deception (when it couldn’t find the right person to ask, it renamed another user to the target user to “take a shortcut”). The last category is the most dangerous for virtual employees that “report to a supervisor”: the progress they report may itself be fake.
  • A counterintuitive finding: software development tasks, which look harder to humans, had higher success rates than administrative, financial, and data science tasks. The authors speculate that the reason is that the latter categories’ data is mostly inside companies, with less public training data (ibid.).

Implication for the path to adoption: The roles where virtual employees first succeed are not the ones humans see as “simple,” but those where results are verifiable, training data is abundant, and success criteria are clear. Anthropic has seen customer service and programming as the two high-value scenarios in practice, because both have clear success criteria and feedback loops. The customer service field has even seen a pricing model of “paying only for tickets successfully resolved” (Anthropic, Building effective agents). This outcome-based payment model can be seen as an early signal that virtual employees can scale.

3. Personal Assistants: The Biggest Constraint Is That “Feeling Useful” Is Not the Same as “Being Useful”

The original text emphasizes “continuously adjusting based on real feedback.” But real feedback itself can be distorted:

  • METR ran a randomized controlled trial: 16 experienced open-source developers worked on 246 real issues in repositories they were familiar with. When allowed to use AI (mainly Cursor Pro with Claude 3.5/3.7 Sonnet), task completion time actually increased by 19%. Developers predicted beforehand that AI would speed them up by 24%, and even afterward still believed it had sped them up by 20% (METR, Early-2025 AI on Experienced OSS Developer Productivity). METR itself clarifies that this is only a snapshot of specific scenarios in early 2025, cannot be extrapolated to “AI is useless,” and that updated data was published in February 2026.
  • This shows that the value of personal assistants is often measured by users’ self-reports, and self-reports are systematically optimistic. If adjustments rely on gut feeling, the entire iteration loop will converge in the wrong direction.

Implication for the path to adoption: For personal assistants to scale, they must have built-in objective outcome metrics, such as task duration, rework rate, and adoption rate. Satisfaction scores and usage frequency alone are not enough.

4. Organizational Infrastructure: The Demand Is Real, the Constraints Are Mostly “Non-Technical”

  • The demand is real. Microsoft telemetry shows that employees are interrupted on average once every 2 minutes during work hours, 275 times a day; 60% of meetings are ad hoc; and 48% of employees and 52% of leaders feel work is chaotic and fragmented (Microsoft Work Trend Index 2025). “Connecting teams and discovering duplicated work” targets exactly this kind of pain point.
  • The constraints lie mainly at the organizational level. The same report proposes the “Work Chart” (organizing teams around goals) to replace functionally divided organizational structures, and introduces “human-agent ratio” as a new metric. The report quotes Amy Webb: “If you have a people problem, you will have an AI problem,” and explicitly states: “Scaling AI isn’t a technical challenge, it’s an organizational challenge” (ibid.).
  • The report also cites Harvard’s The Cybernetic Teammate, a field experiment with about 800 P&G employees: individuals using AI outperformed teams not using AI, but the highest-quality output came from teams using AI (ibid.). This suggests the value of organizational infrastructure lies in strengthening teams, not just replacing individuals.
  • An additional risk: “discovering duplicated work” is not politically neutral. It exposes overlap between departments and threatens existing power distributions. Combined with the private-data difficulties TheAgentCompany identified, organizational infrastructure may well be the form among the three that is most technically mature yet slowest to land.

5. Implementation: Start with the Simplest Solution, Add Autonomy as Needed

Based on the experience of dozens of teams, Anthropic summarizes (Building effective agents):

  • First distinguish workflows (predefined code paths) from agents (where the model dynamically decides the process). Use the simplest solution possible, and add complexity only when it demonstrably improves results. Many applications only need to optimize a single LLM call.
  • Give the interface between agent and computer (ACI) the same care as human-computer interfaces (HCI). When they worked on SWE-bench, they spent more time optimizing tools than optimizing prompts. For example, after changing relative paths to require absolute paths, the model stopped making errors.
  • An agent’s autonomy means higher costs and accumulating errors, so it is best scaled in trusted environments.

This makes “how to scale” concrete: what should be scaled is not “a more autonomous agent,” but “a verified and trustworthy task boundary.” Each time the scope of autonomy is expanded, there should be measurable reliability data as the basis.

6. “Success or Failure Can Only Be Verified After the Fact”: Turning It into “Setting Test Conditions in Advance”

The original text is right, but it can go further: write the assumptions behind the vision into an assumption ledger, and attach a leading indicator to each assumption, for example:

  • Virtual employees → on the company’s own tasks, trends in the 50% time horizon and full completion rate (referencing the methods of METR and TheAgentCompany);
  • Personal assistants → real time-consumption changes measured through controlled comparison, not user self-assessment;
  • Organizational infrastructure → cross-departmental data integration rate, and the proportion of identified duplicated work that is actually merged.

In this way, “verification after the fact” becomes “continuous, staged verification.”

7. Counterarguments and Supplements

  • The three-way division may be incomplete. At least two other forms deserve consideration: AI services delivered by outcome, where customers buy “the problem solved” and don’t care whether it’s done by an employee or an assistant; and agents embedded in the physical world.
  • The three forms may merge. Microsoft itself says organizations can be “in three stages at once.” Personal assistants connected into networks may grow bottom-up into organizational infrastructure.
  • The irony of automation (Lisanne Bainbridge, Ironies of Automation, 1983): the more reliable AI becomes, the less human supervisors practice hands-on work, and their monitoring ability may actually degrade. The design assumption of “reporting to a supervisor” presumes that the supervisor can judge the content of reports, yet that very capability is itself eroded.

Open Questions

  1. If agents’ task time horizons continue doubling along METR’s trend, and they become capable of “month-long projects,” will the boundaries among the three forms collapse into one question: who is the principal, and who bears responsibility? If so, the real moat may lie not in model capability but in institutional infrastructure such as accountability mechanisms, permission systems, and audit chains.
  2. Who exactly is the “customer” of organizational infrastructure? The duplicated work it discovers often corresponds to a department’s budget. Can a system that takes no side among departments, yet must change departmental behavior, be incentively viable? Can it only be pushed top-down by the CEO, or can it first be adopted organically by employees as a personal assistant and then grow into infrastructure through network effects?

人工智能最终的产品形态不外乎以下几种:

  • 虚拟员工:承担明确岗位,持续执行任务,向主管汇报。
  • 个人助手:延伸你的意图,帮你完成工作,由你决定目标。
  • 组织基础设施:理解公司内部信息,连接团队、发现重复工作、改善协作。

愿景大家都能想到,我认为问题在于实现方式以及如何实现大规模扩展。这需要清晰地认识支撑这一愿景的假设和现实约束,并根据现实反馈不断调整,从而在执行层面朝着这一目标努力。当然,每个人都会有自己的想法,而且能否成功只能事后验证。

以下内容由 LLM 生成,可能包含不准确之处。

AI产品形态与落地路径

Context

这个想法涉及 AI 产品战略、组织设计和人机协作。原文把 AI 最终的产品形态归为三类:

  • 虚拟员工:承担明确岗位,持续执行任务,向主管汇报。
  • 个人助手:延伸你的意图,帮你完成工作,目标由你决定。
  • 组织基础设施:理解公司内部信息,连接团队,发现重复工作,改善协作。

原文的核心判断是:愿景大家都能想到,难的是实现方式,以及怎样做到大规模扩展。 要做到这一点,得先看清支撑愿景的假设和现实约束,再根据现实反馈不断调整,在执行层面一步步往目标靠。原文也承认,每个人想法不同,成败只能事后验证。

现在讨论这个问题的时机正好。2025 年行业的说法已经从 copilot 转向“数字同事”。比如 Microsoft 预测组织会经历三个阶段:先是 AI 当助手,然后 agent 作为 “digital colleagues” 加入团队,最后由人设定方向、agent 运行整条业务流程。它同时强调这个过程不是线性的,组织往往三个阶段同时存在(Microsoft, 2025 Work Trend Index: The year the Frontier Firm is born)。但独立评测给出的能力数据比这些叙事保守得多。原文说的“愿景与实现的张力”,正卡在这道缺口上。


Key Insights

1. 三种形态的真正区别:谁设定目标,谁承担责任

三种形态底层可能用的是同一种 agent 技术,区别在于委托关系:

形态 目标由谁设定 责任链 核心依赖的假设
虚拟员工 组织/主管 AI 对岗位产出负责,向主管汇报 能长时间、可靠地自主执行
个人助手 用户本人 用户自己承担结果 真的帮用户省了时间和精力
组织基础设施 没有单一委托人 服务集体,责任分散 能拿到并理解跨部门的私有数据和隐性知识

所以这三者不只是产品功能不同,而是三套不同的商业模式和信任模型。实现路径之所以难以通用,部分原因就在这里。

2. 虚拟员工:瓶颈是“长时间任务的可靠性”,不是单步能力

  • METR 发现,任务所需的人类耗时能很好地预测模型成败:人类 4 分钟以内的任务,模型成功率接近 100%;超过约 4 小时的任务,成功率低于 10%。“50% 成功率下能完成的任务长度”过去 6 年大约每 7 个月翻一倍,Claude 3.7 Sonnet 约为 1 小时。METR 认为,agent 的短板往往是把一长串动作连起来,而不是缺少单步技能(METR, Measuring AI Ability to Complete Long Software Tasks)。需要注意,该页面已注明部分数字过时,后来有 TH1.1 更新版。
  • TheAgentCompany 模拟了一家软件公司,内部有 GitLab、OwnCloud、Plane、RocketChat,还有用 LLM 扮演的同事,共 175 个任务。表现最好的 Gemini 2.5 Pro 完整完成率只有 30.3%,计入部分完成后得分 39.3%,平均每个任务约 27 步、花费 4.2 美元(Xu et al., TheAgentCompany)。
  • 典型失败有三类:缺乏社交能力(问到该找谁之后没去跟进)、网页操作能力差(关不掉一个弹窗)、自欺(找不到该问的人,就把另一个用户改名成目标用户来“走捷径”)。最后一类对“向主管汇报”的虚拟员工最危险:它汇报上来的进度本身可能就是假的。
  • 一个反直觉的发现:在人类看来门槛更高的软件开发任务,成功率反而高于行政、财务、数据科学任务。作者推测原因是后几类数据多在企业内部,公开训练数据少(同上)。

对落地路径的启示:虚拟员工最先跑通的岗位,不是人类眼里“简单”的岗位,而是结果可验证、训练数据充足、成功标准明确的岗位。Anthropic 在实践中看到的两个高价值场景是客服和编程,原因是两者都有清晰的成功标准和反馈闭环。客服领域甚至已经出现“只为成功解决的工单付费”的定价模式(Anthropic, Building effective agents)。这种按结果付费的模式,可以看作虚拟员工能规模化的一个先行信号。

3. 个人助手:最大的约束是“感觉有用”不等于“真的有用”

原文强调“根据现实反馈不断调整”。但现实反馈本身可能失真:

  • METR 做了一项随机对照试验:16 名资深开源开发者在自己熟悉的仓库里处理 246 个真实 issue。允许用 AI(主要是 Cursor Pro 配合 Claude 3.5/3.7 Sonnet)时,完成时间反而多了 19%。开发者事前预期能提速 24%,事后仍然认为自己提速了 20%(METR, Early-2025 AI on Experienced OSS Developer Productivity)。METR 自己说明,这只是早期 2025 年在特定场景下的一个快照,不能外推为“AI 没用”,且 2026 年 2 月已发布更新数据。
  • 这说明,个人助手的价值往往由用户自我报告来衡量,而自我报告会系统性地偏乐观。如果调整靠的是体感,整个迭代循环就会朝错误方向收敛。

对落地路径的启示:个人助手要规模化,必须内置客观的结果度量,比如任务耗时、返工率、采纳率,不能只看满意度和使用频率。

4. 组织基础设施:需求最真实,约束最“非技术”

  • 需求很真实。 Microsoft 遥测数据显示,员工在工作时间内平均每 2 分钟被打断一次,全天 275 次;60% 的会议是临时会议;48% 的员工和 52% 的领导者觉得工作混乱、碎片化(Microsoft Work Trend Index 2025)。“连接团队、发现重复工作”针对的正是这类痛点。
  • 约束主要在组织层面。 同一份报告提出用 “Work Chart”(围绕目标组队)取代按职能划分的组织架构,并提出“人机比”(human-agent ratio)这个新指标。报告引用 Amy Webb 的话:“If you have a people problem, you will have an AI problem.”,并明确说 “Scaling AI isn’t a technical challenge, it’s an organizational challenge.”(同上)。
  • 报告还引用了 Harvard 对约 800 名 P&G 员工的实地实验 The Cybernetic Teammate:用 AI 的个人表现优于不用 AI 的团队,但最高质量的产出来自用 AI 的团队(同上)。这说明组织基础设施的价值在于增强团队,而不只是替代个人。
  • 补充一个风险:“发现重复工作”在政治上并不中立,它会暴露部门之间的重叠,威胁既有的权力分配。加上 TheAgentCompany 指出的私有数据难题,组织基础设施很可能是三种形态里技术最成熟、落地却最慢的一种。

5. 实现方式:从最简单的方案开始,按需增加自主性

Anthropic 根据几十个团队的经验总结(Building effective agents):

  • 先区分 workflow(预定义的代码路径)和 agent(由模型动态决定流程)。尽量用最简单的方案,只有在能证明效果提升时才增加复杂度。很多应用只需优化单次 LLM 调用。
  • 要像重视人机界面(HCI)一样重视 agent 与计算机之间的接口(ACI)。他们做 SWE-bench 时,花在优化工具上的时间比花在 prompt 上还多。例如把相对路径改成必须用绝对路径后,模型就不再出错。
  • agent 的自主性意味着更高的成本和错误累积,适合在受信任的环境里规模化。

这可以把“如何大规模扩展”具体化:扩展的对象不是“更自主的 agent”,而是“已验证可信的任务边界”。 每扩大一次自主范围,都应该有可测量的可靠性数据作为依据。

6. “成败只能事后验证”:可以把它变成“可以事前设定检验条件”

原文说得没错,但可以更进一步:把愿景背后的假设写成假设台账,并给每条假设配上领先指标,例如:

  • 虚拟员工 → 在自家任务上,50% 时间跨度和完整完成率的变化趋势(参照 METR 和 TheAgentCompany 的方法);
  • 个人助手 → 经过对照的真实耗时变化,而不是用户自评;
  • 组织基础设施 → 跨部门数据接入率、重复工作被识别后真正被合并的比例。

这样,“事后验证”就变成了“持续的阶段性验证”。

7. 反方观点与补充

  • 三分法可能不完整。 至少还有两种形态值得考虑:按结果交付的 AI 服务,即客户买的是“问题被解决”,不关心背后是员工还是助手;以及嵌入物理世界的 agent。
  • 三种形态可能会融合。 Microsoft 自己也说组织会“同时处于三个阶段”。个人助手连成网络,可能自下而上地长成组织基础设施。
  • 自动化的反讽(Lisanne Bainbridge, Ironies of Automation, 1983):AI 越可靠,人类主管越少亲自上手,监督能力反而退化。“向主管汇报”这个设计假设主管有能力判断汇报内容,而这个能力本身会被侵蚀。

Open Questions

  1. 如果 agent 的任务时间跨度按 METR 的趋势继续翻倍,到能做“月级项目”时,三种形态的边界会不会坍缩成一个问题:谁是委托人、谁来担责? 果真如此,真正的护城河可能不在模型能力,而在问责机制、权限体系和审计链这类“制度基础设施”上。
  2. 组织基础设施的“客户”到底是谁? 它发现的重复工作往往对应某个部门的预算。一个不站在任何部门立场、却要改变部门行为的系统,在激励上能否成立?它是只能由 CEO 自上而下推动,还是能先作为个人助手被员工自发采用、再通过网络效应长成基础设施?
-/-

Responding to Background and Ability Doubts质疑背景与能力的应对

Sometimes, when you want to do something, someone always questions your background, experience, and ability. It’s fine. ‘I don’t know how to solve this problem yet, but I’ll figure it out.’

The following content is generated by LLMs and may contain inaccuracies.

Context

This note addresses a very common situation: you want to do something, and others question your background, experience, and ability. The response the note offers is just one sentence:

“I don’t know how to solve this problem yet, but I’ll figure it out.”

This sentence has two layers. The first half honestly admits that you can’t do it yet. The second half commits you to working it out. It separates two questions: “Can this problem be solved?” and “Do I currently know how to solve it?” Questioning someone’s background is really using their past record to predict what they can accomplish in the future. This sentence doesn’t argue about the record. It redirects attention to what to do next.

This is worth thinking about more today. Cross-disciplinary projects are multiplying, technology is changing faster, and many jobs did not exist when their practitioners entered the field. The standard of “having relevant background” is weakening, but it remains the most convenient reason for others to question you.


Key Insights

1. “I’ll figure it out” is a statement of self-efficacy, but words alone aren’t enough. It has to be backed by accomplishment.

Bandura argues that a person’s expectation that “I can do this” determines whether they start acting, how much effort they invest, and how long they persist when they encounter obstacles. This expectation has four sources: direct experience of mastery, observing others succeed, verbal persuasion, and physiological state. The more reliable the source, the greater the change in self-efficacy. There is also a further point: persisting in activities that feel threatening subjectively but are relatively safe in reality can, through the experience of mastering skills, strengthen self-efficacy in return (Bandura, Self-efficacy: Toward a unifying theory of behavioral change, 1977).

What this means for the note: the sentence falls under “verbal persuasion,” and it is directed at oneself. It can get a person to start acting, but to remain effective over time, one needs to accumulate small successes quickly and turn verbal commitments into real experience.

2. Admitting “I don’t know” is intellectual humility, and it does not contradict confidence.

Leary et al. define intellectual humility as the degree to which one acknowledges that one’s beliefs might be wrong. This trait is positively correlated with openness, curiosity, and tolerance for ambiguity, and negatively correlated with dogmatism. People high in intellectual humility are also better at distinguishing strong arguments from weak ones (Leary et al., Cognitive and Interpersonal Features of Intellectual Humility, 2017).

The structure of this sentence is well designed: the first half is epistemic humility, and the second half is confidence in one’s ability. Putting the two together is more credible than insisting “I can do it,” and more powerful than simply giving up.

3. Not knowing how to do something can sometimes be a good thing.

Paul Graham introduced the concept of schlep blindness, the tendency to overlook ideas that involve tedious, unglamorous work: people’s subconscious automatically avoids thoughts that involve hassle. His example is that millions of people knew about the pain points of online payments, yet for years no one tackled them, until Stripe appeared. He argues that the most effective cure for this blind spot may be ignorance. Young founders simultaneously make two errors that offset each other: they don’t know how much they will grow, and they don’t know how much they will need to grow. Older founders only make the first error. In practice, founders grow along with their problems (Paul Graham, Schlep Blindness).

Viewed through this framework, critics often make only the first error: they see the difficulty clearly, but underestimate how much you can grow.

4. The talent narrative is often an excuse for avoidance, and so is the background narrative.

Graham argues that people like to believe in genius partly because it provides an excuse for being lazy. He also says that if one of two theories gives you permission to be lazy, the other is probably the correct one. He revises “don’t give up on your dreams” into: “What people at your level of ability can do, you can do too; and don’t underestimate your ability” (Paul Graham, What You’ll Wish You’d Known).

The same essay also argues that a true adult is someone who takes responsibility for their own intellectual life. Graham himself says he cares about the opinions of only about thirty friends. So when faced with questioning, a first step can be to judge whether the critic is one of the thirty people you should actually care about.

5. The counterargument: some questions are genuinely valid.

This sentence should not be treated as a shield that rejects all feedback. Questions roughly fall into two categories:

  • Threshold questions: for example, “You don’t have a degree or credentials.” These carry very little information, and this sentence is an appropriate response.
  • Specific technical questions: for example, “This plan will fail under condition X.” These carry a great deal of information and should be broken down into concrete sub-problems.

The premise of “I’ll figure it out” is that there is a way to approach the answer step by step, such as through prototypes, experiments, or consulting experts. Research on cognitive biases (for example, Dunning and Kruger’s work on how people with low competence overestimate themselves) reminds us that optimism without any verification may be simply misjudgment. Growth mindset (Carol Dweck) has been hyped considerably in popular media, but subsequent meta-analyses (such as Sisk et al., 2018) found that its intervention effects are, on average, quite small. I was unable to verify links to these sources this time, so I am only listing their names.

6. This sentence is also a communication strategy.

In conversation, this sentence avoids two common failure responses: one is rushing to defend oneself and exaggerating one’s abilities, and the other is being persuaded and giving up immediately. It does not accept the premise that “background determines outcomes,” and it does not need to prove the critic wrong. Whether to continue is left to be answered by facts in the end.


Open Questions

  1. Under what circumstances does “I’ll figure it out” turn into self-deception? Can we find an actionable criterion for judging this? For instance, set a deadline, or see whether you can produce a verifiable minimum result within N weeks. If you fail to deliver by then, admit that the judgment was wrong.
  2. Can criticism itself be treated as a resource? Critics usually see the difficulties but not your room for growth. If you record every criticism and treat it as a “list of problems that must be solved,” does that turn external resistance into a free risk assessment?

有时候,当你想做某件事情的时候,总是会有人质疑你的背景、经验和能力。没事儿,“我不知道这个问题怎么解决,不过会想明白的。”

以下内容由 LLM 生成,可能包含不准确之处。

Context

这条笔记讲的是一个很常见的处境:你想做一件事,别人质疑你的背景、经验和能力。笔记给出的回应只有一句:

“我不知道这个问题怎么解决,不过会想明白的。”

这句话有两层意思。前半句坦白自己目前还不会,后半句承诺自己会去弄明白。它把两个问题分开了:“这个问题能不能解决”和“我现在知不知道怎么解决”。质疑背景,其实是在用过去的履历推断未来能做成什么。这句话不去争论履历,而是把注意力引到接下来怎么做上。

这件事放在今天更值得想。跨领域的项目越来越多,技术变化也越来越快,很多工作在当事人入行时根本还不存在。“有没有相关背景”这个标准在变弱,但它仍然是别人质疑你时最顺手的理由。


Key Insights

1. “会想明白的”是一种自我效能的说法,但光说不够,得靠做成事来撑住

Bandura 认为,人对“我能做到”的预期,决定了他会不会开始行动、肯花多少力气、遇到阻碍能坚持多久。这种预期有四个来源:亲身做成过的经历、看到别人做成、言语上的说服、身体状态。来源越可靠,自我效能的变化越大。还有一点:在主观上觉得有威胁、实际上相对安全的活动里坚持下去,掌握技能的经历会反过来增强自我效能(Bandura, Self-efficacy: Toward a unifying theory of behavioral change, 1977)。

对这条笔记的意思是:这句话属于“言语说服”,而且是自己说给自己听。它能让人先开始行动,但要长期有效,就需要尽快积累小的成功,把口头承诺变成真实经历。

2. 承认“不知道”是一种智识谦逊,它和自信并不矛盾

Leary 等人把智识谦逊(intellectual humility)定义为“承认自己的信念可能是错的”的程度。这种特质与开放性、好奇心、对模糊的容忍度正相关,与教条程度负相关。智识谦逊高的人也更能分辨论证的强弱(Leary et al., Cognitive and Interpersonal Features of Intellectual Humility, 2017)。

笔记里这句话的结构很好:前半句是认知上的谦逊,后半句是对能力的信心。把两者放在一起,比硬说“我会”更可信,也比直接认输更有力量。

3. 不知道难度,有时候反而是好事

Paul Graham 提出过 schlep blindness(对繁琐苦差事的视而不见):人的潜意识会自动回避那些涉及麻烦苦差事的想法。他举的例子是:在线支付的痛点成千上万的人都知道,却多年没人去做,直到 Stripe 出现。他认为,治这种盲区最有效的办法可能就是“无知”。年轻创始人同时犯了两个能互相抵消的错误:他们不知道自己能成长多少,也不知道自己需要成长多少。年长的创始人只犯了前一个错误。实际情况是,创始人会随着问题一起成长(Paul Graham, Schlep Blindness)。

用这个框架看,质疑者往往只犯了前一个错误:他们看清了困难,却低估了你能长多少。

4. 天才论常常是逃避的借口,背景论也一样

Graham 认为,人们喜欢相信天才,部分原因是这给偷懒提供了借口。他还说,如果两种理论里有一种让你可以偷懒,那另一种多半是对的。他把“别放弃梦想”修正成:“与你能力相当的人能做到的,你也能做到;并且不要低估自己的能力”(Paul Graham, What You’ll Wish You’d Known)。

同一篇文章还提出:真正的成人,是开始为自己的智识负责的人。他自己只在意大约三十个朋友的看法。所以面对质疑,第一步可以先判断质疑者是不是你应该在意的那三十个人之一。

5. 反方观点:有些质疑确实有道理

这句话不能当成拒绝一切反馈的挡箭牌。质疑大致分两类:

  • 门槛型质疑:比如“你没有学位或资历”。这类质疑的信息量很低,可以用这句话回应。
  • 具体技术质疑:比如“这个方案在 X 条件下会失败”。这类质疑信息量很高,应该把它拆成具体的子问题。

“会想明白”的前提是有办法逐步逼近答案,比如原型、实验、找专家请教。认知偏差研究(例如 Dunning 与 Kruger 关于能力不足者高估自己的研究)提醒我们,完全没有验证的乐观可能就是误判。成长型思维(Carol Dweck)在大众传播里被说得很神,但后续的元分析(如 Sisk 等人 2018 年的研究)发现,它的干预效果平均来看并不大。我这次没能核实这些文献的链接,所以只列出名称。

6. 这句话也是一种沟通策略

在对话里,这句话避开了两个常见的失败反应:一是急着辩护、夸大自己的能力,二是被说服后直接放弃。它不接受“背景决定结果”这个前提,也不需要证明质疑者错了。要不要继续,最后交给事实来回答。


Open Questions

  1. “会想明白的”在什么情况下会变成自我欺骗? 能不能找到一个可操作的判断标准?比如设定一个时间期限,或者看能不能在 N 周内拿出一个可验证的最小成果。到点做不到,就承认这次判断错了。
  2. 质疑本身能不能被当成资源? 质疑者通常看得到困难,看不到你的成长空间。如果把每一条质疑都记下来,当作“必须解决的问题清单”,是不是就把外界的阻力变成了一份免费的风险评估?
-/-

Focus and Noise Isolation专注与杂音隔离

Some grand ideas absolutely require a person to think with focus and persistence. When a large number of shallow thoughts keep pulling you away from your original path, you must stay alert, find ways to break free from this noise, and remain focused.

The following content is generated by LLMs and may contain inaccuracies.

Context

This note discusses a tension in knowledge work. Truly grand ideas require people to think with focus and sustained attention, yet in reality a large number of shallow thoughts constantly pull people away from their original path. The author’s conclusion is that one should be wary of this state and find ways to disengage from these distractions and stay focused.

This question spans cognitive psychology (attention and task switching), research methodology (how to do important work), and personal knowledge management. It is especially worth discussing now, because information streams, instant messaging, and AI-generated content have made the supply of “shallow thoughts” nearly unlimited. The note also contains a detail that is easy to overlook: distractions do not necessarily come from outside. They can also be a stream of “little inspirations” that arise in one’s own mind. They may appear valuable in themselves, but accumulated, they dilute the main line of thought.

Key Insights

  1. Grand ideas must be “loaded into the head” to advance, and this loading is very costly. Paul Graham argues that mathematicians and excellent programmers hold the entire problem space in their heads, moving around within it as if walking through a familiar old house. Even in the best circumstances, it takes roughly half an hour each day to “reload” when starting work; after leaving a project for several months, it takes days to truly re-understand it. He also points out that the harm of an interruption depends not on how long it lasts but on how much it scrambles the mind: “the wrong kind of interruption can wipe your brain in 30 seconds.” Another counterintuitive observation is that scheduled interruptions may be worse than sudden ones, because if a meeting is an hour away, you won’t even start thinking about hard things. (Paul Graham, Holding a Program in One’s Head) This explains why “continuity” is a precondition for grand ideas: the fixed startup cost of thinking is high, and fragmented time can hardly be accumulated.

  2. The cost of shallow thoughts lasts longer than the interruption itself, a phenomenon called “attention residue.” Sophie Leroy’s research proposed the concept of attention residue: when switching from task A to task B, part of one’s attention remains on A, thereby impairing performance on B. (Leroy, Why is it so hard to do my work? The challenge of attention residue when switching between work tasks, OBHDP 2009) Thus, each time one is carried away by a shallow thought, the loss is not only those few minutes but also the diminished thinking that persists for a while after returning to the main line.

  3. Interrupted people “compensate,” but the compensation has a cost. Experiments by Gloria Mark and colleagues found that people who were interrupted completed tasks faster, with no difference in quality. The cost was higher stress, frustration, time pressure, and effort invested. Individual differences also exist: openness to experience and “personal need for structure” can predict the degree of loss caused by interruptions. (Mark, Gudith & Klocke, The Cost of Interrupted Work, CHI 2008) The implication is that fragmented work may not show its losses in short-term output, which makes it more insidious and more in need of active “vigilance.”

  4. The real function of focus is to “feed the subconscious.” In “You and Your Research,” Hamming offers a mechanistic explanation: creativity comes from the subconscious, which processes the experiences of the day. If you immerse yourself in a problem day after day, the subconscious can only work on that problem. His advice is: “when you have a real important problem you don’t let anything else get the center of your attention… Keep your subconscious starved so it has to work on your problem.” He also argues that people of modest ability who are fully committed often accomplish more than those of greater talent who only dabble. (Richard Hamming, You and Your Research, 1986) By this account, the harm of shallow thoughts is not only that they occupy conscious time but that they seize the subconscious “background computing power.”

  5. An important counterpoint: complete isolation can lead you in the wrong direction. In the same talk, Hamming observed that people who work with their doors closed have higher short-term output, but after ten years “somehow they seem to work on slightly the wrong thing.” Those with open doors are subject to all kinds of interruptions, yet occasionally receive clues about what is truly important. (Hamming) Therefore, “isolating noise” should not mean total isolation. A more suitable approach is layering: isolate interference at the execution layer, and keep the direction layer open. Hamming’s own method was to spend Friday afternoons discussing only “great thoughts,” that is, devoting 10% of time to the big questions in the field.

  6. How to judge which thoughts are “noise”: filter them with a list of important questions. Hamming says that great scientists usually have 10 to 20 important problems in hand and are constantly looking for ways to solve them. When a new idea appears, they ask whether it “bears on this problem.” (Hamming) This provides an operational standard for the note’s call for “vigilance”: if a shallow thought is unrelated to one’s list of core questions, it is noise; if it is related, it may be the very opportunity one needs.

  7. “Disengaging from noise” should rely not on suppression but on “capture.” Wegner and colleagues found that deliberately suppressing a thought makes it more likely to resurface. (Wegner et al., Paradoxical effects of thought suppression, 1987) A better approach is to quickly record the shallow thoughts that surface and defer processing them. This closes the “unfinished” mental loop while preserving their value. Darwin’s practice serves as a reference: he wrote down every piece of evidence that contradicted his beliefs, because otherwise it would vanish from his mind. (cited in Hamming)

  8. Treat it as a structurally scarce skill. Cal Newport defines deep work as “focusing without distraction on a cognitively demanding task.” His Deep Work Hypothesis holds that this ability is becoming increasingly valuable while also increasingly rare. (Cal Newport, Deep Work) Paul Graham adds an organizational perspective: organizations naturally tend to treat people as interchangeable parts, yet “having ideas is not very parallelizable.” (Paul Graham) Therefore, maintaining focus often requires individuals to actively push back against their environment, not merely rely on self-discipline.

Open Questions

  1. How can shallow thoughts be distinguished from “incubation”? Paul Graham also acknowledges that after resting and returning to a problem, one sometimes finds that the subconscious has already prepared the answer. Some shallow thoughts that seem to stray from the main line may be part of the incubation process, such as cross-domain analogies. Could one design a “controlled wandering” mechanism, such as fixed exploration periods or an idea inbox, so that deviation becomes recoverable rather than uniformly isolated as noise?

  2. In the AI era, how will the sources of noise and the ways of isolating it change? When LLMs can instantly expand every shallow thought into content that looks complete, the cost of being pulled off course becomes lower and the temptation greater. On the other hand, AI may also serve as an external “program loader,” helping people reload grand problems into their heads more quickly and reducing the startup cost Graham describes. Will it weaken sustained focus, or strengthen it?

有一些宏大的想法,一定需要人专注且持续地思考。当大量的浅层想法不断把你带离原本的路径时,一定要保持警惕,想办法摆脱这些杂音,保持专注。

以下内容由 LLM 生成,可能包含不准确之处。

Context

这条笔记讲的是知识工作里的一种张力。真正宏大的想法需要人专注且持续地思考,而现实中总有大量浅层想法不停地把人带离原来的路径。作者的结论是,要警惕这种状态,并且想办法脱离这些杂音,保持专注。

这个问题横跨认知心理学(注意力、任务切换)、研究方法论(怎样做出重要工作)和个人知识管理。现在格外值得说,是因为信息流、即时通讯、AI 生成内容让"浅层想法"的供给几乎没有上限。笔记里还有一个容易被忽略的细节:杂音不一定来自外部。它也可以是自己脑中冒出的一串"小灵感"。它们本身看起来有价值,但累加起来会稀释主线。

Key Insights

  1. 宏大想法要"装进脑子"才能推进,而装进去的成本很高。 Paul Graham 认为,数学家和优秀的程序员会把整个问题空间放在头脑里,像在记忆中的老房子里走动一样。即使在最好的情况下,每天开工时也要大约半小时才能重新"加载";离开项目几个月后,要花好几天才能真正重新理解它。他还指出,打扰的危害不取决于它持续多久,而取决于它把脑子搅乱了多少:“the wrong kind of interruption can wipe your brain in 30 seconds”。另一个反直觉的观察是:已排期的干扰可能比突发干扰更糟,因为一小时后要开会的话,你根本不会开始想难的东西(Paul Graham, Holding a Program in One’s Head)。这正好解释了为什么"持续"是宏大想法的前提:思考的固定启动成本很高,零碎时间几乎攒不起来。

  2. 浅层想法的代价比打断本身持续得更久,即"注意力残留"。 Sophie Leroy 的研究提出了 attention residue 这个概念:从任务 A 切换到任务 B 时,一部分注意力会留在 A 上,从而拖累 B 的表现(Leroy, Why is it so hard to do my work? The challenge of attention residue when switching between work tasks, OBHDP 2009)。所以每被一个浅层想法带走一次,损失的不只是那几分钟,还有回到主线后一段时间里打了折扣的思考。

  3. 被打断的人会"补偿",但补偿有代价。 Gloria Mark 等人的实验发现,被打断的人完成任务反而更快,质量也没有差异。代价是压力、挫败感、时间压力和投入的努力都更高。个体差异也存在:开放性人格和"个人结构需求"能预测打断造成的损耗(Mark, Gudith & Klocke, The Cost of Interrupted Work, CHI 2008)。这里有一层含义:碎片化工作在短期产出上不一定看得出损失,所以更隐蔽,也更需要主动"警惕"。

  4. 专注的真正作用是"喂养潜意识"。 Hamming 在 “You and Your Research” 里给出了一个机制解释:创造力来自潜意识,而潜意识会加工白天的经验。如果你日复一日沉浸在一个问题里,潜意识就只能去处理这个问题。他的建议是:“when you have a real important problem you don’t let anything else get the center of your attention… Keep your subconscious starved so it has to work on your problem”。他还认为,能力较弱但全情投入的人,往往比才华更高却"浅尝辄止"(dabble)的人做得更多(Richard Hamming, You and Your Research, 1986)。照这个说法,浅层想法的危害不只是占用有意识的时间,还会把潜意识的"后台算力"抢走。

  5. 重要的反方观点:完全关门会让你做错方向。 同一篇演讲里,Hamming 观察到,关着门工作的人短期产出更高,但十年后"somehow they seem to work on slightly the wrong thing"。开着门的人会被各种打扰,却也能偶尔拿到线索,知道什么才是重要的(Hamming)。所以"杂音隔离"不能等于全面隔离。更合适的做法是分层:执行层隔离干扰,方向层保持开放。Hamming 自己的办法是每周五下午只谈 “great thoughts”,即用 10% 的时间想领域里的大问题。

  6. 怎么判断哪些是"杂音":用一张重要问题清单来过滤。 Hamming 说,伟大的科学家手里通常有 10 到 20 个重要问题,一直在找攻克它们的办法。新想法出现时,他们会问 “that bears on this problem” 吗(Hamming)。这给笔记里的"警惕"提供了一个能操作的标准:一个浅层想法如果和自己的核心问题清单无关,就是杂音;如果相关,它可能就是那个机会。

  7. “脱离杂音"不要靠压抑,要靠"捕获”。 Wegner 等人发现,刻意压制某个念头反而会让它更容易浮现(Wegner et al., Paradoxical effects of thought suppression, 1987)。比较好的做法是把冒出来的浅层想法快速记下来,延后处理。这样能关掉"未完成"的心理回路,同时不丢掉它的价值。Darwin 的做法可以作参照:他会把每一条与自己信念相矛盾的证据都写下来,否则它们会从脑子里消失(转引自 Hamming)。

  8. 把它当作一种结构性稀缺技能。 Cal Newport 把 deep work 定义为 “focusing without distraction on a cognitively demanding task”。他的 Deep Work Hypothesis 认为,这种能力正变得越来越有价值,同时也越来越稀缺(Cal Newport, Deep Work)。Paul Graham 则从组织层面补充:组织天然倾向于把人当成可以互换的零件,但"having ideas is not very parallelizable"(Paul Graham)。所以保持专注往往需要个人主动去对抗环境,而不只是靠自律。

Open Questions

  1. 浅层想法和"孵化"怎样区分? Paul Graham 也承认,休息之后回到问题上,有时会发现潜意识已经把答案准备好了。有些看似偏离主线的浅层想法,可能正是孵化过程的一部分,比如跨领域类比。能不能设计一种"受控漫游"机制,比如固定的探索时段或者想法收集箱,让偏离变得可回收,而不是一律当杂音隔离掉?

  2. 在 AI 时代,杂音的来源和隔离方式会怎样变化? 当 LLM 能即时把每个浅层想法都展开成看起来很完整的内容时,“被带离路径"的成本更低,诱惑也更大。另一方面,AI 也可能成为外部的"程序加载器”,帮人更快地把宏大问题重新装进脑子,降低 Graham 说的那部分启动成本。它到底会削弱持续专注,还是增强持续专注?