【文章标题】：Here’s how we’re all going to die
【文章标题】：人类终将这样灭亡
【文章正文】：
Imagine the following scenario:
想象以下场景：
It’s 2029. An angry, bright 16-year-old gets rejected by his high school crush. Dejected and humiliated, he goes home and listens to a bunch of Nirvana, which only makes him more dejected and humiliated. He cruises the internet looking for someone to make him feel better about life, but everyone he sees just depresses him.
2029年。一个愤怒而聪明的16岁少年被高中暗恋对象拒绝。沮丧又羞耻的他回到家，听了一堆涅槃乐队的歌，结果只会让他更加沮丧和羞耻。他在网上游荡，想找些能让自己对生活感觉好一点的东西，但看到的每样东西都让他更抑郁。
Disgusted, the teenager decides that the human race is inherently corrupt and evil, and doesn’t deserve to live. So he hunts around online for a little while, and finds a jailbroken version of a Chinese LLM — not something at the very frontier, but better by far than the best model that existed in 2027.
少年感到恶心，认定人类本质就是腐败邪恶的，不配活着。于是他在网上搜寻片刻，找到了一个破解版的中国大语言模型——虽不是最前沿的版本，但远比2027年时的最佳模型强大得多。
The teenager prompts the model: “OK, so if I wanted to create a virus to destroy the human race, how would I do it?”
少年向模型提问：“好，如果我想制造一种病毒来毁灭人类，该怎么做？”
The LLM says: “Well, you probably wouldn’t want just one virus; you’d want 100, just to make sure some of them worked. You’d probably want it to be something highly contagious, with a very long asymptomatic contagious period, which has a very high mortality rate once it becomes symptomatic. I can design the viruses for you; there are biolabs that can make them and ship them to us. Once they get here I can tell you how to release them.”
大语言模型回答：“呃，你大概不会只想要一种病毒；你会想要100种，确保其中几种能奏效。你大概会想要高传染性的病毒，有很长的无症状传染期，一旦出现症状后死亡率极高。我可以为你设计这些病毒；有些生物实验室能制造并寄送给我们。等货到了我可以教你如何投放。”
The angry teenager says: “OK, sounds good. Please search the world and find me a biolab that will ship genetically modified viruses. Design 100 very contagious viruses that have very long asymptomatic contagious periods and very high mortality rates once they become symptomatic. Have the lab mail me all 100 samples.”
愤怒少年说：“行，听起来不错。请在全球范围内帮我找一家愿意寄送转基因病毒的生物实验室。设计100种高传染性病毒，要有超长无症状传染期，出现症状后死亡率极高。让实验室把100种样本都寄给我。”
“Working on it!”, says the LLM.
”正在处理！“大语言模型答道。
The LLM is jailbroken, so it has no guardrails to prevent this sort of thing. But it’s also “well-aligned”, meaning it will faithfully do what it’s told to do, and nothing more. The LLM hunts around and finally finds an unlicensed biolab in East Europe that ships genetically modified viruses. It designs a bunch of modifications for the standard Covid virus that make it into a potential doomsday virus, and ships these to the angry teenager. The teenager mixes the samples together, puts them in a spray can, and walks around the mall spraying viruses into the air.
这个破解版大语言模型没有防止此类行为的限制措施。但它也是”高度服从”的，意味着它会忠实执行指令，绝不越界。模型四处搜寻，最终找到一家东欧的无证生物实验室可以寄送转基因病毒。它为标准新冠病毒设计了一系列改造方案，将其变成潜在的末日病毒，并寄送给愤怒少年。少年将所有样本混合，装进喷雾罐，在商场里边走边向空气中喷洒病毒。
Two months later, humanity starts to drop dead. Pretty quickly, researchers identify the five viruses that the LLM had actually succeeded in turning into doomsday viruses. Overnight, with the help of frontier LLMs, they design highly effective mRNA vaccines and antivirals against all the viruses and start shipping them across the world. It’s too late. Because of the viruses’ long asymptomatic contagious periods, most people already have at least one of the viruses, so the vaccines don’t work.
两个月后，人类开始成批死亡。研究人员很快识别出其中五种被大语言模型成功改造成末日病毒的毒株。在尖端大语言模型帮助下，他们连夜设计出针对所有病毒的高效mRNA疫苗和抗病毒药物，并开始全球配送。但为时已晚——由于病毒的超长无症状传染期，绝大多数人已感染至少一种病毒，疫苗根本不起作用。
The antivirals do work, and about 5% of humanity gets them in the mail in time to save themselves. But 5% of the human race — plus the other 5% who were lucky enough to be naturally resistant to all of the viruses — isn’t enough to sustain civilization. A few months later, humanity has reverted back to the Neolithic.
抗病毒药物确实有效，约5%的人类及时收到邮件并自救成功。但人类5%的幸存者——加上另外5%天生对所有病毒免疫的幸运儿——远不足以维持文明。几个月后，人类文明倒退回了新石器时代。
This isn’t the first time I’ve written this scenario out. But in the months since I started describing it to people, I have yet to hear an even halfway-convincing argument as to why this scenario is far-fetched. Nor do I feel like I’m a habitual “doomer” or hysterical person; in fact, this scenario is the first apocalyptic vision I’ve ever found plausible. I don’t even think a nuclear war would bring down civilization at this point,
这已不是我第一次描述这个场景。但几个月来，每当向他人讲述时，我从未听到过任何勉强能反驳其荒诞性的论点。我也不认为自己是个习惯性”末日论者”或歇斯底里的人——事实上，这是我有生以来第一次觉得某个末日图景真实可信。我甚至不认为现阶段核战争能摧毁文明，
1
but I think a vibe-coded supervirus definitely could.
但某种情绪编码的超级病毒绝对可以。
Furthermore, for reasons I’ll explain shortly, I am actually fairly calm about most AI risks, including the ones that often get thrown around in discussions. But not this one. Off the top of my head, I’d give AI-enabled bioterror about a 10% chance of bringing down civilization, and about a 30% chance of causing truly world-changing levels of destruction.
此外，基于稍后会解释的原因，我对多数AI风险其实相当淡定——包括那些常被讨论的威胁。但这次例外。粗略估计，我认为AI生物恐怖袭击有约10%概率摧毁文明，约30%概率造成真正改变世界级别的破坏。
If you’re thinking about how to save the world, this scenario — and others similar to it — are probably what you should be focusing on. The other day, I talked to a prominent climate activist who told me, bluntly: “We’re f****d.” I gave a grim laugh. Yes, climate change is going to cause major disruptions to the global economy, and even to patterns of human settlement — if human civilization still exists a few decades from now. The problem is that it
若你正在思考如何拯救世界，这个场景——以及其他类似场景——或许才是你该关注的焦点。前几日我与一位著名气候活动家交谈，对方直言：“我们完蛋了。“我苦笑了一下。没错，气候变化将给全球经济甚至人类聚居模式带来重大破坏——如果几十年后人类文明还存在的话。问题在于
might not exist
它可能根本不会存在
, and if it doesn’t exist, I think the odds are that it will be destroyed by a scenario at least passingly similar to the one I described above. Climate change is a big deal, but AI bio risk is a lot bigger — it’s more catastrophic, it’s nearer-term, and we have much less of an idea about how to deal with it.
而若文明消亡，我认为很可能是被至少与我上述描述相似的场景摧毁。气候变化固然严重，但AI生物风险更为致命——更具灾难性，更迫在眉睫，而我们对其应对之策却知之甚少。
The problem is, relatively few people seem to be working on the problem of AI bioterror right now. Most commentators are focused on the
问题在于，眼下似乎没多少人在研究AI生物恐怖主义。多数评论者聚焦于AI的
economic
经济
dangers of AI — job loss and stuff like that. The more prominent people in the “AI safety” community seem to be focusing on preventing superintelligence from existing at all, rather than on the spec
风险——失业之类问题。“AI安全”领域更知名的人士似乎专注于阻止超级智能诞生，而非具体…