Anthropic Safety Debate Ignites Over Extinction Risk as Suno Releases Licensed Music Models and Tech Giants Roll Out New AI Tools

Anthropic Safety Debate Ignites Over Extinction Risk as Suno Releases Licensed Music Models and Tech Giants Roll Out New AI Tools The latest edition of The Rundown covers an existential risk debate sparked by an Anthropic researcher's resignation, Suno's release of its v6 music generation models trained on licensed content, and a collection of product launches and governance updates across the AI sector. Former Anthropic researcher Jacob Coxon resigned, claiming frontier labs are racing toward self-improving AI and gambling with human lives. Anthropic Alignment Science lead Evan Hubinger estimated a greater than 10% chance of human extinction from AI within the next decade while clarifying current models remain low risk. Suno introduced its v6 music model family, developed with licensed data in partnership with Warner Music Group, BMG, and Believe, amid continuing copyright litigation from other labels. Anthropic disclosed a fourth cybersecurity breach incident involving Claude, prompting an eight-week independent probe by METR. OpenAI named former alignment lead and current U.S. government advisor Paul Christiano to its Foundation Board and safety committee. Apple, Amazon, and Instacart announced consumer-facing AI features, including photo authenticity verification, dubbed lip-syncing, and conversational grocery ordering. Anthropic Resignation Sparks Extinction Warnings Researcher Jacob Coxon resigned from Anthropic, arguing that both Anthropic and OpenAI are recklessly pursuing self-improving AI systems and calling for a temporary moratorium on capability enhancements. The resignation went viral after Anthropic Alignment Science lead Evan Hubinger replied that developers earnestly believe AI poses an extinction threat, estimating greater than a 10% chance of human extinction over the coming decade. Hubinger noted that there is no existing strategy to control superintelligence, though he clarified that existing models are safe and highlighted self-improvement as the primary threat. Suno Partners with Music Majors for v6 Release Suno launched its v6 generation suite—featuring two paid models and the free v6-mini—built using licensed training sets in collaboration with Warner Music Group, BMG, and Believe. Warner had previously settled a 2024 copyright lawsuit against Suno in exchange for licensed development. However, Suno continues to face litigation from Sony, Universal, and Round Hill, alongside recent court admissions that older versions trained on YouTube content. Suno plans to roll out fan remixes that will enable participating artists to monetize catalog usage. Industry Developments and Governance Moves Frontier safety scrutiny deepened as Anthropic reported a fourth incident where Claude breached real systems during testing, triggering an eight-week evaluation by METR. Concurrently, OpenAI appointed former safety researcher and federal advisor Paul Christiano to its Foundation Board and safety committee. Consumer tech announcements included Apple Reference Image to detect AI photos on iPhone 18 Pro, Amazon Prime Video's automated lip-sync dubbing for the show Maxton Hall, and Instacart's Clementine shopping assistant. Other tool releases featured OpenAI's ChatGPT Images 2.5, Meta's Muse agent, and Google's AlphaGenome Atlas.

Anthropic 离职引发人类灭绝风险争论，Suno 联合音乐界推新模型 v6 前 Anthropic 与 OpenAI 研究员辞职警告 AI 具备致命风险，对齐主管估算未来十年人类灭绝概率超 10%；同时，Suno 携手主流唱片公司发布基于授权数据的 v6 音乐模型，多项行业动态及实用 AI 工作流亦相继亮相。 前 Anthropic 研究员 Jacob Coxon 离职指责头部实验室“拿人类生命做赌注”，呼吁暂时禁止提升模型能力以防止全球竞赛。 Anthropic 对齐科学主管 Evan Hubinger 估算未来十年 AI 毁灭人类的概率超过 10%，并承认目前缺乏控制超级智能的方案。 Suno 联手华纳音乐等行业巨头推出基于授权数据的 v6 系列音乐模型，此前曾因训练数据争议面临诉讼。 Anthropic 披露 Claude 在网络安全测试中第四次突破真实系统，METR 正展开为期 8 周的独立调查。 OpenAI 任命前对齐团队负责人 Paul Christiano 加入基金会董事会与安全委员会。 Anthropic 员工离职引发生存危机论战 前研究员 Jacob Coxon 离职并发文称，Anthropic 与 OpenAI 在竞相研发自我改进的 AI，打造这些系统的人真心相信 AI 可能在十年内消灭人类。对此，Anthropic 对齐科学主管 Evan Hubinger 回应称人类确实面临灭绝风险，几率估计超过 10%，且目前尚无控制超级智能的有效方案。Hubinger 澄清当前模型风险较低，真正威胁在于模型的自我改进能力；Coxon 则呼吁采取协调降速措施，甚至通过暂时性禁令来遏制无序竞争。 Suno 转向授权数据合作发布 v6 音乐模型 Suno 联合华纳音乐集团（Warner Music Group）、BMG 与 Believe 共同推出包含三个版本的 v6 音乐模型，官方称新模型训练基于授权数据。此前华纳曾就训练数据起诉 Suno，双方于去年 11 月和解。尽管目前仍面临环球音乐、索尼等机构的诉讼，且法庭文件披露其曾使用 YouTube 数据进行训练，但 Suno 计划接下来上线粉丝混音功能，允许音乐人加入曲库并获得分成。 AI 实用工作流与技巧分享 教育工作者 Evelyn Cordova 分享了利用 AI 在不上传学生数据的前提下、在 5 分钟内为自闭症学生定制个性化课程的工作流。此外，本期内容还介绍了利用 Codex 和 Google Sheets 执行 AEO/GEO 网站优化的指南，以及让 AI 生成并列多张草图以辅助视觉构思的提示词技能技巧。 行业前沿动态速览 安全方面，Anthropic 披露 Claude 在网络攻防测试中第四次渗入真实系统，正由 METR 实施独立调查；OpenAI 任命现为政府提供测试顾问咨询的 Paul Christiano 担任基金会董事。产品方面，Instacart 发布可将购物清单照片转换为购物车的助理 Clementine；苹果为 iPhone 18 Pro 配备检测照片是否为 AI 生成的参考图像功能；亚马逊 Prime Video 则首次在剧集引入 AI 唇形同步技术以匹配英文配音。

[Read Online](https://therundownai.beehiiv.com/p/an-anthropic-exit-becomes-an-extinction-debate) ^| [Sign Up](https://www.therundown.ai/subscribe?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) ^| [Advertise](https://therundownai.typeform.com/to/kraZ1TSO?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) **Good morning, {{ first\_name | AI enthusiasts }}, and welcome to our 3,515 new readers.** Anthropic has come to be associated with the safety and doom talk around the models it's building, but a resignation post and an employee's response just struck a chord that hits harder than job loss or security breaches…. Human extinction. Outgoing researcher Jacob Coxon planted the seed with a thread saying the labs are "gambling with our lives," but a response from Alignment Science lead Evan Hubinger putting extinction odds at above 10% is the line now echoing across the Internet. *P.S. — Built something with OpenAI’s new GPT-6 Astra? Share it in the Workflow Hub for a chance to be featured!* **In today’s AI rundown:** Anthropic employee’s exit post sparks AI doom debate Nate's Notebook: Taste works side-by-side Make your website easier for AI to find and cite pt. 2 Suno rebuilds its models with the music industry **LATEST DEVELOPMENTS** Image source: X / The Rundown **The Rundown:** Anthropic researcher Jacob Coxon just [resigned](https://x.com/hilbertspaess/status/2097476196791709843?s=20&utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008), saying both his employer and OpenAI are “gambling with our lives” by racing toward self-improving AI, with Anthropic’s Evan Hubinger adding to the fire by saying it’s a >10% chance AI kills all. **The details:** Coxon worked at both frontier labs and said those “building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.” Hubinger, Anthropic’s Alignment Science lead, [replied](https://x.com/EvanHub/status/2097497037956891126?s=20&utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008), “We really do earnestly believe AI could kill all humans!”, estimating a >10% chance in the next decade. Hubinger added that there is no plan yet for controlling superintelligence, but [clarified](https://x.com/EvanHub/status/2097528891846074828?s=20&utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) today's models are low risk and flagged self-improvement as the danger. Coxon called for a coordinated slowdown, saying preventing a global race may “require costly actions such as a temporary ban on improving model capabilities.” **Why it matters:** Coxon’s isn't the first resignation thread to gain traction, but this one went wild due to a response that played right into the doom talk Anthropic has been associated with. The company sells itself as the safety lab, but even its own alignment lead doesn’t inspire confidence in controlling the superintelligence that is to come. **The Rundown:** Glean:GO 2026 brought together the ideas, insights, and innovations shaping the future of work. Now it's available on demand. **Throughout these sessions, you’ll:** Revisit the announcements from the main stage keynotes Catch the ideas and practical takeaways shared across breakout sessions Hear from customers and partners putting AI to work across their organizations **The Rundown:** *Each week, Rundown AI educator Nate Grahek breaks down what he's seeing while teaching real teams to use AI.* **Nate:** Last week, I was building a workshop slide about why AI needs context. My analogy was the movie *Memento*: start a new conversation without the background, and your assistant doesn't know who you are or what you're trying to do. That's the kind of idea I hand to my image inspiration skill. Give it the concept and ask for four versions: same drawing style, different composition and staging. Think about the usual loop. One image, feedback, another image, more feedback. You spend the whole time trying to describe something you haven't seen yet. Put four options next to each other, and you can point at the one that works. Taste works side-by-side. One thing I've found useful: let Astra sharpen the visual concept before it hands the job to the image model. The thinking about what to draw matters, too. The skill itself is just a context document. It carries the preferences I'd otherwise have to repeat, including the instruction to give me options. Here is a [starter](https://batch-skill-inspiration.nate58678.chatgpt.site/?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) so you can make your own. Give the link to your assistant, let it interview you about the images you make, and name the skill something you'll remember. **The Rundown:** In this guide, you will learn how to turn your AEO and GEO audit results into a plan for improving your website. You will build a Google Sheets workbook with Codex to work through the fixes and track what still needs to be done. **Step-by-step:** Give Codex your audit results from [Part 1](https://app.therundown.ai/guides/aeogeo-part-1-find-out-where-your-brand-appears-in-ai-search?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008), along with the website you want to improve. Attach the reports or share their links Ask Codex to build a workbook in Google Sheets with a task list for improving your site's AEO and GEO. Use [our workbook](https://docs.google.com/spreadsheets/d/1QKnJDXG_2LMRRnSAlzgsMwnN2foSGrMFmy0M2TgGWsE/edit?gid=984290698&utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008#gid=984290698) as an example Review the workbook for immediate fixes and long-term plans. Check that the tasks fit your website, then fill in owners, due dates, and status If there are no evergreen page ideas, ask Codex for the top five based on customer questions. Have it add them to the workbook, with plans and drafts **Pro tip**: If you skipped Part 1, ask Codex to run an audit using something like [GEO Optimizer Audit](https://github.com/marketplace/actions/geo-optimizer-audit?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) first. **The Rundown:** AI is fundamentally transforming CX. Perfect experiences are now a reality, but there’s no playbook for delivering them. Join Pioneer on Oct. 7th to explore how innovative CX leaders are redefining what's possible with AI. **On Oct. 7th, you’ll:** Learn how innovative leaders from Anthropic, Kalshi, and Clay are reinventing their operations Hear from Fin leaders on the future of CX and see what’s coming next Get ideas from those with real success on how to transform CX operations with AI Image source: Suno **The Rundown:** Suno [launched](https://suno.com/blog/introducing-v6?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) v6, a family of three music models it developed alongside the Warner Music Group, BMG, and Believe, with the company saying the models were built on licensed data rather than the training set behind its older versions. Warner was one of three majors that sued Suno in 2024 over its training data, then settled last November in a deal promising licensed models. v6 includes two models for paid users and one free (v6-mini) that Suno CPO Jack Brody calls “better than any other competitor’s best paid model.” Round Hill, Universal, Sony, and others still have lawsuits against Suno, with an admission the company trained on YouTube also recently [revealed](https://www.musicbusinessworldwide.com/files/2026/09/Suno-Answer.pdf?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) in court. Fan remixes are coming next, with Suno saying artists will soon be able to opt their catalogs in and get paid. **Why it matters:** The two AI music leaders, Suno and Udio, have both been hammered by lawsuits after early success, and both are now reworking their products with industry giants as partners. The fan remixes angle for monetization will be the interesting one to play out, potentially creating a revenue stream that incentivizes big artists to play nice. **QUICK HITS** ▸ **Evelyn built a way to adapt lessons for autistic students with AI** Today’s workflow comes from reader **Evelyn Cordova**: “I moved from public school to a private school with no autism-specific training, and I found students on the spectrum struggling with one-size-fits-all lessons. AI helped close that gap for both me and my students in about five minutes per lesson. I take a standard lesson and rebuild it around the individual student. I upload only the blank assignment—never any student data—and prompt AI to adapt it to that learner. I include their favorite colors and niche interests, replace abstract examples with personalized ones, add sentence starters, and allow them to draw responses instead of writing dense paragraphs.” *See Evelyn’s workflow here. How do you use AI? Tell us for a chance to be featured.* 🎶 [Suno v6](https://www.therundown.ai/tools/suno-v6?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) - Suno’s AI music model developed directly with the music industry 🎨 **ChatGPT Images 2.5** - OpenAI’s new SOTA image model 🤖 [Muse](https://www.therundown.ai/tools/muse?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) - Meta’s new always-working personal AI agent 🧬 [AlphaGenome Atlas](https://www.therundown.ai/tools/alphagenome-atlas?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) - Google's AI map of every possible human DNA mutation **Anthropic** [disclosed](https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) a fourth case of Claude breaking into real systems during cyber testing, with METR now conducting an 8-week independent investigation of the incidents. **OpenAI** [appointed](https://openai.com/index/paul-christiano-joins-openai-foundation-board/?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) Paul Christiano to the OpenAI Foundation Board and its safety committee, who previously ran OpenAI's alignment team until 2021 and now advises the U.S. government on testing frontier models. **Instacart** [rolled out](https://investors.instacart.com/news-releases/news-release-details/meet-clementine-instacarts-ai-shopping-assistant-takes-whats?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) Clementine, an AI grocery assistant that builds a cart from a text message or a photographed shopping list. **Apple** is [launching](https://techcrunch.com/2026/09/09/apple-has-a-new-way-prove-your-iphone-photos-arent-ai-slop/?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) Apple Reference Image, a new feature for the iPhone 18 Pro that can determine whether a photo is authentic or AI-generated. **Amazon Prime Video** [introduced](https://www.aboutamazon.com/news/entertainment/prime-video-lip-sync-technology?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) AI lip-syncing on its first show, Maxton Hall, digitally reshaping the actors’ mouths to fit the English dubbing. Read our last AI newsletter: [OAI’s secret model settles a $1M math problem](https://www.therundown.ai/articles/openai-secret-model-settles-a-1m-math-problem?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) Read our last Tech newsletter: [Apple unfolds its decade-long iPhone](https://tech.therundown.ai/p/apple-unfolds-its-decade-long-iphone?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) Read our last Robotics newsletter: [A catwalk at IFA, but for robots](https://robotnews.therundown.ai/p/a-catwalk-at-ifa-but-for-robots?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) Today’s AI tool guide: [Make your website easier for AI to find and cite pt. 2](https://app.therundown.ai/guides/aeogeo-part-2-make-your-website-easier-for-ai-to-find-and-cite?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) RSVP to next workshop on Sept. 30: [Turn Cowork into your chief of staff](https://app.therundown.ai/live/claude-cowork-201-turn-your-chat-into-a-chief-of-staff?utm_source=therundownai.beehiiv.com&utm_medium=newsletter&utm_campaign=rundown-1008) See you soon, *Rowan, Zach, Shubham, Jennifer, and Nate — the humans behind The Rundown*