Jacob Coxon, a former researcher at both OpenAI and Anthropic, has resigned from his role as chief scientist at Anthropic. He stated his departure on September 8, citing deep concerns that artificial intelligence poses a serious threat to humanity. Coxon described the work at Anthropic as resembling a mini Manhattan project but warned that the consequences of unsafe AI could be catastrophic.
In interviews with WIRED, the New York Times, and TIME, Coxon emphasized that aligning AI with human values requires a fundamental societal shift. He specifically highlighted the risk of autonomous weapons and noted a lack of transparency in how these systems are developed. He believes the current pace of development is spinning out of control and that we may only have a few years left to ensure these systems are safe before they surpass human intelligence.
While Coxon faces resistance within the industry regarding these warnings, his departure has sparked a broader debate among experts about the risks and benefits of AI. Meanwhile, the practical application of AI continues to expand in other sectors, with more than 70% of middle school teachers in the United States now using AI tools to create personalized learning plans and assist with grading.
Educational institutions are also grappling with how to integrate AI into curricula. Harvard's editorial board recently argued against allowing students to write papers using AI, suggesting oral defenses instead to ensure foundational learning. Conversely, the University of Texas at Dallas now offers courses on prompting AI, and the Mark Cuban Foundation has partnered with Samsung to create a free AI curriculum for U.S. schools.
Regulatory challenges are emerging alongside technological adoption. Hawaii County has deferred a ban on AI data centers until October, with officials considering size limits based on electricity needs rather than square footage. In Wisconsin, lawmakers are discussing rules for AI in healthcare, noting the U.S. lacks a national data privacy law compared to 144 other countries. A proposed 2025 bill in Washington would allow AI to prescribe medications, raising questions about liability if mistakes occur.
Security experts also warn that traditional testing methods are becoming obsolete as frontier AI models speed up the creation of weaponized exploits. They argue that fighting AI attackers with AI defense tools is insufficient and call for a new system called Agentic BAS to run tests automatically. Additionally, Italian philosopher Stefano Gualeni published a book titled What We Owe the Dead, which explores AI ethics through a dystopian story about cyborgs serving as security units.
Key Takeaways
- Jacob Coxon, former chief scientist at Anthropic and researcher at OpenAI, resigned on September 8 due to fears that AI poses an existential threat to humanity.
- Coxon warned that the current pace of AI development is spinning out of control and could lead to a scenario where AI surpasses human intelligence.
- He specifically cited the risk of autonomous weapons and a lack of transparency in AI development as major concerns.
- More than 70% of middle school teachers in the United States are now using AI tools for personalized learning and grading.
- Harvard's editorial board opposes students using AI to write papers, advocating for oral defenses to verify student understanding.
- The Mark Cuban Foundation and Samsung have partnered to create a free AI curriculum for U.S. schools.
- Hawaii County has deferred a ban on AI data centers until October, considering limits based on electricity usage instead of square footage.
- Wisconsin lawmakers are debating AI regulations in healthcare, noting the U.S. lacks a national data privacy law.
- A 2025 bill in Washington would allow AI to prescribe medications, prompting concerns about liability for medical errors.
- Security experts recommend a new Agentic BAS system to address the speed at which AI models create weaponized exploits.
Anthropic Researcher Quits Over AI Safety Fears
Jacob Coxon, the chief scientist at Anthropic, has resigned from his job. He told WIRED that he believes it is crunch time for humanity regarding AI safety. Coxon compared the work at Anthropic to a mini Manhattan project but warned that the consequences of unsafe AI could be catastrophic. He stated that aligning AI with human values requires a fundamental shift in how society thinks about technology. Many experts share his concern that we only have a few years left to make these systems safe.
AI Researcher Leaves Industry Citing Threat to Humanity
Jacob Coxon quit his job at Anthropic on September 8 and left the AI industry entirely. He spoke to the New York Times about his fears that AI poses a serious threat to humanity. Coxon cited the risk of autonomous weapons as a major concern and noted a lack of transparency in AI development. He tried to raise these issues within the company but faced resistance. His departure has sparked a debate among experts about the risks and benefits of artificial intelligence.
Former AI Builder Warns Technology Could Kill Us
Jacob Coxon, a former researcher at OpenAI and Anthropic, says he is afraid AI could kill us. He told TIME that the current pace of AI development is spinning out of control. Coxon believes it is only a matter of time before AI surpasses human intelligence, which he calls a recipe for disaster. He warned that we are playing with fire and need to slow down to avoid getting burned. The AI community is divided, but Coxon insists the risks are real and require immediate action.
Hawaii County Delays Ban on AI Data Centers
The Windward Planning Commission in Hawaii County has deferred a ban on AI data centers until October. Council member Heather Kimball introduced Bill 170 to distinguish between large data centers and standard data processing facilities. County Planning Director Jeff Darrow suggested a size limit of 10,000 square feet instead of an outright ban. Opponents worry about high water and electricity usage, while proponents see job creation and tax revenue. The commission is considering limits based on electricity needs rather than just square footage.
AI Era Requires New Approach to Security Testing
Security experts say breach and attack simulation tools must evolve for the age of AI. Frontier AI models have sped up the time it takes to create weaponized exploits, making old testing methods too slow. A recent report found that 49% of detection failures are due to performance issues and 41% are caused by gaps in log collection. Experts argue that fighting AI attackers with AI defense tools is not enough. The industry needs a new system called Agentic BAS that runs tests automatically and fixes gaps in real time.
Harvard Students Oppose Using AI to Write Papers
A Harvard editorial board argues that students should not be allowed to write papers using AI. They responded to a suggestion by Dean David J. Deming that students could use AI if they disclosed it. The board believes outsourcing thinking to AI prevents students from learning how to synthesize sources and build arguments. They also noted that AI detection software is often unreliable and prone to false positives. Instead, they suggest oral defenses to ensure students truly understand the material. The group believes liberal arts education should focus on foundational knowledge rather than just job skills.
Middle School Teachers Adopt AI for Classroom Learning
More than 70% of middle school teachers in the United States are now using AI tools in their classrooms. These tools help create personalized learning plans and provide real-time feedback on assignments. Popular tools include virtual learning environments that adapt to student needs and chatbots that assist with grading. While some teachers worry about losing the human touch in education, others believe AI makes learning more accessible. Proponents argue that these technologies can provide valuable insights to help teachers improve their methods.
Book Explores AI Ethics Through Fictional World
Italian philosopher Stefano Gualeni wrote a book called What We Owe the Dead that explores AI ethics. The story is set in a dystopian future where cyborgs with the memories of deceased officers serve as security units. The novel draws parallels to Cixin Liu's The Three-Body Problem and uses a fictional game to examine cultural obligations. Characters in the book discuss the nature of consciousness and whether machines with human memories can be considered people. Gualeni hopes the story helps readers think about the moral burdens we carry toward our ancestors.
Schools Debate Teaching Students How to Use AI
Schools across the country are discussing whether they should teach students how to use artificial intelligence. The University of Texas at Dallas now offers courses on prompting AI and the ethics of its use. The Mark Cuban Foundation and Samsung have partnered to create a free AI curriculum for U.S. schools. Texas institutions currently offer more than 20 degree programs in AI. This initiative aims to prepare students for a future where AI is a common tool in many industries.
Wisconsin Lawmakers Discuss AI Rules for Healthcare
Lawmakers in Wisconsin met to discuss regulating artificial intelligence in healthcare settings. The United States currently lacks a national law for data privacy compared to 144 other countries. A bill introduced in 2025 in Washington would allow AI to prescribe medications, which some doctors call science fiction. Dr. Ramsey Larson asked who would be responsible if an AI made a mistake in patient care. Experts advised lawmakers to think seven generations ahead before passing new legislation. The legislative study committee plans to meet again this fall.
Sources
- The AI Researcher Who Just Quit Anthropic Says It’s ‘Crunch Time for Humanity’
- 'Gambling with our lives': AI researcher quits Anthropic and leaves AI entirely over what he calls a threat to humanity
- He Helped Build Powerful AI at OpenAI and Anthropic. Now He's Afraid It Could Kill Us
- Hawaii County Defers AI Data Center Ban Until October
- What breach and attack simulation needs to become in the AI era
- Don't Tell Us To Write Our Papers With AI
- Middle school teachers embrace AI as a classroom learning tool
- Reflections on artificial intelligence and the moral burdens of What We Owe the Dead
- Should schools be teaching students how to use AI?
- Wisconsin lawmakers weigh AI regulation in healthcare
Comments
Please log in to post a comment.