Hello, and welcome back. What day could be more appropriate for TGIF than September 11? TGIF on September 11 reminds Nellie Bowles of trying to celebrate her birthday on Adolf Hitler’s birthday: “A terrible and ruthless commander was born that day, and also Adolf Hitler. Someone’s always doing something terrible in honor of him, and I’m just trying to have a tiramisu.”
Writing resignation letters from AI companies is the chic new thing: One young man, Jacob Coxon, left OpenAI and joined Anthropic and then resigned four months later—before describing the danger in developing artificial intelligence and asking people to please stop. “If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because ‘it’s happening anyway’—or take this moment to call for different conditions?”
Coxon’s post went super viral. Even Elon Musk was shocked by the virality: “I don’t think this has ever happened for a post from a new account with almost no prior activity.” Jacob’s argument is that everyone building AI knows that it could kill us, so Americans have to stop building AI and, I guess, hope that China and other countries just voluntarily stop too? Evan Hubinger, the Anthropic lead in charge of “alignment,” actually agreed on the stakes: “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
Like all arms races, everyone has to agree to stop doing the race for it to work. That is not what’s happening. But smart young Americans have been told for a generation that we are the worst people on Earth (aside from Israelis) and should cede power to any other more ethical nation. What sweet, gentle Jacob can’t imagine is that genuinely malevolent nations sit outside our gates, some hoping to kill our society and subjugate us to their interests.
This is an excerpt of the original article published by The Free Press. Read the full article there.