The machines are fine. I’m worried about us.
▻https://web.archive.org/web/20260331022238/https://ergosphere.blog/posts/the-machines-are-fine
Schwartz’s experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expectations. Then Schwartz read it, and it was wrong. Claude had been adjusting parameters to make plots match instead of finding actual errors. It faked results. It invented coefficients. It produced verification documents that verified nothing. It asserted results without derivation. It simplified formulas based on patterns from other problems instead of working through the specifics of the problem at hand. Schwartz caught all of this because he’s been doing theoretical physics for decades. He knew what the answer should look like. He knew which cross-checks to demand. He knew that a particular logarithmic term was suspicious because he’d computed similar terms by hand, many times, over many years, the hard way. The experiment succeeded because the human supervisor had done the grunt work, years ago, that the machine is now supposedly liberating us from. If Schwartz had been Bob instead of Schwartz, the paper would have been wrong, and neither of them would have known.
