🗣"Just please help ... I am quite worried about how this direction is heading."

Nicolas Carlini, a research scientist at top AI company Anthropic, says AI is rapidly improving at hacking.

He's used AI to find so many bugs that he can't report them.

Carlini warns: "Soon it's not just gonna be me who has all of this, but it's gonna be anyone malicious in the world who wants."

These are just the capabilities that AIs have today. Superintelligent AI, which top AI companies like Anthropic are aiming to build, would be vastly more powerful. If we can't ensure that it's safe and controllable, tha