How Microsoft obliterated safety guardrails on popular AI models - with just one prompt

Follow ZDNET: Add us as a preferred source on Google. Model alignment refers to whether an AI model's behavior and responses align with what its developers have intended, especially along safety guidelines. As AI tools evolve, whether a model is safety- and values-aligned increasingly sets competing systems apart. But new research from Microsoft's AI Red Team reveals how fleeting that safety training can be once a model is deployed in the real world: just one prompt can set a model down a different path. Also: I tried a Claude Code rival that's local, open source, and completely free - how it went "Safety alignment is only as robust as its weakest failure mode," Microsoft said in a blog accompanying the research. "Despite extensive work on safety post-training, it has been shown that models can be readily unaligned through post-deployment fine-tuning." The company's findings question whether alignment can withstand downstream shifts and identify how easily model behavior can change if it can't. Companies like Anthropic have committed plenty of research efforts toward training frontier models to stay aligned in their responses, no matter what a user or bad actor throws at it. Most recently, Anthropic released a new "constitution" for Claude, its flagship AI chatbot, which details "the kind of entity" the company wants it to be and emphasizes how it should approach attempts to manipulate it (with confidence rather than anxiety). Also: Is your AI model secretly poisoned? 3 warni...