How Amazon red-teamed Alexa+ to keep your kids from ordering 50 pizzas

RSAC If Amazon's Alexa+ works as intended, it could show how an AI assistant helps with everyday tasks like making dinner reservations or arranging an oven repair. Or things could go terribly wrong: it might turn on the oven and turn dinner plans into a house fire. This is why the e-commerce giant brought in security engineers, including both red teams and penetration testers, to work alongside product developers from the beginning, according to Amazon CISO Amy Herzog. Their job was to anticipate what could go wrong and ensure safety and security guardrails were in place to prevent Alexa+ from jumping the track. "It's funny how, having been in both seats, the product engineer thinks about making the intended thing work, and the security engineer thinks about all the ways that you can game that system," Herzog told The Register on the outskirts of the RSA Conference in San Francisco this week. "Whenever you're talking about a system that can take actions on behalf of someone our immediate [reaction is]: Wouldn't it be good if, like me, as someone who's running this household could just say, This is what I need to go shopping for. These are the dinner reservations I need to make. Go make that happen. Schedule a delivery window," she said. "And then my kid comes into the kitchen and says, and also 50 pepperoni pizzas for me and my friends." The product engineer thinks about making the intended thing work, and the security engineer thinks about all the ways that you can game that...