MINJA sneak attack poisons AI models for other chatbot users

AI models with memory aim to enhance user interactions by recalling past engagements. However, this feature opens the door to manipulation. This hasn't been much of a problem for chatbots that rely on AI models because administrative access to the model's backend infrastructure would be required in previously proposed threat scenarios. However, researchers affiliated with Michigan State University and the University of Georgia in the US, and Singapore Management University, have devised an attack that muddles AI model memory via client-side interaction. The boffins – Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang – describe the technique in a recent preprint paper, "A Practical Memory Injection Attack against LLM Agents." They call their technique MINJA, which stands for Memory INJection Attack. "Nowadays, AI agents typically incorporate a memory bank which stores task queries and executions based on human feedback for future reference," Zhen Xiang, assistant professor in the school of computing at the University of Georgia, told The Register. "For example, after each session of ChatGPT, the user can optionally give a positive or negative rating. And this rating can help ChatGPT to decide whether or not the session information will be incorporated into their memory or database." The attack can be launched by just interacting with the agent like a regular user If a malicious user wants to affect another user's model interaction...