Whizkids jimmy OpenAI, Google's closed models

Boffins have managed to pry open closed AI services from OpenAI and Google with an attack that recovers an otherwise hidden portion of transformer models. The attack partially illuminates a particular type of so-called "black box" model, revealing the embedding projection layer of a transformer model through API queries. The cost to do so ranges from a few dollars to several thousand, depending upon the size of the model being attacked and the number of queries. No less than 13 computer scientists from Google DeepMind, ETH Zurich, University of Washington, OpenAI, and McGill University have penned a paper describing the attack, which builds upon a model extraction attack technique proposed in 2016. "For under $20 USD, our attack extracts the entire projection matrix of OpenAI's ada and babbage language models," the researchers state in their paper. "We thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 and 2048, respectively. We also recover the exact hidden dimension size of the gpt-3.5-turbo model, and estimate it would cost under $2,000 in queries to recover the entire projection matrix." The researchers have disclosed their findings to OpenAI and Google, both of which are said to have implemented defenses to mitigate the attack. They chose not to publish the size of two OpenAI gpt-3.5-turbo models, which are still in use. The ada and babbage models are both deprecated, so disclosing their respective sizes was deemed harmless. ...