ICO issues ChatGPT warning in Snapchat privacy probe

The Information Commissioner’s Office has sent out a warning shot to companies using generative artificial intelligence tools, taking its first action by issuing a preliminary enforcement notice to Snapchat owners Snap Inc and Snap Group UK over a potential failure to properly assess the privacy risks posed by generative AI chatbot ‘My AI’.

The preliminary notice sets out the steps which the Commissioner may require, subject to Snap’s representations on the preliminary notice. If a final enforcement notice were to be adopted, Snap may be required to stop processing data in connection with My AI. This means not offering the My AI product to UK users pending Snap carrying out an adequate risk assessment.

Snap launched the My AI feature for UK Snapchat+ subscribers in February 2023, with a roll out to its wider Snapchat user base in the UK in April 2023.

The chatbot feature, powered by OpenAI’s ChatGPT technology, marked the first example of generative AI embedded into a major messaging platform in the UK. As at May 2023, Snapchat had 21 million monthly active users in the UK.

The ICO’s investigation provisionally found the risk assessment Snap conducted before it launched ‘My AI’ did not adequately assess the data protection risks posed by the AI technology, particularly to children.

The regulator stresses the assessment of data protection risk is particularly important in this context which involves the use of innovative technology and the processing of personal data of 13- to 17-year-olds.

The Commissioner’s findings in the notice are provisional. No conclusion should be drawn at this stage that there has, in fact, been any breach of data protection law or that an enforcement notice will ultimately be issued.

The ICO said it will consider any representations from Snap before taking a final decision.

Commissioner John Edwards said: “The provisional findings of our investigation suggest a worrying failure by Snap to adequately identify and assess the privacy risks to children and other users before launching My AI.

“We have been clear that organisations must consider the risks associated with AI, alongside the benefits. Today’s preliminary enforcement notice shows we will take action in order to protect UK consumers’ privacy rights.”

Edwards’ comments and the ICO’s action appear to be far more cautious than under his predecessor Elizabeth Denham who was often accused of chasing headlines, including the ICO’s actions against Facebook and Cambridge Analytica, as well as GDPR fines for British Airways and Marriott International.

The issue of this preliminary enforcement notice follows an ICO reminder to companies developing or using generative AI that they should be considering their data protection obligations from the outset.

The ICO has issued advice to developers and users of generative AI on the issues that they must consider and will continue to scrutinise the compliance of products and services introduced to market.