Brussels faces industry ire over plans for 'GDPR for AI'

Companies which use algorithms that discriminate against certain groups or calculate credit scores using information about consumers’ online habits could face fines of up to 6% of their global turnover under new legislation to govern artificial intelligence, proposed by the European Commission.

The draft, which will now be debated by the European Parliament and member states until at least 2023 before becoming law, is being seen as the first marker in regulating AI. The Commission is eager to replicate GDPR, although that took more than seven years to implement.

While Britain will not have to adopt the legislation, due to Brexit, the UK’s new “super-regulator”, the Digital Regulation Cooperation Forum (DRCF), has already warned that AI is in its sights.

Under the proposals, the Commission has designed a scheme that splits AI systems into four categories according to their potential risk:

Minimal risk: Examples of these systems include spam filters and video games. The new rules will not apply to these systems because they represent only minimal or no risk for citizen’s rights or safety.

Limited risk: These AI systems include chatbots and the like and will be subject to specific transparency obligations to allow users to make informed decisions, be aware they are interacting with a machine and let them easily switch off.

High risk: The Commission expects high-risk systems to be found in a range of sectors, such as transport, education, employment, law enforcement, migration and healthcare, with the examples it cites including facial recognition, algorithms that evaluate credit worthiness and those that sort CVs from job candidates. These systems will be “carefully assessed before being put on the market and throughout their lifecycle”.

Unacceptable: the Commission will ban AI systems that represent “a clear threat to the safety, livelihoods and rights of people”. Examples include social scoring by governments, exploitation of vulnerabilities of children and use of subliminal techniques.

The most controversial aspect is the question of biometric identification systems, which are based on human characteristics such as facial traits, voice or emotions. These applications can be found when consumers unlock their smartphones or cross country borders.

According to the regulation, all remote biometric ID systems will be categorised high risk and will be subject to strict requirements; the use of these systems by law enforcement will be banned, falling into the “unacceptable” category.

However, the Commission said there would be “narrow exceptions” that would allow the use of biometrics in “targeted” police actions, such as the search for a missing child and the response to a terrorist threat. These uses will have to be authorised by a judicial body and be limited in time and reach.

Governance of the law will be spearheaded by a new European Artificial Intelligence Board that will help national authorities to supervise and implement the rules.

In response to the proposals, Soffos.ai chief executive Nikolas Kairinos said: “The AI industry needs a strong system of checks and balances to win public trust. That said, the proposed regime will not sit well with many in the community.

“Loose definitions like ‘high risk’ are unhelpfully vague. AI today comes in many forms, and the risks and considerations vary across different domains. An ambiguous, tick-box approach to regulation that is overseen by individuals who may not have an in-depth understanding of AI technology will hardly inspire confidence within the industry as it sets about building solutions that are, in many instances, intended to make our lives better, either as businesses or consumers.

“I fear that without clear and fair definitions, ambitious AI developers will be left at the mercy of regulators and risk being barred from the EU market – one that desperately needs to push the needle forwards where innovation is concerned.

“While the EC should be praised for its efforts to proactively raise the standards for AI development, introducing a list of rigid tests and benchmarks also risks impeding progress in R&D. Innovation, by its nature, involves an element of risk – and any attempts to over-regulate will result in high economic and human costs. That is not to say that we should do away with regulation altogether. Rather, we must be careful to avoid applying too broad a brush, and instead find an approach which meets the needs of all stakeholders.”

Meanwhile, Digital Europe, the trade association representing digital companies in Europe said that “the inclusion of AI software into the EU’s product compliance framework could lead to an excessive burden for many providers”.

Director Cecilia Bonefeld-Dahl added: “After reading this regulation, it is still an open question whether future start-up founders in ‘high risk’ areas will decide to launch their business in Europe.”