The first AI Act proposal made product purpose a regulatory choice
Europe's first AI Act proposal linked obligations to the task a system performed. For software businesses, choosing a market also meant choosing a body of evidence to build and maintain.
Recruiting software and a chatbot serving customers might use related statistical techniques, yet perform very different tasks in people's lives. The European Commission's AI Act proposal of 21 April 2021 put that distinction near the centre of regulation. Its approach connected obligations to a system's intended purpose, the setting in which it operated and the consequences of failure.
Negotiations still lay ahead, but the proposal gave founders and investors a product decision to consider: would a company build the evidence, controls and customer support required for the use it was selling? Describing an application as a general AI platform could not replace an examination of the decisions customers expected it to influence.
A market choice could define the evidence burden
The proposal's Annex III identified particular uses in fields including recruitment, education, creditworthiness and essential public services as high risk. A separate route covered AI that was a safety component, or itself a product, under specified product legislation requiring an assessment of conformity by a third party. The classification depended on these distinct tests, with different implications for different uses of machine learning.
Scroll to view all columns
| Comparison | Example use | Route in the April 2021 proposal |
|---|---|---|
| Recruitment | Screening or filtering job applications | An explicitly listed high-risk use in Annex III |
| Customer interaction | A chatbot speaking with a person | Article 52 disclosure where the AI interaction was not obvious; other duties could also apply |
| Product safety | AI acting as a product's safety component | Article 6 high-risk route where both specified product-law conditions were met |
Examples from Articles 6 and 52 and Annex III of the April 2021 proposal.
The definition of intended purpose included instructions, promotional and sales materials, and technical documentation. A supplier promising that its system could screen applicants was therefore making a claim with consequences beyond the sales pipeline. For a startup, discipline in defining the supported task would help align training data, validation and customer expectations. Reusing a model in a new market could require more than changing the website.
The requirements reached into daily engineering
For high-risk systems, Articles 9–15 proposed a continuing process for managing risk, governance of training and testing data, technical documentation, logs, transparency, human oversight, accuracy, robustness and cybersecurity. Data requirements included examining possible bias and whether datasets suited the geographical, behavioural or functional setting. A model's average test score would answer only part of that assessment.
The commercial implication was an ongoing maintenance obligation. A team would need to explain which data and software version supported a performance claim, how a change affected that claim, and what evidence a customer could obtain after an unexpected result. Those capabilities would consume engineering time, but could also make a supplier easier to evaluate in a demanding procurement process. Their value depended on usable records rather than the volume of documents produced.
Human oversight needed a working interface
Article 14 described oversight in operational terms. The person supervising a high-risk system should be able to understand its capabilities and limitations, recognise unexpected behaviour, avoid relying too heavily on the output and, where appropriate, disregard, override or reverse a result. The proposal also envisaged the ability to interrupt operation. A nominal human approval step would provide weak protection if the reviewer lacked the information or authority to intervene.
Consider a hiring product that prioritised applicants for review. Effective oversight would require a customer to understand what the ranking represented and to investigate when qualified candidates were systematically missed. Designing for that possibility would add work beyond improving a narrow model benchmark: the user would need enough context to make an independent decision.
Responsibility could move along the supply chain
The proposal distinguished providers from users, importers and distributors. Article 28 would treat another party as a provider in specified circumstances, including putting a high-risk system on the market under its own name, modifying its intended purpose or making a substantial modification. That provision made integration and resale arrangements relevant to responsibility as well as distribution.
For an AI supplier selling through an enterprise platform, the contract would need to make the supported use, change process and access to evidence intelligible to both sides. A customer that configured the workflow also controlled facts the original developer might never see. The proposal's user duties included following instructions, monitoring operation and retaining logs under the user's control, reinforcing the need for a practical exchange of information after deployment.
The debate exposed limits to a compliance claim
On 18 June, the European Data Protection Board and European Data Protection Supervisor issued a joint opinion. They welcomed the approach of classifying systems by risk but sought changes, including broader restrictions on biometric recognition in public spaces and assessment of high-risk AI by third parties. Their recommendations had not been enacted. The disagreement showed why a financing plan could not assume the Commission's text was the final settlement.
Scroll to view all columns
| Comparison | Fixed ceiling | Company turnover ceiling |
|---|---|---|
| Prohibited practices or Article 10 data requirements | €30 million | 6% |
| Other requirements or obligations | €20 million | 4% |
| Incorrect, incomplete or misleading information supplied on request | €10 million | 2% |
Article 71 of the Commission's original proposal, in the copy transmitted by the Council. For companies, the proposed maximum was the higher of the fixed amount and the percentage of worldwide annual turnover for the preceding year. The circumstances also affected the penalty assessment. These were proposed ceilings, not the final AI Act's penalty schedule or fines already imposed.
Source3
The opinion also emphasised that classifying a system as high risk did not make its use lawful under existing data protection law. A supplier's proposed conformity process and a customer's lawful basis for using personal data were separate questions. Investors therefore needed to assess whether a company could explain and support its product in the setting where it would be used. The proposal gave that assessment a regulatory structure while leaving consequential legislative choices unresolved.
Investment implications
- The proposal classified systems partly through intended use, making sales claims relevant to product governance.
- Proposed high-risk duties connected datasets, documentation, human oversight and monitoring across the product lifecycle.
- European privacy authorities challenged important boundaries in the proposal.
- Meeting a proposed AI conformity route would not by itself establish that processing personal data was lawful.
Sources
- Proposal for a Regulation laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), COM(2021) 206 final (opens in a new tab)
European Commission
- EDPB–EDPS Joint Opinion 5/2021 on the proposal for an Artificial Intelligence Act (opens in a new tab)
European Data Protection Board and European Data Protection Supervisor
- Proposal for an Artificial Intelligence Act, COM(2021) 206 final: Council transmission (opens in a new tab)
European Commission, transmitted by the Council of the European Union
Published
