A natural person is not a rubber stamp

On 8 October the Dutch data protection authority fined Uber €824,990,000. The reason, in the EDPB’s words: Uber “has violated the prohibition of fully automated decision-making”. Drivers’ accounts deactivated by a system, without meaningful human involvement, and insufficient information about profiling. Uber has appealed.

The same day, Italy published its legislative decree implementing the AI Act. According to early reports, at work hiring, dismissal and disciplinary decisions cannot rest solely on an algorithm: the final decision “is in any case reserved to a natural person”.

Two pieces of news, one thesis: you need a human. But this is where, I think, law risks stopping one step short of the problem.

The reverse centaur

For years we imagined the centaur: the person steering, the machine amplifying. In platform practice the opposite often happens. The machine sets the pace, the queue, the priority. The person arrives at the end, with a screen, a score and thirty seconds. Formally they decide. In reality they ratify.

Article 22 GDPR already knew this: it is not enough for a human to touch the decision; they must be able to change it. European guidance has spoken of meaningful intervention for years. The point is that “meaningful” is not proven by a signature. It is proven by the conditions under which the signature is given.

What the decision-maker sees

When I work on documents with the LX Complexity Score I measure something simple: how far a text is built to be understood by those who must use it. In the first study, on consent documents for brain-computer interfaces, no text was below the experimental accessibility threshold. The same question should be asked of human reviewers’ interfaces:

• what does the reviewer see before confirming a deactivation?
• the data behind the score, or just the score?
• how much time do they have? Who measures how often they disagree?
• if they disagree often, are they rewarded or put under watch?

If the answers are “little”, “just the score”, “very little”, “nobody” and “under watch”, we have a natural person acting as a rubber stamp. The legal requirement is met in form and emptied in substance.

In the Habeas Mentem Charter I put it this way: “Whoever must oversee a machine has the right to the conditions to judge it, not merely to sign for it.” It is not a slogan. It is a verifiable criterion.

What I would do tomorrow

Anyone using systems to manage people (in Italy, the UK or the US) should stop asking “is there a human in the loop?” and start asking “can that human say no, and do they know how?”. In practice:

1. override rate: how many system decisions the reviewer changes. Near zero means either a perfect system or an absent reviewer;
2. minimum time per review, written into the procedure;
3. review screen designed for dissent: source data, reason, alternative, not just “confirm”;
4. traceability of human judgment, not only of the output.

The Uber fine does not say algorithms cannot decide. It says a decision without real human judgment is unlawful. That is a different thing, and worth much more.