When AI Meets the World's Strictest Privacy Law: A Deliberate Blank, and an Answer Still Being Written

A little while ago, a friend of mine who works in enterprise services came to me with a complaint.

His company, he said, had recently wanted to use machine learning to build a credit-scoring model for their customers. The moment the model was assembled, legal shut the project down.

Their reason came down to one sentence: your model has to be fed the data of hundreds of thousands of people. When that data was originally collected, the stated purpose was "to provide our services." Now you want to use it for risk assessment — the purpose doesn't match.

My friend was indignant: we collected this data legally. It's just gathering dust in our database. What's wrong with putting it to use?

I didn't rush to answer.

Because I realized he hadn't run into a compliance problem. He had run into the most tangled knot of our era — between the smartest technology and the strictest privacy law, that gap: who, exactly, is supposed to fill it?

That's what I want to talk to you about today.

First, an uncomfortable fact

Did you know that AI, at its core, grows up feeding on data?

What is machine learning, really? Plainly put — you give it a pile of examples, tell it "in this kind of case, the answer is A; in that kind of case, the answer is B," and it teases out the pattern on its own. Next time it encounters something new, it can guess pretty closely.

The more examples, the more accurate the guess.

And so the cycle kicks into motion: the more accurate the model, the more it gets used; the more it's used, the more data gets collected; the more data, the better the model gets again.

This is what's called the data flywheel. AI's appetite is fed by this flywheel.

And where does all this data come from? From every click we make, every location we share, every purchase, every card swipe. Someone has calculated that the number of connected devices worldwide is now approaching 30 billion. Every minute, the data generated on the internet could fill an entire library.

And a sizable share of it is about you.

The privacy law says: you may use it for this purpose, not for that one

This is where the law that made my friend's legal team so nervous enters the picture.

Europe has a privacy law — the General Data Protection Regulation, or GDPR — that took effect in 2018. Anyone in the industry knows how fierce it is: violations can be fined up to 4% of annual global revenue. It laid down several iron rules, two of which are fundamentally at odds with AI.

The first is called purpose limitation. When you collect someone's data, you said it was for delivering goods — so you can only use it to deliver goods. You want to use it for something else? No.

The second is called data minimization. If you could get by with one record, don't collect ten; if you could collect less, never collect more.

Think about it — which of these two isn't a hand around AI's throat?

What AI wants is exactly large volumes of data, exactly cross-purpose use. Your delivery address, your purchase history, your browsing traces — each one on its own is trivial. But once you stitch them together and feed them to a model, it can infer your income, your habits, even your personality.

What the privacy law fears is precisely what AI most wants.

So did the law just ban AI outright?

It didn't.

The law left a door open — but didn't tell you how to walk through it

Read the law carefully and you'll find it isn't actually that rigid.

The purpose-limitation clause comes with an escape hatch: as long as the new use is "not incompatible" with the original one, it's allowed. Doing statistical analysis? Presumed compatible.

The data-minimization clause has interpretive room, too. What it really wants to police isn't necessarily the volume of data, but the data's personal-identifiability — whether you can identify a specific person from that pile of data. If it's been de-identified so that no one can be matched back to a name, that data isn't considered as "dangerous."

The door was left open.

But where the door is, how to open it, how many steps count as crossing the line — the law doesn't say.

It's like traffic rules telling you to "drive safely" without marking the speed limit on each stretch of road or where U-turns are allowed. You sit behind the wheel feeling uneasy.

That's the root of my friend's legal team's agony. The law says "fine, as long as it's not incompatible" — but what counts as "not incompatible"? Does using delivery data for credit scoring count as incompatible? No one has ever given a clear answer.

The law tossed this judgment call to the companies themselves.

A case that should send a chill down your spine

You might say: well then, just use the data boldly — efficiency comes from putting data to work.

Let me tell you a story first.

A few years ago a data company — Cambridge Analytica — built a little personality-test game. You logged in with your social-media account, answered about a hundred-plus questions, and they paid you a few dollars. Roughly 320,000 people took the test.

Sounds pretty harmless, right?

But through those 320,000 people's accounts, the company also scraped the data of everyone in their social graph. The data they ended up with covered more than 50 million people.

And then? It used the 320,000 people's "test results + social data" as a training set, and trained a model — feed it your social behavior (what you've liked, what you've posted), and it could infer your personality type, your political leanings, what kinds of emotions you were most susceptible to.

Next, in a globally watched election, it precisely located the voters who were "persuadable," and served each of them tailored political ads — poking exactly at your soft spots, while you remained completely unaware.

The data of 50 million people became an invisible hand in an election.

When this came to light, the whole world gasped.

You see, data is neutral, and so is AI. But when it can calculate "what kind of person you are" this precisely — and then turn around and manipulate you — this is no longer a question of efficiency.

This is a question of who controls you.

The real danger hides in one phrase: differential treatment

This case made one thing clear to me.

The harm of AI isn't only that it "gets things wrong" or "discriminates" — those are technical problems, fixable.

A deeper danger is that, silently, it sorts people into tiers and ranks.

Take insurance. A company uses AI to analyze health data and predict who's likely to get sick. Once it predicts accurately, it lowers premiums for the healthy and raises them for the people likely to fall ill.

Sounds reasonable, doesn't it?

But think about it — the people who are already sick, the people who are already vulnerable, now have to pay more for insurance on top of everything else. Here AI isn't helping people; it's baking society's unfairness into an algorithm.

Or take hiring. A model notices that "people from a certain neighborhood tend to underperform," and filters out any résumé with that address. What it won't tell you is that the people living in that neighborhood are mostly from a particular ethnic group — and so discrimination puts on a "data-driven" disguise, more hidden than the prejudice in any one human's head.

Even more chilling is pricing. The same app, the same product — the price you see and the price someone else sees may be entirely different. The system has figured out exactly how much you're willing to pay, and that's what it charges you. You think you're choosing freely; in reality, you were set up long ago.

All of this happens in a background you never notice.

So why is that privacy law so strict? Not because it doesn't understand technology, not because it's heartless. It's because it has seen where these things lead.

But strictness can't solve everything

Interestingly, the very "strictness" of this law has itself become a problem.

What companies face is this landscape: vague rules, astronomical fines, and AI that fundamentally needs data. The result — big companies can afford legal teams and slowly feel their way forward (lit., "crossing the river by feeling the stones"); small companies take one look at this and decide not to touch it at all.

The law meant to protect innovation, and instead it scared the small players out of moving.

This is the price of ambiguity. It tosses "what counts as compliant" to every company, without offering an answer key.

There's another blind spot: this law only protects individuals, not groups.

Meaning what? A given AI application does no harm to you personally — it might even be quite convenient. But if hundreds of millions of people are all being silently categorized, quietly influenced by it — this "slowly boiling the frog" kind of collective risk — the law has barely touched it.

A counterintuitive fix: use AI to fight AI

At this point you might be feeling that it's all pretty hopeless. The law can't keep up, companies are feeling their way forward in the dark, ordinary people are completely in the dark.

I did come across one rather interesting line of thinking.

Government and law alone can't stop the abuse of technology. Tech companies hold the smartest algorithms, the most data, the most computing power. Regulators very often can't understand it, and can't keep up.

So what's the answer? Ordinary people need to have AI in their own hands, too.

There are already early signs of this. There are systems that automatically flag unfair contract terms buried in your online-shopping agreements. There are tools that scan your privacy settings and tell you which of your data has been collected. There are people doing bias auditing, deliberately hunting for the prejudice hiding inside AI's decisions.

The logic is plain and simple: only when ordinary people have their own "AI weapons" does real counterbalance begin.

Otherwise it's a game where only one side has guns.

Back to my friend from the beginning

My friend later asked me again: so can my project go forward or not?

My answer was: the law hasn't forbidden you from doing it, but no one has told you how to do it safely, either.

What you can do is take the initiative to treat yourself as "the responsible party" — run a risk assessment, de-identify the data, explain the use clearly to your users, leave room for human review. Not to pass an inspection — but so you can sleep at night.

As for the bigger question — how AI should be governed, which applications should be banned outright, which ones companies can be trusted to experiment with — that's not a question any single law can finish answering.

It needs a conversation where all of society sits down together: lawmakers, companies, scholars — and you and me, too.

This law pointed in a direction: protect people. Don't let people be swallowed by technology.

But the road is still long.

Next time you tap open an app's permission dialog, think for one extra second: this piece of data I'm handing over — what will it eventually become?

That one second is where change begins.