Big firms' big problem: no one reads what AI writes

Sixteen of twenty-seven sources did not check out. They did not exist, led to dead links, or claimed something that was not in them. The report came from one of the largest consulting firms in the world. It was meant to be about cybersecurity. They pulled it.

The study that never existed

The report contained one specific figure. It was supposed to come from a study by a world-famous consulting firm. That study was never published. The figure actually comes from a blog post. Someone took it from the blog, attributed it to the study that did not exist, and put that study into the report as a source. From there a newspaper picked it up, and it spread to more than sixty papers. Today the same invented figure is repeated as fact by ordinary AI models.

Is this the fault of AI? No. A person was responsible for the output. Someone who let AI do the work and did not read what it produced.

It is not one firm

A few months earlier, another firm from the big four handed a government a report for roughly 440,000 dollars. It contained fabricated academic sources and a fabricated quote from a judge. The firm returned 97,000 dollars and added to the corrected version that it had used generative AI.

Two independent firms. Both cases public. This is not an exception. There is more and more of it. It is the professional edge of the dead internet theory: generated content that looks finished, yet no one truly lived or checked it.

Why did no one read it?

The question is not why AI generated it badly. AI predicts the next word of text. It does not verify whether something is true, and when it claims that it does, it does that by predicting the next word too. The question is why no one caught it. And here is the problem that large corporations will not solve so easily.

A large corporation produces an enormous volume of output. It passes through many people, a large share of whom are comfortable and doing work that does not mean much to them. AI gives them an unprecedented tool to get that work done in minimal time, while at first glance the work looks good.

Rubbish that looks like finished work

Before, this kind of approach to work showed at first glance. An empty document, missing numbers, work that was obviously unfinished. Today AI produces text that looks finished. It has the right format, an academic tone, a table of sources. It looks exactly like honest work. But it is not.

The brand that pays for it is not his

For that person the cost is low. When it comes out, his name does not suffer. The brand of the firm suffers, a firm he feels no great bond to. He gets paid all the same. It is even highly likely that it is never discovered. And if it is, he can say that they were urged to do everything faster with AI and to be more productive. He can even put it on his managers and make himself the victim.

A corporation can police output like this only with great difficulty. Not because it does not care. Because at its size, with that many processes and outputs, it would have to check every source in every report after every person. That cannot be done.

Such checking would stand in for the missing personal responsibility of a particular person. That is the core of the problem today.

Going under your own name

Unlike a large firm, an individual who goes under their own name has it easier here.

When his own name is on it, the cost of poor work is higher for him. It is not insurance, it guarantees nothing, he too can miss something. But there is a greater chance that he is careful and reads it back.

What we have been paying big firms for

We pay a premium for a big brand to get a standard, quality output: for the fact that someone stands behind it. That is falling apart more and more. Trust today belongs sooner to whoever stands behind the work with their own name. It is not a 100-percent guarantee, but it is closer to one.