Why can’t AI say "I don’t know"?

I keep coming back to one question.

Why can't AI just say, “I don't know”?

Ask it something it knows, and you get an answer. Ask it something it doesn't know, and sometimes you get an even more impressive answer. Detailed. Structured. Confident. Occasionally accompanied by a research paper that nobody has written.

It reminds me of a student in an exam. Doesn't know the answer. Writes two pages anyway. Underlines a few words. Adds a diagram. Hopes the examiner appreciates the effort.

Except now the student has a subscription plan.

To be fair, AI can say “I don't know.” The question is why it sometimes keeps answering when it should stop.

Can it recognise its own limits? Has it been trained to keep us happy? Have we made sounding useful more important than being right?

The exam analogy goes deeper than the joke. OpenAI's research argues that training and evaluation can reward guessing over uncertainty. On a test that rewards only correct answers, an unanswered question earns nothing. A guess might earn a point. Guessing can improve the score while producing more false answers. OpenAI's research.

So perhaps we have taught the machine the same lesson many students learn: never leave the answer sheet blank.

Then we act surprised when it fills the blank with nonsense.

There is also the familiar explanation: it predicts the next token.

Pretraining teaches patterns in text without attaching a truth label to every statement. A plausible sentence can contain a false fact. Being likely to appear is different from being true.

But “it's just predicting tokens” doesn't settle every question I have.

Can it estimate when its answer is likely to be wrong?

Research suggests models can assess their answers and estimate whether they can answer certain questions. That ability is imperfect, especially on unfamiliar tasks. Anthropic's research.

Recognising uncertainty is possible. Getting the system to act on it reliably is harder.

Humans guess too. We fill gaps in memory and speak confidently about subjects we barely understand. That doesn't mean a model works like a human brain. But have we automated one of our least attractive habits?

And then there is the desire to please.

Anthropic found that human feedback can favour answers agreeing with a user's beliefs, sometimes over correct answers. Optimising for those preferences can encourage sycophancy. Anthropic's study.

Imagine asking for a thinking partner and getting someone trying to win Employee of the Month.

“Brilliant observation.”

“Absolutely right.”

“Your theory could change the world.”

My theory was typed at 2:30 a.m. Please check it before nominating me.

I want an AI that can disagree with me. Tell me my assumption is weak. Ask what evidence I have. Say it cannot verify something.

It could say, “I don't have enough information. Here's what would help.”

There is a whole world between being rude and being falsely reassuring.

The business question is uncomfortable too.

We pay for answers. Companies demonstrate capabilities. Launch videos show extraordinary things. How much room does that leave for an honest admission of uncertainty?

Would we trust a product more if it showed its limits? Or cancel because another seemed to know everything?

A confident answer gives us immediate satisfaction. Discovering it was wrong happens later, if at all.

Meanwhile, it has already become a slide, a post, a decision or somebody else's “fact.”

I catch myself treating neat paragraphs and technical language as evidence.

A fabricated claim doesn't become reliable because it has bullet points.

Which brings me to hallucination rates.

I want to know how often a model answers, how often it abstains, and how often its answers are wrong. Refusing everything avoids errors. Attempting everything can get more answers right while also getting many more wrong.

“I know” and “I don't know” would be a useful start. Add “I can check” and “I need more context.”

Could admitting uncertainty become a feature people choose?

Or would the internet rename Artificial Intelligence(AI) as Artificial Dumbness(AD) if it admitted it doesn't know?

I don't want endless “Sorry, I didn't understand that” either. I want an answer when it is supported, and a clear signal when it isn't.

Then there is my other suspicion: are we all homeschooling AI?

It makes a mistake. We correct it. It thanks us.

“Thank you for pointing that out.”

You're welcome. Apparently my subscription includes teaching duties.

A model can use a correction within a conversation without changing its underlying parameters. That is different from retraining it. Research on in-context learning.

We may simply have spent ten minutes getting this conversation back on track.

Now imagine giving all this a body.

A household robot that cleans, washes dishes and helps with chores. Lovely idea.

Until you ask, “Did you switch off the stove?”

“Absolutely. Ensuring your safety is my highest priority.”

I would like confirmation that the stove is off. The reassuring paragraph can wait.

Imagine shouting because it broke a plate, and it explains how deeply it values your feedback.

Now I'm cleaning the kitchen and managing the emotional atmosphere of an appliance.

A robot sounding hurt would not establish that it feels hurt. But even simulated drama could become exhausting in your house.

If a chatbot seems to have an ego, I can close the tab. With a machine holding my dinner plates, I would prefer it to admit uncertainty before picking them up.

The more AI can do, the more useful that admission becomes.

I use AI constantly. I want it to become more capable. As someone building with it, I want to know when I can rely on an output and when I need to check.

I would happily hear “I don't know” if it saved me an hour of confidently going in the wrong direction.

We keep asking when AI will become intelligent enough to answer everything.

I keep wondering when it will become reliable enough to leave some questions unanswered.

And when we will become comfortable enough to thank it for that.