niiwaa
FREN
View subscriptionsRequest a demo

Poor Albert!

fotor ai 20250413175738 300x300Poor Albert!
Why can't generative AI reliably answer advanced questions?

 

I'm talking about the application here Albert, the AI ​​deployed in theFrench administration and which, according to sources, would offer relevant answers in 65% to 10% of cases.
65% means that 1 in three answers is wrong. And it seems that this is the officially announced rate, a good average for an LLM.
10%, that means that 9 out of ten answers are wrong. And it seems that this has been the rate assigned by beta testers for a few months now. “A little worse.”

Let's be clear, no tool or service is and will not be used if it is not reliable. Reliability is not perfect, but close to it. We therefore expect performances of the order of 95%, even 99% !

So, what's going on?
For us, it’s simple:Generative AI is not designed to achieve 95% reliability in its responses on specific topics !
It is designed to, on the basis of thousands, even tens or hundreds of thousands of sources, offer us a “moderately” acceptable answer.
It's already great. But it is only reliable for situations where there are many data sources (on the internet) which give correct answers. We are not even addressing the notion of fakenews here.

So what's the problem with Albert?
It is frighteningly simple: based on a specific question, the source texts offer “only” a few sources directly related to this question. A few sentences, maybe a few paragraphs, a page in total, at most.
The rest has nothing to do with it. So when we ask him to answer, he bases his answer on an average which, for sure, will not be good (in the statistical sense of the term) since there are many more sources off topic than IN the topic.

The extreme case, there A SINGLE SENTENCE that contains the answer.

There, you understood, it’s dead. The laws of statistics are formidable: no calculation possible with less than THREE sources.

Is it unsolvable?
I don't think so.
And I am sure that generative AI alone will never be able to solve this type of problem.

We specifically designed niiwaa to be reliable in this kind of situation.
Reliable to the point where its response is correct in at least 95% of cases.

niiwaa is a monitoring tool, but its algorithm can be used in other use cases where the reliability of the response is the primary criterion of use.

Niiwaa, outil de veille multilingue

To research :To research

Niiwaa, your unique multilingual monitoring tool in the world! Request a demo

To research

To research

Recent articles