Designing quality signals when AI makes everything look credible

Some time ago, I was leading monetization at a large ed-tech Q&A platform. Millions of students, one job: give them answers they could trust. We charged for access to those answers. For years, that was a perfectly good business. Then generative AI showed up, and the problems began. Not because the competition got better, but because the signal broke. Overnight, any student could paste a question into a chatbot and get back something fluent, formatted, and confident. It looked exactly like one of our answers. Sometimes it was just as good. Often, it was not. But the difference was not always obvious.

That is the part most people miss. AI severed the link between looking credible and being credible. On our platform, confidence used to be a decent proxy for quality. Now, confidence was free.

When everything looks credible, nothing does. And that is a serious UX problem.

The badge was my first instinct, too — and the weakest

My first reaction was like everyone else’s: slap a label on top of it. Tag the AI-generated content as “AI-generated.” Put a “Verified” badge on the expert answers.

It barely moved a thing.

Let’s face it: we have all learned that “Verified” can mean “someone ran a script.” Five gold stars can mean “the seller bought reviews.” A checkmark can mean “this account paid eight dollars.”

---

---

Under AI, badges get even weaker because everyone adds them. On their own, they become visual noise.

So we stopped decorating and started rebuilding. Credibility is not a sticker you apply. It is a system you design. We invested months in understanding what makes an answer trustworthy in students’ eyes and rebuilt our whole approach to providing answers, especially AI-generated ones.

The four signals we rebuilt trust with

Source transparency: Are your sources more than just scraped links?

Authorship is who wrote it. Source transparency is the homework behind the answer: the references a user can actually follow.

This is one we underused for a long time. We added a few web links as references and called it a day, like most of our competitors.

However, with everyone doing that, the actual quality and relevance of sources matter. Scraping the internet and providing random links to random websites does not do the trick anymore.

So ask yourself: What constitutes the most credible source of information for your specific use case? Then invest in grounding your answers in that.

In our case, the answer was simple: our users were students. Students learn from textbooks, and teachers create quizzes from textbooks.

Our metrics — engagement, return rate, satisfaction, and clicks on sources — skyrocketed when we added textbooks as a source, because:

• Textbooks students actually use in class provide more relevant information than random articles
• Vetted sources, like textbooks or research papers, are much more trustworthy than most websites

Having one to three textbook references performed much better than even 20 scattered internet references. Yes, we tested that.

Validation: Who checked this, and when?

Content is not credible because it exists. It is credible because someone vouched for it recently, and users can see the trail.

This was our quiet superpower. Expert-verified answers were the genuine aha moment — the thing students came back for — and the data was clear: more expert-verified answers early in a session meant a higher conversion rate.

So we decided to double down on that. Because again, anyone can slap a “Verified” badge anywhere they want. But giving the verifier an actual name and face changes the perception dramatically.

Our experts were not random people. They were people actually working with school curricula or teaching students.

Just as textbooks were more relevant and trustworthy sources than Wikipedia links, teacher-verified answers were more credible to our students than answers “verified by someone.”

Confidence communication: Can it prove its correctness?

AI models can sound confident even when they are completely wrong. So, how do you make them prove they are right?

Sources are one answer. But let’s also be honest: people are lazy, and they are not always eager to browse through sources to double-check an answer on their own.

So we made “examples and evidence” a dedicated section for each answer.

Whenever users were unsure whether an answer was really correct, they could expand a list of real-life examples and well-known evidence related to the concept. Even though most people did not click into that section, simply adding the option boosted our engagement metrics by more than 20%.

We also took a risky bet and included ChatGPT and Gemini answers in our product. Why? Because we learned that our users were already comparing answers across different platforms. Giving them the ability to check different providers made them trust our answers more.

It also sent a powerful message: “We trust the quality of our answers so much that we are not afraid to put them next to our competitors’ answers. We are the best, and you can check for yourself.”

Similar to the examples and evidence section, very few people actually used this option. But simply giving them the choice increased our metrics significantly.

Reputation: Social proof still matters

Sometimes it feels like we have completed a circle. A couple of years ago, people were adding social proof everywhere.

Then AI came along, and the industry shifted toward metrics, badges, sources, and other trust indicators. But while great sources and expert verification matter, people still like to hear from other people.

Although social proof no longer plays the main role when it comes to AI products, it is still a valuable trust builder.

The same four signals work everywhere

These signals work for most products, not just ed tech. Once you start looking, you can spot them — or their absence — everywhere AI touches content.

Search

Answer engines live and die on source transparency and honest confidence. It is why Perplexity made clickable citations its whole personality, and why Google’s AI Overviews were mocked for cheerfully telling people to put glue on their pizza: a confident answer with nothing behind it that anyone would actually want to check.

Users do not need the summary to sound authoritative. They need to see what it is built on, and they need a clear signal when the model is guessing.

Documentation

As docs become AI-generated, validation and confidence carry the weight.

A “last reviewed three months ago” stamp matters. A version tag matters, so readers know whether the snippet still matches the current API. A quiet “this section was auto-generated; verify before you rely on it” matters.

Auto-generated docs that silently rot are worse than no docs because they still look maintained.

Publishing

Publishing relies on reputation and validation: Does this author have a track record? Did a human actually check this?

Amazon is already drowning in AI-spun books with slick covers and nobody behind them. The fix is the move health sites made years ago: a real byline next to a visible “reviewed by [name].”

Readers can forgive AI assistance. They do not forgive being deceived about it.

Customer support

Customer support needs confidence and source transparency, or you can end up in court.

Air Canada’s support bot confidently invented a refund policy that did not exist, and a tribunal made the airline honor it. A good assistant does the opposite: it cites the exact help center article, and when it is not sure, it says so and hands the user to a human instead of bluffing.

The same pattern appears every time. The products that win trust are not the ones with the most badges. They are the ones that let users check for themselves.

Common pitfalls

Here are a few ways smart teams still get this wrong — mine included.

Badge theater

These are labels users have already learned to ignore. A “Verified” badge that means nothing is worse than no badge at all. It becomes noise that buries the signals that actually matter.

Decorative sources

These are citations that look like homework but are not: links to pages that do not actually back up the claim. Users only need to catch this once before they stop trusting anything you cite.

Stale validation

“Verified” with no date is a promise you stopped keeping. A check from two years ago on a fast-moving topic is worse than no check at all because it still reads as current.

Confidence laundering

This happens when AI is allowed to sound certain about things it is not certain about. If your system cannot express doubt, it is lying by omission every time it gets something wrong.

Whenever I see AI tell me it is not sure about an answer, I trust the other answers a little more.

Signal overload

Forty trust cues on a page is the same as zero trust cues on a page. Pick the few that carry weight and cut the rest.

Lipstick on a weak product

Signals are a multiplier on real quality, not a substitute for it. Bolt them onto a mediocre answer, and you have just helped users find the mediocrity faster.

The takeaway

AI did not break trust. It broke the cheap proxy for trust: the assumption that polished and confident means probably right. That proxy was never reliable. AI just made it worthless fast enough that we could no longer pretend otherwise.

And to be fair, I lean on AI constantly. I also catch it making things up every week. Confidently. In the most assured voice in the room. That is exactly why our interfaces cannot borrow that voice for free.

Stop labeling and start structuring. Credibility was never a sticker. It is a system, so design it like one. Do your homework, then ship.