AI Chat Scams: Human or Bot?

Author

Calvin Chen
March 27, 2025

We previously discussed the growing prevalence of AI in scams and since then, it has made the news again. Given that seniors are disproportionately impacted by scams, we felt it was important to cover this topic again but with more focus on education and protection. The key to understanding how best to break AI-based chat bots used in these scams is to understand how Large Language Models (LLMs) work.

Click the following link to read our previous article: Protecting Seniors from AI Scams.

​Advancements in artificial intelligence (AI) have led to the rise of voice cloning scams, where fraudsters use AI to replicate voices of trusted individuals to deceive victims into providing money or sensitive information. These AI-generated voices are increasingly sophisticated, making it challenging to distinguish them from real ones.

To protect against such scams, experts recommend establishing unique verification methods. One approach is to agree on a "safe word" or ask personal questions that only the genuine individual would know, such as details about recent events or shared experiences. This strategy can help verify the caller's identity and prevent fraudulent activities.

Additionally, being cautious of unsolicited calls, especially those requesting urgent financial actions, is crucial. If in doubt, it's advisable to hang up and contact the person directly using a known, trusted number to confirm the request's legitimacy.

Staying informed about the latest scam tactics and maintaining open communication with friends and family about potential threats can further enhance one's defense against these evolving AI-driven schemes.

ZNEST’S TAKE
Key Takeaways

  • Education is key to fighting scams, especially for seniors, by helping people understand how scam technologies work.

  • Scammers exploit emotions—fear, urgency, excitement—to pressure victims into quick decisions; staying calm and thinking critically is a strong defense.

  • Encouraging open conversations about scams reduces stigma, promotes early reporting, and helps protect others through shared awareness.

  • Large Language Models (LLMs) like ChatGPT drive many AI chat scams.

  • LLMs can’t recall personal memories, making them vulnerable to questions about specific, private moments that only real people would know.

In today’s fast-moving digital world, scams are becoming more convincing—and more dangerous—than ever before. From deepfake audio to AI-generated messages and phishing attacks, fraudsters are constantly evolving their tactics.

Education Is Our Best Defense

Most scams rely on catching people off guard. They work because the victim doesn’t recognize the warning signs. But when people are educated about the common tricks scammers use—like fake emergency calls, suspicious links, or too-good-to-be-true offers—they’re far more likely to pause and assess the situation before falling into a trap.

Scammers thrive on emotional reactions. They create urgency, fear, or excitement to push people into making quick decisions. But when someone has been educated to stay calm and think critically, they’re less likely to get swept up in the scammer’s narrative. A bit of skepticism goes a long way.

Seniors who aren't very tech-savvy are frequent targets. Tailored education—through workshops, online tutorials, or even family conversations—can make a huge difference. When these groups understand the risks, they’re empowered to protect themselves and others.

With that in mind, our article will be focused on:

📱 Tech Knowledge = Scam Resistance

It’s not enough to say “don’t trust phone calls or emails.” Seniors need to understand why and how these scams work. For instance, knowing that scammers can clone a voice or fake a caller ID helps individuals take that extra step to verify a call—even if it sounds exactly like a loved one.

Our goal is to promote a culture of openness. When people are educated about scams, they’re more likely to talk about them without feeling embarrassed—whether it's reporting a fraud attempt or warning others. This openness is powerful. It encourages early reporting, helps others avoid similar traps, and gives law enforcement the information they need to respond effectively.

How Do Large Language Models (LLMs) Work?

Most AI voice and chat scams work by having a scam victim interact with a chatbot of some sort. No matter how good the voice cloning is, it doesn’t work if there isn’t a believable interaction between the bot and the victim. The engine driving that interaction is an AI model like ChatGPT, which can understand and generate human-like text. It might seem like magic, but it’s actually a very advanced guessing game. If you’ve ever used autocomplete on your phone, you already have a basic idea of how LLMs work.

Think of an LLM as a super-powered student that has read millions of books, articles, and websites. But instead of memorizing everything word for word, it learns patterns—how words, phrases, and sentences fit together in different contexts.

For example, if it sees the sentence:

"The sky is ___."

It knows from experience that the most likely word to complete this sentence is "blue."

At its core, an LLM is just a really advanced word predictor. Instead of thinking ahead like a human, it focuses on guessing one word at a time based on what came before.

For example, if you type:

"Once upon a time..."

The model predicts that the next words should be something like:
"... there was a princess."
"... a great adventure began."
"... a boy found a magic sword."

It doesn’t actually know what’s right—it just chooses the most probable answer based on all the text it has seen before.

Understanding Context 

You might be wondering, “If AI just predicts words, how does it understand what I’m saying?”

That’s where context comes in. LLMs analyze the words before and after to figure out what you mean.

For example, consider these two sentences:
1️⃣ "The bat flew through the night sky." 
2️⃣ "The baseball player swung the bat." 

Even though "bat" is the same word, the model understands that one refers to an animal and the other to sports equipment. This ability to adapt based on context is what makes AI responses feel so natural.

How to Beat an AI Chat Scam

Even though LLMs are impressive, they are not perfect and they are no well suited for deep, personalized conversations. Remember, an LLM is essentially a big probability calculator for words, and the way it calculates probability is by using the billions of examples it was trained on. Those probabilities cannot perfectly encompass the details of any single person’s life.

When approached by a potential AI chat scam, the simplest thing to do is to stay calm and ask random questions only you and the other person would know. It cannot be anything generic (which has a high probability of being guessed correctly) or personal details that have been shared on social media such as a favorite team or vacation destination. Instead, ask about a small cherished moment or a trivial detail that is personally important. Asking more questions increases the odds that the chatbot will get something wrong, and when in doubt, hang up and confirm with a trusted source.

Did you find this editorial helpful?

Login or Subscribe to participate in polls.

Senior Living Stocks

Have a topic you would like us to cover? Or just general suggestions? Please let us know!

[email protected]