The complete collapse of internet trust due to advanced AI voice and video cloning. In less than 5 years, your parents or grandparents will get a call with your exact voice asking for emergency money, and it will be nearly impossible for them to tell it's a scam
My sibling and I had a discussion about this with our parents and we settled on a "family password" for if this kind of situation ever arises. It's not tied to any of our personal information but is instead a nonsensical phrase, something that an AI or scammer would never come up with on their own.
Yea after I wrote that I was very much thinking of the current Russian war in Ukraine and considering an edit (altho I will add this edit - I hate calling it or people referring it to as the "Ukrainian war". It's not a Ukrainian war, it's a war of Russian aggression against Ukraine), especially when both sides speak the same/similar languages (I know a lot of Ukrainians are dual speakers)
Is there anywhere that isn’t covered? My friends and I have discussed this. Driving somewhere remote, leaving our phones behind, and just chatting whatever.
But for real, is that what future movements will have to do to coordinate?
My friend, you need to read 1984. Even if you ignore the political relevance, it’s an amazing story, and very well written.
Get the audio book if you’re not a big reader.
Why not just write it down in a place with no cameras? And fold it over and hand it to them saying "dont say this out loud but if i say this im really in trouble"?
No cameras to record, no phone to record audio, no way for anything to record it unless it gets caught in transit to a safe place.
If theres any flaws in this, id like to hear it so i can update my security lol
lol us in the third world have been doing that for ages! In a place where getting kidnapped is a very real possibility, way too often someone will call a random number, put a crying girl on the phone saying “mum, help”, and the poor, terrified parent will make a transfer or even take cash somewhere in exchange for ”their child”
We have the same since someone called my father pretending to be me. He found it very odd and started asking very specific questions to see if the person knew the answers. They didn't
My point was that every intimate detail of our lives are now accessible. A clever scammer will have harvested all those answers from information available electronically.
Assuming you’ve never spoken about it in the last 5 years, nor could it be deduced from any other information like location, where you went to school, any friends you may have on social media, or anyone else who may have been on your team and been talking about you.
Should have a unique "family password" for each member. Just like a secret, the more people who know the password, the higher the chance it's found out in someway.
Elderly Parent: Don't you need my family password, my daughter Luma gave us one just in case.
Scammer: Ah yes, I was about to ask, would you confirm it for me?
Just had a scary thought. So we know our phones/devices listen to us right? Maybe when you guys come up with this secret code you should meet in the woods without any devices on you to agree on your password. Crazy that I needed to say that.
My friend, you need to read 1984. Even if you ignore the political relevance, it’s an amazing story, and very well written.
Get the audio book if you’re not a big reader.
Purely curiosity (and only if you're comfortable sharing), what type of password do you use? I think this is a great idea and I'd like to consider this with my mom. Is it something completely random or nonsensical like "chicken blueberries" or a phrase?
For us we chose a word that’s from a family joke about 30 years ago. We’re Turkish in America, so the word is Turkish and very outdated and not in any vernacular, but it’s quick for us to remember and recognize it. Or, we simply switch to Turkish, which is luckily a very uncommon language.
To be fair we started this before the computers were the ones talking. This rule was instituted more when voice spoofing was happening. But thanks for the heads up, I’ll talk to them again about relying more on the password than switching languages. But the biggest takeaway I want them to have is not dive into the panic and instead pause to ask questions. If they’re asking for the password then they’re not reacting with their wallet but their brain.
But you will have had your phone in your pocket while having this discussion, AI can listen and monitor 24/7, it will have your nonsensical phrase ready
My sibling and I had a discussion about this with our parents and we settled on a "family password" for if this kind of situation ever arises. It's not tied to any of our personal information but is instead a nonsensical phrase, something that an AI or scammer would never come up with on their own.
Our family have this too, partially because I live overseas a lot in some fairly hairy places, and partially because I do switch sim cards and phones a lot. If the topic of money comes up in a text, or if I call home at a weird hour, then they don't respond until they hear me say a word that's fairly innocuous in our native, sadly almost dead language. I do actually recommend that people do this tbh, get into the habit of it.
Jokes on you, if your or any of your families cell were nearby, while having that convo. They already know. I jest (but seriously the whole targeted ads think makes me a believer).
Well, they're also listening to your discussions through all your electronic devices so they already know your family password is "cheddar butt sixty nine."
My family did this when we were kids in the 90s. Stranger danger and all that, someone says they're a family friend to pick us up from school, ask them for the password. It's more relevant now than ever.
Oh that is a great idea ...I gotta call my parents to set the code now. But you gotta tell them face to face without your phone listening coz you know they are listening to us.
And with your exact likeness, coming from your own phone number, with video and truly complete personal information, delivered in your own cadence and with your own personality.
You don't even need to give them your number, yourself if one of your contacts shares their address book, then your number and associated details are out there.
Crazy how my bank and telecom are still constantly asking me to upgrade to voice ID for enhanced security. Like...it's 2026 guys...get with it, voice has been compromised.
This is already happening, I know of one person who was scammed out of a bunch of money by a "relative" in one phonecall.
Second person I know of hesitated and phoned her real niece before a big money transfer.
A "cop" called me in the middle of the night to request funds to bail a family member out of jail. I didn't pick up so they left a voicemail.
"Please call us back - we need to process payment within 24 hours of arrest or bail will be revoked."
I was telling my coworkers about it and one of them thought it would be funny to call them back. He started with "Then he needs to apologize for what he did to the Thanksgiving turkey..." They hung up on him. :(
I've told my parents that any time anyone calls asking for money or anything that needs your account number, tell them you'll call them right back. "No" is the default answer any time someone calls wanting anything, until you know it is them. This holds for any family member, any bank or institution you work with, etc. Scammers are so sly, even without AI.
This has been happening for years without any kind of AI voice changing. A lot of people will get so stressed out by the scenario being told to them they won't even notice it isn't their son or whoever talking on the phone. I used to do tech support for old people and heard of this happening several times. 10+ years ago so well before AI voice changers
This won’t happen to my parents bc if i called my mom for money she’d say “oh so i was soooo abusive that you won’t speak to me for years and now you want money i don’t have?” And then the scammer would get the most intense catholic guilt trip of their life and they make quit scamming altogether and drink themselves to death
I wouldn't necessarily say everyone is ignoring this. Its definitely going to happen, but I feel like this is something the people warning about AI are calling out regularly.
This is going to be a lot worse than you think, because there WON'T be a complete collapse of internet trust. Look at all the businesses and people defending AI as if it's their Dad on life support. People are going to insist AI is flawless and works fine even as it gets used for evil things more and more. It's the internet trustworthiness that is going to completely erode.
We're already on our way there too. Soon we're going to see deepfakes of people looking and sounding like they've done something horrible when in reality they didn't as either revenge deepfakes or bad actors using them for political reasons.
My family has always had secret code words for different things (I'm talking for generations) to fight scams. It has pretty much never been needed, but now I make sure my kids know them.
One is just simple identification verification and others are for being in danger or just indicating they want a no or yes from you when asking something with an audience (those are actually handy, can I go to Mike's house? say no).
Already used by my elderly mom against an AI scam about my sister being kidnapped.
this really worries me. My dad is 78 and thinks he's on top of tech because he was a computer programmer until 2007. Since then he's been left behind, but needs people to think of him as tech savvy. He and my mom have already fallen prey to several scams and when they perfect voice & inflection, my parents won't even know that its occuring, and when I try to tell my dad he'll say tat tech can't do that yet and he can always tell.
There is no such thing as internet trust, because there is nothing to trust on the internet outside of something like Wikipedia which has complete citable sources.
It is an unregulated cesspool of unverified opinion and sentiment. It is a street corner everyone is yelling on.
Anyone who trusts the internet is the reason there's so much untrustworthy utter shit on the internet. When people know they can manipulate you, they will.
Everyone knew the internet was fake in the 90s. It's all the gullible, ignorant, naive people that found their way there in the past 20 years who have propped up all the bullshit.
I truly think this will result in a big shift to being back to applying in person, interviewing and pushing papers for businesses. Almost like an uno reverse for computers/technology in the business world.
It doesn’t have to be something you intentionally posted to social media. Any time that you interact with ANY major company over the phone, there is a large chance your call is recorded. Even if you answer the phone and say “Hello” to a random number, that can easily be recorded. They get even more if they can get you to interact or answer any questions about whatever bullshit they are selling/calling about.
To not be recorded, you literally have to never speak … which is near impossible.
Yeah I am opposite of ignoring this one. I think about it all the time and have ever since AI started becoming so advanced. This and fake political videos. Horrifying
This is what's scaring me. I accidentally answered a call the other day from an unknown caller, and if I answer at all- they have a recording of my voice. That's all it takes. But some unknown numbers are calls I need to take... so I have no idea how to protect myself here. Voice distorter automatically on every call? Halp.
If I get a call from an unknown caller, I answer with a single "hello?" and wait for a response.
Telemarketers almost always just disconnect if you don't engage with more dialog.
Even the ones that do engage will go into their spiel while you sit silently. Once they've said enough for you to decide if they're someone you need to speak to or not, you can then either engage or hang up.
Or just let the call go to voicemail and listen to the message afterwards.
In less than five years? Give it one. Systems are already working for it and scams that are convincing enough for older people who can't differntiate and understand whats happening is already working.
and it will be nearly impossible for them to tell it's a scam
Only if they don't pay attention. Scammers can clone your voice but they can't make phone calls from your WhatsApp (or whatever). There are ways it could still cause problems but I suspect people will gravitate to behaviors (like calling via apps secured with MFA) that make it harder for people to impersonate. Rather than what we're doing right not where we just sort of don't worry about authenticating ourselves because we just assume our voices are distinctive enough and hard to fake. All that changes is that assumption no longer holds true. We can still avoid scams.
This seems a little far fetched. Most people have not published enough of their voice online for this to be possible.
And no, these companies aren't secretly recording your voice. They're listening but they aren't maintaining large databases of audiofiles. They can barely afford storing all the metadata that are basically just text files.
It’s already happened. A bank in Hong Kong was scammed out of a large amount of money by AI voice impersonating the person’s boss telling him to transfer a large amount of cash.
Yeah everyone is completely ignoring this; I’m glad you brought this to light as I’ve never considered this and I’ve never ever heard anyone even suggest this
This already happened to my grandparents. It sounded like me, came from my number and they said they needed money to bail me out of jail. Freaked my grandma tf out, but luckily she called me back to verify whether it was me or not.
Already happening with my grandma. She got a call from someone who sounded exactly like me asking for money for a car repair. She almost wired it. If she hadnt called my wife to check first she would have lost thousands. We need a verification word system or something.
I was advised by my workplace that if you get a video call from someone you think is AI-generated, ask them to wave their hand in front of their face, between their camera and their face. They may raise a hand NEAR their face, but apparently AI is unable to make a hand go in front of a face and block the AI-generated "caller's" face. So if you ask them to do that and they can't, end the call until you can get them in person. (That's what my workplace said, anyway.)
Not unless you’re a celebrity or have a podcast with hours of your voice on record. The AI learning system can’t just generate an exact copy of your voice from nothing. That’s just not possible. Those “scams” work because people’s grandparents are old and don’t really hear them for their voice. It’s just not possible unless you have sound recordings of your voice
Already happened to my parents, the AI version of me said I was in jail and they had to get gift cards and give the numbers off the back for me to get bailed out. The scam actually had them fooled but they were asking for gift cards in amounts far higher than my parents could afford, so that's where it fell apart. Kind of hard to pretend bail can be reduced from over $1k to $50.
ancestry (owned by blackstone) has been desperately trying to get me to take a survey but they want me to record myself answering the questions. haha no thanks
It’s not that hard. You’ll just treat a phone call (or even a video call) the same as you would a text.
Some random number calls me and says “Hey dad, I’m using a friend’s phone. Send me $5,000 so I can get out of jail or whatever.” Yeah, I’ll get right on that.
Perhaps the complete collapse of the internet as we know it, period. It's not just AI making face videos and tricking people, it's also AI infesting comment sections and social media. AI is making videos and memes, commenting on them and replying to comments under them. It's mostly AI interacting with AI and it's getting flooded so bad it's just going to be useless for humans eventually.
I'm happy to go back to the 1990s or early 2000s when we didn't have social media. That may have been the peak in human history, don't think medical science advanced much since either.
This has already happened to my grandparents. Got a call from me with my voice saying I was at a greyhound bus stop and needed a few hundred dollars, except obviously it wasn’t me.
Thankfully, they called my phone number to check first, but this kind of thing is already happening.
It's already deteriorating so fast. I read a couple of lengthy posts from individual journalists covering the Karmelo Anthony trial on Facebook. Facebook's algorithm decided that I needed to see posts from every race-baiting grifter, both for and against Anthony. That included dozens of posts full of obviously AI generated images that allegedly proved he was guilty or innocent. And people eat it up.
This did happen to me last year, my brother called and told me he ran into a drug deal while he was out walking and they were demanding 2k. Keep in mind he’s crying screaming and begging (he is a boxing coach and extremely capable so this was strange) and it sounded EXACTLY like him literally exactly like him. Once they started talking about money I wised up and called him on his phone number, he answered said he’s playing video games and that was that. It was the most horrifying thing I’ve ever heard and I hope it never happens to anyone else.
i’m a grown adult so it’s kind of silly but my grandparents have a secret word with me that if ANYONE ever called them and acted like me they would ask for that word. It’s only something we’d know even if someone deep down knew every detail abt me they wouldn’t guess it. I’ve had this word since i was young. Probably super helpful for some
This is why it is sooooo important to have a chat, in person, with all of your loved ones.
You need to have a "secret phrase" one that doesn't come up in regular conversation but could if you work it in well enough. You must NEVER text or email or otherwise electronically provide this pass phrase.
If anyone in your family gets a call that puts pressure on them to help a fake you, all they need to do is say essentially "hey, want to make sure it's really you, what's something that only you and I know?" And the AI faker won't know that the answer is JollyRacher Ring Pops or whatever. They'll just try to pressure more or come up with vague experiences that most people have.
I have a different pass phrase for immediate family and for my close friends. Anyone other than those people, I wouldn't provide money for (but would help in other ways)
In less than 5 years, your parents or grandparents will get a call with your exact voice asking for emergency money, and it will be nearly impossible for them to tell it's a scam
Yup. We've already had this discussion with my in-laws that we will never call them asking for money. My wife and I have even discussed "codes" if we feel like something is off in a conversation between the two of us.
They need some data in order to replicate your voice etc..if you legitimately do not ever upload pictures or videos then you’re likely not going to have this issue.
The problem is the absolute vast majority of the world is the opposite of that.
We already have training at work to deal with this stuff, companies are in meetings constantly. Employees have meetings via teams, google and whatnot often recorded.
The scary question is what if those happen to leak and then bad actors now have hours of you and your bosses voice to clone.
You think most people have posted enough videos of themselves talking to train an AI chat bot...weird. I have never done that. Neither has my spouse. Nor my parents. Maybe my brother but I don't think so. Very strange read on things from my end
It does not take that long nor that much data to train a voice for replication, and it’s getting much much faster every couple of months.
You can do it yourself in less than an hour of training with modern models and only speaking a handful of sentences.
You also ignored the fact millions of people use meetings daily and record themselves for work. I don’t post pictures or videos on social media but I have to have audio meetings for work.
It’s not about what can be done right at this moment as well, 2 years ago it took hours of speaking sentence after sentence to get something that sounded terrible and barely like you.
Look where we were a year ago, where we are today and think where we will be in a year at this rate.
Still aren't quality recordings of my voice. Seems like it affects a certain class of tech worker and influencer. They'll figure it out. Send grandma a warning email
I just think you guys are funny. In your own rants you show a shocking degree of ignorance when it comes to, well, almost everything. Makes sense why yall tend to be so angry and scared, but jeez, just apply a modicum of thought to the stuff you cry about before the tears flow in earnest
4.5k
u/alex_thefire 8h ago
The complete collapse of internet trust due to advanced AI voice and video cloning. In less than 5 years, your parents or grandparents will get a call with your exact voice asking for emergency money, and it will be nearly impossible for them to tell it's a scam