Sometimes you read something so weird that you take it twice and then find yourself looking for the original source or more information because there’s just no way anyone really said what you just read. I had such a moment this week after seeing someone tweet about Amazon wanting to create a deepfake that would allow Alexa to channel a dead family member.
The report the tweet was based on was made at the Amazon Re:MARS event on Wednesday, where Amazon Senior Vice President and Alexa Chief Scientist Rohit Prasad showed how well Amazon can mimic speech. based on just one. minute sample of someone’s voice. What TechCrunch reports:
In the scenario presented at the event, the voice of a deceased loved one (a grandmother, in this case), is used to read a bedtime story to a grandson. Prasad points out that using the new technology, the company can achieve a very impressive audio output with just one minute of speech.
TechCrunch labeled this as a “potential new feature”. So I contacted Amazon to get a little more information. But I’m not sure if Amazon spokespeople can really answer the bigger question, which is: Why? Why would people want to hear a deep fake from a dead loved one reading them a story? Or give them the weather?
While I too would love to hear the voice of a deceased loved one read me a story, what I really want to say is that I would love for them to be alive again. A fake version of a loved one immediately falls into the uncanny valley and lodges there as the foundation of a new dystopia.
But maybe I’m just old and jaded. So I decided to ask my teenage son what he thought of the Amazon idea. Unfortunately, his first question made it clear that my cynicism had found a home in my offspring. “Would Amazon make this so they can sell it to people because they miss his grandparents?” they asked. “Is it about money?”
I explained that it was a way to show the advances of AI when it comes to being able to replicate people’s voices. Then I asked them if they would like to hear my voice if I were dead or just traveling. “But I would know it wasn’t you,” they said. “This robot would be using your voice.”
In other words, they weren’t having any of it. Now, my son is not a representative sample of the population by any means. But the fact that a 15-year-old was so quick to identify how creepy this example is and express the horror of it within minutes of hearing it made me wonder what, exactly, the guys at Amazon were thinking.
Is this a simple case of the engineering department getting excited about technology and trying to explain why it’s so ingenious in a way that, to many normal people, is deaf? Or exaggerated reports that took a sloppy statement and turned it into something big? Or is Amazon testing the waters to gauge overall readiness for a technology that looks creepy but might actually appeal to some people? (I think of Amazon announcing the flight Always call home camera two years ago as an example of this.)
I often read the news and imagine how new technology can be used against people, or to monetize people while providing them with little real value. For example, I still have deep questions about the effects of paying for hardware as a service and how it affects the ability of consumers to generate value through their assets. And when I look at efforts to collect more and more personal health data in one place by offering new “features”, I tend to think that technologists might be selling us digital snake oil.
But this particular demo seemed so deaf to me that I was left wondering what the positives would be for anyone, even Amazon. if you want sell us more celebrity voices on Alexa, summoning a dead grandmother and a bedtime story seems like a terrible way to go.