The Prime Effect: Alexa, AI And What Makes Jeff Bezos Tick

Sign up for the On Point newsletter here. 

Who is Jeff Bezos? In the second part of our series Amazon: The Prime Effect, we hear what the creation of Alexa tells us about the man who built Amazon.

Brad Stone, senior executive editor for Bloomberg Technology. Author of "Amazon Unbound" and "The Everything Store." (@BradStone)

Janet Slifka, director of research science in the Alexa AI division at Amazon. She's worked on Alexa technology since 2012.

On the drawing that started Alexa

Brad Stone: “The idea for Alexa comes from Bezos, late 2010, and he is looking for a way to take advantage of Amazon's advantage in the cloud. AWS is this cloud computing service where devices or other companies will use Amazon's data centers. And at the time, he is also enamored with voice computing. He's a big science fiction fan. He thinks that the future of computing might be a voice.

"And he is in a conference room with basically his assistant at the time, a guy named Greg Hart. And they're trying to figure out what Hart's next job is going to be, because he's kind of leaving that chief of staff role. And Bezos asked him to realize this vision of a voice computer. And the two of them sketch out this drawing. And it's a lot like what Alexa became.

"Bezos identifies it should have a mute button if people want their privacy, God forbid. And that one of the challenges will be how do you get it on the wireless network if it doesn't have a keyboard. Because you can't yet talk to it to configure it. So that's in the drawing. And Amazon, they're hard to write a book about. They can be kind of secretive. But they actually gave me this drawing.”

Is it fair to say that the idea for Alexa first sprang out from Bezos's head?

Brad Stone: “When you do the archeology on the big ideas at Amazon, and you dig down into the various layers, you often get to a crazy idea from Bezos. And sometimes the ideas go nowhere or they flop. And that's the fire phone, or other prominent examples. And sometimes they're the big ideas. And yeah, Bezos seems to have this magical elixir of, you know, technical insight about what is perhaps not possible today, but could be possible tomorrow. You know, he's very technically savvy. And since he's a science fiction reader and fan, he likes to conjure the ideas that he grew up with, or that he enjoys in science fiction.”

On the core qualities of Bezos that unfold in the story of Alexa

Brad Stone: “We talked about the ideas. Maybe that's even the easiest part. But it is the detailed focus and follow up. And so he puts Greg Hart, his technical assistant, his chief of staff at the time, in charge of the project. But then he starts meeting with them once a week, twice a week. He micromanages the selection of Alexa's voice. And that's a voice actress who I identified in the book, a voice actress named Nina Rowley.

"He selects the name Alexa, and determines that there should be a second name for the device, which they at the last minute call the Echo. The other attribute, I think, is the incredible impatience and stubbornness and high standards. These are not necessarily good adjectives, but I have a scene in the book where he basically challenges his voice engineers about how smart Alexa is becoming, and why it's taking so long.

"And he determines that it's going to take years for them to catch up to, say, Google, and he stands up and walks out. He says, you guys aren't taking this seriously. And, you know, where does that come from? I mean that is maybe the technology titan template, right? The impatience that we saw with Steve Jobs or others. And it makes him difficult to work for. But it also clearly, in some respects at Amazon, produces results.”

You write that Jeff Bezos would not let go of the idea of Alexa being a more sophisticated generalized computer. Can you talk about that?

Brad Stone: “We talked about his science fiction fandom, the origins in the books and the movies that he loves. And he wanted this thing to be conversational. And I have a funny anecdote in the book where he's testing one in his home. And he's doing what your listeners are describing, kind of interrogating Alexa. That seems to be one of the big applications for Alexa, just kind of seeing what it does. And Bezos gets frustrated and he says, Alexa, go shoot yourself in the head.

"And the engineers on the project are terrified because they all think they're going to lose their jobs. But in those meetings, Bezos wanted this thing to be a personal assistant. Among the names that he was contemplating for this thing was Samantha from Bewitched. He really saw it as this personal assistant that would just show up, twinkle its nose and and do everything. And the engineers and marketers were terrified. They knew that when they released this thing, it needed to be practical. So they wanted it to be a music player and they kind of compromised in the middle.”

On how Jeff Bezos’s vision and personality shapes his employees

Brad Stone: “It's very powerful. And the employees, particularly the long-term employees at Amazon, like Janet, they tend to exhibit his mannerisms, they use his vocabulary. They completely inhabit the sacrosanct leadership principles that Amazon posts on its website, and all around its offices. It's amazing.

"In my first book about Amazon, The Everything Store, I use the word Jeff-Bots. And boy, they didn't like that at all. But it captured something. I was trying to be funny. It captured something true about Amazon employees, which is they tend to feel like they've observed their boss quite carefully.”

Bezos is going to step down as CEO this year. What does that mean for Amazon?

Brad Stone: “I say in the book that Amazon is scaffolding built around Jeff Bezos's brain. And when you remove that, you remove the brain, at least as he steps aside, you know, you wonder, does the structure hold up? And he says he's not going anywhere. He's going to become executive chairman. He's going to continue to work on new projects.

"But then you look at the vast array of other responsibilities he has, including giving away that $200 billion dollar fortune. He's building a super yacht. So he's going to be sailing the high seas. He's got a space company. And you wonder, will he drift away over over the next few years?

"What he would say, what they would argue, is that he's put in place a system of invention. And that Amazon can continue without him. But I wonder. Because we've told the story here of Alexa. And he's so fundamental to the company's inventiveness, who will replace that? Probably we won't see the answer to that in the short term, because, as I said, he'll stick around. But in the long term, it's probably the biggest challenge facing Amazon.”

This hour is our second in our series Amazon: The Prime Effect — where we take an in-depth look at the myriad ways Amazon has changed the way we shop, work and how we live. In this episode, we focus on Amazon founder Jeff Bezos — his ambitions, and how he built the company and its culture.

[Tape] BEZOS: Our vision was in the long-term it would become the Star Trek computer. You could ask it anything and ask it to do things for you, to find things for you and it would be easy to converse with in a natural way.

CHAKRABARTI: That’s Jeff Bezos in a 2016 interview, describing his initial inspiration for Alexa, which began development in 2011. Consider the ambition behind that statement. He wanted to develop a computer, like the one in Star Trek. A seemingly all-knowing, linguistically sophisticated computer that you could chat with — frustrated futuristic Scottish brogue or not — but at that point, a computer that only existed in television fiction.

JANET SLIFKA: What was exciting about it [was] Amazon was stepping into that area without having a long history in that. Like at that time in 2012, if you listed the technology companies that worked in language understanding, I don't think Amazon would have been in the list.

SLIFKA: My name is Janet Slifka. I'm a director of research science in our Amazon Alexa AI organization.

CHAKRABARTI: Slifka had the kind of expertise Bezos needed — a career in speech and acoustics that had already taken her to MIT and speech recognition software companies. She joined Amazon in 2012. The Alexa project was still secret.

SLIFKA: It was early, we didn't have anything that worked yet. But lots of enthusiasm to put something together.

CHAKRABARTI: Enthusiasm and urgency, because remember, Jeff Bezos wanted to make something like the Star Trek computer. So, to make that happen, the product had to be smart, it had to be able to learn, which meant, the Alexa team needed a ton of data.

[Tape] BEZOS: Very challenging from a technical point of view. There were probably more than 1,000 people working on Echo and Alexa.

CHAKRABARTI: Data, in this case, meant human speech. People talking.

SLIFKA: The notion of what would delight customers would be that you don't have to go find a device, and unlock it, and bring it up and speak to it, which certainly was the model I had been working on prior to joining. It was this nice, big idea about, Look, I just want to talk. I just want to speak out in the open air and have things happen. Like, that would be magical.

CHAKRABARTI: So, the team handed out Alexa-enabled prototypes to Amazon employees. Including Jeff Bezos, who tested Alexa at his home. It did not go well.

As Brad Stone writes in Amazon Unbound, engineers reviewing the data heard their boss get so frustrated by Alexa’s lack of comprehension, they can hear him telling the machine, 'Go shoot yourself in the head.' But, I mean, when you want the Star Trek computer …

CHAKRABARTI: But then Janet Slifka came in with an idea. It became known internally as AMPED.

SLIFKA: [Laughs] Um I was very involved in that.

CHAKRABARTI: No longer satisfied with employee testing at home — that didn’t give them enough data — the Alexa team went into the field.

SLIFKA: We would call this, in the lingo, a supervised data collection.

CHAKRABARTI: They rented out different homes and apartments initially in the Boston area ... where they paid a constant stream of random people to come in and read a script.

SLIFKA: We can't just use a studio room that we've set up in a lab and think that that's going to be good enough. Right? We wanted to say, let's get out into real spaces that, as you know, can be quite quirky. So you want to have confidence that you're meeting the expectations of all of your customers.

CHAKRABARTI: But of course, the people reading the script had no idea why they were doing it, or that Alexa prototypes, hidden under cloth, were listening. So, in addition to the scripted speech, engineers collected a lot of … let's call it, colorful, frustrated language as well.

CHAKRABARTI: But, by 2014, as Brad Stone reports, AMPED had done its work. Amazon had increased its store of speech data by a factor of 10,000. While Bezos didn’t know about AMPED ahead of or during the program, he was happy with the results.

CHAKRABARTI: The Amazon Echo, enabled with Alexa, launched on November 6, 2014. Janet Slifka remembers the moment.

SLIFKA: The moment of launch you don't know what the first feedback is going to be. And of course, first, feedback's all over the place. But I remember a couple comments were a little bit like, oh, well, what is that? And why would I want that? [Laughs] There was a bit of puzzlement that very quickly turned into delight, like, oh, wait, this is actually awesome.

CHAKRABARTI: Amazon sold about 5 million devices in the first two years. In 2016, Bezos paid $10 million for a Super Bowl advertisement featuring football great Dan Marino and actor Alec Baldwin.

[Tape] 2016 SUPERBOWL AD:

BALDWIN: Alexa, how many championships has Dan Marino won?

ALEXA: Dan Marino has won zero championships.

MARINO: Alexa, how many Oscars has Alec Baldwin won?

ALEXA: Alec Baldwin has won zero ... 

BALDWIN: Alexa stop. Well played, Marino.

CHAKRABARTI: Alexa is one of Amazon’s most successful products. But what, fundamentally, is its purpose? Well, ask, Alexa:

[Tape] ALEXA: I was made to play music, answer questions and be useful.

CHAKRABARTI: Janet Slifka says it’s that last attribute that defines how Amazon corporate culture, and Jeff Bezos’s relentless drive flows into Alexa’s development.

SLIFKA: I think fundamentally that phrase about be useful, there's so much in that ... because it does imply a personalization. Be useful to me as a customer. And that means deliver me the experiences I want, with the language I want, change with me as what useful means for me as a customer. So it might look like a fairly short set of words to be useful, but there's so much in that. And I think that's what we keep striving to deliver.

CHAKRABARTI: For example, right after launch in 2014, Bezos instructed the team to develop new useful Alexa skill sets every week. They pretty much still stick to that, seven years later.

SLIFKA: The way that innovation at Amazon thrives, or at least in my experience thrives, is because of the way the leadership principles capture that philosophy, that proven track record of success for fostering innovation. And I think [that] in a way that's so baked in, it's so integral. And that's just the fabric of every day and is a constant part of Amazon. Independent of how people shift their focus.

CHAKRABARTI: Recall, Janet Slifka is the director of research for Alexa AI So what does she want Alexa to be able to do next?

SLIFKA: Addressing how creative people are with all languages, and mixing languages.

CHAKRABARTI: It sounds so simple. But human language is incredibly complex. Slifka is describing technology vastly more sophisticated than a directive-processing machine. For Alexa to be able to understand how, when, and why people mix languages, the AI will have to understand things like context, culture, cues, idiom, innuendo and meaning.

CHAKRABARTI: Huh. Imagine that capability, that power, leveraged across one of the largest corporations on Earth.

[Tape] BEZOS: We as humanity, as a civilization, as a technological civilization, are still quite a ways away from being about to be as magical and amazing as the Star Trek computer. But I think, I don’t know, when is Star Trek? So, we still have a couple of centuries. I don’t think we’ll need that much time, actually.

Bloomberg Businessweek: "The Untold Story of How Jeff Bezos Beat the Tabloids" — "'Raise your hand if you think you’ve had a harder week than I’ve had.' It was Feb. 14, 2019, in the early afternoon, and for perhaps the first time in the 25-year history of Amazon.com Inc., Jeff Bezos was prepared to explain himself to his employees."

New York Times: "Inside Amazon: Wrestling Big Ideas in a Bruising Workplace" — " On Monday mornings, fresh recruits line up for an orientation intended to catapult them into Amazon’s singular way of working."