Using AI to Help MBA Students Talk Like TED

As I was wrapping up a lesson in my Business Presentation class at GLOBIS last Fall, briefing students on the report due in the following class, I threw out an offer which piqued their interest.

“Who would like to get their presentation evaluated by an Artificial Intelligence, an AI, in our next class?”

Cue the surprised faces.

I explained how I’d been talking to Professor Toshihiko Yamasaki, a machine-learning researcher at the University of Tokyo, who had developed an AI system which evaluates a recorded speech based on how similar it is to TED talks.

Anyone who wants to participate can make an audio recording of yourself as you present in the next class. I’ll send those to Professor Yamasaki and the AI will analyze your speech and tell you your strengths and weaknesses by comparing your presentation to TED talks. Who’s in?”

Not surprisingly, I got a lot of takers.

The GLOBIS Business Presentation course

Being a persuasive and inspiring communicator is a must-have skill for any business leader. You can’t be an effective leader without it. That’s why we offer a course in Business Presentation at GLOBIS, being one of the few business schools in the world that offers a full-semester presentation course.

It’s not a course about PowerPoint. The syllabus covers the importance of understanding the audience and developing storylines which resonate with them based on their needs, plus how to create clear and compelling visuals to effectively share your message.

But in the end it’s the in-person delivery of the presentation that matters the most. That will make or break your presentation. I spend a lot of time discussing presentation delivery and having students present in small groups or in front of the entire class, with feedback and questions from me and their classmates.

Repeated feedback is what helps transform students. Over three months many change from being unconfident and reluctant public speakers, into passionate presenters who persuade and inspire their classmates to action. Then they can do the same in business, leading colleagues, executives, or customers on to greatness.

Now we’ve added a new kind of presentation feedback, a non-human one.

AI & the TED Benchmark

I met Professor Yamasaki In the summer of 2016 at TEDxUTokyo, where he was speaking about his research at a lunch event. Originally created to help his undergraduate students become better speakers, the AI system could learn from each TED talk because each talk comes with rankings and evaluations made by anyone who has watched it. Visitors to can rate each talk in 14 categories, from the positive (inspiring, beautiful, informative) to the less-so (obnoxious, unconvincing, boring).

Here’s how Simon Sinek’s famous “Start with Why” TED talk has been rated by viewers;

The AI system watched 1600 TED talks with ratings like this, learning what makes TED talks so compelling, and reached an accuracy level of is 93%. Give the AI an audio file or yourself speaking, and you’ll find out how close to TED greatness you are.

After trying the system by having it evaluate a TEDx talk I gave, Professor Yamasaki and I discussed testing it out with a new group, MBA students at GLOBIS.

GLOBIS students’ AI Adventure

On Day 4 of the Business Presentation course, students are required to submit a report based on a Harvard Business School case on Pizza Hut. The case studies how Pizza Hut made a huge change to its operations by beginning to offer pizza delivery in the 1980s, and faced internal resistance. Students take on the role of the Pizza Hut CEO, and must analyze their skeptical audience, and construct a storyline and slides to persuade that audience to change. They deliver their presentation in class as the CEO, getting feedback from their classmates.

Each GLOBIS student volunteer recorded their Pizza Hut presentation in class. They later received a map of how the AI evaluated their presentation against the 14 TED categories, like this.

So what did the students think about the whole experience of having a machine evaluate their words?

Incredibly positive.

Many of the volunteers were struck by how well the feedback from the AI matched what their classmates had commented on, like how confident they sounded, or how persuasive. Both human and AI were commenting on the same things.

But what most impressed them was the neutrality of the AI feedback. We humans have a tendency to soften or avoid negative feedback to others. We’re afraid of hurting others’ feelings, so we sometimes hold back and don’t give the constructive criticism needed to learn and grow. An AI doesn’t worry about hurting feelings or needing to be seen as nice; it is impartial and un-emotional.

This was the great strength of the AI system and one of the most interesting parts of this experience. The students implicitly trusted the AI’s feedback and valued it because it was neutral and unbiased.

So the MBA students got a better sense of their strengths and weaknesses, and so can become better communicators. A machine helped them with that.

AI & the Future of Business Education

This was an experiment. I was curious about how AI could supplement the in-class learning experience.  As we learned, AI can be very effective in evaluating presentations, but that doesn’t mean I’m out of a job (yet).

AI can tell you that you’re not being persuasive, but not why or what to do about it. AI is not a coach, it can’t understand each student as an individual and motivate or inspire. That still requires the human touch.

But I see huge potential in having an AI teaching assistant in any training on becoming a more persuasive communicator. AI can provide a new kind of feedback and support learners. AI is disrupting all industries: education is no different. At GLOBIS, we are getting ahead of that by bringing new technologies to the real-world and online classroom. Using an AI to evaluate presentations in an MBA program seems to be a world’s-first, and it’s great to be pioneering the use of AI in business education.

What’s next for this AI? Many students hoped for an app version in future, so they could use it routinely to practice and refine their talks. Professor Yamasaki is starting to work on adding the ability for it to give more concrete feedback.

And I’m looking forward to the day when I can pick up my phone and say “Hey Siri, what did you think about Bob’s presentation?”

Photo copyright: Cathy Yeulet