Would You Be Fooled By An AI Teaching Assistant?
Dave Gershgorn
at 12:01 PM May 11 2016
Public-School Science Teacher
Ryan Heshka

On the internet, you never know who's a robot (or a dog). Computer science students at Georgia Tech were taught this lesson first-hand, when their professor Ashok Goel brought in some extra help to teach his Knowledge Based Artificial Intelligence course. The class already had eight teaching assistants, but as humans they weren't always available to answer questions on the class' online forum. That's where Jill Watson, the A.I. teaching assistant, came in. Since she was an algorithm that lived on a computer, she was available 24x7 to help students. The A.I. was actually so helpful that students started to wonder if Watson was human.

Goel trained the algorithm on 40,000 questions and answers about the course that had been made by human students and teaching assistants since fall 2014. At the beginning of the course in January, Watson posted in a hidden forum to workshop her answers. In March, human teaching assistants would post the algorithm's responses, screening for any mistakes. And then by April, the algorithm was posting by itself. Goel used IBM's Watson platform to build the bot.

Of course, being a class of A.I. students (students learning about artificial intelligence, not students who were A.I.s), some became suspicious. After the A.I. posted a slightly vague answer, another T.A. jumped in, and students started to question whether Watson was a computer.

George Tech
Georgia Tech's A.I. teaching assistant tries to help out on a class forum.

But in other situations, the algorithm was able to answer questions independently without any misunderstandings. The students were not aware that Watson was an A.I. until April 26, when Goel lifted the veil.

“We were taking an A.I. course, so I had to imagine that it was possible there might be an A.I. lurking around,” said Tyson Bailey, who said they had been suspicious of Jill's humanity since February. “Then again, I asked Dr. Goel if he was a computer in one of my first email interactions with him. I think it's a great idea and hope that they continue to improve it.”

George Tech
The A.I. was let loose to answer questions autonomously, and proved up to the task.

Would you be fooled by this A.I. teaching assistant?

comments powered by Disqus
Filed under:
Sign up for the Pop Sci newsletter
Australian Popular Science
PopSci Live