Reading (For Dummies): Turing

February 18, 2003

Turing

Suddenly I realize that Turing got it all wrong.

The definition for a conversational/chatter type robot shouldn't be to fool one of us into thinking it's human, it should be to fool the average idiot who doesn't even know that they're testing a computer. Think about it, who tests modern chatter bots? Computer nerds. What they should be doing is taking clueless people from the street, and saying "interview these two people and tell me if you see anything odd." Because when people know they're testing one computer and one human, they formulate sentences and questions that they know fool a computer.

But if you don't know from the get-go that you're studying one computer and one person, you won't be as prone to asking those questions. Like when we talk to a normal person face-to-face we assume they're human, we don't test the hypothesis. So we should be assuming that both parties in the Turing test are human. That's a better/more realistic test. Isn't there some kind of psychology term for this? Help me out here, Stevis.

Back to studying....

Posted by reid at February 18, 2003 02:45 AM | TrackBack

Comments

Kristen would probably agree with you that a double blind study would be the most effective test. Ideally, the "tester" would not even know which one was the robot.
Jason

Posted by: Jason on February 20, 2003 08:20 PM

Yeah, the medical system does something like what I'm thinking. But not really.

When they do medical studies, don't they usually say "We're giving one group of people medicine to lower their blood pressure, and one group a placebo. We're not going to tell you which on you're getting"? Like, you know your pill is one of two things, placebo or blood pressure medicine. Telling you what the real medicine does is what causes the placebo effect (you think "oh, maybe I'm getting the blood pressure stuff" and for no good reason your blood pressure goes down, even though you really have the placebo). I think my comparison would be equivalent to having a medical study and telling the patient, "We're giving you some medicine, but aren't going to tell you what it does. You tell us," or better yet, not even telling the patient they're getting any medicine :). Medical testing would be a lot more effective (well, scientifically effective) this way, but the FDA seems to think it's a bad idea ;-).

In my Turing comparison, I'm saying that people who test AI's *know* one of the things they're talking to is a computer. That knowledge means they're on the lookout, to disprove one of the contestors is human. Maybe they shouldn't know that it's even a test, because that would make the test more fair (as if "fair" can be quantified :)).

Posted by: Reid on February 20, 2003 08:39 PM