How do you find the jagged edge between AI performance and hallucinations?
About ten years ago, I was in Silicon Valley at an auto technology show. These were days before OpenAI and LLM’s became household names. A researcher was demonstrating AI on an edge device. He used an Android smartphone and a simple classifer application to test cards showing a hand, a cloud, and a cat. With each, the classifier quickly identified the subject of the test cards.
I knew from my own research back in the 80’s that researchers would boost results by using training data for testing. I challenged the researcher, “Well, that’s great. But what if we use a novel image, like my hand?” I put my hand in view of the phone’s camera. On the display, I saw a picture of my hand, and the caption, “Hand.”
AI has grown in leaps and bounds since that day. A fundamental question is: “How do I successfully use AI?” In this blog, we explore the jagged, cutting edge of AI, to find where it is useful, and where it is not.