Trompe l'oeil

What optical illusions tell us about our sensory systems

I’ve always had a strange fascination for optical illusions.

Part of it is the pure fun of having our expectations about what we’re seeing confounded, but a lot of my curiosity about them has to do with what they tell us about how our sensory processing systems work. The answer, unfortunately, is “not very well.”

Oh, they work well enough; most of the time, we perceive the external world with sufficient clarity not to walk into walls or get run over by oncoming trains. But our impression that we experience the world as it is—that our overall ambient sense of everything around us, what the brilliant neurophysiologist David Eagleman calls our umwelt, is a crystal-clear reflection of the real universe—simply is false.

All it takes is messing about with optical illusions to convince yourself of how easy our brains and sensory organs are to fool. For example, in the following drawing, which is darker; square A or square B?

They’re exactly the same. Don’t believe me? Here’s the same drawing, with a pair of gray lines superimposed on it:

You do a great deal of color saturation perception based upon comparison and your knowledge of the surrounding environment. Because your brain decided that B was in the shadow and A wasn’t, then it concluded that A had to be intrinsically darker. What baffles me still about this illusion is that even once you know how the trick works, it’s impossible to see it any other way.

“Our brains are rife with ways of getting it wrong,” said astronomer Neil deGrasse Tyson. “You know optical illusions? That’s not what they should call them. They should call them brain failures. Because that’s what they are. A few cleverly drawn lines, and your brain can’t handle it.”

For example, there’s the famous two-and-a-half pronged fork:

The problem here is that we’re trying to interpret a two-dimensional drawing as if it were a three-dimensional object, and the two parts of the drawing aren’t compatible with each other under that interpretation. Worse, when you try to force your brain to make sense of it—following the drawing from left to right, and trying to figure out where exactly the object goes from two prongs to three—you fail utterly.

We also tend to run into trouble when our brains make assumptions about what is most likely, and then forces what we see to conform to those assumptions. Take, for example, the Kanizsa Triangle illusion:

Almost everyone sees a solid white triangle superimposed on a black-outlined triangle and three circles. The illusion is so powerful a lot of people perceive the edges of the white triangle clearly even though they aren’t there. The explanation for this one seems to be that the brain reasons that a white triangle on top is far more likely than three V-shapes and three PacMan shapes just happening to line up perfectly, so it decides that’s what reality is.

The Kanizsa Triangle illusion is an example of the phenomenon of amodal completion, where we make assumptions about what we can’t see based on what we can. This often works, but sometimes leads us astray:

Amodal completion can fool us in other ways, too. How about this pair of photos?

It seems unlikely that the masking orange bits would just happen to cover up all the clothing by accident, so our brain decides the people in the photograph are naked. Works with men, too:

Of course, this effect is probably enhanced by the fact that a great many of us like looking at sexy naked people.

Some optical illusions have yet to be convincingly explained. An example is the Scintillating Starburst illusion:

Most people see radial white lines coming out from a bright spot at the center, and—you’ve guessed it—those lines and bright spot don’t exist. This one is so puzzling it was the subject of a paper in the journal Perception. The researchers seem as baffled as the rest of us as to why this works:

[The scintillating starburst illusion is] a unique kind of stimulus that evokes ghostly or ephemeral illusory rays that appear to shimmer or scintillate... We ascertained that the [effect] experienced by observers when viewing this stimulus type is modulated by all stimulus dimensions we suspected to be relevant when piloting the study, namely the number of vertices of the polygons, contrast, the line width of the wreaths, the number of wreaths, and whether the polygons are bisecting or not. The strongest effect was yielded by the number of wreaths, followed by whether the strands are bisecting, stimulus contrast, line width of the braids, and the number of vertices of the polygons, in that order... [N]o stimulus dimension by itself produces a strong effect, only the optimal confluence of many stimulus parameters does so. We believe that these results are consistent with probabilistic inference—for instance, the percept of illusory lines from an occluder is more likely if there are more intersection points where the vertices bisect, and if this happens at higher contrast. This is not implausible, as deciding on a coherent interpretation of ambiguous visual information is a fundamental challenge faced by the visual system. Of course, probability by itself is not sufficient—the specific stimulus situation matters—for instance, a row of street lights does not evoke the impression of a bright band that connects them. But in the case of street lights, the bright beacons are broken up by the darkness of the night. This darkness is unambiguously present. However, in the case of Starbursts, the bright beacons are separated by background of the same color, yielding the percept of an occluder of that color on top of the stimulus.

For one more final mindblowing illusion, there are the amazing antigravity cardboard sculptures of Japanese mathematician and artist Kokichi Sugihara. Here’s a link to one of them, in which he makes it look like marbles are rolling uphill. Then he shows you how it’s done by turning the sculpture around. Then he turns it back, and does it again… and you still see it as the marbles rolling uphill. Once your brain has decided on a model for reality, even one that is demonstrably false, even one where your rational brain knows for sure that it’s a trick and understands how it’s done, there’s no arguing with our more primal sense of “whoa, that’s weird.”

All of this adds more layers to the problem of eyewitness testimony. We are all deeply convinced that what we perceive is correct, when in fact we should not be. Add to the sensory/perceptive problems the fact that memory is highly plastic and inaccurate—a topic for a different day—and it behooves us all to be extremely careful about saying, “I know it happened that way! I saw it with my own eyes.”