A floating touchscreen.
For fans of science fiction, this scene is likely familiar: at a distant interstellar base, a suave hero presses a button on a control panel, and a schematic appears in the air. The hero’s nimble fingers touch what appears to be a non-physical display, shutting down the energy shield and continuing with their secret mission. But what you may not know is that while interstellar bases and energy shields may be far from us, floating displays are not.
I am referring to displays that can truly generate floating two-dimensional images in the air and allow interaction, rather than those that project images onto a transparent surface that viewers cannot touch, based on Pepper’s ghost illusion. The optical principles behind creating floating images are easy to understand, and the pandemic has sparked interest in various non-contact controls. Companies like Toppan Printing and Kyocera have attempted to commercialize such aerial displays. However, progress has been slow, and expected applications like elevator controls are not very exciting.
I decided to create my own aerial display, one that fully showcases the charm of science fiction.
I am no stranger to making novel displays. In 2022, I showcased my color electromechanical display in the “Hands-On” section of Technology Review, reminiscent of the early days of television. This time, since I was creating something from the future, I decided to design the system inspired by props from the movie “Star Wars.” But first, I needed to get the optical device working.
Let’s review some optical knowledge. Typically, light from a source (like a display) diverges as the distance increases. If these diverging rays are reflected by a mirror, the eye perceives the display as being behind the mirror. This is a virtual image. However, if we can make the light emitted from the display converge at a point in space and then diverge again, the eye will perceive the display as if it were located at the convergence point, even if that point is in mid-air. This is what we see as a real image.
The key to achieving this convergence in the air is the use of reflective materials. Ordinary mirrors follow the law of reflection where the angle of incidence equals the angle of reflection, meaning that light entering the mirror from the left at a small angle will reflect back at the same small angle and continue to propagate to the right. However, a retroreflector can reflect incoming light directly back to the light source. Therefore, if a retroreflector is installed directly in front of the screen, all diverging light will reflect back along its path, and when they converge at the screen surface, a real image will form. Clearly, this is meaningless by itself, so we need to introduce another optical element—a beam splitter.
This material reflects about half of the incoming light and allows the other half to pass through directly. One clever aspect is that the screen and retroreflector are installed at 90 degrees to each other, with the beam splitter positioned opposite at a 45-degree angle to both the screen and retroreflector. Now, let’s follow the light: the diverging light from the screen reaches the beam splitter, where half of the light is sent to the retroreflector, which reflects the light back to the beam splitter. The beam splitter allows half of the converging light to pass through. When they converge in the air above the display, the light forms a real image.
Clearly, the efficiency of this optical trick is quite low, as most of the original light is lost in the system. However, finding a sufficiently bright small modern flat-screen that can produce a reasonably good aerial image under indoor (or interstellar base) lighting conditions is not difficult. I used a LattePanda 3 to drive this 7-inch display, which is an Intel Core-based single-board computer capable of running Windows or Linux and supports multiple displays.
The biggest challenge I faced was finding suitable retroreflector material. Ultimately, I chose a type of foil, which I cut to the required size, and it can produce a clear image at a low cost. This material is called Oralite 3010 prism optical film, and I spent about $90 on a roll of 77cm x 1m, which is the smallest size available.
The next step was to make the display interactive. After some testing, I chose a $5 laser detection sensor that can report distances measured along a narrow cone. I installed three of these sensors to cover three columns on the aerial display plane, and I connected these sensors to an Arduino Nano via I2C. When a user’s fingertip enters the detection cone of the sensor, the Nano checks whether the fingertip’s distance is within three predefined ranges. The aerial display has three sensors, each divided into three segments, giving the display nine areas that can respond to a finger. The system reports the activated area to the LattePanda via USB.
The optical components and computer are housed in a frame made of aluminum profiles measuring 33cm x 25cm x 24cm. I also installed a small touchscreen at the front, allowing control of what the LattePanda displays on the aerial display. I added side panels to the frame and attached 3D-printed metal strips and other decorative components to make it look like a display from a sci-fi scene.
The completed system works well, and the futuristic feel is just as I envisioned. This also indicates that this technology is within the capabilities of most manufacturers today, without the need for faster-than-light propulsion.
Author: Markus Mierse
IEEE Spectrum
Technology Review
Official WeChat Public Platform
PreviousRecommendationsMetasurface Displays Expected to Surpass LCDsHow an Engineer Writes the Best Sci-Fi NovelsSimple Mechanical Television—The Oldest Television Display Gets a New Twist