Utah engineers & SLC startup develop a high-speed, high-def hyperspectral video camera

Instead of a filter that divides light into three color channels, University of Utah engineers, in collaboration with Salt Lake City startup Lumos Imaging, have developed a diffractive camera element that divides it into 25 colors. (Photo courtesy University of Utah) 

A traditional digital camera splits an image into three channels — red, green and blue — mirroring how the human eye perceives color. But those are just three discrete points along a continuous spectrum of wavelengths. Specialized “spectral” cameras go further by sequentially capturing dozens, or even hundreds, of these divisions across the spectrum. 

This process is slow, however, meaning that hyperspectral cameras can take only still images, or videos with very low frame rates — frames per second (fps). But what if a high-fps video camera could capture dozens of wavelengths at once, revealing details invisible to the naked eye?

Researchers at the University of Utah’s John and Marcia Price College of Engineering have developed a new way of taking a high-definition snapshot that encodes spectral data into images, much like a traditional camera encodes color. Instead of a filter that divides light into three color channels, their specialized filter divides it into 25. Each pixel stores compressed spectral information along with its spatial information, which computer algorithms can later reconstruct into a “cube” of 25 separate images — each representing a distinct slice of the visible spectrum.

This instantaneous encoding enables the researchers’ camera system — small enough to fit into a cellphone — to take high-definition video, and the compressed nature of the component images opens up new real-world applications.

A study demonstrating the camera was led by Apratim Majumder, research assistant professor, and Professor Rajesh Menon, both in the Department of Electrical & Computer Engineering at the university.

The findings, titled “High-Definition (HD) Snapshot Diffractive Computational Spectral Imaging and Inferencing,” were published Sept. 20 in the journal Optics. Co-authors include Fernando Guevara-Vasquez, a UofU associate professor of mathematics. Funded by the Office of Naval Research, the research was conducted in collaboration with the Salt Lake City technology startup Lumos Imaging, which Menon founded in 2015, and the U.S. Naval Research Laboratory.

The camera’s design represents a leap forward in how spectral data can be
captured.

“We introduce a compact camera that captures both color and fine spectral details in a single snapshot, producing a ‘spectral fingerprint’ for every pixel,” Menon said.

Hyperspectral cameras have long been used in agriculture, astronomy and medicine, where subtle differences in color can make a big difference. But these cameras have historically been bulky, expensive and limited to still
images.

“When we started out on this research, our intention was to demonstrate a compact, fast, megapixel-resolution hyperspectral camera, able to record highly compressed spatial-spectral information from scenes at video rates, which did not exist,” Majumder said.

The UofU team’s breakthrough lies in how it captures and processes the data. The key component is a diffractive element that is placed directly over the camera’s sensor. It’s the element’s repeating nanoscale patterns that diffract incoming light and encodes both spatial and spectral information for each pixel on the sensor.

“One of the primary advantages of our camera is its ability to capture the spatial-spectral information in a highly compressed two-dimensional image instead of a three-dimensional data cube and use sophisticated computer algorithms to extract the full data cube at a later point,” Majumder explained. “This allows for fast, highly compressed
data capture.”

Instead of a filter that divides light into three color channels, the UofU engineers have developed a diffractive element that divides it into 25.

The streamlined approach also cuts costs dramatically.

“Our camera costs many times less, is very compact and captures data much faster than most available commercial hyperspectral cameras,” Majumder said. “We have also shown the ability to post-process the data as per the need of the application and implement different classifiers suited to different fields, such as agriculture, astronomy and
bio-imaging.”

Data storage is another advantage.

“Satellites would have trouble beaming down full image cubes, but since we extract the cubes in post-processing, the original files are much smaller,” Majumder added.

To demonstrate the camera’s capabilities, the researchers tried three real-world applications: telling different types of tissue apart in a surgical scene, predicting the age of strawberries as they decayed over time, and mimicking a series of spectral filters that are used in astronomy.

The current prototype takes images at just over one megapixel in size and can break them down into 25 separate wavelengths across the spectrum. But the team is already working on improvements.

“This work demonstrates a first snapshot megapixel hyperspectral camera,” Majumder said. “Next, we are developing a more improved version of the camera that will allow us to capture images at a larger image size and increased number of wavelength channels, while also making the nano-structured diffractive element much simpler in design.”

By making hyperspectral imaging cheaper, faster and more compact, the UofU engineers have opened the door for technologies that could change the way we see the world and uncover details hidden across the spectrum.

This article is based on a release furnished by the University of Utah College of Engineering.