Near infrared imaging: new 3D sensing technology lowers cost and simplifies hardware architecture

Near infrared (NIR) imaging has long been deployed in specialized fields such as scientific research because of the valuable optical properties of light at NIR wavelengths. For example, research laboratories perform spectrometry with NIR light sources and cameras. This advanced form of NIR imaging can detect and measure the specific physical or chemical properties of materials. In medical diagnostics, NIR imaging is used in laboratory equipment to analyze specimens or samples. The data derived from the analysis enable, for instance, the diagnosis of medical conditions.

Now industrial and commercial applications are developing new ways to deploy NIR imaging, using 3D sensing technology pioneered by the smartphone industry.

This trend is backed by manufacturers of NIR illuminators and sensors, which are introducing products that bring 3D sensing in reach of manufacturers of machine vision products and industrial equipment. And designers of machine vision equipment are finding that it can help them to improve functionality, often while reducing system cost at the same time.

Richer information, more valuable insights

Machine vision has traditionally used cameras based on image sensors operating in the visible light spectrum. In factory automation, for instance, global shutter image sensors with a high frame rate enable visual checks for defects and non-conformity in high-speed, high-volume production environments.

A flat 2D visible image, however, can provide only limited information about an object: for instance, to inspect the label printed on a packaging box. A 3D imaging system, by contrast, not only inspects the label but also provides additional data about position and geometry, such as:

• The dimensions of the box
• Deformities that could indicate that the box has been damaged
• Orientation of the box, to ensure that it has not been placed upside-down or on its side

3D sensing today is often implemented as a stereovision set-up: a pair of visible light digital cameras can infer depth from the parallax effect, the fractionally different pixelated representation of the scene that each camera produces. This stereovision technology mimics the way in which the human vision system perceives depth.

3D vision solutions can also be implemented with structured light: here, a pattern projector works in combination with a single camera. The projector uses a diffractive optical element, an optical filter which splits a beam of laser light into multiple tiny beams. This diffracted beam forms a predefined pattern, typically a dot pattern, which is projected on to the observed scene. From the distortion of the projected pattern in the camera’s image of the scene, the size and position of the object can be inferred.

If both 3D technologies are combined, 3D stereovision plus structured light, an even more robust 3D image can be captured. This technique is called active stereovision.

Both these 3D scanning technologies work best in the NIR spectrum, especially at 850 nm or 940 nm, rather than in the visible spectrum with the use of conventional RGB image sensors:

• Sunlight has spectral gaps at 850 nm and 940 nm that produce a better signal-to-noise ratio (SNR), enabling the use of a lower-power illuminator or projector. Such an illuminator will tend to have a lower unit cost than a higher-power light source, and will also consume less power, to provide for longer run-times in battery-powered applications such as remote surveillance
• The light source, a pattern projector or illuminator, is invisible to people
• Some NIR illuminators and most NIR image sensors have a narrow spectral bandwidth: both the output from the emitter and the sensitivity of the sensor can have narrow peaks close to their nominal wavelength. This enables an NIR system to capture clear, detailed image data free of interference from ambient visible light or other light sources

The high SNR of NIR imaging systems is particularly valuable in challenging applications for machine vision such as automatic number plate recognition (ANPR), which has to operate both fast and reliably often in dim or dark conditions, even with a low-power illuminator.

Besides these higher-resolution 3D vision systems, there are also so called time-of-flight (ToF) systems: they measure the differences between emission of short laser pulses, in direct ToF, or dToF, systems, or between continuous phase-modulated light outputs, in indirect ToF or iToF systems, and the reflection from an object in the field of view received by a sensor array. The flight-time differences between the emitted and reflected photons from different parts of a non-flat object can be measured by a ToF sensor, and converted into a depth map.  

Although the image resolution is not as high as with stereovision or structured light systems, ToF is a proven 3D scanning technology. The quality of the scan results depends on the laser system’s ability to produce ultra-short pulses, for dToF, or stable phase modulation for iToF. In applications that do not require a high-resolution depth map, smart and energy-efficient multi-zone dToF sensors can perform histogram calculations and indicate in which zone an obstacle has been detected, and how far away it is.

New components serving the industrial market for 3D sensing

There is a broad choice of components available to implement 3D NIR imaging and sensing, including high- and low-power LEDs, both in the visible and IR spectrums. IR lasers for 3D sensing include vertical-cavity surface-emitting lasers (VCSELs), which are supplied in 850 nm and 940 nm wavelengths, and edge-emitting lasers (EELs), which normally operate at 905 nm. EELs offer a higher power output for use over a longer range, for instance in LiDAR systems traditionally deployed in autonomous guided vehicles (AGVs) and mobile robots. In fact, each 3D sensing technology is suited to a different set of application requirements and ranges. 

3D sensing for face recognition and biometrics typically operates at a range of less than 1 m, and is realized with NIR imagers paired with structured light dot projectors, to provide an anti-spoofing capability. 

Mobile household robots use low-cost multi-zone dToF sensors that have an operating range of up to 5 m, for obstacle detection and navigation.

Higher-end robots or production inspection systems use stereovision or active stereovision cameras to produce a detailed 3D depth map. Paired with higher resolution image data originating from the stereovision set-up, these systems capture a detailed image of the scene over a range from 0.5 m to further than 10 m.

For any of these types of NIR imaging or sensing system, ams OSRAM is a source of the advanced optical components that are required. ams OSRAM, is a company formed by the merger of ams, a specialist manufacturer of optical sensors, and OSRAM, one of the world’s largest producers of emitters, including both LEDs and lasers. 

Fig. 1: The BELAGO1.1 projector, showing the diffractor that produces the dot pattern. (Image credit: ams OSRAM)

The ams OSRAM product range is equally well geared towards the industrial as the consumer market. For instance, the BELAGO1.1 is a proprietary 940 nm dot-pattern illuminator with built-in eye-safety protection, shown in Figure 1. It emits a focus-free, high-contrast dot pattern that algorithms can use to produce high-accuracy depth maps. A particular benefit of the BELAGO illuminator is its characteristic random dot pattern, which can feature between 5,000 and 35,000 dots. The pattern reduces the computational load on the image processor that performs the depth map calculation.

ams OSRAM also supplies the TARA family of IR VCSEL emitters. The TARA2000-AUT is an automotive-qualified version of the emitter, for use in in-cabin sensing applications such as driver monitoring and gesture detection, shown in Figure 2. The VCSEL’s narrow spectral bandwidth ensures freedom from interference caused by sunlight and reflections inside the car. The ultra-compact module, with a footprint of just 4.1 mm x 4.1 mm, is available with various field of illumination options. 

Fig. 2: TARA2000-AUT, an IR VCSEL flood illuminator. (Image credit: ams OSRAM)

For the image sensor element of a 3D optical system, ams OSRAM supplies the Mira family of global-shutter NIR image sensors. The Mira family image sensors offer high quantum efficiency, which enables operation with a low-power emitter and in dim lighting conditions.

A stacked chip design uses ams OSRAM back-side illumination technology to shrink the chip’s footprint, giving greater design flexibility to manufacturers of space-constrained products. The Mira220, a 2 Mpixel image sensor that can be used in structured lighting systems as well as other applications, has a small 5.3 mm x 5.3 mm footprint, and is shown in Figure 3. 

Fig. 3: Block diagram of the Mira220, a 2.2 Mpixel image sensor. (Image credit: ams OSRAM)

The global shutter provides for the distortion-free capture of fast-moving objects, such as products subject to quality assurance in a high-volume manufacturing operation. The Mira220 achieves quantum efficiency of up to 38% at 940 nm. Some applications can work with lower-resolution images, and benefit from less complex algorithms and hence lower power consumption. Here, ams OSRAM provides a 0.5 Mpixel image sensor, the Mira050. This sensor provides additional low-power features to provide wake-up triggers, which would otherwise need to be realized by additional presence detection sensing circuitry.

ams OSRAM also supplies a broad portfolio of single- and multi-zone direct time-of-flight (dToF) sensors, including the TMF8828, which detects objects in a user-configurable array of up to 8 x 8 zones.

These ams OSRAM optical products, as well as illuminators and image sensors from manufacturers including Luminus Devices, onsemi, ROHM Semiconductor and Vishay, are fully supported by the technical service, engineering and sales teams at Future Electronics.