Spectral Imaging

Introduction

Hyperspectral and spectral cameras are used in plant research, crop breeding and in commercial farming, both in the field with drones and in controlled environments. The main limitation to their widespread use is cost.

Traditional spectral cameras have a monochromator and scan the image line by line (see Specim’s cameras, Resonon’s, and others). Some “snapshot” spectral cameras are based on a normal CMOS image sensor with a special filter or filter plus microlenses (see Cubert’s cameras). In both cases the image resolution is relatively low and the cost of cameras very high. These cameras in most cases are not able to “see” ultraviolet radiation. However, they do have good wavelength resolution (up to 2 to 3 nm, and .

The cameras described above depend on discriminating different wavelengths at the camera. A recent commercial camera uses single colour LEDs to obtain spectral images, however, even if using a very simple and cheap camera module its price is very high (see RAYN Vision System Camera). Using monochromatic light to measure reflectance or transmittance is not a new approach in that it is frequently used in spectrophotometers where the monochromator is at the light source.

These two approaches are not fully equivalent as they detect fluorescence differently.

Because of this I decided to try the colour-LED-based approach with a good photographic camera and also test performance in the ultraviolet with a full-spectrum converted photographic camera. Below I describe some of these tests.

Six colours and white

Camera

I used an Olympus O-M1 digital mirrorless camera. This camera has a 20 Mpix sensor. It can take both 20 Mpix images and 80 Mpix images. In the later case using sensor shift and in-camera image merging.

The LED array

Ledengin Osram LZ7-N4M100 7-channel LED array from 2018 or so: six colours + white. Max 20 W total. It has a flat glass window protecting the chips. LES 3.8 mm (all chips are within a 3.8 mm diameter area (Figure 1). Maximum rated current per channel is 700 mA. For the photographs below current was set to 350 mA, the maximum for the driver used.

Figure 1: LZ7-N4M100 LED array from Led Engin Osram, window (= package) size \(7\times 7\,\mathrm{mm}\) mounted on PCB.

library(ggspectra)
library(photobiologyLEDs)
library(photobiologyWavebands)

Of the seven channels, six are “single colour” (Figure 2) and one is white (Figure 3).

colour_channels <- 
  grepl("LZ7", names(leds.mspct)) & !grepl("white", names(leds.mspct))
autoplot(leds.mspct[colour_channels],
         range = c(300, 750), 
         geom = "spct", 
         w.band = VIS_bands()) +
  theme_bw() + theme(legend.position = "none")

Figure 2: Emission spectra of the 6 “single-colour” channels of the led array type LZ7-N4M100 from Led Engin (now Osram). Spectral irradiance is normalized to the maximum of each spectrum.

white_channel <- 
  grepl("LZ7", names(leds.mspct)) & grepl("white", names(leds.mspct))
autoplot(leds.mspct[white_channel],
         range = c(350, 850), 
         geom = "spct", 
         w.band = VIS_bands()) +
  theme_bw() + theme(legend.position = "none")

Figure 3: Emission spectrum of the white channel of the led array type LZ7-N4M100 from Led Engin (now Osram).

Images

For the first test I picked four pansy flowers from the balcony. For each image I also took images with the LED switched off in the same room as dark control (Figure 4).

LED: 396 nm

LED: 454 nm

LED: 498 nm

LED: 520 nm

LED: 620 nm

LED: 629 nm

LED: white

Figure 4: Individual RGB images, not edited. Taken with an off-the-shelf Olympus OM-1 camera and an Olympus D.Zuiko 35 mm 1:3.5 macro objective.

The RGB filters on the camera sensor see to some extent other wavelengths, so white balancing the images based on the grey reference with 18% reflectance used as a background results in false-colour images (Figure 5).

LED: 396 nm

LED: 454 nm

LED: 498 nm

LED: 520 nm

LED: 620 nm

LED: 629 nm

LED: white

Figure 5: Individual colour images white balanced and exposure-matched on grey background. Taken with an off-the-shelf Olympus OM-1 camera and an Olympus D.Zuiko 35 mm 1:3.5 macro objective.

By converting the images to greyscale one can get a estimate of relative reflectance (Figure 6).

LED: 396 nm

LED: 454 nm

LED: 498 nm

LED: 520 nm

LED: 620 nm

LED: 629 nm

LED: white

Figure 6: Individual images white balanced and exposure-matched on grey background (Figure 5) and converted to greyscale. Taken with an off-the-shelf Olympus OM-1 camera and an Olympus D.Zuiko 35 mm 1:3.5 macro objective.

For a comparison we can extract the three channels from the RGB image taken under white light.

LED: white, sensor channel: B

LED: white, sensor channel: G

LED: white, sensor channel: R

Figure 7: The three channels extracrted as greyscale from RGB image taken under white light. Taken with an off-the-shelf Olympus OM-1 camera and an Olympus D.Zuiko 35 mm 1:3.5 macro objective.

Twelve colours

Camera

For the images in this chapter I used a different camera, an Olympus E-M1. This older camera has been modified to increase its sensitivity to ultraviolet and infrared radiation. I also used a different objective, a Sigma 30 mm 1:2.8 Art for MFT. This is one of the few modern lenses with accidental extended transmission into the ultraviolet and good performance in the IR. This is no a macro lens and I used it very near its closest focusing distance. The images were taken on a different so the flowers are not the same as above but they do come from the same plants. The increased sensitivity to red light affects the colour rendering even when using a white LED source with a CRI \(\approx 95\), and a truer rendering can be achieved only with a custom colour profile (Figure 8). After applying the ICC profile the colours match well those shown above for the unmodified OM-1 camera.

LED: white; from camera

LED: white; white balanced

LED: white; ICC profile applied

Figure 8: Colour rendering by the full-spectrum-converted Olympus E-M1 digital mirrorless camera.

I have used LEDs from my collection to characterize the spectral response of the RGB channels of this same camera. The measurements are described in the page and help explain the false colours in the images.

The E-M1 has a 16 Mpix sensor, front side illuminated. The photographs were saved as ORF raw files and edited and converted to lower resolution JPEG compressed files for display in this page.

The LED array type A

A custom assembled LED array with twelve channels with independent wiring. Ten LED chips (1 W each) connected in series and arranged in a line conform a channel. Total nominal power is 120 W, with 10 W per channel. Nominal maximum current per channel is 350 mA and voltage up to 37 V. The array was supplied by Shenzhen Weili Optical with “family type” G-100PW140AG-M but with my choice of wavelengths.

Figure 9: Custom LED array from Weilli light emitting area size \(20\times 22\,\mathrm{mm}\) mounted on PCB. The 120 chips can be seen as well as the wiring. The wavelength increases from bottom to top from 365 nm to 660 nm nominal.

autoplot(normalise(led_arrays.mspct$Weili_120W.array.12ch.custom.A),
         range = c(350, 850), 
         geom = "spct", 
         w.band = VIS_bands()) +
  theme_bw() + theme(legend.position = "none")

Figure 10: Emission spectra of the 12 channels of the custom assembled led array.

Images

LED array type B

This array is identical to type A but differs on the selections of wavelengths in the channels, being more spaced in short-wavelength end of the spectrum and reaching farther in the long-wavelengths end.

autoplot(normalise(led_arrays.mspct$Weili_120W.array.12ch.custom.B),
         range = c(350, 850), 
         geom = "spct", 
         w.band = VIS_bands()) +
  theme_bw() + theme(legend.position = "none")

Figure 11: Emission spectra of the 12 channels of the custom assembled led array.

Images

In the images out of the camera the colour of the LED used for illumination predominates (Figure 12).

LED: 366 nm

LED: 390 nm

LED: 422 nm

LED: 469 nm

LED: 514 nm

LED: 555 nm

LED: 593 nm

LED: 610 nm

LED: 637 nm

LED: 658 nm

LED: 675 nm

LED: 741 nm

LED: white

Figure 12: Individual colour images. Photographs taken with an Olympus E-M1 camera converted to full-spectrum by removal of the sensor-cover IR- and UV-blocking filter and a Sigma 30 mm 1:2.8 Art MFT objective. As the camera has been modified even in white light the white balance is incorrect.

LED: 366 nm

LED: 390 nm

LED: 422 nm

LED: 469 nm

LED: 514 nm

LED: 555 nm

LED: 593 nm

LED: 610 nm

LED: 637 nm

LED: 658 nm

LED: 675 nm

LED: 741 nm

LED: white

Figure 13: Individual colour images white balanced and exposure-matched on grey background. Photographs taken with an Olympus E-M1 camera converted to full-spectrum by removal of the sensor-cover IR- and UV-blocking filter and a Sigma 30 mm 1:2.8 Art MFT objective.

LED: 366 nm

LED: 390 nm

LED: 422 nm

LED: 469 nm

LED: 514 nm

LED: 555 nm

LED: 593 nm

LED: 610 nm

LED: 637 nm

LED: 658 nm

LED: 675 nm

LED: 741 nm

LED: white

Figure 14: Individual colour images white balanced and exposure-matched on grey background and converted to greyscale. Photographs taken with an Olympus E-M1 camera converted to full-spectrum by removal of the sensor-cover IR- and UV-blocking filter and a Sigma 30 mm 1:2.8 Art MFT objective.

UV-A1-induced fluorescence

The images above show mainly reflected light. UV-radiation as well as visble light can induce fluorescence in plants. The UV-A1 radiation from the arrays has enough contamination with other wavelengths that even with a UV-blocking filter on the objective the fluorescence cannot be photographed.

Here I show a photograph of the same set-up of flowers and leaves illuminated with a well-filtered UV-A LED flashlight (Figure 15)

Figure 15: Fluorescence induced by UV-A (365 nm) from a flashlight. Photograph taken with an Olympus E-M1 camera converted to full-spectrum by removal of the sensor-cover IR- and UV-blocking filter and a Sigma 30 mm 1:2.8 Art MFT objective. Firecrest UV400 UV-blocking interference short-pass filter on the camera objective. Small pieces of lint from cloth and paper fluoresce brightly.

Remaing questions/tasks

1. Construct an image cube from these images
2. Compare recovered information against spectral reflectance measured with a spectrometer.