by Kostas Hatzigiannakis, Athanasios Zacharopoulos and Xenophon Zabulis (ICS-FORTH)
Multispectral imaging and spectroscopic analysis are valuable tools for the study of materials of cultural heritage objects. Accurate registration of spectral images into a spectral cube increases the precision and quality of spectral measurements, supporting portable and low cost multispectral imaging systems.
Multispectral imaging (MSI) is a technique which combines digital imaging with spectroscopic analysis to recover spatial and spectral information of an object. MSI is implemented by inserting a monochromator, most often a series of bandpass filters, in the light path either of the illumination system or in front of the imaging sensor. The outcome of this technique is a sequence of successive images, one spectral image per spectral band, called a spectral cube. Processing of the spectral cube enables the extraction of useful information about the materials comprising the object. MSI is applied in medicine, agriculture, remote sensing, the food industry and other fields. MSI primarily applies in cultural heritage, for art conservation, archaeology and art history studies.
Cultural heritage objects are extremely delicate. MSI, being contactless, non-invasive, and non-destructive, can play an invaluable role in stratigraphic analysis, monitoring conservation interventions, unveiling underdrawings and enhancing faint patterns [1]. Spectroscopic analysis for artworks materials study is also an MSI potential. The intensity along a column of the spectral cube, corresponding to a pixel or a group of pixels, is the extracted reflectance spectrum, enabling the composite materials, including pigments, to be differentiated and identified.
IRIS II is lightweight portable system developed at FORTH comprising a camera, a filter wheel able to interchange 28 filter and replaceable c-mount lens. The sensor, in combination with the band-pass filters, allows sensitivity to the UV region (350 nm up to 400 nm, BP10nm), to the visible (400 nm-700 nm, BP25nm) and to the Near IR region (700 nm-1200 nm, BP50nm). The final 3D spectral cube can have maximum dimensions of 2560 × 2048 × 28 data points.
The portability of the system introduces possible camera pose changes and vibrations during the acquisition, which in turn are evident as translation shifts and even rotations between the different spectral images. In addition, the versatility of the system and the properties of the optical elements occasionally introduce the need for refocusing and thereby slight changes in the focal distance and consequently in the field of view.
As a consequence, the spectral images are not accurately aligned. Image registration aligns the spectral images, so that a physical point is imaged at the same coordinates in all images of the spectral cube. The reflectance spectrum is more accurately extracted and, in this way, precise spectral measurements can be achieved. High precision is of particular interest when only traces of a material are present in the artwork.
The registration approach capitalises on the (approximate) planarity of paintings, to simplify the registration problem. In this way, the estimation of the geometrical transformations that warp the multispectral images so that they are registered to each other and form an aligned spectral cube is feasible. A non-linear, robust estimation method that utilises point correspondences across image pairs is employed for the estimation of these transforms [2].
Figure 1: The effect of registration in the acquisition of multi-spectral images. (a) Conventional colour image of the object under study. (b) Synthesis of a colour image by combining the three unregistered spectral images at 450 nm (blue channel), 550 nm (green channel) and 650 nm (red channel) of the spectral cube. The observed blurring is due to misregistration. In (d), spectral images are registered before their combination and no blurring is observed. The area of study is magnified in b) and d) for higher detail. Graphs (c) and (e) plot the extracted spectra at four points, for the unregistered and registered spectral cubes, respectively. Due to registration, in (e), the extracted spectra are better aligned, indicating that the same pigment has been used at the four studied points. The checker markers in (b) and (d) were used only for evaluation and are not required by the registration method.
These correspondences are established based on the detection and matching of keypoint features in image pairs of the spectral sequence. Typically, two arbitrary images from the spectral sequence expected to be quite different for feature matching to be fruitful. However, images acquired at neighbouring spectral bands are more similar and, thus, correspondences are searched in consecutive images. By combining the estimated transforms, all spectral images can be registered upon a reference image in the sequence (i.e., the first, or the middle one), yielding an accurately aligned spectral cube.
The accuracy of registration is evaluated with the help of benchmark datasets, which are annotated with ground truth for registration. The aforementioned registration approach has less than a pixel of error, measured on these benchmark datasets. As such, datasets are not widely available, but a collection of datasets indicating characteristic situations in in-situ MSI are publicly available for the evaluation of existing future MSI registration approaches.
Link:
http://www.ics.forth.gr/cvrl/msi/
References:
[1] V. Papadakis et al.: “‘IRIS’; a novel spectral imaging system for the analysis of Cultural Heritage objects”, Proc. of SPIE Vol. 8084 (2011); doi: 10.1117/12.889510.
[2] M. Lourakis: “homest: A {C/C++} Library for Robust, Non-linear Homography Estimation”, Jul. 2006,
http://www.ics.forth.gr/~lourakis/homest/
Please contact:
Xenophon Zabulis
ICS-FORTH, Greece
