Processing Satellite Imagery to Detect and Identify Non-collaborative Vessels

by Marco Reggiannini and Marco Righi (ISTI-CNR)

In recent years, European maritime countries have had to deal with new situations involving the traffic of illegal vessels. In order to tackle such problems, systems are required that can detect relevant anomalies such as unauthorised fishing or irregular migration and related smuggling activity. The OSIRIS project [L1] aims to contribute to a solution to these problems with the use of large scale data provided by satellite missions (Sentinel, Cosmo-SkyMed, EROS).

Optical/SAR data and system Integration for Rush Identification of Ship models (OSIRIS) is a European Space Agency project launched in March 2016, with the primary purpose of developing a software platform dedicated to maritime surveillance. The platform will be in charge of: (i) collecting maritime remote sensing data provided by satellite missions such as Sentinel-1, Sentinel-2, Cosmo-SkyMed and EROS-B, and (ii) processing the acquired data in order to detect and classify seagoing vessels.

A main goal within OSIRIS is to develop computational imaging procedures to process Synthetic Aperture Radar and Optical data returned by satellite sensors. We propose a system to automatically detect and recognise all the vessels within in a given area; the maritime satellite imagery will be processed to extract visual informative features of candidate vessels and to assign an identification label to each vessel.

Vessel detection and feature extraction
Processing operations first focus on the analysis of SAR amplitude/intensity images, aiming to detect potential vessels within a specific maritime area. A positive detection in this first step returns a subset of the initial image, restricted to an Area of Interest (AoI), where only the candidate vessel is visible. The next stage concerns the recognition of visual attributes within the AoI, providing relevant informative content. Imaging algorithms are applied to extract geometric and radiometric features, which provide meaningful insights about the vessel’s morphology, geometry and dynamics. The input data are processed by standard statistical and morphological filters to extract the features.
 
As an example, consider the SAR intensity AoI image in Figure 1. Proper thresholding techniques (e.g., Otsu method) are applied to the AoI patch to estimate the set of pixels belonging to the detected ship. The identified cluster of pixels is then exploited to compute meaningful ship parameters such as the Length Overall (LoA), and the width and area of the ship. The analysis of the pixel distribution of the segmented ship provides insights about the ship’s orientation. A cascade of imaging algorithms is applied to: (i) reduce the ship’s area to a linear segment by morphological operators (closing, erosion and skeletonisation, [2]) and (ii) identify the bearing angle (e.g., by means of the Radon transform, [2]).

Figure 1:  Cropped SAR patch on an area of interest. Only one vessel is displayed (image taken from [1]).

Figure 2:  Pipeline of processing to estimate the bearing angle of the ship in Figure 1: a) optimal binary thresholding, b) morphological closing, c) morphological skeletonisation, d) line detection by Radon transform.
 
The wavelength of waves observed in the wake generated by the ship’s passage is related to the velocity of the ship itself through known hydrodynamics relationships [3]. Provided the image resolution is large enough to observe the individual wake features, a possible approach to estimate the ship’s velocity v is to perform a frequency analysis of the AoI patch to estimate λ, the wavelength of the wake oscillation (Figure 3).

Figure 3: Kelvin wave oscillation and vessel speed (picture taken from [L2]).

Figure 4: Turbulent wake (yellow lines) and boundaries of the ship wake (red lines). Picture taken from [L3].
 
The amplitude of the central turbulent wake is also known to be related to the ship’s beam [3]; indeed the vehicle’s lateral width can be measured by estimating the width of the turbulent wake. This is performed by applying the Radon transform to extract the two linear borders (right side of Figure 4, yellow lines) enclosing the central turbulent wake and by estimating their separation distance. Additionally, by exploiting the same linear detector, the external boundaries of the wake (right side of Figure 4, red lines) are detected and employed to univocally estimate the direction of the vessel’s speed.

The information collected is then exploited to perform a vessel classification, implemented by feeding a decision tree based classifier with the estimated features. Our software has been evaluated by comparing the estimations with the ground truth information provided by an existing knowledge base. The algorithm performs successfully in 75% of cases, featuring an almost linear time complexity with respect to the input data.

The OSIRIS project consortium comprises the National Research Council of Italy and the Italian companies SisTer and Mapsat (project coordinator).

Links:
[L1] https://wiki.services.eoportal.org/tiki-index.php?page=OSIRIS
[L2] http://www.planetinaction.com/
[L3] https://crisp.nus.edu.sg

References:
[1] X. W. Xing, et al.: “Superstructure scattering distribution based ship recognition in TerraSAR-X imagery”, IOP Conference Series: Earth and Environmental Science, vol. 17, no. 1, 2014.
[2]  R. C. Gonzalez, R. E. Woods: “Digital Image Processing”, Pearson Prentice Hall, 2008.
[3] G. Zilman, A. Zapolski, M. Marom: “The speed and beam of a ship from its wake’s SAR images”, IEEE Transactions on Geoscience and Remote Sensing, vol. 42, no. 10, pp. 2335-2343, 2004.

Please contact:
Marco Reggiannini, Marco Righi
ISTI-CNR, Italy
This email address is being protected from spambots. You need JavaScript enabled to view it., This email address is being protected from spambots. You need JavaScript enabled to view it.