by Christos Anagnostopoulos, Aris Lalos, George Pavlidis, Christos Koulamas, Athanasios Kalogeras (ISI, Athena R.C), Ana Garcia Lopez (University of Granada) and Giacomo Di Benedetto (Enginelife-Engineering Solutions)
The WARMEST project is developing an intelligent decision support system that will result in a low impact tool to suggest improvements in maintenance and risk management procedures in cultural and heritage sites (CHSs).
The decision support system developed by the WARMEST project [L1] will utilise existing data over multiple sources to anticipate future scenarios and indicate actions in order to avoid major risks in maintenance and probable risks related to climate change. The generated maintenance scenarios will have to be both technologically and economically feasible; a significantly difficult task owing to the large variety of loads (different types of sites), strict regulations to consider and the need for continuous operation throughout the year.
The preservation and promotion of culture contributes to Sustainable Development Goals (SDGs), as well as being a worthwhile goal in itself, according to UNESCO [1]. The relationship between heritage and sustainable development is described explicitly in two SDGs. More specifically, in SDG 11, entitled “Make cities and human settlements inclusive, safe, resilient and sustainable”, target 11.4 focuses on strengthening efforts to protect and maintain cultural and heritage sites. In SDG 8 “Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all”, target 8.9 refers to the need for creating policies that will support sustainable tourism through carefully devised maintenance and operational plans. In other words, a set of actions must be carefully prepared that will ensure the viable preservation of the CHS while allowing its touristic exploitation.
In this context, WARMEST will try to satisfy these targets by: (i) collecting updated and detailed data about CHSs; (ii) producing accurate and cost-effective maintenance plans to face current heritage conservation needs and foreseen scenarios; (iii) defining safe and effective visitor routes for sites; (iv) making decisions about excavation campaigns in relation to expected maintenance plans; and (v) supporting new companies or technologies in monitoring and maintaining CHSs. The project plans to use 3D models to document and digitalise parts of the CHSs that take part in the project, identify the current state of the artifacts by analysing heterogeneous data sources like 3D meshes, dense point clouds, and IR photos, evaluate the temporal evolution of the artifacts by studying archive material and weather data information, correlate the identified damage with external factors like weather conditions or accidents, evaluate the success of maintenance actions that have been undertaken and finally deliver a decision support system to deliver cost-effective maintenance plans.
To this end, we propose a novel CHS defect detection methodology by extracting saliency mapping from 3D models. The first step is to collect a big dataset that can provide valuable information. One of the CHS participating in the project is the Patronato de Alhambra, an UNESCO Heritage Site palace and fortress complex located in Granada, Spain. Its most famous courtyard is the Court of Lions, which includes a low gallery supported on 124 white marble columns. We used photogrammetry technologies to construct 3D models of the columns. More than three hundred photos per column were taken at different angles and heights. Photos were taken manually and no UAV was used, in accordance with safety regulations. A low ISO in conjunction with an aperture of around f/11 were selected to reduce image noise and optimise sharpness and depth of field. The photoshoots were conducted just before opening at around 7 a.m., to ensure optimal lighting conditions and avoid crowds. The 3D reconstruction was performed using a structure from motion combined with Multiview stereo (SfM-MVS) technique. SfM-MVS systems are affordable, can be automated and used by non-specialists and produce high quality 3D data. This system is increasingly used in demanding cultural digitisation applications, surpassing laser scanning and structured light approaches. Figure 1 shows a sample of the camera locations and the final 3D object.
Figure 1: Camera positions and 3D reconstructed object (left), saliency map extraction (right).
The 3D reconstructed model is then processed using a data-driven approach that focuses on the extraction of the 3D saliency mapping. Training data were generated from meshes and a given saliency mapping, which was previously generated by traditional methods [2]. After the training process, the generated output can be used to automatically extract saliency mapping for any other new 3D model. We start by separating the whole mesh into n (i.e., equal to the number of centroids) overlapped and equal-sized patches. We then follow two different steps for the estimation of the spectral and geometrical saliency. The final result is a combination of these two values. Once the saliency mapping has been estimated, we use it as input to train the convolutional neural network (CNN) (see Figure 1(b)). This concludes the procedure and gives us the appropriate input data to detect defects on the columns using machine learning techniques.
WARMEST is a H2020 Marie Curie Research and Innovation Staff Mobility Project (H2020- Marie Skłodowska-Curie Actions-RISE-2017). The project is carried out by an international and multi sectorial consortium comprising more than 30 researchers. In close cooperation with the consortium, three heritage sites are involved in the research: Patronato de la Alhambra y Generalife (Granada, Spain), Marzamemi Underwater Museum (Sicily, Italy) and Opera Santa Croce (Florence, Italy).
Link:
[L1] https://warmestproject.eu/
References:
[1] UNESCO, 2003. Convention for the Safeguarding of the Intangible Cultural Heritage. World Heritage Committee. https://ich.unesco.org/en/convention
[2] G. Arvanitis, A.S. Lalos, K. Moustakas: “Saliency mappingfor processing 3d meshes in industrial modeling applications,” in 2019 IEEE 17th Int. Conf. on Industrial Informatics (INDIN), July 2019.
Please contact:
Athanasios Kalogeras
ISI, Athena Research Centre, Greece
