by Olivier Parisot (Luxembourg Institute of Science and Technology)
Capturing deep sky video streams has become accessible and inexpensive thanks to recent hardware and software advances, but the growing number of satellites in Low Earth Orbit (LEO) generates undesired light pollution. Thus, we are currently developing a resource-aware AI system for automatically detecting specific targets like satellite streaks in video streams produced with affordable observation stations.
Amateur and professional astronomers can witness that the number of satellites in Low Earth Orbit (LEO) is constantly increasing, as it has a major impact on both visual observation and telescope imaging [L1]. The public is not necessarily aware of this, due to a lack of interest on the subject, but also because of the supposed absence of technical means to verify it.
However, recent advances in astronomical equipment allow to obtain high-resolution astronomical image streams with little effort. For wide-field observation, there’s now no need to have access to a professional observatory: recent smart telescopes are equipped with sensitive CMOS sensors, and can therefore obtain decent images with exposure times of a few seconds, making it possible to produce continuous streams of raw images - which are either combined later to obtain final quality images (astrophotography), or used as they are for live sky monitoring purposes (transient events like meteors, supernovas). These automated instruments, or all-sky devices (e.g., cameras equipped with fisheye lenses), are relatively easy to use and affordable (prices now start at just a few hundred euros). Even recent smartphones are equipped with sensors sensitive enough to capture images of the night sky, the Milky Way... and satellite streaks (Figure 1). This is accessible to schools, science outreach associations, and even individuals.
Figure 1: A picture of the night sky with a Pixel 4a in ‘astrophotography’ mode (August 22, 2022) – at the right of the image, a satellite track was captured.
These installations can generate large data streams, with a considerable number of high-resolution images. Processing these streams requires specialized software (such as ASTRiDE [2]) to detect and filter satellite streaks: this is often a resource-intensive and offline process, needing significant data storage. In a forthcoming article, we reported that 0.16% of the astronomical images we captured between 2022 and 2023 from Luxembourg Greater Region were affected by satellite streaks. Processing these images required heavy computations on several dozen gigabytes of data.
Nowadays, algorithms and hardware improvements mean that complex calculations can be performed efficiently on data streams for a whole range of tasks: it is now possible to run accurate AI models to process images and videos with low resource usage and latency.
As part of research projects at Luxembourg Institute of Science and Technology (LIST) about AI-powered technologies for the Space domain [L2], we are working on an online detection system to analyse continuous streams of sky images, and we are particularly interested in satellite streaks: this problem is obviously being addressed for large observatories [L1], but we are more specifically targeting methods that can be implemented on systems with limited capacity. In other words, we are targeting stream analysis that does not require significant storage requirements, using a lightweight computing device (such as a Raspberry Pi or even a smartphone).
To this end, we are currently working on this workflow:
- The first step consists of designing and training a supervised deep learning model for detection, for instance by using YOLO (You Only Look Once) [1]: model training and evaluation is based on images that we have collected ourselves over the years using various observation instruments, and then annotated. The tiniest model architectures (with low parameter count, ~6M or less) are preferred: they offer fast inference, lower computational and energy requirements, making it ideal for real-time applications on resource-constrained devices, though with slightly reduced accuracy (it’s a trade-off to find). To go further, an incremental technique like DeepSORT can help avoid recalculating annotations for each image by considering the result of the previous image in the stream [2].
- Then, the trained deep learning model(s) must be adapted to the environments/devices on which it will be executed. This step is done via the compression and the quantification of the trained YOLO models, in order to obtain different lightweight versions of the trained models (with weights encoded in 8-bit integers or 16-bit floats, instead of 32-bit floats).
- Finally, the final application must be based on an intelligent strategy to execute the models obtained according to the resources of the targeted computing device. To this end, we apply ‘Resource-aware control’, focusing on efficient algorithms to process continuous streams of images under constrained computational, memory, and energy consumption [3]. When processing video streams, it is therefore a question of using models according to the actual occupation of the device’s memory (bearing in mind that the largest models are potentially slower and more demanding) and adapting the size of the inputs if necessary for each inference (reducing the size of the inputs also reduces CPU/GPU and memory consumption). All this must be done without compromising the accuracy of detection (by minimising false negatives/positives).
Next step: finalizing and embedding the whole process (in a Raspberry Pi or a smartphone), and then interfacing with a setup capable of generating streams of astronomical images (smart telescope or other), showing in near-real time if satellite streak appears. When running, it will allow to highlight the importance of the number of satellites in the sky (for example, through local impact studies, such as for light pollution linked to urban lighting). This is unlikely to influence the actors the satellite industry, but it could enable educators and enthusiasts to raise awareness among the general public, and why not among politicians.
Links:
[L1] https://www.nature.com/articles/d41586-024-03146-2
[L2] https://researchluxembourg.portals.in-part.com/6a8195cd-5b26-4c8a-80e1-785ba9219ac3
[L3] https://github.com/dwkim78/ASTRiDE
References:
[1] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, 2023.
[2] R. Gandhi et al., “Multiple Object Detection and Tracking Using DeepSORT,” in Communications in Computer and Information Science, Springer Nature Switzerland, Nov. 2024, pp. 438–448.
[3] A. M. Shiddiqi, E. D. Yogatama, and D. A. Navastara, “Resource-aware video streaming (RAViS) framework for object detection system using deep learning algorithm,” MethodsX, vol. 11, p. 102285, 2023.
Please contact:
Olivier Parisot, Luxembourg Institute of Science and Technology, Luxembourg
