Image Preprocessing Pipeline for Edge Computing in Space
Together with armauisse S+T, Ateleris designed and evaluated an image preprocessing pipeline for execution on embedded platforms like an in-orbit payload or a drone. The goal was to observe the necessary steps to calibrate, reconstruct, geo-correct, and geo-project raw data from an optical sensor to use the image for onboard post-processing, analysis, and machine learning.
Generating a calibrated image from an optical sensor involves several processing steps, like debayering, pixel calibration, lens correction, and more. This preprocessing is typically applied to the raw data directly by a hardware component close to the sensor. However, in specific scenarios, greater control over the preprocessing pipeline is desired, like when the data must be high-quality for science or when the data must be geo-corrected (georectification, orthorectification, and georeferencing).
Thus, data collected by earth observation satellites are traditionally sent to the ground for processing, requiring large transmission budgets and adding a longer turn-around time to react to interesting observations. Especially in time-critical scenarios like quasi-real-time disaster detection and recovery, preprocessing and analyzing the data onboard is desired to reduce transmission budgets and turn-around time for decision-making.
In this joint research project with armasuisse S+T, we designed a computation-graph-based image preprocessing pipeline using TensorFlow. We chose TensorFlow to keep the ability to flexibly re-arrange the pipeline, change and evaluate different preprocessing methods, and compare their results. We also compiled a trade-off analysis of other methods and hardware combinations, including image calibration, geocorrection, and geoprojection.
We chose a RISC-V architecture as our test platform to run the TensorFlow-based preprocessing pipeline. We built a wholly virtualized RISC-V-based payload with QEMU to have repeatable tests and integrated it with a multi-layered CI/CD pipeline. This CI/CD pipeline automated the build process, tying the TensorFlow model with the Buildroot-Linux-based onboard software and running the integrated pipeline in the virtualized RISC-V platform on QEMU.
Also see the follow-up project on Machine Learning in Space.
Key Technologies/Terms
- Tensorflow Lite for Mobile and Edge devices
- Buildroot Linux
- QEMU
- RISC-V architecture
- CI/CD pipeline
- Image processing
- Earth observation
armasuisse Science and Technology (S+T) is the technology centre of the Federal Department of Defence, Civil Protection and Sport DDPS. The area of space research (Research Program 8 (Space)) develops and tests the latest technologies for intelligence collection, command support, precision navigation and synchronisation.