Image Preprocessing Pipeline for Edge Computing in Space

Together with armauisse S+T, Ateleris designed and evaluated an image preprocessing pipeline for execution on embedded platforms like an in-orbit payload or a drone. The goal was to observe the necessary steps to calibrate, reconstruct, geo-correct, and geo-project raw data from an optical sensor to use the image for onboard post-processing, analysis, and machine learning.

Generating a calibrated image from an optical sensor involves several processing steps, like debayering, pixel calibration, lens correction, and more. This preprocessing is typically applied to the raw data directly by a hardware component close to the sensor. However, in specific scenarios, greater control over the preprocessing pipeline is desired, like when the data must be high-quality for science or when the data must be geo-corrected (georectification, orthorectification, and georeferencing).

Thus, data collected by earth observation satellites are traditionally sent to the ground for processing, requiring large transmission budgets and adding a longer turn-around time to react to interesting observations. Especially in time-critical scenarios like quasi-real-time disaster detection and recovery, preprocessing and analyzing the data onboard is desired to reduce transmission budgets and turn-around time for decision-making.

In this joint research project with armasuisse S+T, we designed a computation-graph-based image preprocessing pipeline using TensorFlow. We chose TensorFlow to keep the ability to flexibly re-arrange the pipeline, change and evaluate different preprocessing methods, and compare their results. We also compiled a trade-off analysis of other methods and hardware combinations, including image calibration, geocorrection, and geoprojection.

We chose a RISC-V architecture as our test platform to run the TensorFlow-based preprocessing pipeline. We built a wholly virtualized RISC-V-based payload with QEMU to have repeatable tests and integrated it with a multi-layered CI/CD pipeline. This CI/CD pipeline automated the build process, tying the TensorFlow model with the Buildroot-Linux-based onboard software and running the integrated pipeline in the virtualized RISC-V platform on QEMU.

Also see the follow-up project on Machine Learning in Space.

Key Technologies/Terms

  • Tensorflow Lite for Mobile and Edge devices
  • Buildroot Linux
  • QEMU
  • RISC-V architecture
  • CI/CD pipeline
  • Image processing
  • Earth observation

armasuisse Science and Technology (S+T) is the technology centre of the Federal Department of Defence, Civil Protection and Sport DDPS. The area of space research (Research Program 8 (Space)) develops and tests the latest technologies for intelligence collection, command support, precision navigation and synchronisation.

Picture of Ivo Nussbaumer

Ivo Nussbaumer

Any questions? Please send me a message!

Contact Me