Data Cube Builder


Data Cube Builder is a framework to generate Earth Observation (EO) Data Cubes on-demand to work both on-premise and on-cloud AWS.

Basically, the data cube creation consists of the following steps: Warp and Temporal Compose. The Warp step consists in cropping and mosaicking all images that superimpose a target tile of the common grid, for a specific date. This spatial mosaic is reprojected to the target tile reference system and all bands are resampled to a determined spatial resolution through a bilinear function, except for quality assessment band, which is resampled using nearest neighbor to avoid changes on the image values. The result is called Identity Data Cube.

The Data Cube Builder also supports temporal composition functions, which allow generating regular data cubes. The most used function is `STACK`, which consists in aggregating pixels from all images in the time interval according to each image quantity of valid pixels. The STACK composition can be seen in the figure below.

WTLSS

Cube Builder Architecture on AWS

The Data Cube Builder on top of AWS consists in the following structure:

WTLSS

Lambdas: To execute several tasks in parallel for data processing, and it also is used as an API to orchestrate data cube generation.

S3: Data Storage for working directory and published data.

DynamoDB: Store Tasks definitions in JSON.

PostgreSQL: Store data cube published data cubes metadata.

SQS: Used to separate multiple queues for AWS Lambda parallel processing.

Kinesis: Serialize data cube tasks definitions and metadata through a stream to be served and stored in DynamoDB.

Image gallery