Mosaic metadata

[vnewey]

Adding a module for generating odc stac and yaml metadata and thumbnails for our dea coastal mosaic products.

Some additional info on the module:

Mosaic Metadata Generator

Generate ODC YAML and STAC JSON metadata for DEA coastal products.

Installation

Option 1: Install as a command (recommended)

pip install -e .

This makes metadata_generator available as a command:

metadata_generator generate \
    --year 2024 \
    --product ga_s2ls_intertidal_cyear_3 \
    --version 2-1-0

Option 2: Run as a script

python metadata_generator.py generate \
    --year 2024 \
    --product ga_s2ls_intertidal_cyear_3 \
    --version 2-1-0

Quick Start

Intertidal (auto-detected product family):

metadata_generator generate \
    --year 2024 \
    --product ga_s2ls_intertidal_cyear_3 \
    --version 2-1-0 \
    --output-dir metadata

Tidal Composites:

metadata_generator generate \
    --year 2024 \
    --product ga_s2_tidal_composites_cyear_3 \
    --version 2-1-0 \
    --output-dir metadata

Coastal Ecosystems (requires dea naming):

metadata_generator generate \
    --year 2022 \
    --product ga_s2_coastalecosystems_cyear_3_v1 \
    --version 1-0-0 \
    --naming-conventions dea \
    --output-dir metadata

Output to S3:

metadata_generator generate \
    --year 2024 \
    --product ga_s2ls_intertidal_cyear_3 \
    --version 2-1-0 \
    --output-dir s3://dea-public-data-dev

Key Features

• Auto-detects product family from product name
• Generates both ODC YAML and STAC JSON metadata
• Creates product-specific thumbnails (elevation, RGB composite, classification)
• Extracts lineage from tile STAC files
• Validates metadata
• Syncs to S3 or local filesystem

Parameters

Required

• --year: Year of the product (e.g., 2022)
• --product: Product name (e.g., ga_s2ls_intertidal_cyear_3)
• --version: Product version (e.g., 2-1-0)

Optional

• --product-family: Auto-detected if not specified (intertidal, tidal_composites, coastalecosystems)
• --study-area: Study area code (default: AU)
• --freq: Frequency code (default: P1Y)
• --naming-conventions: dea or dea_c3 (default: dea_c3)
• --output-dir: Output destination, S3 or local (default: metadata)
• --product-maturity: Product maturity (default: stable)
• --dataset-maturity: Dataset maturity (default: final)
• --dev/--prod: Use dev or production S3 bucket (default: dev)
• --verbose: Verbose output

Examples

With auto-lineage extraction:

metadata_generator generate \
    --year 2024 \
    --product ga_s2ls_intertidal_cyear_3 \
    --version 2-1-0 \
    --verbose

With custom lineage file:

metadata_generator generate \
    --year 2024 \
    --product ga_s2ls_intertidal_cyear_3 \
    --version 2-1-0 \
    --lineage-file lineage.json

Testing with empty lineage:

metadata_generator generate \
    --year 2024 \
    --product ga_s2ls_intertidal_cyear_3 \
    --version 2-1-0 \
    --auto-lineage

Other Commands

Validate metadata:

metadata_generator validate metadata/*.yaml metadata/*.json

Show paths:

metadata_generator show-paths \
    --year 2022 \
    --product ga_s2ls_intertidal_cyear_3 
    --version 2-1-0 \

[github-actions]

For full integration test results, refer to the Tests directory README.