CAPCON Pipelines Help

Runs the Sentinel-1 processing scripts upstream of the other apps β€” SAR preparation and per-field statistics.
← All apps

CAPCON Pipelines

Runs the Sentinel-1 processing scripts upstream of the other apps β€” SAR preparation and per-field statistics.

The scripts are the existing ones from the capcon scripts/ tree; this app gives them a form, a command preview and a live log. Nothing is re-implemented: what runs is exactly the script you would run from a terminal.

Script groups

GroupPanelsScripts
1 Β· SAR PreparationRegion & Data, SAR Preparationcreate_region/, sar_preparation/
2 Β· Field StatisticsField Statisticscalc_statistic/
3 Β· CAPCON Dataβ€”not connected yet

If the panel says "not configured on this host"

The app can be installed on a server before the processing scripts and their environments are. Overview then lists what is missing (scripts folder, interpreters, data folders), and every Run is refused with the same list. Regions can still be viewed. It is an installation task, not a fault in a step β€” pass the list to the platform administrator.

Scoped to a region, not a project

Select a Region in the sidebar (or create one with + New region…). A region is the data-preparation unit, separate from the platform's projects. Every step takes its year, folders, boundary and grid from the selected region (Region & Data).

How a run works

  1. The form is built from the script's declared parameters; the exact command

is shown under it before you press Run.

  1. The script runs with the platform's interpreter, from the app's own

pipelines/scripts/ folder; the GPU steps inside S1 composites run with the separate GPU venv (/opt/capcon/env-gpu on the server).

  1. The run is detached: closing the page or restarting the service does not

stop it. Jobs lists every run of the region with its log; Stop terminates the script and all its worker processes.

Panels

Use the menu on the left; each panel has its own help topic.

Region & Data

What this panel is for

Everything a region needs before SAR processing: create the region (folders, boundary, 0.5Β° grid), describe it, import the vector layers it uses, and choose which months, dekads and periods the SAR composites are computed for.

A region is not a platform project. It is the data-preparation unit; a project may later take its data from a region. The region is chosen in the Region selector at the top of the sidebar (it replaces the year Β· project Β· period selector in this app).

The selector lists only the regions you have been given access to. The account that creates a region is given access to it; anyone else is added by an administrator. The selection is shared by everyone using the app, so if another operator switches it while you have a form open, your next Run or save is refused with the selected region is now … β€” reload the page and check the region before running again.

Creating a region

Choose + New region… in the sidebar selector.

  1. Region name β€” upper-case letters, digits and _ (e.g. DEMO). It names

every folder and file and cannot be changed later.

  1. Description β€” what the region is for, in English. Shown under the

selector; editable later.

  1. Year and Scene archive region β€” editable later.
  2. Work / Source / Temp folder β€” suggested inside the region's own folder

(<regions base>/DEMO/work, /source, /temp), or, where the server redirects bulk data to another disk, under that base (DEMO_work, DEMO_data_source, DEMO_temp). Change them before creating if needed, but each must stay under the region's folder or the server's base for that kind. They are fixed afterwards.

  1. Region boundary β€” Browse… to the file (GeoPackage, shapefile,

GeoJSON, …) wherever it lies under the allowed folders.

Create region runs create_region.py, which:

geometry fixed (invalid polygons repaired, empty ones dropped);

intersects the boundary, with zona = DEMO.

Re-import boundary / rebuild grid runs it again. The grid is only rebuilt when you tick that box, because rebuilding loses the zone attributes.

Importing a vector layer (Region data)

  1. Browse… to the file; pick the layer if there are several. Its fields

are listed with their types and sample values.

  1. Tick the fields to import.
  2. Name β€” your part of the file name: 0625_parcels becomes

DEMO_0625_parcels_3035.gpkg. CRS β€” EPSG:3035 (default) or 4326, also written into the name.

  1. Features to import β€” the whole layer, or *only the features

intersecting the region boundary*, optionally grown by a buffer in metres (default 0, computed in EPSG:3035). Selected features are kept whole, not clipped to the boundary.

  1. Optionally compute area_ha (0.01 ha) and centr_x / centr_y (integer

metres) β€” always in EPSG:3035.

  1. Kind and Field mapping β€” which of the layer's own fields is the

parcel ID, the crop code, and so on. The file keeps the country's field names; scripts are given the mapped names. A Lithuanian file is mapped automatically (glk_dkl_id, glk_paseli, psl_kodas).

  1. Description β€” what the data is and when it was received.

The import runs as a job (vector_cli.py import), streamed feature by feature so memory use does not depend on the size of the file.

Each imported layer is listed with its name, kind and state; open it to edit the description and the field mapping (no re-import needed), to see its fields, types and sample values, or to Delete it β€” the GeoPackage is deleted from disk and the layer removed from the list.

Zone attributes

Per zone: tick Active, the months and dekads, and the periods; Save zone (or Save ALL zones). The Clear buttons clear the form only; Clear in ALL zones writes to the file at once.

What it writes

layer registry (kind, description, fields, mapping, source).

DEMO_<name>_<epsg>.gpkg. The description is also stored inside each GeoPackage.

columns of the grid β€” the same fields and format as the old tkinter zone_attributes_editor.py.

When it goes wrong

server allows (CAPCON_IMPORT_ROOTS).

file's storage is mounted and readable by the service.

period list β€” usually a typo from an older edit.

SAR Preparation

What this panel is for

Producing the Sentinel-1 statistical rasters for the selected region and its year, in pipeline order: composites, audits, VRT mosaics, indices, exports, RGB composites.

Before you start

The region created and its zone attributes set (Region & Data).

How to use it

  1. S1 composites β€” the main, long run (hours). Incremental: only months,

periods and dekads whose files are missing are computed, so after adding a month in the zone editor just run it again. GPU work runs in GPU-venv subprocesses started by the script itself.

  1. Audits β€” completeness always; the integrity check opens every COG.

With Delete corrupted files = YES, bad files are deleted so the next composites run recomputes them.

  1. VRT mosaics, indices (SM, RVI4S1), index VRTs, export,

RGB composites β€” in that order; each skips what already exists where the script does.

Open Paths & tuning to change a step's worker counts and to see its folders. The folders are shown read-only: they come from the region and the server, so a run can only ever read and write the selected region's data. The command line under each form is exactly what will run.

What it writes

Under the region's work folder: sar_data/<year>/<tile>/, indices/, vrt/<year>_data/, vrt/<year>_index/, merged_raster/, rgb_raster/; audit reports audit_<year>_*.txt in sar_data/.

When it goes wrong

scenes for some month β€” they stay incomplete and are retried on the next run.

incomplete and the run continues with the next block.

Field Statistics

What this panel is for

Per-parcel zonal statistics (exactextract, area-weighted) over the SAR composites (crop statistics, VV/VH) and the SM / RVI indices.

Before you start

statistics, the SM / RVI indices for the index statistics.

parcels), with at least the Parcel ID mapped. The fields mapped in it (parcel id, crop codes, area, centroids) are the parcel columns of the output, under the layer's own field names.

How to use it

  1. Parcels layer β€” the newest imported parcels layer is preselected.
  2. Months and dekads β€” tick the monthly composites (Month comp.) and the

dekads to compute; the month button toggles a whole row.

  1. Month quartiles / Dekad quartiles β€” crop statistics default to Q1–Q3,

SM and RVI to Q2.

  1. Min parcel area β€” 0.1 ha by default (the mapped area field is used;

without one the area is computed in EPSG:3035).

  1. Existing outputs

What it writes

<inputs>/statistic/<year>/crops/ (and rvi/, sm/) β€” one parquet file per raster layer, e.g. S1_VV_D10_Q2_AD_2026_05_26_TC4326_stat.parquet, one row per parcel. The folder name never changes; _source.json in it records which parcels file (path, size, modification time), which fields and which minimum area the statistics were computed from.

A new parcels delivery: import it in Region data (e.g. DEMO_2026_0801_parcels_3035.gpkg), delete the old layer, choose the new layer here. With skip the run stops and says the parcels changed β€” the existing files belong to the old delivery. Run with overwrite: the old files are deleted and the statistics are computed from the new parcels.

Arable land (VV / RVI) still uses the v1 scripts and their dated folders. These tables are not in a sampling period's stats/ folder, so Checks and Models do not see them yet.

When it goes wrong

parcels.

Region data.

Workers, border phase if the machine runs out of RAM on a large region.

Jobs

What this panel is for

Every processing run of the selected region, newest first β€” the way back to a run started from a page that has since been closed.

How to use it

stopped from the console header.

outside, or the machine restarted).

What it writes

Each run is a folder <regions base>/<REGION>/logs/pipelines/<job id>/ with job.json (the exact command, who started it, when), output.log and rc (exit code).