Skip to content

Releases: jldz9/InSARHub

v0.3.2

09 Jun 20:23

Choose a tag to compare

New Features

  • HPC sliding-window submission (isce_base.py) — each step now runs a lightweight sbatch manager job that keeps ≤max_concurrent_hpc child jobs active at all times, refilling immediately on completion. Replaces the old batch-sequential approach. Consecutive steps with equal command counts (e.g. run_13run_16) are merged into a single group-manager. Steps are chained via --dependency=afterok.
  • Per-command elapsed time in sbatch logs (isce_base.py) — sbatch scripts print START/DONE/FAIL with elapsed seconds. Group tasks also print total elapsed across all grouped steps.

Bug Fixes

  • HyP3 file paths (hyp3_base.py) — hyp3_jobs.json now saves to workdir root (was workdir/hyp3/); downloaded ZIPs go to workdir/hyp3/; retry job files save to workdir root; legacy out_dir=workdir entries auto-migrate to workdir/hyp3/.
  • watch command ignores --interval flag (cli/main.py) — _proc_local_watch read args.refresh_interval but argparse stores it as args.interval; interval was always 60 s regardless of user input. Fixed.
  • insarhub processor refresh/download re-processes retry job files (cli/main.py) — _find_job_files globbed hyp3*.json, matching hyp3_retry_jobs_<ts>.json files from past runs alongside hyp3_jobs.json. On refresh/download, stale retry files were loaded as separate processors, causing duplicate downloads or incorrect status. Fixed: retry files excluded from glob.
  • Orbit files downloaded twice (cli/main.py) — -d -O flags triggered two orbit downloads. Fixed: skips explicit download_orbit() call when downloader already handled it.
  • retry() runs locally after HPC submission (isce_base.py, cli/main.py) — hpc_mode excluded from saved config, so retry defaulted to local. Fixed: retry() auto-detects HPC from job metadata (slurm_job_ids/hpc_manager/hpc_array) and writes it back to config.hpc_mode so _step_executor() routes correctly. _load_local_processor also restores max_concurrent_hpc and HPC config fields from insarhub_config.json.
  • Manager job killed mid-run leaves step stuck PENDING (isce_base.py) — elif n_cmds > 0 and not job_ids: was dead code (always False inside if job_ids: block); SLURM-killed managers with incomplete commands never resolved to FAILED. Fixed: condition is now elif n_cmds > 0:.
  • _parse_time_s mis-parses 2-part SLURM time strings (isce_base.py) — "30:00" was treated as 30 h 0 min (108,000 s) instead of 30 min 0 s (1,800 s), overestimating group-manager walltime 60×. Fixed: 2-part strings now parsed as MM:SS per SLURM spec.
  • Group-manager step stuck PENDING when job gone from SLURM (isce_base.py) — if group_task_dir was absent from saved metadata (old jobs), n_cmds resolved to 0, preventing the SUCCEEDED/FAILED transition. Fixed: n_cmds now stored in job metadata at submission; refresh uses it directly with file-count as fallback.
  • refresh() shows only one RUNNING command in manager mode (isce_base.py) — with sliding-window, multiple commands run concurrently but only one showed RUNNING. Fixed: per-command status now derived from .done/.fail files; all in-flight commands show RUNNING.
  • Hyp3_SBAS MintPy output in workdir root (mintpy_base.py) — base class run() passed self.workdir to TimeSeriesAnalysis; Hyp3_SBAS inherits without override, so all MintPy outputs scattered to workdir root instead of workdir/mintpy/. Fixed: uses self.mintpy_dir; same correction for _geocode_diagnostic_files.
  • ISCE cleanup misses merged/interferograms (isce_sbas.py) — cleanup targeted isce/interferograms/ (nonexistent); real stackSentinel output is isce/merged/interferograms/. Large intermediate files were never deleted.
  • HyP3 auth failure submits to wrong user (hyp3_base.py) — when re-auth failed for a pool user, credits=0 was overwritten by self.client.check_credits() on the previous user's client; jobs were then submitted under the wrong account. Fixed: credits check guarded by auth result.
  • self.batchs updated per-loop-iteration (hyp3_base.py) — on multi-user refresh, if any user failed, their batch was silently dropped from self.batchs. Fixed: assignment moved after loop.
  • Missing filename key in HyP3 file metadata crashes download (hyp3_base.py) — direct dict subscript raised KeyError for auxiliary entries lacking filename. Fixed: file_meta.get('filename') with skip on empty.
  • delete_job_folder blocks on ~-prefixed workdir (settings.py) — Path(workdir) without expanduser().resolve() made relative_to() always raise ValueError, returning 403 on every delete. Fixed.
  • _run_folder_select_pairs uses folder.parent as workdir (folders.py) — downloader config received parent directory instead of job folder; sub-paths written one level up, potentially colliding with sibling jobs. Fixed: workdir=folder.
  • Analyzer stop_event leaks on step error (routes/analyzer.py) — early return on step exception bypassed _stop_events.pop(job_id), leaking events indefinitely. Fixed: pop before return.
  • Refresh overwrites .insarhub_cache.json filenames with empty list (routes/processor.py) — if no jobs had SUCCEEDED yet, filenames=[] overwrote a valid cache from a prior successful refresh. Fixed: preserves existing filenames when current refresh yields none.
  • Retry job files appear as selectable job entries in GUI (routes/processor.py) — hyp3*.json glob matched hyp3_retry_jobs_<ts>.json; selecting one for refresh returned only the retry batch status. Fixed: retry files excluded.

GUI

  • Hyp3_S1 max_workers in settings panel (defaultconfig.py) — parallel download threads now configurable via Job settings group (default 4, range 1–16).

Refactor

  • Centralized path layout (config/paths.py) — Hyp3Paths, ISCEPaths, MintPyPaths dataclasses replace all hardcoded workdir / "subdir" strings across hyp3_base.py, isce_base.py, isce_s1.py, mintpy_base.py, hyp3_sbas.py.
  • Path dataclass coverage extended — remaining hardcoded path literals replaced with dataclass properties: isce_sbas.py now uses ISCEPaths for isce_dir, slc_dir, dem_dir; cli/main.py uses Hyp3Paths/ISCEPaths in _has_zips and _find_job_file; utils/batch.py and utils/tool.py use Hyp3Paths.output_dir for ZIP discovery.
  • Hyp3Processor renamed to CloudProcessor (core/base.py) — ABC renamed to reflect generic cloud-backend semantics rather than HyP3 specificity. Updated across core/__init__.py, __init__.py, processor/hyp3_base.py, commands/processor.py, cli/main.py, core/engine.py.

Docs

  • Contributing guide — new tab in MkDocs navigation (EN + ZH). Split into Overview, Backend, and Frontend pages.
  • Backend contributing guide — architecture overview, path conventions, per-section instructions for adding new processors/downloaders/analyzers. Each section includes a "Adding a New Base X" subsection (with code examples for CloudProcessor/LocalProcessor, BaseDownloader, BaseAnalyzer) and an "Extending an Existing Base X" subsection with switch tabs (Hyp3Base, ISCE_Base, ASF_Base_Downloader, Mintpy_SBAS_Base_Analyzer).
  • Frontend contributing guide — conda Node.js install, uvicorn backend startup from InSARHub root, module reference tables grouped by area (Entry & Global, Map, Search & Scene Selection, Jobs & Results, Settings, Utilities), backend communication pattern, settings panel, Vite proxy, build output, code style.

v0.3.1 bug fix

03 Jun 20:56

Choose a tag to compare

Bug Fixes

  • Download result unpacking (commands/processor.py) — processor.download() returns (Path, dict) tuple; DownloadCommand was assigning the whole tuple to output_dir. Fixed: now unpacks to output_dir, dl_stats. CommandResult.data now includes both output path and download stats.
  • HyP3 workflow marker wrong location (hyp3_base.py) — write_workflow_marker was writing insarhub_config.json to workdir/hyp3/ instead of the job folder root, so HyP3 tags never appeared in the job drawer. Fixed: writes to config.workdir.
  • Stale out_dir from saved job file (hyp3_base.py) — old hyp3_jobs.json pointing to a pre-migration path outside current workdir would silently redirect output. Fixed: out_dir rejected if not under current workdir.
  • wslpath unchecked (settings.py) — if wslpath -w failed, PowerShell was called with -File "". Fixed: returncode + empty string guard added.
  • ZIP detection for hyp3/ layout (cli/main.py) — analyzer now checks workdir/hyp3/*.zip first, with fallback to workdir/*.zip for legacy layouts.

Performance

  • Auth status parallel checks (auth.py) — HyP3 credit check, CDSE, CDS, and Earthdata checks now run concurrently via ThreadPoolExecutor instead of sequentially. Typical improvement: 3–5× faster settings panel load.
  • Job folder listing SSH speed (settings.py) — removed all per-folder glob/exists/is_file checks. Now reads only insarhub_config.json per folder. Significant speedup on remote filesystems.

Source

  • hyp3/ subdir awareness (hyp3_sbas.py, mintpy_base.py, batch.py) — all ZIP lookups now check workdir/hyp3/*.zip first, falling back to workdir/*.zip for legacy layouts. Affected paths: _unzip_hyp3, cleanup, and ERA5Downloader.download_batch.
  • Missing .mintpy.cfg guard (mintpy_base.py) — if .mintpy.cfg is not found when run() is called, a warning is printed and the config is written automatically rather than crashing downstream MintPy steps.
  • write_mintpy_config parent mkdir (defaultconfig.py) — outpath.parent.mkdir(parents=True, exist_ok=True) added before opening the file, preventing FileNotFoundError when the output directory does not yet exist.

CLI

  • prep aliasinsarhub analyzer run --step prep now accepted as alias for prep_data. Help text updated to show alias.
  • Default portinsarhub-app now defaults to 8080 (was 8000). Use --port to override.

GUI

  • Subfolder navigation (JobQueueDrawer) — click any folder to drill into subfolders; button to go up. Resets to workdir root on workdir change. Uses /api/browse-subfolders endpoint.
  • Cancel button (JobQueueDrawer) — Cancel action added for local ISCE jobs.
  • Modern folder picker (settings.py) — Windows/WSL now uses IFileOpenDialog COM API via embedded C# in PowerShell. Fixes: DPI blurriness on 2K monitors, Chinese character paths.
  • Add nyan cat

Network Graph (utils/tool.py)

  • Node labels changed from last-8-chars to YYYY-MM-DD dates.
  • Bottom axis: real acquisition dates. Top axis: days since first acquisition (swapped).
  • Left graph title removed.
  • Font sizes increased throughout; date labels rotated for readability.

Docs

  • Port references updated to 8080 across README, quickstart, and frontend docs.
  • file_structure.md/zh: added hyp3/ to directory layout; out_dir examples updated to .../hyp3.
  • cli.md/zh: --credential-pool corrected from "JSON" to plain username:password text file; prepprep_data.
  • index.md/zh: satellite support table added; program structure section moved to new Advanced page with workflow diagram.

v0.3.0

14 May 23:24

Choose a tag to compare

V0.3.0 ISCE2 Local Processor Support/HPC support

Changelog

[0.3.0] - 2026-05-14

New Features

  • ISCE_S1 local processor: New processor backend that runs ISCE2 stackSentinel locally. Supports sequential local execution and SLURM HPC mode (--hpc-mode). Bounding box is auto-filled from the map AOI in the GUI.
  • ISCE_SBAS analyzer: New MintPy SBAS analyzer for ISCE2 stackSentinel outputs. prep_data() auto-discovers interferogram, geometry, baseline, and metadata paths; MintPy outputs written to mintpy/ subdirectory.
  • HPC mode (SLURM): ISCE_S1 can submit each processing step as a separate sbatch job. Per-step resource configuration via sbatch_options.json, editable in the GUI via Sbatch Options modal.
  • Job Folders subfolder browser: The Jobs drawer now lists both folders and files. Click any folder to navigate into it; click ↑ Up to return to the parent. Breadcrumb path shown in the header.
  • Cancel button for local processors: A Cancel button appears in the ISCE_S1 processor panel to terminate the running background process (local) or scancel all active SLURM jobs (HPC).
  • Refresh with per-command detail: ISCE_S1 refresh() now shows per-command status (cmd_NNNN RUNNING / SUCCEEDED / FAILED) for multi-command steps, matching the CLI output.

Bug Fixes

  • ISCE_S1 bbox not passed: Processor.create() was calling cls(cfg) which mapped the config to the pairs argument in ISCE_S1's two-argument constructor. Fixed by detecting pairs in the constructor signature via inspect.signature and using keyword arguments.
  • ISCE_SBAS diagnostic geocoding: avgPhaseVelocity.h5, numTriNonzeroIntAmbiguity.h5, and maskConnComp.h5 are now geocoded automatically after the geocode step. Existing radar-coordinate data is geocoded on demand in the render endpoint.
  • ISCE_SBAS timeseries filter: View Results now returns only geo/geo_timeseries*.h5 (geocoded) when present, not the radar-coordinate timeseries*.h5 files.
  • ISCE_SBAS .mintpy.cfg path: Analyzer route was writing .mintpy.cfg to the job folder root; ISCE_SBAS expects it at mintpy/.mintpy.cfg. Fixed by reading analyzer.cfg_path at runtime.
  • ISCE_S1 submit via GUI missing sbatch options: _run_folder_process now loads sbatch_options.json and calls processor.submit() directly for local processors, bypassing the HyP3-only SubmitCommand/SaveJobsCommand wrappers.
  • cmd index parsing crash: int() raised ValueError on malformed cmd_????.done/fail filenames. Fixed with a safe _idx() helper.
  • Job Folders empty workdir path traversal: An empty workdir in browse-subfolders resolved to CWD, allowing requests outside the workdir. Fixed with an early 400 response when workdir is not configured.
  • Job Folders has_children OSError: subfolder.iterdir() on restricted directories could raise OSError. Wrapped in try/except.

Full Changelog: v0.2.5...v0.3.0

v0.2.5

21 Apr 19:36

Choose a tag to compare

Key updates:

  1. Network scoring system and modification before interferogram generation. Based on S1 global coherence dataset and weather data
  2. add quick_overview in mintpy for quality control
  3. re-oragnize fastaoi into seperate sub modules

Pair Quality Database:

  • DB no longer rebuilds when pair selection parameters change (dt_max, pb_max, min_degree, etc.) — only rebuilds when the scene set actually changes
  • Removing scenes from the network no longer triggers a rebuild; the existing DB already covers all subsets
  • DB now stores _scene_names for exact scene-set comparison instead of relying on count alone
  • Backward compatible: old DBs without _scene_names fall back to count comparison and migrate on the next rebuild
    Performance
  • S1 global coherence prefetch is now parallelized in two phases: season tile S3 downloads run concurrently (up to 4 threads), followed by per-pair numpy coherence evaluation in parallel (8 threads). Expected speedup for 32,000+ pairs: 4–6× on first run, with warm cache unchanged

select_pairs: Avoid Bad Acquisition Days (avoid_low_quality_days)

  • Default changed to True — bad-weather scenes are excluded from the network by default
  • Default precipitation threshold tightened to 25 mm (3-day accumulation)
  • Weather and snow data fetched during bad-scene filtering is now seeded directly into the pair quality cache — the scorer reuses it instead of making a second round of API calls, eliminating duplicate fetches entirely
  • select_pairs returns a prefetch_cache containing the pre-fetched weather/snow keyed by stack, which _run_folder_select_pairs writes into each subfolder's CacheManager before scoring begins
  • Status message "Building weather/snow cache — P{path}/F{frame}…" shown in the GUI while seeding so the user can see progress during long cache writes
    main
    ·
    v0.2.5
    1 paren

Full Changelog: v0.2.4...v0.2.5

v0.2.4

25 Mar 19:28

Choose a tag to compare

[0.2.4] - 2026-03-25

New Features

  • CLI & API: select_pairs() is now a pure computation method — no file I/O inside the class. File writing (JSON, PNG, workflow marker) has been moved to the CLI and API call sites, keeping the core logic reusable and testable
  • Path handling: All functions that accept path arguments now call .expanduser().resolve(), enabling ~ tilde paths everywhere
  • WebUI: Added documentation button in the General Settings panel (bottom-left) linking to the InSARHub docs site
  • WebUI (insarhub-app): Auto-creates the working directory if it does not exist when -w <path> is passed
  • CLI (insarhub-app): Added -v / --version flag
  • Windows fix: insarhub-app no longer returns immediately on Windows — sets WindowsSelectorEventLoopPolicy so uvicorn blocks correctly

Bug Fixes

  • WebUI Processor: Unchecking dry-run after a completed run no longer leaves the button stuck at "✓ Done" — the status resets to idle on checkbox change
  • WebUI Processor: Clicking "✓ Done" after a real (non-dry-run) submit now correctly closes the modal
  • WebUI Processor: "✓ Done" button now shows a pointer cursor on hover
  • Analyzer: Fixed NoneType crash in troposphere correction when Path.mkdir() was called on an already-resolved path
  • CLI credential setup: Removed spurious blank first line from .cdsapirc written by the interactive credential prompt

v0.2.3 bugfix

21 Mar 02:08

Choose a tag to compare

v0.2.3

Bug Fixes

  • Fixed troposphere correction crash — Path.mkdir() returns None, which was incorrectly assigned and passed to pyaps3 as the weather data directory
  • Fixed .cdsapirc file created by CLI prompt having a blank first line
  • Fixed dry run checkbox leaving the submit button stuck at "✓ Done" after job completion
  • Fixed "✓ Done" button not closing the modal after a real (non-dry-run) submission
  • Fixed missing pointer cursor on "✓ Done" button
  • Fixed pixel hover value lookup to use Mercator projection formula instead of linear WGS84 interpolation
  • Fixed unclosed file handle in /api/folder-image endpoint
  • Fixed RuntimeWarning: invalid value encountered in cast when rendering velocity PNG with NaN pixels
  • Added path traversal protection to /api/folder-image endpoint
  • Fixed polling interval leak in frontend job status watcher

Documentation

  • Home page: corrected package name from "InSAR Script" to "InSARHub", added Web UI overview screenshot and button, clarified WSL2 support
  • Header: added Email and Discord icon buttons next to GitHub badge
  • Header: reduced excess whitespace in GitHub badge

Features

  • Added yellow click marker on the map when a time-series pixel is selected
  • Click marker renders above the velocity overlay layer

v0.2.1 GUI release

20 Mar 02:24

Choose a tag to compare

v0.2.1

Web UI

  • Velocity map alignment fixed — corrected a projection error where UTM-projected MintPy outputs were reprojected to EPSG:3857 using wrong source CRS, causing the velocity overlay to appear shifted relative to the basemap
  • Time-series click marker — clicking a pixel on the velocity map now shows a yellow highlight point to indicate the selected location
  • Per-job analyzer status — the Stop button and run status in the Analyzer panel are now isolated per job folder; switching folders no longer bleeds status across jobs
  • Drawer auto-hides on map click — the job drawer collapses automatically when clicking on the map to extract a time-series

Downloader

  • Added --orbit-files option to download precise orbit files alongside scenes
  • Added granule name input: users can now supply a file, list, or string of granule names for custom scene searches
  • Fixed --stacks with multiple entries (e.g. 28:107 28:112) being incorrectly deduplicated by the ASF server
  • Orbit download now falls back to ASF server automatically if the CDSE server is unavailable
  • Fixed duplicate EOF file detection by checking acquire time

Bug Fixes

  • Fixed download error not enough values to unpack in the parallel download result handler
  • Fixed NumPy warning invalid value encountered in cast when rendering velocity colormaps with NaN pixels
  • Pinned CI to Python 3.12 to avoid instability with 3.13/3.14

Documentation

  • Completed Web UI documentation: Search & Download, Processor, Analyzer, and Results Viewer pages
  • Added figures for all UI panels with light/dark mode variants

v0.1.0 Init Release

09 Mar 18:46

Choose a tag to compare

Changelog

0.1.0 - 2026-03-06

Initial Release

First public release of InSARHub — a modular Python framework for automated InSAR time-series processing.


Features

Downloader

  • ASF_Base_Downloader: Search and download Sentinel-1, ALOS, and NISAR SLC data via the ASF Search API
  • Spatial filtering with bounding box, WKT, or GeoJSON/shapefile AOI
  • Post-search filtering by date range, path/frame, flight direction, polarization, season, coverage, and scene count
  • Scene footprint visualization with basemap overlay (footprint())
  • DEM download via dem-stitcher aligned to search footprints
  • Multi-threaded download with Ctrl+C cancellation and partial-file cleanup
  • S1_SLC: Sentinel-1 SLC specialized downloader with orbit file (sentineleof) support

Processor

  • Hyp3_InSAR: Submit, monitor, download, retry, and persist HyP3 InSAR jobs
  • Multi-account credential pool with automatic credit-aware job rotation
  • Batch job persistence (save/load JSON) for resumable workflows
  • watch() mode: polls job status and downloads succeeded outputs continuously
  • Retry failed jobs with automatic timestamp-stamped save files

Analyzer

  • Hyp3_SBAS: End-to-end MintPy SBAS time-series analysis from HyP3 outputs
  • Automatic unzip, file collection, common-overlap clipping, and MintPy config generation
  • Optional pyAPS tropospheric correction with CDS API credential management
  • cleanup() to remove temporary files after processing

Utilities

  • select_pairs: Temporal and perpendicular baseline filtering with configurable targets and tolerances
  • Local baseline computation (zero network calls for Sentinel-1 and ALOS)
  • API fallback with threaded fetching for products without local baseline data
  • Connectivity enforcement: minimum/maximum degree per scene with force-connect option
  • plot_pair_network: Network visualization with per-scene connection histogram
  • ERA5Downloader: Batch ERA5 reanalysis download for MintPy tropospheric correction, MintPy-compatible filenames
  • clip_hyp3_insar: Clip HyP3 zip outputs to a custom AOI before analysis
  • Slurmjob_Config: Generate SLURM batch scripts for HPC job submission
  • earth_credit_pool: Load multi-account Earthdata credentials from a pool file

CLI (insarhub)

  • insarhub download — search, filter, and download SLC scenes
  • insarhub processor submit/refresh/download/retry/watch/save/credits — full HyP3 job lifecycle
  • insarhub analyzer prep/run — prepare and run MintPy analysis
  • insarhub utils select-pairs/plot-network/era5/clip — utility commands
  • Workdir (-w) and credential pool (--credential-pool) flags across all subcommands

Core

  • Auto-registering component registry (Downloader, Processor, Analyzer)
  • InSAREngine: high-level pipeline runner with skip flags and watch mode
  • Unified CommandResult pattern shared between CLI and Panel frontend