Releases: jldz9/InSARHub
Releases · jldz9/InSARHub
v0.3.2
New Features
- HPC sliding-window submission (
isce_base.py) — each step now runs a lightweight sbatch manager job that keeps ≤max_concurrent_hpcchild jobs active at all times, refilling immediately on completion. Replaces the old batch-sequential approach. Consecutive steps with equal command counts (e.g.run_13–run_16) are merged into a single group-manager. Steps are chained via--dependency=afterok. - Per-command elapsed time in sbatch logs (
isce_base.py) — sbatch scripts printSTART/DONE/FAILwith elapsed seconds. Group tasks also print total elapsed across all grouped steps.
Bug Fixes
- HyP3 file paths (
hyp3_base.py) —hyp3_jobs.jsonnow saves to workdir root (wasworkdir/hyp3/); downloaded ZIPs go toworkdir/hyp3/; retry job files save to workdir root; legacyout_dir=workdirentries auto-migrate toworkdir/hyp3/. watchcommand ignores--intervalflag (cli/main.py) —_proc_local_watchreadargs.refresh_intervalbut argparse stores it asargs.interval; interval was always 60 s regardless of user input. Fixed.insarhub processor refresh/downloadre-processes retry job files (cli/main.py) —_find_job_filesglobbedhyp3*.json, matchinghyp3_retry_jobs_<ts>.jsonfiles from past runs alongsidehyp3_jobs.json. On refresh/download, stale retry files were loaded as separate processors, causing duplicate downloads or incorrect status. Fixed: retry files excluded from glob.- Orbit files downloaded twice (
cli/main.py) —-d -Oflags triggered two orbit downloads. Fixed: skips explicitdownload_orbit()call when downloader already handled it. retry()runs locally after HPC submission (isce_base.py,cli/main.py) —hpc_modeexcluded from saved config, so retry defaulted to local. Fixed:retry()auto-detects HPC from job metadata (slurm_job_ids/hpc_manager/hpc_array) and writes it back toconfig.hpc_modeso_step_executor()routes correctly._load_local_processoralso restoresmax_concurrent_hpcand HPC config fields frominsarhub_config.json.- Manager job killed mid-run leaves step stuck PENDING (
isce_base.py) —elif n_cmds > 0 and not job_ids:was dead code (always False insideif job_ids:block); SLURM-killed managers with incomplete commands never resolved to FAILED. Fixed: condition is nowelif n_cmds > 0:. _parse_time_smis-parses 2-part SLURM time strings (isce_base.py) —"30:00"was treated as 30 h 0 min (108,000 s) instead of 30 min 0 s (1,800 s), overestimating group-manager walltime 60×. Fixed: 2-part strings now parsed as MM:SS per SLURM spec.- Group-manager step stuck PENDING when job gone from SLURM (
isce_base.py) — ifgroup_task_dirwas absent from saved metadata (old jobs),n_cmdsresolved to 0, preventing the SUCCEEDED/FAILED transition. Fixed:n_cmdsnow stored in job metadata at submission; refresh uses it directly with file-count as fallback. refresh()shows only one RUNNING command in manager mode (isce_base.py) — with sliding-window, multiple commands run concurrently but only one showed RUNNING. Fixed: per-command status now derived from.done/.failfiles; all in-flight commands show RUNNING.Hyp3_SBASMintPy output in workdir root (mintpy_base.py) — base classrun()passedself.workdirtoTimeSeriesAnalysis;Hyp3_SBASinherits without override, so all MintPy outputs scattered to workdir root instead ofworkdir/mintpy/. Fixed: usesself.mintpy_dir; same correction for_geocode_diagnostic_files.- ISCE cleanup misses
merged/interferograms(isce_sbas.py) — cleanup targetedisce/interferograms/(nonexistent); real stackSentinel output isisce/merged/interferograms/. Large intermediate files were never deleted. - HyP3 auth failure submits to wrong user (
hyp3_base.py) — when re-auth failed for a pool user,credits=0was overwritten byself.client.check_credits()on the previous user's client; jobs were then submitted under the wrong account. Fixed: credits check guarded by auth result. self.batchsupdated per-loop-iteration (hyp3_base.py) — on multi-user refresh, if any user failed, their batch was silently dropped fromself.batchs. Fixed: assignment moved after loop.- Missing
filenamekey in HyP3 file metadata crashes download (hyp3_base.py) — direct dict subscript raisedKeyErrorfor auxiliary entries lackingfilename. Fixed:file_meta.get('filename')with skip on empty. - delete_job_folder blocks on
~-prefixed workdir (settings.py) —Path(workdir)withoutexpanduser().resolve()maderelative_to()always raiseValueError, returning 403 on every delete. Fixed. _run_folder_select_pairsusesfolder.parentas workdir (folders.py) — downloader config received parent directory instead of job folder; sub-paths written one level up, potentially colliding with sibling jobs. Fixed:workdir=folder.- Analyzer stop_event leaks on step error (
routes/analyzer.py) — earlyreturnon step exception bypassed_stop_events.pop(job_id), leaking events indefinitely. Fixed: pop before return. - Refresh overwrites
.insarhub_cache.jsonfilenames with empty list (routes/processor.py) — if no jobs had SUCCEEDED yet,filenames=[]overwrote a valid cache from a prior successful refresh. Fixed: preserves existing filenames when current refresh yields none. - Retry job files appear as selectable job entries in GUI (
routes/processor.py) —hyp3*.jsonglob matchedhyp3_retry_jobs_<ts>.json; selecting one for refresh returned only the retry batch status. Fixed: retry files excluded.
GUI
- Hyp3_S1
max_workersin settings panel (defaultconfig.py) — parallel download threads now configurable via Job settings group (default 4, range 1–16).
Refactor
- Centralized path layout (
config/paths.py) —Hyp3Paths,ISCEPaths,MintPyPathsdataclasses replace all hardcodedworkdir / "subdir"strings acrosshyp3_base.py,isce_base.py,isce_s1.py,mintpy_base.py,hyp3_sbas.py. - Path dataclass coverage extended — remaining hardcoded path literals replaced with dataclass properties:
isce_sbas.pynow usesISCEPathsforisce_dir,slc_dir,dem_dir;cli/main.pyusesHyp3Paths/ISCEPathsin_has_zipsand_find_job_file;utils/batch.pyandutils/tool.pyuseHyp3Paths.output_dirfor ZIP discovery. Hyp3Processorrenamed toCloudProcessor(core/base.py) — ABC renamed to reflect generic cloud-backend semantics rather than HyP3 specificity. Updated acrosscore/__init__.py,__init__.py,processor/hyp3_base.py,commands/processor.py,cli/main.py,core/engine.py.
Docs
- Contributing guide — new tab in MkDocs navigation (EN + ZH). Split into Overview, Backend, and Frontend pages.
- Backend contributing guide — architecture overview, path conventions, per-section instructions for adding new processors/downloaders/analyzers. Each section includes a "Adding a New Base X" subsection (with code examples for
CloudProcessor/LocalProcessor,BaseDownloader,BaseAnalyzer) and an "Extending an Existing Base X" subsection with switch tabs (Hyp3Base,ISCE_Base,ASF_Base_Downloader,Mintpy_SBAS_Base_Analyzer). - Frontend contributing guide — conda Node.js install, uvicorn backend startup from InSARHub root, module reference tables grouped by area (Entry & Global, Map, Search & Scene Selection, Jobs & Results, Settings, Utilities), backend communication pattern, settings panel, Vite proxy, build output, code style.
v0.3.1 bug fix
Bug Fixes
- Download result unpacking (
commands/processor.py) —processor.download()returns(Path, dict)tuple;DownloadCommandwas assigning the whole tuple tooutput_dir. Fixed: now unpacks tooutput_dir, dl_stats.CommandResult.datanow includes both output path and download stats. - HyP3 workflow marker wrong location (
hyp3_base.py) —write_workflow_markerwas writinginsarhub_config.jsontoworkdir/hyp3/instead of the job folder root, so HyP3 tags never appeared in the job drawer. Fixed: writes toconfig.workdir. - Stale
out_dirfrom saved job file (hyp3_base.py) — oldhyp3_jobs.jsonpointing to a pre-migration path outside current workdir would silently redirect output. Fixed:out_dirrejected if not under current workdir. wslpathunchecked (settings.py) — ifwslpath -wfailed, PowerShell was called with-File "". Fixed: returncode + empty string guard added.- ZIP detection for
hyp3/layout (cli/main.py) — analyzer now checksworkdir/hyp3/*.zipfirst, with fallback toworkdir/*.zipfor legacy layouts.
Performance
- Auth status parallel checks (
auth.py) — HyP3 credit check, CDSE, CDS, and Earthdata checks now run concurrently viaThreadPoolExecutorinstead of sequentially. Typical improvement: 3–5× faster settings panel load. - Job folder listing SSH speed (
settings.py) — removed all per-folderglob/exists/is_filechecks. Now reads onlyinsarhub_config.jsonper folder. Significant speedup on remote filesystems.
Source
hyp3/subdir awareness (hyp3_sbas.py,mintpy_base.py,batch.py) — all ZIP lookups now checkworkdir/hyp3/*.zipfirst, falling back toworkdir/*.zipfor legacy layouts. Affected paths:_unzip_hyp3,cleanup, andERA5Downloader.download_batch.- Missing
.mintpy.cfgguard (mintpy_base.py) — if.mintpy.cfgis not found whenrun()is called, a warning is printed and the config is written automatically rather than crashing downstream MintPy steps. write_mintpy_configparent mkdir (defaultconfig.py) —outpath.parent.mkdir(parents=True, exist_ok=True)added before opening the file, preventingFileNotFoundErrorwhen the output directory does not yet exist.
CLI
prepalias —insarhub analyzer run --step prepnow accepted as alias forprep_data. Help text updated to show alias.- Default port —
insarhub-appnow defaults to8080(was8000). Use--portto override.
GUI
- Subfolder navigation (
JobQueueDrawer) — click any folder to drill into subfolders;↑button to go up. Resets to workdir root on workdir change. Uses/api/browse-subfoldersendpoint. - Cancel button (
JobQueueDrawer) — Cancel action added for local ISCE jobs. - Modern folder picker (
settings.py) — Windows/WSL now usesIFileOpenDialogCOM API via embedded C# in PowerShell. Fixes: DPI blurriness on 2K monitors, Chinese character paths. - Add nyan cat
Network Graph (utils/tool.py)
- Node labels changed from last-8-chars to
YYYY-MM-DDdates. - Bottom axis: real acquisition dates. Top axis: days since first acquisition (swapped).
- Left graph title removed.
- Font sizes increased throughout; date labels rotated for readability.
Docs
- Port references updated to
8080across README, quickstart, and frontend docs. file_structure.md/zh: addedhyp3/to directory layout;out_direxamples updated to.../hyp3.cli.md/zh:--credential-poolcorrected from "JSON" to plainusername:passwordtext file;prep→prep_data.index.md/zh: satellite support table added; program structure section moved to new Advanced page with workflow diagram.
v0.3.0
V0.3.0 ISCE2 Local Processor Support/HPC support
Changelog
[0.3.0] - 2026-05-14
New Features
- ISCE_S1 local processor: New processor backend that runs ISCE2
stackSentinellocally. Supports sequential local execution and SLURM HPC mode (--hpc-mode). Bounding box is auto-filled from the map AOI in the GUI. - ISCE_SBAS analyzer: New MintPy SBAS analyzer for ISCE2
stackSentineloutputs.prep_data()auto-discovers interferogram, geometry, baseline, and metadata paths; MintPy outputs written tomintpy/subdirectory. - HPC mode (SLURM): ISCE_S1 can submit each processing step as a separate
sbatchjob. Per-step resource configuration viasbatch_options.json, editable in the GUI via Sbatch Options modal. - Job Folders subfolder browser: The Jobs drawer now lists both folders and files. Click any folder to navigate into it; click ↑ Up to return to the parent. Breadcrumb path shown in the header.
- Cancel button for local processors: A Cancel button appears in the ISCE_S1 processor panel to terminate the running background process (local) or
scancelall active SLURM jobs (HPC). - Refresh with per-command detail: ISCE_S1
refresh()now shows per-command status (cmd_NNNN RUNNING / SUCCEEDED / FAILED) for multi-command steps, matching the CLI output.
Bug Fixes
- ISCE_S1 bbox not passed:
Processor.create()was callingcls(cfg)which mapped the config to thepairsargument in ISCE_S1's two-argument constructor. Fixed by detectingpairsin the constructor signature viainspect.signatureand using keyword arguments. - ISCE_SBAS diagnostic geocoding:
avgPhaseVelocity.h5,numTriNonzeroIntAmbiguity.h5, andmaskConnComp.h5are now geocoded automatically after thegeocodestep. Existing radar-coordinate data is geocoded on demand in the render endpoint. - ISCE_SBAS timeseries filter: View Results now returns only
geo/geo_timeseries*.h5(geocoded) when present, not the radar-coordinatetimeseries*.h5files. - ISCE_SBAS
.mintpy.cfgpath: Analyzer route was writing.mintpy.cfgto the job folder root; ISCE_SBAS expects it atmintpy/.mintpy.cfg. Fixed by readinganalyzer.cfg_pathat runtime. - ISCE_S1 submit via GUI missing sbatch options:
_run_folder_processnow loadssbatch_options.jsonand callsprocessor.submit()directly for local processors, bypassing the HyP3-onlySubmitCommand/SaveJobsCommandwrappers. - cmd index parsing crash:
int()raisedValueErroron malformedcmd_????.done/failfilenames. Fixed with a safe_idx()helper. - Job Folders empty workdir path traversal: An empty workdir in
browse-subfoldersresolved to CWD, allowing requests outside the workdir. Fixed with an early 400 response when workdir is not configured. - Job Folders
has_childrenOSError:subfolder.iterdir()on restricted directories could raiseOSError. Wrapped intry/except.
Full Changelog: v0.2.5...v0.3.0
v0.2.5
Key updates:
- Network scoring system and modification before interferogram generation. Based on S1 global coherence dataset and weather data
- add quick_overview in mintpy for quality control
- re-oragnize fastaoi into seperate sub modules
Pair Quality Database:
- DB no longer rebuilds when pair selection parameters change (dt_max, pb_max, min_degree, etc.) — only rebuilds when the scene set actually changes
- Removing scenes from the network no longer triggers a rebuild; the existing DB already covers all subsets
- DB now stores _scene_names for exact scene-set comparison instead of relying on count alone
- Backward compatible: old DBs without _scene_names fall back to count comparison and migrate on the next rebuild
Performance - S1 global coherence prefetch is now parallelized in two phases: season tile S3 downloads run concurrently (up to 4 threads), followed by per-pair numpy coherence evaluation in parallel (8 threads). Expected speedup for 32,000+ pairs: 4–6× on first run, with warm cache unchanged
select_pairs: Avoid Bad Acquisition Days (avoid_low_quality_days)
- Default changed to True — bad-weather scenes are excluded from the network by default
- Default precipitation threshold tightened to 25 mm (3-day accumulation)
- Weather and snow data fetched during bad-scene filtering is now seeded directly into the pair quality cache — the scorer reuses it instead of making a second round of API calls, eliminating duplicate fetches entirely
- select_pairs returns a prefetch_cache containing the pre-fetched weather/snow keyed by stack, which _run_folder_select_pairs writes into each subfolder's CacheManager before scoring begins
- Status message "Building weather/snow cache — P{path}/F{frame}…" shown in the GUI while seeding so the user can see progress during long cache writes
main
·
v0.2.5
1 paren
Full Changelog: v0.2.4...v0.2.5
v0.2.4
[0.2.4] - 2026-03-25
New Features
- CLI & API:
select_pairs()is now a pure computation method — no file I/O inside the class. File writing (JSON, PNG, workflow marker) has been moved to the CLI and API call sites, keeping the core logic reusable and testable - Path handling: All functions that accept path arguments now call
.expanduser().resolve(), enabling~tilde paths everywhere - WebUI: Added documentation button in the General Settings panel (bottom-left) linking to the InSARHub docs site
- WebUI (
insarhub-app): Auto-creates the working directory if it does not exist when-w <path>is passed - CLI (
insarhub-app): Added-v/--versionflag - Windows fix:
insarhub-appno longer returns immediately on Windows — setsWindowsSelectorEventLoopPolicyso uvicorn blocks correctly
Bug Fixes
- WebUI Processor: Unchecking dry-run after a completed run no longer leaves the button stuck at "✓ Done" — the status resets to idle on checkbox change
- WebUI Processor: Clicking "✓ Done" after a real (non-dry-run) submit now correctly closes the modal
- WebUI Processor: "✓ Done" button now shows a pointer cursor on hover
- Analyzer: Fixed
NoneTypecrash in troposphere correction whenPath.mkdir()was called on an already-resolved path - CLI credential setup: Removed spurious blank first line from
.cdsapircwritten by the interactive credential prompt
v0.2.3 bugfix
v0.2.3
Bug Fixes
- Fixed troposphere correction crash —
Path.mkdir()returnsNone, which was incorrectly assigned and passed to pyaps3 as the weather data directory - Fixed
.cdsapircfile created by CLI prompt having a blank first line - Fixed dry run checkbox leaving the submit button stuck at "✓ Done" after job completion
- Fixed "✓ Done" button not closing the modal after a real (non-dry-run) submission
- Fixed missing pointer cursor on "✓ Done" button
- Fixed pixel hover value lookup to use Mercator projection formula instead of linear WGS84 interpolation
- Fixed unclosed file handle in
/api/folder-imageendpoint - Fixed
RuntimeWarning: invalid value encountered in castwhen rendering velocity PNG with NaN pixels - Added path traversal protection to
/api/folder-imageendpoint - Fixed polling interval leak in frontend job status watcher
Documentation
- Home page: corrected package name from "InSAR Script" to "InSARHub", added Web UI overview screenshot and button, clarified WSL2 support
- Header: added Email and Discord icon buttons next to GitHub badge
- Header: reduced excess whitespace in GitHub badge
Features
- Added yellow click marker on the map when a time-series pixel is selected
- Click marker renders above the velocity overlay layer
v0.2.1 GUI release
v0.2.1
Web UI
- Velocity map alignment fixed — corrected a projection error where UTM-projected MintPy outputs were reprojected to EPSG:3857 using wrong source CRS, causing the velocity overlay to appear shifted relative to the basemap
- Time-series click marker — clicking a pixel on the velocity map now shows a yellow highlight point to indicate the selected location
- Per-job analyzer status — the Stop button and run status in the Analyzer panel are now isolated per job folder; switching folders no longer bleeds status across jobs
- Drawer auto-hides on map click — the job drawer collapses automatically when clicking on the map to extract a time-series
Downloader
- Added
--orbit-filesoption to download precise orbit files alongside scenes - Added granule name input: users can now supply a file, list, or string of granule names for custom scene searches
- Fixed
--stackswith multiple entries (e.g.28:107 28:112) being incorrectly deduplicated by the ASF server - Orbit download now falls back to ASF server automatically if the CDSE server is unavailable
- Fixed duplicate EOF file detection by checking acquire time
Bug Fixes
- Fixed download error
not enough values to unpackin the parallel download result handler - Fixed NumPy warning
invalid value encountered in castwhen rendering velocity colormaps with NaN pixels - Pinned CI to Python 3.12 to avoid instability with 3.13/3.14
Documentation
- Completed Web UI documentation: Search & Download, Processor, Analyzer, and Results Viewer pages
- Added figures for all UI panels with light/dark mode variants
v0.1.0 Init Release
Changelog
0.1.0 - 2026-03-06
Initial Release
First public release of InSARHub — a modular Python framework for automated InSAR time-series processing.
Features
Downloader
ASF_Base_Downloader: Search and download Sentinel-1, ALOS, and NISAR SLC data via the ASF Search API- Spatial filtering with bounding box, WKT, or GeoJSON/shapefile AOI
- Post-search filtering by date range, path/frame, flight direction, polarization, season, coverage, and scene count
- Scene footprint visualization with basemap overlay (
footprint()) - DEM download via
dem-stitcheraligned to search footprints - Multi-threaded download with Ctrl+C cancellation and partial-file cleanup
S1_SLC: Sentinel-1 SLC specialized downloader with orbit file (sentineleof) support
Processor
Hyp3_InSAR: Submit, monitor, download, retry, and persist HyP3 InSAR jobs- Multi-account credential pool with automatic credit-aware job rotation
- Batch job persistence (save/load JSON) for resumable workflows
watch()mode: polls job status and downloads succeeded outputs continuously- Retry failed jobs with automatic timestamp-stamped save files
Analyzer
Hyp3_SBAS: End-to-end MintPy SBAS time-series analysis from HyP3 outputs- Automatic unzip, file collection, common-overlap clipping, and MintPy config generation
- Optional pyAPS tropospheric correction with CDS API credential management
cleanup()to remove temporary files after processing
Utilities
select_pairs: Temporal and perpendicular baseline filtering with configurable targets and tolerances- Local baseline computation (zero network calls for Sentinel-1 and ALOS)
- API fallback with threaded fetching for products without local baseline data
- Connectivity enforcement: minimum/maximum degree per scene with force-connect option
plot_pair_network: Network visualization with per-scene connection histogramERA5Downloader: Batch ERA5 reanalysis download for MintPy tropospheric correction, MintPy-compatible filenamesclip_hyp3_insar: Clip HyP3 zip outputs to a custom AOI before analysisSlurmjob_Config: Generate SLURM batch scripts for HPC job submissionearth_credit_pool: Load multi-account Earthdata credentials from a pool file
CLI (insarhub)
insarhub download— search, filter, and download SLC scenesinsarhub processor submit/refresh/download/retry/watch/save/credits— full HyP3 job lifecycleinsarhub analyzer prep/run— prepare and run MintPy analysisinsarhub utils select-pairs/plot-network/era5/clip— utility commands- Workdir (
-w) and credential pool (--credential-pool) flags across all subcommands
Core
- Auto-registering component registry (
Downloader,Processor,Analyzer) InSAREngine: high-level pipeline runner with skip flags and watch mode- Unified
CommandResultpattern shared between CLI and Panel frontend