test(ZIOApp): add comprehensive process-level test suite (#9909) by 987Nabil · Pull Request #10579 · zio/zio

987Nabil · 2026-03-13T14:01:43Z

Summary

Fixes #9909: Create test suite that tests the correct behaviour of ZIOApp
/claim #9909

This PR adds a process-level test suite that spawns real child JVM processes via ProcessBuilder and verifies ZIOApp behavior externally — testing the actual main() code path including JVM shutdown hooks, signal handling, and System.exit.

What's tested

Natural completion

Successful app emits exit code 0
Failed app (ZIO.fail) emits exit code 1
Defect (ZIO.die) emits exit code 1

Signal handling

Finalizers run when app receives SIGTERM
Multiple finalizers all run on signal (regression ZIO 2.1.18 does not wait on termination of registered finalizers #9901)
Shutdown doesn't hang (completes within bounded time)
gracefulShutdownTimeout is respected (slow finalizer doesn't block exit)

Regressions

No uncaught exception on stderr during shutdown (regression Unclean shutdown as a result of a race between shutdown hooks #9807)
Signal handler works via reflection without NoClassDefFoundError (regression Dependency on sun.misc.SignalHandler causing NoClassDefFoundError #9240)

Approach

Unlike invoke()-based tests (which bypass main(), shutdown hooks, and exit codes), these tests:

Spawn real child JVM processes using ProcessBuilder with the test classpath
Send SIGTERM via kill -TERM <pid> to trigger JVM shutdown hooks
Capture stdout/stderr and verify sentinel markers (e.g., FINALIZER_RAN)
Assert exit codes from the actual process

Files

ZIOAppProcessSpec.scala — 9 process-level tests
testapps/ProcessTestHelper.scala — helper for spawning/signaling child JVMs (~99 lines)
testapps/TestApp*.scala — 7 minimal ZIOAppDefault programs, each exercising one behavior

Test results

+ ZIOAppProcessSpec
  + natural completion
    + successful app emits exit code 0 - 526 ms
    + failed app emits exit code 1 - 477 ms
    + defect (die) emits exit code 1 - 452 ms
  + signal handling
    + finalizers run when app receives SIGTERM - 490 ms
    + multiple finalizers all run on signal (regression #9901) - 489 ms
    + shutdown doesn't hang - 474 ms
    + gracefulShutdownTimeout is respected - 2 s 453 ms
  + regressions
    + no uncaught exception on stderr during shutdown (regression #9807) - 1 s 494 ms
    + signal handler works via reflection without NoClassDefFoundError (regression #9240) - 453 ms
9 tests passed. 0 tests failed. 0 tests ignored.

Stable across 3 consecutive runs with zero flakiness.

Copilot

Pull request overview

Adds a process-level test suite to validate ZIOApp behavior via real child JVM processes, exercising the actual main() path (shutdown hooks, SIGTERM handling, and exit codes) to cover key regressions.

Changes:

Introduces ZIOAppProcessSpec with process-spawning tests for exit codes, SIGTERM finalization, timeout behavior, and regression cases.
Adds ProcessTestHelper plus several minimal ZIOAppDefault “test apps” used as spawned child programs.
Adds regression-focused spawned apps (slow finalizer, multiple finalizers, clean shutdown hook, etc.).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
core-tests/jvm-native/src/test/scala/zio/ZIOAppProcessSpec.scala	New process-level spec that spawns JVMs and asserts exit/shutdown behavior
core-tests/jvm-native/src/test/scala/zio/testapps/ProcessTestHelper.scala	Helper to spawn JVM subprocesses and capture stdout/stderr
core-tests/jvm-native/src/test/scala/zio/testapps/TestAppSuccess.scala	Child app that completes successfully
core-tests/jvm-native/src/test/scala/zio/testapps/TestAppFailure.scala	Child app that fails to validate exit code behavior
core-tests/jvm-native/src/test/scala/zio/testapps/TestAppDie.scala	Child app that dies to validate defect exit code behavior
core-tests/jvm-native/src/test/scala/zio/testapps/TestAppFinalizer.scala	Child app that registers a finalizer for SIGTERM shutdown validation
core-tests/jvm-native/src/test/scala/zio/testapps/TestAppMultipleFinalizers.scala	Child app to validate multiple finalizers run on shutdown (regression #9901)
core-tests/jvm-native/src/test/scala/zio/testapps/TestAppSlowFinalizer.scala	Child app to validate `gracefulShutdownTimeout` behavior
core-tests/jvm-native/src/test/scala/zio/testapps/TestAppCleanShutdown.scala	Child app to validate clean shutdown (regression #9807)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+object ZIOAppProcessSpec extends ZIOBaseSpec {
+
+  private val defaultTimeout = java.time.Duration.ofSeconds(30)


+    var line = stdoutReader.readLine()
+    while (line != null && !line.contains(readyMarker)) {
+      stdoutBuilder.append(line).append('\n')
+      line = stdoutReader.readLine()
+    }
+    if (line != null) stdoutBuilder.append(line).append('\n')


+    val pid = process.pid()
+    java.lang.Runtime.getRuntime.exec(Array("kill", "-TERM", pid.toString)).waitFor()
+
+    val remainingMs = timeout.toMillis - (java.lang.System.currentTimeMillis() - startTime)
+


+      test("successful app emits exit code 0") {
+        ZIO.attemptBlocking {
+          ProcessTestHelper.runApp("zio.testapps.TestAppSuccess", defaultTimeout)
+        }.map { result =>


+  def runAppAndSignal(mainClass: String, readyMarker: String, timeout: java.time.Duration): ProcessResult = {
+    val classpath = System.getProperty("java.class.path")
+    val javaHome  = System.getProperty("java.home")
+    val javaBin   = s"$javaHome/bin/java"
+
+    val pb      = new ProcessBuilder(javaBin, "-cp", classpath, mainClass)
+    val process = pb.start()
+
+    val stdoutBuilder = new StringBuilder
+    val stderrBuilder = new StringBuilder
+
+    val stderrThread = readerThread(new BufferedReader(new InputStreamReader(process.getErrorStream)), stderrBuilder)
+    stderrThread.start()
+
+    val stdoutReader = new BufferedReader(new InputStreamReader(process.getInputStream))
+    val startTime    = System.currentTimeMillis()
+
+    var line = stdoutReader.readLine()
+    while (line != null && !line.contains(readyMarker)) {
+      stdoutBuilder.append(line).append('\n')
+      line = stdoutReader.readLine()
+    }
+    if (line != null) stdoutBuilder.append(line).append('\n')
+
+    process.destroy()
+
+    val remainingMs = timeout.toMillis - (System.currentTimeMillis() - startTime)
+
+    val tailThread = readerThread(stdoutReader, stdoutBuilder)
+    tailThread.start()
+
+    val exited = process.waitFor(math.max(remainingMs, 1000L), TimeUnit.MILLISECONDS)
+    if (!exited) process.destroyForcibly()
+
+    tailThread.join(5000)
+    stderrThread.join(5000)
+
+    ProcessResult(
+      if (exited) process.exitValue() else -1,
+      stdoutBuilder.toString,
+      stderrBuilder.toString
+    )
+  }
+


+    val stdoutReader = new BufferedReader(new InputStreamReader(process.getInputStream))
+    val startTime    = System.currentTimeMillis()
+
+    var line = stdoutReader.readLine()
+    while (line != null && !line.contains(readyMarker)) {
+      stdoutBuilder.append(line).append('\n')
+      line = stdoutReader.readLine()
+    }
+    if (line != null) stdoutBuilder.append(line).append('\n')
+


kyri-petrou

The PR looks good to me minus some minor comments, but when I wrote the issue I also had in mind that the tests would also cover JS and Native.

Do you think it'd be feasible to add those as well?

kyri-petrou · 2026-03-16T11:27:03Z

+
+object TestAppCleanShutdown extends ZIOAppDefault {
+  def run = ZIO.succeed {
+    java.lang.Runtime.getRuntime.addShutdownHook(new Thread(() => Thread.sleep(1000)))


Can we test/assert that whatever is run within the hook is actually executed?

kyri-petrou · 2026-03-16T11:28:36Z

+
+object TestAppFinalizer extends ZIOAppDefault {
+  def run =
+    ZIO.acquireRelease(Console.printLine("APP_STARTED"))(_ => Console.printLine("FINALIZER_RAN").orDie) *> ZIO.never


Let's explicitely use ZIO.scoped instead of relying on the app scope, wdut? (here and elsewhere)

But we test the behavior of the App or not? So we want to make sure that the app scope is doing the right thing. Or am I missing something?

Add process-level tests for ZIOApp behavior, verifying correct exit codes, finalizer execution on SIGTERM, gracefulShutdownTimeout, clean shutdown (regression zio#9807), signal handler reflection (regression zio#9240), and multiple finalizer execution (regression zio#9901). Tests spawn real child JVM processes via ProcessBuilder, send SIGTERM via kill -TERM, and assert on stdout/stderr/exit code. All test files are in core-tests/jvm/ (JVM-only) since they require a JVM to spawn.

Add invoke()-based tests for multiple-finalizer execution and LIFO ordering on interruption. These run on all platforms (JVM, JS, Native) via the shared ZIOAppSpec, covering the ZIO-level logic behind the JVM-only regression scenarios. Process-level tests remain JVM-only since shutdown hooks are no-ops on both JS and Native (PlatformSpecific.addShutdownHook discards the action on both platforms).

github-actions · 2026-05-04T11:37:26Z

🚀 Preview deployed to Netlify: https://pr-10579--zio-dev.netlify.app

algora-pbc Bot added the 🙋 Bounty claim label Mar 13, 2026

algora-pbc Bot mentioned this pull request Mar 13, 2026

Create test suite that tests the correct behaviour of ZIOApp #9909

Open

987Nabil requested a review from Copilot March 13, 2026 14:05

Copilot started reviewing on behalf of 987Nabil March 13, 2026 14:07 View session

Copilot AI reviewed Mar 13, 2026

View reviewed changes

987Nabil force-pushed the feat/zioapp-test-suite-9909 branch from 0121ad8 to a4db82d Compare March 14, 2026 13:29

kyri-petrou reviewed Mar 16, 2026

View reviewed changes

987Nabil force-pushed the feat/zioapp-test-suite-9909 branch from a4db82d to 85d7313 Compare March 17, 2026 16:29

987Nabil and others added 2 commits March 18, 2026 15:45

Merge branch 'series/2.x' into feat/zioapp-test-suite-9909

98bef15

guizmaii removed the 🙋 Bounty claim label May 26, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(ZIOApp): add comprehensive process-level test suite (#9909)#10579

test(ZIOApp): add comprehensive process-level test suite (#9909)#10579
987Nabil wants to merge 3 commits into
zio:series/2.xfrom
987Nabil:feat/zioapp-test-suite-9909

987Nabil commented Mar 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

kyri-petrou left a comment

Uh oh!

kyri-petrou Mar 16, 2026

Uh oh!

kyri-petrou Mar 16, 2026

Uh oh!

987Nabil Mar 20, 2026

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		object ZIOAppProcessSpec extends ZIOBaseSpec {

		private val defaultTimeout = java.time.Duration.ofSeconds(30)

Conversation

987Nabil commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's tested

Natural completion

Signal handling

Regressions

Approach

Files

Test results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

kyri-petrou left a comment

Choose a reason for hiding this comment

Uh oh!

kyri-petrou Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

kyri-petrou Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

987Nabil Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

987Nabil commented Mar 13, 2026 •

edited

Loading