Skip to content

TestKit deadlock during parallel test execution with TestActor initialization #7770

@Aaronontheweb

Description

@Aaronontheweb

Description

When running tests in parallel using Akka.TestKit (especially with Akka.Hosting.TestKit), a deadlock can occur during TestActor initialization. This causes tests to fail intermittently when run in parallel.

Root Cause

The deadlock occurs due to the following sequence:

  1. TestActor is created using SystemActorOf which calls AttachChild(props, true, name) with async=true
  2. This creates a RepointableActorRef that calls Initialize(async: true)
  3. The Initialize(true) method sends a Supervise system message but does NOT immediately call Point()
  4. The TestActor runs on the CallingThreadDispatcher
  5. The Supervise message needs to be processed on the calling thread to complete initialization
  6. In Akka.Hosting.TestKit, this happens inside a WithActors callback which blocks the startup thread
  7. Deadlock: The thread is blocked waiting for initialization, but initialization needs that same thread to process the Supervise message

Reproduction

See reproduction test in Akka.Hosting PR: akkadotnet/Akka.Hosting#643

The issue manifests as:

System.Exception : Timeout waiting for test actor to be ready
  at TestKitBase.InitializeTest()

Symptoms

  • Tests fail randomly when run locally with parallel execution enabled
  • Tests may pass in CI/CD pipelines with different threading characteristics
  • Changing XUnit's parallelAlgorithm setting doesn't fully resolve the issue
  • Issue is more likely to occur with high test parallelism (e.g., 40+ concurrent tests)

Technical Details

The core issue is in TestKitBase.CreateTestActor:

  • Uses SystemActorOf which creates a RepointableActorRef with async initialization
  • The TestActor uses CallingThreadDispatcher (configured in Reference.conf as akka.test.test-actor.dispatcher)
  • During async initialization, the Supervise message must be processed on the calling thread
  • If that thread is blocked waiting for initialization, we have a circular dependency

Affected Components

  • Akka.TestKit
  • Akka.TestKit.Xunit2
  • Akka.Hosting.TestKit
  • Any test framework using TestKitBase with parallel execution

Environment

  • Akka.NET: 1.5.45+
  • Affects all platforms (Windows, Linux, macOS)
  • More pronounced with high CPU core counts and aggressive parallelism

Workaround

Current workaround is to disable parallel test execution in xunit.runner.json:

{
  "parallelizeTestCollections": false
}

However, this significantly increases test execution time.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions