-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Closed
Copy link
Labels
Milestone
Description
Description
When running tests in parallel using Akka.TestKit (especially with Akka.Hosting.TestKit), a deadlock can occur during TestActor initialization. This causes tests to fail intermittently when run in parallel.
Root Cause
The deadlock occurs due to the following sequence:
- TestActor is created using
SystemActorOfwhich callsAttachChild(props, true, name)withasync=true - This creates a
RepointableActorRefthat callsInitialize(async: true) - The
Initialize(true)method sends aSupervisesystem message but does NOT immediately callPoint() - The TestActor runs on the
CallingThreadDispatcher - The
Supervisemessage needs to be processed on the calling thread to complete initialization - In Akka.Hosting.TestKit, this happens inside a
WithActorscallback which blocks the startup thread - Deadlock: The thread is blocked waiting for initialization, but initialization needs that same thread to process the
Supervisemessage
Reproduction
See reproduction test in Akka.Hosting PR: akkadotnet/Akka.Hosting#643
The issue manifests as:
System.Exception : Timeout waiting for test actor to be ready
at TestKitBase.InitializeTest()
Symptoms
- Tests fail randomly when run locally with parallel execution enabled
- Tests may pass in CI/CD pipelines with different threading characteristics
- Changing XUnit's
parallelAlgorithmsetting doesn't fully resolve the issue - Issue is more likely to occur with high test parallelism (e.g., 40+ concurrent tests)
Technical Details
The core issue is in TestKitBase.CreateTestActor:
- Uses
SystemActorOfwhich creates aRepointableActorRefwith async initialization - The TestActor uses
CallingThreadDispatcher(configured in Reference.conf asakka.test.test-actor.dispatcher) - During async initialization, the
Supervisemessage must be processed on the calling thread - If that thread is blocked waiting for initialization, we have a circular dependency
Affected Components
- Akka.TestKit
- Akka.TestKit.Xunit2
- Akka.Hosting.TestKit
- Any test framework using TestKitBase with parallel execution
Environment
- Akka.NET: 1.5.45+
- Affects all platforms (Windows, Linux, macOS)
- More pronounced with high CPU core counts and aggressive parallelism
Workaround
Current workaround is to disable parallel test execution in xunit.runner.json:
{
"parallelizeTestCollections": false
}However, this significantly increases test execution time.