Add configurable logging to withArg & withNullableArg #1441

OsaSoft · 2025-10-23T13:17:22Z

Problem decription:

After upgrading to a newer version I noticed a LOT of spamming in the console logs, enough to significantly slow down test execution and even crash the IDE. After some research I narrowed it down to this PR: #1395

The issue I encountered is caused by a combination of these factors:

using property based testing (many generated test cases)
verifying different calls to the same mock
the change to withArg and withNullableArg dumping the error stack trace on any failed match, even if the test itself passes

Proposed solution:

Add an argument flag to the withArg and withNullableArg functions to toggle the stacktrace dump behaviour

Reproduction:

build.gradle.kts

import org.gradle.internal.impldep.org.junit.runner.Request.runner
import org.jetbrains.kotlin.gradle.plugin.KotlinPlatformType

plugins {
    kotlin("jvm") version "2.2.20"
}

group = "org.example"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
    mavenLocal()
}

dependencies {
    testImplementation(kotlin("test"))

    testImplementation("io.kotest:kotest-assertions-core:6.0.3")
    testImplementation("io.kotest:kotest-assertions-core-jvm:6.0.3")
    testImplementation("io.kotest:kotest-property:6.0.3")
    testImplementation("io.kotest:kotest-framework-engine-jvm:6.0.3")
    testImplementation("io.kotest:kotest-runner-junit5:6.0.3")

    testImplementation("io.mockk:mockk:1.14.7-SNAPSHOT-local")
}

tasks.test {
    useJUnitPlatform()
}
kotlin {
    jvmToolchain(21)
}

Foo.kt

package cloud.osasoft.mockkTest

interface Bar {
    fun doSomething(number: Int)
}

class Foo(private val bar: Bar) {

    fun fooBar(a: Int, b: Int) {
        bar.doSomething(a)
        bar.doSomething(b)
    }
}

FooSpec.kt

package cloud.osasoft.mockkTest

import io.kotest.assertions.shouldFail
import io.kotest.core.spec.style.FreeSpec
import io.kotest.matchers.ints.shouldBeLessThan
import io.kotest.matchers.shouldBe
import io.kotest.property.Arb
import io.kotest.property.arbitrary.int
import io.kotest.property.arbitrary.positiveInt
import io.kotest.property.checkAll
import io.mockk.mockk
import io.mockk.verify

class FooSpec : FreeSpec(
    {
        "This spams log a lot" {
            checkAll(
                Arb.int(1..10),
                Arb.int(1..10),
            ) { a, b ->
                val bar = mockk<Bar>(relaxed = true)
                val foo = Foo(bar)

                foo.fooBar(a, b)
                verify {
                    bar.doSomething(
                        withArg { it shouldBe a }
                    )
                    bar.doSomething(
                        withArg { it shouldBe b }
                    )
                }
            }
        }

        "This does not spam and successfully passes" {
            checkAll(
                Arb.int(),
                Arb.int(),
            ) { a, b ->
                val bar = mockk<Bar>(relaxed = true)
                val foo = Foo(bar)

                foo.fooBar(a, b)

                verify {
                    bar.doSomething(
                        withArg(logAllAssertionErrors = false) { it shouldBe a }
                    )
                    bar.doSomething(
                        withArg(logAllAssertionErrors = false) { it shouldBe b }
                    )
                }
            }
        }

        "This does not spam and fails" {
            checkAll(
                Arb.positiveInt(),
                Arb.positiveInt(),
            ) { a, b ->
                shouldFail {
                    val bar = mockk<Bar>(relaxed = true)
                    val foo = Foo(bar)

                    foo.fooBar(a, b)

                    verify {
                        bar.doSomething(
                            withArg(logAllAssertionErrors = false) { it shouldBe a }
                        )
                        bar.doSomething(
                            withArg(logAllAssertionErrors = false) { it shouldBe b }
                        )
                        bar.doSomething(
                            withArg(logAllAssertionErrors = false) {
                                // This will fail
                                it shouldBeLessThan 0
                            }
                        )
                    }
                }
            }
        }
    }
)

Here the first testcase spams the console with messages like this:

Possible future improvements:

Would be nice if the behaviour could be configured project-wise via mockk config (would love some pointers on how to achieve this).
Would be nice if the stacktrace dump actually used a logger instead of directly dumping to console.

Raibaz · 2025-10-23T15:22:01Z

Thanks for the investigation!

What if instead of just passing a flag to turn on/off the printStackTrace calls we gave the option to pass a java.util.logging.Logger to use to log the stacktraces?

That would also solve at least one of the future improvements you identified.

OsaSoft · 2025-10-24T14:24:29Z

Hi @Raibaz . Thats an interesting idea. How do you think we should handle the default there to maintain the current functionality? Nullable with default null, and if its null keep current behaviour, otherwise use the provided logger? Should we use a WARN or maybe even INFO level?

Or it could be something like this, that way when we dont want any logging, we can even completely skip the logger overhead, which for a lot of larger strings (like in this case) can actually have a noticable impact and we can leave it up to the user to use whatever log level they deem appropriate
assertionErrorLogger: (AssertionError) -> Unit = { e -> e.printStackTrace() },

then we could call the function e.g. like this:

withArg(assertionErrorLogger = logger::debug) {
  a shouldBe b
}

edit: actually that wouldnt work, since the receiver param would be the assertionError 🤔 would work with a String so then in the matcher we would have to call it with the exception converted to string, but that could lose the exception instance, if we wanted e.g. to log it like logger.error("Failed matching arg", ex)

Raibaz · 2025-10-24T14:47:53Z

Uhm, why wouldn't it work?

We could implement the FunctionMatcher class like this:

data class FunctionMatcher<in T : Any>(
    val matchingFunc: (T) -> Boolean,
    override val argumentType: KClass<*>,
    private val assertionErrorLogger: (AssertionError) -> Unit = {e -> e.printStackTrace()}, // Existing behavior by default
) : Matcher<T>, TypedMatcher, EquivalentMatcher {
    override fun equivalent(): Matcher<Any> = ConstantMatcher(true)

    override fun match(arg: T?): Boolean {
        return if (arg == null) {
            false
        } else {
            try {
                matchingFunc(arg)
            } catch (a: AssertionError) {
                assertionErrorLogger();
                false
            }
        }
    }

And change the withArg function to accept a (AssertionError) -> Unit param to be relayed to the FunctionMatcher.

I quite like it, as it offer good customizability; I actually like it way more than just passing a logger.

OsaSoft · 2025-10-27T20:57:36Z

i meant that a very short and clean reference pass wouldnt work, ie withArg(assertionErrorLogger = logger::debug), but yeah, with lambda bodies it should work fine. Let me do a few quick tests, see what works best for a nice syntax and ill get back to you once i push the changes

OsaSoft · 2025-10-27T21:58:21Z

ok, so i think this signature works quite well, some example usages

// quiet
withArg({}) { it shouldBe a }

// logger
withArg({ logger.debug(it) }) { it shouldBe a}

// custom reference
withArg(::logAssertion) { it shouldBe a }
fun logAssertion(error: AssertionError) = ...

one more thing Im considering: since right now the default function body is defined the same in quite a few places, could it be moved to possibly some static value? Where could this go, is there some file containing different util functions etc? Or maybe some config class that could be a good place to put it?

Raibaz · 2025-10-28T08:14:15Z

Can you please run gradle :mockk-dsl:apiDump to update the api definition files and make the builds pass?

OsaSoft · 2025-10-28T21:54:33Z

ok, should be done. Thanks for letting me know which exactly to run :)

Raibaz · 2025-10-29T11:27:41Z

Thanks!

hho · 2025-12-15T13:52:53Z

modules/mockk-dsl/src/commonMain/kotlin/io/mockk/API.kt

+     *
+     * @param logAllAssertionErrors if true, all assertion errors inside the captureBlock will be logged.
+     *                              This can cause multiple errors to be logged if the matcher is evaluated
+     *                              multiple times.


This doesn't match the actual parameter of the method?

hho · 2025-12-15T13:54:22Z

modules/mockk-dsl/src/commonMain/kotlin/io/mockk/API.kt

+     * @return the captured argument
+     */
+    inline fun <reified T : Any> withArg(
+        noinline assertionErrorLogger: (AssertionError) -> Unit = { e -> e.printStackTrace() },


This breaks compilation for uses of withArg with a variable (instead of a direct given block)...

@hho can you provide an example please?

Sure. Here's what broke for me (only anonymized):

My capture block is extracted into the someDetailValidator variable. This worked fine in 1.14.6, but breaks in 1.14.7, because now I need to either pass an assertionErrorLogger or spell out the captureBlock parameter name.

I think the problem might've be avoidable, if the new signatures (with the new assertionErrorLogger parameter) would've just been added as separate methods, without a default for the logger?

hho · 2025-12-15T13:55:43Z

Maybe I don't completely understand what's going on here, but I believe such a breaking API change shouldn't be hidden in a patch release?

add configurable logging to withArg

e62d62e

update arg from boolean flag to function

1bb4349

update api definitions

eab8320

Raibaz merged commit 00ea4a8 into mockk:master Oct 29, 2025
22 checks passed

hho reviewed Dec 15, 2025

View reviewed changes

Uh oh!

Add configurable logging to withArg & withNullableArg #1441

Add configurable logging to withArg & withNullableArg #1441

Conversation

OsaSoft commented Oct 23, 2025

Problem decription:

Proposed solution:

Reproduction:

Possible future improvements:

Uh oh!

Raibaz commented Oct 23, 2025

Uh oh!

OsaSoft commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Raibaz commented Oct 24, 2025

Uh oh!

OsaSoft commented Oct 27, 2025

Uh oh!

OsaSoft commented Oct 27, 2025

Uh oh!

Raibaz commented Oct 28, 2025

Uh oh!

OsaSoft commented Oct 28, 2025

Uh oh!

Uh oh!

Raibaz commented Oct 29, 2025

Uh oh!

hho Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

hho Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OsaSoft Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

hho Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

hho commented Dec 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

OsaSoft commented Oct 24, 2025 •

edited

Loading

hho Dec 15, 2025 •

edited

Loading