Skip to content

Issues with interaction of meck with cover + eunit #203

Open

Description

@sjplatt

Mocking a covered module multiple times in unit tests cause the tests to fail.

Reproduction Steps

Sample code:

-module(my_repro_module).
-include_lib("eunit/include/eunit.hrl").

first_test() ->
    cover:compile_beam_directory("PATH_TO_DIRECTORY which contains test_module"),
    meck:new(test_module),
    meck:expect(test_module, read, fun(_, _) -> ok end),
    meck:unload(test_module).

second_test() ->
    meck:new(test_module),
    meck:expect(test_module, read, fun(_, _) -> ok end),
    meck:unload(test_module).

Result:

eunit:test(my_repro_module, [verbose]).

module 'my_repro_module'
  my_repro_module: first_test...[0.365 s] ok
  my_repro_module: second_test...*skipped*
undefined
*unexpected termination of test process*
::{terminated,[{io,format,
                   [<0.5771.0>,
                    "WARNING: Deleting data for module ~w imported from~n~tp~n",
                    [test_module,
                     ["PATH_...test_module.1438497.coverdata",
                      "PATH_...test_module_meck_original.1438497.coverdata"]]],
                   []},
               {cover,remove_imported,2,[{file,"cover.erl"},{line,1362}]},
               {cover,fix_state_and_result,3,[{file,"cover.erl"},{line,1496}]},
               {cover,main_process_loop,1,[{file,"cover.erl"},{line,664}]}]}

=======================================================

Observed behavior

If I comment out meck:unload(test_module). in second_test(), the test will pass.
If I add c:l(test_module) at the start of second_test(), the test will pass.

Versions

  • Meck version: 0.8.12
  • Erlang version: OTP 21

[Add any other environment or test framework related information here, such as
what OS you are using and how Erlang is installed]

Activity

  1. eproxus commented on Jan 24, 2019

    @eproxus
    Owner

    @sjplatt Finally had some time to look at this issue. Wow, what a ride 😅

    So, I can trigger some weird behavior which I think is the same, just not as visible (I think Meck exposes this by having another linked process running on the side, which shows some more stack traces and exceptions).

    The repro_test module below can trigger a killed error. It is placed in an example project under ./test. In ./test/modules/test_module.erl I have an example test module.

    -module(repro_test).
    -include_lib("eunit/include/eunit.hrl").
    
    first_test() ->
        erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test")),
        erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test")).

    Running this I get:

    $ rebar3 eunit
    ===> Verifying dependencies...
    ===> Compiling meck
    ===> Compiling meck_repro
    ===> Performing EUnit tests...
    [{ok,repro_test},{ok,test_module}]
    
    Pending:
      undefined
        %% Related process exited with reason: killed
    
    
    Finished in ? seconds
    2 tests, 0 failures, 2 cancelled
    ===> Error running tests
    

    (Notice how EUnit can't even 😛 )

    If I change the test case lines to point to the directory where only the example module is:

        erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test/modules")),
        erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test/modules")).

    It works:

    $ rebar3 eunit
    ===> Verifying dependencies...
    ===> Compiling meck
    ===> Compiling meck_repro
    ===> Performing EUnit tests...
    []
    []
    .
    
    Top 1 slowest tests (0.004 seconds, 17.4% of total time):
      repro_test:first_test/0: module 'repro_test'
        0.004 seconds
    
    Finished in 0.023 seconds
    1 tests, 0 failures
    

    What's happening is that you're recompiling the test case module itself with cover, while running inside the test case. This means Erlang is killing the process running the test case (because the module is recompiled twice, removing the oldest running version and killing the process inside that code). What you see from Meck and EUnit is just the linked errors eventually cascading through the other linked processes.

    Thus, you can fix your test problem by moving the test_module into it's own directory that doesn't contain the test code itself, and only cover compile that.

    I'm closing this as invalid for now, but please re-open if this is not the case. 😉

  2. sjplatt commented on Jan 24, 2019

    @sjplatt
    Author

    Hi,

    Thanks for digging into this. Unfortunately, this doesn't seem to be the issue I am running into.

    In my situation the two modules are in distinct directories:
    EX: ./test/a/test_module.erl and ./test/b/my_repro_module.erl

    They are compiled into two separate directories:
    EX: ./test/a/ebin/test_module.beam and ./test/b/ebin/my_repro_module.beam

    Then in the test we are only calling cover:compile_beam_directory() for the path ./test/a/ebin (as an example).

    (Sorry if this is vague, we have a lot of machinery/makefiles for compilation so it is difficult to get a super clean example).

  3. sjplatt commented on Jan 24, 2019

    @sjplatt
    Author

    @eproxus I don't think I can re-open the issue bc you closed it.

    I will also see if I can get a better/more concise repro although might be difficult.

  4. eproxus commented on Jan 24, 2019

    @eproxus
    Owner

    Are any of the modules used in processes that are linked to the test case?

  5. sjplatt commented on Jan 24, 2019

    @sjplatt
    Author

    Are any of the modules used in processes that are linked to the test case?

    As far as I know no (the test case also doesn't do anything, it simply mecks and unmecks a file).

    One other way to potentially repro this (this is how we originally found it, the above example was just a simpler way to repro) would be to start a node, cover compile everything, and then run the above test (without the cover:compile_beam_directory line).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions