I'm still working on the tests.

This is a related issue to a test failure christhekeele/erlex#6

first of all thanks you are working in it. I think it's important to say in documentation/readme or somewhere that it use _build folder for cache. So when developers use common approach with cache they cache deps and _build folder before dialyzer step/job. But common practice is to use cache versioning based on mix lock file or something like that. In this case it would mean that dialyzer cache will be stored only in case deps are changed. So not big benefits for incremental feature. I think this is important to explain that developers have to check properly their cache logic what they have. I would suggest to introduce new cache which will directly cache folder where new file dialyzer files are stored. Because this new cache has to be updated everytime with some logic. For example you don't want to store new dialyzer's files from some branch same way as you store main branch to avoid usage branch's cache in main. But you want to store branch files due to multiple runnings of ci/cd if it's common for your development flow to have multiple runs of ci/cd per branch. So for example in GitHub Action you can use branch name in cache name with fallback from main branch, for runs from main only cache with main cache. Also I would suggest to use ci/cd runs number which should be incremental every run, because thanks to that it should store cache everytime when it runs, not only when key is changed. But this has to be checked because last time (a few months ago) when I checked GH Action Cache app it had bug/removed feature for flagging cache to be stored everytime when it runs, because we can assume that every run (often) should contains some code change. Is it know exact folder path for cached files?

I would suggest to introduce new cache which will directly cache folder where new file dialyzer files are stored. Because this new cache has to be updated everytime with some logic. For example you don't want to store new dialyzer's files from some branch same way as you store main branch to avoid usage branch's cache in main. But you want to store branch files due to multiple runnings of ci/cd if it's common for your development flow to have multiple runs of ci/cd per branch. So for example in GitHub Action you can use branch name in cache name with fallback from main branch

I was thinking about this. Is you thought to just document this behavior or to add some mechanism to the library to better facilitate this caching approach?

Also I would suggest to use ci/cd runs number which should be incremental every run

this is on my todo list.

I think adding good documentation is base. Right now I don't see how library can give some tool or approach for it. Because it depends on own CI/CD configurations how people use cache. Something can be handled by library only in case library is able to set how to store these incremental files. Thanks library can partly allow to configure it for developers or use some common cache so developers will not have to configure new cache or something.

@jeremyjh do you have any pointers for getting the tests to not time out in CI? They all work for me locally.

@jeremyjh do you have any pointers for getting the tests to not time out in CI? They all work for me locally.

@Ch4s3 Thanks for working on this!

The problem must be in this branch; I reran master - it passed and test times haven't changed. #578

I don't immediately see the issue; there are tests timing out that aren't really doing much. I think I'd start by commenting all the new tests; if it passes then uncomment just the new output test.

from documentation:

--incremental - The analysis starts from an existing incremental PLT, or builds one from scratch if one does not exist, and runs the minimal amount of additional analysis to report all issues in the given set of apps. Notably, incremental PLT files are not compatible with "classic" PLT files, and vice versa. The initial incremental PLT will be updated unless an alternative output incremental PLT is given.

it means that file location is same but content is different. I tried your PR and when I use plt_local_path and plt_core_path it use same paths so basically no changes for CI/CD behavior I guess. Or do you have different experience? I think only important part is to remove them before running incremental, because these files are not compatibile with each other.
BTW: I can not confirm that it makes some speed up, it felt same with flag or without flag, with true or false in config.

BTW: You wrote you fixed Fix critical API usage bug in Dialyzer.Runner are you sure you put that change in this module in this PR?

BTW: You wrote you fixed Fix critical API usage bug in Dialyzer.Runner are you sure you put that change in this module in this PR?

I think that’s leftover from an incremental step in some commits I squashed. I need to rewrite the PR description.

it means that file location is same but content is different. I tried your PR and when I use plt_local_path and plt_core_path it use same paths so basically no changes for CI/CD behavior I guess. Or do you have different experience? I think only important part is to remove them before running incremental, because these files are not compatibile with each other.

Let me double check that I didn't have some environment config for erlang causing an issue here

BTW: I can not confirm that it makes some speed up, it felt same with flag or without flag, with true or false in config.

I can definitely confirm that on my production code base there's a speedup the measurement I included is from that app. I should clarify it is NOT faster to general the PLT and initial run, it is only faster for subsequent runs.

@oliver-kriska once I have tests working, I'm going to test this in CI on my app and gather timing info as well as double checking the path information and I'll update docs here.

I think I found the main problem

I'm still working on this, I just had to take a break do to work/life business.

I think the issue here was trying to test the system halt call on older OTP versions. Since the other code path that calls a halt isn't tested, I'm removing the test that verifies a halt on OTP < 26. Hopefully that's an acceptable tradeoff.

I'm fiddling with how to correctly use this in CI. It seems like just mix dialyzer --incremental plus any formatting should be run without a discrete --plt step. It doesn't seem like OTP generates an incremental plt if you use both flags.

Yes, with the new incremental mode you don't need a separate PLT step - an incremental run is both generating a new PLT and checking your code - internally it was always the same operation, this just explicitly combines them.

@michalmuskala thanks for weighing in. Am I understanding correctly that incremental mode doesn't build the core language PLT so you need to also have that if you want to avoid getting errors from missing language functions?

Or should we be using --apps and --warning-apps to deal with this?

Am I understanding correctly that incremental mode doesn't build the core language PLT so you need to also have that if you want to avoid getting errors from missing language functions?
Or should we be using --apps and --warning-apps to deal with this?

Incremental mode analyses whatever modules you give it, including OTP modules (they don't get any special treatment). That means that you do want to use --apps and --warning-apps to make sure everything is included. For code that you don't own (OTP, libraries made by others), I'd suggest using --apps, so Dialyzer knows about them and can comment on their usage, but won't complain about them directly. Then use --warning-apps for code you actually want issues to be raised for.

Sadly, it's not quite as clean and orthogonal as it sounds, because if there is some sort of discrepancy in your --apps, you won't get a direct error reported for it, but your usages of that code from your --warning-apps might end up generating warnings due to your calls into broken code. As such, it's worth having that in the back of your mind when investigating Dialyzer warnings.

For the mental model for how to use --incremental mode, try to forget your existing knowledge of how Dialyzer works, and instead just list all the code you care about as either --apps or --warnings-apps (including OTP and libraries). You call the same command each time, asking Dialyzer to analyse your--apps and --warnings-apps. The incremental command is given a path to a PLT file, and it will worry about building, changing and updating the cache (PLT) file to minimise the work it needs to use between runs. You don't need to do anything explicitly to manage that lifecycle apart from making sure you point it to a consistent PLT file (and as someone pointed out above, perhaps you want to point to a different one per build configuration/branch if you're often moving between them).

Hi there 👋 I made Dialyzer's incremental mode. I tried to add a bit of context where I could on this PR. I hope that helps.

Am I understanding correctly that incremental mode doesn't build the core language PLT so you need to also have that if you want to avoid getting errors from missing language functions?
Or should we be using --apps and --warning-apps to deal with this?

Incremental mode analyses whatever modules you give it, including OTP modules (they don't get any special treatment). That means that you do want to use --apps and --warning-apps to make sure everything is included. For code that you don't own (OTP, libraries made by others), I'd suggest using --apps, so Dialyzer knows about them and can comment on their usage, but won't complain about them directly. Then use --warning-apps for code you actually want issues to be raised for.

This is what I'm figured out after some trial and error and the latest changes are an attempt to make passing those through as straightforward as possible. Thanks for further clarification.

Sadly, it's not quite as clean and orthogonal as it sounds, because if there is some sort of discrepancy in your --apps, you won't get a direct error reported for it, but your usages of that code from your --warning-apps might end up generating warnings due to your calls into broken code. As such, it's worth having that in the back of your mind when investigating Dialyzer warnings.

Makes sense.

For the mental model for how to use --incremental mode, try to forget your existing knowledge of how Dialyzer works, and instead just list all the code you care about as either --apps or --warnings-apps (including OTP and libraries). You call the same command each time, asking Dialyzer to analyse your--apps and --warnings-apps. The incremental command is given a path to a PLT file, and it will worry about building, changing and updating the cache (PLT) file to minimise the work it needs to use between runs. You don't need to do anything explicitly to manage that lifecycle apart from making sure you point it to a consistent PLT file (and as someone pointed out above, perhaps you want to point to a different one per build configuration/branch if you're often moving between them).

Awesome, this is such helpful context! I think I'm pretty close on this, I have the branch working against a production repo and giving me meaningful output. Thanks again, and also for your work on incremental mode.

@TD5 could you weigh in again? I'm now just passing --apps and --warning-apps but not the files in incremental mode and I'm not seeing errors for code with trivial dialyzer errors picked up by classic mode.

Based on this test setup from OTP, and the original commit message I'm assuming --apps and --warning-apps have to overlap. I'm not sure how I missed this before.

Based on this test setup from OTP, and the original commit message I'm assuming --apps and --warning-apps have to overlap. I'm not sure how I missed this before.

That sounds plausible. Honestly, I wrote the code long enough ago that I can't remember the motivation there.

That sounds plausible. Honestly, I wrote the code long enough ago that I can't remember the motivation there.

I know the feeling! Thanks for the input.

@jeremyjh I think this is reviewable now. Below is a sketch of how to read through the changes or as best I could reason about how to approach it.

1) Start with the intent and surface changes

• Changelog: CHANGELOG.md — confirms incremental mode feature addition and headline behavior.
• README incremental section: README.md — updated guidance for apps/warning_apps, incremental usage, caching, and macOS ulimit tip.

2) Core entry points (behavioral changes)

• Mix task front door: lib/mix/tasks/dialyzer.ex
  • CLI parsing and incremental gating: R226.
  • Selection assembly and resolver wiring: R252.
  • Argument assembly for Dialyzer: R319 and dialyzer_args/4 around R488.
• Dialyzer runner: lib/dialyxir/dialyzer.ex
  • Incremental analysis toggle and formatter parsing: R12.

3) Centralized app/warning_app resolution

• New resolver module: lib/dialyxir/app_selection.ex
  • Single place that merges CLI + config, expands flags, normalizes atom/string/charlist, filters non-project warning_apps with warnings, and merges warning_apps into apps.
  • Side-effect scope: only emits warnings via Dialyxir.Output.
• Project-level helpers: lib/dialyxir/project.ex
  • resolve_apps/1 and resolve_warning_apps/1 unify config resolution; see R307 and R337.
  • fallback_list/3 and resolve_list_value/2 handle list configs and backward compatibility; see R354.
  • expand_transitive_apps/1 extracts :transitive expansion to avoid duplication; see R375.
  • Semantics: apps expands :transitive (deps + project) but still requires explicit OTP apps; warning_apps lists are project-only, merged into apps later; nil → empty list for backward compatibility.

4) Incremental PLT handling

• PLT file handling: lib/dialyxir/plt.ex
  • Confirm file enumeration and incremental PLT handling are compatible with incremental analysis (no regressions in classic PLTs): see R244 for classic vs incremental PLT file handling.

5) Docs and CI caching

• CI examples: docs/github_actions.md, docs/gitlab_ci.md, docs/circleci.md
  • Clarified cache keys (OTP/Elixir/mix.lock, optional branch keys) and guidance that incremental artifacts live in priv/plts.
• MacOS ulimit note: README.md
  • EMFILE avoidance tip for large codebases during MD5 hashing.

6) Tests (behavioural coverage)

• Resolver unit tests: test/dialyxir/app_selection_test.exs
  • Covers incremental gating, CLI overrides, filtering, and merging.
• Project helpers: test/dialyxir/project_test.exs
  • Expanded scenarios for apps/warning_apps resolution and :transitive expectations.
• Task integration: test/mix/tasks/dialyzer_test.exs
  • Ensures flags flow into dialyzer args correctly, including incremental/app-mode paths.
• Dialyzer runner: test/dialyxir/dialyzer_test.exs
  • Confirms runner behavior in incremental contexts.

7) Fixtures and samples

• Incremental/app-mode fixtures: under test/fixtures/* (e.g., apps_transitive, warning_apps_project, incremental_*)
  • Check they mirror the intended semantics (OTP apps explicit, warning_apps limited to project).

8) Thing to look out for

• CLI vs config precedence in AppSelection: CLI should win; warning_apps filtered to project apps only; apps expanded for :transitive.
• Incremental vs classic: incremental should skip classic PLT build when --incremental; classic flow unchanged when not incremental.
• Caching guidance: priv/plts is the cache root for both classic and incremental artifacts; keys tied to OTP/Elixir/mix.lock (optional branch suffix).
• ulimit/EMFILE: documented; no code changes, just awareness for large repos.

@jeremyjh there are some sort of tricky tradeoffs to consider here. The existing code for resolving dependencies is built around the way classic PLTs work and assumes that things like :erts, :elixir, :kernel, and so on are already in the PLT. This means that trying to add those items in automatically through the old code causes runtime warnings about loading deps like the following:

Error loading sasl, dependency list may be incomplete.
 {~c"no such file or directory", ~c"sasl.app"}

I can solve that by resolving deps in a less efficient way in a new code path, or by requiring the end user to declare them explicitly in apps: [...]. Any other approach would have to cut into a lot of older code in a way that feels dicey in terms of broad ecosystem support (to me at least).

Update

I think this is ready for re-review. My compromise here is that users have to declare core language dependencies like [:erts, :kernel, :stdlib, :elixir, :logger]. I don't love it but trying to make it work automatically with existing dialixyr code using an existing flag would require a lot of changes to dependency resolution. You can see it in use here

@Ch4s3 is there an issue with always adding [:erts, :kernel, :stdlib, :crypto, :elixir] to the app list when :app_tree or :apps_direct are used? If a user doesn't want some of those, they could just provide a complete list of the apps they want instead of using either of those methods. I'm not sure that is an edge case that even exists.

@Ch4s3 is there an issue with always adding [:erts, :kernel, :stdlib, :crypto, :elixir] to the app list when :app_tree or :apps_direct are used? If a user doesn't want some of those, they could just provide a complete list of the apps they want instead of using either of those methods. I'm not sure that is an edge case that even exists.

@jeremyjh if the hard coded value is good with you I can absolutely add it to app_tree. I was just trying to avoid hard coding if I could.

@jeremyjh if the hard coded value is good with you I can absolutely add it to app_tree. I was just trying to avoid hard coding if I could.

It is already hard coded, those are the default apps that go into the Erlang/Elixir PLTs and are included in the project one.

@jeremyjh I assumed since it was deprecated that I should do something different. I'll add it into my code path.

If you're you ok using the new apps_project flag for the warning apps then it's read for review.

Hey folks, how is this work going? I'm excited to have this ready to use and try it out! Is there something we can do to make that happen?

@Wigny I'm just waiting for now. I've released an incremental only dialyzer wrapper at https://github.com/Ch4s3/assay that's experimental if you want to try that while you wait for this PR to be merged.

I did some testing in a project I'm working on, and got some issues with the new configs.

When I tried to use :app_tree I got an error because of sasl:

:dialyzer.run error: No such file, directory or application: "sasl"

When I tried to use :apps_direct I got an error because of credo:

:dialyzer.run error: No such file, directory or application: "credo"

Dialyzer is running with prod env, and credo is available only on test env.

I did some testing in a project I'm working on, and got some issues with the new configs.

When I tried to use :app_tree I got an error because of sasl:

:dialyzer.run error: No such file, directory or application: "sasl"

When I tried to use :apps_direct I got an error because of credo:

:dialyzer.run error: No such file, directory or application: "credo"

Dialyzer is running with prod env, and credo is available only on test env.

Interesting, I must not have considered running this with MIX_ENV=prod. Is that how you were running dialyzer before?

Yeah, in this project we only run dialyzer with MIX_ENV=prod, and it works with just this setting dialyzer: [plt_add_apps: [:mix, :phoenix_storybook]]

Yeah, in this project we only run dialyzer with MIX_ENV=prod, and it works with just this setting dialyzer: [plt_add_apps: [:mix, :phoenix_storybook]]

Gotcha, I hadn't considered running in prod, I'm a little surprised it ever worked. This is the line that causes the issue. I'm not sure there's a way to meet @jeremyjh 's desire to make the helpers "just work" and to have incremental mode work in MIX_ENV=prod. Incremental mode in dialyzer doesn't have plt_apps, and I use these helpers like app_tree to sort of fake it.

If you can suggest a clean fix, I'm open.

plt_ignore_apps not respected in incremental mode

I tested this PR branch on our production codebase (Elixir 1.19.5, OTP 28) and incremental mode works — great work! The speedup is significant:

However, I found an issue with plt_ignore_apps being silently ignored in incremental mode.

Our config:

dialyzer: [
  plt_ignore_apps: [:gpt3_tokenizer, :memoize],
  # ...
  incremental: [
    enabled: true,
    apps: [:elixir, :logger, :crypto, :public_key, :ex_unit, :credo, :tidewave] ++ [:app_tree],
    warning_apps: [:enaia]
  ]
]

What happens:

In classic mode, plt_ignore_apps excludes gpt3_tokenizer from the PLT (it caused an infinite loop during classic PLT building). Since dialyzer doesn't know about the module, calls to Gpt3Tokenizer.token_count/1 produce a "function does not exist" warning, which we suppress via a regex filter in our ignore file.

In incremental mode with :app_tree, the new AppSelection module resolves apps independently and does not subtract plt_ignore_apps. So gpt3_tokenizer gets included, dialyzer sees the function exists, no warning is generated, and our ignore filter becomes unused — which causes list_unused_filters: true to fail CI with exit code 1.

Looking at the code, lib/dialyxir/project.ex:91 applies plt_ignore_apps() in the classic path via cons_apps/0, but lib/dialyxir/app_selection.ex has no reference to plt_ignore_apps at all.

Suggested fix:

AppSelection should read plt_ignore_apps from config and subtract them from the resolved app list, consistent with the classic path. This would make migration from classic → incremental seamless for users who rely on plt_ignore_apps.

In our case the infinite loop turned out to be PLT-specific and didn't occur in incremental mode, but the behavioral inconsistency is still a problem for CI pipelines that use list_unused_filters: true.

fyi: Claude Code generated this comment

@oliver-kriska I didn't do this initially because it required a lot of file handling. I could possibly do it in the future but I'm waiting on @jeremyjh to take another look at the PR. This has been my approach in the mean time

@jeremyjh any chance we can move this forward? large dialyzer project are slow and while it s harder to justify dialyzer with type safety coming, it would be nice to still support this

@epinault I built this to work on a large umbrella and it's used on a few other large apps https://github.com/Ch4s3/assay

Is that a replacement then for dialyxir then?

@epinault it is for my use case. It's mostly a dumb wrapper around iplt generation, checking, and a formatter. Its a little rough, because I'm still interested in merging this PR to dialyxir. If you have any issues trying it, file an issue over there.