Performance opportunity for yarn 3 cache #325

belgattitude · 2021-09-03T09:39:04Z

Thanks for the new cache feature. Much easier.

After few weeks, I realized that while it supports yarn, there's some improvements that can be made.

Yarn 3 (probably yarn 2+ too) manages downloaded archives pretty well (.yarn/cache/*.zip) and invalidating on yarn.lock changes does not take that into account and I saw a lot of cache misses.

As an example I converted back to action-cache to illustrate and test.

I'm wondering if a similar approach could be done with setup-node ?

Updated example with action cache

Updated on Nov 23th, taking into account @merceyz comment.

Setup action

      # Get the yarn cache path.
      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
    
      - name: Restore yarn cache
        uses: actions/cache@v2
        id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
          key: yarn-cache-folder-${{ hashFiles('**/yarn.lock', '.yarnrc.yml') }}
          restore-keys: |
            yarn-cache-folder-

Testing a cache hit after adding a dependency

With setup node cache, as the yarn.lock have changed all packages would be fetched again (1m 28s)
rather than (1s 124ms). Environmentally friendlier 🌳

yarn install --immutable
➤ YN0000: ┌ Fetch step
  ➤ YN0013: │ 1719 packages were already cached, one had to be fetched (superjson@npm:1.7.5)
➤ YN0000: └ Completed in 1s 124ms

Cache Size: ~127 MB (133261279 B)
Cache saved successfully
Cache saved with key: yarn-cache-folder-os-Linux-node--f118ea4bee07eada9df36ad2e83fd6febcbaf06b5b8962689c7650659e872ad3

PS: Key points

~Example with action cache~ (old version, before @merceyz improvements)

      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
      - name: Restore yarn cache
        uses: actions/cache@v2
        id: yarn-cache 
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}       
          key: yarn-cache-folder-os-${{ runner.os }}-node-${{ env.node-version }}-${{ hashFiles('**/yarn.lock', '.yarnrc.yml') }}      
          restore-keys: |
            yarn-cache-folder-os-${{ runner.os }}-node-${{ env.node-version }}-
            yarn-cache-folder-os-${{ runner.os }}-

Note

Here's a gist with an optimized install example: https://gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a

The text was updated successfully, but these errors were encountered:

merceyz · 2021-09-10T13:25:14Z

It can be even more efficient by not including the OS and Node version in the cache key as well - #272 (comment)

dmitry-shibanov · 2021-11-12T15:08:22Z

Hello @belgattitude. Thank you for your feature request. Could you please describe it little bit. Do you want to add restore-keys to the action to take previous cache or you also want to add .yarn/cache/*.zip to cached directories ?

dmitry-shibanov · 2021-11-22T15:22:44Z

Hello @belgattitude, just a gentle ping.

belgattitude · 2021-11-22T16:09:07Z

Let me check something in a couple of days. I'll be back asap.

belgattitude · 2021-11-23T14:08:06Z

@dmitry-shibanov first of all thanks for considering this issue. I've edited the P/R desc with latest comments from @merceyz

Do you want to add restore-keys to the action to take previous cache

Yes that's the idea, that way we take advantage of the built-in yarn 2+/3+ cache management. Yarn will prune old packages and only fetch new ones keeping the packages zipped into cacheFolder in sync with package.json (works with workspace/monorepo too).

Note that if multiple actions are run in parallel, only the first one will be able to reserve the cache, others will print a warning after:

> Post restore yarn cache
   Unable to reserve cache with key yarn-cache-folder- 
   cb79cb8e7c0fdd859237ac516da53bc2fdafceb3230fd68444aa4f55c9238980, another job may be creating this cache.

As far as I tested it does not create problems and can be safely ignored..

you also want to add .yarn/cache/*.zip to cached directories

Actually this seems to be what matters most. (AFAIK setup/node try to cache only the node_modules folders, right ?)

There's some kind of guarantee that yarn cache is portable across OS and node versions as @merceyz pointed out, while node_modules might not (native binaries, postinstall tricks).

As the cache folder is configurable (through yarnrc.yml or YARN_CACHE_FOLDER env), a good way to read it

      # Get the yarn cache path.
      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn config get cacheFolder)"

The restore key could be a constant:

      - name: Restore yarn cache
        uses: actions/cache@v2
        id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
          key: yarn-cache-folder-${{ hashFiles('**/yarn.lock', '.yarnrc.yml') }}
          restore-keys: |
            yarn-cache-folder-

Interestingly pnpm has a benchmark page with some insights about the different modes (cache+node_modules, cache only)

Looks having both node_modules + yarn cache will be faster (excluding action/cache restore/generate...). I guess even more as yarn won't probably have to re-run the link phase (generally slow with native binaries: esbuild, swc, sharp, prisma...). But with caching node_modules I'm not sure of the portability though (it depends also of setup/node). Note also that node_modules folder is only used in nmLinker: node-modules or nmLinker: pnp modes (otherwise it's assumed to be yarn pnp)

I would say a safe way would be to just save 'yarn cache folder' and not node_modules.

Let me know if it answer your questions.

Have a great day

PS: @merceyz thanks for your comment about portability, I was wondering if the newly introduced supportedtArchitecture change something ?

devgioele · 2022-05-05T10:28:10Z

Do you recommend using actions/setup-node to cache node_modules and to use actions/cache to cache "the yarn cache"?
Caching both would allow us to skip the linking step too, right? I don't see why it cannot be done.

panticmilos · 2022-05-09T11:10:56Z

Hi @devgioele,

Caching node_modules is not possible using setup-node. This action is only caching on a global level. If you would like to cache node_modules, please use actions/cache.

Cheers

mklueh · 2022-07-10T17:48:29Z

Deleting yarn.lock and running yarn install helped, but in my case with yarn caching enabled

See: actions/setup-node#325

nickserv · 2022-09-24T07:24:00Z

I think this is a duplicate of #328, which is similar but not specific to yarn

belgattitude · 2022-10-09T08:17:09Z

Just to share, here's my updated composite action for

yarn 3+ with node-modules linker: https://gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a
pnpm 7+: https://gist.github.com/belgattitude/838b2eba30c324f1f0033a797bab2e31

Simplify dependency caching for Yarn 3+ with Zero Installs. Inspired by actions/setup-node#325 and https://gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a.

Simplify dependency caching for Yarn 3+ with Zero Installs. Inspired by actions/setup-node#325 and gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a.

marcofugaro · 2022-11-08T12:24:47Z

Would be good to have support for Yarn 3 caching

merceyz · 2023-01-11T09:13:40Z

It can be even more efficient by not including the OS and Node version in the cache key as well - #272 (comment)

This got a bit more complicated in Yarn v3.1.0 with the introduction of supportedArchitectures, if current is included in any of the options (the default) then the amount of packages fetched by Yarn can be different based on the host. For example by default fsevents will be fetched on a macOS runner but not Ubuntu.

nijikon · 2023-04-05T21:34:39Z

Is there any plan to add this anytime soon?

marko-zivic-93 · 2023-04-06T14:24:07Z

Hello @nijikon,
We have great news for you. We will be taking this feature and working on it in the upcoming quarter, so you can expect this to be available soon... 😃

dsame · 2023-04-21T19:32:28Z

Hello @belgattitude, please confirm the problem we want to solve is eliminating the whole cache reset on any lock file invalidating.

I mean that if only one (of 2) dependency changed akv-demo/setup-node-test@44f68b4#diff-51e4f558fae534656963876761c95b83b6ef5da5103c4adef6768219ed76c2deL19

the whole cache is reset https://github.com/akv-demo/setup-node-test/actions/runs/4767849996/jobs/8476525793#step:3:82

and all (of 2) dependencies must be fetched https://github.com/akv-demo/setup-node-test/actions/runs/4767849996/jobs/8476525793#step:4:28

Is your suggestion to solve the described problem with restore-keys-like behaviour?

belgattitude · 2023-05-01T10:12:40Z

Hey @dsame

please confirm the problem we want to solve is eliminating the whole cache reset on any lock file invalidating.

Yes.

But https://github.com/akv-demo/setup-node-test/actions/runs/4767849996/jobs/8476525793#step:4:28 does not seem to work ? Am I wrong ?

belgattitude · 2023-05-01T10:16:14Z

BTW you can have a look to some ideas

CI: speed up CI time by improving yarn install and caches (iteration 1 / >30%) strapi/strapi#16581 (I've commented in the code see the changed files)
https://github.com/belgattitude/compare-package-managers
https://github.com/belgattitude/compare-package-managers/tree/main/.github/actions (see yarn-nm-install and pnpm-install)
https://gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a ( a yarn install example)

That said in my experience couldn't find one solution for every use-case. Comments in there strapi/strapi#16581

TLDR:

Ideally with yarn, the following steps/files/folders can be cached

(1) required: yarn cache .yarn/cache.zip (reusable on lock changes)
(2) optional: install_state (invalided on lock/os/node-version change)
(3) optional: node_modules/** (invalided on lock/os/node-version change)

For 2 and 3 it's optional and should be measured on a specific repo. Cause saving node_modules is heavy (see post restore).

For a concrete example see also a small sized monorepo but with a lot of deps: https://github.com/belgattitude/nextjs-monorepo-example. It seems doable to get an install under 20s

There's more to say about supportedArchitectures (yarn 4), prisma postinstall tricks...

dsame · 2023-05-03T18:54:54Z

Hey @dsame

please confirm the problem we want to solve is eliminating the whole cache reset on any lock file invalidating.

Yes.

But https://github.com/akv-demo/setup-node-test/actions/runs/4767849996/jobs/8476525793#step:4:28 does not seem to work ? Am I wrong ?

Yes, it is a proof the problem exists build - pay attention it uses v3(i.e. current) action, this way i confirmed we really need the feature.

dsame · 2023-05-03T19:19:22Z

@belgattitude with the close look i started to doubt about the feature.

First of all yarn has 2 modes - local and global cache depending on enableGlobalCache option in .yarnrc.yml

With the global cache yarn does not purge the obsolete dependencies and we can not restore the previous cache in order to avoid the endless grow

proof: you can see different versions of lodash

with the local cache yarn does purge the obsolete dependencies BUT, in this case you should not use actions cache at all - you should keep .yarn in your repo instead as [it is recommended in the yarn docs]

(https://yarnpkg.com/features/offline-cache).

If we even apply the feature only for local caches we are getting into the danger of clashing .yarn from cache with .yarn from repo. It is still possible to workaround this but it looks to be useful only for marginal case: local cache + .yarn not included to the repo

Can you please provide more justification to use caching instead of adding .yarn to the repo?

Also i have a question about node modules? Do we really need it to cache it with yarn? Since PnP has been introduced yarn does not have it at all, but have .pnp.cjs instead, which is assumed to be added to the repo as well.

belgattitude · 2023-05-04T17:01:33Z

From my phone, so writing is minimal.

Override global cache with YARN_ENABLE_GLOBAL_CACHE=false. That gives you an uniform way to do it for ci. The yarn cache will be stored in .yarn/cache by default but you can get it from config as well.

Just cache .yarn/cache not .yarn

Better to avoid caching node modules with this approach. It generally does not bring speed up due to action/cache extra work. That also solve the pnp support.

dsame · 2023-05-05T06:17:49Z

Hello @belgattitude - one more clarification: is there any reason to do not add .yarn/cache to the repository as it is recommended in the yarn docs and how the yarn team do for their own repo ?

Having yarn/cache saves the Git storage and gives better performance than caching, but i suspect there may be some specific reason to use cache instead of adding to the repo?

belgattitude · 2023-05-05T08:13:08Z

The initial offline mode idea isn't probably used. Storing large binaries in git and their mutation is probably not a good idea. I guess after few updates you'll end up with a git repo of 10's of Gb.
Git isn't a cache for large binaries. Rewritting history to purge obsolete entries isn't easy.

That said I can see why they explored this. Yarn default is to not commit the cache (doc is confusing).

But it's true setup/node action cache feature shouldn't be enabled by the user if he's using full offline.

dsame · 2023-05-05T18:27:56Z

Thank you @belgattitude for your opinion, now i agree it is worth to continue with the feature in order to avoid huge repo.

Can you please take a look at this Proof of concept draft PR and tell you opinion about does it make a sense to include node version and step id into the cache key keeping in mind having many huge cache will hit storage limit soon

https://github.com/actions/setup-node/pull/744/files#diff-55f15e2366942ad15f71a41ac983f8ce9a9882b28b7fd9082f3a26c799783064R42

dsame · 2023-05-05T18:56:01Z

@belgattitude can you please also confirm the following :

We have to use 3rd parameter of restoreCache in case if all of 3 conditions met:

The 3rd parameter of restoreCache has to be set to not null in case if all of 3 conditions met:

package manager is yarn
enableGlobalCache is omitted or false
there's no .yarn/cache directory in the working dir

belgattitude · 2023-05-06T13:50:01Z

@dsame

only from v2 (v1 behaviour should fallback to current way of doing)
(mmmh). on CI this should be overriden to enableGlobalCache: false through env. Don't run with true on CI (see previous comment in Performance opportunity for yarn 3 cache #325 (comment))
yes

I have looked into your PR. stepId not necessary in my opinion. But there's something missing to ensure yarn 4. Not a lot of time the next days... I'll figure out a way to share info asap

belgattitude · 2023-05-06T13:50:54Z

In the meantime I've tried to answers few questions in

Might worth to have a quick glance

dsame · 2023-05-09T07:04:34Z

The next iteration of PoC PR showed the detection of all the conditions implies the significant changes in code base and makes it unreasonable complicated.

@belgattitude can you please take a look at this issue - does not it seem the goal of reusing the existing cache can be achieved with resolving that issue in the more straightforward and common way?

I looked through the links you've sent,

according to the key-prefix idea, it is clear but you've provided a custom solution for the specific build while we are trying to provide universal solution for the whole action - this is why it become complicated
according to including .yarnec.yaml in hash calculation, it is the overkill and cause unnecessary cache refreshes
according to node_modules caching it brings performance gain only for cases of some binaries have to be rebuilt from sources and it is not considered now to be effective enough for most users - downloading and unpacking lot of files can be event slower than npm install ... if the dependencies are already cached.

Summary: can you please confirm or deny that key-prefix input added to the setup-node cache can boost the yarn3 performance and achieve the same goal as auto-generating the same key-prefix if the conditions are met?

belgattitude · 2023-05-10T17:29:52Z

Sorry I couldn't read properly but some quick toughts:

an you please take a look at this actions/setup-go#358 - does not it seem the goal of reusing the existing cache can be achieved with resolving that issue in the more straightforward and common way?

I would agree that params helps.

ccording to including .yarnec.yaml in hash calculation, it is the overkill and cause unnecessary cache refreshes

yarnrc.yml isn't something that is mutated often, so I wouldn't worry about it to invalidate things often.

Not accounting for it looks fragile in my impression, but I'm not 100% sure. I would let the benefit to doubt and include it in the hash (ie someone changes an advanced param...)

What do you mean by overkill ? is the hash call slow ?

according to node_modules caching it brings performance gain only for cases of some binaries have to be rebuilt from sources and it is not considered now to be effective enough for most users - downloading and unpacking lot of files can be event slower than npm install ... if the dependencies are already cached

For yarn 2+, I would not cache node_modules at all. (only few edges case would benefit from it, in this case they might use a totally different action than the one in setup/node)

Can you please confirm or deny that key-prefix input added to the setup-node cache can boost the yarn3 performance and achieve the same goal as auto-generating the same key-prefix if #325 (comment)?

The general idea seems good, but as always a deep testing is required. Do you expect me to do it ? In that case I'll need some time.

showed the detection of #325 (comment) implies the significant changes in code base and makes it unreasonable complicated.

Can you precise the meaning of ? "Unreasonable" => "in your opinion you wouldnt include the feature ?". Or you're still wondering ? I would understand of course.

But as a small personal note, it might have an very nice impact on global CI time... . Not sure about how many projects uses yarn 2+ and setup/node cache, but that's an interesting feature imho

dsame · 2023-05-10T17:35:31Z

Hello, @belgattitude

The general idea seems good, but as always a deep testing is required. Do you expect me to do it ?

Not at all, i've requested you opinion just to confirm we are no the same page and to get rid off some my hesitations. Now there's a plan an we are on the road. Thanks for the cooperation.

nijikon · 2023-07-18T17:47:15Z

@marko-zivic-93 back in April #325 (comment), you wrote that you will be having a look at this in the upcoming quarter. Were you talking about Q2 or Q3 of 2023?

dsame · 2023-07-19T07:23:13Z

Hello @nijikon the requested changes are merged into the main branch and are available since v3.7.0 release

nijikon · 2023-07-19T09:02:42Z

@dsame thanks. I will have a look if that helps my case.

dsame · 2023-07-25T09:08:50Z

Hello @nijikon, i am going to close this issue because the PRs are merged and due to inactivity, but please feel free to reopen it or create new issue in case if the problem still exists

nijikon · 2023-07-25T13:43:49Z

@dsame I think I'm good. My current problem is that it does not cache the node_modules folder, but I read that it's expected.

Bumps [@types/js-yaml](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/js-yaml) from 4.0.4 to 4.0.5. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/js-yaml) --- updated-dependencies: - dependency-name: "@types/js-yaml" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [actions/setup-node](https://github.com/actions/setup-node) | action | major | `v2.5.2` -> `v4.1.0` | --- ### Release Notes <details> <summary>actions/setup-node (actions/setup-node)</summary> ### [`v4.1.0`](https://github.com/actions/setup-node/releases/tag/v4.1.0) [Compare Source](actions/setup-node@v4.0.4...v4.1.0) #### What's Changed - Resolve High Security Alerts by upgrading Dependencies by [@aparnajyothi-y](https://github.com/aparnajyothi-y) in actions/setup-node#1132 - Upgrade IA Publish by [@Jcambass](https://github.com/Jcambass) in actions/setup-node#1134 - Revise `isGhes` logic by [@jww3](https://github.com/jww3) in actions/setup-node#1148 - Add architecture to cache key by [@pengx17](https://github.com/pengx17) in actions/setup-node#843 This addresses issues with caching by adding the architecture (arch) to the cache key, ensuring that cache keys are accurate to prevent conflicts. Note: This change may break previous cache keys as they will no longer be compatible with the new format. #### New Contributors - [@jww3](https://github.com/jww3) made their first contribution in actions/setup-node#1148 - [@pengx17](https://github.com/pengx17) made their first contribution in actions/setup-node#843 **Full Changelog**: actions/setup-node@v4...v4.1.0 ### [`v4.0.4`](https://github.com/actions/setup-node/releases/tag/v4.0.4) [Compare Source](actions/setup-node@v4.0.3...v4.0.4) #### What's Changed - Add workflow file for publishing releases to immutable action package by [@Jcambass](https://github.com/Jcambass) in actions/setup-node#1125 - Enhance Windows ARM64 Setup and Update micromatch Dependency by [@priyagupta108](https://github.com/priyagupta108) in actions/setup-node#1126 ##### Documentation changes: - Documentation update in the README file by [@suyashgaonkar](https://github.com/suyashgaonkar) in actions/setup-node#1106 - Correct invalid 'lts' version string reference by [@fulldecent](https://github.com/fulldecent) in actions/setup-node#1124 #### New Contributors - [@suyashgaonkar](https://github.com/suyashgaonkar) made their first contribution in actions/setup-node#1106 - [@priyagupta108](https://github.com/priyagupta108) made their first contribution in actions/setup-node#1126 - [@Jcambass](https://github.com/Jcambass) made their first contribution in actions/setup-node#1125 - [@fulldecent](https://github.com/fulldecent) made their first contribution in actions/setup-node#1124 **Full Changelog**: actions/setup-node@v4...v4.0.4 ### [`v4.0.3`](https://github.com/actions/setup-node/releases/tag/v4.0.3) [Compare Source](actions/setup-node@v4.0.2...v4.0.3) #### What's Changed ##### Bug fixes: - Fix macos latest check failures by [@HarithaVattikuti](https://github.com/HarithaVattikuti) in actions/setup-node#1041 ##### Documentation changes: - Documentation update to update default Node version to 20 by [@bengreeley](https://github.com/bengreeley) in actions/setup-node#949 ##### Dependency updates: - Bump undici from 5.26.5 to 5.28.3 by [@dependabot](https://github.com/dependabot) in actions/setup-node#965 - Bump braces from 3.0.2 to 3.0.3 and other dependency updates by [@dependabot](https://github.com/dependabot) in actions/setup-node#1087 #### New Contributors - [@bengreeley](https://github.com/bengreeley) made their first contribution in actions/setup-node#949 - [@HarithaVattikuti](https://github.com/HarithaVattikuti) made their first contribution in actions/setup-node#1041 **Full Changelog**: actions/setup-node@v4...v4.0.3 ### [`v4.0.2`](https://github.com/actions/setup-node/releases/tag/v4.0.2) [Compare Source](actions/setup-node@v4.0.1...v4.0.2) #### What's Changed - Add support for `volta.extends` by [@ThisIsManta](https://github.com/ThisIsManta) in actions/setup-node#921 - Add support for arm64 Windows by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#927 #### New Contributors - [@ThisIsManta](https://github.com/ThisIsManta) made their first contribution in actions/setup-node#921 **Full Changelog**: actions/setup-node@v4.0.1...v4.0.2 ### [`v4.0.1`](https://github.com/actions/setup-node/releases/tag/v4.0.1) [Compare Source](actions/setup-node@v4.0.0...v4.0.1) #### What's Changed - Ignore engines in Yarn 1 e2e-cache tests by [@trivikr](https://github.com/trivikr) in actions/setup-node#882 - Update setup-node references in the README.md file to setup-node@v4 by [@jwetzell](https://github.com/jwetzell) in actions/setup-node#884 - Update reusable workflows to use Node.js v20 by [@MaksimZhukov](https://github.com/MaksimZhukov) in actions/setup-node#889 - Add fix for cache to resolve slow post action step by [@aparnajyothi-y](https://github.com/aparnajyothi-y) in actions/setup-node#917 - Fix README.md by [@takayamaki](https://github.com/takayamaki) in actions/setup-node#898 - Add `package.json` to `node-version-file` list of examples. by [@TWiStErRob](https://github.com/TWiStErRob) in actions/setup-node#879 - Fix node-version-file interprets entire package.json as a version by [@NullVoxPopuli](https://github.com/NullVoxPopuli) in actions/setup-node#865 #### New Contributors - [@trivikr](https://github.com/trivikr) made their first contribution in actions/setup-node#882 - [@jwetzell](https://github.com/jwetzell) made their first contribution in actions/setup-node#884 - [@aparnajyothi-y](https://github.com/aparnajyothi-y) made their first contribution in actions/setup-node#917 - [@takayamaki](https://github.com/takayamaki) made their first contribution in actions/setup-node#898 - [@TWiStErRob](https://github.com/TWiStErRob) made their first contribution in actions/setup-node#879 - [@NullVoxPopuli](https://github.com/NullVoxPopuli) made their first contribution in actions/setup-node#865 **Full Changelog**: actions/setup-node@v4...v4.0.1 ### [`v4.0.0`](https://github.com/actions/setup-node/releases/tag/v4.0.0) [Compare Source](actions/setup-node@v3.8.2...v4.0.0) #### What's Changed In scope of this release we changed version of node runtime for action from node16 to node20 and updated dependencies in actions/setup-node#866 Besides, release contains such changes as: - Upgrade actions/checkout to v4 by [@gmembre-zenika](https://github.com/gmembre-zenika) in actions/setup-node#868 - Update actions/checkout for documentation and yaml by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#876 #### New Contributors - [@gmembre-zenika](https://github.com/gmembre-zenika) made their first contribution in actions/setup-node#868 **Full Changelog**: actions/setup-node@v3...v4.0.0 ### [`v3.8.2`](https://github.com/actions/setup-node/releases/tag/v3.8.2) [Compare Source](actions/setup-node@v3.8.1...v3.8.2) #### What's Changed - Update semver by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#861 - Update temp directory creation by [@nikolai-laevskii](https://github.com/nikolai-laevskii) in actions/setup-node#859 - Bump [@babel/traverse](https://github.com/babel/traverse) from 7.15.4 to 7.23.2 by [@dependabot](https://github.com/dependabot) in actions/setup-node#870 - Add notice about binaries not being updated yet by [@nikolai-laevskii](https://github.com/nikolai-laevskii) in actions/setup-node#872 - Update toolkit cache and core by [@dmitry-shibanov](https://github.com/dmitry-shibanov) and [@seongwon-privatenote](https://github.com/seongwon-privatenote) in actions/setup-node#875 **Full Changelog**: actions/setup-node@v3...v3.8.2 ### [`v3.8.1`](https://github.com/actions/setup-node/releases/tag/v3.8.1) [Compare Source](actions/setup-node@v3.8.0...v3.8.1) #### What's Changed In scope of this release, the filter was removed within the cache-save step by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#831. It is filtered and checked in the toolkit/cache library. **Full Changelog**: actions/setup-node@v3...v3.8.1 ### [`v3.8.0`](https://github.com/actions/setup-node/releases/tag/v3.8.0) [Compare Source](actions/setup-node@v3.7.0...v3.8.0) #### What's Changed ##### Bug fixes: - Add check for existing paths by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#803 - Resolve SymbolicLink by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#809 - Change passing logic for cache input by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#816 - Fix armv7 cache issue by [@louislam](https://github.com/louislam) in actions/setup-node#794 - Update check-dist workflow name by [@sinchang](https://github.com/sinchang) in actions/setup-node#710 ##### Feature implementations: - feat: handling the case where "node" is used for tool-versions file. by [@xytis](https://github.com/xytis) in actions/setup-node#812 ##### Documentation changes: - Refer to semver package name in README.md by [@olleolleolle](https://github.com/olleolleolle) in actions/setup-node#808 ##### Update dependencies: - Update toolkit cache to fix zstd by [@dmitry-shibanov](https://github.com/dmitry-shibanov) in actions/setup-node#804 - Bump tough-cookie and [@azure/ms-rest-js](https://github.com/azure/ms-rest-js) by [@dependabot](https://github.com/dependabot) in actions/setup-node#802 - Bump semver from 6.1.2 to 6.3.1 by [@dependabot](https://github.com/dependabot) in actions/setup-node#807 - Bump word-wrap from 1.2.3 to 1.2.4 by [@dependabot](https://github.com/dependabot) in actions/setup-node#815 #### New Contributors - [@olleolleolle](https://github.com/olleolleolle) made their first contribution in actions/setup-node#808 - [@louislam](https://github.com/louislam) made their first contribution in actions/setup-node#794 - [@sinchang](https://github.com/sinchang) made their first contribution in actions/setup-node#710 - [@xytis](https://github.com/xytis) made their first contribution in actions/setup-node#812 **Full Changelog**: actions/setup-node@v3...v3.8.0 ### [`v3.7.0`](https://github.com/actions/setup-node/releases/tag/v3.7.0) [Compare Source](actions/setup-node@v3.6.0...v3.7.0) #### What's Changed In scope of this release we added a logic to save an additional cache path for yarn 3 ([related pull request](actions/setup-node#744) and [feature request](actions/setup-node#325)). Moreover, we added functionality to use all the sub directories derived from `cache-dependency-path` input and add detect all dependencies directories to cache (related [pull request](actions/setup-node#735) and [feature request](actions/setup-node#488)). ##### Besides, we made such changes as: - Replace workflow badge with new badge by [@jongwooo](https://github.com/jongwooo) in actions/setup-node#653 - Fix a minor typo by [@phanan](https://github.com/phanan) in actions/setup-node#662 - docs: fix typo in advanced-usage.md by [@remarkablemark](https://github.com/remarkablemark) in actions/setup-node#697 - bugfix: Don't attempt to use Windows fallbacks on non-Windows OSes by [@domdomegg](https://github.com/domdomegg) in actions/setup-node#718 - Update to node 18.x by [@feelepxyz](https://github.com/feelepxyz) in actions/setup-node#751 - Remove implicit dependencies by [@nikolai-laevskii](https://github.com/nikolai-laevskii) in actions/setup-node#758 - Fix description about ensuring workflow access to private package by [@x86chi](https://github.com/x86chi) in actions/setup-node#704 #### New Contributors - [@jongwooo](https://github.com/jongwooo) made their first contribution in actions/setup-node#653 - [@phanan](https://github.com/phanan) made their first contribution in actions/setup-node#662 - [@remarkablemark](https://github.com/remarkablemark) made their first contribution in actions/setup-node#697 - [@domdomegg](https://github.com/domdomegg) made their first contribution in actions/setup-node#718 - [@feelepxyz](https://github.com/feelepxyz) made their first contribution in actions/setup-node#751 - [@nikolai-laevskii](https://github.com/nikolai-laevskii) made their first contribution in actions/setup-node#758 - [@x86chi](https://github.com/x86chi) made their first contribution in actions/setup-node#704 **Full Changelog**: actions/setup-node@v3...v3.7.0 ### [`v3.6.0`](https://github.com/actions/setup-node/releases/tag/v3.6.0): Add Support for Nightly, Canary and RC builds for Node.js [Compare Source](actions/setup-node@v3.5.1...v3.6.0) In scope of this release we added support to download nightly, rc (actions/setup-node#611) and canary (actions/setup-node#619) Node.js distributions. ##### For nightly versions: ```yaml jobs: build: runs-on: ubuntu-latest name: Node sample steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: '16-nightly' - run: npm ci - run: npm test ``` ##### For canary versions: ```yaml jobs: build: runs-on: ubuntu-latest name: Node sample steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: '16-v8-canary’ - run: npm ci - run: npm test ``` ##### For rc versions: ```yaml jobs: build: runs-on: ubuntu-latest name: Node sample steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: '16.0.0-rc.1’ - run: npm ci - run: npm test ``` Note: For more examples please refer to [documentation](https://github.com/actions/setup-node#advanced-usage). Besides, we added the following changes as: - Updated minimatch: actions/setup-node#608 - Fixed extra newline character in version output when reading from a file: actions/setup-node#625 - Passed the token input through on GHES: actions/setup-node#595 - Fixed issue with scoped registries are duplicated in npmrc: actions/setup-node#637 ### [`v3.5.1`](https://github.com/actions/setup-node/releases/tag/v3.5.1): Update @actions/core and Print Node, Npm, Yarn versions [Compare Source](actions/setup-node@v3.5.0...v3.5.1) In scope of this release we updated [actions/core to 1.10.0](actions/setup-node#587). Moreover, we added logic [to print Nodejs, Npm, Yarn versions](actions/setup-node#368) after installation. ### [`v3.5.0`](https://github.com/actions/setup-node/releases/tag/v3.5.0): Add support for engines.node and Volta [Compare Source](actions/setup-node@v3.4.1...v3.5.0) In scope of this release we add support for engines.node. The action will be able to grab the version form package.json#engines.node. actions/setup-node#485. Moreover, we [added support for Volta](actions/setup-node#532) Besides, we updated [@actions/core to 1.9.1](actions/setup-node#574) and [@actions/cache to 3.0.4](actions/setup-node#573) ### [`v3.4.1`](https://github.com/actions/setup-node/releases/tag/v3.4.1): Fix pnpm output and node-version output issues [Compare Source](actions/setup-node@v3.4.0...v3.4.1) In scope of this release we fixed bugs related to the pnpm 7.5.1 output issue from `pnpm store path` actions/setup-node#545. Moreover we fixed the issue with falling on node-version output actions/setup-node#540. ### [`v3.4.0`](https://github.com/actions/setup-node/releases/tag/v3.4.0): Add support for asdf format and update actions/cache version to 3.0.0 [Compare Source](actions/setup-node@v3.3.0...v3.4.0) In scope of this release we updated `actions/cache` package as the new version contains fixes for [caching error handling](actions/setup-node#526). Moreover, we added support for asdf format as Node.js version file actions/setup-node#373. Besides, we introduced new output [node-version](actions/setup-node#534) and added `npm-shrinkwrap.json` to dependency file patterns: actions/setup-node#439 ### [`v3.3.0`](https://github.com/actions/setup-node/releases/tag/v3.3.0): Add support for lts/-n aliases [Compare Source](actions/setup-node@v3.2.0...v3.3.0) In scope of this release we added support for `lts/-n` aliases, improve logic for `current`, `latest` and `node` aliases to handle them from `toolcache`, update `ncc` package. ##### Support of lts/-n aliases - Related pull request: actions/setup-node#481 - Related issue: actions/setup-node#26 ```yaml steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: lts/-1 - run: npm ci - run: npm test ``` ##### Minor improvements - Update zeit/ncc to vercel/ncc: actions/setup-node#476 - Get latest version from cache if exists: actions/setup-node#496 ### [`v3.2.0`](https://github.com/actions/setup-node/releases/tag/v3.2.0): Add current, node, latest aliases [Compare Source](actions/setup-node@v3.1.1...v3.2.0) In scope of this release we added new aliases to install the latest Node.js version. actions/setup-node#483 ```yml steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: current - run: npm ci - run: npm test ``` ### [`v3.1.1`](https://github.com/actions/setup-node/releases/tag/v3.1.1): Update actions/cache version to 2.0.2 [Compare Source](actions/setup-node@v3.1.0...v3.1.1) In scope of this release we updated `actions/cache` package as the new version contains fixes related to GHES 3.5 (actions/setup-node#460) ### [`v3.1.0`](https://github.com/actions/setup-node/releases/tag/v3.1.0): Add caching support on GHES 3.5 [Compare Source](actions/setup-node@v3.0.0...v3.1.0) In scope of this release we added [support for caching from GHES 3.5](actions/setup-node#452) and fixed download issue for files > 2GB during restore. Besides, we updated `actions/cache` dependency to 2.0.0 version. ### [`v3.0.0`](https://github.com/actions/setup-node/releases/tag/v3.0.0) [Compare Source](actions/setup-node@v2.5.2...v3.0.0) In scope of this release we changed version of the runtime Node.js for the setup-node action and updated package-lock.json file to v2. ##### Breaking Changes - With the update to Node 16 in actions/setup-node#414, all scripts will now be run with Node 16 rather than Node 12. - We removed deprecated `version` input (actions/setup-node#424). Please use `node-version` input instead. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://gitea.bruyant.xyz/alexandre/PaletteSwitcher/pulls/56 Co-authored-by: Renovate <renovate@bruyant.xyz> Co-committed-by: Renovate <renovate@bruyant.xyz>

dmitry-shibanov added the feature request New feature or request to improve the current logic label Nov 12, 2021

dmitry-shibanov mentioned this issue Nov 26, 2021

Dependencies are still installed after cache flag is true #349

Closed

5 tasks

cypressf mentioned this issue Jan 27, 2022

cache frontend builds mit-jp/mit-climate-data-viz#248

Closed

belgattitude mentioned this issue Mar 6, 2022

Docs: Which Package Manager should I use? vercel/turborepo#785

Closed

kachkaev mentioned this issue May 10, 2022

cache: yarn not working for Yarn 3 (Berry) in a subfolder #488

Closed

5 tasks

belgattitude mentioned this issue May 23, 2022

Make the local cache opt-in (rather than opt-out) yarnpkg/berry#4252

Merged

3 tasks

mrtnzlml added a commit to adeira/universe that referenced this issue Jul 15, 2022

CI: use custom (better) Yarn cache strategy

7ec627f

See: actions/setup-node#325

mrtnzlml mentioned this issue Jul 15, 2022

CI: use custom (better) Yarn cache strategy adeira/universe#4743

Merged

kodiakhq bot pushed a commit to adeira/universe that referenced this issue Jul 15, 2022

CI: use custom (better) Yarn cache strategy

82d9052

See: actions/setup-node#325

This was referenced Aug 24, 2022

CI / monorepo improvements trezor/trezor-suite#6103

Open

chore(ci): better yarn github cache trezor/trezor-suite#6123

Merged

marcusrbrown mentioned this issue Oct 22, 2022

feat(ci): update Yarn caching marcusrbrown/translate-app#8

Merged

Michsior14 mentioned this issue Nov 10, 2022

feat: support yarn berry nrwl/ci#43

Merged

Gudahtt mentioned this issue Nov 10, 2022

Use actions/setup-node cache MetaMask/metamask-module-template#145

Merged

merceyz mentioned this issue Jan 11, 2023

chore: Use setup-node action to cache dependencies nodejs/corepack#230

Closed

dsame self-assigned this Apr 21, 2023

dsame mentioned this issue Apr 24, 2023

Do not ivalidate the cache entirely on lock file change #744

Merged

2 tasks

This was referenced May 6, 2023

CI: speed up CI time by improving yarn install and caches (iteration 1 / >30%) strapi/strapi#16581

Merged

Ci: improve install time iteration 2 strapi/strapi#16638

Closed

belgattitude mentioned this issue Jun 20, 2023

feat: cache npm metadata yarnpkg/berry#5491

Merged

7 tasks

dsame closed this as completed Jul 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance opportunity for yarn 3 cache #325

Performance opportunity for yarn 3 cache #325

belgattitude commented Sep 3, 2021 •

edited

Loading

merceyz commented Sep 10, 2021

dmitry-shibanov commented Nov 12, 2021

dmitry-shibanov commented Nov 22, 2021

belgattitude commented Nov 22, 2021

belgattitude commented Nov 23, 2021 •

edited

Loading

devgioele commented May 5, 2022 •

edited

Loading

panticmilos commented May 9, 2022

mklueh commented Jul 10, 2022

nickserv commented Sep 24, 2022

belgattitude commented Oct 9, 2022 •

edited

Loading

marcofugaro commented Nov 8, 2022

merceyz commented Jan 11, 2023 •

edited

Loading

nijikon commented Apr 5, 2023

marko-zivic-93 commented Apr 6, 2023

dsame commented Apr 21, 2023 •

edited

Loading

belgattitude commented May 1, 2023

belgattitude commented May 1, 2023 •

edited

Loading

dsame commented May 3, 2023

dsame commented May 3, 2023

belgattitude commented May 4, 2023

dsame commented May 5, 2023

belgattitude commented May 5, 2023 •

edited

Loading

dsame commented May 5, 2023

dsame commented May 5, 2023

belgattitude commented May 6, 2023 •

edited

Loading

belgattitude commented May 6, 2023 •

edited

Loading

dsame commented May 9, 2023 •

edited

Loading

belgattitude commented May 10, 2023 •

edited

Loading

dsame commented May 10, 2023

nijikon commented Jul 18, 2023

dsame commented Jul 19, 2023

nijikon commented Jul 19, 2023

dsame commented Jul 25, 2023

nijikon commented Jul 25, 2023

Performance opportunity for yarn 3 cache #325

Performance opportunity for yarn 3 cache #325

Comments

belgattitude commented Sep 3, 2021 • edited Loading

Updated example with action cache

Setup action

Testing a cache hit after adding a dependency

Note

merceyz commented Sep 10, 2021

dmitry-shibanov commented Nov 12, 2021

dmitry-shibanov commented Nov 22, 2021

belgattitude commented Nov 22, 2021

belgattitude commented Nov 23, 2021 • edited Loading

devgioele commented May 5, 2022 • edited Loading

panticmilos commented May 9, 2022

mklueh commented Jul 10, 2022

nickserv commented Sep 24, 2022

belgattitude commented Oct 9, 2022 • edited Loading

marcofugaro commented Nov 8, 2022

merceyz commented Jan 11, 2023 • edited Loading

nijikon commented Apr 5, 2023

marko-zivic-93 commented Apr 6, 2023

dsame commented Apr 21, 2023 • edited Loading

belgattitude commented May 1, 2023

belgattitude commented May 1, 2023 • edited Loading

dsame commented May 3, 2023

dsame commented May 3, 2023

belgattitude commented May 4, 2023

dsame commented May 5, 2023

belgattitude commented May 5, 2023 • edited Loading

dsame commented May 5, 2023

dsame commented May 5, 2023

belgattitude commented May 6, 2023 • edited Loading

belgattitude commented May 6, 2023 • edited Loading

dsame commented May 9, 2023 • edited Loading

belgattitude commented May 10, 2023 • edited Loading

dsame commented May 10, 2023

nijikon commented Jul 18, 2023

dsame commented Jul 19, 2023

nijikon commented Jul 19, 2023

dsame commented Jul 25, 2023

nijikon commented Jul 25, 2023

belgattitude commented Sep 3, 2021 •

edited

Loading

belgattitude commented Nov 23, 2021 •

edited

Loading

devgioele commented May 5, 2022 •

edited

Loading

belgattitude commented Oct 9, 2022 •

edited

Loading

merceyz commented Jan 11, 2023 •

edited

Loading

dsame commented Apr 21, 2023 •

edited

Loading

belgattitude commented May 1, 2023 •

edited

Loading

belgattitude commented May 5, 2023 •

edited

Loading

belgattitude commented May 6, 2023 •

edited

Loading

belgattitude commented May 6, 2023 •

edited

Loading

dsame commented May 9, 2023 •

edited

Loading

belgattitude commented May 10, 2023 •

edited

Loading