Blog

2025 in retrospect

Jan 2, 2026

NomadicCore

Comms

Ermo

Co-founder

2025 has been a year of significant change for the AerynOS project, not just in terms of development itself but also in name and in the staff working on the project.

We know many people will be following along, waiting for our beta and/or stable releases but for now we want to take a look back at 2025 and summarize what we have delivered and how that is positioning us strongly for 2026.

Camp Fire

The TL;DR summary:

We changed our name from Serpent OS to AerynOS. Aeryn is a different form of Erin, which means Ireland in Irish Gaelic.
The project founder, Ikey, stepped away from the project in mid April. A few weeks later, he sent an invitation to project co-founder and co-architect, Rune Morling (ermo), to be his Github account successor.
After these events, ermo asked the team if they would be interested in carrying on working on the project under his stewardship while waiting for Ikey to potentially make contact, and the team voted unanimously in favour of doing so.
Since then, the team has continued to deliver per the project roadmap:
- We completed the transition of all core tooling to Rust.
  - The new Rust-based infra has now delivered thousands of package builds while proving to be much more stable and robust than the previous implementation.
  - Leveraging Rust has enabled us to confidently deliver new tooling solutions like versioned repositories and a system-model approach to declaratively describe system installs.
- We improved upon the Cosmic DE offering whilst delivering new KDE Plasma and Console-based install options
- We onboarded several new staff members onto the project over the course of the year with differing strengths and interests
- We reworked our cloud hosting setup to deliver more build capabilities at lower cost base
- We renewed our media strategy through increased blog posts and engagement across social media platforms

What’s in a name

The first and biggest change we made this year was changing our name from Serpent OS to AerynOS. We are aware that, to some, this move was somewhat controversial. Our stance has been simple: A name takes on the meaning you give it.

Over time, we have continued to work to deliver on our goals for AerynOS. In return, people have become less focused on the name and more focused on what we are setting out to do, and what we are continuously delivering.

As an aside, we originally utilised AI tooling to create our logo, but this is not something we are overly comfortable with for the long run. We are looking at our options around either iterating on or creating a new “human made” logo where we can feel a lot more confident around licensing and ownership of the artwork itself.

The project founder steps away

In mid April of this year, Ikey went offline as part of a move, and subsequently never returned.

At the time, the team tried to reach out in multiple ways, however there has sadly been no reciprocation, other than ermo receiving a Github request sent from Ikey’s Github account to become Ikey’s account successor towards the end of the first week of May.

Taking into account Ikey’s experiences up until his move, we have made the assumption that his stepping back from the project was related to ongoing personal health problems for him and his family, which were compounded by significant financial strain as we understand it.

Note that, out of respect to Ikey and his family, we will not be elaborating any further on this. Needless to say, we wish him and his family the best and hope that they will have found a way through their travails.

As Ikey has left us with no way to contact him, we cannot share with you how to directly support him financially. All we can say is that you may be able to do so via his personal GH sponsors account, @ikeycode, though we stress that we have no way of knowing whether he will actually be receiving any money you decide to send that way.

Team regroup and reset

To ensure the project would be able to continue in the short and medium term, and on the basis that he felt that the promise of the tech and the underpinnings of AerynOS were too good to be left to languish and wither away, ermo asked the team whether they would like to continue working under his stewardship while we awaited news from Ikey. The team voted unanimously in favour of doing so.

Hence, for the initial couple of months, our focus was to continue delivering as a project, while allowing Ikey the space and the time to return once he’d had a chance to regroup and recover. This included letting all project sponsorship (including Ko-fi sponsorship) flow directly to Ikey.

As time progressed and the likelihood of Ikey getting in contact and returning grew dim, we needed to be in a position to actually have control of project assets and went about gaining said control in order to be able to keep the proverbial lights on.

With the exception of the original AerynOS X account, we have regained access to all relevant accounts, and have continued delivering on the project goals in the meantime.

We have done our best to not let any of this negatively impact our early alpha testers’ experience of the project, and feel that we have largely succeeded in doing so. The only real hiccup was a few months of reduced package delivery from mid April to late May, where we worked flat out to restore our ability to deliver package updates via the then under-development Rust infra port, after the old Proof-of-Concept DLang infra broke down completely.

We are now 9 months on from Ikey stepping back, and taking into account that we have not heard from him and that he explicitly designated ermo as his GH account successor, we are in the final stages of offboarding him into an inactive project alumni role.

We would like to stress that, as a project, we hold no ill will towards Ikey, and that we simply wish him the best. There is no hiding that his work was foundational to AerynOS with respect to both the tooling, the distribution itself, and the vision & ethos he promulgated for the project. We simply would not be where we are now as a project without his years of forward-looking engineering efforts.

At the same time, we think the present blog post will serve to outline why, based on our ability to deliver up until now, we are confident in our ability to continue developing AerynOS and its tooling into the future.

Infrastructure & Tooling development

Late March and April also saw the start of the porting effort of our infrastructure from DLang to Rust. The approach for this porting effort was to develop a so-called “Minimum Viable Product” (MVP) code base that could iteratively be improved.

After several months of development, in late May we started delivering packages to users again, which had all been built from the new Rust based infrastructure code base. This effort included a full rebuild of the whole recipe repo at the time to ensure ABI sanity of the newly built packages.

The development of our infrastructure didn’t stop there. Throughout the year, the code has been continually developed and expanded upon to land new features. In the last three months, the team has successfully delivered an MVP of the Versioned Repositories feature and an accompanying system-model feature as two very important examples of this.

Versioned Repositories

When fully fleshed out, Versioned Repositories will allow the team to seamlessly deliver improvements to our os-tooling (moss and boulder) in a controlled fashion. This is important as AerynOS is a rolling release distribution, meaning we need to be able to deliver improvements to users’ systems without breaking them or requiring complex manual upgrade procedures.

From a user perspective, it will open up the ability to decide which version or “update stream” of our repositories they want to use, for example opening up the capability to “lock” their system to a specific release or point in time, or eventually opening up the capability to use release-candidate or in-testing package builds that wouldn’t otherwise make their way into the “standard” AerynOS Repository.

In short, users will be able to use Versioned Repositories to easily choose a more stable and cautious repository or conversely to choose a more bleeding edge repository based on their needs and desires for their systems.

More broadly speaking, Versioned Repositories will also eventually allow for layered repositories for x86_64-v3 and -v4 packages to sit above our baseline x86_64-v2 repository where we see worthwhile improvements in building -v3 and -v4 packages. AerynOS’s approach here will not be to rebuild full repositories of -v3 and -v4 packages, but instead to overlay these packages on top of the x86_64-v2 repository only where there are verifiable gains to be found by doing so.

In a similar vein, this development workstream will eventually enable us to build repositories targeting ARM and RISC-V instruction sets. This will be a ways down the road and the team would need to acquire ARM and RISC-V builders at the appropriate time, but we want to foreshadow this to highlight that our current up front planning of our infrastructure development roadmap will enable very flexible use cases down the road.

system-model

The system-model approach we are currently developing, is at its heart a declarative definition of the packages comprising a system and where they come from. The team has just landed the first iteration of this to users, and this — along with Versioned Repositories — will continue being a development focus into 2026. The system-model approach declares the repositories & packages a user wants to sync their system against. This is currently an opt-in solution with moss now automagically generating a system-model.kdl file each time a moss transaction successfully completes.

The team has multiple future ideas of how this system-model can be used with a few examples including:

Sharing your system-model.kdl file for reproducing your system state for issue tracking and resolution
Using your system-model.kdl file as part of your install to get your system exactly how you want it at first boot
Changing your system by updating packages and then reverting back to your desired system-model on next sync
Being able to pin to older Versioned Repositories (for stability’s sake) or if issues occur on the latest rolling Versioned Repository until it’s fixed

All of these features, particularly where users could potentially face issues with their installs, are complemented by our offline rollback feature, which gives the ability to manually go back to either of 5 earlier states from the bootloader. This will ensure that users have both offline and online protections to help them recover from a misbehaving system.

Team organisation

We kicked off the year with looking for support in 4 key areas:

Community management
Documentation
Translation
Web development

As the year has gone on, NomadicCore, Jplatte, bhh32 and CookieSource have joined the team bringing with them a vast amount of experience and enthusiasm.

It’s fair to say that community management and documentation development have taken significant steps forward. Web development is a work in progress area with plans being formulated for an eventual re-write of our main website, which frankly needs a lot of work. Translation efforts have been paused (or more accurately never started). The team is prioritising building out the core infrastructure and tooling first, with distro polish taking a back step until the tooling is ready for it.

Website development

The current state of our website leaves a lot to be desired. We are aware that it needs a full refresh, not only for presentation, but also to include important information as a bare minimum.

The team are looking at options such as Zola and Hugo as alternative Static Site Generators however this is a low priority workstream for now. We are open if anyone has experience in website design and would like to work with the team to develop our website into something to be proud of.

Distro updates

DE and WM updates

Over the course of 2025, the team have continued to keep Gnome packaging updated whilst improving upon Cosmic through to its latest stable release. In addition, Reilly Brogan was able to get KDE Plasma packaged up, which expanded our recipe repo size by almost 50%.

All three desktop environments have proved to be stable, though it needs to be said that Cosmic overall has a way to go for true maturity.

We also decided to include a terminal-only option in our installer, which can be used to install and tweak e.g. Sway, Niri (and soon MangoWC) based custom Wayland environments almost from scratch.

For now, we don’t have any concrete plans to package up additional DEs or WMs for AerynOS. There are “enough” options for early alpha testers and our core team to work from, and any additional options would simply add an extra maintenance burden on the team that would in turn detract from necessary tooling, infra and overall feature and integration work.

In terms of future potential, it’s worth mentioning that AerynOS is first and foremost a forward-looking, Wayland-aligned project, which means that Wayland session support is a minimum requirement for any future DE or WM additions to the repository.

Installer

At the start of this year, Ikey shared initial screenshots of a GUI based installer that would eventually replace our text based installer as the primary installation method. With Ikey stepping back from the project, the development of this GUI installer took a back seat to allow the team to focus on higher-priority items such as delivering working infra.

Recently, one of our new staff members, Bryan Hyland, began working in the background on a new GUI based installer to eventually fill this role.

To put it simply, the idea is to eventually use the newly developed system-model approach to define the packages that would be installed per DE / WM and also allow users to use their own custom system-model.kdl files as the basis of an installation.

Sponsorship

With Ikey stepping away earlier this year, for a period of time, it was unclear whether he would be returning or not. As the months passed by, the prospect of his return grew less and less likely. Therefore, as covered earlier in this blog post, we are now in the latter stages of formally retiring Ikey from the project.

As mentioned previously, at the start of this year, the sponsorship was set up in such a way that all funds were sent directly to Ikey in order to enable him to afford to spend time working on the project and its underpinnings.

However, in light of Ikey’s continued absence, the team has taken control of the AerynOS Ko-fi account and redirected the funds from it so they can be used to cover ongoing project running costs.

All other references to sponsorship routes have been removed, leaving Ko-fi as our only official sponsorship medium.

Due to the way Ko-fi sponsorships work, at the point of transition, we had to cancel all existing sponsorships. We are not yet at the point where we are at parity in sponsorship income compared to before they were cancelled, however, we are fully covering direct project costs.

Part of this ability to cover direct project costs can be attributed to how we are spending our money and how we are utilizing as much of our own internal resources (such as ermo’s machines) for build capacity. We have also been developing our infrastructure code base to make it as flexible as possible in terms of adding new builders and scheduling builds efficiently based on individual builder resources.

Our sponsorship goals in 2026 will be to continue growing our recurring sponsorships to help cover the backlog of historic project costs, and also to build up towards future hardware investments such as an x86_64-v4 capable builder.

If any hardware vendors are interested in sponsoring the project either financially or through hardware sponsorship, this would be warmly received.

Similarly, we are interested in and open to CDN sponsorship as a way of strengthening our package delivery and ISO download capabilities. If you are in a position to help make this happen, we would love to hear from you.

If you wish to discuss sponsorship details, please reach out to us at contact@aerynos.com.

Sponsor AerynOS

Going into 2026

For now, there is a deliberate, continued focus on our os-tooling and infrastructure code bases, somewhat at the expense of delivering a fully featured Linux distribution for end users.

As time goes on, the goal is for this balance to naturally shift as our code bases become more capable and featureful.

Over time, and as logical consequence of this shift, we hope to be able to onboard more contributors to help us scale out the recipe repo, once the tooling is in a state where it conveniently allows for this.

If any of this has piqued your interest and made you want to contribute, we invite you to join us over at our Zulip chat platform.

Hope to see you there!

November + early December 2025 project update

Dec 9, 2025

NomadicCore

Comms

Ermo

Co-founder

AerynOS: November + early December 2025 project update

As we near the end of the year, the team has been reflecting on how best to position itself going into 2026. Over the course of November and early December, the team has made a number of changes, some of which have been public facing which you may have already seen if you’ve been following our socials.

On the public side, the two most visible changes has been our transition away from Matrix to Zulip for our instant messaging chat platform, and our transition to netcup for our server requirements. In the background, the team has also changed email hosting providers, however no one would really have noticed any difference there.

Packaging updates are progressing nicely with a nice rhythm for Cosmic and KDE stack updates, which along with fixes to reported issues is all helping to improve the overall experience for the people testing and checking out AerynOS.

On the infrastructure side of things, we have landed simple auto-pruning logic into our Vessel repository manager. This sort of automation ensures that we keep our repository lean and mean.

Things are progressing nicely in terms of getting phase 1 of our Versioned Repositories infra feature ready for production use. It was designed to from the outset to mesh nicely with our Moss system-model feature. Internal testing on a private infra instance has been running smoothly with these new features so far.

What’s new in the distro

Package / stack updates for this iteration include:

COSMIC Beta9
GNOME 49.2
KDE Frameworks 6.20.0
KDE Gear 25.08.2
KDE Plasma 6.5.4
bash 5.3.8
buildah: Add at v1.42.2
docker 29.0.4
ffmpeg 8.0.1
firefox 146.0
gamemode: Add at 1.8.2
linux 6.17.10
llvm 21.1.7
mesa 25.3.1 (with Vulkan anti-lag support enabled)
openvpn 2.6.17
prism-launcher 9.4
scx-scheds 1.0.17
sudo-rs 0.2.10
uutils-coreutils 0.4.0
vim 9.1.1896
vscodium v1.106.37943
wine 10.19
xwayland-satellite 0.8
zed 0.210.4
zlib-ng: 2.3.2

… along with sundry additions and updates.

Desktop Updates

Cosmic

Cosmic Install

Our Cosmic packaging team have automated much of the update process based around the more frequent git tags that the System76 team are now publishing for the Cosmic Beta phase. As such, we are more quickly able to push updates out to our volatile repository for eventual availability in our unstable repository.

As an aside, we believe we have fixed the issue for Cosmic DE sessions where USB devices were not automatically showing up, by adding gvfs as part of the Cosmic pkgset. This is a fairly important usability feature so great to have it fixed.

We have also fixed the issue we reported in our October Project Update blog post about sudo-rs not functioning correctly on Cosmic Terminal and Ptyxis in our Cosmic session.

Overall, Cosmic is progressing nicely in general and we are polishing the experience on AerynOS in lockstep with Cosmic Beta9 having landed in our repositories.

Additional details about Cosmic can be found on System76’s website.

Gnome

Gnome Install

As usual, there isn’t really much to say wrt. GNOME. The team has updated it to 49.2 and it is running nicely as expected.

KDE Plasma

KDE Install

KDE Plasma has been updated to 6.5.4, KDE Frameworks to 6.20.0.

As part of these updates, Reilly added a few custom patches to better support monitors with high refresh rates.

Infrastructure Updates

Auto-pruning in vessel

The new auto-pruning feature in our Vessel repository manager service means that our infra will periodically (currently daily) review what packages are present on our server storage versus which packages are reachable via a repository index.

If packages are no longer reachable via a repository index, they are considered surplus to requirements and pruned.

Automating this workload ensures that our storage doesn’t fill up and that our repository looks after itself over time, freeing the team from encountering unpleasant surprises in the form of having to take corrective action as storage fills up.

Wider Project Updates

Transition to netcup for our server requirements

As part of a wider review of our requirements, the team looked at how we were using our server and whether it was “right-sized” for where we are as a project. The project has shifted over the last 7 months to more heavily using ermo’s four private servers, meaning the “main” AerynOS build server hasn’t been utilised as effectively. Therefore, the team took the decision to sunset this server and look for a server better specified for repo hosting and infra coordination. To this end, we settled on a netcup “root server” based in Germany (where the old server was based in Finland). The netcup server has a 2.5Gbit NIC and has — for European users at least — yielded significantly faster download speeds.

It is still early days, however we are very pleased with our transition to netcup. Their support offering during our initial set up phase was both responsive and very helpful.

Transition from Matrix to Zulip

We want to preface this by saying we love what Matrix is doing by providing a federated open source instant messaging chat solution for users.

However, for AerynOS specifically, we have been having a few issues with our matrix space, partially in administration and partially around federation and delays in messaging (or in some cases, some users not able to see messages from other users in one room but they could in others).

As a team, we looked at what other options were available, and considered what our requirements were before eventually deciding to try Zulip on for size.

After trying it internally for some time, it quickly became clear that it offers great instant messaging capabilities like Matrix whilst also offering many other features such as asynchronous conversations (threads/topics) that allow users, contributors and staff to better focus on the various project conversations that fall in their sphere of interest or responsibility.

Like Matrix, Zulip is an Open Source Project and it also has hundreds of integrations that help elevate the overall User and Moderator experience.

Initial feedback from our wider user base has been overwhelmingly positive and we are excited to continue on this journey!

For transparency, Zulip has a policy of supporting Open Source Projects, and has generously sponsored the AerynOS project with a standard cloud server instance at no cost.

Please feel free to join our Zulip server and get to know the community that is building up around AerynOS.

Transition to Migadu for emails

The team had a look at the options available in the market for emails and decided to make a transition to Migadu’s offering, after Migadu offered us an OSS project discount.

While it does require a little more set up than other providers out there, it has the substantial benefit that you can set up as many email addresses on your account as you want or need, without this impacting the cost base.

This will undoubtedly prove helpful for the project going forward.

Logo / Branding / Website progress

The team has been looking at our branding and reviewing our requirements. It’s safe to say that if we didn’t have to update these, we would keep them as is as there are other higher priority activities.

However, if we take the logo itself, it was created using AI tools, so doesn’t really line up with our project ethos and does leave questions over licensing and ownership.

As such, we have been looking at what we might want from a logo and how it can best represent AerynOS. In keeping with this, we have also been looking at our wider branding including the colour schemes that we use.

Given that the naming conventions for AerynOS’s infrastructure and tooling projects all link to nature, we are leaning towards a nature oriented colour scheme going forward.

Lastly, we have been looking at our website which is in dire need of work. We currently use Astro for the website, however we have found that to be a complex solution. This is especially true as none of the current team / wider contributor base has much experience with Astro. We have been looking at the field of Static Site Generators (SSGs) and have shortlisted it down to Zola and Hugo, both of which are meant to be much easier to work with than Astro.

Zola is the preferred option with it being Rust based, however it’s fair to say that it is a much smaller project so doesn’t have anywhere near as extensive a theme library to choose from. Hugo is a larger / more mature project with greater theming, but is tied to Go which would create an additional maintenance burden outside of the team’s core focus.

Outside of existing themes, one option is to create (and subsequently maintain) a brand new theme, but this requires expertise and would place a burden on the team to ensure compatibility and that everything is updated within the template as time progresses.

If you have experience in website design and would like to help create / shape a redesign for our main website, then please join us on our Zulip server so that we can discuss how to move things forward.

Documentation website progress

With new contributors coming on board, our documentation website has seen significant updates in both content and in presentation. Whilst still a work-in-progress project, it is becoming more usable with less gaps. The team and wider contributor base continues to push updates, the big outstanding one being how to create new package recipes from scratch.

Once these gaps are filled in, it will become an iterative exercise to make sure our documentation stays updated and potentially go down into further detail where feedback suggests this is necessary.

Download choices and BitTorrent

Another area we have been reviewing is how we serve ISO downloads to our users. We currently only offer a direct download via our package server behind a CDN. As a small operation, this has suited us well enough, however users in certain locations don’t necessarily benefit from this solution.

We have had the BitTorrent option on the board for a long while now, and over the last couple of weeks, we silently added a torrent link for the AerynOS 2025.10 ISO. With the 2025.12 ISO we are taking this approach forward. This will have a secondary benefit of reducing the load on our server / CDN at the point of ISO releases, as bandwidth is naturally distributed due to the torrenting approach. We will continue to offer direct downloads for anyone who prefers this option.

ISO refresh

We are releasing our newest Alpha ISO, AerynOS 2025.12, which includes the updates we’ve worked on during the month of November and the first week of December, and which features the 6.17.10 kernel at the time of upload.

As usual, this is a Live GNOME ISO that contains our Alpha/PoC lichen installer. Hence, installing AerynOS requires a network connection over which the latest pkgsets can be downloaded and subsequently installed onto a dedicated AerynOS target hard drive.

The link for our 2025.12 ISO can be found at our download page.

Next Steps

The rest of december is dedicated to bringing phase 1 of the Versioned Repositories feature PR here and its associated Moss system-model companion feature PR here up to snuff and landed for the general public.

As mentioned in the introduction, these PRs are already in testing, and merely need some final bits and bobs wrapped up before they can be landed and put in production.

The fully realised Versioned Repositories feature will enable us to deploy new Boulder and Moss features in a seamless fashion. It will allow us to introduce breaking code and on-disk format changes, that would otherwise cause installed systems to require manual intervention for them to continue to receive updates.

Once Versioned Repositories are in place, the goal is that users will be able to simply update and sync their system as normal via the sudo moss sync -u command.

With this:

Users will be upgraded to the new versions of Moss that uses a new repository format, without having to pay special attention.
It will enable AerynOS to iteratively expand the capability of Moss and Boulder on existing systems without breaking user systems in the process.

Going further, we consider Versioned Repositories a pre-requisite for what we call “try-builds” and eventually for multi-arch support.

Automated try-builds denotes the process whereby the infrastructure discovers an update to the upstream source repository of a package, attempts to auto-update the recipe and then attempts to build the updated package recipe in question.
We think this will be a useful tool for contributors as it will automate some of the packaging tedium related to simple package version updates. It will also help enable automated regression testing and build flag optimisation in a future workstream.
Included under the multi-arch umbrella is our ability to target ARM, RISC-V, and different x86 architecture levels such as x86-64-v3 and v4.

Supporting the project

Following the October blog post, we had a flurry of new donors whom we want to thank for supporting our project. Their support is greatly appreciated, especially given the global cost of living crisis leaving less money in peoples pockets each month. Your support means a lot to everyone on the project as it shows the faith you have in the work we are doing!

2025 has been a significant year for the project, with the team having landed our new Rust based infrastructure, repository-wide rebuilds along with landing KDE and significantly improving our Cosmic offering whilst also landed new features into our tooling. We see our Versioned Repository work continuing into 2026, and this will eventually help lift AerynOS into becoming a serious offering in the Linux distribution space. Onwards and upwards!

If you are following along with our project and are in a position to support us, please consider donating via our Ko-fi page.

Musings on Inode Watchers and Atomic Live Upgrades

Nov 4, 2025

Joey Riches

Tooling & Packaging

This blog post will focus on inode watcher applications as well as the difficulties they present for live atomic updates for our moss package manager.

What is Moss?

For those not in the know moss is an atomic package manager that allows for live updates i.e. not needing to reboot to apply updates.

Although moss presents itself as a traditional package manager, under the hood, it works quite fundamentally differently.

When you install a package with moss, it actually downloads and extracts the package to a Content Addressable Store (CAS) in /.moss/assets. It then constructs a new virtual filesystem in memory comparing the current installed state to the new desired installed state containing the additional package. From the CAS it then constructs a /usr tree in /.moss/root/staging containing the additional package using hardlinks into the CAS. Then, using the renameat2 Linux kernel syscall with the RENAME_EXCHANGE flag set, moss atomically promotes the current /.moss/root/staging tree to be the new /usr tree, and simultaneously demotes the current /usr tree back to being an inactive, numbered filesystem transaction (fstx) tree in /.moss/root/<fstx id>. Finally, moss then updates the bootloader configuration to reference the five newest numbered filesystem transactions for rollback purposes.

Compared to other atomic distributions on the market, the ability to live-update without needing to reboot is an important usability requirement for us, such that the user experience remains friendly to downstream users.

Enter Inode Watchers

Although quite a novel approach, allowing atomic updates of a live system leaves us with an interesting problem: After any moss transaction activating a new /usr tree, any running applications holding an underlying inode to a filesystem path will continue to watch the file in the now archived state.

For example:

$ inotifywait -m /usr/bin/le-foo &
# moss remove 'binary(le-foo)'

In this case, if you hold an inode to a path that is deleted after the the /usr tree is atomically swapped as part of a moss transaction, you will continue to hold the inode to the file in the archived state e.g. /.moss/root/<fstx id>/bin/le-foo. This is due to the fact that the underlying file in the CAS that was referenced from the previous /usr tree was not removed from the system; it still exists in the now archived previous /usr tree.

For any running applications whose functionality depends on these inode watchers, it can leave the system in a weird state as the application has no real way to know that the “real path” has now changed from /usr/<something> to /.moss/root/<fstx id>/<something>.

The most obvious example in which this presents to users is in our GNOME Edition. GNOME Shell uses a inode watcher on /usr/share/applications/ to watch for any changed .desktop files as applications get installed or removed. This design works pretty well for a traditional installation, and reduces the number of expensive stat calls required to see what applications are available to launch. However, notably this design does not work with the design of moss, in that when applications are freshly installed in GNOME they simply do not show up in the application launcher as GNOME Shell instead continues to hold the inode to the archived path. e.g. /.moss/root/<fstx id>/usr/share/applications/, once a mutating moss operation is performed.

Whilst we could patch GNOME Shell instead to stat for new changes in /usr/share/applications/, patching every application that has issues with picking up file-system changes is not feasible across the ecosystem.

Alternative Approaches?

One suggested alternative has been to explore so called “mount-tucking” and EROFS images.

Mount-tucking is a fairly new addition to the Linux kernel where you can mount a new image beneath a currently mounted image for a path.

For example

# mount my-image.img /mnt
# mount --beneath my-new-image.img /mnt
# umount /mnt

In this example, once /mnt is unmounted it will unmount my-image.img and leave my-new-image.img mounted in its place. If any files from my-image.img are currently open, then a lazy unmount of the /mnt path is required.

When combined with EROFS (extended read-only filesystem), we can construct a lightweight, metadata-only EROFS /usr tree image which then links into the underlying CAS to form the new /usr tree instead, then mount it beneath the currently running /usr tree lightweight EROFS image. Lastly, we can then lazy unmount the current running /usr image so the new image will apply in its place. As an additional benefit, this approach ensures that the /usr tree is also immutable whilst still remaining atomic.

However, this approach has now been explored, and the fundamental problem remains that any running applications holding an inode to a changed file will simply not see any change.

Are We Stuck with Needing To Reboot?

Possibly.

The Linux kernel does not offer any mechanism to forcibly “revoke” inodes. If it did we would potentially build up a list of changed files, find their inodes, and then “hint” that they have changed after the /usr trees are swapped. Any running applications that were then watching the inodes could pick up the changes.

For this particular problem, ideas are starting to run thin. Whilst it is important to us that live atomic updates remain possible instead of requiring the user to reboot, solving this problem currently has us scratching our heads.

TL;DR: More research needed.

October 2025 project update

Oct 31, 2025

NomadicCore

Comms

Ermo

Co-founder

We are firmly in the final quarter of 2025 and what a year it’s been so far. In our last blog post at the end of September, we mentioned that we were delaying the release of our next ISO into October to give our Gnome 49 stack (and the wider extensions ecosystem) more time to mature.

Coming into the final week of October, the team made the decision to transition from clang’s libc++ to GNU libstdc++ and work through the associated rebuild requirements across our repository.

We also mentioned in our September blog post that we had reached the point of having stable build infrastructure. However, over the course of October, we had an old problem re-appear, which proved somewhat vexing to solve initially.

After the issue was debugged, and a fixed version of boulder was deployed to the build infrastructure, we used the transition back to libstdc++, which included hundreds and hundreds of rebuilds, and the landing of the KDE Plasma 6.5 stack to verify that that we have put this particular issue behind us.

In addition to the build infrastructure related boulder fixes, the team has also found the time to hash out a design for, and prepare an early prototype PR of, the moss declarative system-model feature, as well as landing a few small user experience improvements and correctness fixes to both moss and boulder.

With that all said and done, we are ready with our 2025.10 GNOME Live ISO refresh for you to enjoy along with this blog post.

Whats new in the distro

Package / stack updates for this month include:

KDE Plasma 6.5.1
Cosmic Beta3
Gnome 49.1
Linux 6.16.12
Mesa 25.2.5
KDE Frameworks 6.19
KDE Gear 25.08.2
llvm 21.1.4
uutils-coreutils 0.3.0
sudo-rs 0.2.9
ffmpeg 8.0
pipewire 1.4.9
Wine 10.17
nodejs 22.21.0
zed 0.206.6
virt-manager 5.1.0
bash 5.3.3
scx-scheds 1.0.16
vim 9.1.1829
systemd 257.10
uv 0.9.5
perl-module-build: Add at v0.4234
libdovi: Add at v3.3.2
openconnect: Add a v9.12

… along with sundry additions and updates.