Not a main developer, but Rust does not have CVE tracking or a standards process. Can live with uv if main developers choose to merge this, but hatch or PDM would be much preferred in my opinion.

Using pyproject.toml is good though.

uv primarily defaults to using Hatchling as its build backend, same as hatch.

This is because Hatchling is a modern, standards-compliant, and minimal build backend maintained under the PyPA (Python Packaging Authority) umbrella, which aligns well with uv's goals of speed and efficiency.

uv is also the only package manager among others (Poetry, PDM, Hatch, Pipenv) that doesn't require Python to be pre-installed on the system.

It's hard to be truly persuasive. The best approach is for people to try it out for at least a month to see if it works for them.

The problem with the long Rust dependency toolchain:

Am expecting a stream of incidents such as xz utils is required before Rust developers see the light.

I understand the concern about dependency chains, especially in light of recent incidents like xz utils. While xz utils itself is written in C (as is much of Python's core, for that matter), such incidents underscore that the security of the software supply chain is a shared challenge across all ecosystems. I believe the Rust community, like many others, is actively working on enhancing security measures within their tooling and libraries.

My 2c: Having a pyproject.toml is great because it's a common standard that several tools can use. If we have concerns about uv not being sufficiently secure enough for our infrastructure, this doesn't seem like it's too tied to uv in as much as those could be replaced by another manager. I don't have too strong an opinion on what's used: I'm sure uv's fast but I'm not sure that the biggest bottleneck in this project's builds is setting up the build environment.

Thanks so much for taking the time to look into improving InkStitch's build system reproducibility! It's fantastic to have more eyes on this crucial area.

Regarding some of the recent discussions, it's important to clarify that CVEs identify vulnerabilities in concrete software implementations, rather than in the programming languages in which they're written. Similarly, package managers like uv, Poetry, PDM, and Hatch don't have their own centralized, independent CVE tracking systems for the entire Python ecosystem. Instead, known vulnerabilities are assigned to specific packages or tools (for instance, PDM has had CVEs assigned to it in the past, e.g., CVE-2023-45805 related to lockfile manipulation).

As far as I understand, it doesn't matter which tool you use to create your virtual environment (be it uv, Poetry, PDM, or venv directly). What's critical is to then verify its contents and dependencies using auditing tools like pip-audit or safety. The creation tool sets up the initial environment, but the actual security validation comes from scanning the installed packages against known vulnerabilities.

Once the virtual environment is built, you generally don't need the package manager (like uv, Poetry, or PDM) for InkStitch's runtime functionality.

You're right, environment setup isn't always the only bottleneck. However, uv addresses crucial areas that lead to hidden issues later. Like other modern managers, uv's uv.lock guarantees reproducible builds across all environments. Combined with its .python-version integration, this simplifies Python management and provides a strong foundation of stability and consistency, saving us significant debugging time and making our CI/CD far more reliable in the long run, contributing to overall faster project progress.

The problem is that rust packages use specific commits of dependencies. These need not be updated. NPM from javascript does the same, though at least has vulnerability tracking.

For a study in the Android ecosystem see https://media.ccc.de/v/38c3-ultrawide-archaeology-on-android-native-libraries

Thank you so much for this valuable information and for sharing the insightful video – it was indeed very educational!

You've highlighted a crucial point regarding dependency management, particularly the use of specific commits. You're absolutely right that in Rust's Cargo.toml (its dependency manifest), one can specify dependencies not just by version ranges (e.g., ^1.0.0), but also by exact Git commits, branches, or paths. This approach is often used for unreleased features, internal libraries, or when a project needs a very specific, immutable version of a dependency.

We actually employ a very similar practice in InkStitch ourselves (e.g., in requirements.txt you might see inkex @ git+https://gitlab.com/inkscape/extensions.git@1792934a09046fdef8aab20d05aad9c47f825bf5).

As you rightly pointed out, when a dependency is pinned to an exact commit, it means that even if a new version of that dependency is released (perhaps with a security fix), your project won't automatically pick it up. You would indeed need to manually update the commit hash in your configuration (whether Cargo.toml, Cargo.lock, or our requirements.txt) and then re-evaluate. This is certainly a significant inconvenience and auditing tools might not always detect vulnerabilities in such highly specific, non-standard versions.

However, this isn't a challenge exclusive to Rust. In any programming language, one can choose to download and use an arbitrary commit of any library, which would similarly prevent automatic updates. It's more a matter of project approach, best practices, and the deployment of appropriate tools and processes to either completely exclude such issues or at least reasonably minimize them.

Your insights are invaluable as we strive to make InkStitch's build process as robust and secure as possible. Thank you again for sparking this important discussion!

We have been trying to reduce dependencies in Ink/stitch. Inkex is developed as part of Inkscape. Some of the newer unreleased features are used in Ink/stitch. The hope is to encourage Inkscape to do minor releases of Inkex between major releases of Inkscape as projects dependent on it often are incompatible with most recent release. In the process of making inkstitch-pyembroidery available on pypi. Colormath2 was forked from colormath as it was not receiving updates and besides its use in Ink/stitch may also find use elsewhere.

That's an excellent summary of our current situation with dependencies, and you've highlighted some key challenges.

Indeed, InkStitch's dependency choices are often pragmatic, dictated by the upstream development and release cycles of critical components like inkex. We're actively working towards improving our dependency hygiene, for instance, by publishing inkstitch-pyembroidery to PyPI and by forking colormath2. We also hope that upstream projects will adapt to facilitate better dependency management, perhaps through more frequent minor releases for inkex. This context is crucial for understanding why we sometimes deviate from standard versioning practices and use specific Git commits.

Another significant challenge arises with certain packages that depend not just on the operating system but also on the specific OS version, such as PyGObject (used by both inkex and wxPython). For example, versions greater than >3.50 can't be used on Ubuntu 22.04 and older.
Newer PyGObject versions require newer underlying system libraries (such as libgirepository-2.0). If these aren't available or installable on older operating system releases, PyGObject won't function, indicating a dependency on the system's C libraries rather than an issue with PyGObject's Python code itself.

The difficulty here is that pyproject.toml doesn't currently allow for detecting the specific OS version in use, nor is this functionality planned for inclusion (see PEP 621 and PEP 508). So far, we've been fortunate that version 3.50 appears to remain compatible with newer OS versions (like Ubuntu 24.04), but I haven't exhaustively checked this, and it might not necessarily hold true indefinitely.

Another significant challenge arises with certain packages that depend not just on the operating system but also on the specific OS version, such as PyGObject (used by both inkex and wxPython). For example, versions greater than >3.50 can't be used on Ubuntu 22.04 and older. Newer PyGObject versions require newer underlying system libraries (such as libgirepository-2.0). If these aren't available or installable on older operating system releases, PyGObject won't function, indicating a dependency on the system's C libraries rather than an issue with PyGObject's Python code itself.

The difficulty here is that pyproject.toml doesn't currently allow for detecting the specific OS version in use, nor is this functionality planned for inclusion (see PEP 621 and PEP 508). So far, we've been fortunate that version 3.50 appears to remain compatible with newer OS versions (like Ubuntu 24.04), but I haven't exhaustively checked this, and it might not necessarily hold true indefinitely.

What OS do you run Ink/Stitch on? It would be nice to have it correctly packaged for Ubuntu. Am doing this for Fedora, so much of the effort can be reused once inkstitch-pyembroidery is available on PyPI. At the moment, many system libraries are distributed with official GNU/Linux releases and expect not everyone will want to use a development installation. There is still the question of what operating systems and operating system versions to support, but this depends on community interest - it is nice that releases are made for Linux, Mac and Windows, but it maybe helpful to know what people are willing to test on as developers can only do so much for operating systems they do not use or use regularly.

Another significant challenge arises with certain packages that depend not just on the operating system but also on the specific OS version, such as PyGObject (used by both inkex and wxPython). For example, versions greater than >3.50 can't be used on Ubuntu 22.04 and older. Newer PyGObject versions require newer underlying system libraries (such as libgirepository-2.0). If these aren't available or installable on older operating system releases, PyGObject won't function, indicating a dependency on the system's C libraries rather than an issue with PyGObject's Python code itself.
The difficulty here is that pyproject.toml doesn't currently allow for detecting the specific OS version in use, nor is this functionality planned for inclusion (see PEP 621 and PEP 508). So far, we've been fortunate that version 3.50 appears to remain compatible with newer OS versions (like Ubuntu 24.04), but I haven't exhaustively checked this, and it might not necessarily hold true indefinitely.

What OS do you run Ink/Stitch on? It would be nice to have it correctly packaged for Ubuntu. Am doing this for Fedora, so much of the effort can be reused once inkstitch-pyembroidery is available on PyPI. At the moment, many system libraries are distributed with official GNU/Linux releases and expect not everyone will want to use a development installation. There is still the question of what operating systems and operating system versions to support, but this depends on community interest - it is nice that releases are made for Linux, Mac and Windows, but it maybe helpful to know what people are willing to test on as developers can only do so much for operating systems they do not use or use regularly.

Currently, I'm primarily on Linux Mint Xfce (based on Ubuntu 22.04), with other OS testing done in VirtualBox. I've spent some time with PyGObject and haven't found a way to add libgirepository-2.0 to Ubuntu 22.04 or older without a full system upgrade to Ubuntu 24.04.

This creates a challenge for dependency management:

• If pyproject.toml lists pygobject without a version constraint, uv (or any other package manager) will automatically pull the latest version (currently 3.52), which doesn't work on older Ubuntu releases.
• Even if I pre-install an older pygobject version (e.g., using uv pip install pygobject<=3.50) before synchronizing with pyproject.toml, the inkex dependency (which specify a pygobject>=3.36 version constraint) will cause the latest 3.52 to be installed, overwriting the pre-installed version, and again, it fails. pyproject.toml takes precedence over manual installations.

My initial thought was to generate pyproject.toml dynamically, but that runs into the problem that package managers expect pyproject.toml to be static and consistent across all operating systems.

One potential workaround is to remove inkex and pygobject from pyproject.toml (at least for Linux users) and then pre-install specific versions of pygobject and inkex (via uv pip install ...) before the main pyproject.toml synchronization. It might be possible to enforce the versions this way, but it feels like a less robust solution compared to proper pyproject.toml declarations.

Currently, I'm primarily on Linux Mint Xfce (based on Ubuntu 22.04), with other OS testing done in VirtualBox. I've spent some time with PyGObject and haven't found a way to add libgirepository-2.0 to Ubuntu 22.04 or older without a full system upgrade to Ubuntu 24.04.

Helped create build for 32 Bit Linux Mint Debian edition, so can create an unofficial deb package. Am not a Debian/Ubuntu packager though. Would you consider becoming one?

This creates a challenge for dependency management:

• If pyproject.toml lists pygobject without a version constraint, uv (or any other package manager) will automatically pull the latest version (currently 3.52), which doesn't work on older Ubuntu releases.
• Even if I pre-install an older pygobject version (e.g., using uv pip install pygobject<=3.50) before synchronizing with pyproject.toml, the inkex dependency (which specify a pygobject>=3.36 version constraint) will cause the latest 3.52 to be installed, overwriting the pre-installed version, and again, it fails. pyproject.toml takes precedence over manual installations.

My initial thought was to generate pyproject.toml dynamically, but that runs into the problem that package managers expect pyproject.toml to be static and consistent across all operating systems.

One can query the operating system and install appropriate packaged library versions. Package managers provide a better user experience though for someone that is not a developer.

Helped create build for 32 Bit Linux Mint Debian edition, so can create an unofficial deb package. Am not a Debian/Ubuntu packager though. Would you consider becoming one?

That's a very kind offer, and I truly appreciate you asking!
I must admit that my experience with creating official distribution packages (like .deb files for Debian/Ubuntu) is very limited. I've only engaged in packaging efforts out of absolute necessity and in exceptional cases, rather than as a regular practice. To be honest, it often feels like a bit of a "black box" to me; I'm more accustomed to fixing existing packages than creating new ones from scratch.
I'm not sure if maintaining .deb packages would be an area I'd genuinely enjoy or excel at in the long term, and I don't believe I'd be the right person to take on those responsibilities at this time.

It seems the SignPath.io signing limit has been exhausted. We have some restrictions there? Do we have restrictions for other signing types as well?

Signing on windows needs manual approval and also we do have a limit. That's why we only sign (beta) releases.

On macOS we sign everything, but only notarize on demand or for releases.

Thank you for the explanation. I am entirely unfamiliar with these signing and notarization mechanisms on Windows and macOS, as I have not previously encountered such processes.

To summarize the signing processes:

1. Windows Signing

• Sign only release versions (like beta releases, each v* tag).
• Both the program itself and its installer must be signed.
• The signing process requires manual approval, and there is a limit on the number of times they can sign.

2. macOS Signing and Notarization (Apple security scan)

• Sign every build; there is no limit.
• Notarize only for releases (like beta releases, each v* tag) and on demand.
  • What does "on demand" mean? It means the notarization process is initiated manually or by a specific request, rather than automatically for every single build.
• Both the program itself and its installer must be notarized.

3. Linux Signing

• Generally, all builds are signed.

Notes for discussion/reconsideration:

• For builds on forks of InkStitch, the entire signing process needs to be disabled due to lack of access to the necessary secrets.
• Local builds are always performed without signing.
• Does "on-demand signing" imply initiating a build with special parameters?
• Could we implement on-demand signing in a separate workflow? This would allow builds with limited use (e.g., development or specific tests) to always be created without the complexities of full signing or notarization.
  • Signing would only occur once the packages are verified to be installable and functional.
• Should we also consider enabling on-demand signing for Windows?
• How can we perform a test signing to verify the entire build process, including the signing step?

Just checking in: are there any other aspects that still need to be verified or tested? My aim is to avoid introducing too many new features at this stage, so we can keep close to our existing workflows.

I think this needs more work on the documentation. As mentioned above, I'd like to not have a draft file describing the changes, but rather have straight forward guides on how things are done. I guess this is still work in progress, considering the empty PYPROJECT_UV_BUILD file. So I might be a little bit fast with this comment.

You're right. I'm sorry about that!

I renamed the original file to DEVELOPMENT_UV_GUIDE.md but forgot to delete the empty PYPROJECT_UV_BUILD.md file, which caused the confusion. I'll remove the old file to clear this up.

see DEVELOPMENT_UV_GUIDE.md

Thanks :) ... Although I still think that we need to improve the documentation here ...

I find the pyproject draft file very confusing. I'd really like the description to focus on how things are done and less on what has changed - which won't be interesting after this branch got merged. To have to read through it when you just search for a guide could be a bit annoying as the last headlines become important again (release, release testing). So this makes it harder to find the bits that are relevant for future developers. Also I think this file could use an other name - or even needs to be split up.

Still it makes sense to document the changes somewhere. Maybe in this thread.

Thank you so much for the feedback! It’s incredibly helpful, and I completely agree with your points.

The DRAFT file was created as a working log for my own process. Since the solution is still very much in flux, I felt it was premature to create a final, polished guide. I needed a single place to track my progress, and a discussion thread didn't seem practical for that.

Your fresh perspective is exactly what I need. I'm so deep in this that many things seem obvious to me, so a collaborative approach would be most effective. My plan is to first build out a clear guide for developer users based on your feedback. After that's finalized, a separate guide for github action users on creating a release would be next.

Ultimately, I will copy the final contents of DRAFT here into the thread and then remove the file from the repository.

What are the files in bin/msys2 for? They seem generic, not related to UV or Ink/Stitch?

Would there be a possibility of considering meson-python instead of uv?

The dependencies are on PyPI, so publishing to PyPI when there is a release would be helpful. Some extensions which
are on PyPI:

• https://pypi.org/project/inkcut/
• https://pypi.org/project/svg2tikz/
• https://pypi.org/project/inkscape-scripting/

PyInstaller builds could the be done weekly and when there is a release rather than on every pull request.

Would there be a possibility of considering meson-python instead of uv?

The dependencies are on PyPI, so publishing to PyPI when there is a release would be helpful. Some extensions which are on PyPI:

* https://pypi.org/project/inkcut/

* https://pypi.org/project/svg2tikz/

* https://pypi.org/project/inkscape-scripting/

PyInstaller builds could the be done weekly and when there is a release rather than on every pull request.

You can easily install meson-python using uv with this command:

uv pip install meson-python

and there shouldn't be any conflicts, as uv fully complies with the PEP 517 standards used by meson-python.
Just keep in mind that both projects are being rapidly developed.

What are the files in bin/msys2 for? They seem generic, not related to UV or Ink/Stitch?

The files in the bin/msys2 directory are included to simplify the installation process on Windows. They are generic tools (specifically from the MSYS2 environment) that provide a necessary Unix-like build environment for compiling and running native components. I include them so the user doesn't have to manually locate or configure them, as Windows installation is often less straightforward than on Linux or macOS.
I'm considering placing these files in a more dedicated folder, like install/windows or a similar structure, to better separate these generic utilities from the main project code.

PyInstaller builds could the be done weekly and when there is a release rather than on every pull request.

We have a number of non techy users who enjoy to test new features or specific bug fixes. We definitely don't want to exclude them from the testing area. Therefore I think it is actually great to be able to link to a specific branch release and ask for testing and I definitely don't want to lose this possibility.

PyInstaller builds could the be done weekly and when there is a release rather than on every pull request.

We have a number of non techy users who enjoy to test new features or specific bug fixes. We definitely don't want to exclude them from the testing area. Therefore I think it is actually great to be able to link to a specific branch release and ask for testing and I definitely don't want to lose this possibility.

Ok.