Skip to content

Conversation

@tigarmo
Copy link
Collaborator

@tigarmo tigarmo commented Mar 4, 2024

This commit adds code in the post-prime callback to handle the specific combination of "ubuntu@24.04" base + "python" plugin. The handling detects and removes a primed "lib64" symlink to "lib", which can't stay because it breaks the "lib64" symlink to "usr/lib64" on the base layer, making the rock unusable.

Read on for more context on why this started happening now:

The Python plugin works by creating a virtual environment on the part's install dir (which eventually gets primed). By default, the creation of the virtual environment will setup a directory structure like this:

<install-dir>
     |- bin/
     |- lib/
     |- ...
     |- lib64 -> lib

That is, the creation itself places a "lib64" symlink pointing to "lib". If this gets primed as-is, this symlink will override the base layer's own symlink of "lib64" to "usr/lib64", which renders the rock unusable because all binaries expect the loader to exist in "lib64/ld-linux-x86-64.so.2", when the loader's "true" location is in "usr/lib64/ld-linux-x86-64.so.2".

This wasn't an issue prior to 24.04 because of the way we extract stage packages: since the base layer doesn't include the Python interpreter, rockcraft projects need to provision Python themselves, typically via the "python3-venv" stage-package. This causes the inclusion of the package's dependencies, ultimately pulling in "libc6".

On Ubuntu versions older than 24.04, the "lib64" package includes the "ld-linux-x86-64.so.2" loader as "/lib64/ld-linux-x86-64.so.2". This means that by the time the virtual environment creation starts the install dir already has a "lib64" directory, which I guess the venv just accepts and leaves alone. The end result is that "lib64" is not a symlink to "lib", and the usrmerge handling code that we already have when packing takes care of packing the files in "lib64" as inside "usr/lib64". The install dir looks like this:

<install-dir>
     |- bin/
     |- lib/
     |- ...
     |- lib64/
          |- ld-linux-x86-64.so.2

However, 24.04 changed the usrmerge handling and now the libc6 package includes that loader as "/usr/lib64/ld-linux-x86-64.so.2", which means that the "lib64" directory doesn't exist anymore when the venv is created. Thus, "lib64" is created as a symlink to "lib" and breaks the base layer:

<install-dir>
     |- bin/
     |- lib/
     |- ...
     |- lib64 -> lib
     |- usr
          |- lib64
               |- ld-linux-x86-64.so.2

Package listings:
Jammy: https://packages.ubuntu.com/jammy/amd64/libc6/filelist
Noble: https://packages.ubuntu.com/noble/amd64/libc6/filelist


This commit adds code in the post-prime callback to handle the specific
combination of "ubuntu@24.04" base + "python" plugin. The handling detects
and removes a primed "lib64" symlink to "lib", which can't stay because
it breaks the "lib64" symlink to "usr/lib64" on the base layer, making the
rock unusable.

Read on for more context on *why* this started happening now:

The Python plugin works by creating a virtual environment on the part's
install dir (which eventually gets primed). By default, the creation of
the virtual environment will setup a directory structure like this:

```
<install-dir>
     |- bin/
     |- lib/
     |- ...
     |- lib64 -> lib
```

That is, the creation itself places a "lib64" symlink pointing to "lib".
If this gets primed as-is, this symlink will override the base layer's
own symlink of "lib64" to "usr/lib64", which renders the rock unusable
because all binaries expect the loader to exist in "lib64/ld-linux-x86-64.so.2",
when the loader's "true" location is in "usr/lib64/ld-linux-x86-64.so.2".

This wasn't an issue prior to 24.04 because of the way we extract stage
packages: since the base layer doesn't include the Python interpreter,
rockcraft projects need to provision Python themselves, typically via
the "python3-venv" stage-package. This causes the inclusion of the package's
dependencies, ultimately pulling in "libc6".

On Ubuntu versions older than 24.04, the "lib64" package includes the
"ld-linux-x86-64.so.2" loader as "/lib64/ld-linux-x86-64.so.2". This means that
by the time the virtual environment creation starts the install dir already
has a "lib64" directory, which I guess the venv just accepts and leaves alone.
The end result is that "lib64" is not a symlink to "lib", and the usrmerge
handling code that we already have when packing takes care of packing the
files in "lib64" as inside "usr/lib64". The install dir looks like this:

```
<install-dir>
     |- bin/
     |- lib/
     |- ...
     |- lib64/
          |- ld-linux-x86-64.so.2
```

However, 24.04 changed the usrmerge handling and now the libc6 package includes
that loader as "/usr/lib64/ld-linux-x86-64.so.2", which means that the "lib64"
directory doesn't exist anymore when the venv is created. Thus, "lib64" is
created as a symlink to "lib" and breaks the base layer:

```
<install-dir>
     |- bin/
     |- lib/
     |- ...
     |- lib64 -> lib
     |- usr
          |- lib64
               |- ld-linux-x86-64.so.2
```
@tigarmo tigarmo marked this pull request as ready for review March 4, 2024 15:18
@tigarmo tigarmo requested review from lengau, mr-cal and sergiusens March 4, 2024 16:13
Copy link
Collaborator

@lengau lengau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

Copy link
Contributor

@mr-cal mr-cal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed explanation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants