Hi @pstark-code, please sign off the commit.

signed-off and rebased on top of the upstream default branch.

The PR might not be enough since some paths in the longhorn-manager or other component images are hardcoded with the /var/lib/longhorn prefix.

The PR might not be enough since some paths in the longhorn-manager or other component images are hardcoded with the /var/lib/longhorn prefix.

This was brought up in multiple places. Please read my response. This change does not change the path it gets mounted at in the container. So for the Longhorn Manager software, nothing changes.

I also added the ability to configure extra volumes to be mounted into the longhorn-manager container. This mirrors the longhorn option to have multiple "disks".

The PR might not be enough since some paths in the longhorn-manager or other component images are hardcoded with the /var/lib/longhorn prefix.

This was brought up in multiple places. Please read my response. This change does not change the path it gets mounted at in the container. So for the Longhorn Manager software, nothing changes.

The binaries stored in the host’s /var/lib/longhorn/engine-binaries are managed by the engineImage controller (code) and the EngineImage DaemonSet.

The change introduced in the PR makes the spec.volumes['longhorn'].hostPath.path of the longhorn-manager DaemonSet configurable. However, the EngineImage DaemonSet is not aware of this change, which results in the controllers being unable to locate the binaries at the expected path inside the container (/var/lib/longhorn/engine-binaries). Thus, updating the longhorn-manager codes is still required.

The binaries stored in the host’s /var/lib/longhorn/engine-binaries are managed by the engineImage controller (code) and the EngineImage DaemonSet.

Ah! That explains why I couldn't figure out how they get there!

Okay, then I will remove the configurable hostPath and leave the extraMounts and extraVolumeMounts options, because that, together with the default settings.defaultPath resolves all the issues I was looking to address.

Just as a curious question: why are the engine binary fetching containers not part of the longhorn-manager pods? InitContainers maybe? It seems unintuitive to have this tight coupling between two seemingly disparate DaemonSets.

Okay, then I will remove the configurable hostPath and leave the extraMounts and extraVolumeMounts options, because that, together with the default settings.defaultPath resolves all the issues I was looking to address.

Is narrowing down the scope to extraMounts and extraVolumeMounts enough for addressing the issue?

Just as a curious question: why are the engine binary fetching containers not part of the longhorn-manager pods? InitContainers maybe? It seems unintuitive to have this tight coupling between two seemingly disparate DaemonSets.

I think this is due to Longhorn can only one longhorn-manager daemonset but can support multiple engineimage daemonsets. Longhorn can support multiple egnineimages simutaneously.

Is narrowing down the scope to extraMounts and extraVolumeMounts enough for addressing the issue?

I believe so. I wanted to resolve two things:

1. I thought it was a silly limitation to require a very specific path on the host to contain longhorn things.

• but as it turns out there is a good reason to not change this. Changing that would require far-reaching refactoring within the longhorn-manager, as far as I can tell

2. I was trying to work around a talos-specific issue I was having.

• Context: Talos runs the kubelet inside a container and so any paths that need to be visible to the kubelet need to be bind-mounted into that namespace. This gave me some issues with the new UserVolume mechanism for exposing extra disks to talos nodes. And the only reason I was trying to do that is because the longhorn chart did not have any other way to access extra storage.
• The extra disks are already automatically available inside the Talos kubelet namespace at /var/mnt/${user-volume-name}. So if I can just specify another Volume with that hostPath, and then add a longhorn disk with that path for storage, everything should work. (or just set it via the defaultSettings.defaultDataPath)

To summarize: Yes I think limiting the scope will be sufficient. In fact, I think making the hostPath of /var/lib/longhorn configurable is useless and will fundamentally (although temporarily) break longhorn deployments.

Talos runs the kubelet inside a container and so any paths that need to be visible to the kubelet need to be bind-mounted into that namespace.

• If all paths used by longhorn-manager should be bind mounted to the kubelet container, this change is probably not enough since longhorn-manager hard codes some paths when operating files, e.g., the socket files

if I can just specify another Volume with that hostPath, and then add a longhorn disk with that path for storage, everything should work. (or just set it via the defaultSettings.defaultDataPath)

• If you are trying to use a customized data path/disk path as Longhorn Disk, the main ways are modifying the customized setting before launching Longhorn or editing the Longhorn Node after launching Longhorn. IIRC, changing the volume host path mounted by longhorn-manager won't help automatically add it as Longhorn Disk, because it's InstanceManager Pod being responsible for managing those data paths.