Oro's Stuff

I lost my mind trying to bootstrap rustc with mrustc and you will too

2023-06-24T00:00:00+00:00

I SPENT THE LAST 10 HOURS TRYING TO BUILD RUSTC AND HOLY FUCK EVERYTHING IS SO OLD AND HACKY AND FUCKED UP THAT I DONT KNOW WHAT JUST HAPPENED BUT NOW YOURE GOING TO GO THROUGH IT WITH ME AGAIN

THATS RIGHT

IM GOING TO SUFFER THROUGH BUILDING RUSTC FROM SOURCE AGAIN, WITHOUT RUSTC ITSELF, WITHOUT THE ORIGINAL RUST COMPILER, ALL FOR THIS FUNNY INTERNET BLOG POST THAT FIVE PEOPLE WILL READ

UHUH

BUILDING RUSTC WITHOUT RUSTC AND LOSING MY SANITY AGAIN IN THE PROCESS

for the love of god and all that is holy please let this go well

GETTING STARTED</h2>
TO START US OFF lets make ourselves a container
THATS RIGHT A CONTAINER
WE GONNA GET GOOFY TODAY
distrobox create -n CrystalBoxFUCK -i registry.getcryst.al/crystal/misc/docker:latest --home $HOME/distroboxes/CrystalBoxFUCK </code></pre> Why this niche distro in particular you might ask? No, because FUCK YOU NOW LETS GET STARTED WITH MRUSTC and ENTER THE CONTAINER distrobox enter CrystalBoxFUCK git clone https://github.com/thepowersgang/mrustc.git cd mrustc git checkout b4503ee66847581e0483b6cff0ebc3a3d99fb4ff # because we don't want you cheating and taking advantage of any fixes until we fix them ourselves, now do we? </code></pre> NOW lets BUILD IT. We'll go with 1.39, because thats what I used first and I don't want to spend ANOTHER TEN HOURS finding what needs to be include</code>d ./build-1.39.0.sh </code></pre> wait fuck we need to download the base build dependencies or something and whatever the fuck else ame ins cmake python --noconfirm </code></pre> NOW if you wait a while for it all to build and fuck around youll see that mrustc likes to complain when it's not on <= gcc12 lets tell it "fucking take it, I don't want this include</code>" we're gonna add #include <cstdint></code>: src/mir/mir.hpp</code> /* * MRustC - Rust Compiler * - By John Hodge (Mutabah/thePowersGang) * * mir/mir.hpp * - MIR (Middle Intermediate Representation) definitions */ #pragma once #include <tagged_union.hpp> #include <vector> #include <string> #include <memory> // std::unique_ptr #include <hir/type.hpp> #include "../hir/asm.hpp" #include <int128.h> #include <cstdint> </code></pre> now lets continue building again with ./build-1.39.0.sh</code> OH YOURE GETTING SMART WITH US YOU WANT ANOTHER INCLUDE src/hir/type.cpp</code> /* * MRustC - Rust Compiler * - By John Hodge (Mutabah/thePowersGang) * * hir/type.cpp * - HIR Type helper code */ #include "type.hpp" #include <span.hpp> #include "expr.hpp" // Hack for cloning array types #include <cstdint> </code></pre> NOW. AGAIN. LETs BUILD oh. my. fucking. god. one more, alright. tools/common/toml.h</code> /* * mrustc common tools * - by John Hodge (Mutabah) * * tools/common/toml.h * - A very basic (and probably incomplete) streaming TOML parser */ #pragma once #include <fstream> #include <vector> #include <string> #include <unordered_map> #include <cstdint> </code></pre> would you believe me if I told you this is the last patch needed? no? well, I don't blame you. but it is anyways, so we're lucky. now we can actually build rustc. building rustc</h2> We're building core, and the process only just begins. We get through the first couple dozen steps, alright, alright, that's good. wait till we get to LLVM. Then the fun begins. now wait for a long fucking time until it also complains about some compile errors about unused imports. what will we do about this? FUCK IT LETs PATch IT AGAIN rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/benchmark_register.h</code> #ifndef BENCHMARK_REGISTER_H #define BENCHMARK_REGISTER_H #include <vector> #include <cstdint> #include "check.h" </code></pre> Now the final error is upon us. After this, it's a clear road, all sunshine and rainbows. openssl</h3> nooOOOOOOOOOOOOOOOOOOOOOOOOOoooOOOOO I;'m not even going to begin to start with openssl, so take this PKGBUILD and FUCK OFF pkgname=openssl-1.1 _ver=1.1.1q # use a pacman compatible version scheme pkgver=${_ver/[a-z]/.${_ver//[0-9.]/}} pkgrel=1 pkgdesc='The Open Source toolkit for Secure Sockets Layer and Transport Layer Security' arch=('x86_64') url='https://www.openssl.org' license=('custom:BSD') depends=('glibc') makedepends=('perl') optdepends=('ca-certificates' 'perl') replaces=('openssl-perl' 'openssl-doc') backup=('etc/ssl/openssl.cnf') source=("https://www.openssl.org/source/openssl-${_ver}.tar.gz") sha256sums=('SKIP') validpgpkeys=('SKIP') prepare() { cd "$srcdir/openssl-$_ver" } build() { cd "$srcdir/openssl-$_ver" # mark stack as non-executable: http://bugs.archlinux.org/task/12434 ./Configure --prefix=/opt/openssl-1.1 --openssldir=/etc/ssl-1.1 --libdir=/opt/openssl-1.1/lib \ shared no-ssl3-method enable-ec_nistp_64_gcc_128 linux-x86_64 \ "-Wa,--noexecstack ${CPPFLAGS} ${CFLAGS} ${LDFLAGS}" make depend make } package() { cd "$srcdir/openssl-$_ver" make DESTDIR="$pkgdir" MANSUFFIX=ssl install_sw install_ssldirs } </code></pre> I literally just took the Arch openssl pkgbuild</a> and hacked it together to make it work. and it works. so run makepkg -sif</code> and leave me alone. Oh yeah and then run set -x OPENSSL_DIR /opt/openssl-1.1/</code> (or export OPENSSL_DIR=/opt/openssl-1.1</code>)to make rustc see openssl, for some reason it refuses to see it otherwise. I'm too lazy to debug that. The End.</h2> You made it. Congratulations. Now you can work yourself up to the latest Rust compiler, cutting out a lot of the versions that came before, making it faster to get a fully working Rust env from scratch, no weird binaries needed. This was so annoying to work with, and now I have to do it for even more Rust versions. Holy fuck. At least 10 hours wasted, easily. But it all looked so simple, right? Nope. I pulled an all-nighter trying to get this to work :P Hopefully, in the future, mrustc can more easily go right to the newest/newer Rust versions so you don't need to rebuild too much. In the meantime, have fun with this documentation regarding how I lost my mind and needed hours of sleep just to get rustc building. Some things to keep in mind, from the mrustc developer: If you're building a rustc for use, then build 1.54 and step up from there I keep 1.19/1.29/1.39 as build targets because they're good test coverage (and someday I'll get time to add 1.69) </blockquote> So progress :) I'll be submitting some patches upstream to help with mrustc, including this one, which involves the mrustc changes listed here: https://github.com/thepowersgang/mrustc/pull/305</a>



Flatpak - an insecurity nightmare
2023-06-15T00:00:00+00:00
Not to be confused with Flatpack</a>.</em></p>
For over 20 years, Linux has been used everywhere, from servers to embedded devices. One area it has been lacking in, however, is the desktop space. Windows and macOS have constantly been keeping Linux out of the market, with gaming primarily being a Windows thing and content creators utilizing macOS.</p>
Over the last decade, the Linux desktop has been improving dramatically: Flatpak, Wayland, Portals, PipeWire, and FreeDesktop have all been working to make Linux an operating system better suited to modern (desktop) systems. I'm not going to go over all of those though; as you could probably tell by the title, this is going to be about Flatpak!</em></p>
History</h2>
Let's take a quick look at the server space. It's not uncommon to see them running decade-old (sometimes even decades</em>-old!) kernels and libraries. This makes sense, as their boot times are long and on every single system update, there's a risk of breakage, and needing to debug that; valuable uptime lost, especially for a service as large as Google. Naturally, they evolved a way to quickly patch the most relevant parts of the system, so security isn't too much of an issue.</p>
However, the software they run may only be able to utilize newer features from newer libraries, not the old and crusty software the servers run. So how did they solve that?</p>
Containers. Normally used to ship software to several customers in a consistent and safer way, they allow you to isolate software into a little box with all of the needed dependencies, at their specified versions, with limited access to the host system.</p>
The present</h2>
We have Android, macOS, and iOS, all with limited access to the system, requiring end-user intervention to access certain resources. But what about the Linux desktop?</p>
In 2015, Flatpak was released, taking all of the technology used from containerization on servers, and applying it to desktop systems. With it came Portals</a>, and the Freedesktop SDK</a>. This gave a consistent and generic platform and API to developers, a drastic change from what was previously a battleground of varying dependency versions, core system libraries, and packaging formats.</p>
Of course, after having the distribution and development situation stay the same for a few decades, this was too large of a change to make at once. Many of the needed APIs exist, but developers need to take time out of their day to migrate to them, and some users (specifically, Flatpak-hating ones) vehemently refuse to help improve the situation, insisting we stick with the old way of doing things, and repackage software hundreds of times for thousands of varying systems.</p>
To put it simply, naturally, nobody in their right mind gave a shit. Many new application developers are creating their apps with a confined environment in mind, and shipping their software to Flathub, allowing the software to get to end users in a fast and secure way.</p>
Application confinement</h3>
What about the old applications, though? You can find 10-year-old apps shipped on Flathub, after all. From these applications spawned articles such as Flatpak - a security nightmare</a> hey, you might recognize that name!</em>, and various responses from well-known individuals</a> within the FOSS space intended to counteract misinformation and aggression from those articles.</p>
For these apps, the FUD articles like to cherry-pick the worst offenders and use this to say that Flatpak isn't worth pushing for at all. So I'd like to address that.</p>
These applications were designed with an unconfined environment, and the APIs necessary for them either didn't exist at the time or they weren't updated to use the new API. Cherry-picking them to show that the Flatpak sandbox is "not a sandbox"</a> is not a valid argument, and only shows that the applications themselves are insecure.</p>
Of course, older apps aren't the only offenders. As the new APIs might not be mature enough, both newer and older apps may have to resort to static permissions</a> in order to work properly. For most newer applications, this is fine, as they can restrict their permissions a lot</em> more than older applications would be able to.</p>
Flatpak is</em> a sandbox. And Flatpak is</em> secure. The applications</em> are not</a>. It doesn't help anyone to change the meaning of the word "sandbox". Just because the sandbox that some apps use isn't on par with iOS or Android, it doesn't disqualify it as a sandbox. If you'd like to blame Flatpak for that, don't</em> be my guest. The fact that the apps rely on static permissions shows that the applications are insecure and rely on inconsistent interfaces, and in some cases, that the current APIs have yet to improve enough for their use cases</a>. We should be focusing on improving that situation, not trying to tear it down.</p>
If you want a real-world example of how a partial sandbox is still effective: the "fractureiser" malware</a>. Even if the malware was nonfunctional on Linux in the first place, it's still an example of how traditional packaging fails to address the fact that modern desktop systems have plenty of ways for a system to be compromised. Running software "that you trust" isn't enough to protect yourself. To quote the relevant notes from the linked post:</p>

If you use Linux, use the PrismLauncher Flatpak</a> and be very careful of what you give it access to through utilities such as Flatseal. By default, it has no access to the needed directories to compromise the system.</p>
</blockquote>

Taking a quick look over the decompiled source code, it will indeed fail to function inside of the default PrismLauncher Flatpak sandbox; the current malware hardcodes the user's ~/.config/ directory. The creation of files inside of the Flatpak sandbox, if the app does not have access to that real path, will result in it being written to a tmpfs that gets wiped on a sandbox restart. systemd is also not available inside of the Flatpak sandbox, so executing that command will fail. The malware seems to not attempt to work around these limitations, and assumes it is running unsandboxed.</p>
</blockquote>
Disclaimer: I wrote the following notes :)</em></p>
For a more in-depth view of static permissions and Portals, see Overview of Flatpak’s Permission Models</a> by TheEvilSkeleton</a>.</em></p>
Outdated dependencies</h3>
Some also like to bring up outdated dependencies to the argument. No, this doesn't matter. The runtime itself is updated whenever a needed security update is available (sometimes even breaking the ABI</a> if it's a big enough deal) and application developers don't have to think about manually updating bundled dependencies with automatic update checking</a>. And many major libraries, such as your GPU drivers, can be updated separately from your runtime if you so choose. Outdated dependencies isn't even an argument in the first place.</p>
In the event an application does</em> use outdated libraries, be it because an update broke the build process or the developer forgot to set up the data checker, there is the sandbox placed around all Flatpak applications that can help mitigate these issues while the developer addresses them.</p>
The future</h2>
The hope is for the Linux desktop to migrate to developer- and more sandbox-friendly methods of software distribution and programming. We've already made great strides with PipeWire, which paves the way for many</em> nice features with audio and video (including several apps using your camera at the same time!), and FreeDesktop, but we undoubtedly have a ways to go before Linux is a viable platform for developers to target and ship software to.</p>
We can't move forward without modern software like Wayland and Flatpak, no matter how much one might want to repair the situation with Xorg and traditional packaging. If we keep trying to revert to the old way of doing things, all that's shown to developers is that Linux isn't a platform worth considering, and it ends up being a worse situation for end users; turning back into a "which came first: the chicken or the egg?" situation, and resulting in a very</em> bad UX.</p>
We don't want that happening.</p>
More reading</h2>
If you want more information on Flatpak and in general more information on how it works and the current situation, the following articles are worth a read:</p>


Response to flatkill.org - TheEvilSkeleton</a></p>
</li>

Response to “Flatpak Is Not the Future” - TheEvilSkeleton</a></p>
</li>

Traditional Packaging is not Suitable for Modern Applications - TheEvilSkeleton</a></p>
</li>

Overview of Flatpak’s Permission Models - TheEvilSkeleton</a></p>
</li>

Response to “Developers are lazy, thus Flatpak” - TheEvilSkeleton</a></p>
</li>

Distribution packaging for Linux desktop applications is unsustainable - memoryfile</a></p>
</li>

You are not actually mad at Flatpak - Jordan Petridis</a></p>
</li>
</ul>


*[FUD]: fear, uncertainty, and doubt</p>