casraf.dev

Flutter Tutorial - Creating a Wheel Spinner Widget

Fri, 22 Mar 2019 10:00:00 GMT

Flutter is an amazing framework for app development, and gaining popularity every second. Being component and composition based, and with the fact that it includes many built-in widgets to work with, it’s really easy to create beautiful widgets, either simple or complex.

Today we’ll be learning how to create a real-world case widget for updating a numeric value - a pitch-knob-like control. Like the ones you can usually find in a studio. It will also support letting the finger go to keep rolling in the same direction, and using the velocity it was left with.

In this tutorial, we will:

Create a stateful widget
Use GestureDetector to detect scroll gestures for us, and update the callbacks appropriately.

This tutorial assumes you have some experience with Flutter and understand basic Widget composition and basic state management.

Step 1: Create the Widget

We’ll start basic. We have one base stateful widget, which will hold everything. In this example, I’ve already added some parameters we will need to accept:

class WheelSpinner extends StatefulWidget {
  final double max;
  final double min;
  final double value;
  final Function(double value)? onSlideUpdate;
  final Function(double value)? onSlideDone;

  WheelSpinner({
    required this.value,
    this.max = double.infinity,
    this.min = double.negativeInfinity,
    this.onSlideDone,
    this.onSlideUpdate,
  });

  @override
  _WheelSpinnerState createState() => _WheelSpinnerState();
}

class _WheelSpinnerState extends State<WheelSpinner> {
  late double value;

  @override
  void initState() {
    value = widget.value;
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return Container(child: Text(value.toString()));
  }
}

As you can see, we are accepting a min, max, and current value to use with the widget. We are also accepting callbacks for when updating (while sliding) and when done (when finger is let go and the “fling” duration is over), which we will call once we update the actual value.

Let’s give it some shape

Right now our widget is completely empty, except for that little Text widget to see our current value, so let’s do some styling. For this example, We will be creating a rounded, tall box, which will contain some separation lines, and maybe some shade or gradient.

Let’s update our build method:

@override
Widget build(BuildContext context) {
  double shadowOffset = 0.2;
  return Container(
    width: 60,
    height: 100,
    decoration: BoxDecoration(
      gradient: LinearGradient(
        begin: Alignment.topCenter,
        end: Alignment.bottomCenter,
        stops: [0.0, shadowOffset, 1.0 - shadowOffset, 1.0],
        colors: [
          Colors.grey[350],
          Colors.grey[50],
          Colors.grey[50],
          Colors.grey[350]
        ],
      ),
      border: Border.all(
        width: 1,
        style: BorderStyle.solid,
        color: Colors.grey[600],
      ),
      borderRadius: BorderRadius.circular(3.5),
    ),
  );
}

We should see something like this:

Still no divider lines, though. Let’s say we want to divvy up our box to 10 segments, so that each time one of them goes outside the bounding box, we increase or decrease the value by 1. In this example we create a Stack, with 11 lines (10 for each division + 1 extra for the scroll effect) going from top to bottom (note the lineTopPos function that gets the correct y value):

@override
Widget build(BuildContext context) {
  // ...
  decoration: // ...,
  child: Container(
    child: Stack(
      children: List<Widget>.generate(
        11,
        (i) {
          var top = lineTopPos(value, i);
          return Positioned.fromRect(
            rect: Rect.fromLTWH(0.0, top, 60, 0),
            child: Divider(
              color: Colors.grey[600],
            ),
          );
        },
      ).toList(),
    ),
  ),

  // ...
}

double lineTopPos(double value, int i) {
  double valueFraction = (value.ceil() - value) * 10.0;
  double indexedTop = 10.0 * i;
  double top = indexedTop + valueFraction;
  return top;
}

Note the line that sets valueFraction. We take our value.ceil() and reduce the current value. This always gives us a number between 0.0 and 1.0 that tells us how much of the next segment to show. In reality, whenever we update value, we will always consider the small fraction we are scrolling into, which means we don’t jump by 1 every time, which will cause the lines to represent the value correctly, and also smoothly move as we input our scroll.

Now, we have something like this:

And now that it’s all nice and pretty, let’s start handling the logic.

Step 2 - Detecting gestures and updating the value

We can now wrap our widget with a GestureDetector. This is a built-in widget that lets you capture and use scroll, tap and multi-tap gestures on the child widget, and its decendants (that last part depends on the behavior parameter).

//...
GestureDetector(
      onVerticalDragStart: onDragStart,
      onVerticalDragUpdate: onDragUpdate,
      onVerticalDragEnd: onDragDone,
      child: /* our widget */
),
//...

And of course, we need to actually define onDragStart, onDragUpdate and onDragDone.

1. onDragStart

We’ll start by capturing on what value and position the finger first started dragging. For that, we will save them in our state:

Offset dragStartOffset;
double dragStartValue;

// ...

void onDragStart(DragStartDetails details) {
  setState(() {
    dragStartOffset = details.globalPosition;
    dragStartValue = value;
  });
}

2. onDragUpdate

On every update, aka when the finger slides up and down, we want to take the distance between the original start point, and use that to calculate our new value. If the finger scrolled up an amount equivalent to 10 separator lines, we increase/decrease by 10 accordingly. Of course, these numbers will be much smaller since we are updating a double, on a subpixel basis.

void onDragUpdate(DragUpdateDetails details) {
  var newValue = clamp(
      dragStartValue - (details.globalPosition - dragStartOffset).dy / 20.0,
      widget.min,
      widget.max);
  setState(() {
    value = newValue;
  });
  if (widget.onSlideUpdate != null) {
    widget.onSlideUpdate(value);
  }
}

We set the new value to use the dragStartValue and decrease by the distance of the scroll so far, divided by 20 to scale appropriately with the separator lines. Then we update using the callback, if that’s relevant.

Note: the clamp method is a just a convenience method to keep a number between 2 boundaries. Here is a basic implementation:

double clamp<T extends num>(T number, T low, T high) =>
    max(low * 1.0, min(number * 1.0, high * 1.0));

3. Testing the current widget

We can already test out our widget - we still can’t “fling” the finger, but we can drag our finger up and down to see the value updating.

Let’s add a Text widget to our parent build method, to see the value from the state that’s calling it. Here is an example of calling our widget:

class _MyHomePageState extends State<MyHomePage> {
  //...
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisSize: MainAxisSize.min,
          children: <Widget>[
            WheelSpinner(
              value: _counter.toDouble(),
              min: 0.0,
              max: 100.0,
              onSlideUpdate: (val) => setState(() {
                    _counter = val;
                  }),
            ),
            Padding(
              padding: const EdgeInsets.only(top: 16.0),
              child: Text(_counter.toStringAsFixed(2), textScaleFactor: 2.0,),
            ),
          ],
        ),
      ),
    );
  }
  // ...
}

Now it should work and look like this:

4. onDragDone

Our last piece is also the most fun. Here we will start handling our “fling” physics. We need 2 things here:

Use the velocity of the letting-go gesture to figure out how much to add/reduce from the value
Dampen this value slowly to create an eased roll effect

Luckily, animations are super useful in a case like this, for applying curves to a path between 2 number values - we don’t need to calculate the damping ourselves.

So first thing’s first, we need to define and create a new AnimationController, and Animation<double>, and let’s also set their initial value in our initState method:

AnimationController flingController;
Animation<double> flingAnimation;
void Function() currentFlingListener;

@override
initState() {
  flingAnimation = AlwaysStoppedAnimation(0.0);
  flingController = AnimationController(vsync: this);
  // ...
}

Also, since we use animations now, we will want to mixin SingleTickerProviderStateMixin, which will manage a ticker for us, for the animation to use:

class _WheelSpinnerState extends State<WheelSpinner>
    with SingleTickerProviderStateMixin {
      // ...
}

Then, we can start with our new method:

void onDragDone(DragEndDetails details) {
  setState(() {
    dragStartOffset = null;
  });
  double velocity = details.primaryVelocity;
  if (velocity.abs() == 0) {
    if (widget.onSlideDone != null) {
      widget.onSlideDone(value);
    }
    return;
  }
}

In the above lines we simply reset the drag start offset, as it’s no longer relevant, now that the finger was let go. Then we get the velocity of the drag, and if it’s 0, we return early and submit our callback, onSlideDone with the latest value.

Now, we can proceed with handling the “fling”.

We’ll start by saving the value that was set when we first let go of the finger.

void onDragDone(DragEndDetails details) {
  // ...
  double originalValue = value;
  // ...
}

Now, we want to add a listener to our animation value. We will generate a listener based on the value that was let go at, because we will need it to calculate the updated value:

flingListener(double originalValue) {
  return () {
    double newValue =
        clamp(originalValue - flingAnimation.value, widget.min, widget.max);
    if (newValue != value) {
      setState(() {
        value = newValue;
      });
      if (flingAnimation.value == flingController.upperBound) {
        if (widget.onSlideDone != null) {
          widget.onSlideDone(value);
        }
      } else {
        if (widget.onSlideUpdate != null) {
          widget.onSlideUpdate(value);
        }
      }
    }
  };
}

In the line:

double newValue =
        clamp(originalValue - flingAnimation.value, widget.min, widget.max);

You can see we use the animation value and simply decrease it from the original value. Depending on the fling direction, this will either continue up or down along with the animation value.

Now we can set our listener to that function once we call it. Saving it to an instance variable will allow us to remove the listener on dispose, as we will have a reference to the same listener function.

  // ...
  currentFlingListener = flingListener(originalValue);
  // ...

Then we start a Tween animation: we start at 0.0, and end at velocity. Whether the velocity is positive or negative, the math will work to reach the final number we want.

We set the curve as we want (in this case, Curves.decelerate), and attach the parent animation controller; attach the listener, and finally when we’re done, we can use forward() to start animating.

  flingController.duration = Duration(milliseconds: velocity.abs().toInt());
  flingAnimation =
      Tween(begin: 0.0, end: velocity / 100).animate(CurvedAnimation(
    curve: Curves.decelerate,
    parent: flingController,
  ))
        ..addListener(currentFlingListener);
  flingController
    ..reset()
    ..forward();
}

We added reset() just before forward(), to make sure no previous animations are lingering for some odd reason. In fact, let’s add a stop() calls to onDragStart and onDragUpdate, as well, and also reset the animation itself:

void onDragStart(DragStartDetails details) {
  flingController.stop();
  flingAnimation = AlwaysStoppedAnimation(0.0);
  // ...
}

void onDragUpdate(DragUpdateDetails details) {
  flingController.stop();
  flingAnimation = AlwaysStoppedAnimation(0.0);
  // ...
}

And now that it’s all out of the way, our widget should be fully working:

Done!

This should be pretty much it! You can of course style it and expand on it, but I’ve already made this a package, so any improvements you have, or if you just want to use it, head over to the package on Dart Pub, or feel free to contribute at the source on GitHub.

The source of the example app used for this tutorial right here, if you want to take a look and compare.

Feel free to ask questions, provide feedback or correct my mistakes in the comments - I’m sure there are some.

Simple Scaffold: Generating multiple files for faster coding

Wed, 06 Mar 2019 13:43:40 GMT

As most people who have been working a lot in front-end development, I will acknowledge that no matter what framework, library set or tools you work with, you end up replicating a lot of files, especially with modern component-based frameworks, such as React, Angular and Vue.js.

They usually grow to be very different, but they almost always start with the same some sort of base-skeleton, which gets built-upon later. A lot of pipeline code written, and time wasted.

What have we got?

There are some existing solutions to these problems, but they usually are not very flexible, or are tied to one library or implementation. Let’s take a look at a few examples.

IDE snippets, such as VS Code’s easy snippet extensions/configurations are very helpful. Basically, you can either use configuration JSON files to add snippets to your code quickly, or use extensions that simply fill these up for you.

There are nice benefits to this, as you can start typing the ‘prefix’ of the snippet and hit Tab quickly to add small snippets of code. You can also predefine stopping points, which you can navigate using Tab after inserting the snippet, and insert arguments, method names, etc.

These are nice, but only for small snippets. When you have to create an entire component, made by multiple files, you’re stuck with the option to create multiple files yourself, name them appropriately, and paste different snippets on each file.
Some NPM libraries provide ways to scaffold files based on templates, but from what I’ve seen
- they are tied to specific task runners, libraries or frameworks
- they are difficult to set up and customize
- combining your entire stack will be difficult

Enter: Simple Scaffold

Read the Simple Scaffold documentation

I was frustrated with this, and all I wanted was to have different sets of files I could generate, simply by copying them at predefined directory structures, and fill them with some variable data.

For instance: I create a lot of React components at work. We have recently moved to a single-file component structure for starting components; but we have different types of components that we want to generate based on context, like different file contents for page containers versus general-use components.

Being fed up with my options, I made this small NPM package, that does just this, in a very easy and quick way to setup. Simply put your files wherever you want, and either use the CLI tool or the package as an import - and you’re good to go.

Example: Jekyll Posts

Creating Jekyll posts is very simple! However I still had a little bit missing - I have to manually add the date every time? Oh, man. What about the file identifier name, and title? My excerpt_separator?

So I just made the simplest script in Ruby (to fit in with the Jekyll theme) to run the Simple Scaffold CLI with some custom parameters.

#!/usr/bin/env ruby
require "json"

SCAFFOLD_DIR = "scaffold_templates"
*name = ARGV

data = { dateFmt: "yyyy-MM-dd", fullDateFmt: "yyyy-MM-dd HH:mm:ss XX" }

puts `
  npx simple-scaffold@latest "#{name.join(" ")}" \
    --templates #{SCAFFOLD_DIR}/**/* \
    --output _drafts \
    -w true \
    -s false \
    -d '#{JSON.dump(data)}'
`
puts 'Done'

Let’s run by this real quick:

Lines 1-7 - setting up locals such as template directory base, and some variables to pass to the templates.
Lines 9-16 - We pass the parameters via a shell call (back-ticks in ruby), and immediately run using npx.

Locals are passed to Handlebars, and can be used both in file/directory names and file contents.

Now all I had to do is create a place for my templates, and add a template inside:

- scaffold_templates/
  -- {{now dateFmt}}-{{kebabCase name}}.md

And fill it up with some basic post:

---
layout: post
title: '{{ startCase name }}'
date: { { now fullDateFmt } }
excerpt_separator: <!-- more -->
categories:
---
Post content goes here

And voila! Running the script, along with a name:

./scaffold.rb "Billy Jean is not my lover"

Generates the following file structure:

- _posts/
  -- 2019-03-06-billy-jean-is-not-my-lover.markdown

With the following content:

---
layout: post
title: 'Billy Jean is not my lover'
date: 2019-03-06 16:43:38 +0200
excerpt_separator: <!-- more -->
categories:
---
Post content goes here

You can do more

Take the same approach and think what you could create generators for with 0 effort. React or Vue components? You can bundle each in a folder with all the imports, exports, base props, etc all lined up with your component name.

Add a script to your package.json, a couple of files, and you’re good to go:

package.json

{
  "gen:component": "npx simple-scaffold@latest -t scaffold_templates/react-component -o src/components -s true -w true '{\"className\": \"myClassName\",\"author\": \"Chen Asraf\"}'"
}

scaffold_templates/react-component/{{pascalCase name}}.tsx

/**
 * Author: {{ author }}
 * Date: {{ now "yyyy-MM-dd" }}
 */
import React from 'react'
import { ComponentProps } from 'utils/types'

export interface {{pascalCase name}}Props extends ComponentProps {
  className?: string
}

export default {{camelCase name}}: React.FC<{{pascalCase name}}Props> = (props) => {
  return (
    <div className="{{className}}">{{camelCase name}} Component</div>
  )
}

scaffold_templates/react-component/index.ts

export * from './{{pascalCase name}}'

Running npm run gen:component MyComponent will quickly generate your new components:

src/components/MyComponent/MyComponent.tsx

/**
 * Author: Chen Asraf
 * Date: 2022-08-10
 */
import React from 'react'
import { ComponentProps } from 'utils/types'

export interface MyComponentProps extends ComponentProps {
  className?: string
}

export default MyComponent: React.FC<MyComponentProps> = (props) => {
  return (
    <div className="myClassName">MyComponent Component</div>
  )
}

src/components/MyComponent/index.ts

export * from './MyComponent'

Why stop there? Create entire app boilerplates using the same method - create your boilerplate app, add all your libraries, and replace your app name with {{name}}. That’s it! You can run this package with input from any local files and output them in (or as) your next project.

Check out the documentation by clicking this link!

Placeholder.photo: An easy way to add mockup images to your UI and web development workflow

Thu, 21 Apr 2022 12:55:03 GMT

When prototyping UI designs or code, we often need images to put in places when we have no finalized images in the design, or in order to test out different sizes and colors when only some are provided.

I find myself needing to find an image to use when developing websites and apps often, and finding a mock image to upload is time consuming - not to mention leaves possibly unwanted files in your code or design.

So what do we do?

Enter: placeholder.photo.

What is placeholder.photo?

Placeholder.photo is a service that lets you use placeholder images anywhere, saving you the hassle of creating images in the required sizes for temporary use-cases.

It is a tool that gives you an image URL to use anywhere, from any web or other origin.

Why would I need this?

Say you are working on implementing a website design in HTML/CSS and more languages of your choosing.

When confronted with the need to use an image, for, say, an image that exists on the design - usually I would put it in the appropriate asset location in the project.

But what if any of these is true…

The image represents user uploaded content, so the size is variable?
The image is not finalized and ready for use in the design?
The image does not exist at all and a placeholder box is given in the design docs?

How does placeholder.photo help me solve this?

Now instead of having to go ahead and either find images of appropriate sizes, making sure they are uploaded to an accessible location online, or in the case of updates to the spec, now having to update said images…

You can just take a URL from placeholder.photo.

The preview tool lets you decide how the image looks by letting you customize size, background and text colors, the text content and font size.

But really, you can just customize the URL yourself and use the images as necessary without having to rely on it.

A URL like this:

https://placeholder.photo/img/255?bg_color=fb9e00&fg_color=ffffff&text=My+Text

Will generate this image:

When generating from the tool, you can easily copy the image URL you made using the copy button:

Or drag & drop to a supporting tool:

And that’s it! Enjoy easier time coding & designing.

The 5 Steps of Completion: Learning to break coding problems down

Sun, 01 May 2022 17:00:00 GMT

When developers first start their journey, sometimes syntax is easy. You’ve memorized a few commands, classes, and method names, and you can make simple applications quite easily. Also, there is plenty of docs and tutorials online - as well as Stack Overflow - surely copying-and-pasting your way through would be enough, right?

Let’s try to answer the question: how to start approaching problem solving for any given task?

Well, the real problem most actually face on a day-to-day basis when trying to program is not determining which lines of ifs and for loops go where. The problem starts at… well, the problem. Where to start? How to approach a solution to a given task or problem you are facing?

Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.

~ Jamie Zawinski

Learning to put theory to practice

In my time developing I tried hard to learn to recognize the most important skills a developer has in their tool set - and one of the biggest hurdles for starting developers - is the ability to know how to face any problem - at least, in the context of development.

When an up-coming developer gets a new task – either from someone else (a boss, a colleague) or that they themselves define (for challenge, for proof of concept, for learning) - they often get stuck at the first hurdle - “where do I start?”

Now that you can create simple sentences – you have no idea where to begin when trying to write entire paragraphs or essays. How does it my “hello world” actually do what it should? How do I take what I know and combine the parts so they work with each other?

Here, I will try to give you some thought experiments and tools so you can be self-sufficient when it comes to starting a new coding task, whether it is adding to existing code or beginning from scratch.

We will start very simple, and then try to apply the same ideas to more complex tasks later on. So while it may seem very stupid and superficial at first, it will come together soon enough.

Defining a “problem”

A problem in our context is any task that has been given, however large or small.

For example, here are some small task examples:

Add a “Contact” button to the home page that links to the contact page [design attached]
Create a program that calculates the area of a rectangle

And here are some larger tasks:

Create a snake game
Create a to-do list application

All tasks are completely unique, and yet somewhat alike. How do we approach each of these from a development perspective?

The methodology - The 5 Steps of Completion

Let’s start introducing our frame of thought. We need to define the context of our task, so we can ask the proper questions right after.

1. Ask Questions

Let’s take a basic example to start.

Given story: Add a “Contact” button to the home page that links to the contact page

Ask yourself the following questions:

What is the current state? (i.e., from what point do I start?)
This won’t become a task later, but will be important for you to picture a “delta” of sorts, which draws a path in your mind from A (what there is) to B (what you want achieve).
What does the user expect to see change?
What interactions can the user make with the change?

2. Give Answers

Let’s attempt to answer these questions for this case, one by one.

Q: What is the current state?

A: The user can see a website, with various sections, images and links. In the home page, there is no button that links to the “Contact” page.
Q: What does the user expect to see change?

A: The user should see a new button with the text “Contact”
Q: What interactions can the user make with the change?

A: The user should be able to click the button. Clicking it will point to the “Contact” page.

We didn’t learn a lot of new information by breaking down these questions. But we did gain separation which is critical. Now we have a list of tasks, instead of one task.

Why is that helpful?

3. Break it down

Large and small tasks alike – even the smallest of addition to your code, can be broken down even further. It is crucial that we learn to break things down as developers, because then we can create actionable items which we can start working on.

The example above might be a bit trivial, but it gets the point across - we can break everything down further, into infinity. Really, this has no limit. But we should stop somewhere…

So what next?

4. Create Actionables

Now what we want to do is take the answers from above, and compile them into a list of actions. The first question is redundant as it describes what is going on right now. So we will list the rest and try to create action items from them:

The user should see a new button with the text “Contact”
1. Show a button on the page
2. Have the text say “Contact”
3. Design the button to fit with the design
The user should be able to click the button. Clicking it will point to the “Contact” page.
1. Attach a click “handler” to the button
2. Make it redirect the page to the desired URL

We are making some progress. We now have a list of tasks to work with. See how 2 small questions already turned into 5 tasks, and that is for a tiny problem.

5. Solve It

Now that we have tasks - we can get crackin’.

For the sake of this let’s assume we are developers that are relatively new to web development. What would we do to approach this task?

When in doubt, look it up

Google, Duck Duck Go, Brave, whatever. What do you choose? Your preference. What would we search for here exactly?

The trick is to be as broad as possible, and when the answers don’t help, get more specific from there. This is different from debugging - where you should be as specific as possible and search for exact matches. But I digress. Oh, and always prepend/append the programming/hypertext language you are referencing.

The action item is “Show a button on the page”. So let’s try see: how do we display a button in HTML? and how do I set its text? Now that we have these simple questions, we can remove some excess words and we remain with 2 great keywords to search which should get us starting on the topic, at the most general sense:

html button

Now to answer the next burning question - how to make the button fit with the design?

We have another tricky task here.
Say our design looks something like this:

Here, if we are unfamiliar with everything, breaking down even further can really help. Let’s look at what defines the style of this button:

It has a gradient fill/background
It has white text
It has rounded corners
It has a shadow behind it

We will want to search for more specific things, which will also be our broken down tasks; such as:

Try some more simple task ideas, and practice breaking down tasks into chunks that you can definitely complete. As tasks become more complex, they will sometimes become harder. But sometimes you will find that they don’t, really — they just become longer, and larger, which may seem daunting, but all big tasks can sometimes be broken up into many, many tiny tasks, which will be easy to actually get done.

Any progress is important: No task is too small

One of the important concepts behind this approach is that tasks should be broken down until you can picture in your mind the entire task’s execution, to some extent. It’s not really a rule to follow, but more of a guiding line. Tasks that you can picture yourself doing almost fully, you can probably complete. And if the smallest task is still beyond your knowledge, then you try to break it down further, or must go out and seek that knowledge.

Which is to say - in order to make your tasks useful, you must make them doable. Once you can tick a checkbox that says “I’m done”, no matter the size of the contribution to the total pool of tasks, you still made some progress. That progress piles up slowly, and soon you will see your projects come to life.

The next steps

Practice makes perfect - so whatever side project you have, apply this new knowledge to it. Take your tasks, ask the questions, and turn them into smaller ones.

In the next part of this series, we will take a look at some more complex examples - abstract, large ideas that need to become a coded reality. For the meantime, a good grasp of simple tasks breakdown will shape your minds properly for the next difficulty that lies ahead.

How to use React memoization hooks for increased performance

Wed, 10 Aug 2022 13:41:15 GMT

As React apps grow bigger and more complex, performance becomes more and more of a problem. As components become larger and contain more and more sub-components, rendering becomes slow and turns into a bottleneck.

How do we tackle this? If you haven’t used useMemo and useCallback, we can start with those.

In this tutorial, we will take a look at how these 2 very easy and handy callbacks work, and why they are so useful. In fact, these days my eyes get sore when I don’t see them used. So let’s dive in to what they do.

React.useMemo

This React hook’s one goal is to save a value for later use, and not re-calculating it on the spot.

Let’s take an example of some expensive logic that runs in our render function:

const ExpensiveComponent: React.FC = (props) => {
  const [list, setList] = React.useState([])
  const [counter, setCounter] = React.useState(0)
  const multiplied = list.map((i) => (i * 972 + 1000) / 5213).join(', ')

  function addRandom() {
    setList((prev) => [...prev, Math.floor(Math.random() * 10000)])
  }

  function increaseCounter() {
    setCounter((prev) => ++prev)
  }

  return (
    <div>
      Counter: {counter}
      <br />
      Multiplied: {multiplied}
      <br />
      <button onClick={addRandom}>Add Random</button>
      <button onClick={increaseCounter}>Increase Counter</button>
    </div>
  )
}

Doesn’t seem very problematic, but take a look at the multiplied variable. The logic in this example isn’t too bad, but imagine working with a giant list of special objects. This mapping alone could be a performance problem, especially if it’s looped in a parent component.

In this case, there is another state hook - counter. When setCounter is called, multiplied will be calculated all over again, wasting previous resources, even when an update in that case is not needed as these variables are independent of each-other.

That’s where useMemo comes in hand (read the official docs).

You can use this hook to save the value, and retrieve the same object until a re-calculation is needed.

Here’s how it’s used, the only line we need to change is the multiplied definition:

const multiplied = React.useMemo(() => {
  console.log('recalculating multiplied:', list)
  return list.map((i) => (i * 972 + 1000) / 5213).join(', ')
}, [list])

The useMemo hook takes 2 arguments:

The create function - used to return the calculated value of the variable we want to eventually use
A list of dependencies. The dependency list is used to determine when a new value should be calculated - i.e, when to run the create function again.

We added a console.log call here just to note when a new value is being calculated.

And with those changes, we can try our component again (here is the updated code just in case):

const ExpensiveComponent: React.FC = (props) => {
  const [list, setList] = React.useState([])
  const [counter, setCounter] = React.useState(0)
  const multiplied = React.useMemo(() => {
    console.log('recalculating multiplied:', list)
    return list.map((i) => (i * 972 + 1000) / 5213).join(', ')
  }, [list])

  function addRandom() {
    setList((prev) => [...prev, Math.floor(Math.random() * 10000)])
  }

  function increaseCounter() {
    setCounter((prev) => ++prev)
  }

  return (
    <div>
      Counter: {counter}
      <br />
      Multiplied: {multiplied}
      <br />
      <button onClick={addRandom}>Add Random</button>
      <button onClick={increaseCounter}>Increase Counter</button>
    </div>
  )
}

If you now change the counter by using the “Increase Counter” button, you will see our console.log call is not being invoked again until we use the other button for “Add Random”.

React.useCallback

Now we have the other hook - useCallback (read the official docs).

This works exactly like the useMemo hook - except it is for functions instead of variable values.

We can take our button functions, and wrap each in this hook to make sure our function reference only changes when needed.

const ExpensiveComponent: React.FC = (props) => {
  const [list, setList] = React.useState([])
  const [counter, setCounter] = React.useState(0)
  const multiplied = React.useMemo(
    () => list.map((i) => (i * 972 + 1000) / 5213).join(', '),
    [list],
  )
  const addRandom = React.useCallback(
    () => setList((prev) => [...prev, Math.floor(Math.random() * 10000)]),
    [setList],
  )
  const increaseCounter = React.useCallback(() => setCounter((prev) => ++prev), [setCounter])

  return (
    <div>
      Counter: {counter}
      <br />
      Multiplied: {multiplied}
      <br />
      <button onClick={addRandom}>Add Random</button>
      <button onClick={increaseCounter}>Increase Counter</button>
    </div>
  )
}

Now both our variables and functions are memoized and will only change reference when their dependencies dictate they do.

Caveats

Using these hooks don’t come without their share of problems.

Consider whether this actually improves performance or not in your specific case. If your state is pretty regularly changed and these memoizations have to run pretty often, their performance increases might be outweighed by the performance costs of actually running the memoization logic.
Dependency checking and generating can be expensive. Be careful what you put in the dependency lists, and if needed, make some more memoization and map your objects and lists in deterministic ways so that they are statically inspectable easily. Also avoid using expensive methods such as JSON.stringify to create those memoizations or dependencies, as it might also be too expensive to be worth the trouble and might make things worse.

Other things to consider

You might want to make sure your project uses lint rules that enforce exhaustive dependencies, as they make tracking these things much more easy.

In some cases, you might want to add ignore comments in very specific places, but it makes it very clear that this part is built that way intentionally and prevents more confusion about when to update the dependencies.

Hopefully you find this useful. There are many other hooks to learn about, but these 2 are very useful and often ignored, so I thought it would be good to highlight them. If you are interested you might want to look up useRef and how it differs from useMemo, or maybe I’ll make another part about it in the future. Who knows?

React: Using useEffect Effectively

Thu, 18 Aug 2022 21:00:00 GMT

Whether you’re new to React or you are recently migrating from class components (this late?), you might not be familiar with how useEffect works.

Actually, you might be using it still from other existing (or copied 😉) code, but have no idea (or just a small idea) what’s actually happening.

In this post we’ll dive in to this very flexible hook, some example, and some real, common use-cases.

What is `React.useEffect`?

This is a React hook that lets you introduce side-effects into your components.

Here is some example code of a common useEffect hook:

const MyComponent = () => {
  const [posts, setPosts] = React.useState([])

  // This will fire only once when the component mounts
  React.useEffect(() => {
    PostsAPI.getPostsList().then((data) => setPosts(data))
  }, [])
  return <div>{JSON.stringify(posts)}</div>
}

What are side effects?

Well, side effects are a general programming term, under the larger term “effects”. Effects have a wide meaning, and side-effects are a subset of that. The short version of the definition of a side-effect is “a function (or piece of code) that introduces changes to any value that is not inside its scope” - whether input arguments, or outside scoped variables (or global variables).

In React components, that mostly means any function inside a component that introduces a change that is outside the scope of rendering itself.

Are you calling an outside API? You are introducing an effect. Saving a value to the state? Introduces an effect. As long as it’s outside the scope of rendering the component itself, it’s a side-effect.

Side-effects are not bad per-se - but introducing them needs to be done with intent, and with caution.

React side-effects example

React.useEffect is a hook that lets you wrap side-effects in a safe way so that you can avoid unnecessary renders, and use the necessary ones correctly. Consider the following code:

const MyComponent = () => {
  const [count, setCount] = React.useState(0)

  CounterAPI.getCount().then((data) => setCount(data))

  return <div>{count}</div>
}

If you haven’t noticed th the problem at first glance - don’t worry.

What happened here is we implicitly introduced a side effect right inside our rendering.

As soon as the component renders, the CounterAPI will get called, return something, and setCount will trigger a state change, which causes a render.

Then immediately, the new render runs, and another API call is made… This will go on forever, so React (or your browser) will crash in some way, killing the page.

This is why we need to guard against side-effects. Here is a simple change to fix this:

const MyComponent = () => {
  const [count, setCount] = React.useState(0)
  React.useEffect(() => {
    // add this line
    CounterAPI.getCount().then((data) => setCount(data))
  }, []) // and this line
  return <div>{count}</div>
}

This will mean our API only calls once.

A replacement for lifecycle methods

If you have ever used class-based components in React, you might be familiar with the term. In these types of components, sometimes we would need to fire events when the component mounts for the first time, or when one of the props changes, to sync our state properly with our business-logic.

This is the same effect from before, but in the old* method:

class MyComponent extends React.Component {
  render() {
    // return ...
  }

  componentDidMount() {
    MyAPI.getResults().then((data) => this.setState({ apiData: data }))
  }
}

* It's not deprecated, and React will support it for a while to come. But class-based components are generally considered to be outdated, a bit less easy to predict sometimes - especially for newcomers, and generally too verbose for sometimes really simple code.

The useEffect hook is also the de-facto replacement to the lifecycle methods, which are unavailable when you are using function components.

Let’s dive into how it’s used.

Structure of the `useEffect` hook

A useEffect call takes 2 arguments.

React.useEffect(
  () => {
    // the side-effect code

    // Optionally return a clean-up function
    return () => doCleanup()
  },
  [
    /* dependencies */
  ],
)

The first is a function to run at… “some time”. The time in which to run this effect differs on your use case, and your usage of the 2nd argument. We will talk about how it works shortly.

The second is optional - but in my opinion, very required - the dependency array. Like many React hooks (see my post about useMemo and useCallback), it is a list of changes that the component listens to, only running the effect after changes to the dependencies were detected.

In the case of useEffect, the effect always runs at least once, even if that means all the values it depends on might be undefined. So wait, how does this work?

Thinking “Reactive”

This is a bit confusing. We are listening to changes now - okay. I will elaborate a bit.

If in the past we used to rely on the framework to call specific methods in different lifecycle times, now we are going to change our way of thinking a bit. This method introduced many problems, especially because developers were often introducing side-effects where they shouldn’t be possible, introducing bugs and other problems.

Many hooks now rely on dependency arrays to define when they are being re-built. Our way of thinking will be similar - our function will run when changes are made, and they’re always made at least once - in the mounting phase.

If we want to fire code once when our component is mounted, we can introduce an effect with 0 dependencies. This causes it to only run once, as no changes will ever be introduced to this very constant array. This causes the hook to behave exactly as componentDidMount did.

Dependencies: `[] !== undefined`

It’s important to note that supplying no array in the dependency argument, and supplying an empty array are 2 different things! Try it out yourself to see the difference.

const MyComponent = () => {
  const [count, setCount] = React.useState(0)

  React.useEffect(() => console.log('no array'))
  React.useEffect(() => console.log('empty array'), [])

  return <div onClick={() => setCount((prev) => ++prev)}>{count}</div>
}

Running a component with this code will show different results.

Immediately, we will see this in the log:

no array
empty array

Nothing special yet. Let’s click the div and see what happens.

no array

Ahh… What’s this? It ran again? But we supplied no dependencies?

Supplying no value at all means it will update every time the component renders, not just once. The dependency needs to be an array for any comparison to happen. If it’s undefined, comparison is skipped and the effect runs every time.

How can we use this now?

Let’s review the basics we learned about this hook.

It runs at least once
It then runs:
1. every time its dependency array contents change, or
2. if there is no array, after every render

Component mounting

The simplest effect we learned about is commonly referred to as a mount effect. It runs once since you supply no dependencies.

Let’s say we are viewing a list of posts.

const PostList = () => {
  const [posts, setPosts] = React.useState([])
  const [page, setPage] = React.useState(0)

  React.useEffect(() => {
    setLoading(true)
    PostsAPI.getPosts({ page: page }).then((data) => {
      setPosts(data.posts)
      setLoading(false)
    })
  }, [])

  return (
    <div>
      {loading
        ? 'Loading...'
        : posts.forEach((post, i) => {
            return <div key={post.id}>{post.title}</div>
          })}
    </div>
  )
}

This will fetch the posts once, and once only.

Then we have more cases, where we decide what to supply, and we can make our component react to changes in either the props or change, and introduce the effects we need.

Let’s consider an extension to this use-case to test with.

Reacting to a state change

We will add a page selector, which lets us change which page of the posts we are viewing. For the sake of simplicity, the count per page is implicit and hard-coded in the backend.

  const PostList = () => {
    const [posts, setPosts] = React.useState([])
    const [loading, setLoading] = React.useState(true)
+   const [page, setPage] = React.useState(0)

    React.useEffect(() => {
      setLoading(true)
      PostsAPI.getPosts({ page: page }).then((data) => {
        setPosts(data.posts)
        setLoading(false)
      })
-   }, [])
+   }, [page])

    return (
      <div>
+       <span onClick={() => setPage(prev => ++prev)}>Next page &raquo;</span>
        {loading
          ? 'Loading...'
          : posts.forEach((post, i) => {
              return <div key={post.id}>{post.title}</div>
            })}
      </div>
    )
  }

As you can see, we supplied our page as a dependency for the API call. This means that once the component mounts, and then every time this count changes afterwards, React will run our effect again, fetching the posts from the API. Neat!

The cleanup function, timeouts and intervals

Another great use-case is to run some logic on a fixed interval, or after a timeout. Let’s say we want to increase a counter every second.

const Counter = () => {
  const [count, setCount] = React.useState(0)

  React.useEffect(() => {
    setInterval(() => {
      setCount((prev) => ++prev)
    }, 1000)
  }, [])

  return <div>{count}</div>
}

This works great! Every second, we get a new count with a larger number.

But this introduces a bug: navigate to another route, or get this component removed in some way. You will notice the interval keeps running!

In a case like this, we want to make sure our interval or timeout stops when the component is no longer mounted, in order to prevent a memory leak.

This is where cleanup functions come in handy. A useEffect call can also accept a function as a result of its callback.

That function will run once one of the dependencies change, before the “new” effect runs. It will also run once the component unmounts, i.e., is being removed from the tree in some way. These cases are the same phase that calls the cleanup function, with the only difference being whether a new effect will replace the discarded one, or will the effect end its life there.

This allows us to do things like cancel timers, API requests, or make some other cleanup to make sure nothing happens once it’s no longer necessary. React class-component users will recognize this as being a similar use-case for using componentWillUnmount lifecycle method. Let’s see how we can fix our previous example:

  const Counter = () => {
    const [count, setCount] = React.useState(0)

    React.useEffect(() => {
-     setInterval(() => {
+     const id = setInterval(() => {
        setCount((prev) => ++prev)
      }, 1000)
+     return () => clearInterval(id)
    }, [])

    return <div>{count}</div>
  }

As you may or may not already know, setInterval and setTimeout return an id token that we can later use to cancel it. By supplying it as a return function to useEffect, we are telling it to destroy our timer every time it needs to. If it is supposed to create a new one due to dependency change, it will still create it and our code will still work.

External listeners

The last prominent use-case we will show here is using external listeners inside components.

This is similar to our previous use-case, where we have a function with a side-effect that needs to be cleaned up. In fact it’s pretty much exactly the same. Let’s say we want to display the window resolution in our component (or make some condition depend on a value from there). Let’s use our previous knowledge to make a component that works for this.

const WindowSize = () => {
  const [width, setWidth] = React.useState(0)
  const [height, setHeight] = React.useState(0)

  React.useEffect(() => {
    const listener = () => {
      setWidth(window.innerWidth)
      setHeight(window.innerHeight)
    }
    window.addEventListener('resize', listener)
    return () => window.removeEventListener('resize', listener)
  }, [])

  return (
    <div>
      Width: {width}px
      <br />
      Height: {height}px
    </div>
  )
}

Now we have an effect that listens to window changes, and makes sure to update the state with it. Another success! And as you can see, it’s not much different than the interval/timeout example.

Conclusion

I hope that now you might see a bit more clearly how this hook is useful, and how to actually put it to use properly.

Experiment with using useEffect in different cases, and experiment with thinking more reactively about changes - making dependencies change things as they need, instead of always chaining function calls whenever you change values just to replicate the same behavior.

Create your own useOptimistic hook

Thu, 09 Nov 2023 21:38:07 GMT

Recently, I came across this video by Web Dev Simplified, which introduced me to React’s new useOptimistic hook.

I began to wonder whether this is even needed. React has never been a library to provide more than bare-bones controls and hooks. The useCallback, useMemo, and useState hooks are most of the built-in hooks and they provide a solid base upon which you can build pretty much any logic inside your components or custom hooks.

So since when does React provide more high-level abstractions above hooks? At that point, they become even more opinionated than they already are. Not all use cases are the same - but this is a pretty generic hook, all things considered. Apparently, it is still in the experimental stage.

I decided to experiment with creating this hook on my own, to understand if it was overcoming some internal system limitation. I looked at various examples of existing optimistic hooks on NPM, but I didn’t really find one that falls in line with more modern coding styles, or they were completely outdated and abandoned.

Why create your own?

Ideally, React should generally keep its size down as much as possible, and not increase the bundle with more bloat that won’t be used by everyone. I believe this hook is a very nice idea, but it is not relevant for everyone, and I think if you find it does fit your needs, you don’t have to wait for official support - you can do it (or copy the final code here) in relatively short time, and feature-complete (and added).

In addition, I thought it would make a good thought experiment and some coding practice, it can’t hurt!

Looking at React’s original implementation

I decided to first take a look at React’s official implementation of this. Not the source code, but I did want to get a sense of whether I like the consensus they came up with, or whether I feel it has room for improvement.

Here is a basic usage example, verbatim from React’s docs:

const [optimisticState, addOptimistic] = useOptimistic(
  state,
  // updateFn
  (currentState, optimisticValue) => {
    // merge and return new state
    // with optimistic value
  },
)

Seems nice, but I think we can do better.

Off the top of my head, I don’t see a way to get the status of the optimistic value. Can I know when it’s been finalized without having to maintain a separate state?

Also, there doesn’t seem to be a way to roll back to the original value if an error has occurred. Or at least, the docs don’t yet specify any. We can fix that, and provide a more “holistic” approach, which in my opinion doesn’t leave the scope of this hook. Say I push an optimistic update, and it fails, how can I roll it back?

We’ll try to make sense of where it should go as we go along.

Creating a hook

Let’s start with the signature, and make sure we have our types going for us.

type StateUpdater<T, A = T> = (currentState: T, newState: A) => T

export function useOptimistic<T, A>(
  initial: T,
  merge?: StateUpdater<T, A>,
): readonly [T, React.Dispatch<React.SetStateAction<T>>] {
  // ...
}

We start with the basics. An initial value, of type T which is generic, and an optional merge function. If it’s not provided by the user, we’ll just have it replace the old state with the new when an update comes. The merge function can have any type in the argument for the final dispatch, but we assume T unless specified.

Now we should add the state that maintains the optimistic value.

export function useOptimistic<T>(
  initial: T,
  merge?: StateUpdater<T, A>,
): readonly [T, React.Dispatch<React.SetStateAction<A>>] {
  const [state, setState] = useState(initial)
  return [state, setState]
}

We… have a very simple hook working right now. It doesn’t really do anything special at all. Very bland.

Ok so, what’s the most basic requirement? Well, for starters, it should let us merge the state if we provide a merge function, which we haven’t use yet from our arguments. Then, we should take into account that updating the parent state (held in initial) should also update the state itself, so that it will be up-to-date with the latest server value.

Creating the update functions

We’ll start by adding the useEffect hook, that will update from the parent initial prop when it updates.

useEffect(() => {
  setState(initial)
}, [initial])

Now, when the initial value is updated, the internal state will be kept up-to-date with the change.

We can also implement the merge functionality. For that, we need to provide another implementation of the setState function, which uses it internally and adds some more logic.

const update = useCallback(
  (value: SetStateAction<A>) => {
    const newValue = typeof value === 'function' ? (value as (value: T) => T)(state) : value
    const merged = merge ? merge(state, newValue as A) : (newValue as T)
    setState(merged)
  },
  [state],
)

return [state, update] // instead of [state, setState]

Okay, it’s a little more to break down here.

First, we have an identical callback to setState now, except it also runs the merge function before sending the final value.

In useState, in addition to passing a new state value, the setState function can be used with a callback function, which takes the current state as a parameter, and should return the new state. The variable newValue is populated with the result of that process.

Then we just stack the merge function on top of that, if it exists.

The usage for this would still be:

const [count, setCount] = useOptimistic(0)

// option 1
setCount(1)

// option 2
setCount((c) => c + 1)

All the changes happen internally. Nice.

That’s pretty much it as it stands right now. React’s implementation is pretty similar to this, with maybe a slight variation on how the change is calculated with the parent’s update, to determine when the value should be brought up to date.

A bit anti-climactic? Well, let’s add a few bells and whistles. And we can now, because we don’t need React to build it for us. We’re strong!

What else can we add? Rolling back on failure

The first idea for improvement I have is to be able to catch an error, and roll back the value if it fails. Let’s begin implementing this.

First, we need to have a value to go back to. So every time the developer updates the optimistic state to a new value, and an operation related to it fails, we want to provide a way to revert to the original value.

For a recap, here’s the entire hook as it now is, before we make changes:

type StateUpdater<T, A = T> = (currentState: T, newState: A) => T

export function useOptimistic<T>(
  initial: T,
  merge?: StateUpdater<T, A>,
): readonly [T, React.Dispatch<React.SetStateAction<A>>] {
  const [state, setState] = useState(initial)

  useEffect(() => {
    setState(initial)
  }, [initial])

  const update = useCallback(
    (value: SetStateAction<A>) => {
      const newValue = typeof value === 'function' ? (value as (value: T) => T)(state) : value
      const merged = merge ? merge(state, newValue as A) : (newValue as T)
      setState(merged)
    },
    [state],
  )

  return [state, update]
}

Now, let’s say the developer comes across a case like this:

function MyApp({ comments }) {
  const [oComments, addComment] = useOptimistic(
    comments || [],
    (existingComments, newComment) => [...existingComments, newComment],
  )

  const sendComment = async (comment: Comment) => {
    addComment(comment)
    await sendCommentToApi(comment) // could fail and throw
  }

  return ...
}

In this case, we are sending some API request to send the comment content to the backend and make sure it’s already there.

But if it fails, we will have to revert it to its original content manually by keeping a separate state for the original value, and set it back if it failed.

Instead, we can make our new hook handle this.

Let’s add another state that maintains the previous value:

const [rollbackState, setRollbackState] = useState<T | null>(null)

And now that we have that, we should set this rollback value once we set a new value through the regular setter:

const update = useCallback(
  (value: SetStateAction<A>) => {
    const newValue = typeof value === 'function' ? (value as (value: T) => T)(state) : value
    const merged = merge ? merge(state, newValue as A) : (newValue as T)
    setRollbackState(state)
    setState(merged)
  },
  [state],
)

We take this value, save it, and… Do nothing with it. We should probably write that logic somewhere. How’s this:

const rollback = useCallback(() => {
  if (rollbackState == null) {
    // you can either return here, or throw an error, and be more strict about false rollbacks:
    // throw new Error('There is no state to roll back to')
    return
  }
  setState(rollbackState!)
  setRollbackState(null)
}, [rollbackState])

And we can also add a done callback, which ditches the retained value, just for clean up. Something like this:

const done = useCallback(() => {
  setRollbackState(null)
}, [])

Let’s also return these in our hook:

export function useOptimistic<T, A = T>(
  initial: T,
  merge?: StateUpdater<T, A>,
): readonly [
  T,
  React.Dispatch<React.SetStateAction<A>>,
  {
    readonly rollback: () => void
    readonly done: () => void
  },
] {
  // ...
  return [state, update, { rollback, done }]
}

Nice, right? We just revert. Now to actually use it, we can update the component:

const [oComments, addComment, { done, rollback }] = useOptimistic(
  //                          ^^^^^^^^^^^^^^^^^^
  comments || [],
  (existingComments, newComment) => [...existingComments, newComment],
)

const sendComment = async (comment: Comment) => {
  try {
    addComment(comment)
    await sendCommentToApi(comment) // could fail and throw
  } catch (e) {
    rollback()
  }
}

That’s not too bad. If we fail, we use rollback and get back our previous value, and we never had to save it separately, which is good for scaling this to multiple places.

But… if you give it a go, you will notice a problem.

Let’s do a small test:

const fail = async (comment: Comment) => {
  try {
    addComment(comment)
    await new Promise((res) => setTimeout(res, 1000))
    throw new Error('Oops!')
    done()
  } catch {
    rollback()
  }
}

Make this run somewhere and you will realize the value reverted back to null instead of the old value, or simply doesn’t roll back. What gives?

Here is the problem: when we are using rollback, it uses the state variable, rollbackState, to set the value back on the main state. But here is the thing - this variable is a dependency in the callback hook on rollback. Take a closer look:

// before:
// const [rollbackState, setRollbackState] = useState<T | null>(null)
//
// after:
const rollbackState = useRef<T | null>(null)

const rollback = useCallback(() => {
  if (rollbackState == null) {
    return
  }
  setState(rollbackState!)
  setRollbackState(null)
}, [rollbackState])
//  ^^^^^^^^^^^^^

So why is that bad? Well, in our fail test above, we used addComment which is our update function with a fancy name. When the API call fails we call rollback. But rollback was created bound to the rollbackState that was there at the time of its creation. We wait for a promise to return, and then use the done or rollback functions, that were created before the rollbackState was even set away from null and into a value in the first place. So basically we are re-using the value from before we put anything in it, which means it’s null, which is the value we roll back to when using rollback.

How do we fix this?

Fixing the stale value

There are probably many possible solutions, but here is my very naive one.

How do I make sure I get the correct reference to the updated value?

In reality, I don’t need rollbackState to be an actual state. Why can I say that with confidence? Well, I don’t need to call a re-render when it updates, as it is only there to reference a value in another function when called later, and it’s not displayed in the UI - meaning it has no need to call for a render when I update it. Also it happens on literally every update call as well, so I am trying to uselessly mark the frame dirty for re-render twice when one time is enough.

With that in mind, I can now safely switch the rollbackState from a state hook to a useRef hook.

Why would that work? Because a useRef reference is always a constant object. But the current value inside that object can be changed. In effect, when we reference a ref value, we can say we are sure it is the most recent, as the ref never needs to change and no mismatches can occur.

Here’s how it looks after the fix:

const update = useCallback(
  (value: SetStateAction<A>) => {
    const newValue = typeof value === 'function' ? (value as (value: T) => T)(state) : value
    const merged = merge ? merge(state, newValue as A) : (newValue as T)
    rollbackState.current = state
    setState(merged)
  },
  [state],
)

const rollback = useCallback(() => {
  if (rollbackState.current == null) {
    return
  }
  setState(rollbackState.current!)
  rollbackState.current = null
}, [])

const done = useCallback(() => {
  rollbackState.current = null
}, [])

This way, no matter when I call the rollback or done functions, they will always try to access the same parent object. With that object, they can modify the inner property and we don’t need to worry about the data being stale.

Simplifying usage

You might notice that now you can have a pretty redundant pattern of try {...} catch() {...} where you would rollback and done at the end of each section. Well that’s no fun, we want less code.

Why not add this to our hook?

const run = useCallback(
  async (cb: () => Promise<unknown> | unknown) => {
    try {
      await cb()
      done()
    } catch (_) {
      rollback()
    }
  },
  [done, rollback],
)

Simple yet elegant. If we export this from the hook:

return [state, update as React.Dispatch<React.SetStateAction<A>>, { done, rollback, run }]

Now it will now let us do:

const sendComment = async (comment: Comment) => {
  run(async () => {
    addComment(comment)
    await sendCommentToApi(comment)
  })
}

Much simpler, right?

Saving the promise state

One last thing I want us to add is the ability to know when our state is loading. This way, even without using useMutation from @tanstack/react-query, we can get a loading state of any promise, while working directly with the useOptimistic hook instead.

For that we can simply add a state or a ref (decide which you want depending on your wanted DX), which sets a pending state to true and false when we update the state, and when we call done/rollback, accordingly.

Here is the full entire hook with all the things we made here:

type StateUpdater<T, A = T> = (currentState: T, newState: A) => T

export function useOptimistic<T, A = T>(
  initial: T,
  merge?: StateUpdater<T, A>,
): readonly [
  T,
  React.Dispatch<React.SetStateAction<A>>,
  {
    readonly done: () => void
    readonly rollback: () => void
    readonly pending: boolean
    readonly run: (promise: () => Promise<unknown> | unknown) => void
  },
] {
  const [state, setState] = useState(initial)
  const rollbackState = useRef<T | null>(null)
  const [pending, setPending] = useState(false)

  useEffect(() => {
    setPending(false)
    setState(initial)
  }, [initial])

  const update = useCallback(
    (value: SetStateAction<A>) => {
      const newValue = typeof value === 'function' ? (value as (value: T) => T)(state) : value
      const merged = merge ? merge(state, newValue as A) : (newValue as T)
      rollbackState.current = state
      setState(merged)
      setPending(true)
    },
    [state],
  )

  const rollback = useCallback(() => {
    if (rollbackState.current == null) {
      // throw new Error('There is no state to roll back to')
      return
    }
    setState(rollbackState.current!)
    rollbackState.current = null
    setPending(false)
  }, [])

  const done = useCallback(() => {
    rollbackState.current = null
    setPending(false)
  }, [])

  const run = useCallback(
    async (cb: () => Promise<unknown> | unknown) => {
      try {
        await cb()
        done()
      } catch (_) {
        rollback()
      }
    },
    [done, rollback],
  )

  return [
    state,
    update as React.Dispatch<React.SetStateAction<A>>,
    { done, rollback, pending, run },
  ] as const
}

Where to go from here?

There are many ways that you can improve this hook, but I feel like it now reached a fairly usable state which we can all build from in our own special ways.

Until React decides (or decides against) promoting useOptimistic to non-canary channels, maybe we can all benefit from a relatively simple hook that does exactly this without having to wait.

I hope you found this article useful or interesting. Feel free to ask questions or suggest improvements in the comments!

Create an Automated Portfolio Using GitHub and Astro

Tue, 27 Aug 2024 09:00:00 GMT

I have updated the code in this article to use Astro loaders/Content Layer, and release it as a package on NPM!

I have a tendency to start a lot of projects. I don’t necessarily finish them all — some software is never truly finished, sometimes they become deprecated, and sometimes, I just end up bored, or simply forget about them.

As a contractor/potential employee, I find it useful to redirect people to my GitHub page, where my projects are most prominently displayed and easily accessible, and they can get a quick glance at what I like to do and how my code and conduct look.

Since I have many rotating projects, it’s becoming quite difficult to keep track of which projects were recently updated, added, or removed. Not to mention, 6 pins on the GitHub profile really isn’t that much.

In this article, I will try to automate this process just a bit. This blog seems like the perfect place to make things a bit more organized but also make sure they can be updated easily. It gives me more control than GitHub, and I already have a projects page here, which would be a great target to replace and put something a bit less static in.

Hopefully, this article will give you some nice ideas on how you can spruce up your website with some semi-dynamic content, or how you can also help yourself display your featured work with a lot less manual labor.

The goal

This blog is an Astro project hosted as a static website. I truly recommend Astro; it’s an incredibly easy, fun, and flexible way to create a website or a blog that isn’t meant to be some kind of large web application — but I digress.

What I’m trying to accomplish here is essentially the following:

Get all my GitHub repos
Filter out uninteresting ones and prominently display the best ones
List others by type/category/language/some other grouping

Now, since this is a static website generator that I am using, there are a few limitations:

There is no “server side” during runtime — any processing is being done during build
I want the system I am creating to play nicely with the existing code
I want it to be easy to update, manage, add overrides to, and hide certain repositories

The plan

As of writing this, I haven’t yet changed a single character of code. I first want to get a glimpse at what I currently have and try to see if I can come up with a plan for how to get through this.

I already have a projects page; it’s simply an index with a list of some projects. Each project links to its own page, which might or might not have more information. This isn’t ideal, but I can already see a way to improve here — GitHub holds a README, a project description, releases, and links. It would be nice to implement this information somehow into the page.

In Astro, pages are created inside the src/pages/ directory of a project. Content that can be generated for those pages using simpler formats, such as Markdown and MDX, is available inside the src/content directory. With that in mind, let’s take a peek at the project-related files:

src/
├── components
│   ├── ProjectLinks.astro
├── content
│   ├── config.ts
│   └── project
│       ├── dart-script-runner.md
│       ├── dungeon-paper.md
│       ├── flutter-wheel-spinner.md
│       ├── mudblock.md
│       ├── simple-scaffold.md
│       └── unaconfig.md
└── pages
    └── projects
        ├── [...slug].astro
        └── index.astro

As you can see, there are a few files that are relevant to projects.

A component that displays a project
An index page
A page for each project ([...slug].astro)
Contents for each project (the .md files)

So, for each repository I need to:

Generate a .md file
Update the index page to display them with new information & sorting

This will get me most of the way there. After that, it will be mostly connecting some pipes to make sure it’s easy to reproduce this from, say, a CI environment, for easy deployment.

Getting the data

Well, I can’t display GitHub projects without getting their information, right?

For this, I’ll need a GitHub token that works. I’ve already got one set up — perfect.

Right now, the top of the src/pages/[...slug].astro file looks like this:

export async function getStaticPaths() {
  const posts = await getCollection('project')
  return posts.map((post) => ({
    params: { slug: post.slug },
    props: post,
  }))
}
type Props = CollectionEntry<'project'>

const post = Astro.props
const { description, links } = post.data
const { Content } = await post.render()

The function getStaticPaths is a special function in Astro that is responsible for telling Astro which pages to generate, and it is using the content directory to do so. I am passing slug as a param to the page generator, which means it can be accessed in the name’s placeholder — that’s what the file is called [...slug].astro. [] denotes param tokens to be replaced, and the ... means they can have a directory path inside.

The top of the src/pages/index.astro file looks like this:

const projects = await getCollection('project')
projects.sort((a, b) => {
  return a.data.order - b.data.order
})

Which is where we load all the projects for the index page. Note that they don’t necessarily have to correlate; I can have pages without links routing to them or projects without pages to link to. This is not our case, but it’s good to know.

Practically anything can be loaded here — which is where I intend to invoke the generator of new pages for the GitHub projects. So, let’s work on getting that to work.

Using GitHub API to grab repositories

A quick search and we can easily find the GitHub API specification for repositories for a user.

This is where I need the API key.

Let’s create a new TypeScript file and put whatever logic we need there in its own box.

I’ll start simple and work up from there. Let’s make a request to the repos API, and for each repo also grab the README file.

// src/github-projects.ts
import { GITHUB_TOKEN } from './consts'

const GITHUB_USERNAME = 'chenasraf'
const headers = new Headers()
headers.set('Authorization', `Bearer ${GITHUB_TOKEN}`)

export async function getProjectsList(): Promise<GitHubProjectSchema[]> {
  // Fetch the data
  const response = await fetch(`https://api.github.com/users/${GITHUB_USERNAME}/repos`, { headers })
  const repos = await response.json()

  const projects: GitHubProjectSchema[] = []

  // Create a new object for each repo + get README
  for (const repo of repos) {
    const project = GitHubProjectSchema.parse({
      name: repo.name,
      url: repo.html_url,
      description: repo.description,
      stars: repo.stargazers_count,
      order: -repo.stargazers_count,
    })

    // Get the README.md static file
    const readmeResponse = await fetch(
      `https://raw.githubusercontent.com/${GITHUB_USERNAME}/${repo.name}/${repo.default_branch}/README.md`,
      { headers },
    )
    const readme = await readmeResponse.text()
    project.readme = readme
    projects.push(project)
  }

  return projects
}

I’ll also create a type for it:

// src/types.ts
export const LinkSchema = z.object({
  title: z.string(),
  href: z.string(),
  icon: z.string(),
})
export type LinkSchema = z.infer<typeof LinkSchema>

export const GitHubProjectSchema = z.object({
  name: z.string(),
  title: z.string(),
  description: z.string().nullable(),
  url: z.string(),
  stars: z.number(),
  readme: z.string().optional(),
  order: z.number(),
  links: z.array(LinkSchema),
})
export type GitHubProjectSchema = z.infer<typeof GitHubProjectSchema>

Seems like quite a bit of code, but it’s really not complex at all. Here’s a rundown:

Fetch the data from /users/${GITHUB_USERNAME}/repos, which gives us all the basic information about each repository
Fetch the README from the raw file link for each repository
Put it all together and push into an array

Now, I’ll just use that instead of the original in index.astro, just so I can test what it shows and that everything is working.

const projects = await getProjectsList()
projects.sort((a, b) => {
  return a.order - b.order
})

// in the template itself:
projects.map((project) => (
  <code>
    <pre>{JSON.stringify(project, null, 2)}</pre>
  </code>
))

I removed the previous ProjectCard component for now because we haven’t updated it yet, and I just wanna make sure things are working.

Let’s refresh the page, and… Well, this is taking a couple of seconds, but okay, nothing major at all. It loaded! Wait… this is only 30 projects. Pretty sure I have more than that, even just the public ones.

Hmm… Oh, here, I found the problem.

Handling pagination

Seems like I’ve hit a small snag, but it’s all well documented, so I can only blame my own laziness.

Here is an excerpt from the GitHub API documentation for /{user}/repos:

page - integer The page number of the results to fetch. For more information, see “Using pagination in the REST API.”

Okay, let’s implement pagination to get everything:

async function fetchRepos() {
  const repos = []
  let page = 1

  let response = await fetchReposPage(page)
  repos.push(...response.repos)

  while (response.url) {
    response = await fetchReposPage(++page)
    repos.push(...response.repos)
  }

  return repos
}

async function fetchReposPage(page: number) {
  const response = await fetch(
    `https://api.github.com/users/${GITHUB_USERNAME}/repos?page=${page}&per_page=100`,
    { headers },
  )

  const repos = await response.json()
  const links = response.headers.get('link')

  let url: string | null = null

  if (links) {
    const next = links.split(',').find((link) => link.includes('rel="next"'))
    if (next) {
      url = next.split(';', 1).toString().slice(1, -1)
    }
  }
  return { repos, url }
}

I took some shortcuts — I used an iterating number instead of using the actual page link. It should still work, and there is no unknown data in the actual link. I just need to make sure it exists before trying.

Also, I could have totally made the fetching logic recursive, which might have looked cleaner than a while loop. But I don’t wanna stress over this too much. Overengineering is bleh.

Let’s try it now, and… it works! That didn’t take me multiple iterations to get it right at all 😇.

Well, I have all the data I could want for now. On to the next step.

Generating Markdown files

Astro consumes Markdown files, which allow me to also do some fancier rendering for each project rather than just showing text. You can add metadata (often called “frontmatter”) at the top of the file, like so:

---
title: Simple Scaffold
order: 200
links:
  - href: https://www.npmjs.com/package/simple-scaffold
    icon: logo-npm
    title: NPM
description: |-
  <p class="mb-0">A simple command to generate any file structure, from single components to entire app boilerplates.</p>
  <p>
    See
    <a href="/2019/03/simple-scaffold">my post</a>
    detailing more, or the
    <a href="https://chenasraf.github.io/simple-scaffold">documentation</a>
    /
    <a href="https://www.npmjs.com/package/simple-scaffold">NPM</a>
    page to start using it!
  </p>
---

<!-- Markdown content may go here -->

So, I want to generate a similar file, but populated with our new data from GitHub.

I also want to be able to add custom information or add overrides, which won’t be destroyed if I happen to want to re-fetch repositories. I’ll think about that later; for now, let’s get this new task done.

Creating the file

While this is Markdown at the bottom, the top looks a lot like a YAML file.

First thing’s first: I’ll move the original files to a different directory, so I don’t destroy them right now while working. Also, I will probably want to keep some of the data there as overrides or additions to what the repo info gives me.

The next step is to convert the structure I already have to YAML. I am trying to decide whether to use a package or just implement a simple string file writer for this. The latter option is cool, but I’m sure there will be many problems with correctly escaping quotes and other YAML tokens in the output, so I prefer just using a pre-made package for this. I quickly found yaml, and it seems like a perfect option for that.

This part was surprisingly simple — only a few lines:

async function generateProjectFile({ readme, ...project }: GitHubProjectSchema) {
  const file = path.join(projectsDir, `${project.name.toLowerCase()}.md`)
  const lines = ['---']
  lines.push(yaml.stringify(project).trim())
  lines.push('---')
  if (readme) {
    lines.push(readme)
  }
  return fs.writeFile(file, lines.join('\n'))
}

export async function generateProjectsList() {
  const projects = await getProjectsList()
  await Promise.all(projects.map(generateProjectFile))
}

That already generates all the files for me. Success! I can now revert some of the index code back to load it back from the original collection:

await generateProjectsList()
const projects = await getCollection('project')
projects.sort((a, b) => {
  return a.data.order - b.data.order
})

And also update the correct types for Astro:

// src/content/config.ts

const project = defineCollection({
  schema: GitHubProjectSchema,
})

export const collections = { project }

And now my files look like this:

---
name: simple-scaffold
description: >-
  Generate any file structure &mdash; from single components to entire app boilerplates, with a
  single command.
url: https://github.com/chenasraf/simple-scaffold
stars: 53
order: -53
links: []
---

<p align="center">
  <img src="https://chenasraf.github.io//simple-scaffold/img/logo-lg.png" alt="Logo" />
</p>
<!-- Rest of contents of the README.md (too big to show here) -->

This already looks pretty solid. If I just update the ProjectCard props and use it again, I should already see a nice list of projects. Let’s do that:

---
// src/pages/projects/index.astro
---
<div class="grid grid-cols-1 md:grid-cols-2 gap-8">
  {projects.map((project) => <ProjectCard project={project} />)}
</div>

---
// src/components/ProjectCard.astro
import type { CollectionEntry } from 'astro:content'
import ProjectLinks from './ProjectLinks.astro'
type Props = { project: CollectionEntry<'project'> }
const { project } = Astro.props
const { title, description, links } = project.data
---

<style>
  .project-card p {
    @apply mt-0;
    @apply mb-0;
  }
</style>
<div
  class:list={[
    'project-card cursor-default transition duration-300',
    'border dark:border-none',
    'shadow-generic hover:shadow-generic-hover',
    'dark:shadow-generic-dark dark:hover:shadow-generic-dark-hover',
    'rounded-2xl p-6',
  ]}
>
  <h3 class="mt-0">
    <a href={`/projects/${project.slug}`}>{title}</a>
  </h3>
  <div set:html={description} />
  <ProjectLinks links={links} />
</div>

Success! I see my GitHub projects all nicely laid out for me on the projects page.

Let’s update the details page so that clicking it will lead to the actual information.

There are 2 main things to change; first, let’s update the props so they reflect the new md files:

// src/pages/projects/[slug].astro
const post = Astro.props
const { name, description, links } = post.data
const { Content } = await post.render()

You will notice the const { Content } = await post.render() line there at the bottom.

When importing the md file as part of a collection, the actual contents of the md file are already parsed for Markdown and are returned as a component I can inject into the page.

The metadata at the top (the frontmatter) is added alongside the content as additional props for use.

Now we can simply plug it into the template:

---
// src/pages/projects/[slug].astro
// ...
---
<Page showTitle prose title={name} description={description ?? ''}>
  <p set:html={description} />
  <div class="no-prose">
    <ProjectLinks links={links} />
  </div>
  <Prose><Content /></Prose>
</Page>

We now have a fully functioning projects list, along with a singular project page for each repo. Great!

Enriching the data

The proof-of-concept is done. Now we can enrich the data a bit more, and maybe update the design. I also already cached the repository responses into JSON files, so that the process can start over later without having to re-fetch everything.

Let’s add some kind of filters to the project page. The easiest win is to add the GitHub link to the links array:

async function getProjectsList(): Promise<GitHubProjectSchema[]> {
  // ...

  for (const repo of repos) {
    const project = GitHubProjectSchema.parse({
      name: repo.name,
      title: repo.name,
      url: repo.html_url,
      description: repo.description,
      stars: repo.stargazers_count,
      order: -repo.stargazers_count,
      // There 👇🏼
      links: [{ href: repo.html_url, icon: 'logo-github', title: 'GitHub' }],
    })
  }

  // ...
}

Now, I’ll put my old files in a directory called project-overrides. We will use this to manually add override fields to any repo we choose.

We’ll start by loading the corresponding file:

if (await fileExists(overridesFile)) {
  const content = await fs.readFile(overridesFile, 'utf8')
  const allLines = content.toString().split('\n')
  const lines = allLines.slice(0, content.lastIndexOf('---')).join('\n')
  const obj = jsYaml.parse(lines) as GitHubProjectSchema
}

Then loading the overrides information there:

for (const link of obj.links ?? []) {
  const found = project.links.findIndex((i) => i.href === link.href)
  if (found >= 0) {
    project.links.splice(found, 1, link)
  } else {
    project.links.push(link)
  }
}
for (const key of ['title', 'order', 'description'] as const) {
  if (obj[key as keyof typeof obj] != null) {
    project[key as keyof typeof project] = obj[key as keyof typeof obj] as never
  }
}

I wanted to avoid duplicated links, so I let the overrides take precedence if there are any conflicts.

Filtering out projects

I also want to filter out any irrelevant projects. I’ll do that by making a few checks before I push into the projects array.

Something like this:

const overridesDir = path.join(process.cwd(), 'src', 'content', 'project-overrides')
const projectIgnoreFile = path.join(overridesDir, '.projectignore')
let projectIgnore: string[] = []

fs.readFile(projectIgnoreFile, 'utf8').then((content) => {
  projectIgnore = content
    .split('\n')
    .map((i) => i.trim())
    .filter(Boolean)
})

function projectFilter(project: Record<string, any>): boolean {
  if (projectIgnore.includes(project.name)) {
    return false
  }
  return [
    !project.fork,
    project.stargazers_count > 0,
    //
  ].every(Boolean)
}

As you can see, I exclude:

Project names listed in the ignore file
Forks
Projects without stars

I can always tweak this further if I need to, but the regular filters got me most of the way, while I filled the gap with old (but starred) projects in the ignore file.

Now, I can add it to the main repo loop, simply using continue to skip the current project:

async function getProjectsList(): Promise<GitHubProjectSchema[]> {
  const repos = await fetchRepos()
  const projects: GitHubProjectSchema[] = []

  for (const repo of repos) {
    if (!projectFilter(repo)) {
      continue
    }
    // ...
    projects.push(project)
  }
  // ...
}

Marking featured projects

Well, that’s easy — I can just add a new property to GitHubProjectSchema, called featured. It’s a simple boolean, and when I want, I can add an override for that field; there shouldn’t be many of them anyway, and it’s more likely they will already have overrides. Even if not, having a file with 3 lines isn’t that terrible.

Let’s update the types:

// src/types.ts
export const GitHubProjectSchema = z.object({
  name: z.string(),
  title: z.string(),
  description: z.string().nullable(),
  url: z.string(),
  stars: z.number(),
  readme: z.string().optional(),
  order: z.number(),
  links: z.array(LinkSchema),
  // here:
  featured: z.boolean().optional().default(false),
})

And add the implementation to the overrides loading:

// just add the 'featured' property to the list of overridable fields:
for (const key of ['title', 'order', 'description', 'featured'] as const) {
  // original logic
}

As for changing how featured projects are displayed, that’s up to you. Personally, I partitioned them into 2 arrays, and had the featured ones at the top with a bit of a difference in the design, while the other ones were separated to the bottom and also a bit smaller in size.

Finishing touches

I have the basic structure working well. I could call it a day, but I will also update some styles in the new pages, re-order the information, stuff like that.

There are also some other ideas I can think of to improve this:

Adding releases links
Adding license
Smarter filtering
Search feature
Show project tags
Populate with project URL from the repo info

But all of these can wait for another day.

Wrapping it all up

The last thing I need is to make sure it’s easy to update the project files whenever I need to make an update.

I have a CI that deploys the site on every push to master
I would like a way to trigger the changes without necessarily making an actual change to a real file

I have a few ideas in mind. To manually trigger a rebuild, I could do one of the following:

Use manually triggered GitHub Actions
Commit some sort of hash file, which will change when needed, causing the build to trigger
Add the files to gitignore, which will make sure every GitHub Action run goes and fetches the most up-to-date information

I think adding the files to gitignore and manually triggering GitHub Actions would solve both of my issues.

Updating the workflow is easy; all I have to do is add the workflow_dispatch event to be able to manually trigger my deploy action:

# .github/workflows/deploy.yaml

name: Deploy
on:
  push:
    branches:
      - master
  workflow_dispatch:

And to add the files to gitignore:

# .gitignore
src/content/project

Uh-oh

Seems there is a major snag.

In Astro, this all works pretty well in dev mode, but when using the astro build command, the newly created files aren’t noticed by the build system (actually not really, I should have noticed I need an extra refresh after building all the files — it was a hint).

Apparently, the file list needs to be refreshed, and I don’t think that’s quite possible.

So… what now?

I decided simpler is better. Let’s add this build step as a prebuild command, separate from Astro. This will make it work.

There is one caveat, and that is we can’t use any of Astro’s stuff. So, if I want the types in the script, I will have to replicate them (and worry about inconsistencies in the future) using my own zod import, not the one that is bundled with Astro:

// src/github-projects.ts
const LinkSchema = z.object({
  title: z.string(),
  href: z.string(),
  icon: z.string(),
})
type LinkSchema = z.infer<typeof LinkSchema>

const GitHubProjectSchema = z.object({
  name: z.string(),
  title: z.string(),
  description: z.string().nullable(),
  url: z.string(),
  stars: z.number(),
  readme: z.string().optional(),
  order: z.number(),
  links: z.array(LinkSchema),
  featured: z.boolean().optional().default(false),
})
type GitHubProjectSchema = z.infer<typeof GitHubProjectSchema>

Also, I needed to change the way I access env vars:

// from:
const GITHUB_TOKEN = import.meta.env.GH_TOKEN || import.meta.env.GITHUB_TOKEN!
// to:
const GITHUB_TOKEN = process.env.GH_TOKEN || process.env.GITHUB_TOKEN!

And then I created a standalone script to run this:

import { generateProjectsList } from '../github-projects'
generateProjectsList()

And made it run on pre-build in package.json:

{
  "scripts": {
    "prebuild": "tsx src/scripts/generate-projects.ts"
  }
}

I also removed the generate call from the astro pages.

And now it works!

I have also released this as a package. It has been updated and you can have more control over the filters, and it generally works better as it integrates directly into Astro. Check it out, or install right now using:

npm

npm i github-repos-astro-loader

pnpm

pnpm i github-repos-astro-loader

Yarn

yarn i github-repos-astro-loader

Conclusion

That’s it! You can take a look at my Projects page to see how it turned out.

While it’s not perfect, I think it’s an interesting direction to take, and there are many possibilities for improving how and what to display. Have you done anything similar for yourself? How do you make sure to keep your projects up to date? Leave a comment below telling us more.

Pathfinding Guide for 2D Top-View Tiles in Godot 4.3

Tue, 03 Sep 2024 12:00:00 GMT

Making your enemies run after your player character is a common task in 2D games. If you are using Godot and having trouble implementing pathfinding while respecting your TileSet collisions, you’ve come to the right place!

I love Godot. It’s a really good game engine. It has its flaws, sure; but it’s very flexible, it’s designed to give the developer a lot of control, and it’s still very easy to use. That said, 2D top-down navigation on TileMaps has proven to be a difficult task to research and plan correctly.

In this tutorial we will learn how to implement pathfinding, in a flexible and portable way, with avoiding obstacle tiles being the main focus point which was a pain for me.

Motivation

Godot 4.3 introduced a lot of new features. Among them are TileMapLayers nodes, which replace the old TileMap node’s layers property. This isn’t actually part of the problem; they mostly work the same as before.

The problem is, I was under the impression that if I give some tiles collision information, it would be used by the pathfinding and be avoided by the moving actors. That is not the case!

I actually got the most useful information in understanding how to use pathfinding by watching this video by Coding Quests on YouTube. His video provides a great basis, and I recommend you watch it.

This tutorial will do much of the same, but I will try to give you a more flexible way to handle obstacle tiles, so that it is more scalable for your game, and easier to manage and expand on.

Starting set-up

First thing’s first, we need to create some structure and understand how the nodes interact.

Let’s assume we have the following scene tree (I only included relevant nodes for brevity):

SceneBase (Node2D)
├── GroundLayer (TileMapLayer)
├── ActorLayer (TileMapLayer)
│   ├── Player (CharacterBody2D)
│   └── Enemy (CharacterBody2D)
├── Trees (TileMapLayer)
└── Rocks (TileMapLayer)

This is the base starting point. We have a ground layer, which has tiles like grass and dirt. We have an actors layer, inside it, we have two actors: the player and an enemy. We want the enemy to chase the player around, finding their path and avoiding blocked tiles on the way.

Below those, we have 2 more layers just so we can make sure our pathfinding is working across multiple layers automatically in a more generic, less constrained way than having to manually link them for each map scene.

Preparing the TileSets

For any navigable TileSets you plan to use, you will need to add navigation support to it, to allow pathfinding to work.

To do that, first, find one of the layers that uses the desired TileSet, and expand the TileSet’s details.

Then, expand the “Navigation Layers” section, and add a layer with the bitmask of your choice (just leave at the default if you’re unsure).

Once that’s done, go to the TileSet editor. Go to the “Paint” tab, and select a walkable tile, such as grass or dirt.

From “Paint Properties”, under “Navigation”, select the navigation layer you created (it should be the only one).

You can start painting on a few more tiles as you see fit. Any tile you mark will be walkable.

Implementing pathfinding

Now that the basic setup is out of the way, we can carry on. Pathfinding isn’t too tricky to implement in this case for just the base.

First, let’s add a child node to the enemy, of type NavigationAgent2D.

You may also create a nav node via code, or use an @export var as alternate methods to get a navigation node; but we’ll keep it simple in this tutorial.

Then, let’s attach a (or modify our existing) script for the enemy.

extends CharacterBody2D
class_name Enemy

var speed := 50.0 # can be anything

@onready var nav: NavigationAgent2D = $NavigationAgent2D # change if needed
@onready var player: CharacterBody2D = %Player # change if needed

func _ready() -> void:
	actor_setup.call_deferred()
	nav.velocity_computed.connect(_velocity_computed)

func actor_setup():
	# Wait for the first physics frame so the NavigationServer can sync.
	await get_tree().physics_frame

	# Now that the navigation map is no longer empty, set the movement target.
	set_movement_target(player.position)

func set_movement_target(movement_target: Vector2):
	nav.target_position = movement_target

func _physics_process(delta: float) -> void:
	_move_towards_player()

func _move_towards_player():
  pass

Before we implement the actual movement function, let’s go over what we have added so far.

In the _ready() function, we do 2 things:

Call the actor_setup function so the nav node is ready. We use call_deferred() to make it wait for the next frame before it is invoked. This assures us that the nav node is ready.
We connect the velocity_computed signal so we can get the nav node’s velocity for avoidance (if enabled).

The actor_setup function also does a couple of things:

Waits for 1 physics frame to wait for the nav agent to be ready
Sets the navigator’s target position to the player position

What does avoidance do?

What is avoidance, you ask? It is a feature of the NavigationAgent2D that allows it to avoid other navigation agents. You can define a radius to circle around agents on your path, and the path will be recalculated to avoid it. However, it does not relate to collision tiles or static obstacles. We will handle that a bit later on.

Implementing the movement logic

Getting back on topic, we will go ahead and implement the logic to move the actor using the navigation agent.

This is most of the logic involving the actual navigation agent, so let’s get going.

func _move_towards_player() -> void:
	# Update the player position
	set_movement_target(player.position)

	# If we're at the target, stop
	if nav.is_navigation_finished():
		return

	# Get pathfinding information
	var current_agent_position: Vector2 = global_position
	var next_path_position: Vector2 = nav.get_next_path_position()

	# Calculate the new velocity
	var new_velocity = current_agent_position.direction_to(next_path_position) * speed

	# Set correct velocity
	if nav.avoidance_enabled:
		nav.set_velocity(new_velocity)
	else:
		_velocity_computed(new_velocity)

	# Do the movement
	move_and_slide()

func _velocity_computed(safe_velocity: Vector2):
	velocity = safe_velocity

The comments should help you understand what we do on each section, but I want to explain a bit more about the avoidance lines:

func _move_towards_player() -> void:
	# ...
	# Set correct velocity
	if nav.avoidance_enabled:
		nav.set_velocity(new_velocity)
	else:
		_velocity_computed(new_velocity)
	# ...

func _velocity_computed(safe_velocity: Vector2):
	velocity = safe_velocity

If the nav’s avoidance is enabled, we don’t directly set the velocity of the actor. We instead pass the calculated velocity to the nav agent itself. The agent will use it to handle the next calculations in order to avoid other nav agents, and report back when it has a new “safer” path.

When the _velocity_computed function is called (either by us or by the velocity_computed signal of the nav), we simply set the velocity of the actor. Avoidance will make sure to call the velocity_computed when necessary, which is why we don’t manually set it in the case it is enabled.

It’s important to note that only NavigationAgent2D are considered for avoidance, not any obstacle. For that, just continue reading.

Testing our setup

This setup should work for the initial test. Just run the game and see if it works.

To make it more visual, you can go to the navigation agent of the enemy, and check the “Debug” property; which will show us the paths the enemy is trying to follow. Also, in Godot’s top menu, under “Debug” -> check “Visible Navigation”, which will highlight which tiles are used as possible paths and which aren’t.

Here’s how it looks right now:

Ok, that works nicely. How would we go about removing obstacle tiles, though?

Obstacle tiles

Some tiles, such as rocks and trees, are meant to be obstacles which block your (and others’) path. But unfortunately (and this is an area where Godot can definitely improve), linking the navigation area to tile collisions isn’t possible out of the box without some code.

Let’s see how we need to implement that.

Preparing the TileSet

One thing we need is to set some collision on the relevant tiles.

In your TileSet’s inspector, add a Physics layer, similar to how we added a Navigation layer:

Then, once again, go to the TileSet panel, click “Paint”, but this time, under “Physics Layers” choose the layer you just added:

Now when you paint tiles you will make them block your path and your enemies’, if they try to walk through it.

They will still try to walk through these tiles, because the navigation isn’t aware of the physics there. This is where we do a little bit of magic.

To your ground layer, attach a new script. Call it whatever you like, I picked NavigationTileLayer.

In this script we need to do a few things:

We need to get information about all the relevant layers (ones that contain obstacles)
We need to tell the navigation system that any tiles with collision should be removed from the nav area

In order to do that, we’ll write a little bit of code. Not much more, I promise!

To get the information on relevant layers, there are many possibilities:

Put them all in the same group, and use get_tree().get_nodes_in_group()
Find any sibling nodes with the TileMapLayer type
Use an @export var and manually put the relevant layers in there (less recommended)

To make it super simple, I think we’ll go with the first option.

First, select your TileMapLayers in the scene. Then in the inspector pane, click the “Node” tab, then “Groups”.

Simply add a new group with the ”+” button:

and add them to the Obstacles group (change the name if you want but just make sure to also change it in the script).

This is how it will look when added (it’s your choice whether to make this group global or scene local):

Great, now let’s add a function for that in our script:

extends TileMapLayer

var _obstacles: Array[TileMapLayer] = []

func _ready() -> void:
	_get_obstacle_layers()

func _get_obstacle_layers():
	# make sure the name here is the same as the group's
	var layers = get_tree().get_nodes_in_group("Obstacles")

	for layer in layers:
		if layer is not TileMapLayer: continue
		_obstacles.append(layer)

Ok, great! Now the next step is to remove navigation from collision tiles.

We will need a couple of things for that. First, we need to implement the _use_tile_data_runtime_update function.

func _use_tile_data_runtime_update(coords: Vector2i) -> bool:
	return _is_used_by_obstacle(coords)

This function is an override for a built-in for TileMapLayer. It should return a bool, which represents whether the cell at coords needs to have its data updated in runtime.

It makes sure we don’t run excess updates on every cell and only update what we need. You probably noticed I added a stub _is_used_by_obstacle function there, let’s implement that now:

func _is_used_by_obstacle(coords: Vector2i) -> bool:
	for layer in _obstacles:
		if coords in layer.get_used_cells():
			var is_obstacle = layer.get_cell_tile_data(coords).get_collision_polygons_count(0) > 0
			if is_obstacle:
				return true
	return false

Our logic here is pretty simple:

Iterate over every layer in _obstacles
For each layer, check if the coords correspond to a cell
- If there is a cell in that layer on those coords, check if it is an obstacle:
  - We do the check by getting the tile data, and retrieving the collision polygons count for it
  - If it has more than 0 (i.e., it has a collision shape), it is an obstacle
  - The 0 we pass to the function is the index of the physics layer we created on the TileSet, so if you had previous layers, this is what you would want to change
  - Return true if it is an obstacle
  - Continue otherwise
If no obstacles were found and we reached this point, then we return false

It’s an efficient way to attach the information from the TileMapLayer into the navigation area. We don’t need any custom data layer or any additional setup for the tiles other than the collision (which we would want anyway, so our player would be blocked).

Now for the last piece of the puzzle, we just need to actually set the navigation polygon on the tile to accommodate to the walkable state of the tile. Add a new method on the script:

func _tile_data_runtime_update(coords: Vector2i, tile_data: TileData) -> void:
	if not _is_used_by_obstacle(coords):
		tile_data.set_navigation_polygon(0, null)

This is another built-in method in Godot that will only run on the tiles that have returned true inside the previous _use_tile_data_runtime_update method. It will only try to update the tiles that are relevant, in order to save on memory. Since it will only run on tiles we already queried that are not walkable, it won’t try to update every tile.

Last test

Let’s run the game one more time - we can see that any tiles that contain collision are not highlighted in the navigation debug. We can also notice our agent finding ways around the obstacles. Success!

Conclusion

I believe this is a pretty flexible system. Basically, if we need to add more TileSets or tiles to the same set, we just need to make sure of two things:

The layer is grouped (or otherwise connected to the NavigationTileLayer script’s obstacles)
We have collision polygons on the required tiles

Other than those, no special set up is required. This means you can essentially put the script anywhere, even on a completely separate node, and re-use that same node across all your maps with minimal effort. And if you don’t want to use groups, you can find other ways to get the correct layers to use.

Hopefully this tutorial helps you understand a bit more about the TileMapLayer nodes, about NavigationAgent2D, and about how to use tile data to your advantage. Personally, I had trouble finding resources that properly explain how to handle 2d obstacles using pathfinding, so I thought it could be a nice write-up for others to learn from.

Do you have ideas for improvement on this system? Do you have any questions? Please leave a comment!

Distribute Open-Source Tools with Homebrew Taps: A Beginner's Guide

Mon, 27 Jan 2025 08:00:00 GMT

Often, when I create open-source tools or apps, it’s tricky to distribute them easily. You usually need to specify where to download from, and where to place the extracted files, which is a manual process that doesn’t usually translate into a simple script.

That’s where Homebrew comes in. It makes it easy to package your tools in a way that works on both macOS and Linux and installs them for you. Homebrew taps are user-created repositories that simplify distribution. Instead of asking your users to manually download and configure files, you can provide them with a single command to install your tool. This seamless experience can make your app more approachable for users and easier for you to manage.

There are plenty of package managers – but I think Homebrew is one of the simpler ones to publish for, believe it or not, which aren’t tied to a specific development language or platform (such as Node’s NPM, or Dart Pub).

In this post, we’ll go over how to get your own tap up and running, without having to rely on the core repository, or having to fulfill the minimum requirements to have your app uploaded to it.

What are Homebrew taps?

A tap is Homebrew’s term for user-made repositories of software, separate from the main Homebrew repository.

Without having to be verified by Homebrew’s team, and without the minimum standards (your software’s popularity, for example), you can publish your apps on your own repository, and easily point your users to install using 1 or 2 lines of shell code.

For example, to use my own tap, you can either add it as a repository to Homebrew:

brew tap chenasraf/tap

And then install a package:

brew install sofmani

Or, you can directly install from the tap without adding it:

brew install chenasraf/tap/sofmani

So, how can you make this work for your own apps?

Step 1: Setting up a tap repository on GitHub

First thing’s first, you need a GitHub repository to hold your tap. The first requirement is properly naming your repository.

If you prefix the name of your repository with homebrew-, the rest of the name becomes your tap name, which will be under your username.

For example, for my sample above, the repository I created was named chenasraf/homebrew-tap, which then translates into chenasraf/tap for brew.

I suggest a similar naming scheme. I personally don’t think it’s a good idea to have more than 1 main tap, as publishing and maintaining multiple repositories might prove to be a hassle. Also, if all your apps are accessible under the same tap, you can direct your users more easily and have them install with less possible commands and actions.

Step 2: Adding GitHub Actions

If you’re not familiar, GitHub Actions are a way to run automatic processes on your repository, in the server, which also allows you to report success or failure and block PRs or have more actions done based on the results. It’s often used to automate testing and releasing software on GitHub whenever something is pushed to a repository or if it fills certain criteria.

Homebrew supplies us with a few GitHub actions to help us with the publishing process. These, in addition to publishing, run some tests and pre-requisites to make sure everything is set up properly and that your tap will be accessible and usable without errors.

The first action is the brew test bot pipeline. This action will run some tests whenever you push to your repository, making sure test run and files are formatted properly. This action also creates Bottles for your app for various platforms. We will discuss Bottles in just a bit.

To add this action, you can paste this into .github/workflows/test.yml:

Test Action

name: brew test-bot

on:
  push:
    branches:
      - master
  pull_request:

jobs:
  test-bot:
    strategy:
      matrix:
        os: [ubuntu-22.04, macos-13, macos-14]
    runs-on: ${{ matrix.os }}
    steps:
      - name: Set up Homebrew
        id: set-up-homebrew
        uses: Homebrew/actions/setup-homebrew@master

      - name: Cache Homebrew Bundler RubyGems
        uses: actions/cache@v4
        with:
          path: ${{ steps.set-up-homebrew.outputs.gems-path }}
          key: ${{ matrix.os }}-rubygems-${{ steps.set-up-homebrew.outputs.gems-hash }}
          restore-keys: ${{ matrix.os }}-rubygems-

      - run: brew test-bot --only-cleanup-before

      - run: brew test-bot --only-setup

      - run: brew test-bot --only-tap-syntax

      - run: brew test-bot --only-formulae
        if: github.event_name == 'pull_request'

      - name: Upload bottles as artifact
        if: always() && github.event_name == 'pull_request'
        uses: actions/upload-artifact@v4
        with:
          name: bottles_${{ matrix.os }}
          path: '*.bottle.*'

A brief overview of the above:

We run this job on 3 different platforms, Ubuntu 22, and macOS 13 and 14.
We set up Homebrew, and caching to quicken the process
We run some of brew’s test-bot functionality, which includes making some tests and building the binaries for each of the matrix platforms

Now every push to your repository will run the tests, and create bottles for your app. These bottles will be re-used when you release the next version of your app.

Publish Action

The next action is where the magic happens. Paste the following into .github/workflows/publish.yml:

name: brew pr-pull

on:
  pull_request_target:
    types:
      - labeled

jobs:
  pr-pull:
    if: contains(github.event.pull_request.labels.*.name, 'pr-pull')
    runs-on: ubuntu-22.04
    permissions:
      contents: write
      pull-requests: write
    steps:
      - name: Set up Homebrew
        uses: Homebrew/actions/setup-homebrew@master

      - name: Set up git
        uses: Homebrew/actions/git-user-config@master

      - name: Pull bottles
        env:
          HOMEBREW_GITHUB_API_TOKEN: ${{ github.token }}
          PULL_REQUEST: ${{ github.event.pull_request.number }}
        run: brew pr-pull --debug --tap="$GITHUB_REPOSITORY" "$PULL_REQUEST"

      - name: Push commits
        uses: Homebrew/actions/git-try-push@master
        with:
          token: ${{ github.token }}
          branch: master

      - name: Delete branch
        if: github.event.pull_request.head.repo.fork == false
        env:
          BRANCH: ${{ github.event.pull_request.head.ref }}
        run: git push --delete origin "$BRANCH"

This action will create a release for you when your app is ready to be released.

A brief overview of the above:

We set up homebrew & git
We pull the previously-created bottles, so we can use them in this action
We tell Homebrew to push new commits containing the downloaded bottles, and previous existing commits
We then delete the source branch

Now, push your changes to your repository, and then create a new branch. This branch will be used to create a PR that will be the release PR for your app.

Step 3: Creating your first Formula

In Homebrew, “Formulae” (plural) or “Formula” (singular) are the actual individual app installers. A formula tells brew where to fetch your code from, and how to install it into the user’s system.

Each of your apps should have a formula. For our example, I will show you how to create a build for a Go app – but this can be modified as needed for any other type of app.

For this, you will need a tagged release for your own app, whatever it is written in. For our example, we can reference sofmani v1.4.0.

A few notes:

A downloadable release is not required – we will be instructing Homebrew to install from the source tar file which we will fetch.
A tag is not required, any ref can work, such as a branch or commit hash – but it’s recommended to use a tag to make it easy to follow the correct version.

Creating a formula file

To create a new formula file, you can use the brew create command. This will create a new formula template that you will have to modify.

The command should be run as follows:

brew create <source tar.gz URL>

To use our example from before, in our case we will use:

brew create https://github.com/chenasraf/sofmani/archive/refs/tags/v1.4.0.tar.gz

This will start a prompt that will ask for some details about your app. Fill them up and you will be brought into your shell’s editor to edit the formula file. Your file should be located at /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/<first-letter-of-name>/<app-name>.rb, or a similar directory somewhere else depending on your Homebrew setup.

Take note of your file and move it into your repository. In the state it’s currently in, it is meant to be pushed to your fork of Homebrew, so that you can create an official PR to add it to the core repository – but for our case, we are only publishing this for our own repository.

Your file should be placed in the Formula directory of the repository. To make it easier to sort a lot of apps, you can divide your formulae to separate directories, where each directory is the first letter of the apps below it.

As an example, for our case we can use Formula/s/sofmani.rb or just Formula/sofmani.rb. I chose the latter because I don’t think I am close to having enough apps there to require more splitting up.

Updating the file

Now that we have a file in place, it should look like this:

# Documentation: https://docs.brew.sh/Formula-Cookbook
#                https://rubydoc.brew.sh/Formula
# PLEASE REMOVE ALL GENERATED COMMENTS BEFORE SUBMITTING YOUR PULL REQUEST!
class Sofmani < Formula
  desc "Installs software from a declerative config on any system "
  homepage "https://chenasraf.github.io/sofmani/"
  url "https://github.com/chenasraf/sofmani/archive/refs/tags/v1.4.0.tar.gz"
  sha256 "0cb4d1f51b23f272fd8f5146e30f937e7cd9ff1275fbcc732a6f27e017c0426a"
  license "MIT"

  # depends_on "cmake" => :build

  def install
    # Remove unrecognized options if they cause configure to fail
    # https://rubydoc.brew.sh/Formula.html#std_configure_args-instance_method
    system "./configure", "--disable-silent-rules", *std_configure_args
    # system "cmake", "-S", ".", "-B", "build", *std_cmake_args
  end

  test do
    # `test do` will create, run in and delete a temporary directory.
    #
    # This test will fail and we won't accept that! For Homebrew/homebrew-core
    # this will need to be a test that verifies the functionality of the
    # software. Run the test with `brew test sofmani`. Options passed
    # to `brew install` such as `--HEAD` also need to be provided to `brew test`.
    #
    # The installed folder is not in the path, so use the entire path to any
    # executables being tested: `system bin/"program", "do", "something"`.
    system "false"
  end
end

Note that yours will differ slightly, but the gist is the same.

There are 3 main sections we currently want to update.

The install block
The test block
The depends_on block

Install block

The install block will be used to install the software. This is where you tell Homebrew how to build the app from source. Since we are demonstrating a go app, we can replace the block with the following:

def install
  system "go", "build", "-buildmode", "exe", "-o", "sofmani", "." # Build from source
  bin.install "sofmani" # Install the bin into the required directory
end

It’s pretty straightforward, but just to make sure we understand what’s going on:

First, we use system to run a shell command. This would be the regular build command for your app. You can run several commands here, use conditions, and do whatever else you need to get your app properly built from source. You will notice I used "." for the current directory as target.

Second, we use bin.install which tells brew to act on the directory bin, and installs a binary into the homebrew prefix location that it needs to be in for that directory. In our case we supplied it the output binary name, "sofmani".

Test block

Here we just need to test that the software is installed and works as expected. This isn’t the best practice but for our example we will make a short call to the app’s help and check that the output doesn’t crash.

test do
  assert_match "Usage: sofmani [options] [config_file]", shell_output("#{bin}/sofmani -h")
end

This adds a simple string match check against the help output.

Fore more information about adding tests and more matchers, you can refer here: Formula Cookbook — Add a test to the formula

Depends on block

We also need to tell brew that if the user builds from source, they would require go to be present in the system:

depends_on go: [:build]

This tells Homebrew to require the go system-wide executable, which can be added either by Homebrew or in a different method by the user. Limiting it to :build makes sure that the requirement is only relevant if building from source, not when downloading from bottle.

For more information about how the dependencies work (for example, requiring a specific version, or other steps or options) you can read more here: Formula Cookbook — Check for dependencies

Step 4: Releasing a new version

Make sure to update any other required information, remove all the unnecessary comments from the code and push your changes. Make sure you are using an alternative branch and not your default branch.

For every new version of app, you should do the following steps:

Make your changes
Push to a new branch
Create PR from your branch to the default branch

This is the process to start a release.

Once all the tests on your PR pass properly (push new commits with fixes if needed), all you have to do is add the pr-pull tag to your PR.

This will instruct the release action (release.yml) to start a release process. It relies on previous actions to be completed, as it will also attach bottles to the release if they were properly created.

When the action is done, you will notice your PR was closed without merging. This is working as expected, and is a side-effect of the brew bot creating additional commits for you and then pushing them to the target branch directly alongside your original commits, instead of merging from your branch through the PR.

Bottles

You might have noticed that in the extra commit that the bot created, it added a bottles section to the Formula file, which contains the pre-built binaries for your app:

bottle do
  root_url "https://github.com/chenasraf/homebrew-tap/releases/download/sofmani-1.4.0"
  sha256 cellar: :any_skip_relocation, arm64_sonoma: "4941252086eb62660b684f1210dbec8d9bfd7a343f2fca43a8cc19dee64bb38b"
  sha256 cellar: :any_skip_relocation, ventura:      "7b7c96d587d956219f30089af88e416adfef5b039f2f844b241a766d6e8016d1"
  sha256 cellar: :any_skip_relocation, x86_64_linux: "39fd3e306f66b7c04a3abe0e8e5dbe2070ef9c72fdd1bdefde2a1807d22082ae"
end

Bottles are simply pre-built binaries that you can install from the tap directly, without building from source.

If a user tries to install your tap on one of the supported platforms, they will have a faster process by downloading the matching binary that was “bottled up” during the release process.

If they can’t find a proper match, or if they choose to --build-from-source, then they will build the app from source for their system, including any necessary dependencies.

Addendum: Updating an existing app

Some time has passed, and we want to update the app to version 1.5.0.

For this we need to change 2 things:

The download URL for the tar file
The SHA256 checksum for the tar file

-  url "https://github.com/chenasraf/sofmani/archive/refs/tags/v1.4.0.tar.gz"
-  sha256 "0cb4d1f51b23f272fd8f5146e30f937e7cd9ff1275fbcc732a6f27e017c0426a"
+  url "https://github.com/chenasraf/sofmani/archive/refs/tags/v1.5.0.tar.gz"
+  sha256 "7a488daa01d999ff2e1a310ce351e3d0d3add4e18c709625b61e7b67b554b90b"

To generate a new checksum, you need to download the tar file, and get the checksum for that:

curl -L https://github.com/chenasraf/sofmani/archive/refs/tags/v1.5.0.tar.gz | shasum -a 256

Once you made these changes, you are ready to push them to a new branch, as before, create the PR, and then when the actions are completed successfully you just add the pr-pull tag to your PR.

And there you go! You have released your app to the world.

Addendum: Simplifying the process by automating more of it

I like to automate stuff more, so I created 2 scripts to help:

Download the latest release, get a checksum, and update the file
Create a PR with the proper branch

Here are the 2 scripts I use:

#!/usr/bin/env bash

if [[ -z "$REPO_NAME" ]]; then
  read -r -p "Enter repository path: " REPO_NAME
fi

if [[ -z "$REPO_NAME" ]]; then
  echo "Repository path is required"
  exit 1
fi

VERSION=$(curl -s "https://api.github.com/repos/$REPO_NAME/releases/latest" | jq -r .tag_name)
URL="https://github.com/$REPO_NAME/archive/refs/tags/$VERSION.tar.gz"
APP_NAME=$(basename "$REPO_NAME")

echo "Version: $VERSION"
echo "URL: $URL"

curl -Ls "$URL" -o "$APP_NAME-$VERSION.tar.gz"
echo -n "SHA256: "
hash=$(sha256sum "$APP_NAME-$VERSION.tar.gz" | awk '{print $1}')
echo "$hash"
rm "$APP_NAME-$VERSION.tar.gz"

sed -i.bak "s/sha256 \".*\"/sha256 \"$hash\"/" "Formula/$APP_NAME.rb"
rm "Formula/$APP_NAME.rb.bak"

sed -i.bak "s| url \".*\"| url \"$URL\"|" "Formula/$APP_NAME.rb"
rm "Formula/$APP_NAME.rb.bak"

I can either run this directly to get the interactive version or add the REPO_NAME variable ahead of time to pre-populate it and make it completely automatic.

REPO_NAME=chenasraf/sofmani ./scripts/update-hash.sh

And here is the second step of the process, which is to create the PR properly:

#!/usr/bin/env bash

if [[ -z "$REPO_NAME" ]]; then
  read -r -p "Enter repository path: " REPO_NAME
fi

if [[ -z "$REPO_NAME" ]]; then
  echo "Repository path is required"
  exit 1
fi

VERSION=$(curl -s "https://api.github.com/repos/$REPO_NAME/releases/latest" | jq -r .tag_name)
APP_NAME=$(basename "$REPO_NAME")

echo "Version: $VERSION"
BRANCH="feature/$APP_NAME-$VERSION"

if ! git switch -C "$BRANCH"; then
  echo "Branch already exists, aborting"
  exit 1
fi
git add "Formula/$APP_NAME.rb"
git commit -m "feat: update $APP_NAME to $VERSION"
git push --set-upstream origin "$BRANCH"

if gh pr create --fill; then
  open -u "$(gh pr list --json url | jq -r '.[0].url')"
  git switch master
else
  echo "Couldn't create PR, aborting"
  exit 1
fi

Again, I can use the variable to make life a bit easier:

REPO_NAME=chenasraf/sofmani ./scripts/create-pr.sh

We can combine the 2 and automatically get the latest release pushed for any app:

REPO_NAME=chenasraf/sofmani ./scripts/update-hash.sh && ./scripts/create-pr.sh

How’s that for making it simple? With one line (which you can wrap in a Makefile or any similar choice if you want), you can update the tap and create a PR in one step.

Conclusion

We created a tap, pushed a new app to it, and also pushed an update.

Now that you have an app on your tap, you can simply instruct your users to use it instead of downloading the release manually or figuring out what to do with the files.

It’s also pretty widely used on macOS and so it’s almost guaranteed to simplify things for a lot of your potential users, or for yourself!

Share your taps with us in the comments, or if you have more tips, or questions feel free to reply below to let everyone know!

casraf.dev

Flutter Tutorial - Creating a Wheel Spinner Widget

In this tutorial, we will:

Step 1: Create the Widget

Let’s give it some shape

Step 2 - Detecting gestures and updating the value

1. onDragStart

2. onDragUpdate

3. Testing the current widget

4. onDragDone

Done!

Simple Scaffold: Generating multiple files for faster coding

What have we got?

Enter: Simple Scaffold

Example: Jekyll Posts

You can do more

Placeholder.photo: An easy way to add mockup images to your UI and web development workflow

What is placeholder.photo?

Why would I need this?

How does placeholder.photo help me solve this?

The 5 Steps of Completion: Learning to break coding problems down

Learning to put theory to practice

Defining a “problem”

The methodology - The 5 Steps of Completion

1. Ask Questions

2. Give Answers

3. Break it down

4. Create Actionables

5. Solve It

When in doubt, look it up

Any progress is important: No task is too small

The next steps

How to use React memoization hooks for increased performance

React.useMemo

React.useCallback

Caveats

Other things to consider

React: Using useEffect Effectively

What is React.useEffect?

What are side effects?

React side-effects example

A replacement for lifecycle methods

Structure of the useEffect hook

Thinking “Reactive”

Dependencies: [] !== undefined

How can we use this now?

Component mounting

Reacting to a state change

The cleanup function, timeouts and intervals

External listeners

Conclusion

Create your own useOptimistic hook

Why create your own?

Looking at React’s original implementation

Creating a hook

Creating the update functions

What else can we add? Rolling back on failure

Fixing the stale value

Simplifying usage

Saving the promise state

Where to go from here?

Create an Automated Portfolio Using GitHub and Astro

I have updated the code in this article to use Astro loaders/Content Layer, and release it as a package on NPM!

The goal

The plan

Getting the data

Using GitHub API to grab repositories

Handling pagination

Generating Markdown files

Creating the file

Enriching the data

Filtering out projects

Marking featured projects

Finishing touches

Wrapping it all up

Uh-oh

npm

pnpm

Yarn

Conclusion

What is `React.useEffect`?

Structure of the `useEffect` hook

Dependencies: `[] !== undefined`