Backstory​

A long time ago, we've come to the point where our monolithic Ruby backend in Productboard stopped scaling as the company grew so the decision was made to introduce microservice architecture. That always poses some challenges, yet the tooling to deploy multiple services into the Kubernetes cluster was established quite soon. But what about running the application locally during development?

Naturally, it was no longer feasible to teach everyone how to run every service on bare metal, and expect to make it work without hassle. So dockerization was the most convenient next step. To automate the whole local setup, we've created "toolkit", a set of bash scripts and docker-compose files to install commonly used binaries, spin up the application in Docker Desktop, initialize the database, and other tasks.

But with time and an increasing number of services even this solution stopped working - we had to maintain two different formats of the configurations for every service, maintenance of routing rules for nginx container lagged behind ingress definitions and Kong decK configs and the overall developer experience wasn't very high to say at least.

For these reasons, we decided on the next step in the evolution: use Kubernetes for the local environment as well to reuse what we already have for staging and production.

But is that enough to increase the developer experience and cut the inefficient time spent by developers to fix their environment? When you start with Kubernetes as a developer, you suddenly have to grasp an understanding of a plethora of abstractions: what is the pod, deployment, service, ingress, volume claim, secret and similar.

We desired to strip our colleagues from product teams from all the complexity of the infrastructure so they can be focusing on features, customer feedback, and user experience. We wanted to offer them a better interface than that provided by bash scripts so we picked Tilt.dev as a layer between our customers - developers and the harsh underworld beneath.

About Tilt.dev​

At the first glance, Tilt is similar to tools like Skaffold, it just does much more for us. When you run tilt up to trigger the local deployment, you aren't provided with much terminal output, you get the link to the Web interface instead:

This is in my opinion the direction such tooling around Kubernetes should follow. Even without a deep understanding, I immediately have an overview of my environment, how many services are deployed, and which are still spinning up. And if something fails, I can trigger a redeploy.

Flexibility of Tiltfile​

We've been relying on Tilt for over the last 6 months and one of the things we appreciate about it since then is that it uses Starlank for configuration, a language originally built for the Bazel build ecosystem. Having the power of Python-like language shines especially in comparison to alternatives like DevSpace which uses yaml for configuration.

For example, you can use conditional statements and it all feels natural:

if current_namespace() != "productboard":
    fail('You must be on "productboard" namespace! Switch by executing "kubens productboard".')

Another great case is that you can create a new repository with shared functions and load them in every project as an extension:

v1alpha1.extension_repo(name = "pb-extensions", url = "https://github.com/productboard/our-tilt-extension")
v1alpha1.extension(name = "k8s", repo_name = "pb-extensions", repo_path = "k8s")
load("ext://k8s", "add_helm_repos")

add_helm_repos({
    "bitnami": "https://charts.bitnami.com/bitnami",
    "ingress-nginx": "https://kubernetes.github.io/ingress-nginx",
    "elastic": "https://helm.elastic.co",
})

If you already tried Tilt, you might notice there is already helm_repo extension provided by the team behind Tilt that does the same thing. This is actually one of our "specialties" we came to as helm_repo adds a new resource to UI:

Somehow most of your engineers found it quite confusing, therefore our custom-made add_helm_repos function uses local function underneath to add a helm chart:

local("helm repo add {} {}".format(name, url))

The function local runs an arbitrary command on a host machine before deployment and always outputs to a special (Tiltfile) resource which makes it great for any tasks to prepare the environment without distracting developers.

Tilt Web UI​

Speaking of Web UI with a list of resources, we probably appreciate the most a button to rotate specific services when things go awry:

Moreover, UI can be extended with additional buttons that can trigger specific actions on these services. My personal favorite is adding "Copy connection string" on database resources:

cmd_button(
    name = "{}-connection".format(database_service_name),
    argv = [
        "bash", 
        "-c", 
        "echo -n postgres://{}:{}@localhost:{}/{} | pbcopy".format(user, password, port, dbname),
    ],
    text = "Copy connection string",
    icon_name = "link",
    resource = database_service_name,
)

A very simple thing, but greatly improves the experience when one wants to swiftly connect to the DB.

UI button can be also extended by input fields, but I found its interface rather counterintuitive at this moment for most of the use cases - we would like to run some of the commands with parameters provided by the developer but instead of prompting whenever they click, it's necessary to click on the arrow to open the dropdown, fill in the values, close it and then click on the button to trigger the action.

Additionally, it would be cool to have an overview of all such action buttons in one place, similarly to commands in DevSpace:

Nevertheless, we expanded our usage of Tilt far beyond Kubernetes, e.g. this is how we run webpack server for the frontend application:

def run_with_nvm(script):
  return 'export NVM_DIR="${{NVM_DIR:-$HOME/.nvm}}" && source $NVM_DIR/nvm.sh && nvm install && corepack enable && nvm use && {}'.format(script)

def run_webpack(feApp, port):
  script = run_with_nvm("yarn nx serve --port {} {}".format(port, feApp))

local_resource(
  'Main app: webpack',
  cmd=run_with_nvm('yarn install'),
  serve_cmd=run_webpack('main', 4200),
  readiness_probe=probe(
      period_secs=5,
      http_get=http_get_action(port=4200, path="/")
  ),
  allow_parallel=True
)

We use the same local_resource function also for other tasks like running the Relay compiler in watch mode to recompile GraphQL queries:

local_resource is quite a versatile function that can spin up both long-running programs and one-time scripts. The only peculiarity is how to stop such long-running programs - you have to disable them:

There's one last thing we're missing a lot and that's the ability to execute commands in the pods directly from UI. Tilt shows pod ids conveniently but having such a possibility would mean that we no longer need to leave Web UI and switch back to the terminal to use kubectl exec command. This is on the contrary feature solved well in DevSpace.

These are some nitpicks that would deserve further improvements but running both backend services and local processes like Relay compiler above or unit tests in a single unified UI is a great step.

Summa summarum: Tilt proved to be an irreplaceable part of our platform for the local environment and I believe it's exactly the direction where things should move.

There are people with various roles in the company and only a fraction of them understands Kubernetes in depth. The goal of the majority of people is to deliver delightful product and not to waste time on debugging their local environment. Tilt so far seems to be a good tool to bring up the working station to life, oversee all deployed services, and spot potential problems soon.

Docker has long established itself as a standard for containerization and local development. So much that one would almost take it for granted, just like bash or basic UNIX commands. But with announced updates to Docker Desktop terms & conditions, it made me think: Can we even breathe without it on Macs, especially when we need to run multiple services locally? Let's explore possible alternatives...

We have to start first by making a distinction here - license changes apply to Docker Desktop, the application that enables us to use Docker on Macs or Window machines, not to Docker Engine itself, which is an actual open-source client-server application to build and run containers.

Secondly, we have to ask ourselves for which tasks do we need Docker? Most likely in two scenarios:

• to build images
• to run containers in local k8s cluster

We could start with Minikube and HyperKit which seemed to be the most frequent alternative. Unfortunately, it doesn't play nice with the new Apple M1 chips. Driven by impulse from colleagues, it pushed me to further and discover Multipass VM with MicroK8s.

Multipass VM​

Multipass is a tool from Canonical to run the virtual machine (VM) that can be managed from the terminal with a single command. Let's get it with Homebrew:

brew install multipass

Now we're able to spin up our VM. What's more, to set it up, we can use cloud-init:

multipass launch --name microk8s-vm --mem 4G --cloud-init local-k8s.yml

I've prepared all that's needed in local-k8s.yml as below:

#cloud-config

# packages to install (via snap)
snap:
  commands:
    - [install, docker]
    - [install, microk8s, --classic]

# create the docker group
groups:
  - docker

# Setup default ubuntu user & enable SSH access
users:
  - name: ubuntu
    groups: sudo, docker
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh-authorized-keys:
      - ssh-rsa <your-public-key>

# make sure firewall is not a problem
runcmd:
  - sudo iptables -P FORWARD ACCEPT

Once the execution is finished, we should get all the required pieces: VM with Docker Engine installed, user rights set according to recommended setup and MicroK8s to run local k8s cluster. My DevX heart rejoices.

Let's perform a quick check:

multipass exec microk8s-vm  -- docker --version

Voilá! It should print Docker version:

Docker version 20.10.8, build 3967b7d28e

MicroK8s​

MicroK8s is lightweight setup of Kubernetes. If you check our cloud config again, you can see that we've already installed it from snap, therefore as the next step, let's also set up microk8s command to manage that Kubernetes cluster within the VM:

brew install ubuntu/microk8s/microk8s

Because we named our Multipass VM as microk8s-vm, we're ready to go. Verify it as below:

microk8s status

There is one gotcha though if we try to run kubectl:

$ microk8s kubectl get pods
error: stat /Users/jaroslavkubicek/.microk8s/config: no such file or directory

Because we haven't initiated the VM by microk8s install as stated in official docs, we need to create the config ourselves. Fortunately, it's not much a fuss:

microk8s config > ~/.microk8s/config

Building Docker images in Multipass VM​

If you look back into cloud-init config, you may also notice we provided SSH public key. We did so to be capable of building Docker images outside of Multipass VM, on our Mac.

Under normal circumstances, Docker client is likely configured to talk to Docker Engine daemon from Docker Desktop. When Docker Desktop is not running or not installed, it's not possible to build images:

docker build .

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. 
Is the docker daemon running?

But Docker client command supports contexts. So in order to build images in our VM, we can create a new one:

docker context create multipass \
  --description "Docker Engine in Multipass" \
  --docker "host=ssh://ubuntu@192.168.64.2"
docker context use multipass

Docker client will now use ssh to build images inside the VM. Note that it's necessary to ssh into the machine at least once beforehand to confirm the authenticity:

$ ssh ubuntu@192.168.64.2 # IP of VM, check yours with "multipass list"

The authenticity of host '192.168.64.2 (192.168.64.2)' can't be established.
ED25519 key fingerprint is SHA256:xxx.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

If we run docker build . again, it should work now.

Time to celebrate: we have everything ready for local development. 🎉

Putting it all together​

Let's start by enabling the local Docker image registry in microK8s cluster:

microk8s enable registry

As a result, MicroK8s will run a registry and expose it for us at localhost:32000.

In the next step, we're going to create Dockerfile. For the sake of our hello world example, it can be as short as the one below:

FROM nginx

Now we can proceed with building the image and pushing it to the registry:

$ docker build . -f Dockerfile

...

Status: Downloaded newer image for nginx:latest
---> 605c77e624dd
Successfully built 605c77e624dd

$ docker tag 605c77e624dd localhost:32000/nginx-hello
$ docker push localhost:32000/nginx-hello

And as the final touch, let's deploy our image into MicroK8s cluster. Considering we've crafted a file called nginx.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx-hello-world
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx-hello-world
    spec:
      containers:
        - name: nginx
          image: localhost:32000/nginx-hello
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-hello-world
  labels:
    app: nginx-hello-world
spec:
  type: NodePort
  ports:
    - port: 8080
      targetPort: 80
      protocol: TCP
      name: http
  selector:
    app: nginx-hello-world

... we're ready to deploy it with:

microk8s kubectl apply -f nginx.yaml

... get NodePort of the service:

microk8s kubectl get svc

... and see nginx hello world, e.g. at http://192.168.64.2:31493:

Summary​

We've managed to build docker images and deploy them into the local Kubernetes cluster, all without the need for Docker Machine. What's more, the whole setup is achievable with just a handful set of commands and since we've utilized cloud init, it can be shared easily within the whole organization.

Further resources​

• Small Kubernetes for your local experiments: k0s, MicroK8s, kind, k3s, and Minikube
• Docker Context
• Docker architecture
• Install Docker Engine on Ubuntu
• How-To: Docker on Windows and Mac with Multipass
• MicroK8s: How to use the built-in registry

📝 TL;DR​

• Get the maximum from ESLint.
• Use getBy* queries together with @testing-library/jest-dom assertions.
• Try to not use getByTestId, aim for accessibility with getByRole.
• Don't initialize global context in each test. Define custom renderer.
• Use the matcher function to query text broken up by multiple elements.
• Keep your test report clean and disable console logs in jest config.

All the examples relate to jest and @testing-library libraries we've been using for a while. If you aren't, start with their adoption :).

1. Use appropriate assertions​

Having a descriptive error message at hand is always a good start for any debugging. Consider a test where we check for the presence of a particular element in a component:

describe("NavBar", () => {
  it("renders menu", () => {
    const onMenuOpen = jest.fn();
    render(<NavBar onMenuOpen={onMenuOpen}/>);

    const menu = screen.queryByLabelText("Open menu");

    expect(menu).not.toBeNull();
  });
});

...which gives us the output below. Not very helpful after six months of not seeing the related code:

● NavBar › renders menu

expect(received).not.toBeNull()

Received: null

  11 |     const menu = screen.queryByLabelText("Open menu");
  12 |
> 13 |     expect(menu).not.toBeNull();
     |                      ^

It get's better just by using getBy* type of queries:

describe("NavBar", () => {
  it("renders menu", () => {
    const onMenuOpen = jest.fn();
    render(<NavBar onMenuOpen={onMenuOpen}/>);

    const menu = screen.getByLabelText("Open menu");

    expect(menu).toBeInTheDocument();
  });
});

● NavBar › renders menu

TestingLibraryElementError: Unable to find a label with the text of: Open menu

<body>
  <div>
    <nav
      class="NavigationBar__StyledNavigationBar-sc-17lan40-1 pjXCn"
      data-test="NavBar"
    >

Conclusion:

• Always use getBy* to query for the element.
• Prefer assertions from @testing-library/jest-dom over default ones.
• queryBy* has only one purpose - to test non-existence in pair with .not.toBeInTheDocument() assertion.
• Make tests fail at least once before committing to check that error message is human-friendly.

2. Enforce strict ESLint rules​

Look at the following test. Can you spot the problem?

describe("NavBar", () => {
  it("can open 🍔 menu", () => {
    const onMenuOpen = jest.fn();
    render(<NavBar onMenuOpen={onMenuOpen} />);

    userEvent.click(screen.getByLabelText("Open menu"));

    expect(onMenuOpen).toHaveBeenCalled();
  });
});

It all seems just fine at first glance, correct? What if I told you it doesn't work in the application. You've been just assigned to a bug where customers complain they can't open the menu.

Let's check the code:

const MenuButton = ({ onClick }) => (
  <Wrapper onClick={onClick}>
    <Button onClick={onClick} icon={<Hamburger />} />
  </Wrapper>
);

Our test was indeed right. The callback got called. Except it likely happened twice, and the second execution canceled out the effect of the first one. Fortunately, there are more robust assertions to cover this, e.g. toHaveBeenCalledTimes.

What's more, we can enforce stricter variants with prefer-called-with ESLint rule.

Conclusion:

• Use eslint-plugin-jest and eslint-plugin-testing-library ESLint plugins to prevent common pitfalls from slipping under your radar.

3. Avoid test-ids whenever possible​

It's rather tricky in certain situations to select the element. In such cases, our developer instinct drives us to put there test id (e.g. <button data-test="menu" />). The chances are that the element is inaccessible not only to our unit test but to a portion of our audience as well.

If that's the case, consider UX improvement over crutch for a developer. The button in the picture has no visible text label, just an icon that cannot be easily interpreted by assistive technology. Both screen reader and unit test will become happier if we put there a label:

const Button = ({ onClick }) => (
  <button onClick={onClick} aria-label="Open menu">
    <HamburgerIcon />
  </button>
);

And then in the test:

render(<Button onClick={onClick} />);

userEvent.click(screen.getByLabelText("Open menu"));

Conclusion:

• Prefer getByRole query over getByTestId to spot accessibility issues.
• Check more on accessibility in the previous article.

4. Reuse global context with custom renderer​

Usually, we don't thrive for 100 % coverage but adequate coverage: Having tests for all components where some interesting logic is involved. And in such cases, global context often plays a role - checking whether a user is authenticated, the proper event is tracked, and similar. Many times we then end up with code like this:

it("shows remaining credits for authenticated user", () => {
  const auth = {
    onSignIn: jest.fn(),
    onLogout: jest.fn(),
    user: { name: "Joe", settings: { credits: 23, currency: "USD" } },
  };

  render(
    <AuthProvider value={auth}>
      <Billing />
    </AuthProvider>,
  );

  expect(screen.getByText("Amount: $23")).toBeInTheDocument();
});

Besides the repetitiveness in each test, it may also discourage anyone from writing more of them, knowing how much boilerplate is necessary for setup. Luckily we can leverage that render function from @testing-library takes optional wrapper, and define our custom renderer:

type ExtendedOptions = {
  auth?: Partial<AuthContext>;
};

type Props = ExtendedOptions & {
  children: ReactNode;
};

const TestProvider = ({ auth, children }: Props) => {
  const defaultAuth = useAuth();
  const authValue: AuthContext = {
    ...defaultAuth,
    user: defaultUser,
    ...(auth ?? {}),
  };

  return (
    <ThemeProvider value={defaultTheme}>
      <AuthProvider value={authValue}>{children}</AuthProvider>
    </ThemeProvider>
  );
};

type Options = Omit<RenderOptions, "wrapper"> & ExtendedOptions;

export const renderWithProviders = (
    component: ReactElement, 
    options: Options = {},
) => {
  render(component, {
    ...options,
    wrapper: ({ children }) => 
        <TestProvider auth={options.auth}>{children}</TestProvider>,
  });
};

Our updated test then can look as follow:

it("shows remaining credits for authenticated user", () => {
  renderWithProviders(<Billing />, {
    auth: { user: { name: "Joe", settings: { credits: 23, currency: "USD" } } },
  });

  expect(screen.getByText("Amount: $23")).toBeInTheDocument();
});

Notice how we don't need to initialize the whole context. We're passing only relevant options without sacrificing Typescript types.

Conclusion:

• Define a renderer, which encapsulates the common global context.
• Provide reasonable defaults, for example, default user when most of your components are behind authentication.

5. Match the whole text with elements​

It might become trickier with our Billing component from the previous example when the text contains HTML elements:

<div>
  Amount: <strong>$23</strong>
</div>

Standard querying by text stops working while assertion on single words isn't something we would appreciate:

// does not work
screen.getByText("Amount: $23");

// works but sucks
screen.getByText("Amount:");
screen.getByText("$23");

@testing-library knows its folks, so it even gives us a clear hint on what to do in error:

This could be because the text is broken up by multiple elements. 
In this case, you can provide a function for your text matcher 
to make your matcher more flexible.

So let's start with our reusable matcher function:

import { MatcherFunction } from "@testing-library/react";

export const matchRichText =
  (text: string): MatcherFunction =>
  (content, node) => {
    const hasText = (node?: Element | null) => node?.textContent === text;
    const nodeHasText = hasText(node);
    const childrenDontHaveText = 
        Array.from(node?.children ?? []).every((child) => !hasText(child));

    return nodeHasText && childrenDontHaveText;
  };

No more compromises in getByText query:

screen.getByText(matchRichText("Amount: $23"));

Conclusion:

• @testing-library simply rocks. And it pays off to read error messages carefully to their end.

6. Disable console logs​

Some time has passed, and while we keep our test suite green, a few console logs appear during each execution. Although they usually don't cause any bugs, they may warn us about deprecations, unmocked IO operations, missing contexts, and similar imperfections:

    console.error
      useTranslator must be used within a TranslatorContext

    > 22 |  const translator = useTranslator();

When their numbers grow, they can signal rising technical debt, but first and foremost, they spam our terminal output!

To counterattack this, we've extended our jest configuration to disable console logs altogether and make tests fail if there are any. Check out how it's done in React repository or Adeira Universe.

Whenever you encounter the console log then, you have to mock it explicitly or remove the cause:

it("handles error", async () => {
  const consoleSpy = jest.spyOn(console, 'warn').mockImplementation(() => {});
  render(<FaultyButton />);

  userEvent.click(screen.getByRole("button"));
  
  await screen.findByText("Something went wrong.");
  
  expect(consoleSpy).toHaveBeenCalledWith('Request failed, retrying...');
  consoleSpy.mockRestore();
});

Read more:​

• Common mistakes with React Testing Library
• Five Things You (Probably) Didn't Know About Testing Library

Fortunately, we realized emails today are nothing but an HTML page. And Cypress excels in that.

Ideation​

After quick brainstorming, we came up with the following plan:

1. We'll call REST API to send a specific email (e.g. boarding passes)
2. Email should be delivered to our inbox
3. We render its content as an HTML page to assert it contains certain elements

Preconditions​

To not end up with flaky tests, we must assure the system's high predictability, and reliance on an email message being transmitted over the network will always pose some risk to automation.

Additionally, there might be another burden of maintaining a solution for our test inbox: generation of unique email addresses for API calls that are not idempotent (i.e. actions that can be triggered only once to produce the same expected outcome), filtering out messages correctly when multiple tests are running or waiting for email delivery.

Fortunately, there are ready-made solutions for QA purposes. We selected Mailosaur as it has all the required functionality, automatically parses the content, and offers Cypress client library on top of it.

Picking up an email from inbox​

Let's take off first with requests to our API. In our scenario, we want to message customers with boarding passes. This is a matter of calling cy.request command:

const payload = { bookingId: 123456 /* ... */ };
const url = `${Cypress.env("MAILING_BASE_URL")}/api/emails/boarding`;

cy.request({
  method: "POST",
  url,
  body: payload,
  headers: {
    Authentication: Cypress.env("MAILING_API_AUTH"),
  },
});

Notice our mailing API expects bookingId, not email. Internally it checks that boarding passes indeed exist for a given id, and then it sends the desired email.

The booking id in our test relates to the email address we generated in Mailosaur. Picking it up from the inbox is then just a matter of one command:

cy.mailosaurGetMessage(
  serverId,
  {
    sentTo: email,
  },
  {
    receivedAfter: timestamp,
    timeout: 1000 * 60 * 2,
  }
).then(message => {/* ... */});

Wait a sec, how did we get mailosaurGetMessage? This comes from the official cypress-mailosaur plugin. It wraps Mailosaur APIs into convenient commands. For example, we use fixed email here, but for registration, we could use another such command, mailosaurGenerateEmailAddress, which would provide us with a new unique address for every test run.

And because most of our tests will follow this pattern of calling the API and then accessing Mailosaur inbox, we shouldn't miss an opportunity to define it as a single custom command, here's the full source code:

const sendEmail = ({ email, payload, apiEndpoint }) => {
  const now = new Date();
  const serverId = Cypress.env("MAILOSAUR_SERVER");
  const baseApiUrl = Cypress.env("MAILING_BASE_URL");
  const url = `${baseApiUrl}${apiEndpoint}`;

  cy.request({
    method: "POST",
    url,
    body: payload,
    headers: {
      Authentication: Cypress.env("MAILING_API_AUTH"),
    },
    // Cypress dumps the whole request including auth headers on error by default
    // So in order to avoid credential leaking we handle errors on our own
    failOnStatusCode: false,
  }).then((response) => {
    if (!response.isOkStatusCode) {
      const { status } = response;

      throw new Error(
        `Unexpected Mailing API error, url: ${url}, status: ${status}`,
      );
    }
  });

  return cy.mailosaurGetMessage(
    serverId,
    {
      sentTo: email,
    },
    {
      receivedAfter: now,
      timeout: 1000 * 60 * 2, // wait for email to arrive for 2 mins
    },
  );
};

Cypress.Commands.add("sendEmail", sendEmail);

Nice! At this point, we know how we're going to send and receive emails for little costs. Mailosaur gives us HTML as a string, and now we need to figure out how to convert it into an actual page that Cypress can visit.

Visiting email as HTML page​

Alright, we have email content, and we want to be minimalists during this step too. At the same time, Cypress needs a URL that we can visit to run our assertions.

For that purpose, we decided to store the email content on the filesystem and run a local static server to serve it as an HTML web page.

It's impossible to access the filesystem from Cypress directly as tests are executed in the browser, so defining the task is necessary to write to the disk. In our plugin file:

const serveEmail = ({ name, emailContent }) => {
  const filename = `${name}_${uuidv4()}.html`;
  const filepath = path.resolve(__dirname, "../../downloads", filename);

  fs.writeFileSync(filepath, emailContent, { encoding: "utf-8" });

  return filename;
};

module.exports = (on, config) => {
  on("task", {
    serveEmail,
  });
};

Each time we call cy.task("serveEmail", { name: "boardingDocuments", emailContent }) from our tests, email is saved into the download folder, and its unique generated path is yielded.

Now we need a server to visit the file as an HTML page. The most straightforward way we found is to use http-server package. We execute the following command before cypress tests:

yarn http-server downloads --port 8080 &> /dev/null &

With this in place, let's load the email content as if it was just a regular web page:

cy
  .task("serveEmail", { name: "boardingDocuments", emailContent })
  .then(path => cy.visit(path))

Voilá! 🎉

Mind the fact that CYPRESS_BASE_URL should be set to localhost:8080 to successfully load the web page.

Putting it all together​

One note, though: we will probably repeat these steps in all other tests: first calling cy.sendEmail() command to send and receive emails, followed by cy.task("serveEmail") to store and visit them. So why not combine it all into one master command?

const getEmailContent = ({ email, name, payload, apiEndpoint }) => {
  return cy.sendEmail({ email, payload, apiEndpoint }).then((message) => {
    cy.mailosaurDeleteMessage(message.id);

    return cy
      .task("serveEmail", {
          name,
          email: message.html?.body || "",
      })
      .then((visitPath: string) => {
          return { message, visitPath };
      });
  });
};

With these optimizations, all we need to do in our code to retrieve the email we've just sent is then limited to a single command call:

describe("Boarding documents", () => {
  it("email with boarding travel documents has correct content", () => {
    const recipient = "some-generated-email@mailosaur.io";
    const apiEndpoint = "/api/emails/boarding";
    const name = "boardingDocuments";
    const payload = { bookingId: 123456 /* ... */ };

    cy.getEmailContent({ email: recipient, apiEndpoint, payload, name }).then(
      ({ visitPath, message }) => {

        // We can still assert on message object from Mailosaur
        expect(message.subject).to.equal(
          `Booking ${payload.bookingId}: Here are your travel documents`,
        );

        // or visit the email and assert on the content
        cy.visit(visitPath);
        cy.findByRole("link", { name: /Download/ }).should("be.visible");
        cy
          .findByText("We've attached your travel documents.")
          .should("be.visible");
      },
    );
  });
});

Final remarks​

I've shown you how to assert that email content meets certain criteria. We could also verify the correctness of links leading back to the application. Still, as Cypress is currently limited to visiting single origin per test and we already went to localhost, we can't do it right here.

For such purposes, we have another set of tests, but that's already a different story.

Special thanks to Furbo.

Drawings made with the help of Freepik icons from www.flaticon.com.

I like being involved in architectural decisions, trying to crack various programming nuts around, and have multiple objections on how things should be delivered, tested, or tracked. So it was natural for me to enlarge and strengthen my plate.

And then people happened.

Beginnings didn't show any adverse symptoms. We formed the team, delivered the first release, folks happy. However, later on, someone started mentioning how he isn't pleased about the product vision and that we have tasks defined just for next week at most. Another got into trouble with his coworker from a different team, and other similar human psyche mysteries unfold. And what became even more surprising - it was me who was supposed to act!

Indeed, there were many aspects of being a tech lead I hadn't anticipated. Eventually, it led me to step down and pursue a carrier as an individual contributor. Nonetheless, I learned hell a lot from this experience, and in the next lines, I would like to share my "discoveries".

Mountain summit is narrow and far from others.​

A peculiar sense of loneliness crept first into the surface. As a developer, you never work alone but in a team with other developers, but once I got into the new role, there was only a single instance of tech lead - me. If you're considering a managerial position, be prepared for this. As you climb up, the path begins to disappear.

Sure, most colleagues around you will help you, but you must reach out farther than before to meet people at the same level. More often, even beyond your company. Building a stronger connection to people in the same role, although they might be sitting at a different location or even country, is something I undervalued initially, but I shouldn't.

Being direct, like for real, grants you safer landings.​

Early in the morning of August 6, 1997, a Korean Air flight no. 801 crashed into a mountainside on Guam. Why did it happen? Pilots decided on the visual approach to the runway, despite the weather not being optimal. It was raining above the island. During the landing procedure, the captain mistakenly believed there're closer to the airport.

What's more, the first officer and flight engineer realized something is not right. But they didn't speak up clearly to the captain. Instead, they inclined to indirect suggestions and hints: "Don't you think it rains more?", "Not in sight?" or "Captain, the weather radar helped us a lot."

Both men were aware of imminent danger, yet the plane still crashed! They didn't want to question the captain's authority. Korean culture is traditionally hierarchical. Not by any coincidence, Korean Air recorded much more accidents than airlines from western countries by that time before they brought David Greenberg from the US to reform their operations.

Why am I saying this story here? (Which you can read about in the book Outliers btw.)

I made a similar mistake in some one-to-one meetings, for example, when I needed to motivate my colleague to change his attitude to one particular problem. In the end, a message never landed as it was too ambiguous, while I got upset that there's no visible improvement. I was trying to be too friendly as I liked having him in the team otherwise and didn't like the idea of him getting angry and leaving for a different team or even company.

The fact is that this was purely a construct of my mind, which was counterproductive for all involved. Providing honest feedback is usually the best we can do. What I did was ruinous empathy. If in doubt, I highly recommend the book Radical Candor.

Sharing is caring.​

As a developer, I always wanted to be involved in all hard to crack issues. And rarely said NO when such opportunity had come. Fortunately, the team and scrum master usually made sure I'm not busy working on multiple tasks simultaneously. Nonetheless, with stepping into the tech lead role, this barrier had fallen (we didn't have any scrum master or agile coach to stop me), and I had to learn the hard way context switching is not an option.

Before I fully acknowledged that truth, I put the whole team's work into danger. After initial successes, our velocity significantly dropped as I kept myself preoccupied with several challenging tasks that took forever to deliver for real. Meanwhile, colleagues weren't moving much with their business domain knowledge: I ate most exciting stuff for myself.

Some tech debt also quickly accumulated because I was too busy to do in-depth reviews of other application parts.

Lesson learned: share the workload and delegate, focus on finishing the work, not starting. And knowing when to say NO is an important skill.

By the way, that hardly gained experience helped me eventually in another aspect. I'm no longer behaving like a babysitter at the workshops towards juniors. Instead of solving ostensibly unrelated issues for them, I let them try by themselves first - or at least more often.

Don't shoot too fast.​

Another point relates to the one above. Don't say out loud hasty conclusions too quickly. When it comes to gross estimation of future plans, product owners and stakeholders can shift their priorities if it seems it will take too long, even before grooming sessions. It's always good to remember team capacity and individual skills to avoid the trap of overconfidence.

Here are also some practical lessons I learned when I was part of some decision making. Because sometimes, things go awry even though not a single experienced participant saw any risks. If the new proposal looks too promising, try the following:

• Imagine the project or new approach fails, why it would happen?
• If there is an alternative, compare the proposal with that instead of old.
• If it still looks so great, ask yourself: Do I consider all sides, or I'm putting all weight on facts confirming my belief in the idea?

Retrospect in big.​

I designed the article mainly as a retrospective of my tech lead intermezzo. Plenty of shortcomings I committed I saw before, and yet I did them too.

But I believe there were some good parts too. One thing that worked particularly well was organizing a Squad Health Check. As we're usually busy doing all the stuff, we sometimes forget to stop, take a breath and step aside for a while to check the overall picture. The health check proved to be the best mechanism to capture it.

Everything is well described in the Spotify blog post, so here are some additions:

• Turn disadvantages to advantages. For example, we were a remote team, but we scheduled it when we met at the same place and joined it with informal follow up at the restaurant, discovering local cuisine. And discussions got richer. Similarly, if it's still corona time, you can ship everyone, e.g., a bottle of wine, and enjoy some slack time on how you spend time under lock.
• Don't throw it into a drawer. It's great to talk things out, but it's much better when actual issues got addressed. Dedicate one retrospective or additional time between two health check sessions to briefly reiterate how you are standing.

Read & listen more:​

• Squad Health Check model – visualizing what to improve
• GOTO 2020 • Talking With Tech Leads • Patrick Kua
• GOTO 2017 • How to Take Great Engineers & Make Them Great Technical Leaders • Courtney Hemphill
• 5 Practical Software Development Metrics Every Agile Leader Needs
• You're About to Make a Terrible Mistake
• Make It Stick

How much time do we spend on making the web more accessible? And how much on supporting outdated browsers like IE? The chances are that more of our visitors would likely benefit from improvements in the accessibility area, but they are invisible to our statistics and error reporting.

Nonetheless, by today's post on this matter, I want to show you that it's not, in fact, that complicated to improve the audience's experience.

We're going to look at forms specifically, using the awesome Orbit component library in code examples, but all rules mentioned here apply in general.

Who is accessibility for?​

Before we start with an example, defining what we understand under the term accessibility is essential. The truth is that many developers and product owners put an equation between accessibility and blindness difficulties, treating this topic as a "nice to have" feature which can wait.

But accessibility is not just how the screen reader interprets the web content.

We can split various impairments into categories as follows:

• visual - e.g. full, partial, or color blindness, photosensitive seizures, astigmatism

  Even in the strict interpretation, we can't treat accessibility only as a way of making web page more screen reader friendly. Such a tool will probably be used only by people with the most severe eyesight problems, while chosen colors and font matters to others.

• motoric-coordination difficulties

  Some visitors, especially at higher age or with lasting injury, can get stuck on elements that move too fast, shift their position, or don't work with a keyboard and require precise mouse navigation.

  You should keep in mind many aspects: foremost, you should observe Cumulative Layout Shift (CLS) metric from web vitals on your web as buttons that jump from their position just before tapping them irritates everybody. Also, enabling the selection of items from autocomplete and dropdown using keyboard only is hardly a mistake.

  And last but not least, a significant improvement happens even with something as small as putting an authorization code at the beginning of SMS instead of its end. All users appreciate it if they have such information available right on the lock screen instead of doing extra steps with a mobile phone because the text in the notification was truncated.

• cognitive limitations

  Line-height, font size, or paragraph length are all things that determine how well people, for example, with dyslexia, may understand the content. Communicate information through illustrations, not only by text, to ease out understanding.

As you can see now, accessibility spans multiple areas and can be perceived more like a UX extension. And with this being said, we shall acknowledge that it's not some nitpicking but a problem that actually affects a significant portion of our audience.

The good thing is that it's possible to improve the overall experience a lot with just a few fundamentals in mind. And it includes developers themselves, as you will see at the end.

Importance of form element​

Let's now look at the form where users set their new password, or they're able to skip such a step entirely when authenticating for the first time.

A simplified implementation that uses Orbit components could look more or less as follows:

const SetPassword = ({ onSubmit, onSkip }: Props) => {
  return (
    <Modal size="small">
      <ModalHeader />
      <ModalSection>
        <Stack>
          <InputField type="password" label="Create new password" />
          <InputField type="password" label="Confirm password" />
          <Button type="primary" fullWidth onClick={onSubmit}>
            Continue
          </Button>
          <Button type="primary" fullWidth onClick={onSkip}>
            Skip
          </Button>
        </Stack>
      </ModalSection>
    </Modal>
  );
};

For sure, it's going to work, and we might put the task into the "done" column, yet we made a big mistake by omitting the form element.

A long time ago, all inputs had to be wrapped in <form /> to function correctly, but with the rise of single-page applications, actions are now usually handled by JavaScript, and it seems to be redundant and often forgotten. Form submit is then performed by an onClick callback attached to a specific button.

But as long as we leave out the <form /> tag, text input can't be submitted by the enter, and we also didn't specify what's the primary button if there are more.

Let's refactor the code now:

const SetPassword = ({ onSubmit, onSkip }: Props) => {
+  const handleSubmit = useCallback((e) => {
+      e.preventDefault();
+      onSubmit();
+    },
+    [onSubmit],
+  );


  return (
  
  <ModalSection>
+   <form onSubmit={handleSubmit}>  

-            <Button type="primary" fullWidth onClick={onSubmit}>
+            <Button type="primary" fullWidth submit>


+   </form>

With such a simple change, we've made our form submittable by the enter, and assistive technologies will better understand its context.

Progressive disclosure​

We've improved the keyboard experience, but let's revisit the visual side - our form has two primary buttons. That means they compete with each other over the user's attention. And as various psychological studies proved it, every person has the limited brain capacity to perform many little decisions during the day. In other words, we shouldn't make people think about our interface. Otherwise, they might get overwhelmed and eventually bounce out of the web.

There are several solutions: we could choose only one of the buttons as primary (if it has lesser importance) or make both of them subtle:

We got it by switching button type from "primary" to "primarySubtle":

<Button submit type="primarySubtle" fullWidth>Continue</Button>

Alternatively, we could show only information relevant at the time:

At first, the subtle "Skip" button is shown, but right after we start filling the form, we replace it with the submit "Continue" button.

This approach is called progressive disclosure. To illustrate it in a more representative example, consider a shopping cart where users have to select shipment and payment methods. Instead of overwhelming them with the matrix of all possible options, you can, for example, display and preselect a single variant and offer other methods from an expandable list that is hidden by default.

Better mobile phone experience​

Let's now suppose that we ask for a security code of credit card in our application, a 3-digit verification number. No other characters are involved. Yet what type of keyboard we usually display to the user on mobile devices?

We could potentially switch input type to "number", but this is still too suboptimal. Although keyboards containing numbers are shown, most of the valuable space on IOS is still occupied by special characters:

On top of that, we got native decrement and increment buttons inside the input, which do not make sense there.

There is a better way. If we set inputmode="decimal" on the input element, we get exactly what we expect:

And as a cherry on the top, we can reward those who responsibly rely on password managers. By adding autocomplete="cc-csc" parameter, we help autocomplete tools of password managers recognize the true meaning of the field, so they make the right suggestions.

Check out the awesome talk by Alex Holachek and her demo app to learn more about autocomplete and input mode options.

Buttons with no label​

Nice, we've made a significant improvement! But before we wrap up the topic of accessibility, let's check buttons. If they contain text label, we're good to go, but what if we have a search bar that looks like this:

The meaning is clear for most of us, not so for people with visual impairment since the button label consists solely of an SVG icon. Users relying on a screen reader will hear just a "button".

Fortunately, all that we have to do is to use "aria-label" properly in such cases:

const SearchBar = () => (
  <Stack direction="row" spacing="XXSmall">
    <InputField />
    <Button iconLeft={<Search ariaLabel="Search" />} />
  </Stack>
);

As you can see, it's not that a hassle to improve the experience, even for screen readers.

Bonus: testing​

There is an additional hidden secret known only to developers who responsibly test their code: accessibility affects unit and E2E tests. Just like some visitors can't see your content on the page, we can't effectively identify elements to select them if any "anchor" is missing.

If you are using @testing-library, think about how you would select a button with a search icon only from the previous example if there are many buttons like that. With aria-label, the button becomes accessible even in your tests:

const { getByLabelText } = render(<SearchBar />);

expect(getByLabelText("Search")).toBeInTheDocument();

Further resources​

• A11y Coffee ☕️
• Building & Maintaining Accessible Experiences at Scale
• Accessible Icon Buttons
• Beyond Responsive Design: Building Mobile-Optimized Websites in React
• Accessibility as a First Class Citizen
• Accessibility Developer Guide

In the previous article "Cypress: Setting up the first acceptance tests in GitLab CI pipeline", I showed you how to scaffold Cypress tests with TypeScript support, run tests in typical GitLab pipeline and avoid some common pitfalls.

Such initial setup might be already all you need to add more tests to cover relevant user scenarios. Unless your whole application is hidden behind OAuth2 flow or single sign-on, in our case managed by Google Identity and Access Management (IAM).

Authentication as barrier​

To get through such an authentication process, it's best to make the application think we're coming to the web already as signed-in user. Cypress has known limitations to visit only one superdomain per test and any redirection to third-party service presents an impassable barrier.

On top of that, our single sign-on used for internal apps requires the usage of two-step verification via mobile application.

We will take the following steps to go around this process:

1. Retrieve token through REST API.
2. Send token in HTTP headers when visiting the page.
3. Forward token also to any other API call made by the application.

Adding custom command​

Since all our tests are going to share the need for bypassing the authentication, we will implement it as a custom command which we can then execute instead of cy.visit:

Cypress.Commands.add(
  "login",
  (path?: string, visitOptions?: Cypress.VisitOptions) => {
    // some magical implementation here
  },
);

And because we use TypeScript, we add the type definition for the new "login" command too:

declare global {
  namespace Cypress {
    interface Chainable<Subject = any> {
      login(
        path?: string, 
        options?: Cypress.VisitOptions,
      ): Chainable<AUTWindow>;
    }
  }
}

You can place it into the same file, src/support/commands.ts, just above the command or choose your own way, just make sure these type definitions are properly loaded.

API calls from Cypress​

Now the implementation itself. First, we call the API with cy.request to retrieve the token. Then we can utilize cy.visit as we know with the token in HTTP headers:

Cypress.Commands.add(
  "login",
  (path?: string, visitOptions?: Cypress.VisitOptions) => {
    const options = {
      method: "POST",
      url: "https://www.googleapis.com/oauth2/v4/token",
      body: {
        client_id: Cypress.env("CLIENT_ID"),
        client_secret: Cypress.env("CLIENT_SECRET"),
        refresh_token: Cypress.env("REFRESH_TOKEN"),
        grant_type: "refresh_token",
        audience: Cypress.env("IAP_AUDIENCE"),
      },
      // Restrict cypress from showing errored response by default.
      // It would dump the whole request object, including env values.
      failOnStatusCode: false,
    };

    return cy.request(options).then((response) => {
      if (response.status !== 200) {
        throw new Error(
          `Request to get auth token failed, response: ${JSON.stringify(
            response.body,
          )}`,
        );
      }

      const { id_token: token } = response.body;
    
      return cy.visit(path || "/", {
        headers: { Authorization: `Bearer ${token}` },
        ...visitOptions,
      });
    });
  },
);

One thing you may notice in the code is failOnStatusCode: false. Cypress cy.request command fails by default if API returns 4xx or 5xx HTTP error code. Unfortunately, it dumps the whole request and response objects into console, including environment variables in the payload.

For that reason, we're handling errors by ourselves.

Now we can try to use our fresh command in tests:

describe("Dashboard", () => {
  it("contains latest bookings", () => {
    cy.visit("/dashboard");
    cy.findByTestId("Overview").should("be.visible");

    // ...
  });
})

Our morale drops because test run very likely fails. It turns out that the application sends a request to another backend to fetch the data for the dashboard, but this service doesn't have our auth token!

Network stubbing​

If we worked with just normal OAuth 2 flow, we could redeem ourselves by simply storing token in cookies, but we haven't found a reliable way to make Google IAM happy this way.

For that reason, we decided to use cy.intercept function to stub network requests instead. We put the following code just before cy.visit statement:

const baseUrl = Cypress.config().baseUrl || "";

cy.intercept("**", (req) => {
  if (req.url.includes(baseUrl)) {
    req.headers["Authorization"] = `Bearer ${token}`;
  }
});

Voilà! Now every request made by the application automatically gets our token too. We've won our struggle against relentless authentication.

Note: This section on network stubbing was rewritten to reflect the changes which came out with the Cypress version 6.0.0 and a new cy.intercept() function.

cy.server({
  onAnyRequest: (_route, proxy) => {
    const baseUrl = Cypress.config().baseUrl;

    if (proxy.url.includes(baseUrl)) {
      proxy.xhr.setRequestHeader("Authorization", `Bearer ${token}`);
    }
  },
});

<a href="/bookings/detail/1234567">1234567</a>

{
  "experimentalNetworkStubbing": true
}

const baseUrl = Cypress.config().baseUrl || "";

cy.route2("**", (req) => {
  if (req.url.includes(baseUrl)) {
    req.headers["Authorization"] = `Bearer ${token}`;
  }
});

Conclusion​

Network stubbing was historically one of Cypress weaker points as it lacked first-class support for intercepting any application request. Although the API is still experimental, and we still experience issues with requests made in service workers, for example, it"s great to see the big progress which comes with cy.intercept function.

Full implementation​

const login = (
  path?: string,
  visitOptions?: Cypress.VisitOptions,
): Cypress.Chainable<Cypress.AUTWindow> => {
  const options = {
    method: "POST",
    url: "https://www.googleapis.com/oauth2/v4/token",
    body: {
      client_id: Cypress.env("CLIENT_ID"),
      client_secret: Cypress.env("CLIENT_SECRET"),
      refresh_token: Cypress.env("REFRESH_TOKEN"),
      grant_type: "refresh_token",
      audience: Cypress.env("IAP_AUDIENCE"),
    },
    // Restrict cypress from showing errored response by default.
    // It would dump the whole request object, including env values.
    failOnStatusCode: false,
  };

  return cy.request(options).then((response) => {
    if (response.status !== 200) {
      throw new Error(
        `Request to get auth token failed, response: ${JSON.stringify(
          response.body,
        )}`,
      );
    }

    const { id_token: token } = response.body;

    const baseUrl = Cypress.config().baseUrl || "";
    // ignore requests to assets specified by their extension
    //
    // will be ignored:
    // https://example.com/file.svg 
    // https://example.com/file.css?v=a12c
    const extensionsToIgnore = ["jsx?", "svg", "s?css", "jpe?g", "png"];
    const patternGroup = extensionsToIgnore
      .map((ext) => `.*\\.${ext}(\\?.*)?$`)
      .join("|");
    const matchURL = new RegExp(`^(.(?!${patternGroup}))*$`);

    cy.intercept(
      {
        url: matchURL,
      },
      (req) => {
        if (req.url.includes(baseUrl)) {
          req.headers["Authorization"] = `Bearer ${token}`;
        }
      },
    );

    return cy.visit(path || "/", {
      headers: { Authorization: `Bearer ${token}` },
      ...visitOptions,
    });
  });
};

declare global {
  namespace Cypress {
    interface Chainable<Subject = any> {
      login: typeof login;
    }
  }
}

Cypress.Commands.add("login", login);

We at Kiwi.com use Cypress.io for some time, and we already had to rethink how we write such tests to keep them efficient and stable. One topic that plays a role in it sometimes remains forgotten: configuration. In this post, we would like to share with you some tips on how we now set up Cypress tests with Typescript support for a typical GitLab CI pipeline.

Before we start…​

You can see all topics discussed here in the following repository: https://gitlab.com/jaroslav-kubicek/cypress-example

Back to Basics​

Beginnings are fairly easy and well described in the documentation. First, we install cypress from npm and run yarn cypress open to get the autogenerated skeleton. After adding the typescript, renaming files to .ts and reshuffling, we get a structure like this:

Notice that we use monorepo setup with yarn workspaces — our tests are placed there along with our frontend. To get that, we had to change some paths in cypress.json:

{
  "fixturesFolder": "src/fixtures",
  "integrationFolder": "src/tests",
  "pluginsFile": "src/plugins",
  "supportFile": "src/support",
  "screenshotsFolder": "assets/screenshots",
  "videosFolder": "assets/videos"
}

Every time we run a test, all screenshots and videos are stored in the assets folder. Trust me, you don’t want to commit it accidentally, so we will add the following lines to .gitignore with possible env file:

assets
cypress.env.json

As we want to follow the best practices and write clean code, we will add ESLint too. Since it’s already there for the whole repository, we need to just extend it: yarn workspace cypress-tests add eslint-plugin-cypress --dev to install eslint-plugin-cypress in our cypress workspace and then we create .eslintrc.js:

module.exports = {
  extends: ["plugin:cypress/recommended"],
  plugins: ["eslint-plugin-cypress"],
  rules: {
    // does not make sense in Cypress world
    "promise/prefer-await-to-then": OFF,
  },
};

Your exact configuration may vary, but the philosophy is the same — you should have one root ESLint configuration and override only what’s necessary for the given workspace. For example, we force devs to use async/await in unit tests but not here in cypress tests, so we turned off the promise/prefer-await-to-then rule as cypress commands are not regular promises.

Let’s write the first test​

Now it’s about the time to finally add some tests. For the beginning, we will check just that the Kiwi.com homepage is loaded, the navbar on top is shown and a user can open the hamburger menu:

describe("Landing page", () => {
  it("shows navigation menu", () => {
    cy.visit("/");
    cy.get("[data-test='NavBar']").should("be.visible");
    cy.get("[data-test='NavBar-SideNav-Open']").click();
    cy.get("[data-test='NavBar-SideNav']").should("be.visible");
  });
});

And the resulting code is…ewww, not great, not terrible.

• We follow best practices by using data-test attributes for selectors, but it’s a lot of writing, and we have to use quotes inside the string.
• After a couple of weeks or months, it might not be clear what exactly is happening in the test.

We can do a lot better! In the beginning, we might be tempted to add a custom command to select elements more easily, something like cy.getByDataTest("NavBar"). It turns out there is even a better solution. We use Testing Library for unit tests and there is a variant for Cypress too. Let’s use it:

1. yarn workspace cypress-tests add @testing-library/cypress to install the lib.
2. In tsconfig.json, we add support for types: "types": ["cypress", "@types/testing-library__cypress"].
3. We upgrade src/support/index.ts file to make use of it:

import "@testing-library/cypress/add-commands";
import { configure } from "@testing-library/cypress";

import "./commands";

configure({ testIdAttribute: "data-test" });

And this is how we rewrite our test:

describe("Landing page", () => {
  it("shows navigation menu", () => {
    cy.visit("/");
    cy.findByTestId("NavBar").should("be.visible");

    cy.log("🍔 menu can be opened");
    cy.findByTestId("NavBar-SideNav-Open").click();
    cy.findByTestId("NavBar-SideNav").should("be.visible");
  });
});

Small changes make wonders. Selecting elements on the page is no longer cumbersome. We also increased readability by adding cy.log.

cy.log may seem unnecessary but we learned this a hard way. The complexity of both application and user scenarios tends to increase over time, real specs files are usually longer and when the test suddenly starts failing after months, it might be challenging to understand at first what the hell we do at that exact step, even with a screenshot or video recording. So make your future self happier by documenting each logical user step… or consider the BDD approach with cucumber.

Going live in GitLab CI​

Alright, we’ve implemented the first test and it passes locally, now it’s important to incorporate it to our continuous integration to make sure our application works before we merge our changes.

As we use GitLab CI, we’re gonna extend our .gitlab-ci.yml file:

variables:
  CYPRESS_CACHE_FOLDER: $CI_PROJECT_DIR/.cache/Cypress

cache:
  key:
    files:
      - yarn.lock
    prefix: $CI_PROJECT_PATH_SLUG
  paths:
    - .cache
    - node_modules
    - packages/*/node_modules
  policy: pull

cypress: 
  stage: verify
  environment:
    name: review/$CI_COMMIT_REF_SLUG
  image: cypress/browsers:node12.18.0-chrome83-ff77
  script: 
    - yarn --frozen-lockfile
    - export CYPRESS_BASE_URL="https://$CI_ENVIRONMENT_SLUG.$KUBE_BASE_DOMAIN"
    - yarn workspace cypress-tests cypress run -b chrome --headless
  artifacts:
    when: on_failure
    expire_in: 3 days
    paths:
      - packages/cypress/assets

Again, your exact configuration will vary depending on your use case and used tech stack, but here are some important points:

1. Cypress needs binaries for its execution, so if we cache node_modules dependencies in GitLab CI cache, we define where such binaries should be stored with a CYPRESS_CACHE_FOLDER variable. Then we have to include this path in the cache.
2. Notice we use yarn.lock as a key. This enables us to reuse cache even between different branches if dependencies haven’t changed.
3. We run our tests in headless chrome and store screenshots and video recordings if it fails.
4. To run tests successfully, we had to tell Cypress against which URL it should be run. Such URL is usually created dynamically depending on the branch name. We set it through CYPRESS_BASE_URL variable. Notice we had to specify the environment name in the job definition too: review/$CI_COMMIT_REF_SLUG in our case. This is necessary to have access to the environment variables related to the specific environment and should usually correspond 1:1 to the job definition of your deployment.

…annnnd it fails​

This article was written before version 6.0.0 came out which now passes disable-dev-shm-usage launch switch to chrome browser by default. Following workaround is therefore no longer necessary as it's done automatically for you.

If you are still facing such issues, check out the amount of available resources in your CI or try xvfb mentioned below.

Suppose you pushed commits to the repository, a merge request went through review and was merged. Everything was up & green. Then you went to the office the next day and your colleague asked you to help with the failed job.

The output shows a weird error:

Or almost nothing while the job execution hung up:

The usual common denominator of these issues occurring only in GitLab CI is virtual shared memory. The job runner always runs some Docker image and the default setting (64 MB) is too small for a modern browser.

You have several options: either ask DevOps to change GitLab configuration to run Docker images with bigger shared memory (with --shm-size an argument to docker run command where it’s questionable what’s enough), or add shm to volumes (/dev/shm:/dev/shm), or change IPC mode (--ipc=host which might be refused due to security concerns), or you can just disable it for chromium browsers.

To do so, use Cypress Browser Launch API to alter command-line arguments:

module.exports = (
  on: Cypress.PluginEvents, 
  config: Cypress.PluginConfigOptions,
) => {
  on("before:browser:launch", (browser = {}, launchOptions) => {
    if (browser.family === "chromium") {
      launchOptions.args.push("--disable-dev-shm-usage");
    }

    return launchOptions;
  })
};

Chrome will fall back to using temp files instead.

Growing up​

What started with a single test grew into a full suite. Now one of the hottest concerns in your team is probably the speed of tests, prolonging the execution of the whole pipeline for every merge request.

Although it’s not the only way how to tackle it, it’s definitely one of the easiest — parallelization using the paid Cypress Dashboard. It requires two lines of code change in .gitlab-ci.yml:

# other lines remain the same
cypress:
  script:
    - yarn workspace cypress-tests cypress run -b chrome --headless --record
  parallel: 3

With parallel: 3 GitLab CI will schedule three parallel jobs that will execute the same script. That would normally lead to executing tests three times, but thanks to --record flag, Cypress will send all spec files to Dashboard service first and Dashboard redistributes tests back to GitLab runners based on previous runs.

Don’t forget that you have to set CYPRESS_RECORD_KEY environment variable in GitLab to be able to use Dashboard with --record flag:

Given you have twelve spec files, each worker will likely get four files back to execute. But it might be just two: for example, if one of your tests is time-consuming and gets several minutes to execute, it’s usually scheduled the first and the runner receiving such spec might get only one more. The exact behavior depends on our context and current magic hidden behind Dashboard.

Parallelization through Cypress Dashboard this way has one crucial drawback: if any of your tests fail and you trigger a rerun, all tests will get executed again.

Luckily, Cypress.io team has just released version 5.x with test retries. So little momentary hiccups do not jeopardize your execution: if some of your tests are a little flaky and fail, they’re retried again before reported as failed.

If you are still on the older version and can’t update, cypress-plugin-retries plugin does the same as test retries in 5.x.

Neverlethess, as we learned the hard way, it might not be that easy even with retries — especially if you didn’t follow the best practices before. In case we would implement our first test as follow:

describe("Landing page", () => {
  before(() => {     
    cy.visit("/");
  });

  it("shows navigation menu", () => {
    cy.findByTestId("NavBar").should("be.visible");

    cy.log("🍔 menu can be opened");
    cy.findByTestId("NavBar-SideNav-Open").click();
    cy.findByTestId("NavBar-SideNav").should("be.visible");
  });
});