At Smartrr, Temporal helped us with synchronization problems between Shopify and our systems that had the usual failure modes of distributed systems. When I joined Convergint, I brought that experience with me and started seeing "Temporal-shaped problems" again. My team has since started using Temporal for our synchronization jobs while helping our colleagues adopt it as a core primitive of our new engineering platform.

I recently attended Temporal Replay 2026 in San Francisco to learn how far the ecosystem had come and what it looks like to support Temporal well once more than one or two teams care about it. The conference was larger than I expected, the talks were hard to choose between, and the hallway conversations made the Temporal community feel much bigger than it feels when only interacting online.

I came home convinced that I should push harder for Temporal adoption at Convergint, and clearer about what that work should look like. We're still early, so I want us to get better at the parts that make Temporal trustworthy in practice, support the teams already using it, and defer building more until the need arises.

Temporal-shaped problems

Engineers often come to our platform team looking for help matching a problem to the right technology, preferably something we'll help them operate. Sometimes they need a database, a public-facing web presence, a way to broadcast events to other teams, or help with their CI/CD pipeline. Sometimes they have a problem where retries, partial progress, rollbacks, and complex collaboration are already part of the work, but nobody has called it "orchestration" yet.

That's the class of problem I mean by "Temporal-shaped." Replay helped me broaden my sense of that shape. I heard engineers talk about using Temporal for AI agent loops, self-service infrastructure changes, data pipelines, payments, and managing long-running execution sandboxes. The details varied, but in each case, the same themes kept showing up: managing state, gracefully handling failure, and work that doesn't finish in a single request.

Shabnam Emdadi of Shopify shared a clear picture of this in her talk From Normalizing Complexity to Recognizing the Price. She highlighted how teams often delay orchestration because they think they don't need it yet, then end up building pieces of it anyway. A retry here, a status table there, a support runbook off to the side, and eventually the team owns a small workflow engine they never intended to build.

That's the habit I want us to get better at spotting. When teams are solving reliability problems with point solutions, we should be able to recognize the shape early and help them decide whether Temporal belongs in the design.

Temporal employees pointed me toward the official samples repositories for TypeScript, Go, and Java, along with the Temporal Code Exchange and Validated Patterns, as good places to mine for common patterns. I want us to spend more time with those examples, including asking AI to explain the patterns back to us, so we're better prepared to recognize the same shapes in our own work and the work of our colleagues.

Ensuring Temporal's "reliable as gravity" promise

Temporal's big promise is that it drastically simplifies running reliable, scalable, and resilient applications. This works when developers use Temporal's framework well, but learning to do that takes time. As platform engineers, we can help developers get there more quickly by educating ourselves in the core Temporal primitives that drive this reliability.

At first, this means understanding how to decompose work into activities and then stitch them back together with workflows. Temporal's error classes let activities signal when an action should or shouldn't be retried. Timeout and retry policies in both the client and server configuration give developers control over how hanging actions are detected, how soon to retry, and how many times to retry before giving up.

As developers live with Temporal for a while, they'll run into additional concerns that are just as important and less familiar to our team today. When the code inside a workflow needs to change, workflow versioning, worker versioning, or both can preserve deterministic execution as the new code path is deployed. At scale, the newly announced task queue priority and fairness configurations can help allocate resources across a multi-tenant fleet of workflows. Finally, as we increase the number of teams actively deploying Temporal workflows, we'll likely want to investigate the Temporal Worker Controller to understand its capabilities and decide if we'd benefit.

Just-in-time platform building

I'm a big proponent of incremental value creation. Call it Lean, Agile, whatever you want, but building just enough of a thing while keeping an eye on the bigger picture gives us an opportunity for early feedback and unblocks simpler use cases while retaining the extensibility needed to fulfill the grander vision.

Listening to Rob Zienert's talk The Path to Temporal General Availability at Netflix helped me see how that would play out long term as we build out Temporal on our platform. So many of his early sentiments matched my own experience: patient support work, careful scope control, and being real with developers about what's currently supported and what isn't. In his talk, he described the slow journey from scratching his own itch to solve reliability problems in Spinnaker, to offering Temporal to his colleagues on a best-effort basis alongside his normal Spinnaker work, to finally having enough enterprise integration and mindshare at Netflix to warrant building a small team around the offering and admitting it to the Netflix "paved path".

The talk wasn't super technical. It was encouraging to hear about his journey and how platform work can be slow, methodical, and empathetic.

This reinforces my "just-in-time platform building" conviction. At Convergint, we're early in our Temporal journey. Temporal is functional and helping teams, and there are some immediate places we can help teams understand the "Temporal 101" reliability primitives and try out new onboarding features like Standalone Activities.

Being early in our journey also means some things can wait, such as support for exposing Temporal Workflows across namespaces via Nexus. As demand for additional Temporal use cases grows, we'll be there to meet it, but if not, we won't have overbuilt.

Moving forward with Temporal at Convergint

The key takeaway after leaving Temporal Replay this year is that "Everything is Fine"™, and we should keep going: learn more about Temporal, teach it to my colleagues, and advocate harder for its adoption. It's a genuinely powerful tool that's underused, and we're at the right stage to make it easier for more teams to reach for it without needing to do a deep platform build-out.

Here's a recording of the talk, thanks to the DevOps Days Birmingham organizers!

Introduction

Today, I'm going to share with you some of what I know about DevOps, Staff Engineering, and Platform Engineering. In particular, I want to explore whether these concepts have a place in a Startup.

I argue that they do. In the end, I hope you'll agree. 🙂

My name's Chad McElligott, and I'm a Senior Staff Engineer at the New York City-based startup Smartrr, where we provide subscriptions and loyalty functionality as a service to Shopify brands. I work remotely from Huntsville, Alabama, and I've been with Smartrr for a year now. We have 18 people across our product and engineering group, which is more than half of the company as a whole.

Smartrr is a "Series A Startup." This means we've proven to investors we've found product-market fit to receive a round of funding, and we're now working to grow our customer base, broaden our reach, and stabilize our core offering. I joined Smartrr a few months after we closed our Series A. I have a general Staff engineering role, working on our cloud infrastructure but also helping to lead the product teams alongside our CTO and our Director of Product. This role is one of "leadership without authority" - I help steer the direction of the engineering group through sound reasoning, leading by example, and earning the trust of my fellow engineers. I consider these concepts of DevOps, Staff Engineering, and Platform Engineering as tools to use to lead and build that trust.

Explaining My Core Engineering Tenets

But what do these concepts of DevOps, Staff Engineering, and Platform Engineering mean?

DevOps

Maybe, being at a DevOpsDays conference, you feel you have a pretty good understanding of what DevOps means. But even among us, it's unlikely we'll universally agree on a definition. To some, it's a job title. To others, it's a way of working. Perhaps you're a software engineer who's experienced "DevOps as No Ops," resulting in you having to handle all Ops-related concerns yourself. Maybe you enjoy that, or maybe you feel it's moving us backwards.

To me, DevOps is a mentality of collaboration, automation of toil, and an embrace of modern tooling to sustain a fast flow of value to customers. It's not just leveraging the cloud, building infrastructure using code, or setting up CI/CD pipelines. And it's not just breaking down barriers to communication and collaboration. DevOps is a collection of these things, a recognition of how far we've come as an industry and the lessons we've learned, and taking advantage of that to achieve better outcomes for our customers and for ourselves.

Platform Engineering

Platform Engineering is a newer concept, made popular by the book Team Topologies by Manuel Pais and Matthew Skelton. I gave a talk about my experience as a Platform Engineer at last year's DevOps Days Birmingham, which is available on YouTube if you haven't had a chance to hear it yet.

I think of Platform Engineering as a technique for reducing the cognitive load on developers. It aims to increase product development velocity AND system stability by putting in place foundational components that support the developer's activities. Listening to podcasts like the Engineering Enablement podcast, you can hear many stories about how large organizations are building platform engineering teams to reap efficiencies and improve the developer experience.

Staff Engineering

Lastly, Staff Engineering isn't a mindset or a technique, but a role a software engineer fills within an organization, sometimes also called "Staff plus".

A progression past the "career" senior software engineer role, Staff Engineering may be best described as "servant leadership for software engineering". Senior Software Engineers are responsible for the outcomes of their work and the work of their direct team. The Staff+ role can grow that responsibility to the entire organization, but it may lose some of the individual contributorship a Senior Engineer may have grown accustomed to previously. Staff+ Engineers are often partnered with managers, directors, or VPs to bring IC-style "boots on the ground" perspective, energy, and action to their agendas. There is a lot of overlap between the "Architect" role of past generations and Staff+ engineering, and Will Larson even calls out "Architect" as one of the four "Staff archetypes" – the others being "Right Hand", "Tech Lead", and "Fixer" – in his book Staff Engineer.

Startup Culture

So what's different about working at a venture-backed startup, anyway?

Well, to grow the company, the leaders hire up, often taking on more expenditures than the company's revenue, leaning on the capital gained from investors to provide a so-called "runway" to facilitate rapid growth. As a result, the company must move fast toward growth and profitability before that runway runs out. This dynamic leads to two cultural qualities that a startup exhibits:

1. There's no time to waste. While waste is never okay, wasted time is particularly detrimental for a Startup. The development team's activities have to be hyper-focused on the organization's strategic goals while there's still enough runway to execute them. Everyone on the team needs to consider whether their activities are aligned, and, if they think they're off course, to check in and reorient. Experiments often will fail, but this isn't a waste as long as they're structured for learning and that desired lesson is learned.

2. We all wear many hats. When an organization is small, there's a lot of work to be done and seemingly nowhere near enough people to do it. As a result, someone who identifies as a front-end developer may also be performing the activities of a product designer, technical writer, product manager, quality assurance, third-party integrator, and more. As new ideas are brought to the table of how to further improve the customer experience, those who brought these ideas may end up also being responsible for prototyping or bringing the ideas to life.

These cultural qualities inform what is needed from a product development group and, in particular, a software engineering leader within that group. They can also give us hints about how DevOps, Platform Engineering, and Staff Engineering may need to be adapted to this environment.

Adapting My Engineering Tenets to Startup Culture

So, let's talk about that! How do our 3 engineering concepts change when considered through the lens of a startup culture?

DevOps at a Startup

For DevOps, it's important to keep in mind that processes are easy to change at a startup because there are so few people who need to alter their behavior. When looking to solve some problem, particularly when a tool is involved, it's best to alter your process to the tool for best results. Keeping a rigid process would require you to customize tools and possibly even build additional pieces of software, all of which are a drain on the time you could be spending creating value elsewhere.

Speaking of tools, you should try to avoid as much custom tooling as possible. This can take many forms, but may include using serverless infrastructure, leveraging SaaS tools, or choosing an open-source library. Try to go with the grain of technology where deviating doesn't provide a competitive advantage for your company, as it'll save you a lot of time and energy. Martin Fowler has a great article on his blog about this, titled Utility vs Strategic Dichotomy, if you'd like to learn more.

Lastly, when you are choosing "utility" technology, you'll want to make sure it is boring. Don't make big bets on solutions that are unfamiliar to the team or that have no community around them. If you encounter a problem, you want to be able to find a solution to it online, not be forced to code one up from scratch. Dan McKinley has a full talk dedicated to explaining this idea at boringtechnology.club.

Platform Engineering at a Startup

So how does Platform Engineering change at a Startup?

On Rebecca Murphey's new Engineering Unblocked podcast, something she said that I love is that "A company's developer experience is a product, whether anyone designed it or not." This is true even at Startup scale. You still need to monitor, deploy, track errors, read logs, and flip feature flags. The question is - Is it a pain to do these things?

When thinking about Platform Engineering at an early-stage startup, it'll certainly be a part-time hat you wear. While at one point, you may need an integrated test environment separate from production, later you may find your Platform-related concerns lower down the list of priorities. You'll find yourself putting on the Platform Engineering hat only as the need arises.

You're also not likely to have time for long-term platform-related projects. Instead, think of how to implement some value-adding component of the broader solution you have in mind. You can think big picture, and share that grander vision with your team, but you need to work in small batches, and help people understand how these smaller pieces connect to that broader vision.

Lastly, Platform Engineers at larger companies spend considerable effort ensuring smooth transitions from legacy software to modern tools and techniques. By contrast, doing Platform work at a startup mostly involves putting components in place that did not exist at all before. You'll be blazing a trail, unlocking new productivity and approaches that were previously not possible. This is a great feeling, but it can also be tricky to know - of all the possible things I could be doing, what should I do? When deciding whether to take on a platform-related project, take care to ensure you are solving the problems your organization currently has or those you see on the immediate horizon. Through boots-on-the-ground work, talking with your fellow engineers, paying attention to retrospective feedback, and periodic assessment of your Software Development Lifecycle metrics, you'll surface work that deserves attention. Sometimes though, it'll be best to leave the platform work for another day, and focus on needs elsewhere. Try to be flexible.

Staff Engineering at a Startup

Lastly, the Staff Engineer role is already well suited for work within a startup. The inherent deep technical expertise, the interest in having a broad impact, and the willingness to go where needed to solve business problems all resonate strongly in this environment.

There's a common saying in large organizations that certain senior staff "know where the bodies are buried", suggesting that they understand where technical debt lies and the context around it. It's often the role of a Staff engineer to go and find those people and write down their knowledge so decisions can be made on how to proceed. At a startup, by contrast, you can often look around after joining and easily see all the various things that need attention. You might even say there's a "trail of unburied bodies" at a startup! Startups provide the opportunity for a Staff engineer to have a tremendous impact through execution alone by reigning in this chaos and putting in place solutions that will serve the organization for years to come.

Staff engineers won't be able to stay in any one "archetype" at a startup. It is likely, due to the small size of the group, that you will be expected to be hands-on in addition to your other typical Staff+ activities. You'll be called upon for architecture work, leading projects, performing tactical tasks, and will be expected to come up with your own ideas of how to improve things and implement them without being told. Fluidity and ownership are key traits of a Staff+ engineer at a Startup.

Lastly, one perhaps less mentioned aspect of Staff+ engineering is their work doing mentorship and sponsorship to help other engineers. This can be especially important at a startup with junior talent, as they need the support, guidance, and guardrails that a staff engineer can provide.

Applying Modern Staff Engineering at a Startup

So now that we've looked at what DevOps, Platform Engineering, and Staff Engineering look like at a startup in the abstract, I'd like to share two stories with you about projects I worked on during my first year at Smartrr and how I applied these modern engineering practices we've discussed thus far.

My first story is from the beginning of my time at Smartrr. It's about the improvements we made to our QA process and how we no longer impose a merge freeze while regression testing our releases.

Second, I'll share how we've evolved our processes as we strive to continuously improve how we organize our work.

So, let's dive in!

Story 1 of 2 - Improving DevEx by Removing a Merge Freeze

When I joined Smartrr, one of my first tasks was to help finish a project to provision a new integrated testing environment in Google Cloud.

At that time, the process for performing releases was to stop merging patches into the main branch for 2-3 days while the QA team walked through their manual regression testing process using Qase, a popular QA test management tool. Any bugs found would be fixed immediately, and their patches would be merged into the main branch prior to a release being cut and deployed to production. During the merge freeze, the developers would continue building new patches for the main branch, but leave the merge requests open until the release was deployed. Then, developers would land their patches, and development would continue normally until it was time for a new release. The team was operating using a 2-week sprint, and the regression testing process would start roughly 3 days before the sprint ended.

While this process was working, it had its drawbacks. The fully manual regression testing process was slow, and often had unpredictable results because of a lack of unit tests in fundamental areas of the codebase. The development team understood the importance of quality, but was understandably frustrated by the merge freeze, which often resulted in difficult-to-resolve merge conflicts.

Context for my First Task

For context, the organization had recently renewed its focus on quality and stability when I joined, leading them to reevaluate every aspect of the Software Development Lifecycle. This resulted in a few new initiatives. Among these, they wanted to convert their manual regression testing suite into an automated suite that could be run more often and more efficiently. The QA engineers building this wanted a stable integrated test environment to run it against so that the application wouldn't change while the suite was running. So, this is where I come in, to help build this new environment.

The application to be deployed to this environment was a monolithic Node.js web application, along with a collection of cron jobs, some Google Cloud Run instances, a PostgreSQL database, a Redis instance, some Google Cloud PubSub queues, and DNS. The application and its cron jobs were deployed to Google Kubernetes Engine. The source code was hosted in Gitlab. The GitLab CI config file was considered "arcane knowledge," and most were afraid to touch it for fear of breaking something.

Progress had already been made in building out the new environment. There was a new GCP project and a new cluster, and much was working - even Terraform was already being used to build out the infrastructure. I took this work and applied what I felt was best given the team, the circumstances, and the timeline.

Pulling the Infra Code into our Monorepo

First, I felt it was important that the infrastructure code be colocated with the applications, so the developers would treat it as a first-class citizen. I moved the Terraform project into our monorepo and tied it into our automatic linting and dependency management tooling.

Terraforming All Environments

While choosing to build the new environment with Terraform was great, I felt it didn't go far enough because the other environments also needed this treatment. I went ahead and captured the existing staging and production environments in Terraform as well, then built the new environment's infrastructure by applying a different set of variables to those definitions. This ensured parity between the environments and helped us "heal" the places where they had diverged.

Simplifying our CI/CD

When Smartrr's CI was first setup, it used GitLab's Auto DevOps templates to get started quickly. Over time, heavy customization of this partially-obfuscated configuration made the CI process difficult to understand, so we decided to cut Auto DevOps out and explicitly define everything. We cloned the existing monorepo project to a new project in GitLab that had Auto DevOps disabled, then built out a new .gitlab-ci.yml file that retained the Gitlab specifics of the build, test, and deploy process, but split the majority of the commands off into a Makefile so they could be run manually. Once we had this working, we merged it back into the main monorepo and disabled Auto DevOps.

Using a Proper Secrets Store

Lastly, we decided to further decouple from Gitlab by moving our secrets. While rebuilding the deployment scripts, I moved all the application's secrets to Google Cloud Secrets Manager and automated the process of updating the Kubernetes ConfigMap from these values on each deployment.

Out of Scope

And there were, of course, some things we decided not to do.

Running the application in Kubernetes felt like overkill since we had a monolith. We decided to leave it this way, though, as it would have slowed our progress to migrate it elsewhere, and running it there wasn't actively causing any harm.

We also didn't set up any automated Terraform workflows since our infrastructure is relatively simple and does not change that often. We considered it a "nice to have" to automate it, but it's still a manual process, even today. I have it on my list to investigate Google Infrastructure Manager some time.

And while I did move our secrets into Google Cloud Secrets Manager, I decided not to setup a Kubernetes-native way to access these secrets. We weren't sure if we were going to continue to use Kubernetes moving forward, so I punted on this decision and setup a stop-gap. Time has shown that this decision was fine - and we've since learned we are sticking with Kubernetes, so I'll probably set up something like external-secrets operator the next time we go to deploy new workloads.

Sharing the Broader Vision

While we were still working on the new environment, I documented and shared our vision for eliminating manual regression testing entirely from our release process. Many were skeptical that we would ever achieve this, but folks were enthusiastic about moving QA to their own environment and removing the merge freeze, which was the first step to this larger plan anyway.

Mission Accomplished

So after much iteration and a careful rollout of the infrastructure from staging, to sandbox, and finally to production, we called the project a success. The environment was released about a month after I joined and the QA team began using it in earnest.

And once the new environment was available, we updated our release process, so the QA team could block deployments from happening in their new environment while performing regression testing there. This new process freed up developers to merge their changes to main without restriction.

This is the release process we still use today. I do maintain that we will be able to achieve the vision of continuous delivery, it will just some iteration to get us there.

Principles Applied in Story 1

So, back to our 3 engineering concepts that we're exploring - DevOps, Platform Engineering, and Staff Engineering. How were these helpful in this story?

Well, Staff Engineers are eager to seek out anyone who can help, no matter what team or role they have at the company. While working on this project, I spoke with practically everyone in the company to clarify, verify, and lead with intent. I played a "tech lead" role on this project, driving the project to completion, managing scope, and making tough decisions along the way. Finally, I leveraged the project to make our infrastructure and CI/CD process more accessible and easier to understand, both of which are universal goals of a Staff Engineer.

While this was initially what you may think of as a "DevOps" project, I turned it into a platform project by altering the scope to fully manage our infrastructure with code instead of just the new environment. By making that change, Terraform is now the clear choice for all infrastructure moving forward. We're also less likely to encounter differences between environments as our infrastructure evolves, giving us a solid foundation to build upon. I did have to walk away from some nice-to-haves, but I felt these were good trade-offs given the lateness of the project and pressing priorities elsewhere.

DevOps principles were core to this project, of course. We leveraged infrastructure as code to manage our cloud infra consistently. We simplified and improved our release process once the necessary tooling was in place to support it. And I introduced the Request For Comment document format to the org to facilitate inclusive decision-making as we defined our new process.

So, that was my first project at Smartrr. It felt really good to hit the ground running and have a win so early on. QA got the environment they needed to run their automation without disruptive mid-suite changes to the application. Developers no longer needed to artificially delay merging their patches to the main branch. And the whole organization sped up a bit due to better tooling and processes for collaboration. I'd love to iterate on this process more some day, such as by building ephemeral preview environments or fully automating our regression testing. If these topics interest you, come find and we can chat!

Next, I want to shift gears a bit and show you another side of Modern Staff Engineering: getting involved in shaping the processes used to organize our work.

Story 2 of 2 - Iterating Towards a Productive Engineering Process

Over the course of my year with the company, I have pushed our leadership team to continuously iterate on the processes and practices the engineering group uses to organize itself to ship software. I believe there is no one-size-fits-all approach to building software, so I felt that by experimenting with different approaches I had seen work well in the past, we could eventually collect enough signals to lead us to practices that result in productive collaboration.

When I first joined Smartrr, the product development group all worked from a single board and would meet every day and walk through it, delivering status updates to the rest of the group. There was a lot happening! But when looking out over the Google Meet video call, it was clear that a lot of folks were checking out when it wasn't their turn to speak. As new features were being built, the responsibility of seeing them fully shipped felt diffuse - no single engineer felt this was their responsibility, so it often fell to the product manager. There was also no clear understanding of the technical architecture that was being constructed during feature development. In summary, there were processes in place, but I felt we could do a lot better.

Agile all the things

I've worked places that felt engaging, empowering, and thoughtful each day. I wanted to bring this feeling to my new group, and given my new role and the Startup environment and culture, I felt empowered to do so.

Smartrr does not have any formal engineering management in place, so I collaborate with my amazing colleagues, CTO James Turnbull and Director of Product Bianca Tompkins, to fill the gap and provide support to the team. Together, we regularly discuss ways to improve in all of these areas and more, then I draft proposals for any changes we want to try out. Each time we implement a new idea, we frame the change as an "experiment," which I've found reduces the friction of change and helps folks understand they are not a one-way street. We always make the goal of the change clear, so everyone can decide for themselves if we're hitting the mark or not. In this way, we make any sort of process change a group endeavor, and in our retrospective meetings, we discuss whether a given change is having its desired effect or not.

I'd like to share with you some of the process experiments we have run, the goals of these changes, and the outcomes we saw.

Experiment: Short-lived Feature Teams

First, we ran an experiment to move from individuals working tickets across the board to instead running short-lived feature teams. The goal of this is experiment was to build a sense of direct responsibility for the full lifecycle of feature development within engineering. We chose a lead for a feature, who in turn would pick a cross-functional team – backend, frontend, and QA – to work with them to implement the feature to it's outlined acceptance criteria. The outcome of this experiment was mixed. We definitely saw an improvement in team members' sense of ownership over a given feature, but not everyone wanted to be a lead or felt well suited to be one.

After trying this for a few months, we decided to move to a new fixed-team model with static leads that we call "squads". These squads run their own planning and retrospectives, giving them more autonomy and ownership over their process.

Experiment: Epic Kickoff Documents

Second is our epic kickoff document experiment. The goal of this experiment was to ensure that requirements are clear, and the entire team is on the same page regarding the scope of a project and the approach that will be taken prior to starting work. To make this change, we provided a template for teams to populate during a meeting together where they cover all agenda items highlighted in the document and any additional context they feel is important to share. This document can be referred back to throughout the project to clarify the original plan.

The outcome of this change was very positive! The document only served as a prompt, a way for people to not feel lost when first beginning these meetings and opening up communication between each another. The team members now feel these meetings are a good value for the time they spend, and they operate better earlier on in the project because of them.

Experiment: Group Code Review

Lastly, I'm excited to share my favorite experiment we've run – our group code review. The goal of this experiment was to reduce code review turnaround time, encourage discussion of code, and share skills between the engineers. To achieve this, we decided to try meeting twice a week in an optional 1 hour Slack huddle where developers can join to either review code or have their code reviewed by others on the call. A driver shares their screen and walks through a pull request, and the attendees share their thoughts and make suggestions.

This has had a huge positive impact on our engineering group. By reviewing each other's code in person, they naturally treat each other humanely and respectfully on the call and in comments outside the call. Comments made in the call result in thoughtful discussions about techniques and trade-offs, which leads developers to learn a lot from one another. Pull Requests never sit long because, in each session, we work to reach Inbox 0. More code reviews are happening now, even outside the group review sessions!

This has also given the team exposure to the benefits of pairing or grouping up for coding sessions, which I'm hoping to push more for as time goes on.

Principles Applied in Story 2

Let's check in with our modern engineering concepts again, and see how they have influenced this work.

Adjusting process for teams is usually thought of as the role of an Engineering Manager or Director, but in their absence, a Staff engineer with an interest in these sorts of improvements can get involved and help. Once the culture of continuous improvement has caught on within the teams, they will feel empowered by their agile processes instead of burdened and can take on the challenge of further process improvements themselves. Sometimes teams get stuck in process ruts that are dragging them down, but they're not aware or are used to it. Having a trusted Staff engineer shake things up and show a better way can be just the medicine the team needs to realize their full potential.

While much of the buzz around Platform Engineering tends to revolve around tools, my hot take is that process is a part of your platform. Process plays a large part in the speed and effectiveness of your engineering organization, so it should be thought of as an enabler alongside your tooling. An ineffective process is an impediment to a team's developer experience, so replacing it can be just as impactful as improving build times or reducing toil during deployment. Process is also a lot easier to change than tooling!

As for DevOps, well, consider the acronym CALMS, which is often cited when DevOps is defined: It stands for Collaboration, Automation, Lean, Measurement, and Sharing. The Lean part refers to using Lean processes to eliminate waste in the value stream. Iterating on your processes through agile and teaching others how to do so is a core aspect of DevOps!

I strongly encourage you to take a hard look at your processes and decide if you have room for improvement. It's a low-effort and high-impact change you can make that will benefit everyone.

Conclusion

I'm really enjoying my time working in a startup environment. At this stage of the company, I feel I'm able to bring my full self to the problems we are solving and can work across disciplines to help bring solutions to life. The ever-present time constraints force pragmatism, an important skill to cultivate no matter what industry or environment you're working in. The embryonic nature of the company gives me the opportunity to make a huge impact and set the company up for success by applying a modern approach to software engineering.

Staff Engineers often require both breadth and depth of experience to be successful in their role. Working at a Startup is a great opportunity for a Staff+ engineer to further develop their breadth of knowledge and perhaps try out an area that they may not have had a chance to dig into before. Have you primarily been working as a backend engineer, but you're interested in developing some React or BigQuery skills? A startup is a great place for this.

Platform Engineering will look different at different scales of companies. At a startup, you'll be able to sit down 1:1 with the developers to understand their pain points, then take on small projects to move the needle in a better direction. Building fast feedback loops into the development process is a great way to improve devex and help the developers help themselves in the future. And you'll want to ensure all your basic Software Development Lifecycle concerns are covered with industry-standard tools and techniques. Platform Engineering won't be a full-time job at a startup, but a technique you apply when the need arises. Remember, "A company's developer experience is a product, whether anyone designed it or not." So take a bit of time to design it.

I hope you agree with me that DevOps is very relevant at a startup. Putting in place the culture, processes, and tools that accelerate value delivery to customers and make a company a humane place to work is very rewarding for everyone involved. So if you're a DevOps enthusiast and are considering working at a startup for the first time, I think you'll find you have a lot to bring to the table.

Thank you 🙏.

Many thanks to Chris Hodges, Kyle Kurz, James Turnbull, David Lee, Patrick Steadman, and Samuel Galarneau for reviewing and providing feedback on the early drafts of this talk.

In 2022, I gave my first-ever public conference talk. My confidence grew further from that experience, and this year my conference talk was selected as the day two keynote! I'll share more about my talk another day, as in this post I want to highlight what I learned from others, how I felt walking away from the conference, and my thinking about future conferences.

Format Changes from Previous Years

In previous years, the conference was two days of talks. This year, the organizers decided to make a couple of changes to the format, all of which I felt were great choices and enhanced the quality of the conference:

1. A day of workshops was added at the beginning of the conference, making the conference now three days instead of two.
2. "Open Spaces" topics were proposed on stage and voting happened as people were heading into the hall, so participation in this process was a lot better.

Day 1 - Workshops

Hands-on Hashicorp Vault

The first workshop of the conference was a set of hands-on activities setting up Hashicorp Vault for the first time, adding some secrets to it, setting up user management, and applying permissions to secrets. I enjoyed this workshop because I had been wanting to get some hands-on experience with Vault for a while, but hadn't had a chance until now. The workshop was administered as an Instruqt course, which provided a pre-configured web-based environment with a built-in editor and command line for running vault and various cli utilities as well as viewing the admin web interface. The Hashicorp developer advocate running the workshop, Michael Kosir, explained the various concepts well and answered all my myriad questions. I learned from this experience that Vault is really best suited for organizations that are running workloads in multiple cloud providers, as it provides a layer of abstraction between your cloud provider that gives you a single place to manage access control, rotation policies, and more. For me at a startup that is only working in Google Cloud, I think it is best for us to stick to Google Cloud Secrets Manager and, when providing access to Kubernetes workloads, some sort of operator for bridging between the Kubernetes world of workload identity (i.e. service accounts and namespaces) and the secret, such as External Secrets Operator.

Trufflehog

At lunch, I met Dwayne McDaniel, Developer Advocate at GitGuardian and a fellow speaker at the conference. During a conversation about secrets management, he shared a CLI utility called trufflehog that performs secrets scanning on a Git repository. My current company Smartrr uses gitleaks, but supposedly trufflehog is better so it sounds like it's worth a look!

PCM and the Perils of Multitasking

There were two workshops I really enjoyed in the afternoon: one by Aleksandra Lemańska about Process Communication Model (PCM) and an agile workshop by Steve Hallman.

In Aleksandra's workshop, she gave us a taste of part of the PCM, sharing how everyone has a "dominant" communication style and by learning the telltales, we can employ an appropriate method of communication to successfully be understood by them. It's an intriguing proposition and one I'd like to look into more.

Steve's workshop had us participate in a game called "How long does it take to write a name?", originally developed by Henrik Kniberg. It was an eye-opening first-hand experience illustrating how inefficient multitasking is. He then went on to explain that even better than single-piece flow is applying multiple people to a single piece of work, commonly known in software engineering as "Mob Programming". Steve recommended the work of Woody Zuill for further education on the topic. Here's a video from Goto Conference 2017 where Woody explains his approach, and he wrote a book on the topic as well!

Another great resource Steve shared with me is his Software Practices Timeline, detailing various practices developed within the software industry over the years to improve effectiveness. He described how organizations can trace their maturity through the timeline, finding which pieces they may still be able to employ to "unlock" new efficiencies and capabilities. I don't want to link directly to it here, as I think Steve is still developing it, but he's looking for folks to beta test it with him so reach out to him if this sounds useful to you!

Lastly, Steve spoke about Little's Law and how it models the behavior of queues, which is very relevant to a software engineer's daily work. Check it out to get some theory behind the value of low work-in-progress (WIP)!

Day 2 & Day 3 - Talks and Hallway Conversations

There were many talks at the conference, both on the schedule and in open spaces, which were ad-hoc sessions about topics the attendees wanted to discuss. I thought all of the talks were well done. I had a couple of favorites of the conference: "A successful platform is an opinionated platform" by Ben Goodman, Senior SRE at Rokt, and Steve's agile talk about improving your team's efficiency by pulling techniques from the "free toolbox" instead of adding headcount.

I also had some great hallway conversations.

One was with Will Blackburn and Peter Inge of DevClarity, a freshly minted startup focusing on improving the effectiveness of engineering management. They launched their product at DevOpsDays and were able to secure a pre-seed round of funding the same day, too! I really resonated with their mission and the problems they are looking to solve, as I feel them acutely in my work. I look forward to seeing how their product progresses!

Another conversation was with Johann Schleier-Smith about the work he is doing to provide automated ops for PostgreSQL with the startup he has founded called CrystalDB. I shared the pains Smartrr has been having with performance and how having some baked-in intelligence would help us a lot. He's got an extensive blog post detailing the opportunity they see for applying AI & ML to operating PostgreSQL and are developing both open source and subscription-based solutions for the PostgreSQL user community.

Lastly, I had a fun (but nerdy) conversation about architecture diagramming in an open spaces conversation. I shared some tools I had been using lately for sharing knowledge with others (eraser.io and gamma.app) and the pains I have had with excalidraw, and was recommended to give Miro another try for architecture diagrams. I've been using Miro for other things but not for architecture, so I'm looking forward to trying it out the next time I need a diagram.

Support Your Local DevOpsDays!

DevOpsDays is a not-for-profit organization that is run by volunteers and is solely focused on bringing technologists together. Supporting your local DevOpsDays helps build your local tech scene. This in turn encourages innovation that you can take part in as an employee or co-founder, creates a community you can source solid candidates from as colleagues or learn from, and provides you opportunities to grow yourself by teaching, speaking and mentoring others. I strongly recommend DevOpsDays and am thankful for the organizing committee of DevOpsDays Birmingham, their sponsors, and my employer Smartrr for sending me. 👏

These podcast episodes have been available for a while, but wanted to share them here as well for posterity! I hope you enjoy - hit me up on LinkedIn if you have thoughts you want to share, happy to chat.

Level Up Podcast - How product engineering made Chad McElligott a better platform leader

Day Two DevOps - Platforms Reduce Cognitive Overhead

Reviewing your own work means taking off your "author" hat and putting on your "reviewer" hat, inspecting it from a detached perspective and evaluating its merits, style, and other qualities as if you were reviewing the same from someone else. You want to do this prior to sharing your work with others so that they don't also have to muddle through all the mistakes you will find on your own in your self-review. You will also want to hold yourself to a higher standard than you would hold others to, to grow yourself and set a good example for others.

While this may sound obvious, I rarely meet someone that also follows this practice, so I'm going to walk through two examples of how this looks and what you may gain from the practice.

Review Your Own Merge Requests

When preparing a code merge request into your version control, create a full draft of the request and then fully review it yourself before requesting others to do the same. It's super easy to work on a branch for days or even weeks before opening a merge request, but a lot more difficult for someone to come along and provide a meaningful review of that work. If you review this work yourself prior, it will naturally encourage you to build merge requests that are great to review - smaller, well-scoped, well-explained, and relevant.

Performing a code review of your own work will naturally force you to ask yourself: "What are the important pieces of this request that should be reviewed?" Since it is your own code, you have a different sort of bias than you have when you review others' code. By deliberately disconnecting yourself from your own code when you self-review, you combat your bias to cut corners or do work that you would frown upon if done by others. The result here is that you not only deliver more rigorous work, but you also begin to acknowledge where your judgment of others' work may be nitpicking or otherwise not helpful and can cut it out!

There are tons of good practices you should be looking for when you review code, but when you are reviewing your own code especially, you may ask yourself questions like:

• Does this Merge Request description / commit message accurately describe why the change is being submitted and any additional context that may be helpful when reviewing? Consider the advice from Chris Beams.
• Is this work complete?
• Are there any important test cases missing?
• Are the functions and structures well-factored, well-named, and well-documented?
• Is this demonstrably working? Could the merge request description indicate how it was end-to-end tested?
• Are there any other commits, documents, or tickets that would be useful context when reviewing this? Are they in the description?
• Is this the right size? Does it capture a full piece of work, or if it's only a segment of work, can it stand on its own without breaking anything?
• Is this too big? (hint: if you can't review it in one sitting, it's too big)

This is only a short list as an example. You should decide on your own review criteria, but it should encompass at least what your organization considers the "best practices" for merge requests.

Review Your Own RFCs

Request For Comment (RFC) documents are used to gather feedback on a proposal that you have the autonomy to implement, but want to gather thoughts and ideas from others to round out your own. Before you share an RFC widely, put on your reviewer cap and evaluate it critically. You're about to ask many people to build a mental model of your proposal and provide helpful feedback, which is hard work! To help them out, you can make sure your document succinctly captures your idea and how it solves a specific, well-understood problem.

A great RFC will typically capture the need or problem that is being solved, the proposed solution, alternatives that were considered, and why the specific solution was the best among the alternatives. This will help your reviewer feel you have considered the problem from all known angles. Some questions you may want to ask yourself when self-reviewing are:

• Is the problem clearly stated? Am I assuming some amount of knowledge that could instead be explicitly stated?
• Are there any additional approaches that were considered or tried prior to this proposal that I omitted?
• Is my proposal considering all those that will be affected by this change? If not, how can I expand the proposal to elaborate on this wider impact?
• Is there any way to make this shorter? Am I being redundant anywhere that could be trimmed?
• Is there enough detail in my proposal to allow reviewers to evaluate it fully? If not, is there some way to make it explicit what kind of feedback I am looking for given the amount of detail?
• Am I being biased in my description of the alternatives? Are the true merits and drawbacks given or are they cherry-picked to support my proposal?
• Is this proposal truly better than "doing nothing"? Is this made plain in the document?

Again, these are merely some criteria to use as a jumping-off point for reviewing your own RFCs in case you aren't sure where to start. Definitely consider what you think makes an RFC "great" to you, and hold yourself to that high bar.

Conclusion

Reviewing your own work is an important part of respecting others' time and making sure you are getting the most out of the time they do dedicate to helping you. It also helps you to be more thoughtful about the value you are creating within your organization which results in better results for everyone. Before asking for someone else to review your work, put the effort in to review it yourself - you won't regret it.

Here's a recording of the talk, thanks to the DevOps Days Birmingham organizers!

Introduction

Hi! I'm Chad. I work at Sotheby's as a Staff Infrastructure Engineer on our Platform Engineering team, known internally as the ThunderCats.

Sotheby's is primarily an Auction house for luxury goods like art, jewelry, apparel, and cars. We have about 2,000 employees worldwide and did about 8 billion in sales in 2022. We have about 70 engineers within workstreams supporting our mobile app, auction experiences, financial services, and internal business tools. There are 4 of us in Platform Engineering.

I've been at Sotheby's for about a year and a half now, and I'd like to share with you what I've learned about how Platform Engineering works.

As a former Product Engineer, being a Platform Engineer at Sotheby's this past year has been a huge opportunity for me to learn and stretch my skills. The insights I have gained are forming the foundation upon which we're building our platform and our engineering organization as a whole. I'm excited to share with you what I've learned… about what defines the core of Platform Engineering and how I've learned to apply these principles in my day-to-day work.

Upon becoming a Platform Engineer, I asked my boss "What does Platform Engineering mean to you?" He succinctly responded "Velocity and Stability". This made sense to me. "Sure, I get it" I thought. But as I went about my work, I revisited this description over and over, and found there to be a lot of depth to these words.

So now a year later, as I think about what Platform Engineering means to me, this is what I think of.

Platform Engineering is the application of a Product Mindset to supporting your engineering organization's software delivery velocity and system stability.

Not as catchy, but I think bringing a Product Mindset into the definition is a critical aspect to doing our jobs well. So let's dive in and talk some about these three main qualities: velocity, stability, and a product mindset.

Velocity and Stability are the Core Outcomes of Platform Engineering

Velocity and Stability describe the core outcomes a Platform Engineering team is driving at.

Velocity represents how fast we can provide a solution to meet a customer's needs, from idea to delivery.

Many aspects of the Software Development Lifecycle can be changed to improve delivery velocity, such as project management or agile practices, the local development environment, technology choices, review processes, the deployment pipeline, and more. Many sub-disciplines of Platform Engineering exist that take improving delivery velocity as their primary goal, such as "Build Engineering", "DevOps", "Developer Experience", and "Developer Productivity" teams.

System stability, on the other hand, represents the work required to provide a consistent experience to your company's customers.

While mostly affected by operational practices, stability is also affected by cultural norms such as incident response processes and learning from incidents through post-mortem retrospectives. A common sub-discipline of Platform Engineering that specializes in Stability is "Infrastructure Engineering". Security practices commonly found as components of "DevSecOps" also support System Stability under the Platform Engineering banner, and SRE, while not necessarily a sub-discipline, is strongly aligned.

You may recognize the metrics referred to here as the DORA "4 key metrics".

The DORA State of DevOps Report has proven year after year that maintaining both velocity and stability is key to sustaining a high-performing software delivery organization. This is WHY it is so important to have a Platform Engineering discipline in your organization. Attention to maintaining these characteristics can slip as teams focus on delivering the company's core value. Having a Platform Engineering team supports our organization's ability to focus on delivering its value, while enabling a continuously improving delivery velocity and system stability for all teams.

But Platform Engineers have many challenges to overcome to be successful. They are doing primarily internal-facing work, have fewer engineers to work with, and often lack much of the management support other team archetypes receive such as product and project management. Platform Engineering is also typically an "engineering-led" discipline, and engineers are notorious for overbuilding, under-designing, "chasing the new and shiny tech", and falling into other pitfalls resulting from a lack of customer focus.

In order for engineers to lead Platform Engineering efforts successfully, we have to learn to wear many hats. But above all, we must be hyper-focused on our customers, pragmatic about what we can achieve, approach our improvements iteratively, and set our goals on outcomes, not outputs. In short, we must adopt a Product Mindset.

Adopting a Product Mindset is how Platform Engineering Works

Adopting a Product Mindset is how Platform Engineering works.

Founder engineers know this all too well - building a product as an engineer requires not only having a good technical idea and the skill to deliver it, but also Product-Market fit, which requires customer feedback, marketing and support. Founder engineers rarely have the luxury of being able to rely on others to do these things for them, and Platform Engineers often find themselves in this same boat.

Much of the buzz around Platform Engineering suggests we think of "Platform as a Product", as if we are building what amounts to an internal SaaS product like Heroku. While this may be what some organizations' platforms look like, the deeper meaning here is that we think about our Platform through the lens of a Product Manager. This means we need to slow down and think holistically about how to deliver value to our customers - the software engineers within the organization, but more broadly, all our colleagues and the customers our company is striving to serve. This results in better outcomes because it keeps the focus on solving the problems your organization is experiencing right now.

My team has been living with this charter of improving velocity and stability along with the understanding that we must adopt a product mindset to be successful, and I want to share how we have been applying these principles to our work. My hope is that our lessons learned can add some color to the theory and provide a peek behind the curtain of what Platform Engineering looks like day-to-day.

Lesson 1: Set Goals on Outcomes, Not Outputs

The most important lesson I've learned this past year is the importance of setting goals on outcomes, not outputs.

"Outcomes over output" is a common saying we've learned from the lean product management world that helps us set meaningful goals. We definitely need outputs to achieve our outcomes, but this is meant to remind us to avoid setting goals against activities; otherwise, we may end up in a sort of "productivity theater", where there is plenty of work getting done without meaningful results. We need to ensure that whatever outputs we are creating are achieving our desired outcomes, and if not, be willing to reassess and experiment with entirely different approaches.

At Sotheby's, our Platform Engineering team integrates this outcome-based goal setting into our yearly OKR process, our quarterly planning, and our story kickoff meetings.

Use OKRs for Strategic Planning

For our OKR process, we get together in person, typically in Iceland, where two of our team members are based. During this week, we spend time revisiting our team charter, reflecting on our last year's work, and building a vision for what outcomes we want to accomplish as a team and how we want to see the engineering organization progress.

Because our goals are outcome-oriented, it gives us the freedom to be creative in how we achieve them. Building them as a team draws ideas from everyone and aligns us around our chosen path forward. This engagement results in happier engineers which in turn drives better outcomes.

Perform Quarterly Tactical Planning

Our quarterly planning follows a similar process, but with more granularity. A quarter retrospective leads to collaborative brainstorming sessions, which leads to a final prioritization exercise to add projects to our roadmap. During our collaboration sessions, we examine feedback we've received throughout the quarter, discuss patterns we've been seeing, and dig into promising ideas. These sessions result in planned initiatives for the coming quarter.

Use Story Kickoffs for Alignment

Finally, our story kickoff is a meeting we have at the beginning of each major project. We clarify what outcome we're hoping to achieve by completing the work, what's in scope and out of scope, and who's participating in the project. Finally, we nominate a leader for the story that acts as a "delivery lead", keeping everyone on track and moving towards the goal. With this meeting at the beginning of each project, we make sure that everyone working on the project is aligned around the work and the outcome we are driving towards before digging in.

These 3 core iterative processes help us to self-organize around our goals. They keep our work focused on our desired outcomes while continuously inspecting and adapting throughout the year to ensure we're making progress and our goals are still relevant.

This outcome-based goal setting drives the importance of the next lesson I want to share with you, which is truly knowing your customer.

Lesson 2: Truly Know Your Customer

Even though we're engineers with engineers as our customers, this doesn't mean we can assume we know what problems matter most to them. As Platform Engineers, we're typically not doing the same class of work as other engineers, so it's very likely their challenges look different from ours. To stay connected to their real needs, we need to acknowledge this gap and work to bridge it.

At Sotheby's we're using three main strategies for keeping a finger on their pulse - providing engineers direct support, collaborating closely with them through short-term consulting engagements, and running a biannual survey.

Support Engineers on Slack

Platform Engineers are often some of the more senior engineers in the organization, especially at smaller places like ours. This means our help is often sought out, particularly in our areas of expertise like infrastructure, cloud, databases, and for tooling we support such as GitHub Actions, Terraform, and Kubernetes. To support these needs, we host a shared Slack channel. For each request, we guarantee a response within 1 business day but typically respond much sooner. This direct, low friction, and SLA-based approach to support gives our engineers the help they need in a timely manner to keep them from being blocked. Being a public channel, engineers can also help each other if they see someone ask a question they know the answer to.

This is an important way we learn about the sources of friction in their daily work, which feeds back into our prioritization and planning. But providing asynchronous support like this only gets us so far.

Consult, for a White-Glove Approach

Sometimes a white-glove approach is a better fit for supporting engineers in achieving our organizational goals.

For example, we recently "embedded" 4 engineers on 2 different teams for a month to support them in building out Service Level Objectives. Through active pair- and mob-programming sessions, we were able to explore implementation strategies together, which shared our mental model of SLOs. We also directly helped them to implement SLIs, datadog dashboards, and even new Platform services to support ingesting SLI telemetry from our frontend applications. This collaborative way of working strengthened relationships between our teams, building trust and mutual understanding.

It also gave the Platform team a closer look at life as a product engineer and respect for the challenges they are facing. We discovered they were having issues debugging their applications and jumping to implementations of framework code in their IDEs, which was hampering their day-to-day productivity. We also found testing out gRPC endpoints was frustrating to them. These discoveries have since fed back into our planned work, where we have enabled gRPC reflection on all services, and will be spending time writing IDE setup guides to clarify configuration of a productive build/debug/test flow.

With both low-touch and high-touch supporting activities, it gives us an opportunity to see first-hand what kinds of solutions are needed to improve the velocity of our engineering teams. But there is still a gap in our understanding of their needs - what about the people that are not actively asking for support, but still have feedback or unmet needs?

Survey for a Broad Perspective

This is where it can be really helpful to run an engineering-wide survey.

We ran our first engineering survey this spring to get an unbiased understanding of the state of various high-level aspects of our engineering culture. We decided to use Qualtrics for this as a lightweight way to get starting with surveying. Google or Microsoft Forms can also work. The hard part about running surveys is primarily knowing what questions to ask, but a combination of the SPACE framework, the DORA capabilities model, and a read through the books Accelerate by Nicole Forsgren et al. and Drive by Dan Pink can go a long way to informing what a high-performing engineering culture looks like. It can still be really helpful to get support from a UX researcher for this, even if only to ensure your approach to surveying is statistically sound. If you are looking for a turnkey solution to surveying and have some budget for it, I found through my research that DX @ getdx.com is a solid choice.

Prior to this survey, we felt we may have been over-prioritizing issues from a vocal minority, while other more pressing issues were silently hampering productivity. Our first set of survey results helped us to feel confident that our roadmap is aligned with the most pressing issues. We have since prioritized reducing the amount of toil around managing infrastructure, which was a top issue highlighted by the survey.

So through surveying, white-glove consulting, and collaborating with engineers day-to-day in Slack, we can piece together a relatively comprehensive picture of where we should be applying our effort. But problems don't always come paired with solutions. The problems that most need solving are often the ones without an obvious solution. That's why, in addition to understanding what problems we should solve, another important component of Platform Engineering is inventing on behalf of our engineering organization.

Lesson 3: Don't Just Respond, Invent

By inventing, I don't mean inventing something globally new, but something new to the organization. Because we can dedicate time, energy and resources to solving these cross-cutting concerns, we can bring great solutions that serve and delight our engineers. You could call this the fun part of Platform Engineering! Indeed, this is where you'll apply your creativity and technical chops to create real value for your organization. Some ways we can do this are by introducing known-good practices from the wider tech community, bringing new ideas to existing solutions for better outcomes, and experimenting with promising next-generation technology.

Introduce Known-Good Practices

When introducing known-good practices into your organization, it helps to have a broad understanding of the kinds of problems organizations face as well as a solid understanding of the solution space to pattern match against. Nothing is a substitute for experience with good practices, but some great way to bring more onto your radar is to read books, listen to podcasts, and explore the solutions offered by your cloud vendor of choice. I've written in depth about techniques for staying in touch with the tech community if you're interested in learning more.

At Sotheby's, we recognized a lack of knowledge sharing between senior engineers across teams, and a strong interest for it. We decided to introduce a discretionary "Request for Comments (RFC)" process. My team began sharing our big ideas as RFCs to both socialize them and solicit feedback. We also created an RFC template and some lightweight documentation that engineers could lean on if they wanted some help onboarding into the process. Finally, we created a Slack channel to socialize and discuss the proposals. We didn't invent the concept of RFCs nor did we even invent the specific RFC template we decided to use, but instead used one that had worked for me in the past at a previous organization. This in turn was drawn from the excellent work of Phil Calcado and his experience working at Meetup that he shared on his blog. Good ideas can, and should, spread.

Remember though that no matter what idea it is you are looking to introduce from outside your org as an innovation, it is important to tread lightly - be careful about introducing too much at once and make sure you are addressing problems that exist today - otherwise, your changes may be seen as cargo-culting and may not be taken seriously or adopted. In my case, I decided to introduce the RFC process as opt-in, to test the idea out and see if it caught on naturally. If instead I had tried to mandate it in some way, the initiative may have failed from the start or damaged my goodwill with engineers. Instead, it is slowly gaining traction as engineers begin see they are making better decisions with the help of this tool.

So as you're solving the problems of your engineers, look for gaps in process or tooling that you are aware of an existing, known good solution for, and introduce them! But the majority of your time will be spent iterating on existing solutions.

Iterate on Existing Solutions

Iterating is at the heart of a Product Mindset because it moves us away from the "one and done" project mindset of the past. As they say:

Software is never done, only abandoned

My team has iterated on a few of our offerings over the past year - some less exciting work such as keeping 3rd-party software up-to-date, but one project, in particular, stands out: our migration from Jenkins to GitHub Actions. In early 2022, we added Service Level Objectives to our CI/CD system. One of these was an objective to have fewer than 50 main branch build failures each month, which we found we were missing more than we liked. In response to this, we decided to migrate away from Jenkins and the Kubernetes plugin we were using for our builds, and start using GitHub Actions instead. This not only addressed our flaky builds but also removes the burden of managing Jenkins, provides a better UX for engineers for debugging their builds, and makes the build process easier to understand and customize. Our first iteration of this migration was to open up GitHub Actions use for what we call "Standalone" builds, which are builds outside our main Bazel build system. This smaller, "vertical slice" of GitHub Actions support gave us experience integrating with it and also let us test out how our engineers would respond to it. The feedback we received has been overwhelmingly positive, so this coming fall we plan to finish our migration by moving our Bazel builds over and turning down our Jenkins setup.

Of course, we won't always have prior experience to pull from, nor an existing solution to iterate on. Or maybe you feel like you're hitting a local maximum in your existing solution, and need to make a leap to an entirely new solution to the same problem. In this case we need to experiment.

Experiment With New Technology

It is healthy to consider any new change as an experiment. The riskier the change, the more experimentation eases tensions and encourages action. At Sotheby's, this year we plan to experiment with a new method of provisioning infrastructure that I have been calling "Manifest Infrastructure". This involves using Kubernetes Operators such as the open-source AWS Controllers for Kubernetes (ACK) or Crossplane to define the Infrastructure as Kubernetes manifests and have the infrastructure itself be managed by the operator. We're excited about this solution because it will provide the opportunity to build higher-level abstractions for engineers to interact with, leaving the plumbing an implementation detail. Whitney and Mauricio's keynote from KubeCon Detroit 2022 illustrates this well. We plan to try this on a subset of our applications, then deploy it more broadly if things go well.

Experimentation is a key tool in your toolbelt as a Platform Engineer, both for your own work and to empower your engineers. Consider how you can enable things like blue/green or canary rollouts, feature flagging, "two-way door" decisions, and incremental delivery. Your team and your engineers will thank you for the psychological safety, learning, delivery velocity, and system reliability that result.

So by applying these core principles we're making sure we're driving our work towards specific outcomes, we're informing our work with the voice of our engineers, and we're bringing new ideas to bear to solve for our organization's needs. The final key component to being an effective Platform Engineer is to make sure we continue to have bandwidth by scaling our impact.

Lesson 4: Scale Your Impact

The power of software engineering is the ability to "automate away" our problems. Traditional IT Operators were not trained in software engineering, but Platform Engineers are, so there is an expectation that this new discipline will scale sub-linearly with the size of the engineering organization. But we shouldn't be "building all the things" just because we can. For this model to work, Platform Engineers have to internalize that we should be biased towards "doing things that scale".

We can again apply our product mindset to this by being pragmatic about what we build vs. buy and guarding our time, so we can focus on what matters.

Make Good Build vs Buy Choices

While working to solve problems within our charter, we want to try to build as little software as possible, leaning on established products and open-source solutions when available and feasible. Building and maintaining software is labor-intensive, so we want to reserve this for only those problems where the benefit will far outstrip the cost and where no alternative exists. It can feel painful to accept a solution that's only ~80% of our vision of how we may want to solve a problem, but doing so can often be "good enough" for now and free us up to solve other problems. One particularly relevant example of this was our choice to adopt OpsLevel over a more custom tool such as Backstage. Our business goals for adopting such a tool were to build out a service catalog to provide a broader understanding of the systems in our organization and how they fit together. While not nearly as customizable as building a developer portal using the Backstage framework, adopting this tool solved for our primary business needs. And OpsLevel is constantly working to improve it and add features - work that our Platform Engineering team doesn't have to do. Our organization has taken this same approach elsewhere by adopting a mix of SaaS Products (such as Datadog, Algolia, and Auth0) and open-source tools (such as Linkerd, Jaeger, and Spinnaker) to provide robust solutions to common needs. We still build software, but it is typically for stitching services together for an integrated, polished feel, such as Slack bots, CI/CD customizations, and abstractions to create a great developer experience.

Knowing when it makes sense to build vs. buy is challenging to me, especially making the business case for buying a solution when it makes the most sense. The bigger the platform team or group, the more bandwidth there will be for custom development work, but this will always be a critical skill for a Platform Engineer to grow.

Avoid Interrupt Overload

Another important way we scale our impact is by protecting our time, so we can get planned work done. We prevent our shared support channel from distracting us too much by having only a subset of our team responding to questions each week. There will be days when we are "fire fighting", but we have to be mindful when this happens and prioritize addressing any recurring issues. We also constantly work to clarify and communicate our team charter so people know which conversations to bring us into - otherwise, our time gets drained by meetings and "side quests" that we contribute little to, leaving us little room to deliver on our broader initiatives.

Making wise build vs. buy decisions and protecting our time are only two of many possible ways you can ensure you scale your impact. If you find yourself down in the weeds on some super-gnarly problem, it might be totally okay! Your work may unlock huge value for the business and be totally worth the effort. But don't forget to come up for air sometimes to sanity check what's happening with your team, so you can decide how best to proceed, together.

Summary

I feel confident you will have a solid foundation upon which to build an effective Platform Engineering team if you follow these four principles:

1. Set goals on outcomes, not outputs
2. Truly know your customer
3. Invent on behalf of your customer
4. Scale your impact

Using these, Platform Engineering really works. We are able to support our engineers in delivering their software fast and stably, and do so at a sustainable rate with a happy, engaged platform engineering team.

After all this talk about the importance of a product mindset, I wouldn't fault you for thinking I'm a product manager in engineer's clothing 😅. But really, I just want to make sure what I'm working on is impactful. The results are worth the effort.

I hope my talk has helped to shape your perspective on how Platform Engineering works and, if you're a Platform Engineer, I hope it will help you more effectively support your own engineering organization.

The Consequences of Poor Communication

Being a software engineer in a business context is inherently a group endeavor. You will inevitably be required to collaborate to achieve meaningful goals at work, and having poor communication skills is like rust in the gears of that collaboration. This effect also compounds when multiple people that need to collaborate have poor communication skills. Things that would take days or even minutes with a high-performing, communicative team would take weeks or months - or worse, be stalled - for a team that has communication issues.

One of the cornerstones of modern agile software development is rapid iteration and feedback, but this is all but made impossible when individuals involved in the process are poor communicators. Frequently, high-performers will "route around" poor communicators to get things done, cutting them out of the process. If the poor communicator is a key contributor somehow, it holds the team back, stunting team velocity and dragging everyone down.

Besides these disastrous team outcomes, poor communication can also derail an individual's ability to even gain a position doing what they love to do. Without being able to effectively communicate your value to others, you make it difficult for the hiring manager to understand if you are a good fit for the need they are trying to fill, and as such, will likely pass over you for another candidate that makes their case more clearly.

All in all, you should think of your communication skills as one of the most important skills you can develop over your career.

The Hallmarks of Good Communication Skills

"So what do good communication skills look like?" you may be wondering. I don't claim to be the best communicator - and my spouse would attest to this - but I do see it as a core area that I work to improve upon every day. And while I may not know every trick in the book, there are a few main points I adhere to when collaborating with others.

💭 Bias towards over-communicating

In general, it's better to make your point multiple times than to leave any question. It can also be helpful to communicate your message in more than one way - in writing, speaking, using diagrams, or even examples. For communication, the DRY principle does not apply: repeat yourself frequently, saying things in different ways, as often as you need to convey your message. The more important the message, or the more fundamental, the more you should repeat it.

👂🏼 Listen more than you speak

A key component to being an effective communicator is understanding. To understand the people you are speaking to, the problems you are solving, or the context within you are working, listening to those around you and asking clarifying questions are your best tools. Sometimes the answers to the questions you want to ask have to be sought, so you'll have to invite others to the conversation. Even in 1:1 conversations, leaning towards listening more than speaking will make you a better communicator overall, as you will naturally allow the conversation to flow both ways.

🪟 Prefer transparency

Everyone makes better decisions when they understand the full context within which they are working. Being transparent means sharing anything and everything that may be meaningful or relevant to a given problem domain, where possible. It also means sharing your hunches, gut feelings, and intuition, as often these can be as telling as the facts. Share the history of a problem, the history of the people involved, previous attempts to solve this... anything that may be helpful. Speak your mind.

🙇 Be respectful

As a corollary to the advice to "speak your mind", always try to frame what you share in a way that is respectful and constructive. Look for ways to contribute to the overall goals of the group, not drag it down, demotivate, or otherwise be unhelpful or rude. Assume good intentions. Being respectful should always be at the top of your mind when communicating with others, especially within a business context.

🧠 Keep a beginner's mind

Even if you feel you are a good communicator now, always be looking for disconnects or situations in which you feel you are not getting your point across. Pay attention to the communication tactics employed by those you see as influential or effective, and see if you can incorporate some of them into your own communication style. Overall, stay humble and open.

Good Communication in Software Engineering

It's one thing to talk about good communication skills, but understanding how to apply those skills in your day-to-day work is another thing entirely. Software engineers have lots of opportunities to communicate smoothly - or fail to do so. Let's go through many of them and look at what good communication means in each context.

Code review

A company's code review process is one that every software engineer will be expected to participate in from day one. A less obvious aspect of the code review process is that its function is more social than for validation. By putting up code for review, the committer is sharing a change along with the context within which the change is being made, and is asking the reviewer to understand it and provide any commentary that may help the committer do their best work. This is a communicative process, both by the committer and the reviewer.

From the side of the code committer, they must do their best to allow the review to stand on its own. By providing clear context in the review request description, the committer shares everything the reviewer will need to know to understand why this change is being made and the considerations that went into designing the solution that is presented. They are also charged with writing code that is clear, concise, and directly addressing the requirements - avoiding changing unrelated code, unnecessary refactoring, or other distractions. A code committer is communicating well to their reviewers when the reviewers feel well equipped to play their role. When addressing any comments on the review, the committer should aim to decouple their feelings from their code and address them thinking about the bigger picture of the problem at hand.

A code reviewer is doing a good job communicating when they are directly contributing to the goals of the code committer or the goals of the team as a whole. This could be by providing useful critique on aspects of the design the team cares about, such as code quality, performance, separation of concerns, etc. It could also be providing valuable context the committer is missing, which would allow them to build a better solution. If the code does not warrant any comment, it can also be helpful to provide a supportive comment saying so! A reviewer should avoid being confrontational, combative, accusatory, or disrespectful.

Receiving help from others

Asking for help can be difficult when you see others around you excelling. It's important to remember that we all started somewhere, and you can rest assured that we all received help from others along the way. When receiving help from others, it's important that we do our best to make the process as easy as possible for the person helping us, so they have a good experience and are happy to help us in the future. If we make the process difficult by being uncommunicative, confusing, combative, or defensive, it creates an unpleasant memory for the person helping us, and they're less likely to want to assist us later.

To smooth the process of receiving help from others, learn to state as clearly as possible what you are trying to do and where you are stuck. Providing this needed context is critical, as you may be going down a bad path and getting stuck long after you should have course-corrected. A skilled mentor will know how to lead someone towards the right solution and where they took a bad path, but it is also on us to make their job easier.

Listening is critical to receiving help. It is frustrating to a mentor when they are asked for help and provide it, only for the same mistakes to be made again the next time around. We should do our best to internalize the lessons we learn when receiving help - perhaps by taking notes, recording them and listening later, or even just paying close attention and not letting ourselves get distracted while we're being helped.

Mentoring

Mentoring - the process of teaching and guiding others to learn skills you have experience in - is an important and expected activity of senior software engineers. In truth, anyone can mentor anyone in anything they have experience with.

Teaching is a skill all its own, and should not be taken lightly. The same skills you used to learn may not be what someone else needs, but the burden of translating from your own knowledge to the learning style of your mentee is on you. Varying your communication style is very helpful here - your skills in React frontend development may have been the result of reading a few tutorials and then jumping in the deep end, but your mentee may need a more curated learning experience, such as classes, walk-throughs, or regular pairing sessions. Asking questions, finding out how they learned other skills, and paying attention to how they respond to you trying different teaching tactics are all ways to tune in to how best to help them learn.

When answering one-off questions, your best bet of helping them is to ask them to completely explain the problem and where they are stuck, and anything they have tried so far. If it is clear to you what needs to be done, it can be helpful to guide them to the answer themselves instead of giving it to them directly, by asking leading questions. At times when the answer is not so clear, pairing with them and being very vocal about what you are thinking and the things you are considering will help them develop their own mental model for solving the problem in a similar way.

Most importantly of all, never treat someone as if they should already know. Comments like "you have never viewed the Logstash logs? Where have you been!" or "I would have thought you'd know this by now..." are toxic and only serve to make the other person feel bad. Be supportive and kind, and always encourage their questions.

Pair programming

Pair programming is, as its name suggests, highly collaborative. Your experience of this practice can fall anywhere between infuriating and illuminating, and the pair's communication skills have a lot to do with that.

One of the best things a pair can do when first starting out is establish how the pairing session will work. Will one person "drive" while the other "navigates"? When will you swap who is at the keyboard? How long will you go before a break? How long of a break should you take? Should the "navigator" use a computer, or should both purely work from a single computer? If any of these questions is ambiguous, it can make the experience confusing for one or both of the pair. Having clear ground rules for the session smooths the experience greatly, so take the time at the start to lay them out and form an agreement.

While pairing, it is critical that both of the pair learn to think out loud. Neither can read the other's mind, and it is natural for software engineers to retract into themselves when they are deep in thought. During a pairing session, though, speaking what you are thinking as you are thinking it allows both engineers to follow the train of thought and keeps the flow of the session moving. Without this thinking out loud, a lot of starting and stopping happens, and it can lead to one of the pair being left out, especially when the driver is not collaborating.

After a pairing session and before a break, it can be beneficial to discuss where you should pick back up when you get together again, and write that down so you don't forget. This will allow you to jump directly back into the pairing session without having to retrace your steps too much.

Retrospectives

Retrospectives lay the foundation for continuous improvement on a Scrum software engineering team. In my view, though, all software engineering teams should be running regular team retrospectives, even if they aren't using Scrum. Retrospective meetings are only effective, though, if the entire team is engaged in the process and participating by sharing their thoughts, raising concerns, and celebrating wins. By continuously engaging in this process of continuous improvement with an open, communicative group, a team can quickly go from good to great in only a matter of a few months.

Retrospectives vary in format, but a common one asks the questions: "What went well this iteration? Where can we improve?" The entire team is responsible for sharing anything they have seen in their day-to-day work - without this sharing, the exercise falls flat and the meeting results in wasted time for everyone involved. So the main thing software engineers should bring to retrospective meetings is their whole self.

Taking note of any relevant topics between retrospective meetings is a great way to bring your thoughts and experiences to bear the next time the group meets. It can also be helpful to engage other engineers in the meeting and try to drive towards concrete actions the team could take to address an issue that someone raises. If you tend to be one of the more outspoken members of your team, don't forget to save some air for others to speak their minds. Perhaps instead of speaking directly, you could encourage a teammate to share their opinion on a given topic (assuming you genuinely want to hear it).

Above all else, maintain the integrity of the meeting. Be supportive of everyone speaking their mind as long as it is on-topic. Don't blame or shame - keep the space supportive, encouraging, and safe. Don't be afraid to be lighthearted and fun! And always remember that retrospectives are intended to be in service to the development team. If it starts to suck, change it up.

Incident response

Responding to an active incident - the site is down, there's ransomware on the network, the last commit broke order submission - is some of the most stressful communication we are a part of as a software engineer. Incidents are also uncommon, so we don't get much practice. Nevertheless, having strong communication skills in these moments can make the difference between a smooth recovery and a frustrating nightmare.

Before an incident actually happens, it is helpful to read ahead of time about incident response and best-practices to follow. My favorite write-up on this topic is the PagerDuty Incident Response Guide. Having a clear picture of how incident response should go will help you frame how you may best participate and contribute.

When you are called in to be a part of an incident response, my best advice is to stay on topic. If you are unsure what is going on, ask someone how you can best get up to speed - typically there are documents being written to capture the events as they happen. Once caught up, asking where help is needed, announcing what you are doing, and announcing any findings you think may be relevant are how to stay on topic. Keep any extra chatter out of the main incident response call, and maybe even the main incident response chat room, if it is not directly related to resolving the issue.

Besides staying on topic, try to stay calm. Despite the highly stressful situation, remember that everyone is on the same team and is working towards the same goal. If needed, you can gently nudge people to take off-topic conversations off the call.

Incident postmortem

Incident Postmortems are how we reflect on all aspects of an incident and make changes to improve as an organization. This can be thought of as a "retrospective for incidents", and should be treated similarly. In essence, the postmortem is intended to be for the organization, helping it to improve and learn. A postmortem is not a root-cause analysis, a chance to lay blame, or anything other than a feedback mechanism for learning. All the rules around effective retrospectives apply here, with an emphasis on creating a safe space for sharing, because there are more likely to be sensitive topics in a postmortem than in your typical retrospective.

Strong Communication Sets You Apart

These 7 activities are only a sample of the times when our profession as software engineers calls for plentiful, clear, empathetic, and respectful communication. More can still be said about other activities such as sprint planning, project planning, collecting product requirements, cross-team collaboration, and more, but I think the point is made. Communication is a critical skill for software engineers, and it should be in the core set of skills that we work on throughout the course of our career. Strong communication skills are a superpower that will set you apart and establish you as reliable, a mentor, an asset, and a leader.

_{^{Opengraph background photo by Christina @ wocintechchat.com on Unsplash}}

What is Staff Engineering?

In the book, Will describes the Staff Engineer role as an answer to the question:

What if you want to advance your career without becoming an engineering manager?

A Staff Engineer is a software engineer that is operating at a scale beyond that of a Senior Software Engineer. It is common for there to be many levels of Staff engineers depending on the size of the company. They are named different things depending on the level and where you are: "Tech Lead", "Staff Engineer", "Principal Engineer" and "Distinguished Engineer" to name a few.

At most companies, "Senior Software Engineer" would be considered a "career" level, intended to be the level that most engineers stay at. Moving from Senior to Staff is not expected, and is analogous to moving from Senior into management. A Staff Engineer is not a "Senior Senior Software Engineer", but instead shifts their role to one of leadership, facilitation, sponsorship, exploration, glue work, and reach. As for coding, well...:

Most write some [code], some write none, but none write as much as they used to earlier in their career. [...] Even if you're not writing much, you'll be reading a ton of your coworkers' code and doing a fair number of code reviews.

Will interviewed many Staff+ engineers for the book and four common archetypes stood out: Tech Lead, Architect, Solver, and Right-Hand. He writes in depth about these, and highlights that this taxonomy is neither exclusive nor comprehensive. From reading through the interviews in the book, it was clear to me that each Staff Engineer had their own take on how best to play their role. The autonomy and self-direction that is common throughout the interviews resulted in Staff Engineers frequently customizing their role to play to their strengths while delivering value to their organization.

Becoming a Staff Engineer does not necessarily mean that engineering management is out of the question. Many of the Staff+ engineers interviewed had either been managers in the past, or occasionally revisit the decision. In her article The Engineer/Manager Pendulum, Charity Majors writes about the benefits of moving between the roles, making the case that this makes you better at both. On top of that, Staff Engineering requires many of the same skills that management requires, so choosing this path over management is not the obvious choice that it may initially seem.

About the Book

Staff Engineer is both a physical book and a freely available website.

The website, staffeng.com, was built as Will was building content for the book and remains freely available. It contains most of the content in the book, minus the introduction and conclusion. The site has some content that didn't make it into the book too, such as a handful of the interviews. It is a valuable resource for those that do not purchase the book, and a handy reference to the content for those that do purchase it.

The physical book is broken into 3 main sections: How to operate as a Staff Engineer, how to become a Staff Engineer, and interviews of Staff Engineers from the industry. In the "Operating as Staff" section, Will provides a wealth of advice on being effective and scaling yourself, as well as highlights many pitfalls to avoid. In the sections "Getting the title where you are" and "Deciding to switch companies", he outlines different strategies for obtaining a Staff role depending on your circumstances. Lastly, in the "Stories" section, Will provides 14 interviews of Staff Engineers working at companies like Stripe, Slack, and Auth0.

How this Book has Helped Me

This book has been a guiding light as I chart my own path as a Staff Engineer at my organization. I have few peers to look to for guidance in this role, so having available the collected experiences of Will and the many Staff Engineers he interviewed has been a lifeline to me. While it would be easy to coast along, jumping at the "next most important" task, I am instead crafting a well-rounded framework of priorities within which to work. I want to bring my best to my Principal Engineering role, and with this book's help, I feel more confident than ever that I can chart my own path.

Another benefit of reading this book has been the normalization of many changes I am going through. From coding less, to writing more, to feeling like I'm not contributing like I used to, reading the stories of the many Staff Engineers in this book helped me to realize that this is a natural transition I am going through and nothing to get discouraged about. This emotional support cannot be understated.

Finally, beyond the content in the book itself, Will and his interviewees provide a wealth of resources to dig deeper into. From book recommendations, to key blog posts, this curated list of resources will be a fountain of learning that stretches long past my finishing this book.

Conclusion

My recommendation: Read this book! 👍

This book is fantastic, and I recommend it. There's value here for software engineers of any level, and managers too. Will's thoughtful, researched approach to sharing guidance for this role is a massive step forward for the industry. My hope is that, with a resource such as this available, we will see an increase in opportunities available for these roles within the industry.

P.S: I also recommend the StaffEng podcast. While not hosted by Will directly, the hosts regularly interview Staff Engineers from across the industry in a similar format to the interviews from the book.

Thanks to the power and freedom of the internet, community no longer only refers to the people living a short car or bus ride away. Instead, we can connect with a vast worldwide community of like-minded professionals. Millions of people in the world are building software, and even with a small fraction of those people actively engaging in the tech community, the result is a bustling, healthy ecosystem of energetic engineers eager to share their experiences and learn from others. Connecting with this community, even as a passive observer, is a fantastic way to keep your mind sharp and your skills continuously improving.

I want to share with you some ways that I stay in touch with the tech community. Over the years I have tried out many tactics, but only a few have proven sustainable. By employing the techniques I will describe, I always have a wealth of quality knowledge at my fingertips, so that when I am ready, I can browse it at my leisure and get the most out of the time spent. While not a substitute for hands-on experience, my hope is that some of these techniques will help you, too.

Tune In to Sources with High Signal

Reserving your attention for valuable content is the most important point.

A lot is happening in the tech community, and there's no way you can keep track of everything by "drinking from the fire hose". Instead, tailor your connections to your core areas of interest. Initially you can cast your net wide, following various sources that pique your interest and trying out all possible mediums. As you begin to read, listen, watch or attend to your various outlets, the key is to pay attention to where you get the most value for your time spent.

This process is like measuring your personal "Signal to Noise Ratio", where "signal" in this case is information that is valuable to you, and "noise" is everything else. Your goal is to maximize the amount of signal you get from the content you consume, so you aren't wasting your time. The best way to do this is to simultaneously remove sources of noise and add new sources that could provide signal. Over the long term, this will result in most of your content sources being high quality and tailored directly to your interests.

Throughout the remainder of these tips, you'll notice this trend in action. In every case, the goal is to create a pool of potential content to tap into, let the best material bubble up to the top, and occasionally trim the remainder away that you didn't find useful.

Subscribe to Audio Content

Listening to audio content while you attend to your life is a great way to stay on top of your interests while also getting routine things done.

When I am exercising, working in the garden, cleaning the house, driving around by myself or folding laundry, I like to listen to things. I have found that podcasts, conference talks and audiobooks are all excellent to listen to during these activities.

Podcasts

Podcasts are my top choice for listening. I have an iPhone, and my podcast player of choice is Pocket Casts. The podcast format is generally more casual and conversational than a blog post. You can find podcasts in practically any genre you can imagine.

To find podcasts, I search in my web browser using keywords. Searching this way finds podcast websites, surrounding content and relevant links, which usually works better than an in-app search. Once I've found a podcast that looks interesting, I look it up directly in my podcast app, then subscribe to it. Subscribing to a podcast "bookmarks" that podcast in your player, so that you have easy access to its episodes. In Pocket Casts, you can then configure a filter to show you a list of all episodes from all podcasts going back in time. I will occasionally scan this list for episodes that stand out, and add those to my "Up Next" list. Finally, when I want to listen to something, I play that list.

At any given time, I usually have 15-20 episodes queued in my "Up Next" list, which means I can quickly throw my headphones on or plug my phone into my car and start listening. For my favorite podcasts, I even have them configured to automatically queue new episodes to the top of my "Up Next", ready to go.

Here are a few of my favorite podcasts:

• StaffEng podcast. It features conversations with software engineers who have progressed beyond the career level, into Staff levels and beyond.
• Maintainable podcast, where Robby Russell speaks with seasoned practitioners who have worked past the problems often associated with technical debt and legacy code.
• The Changelog podcast. Weekly discussions with people across the open source and software engineering world.
• The InfoQ Podcast, with conversations geared towards architects and senior developers.
• Chats with Kent podcast, where Kent C. Dodds (a React software development trainer) chats with developers about life, career and code. His 3 minutes with Kent microcast is also good.
• Soft Skills Engineering podcast is a lighthearted show where they respond to listener questions with witty banter backed up by industry software engineering management experience.

Conference Talks

Conference talks aren't a major source of content for me, but occasionally I will go find the YouTube channel of a specific conference that I am interested in. Typically, conference YouTube channels will create playlists that correspond to the group of videos that match a given conference. For example, this playlist from the 2020 GOTO Conference in Chicago, Illinois, USA. I like to browse through these playlists, adding any talks that stand out as particularly interesting to my "Watch Later" queue on YouTube. That way, anytime I am interested, I can open up the YouTube app on my phone and play the watch later queue.

This experience is improved greatly by my subscription to YouTube Premium. I primarily subscribe to YouTube for its integration with my Google Home, but an additional perk of this service is the ability to listen to YouTube videos when the mobile app is in the background. It also removes any advertisements! The result is a smooth listening experience whether I have my phone in my pocket or hooked up to my car.

Here are some channels that have good content I like to listen to:

• Rust channel. Example: Rust LATAM 2019.
• AWS Events channel. Example: AWS re:Invent 2020 Breakout Sessions | Architecture.
• Strange Loop Conference channel. Example: Strange Loop 2019.

Audiobooks

Audiobooks are my least preferred form of listening. Not only are they rarely freely available, but technical books usually do not translate well to audiobook format. Additionally, I am a bit sentimental with books. I like to have physical copies when possible, so that I can write in them, refer to them, and loan them out to friends and coworkers.

All that being the case, I do occasionally purchase audiobooks that discuss soft skills such as leadership, career growth, and personal growth. Sometimes Audible will run promotions that gives a good deal on some audiobooks, so I will subscribe for a time to take advantage of it and then unsubscribe later. In these cases, I will pick a book from my wish list (more on this later) that fits the medium, and purchase it in audiobook format instead. Some audiobooks I have enjoyed in the past are:

• The Effective Manager by Mark Horstman.
• The Five Dysfunctions of a Team by Patrick Lencioni.

Save Interesting Articles to Read Later

I enjoy reading, and split my reading between online articles and books. Online articles, usually blog posts, contain the freshest, most relevant content available. These could be news-related content, opinion articles, deep technical overviews, software documentation, and more. While running across good articles randomly does happen (looking at you, work Slack 👀) I have found that mailing lists and blog subscriptions provide a consistent stream of high-quality and interesting articles to read.

(Psst... You can see some of my favorite articles in my Pocket recommendations page! 🔖)

Mailing Lists

Tech-oriented mailing lists can be great, but the first step to benefiting from them is to tame your email inbox 😅. Once you are there, though, mailing lists provide a curated view of the tech world through the various lenses your chosen lists provide.

The most common kind of content I receive via mailing lists is news. The benefit is that I no longer have to go search to find out what is happening in the tech industry - each week, the newsletters deliver that news straight to me. The key is finding the right newsletters so that all your main interests are covered.

When I receive a newsletter from a mailing list, I typically will skim over it, saving any interesting articles to my Pocket account. Then I will archive the email and go about my day. Later, when I have time, I will go to Pocket and pick an article from the top of the stack to read. This is usually between tasks at work, after lunch, or before going to sleep. My Pocket list is massive... I generally have way more articles to read there than I have time for, but the most interesting content will get read. Pocket also gives me a way to track all the articles I read, so that I can refer to them later if I want to share with others.

The mailing lists I find most interesting are:

• Changelog Weekly, general news and interesting projects in the open source and software development world.
• Node Weekly, delivering Node.js-related news and topics.
• StatusCode Weekly. What's happening in web development, ops, platforms and tools.
• Hacker Newsletter, a weekly curated list of the most interesting topics from the startup and technology board Hacker News.
• Software Lead Weekly, a weekly email for busy people who care about people, culture and leadership.
• ThoughtWorks Technology Radar, a bi-annual commentary on tools, techniques, platforms and languages that are trending in the tech industry.

Blog Subscriptions

Subscribing to an individual blog is more personal than a mailing list. When I subscribe to an individual blog, it is someone that I respect and am keen to read anything they write. Most blogs allow you to subscribe to them in one of two ways: email subscription or RSS feed. An email subscription is just like signing up for a mailing list, except it is only that person's content you will be receiving. An RSS feed has to be treated differently.

To subscribe to an RSS feed, you would normally need an RSS feed reader application. The most popular feed reader used to be Google Reader, but that was discontinued in 2013. Today, some people use apps like Feedly, but I use Blogtrottr because I like to triage new blog posts the same way I triage mailing lists: in my email inbox.

I follow many blogs, which all have varying rates of activity. My favorites are:

• Michael Nygard. Author of Release It!, he writes about DevOps, Architecture and software craftsmanship.
• Martin Fowler. Enterprise application architect, author and speaker.
• Keavy McMinn. Staff-level software engineer. GitHub alumni.
• Michael Feathers. Author of Working Effectively with Legacy Code, he writes about enterprise application development.
• Jessie Frazelle. Serial silicon valley software engineer. All around hacker. Also has a column in the ACM Queue magazine.
• Camille Fournier. Author of The Managers Path, she writes about software engineering careers, management and leadership.

Keep a Book Nearby to Read

While online articles are amazing (and predominantly free), books provide a deeper look at specific subjects. The level of detail you get from a book is not likely to be found in your everyday online publication, so there is a time and place for reading books.

My strategy is to make a book wait. If I want to read a book, I will find it on Amazon and add it to my "Books for Work" list. When the time comes to buy books, I will choose from the list based on what topics are relevant to my current interests. Sometimes books sit on that list for months or years before they are purchased, but if they are good they will frequently bubble up to the top as I add them again (not knowing they are already on the list!) or when they come to mind.

Here is a sample of some recent books I liked:

• DevOps for Dummies by Emily Freeman. The definitive DevOps book.
• The Manager's Path by Camille Fournier. Excellent software engineering career manual.
• Fundamentals of Software Architecture by Mark Richards and Neil Ford. Covers software architecture terminology, patterns, and case studies.
• The Rust Programming Language by Steve Klabnik and Carol Nichols. An eloquent and comprehensive guide to Rust. Also available as a free book online.

Join Local (or Online) Meetups

Finally, reach out in the real world! In most medium to large -sized cities, meetups exist for many technological topics. These meetups are usually a chance to get to know other like-minded people, hear presentations about relevant topics, and maybe even give a talk of your own. Getting to know people in your area has the added benefit of being a network you can draw from if you need a new job.

Due to COVID-19 pandemic, I have been detached from my local community, but I'm looking forward to getting back to it. There's a Cloud Native meetup in my town that tends to share DevOps-oriented topics, and I am going to start looking for an ACM or IEEE meetup soon.

Don't Stop Learning

If you're a software engineer, the goal posts are always moving. Staying in touch with how software is built, learning new techniques for solving tough problems, and understanding trade-offs of new technology are all important parts of building a strong, lasting career. And they all stem from continuous learning.

So go forth, connect with your community, and grow.

These documents are released twice a year, and I generally try to read through them. Due to ThoughtWorks working with lots of clients (they're a huge consultancy: ~8000 employees in 48 offices around the world!), they have a perspective that those of us working at a single company don't have. The Radar is contributed to by many bright engineers, as well; people like Martin Fowler, Neal Ford, Sam Newman, Pat Kua and more. The way I like to think of these documents is as a fireside chat with veterans of the industry: they've been around the block a few times and have interesting things to share. Maybe they're useful to me, maybe not, but it's still great to hear these stories and allow them to shape my own perspective.

Below you will find the "blips" - individual trends in their assessment - that I found particularly interesting or that resonated with me.

Adoption Recommendation Highlights

The ThoughtWorks definition of their "adopt" classification is:

We feel strongly that the industry should be adopting these items. We use them when appropriate on our projects.

Of all the suggestions for adoption, four stood out to me.

Technique Adoption - Design Systems

The Technology Radar has this to say about design systems:

As application development becomes increasingly dynamic and complex, it's a challenge to deliver accessible and usable products with consistent style. This is particularly true in larger organizations with multiple teams working on different products. Design systems define a collection of design patterns, component libraries and good design and engineering practices that ensure consistent digital products. Built on the corporate style guides of the past, design systems offer shared libraries and documents that are easy to find and use. Generally, guidance is written down as code and kept under version control so that the guide is less ambiguous and easier to maintain than simple documents. Design systems have become a standard approach when working across teams and disciplines in product development because they allow teams to focus. They can address strategic challenges around the product itself without reinventing the wheel every time a new visual component is needed.

Design Systems are fantastic and especially help backend-heavy engineering teams make their applications look better, have ergonomic UX, and generally shine compared to how they would be without a design system. This is a great way to scale a smaller UI/UX team to have larger, organization-wide impact.

Building the design system is only one piece of the puzzle, though. To be successful, it needs adoption. Without organizational mandate, the design team needs to socialize it, make it easily accessible, and advocate for its use and growth. As teams begin to use it and see its value, organic growth and feedback will kick in to foster its continued use and adoption.

Our company is just beginning to build out its own design system with React and Storybook. I'm really looking forward to seeing if it gains traction and adoption outside the immediate team that is building it.

Technique Adoption - Applying The Expand-Contract Pattern to APIs

API versioning is hard. I was encouraged to see this one in the radar's "adopt" section this year:

The API expand-contract pattern, sometimes called parallel change, will be familiar to many, especially when used with databases or code; however, we only see low levels of adoption with APIs. Specifically, we're seeing complex versioning schemes and breaking changes used in scenarios where a simple expand and then contract would suffice. For example, first adding to an API while deprecating an existing element, and then only later removing the deprecated elements once consumers are switched to the newer schema. This approach does require some coordination and visibility of the API consumers, perhaps through a technique such as consumer-driven contract testing.

I'm not a fan of "versioning schemes" in REST APIs, but I am a proponent of upholding a contract with my consumers. Prepending versions onto an API (like www.myservice.dev/api/v1/...) is ugly and frequently only ever ends up being /v1/ for the application's entire life. Other versioning schemes, such as requiring a specific Accept header, I have not seen much of in practice.

The API expand-contract pattern, on the other hand, offers a pragmatic approach to introducing changes using a migration strategy embracing the concept of evolutionary architecture. Instead of cut-overs that require lock-step collaboration, we can instead think of our changes as a slowly evolving system. Using the expand-contract pattern, we can change our system organically, allowing our consumers to migrate at their own pace. This is the path that most aligns with my experience and preference.

How the Expand-Contract Pattern Works

One of the best practices for consuming a JSON API is to ignore any keys you don't use. In other words, If you are consuming an API that has a signature like this:

{
  "id": 123,
  "author": {
    "name": "Ada Lovelace",
    "email": "ada@chadxz.dev"
  },
  "commit": "dd252e7"
}

... you should avoid doing anything that would assert the exact structure of the payload. Doing so would couple your implementation too closely to the response, whereas you could instead code only to the keys you need and ignore the remainder. That way if the payload changed to add a new key:

{
  "id": 123,
  "author": {
    "name": "Ada Lovelace",
    "email": "ada@chadxz.dev"
  },
  "commit": "dd252e7",
+  "hash": "dd252e7478279c6391b50421cec801a652040986"
}

... your code continues to work without any changes. This allows the API producer to evolve an API without breaking you. They can then deprecate the commit field and schedule it to be removed, allowing their consumers to migrate to the new hash field if they need it.

This pattern could also be followed by adding a new API while keeping the deprecated API in place for a while. For example, in the gRPC ecosystem with the protobuf RPC format, they recommend never removing fields, renaming fields or messages, changing field types, etc. Eventually when you do need to cull the old versions, you would release a new version of your service with a new protobuf version, deprecate the old API, and allow the two to live alongside one another until the old API's End of Life (EOL) date hits.

You can read more about the expand-contract pattern in the corresponding Martin Fowler wiki article.

Other Noteworthy Adoption Recommendations

• technique adoption - Platform Engineering Product Teams - Platform Engineering stands out to me as a model for IT to adapt to the cloud while continuing to provide value to the engineering teams they aim to support. Building out the platform as a "product" with internal "customers" is the only way to ensure that what is built is serving, instead of hindering, the teams that will be using it. Building an internal IT platform as a "product" involves much of the same collaboration and mutual consideration that the DevOps movement is all about, but it is also a stark paradigm shift from the normal workflow of an IT professional. While challenging, I believe this kind of endeavour is the future of engineering-focused IT teams.

• tool adoption - Sentry - this was called out as a go-to tool for error reporting. Not much to say here, other than *nodding head furiously*. I highlighted Sentry as my preferred bug tracker in my 2021 Node.js Best Practices post. It's a great tool, and people should definitely be using it if they aren't already entrenched using something else.

Trial Recommendation Highlights

The definition of their "trial" classification is:

Worth pursuing. It is important to understand how to build up this capability. Enterprises should try this technology on a project that can handle the risk.

There were 2 main techniques they qualified for trial that I strongly agree with, and others that piqued my interest that I want to learn more about.

Technique Trial - Lightweight Approach to RFCs

This recommendation was confusing to me:

As organizations drive toward evolutionary architecture, it's important to capture decisions around design, architecture, techniques and teams' ways of workings. The process of collecting and aggregating feedback that will lead to these decisions begin with Request for Comments (RfCs). RfCs are a technique for collecting context, design and architectural ideas and collaborating with teams to ultimately come to decisions along with their context and consequences. We recommend that organizations take a lightweight approach to RFCs by using a simple standardized template across many teams as well as version control to capture RfCs.

It's important to capture these in an audit of these decisions to benefit future team members and to capture the technical and business evolution of an organization. Mature organizations have used RfCs in autonomous teams to drive better communication and collaboration especially in cross-team relevant decisions.

It's unclear what they are comparing lightweight RFCs to... "Heavyweight" RFCs? There's no description or elaboration on what the problematic RFCs are and what to avoid. Despite this, RFCs are an area of active interest for me, so I wanted to highlight this.

At work, we have begun to try Architecture Decision Records to capture some of our tribal knowledge. ADRs are documents that capture the context behind major decisions made about an application's architecture. Things like "why was framework X chosen?", "why is this service using basic authentication vs OAuth2?", or "why are we using X test framework?" Along with the why, other examples of topics covered by these documents are alternatives that were considered or pressures placed on the team when the decision was made. The documents are stored in the source repository with the application, and aim to share the knowledge that would otherwise be unavailable unless the engineer that made that decision is still on the team. I like ADRs so far, but it is still a new process for our team, so we're evaluating it to see if it is a good fit.

RFCs and ADRs are mostly designed to serve the same purpose, but with RFCs the emphasis is on seeking feedback and/or approval of your proposal. This emphasis on collaboration benefits all engineers in the company by enhancing accountability and cross-team knowledge sharing. I would relish the opportunity to adopt an RFCs model for cross-team collaboration on best practices, technology adoption, and major architecture changes!

For the time being, I'm going to continue to encourage ADR use in our greenfield applications, advocate for designs to be documented on our internal wiki, and ensure those designs are shared out to those within our team and on other teams that may have some expertise on a given subject. I'm hoping these actions will further develop our culture of documentation and collaboration. Long-term I hope to introduce a process for RFCs.

To learn more about RFCs, I recommend this article by Gergely Orosz.

Technique Trial - Hypothesis Driven Legacy Renovation

Hypothesis-driven legacy renovation is described this way:

We're often asked to refresh, update or remediate legacy systems that we didn't originally build. Sometimes, technical issues need our attention such as improving performance or reliability. One common approach to address these issues is to create "technical stories" using the same format as a user story but with a technical outcome rather than a business one. But these technical tasks are often difficult to estimate, take longer than anticipated or don't end up having the desired outcome. An alternative, more successful method is to apply hypothesis-driven legacy renovation. Rather than working toward a standard backlog, the team takes ownership of a measurable technical outcome and collectively establishes a set of hypotheses about the problem. They then conduct iterative, time-boxed experiments to verify or disprove each hypothesis in order of priority. The resulting workflow is optimized for reducing uncertainty rather than following a plan toward a predictable outcome.

At work our team is responsible for dozens of legacy applications, and we employ this technique! Here is an example.

We have a legacy API that has served the company well and largely been stable for years. It had recently been modified to accept events from Salesforce's Outbound Message Queue and save them to a database. Subsequently, we found that our Sales team would occasionally run mass updates in Salesforce, resulting in a torrent of calls to the API that would bog it down for hours. Each time this happened, we would sit down after remediation to discuss what went wrong and how we might improve the system. Most of our initial changes did not result in significant improvement, so we decided to turn the problem into a project for someone to dig in to. The project owner did some deeper performance analysis, wrote up their hypotheses about the cause of the problem, and we regrouped as a team to discuss the proposals and decide what actions to take.

This approach to problem-solving and "legacy renovation" is a great team-building technique! It also gave the individual leading the project a chance to own something important to the team, drive the discussion and perform the experiments. Overall, I recommend this approach!

Other Interesting Trial Blips

• tool trial - SWR - SWR is a library for making http requests from a client side web application and has built-in support for the "Stale While Revalidate" caching strategy. I've been away from React for a bit, but I want to keep the knowledge of this tool fresh for the next time I dig in.

• tool trial - ESBuild - The ESBuild Javascript build tool keeps popping up on my radar. It seems to be a Webpack/Parcel replacement that improves compilation performance. Certainly worth a look the next time you go to add a Javascript compiler to your build toolchain, especially if you don't need some of the more advanced features these other tools provide.

• technique trial - ethical-explorer - Ethical explorer aims to encourage teams to think about the ethical implications of the systems they build. This kind of introspection would make a great addition to a toolbox of team retrospective prompts.

• technique trial - distroless-docker-images - The idea of building a Docker image with the bare minimum underlying dependencies is great! Looking forward to trying this out.

Assessment Recommendation Highlights

Worth exploring with the goal of understanding how it will affect your enterprise.

The assessment recommendations that stood out to me to highlight were either things new to me that I am aware of and already excited about, or things I'm actively interested in.

• technique assessment - privacy focused web analytics - This is a trend that is emerging from the wake of GDPR and the corresponding surge in obtrusive cookie warnings and opt-in requests. Privacy-focused analytics instead says "what if instead we strive to anonymize analytics for a better user experience?" There are client-side solutions available like the Plausible service mentioned in the blip, but also competitor Fathom. For my site, I'm using Netlify's server-side analytics, which is rad because it doesn't require any client-side tracking - everything is pulled from server logs. I don't know how robust a solution this is, though, because without the client tracking you probably can't do more advanced analytics. Still worth exploring.

• tool assessment - Buildah and Podman - These tools are the Redhat tooling for building container images and running containers, respectively. I've known about their existence for a while, but have continued to use Docker for these needs. Docker has been working well and I haven't felt the need to look elsewhere, but I wonder why people are adopting these tools. Am I missing out?

• tool assessment - GitHub Actions - I explored GitHub Actions when it first went public, but ended up getting distracted and never returned to it. Recently I was building some boilerplate for a Typescript library I was planning to build, and found a nice template for a GitHub Actions Typescript build pipeline within the tsdx repository. Using this template for my build pipeline made me realize that I wouldn't ever need to look outside GitHub for my CI needs anymore. This tool is a huge quality of life boost, and combined with the new Github Packages feature, has me in awe of the features Github is bringing to bear of late. If you haven't checked out Github Actions lately, you definitely should!

• platform assessment - Kafka API without Kafka - As part of an ongoing effort to familiarize myself with Kafka, this blip stood out to me. I've never used Kafka, but I read up it occasionally to develop a mental model of where it would be a good architectural fit (beyond simply thinking "Event driven == Kafka").

• framework assessment - React Hook Form - This suggestion I want to keep an eye on, for when I get back into React development.

Hold Recommendation Highlights

Proceed with caution.

There were two main hold recommendations that stood out to me that I want to highlight, but a few additional ones that surprised me or felt were noteworthy.

Hold This Technique - Peer Review == Pull Request

This anti-pattern hits particularly close to home for me:

Some organizations seem to think peer review equals pull request; they've taken the view that the only way to achieve a peer review of code is via a pull request. We've seen this approach create significant team bottlenecks as well as significantly degrade the quality of feedback as overloaded reviewers begin to simply reject requests. Although the argument could be made that this is one way to demonstrate code review "regulatory compliance" one of our clients was told this was invalid since there was no evidence the code was actually read by anyone prior to acceptance. Pull requests are only one way to manage the code review workflow; we urge people to consider other approaches, especially where there is a need to coach and pass on feedback carefully.

I have been a long-time advocate of adapting code review to the work being done. Many times a pull request is fine, but software engineers should familiarize themselves (and practice!) other forms of peer review when appropriate. Pair programming, mob programming, and RFCs are other forms of peer review that should be considered depending on the scope of the review that is desired.

It is also critical for junior engineers to have active mentorship. It is not enough for them to rely solely on pull requests for feedback and learning. Everyone learns differently, and teams need to consider that the way they normally provide each other feedback may not work for an engineer that is just getting started in their career.

So branch out and try a new form of peer review! You might be surprised to find that it is a much more rewarding experience than what you are used to.

Hold this Technique - GitOps

I was surprised to see GitOps on the hold list, but their explanation makes sense:

We suggest approaching GitOps with a degree of care, especially with regard to branching strategies. GitOps can be seen as a way of implementing infrastructure as code that involves continuously synchronizing and applying infrastructure code from Git into various environments. When used with a "branch per environment" infrastructure, changes are promoted from one environment to the next by merging code. While treating code as the single source of truth is clearly a sound approach, we're seeing branch per environment lead to environmental drift and eventually environment-specific configs as code merges become problematic or even stop entirely. This is very similar to what we've seen in the past with long-lived branches with GitFlow.

At work, we use Git to store our infrastructure as Ansible playbooks, and the different environments are represented as separate inventories you can apply a single deployment playbook to. Our development workflow involves short-lived feature branches, but all environments are deployed from the main branch. This helps us avoid the drift problem described above and makes us think carefully about when and where we have our environments differ from one another. It is also the "painful path" to allow environments to diverge using this model... a good thing!

Other Noteworthy Hold Recommendations

• hold this technique - ticket driven platform operating models - When I saw this I agreed wholeheartedly. We should be looking to reduce gating and friction!

• hold using this tool - AWS CodePipeline - This surprised me! I don't often see advice to not use an AWS service, so I wanted to bookmark this as a data point for consideration should CodePipeline ever come up in conversation. By then, though, CodePipeline may have changed. AWS seems to iterate heavily on their services.

• hold this technique - separate code and pipeline ownership - This is what I often hear being called "DevOps". If anything, a DevOps team can help set up CI, but then advocate for teams to learn the tooling, processes, and administration of these pipelines, leading to the DevOps team relinquishing it to the consumers. At my company, we have adopted a process where each team owns the operation and maintenance of their CI build plans, deployments, and build agents. We wish that a "platform team" would own the operation of the server itself (or a cloud service), but for now we're also stuck maintaining that portion. Hopefully we'll be migrating that to a cloud service soon.

Conclusion

The ThoughtWorks Technology Radar is a great resource for learning about the experiences of others in our industry. It is one of the many ways that I keep up, so that I can take those learnings back to my company and incorporate them where it makes sense. If you find these highlights interesting too, you can read the entire radar at the website, and signup there to be notified when new versions are available.

In the future I plan to write about other ways that I keep up with news, industry topics, and educational resources. If you'd be interested in that, stay tuned.

I say "in 2021" because the concept of a "best practice" is a constantly moving target. As we build we learn, but the ecosystem is also shifting under our feet. An important skill we can develop as a software engineer is keeping an eye on the language ecosystem that we are working in, watching for new developments that supplant older tools with better ways to do things. Consider this document a "snapshot" of the practices that work well for me today.

Follow the 12-Factor App Guidelines

Around 2011-2013, the concept of a 12-factor app made the rounds. Originally written by Adam Wiggins who was then working with Heroku, the site describes guidelines for building applications that are designed to be run in the cloud. Nearly a decade later, these guidelines are still largely applicable to the web software that we build today. Key ideas covered include:

• Dropping reliance on flat files for configuration, logs, state, etc.
• Using a deployment model that explicitly manages dependencies
• Building with horizontal scalability in mind
• ... and more

Even though these "12 factors" map cleanly to Heroku's deployment style and therefore are an obvious on-ramp for using their service, they are undoubtedly relevant even when you deploy to another cloud service or self-host.

Many of the "factors" outlined on the 12-factor site will be touched on later in this document. If you are building a Node.js API, it is a good idea to be familiar with the 12-factor principles and apply them where it makes sense.

Use a Consistent API Design

When building a web API, it is easy to craft endpoints that, in aggregate, are haphazard and messy to work with. Inconsistencies in response codes, response format, acceptable request content types, and more can all be avoided with some up-front thought about the patterns and standards you want to employ.

The guide that I have found to most closely match my mental model for how best to design a web API are found in the HTTP API Design Guide. I have never implemented 100% of the recommendations made in the document, but the "request" and "response" sections are gold and should be seriously considered when building your own web API. These make suggestions such as:

• use plural resource names
• clearly delineate actions when they do not map cleanly to HTTP verbs
• downcase paths and attributes
• minimize path nesting
• ... and more

Consistency is a key part of building an ergonomic web API. So read the guide, decide on your own standard, and stick to it!

Understand the Meaning of "RESTful"

Almost none of the "REST APIs" on the web are 100% "RESTful". How much or how little you adopt the tenets of RESTful API design is entirely up to you, but it helps tremendously to understand the core architectural pattern of building a web API before you make your own adjustments.

The primary architectural constraints of a RESTful web API are:

1. a client-server architecture that allows each component to evolve separately
2. stateless requests, where any state required to process a request is sent along with the request (think: cookies)
3. cacheable responses, improving client performance and server stability
4. a layered system, where the client cannot distinguish if it is speaking directly to your API or if an intermediary such as a proxy is present
5. a uniform interface providing a consistent user experience

These principals are relevant because they inform much of the design of the web as we know it, as well as many of what are considered standard practices today. Understanding these things will help you make good decisions when building web APIs and know when certain patterns deviate from the norm.

Some resources I used to learn the fundamentals of RESTful web API design are the corresponding Wikipedia article, the resfulapi.net tutorial, and the book RESTful Web APIs by Leonard Richardson & Mike Amundsen.

Use the Express Framework as a Starting Point

While it is possible to build a web application in Node.js using only the built-in functionality the standard library provides, you will end up writing a lot of code that is common to every web application and is sometimes tricky to get right (such as routing and error handling). Using a web framework allows you to be more productive by abstracting away these common concerns and giving you powerful patterns to build on top of.

Express.js has been the de-facto standard for building Node.js web applications for a long time. The State of Javascript survey found it to be the most used Javascript web framework by a large margin for the past 4 years. The wide breadth of libraries and addons for Express make it a good choice for any kind of web application. It is being well maintained, is under the stewardship of the OpenJS Foundation, and is largely considered to be a stable basis to build upon.

One drawback of using Express, though, is the need for an additional module to provide an async/await-friendly way to write your route handlers. I use express-async-handler package for this, but it would be nice if they would bake this directly into the framework.

The Express.js documentation is a solid resource for learning how to get up and running with the framework. If you are interested in a book, Express in Action appears good (Note: I have not read this, so take my recommendation with a grain of salt). An important aspect of building an Express application is stitching together all the necessary components such as session handling, secure headers, body parsing, and more. The best way to learn which to use here is to look at open source applications. Some examples I can recommend are sahat/hackathon-starter and TryGhost/ghost.

Deploy Your Application as a Docker Container

To me, Docker is about consistency. When deploying an application to a remote server, ideally you would not require any dependencies or configuration... you could simply push your binary to the server and run it with no arguments and it would behave exactly as you expect. Unfortunately, the reality is far from this ideal. Most applications require some form of configuration, and Node.js applications additionally require the Node.js runtime and package dependencies to be installed at the minimum.

By packaging your application as a Docker container, you not only bundle all of the dependencies it requires into a single executable Docker "image", but you also can deploy it onto a system alongside a mix of other applications that don't necessarily use the same Node.js version, or Node.js at all! In addition to the convenience of a single consistent package, you can also package your application to adhere to the typical "Docker" contract: that is, writing your logs to STDOUT and exposing your application's listening ports via the Dockerfile expose keyword. The result is that many of the typical operational concerns you have to deal with when managing a web application on a linux environment are standardized to reduce the cognitive burden of maintaining them.

There are many providers that support using Docker images to run your application directly. Docker now has built-in support for running your application on AWS ECS / AWS Fargate, but if you wanted to run the host yourself, I recommend using Ansible to provision the host, setup the Docker runtime, and deploy the application.

Using Docker also provides an easy way for developers to collaborate on a given application using Windows, Linux, or MacOS as their desktop environment. Having all the packages installed within the Docker image largely eliminates the "works on my machine" problem, but some care does have to be taken for applications that have to be reloaded when their code changes.

To learn Docker, I primarily relied on the Docker documentation. There are also myriad tutorials and classes available online.

Use node-config for Configuration Management

In the 12-factor guidelines, it talks about storing your configuration in your environment. I have also found this to be a solid approach that pairs nicely with running your application in a Docker container. Configuring your application with environment variables in development can be a hassle, though, so it is good to find a configuration tool that allows using both environment variables and flat files.

In the Node.js ecosystem, I have been using node-config for years and have never had any desire to move away from it. It is simple, flexible, and powerful. It supports different flat file configuration formats, but I tend to use YAML.

To configure your application with node-config, you create a config/ directory at the root of your project, then place flat files there that map to values of the various environments you may run your application in with the NODE_ENV variable (for example, config/production.yml is used when you run your application with NODE_ENV=production node server.js). Additionally, you can specify a file that maps your configuration values to whatever environment variable name you want. The way it handles defaults and overriding makes application configuration simple and painless. Then, in your application code, you simply const config = require('config'); and then config.get('my.config.option') to retrieve the value. The values are easy to override in your tests if needed, and the application will crash (as you would want) when a configuration option is expected but not present.

Always use node-config in your Node.js web APIs, unless you are using a high-level web framework that is handling configuration management for you.

Use express-openapi For Swagger Docs

API Documentation is difficult to keep up-to-date, but is nevertheless important for any non-trivial application. Swagger, or more precisely OpenAPI 3, is a format for specifying the API contract of a web API. When I use it, I define my API endpoints, authentication, inputs and outputs in a YAML file. This file can then be shared with my API consumers using Swagger UI to provide rich documentation and an interactive UI for trying out the API!

To ensure that this specification stays in sync with the actual application endpoints, I use the express-openapi library to build the main Express API router, request/response validation, and authentication mechanisms directly from the OpenAPI specification file. This tight coupling gives me 100% confidence that my documentation is up-to-date. The built-in JSON Schema validation and type coercion removes much of the boilerplate required to start working with my API inputs, and the response validation lets me know if any of my responses stray from my established contract.

Overall though, I wish there were better support for Swagger and OpenAPI in the Node.js ecosystem. While I do like express-openapi, it is primarily run by a single maintainer, which is not viable long-term. I occasionally run across bugs in the framework too, such as issues parsing $ref or request data not being coerced properly. But overall, it is the best tool available that I'm aware of, and I would much rather use it even under these conditions than go back to not having the OpenAPI documentation.

Here's hoping more quality OpenAPI tools arrive in the Node.js community.

Use Objection for Your MySQL Database ORM

I've used a few different ORMs in the time I've been writing code in Node.js: Waterline, Mongoose, Sequelize, Bookshelf, and now Objection. I have also eschewed ORMs entirely and directly used Knex to build my database-connected models. Of all these approaches, my favorite so far is using Objection.

Objection is an ORM for SQL databases. It is a thin layer on top of Knex, providing just enough of an abstraction to lend a hand with building models and the relationships between them. It is thoroughly documented and feels ergonomic to program with. It keeps the best parts of Knex available where you need them. For now, I'll be using Objection when I am building a SQL-backed app.

Report Your Errors Somewhere You Can See Them

Errors do happen. The key is that you will want to know about them so you can fix them!

At work, we use Airbrake for our error reporting. It is a reasonable service and has served our needs sufficiently. For my personal projects, I prefer Sentry. It is similar in functionality and allows you to either use their cloud service or self-host the server that the errors are reported to.

For either of the above services, the general method for using them is the same. You create the client and register them to listen for errors in your application, and when they happen, the errors are sent to an aggregation system for reporting and alerting.

I wholeheartedly recommend using some sort of error reporting in every Node.js application. Pick one and implement it, and you will be in a much better position to support your application for your users.

Lint and Automatically Format Your Code

In most languages there is the concept of a "linter": a tool that checks your code for problems. The problems that a linter looks for are issues that a normal compiler would allow, but that are disallowed by some standard for writing software. That standard could be local to a given team, to a company, or to a language ecosystem as a whole.

It has also increasingly become commonplace for languages to have a "formatter", which rewrites your code in a consistent way.

I recommend ensuring you have both setup when writing Javascript. At work, we use a combination of ESLint, Standard, and Prettier to lint and format our code. Using these tools helps to guide your code down a "happy path" and keeps the code consistent across multiple developers. We chose the Standard set of lints so that we could have some baseline linting that was minimally opinionated but that gave strong guidance away from common issues experienced when writing Javascript. Things like accidentally forgetting your break statement in a switch branch, or using a coercing equality operator (==) instead of a strict equality operator (===) are checked for and warned about. These sorts of warnings help avoid common issues and result in more efficient development.

Since Standard and Prettier have some overlap (Standard has some rules defining formatting), you can use prettier/eslint-config-prettier to disable any style-related rules Standard would setup to allow Prettier own style-related concerns. You can also use prettier/eslint-plugin-prettier to get warnings from eslint if your code is not formatted as your prettier rules say they are. Finally, you can combine all of this with tools like Husky and lint-staged to ensure that both eslint and prettier are running prior to your code being committed to version control.

Set these up once at the beginning of your project, then let them guide you and keep you from thinking about this stuff any more!

Use Jest to Test Your Code

There are dozens of test frameworks available in the Javascript ecosystem. This is great! Having the ability to choose which one matches your mental model closest is a good thing. For the work that I do, I have been setting up Jest for new projects and have been really happy with it.

Jest is a test framework that emerged out of the React ecosystem. It is a "batteries-included" test framework, combining testing, mocking, and test coverage analysis all in one. Previous to using Jest, I had been using a combination of Mocha, Sinon, and Istanbul to cover all of these needs. Additionally, Jest has really nice assertion failure output, helping you find exactly where the problem is in your code. No other test framework that I have worked with has this level of quality in their assertion output. Overall, using Jest is a quick way to get up-and-running with tests in your code.

Additionally, I recommend co-locating your tests with your code. That is, putting your test files in the same directory as your actual code files. For example, if I have a file named src/models/PhoneConfiguration.js, I would also have a src/models/PhoneConfiguration.test.js file containing my Jest tests. This pattern helps speed up your development flow, and quickly highlights where test coverage is completely missing from certain files.

Test your code! And use Jest.

What About Typescript?

While reading through this, you may have been wondering:

What about Typescript? Isn't that how people are writing Javascript these days?

It is true that Typescript has risen massively in popularity over the past few years. Indeed, even though I have not been writing any Typescript directly in my day-to-day, I get a lot of value out of the typings that libraries provide, both those bundled with the libraries and those I add as dependencies from the DefinitelyTyped repository. These help my editor give me advanced code completion for those libraries and speeds up development.

But at my company, Node.js is still on an adoption/learning curve. We are wanting to make that curve as shallow as possible for new contributors to the Node.js applications, because they only have so much time in the day to be learning new things. Their attention is split across many different technology stacks, ranging from legacy PHP with custom-rolled web frameworks to Ruby on Rails applications, to the one-off Node.js applications that are beginning to gain more traction. We are getting a lot of value out of JSDoc to explicitly declare the inputs and outputs of functions, and since that is entirely opt-in, it does not get in the way of developers ability to get their work done.

Over time I expect that we will write more Typescript. But for now, it's not a priority.

Closing Thoughts for 2021

So that is my list of recommendations for building a Node.js API in 2021. While these practices are working well for me now, I'm also paying attention to where there are gaps or friction. I'm also watching out for new, innovative ways to build the sorts of applications I'm building. On my radar are things like GraphQL, better logging libraries infrastructure (like the Pino library and Grafana Loki), and distributed tracing tooling (the OpenTracing project). I am also constantly learning architectural patterns that are in use in our industry, so that I can be familiar enough with them to know when they may apply to any given problem that I'm attempting to solve.

The main improvement to this stack in the last year is the use of Objection. I had been keeping an eye on it for a while and only this year decided to give it a try. I was really happy with how that application turned out, and look forward to continue working with that library in the future.

The main area that I see that is lacking is the OpenAPI support. express-openapi is a good tool, but I would like to see wider adoption and an increase in the contributors to that project, or see another tool rise up in its place that has a stronger backing and wider adoption. I feel OpenAPI is severely underrated for what it provides.

So, go forth and build. And remember: "Best Practices" come and go. Keep your eyes open!

This time I am determined to take a different approach. While I do want to leave the door open to customizing all aspects of the site, I am starting from the beginning with blogging as the primary focus. I really want to break out of the discomfort of blogging, and I know the only way to make that happen is to... just do it. It'll get easier as I go, right?

In this post, I want to share my initial setup with you.

Choosing 11ty as My Blogging Platform

I had already decided that I wanted to self-host and use a workflow driven by flat files and static-site generation. I really like the git-based flat files because they are simple, portable, and mine. I'm also really comfortable working in Markdown and have tools (like IntelliJ editors and Marked.app) that have strong support for Markdown. I knew there were good free options for hosting static sites and that more have emerged in recent times, so building this way would be a low-overhead way to start blogging. The only decision left was to choose which SSG to use.

I narrowed my choices of blogging platform down to 3: Zola (written in Rust), Hugo (written in Go), and 11ty (written in Node). There are myriad others available as can be seen on jamstack.org, but these were the ones that were already top-of-mind to me. Zola because I have recently been learning Rust and it is a popular choice in that community; Hugo because I had run across a Hugo template I liked recently (hugo-coder); and 11ty because it is known to be a "simple but powerful" SSG compared to the likes of Gatsby and Next.js.

I chose 11ty in the end because, even though I liked the hugo-coder template, I felt like I would want to build up my own template from scratch. I also felt like having a SSG that was installable by npm install along with all the other dependencies I would want to use like Prettier, Husky, and lint-staged, would be a really smooth experience. I was also really happy that 11ty supported Nunjucks templating as a first-class citizen. Nunjucks is essentially a port of the Jinja2 templating language, which I'm very familiar with from working so much with Ansible.

So after getting some boilerplate setup for 11ty (this blog post from @mtmdev_ was really helpful), the next step was deciding where to deploy it.

Deploying My Site to Netlify

I have always been a fan of platforms like Heroku and Github Pages that take so much of the toil away from deploying applications. I have deployed web applications to both, and have been very happy with them. BUT! You'd have to be living under a rock to not see all the buzz about Netlify these days, so I wanted to give it a try to see for myself.

The short story about Netlify is that the buzz and attention they get are fully deserved. After publishing my application to a repository on Github, signing up with Netlify took seconds. Walking through their on-boarding process is quick and before I knew it my application was fully available at a random netlify url. I could hardly believe how easy it was.

Netlify also offers server-side analytics for $9/month, which I thought was a good deal. Yes, Google Analytics is free, but I wanted something that was more privacy conscious. My runner-up was Fathom, but it was more expensive, and I really liked that server-side analytics don't require client-side scripts to be effective.

So now that my application was up, a new concern arose: What domain did I want to use?

Grabbing a New Domain Name: chadxz.dev

My previous personal websites were all hosted under the domain name chadmcelligott.com. While straightforward, this domain name is difficult for most people to spell or remember, so I wanted to change it up and go for something shorter and simpler.

To purchase the new domain name, I used Namecheap. In my experience they're a good quality registrar. I have numerous domains I have purchased with them throughout the years and have never had any issues. When I went to choose the domain, I knew I wanted a .dev domain because I like that they are https only and stands for "developer" (hey, that's me!). I chose chadxz.dev because chadxz is my Github username, it was available and cheap, and it felt like a good fit.

After purchasing my domain, I moved the DNS over to Netlify DNS service. Using the Netlify DNS is free and is configurable if you want to set up additional records besides the ones necessary to host your site. As a bonus, once your custom domain is configured, Netlify automatically provisions a SSL certificate for free.

So now that my application is up, what other functionality did I feel my site needed to have a solid foundation?

Establishing My Newsletter with buttondown.email

One aspect I have always appreciated about some blogs is the ability to follow bloggers whose content I enjoy. I knew when setting up my blog that I wanted to make it super easy for people to follow my content if they wanted to. On this front, I decided on two pieces to tackle this:

1. Ensure the blog has an RSS/Atom feed setup
2. Make it painless to receive new content via email

The RSS/Atom feed was super easy - 11ty has a 1st-class plugin for generating Atom feeds, and the walk-through on their site shows how to build a basic template for one.

Adding the ability to subscribe to my blog content via email was more work and required vendor selection. I knew I wanted to find something free and easy, because I was only getting started and knew my needs would not be complex. As it turns out, earlier in the day I had subscribed to Cassidy Williams' newsletter "rendezvous with cassidoo", which uses buttondown.email. Taking a look at their site, they do everything I wanted and have a generous free tier (up to 1,000 subscribers for free), so I decided to roll with them. My newsletter is now available at https://buttondown.email/chadxz, and I'll be adding a small unobtrusive signup form to the bottom of my post template in the future.

Now while setting up my newsletter, I realized I wanted it to be sent by me and from my new domain. In order to make this work, I would need some way to receive email at my domain.

Receiving Email at My Custom Domain With ImprovMX

While doing some research into whether Netlify DNS supported sending and receiving email, I ran across an official recommendation on a Netlify support thread for using the service ImprovMX. ImprovMX has a dead-simple setup and allows setting up to 5 email addresses (including a wildcard) to forward emails sent to your custom domain to the email address of your choice. For free. If you want to send emails from your custom domain, you have to cough up some $$, but for now I'm going with their free tier.

The Foundation has been Laid

So with all of that done, we have:

• A static, Markdown-based site on Github...
• That deploys automatically on push to the main branch...
• Behind a custom domain with https...
• With an RSS feed and an email newsletter...
• And a custom email address that can receive email on my new domain.

The site itself isn't much to look at yet, but with all of this preliminary work done, design and extra features can be added. The most important thing is that all the barriers to publishing are gone!

I'm looking forward to designing the site and writing more on a variety of topics: other projects I have going on (like learning Rust), books and articles I am reading, thoughts about challenges I face at work, and more.