DWF’s Journal

The Pair's The Thing - Introducing PXP

2026-06-28T14:32:00+00:00

My experience with chat-based AI code generation in 2025 was…fine. I liked it, but didn’t love it. I didn’t see what all the hype was about. And then the agents entered the chat.

Linked-Intent Development

I had my first agentic coding session in early 2026. I used my coworker Jess Szmajda’s Linked Intent Development skill and AI coding clicked for me. LID¹ is a multi-phase Spec-Driven-Development system. You write the high-level description for the work you need to do in Markdown. Then you and your agent refine the work into high-level and low-level design documents. The agent slices the low-level design into EARS Specs, then TDD’s each spec. I worked on a toy exercise for an afternoon and felt productive.

Iterating

After a few more cycles on a couple of different projects and I wanted something different than LID. I felt like I was spending a lot of time in design documents. I like that the agent keeps all the levels of documents connected. But I prefer emergent design - I want the tests to tell me what the design should be. I caught my agent, more than once, writing entire classes in the low-level-design doc. How can you TDD if the code was already written?

It was time to write my own skill.

What I Wanted

My primary goal was to build a skill that felt as familiar to me and other pivots as possible. How did we work? How would this adapt to having an agent? I knew I wanted:

Real TDD. Red-Green-Refactor and no writing of any implementation without a test. Tests need to fail for the expected reason.
Small feedback loops. The agents should be writing small parts of the solution at a time. One function, or maybe one class at a time. Definitely not whole UI flows before a developer reviewed the work.
Idea-to-task flow that felt familiar. I wanted the workflow to feel like a Pivotal Iteration Planning Meeting (IPM). You start with an Epic or story and then work down to dev task tasks and TDD.
Works for one person. That is, I’m going to be the Product Manager and the developer at different times. The agent had to keep us honest regardless of the hat I was wearing. We should have enough tracking documentation to allow for interrupted work. Side projects start and stop. API limits can mean pausing for a few hours.

What I Didn’t Want

I had to avoid the sneaky “write the code first” behavior. I wanted to keep things simple, so I wanted to avoid any multi-agent orchestration. One at a agent at a time was fine for now. Maybe forever. And I wanted to avoid the long-running agent problem. I didn’t want to have to review pages and pages of code to see if it met the need and avoided hallucinations.

Duh

I wanted a pair. Both when I was in Product Manager mode and especially when I was coding. Claude, want to pair on this?

Introducing the Pivotal XP Skill

AAnd here is the result - PXP. It is agentic skill, proven to work well with Claude, that takes you through an entire feature from IPM to TDD. Together, you and your agent write and refine documentation in your repo that gives an SDD flow. It strives to feel like you are working from a Pivotal Tracker Backlog.

Epics - Documents that describe each batch of work - one shippable feature or set of bug fixes. It tells you what is in and out of scope for the work. You write this and refine it with your agent pair.
Stories - Your agent then slices the epic into actionable stories. Each story has a set of Gherkin/GWT Acceptance Criteria. The Epic maintains links to each of its stories.
EARS-based Tasks - Each Acceptance Criteria gets its own named, spec-driven task. These are broken down into an implementation order in the story doc.
TDD / Red-Green-Refactor - Each EARS gets TDD’d one at a time. The agent annotates tests and implementsion, with the EARS name in comments.

I’ve been using and iterating on this skill for the past 2-or-3 months. We’ve written a lot of code. Hallucinations have been low. I tend to commit as each story is complete - though bigger stories have had some WIP commits at a EARS/task boundary. I am very much in the loop on every step of the way and still feel crazy productive. I’m happy with the code results. Refactoring has gone well. I’ve had similar success with Ruby, Rails, and JavaScript (and Google AppsScript) projects. I also dipped my toes into an iOS project, where I know next to nothing.

I completed a minor redesign of this blog using PXP. It included a complex CSS simplification refactor pass. You’re soaking in it.

Claude did have a moment where it really wanted to have table-driven tests. Um, nope. Stop that. And it did.

The token count is not low, but the README has some tips on model choice to help manage your API usage.

It’s available via GitHub right now. I’ve submitted it as a marketplace plugin. I hope it will get approved this week. Kick the tires and let me know what you think!

I get partial credit for the name ↩

On Interviewing and Hiring Software Engineers

2025-01-10T00:00:00+00:00

I didn’t understand why one of the hiring managers on my staff had turned down the candidate.

The interviewee had 10+ years of experience with Ruby on Rails. He was a referral from a recent hire, who was already a strong contributor. The candidate had worked on interesting projects. At his most recent, quite large, employer, he’d been promoted over the past few years from a senior engineer all the way to a senior manager, taking care of multiple teams. If he could code well, I suspected he was going to be a good fit for the team. Why was making an offer not the obvious next step?

Hiring is hard. I often say that the only thing harder than hiring is firing. But that’s another post.

There should be reluctance when you see risks with a candidate. So how do you minimize those risks when interviewing?

First, let’s acknowledge that today your candidate funnel is clogged¹. There are enough people looking for software engineering roles that as a hiring manager you no doubt have several good resumés and you’ve only reviewed a subset of your queue.

So what does your interview cycle look like?

Hiring Today

Others are writing about the problems with interviewing, so I won’t go too deep here. But I think these are the most important anti-patterns to avoid.

L33tCode as a Filter

Why put a candidate through the gauntlet of mystical computer science problems that are known to be “hard”? When so much programming information and/or AI coding assistants are free, what are you learning about the candidate from “good” solutions to these problems? Many of these exercises use problems that your developers will never run into on the job. After all, Linked Lists Aren’t Real.

Ignoring Referrals

We used to say, “the front door is a brick wall,” and so as an applicant you would find any way to get an employee to submit your resume for a job. For recruiters, referrals were a welcome blessing. Referrals got at least a short phone call. Real experiences about the candidate from existing employees have enormous value.

Today, it seems like a referral might get your application closer to the top of the stack, but doesn’t guarantee any contact. In the past year, I’ve had my referred application auto-rejected or worse, flat-out ghosted. Don’t rest on a candidate who comes highly recommended.

Over-valuing Complex Problems and Solutions

YAGRII (You Ain’t Gonna Run Into It) might as well be a corollary to YAGNI. Measuring one person on how well they can design a massively distributed, micro-serviced-out-the-yin-yang, multi-cloud solution in an hour is unfair to everyone involved. Your product is very, very unlikely to run into that as relevant situation. Why are you evaluating on them?

If your product does run into such a scaling problem, you’re deep in the Expand phase of 3X and won’t have enough capacity to interview a new person to solve that problem for you. If you are at YouTube or Twitter or Facebook, you’ve already solved these problems and have entirely new ones.

What are the Desired Outcomes?

So interview cycles are a mess. How can you change yours to avoid these traps? Let’s take some pointers from Product. What is the desired outcome of a hiring process?

I contend that the outcome you want is a new team member who’s going to increase the team’s ability to execute. That’s a pretty broad statement. To meet this outcome, the measurable characteristics and behaviors I look for are:

Ability to code and reason about code.
Ability to work with other people.
Knowledge acquisition skills.

Screen for Coding Skills

Yes, you do need to have candidates code during an interview. I’ve personally screened candidates with good looking web development resumés with JavaScript projects that couldn’t write a function that would even run.

Pick an interesting, but not architecturally puzzling, problem. Pick a language and domain that’s easy to work in and with over a video call. Have your screeners pair with the candidate on the exercise. Keep a strict time limit. Don’t expect or measure solutions. Instead measure if they can code, ask good questions, and then talk about why they suggested or wrote the code they did.

Stick with this setup. Keep it consistent across candidates. Keep your screener pool small. Strive to make the results more comparable and thus help you compare candidates more fairly.

Live Work Experience on a Real Problem with Others

If a candidate passes the screen, you want to evaluate them with other team members on a problem that’s closer to your real work. You want to see how the candidate will work in an environment closer to the team’s day-to-day.

This could be a story from the team’s backlog - but that could be inconsistent and might lead to IP risks. More practically, build a sample project that is using (some of) the stack your team uses. Pick the language and framework they use every day. The work should feel like a story from the backlog, even if the domain is not exactly the same as your product work. Have there be an obvious end point with real results.

Have them work with another team member on this exercise - just as if they were pairing on normal work. Let the candidate lead on the solutions while they are being guided on context and style with the interviewer. This should be longer than the screen, but still time limited. This exercise is less about coding, and more about how well they work with others. You want to evaluate them working as part of the team. Software product development remains a team sport.

Knowledge Acquisition

My key thesis with interviewing is that you should, nearly above all else, be hiring for knowledge acquisition skills. Software development is dynamic; new problems or context show up all the time. Information about your language, dependencies, and frameworks is all readily available. How are they building on their knowledge to make themselves and the team better? How well can they work with others to figure out to even start solving a seemingly random, new problem? How are they going to learn when an unexpected complication hits the business (which is inevitable) and increase the overall knowledge of the team?

If you hire people that are continuously learning, and also continuously teaching², you’re hiring force multipliers onto your team.

Iterate

From here? Iterate. Tweak metrics. Train your interviewers. Do what works and toss the rest. Adjust the process until you get something that’s repeatable in execution and consistent in measurement. Do this and your team will get stronger. And stronger.

Back to the Candidate

We didn’t have a candidate problem. We had an interviewing problem. I worked over a few months to rebuild our interviewing rounds along the lines of I’ve mentioned above. Over time, with some measuring, we interviewed more consistently, increased our hire/offer ratio, and built stronger teams overall.

Meanwhile, we had open reqs and plenty of work to do. I got back to the candidate and did a tech screen. He did well. We talked more about the company and our needs. He had the relevant experience. He talked about his goals. He was looking to get closer to execution, wanting to code as an individual contributor for a while before considering managing again. The process said he was going to be a good hire.

I hired him. He did hit the ground running and didn’t stop. He referred a couple of other engineers and we hired one of them. He moved back up into management within a year, led one of the more productive teams, and led a strategic technical spike.

Reader, he was a good hire.

This too, is another post. But for now, it’s clear that available AI resume and application tooling is flooding application tracking systems and the poor recruiters stuck using them. It’s not just you. ↩
Yup. Yet another post. ↩

Synology on Rails Part VI: An Improved Docker Registry

2024-09-25T00:00:00+00:00

After a few months of pushing container images to my home grown registry, I was noticing that my server was filling up. I wasn’t worried about running out of space soon, but what about a few months from now?

There are a handful of UI projects out there that run on top of the Docker Registry and provide UI to allow management of old images, retention policies, etc. Just what I needed.

I looked into deploying a few of them and ran into various complexities. Did I have all the dependencies? I think it’s running? But I’m getting errors. I was ready to throw up my hands.

A New Project has Entered the Chat

I found a project that unified the default Docker Registry image and the Registry UI project from Quiq into one docker-compose.yml. This looked promising.

My only concern was that the last commit was 4 years ago. And then I couldn’t get it all running correctly in the latest Synology DSM, version 7.x. I lost about a half of a day figuring out what was wrong.

A DSM 7.x Solution

But after that work, I was able to bring this project up-to-date in a fork. The README has more detailed instructions. Check it out!

The full details are in the README, but from a high level:

One Container Manager Project that downloads and boots both the Registry and the UI in their own containers
A small workaround (an extra reverse proxy) due to a Web Station limitation
All the management is done through the Container Manager UI (just like your apps you’ve been pushing)

I’ve submitted a PR to the original project, but given the age, I’ll keep an eye on my fork for feedback.

Synology on Rails Part V: Automate the Rest

2024-09-06T00:00:00+00:00

When last we left our intrepid heroes, the final couple of steps of the Rails app deployment to a Synology NAS were still manual. It still took mouse clicks in the Container Manager to pull the image from the registry and rebuild the Project.

Good News, Everyone!

Pivot Luke Winikates has put together the SynoCtl project which can drive the Synology DSM via its own API. He snooped the http traffic from the web UI, checked out some other “let’s automate DSM” projects (the project page has some good references), and got to work. The result is a Golang SDK and an executable that I was able to use in my rake tasks to automate the remaining deploy steps.

What’s New?

I’ve updated the rake tasks project to use synoctl. There are now rake tasks for pulling the image into the Container Manager Project and rebuilding/restarting the Project.

Synoctl requires more configuration values than I had, which made the tasks more messy. Thanks to some inspiration from Bradley Schaefer, I moved to an RSpec-like configuration in a separate file.

Lastly, I added a namespace for the tasks. Which meant a bit more explanation in the README. And,yes, I’ve already moved another app to using this file.

Now with one rake command and a few minutes, we can deploy a new version of a Rails app to the Synology Container Manager and get it restarted. This is a happy place to be.

ps: Big thanks to Luke and Bradley!

That Time I Interviewed Pat Gelsinger for a Bit

2024-08-03T22:01:00+00:00

“You think that Pat Gelsinger looks like Wayne Gretzky?!!”

Pat Gelsinger, somewhat famously, looks a bit like Bill Gates. And he also looks somewhat like a local Portland auto dealer, Scott Thomason, who is on billboards all around town. But to obey the comedy rule of threes we needed a third.

“He looks a little like Gretzky. We need three - we’ll use him first. By the time we get to the end, nobody will question it.”

It’s 1994. Portland, Oregon. Intel is merging three or four teams together, uniting efforts to commercialize video conferencing products under one leader. Pat Gelsinger is that leader.

To “celebrate” - read: unify - this new group of about 500 people, there was going to be a picnic in downtown Portland, Oregon, and a “talent” show. Any team could enter a skit. The winning team, by audience vote, would get an extra vacation day.

Thirty years ago, before Non-Accrued Vacation became an accounting-friendly norm, Intel counted every, single, vacation day. So this was huge. And a few of us thought ourselves very funny.

So here we are, in a conference room over lunch, pitching ideas to each other, looking for a skit idea that would kill. All for an extra Friday off.

We have our number three. What’s the comedic situation?

We settled on an on-stage interview with Pat with some planted audience questions. I volunteered to be the interviewer, Letterman-style. A quick email to Pat’s admin and we had his participation committed.

“Pat’s in. Everyone have their props list? Are we good? Let’s run through it one more time.”

The Similarity

A couple of weeks later, we’re in an auditorium in downtown. Five-hundred or so folks, full of a barbecue lunch and a couple of beers, are in the audience.

We meet Pat backstage and he says he’s ready; that our script makes sense. We come out on stage. It went a little something like this…

The Roughly Remembered Script

Two comfortable chairs downstage, turned slightly towards each other. Davis and Pat walk on stage. Davis, with a clipboard, sits stage left. Pat sits stage right.

Davis: Welcome, everyone, to this fireside chat with our new General Manager, Pat Gelsinger. Welcome, Pat.

Pat: Thanks. I thrilled to be here so that everyone can get to know each other so we’re ready to make great products together.

Davis: Awesome. Does anyone want to ask Pat a question?

A few people standup. One is wearing a hockey jersey.

Davis: You, there, in the LA Kings jersey.

Ken: Mr. Gretzky! Mr. Gretzky! I’m a huge fan going back to Edmonton. You were great this year, but sorry the Kings didn’t make it further. (Ken points to the LA Kings logo on his chest) Will you sign my jersey?

Confused mumbling and maybe a snicker from the crowd.

Pat: Um. I think you’re confused. I’m Pat Gelsinger, head of this new Intel Division.

Davis: That was weird. Let’s take another question. You on the second row.

Blake: Mr. Gates! Mr. Gates! I’m a huge fan. (Blake raises a software box) Will you sign my copy of Microsoft C/C++ version 1.0?

Solid laughter as the crowd gets where this is going.

Pat: I’m not Bill. I’m Pat. Your new GM.

Davis: Odd. But that’s an honest mistake. We have time for one more question. You over on the right…

Third Questioner, whose name I can’t remember: Mr. Thomason! Mr. Thomason! What is the best deal you can give me on a new Camry sedan?

The crowd loses it, raucous laughter. They love the punchline.

Pat: (Pat stands, walks to center stage, and acts a little irritated, then ends enthusiastically) People. I’m not Wayne Gretzky. I’m not Bill Gates. And I’m not Scott Thomason. I’m Pat. Gelsinger. And this is PCEG. And we’re going to make video conferencing happen on Intel PCs!!

Crowd howls and applauds. Davis and Pat shake hands, leave the stage triumphant.

The Aftermath

We totally killed! The crowd loved us! We had the best reaction of the day. Instead of hinging on inside-to-the-team jokes that got a dozen laughs, we had a premise that was for the whole audience. We unified the new org under a now-more-familiar leader. We were ready to call caterers for a team party on our vacation day.

But like all tragic stories, there was one skit left to go.

I wish I could tell you what the last skit was about. But it had dudes wearing dresses, Monty-Python-drag style, with one of them wielding an electric chainsaw.

We never had a chance.

Interestings for Iterators Vol I, #7

2024-06-14T10:00:00+00:00

I’m always looking for feedback about these posts. Ping me on Mastodon or send an email (see About Me).

Ruby Links

SF Ruby Meetup I gave a Ruby-focused version of my tech talk, Three Cheers for Release Engineering, this week at the SF Ruby Meetup. This meetup is now monthly and regaining steam - there are a lot of interesting Ruby teams here! If you’re a Rubyist in the SF Bay Area, please join the Luma event for July and propose a talk. See you next month!

Ruby on Rails Jobs Weekly Mark Moura is making this weekly post of Ruby-focused job listings to LinkedIn’s Rubyists group. It includes remote and remote-friendly jobs. I’ve found an interesting correlation between Iterator-friendly teams and those who use Ruby and Rails as tech choices. So this makes for a good list of companies to research.

RubyConf 2024 is Chicago, IL, November 11-13. The CFP is now open, with help for first-time CFPers/presenters. Go submit a talk!

Tools

MarkMap When I’m dumping my brain using outlines, I sometimes want a mind map view. The MarkMap project is a renderer that can take any Markdown doc and give a tree view. There’s a REPL at their site, as well as a VSCode extension LINK. I’m an Obsidian power user, but I found a split screen (Obs on the left, VSCode/MarkMap on the right) useful a few times in the past week.

Agilytics Darryl Snow, a pivot in Singapore, has made a JIRA extension to approximate some of Pivotal Tracker’s analytics and other features. It’s early, but if you’re in the “I miss Tracker but live in JIRA” camp take a look and kick the stories - I mean tires.

Random

How to Cut an Onion J. Kenji Lopez-Alt is my favorite cooking geek out there. He’s talked about this technique for years on his YouTube channel. This article has the best explanation yet, complete with diagrams and a little math.

Wrapping up this week with smiles from two geek-comedy bits for you. Click for smiles and groans.

Interestings for Iterators Vol I, #6

2024-06-07T19:56:00+00:00

Test-Driving HTML Templates - Martin Fowler tooted about this article which shares a model for writing HTML rendering tests, including examples in Go and Java. I’ve leaned away from view tests for years and I tend to be very wary of parameterized tests. But I think this is worth a read to digest the techniques and see if/how they are worth applying in my own work.

How Structures Affect Outcomes: Insights from Software Engineering Leader Elisabeth Hendrickson - In my queue to listen to this week, because Elisabeth.

How I Saved 100GB of Space on My Mac - You thought you only needed a 512GB SSD on that new MacBook. Then iCloud synced 100GB of Messages and Photos. This is a good toolkit for finding a lot of developer cruft hiding on your system. I’ll add that Daisy Disk is a MacOS app I’ve used for years that’s similar to one of the tools mentioned in this post.

A few years ago, thanks to Mike Dalessio, I started using Excalidraw for all my vector graphic needs. About the same time, I found Mermaid, which is a set of DSLs to make common types of charts and diagrams. The universe has again given us two great tastes that taste great together¹: Excalidraw now can import and render Mermaid charts. Now you can draft up a chart in Mermaid text, get to a “logically correct” diagram, then paste in Excalidraw to make it visually what you want (or just export it for inclusion elsewhere - h/t to RJ Zaworski ). Play with it here.

Squeezing the Feedback Loop - Speaking of RJ, it’s always great when folks outside of my direct Pivotal influence talk about shrinking feedback loops.

The Cloud Under the Sea - The Verge published this in April, but I finally got to reading it this past week. It’s an amazing story that merges the history of undersea cables with what crews are doing today for maintaining intercontinental fiber connectivity (make sure to read in-browser as it’s full of neat images and scrolling techniques). It reminded me of Mother Earth, Mother Board, Neal Stephenson’s epic article from 1996(!) tracing what was then the longest fiber cable project. These articles make a great pair.

Lastly, from the deep archives of the internet (check out the ~ in the URL), I stumbled across Donald Knuth telling the world he doesn’t have time for their email. Come for the etymology (dropping the hyphen from e-mail) and stick around for the reminder to claim your focus and attention from the world so you can do your chosen, important work.

Submitted without commentary. ↩

Interestings for Iterators Vol I, #5

2024-05-31T19:56:00+00:00

Codex Updates

A couple of updates on the Agility Resources/Practices page:

I can confirm that Broadcom dropping their Miro subscription has not affected the Tanzu/Pivotal Labs contributions to the Miroverse. This solid set of great starter boards for various sessions live here happily.

New to me is the the Tanzu Library board: a list of recommended books, with category tags, and room for short reviews. This would be great for a team’s book club.

Also, Ben Christel has added his Automated Project Management presentation, which shares how he talks about and uses Pivotal Tracker.

In The Wild

Planet Money has a podcast episode about Open Source software, told with the framing story of the XZ hack from earlier this year. If you’re reading this, the podcast episode won’t give you much new - though I didn’t know about the project mentioned as the “first” OSS project. I expect it will be a good primer on OSS for anyone you know who’s not in the software business.

And if you’re sending this podcast to one of those folks, here’s the referenced XKCD “Jenga” comic, titled, appropriately, Dependency. Don’t miss the hover!

Interestings for Iterators Vol I, #4

2024-05-17T19:22:00+00:00

I was “off” last week as I was at RailsConf 2024 in Detroit with several hundred other Rubyists on Rails. It was a fun and interesting time. I thought the program committee did an exceptional job this year with the technical topics and inspirational keynotes. Having a day that was just workshops and OSS project hacking was genius. I hope to see this continue at RubyConf and other conferences in the future.

Attendees have been posting their reflections all week - check ruby.social - and the talks should be up in a month or so. I’ll be linking to my favorites then.

Great job, team! And thanks!

Codex Updates and Highlights

Not much new added to the Pivotal Alumni Codex in the past couple of weeks. Instead, here’s a highlight!

This week I’m going to point to the Labs Practices site, and directory of open-sourced-resources that Pivotal Labs, and VMware Tanzu Labs gathered over the years for our clients. This is worth bookmarking for facilitation guides, technique descriptions, and learning paths.

In The Wild

How Slack Automates Deploys H/T to Josh Aresty for pointing me at this. It’s a good story about how Slack went from a very manual to a very automated deploy system. This felt very familiar to my recent work, yet I’ve got a few new techniques to try when advising a release team again.

BackgroundMusic for MacOS A new-to-me, free homebrew-installable, audio utility. Best reason to install it: the virtual microphone that means you can use ScreenRecorder to get system audio in your video. Best reasons to keep it: independent, relative volume controls for open apps and auto-muting of music when you move to a video call.

20 Things I’ve Learned as a Systems (Over) Thinker by John Cutler. Another winner from John. As I said when I re-shared this on LinkedIn - “It me.”

A Radical Enterprise Quick Start Guide for Business Leaders A good jumping off point before you dive into Matt Parker’s book A Radical Enterprise. Which reminds me a bit of…

Doing Agile Right by Darrell Rigby, Sarah Elk , and Steve Berez. When this book came out, I attended a Q&A with one of the authors and was prepared to be constructively confrontational. My experience with management consultants and “Agile Transformations” was…not great. But what I learned from Sarah’s answers, and this book - much like Matt Parker’s book above - is that the Agile Software Manifesto has reflections everywhere. And it’s a systems-thinking mindset, a desire to shrink feedback loops, and reminder that you’re working with people that can help any project move faster with less waste.

Better Together! Jason Fraser posted about Move Fast and how it’s a solid companion to his and Janice Fraser’s book [Farther Faster]. I’m adding Move Fast to my queue.

Let me know if you’re enjoying these. Feedback is much appreciated. My social links are above.

Interestings for Iterators Vol I, #3

2024-05-03T00:00:00+00:00

Editor’s Note: I’m going to be in Detroit at RailsConf the week of May 6th. If you’re going to be there, let me know.

Codex Updates

Last post, I mentioned the creation of the Pivotal Alumni Codex. Even if you’re not a Pivotal alum, it’s still a good resource for all iterators. Take a look. Pull requests welcome.

When there are interesting updates to the Codex, I will make sure to call them out here.
This week, we added a Newsletters section, with sections pivot-authored newsletters, as well as others you might find helpful.

In The Wild

Finding Your Future - Session 3 Jason Fraser is running another workshop to help you find your next job on Monday, May 13th. This will be especially helpful if you’re not sure about that next career step. Sign up at the link.

What is leadership? - With David Marquet This is a 10 minute sketched animation of David giving the highest-level summary of Intent-Based Leadership, the core principle he covers in one of my favorite business books, Turn This Ship Around. He posted this to Threads this week, mentioning that he traded his keynote speech fee for this animation. I agree that it was totally worth the exchange. This is a both a great primer and a quick refresh.

Why Aren’t We Talking About Continuous Improvement? John Cutler’s post this week helped me pause and think about why it’s so hard to get teams and orgs to even talk hypothetically about improving how they work. He has hope. So should we all.

Mik Freedman on the WhatTheDev? Podcast Mik talks about detecting Agile anti-patterns. It’s short, and full of very familiar concepts. Hearing them again, and more of a “whole org” context, it provides a bit of that hope that John Cutler mentions.

Synology on Rails Part IV: Simplifying Deploys

2024-05-01T00:00:00+00:00

Now that we have a local container registry running on the Synology NAS, how do we use it? We want to push updated containerized app builds to this registry, as well as set up the Synology Container Manager for low-fuss deploys.

Pushing to the Registry

Pushing is simple. Following the Docker instructions, there are two small steps - first is tagging the image with the registry URL. Then pushing the new tagged image:

$ docker tag dwfrank/meals registry.local/dwfrank/meals
$ docker push registry.local/dwfrank/meals

Recall from the previous post that I have local DNS set up for registry.local. If you’re following along and don’t have the ability to use local DNS, an IP address + port will work.

To verify that the push worked, I did the following:

Go to the Container Manager / Registry pane
Click the Settings button and to see my local registry in the list
Click the local registry row and then click the Use button; it now had a green checkmark
Close this dialog

And there was dwfrank/meals in the local registry. Boom.

Moving to Docker Compose

In earlier exploration, I had some problems using docker-compose.yml to configure my app. But since I got the Container Registry working, I understand Synology’s terminology better. And it turns out that using a Container Manager Project w/ a docker-compose.yml makes updates very simple.

1. Delete the Old Versions of the app

I deleted the current image, container (both of these in Container Manager), and the app’s Web Portal (in WebStation).I’m not going to be using them any more.

2. Make the docker-compose File

Next I made a docker-compose.yml file in my app’s root directory (so it was in git):

version: "3.5"  
services:  
  web:  
    image: registry.local/dwfrank/meals  
    container_name: balboa-meals
    network_mode: bridge  
    volumes:  
      - /volume1/docker/meals/db:/rails/storage  
    ports:  
      - 3000:3000  
    command: bash -c "rm -f tmp/pids/server.pid && ./bin/rails server"

And then I copied this to the app’s NAS directory at /Volumes/docker/meals on my desktop, where it’s mounted from /volume1/docker/meals.

3. Create a Container Manager project

Again, just like the registry, I created a new Container Manager Project and pointed it at /volume1/docker/meals. It found the docker-compose.yml, then downloaded the image from the local registry, made the container, and then launched WebStation’s “create a Web Portal” dialog.

I gave it the local URL as a name as before. And the app came right up.

Deploying to Tie It All Together

The full deploy cycle is now very simple.

From my desktop (using my deploy tasks linked above):

$ bundle exec update_version deploy

Then on the Synology¹:

Go to the Container Manager / Registry pane
Double click on the image I want to update
1. From the dialog, choose the “latest” tag and hit Apply; this downloads the updated image
2. Wait for the download to complete
Container Manager / Project Pane
1. Click the Project
2. From the Actions button, choose Stop; wait for the dialog to say it’s complete
3. From the Actions button, choose Build; wait for the dialog to say it’s complete

And that’s it. Far less clunky and error prone. And all of the configuration, thanks to the docker-compose.yml file is in source control.

In case it helps your projects, I’ve extracted my deploy tasks to this repo on GitHub for easy inclusion in other Rails apps. These are not Synology-specific, and tie together all the knowledge from this series.

This guide (Internet Archive version) has a good step-by-step. However, Container Manager doesn’t show which projects have images with updates in the local registry. But that’s not bothering me for now. ↩

Synology on Rails Part III: Improve Deploying with a Local Registry

2024-04-28T00:00:00+00:00

2024.09.25 UPDATE

This article is superseded by Part VI, which starts up a registry and UI for that registry. This article is mostly useful for context.

Well, my app is deployed and running. But after several months of continued development and maintenance, I’m not happy with how I deploy this app. The required manual steps in the Synology DSM UI have proven error prone. I often forget what the steps are and have to click around until it’s working. How might we improve deploying my containerized Rails app?

After talking to others and researching some more, I have two things to try in order to make deploying a container image to my Synology NAS.

A Local Registry

If you recall the previous posts, getting the image from my desktop to the NAS has these steps:

Export the image from Docker into a tarfile
Copy the tarfile to the NAS via the Terminal
Delete the tarfile.

That has always felt clunky.

I’m not the only one that thinks there’s no need to copy a 500GB image up to the Internet, only to copy it back down again. There’s consensus around running a local container registry, in a container, on the Synology NAS. Why not? This would drop the export and delete steps from my current workflow. And is makes more sense when deploying via containers.

Good news, everyone! This is super easy. Docker Hub hosts a registry image that nearly drops right in.

1. Make a new shared directory

I made a new directory on the NAS at /volume1/docker/registry. I used this folder because I’m already sharing /volume1/docker to my network. All my deployable apps have their mount points there. On my Mac, this mounts as /Volumes/docker.

2. Make a new docker-compose file

On my desktop I created this file: /Volumes/docker/registry/docker-compose.yml - and filled it with this:

version: "3.5"
services:
  registry:
    image: registry
    container_name: registry
    network_mode: bridge
    volumes:
      # map your volume on the left as makes sense for you
      - /volume1/docker/registry/:/var/lib/registry
    ports:
      # important b/c Synology runs DSM on port 5000
      - 5050:5000
    restart: unless-stopped

The first key thing here is to map the volumes correctly - the left-hand size of the colon needs to be the Synology path to the directory I created in the first step. The Docker registry will see this at its /var/lib/registry and create files there.

The second is that I needed to map the port. The registry container is listening on 5000. But Synology uses 5000 for the DSM web interface. So I chose 5050, which does not conflict.

3. Create the Container Manager Project

In Container Manger, I made a new Project. I called it “registry”. I used the path from first step. Container Manager found the docker-compose.yml and asked to use it.

It downloaded the registry image from Docker Hub, created the container, and started it. Container Manager calls this “building” the project.

I got a Container Manager dialog that had a checkbox to “allow insecure registry.” I said yes. I’m not opening this registry to the Internet, so I’m fine with this.

4. Create the Web Portal

Next, Container Manager brings up the Web Station the dialog for creating the Web Portal for the registry. I gave this a domain name of registry.local.

I am able to create custom DNS “records” on my home firewall/router (shout out to Firewalla), so I set registry.local to map to my Synology NAS’s IP address - which is a reserved address in my DHCP server.

5. Run and Test

Then I visited http://registry.local/v2/ and got a response of {}. So, yay! A registry!

Next up is using this registry at deploy time.

Interestings for Iterators Vol I, #2

2024-04-26T00:00:00+00:00

Welcome to week 2!

The Pivotal Alumni Codex Pivotal (former) employees have had a free Alumni Slack for several years now. Free Slack instances have a two key problems. First is the 90-day history which is makes the second problem, a middling search capability, even worse. We’ve solved that with the Alumni Codex, a collection of Pivotal History, tools, and techniques. This is a public website and if you’re part of the Iterators community, have a look. Note - we do take pull requests.

Radio Free XP This podcast, hosted by Pivots Tony Hansmann and Jesse T. Alford, is a set of interviews mostly about pair programming and Pivotal history. Add this to your favorite podcast app and get listening.

You Probably Don’t Need Media Queries Anymore I started with the question of “What are the Media Queries I should be using to target different phone sizes?” and got this recommendation (from the Pivotal Alumni Slack). There are some great tips here about some CSS features I hadn’t kept up with: new units (fr, vh, and vw), interesting functions (calc() and clamp()) , and a mindset that we can think of things more continuously instead of with constantly shifting size breakpoints. I solved three problems in about an hour with these insights. Though I’m not convinced about display: grid just yet. I’m still very much #teamflexbox.

NASA’s Voyager 1 Resumes Sending Engineering Updates to Earth This news was all around this week. But this official telling at NASA hints at what must’ve been some very impressive code reorganization and refactoring in order to use what’s still working on that craft. Forget the immense latency and feedback loops for a moment and think about moving 50-year-old+ machine code to memory that’s working. Mind blowing, but essential work to ensure we get to V’ger.

SQLite on Rails: The how and why of optimal performance The performance improvements keep coming to Ruby and Rails. Come for the textbook discussion of a performance problem, along with data and graphs. Leave with a one-liner to improve the number of continuous connections your Rails-app-using-SQLite can have to its database.

Subscribing?

I got some feedback this week along the lines of “if I can’t subscribe to this ‘newsletter’ how am I supposed to read it?” If this takes off, I will look into making this email subscribe-able. But until then, RSS is your best bet. Here is a list of readers that folks are using (RIP Google Reader):

Thanks! Feedback welcome

Phew, that’s two! Feedback welcome (see my about page)!

Interestings for Iterators Vol I, #1

2024-04-19T00:00:00+00:00

First, this is a newsletter experiment. My sourdough starter is long dead and my exercise routine is solid. So that means it’s time for an attempt at a newsletter. This experiment won’t have a subscription feature just yet. But if this gets some traction, I’ll move in that direction.

Second, this will be a “link dump” style series. I’ll keep the rants and ramblings as normal, maybe with a link to them here. This series’ contents will be things that I would bring up during the Interestings section of our Standup.

Last, I’ve been looking for a term to use for those of us in the Software Products domain who believe in short feedback loops and improving our practices based on evidence. At Pivotal this started with Extreme Programming, and we added Lean Startup, then Balanced Team. But Pivotal is now part of (a wonderful and challenging) history. And these principles and practices are not exclusive to Pivotal (or its customers, clients, or follow-on entiries).

How might we refer to ourselves, then?

I propose Iterators.

Let’s see if it sticks.

With that aside, here are my Interestings for Iterators from the past week:

Essays & Posts

Jason Fraser wrote My Job Hunt Strategy about re-thinking his job search using the principles from Richard Rumelt’s Good Strategy/Bad Strategy. I found this a refreshing take on how to approach a problem all too many of us have right now.

Tools

Window Managers for Apple Silicon

If you’re a fan of keyboard-based window managers so that you can shortcut your way to moving all of your windows around, you may be sad. The Apple Silicon transition has rendered some of these tools useless. This comes up quarterly on the Pivotal Alumni Slack, so I’m sharing some of the solutions:

ShiftIt’s - one of the moribund solutions - recommends to use Hammerspoon and a script
Onsi Fakhouri forked a fork to give us Mosaic
Phoenix is another one
Personally, I was a big fan of Slate (untoched for 12 years!). Fertigt forked and recompiled it for arm-64 at Slate. There’s even an homebrew cask for it.
- And here’s my config file for Slate

Experimentation Cheat Sheet & Template

John Cutler’s done it again with a straightforward experiment template. I’ve already used this twice this week!

Software Development

Yet Another CSS Reset & Starter

There are so very many of these out there. I’m a big fan of Holiday.css. Our fam at Initial Capacity have an HTML/CSS reset project called CSS Starter. It’s not just a great set of CSS defaults, it also has links to key CSS education and a set of patterns on how to grow your CSS sustainably and maintainably over time.

My favorite SOLID talk

When complex software problems come up, I almost always refer folks to the talk SOLID in the Wild from SpringOne Platform 2017. Pivots David, Mike, and Zoe knock out of the park with a real-world example utilizing SOLID principles to make software adapt. It’s very much in the spirit of Kent Beck’s “make the hard change easy, then make the easy change.” I shared this again this week and it saved someone about a week’s worth of work on a refactor.

Feedback

Thanks for reading! And thanks to Nat Bennett for the inspiration. Let me know what you think about this (see my about page).

Why Do We Miss Pivotal Tracker?

2024-03-06T00:00:00+00:00

UPDATE: Some folks have started StoryTime, a forum to talk about the transition away from Pivotal Tracker. It’s free to join.
UPDATE UPDATE: Pivotal Tracker is gone. Pour out a few story points for stories long gone.
—
Among my favorite stories about the “early” days of Pivotal Tracker was the argument of whether or not Bugs and Chores should have (story) points. Tracker originally said no.

Parker Thompson had a different take. At the time, his client thought in terms of capacity planning - not velocity volatility - and so he wanted points, dammit. After weeks of this being a persistent breakfast and lunch topic, he escalated to Rob Mee. Which led to a semi-intense argument.

Pivotal Tracker ~is about to go away for all but Broadcom Enterprise customers~ has gone away for all customers. Many of us who love Tracker ~are~ were still using it for personal projects. A few companies ~are~ were still using it for shipping products - how awesome is that? So now we pivots are being reflective about Tracker.

So why do I miss Pivotal Tracker so much? I boiled it down to two-ish things.

Small and Opinionated

Pivotal Tracker has a tight UI with lots of opinions. It’s only one tab (per project), stories open in-line (so it’s harder to get lost), and even today the fields on a story are not overly complex or rich.

There are only five types of stories, instead relying on tags (Tracker calls them Labels) to categorize lightly. When Epics were added, they were orthogonal - essentially labels that had the same fields as a story. Stuff you want to deal with later - just throw it in the Icebox. And when the icebox got too full and freezer rot happened, it was easy to purge.

All of these choices make it very easy to use 80% of the time. IPMs - Iteration Planning Meetings - stay very fluid. Developers picking up stories knew what to do - pick the next story, open, re-read, ask everyone for clarifications, document anything new, and then start work. Product Managers knew what stories they could accept. Anyone with project context could figure out what was happening with a very quick glance.

Tracker has an opinion about your backlog. And your stories. And your bugs and chores. Not allowing points on bugs - features that used to work, but now don’t (this is important) - and chores - developer work that had to be done, but didn’t need PM acceptance (say, updating a certificate) - was intentional. If you’re completing lots of stories, but your velocity is dropping, that should drive multiple why questions:

PM: Our velocity is down. Why?
Devs: We had lots of bugs this week.
PM: Why?
Devs: We had to upgrade a dependency, which led to incompatibilities we didn’t anticipate. So. Bugs.
PM: Why?
Devs: Well, we were missing some tests. Which we have now backfilled.
PM: Great! Did we learn anything else for the next time?
Devs: A few things. We’re ready for next time.

Tracker is simple enough to teach a team the basics in an afternoon. As a team gets better at estimation and monitoring their velocity, Tracker reinforces those practices and helps everyone stay productive.

Plenty of Problems

Sure there are issues. Tracker still doesn’t handle interdependent projects well. It’s not smooth when, as a PM, you have lots of draft stories - I found myself working in other apps or keeping future epics “hidden” in another project until they are ready to be discussed at IPM. The integration with source control or CI systems is better than it was, but still can be clunky. There are other features and quirks that bother me and likely some that bothers you.

Backlogs, however, remain awesome. And that leads to Tracker’s absolute best feature.

Release Prediction

The most common question teams get is, “When is it going to be done?” Pivotal Tracker can not only answer that question, but it enables the necessary mitigations to keep a project to any date commitments.

Tracker has a type of story called a Release. Its form is the same as the others. But it has a target date (you read that right: an agile project planning tool with a date; stick with me here). You create a release, fill out the rest of the UI (description, etc.), add a date, and then drag the release down to just past the last story you want completed for that release.

Tracker cares about the project’s current expected weekly velocity and the estimations of all of the stories before the Release in the backlog. If the sum of all of the estimations says you are likely to get all the work done by that date, nothing changes. But if the velocity tells you that the work won’t be completed by that date the release marker turns red. Literally. The background goes from blue to crimson. You’re not going to make that date and you need to take action to get the Release de-risked.

Much like the velocity discussion, the team gets to have a release discussion. There are two choices¹ -

Cut scope.
Change the date.

Both involve work. Cutting scope means making product and technical choices. Changing the date means having hard discussions with management and customers. Likely, the team is going to use some mix of the two. But when you make decisions and update the backlog, Tracker will reward you by turning that Release’s background color from red back to blue.

Back to Bugs & Chores

In the end, Parker got his points for bugs and chores - one of the few customizations that Tracker allowed at the time. It’s project-specific. And yes it puts you into a capacity-planning mindset - which can work. And the above features and team responses still apply. But no, I don’t put points on bugs and chores.

Will Other Apps Ever Catch Up?

I’ve used other backlog tools: Linear, Shortcut, and Jira. Each has pros and cons. I’ve even worked in Jira backlogs that aren’t awful - just not as good at the above as Tracker. But none of these apps has “release prediction” functionality, reinforcing good velocity and estimation practices, anywhere close to Tracker’s.

With Tracker waltzing over the horizon, I wonder if we’ll ever see something close to its superpowers².

Sure, you can “work harder.” But that’s less predictable and not sustainable. Wenodis. ↩
I’d love to be wrong on this. Reach out you’ve found an app that does release prediction well, or know how to configure one of the mentioned apps to approximate this. ↩

Synology on Rails Part II: Deploying

2023-12-15T20:34:00+00:00

This is part II. Visit rest of this series to find out how we got here.

Now that I had a very manual set of steps to get my Meal Planning app to run on Synology DSM, I wanted a more automated set of deployment steps - something I could deploy with a single rake task.

What I Wanted

No DockerHub round tripping
- Why send 1GB up to the cloud just to come back down again?
Local image building
- MacOS/M2 build times were under 3 minutes vs. over 20+ minutes on the NAS
One rake task to do it all from clean git to new version of the app up and running

What I Had

Only a partial understanding of how the Synology GUI maps to Docker underneath
Zero clue if this was possible

What I Did

Let’s see what happened.

TL;DR Here’s a repo of the final files.

1. Get the Image to the NAS

Once I made the decision to not involve Docker Hub, or any registry, I figured this was going to be simple enough copy across my network, right? I had a folder on the NAS that was accessible.

I just needed to save the image out to a .tar file and then copy it to the server.

$ docker save dwfrank/meals > tmp/dwfrank-meals.tar
$ cp tmp/dwfrank-meals.tar /Volumes/docker/meals

There are two key things here:

I used docker save to get the built image from local Docker. Export also worked, but caused problems later.
I saved the tar file in ./tmp, which is in the .dockerignore file, which means it’s in the development file system, but not the container image.

2. Get the Container Loaded

The Synology Container Manager GUI will let you import an image in a tar file. That way was easy. But how could I do this from the command line?

I learned two things:

The Container Manager GUI is really just running docker under the hood.
The docker and docker-compose executables are installed, but only for root or sudo access.

I enabled the Synology’s SSH service, on an alternate port, and made sure my NAS’s deploy user had administrator rights. Then I set up a key pair so I could ssh without a password.

After some trial and error, I found that I needed to use Docker’s load command:

$ sudo docker load --input /volume1/docker/meals/dwfrank-meals.tar

…and that I had to have the full path to the image.¹

The image showed up in Container Manager as expected. Progress!

3. Injecting Values into the Dockerfile

I decided that the most appropriate place to add the RAILS_MASTER_KEY environment variable was when the image is built. It keeps all the environment vars together, after all.

So back to the Dockerfile and its env vars section:

ARG MASTER_KEY=""  
  
# Set production environment  
ENV RAILS_ENV="production" \  
    BUNDLE_DEPLOYMENT="1" \  
    BUNDLE_PATH="/usr/local/bundle" \  
    BUNDLE_WITHOUT="development" \  
    RAILS_MASTER_KEY=$MASTER_KEY

I added the empty Docker ARG for the master key, and then pass it in at build time.² This meant writing the first rake task that does any of this work so I could read the key from the Rails config directory:

key = File.read("config/master.key")  
system "docker buildx build --build-arg=\"MASTER_KEY=#{key}\" --platform linux/amd64 -t dwfrank/meals ."

I round-tripped the build again, and after running sudo docker load, the RAILS_MASTER_KEY was right there with the rest of the variables.

4. Adding the Database Volume Mount Point

It looked like docker-compose was the right way to add volumes. So I created a docker-compose.yml in my app’s root with the image name, the name for the container when it’s running, and the database mount point:

services:  
  web:  
    image: dwfrank/meals  
    container_name: balboa-meals  
    command: bash -c "rm -f tmp/pids/server.pid && ./bin/rails server"  
    volumes:  
      - /volume1/docker/meals/db:/rails/db/production

Then I copied this file to the NAS, ssh’d over, and then called docker-compse:

$ cp docker-compose.yml /Volumes/docker/meals
$ ssh deploy@192.168.1.100:822
$ # Now on the NAS
$ cd /volume1/docker/meals
$ sudo docker-compose create .

…and I had the container loaded but not running. And all of the environment variables were there as I expected.

5. Connecting the Container to the Web Station

The last step was going to have to be manual for now. In the DSM GUI:³

Container Manager - run the container
Web Station - reconnect the meals.local portal, which won’t have a container associated with it, to the container I just ran.

So let’s stitch this all together!

6. Rake All the Things!

To recap the work so far, I did the following:

Built a production.sqlite3
- Moved to a subdirectory under ./db
- Fixed ./config/database.yml
Added a ./.dockerignore
- Excluded ./tmp
- Excluded ./db/production/
Updated the Rails ./Dockerfile
- Commented out asset builds
- Added NAS user and group IDs to the chown commands
- Added injecting the RAILS_MASTER_KEY during the image build
- Used this Dockerfile to build the container image from the command line
Added a ./docker-compose.yml
- Picks the image name
- Sets a container name
- Sets the database volume mount point

The remaining work that needed to be done on the NAS, via sudo, to deploy the app was to load the image, with docker, and then execute docker-compose. Local commands are easy. But what about the sudo commands?

Enter SSHKit. This is what Capistrano and Kamal use under the hood in order to do their work. And SSHKit has a Sudo module. So I was able to use all this for the last remote commands.

I made two additions to the Gemfile development group:

gem "sshkit"  
gem "sshkit-sudo"

And then these commands are in a Rake task:

require "sshkit"
require "sshkit/dsl"
require "sshkit/sudo"

desc "Loads the built image into Docker on Filgate"  
task :deploy_to_synology do
 include SSHKit::DSL

  pw = "kwijibo" # get your password from a password manager, etc.
  send_password = SSHKit::MappingInteractionHandler.new("Password: " => pw)

  bin = "/usr/local/bin"
  docker_compose = "#{bin}/docker-compose"
  docker = "#{bin}/docker"

  on "deployuser@192.168.10.124:622" do
    # acutal mounted path for the docker folder
    within "/volume1/docker/meals" do
      sudo "#{docker_compose} down",
        interaction_handler: send_password
      sudo "#{docker} image rm --force dwfrank/meals",
        interaction_handler: send_password
      sudo "#{docker} load --input /volume1/docker/meals/dwfrank-meals.tar",
        interaction_handler: send_password
      sudo "#{docker_compose} create",
        interaction_handler: send_password
    end
  end
end

Of note above:

SSHKit wasn’t loading the PATH as expected, so I used absolute paths to the executables
It stops the running container down and then deletes its image
Then it uses docker load to add the image to Container Manager/Docker
- The assumption is that the image tar is present
Then it uses docker-compose to create the new container from that image

So when you look at the repo, you’ll see these tasks:

Ensure that git is clean and everything is committed.
Rebuild the Rails assets in ./public.
Build the image.
Copy the image and docker-compose.yml to the NAS.
Do the Docker work on the NAS:
- Top the current container
- Remove its image
- Load the new image
- Create the new container

Phew! And now I’m wondering how I can get the last manual steps into Rake tasks. Stay Tuned!

My Synology NAS mounts most of the file system to /volume1. Your mileage will vary and will be different from what you see in the DSM File Station. ↩
I also went back and uses the same technique for the UserID and GroupID for the chown command. You can see that in the code in the GIST. ↩
I’m guessing that Web Station sits on top nginx configuration files, but I’ve not found them or the right mix of commands yet. I will update this post, and/or add a Part III, once I figure this out. ↩

Synology on Rails Part I: Running

2023-12-12T20:35:00+00:00

Following my last post, I was ready to deploy an early version of my Meal Planning/Recording Rails app. Given this is a small app, with really only two users, I didn’t feel it necessary to pay to deploy it to the cloud and to do the work needed to secure it for cloud hosting.

I knew that I could run containerized apps on my Synology NAS. So I just needed to figure out howto get Rails and my app containerized and deployed.

What I Wanted

Rails app running in a container on the NAS
The SQLite file not in the container
Access inside my home network
Automatable deployment

What I Had

A Rails app using a SQLite database
A Mac Mini M2 for development
A Synology NAS with an Intel processor running DSM 7.2
Me, who despite working at Pivotal on container-based technology, only had a conceptual understanding of all of these pieces working together.

What I Did

First, I needed an image.

1. Install Docker on MacOS

I installed the Homebrew Cask version of Docker

$ brew install --cask docker

I ran Docker, keeping the Docker daemon running in the background on MacOS.

2. Getting The Rails 7.1 Dockerfile

Rails 7.1 ships with a Dockerfile. I built the app with Rails 7.0, but have since upgraded to 7.1, so I generated a new app and copied these two key files:

./Dockerfile
./bin/docker-entrypoint

I’m using cssbuild to turn my SASS into CSS. I decided to keep Node and Yarn out of the container for size, which means I made this change to the Dockerfile to comment out the asset compilation:

# Precompiling assets for production without requiring secret RAILS_MASTER_KEY 
#RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile

I’ll deal with assets later.

3. Building the Image

It’s possible to build an Intel-targeted container on Apple silicon. Very simple:

$ docker buildx build --platform linux-amd64 -t <tag-name>

However, the when bundling, Bundler complained that the Gemfile needed to support Intel processors. This line adds this support:

$ bundle lock --add-platform x86_64-linux

Then the image build just worked and was in Docker locally. Then I was able to push the image to Docker Hub in the GUI.

4. Running the Image on Synology DSM

I followed this guide to better understand how containers, and the Container Manager, work on Synology. I applied this knowledge by:

Updated Synology DSM to 7.2+
Installed the Container Manager package
Installed the Web Station package
Imported my image from Docker Hub
Ran that image to get a booted container

This almost worked. Everything was good until step 5. The Container Manager had all of the environment variables. Port 3000 was exposed.

I followed the prompts in the UI and made a named Web Station Web Portal - I called it meals.local that pointed to the Container. I also updated my local DNS - I use a Firewalla at home - so that meals.local pointed to my NAS’s IP address.

5. Whoops - RAILS_MASTER_KEY

But Rails wouldn’t start because I didn’t have the master key. This was a simple setting in the Container Manager app, adding a RAILS_MASTER_KEY environment variable and manually copying the key over from config/master.key.

I saved and restarted the container.

6. Whoops - The Database

Now Rails wouldn’t start because there was no database. I did the following:

Created and migrated the production database file in db/production/production.sqlite3 on my dev machine
Updated config/database.yml:

production:  
  <<: *default  
  database: db/production/production.sqlite3

Downloaded a Rails .dockerignore (thanks yizeng!) and added db/production to it
- This keeps the production SQLite file out of the container
Created a directory for my app on the NAS
- Synology Container Manager makes a directory called /docker, so I made /docker/meals
Shared /docker so I could see it on my network, thus able to copy files from my development machine
Copied production.sqlite to /docker/meals/db/production.sqlite3
Rebuilt the image and uploaded to Docker Hub

From there I was able to stop and delete the container, then update the image from Docker Hub. Before I started it I had to make a container Settings change in Container Manager:

Map a mount point in Container Manager (Container - Settings - Volume Settings in the GUI) for the container so that /rails/db/production in the container pointed to /docker/meals/db on the NAS.

I rebuilt and restarted again.

7. Whoops - Users and File Permissions

Rails still wouldn’t start. This time it was because the rails user, which the Dockerfile creates, didn’t exist. Back to Synology DSM.

Created a user named rails in the users group
Fount the rails user id and group id - I had to sudo to the NAS for this
Updated the Dockerfile’s “useradd” area like this:

# Run and own only the runtime files as a non-root user for security  
# Synology DSM - make a user called 'rails' and save its id (will need to ssh/terminal for this)  

ARG UID=132 # sudo and find this  
ARG GID=100 # likely this value, but sudo and set properly  

RUN useradd -m -u $UID -g $GID --create-home --shell /bin/bash rails && \  
    chown -R rails:users db log storage tmp public

# Switch to this user
USER rails:users

Then I had to do the manual cycle of build image, push image, stop container, delete container, update image, run image, change container settings.

But the app booted and worked! I could visit https://meals.local and see my app. Victory!

This was a lot of trial and error. And I was tired of all the manual building, copying, and GUI work.

Next post: automating the deploy.

Dynamic Polymorphic Forms in Rails 7

2023-11-15T16:18:00+00:00

It was time to move my Meal Planning app from AirTable to a custom application. AirTable’s pricing for the size of the database wasn’t awful, but I’d moved past the free account limits. And there were some features I wanted to add that were hard with the basic AirTable features.

It’s been a while, so let’s write a Rails app!

I exported CSVs from AirTable, ran bundle && rails new and got to test-driving. Rails 7 and Resource scaffolding just worked as I expected. Recipes and Restaurants came together very quickly.

As soon as I tried to model a calendar of days, where days could have one or more restaurants and one or more recipes (a key feature of the current solution), things got complex.

I ran into three problems:

Rails and forms with has_many associations
But the association is polymorphic
But the form really needs to be dynamic in order to add/remove things from the association

Complex Forms

I had my Recipe and Restaurant models and full controllers. Now I wanted to add a Day model that has a date as well has has_many :recipes as a starting point.

Rails has gotten pretty mature when it comes to the fields_for helper that basically gives you a nested form that includes your association. The Complex Forms section of the Action View Form Helpers Rails Guide talks about People with more than one address. This is great for explaining how form_with and fields_for work together.

To keep things simple, I only tried to add and edit one Recipe on a Day. It was working great. Now it was time to make the association polymorphic.

Polymorphic has-and-belongs-to-many

I needed to be able to add one recipe or one restaurant. I tried several different modelings of the associations to get a polymorphic has_many to work and couldn’t get all the tests passing. Then I found this StackOverflow post that was. I needed a separate join table with some has_many through’s’.

Of course! I needed something that looked more like:

Day <-> Meal (Course <-> [Recipe | Restaurant])

Which got me to these models:

class Day < ApplicationRecord  
  validates :date, uniqueness: true  
  validates :date, presence: true  
  
  has_many :meals, dependent: :destroy  
end

class Meal < ApplicationRecord  
  belongs_to :day  
  belongs_to :course, polymorphic: true  
end

class Recipe < ApplicationRecord  
  validates :name, presence: true  
  
  has_many :meals, as: :course  
  has_many :days, through: :meals  
end

class Restaurant < ApplicationRecord  
  validates :name, presence: true  
  
  has_many :meals, as: :course  
  has_many :days, through: :meals  
end

(Note: I’m not thrilled with the name of course, but it’s close enough.)

Now with the form, I could put one Recipe select and one Restaurant select and be able to have one or the other. There were some bugs, sure. But I was creating, updating, and saving Days. Progress!

Next question - how do I get the form properly dynamic, adding as many recipes and restaurants as I wanted?

Dynamic Complex Polymorphic Forms

There’s a nice caveat at the Rails Guide for forms:

Rather than rendering multiple sets of fields ahead of time you may wish to add them only when a user clicks on an “Add new address” button. Rails does not provide any built-in support for this.

Whoops. I’m on my own.

To recap, I knew that what I wanted was for a Day form to start with no Meals, but then be able to add or delete many Recipes and Restaurants and submit that form at the end to create or update the Day.

Time to learn Turbo and Hotwire.

I found two great pages that helped immensely. First, was Alexis Chávez’s guide to Turbo & Hotwire. The concepts were clear - it’s a bit like old RJS, but more formal thanks to how Controllers have evolved. But how do I structure the form?

Back to StackOverflow, and a nicely detailed question Rails 7 Dynamic Nested Forms with hotwire/turbo frames. It models cocktails with ingredients and looked a lot like what I needed.

I had some patterns! Time for some code.

DaysController gets new methods

First, a couple of new routes:

resources :days do  
  get :new_meal, on: :collection, path: "new_meal/:course_type"  
  delete :destroy_meal, on: :collection, path: "destroy_meal/(:id)"  
end

Then in the Day form I added two Turbo links that call the #new_meal method: one to add a Recipe and one to add a Restaurant.

.meals
  ...
  = link_to new_meal_days_path("Recipe"), data: { turbo_method: :get } do  
    %i.fa-regular.fa-square-plus  
    = "Add Recipe"  
  = link_to new_meal_days_path("Restaurant"), data: { turbo_method: :get } do  
    %i.fa-regular.fa-square-plus  
    = "Add Restaurant"

Then in DaysController:

def new_meal  
  helpers.fields model: Day.new do |f| 
    f.fields_for :meals,  
                 Meal.new,  
                 child_index: 
                   Process.clock_gettime(Process::CLOCK_REALTIME, :millisecond) do |ff|  
      render turbo_stream: 
               turbo_stream.append("meals", 
                                   partial: "meal_fields", 
                                   locals: {f: ff, klass: params[:course_type] })  
    end  
  end
end

Copying from the “Cocktails” post, this method:

Creates a form helper that will build the tags for a Day and then one Meal
Passes the fields_for form helper (ff) to the view, along with the course_type that came in from the GET

And then the meals_fields view…

- div_id = "meal_#{f.index}"  
  
%div{id: div_id}  
  = f.hidden_field :id  
  = f.hidden_field :course_type, value: course_type  
  .course_edits  
    = f.collection_select :course_id, 
                          course_type.constantize.order(:name), 
                          :id, 
                          :name, 
                          prompt: "Select a #{course_type}..."  
    = link_to destroy_meal_days_path(id: f.object.id, div_id: div_id), 
              data: { turbo_method: :delete } do  
      %i.fa-regular.fa-square-minus

This builds a div with a unique ID (from the timestamp in the controller method) that renders:

Hidden fields for the Meal id and course_type
A select with that type’s list (yay, String#constantize!)
A link to destroy this section of the form, if the user so chooses

And that link routes back to DaysController#destory_meal

def destroy_meal  
  Meal.find(params[:id]).destroy! if params[:id]  
  
  render turbo_stream: turbo_stream.remove(params[:div_id])  
end

Which will…

Find and destroy the Meal if this is a Day#edit and so an id is provided; but ignores it if not
Removes this Meal’s div inside the Day form

It’s Alive!

Phew! Thanks, Internet!

Guest on Radio Free XP Podcast

2023-10-28T18:15:00+00:00

For your weekend podcast listening pleasure - I was the guest on Radio Free XP this week¹.

I got to talk to Tony and Jesse about my Pairing Journey. There is probably the most resonance with folks who worked at or with Pivotal/Labs, but I got to tell some fun stories. We talked about parallels between pairing and collaborative gaming - including a shout-out to the classic Infocom text adventures and the more recent Overcooked - and pairing stories from the real world that worked and didn’t. Then we wind up by talking to our former coworkers who are impacted by the Broadcom/VMware acquisition.

Share and enjoy!

Recorded on 2023-10-25 ↩

Balanced and Empowered

2023-10-05T00:00:00+00:00

This article is a spiritual Part II to the post The Team is the Agent of Work.

When I joined the PKS Release Engineering team, they had two dense tracks of work.

Track One was developing all of the continuous integration pipelines and tooling that tested and packaged the PKS product. This was a very complex system of nearly 500 Concourse pipelines, running on 4 (eventually 5) cloud infrastructures, for 4 versions of the product, and between 3-8 different scenarios. These ran on every pull request that came from the PKS development teams.

Track Two was operating all of those pipelines - updating pipelines as business needs changed, triaging failing builds, fixing any infrastructure causes, and notifying the dev teams when it was a legitimate product failure.

This was hard.

To make it easier, the team was building the system so a developer could run a subset of pipelines, locally, on one infrastructure. The hypothesis was that running green locally increased the chances that the run of the full pipeline set would be green and the PR was acceptable. It was a fine hypothesis.

The complication came from the volume of RelEng feature work needed before that hypothesis could be validated. RelEng as a team was empowered to do this work for the PKS dev teams. But unvalidated hypotheses are the devil’s playthings.

The stakeholders didn’t care about a grand set of tooling that might eventually make the developers’ job “easier.” They wanted to ship PKS to customers NOW. New features, new versions of Kubernetes, and bug fixes should be shipping frequently. RelEng wasn’t “moving fast enough” (read: as fast as they expected).

Which is why one of the VPs wanted to fire me. I was leading this team now. And I had a very frustrated stakeholder.

Balance the Team

With the team being the agent of work, how do you set them up to deliver to stakeholders’ expectations?

Empowered teams are impressive for the amount of work they can do while staying healthy. What is sometimes missing in the discussions of empowered team theory¹ is how to keep the team focused on the correct work - the work that actually meets the stakeholders’ and customers’ needs and expectations - at the right time - executed in an efficient order. How do you keep everyone talking, the team productive, and prevent unnecessary work?

One organizational approach for empowered teams is Balanced Team. Very simply put this approach gives you three key roles: Designers, Developers/Engineers, and Product Managers. Each comes with a set of modern practices. Designers bring User Centered Design, Developers bring XP (TDD, Pair Programming, Continuous Integration, etc.), and PMs bring Lean Product Development.

Designers decide what’s desirable to customers that solves their problem, Developers decide what’s feasible to build, and Product Managers decide what is viable for the business and stakeholders. The intersection of the three areas is where the team will build the most value. The interactions between the roles reinforce each other to keep the team in the value sweet spot.

This sounds great. But how do you keep your stakeholders involved? By owning viability for the business, Product Managers are the conduit between the team and the stakeholders. They need to understand the business context well enough to help define stories and epics, especially story slicing and backlog prioritization. They need to understand design and engineering well enough to evaluate and compare the costs of potential solutions.

And it’s not just a “build the backlog” role. They need to accept completed stories as a proxy for the customers and the stakeholders, rejecting work when it’s incomplete or bugs are found. They need to react to changes in the business climate when things like budget or headcount changes or new competitors enter the market. PMs also need to reset stakeholder expectations when challenges like API limits, hosting costs, or new user insights present themselves.

A strong Product Manager reduces the stress of stakeholders, enables designers to focus, and frees up developers to code.

Reset the Team

The VP (who, again, was the one who wanted to fire me) was telling us, very directly and indirectly, that we were not meeting their needs and expectations. Our Empowered RelEng team was building what we thought would make the whole org move faster. Instead, it was a big, complex solution with impact that was months away from paying off.

I was acting in the balanced team’s role of Product Manager. I needed to reset the relationship with the VP, the dev teams, and the rest of the stakeholders. And I needed to do this in a way that would unlock the developers’ skills to tell us how we could start moving the entire org to higher productivity.

First, we broke our existing system down into all of its steps - from pull request to published package - and timed them all. We listed all of the potential failure cases along the way. We put it all on a public whiteboard and shared it with the entire org.

With some introspection, we saw that we were focusing on building an ideal toolset for the PKS dev teams - Track One. Track Two had us mostly focusing on frequent flaky tests running in the pipelines. The former was something that nobody asked for. The latter was wasting time and effort, because at the end of the day RelEng had very little control over making the tests more reliable.

Point at the Summit

We held a mountain climber retrospective and put the goal at the top of the mountain of “1 pull request per day per version.” We listed our “boulders” - all of the bottlenecks from the whiteboard. We talked about making better “ropes” - code we could write that would get us to the top more reliably and faster. We spotted some “First Aid kits” - places where some additional help from other teams would make us stronger.

With the new focus, we prioritized smaller efforts that shrank the feedback loops for every pull request. We did work in the pipelines themselves. We found the easy wins improving RelEng’s code and tooling that removed, or heavily reduced the cost of managing, failure cases.

Some highlights²:

Parallelized our pipelines such that a 35 hour pipeline run shrunk to 7 hours
Built our pipelines from smaller, well-abstracted YAML files, reducing manual errors during pipeline operation to zero
Moved our public cloud resource management to well-factored Terraform, eliminating errors during cloud creation & deletion (this happened hundreds of times a day)
Worked with an internal team to right-size our private cloud resources, reducing errors due to lack of server availability
Deleted code that generated some product configuration YAML, replacing it with editable YAML files, reducing errors during our final packaging

We updated the public whiteboard frequently and improved org-wide communication on our results. We got very close to 1 PR per day.

Oh, and I kept my job.

Our empowered team became a balanced team. We treated Release Engineering as a product. We interviewed our users - the PKS developers. We met more often with our stakeholders. We focused on what we needed to build instead of what we wanted to build.

We solved problems. We shipped. And people - customers, stakeholders, and the RelEng team - got happier.

Shoutout to RJ Zaworski for this take on Empowered Teams. And extra points for the hot-take-title. ↩
For more technical details of some of this work, see Three Cheers for Release Engineering. ↩

The Team is the Agent of Work

2023-09-18T22:11:00+00:00

Once, a VP walked into my boss’s office and said that I needed to be fired.

I had been on the team for about four months. The first few months I was an engineer on the team. Then I had moved into the role of Product Manager - in the Balanced Team/Pivotal sense, defining and prioritizing the outcomes for the team of 3-to-4 pairs of engineers.

This was a Release Engineering Team and we weren’t releasing incremental builds to the rest of the organization very fast or efficiently. We were finding problems and fixing them, and thus increasing our frequency of output. But we had a long way to go.

But for some in the org management chain, we weren’t moving nearly fast enough. And since I was nominally “in charge,” they decided that I was the problem. Replacing me was the essential course of action.

Finding Accountability

Release Engineering for PKS was an intentional specialization, abstracting away the complexity of Kubernetes, 5 different cloud infrastructures, and multiple interdependent products from related teams. The team was building all the tooling to run a very complex set of continuous integration pipelines for 4 versions of the product simultaneously.

The org had over 30 engineers and multiple downstream teams who needed CI to be green and an installable product package in order to get their jobs done. RelEng needed to be smooth and efficient and we weren’t there yet.

The VP had very different expectations about how long CI and packaging should take. Since I couldn’t meet these expectations after four weeks leading, I needed to be held accountable. Removing me was necessary to make the team more productive.

Somehow.

Looking back, this would have been like a Major League Baseball team replacing its manager in the middle of a losing season and then expecting a World Series win come October.

Keeping the Black Box Open

Andy Grove valued middle management because they could cut a hole in the (black) box that is a system to find out a team’s problems, then drive them to fix those problems. John Cutler has talked about what happens when you stop looking inside the box - you return to status-report-driven drudgery without improvements.

Elisabeth Hendrickson taught me about a model of management that’s builds well on top of these ideas - that the team is the agent of work. They do the work. They own the desired outcomes. Together. No single person, and certainly not the manager, is essential to team success. The team succeeds and fails together.

If a team is struggling, if they are not meeting expectations, a manager should ask them why. A lot. The team has the most context about what is happening and why. They know where their dependencies are. They know what work they are doing or not able to do. The manager’s job is to ask, listen, understand, and then use their own context and experience to enable the team to improve.

Because it’s the team doing the work.

Flashlight Time

Back to the story.

My boss asked me a boatload of questions about what was really going on. Then she pulled some organizational aikido and had me lead a series of meetings with the VP and other team leaders where I reviewed, in excruciating detail, the entire Release Engineering pipeline. We talked about code and dependencies. I showed average timings of each step over the past few weeks. I talked about the causes of our top five bottlenecks and what our roadmap was to fix them. I also pointed out where other parts of the org could help RelEng go even faster.

I kept the job. The team kept moving. We even got a bit more support from the rest of the organization. And we did get faster over the coming months.

(It’s worth noting I didn’t hear about the whole firing thing from this manager until a year or two later. Great managers know how to get the most out of a team.)

Save the Team, Save the Outcomes

So when a team is struggling, if it seems sudden or perennial, the solution is almost never to replace the manager.

If you want to help as a leader, remember that building software is a system like any other. Dig into the context with the team. Ask why enough times until you understand what is preventing the outcomes you and the organization expect.

Then use your experience and influence to increase the team’s chances of success. Remove obstacles. Change goals. Find other black boxes, cut more holes, and shine more lights. And if you have data that tells you to, make staff changes.

But remember that a new manager isn’t going to win you the World Series. Just ask New York Yankees fans about Billy Martin.

The team is the agent of work.

Improved 1-1 Note Taking

2023-09-13T17:54:00+00:00

I spent the better part of 18 months getting really good at using Obsidian. Enough that I wrote a whole blog series about it! Daily Journaling, general knowledge dumping, and longer-form writing became very natural.

The job I started after those months of Markdown-ing-more-than-ever-before was remote-first. I was ready to use my new-ish note taking superpowers in another context.

I set up my home office with two screens. I had a 27” monitor full of browser tabs where I did a bunch of Zooming, Looming, and a little coding. And I had my employer’s laptop with Obsidian maximized so I could take notes as I did all the work. Things started well.

But a few months into the job, Obsidian and I started having problems. Smooth writing became clunky. What happened?

It turns out that in a job on a team of other people, collaborative writing is a thing. Duh.

I started any document in Obsidian as always, but when I was ready to get comments or work with others on a shared doc, I moved text over to Google Docs, or later when our team started using it more, to Notion. I needed comments and co-writing, so this made sense.

I learned to delete an Obsidian draft and move it to Notion sooner. I even got better at starting a doc in Notion, preferring to work, write, and think more in the open. But there was still a case where my Obsidian patterns weren’t working: 1-1’s.

1-1 Notes as a Collaborative Document

As part of my Obsidian series, I wrote about how I tracked 1-1’s. I was pretty happy with how this worked. I kept action items. I linked individual notes into the Daily Journal. It really worked for me most of the time.

However, it turns out that 1-1’s are an opportunity for collaboration. Duh, again. The whole point of 1-1 meetings, especially between a manager and a report, is to work through issues, career plans, and meta concerns together.

An Obsidian vault is not easily a place for “together.”

A Notion of a Solution

I spent a few weeks iterating using Notion for 1-1’s with the other people involved (one at a time). Notion turned out to be fantastic place to experiment with meeting these needs:

Keeping all of my 1-1’s well organized, making it easy to find the next meeting with a person
Allow for different meeting templates per person
Easy to create a new regular 1-1; my staff changed occasionally, so adding/removing meetings was useful
Keeping track of action items the two people were creating and managing in each meeting
Keeping the 1-1 notes and action items just for the two people to collaborate

I got to a pretty robust solution. Enough that I’ve productized it!

So if you are someone who:

Uses Notion as your knowledge base
Has several regular 1-1 meetings with others in the same Notion instance
Have had challenges keeping the notes and action items from these meetings organized
Like, or are intrigued by, the idea of 1-1 content being the result of collaborative editing

Then please have a look at my 1-1 Tracker (purchase link at Gumroad). And if you like what you see, you can buy it. And if it works well for you, you can rate it.

I hope this helps you. Five Stars. Will take notes again.

Three Cheers for Release Engineering

2023-08-14T14:16:00+00:00

The first time I was on a modern Release Engineering team, I was skeptical. I had worked with, and managed, “build” or “installer” teams. They often felt slow - a relic of the years of shrink-wrapped software. Why did we need them for a web-downloadable or web-deployable product?

I understood right away. Even web software could reach a level of complexity where additional engineering just to release was necessary. I wanted to share my realizations with everyone!

The analogy of beer brewing hit me one afternoon and this talk came together. The first versions included the pun of Release Engi-beer-ing. But like most puns, that didn’t last long. I’ve given versions of this a few times for internal audiences at Pivotal, and then at Fountain. But this is the first time I’ve shared this publicly.

Come for the analogy, stay for the war stories and insights.

Three Cheers for Release Engineering

Curiosity and Impatience

2023-07-18T00:00:00+00:00

Some years ago, during a particularly complex client engagement, we got thrown a curveball. The team was about 10 developers, cranking on a greenfield project. We were a few months in and despite a rocky start, were feeling pretty productive. CI stayed green, we had code in Production, and we were iterating quickly. Then came a random stakeholder request.

The client wanted us to build an upcoming feature set on top of their research teams’ experimental APIs. We were building a Ruby on Rails app, deployed on Linux. This API was a collection of Windows DLL’s, running on a PC in a lab, in the client’s office, in another state.

Did we make it work? Totally. It’s just software.

It was a bit of a slog. We had to set up tunnels between our cloud deployments and that office. We stalled for days at a time while we waited on the client’s IT team to get to our configuration request tickets. We got there. But it was not fun. Or reliable.

The tunnels were flaky, dying often. The client’s IT team rotated IP addresses on an unknowable schedule. This would cause CI to fail. Or even worse, cause demos for the client to crash. At least once we had to find someone to go reboot the Windows box. Not. Fun.

After a few weeks of this, we added some lightweight observability alarms. We invested in some scripts to restart tunnels and update some config files. These were bandages to make things somewhat manageable so we could continue development.

But our morale, productivity, and credibility all took a hit every time this service went down. It was a tax, slowing down work the team was doing. But we lived with it in service of doing new work.

Empathy is a hot topic these days - and for good reason. Software is made of people. Even when we are remote-first and heavily asynchronous, we still have to work together to build software products. Empathy is key to that. Nat Bennett wrote about screening for empathy in Pivotal’s hiring process and tied it to one of Pivotal’s values - “Be Kind”.

Pivotal had two other values: “Do the Right Thing” and “Do What Works.” Over time I’ve realized that a good way to practice these values is to be curious and impatient.

A productive team is impatient. They pay attention to anything that feels awkward, slow, or inefficient. It could be production code, a CI that is flaky, a manual task list for every deploy, or that Finance keeps complaining about your infrastructure bill. These are taxes on your team’s productivity.

It’s not enough to notice, or worse, to complain that these are slowing you down. You need to be curious enough to find out why the tax exists. And then you need to find an alternate way to achieve the same outcome - only with less distraction and more efficiency.

This is work like any other the team does. The team needs to plan and manage it against other priorities. But it’s work that you need to do. And it can be just as important as new feature work.

John Cutler mentioned this in his newsletter post Your Calendar = Your Priorities. There is value in doing the work that improves your efficiency - what John calls “managing complexity.” If you ignore these issues, they inflate and turn into long, expensive interrupts. That work becomes an even bigger drag on your team.

Back to the project. The issues of this Windows machine spilled over at a retrospective a couple of weeks later. Nobody was happy with our solutions we’d tried so far or the impact it was having on the team. Time to find a better way.

We gave a pair of developers a couple of weeks to dig deep into the problem space and experiment. It wasn’t easy, but they found a way. They negotiated for admin rights to the vSphere instance that was managing all our VMs for our Rails app. They scripted vSphere (using a Ruby gem) to provision Windows VMs. They automated deploying the latest version of the DLLs to those VMs and then connected them to our Linux Rails deployment. They test drove this work and integrated it into our CI and deployment scripts. After this investment, the Windows code never caused CI to fail, or a Production demo to crash, ever again.

It’s hard to map curiosity and impatience directly to the values of “Do the Right Thing” and “Do What Works.” When teams acted according to these values, they fixed small problems early and stayed productive. They stayed productive by making themselves even more productive.

Pay attention to your slow tests. Or flaky tests. Don’t waste your infrastructure. Keep your dependencies up-to-date.

Do the Right Thing. Do What Works.

Stay impatient. Stay curious.

A Little More Yak Trimming

2023-07-15T21:45:00+00:00

I continued to trim around the ears of this blog since last week. And a few things stood out that I felt worth sharing.

CSS Factoring

CSS, in this case SCSS, is source code worth factoring just like any other code. Now that I’d trimmed the stylesheets significantly, I felt it worth organizing them better for later maintenance. Maybe I won’t feel the need to throw out and start over in another year!

I’m using Google Fonts here. While I was pretty happy with the indirection of SASS variables for font family, I did go a little further and have variables for color and weight. Then I pulled out a fonts-only file. While I stopped short of writing a function that generated the Google Fonts URL, I still can change all font attributes in one place if I choose. That’s a fine win.

I also made sure all my font units were in rem, making it easier to scale things for mobile as I tweak sizes over time when fonts change.

I pulled out other sections, roughly mapped to the different types of pages, each into their own files. And I put special media queries at the bottom of each file. But I do have 1 key complex media query for portrait phones in its own file. The longest file is 85 lines. My final css file is only imports.

Better Syntax Highlighting

Lastly, due to my Obsidian series, there are a lot of code blocks that are just Markdown. I wasn’t happy with how simply Rouge (and thus Pygments) supported Markdown - the tagging is simpler than I’d like. Kramdown supports Rouge and [Coderay][c]. But I wanted a converter that had more granularity for MD.

I converted to using Prism, which highlights <code> blocks with JavaScript. This required:

Downloading a Prism bundle that included the language support I expect
Finding and tweaking a Prism theme
Adding these to the repo
Adding these to the layout files
Turning off syntax highlighting for Kramdown with a well-placed nil in Middleman’s config.rb like so:

set :markdown_engine, :kramdown
set :markdown,
    auto_ids: false,
    input: "GFM",            # GH emoji + ``` code blocks
    layout_engine: :haml,
    tables: true,
    autolink: true,
    smartypants: true,
    fenced_code_blocks: true,
    syntax_highlighter: nil  # turns off Rouge so we can use PrismJS

While Kramdown won’t parse the contents of the <code> block, it does set a CSS class on it (and the parent <pre> tag) based on the language. Prism uses that class when converting to HTML for coloring.

That’s enough blog yak shaving for a while. Back to writing…

Anchor's Away

2023-07-12T21:51:00+00:00

I spent the summer of 1990 in Europe, mostly in Leicester, England, taking classes at what is now DeMontfort University. That meant a summer drinking a lot of beer - English beer, mostly Leicester’s Everard’s Old Original because, hey, Drink Local!

I was dreading heading back to Athens, GA, and being stuck with the beer choices we had. The microbrew revolution - thanks, President Carter! - ironically hadn’t made it to Georgia just yet. Which meant my beer choices were slim. So slim that were were happy with Killian’s Irish Red and ecstatic with the occasional splurge on Samuel Smith’s 4-packs.

Home was inevitable, so off to Gatwick and the standard US beer aisles. I had a minor surprise on the flight back to Atlanta. We were shown an episode of Michael Jackson’s The Beer Hunter. It featured Fritz Maytag and Anchor Brewing. I learned the term “microbrew” and that Anchor Steam had wide distribution. Maybe I had hope for hops!

This is how Anchor Steam became my favorite beer. It was pretty easy to get. It was less expensive than Sam Smith’s. It was exotic because it came from California. I was an oddball asking for it.

The Anchor Steam beer label up until the rebrand of 2021.

A classmate had a t-shirt of this (now) “old” label. He never would sell it to me. And I never managed to steal that shirt from him. It took until 1995, on the one time I made it to the brewery, for me to get my own shirt. And I wore it to threads a few years ago.

I’ve fondly enjoyed just about every Anchor style - Liberty Pale Ale just never worked for me. The California Lager was really nice in the summer. Especially after they stopped brewing Anchor Wheat, after the rebranding of it to Anchor Summer. I still have a magnum of the 2019 Anchor Christmas Ale and a six-pack of Christmas 2021. Wow those are good. I’ll save them even longer now - they keep and age well.

Even as I moved to the west coast in 1993, and the number of beer options became anxiety-inducing, I still had Anchor fairly often.

When I had season tickets to the San Francisco Giants, there was an Anchor Steam tap at the top of my aisle. It was my first stop at a game for ~20 years. I called it the freshest beer in the stadium since it was brewed a mile-and-a-half away. I even imagined pipes running from 3rd & King down to the vats in Potrero Hill.

The brewpub across from Oracle Park never materialized. Yet another project where the developer and the City of SF couldn’t get aligned. It died when Sapporo bought Anchor. It would have been amazing.

They opened the Anchor Public Taps instead, and during the COVID semi-lockdown, we drove up for crowlers-only brews a couple of times. Those always felt special.

I woke up today to the news of Anchor Brewing ceasing operations. I’m sure the facilities and the brand are worth something. Maybe Anchor just needs to be saved one more time. After all, beer.

I’m probably part of the problem. But not today. So off to the grocery for one last run for some Steam.

The last six of Anchor Steam at my grocery, 2023 July 12

It was a misfit six-pack, the cardboard handle torn. It felt fitting that in the weight of this moment - the last Anchor Steam at this store - the cashier had to scan the barcode by lifting it with both hands.

If this is the end, Anchor, so long and thanks for all the beer.

Notes

For more escapades from that summer see Goodbye Chris Cornell.
The SF Chronicle has another article about Anchor’s closing.
The judges also would have accepted the title of “Anchor Finally on the Fritz”.

Shaving The Blog Yak

2023-07-11T16:43:00+00:00

I thought some would be interested in my blog “refresh” story. It’s got several yaks that needed everything from a light trim to a buzz-cut/rogaine special. Like any good technical project rathole, I started with a dependency update…

This roughly 3-year-old site is a static site built with Middleman because I prefer Ruby, HAML, and Markdown. I wrote a few helpers, including a moderately interesting one for json+ld. I knew I wanted to do some restyling eventually. I needed to do a bundle update and to see what needed to change in the “guts” before getting to the markup/CSS.

Middleman updated to 4.5, HAML to 6.x, and I moved to Ruby 3.2.2. I then ran into a couple of intertwined problems with plugins and HAML. That led me to reconsider Middleman.

Bridgetown was the obvious next stop. But after a week of wrestling with its HAML support, I gave up. It wasn’t hard to migrate the content, but without full support of HAML components, I was building lots shim Liquid templates to keep most partials in HAML. It felt exactly like the type of work I would want to rip out later. It made more sense to me to stay with Middleman and fix the issues I’d found.

The dependency problems were with two plug-ins. I was using middleman-vcs-time for getting published times for feeds and json+ld. That plug-in has been archived, so I moved to File.mtime for now.

Next was middleman-syntax which has some problems with HAML 6. Without a fix, I deferred this and jumped ahead.

I had to make two changes for HAML 6 - succeed (thankfully) is gone, so I moved to string concatenation. I had a couple of places where I needed unescaped strings (thanks != !) (see again, json+ld).

I knew I wanted to clean up my HTML to ASAP - As Semantic As Practical - so I did some research into classless CSS resets and settled on Holiday. It has some very nice, sensible defaults, making the kinds of changes I wanted to make clean and simple. This let me kill all of my old CSS, including my FontAwesome version-4-era self hosting, instead moving to their JS-based hosted free solution.

Next up was a bunch of cleanup of layouts, HTML, and thus CSS. I had a few design changes I wanted to make that drove this. This was a lot of iterative tweaking, but I think my layouts and component partials are better factored than before. More deleted files!

During the restyling, I found that even with Safari’s Responsive Developer Mode, the media queries aren’t quite right. This meant having the iPhone and iPad around with a cable for working the Console from my desktop for element inspection. You have to install the full Xcode to do this, but it is worth it.

All that done, I took a look at how I was customizing and configuring Middleman. I changed how I store and build my blog series (a custom collection). I do want these presented a little differently, and I had over-complicated them before. Now the series introduction page\s are just in Markdown, like other articles/posts, and share a simple HAML template.

Next up - the motivator for all of this work - was tweaking my resume. Before my resume was in HAML. I wanted to move this to (nearly) 100% Markdown to make maintenance easier. And I got there by making a custom HAML layout, which allowed me to move the social links (using Font Awesome) from HTML-in-Markdown to HAML.

I was using RedCarpet for Markdown-to-HTML because it’s what middleman-syntax recommended. RedCarpet doesn’t support extended Markdown for HTML Definition Lists, which I use in my resume. Back to Kramdown (the Middleman default) for <dl>’s, installing [kramdown-parser-gfm][gfm] so I didn’t have to convert all my syntax-highlighting in articles to ~~~ (and change my brain), and finding a Pygments CSS file I liked. All pretty easy, but I’m still not thrilled with how the syntax highlighting looks for Markdown, so I may be revisiting this later.

I version my resume with dates - this helps me personally and when talking with recruiters. Today this is manual and I wanted to make it automatic. Back to the resume HAML layout, where I added a line to shell out and get the last git commit date of the resume Markdown file with

git log -n 1 --pretty=format:%cs -- FILENAME

Then I added a step to my rake-based build and publish that fails if there are uncommitted files. I should’ve done that ages ago.

After all this work, I’m pretty happy with the result. I learned a bit. Solved some puzzles. Cleaned up some redundancy and poor factoring.

I still have a handful of small things to do that I’ll work on as snacks here and there. A dark mode for CSS, fixing the most-recent-update date on articles using the git commit, some helper refactoring…

Was this yak shaving? Absolutely. Was it fun? Also absolutely.

GoGaRuCo 2013 Storytime

2023-06-28T15:09:00+00:00

Way back in 2013 I got a last minute request from Sarah Mei for a 10 minute talk during the Golden Gate Ruby Conference’s “Story time.” IIRC, it was meant to be a “break time” talk. Something light and fun so that the attendee’s brains could catch up a bit between deep technical sessions.

I knew exactly which stories I wanted to pull together. And it was a lot of fun to give.

I had mostly forgotten about this talk and totally forgotten that it was recorded. So imagine my surprise when I
uncovered the video this week. And that it had 200+ views!

If you’re curious, or you were there and want the 10-year flashback, or maybe just want a ten-minute break today, here it is. I hope you enjoy it.

GoGaRuCo 2013 - Afternoon Storytime - How I test-drove my Career (YouTube)

3X but It's People

2022-06-19T21:58:00+00:00

Kent Beck’s “3X” model is an interesting way to think about software work over the life of a product. Read the article and watch this video for a catch-up.

My high-level paraphrasing is that every “project” goes through three phases, in order:

Explore - you’re experimenting with solutions, including choosing the problems to solve, trying to find a good fit
Expand - you found a problem/solution fit, it working well enough that everyone wants it and you’re scrambling to scale it
Extract - you have scaled, now you need to take a deep breath and extract all the value you can from it

Lather, Rinse, repeat.

I put projects in quotes because the model scales from a work ticket all the way up to a startup business. As Kent says, it’s fractal - 3Xs all the way down. Pick your altitude(s) when applying to your own context.

3X is Made of People

I’ve been thinking about the 3Xs from a different direction. What does 3X look like when looking at it people-first? Who are the people who thrive, or don’t, at each stage? Hypotheses ahead…

Have you worked with people who seem to hop from startup to startup? How about folks who thrive on the chaos of sudden success? Ever seen people who drop into any protect and tidy up all the time, making things simpler and smoother every day?

Using 3X as a guide, what you have seen are Explorers, Expanders, and Extractors. They are people who are really good at contributing at each stage. And maybe less so in others. They show up in all roles on software product teams. How does this idea change your thinking about the people on the project?

I once worked with a developer I liked a lot named Freddie. I learned lots of great coding techniques and deep domain knowledge from him. We got so much done when Freddie was on a project. But I did notice a pattern. When he stayed on a project for a long time, new people had a hard time ramping up to the patterns in the code. Freddie would keep getting lots done, but the team as a whole would slow down. This didn’t make sense.

Freddie was an Expander. He was great at dropping in to a project with a lot of chaos, taming the really hard, immediate problems while ignoring the things that didn’t matter as much. He loved the variety and the urgency. He helped teams get out of the panic and closer to Extract territory. But when the project crossed over into the Extract phase, he wasn’t as effective. His solutions worked, but were hard to learn about and to maintain day-to-day.

Once management realized that he had a sweet spot, we shifted his role and moved him between projects often. More teams got “Freddie Time” when they had scaling issues. He stayed engaged and helpful on each project, teaching his Expander skills all around the department. Everyone was happier.

Explorers are the folks who excel at starting new projects. They deal with bootstrap and boilerplate to get the team moving, help set architectural direction, and encourage experimentation with new tools and techniques. All of this works well when the greenfield is new and lush. But they it doesn’t work as well when scaling challenges start - that’s when you need a Freddie.

Extractors are the ones who want to know everything before they get started helping a team. Sort of like Marie Kondo. They want to march through big swaths of the code, looking at patterns and tools. They delete the unnecessary complexity with extreme prejudice. They spot all the opportunities to refactor, encourage test suite maintenance, and make the project simpler.

(I am full of Extractor tendencies when it comes to software projects. That doesn’t hold true in the real world. Do not look in my garage.)

What, Then?

Every team needs Explorers, Expanders, and Extractors. They are each handy in their own ways and at key times. So how can you apply this new perspective?

If you don’t manage people, maybe this model can help you understand yourself a little better. You used to love your project, but it now seems like a slog. Or your performance feedback went from very positive to meh. Perhaps your project has moved between stages and needs you to change your mindset. Or maybe there’s another project when you can help even more.

If you do manage people, take a look at your teams and their projects. What stage of 3X are they in? Are they suddenly struggling? Or are they dealing with change very well? How are the people on those teams faring on the projects over time? Will some coaching about what the priorities need to be help them? By moving people between projects, can you help a career and a project (and thus a business) at the same time?

Thinking about the 3X’s of people can help you staff your projects for maximum productivity and happiness. It turns out software is made of people.

What Brown M&M's Taught Me About Software

2022-01-17T02:51:00+00:00

I wrote this originally in 2013, and it’s been hosted on the Pivotal Labs, then Pivotal Software, and now VMware Tanzu blog ever since. But I wanted to keep a copy of it around. So here you go, with some light editing.

I had no clue how we were going to afford Branford Marsalis.

I’m not sure who was the instigator, but there was consensus. The Contemporary Concerts team wanted Athens, Georgia, home to an R.E.M.- and/or NCAA-Division-I-football-crazed University of Georgia campus, to host the Sting side-man and New Orleans Jazz scion to perform. I had been on the Concerts team since I was a freshman and was wrapping up my leadership year. A Branford show was a good “last show”: someone famous, interesting, and diverse.

The booking agent had just quoted a larger fee than expected. I had not negotiated the finances of a show before. This skill was in the job description but was rarely exercised. Students’ wallets only open so far. This wasn’t going to work. I was stumped.

This all came rushing back recently when Google announced that their traditional methods of recruiting and interviewing were not predictive of new college graduates’ success. They see the academic computer science environment as being very different than the corporate software environment. This was not much of a surprise to me. Submitting a class project is not the same as shipping production code. The job of a software engineer is to design, develop, and ship while working on project teams with other people, high stress, and big consequences.

Said another way, there the real world is not in a computer lab.

But my entire college career was not just in the computer labs on UGA’s South Campus. I was one of the few geeks who ventured to the middle of campus and got involved with the University Union. I “worked” with mostly North Campus majors – Journalism, English, Drama, and Art – at the Tate Student Center, booking movies, concerts, and lectures on campus. At the time, there were very few terminals, algorithms, or compilers around.

Computers were starting to proliferate around campus – I spent plenty of time teaching MS Word shortcuts, recovering deleted files, and solving PC LOAD LETTERs. But aside from one concert where I was able to re-program Dee-Lite’s lighting system – I was the only person who could count in binary and knew how to flip some barely-accessible dip switches – my CS knowledge did not get much use around the Tate Center.

I started as a freshman moving speakers and taking tickets. I worked my way up the ranks, finding artists on tour, working security, and buying the obscure things that were buried on contract riders. By my last year I was managing the sound and light crew and finding runners (who had approval to drive from the University motor pool) to pick up food for 40 people so we ate before show time.

There are so many similarities between producing a concert and shipping software. Each is a large project that involves a lot of moving parts: complex dependent subsystems, opaque politics, resource budgets, and vocal customers. I spent time coordinating multiple service vendors, student teams, and a musical artist. I managed budgets, ran marketing campaigns, and represented student interests to the school administration. I learned to be ready for anything – to stop, assess and find simple, effective solutions that achieve some manner of consensus.

I essentially double majored in Computer Science and Project Execution. My “classes” at the University Union provided a lot of practical experience that gave me a leg up that first day after graduation. I was ready when my project was canceled and we had to figure out how to document everything and close it down. I was unsurprised when I was asked to be on a plane in 4 hours to courier hardware to a business partner in Tokyo. I adapted each time I was laid off with little (or poor) explanation.

So what’s the big deal about a little negotiation? With no cards to play with Branford’s agent, I went for honest.

“I’m not sure we can afford this show after all. Our venue only seats 800 and at the ticket price we’d charge, I’d have a student revolt. Can you come down on the fee at all? Is there another way to make this happen? The team really wants some jazz on campus.” It was worth a shot – you don’t get answers if you don’t ask questions.

“Hang on…can we move to Saturday? That gives the guys a day off after your show and that means we can do two sets for you at the same price.”

I checked our schedule. The venue was open on the newly proposed day. It moved to a weekend which meant a nearly certain sell-out. And at half the ticket price with double the attendance. Everybody wins.

Then I responded, “That totally works. FedEx me the contract.”

Today I’d just say “Ship it!”

Job Search Journaling with Obsidian - updated

2021-09-17T21:36:00+00:00

I wrote an article a few months ago about how I was using Obsidian to take copious notes during my job search. It covers research into companies, connections in my network, and keeps true to Obsidian daily notes patterns. I’ve kept up the practice and it has served me well. But I found a small tweak I wanted to share.

Interestingly, a company that had turned me down early in my search got back in touch about a new role. When letting you down, internal recruiters always say, “We’ll keep you in mind in the future!” I’ve always written this off as “nice,” but here we are, several months later, and I have a re-screen/catchup chat on the calendar. Quick! To Obsidian!

I immediately ran into a problem. It was harder than I expected to walk the timeline of all of the different calls, meetings, and emails, refreshing my memory. Information was scattered around my daily notes, my per-company-note, and random notes here and there. Even with search and unlinked mentions, it was not easy at all.

Problems to Solve

An interview cycle with a company is a series of interactions that need to be tracked in “one place” for easy retrieval. This system needs to make it easier to:

Have a complete view of all interactions with and about the hiring company
Know when interactions happened
Retrieve the notes at each interaction
- Notes I take during a call
- Key points from emails (e.g., next steps, rejection details)

A Solution

My daily note template has a Calendar section. On the day of a meeting, I will have a heading in that section like this:

# Calendar

## 0230p Christina @ [[Everything Imaging]]

(notes from this call go here)

That Obsidian link goes to the Company research note.

The Company research note, then has the backlink to the daily note in a new section labeled Timeline:

# Timeline

- [[2021.08.05]] Screening Phone call with Billy
- [[2021.08.17]] Christina interview via Zoom

The Files

The repo & files live in “Release 2” at GitHub.

The changes:

The research template now has the “Timeline” section (replacing “Application”)
Added how to use this section in a few places.
Renamed the company/role note as “Research Note”

What I Talk About When I Talk About Management

2021-06-02T14:59:00+00:00

I stepped in it (again) on Dev Twitter, jumping into a discussion about management on agile teams. I saw this thread from @allenholub that had an interesting reply from @alexbunardzic:

The more I work on implementing XP in the teams I'm leading, the more I'm starting to strongly realize that a team is the smallest unit of work. Bypassing the team to hunt down & corner individual team members for some inane "performance appraisal" is so wrong on so many levels.
— Alex Bunardzic #BlackAndAsianLivesMatter (@alexbunardzic) May 23, 2021

I like the first sentence a lot. I am a strong believer in the team being the agent of work on an agile product team. I have a more complex reaction to the second. Nobody likes an “inane ‘performance appraisal’.” But there are times when people need individual help. So I jumped in…

It is possible and necessary to separate individual management from team execution, while still treating the team as a unit of work.
— Davis W. Frank (@dwfrank) May 23, 2021

Which led to this great analogy:

It all boils down to what is being evaluated. An analogy: pass the baton race. Dysfunctional orgs keep a watchful eye on the runners. Mature orgs only care about the baton. Knowing that, teams who are lucky to work in mature orgs organize themselves to optimize the flow of baton.
— Alex Bunardzic #BlackAndAsianLivesMatter (@alexbunardzic) May 23, 2021

I find the assertion “Mature orgs only care about the baton” myopic. But let’s run with this one.

Let’s say we have a 4x400 relay squad who has a set of goals for the season and they have self organized. They are running agile. They are doing well. They have gone from the back of the rankings to contending in most meets. Medals! Success! Fantastic!

What about the individual runners?

This team probably has a coach, or even coaches. The coaching staff worries about all sorts of problems: individual runners’ practice schedules, exercises, hydration, injuries, and egos. That sounds a lot like management to me.

Is the coach, then a member of the squad? Sure. Slice the team however you want. I think the point stands that not all team activities, or team members, are directly about execution to outcomes. There are occasions for additional support to help the runner and the team.

Let’s pop up a level. Say our relay squad is part of an entire Track & Field (Athletics) team. There are more running events. There is the heptathlon and decathlon. The Athletics team has a desired outcome to maximize medals across all of the events. This means more fans and more sponsorships and a better future for all of the team members.

What if swapping a runner off of the 4x400 squad improves the chances of more medals for the team? Or that by moving a runner off of the 4x400, that squad’s time improves? And then that frees up a runner who can enter another event? Or if one of the runners misbehaves outside of their work and sponsors threaten to take away necessary support? How does the team handle these situations?

Running Turtles

When teams are given desired outcomes and given the agency to self organize, they can do amazing things. This is what Agile is all about. But teams do not exist in isolation. They are made of individuals. In larger organizations, teams are working with and/or alongside other teams.

There is self similarity here. It’s turtles - I mean teams - all the way down. A person is a team of one. An organization is a team of teams.

If you ignore the people on a team, you run the risk of losing people from the team and perhaps miss your goals. If you ignore the context in which your team operates, you may wander away from desired outcomes and lose funding. Oops.

This does not have to be due to malicious ignorance. It can come just from natural tendencies to over focus “on the baton” or team members who do not have skills on the team to notice and deal with the problems. So how can you fix this and “stay agile?”

Defining Terms

Let’s expand the model a bit. I’m going to avoid using the term Management and instead define some new-ish ones.

Coaching is the set of solutions that focuses on the needs of the individual within the context of their team. This includes helping a person reach their maximum potential within that team and the organization.

Teams can do a good job coaching directly within the goals of the team. But there are other situations - toxic personalities, complex feedback, or individual goals that are potentially outside the scope of the teams’ goals - that sometimes need an outside perspective.

Governance is the set of solutions that focuses on the needs of the teams within the context of their organizations. This includes helping teams reach their maximum potential within the organization while considering the needs of the organization. This includes supporting coaching.

A team only exists while it is funded. And it’s funded to solve a problem, usually to make a thing that solves a problem. When in the course of solving that problem, their actions or solutions detract from the organization’s goals, then there is a need for at least a discussion, and possibly a correction.

Some situations - both team and individual - can be solved by moving individuals between teams. These situations need input from a lot of directions - involving coaching and governance - in order to get resolved.

This model is the result of observation and iteration that happened during my time at Pivotal. We grew from roughly forty consultants at Pivotal Labs San Francisco in 2008 to the 2000+ employees who VMware acquired at the end of 2019. Coaching and Governance were a key part of that growth.

Implementing and Iterating

In 2010 or so, Pivotal Labs had several high performing agile teams - one for each client of our consulting practice. We had identified the need for team-independent individual coaching. We created a new role to fill that need and gave it the title of “manager.”

Managers remained consultants on projects about 80% of the time. They mostly, but not always, coached people on other teams. Which meant spending about 20% of their time coaching people in 1-1’s outside of the team context. Individuals “reported to” managers in the HR/hierarchy sense.

Some governance was simple. Our clients provided most of the governance for any one team. After all, we were teaching them agile practices for their products. They had their own goals and the teams managed themselves to them.

Pivotal’s need for governance across teams presented itself around staffing changes. We had to staff new projects and deal with projects ending. We needed to ensure we were growing individuals’ skills, onboard new people, and fill seats when someone went on vacation. People quit. Some people did not work well together. This governance was the responsibility of “directors”

We called the process of moving people around between projects “Allocations.” It was a calculus that took input from the clients, all of the teams, and all of our staff to determine who, if any, moved between projects each week. This was initially a meeting of two people and a whiteboard. Over time it grew to a continuous discussion between the team of directors and the managers. The managers were the proxies for their people in these discussions.

The managers were an agile team across all of the other teams. They needed coaching as they grew, met new challenges, and adapted. Managers reported to Directors who provided that coaching.

These management practices scaled, with several tweaks over time to, 17+ Pivotal Labs offices, the Cloud Foundry product teams, and had some influence over the rest of Pivotal R&D. I spent time managing in both organizations. By the time I left in early 2020, there was a lot that was working well. And there was plenty that was not. Some things remained very hard.

Annual Performance Reviews - which is a very non-agile implementation of recognizing achievement - was definitely one of them. It sort of worked. We did not bypass the team to get information and we cornered nobody. But it was not a smooth process and was a contributor to another problem: manager burnout. The burnout came came from other sources, too. Learning to be a manager was challenging and often amorphous. For managers of developers, I think the balance of time on teams vs time as coach was not sustainable.

Gathering, filtering, and delivering feedback was vital to coaching individuals to success. Bad actors were managed out of the company, people found teams where they could contribute more, and I know of a couple of careers that blossomed thanks to how we handled feedback. But as well as we did this, as a solution space it needs a lot more cross-discipline work before I would consider it smooth.

Interface and Implementation

All this said, I think our iterations and adaptations to add coaching and governance were valid and helpful to the people, teams, and company. I will use this experience to help any team I’m a part of.

What I hear a lot, and part of the arguments in the tweet thread, is that agile teams have an interface line (goals/metrics/outcomes) that management cannot cross. That any suggestions about changing things inside the team by a hierarchical manager is anti-agile! That teams could get so much more done without such meddling!

I agree that meddling and micro-management, what Elisabeth Hendrickson (channeling Jerry Weinberg) calls “inflicting help,” are anti-patterns. Agile methods aim to prevent these practices.

In my experience, to ignore the team-independent needs of individuals leads to other anti-patterns like burnout, resignations, and general unhappiness. And to ignore the team-interdependent needs of an organization can lead to funding being pulled, the wrong product being built, or organizational collapse.

Adding concepts like coaching and governance can help people and teams to do their best. And it is possible to add them incrementally, and process feedback about using them, just like you can with any agile practice.

On Pair Programming

2021-05-26T14:51:00+00:00

The internet had a bit of a buzz about Nat Bennett’s thoughtful post on their pairing experience. I think they did a great job talking about the personal impact of the pros and the cons of how we practiced pair programming at Pivotal.

Dev Twitter can tell you all you ever wanted to know, and maybe much you are worried about, on the topic of Pair Programming. Dev Twitter will also swarm you with claims that differ from your experience. Because, opinions, man.

Naturally, here are some of mine.

Pairing at Pivotal Labs

The Pivotal Labs consulting practice was built to immerse product teams in agile practices. Those practices started with Extreme Programming (XP), and added Lean Startup and Balanced Team over time. Clients paid for and expected the immersion. The work was (almost exclusively) in our offices, thus away from their normal day-to-day routines, to maximize exposure to new ways of working.

For developers, XP includes pairing. So developers - pivots and clients - paired from 9am to 6pm. Every day sitting at your own keyboard, mouse, and screen, but at the same computer as another developer (who has their own keyboard mouse and screen). The coaching and learning and dead ends and green-test-runs and high-fives were constant. Breaks happened - people got up to walk around the block, play ping pong, get coffee, etc. But still, it was exhausting.

When someone is new to pairing I use the metaphor of martial arts. You are teaching yourself a new set of skills. Some are mental - thinking aloud, active listening, and figuring out how your pair does the same. Some are physical - negotiating for keyboard control, using a new set of keyboard shortcuts, and navigating a different sense of personal space. You will be tired - a lot - in the first few days-to-weeks because you are thinking and doing differently than before. But with regular practice, and coaching, and breaks, you will build what feels like super powers.

Projects would start with pivot developers pairing with client developers. Over time client developers would pair with each other. The ratio of clients-to-pivots would increase over the life of the project. The client developers became the coaches. It was amazing to watch.
Pivotal Labs was successful at teaching teams how to “be agile” while shipping software. Pair programming was a foundational element of this success.

My Personal Experience

My first month pair programming was draining. I would leave the office spent. And I slept very, very well each night. It was a lot of “wax on, wax off.” But after a few weeks, I did have a well-painted fence. And the finish on the cars was concours-quality.

When I left that job (after about a year) and started at Pivotal Labs, it was like starting over painting an entirely new fence around an entirely new parking lot of old cars. It was hard work, but I learned so much about myself and about the craft of software.

I stayed for 12 years. I worked through lots of change in the company and various roles on different teams. But when I was a developer, I paired by default every day. Pairing works for me. I find I’m even more productive when I pair. I enjoy it. Even fourteen years later.

Introducing Pairing

If pairing sounds interesting, you might wonder how pairing could help your team. Here are some questions to ask:

How long does it take for new developers to ramp up and be effective? How long until they can lead new work?
How much time do your developers spend being coached in the craft? Are they asking for more?
How does your team review code? Does this process work well? How long does it take to get code to production?
What happens to the pace of work when integrating new, large blocks of features into the product? Or when finding and fixing a particularly thorny bug? How long does it take that pace to recover?
Are there knowledge silos on the team? How large is the risk of losing one “key” person due to vacation, resignation, or other reason for leaving the team?

With practice, paring can improve these metrics while continuing to ship software predictably. Teams can grow, and shrink, with low disruption. Developers are learning from each other, improving their craft. They ship lots of business value with high confidence.

TL;DR - Pairing can work. Some developers like it. A lot. But that’s not enough.

Deciding to Pair

But how does a team start pairing?

The team has to decide, as a group, to try pair programming. It’s too big a change to the status quo, from developer up into management, to “just start.” If you want some help here, begin with Janice Fraser and her U-BAD model of decision making : Understanding, Belief, Advocacy, and Decision.

You need a solid foundation. The team needs to understand what pairing is and how it will work. They also need to believe that it could help them. And that they will have support from each other and management to make it work. Pairing can help. A lot. But the team has to make the decision.

Sustainable Pairing Practices

Once the team has made the decision and set themselves up for success, what next? I recommend that everyone starts the day pairing. Having a standup meeting first thing gives everyone a daily reset and catch-up. It’s time to ask for help and plan the rest of the day (see my post Improving Standups for tips).

Among the messages Nat gave us from their post, the key one is sustainability. The team should set itself up so that individuals ease into, and recharge from, pairing time. This increases the chances of adoption and thus the impact on the team and product.

It is important to set the expectation that the entire day will not be pairing-on-code time. As the guide mentions, there are plenty of non-pairing situations. The pair can and should plan their work around the meetings and 1:1’s. This planning is as much about making sure these happen as it is to avoid abandoning someone in a big coding problem.

Encouraging and taking breaks is vital. People get mentally tired. There is a reason you may have heard that we played a lot of ping pong at Pivotal. It is a paired activity that gives your coding brain some rest. I have also found that walks outside - or in a pinch, staring out the window - can be restorative.

The team also should schedule time for solo work. Deep research, reading, and writing are necessary work that are also best as solo activities. Some teams schedule regular solo “spike” or “exploration” days that allow for personal investigation.

Determining when you break up pairing sessions, be it for meetings or walks or table tennis, is going to be a personal or team decision. Some people will like long, unbroken stretches of pairing time. Others will like three or four smaller stints. Experiment. Iterate. Have focused retrospectives on pairing. Make tweaks.

Nat reminds us that teams are made up of individuals. And individuals are, well, individual. Each person’s reaction is going to be different. Pairing won’t work for everyone. And that is OK. Prepare for some people to opt out - be ready to move them to other teams.

In my experience I’ve seen pair programming work in a lot of contexts. But for pairing to last in a lot of contexts, you need to remember that your developers are people. And you need to take care of people first so that they can take care of the product and themselves.

My Obsidian for a Job Search

2021-05-01T00:03:00+00:00

My friends have been great about asking me how my current job search is going. Twice in the last month, people have asked a follow-up, “How are you tackling the problem?”

The answer this time is some new habits, a couple of apps, and Obsidian (duh).

The Short Version

I have a daily routine that looks roughly like this:

Search for Jobs
For each interesting job, write up research notes
Track the status of the job application on a Kanban board (I use Todoist)
For each application, customize a generic resume and cover letter (Google Docs)
Take notes on each discussion and interview
Keep some common interview questions and answers in a note
Keep the Kanban updated as applications happen and change status

Naturally, I take all the notes in Obsidian.

It’s hard to write this up linearlly. This summary wasn’t feeling helpful. If only there were some way to deliver a folder of notes, including some templates, with some linking.

Oh, wait.

But Why Not an Obsidian Vault?

Here it is - a folder of Markdown files that walks you my current job search process. It handles all of the steps above. It allows for lots of personalization. And if you open these files in Obsidian - as their own Vault, or just copy them to your main Vault - then you get even more power out of your second brain.

It’s also a GitHub repo. So issues welcome. Though I will be picky about PR’s. Make this your own.

Cmd-O is My New and Old Friend

2021-05-01T00:00:00+00:00

When I was a big nvALT user, I got very used to a specific set of actions whenever I needed to take notes on something I would…

Cmd-Tab over to nvALT
Cmd-L to get to the search bar
Type the thing on my mind
Scan the search results
Choose a note Edit/Append to the note as needed

Hit Enter to make a new file to empty my brain into

This was deep, deep muscle memory for me. “This feels familiar. Did I ever write anything up about…” then Cmd-Tab, Cmd-L, type-ity-type-ity, and continue on. This was very smooth.

As much as I’ve raved about Obsidian in this series, I was missing this feature.

But it turns out that Cmd-O is all you had to say. It’s nearly the same functionality. The search happens in a modal dialog and gives you the option of navigating down to the note you want or just creating a new note with the search contents as the title.

And yes, Windows & Linux users, Ctrl-O.

I now keep my left pane collapsed by default and only use the search tab for looking for tags.

Just glorious.

History of The Big Pencil

2021-03-21T18:22:00+00:00

Whenever someone asks me about the bigpencil.net domain, I get to tell this story.

In 1965, my dad was a business major, finishing up his senior year at The University of Georgia (ɢᴏ ᴅᴀᴡɢs).

Things were going well for Dad. He had just accepted a job with IBM in Atlanta. He was excited and wanted to share this success with his advisor. So he headed to North Campus and the professor’s office.

Dad was not prepared for the reaction.

“You’re doing what?? “ exclaimed the advisor, eyes squinted and brow furrowed.

My dad replied, earnestly, “I’m going to be working at IBM. Computers really feel like they’re going to be a big deal.”

The advisor continued to stare at my dad. The silence was far from comfortable.

“Really? Do you know what a computer is?” He scribbled on a notepad, ripped off the top sheet, and slid it across the desk, and declared:

“A computer is nothing. But a BIG. PENCIL.”

That note has been in this frame ever since.

Thoughts on Vaults

2021-03-21T16:14:00+00:00

I keep nearly all of my notes in one Obsidian vault. I’ve found that cross-linking and my daily journaling have benefited the most from keeping everything in one directory and thus vault.

I do have separate vaults for dedicated topics of longer form writing. I have an ongoing blog series that has its own vault. And I have another for a topic that may turn into a book.

This series started as one note in the main vault.

I keep the all of my vaults as subdirectories in one directory that lives online for ease of access from different computers and mobile devices. At the moment this is on Dropbox.

Community Plugin: Calendar

2021-03-21T16:14:00+00:00

This plug-in works with the Daily Notes plug-in. It adds a new tab in the right pane that gives a single-month view. Each date links to that day’s daily note, with a cute dot notation for the length of that note. You can also browse back and forth between months.

I find this is faster for zeroing in on historical notes. And sometimes you just need another calendar right in front of you during the day.

Don't Sleep on the Command Palette

2021-03-21T16:14:00+00:00

While I’m still learning how to use Obsidian, I find the Command Palette amazing. Ctrl+P/Cmd+P brings up a searchable list of commands. I find new actions here all the time. And it’s the fastest way to put an Obsidian URL on your clipboard.

Obsidian URLs

2021-03-21T16:14:00+00:00

Obsidian’s URL scheme is fabulous. A link of this form:

obsidian://open?vault=<vault_name>&file=<note_name>

..will open Obsidian with that vault and that note selected.

I use this a lot with my task manager, linking back from projects and tasks to longer thoughts on the topic. I also link to notes other vaults as needed.

With any note open, “Copy Obsidian URL” is available via the Command Palette and the note menu.

Note Taking in 1-1's

2021-03-21T16:14:00+00:00

I have several regular meetings with a handful of people. These meetings are often peer-to-peer or manager-to-person type meetings. Here is how I use Obsidian to take notes during these meetings.

Create the Person’s Note

Before the first meeting, I make a note with the person’s name. Naturally, I use a template for this text.

# {{title}} Background

# Topics

- [ ]

# Action Items

## Me

- [ ]

## {{title}}

- [ ]

Over time, I fill this template in like this:


# Andy Nelson Background

- University of Bedford
- Likes jazz drumming
- Partner is Wendy Lam

# Topics

- [x] Next semester's classes
- [x] LinkedIn Tutorial
- [ ] Career goals review
- [ ] Miles Davis album list

# Action Items

## Me

- [ ] Send goals template

## Andy

- [ ] Write Career Goals

Background is for the anything interesting about them. I usually pull most of this from our first meeting, but I’ve been known to add to it.
Topics is for anything I want to make sure to mention at our next meeting. I use checklists for these.
Action Items gets any tasks we assign to each other during the meeting. Again checklists for these.

Meeting Day

Before the Meeting
- List the meeting in the Calendar section of the today’s note, with the time and a link to this file
- Copy the open Topics from this note to today’s note as an agenda
During the meeting
- Take notes as needed in today’s note
- Mark discussed Topics and Action Items complete
After the meeting
- Brain dump any notes that didn’t make it to the Daily Note
- Mark discussed Topics and Action Items complete in this file, too
- Add any new Topics, Action Items, other background items, or expand this file

Between Meetings

I will add new Topics to the Person’s note as they come up. The Linked Mentions tab becomes a great historical list of our 1-1 meetings.

Using Links

2021-03-21T16:13:00+00:00

For internal links, I use the [[wiki links]] style. This is the default option and I find it’s just faster when I’m typing and am pretty sure there’s a note I want to link to. And if I’m wrong, it’s one click to make that note.

For external links, I have a strong preference for reference-style links in my notes. Given that I spend so much time in Edit Mode, I find this keeps the notes more readable.

I use named references, usually only one word, and stay away from numbers. Why? Because I’m always re-ordering them.

Obsidian renders the references in smaller text, making them easy to ignore most of the time. But when you need them, they are easy to find.

I also like putting them at the bottom of the current note, below a Horizontal Rule, where they are out of the way. I have experimented with putting all of the references at the top of the file, but for now, I keep them at the bottom.

Using Tags

2021-03-21T16:13:00+00:00

I am not a huge tag user. I find the search very good and the Linked/Unlinked mentions tab enlightening.

However, I do have a few categories of notes I want called out. So I will drop a hashtag at the bottom of the file, below a Horizontal Rule, where it is out of the way. I’ve found I like this better than using YAML front matter.

As I type this, I realize that all of the tags I care about are types of notes and thus have templates associated with them. So each template includes the tag. I barely even think about them except when searching.

Core Plugin: Templates

2021-03-21T16:12:00+00:00

I enabled the Core plugin for Templates. This allows you to specify a directory of templates - just Markdown files - for when you create new notes.

You get a new button on the left gutter called Insert Template for inserting templated text.

Create a new file
Optionally name it
Click the Insert Template button

When you click this button, Obsidian asks you which template to insert, shows you a searchable list of all of your template files. Pick one and it’s inserted.

The main option is a directory to hold your template text notes. I have named my template directory _templates_ so the folder pops to the top of my Notes tab. I keep it closed so I barely think about it. I’ve named each template _Template<topic> so they are differentiated in any file list.

There are a few variables it will evaluate, including inserting the title, date, and time, for customization.

And yes, the Daily Journal plugin is fine with the daily template living in this directory.

Core Plugin: Daily Journal

2021-03-21T00:27:00+00:00

I use the Core Daily Journal plugin every day.

This gives you a new button in the left gutter. Click on it and you get a blank, optionally templated, file named with today’s date, for taking notes for well, today. No matter what file you’re in, clicking that button will take you back to this file.

The plugin allows for an optional template, stored in a file as a note. I find this invaluable for jump starting my day.

I use the date format of YYYY.MM.DD, which helps with sorting when looking for a day’s note outside of Obsidian - say, on mobile or via the file system.

As of this writing, my template looks like this:

# Exercise

# Calendar

# Work

---
#dailynotes

I like logging my daily exercise. So that’s obvious. I tend to exercise first thing in the morning, so when I open this note, I log the workouts right away.

The “Work” section is for stuff I did today that needs some context - beyond, say, what’s in my personal task manager. I often will have items here like “wrote some for the blog” or “talked to neighbor about the fence” just to tickle my brain when I come back to this file.

The “Calendar” section is full of entries of the meetings I know I’m going to have, or if spontaneous ones happen, I add them, too. They look like this:

# Calendar

## 1100a Talk with [[Andy Nelson]]

## 0230p 1-1 with [[Kyle]]

I will take bulleted list notes in these sections during the meeting if possible. And anything that I need to dump out of my brain after the meeting, I’ll flesh out the section.

I talk about linking to people in Obsidian 1-1’s.

Driven to Tests

2021-02-02T21:53:00+00:00

What are the signs that your software product team has experienced a big issue during development? Team velocity on new features has slowed down compared to earlier iterations. Tech debt is taking a long time to pay down. There’s a bug in production - or worse, some sort of production outage. How does your team adapt to the situation?

No matter the cause, there is hard coding work ahead. You’re worried about the speed and effectiveness of the team as they work. And you want to finish the task with no risk to the rest of the product.

Are your tests helpful in these stressful situations? Are they just a quality check before shipping? Or are they more integrated into your workflow? How can you increase velocity of new effort, bug fixing, and production fixes? And how can your test suite increase your confidence that recent work is correct while not exposing the rest of the product to risk? How can your test suite factor into Continuous Delivery?

If these questions sound interesting, can I interest you in test-driven development?

Why Test First?

If you’ve never heard or seen test-driven development (TDD), it’s exactly what it sounds like. Developers write a test, run it to watch it fail, then write the implementation code to get the test to pass. Repeat.¹ Over time, these tests build into to a robust suite that provides your product with several benefits.

First is the impact on the amount of implementation code. By writing only the code to make tests pass, you have “just enough implementation” - less code and less complicated code. The codebase will not have unused code paths cuttering developers’ editors and brains. Smaller, simpler code is easier to change in the future.

Yes, there will be a lot of code written in tests. Likely more than what you may have seen in a traditional automated test suite. But a passing TDD’d suite provides a higher sense of confidence than a typical test suite that the product will work as described. That confidence says that when the suite is passing, the team can ship that version of the product.

The tests also allow for change in the future while maintaining that confidence. If tests are passing “near” the work a developer is doing, but failing elsewhere, that is very useful information. You know not to ship until the entire test suite is green. You know about a less obvious code depencency. This is handy when writing new code or fixing bugs. It is amazing when refactoring.

Another benefit is useful documentation. Code documentation is often a separate effort, written after code is complete and shipped. Thus they tend to be, and stay, out of date. In a TDD project, the test suites have descriptive test names and clear implementation that tell the reader exactly how the code works. So the suite serves as documentation that is complete before any code is checked into version control. Test-suites-as-documentation are useful to teams immediately and they stay that way over time.

It’s also possible to build a test suite that does even more. The Jasmine tutorial is end-user documentation that is also a test suite.² This executable documentation has been part of Jasmine’s continuous integration process for years. Jasmine doesn’t ship without knowing that version works for anyone trying the tutorial.

What About Continuous Delivery?

That all sounds great. How does TDD relate to CD?

If we go back to the metaphor of software development as manufacturing line (from The CD Test), then how does a test-driven suite help eliminate wasted effort and keep new software leaving the factory?

New tests reduce effort by forcing you to express how code should work before you’ve written it. This is often easier than imagining the code that will accomplish the given task. It’s like a well-prepared grocery run. You leave home with a shopping list of ingredients grouped by the shops you need to visit to find them. Overall it is faster to do the work at hand.

This is also true for production issues. Once you know how to reproduce a bug, and you express that bug in a new test, it becomes straightforward to solve. The bug may still be complex to fix, but you will know where to start and when you’re done. That test then stays in the suite as long as necessary, preventing regression.

Existing tests do even more to keep team momentum high. Revisiting old code, whether to fix bugs, refactor, or add new functionality, often has a high start-up cost. Reading old code to understand how it works, and why, takes more time the longer its been since you visited that code. But with a test-driven suite, that effort is lower. The how and why is documented right there. If your team is growing, new developers can get oriented just by reading the tests and following the implementation as needed, building concrete understanding of the system. The test suite becomes a knowlege refresher and accelerator for anyone on the team.

Going back to the CD analogy of the manufacturing line, a failing test is like an Andon cord. If there is a failing test in your suite, the product should not ship. The team needs to write code to get all the tests passing again. And once all the tests are green, the manufacturing line can continue. A test suite that is passing and stays passing by default increases the overall confidence in the product.

A test-driven suite removes waste and inefficiencies from the software development process. That leads to getting the team into a regular cadence of releases. And that gives you the benefits you want of Continuous Delivery.

Start by Starting

Getting started is easy. It is as simple as writing code in the “test, fail, implement, pass” pattern. And playing with some refactoring along the way. Where to start is harder.

If you’re a book person, Kent Beck’s TDD By Example is great. If you’re a tutorial type, take a look at James Shore’s Let’s Play casts. If you’re a doc-crawler, try Test Double’s TDD wiki.

If it feels too daunting to start with your product code base, try smaller, isolated problems. There are plenty of TDD exercizes out there (including the Test Double wiki above). Or you can pick up something like Advent of Code, which has fun problems with test data and test answers. When comfortable with your new habits, take them back to your product development.

Justin says in his post The Failures of “Intro to TDD”, early experience will teach you the mechanics. But like any new craft you need lots of practice to get smooth and maximize the benefits. With time, experience, and research you will see TDD as more of a design technique (ps: follow @GeePawHill). Your tests and code will get even better.

Isn’t this Expensive?

In the Joel Test, he talks about having dedicated testers for a software team (he expands on this in a later post). He points out that not having testers in 2000 was a false economy. Finding issues, bugs, and feature gaps earlier in the development process was, and is still, less expensive than letting users find them for you.

Twenty years on, more and more testing is done with code. These tests are repeatable and essential to your product. Moving to test-driven development provides even more benefits. It is an investment in your product that results in lower costs in code design, higher team productivity, and fewer bugs for your customers. When it comes to Continuous Delivery, it can increase the momentum of the team, making it easier to ship early and often.

If this is hard to imagine, Dagmara Szkurłat’s wrote a fun real-world analogy that is a good next stop. ↩
Thanks to Michael Jackson who suggested this at the speakers’ summit prior to the first Fluent Conf in 2012. ↩

Start with Retrospectives

2020-11-30T00:00:00+00:00

You have read The CD Test and it resonates with you. Continuous Delivery sounds great as a concept. Your team is doing some of these practices, but not all. Your list of questions with “no” answers sounds like a good to do list. You know that changing how the team works could improve the team’s momentum. That means shipping new features to your customers more often. And that means increasing business value of your product smoothly and steadily.

But there are many unknowns. You’re not sure how other team members feel about the state of the team, the current process, and how work is shipping to customers. You’ve heard the phrase Continuous Delivery in passing, but the team has not talked about how it could apply. Or if it should. You don’t know how your team moving to CD would affect other teams. You aren’t familiar with each team member’s goals or needs, and how any changes to how they work would affect them. Continuous Delivery means change. And you don’t know how willing your team is to embrace change.

How can you align your team around the need for change, build a culture of process adaptation, and improve your CD Test score? The short answer is another CD Test item - the Retrospective. Let’s talk about how to use retros to build this culture in your team.

Retrospectives

First, a quick intro to (or refresher about) retrospectives. A retro is a meeting where the team reflects on their team’s “story so far.” It is a time to talk about what is working and find ways to amplify that strength. Or it’s a time to talk about what is not working, rank the problems’ importance, and suggest fixes.

If your team does not have regular retrospectives, Nicola Rushton’s quick start guide is a great introduction. The book Agile Retrospectives is full of more advanced retro topics. It has good discussions on how to set the team’s expectations for the meetings, how to facilitate well, and how to process the outcomes. There are also mini-discussions of different retro types and their preparation activities.

As the team builds a set of good retro habits, there will be challenges. Not everyone will want to share their feelings. People avoid talking about hard problems. You may get surprised when a retro uncovers a problem you didn’t realize you had. And there are often initial retros when nobody wants to say anything out of fear or discomfort.

There is no shortage of tips for dealing with common retro problems. How do you help people become more comfortable with talking? How do you talk through the hardest discussions? Or how to combat more general problems? Work through these together and the team will build trust.

Once your team is experienced and comfortable with retrospectives, what’s next on your CD journey? It’s introducing a new mindset about how the team works.

Process as Product

A software team’s development process is as much a product of their work as what they build for their customers. The process is a set of features - design, planning, implementation, deployment - that the team uses to build business value. Like any product there can be bugs, new features, or performance problems. Development tools can change or break. Staffing changes when someone new starts or a veteran leaves. Test runtimes or validation cycles grow too long and features are not shipping on time. Business realities determine new software work for the team. And team realities can suggest process improvements and adaptations.

How would your team receive this idea of “process as product?” Does the team have a similar shared value of continuous improvement? Would they be able to review the CD Test questions together, discuss opportunities, and prioritize any process changes?

Or is this a new concept for the team, just beginning to consider spending effort on their process? Are team members willing to discuss things that are working, and those that aren’t? Are they willing to use those topics as leverage to make the team happier and more productive?

A retrospective focused on sharing this idea, building alignment across the team, and planning process changes will help any team no matter how familiar they are with the concept.

A Retrospective for Alignment

Most retro guides talk about having a prompted or focused retro. There are many agendas to choose from (again, see Agile Retrospectives for some options). A good choice for alignment building is the “Catapult”.

This retro scenario introduces the metaphor of using a catapult to fling yourself over a mountain to a destination. You draw the catapult, the mountain, and the destination all on your whiteboard.

(If you don’t like the idea of being flung over a mountain, there are different options. The Sailboat, The Balloon, and The Mountain Hiker are similar agendas that will work just as well.)

You discuss your current state. How is the team working today? How have you met team goals? How have you met product and business goals? Each key point gets a sticky note and goes near the catapult. Next, you discuss your desired, future state. What new product features are coming? And what business goals will they help achieve? Again, key points get stickies and goes to the board. The discussion here is important. You are building alignment about today and tomorrow.

Once there is alignment, the team lists the risks and challenges that could slow or prevent the journey. These stickies go on the mountain. Then list the helpers or boosters on stickies and put them “over” the mountain. At the end of the exercise, the team should have a shared understanding and a list of process changes (or next steps) for its CD journey.

Schedule this retro as part of the next release kickoff meeting or quarterly review. Find an experienced outside facilitator. Work with them on how best to introduce “process as product” and develop alignment around the concepts. A more aware team may be fine with Continuous Delivery as the desired state. A team which is earlier in their reflective journey may take better to CD ideas as part of the solutions to the agreed challenges. Lean on your facilitator for help navigating this space.

By the end of the retro you should have a good sense of your team’s concerns about the future, what they see as challenges, and their appetite for change. From here, you should be able to build a better team, process, and product.

Building a Retro Habit

Your team now has (even more) experience with retrospectives. And you have at least some team alignment about the process being a product that can get better with iteration. Harness this momentum and make your retros weekly. Keep the discussion and experimentation going.

Keep weekly retros on the team calendar. Introduce more concepts from Continuous Delivery as prompts for discussion. Or suggest them as potential solutions to the problems the team discusses. Process through the artifacts from your Catapult retro. Use different retro formats and new prompts to keep retros lively. Show the team that working together means talking about working even better than they are today.

Retrospectives are the foundation for any great team. They build trust. They surface issues when they are painful. And they allow a team to improve continuously as they deliver continuously.

The Name Game

2020-11-22T00:20:00+00:00

From the “Can you pronounce my name” game/meme from Twitter early fall, 2020. I love this one. —dwf

Davis: 10/10 All day

David (if your name is David): 9/10 You’ll get it

David (1st time): 5/10 Likely a typo

David (>2nd time): 2/10

Frank: 0/10 There is no comma in my name

Dave: 6/10 But it’s never stuck

D: 2/10 Do I know you?

DWF (rhymes with “woof”): 12/10 💪 I have cooked for you

Introducing The Continuous Delivery Test

2020-11-05T00:00:00+00:00

The spice must flow…

Frank Slootman’s Amp It Up advocates removing “slack” (the concept, not the app) from a team to get greatest productivity. There’s pushback that this approach is toxic and/or it leads to colossal burnout. I agree. Low slack is not the same as high productivity. Farhan Thawar shared a list of characteristics or practices that lead to productive teams. I like this list a lot.

I like his list no doubt because Farhan and I both used to work at Pivotal Labs. But I’d like to take a different approach to talking about software team productivity.

Instead of focusing on removing slack from a team, the best way to make a team productive is Continuous Delivery. That is, setting the team up such that new work is flowing from concept to development to production smoothly and often. There is continuous delivery of value to customers. When work ships, it works as expected, it’s reliable, and there is little re-work. The team, by default, is always looking forward to the next new value they can be delivering. There is a low amount of waste during development. When there is waste, the team adapts to remove it.

A Continuous Delivery team looks much like a modern physical goods manufacturing line. From the outside it looks like requirements go in and work - features and fixes - come of out the other side. Inside there are conscientious workers, well-maintained machines, and low inventory. People are improving their workflows for higher efficiency. When there are problems, Andon cords get pulled, and the team stops to fix them. A high functioning CD team has high productivity with low stress. And they are improving themselves all the time.

How close is your team to this ideal?

Joel Spolksy gave us the Joel Test for evaluating software teams 20 years ago. It is still relevant, but this test is of a time of shrink-wrapped, desktop software. Bug databases and source control have become defaults. Most software is web-deployed applications and services.

With apologies to and inspiration from Joel, I give you The CD Test — a list of simple yes/no questions to check your team. The score is the sum of yes answers. A high score doesn’t mean your team is perfect at Continuous Delivery. But if you review your “no” answers I’m sure you’ll find discussion points for improving your team. Here goes…

The Continuous Delivery Test

Does the team have a single product owner?
Is there a single backlog of stories, stack ranked in priority order?
Does the team have weekly planning meetings to review the backlog?
Are stories completed in a day or two?
Do developers take the next story from the top of this backlog?
Do developers test-drive their code?
Is the test suite fast enough that you don’t complain about them?
Is there a Continuous Integration system that runs on every commit?
Does every CI green build get deployed?
Does the Product Owner accept work by exercising it in the product and with low latency?
Is it simple and “low drama” to make a production deployment?
Do features make it to production frequently?
Does the team have daily stand-ups, with a goal of communicating change and unblocking issues?
Do people work on one project at a time?
Is it easy for developers to move between teams?
Are teams composed of team members of diverse skill levels?
Is asking for help, thus leading to knowledge acquisition, seen as a strength?
Is Quality an exploratory function (and not a gatekeeper to release)?
Does the team meet weekly for Retrospectives to discuss team issues, leading to new work that improves team progress?
Does the team complete their action items from Retrospectives?

Does the team have a single product owner?

Somebody has to be The Decider. For CD teams, this is the Product Owner (or Product Manager). Everyone from customer support to leadership should be making their requests for new work. But the Product Owner needs to be the Decider. The Product Owner takes all this input and prioritizes it in the backlog. There will always be give and take on work priority - that is healthy. Often this means deciding what doesn’t get done. Too many cooks spoil the broth and too many stakeholders distracts a team.

Is there a single backlog of stories, stack ranked in priority order?

If work definition lives across many pieces of software and many people’s brains, how does the team know what they should be working on? Once the Product Owner decides and prioritizes work, and the development team estimates it, this needs to be communicated. There should be a single, team-visible, stack-ranked backlog of work to do. Each piece of work - ok, I’ll call them stories. Each story should have enough information so that work can start.

Each story should include use cases, acceptance criteria, estimates, design artifacts, and implementation ideas. The key is to put it all in one place with clear priority order. This makes it easy for the team to pick up the next work to do.

Does the team have weekly planning meetings to review the backlog?

The world around your product is changing all the time. The Product Owner needs to share these changes and how they affect the upcoming work. The rest of the team needs to digest and contribute their views to team understanding. The team needs to review designs and estimate implementation. This helps the team increase their shared understanding of the product. And it helps the Product Owner change priorities. Having this meeting weekly allows everyone to make small corrections and stay productive.

Are stories completed in a day or two?

Long stories take a lot of time to understand, design, and implement. They seem perpetually “almost done.” The work is not yet available to verify and not yet in front of users. Some of the team is working on them and nothing else. Nobody is happy.

Keeping stories small enough to complete in a day or two is the antidote to this problem. The stories are often simpler, making them easier to complete. It is easier for the Product Owner to verify smaller pieces of work, thus confirming progress. The team will build a sense of progress and maintain it. And customers will get more value more quickly.

Do developers take the next story from the top of the backlog?

When a developer completes a story, how do they decide what work is next? If developers aren’t getting their work from the Product Owner, what are they working on? Any other source of work and there is a high risk that it’s not the next most important work to provide user value. And there is very high risk that this other work could provide little value to the product.

If the backlog is well prioritized and well understood, then the next piece of work is obvious. This keeps the team moving forward in a uniform way.

Do developers test-drive their code?

Writing tests for code has become common. Test code has become part of the product. Writing tests that run several times a day helps find bugs early where and when they are cheapest to fix. Joel was spot-on for this one.

But writing tests first is even better. It helps the team write only the code they need to finish a story. This means less waste in the development process. It also increases test coverage because tests define which code gets written. It will be easier to find bugs later when refactoring or when dependent code changes. And in another nod to Joel, well written tests are even better than product specs. They tell “you in the future” how the code actually works.

Is the test suite fast enough that you don’t complain about it?

If the test suite is too slow, then development will be slow. The team will tend to run it less often, lengthening the feedback loop when they add new code. Or even worse, if the test suite takes too long to run the information it provides will be less relevant. You want a test suite and you want to run it often. So keep it fast.

The team can keep their test suites fast with several techniques. They can reduce over-testing (hitting the same code over and over again). They can write only enough tests to keep confidence in the product high. They can parallelize tests. They can use the best hardware. They can limit testing external dependencies (e.g., external API’s). And since test code is product code, developers can refactor it as needed to keep performance at peak.

Is there a Continuous Integration system that runs on every commit?

Even today “it works on my machine” is still a problem. There can be accidental inconsistencies between development systems and production environments. Developers can be working on semi-conflicting changes without realizing it. The product can be working, with test suites passing on one developer’s machine. But that same work could fail shortly after check-in.

Continuous Integration (CI) takes every code commit, integrates the changes, and runs the full test suite. A “green build” means there are no conflicts. This means work can continue. When there are errors, the developers can fix them right away, dropping new work to fix the build. CI keeps the team flowing, preventing some bugs from making it to production.

Does every CI green build get deployed?

A green CI build tells that the team that the product currently works as expected. It should “ship” to a deployment immediately for review. A team which is deploying every green CI build is acting in the spirit of The Joel Test’s daily builds. Less complicated products might do this on a staging deployment. More complicated products may deploy to production, but with feature flags. This prevents users from using new work before validation.

It does not matter where the green build deploys. Regular deployment shows that the team knows how to build regularly and repeatedly. They find issues early. “Early” means after the tests have passed, but before users can exercise it.

Does the Product Owner accept work by exercising it in the product and with low latency?

A test suite tells everyone the current status of the product. CI will catch integration problems, broken features, or bug regressions. But what about that status of new work?

New work should only ship to production when the Product Manager approves it. A staging deployment that has the latest green build of the product enables them to review new work. This is the opportunity to tell developers, “Great work! Let’s get it in front of users,” or, “Oops. This doesn’t work as we discussed. Let’s try again.”

If the Product Owner is reviewing work early and often, it keeps the feedback loops short. Ideally they are validating new work daily. Fixing bugs and problems can happen more quickly. Or, if the work is as desired, the Product Owner can plan for how to get this new work to production and to users.

Is it simple and “low drama” to make a production deployment?

Getting new work to users is one of the most important things a product team can do. It is often one of the most fragile processes: too flaky, too manual, and resulting in extra hours worked.

Instead, teams should treat deployment like a feature of the product. The Product Owner puts deployment stories in the backlog. Development implements them. The product ships to staging and production often. Changes in deployment, like changes in infrastructure or a database upgrade, suggest new stories in the backlog. Deployment is then simple, reliable, automated, and predictable. Customers get their new features consistently. And the team can go home on time.

Do features make it to production frequently (> 1x every 3 mo)? 

Shipping work to customers is important. It should happen often. Completed work that is not yet in production is potential waste. Without feedback from users you don’t know if the work was valuable. Any time that features “sit on the shelf” is time that a competitor could be using to take your customers.

As soon as a feature, or set of features, is practical for users to exercise it should get deployed to production. CD teams aim for deploying to production about once a week. This keeps feedback loops short and tight, delivering value to customers sooner. It also makes it easier for your developers to make changes when there is a bug.

For some products this is hard or seemingly impossible. Teams should drive to the highest frequency possible for their situation.

Does the team have daily stand-ups, with a goal of communicating change and unblocking issues?

Standups have become popular. They can add a lot of value to keep the team up to date. But they can also devolve into long, meandering status reports. Team members zone out, get distracted, and start to complain at retrospectives.

The most effective standups aren’t just quick. They focus on new information. They review changes to the backlog or the business. They discuss important external items (e.g., interview schedules, upcoming vacations). And they stay short.

Standup is a vital opportunity to ask the entire team for help. Using the team’s collective knowledge is good way to unblock hard problems. Unblocking progress is an essential step for a productive day.

Do people work on one project at a time?

Splitting a person across multiple projects is a distraction waiting to happen. Humans are poor at multi-tasking and at adjusting expectations based on task splitting. Letting team members work on many projects at once adds task-switching costs to a project. This is waste. It may speed one team up, but will slow down the others others.

Keeping people working on a single team at a time allows for focus. Focus leads to personal flow. Personal flow leads to team flow. And team flow leads to getting new business value to customers.

Is it easy for developers to move between teams?

If your company is only one team right now, this will not apply. But any larger organization often splits into many, smaller teams. Keeping these teams siloed from each other can slow a product down. Or worse, it can slow the entire company down. Information and expertise stays locked up, reducing organization flexibility. This will slow down all the teams.

Allowing and encouraging team member rotation keeps the organization flexible. Information, knowledge, and techniques will propagate around the organization. When one team is behind, managers can reallocate people to catch them up. And all the while everyone will learn.

Are teams composed of team members of diverse skill levels?

Show me a “10x Engineer” and I’ll show you a single point of failure. What happens when that engineer feels burned out? Or if they have a new work opportunity or a life change? Or if they go on vacation? Single points of failure put entire businesses at risk.

The alternative is to have teams with diverse sets of experience levels. Have the senior people teach the junior people. That turns them into senior people. A team where everyone is learning all the time is a happy team. And happy, knowledgable people do great work. They mitigate risk. They also spot problems and fix them. They keep the product at high quality and shipping.

Is asking for help, thus leading to knowledge acquisition, seen as a strength?

It is not enough to put junior and senior people together on teams. If the junior people do not feel welcome and safe to ask questions, then they will never gain knowledge. This is a risk to your business and your team will work more slowly in the long term.

Hire people who are good at acquiring knowledge. Put them on teams that will immerse them in knowledge and help them apply it. Give the “junior” folks the opportunity to try and fail. Celebrate asking questions. Encourage this behavior at all levels. These teams will grow and evolve over time into productive engines for your product.

Is Quality an exploratory function (and not a gatekeeper to release)?

If your team is waiting for a QA team to tell you the product is ready, you have already lost. Features and bug fixes alike will sit in the queue to production. Everyone will be waiting for some absolute level of confidence that can never come. Meanwhile, your customers are waiting.

Moving the testing “left,” closer to your developers, makes it cheaper to find and fix bugs. A good test suite tells you all you need to know about the functionality of the product. The Product Owner should be able to use the product to decide that work is ready.

“But what about bugs?” Instead of worrying about bugs, think about Exploratory QA. Have people who use the product and find behavior that the team did not expect or test for. Sometimes these are bugs where the product is broken. Sometimes these are new use cases the team missed. Learn more by reading Elisabeth Hendrickson’s Explore It!.

Fixing this behavior is then new work for Product Manager to consider for the backlog. And like any story in the backlog, it starts with a test. And that test ensures a fix or feature that should never regress.

Does the team meet weekly for Retrospectives to discuss team issues, leading to new work that improves team progress?

Problems, issues, and inefficiencies occur naturally on all teams. Things that worked when the team was small start to fail as the team grows. Technology choices don’t pan out. Experiments in process change fail. And there are countless other scenarios.

Some process, techniques, and behaviors are the opposite. They work very well, solving problems and making the team stronger. They should be recognized and further promoted.

A weekly retrospective helps a team identify both problems and successes.

Teams should celebrate every win. This makes team members feel good about their work. It cements good patterns for potential reuse.

Discussing problems also has many good effects on the team. Everyone gets on the same page. It vents frustration and surfaces anxiety. They can discussion potential solutions, building consensus on which ones to attempt. And they can track action items to implement them.

Does the team complete their action items from Retrospectives?

It’s not enough to talk about problems at a retrospective. The team has to suggest and agree on solutions. Then they need to track action items for their implementation. This means having the action items be clear. And each must have an owner and a due date.

The team should review the open action items at the beginning of each retro. This serves as a reminder to the owners. They are helping the team fix a problem and making everyone more productive and happy. When an action item is complete, the team should celebrate!

COVID-19-Inspired Meal Planning

2020-09-24T00:00:00+00:00

Originally published as a tweetstorm, I expanded this one a little to make it more blog-worthy.

—dwf
—

We cook a lot at our house. We see cooking as a life skill and family activity. Being at home full time means we have a little bit more time to cook things well. Excellent!

We mix in some local favorite restaurants - after all, we want them to be open post-Covid - but there are problems.

Meal planning is still meal planning. It’s a bit of a chore. And it’s very easy for someone to complain about repetition, or general lack of variety. What are we to do??

Welcome to How we plan home meals: A software story.

Problems to Solve

What are we meals are eating this week?
Where should we take out from tonight?
What recipes/restaurants do we like? And not like?
How can we reduce repetition of meals/restaurants? (We like variety!)
How can we minimize time spent menu planning or choosing a restaurant?

Iteration 1: Spreadsheet

Duh.

In April I started a spreadsheet with one night per row, recording the date, the recipe, a link to the recipe, and if we liked it. If we brought in from a restaurant, we recorded the restaurant and a link to their menu.

This was just simple columns. I did some conditional formatting for restaurants vs. home-cooked. But nothing more than this.

When it was time to plan food for the week, I’d look over the sheet and pick 4 or 5 meals, prioritizing time since last cooked. I’d add 1 or 2 new recipes each week (from various sources online).

Similar to recipes, when we decided we wanted to eat out, we’d look at the top of the sheet and look for restaurants we’d not visited in a while.

This solved most of these problems, but the planning time could have been better.

Problems to Solve

We were guessing on age since last cooked or last visited. This was not calculated.
We were not counting the # of times cooked/visited.
Scrolling up and down spreadsheet rows make it hard to see history.
We discovered we would like to rate recipes and restaurants - we wanted to be able to choose not just based on most-recently-eaten

Iteration 2: AirTable

People had been suggesting AirTable for a while - the “NEVER use a spreadsheet to solve a software problem” crowd. 🙄

I definitely have built my share of spreadsheets that slowly acrete functions, and SQL-esque queries, and yes, even JavaScript. And I also hate myself. And the [sunk-cost fallacy][sunk].

Spreadsheets are great. Until they aren’t. Which meant for iteration 2, I was going to try AirTable.

Here’s a rough sketch of the implementation. I tweaked it regularly to refine, but here’s where we are today.

3 Tables

Restaurants: Name, Rating (1-3 stars), Notes, Count of times linked to Meal, Roll-up to get most-recent-visit
Recipes: Name, Rating (1-3 stars), Web Link, Notes, Count of times linked to Meal, Roll-up to get most-recent-visit
Meals: Date, link to Recipe (can be multiple), Link to Restaurant (can be multiple)

A Meal is something we ate on a specific date - really just a join, but AirTable calls them “links”. Each day can have multiple meals and/or restaurants.

AirTable has Views, which are sort of like database views. But they are also the visual UI for working with a given table.

Meals has a Calendar view. This is amazingly cool for planning. Meals has a web form for recording what/where we ate when, complete with calendar widget. Restaurants & Recipes each has a web form for a new record.

Restaurants & Recipes each has a “Choice View” that sorts first by least-recent-meal (ascending “roll-up” by date), then by rating (descending star count).

All views hide any extra fields that don’t help with the interaction at hand.

Restaurants vs. Recipes

We wanted to be able to tell when a meal was home-cooked or a restaurant visit. The idea was to be able to scan and see how much we were eating out.

So the “name” field on Restaurants is now a formula that prepends 🍱 to every restaurant name. It’s now very easy to scan the calendar. Unicode FTW!

Multiple Meals in a Night

One night we couldn’t decide on a pizzeria. It was getting knock-down, drag out. So we ordered from both. Another night the kids wanted to make a Taco Bell run. So my wife and I decided to cook something we knew they hated.

AirTable allows for multiple links to the same date. Meals now allow multiple-links records to Restaurants & Recipes. This gave us the bonus of being able to start tracking multiple side dishes on cooking nights.

🌮TACO TUESDAY

We decided to embrace Taco Tuesday! Every Tuesday we have Taco Night, which means wanting to track recipes that are taco-friendly.

We added a Taco? yes/no “choice” field to recipes. And we added a Taco-specific view for when we wanted to pick this week’s taco proteins. And since we’d learned about Unicode support, Taco recipe names are always prepended with 🌮, based on the field.

A Restaurant Closed?!

Oops. COVID-19 claimed a business. We added a “Closed/Avoid” selection field that allows us to filter the closed places out of the Choice view. And we can use this if we have a lousy experience at a restaurant.

Summing Up

Planning is now a simple pass over the Choice Views, looking at what we like but have not done recently. Pick 4-5, iterate. Option anxiety reduced. It’s also easy to answer, “DIDN’T WE JUST HAVE THAT?”

What did I learn? A spreadsheet is a great place to start with any new app that is data-related. There’s a threshold you cross where project’s complexity begs for multiple tabs, conditional logic, and gasp queries. When you see that threshold, taking a look at Airtable, or maybe the newly announced Google Tables¹, is a good investment.

Edited 2026.06.23, Google Tables, an experimental competitor to Airtable, was shut down at the end of 2025. ↩

Improving Standups

2020-09-22T00:00:00+00:00

Your software development team has a daily standup meeting. How’s that going?

Teams at Pivotal had a fairly common agenda for standup. We reviewed Helps (“I need help with…”), Interestings (“Yesterday I learned that…), and Events (“What are we doing besides coding today?”). We rarely got into the details of specific work unless there were team-wide concerns or if someone had a specific question.

Keeping to this agenda daily and keeping the ritual alive over time takes effort. We wrote a Rails app to help. Over time we realized that there are additional, some emergent, problems that cropped up during our meetings.

Standup Problems

I have seen many problems in common across several teams. Some are just informational:

What is the agenda for today’s standup?
When I find an item to discuss at standup tomorrow, how do I not forget to bring it up?
Which events are coming up in the next week?
Who’s on vacation and when?
What stories (or epics, or tracks) is the team working on? Which pairs are working on them?
What are today’s pair assignments?
What are the active action items I have from team retrospectives?

Others problems are more structural:

There is enough structure to make the meeting productive, but not so much to be a burden.
The ritual is lightweight enough to allow for iteration, but solid enough to perpetuate.

Having some sort of physical board present in a team’s area can work well. Put this information in one place, use it to drive the daily ritual, and it lives the rest of the day as an information radiator for the team and the rest of the office. Some Cloud Foundry teams even had Fuji Instax photos of each team member and moved them around to identify pairings and which stories they were implementing.

The physical board breaks down as soon as the team has one remote team member. Given how COVID-19 has made so many teams fully remote, what can we do?

We move the board online.

(Credit to the Cloud Foundry CAPI team - a semi-remote team even in 2018 - where I first saw this solution.)

Setup

What you need:

An online kanban board
Digital avatar photos of each team member
A standup agenda.

I’m going to use Trello in this example, but things like Miro or even a Google Spreadsheet can work. For the agenda, I’m going to use Helps/Interesting/Events/Pairing Assignment.

Then you should set it up like this:

The Left Column is “Standup”; it holds divider cards for each agenda item.
The next Columns are just labeled “Track”; we’ll have 2 tracks in this example
The next column is the “Pairing Parking Lot”
Lastly, a column for the “Retro Action Items”

Now you can add one card for each team member to the Pairing Parking Lot. Your board should look something like this:

Everyone on the team should have access to this board so they can add cards as needed during the day. Peer teams can have read or read/write access, depending on your organization’s norms.

Great! How do we use it?

Standup

Standup is probably in a video call with everyone present. The emcee can now share their screen, reviewing cards in the lefthand column, top-to-bottom. Trello allows for a single keystroke to archive cards as you finish them.

Pair Assignments

After the lefthand column is complete the team can discuss the day’s work. This means shuffling and assigning pairs to the available tracks of work. Whatever your pair rotation strategy - daily splits, track anchoring, etc. - your decisions should be reflected by moving the cards around.

The “Pairing Lot” is a place to put people’s cards when they are away from the team. Or while you decide assignments for the day (hence the example’s dividers for available/unavailable).

Retrospective Action Items

This board can also be a place to track and remind team members about their action items from retrospectives. Even if you use software to run your retros, this board can surface them daily. Action items that stay top-of-mind are more likely to be completed.

As Things Change

In full use, the board will look more like this:

(or play with the live example)

After standup, changes that affect the team should lead to updating the board.

Vacation on the horizon? Or team rotation about to happen? Add a card to Events.
Find a help or interesting during the day? Add a card.
New team member? Add their card. On their first day, use the exercise of adding their photo to verify they have access and then explain the ritual.
Starting a new story or track? Rename the Track column. Or even better? Add a card.
Another team member action item pops up? Add a card.

Iteration

A developer may look at this and see opportunities for automation. Or maybe frustration that software isn’t choosing pairings, or tracking frequency of certain pairs…

This is just a starting point. It is ripe for innovation and iteration. Knock yourself out. But be careful of the yaks you choose to shave - it’s just for five minutes out of your team’s day. Do what works.

Foo Fighters 1995

2020-09-15T00:00:00+00:00

A tweetstorm I wrote up in September, 2020, from being at a very early Foo Fighters’ show. I’ve migrated this away from being embedded tweets because Twitter/X. Refreshed with links and some text formatting. –dwf

My Amazing Rock’n’Roll moments. Next in a series.

Foo Fighters first tour. May 13th, 1995 (or maybe the July 24th show, I’m not sure¹). Portland, Oregon. La Luna. An amazing venue that doesn’t do rock shows anymore.

But an absolutely AMAZING mosh pit. Near the end of the show, the band took a short breather to change guitars or something. And we all stood there, spent.

Then they started “Exhausted.” But nobody was dancing. We were all too tired.

I was in the middle of the room at the edge of a big circle of people. I wanted to keep the excitement. After a 30 seconds or so, when the song is still slow, I tucked in my arms and started rolling slowly to the tempo off of everyone along the inside of the circle.

Just me became two or three, then four people. Not crazy dancing, but just building. And building. Just like the song. By the crescendo we had the pit at full strength. That song felt like it went on forever. Just forever. It was so special.

Fast forward about 4 months. The tour is wrapping up. And I’m listening to a Westwood One radio interview with the band on a Sunday night. They start taking calls from the audience. “What was your favorite show during this tour?”

Dave Grohl - “It’s just so great to be on tour. That we’ve picked up fans and they’ve embraced us. And the shows everywhere have been beyond what we’ve could have ever expected.”

Then one of the other band members jumps in. “Remember that one night in Portland? When they started moshing? To f’n Exhausted ? That never happens.”

Dave - “Oh, yeah. So cool. By the end of the song that was nuts. What a show!”

I think it was likely the May show, as I recall it being short. Regardless, it makes a better story with Exhausted being the last song. ↩

Drake Your Job

2018-08-03T00:00:00+00:00

I originally published this on Medium in 2018. I’ve made a few copy edits, but the content and intent is the same.

–dwf

How much do you love your job? You can measure some job characteristics like salary, benefits, or round-trip commute time. But what about the intangibles? Do you have a good manager? Are you learning? Are you happy? Measuring these abstract things seems hard. But if we can’t measure it, how can we improve it?

What if you measured everything about your job and calculated an overall score? You could pick an area to change to improve your score. Or you could compare the score of your current job and new potential jobs. How can you turn this mix of concrete numbers, fuzzy feelings, and abstract concepts into useful information in order to help direct your career?

The Drake Equation

At the birth of SETI.org, Dr. Frank Drake decided to estimate the number of potential planets with intelligent life. And while it’s a very rough estimate, the birth of the Drake Equation calculated a number that inspired scientists to reach for the stars.

What does this have to do with valuing your job?

Michael Dearing of Harrison Metal has a video and a blog post that shares the Drake Equation’s history and suggests that it’s applicable to any seemingly unquantifiable thing. What would it look like to build a “Drake Job Equation,” taking into consideration all the components of your job? Let’s give it a try.

What’s Important?

First, list which job characteristics are important to you. Take some time and make a list. Here’s what I came up with, based on what matters to me:

Compensation
- Money matters! Base salary, any bonus, stock options or shares, and benefits.
Organization
- How do I feel about the company or organization? Do my leaders inspire me? Do my values align?
Manager
- Is my manager helpful in my career? Do we meet regularly? Do they listen? Do they explain the “Why?” as well as the “What?” when I have questions?
Peers
- Do they have my trust and respect? Do I have coffee and lunch buddies? Do I invite them over for dinner?
Commute
- How long do I spend getting to the job every day? Does it leave me enough time for the rest of my life?
Office location
- Is the office in a place that inspires me daily? Is it easy to schedule time with my friends and mentors at other companies? Can I run errands at lunch?
The company’s main product or service
- Is my work going toward a product or service that I admire or respect? That I would use regularly? Would I be proud to recommend it to friends and family?
The work
- Is the work I’m doing energizing to me? Is it fun? Is it interesting?
Opportunity to learn
- Are I adding to my skillset? Is this job going to help me get the next one?

The Drake Job Equation

Now that you have a list of potential terms, you need to combine them into a Drake equation. Let’s assume that the compensation is the base of your score. The other terms should increase or decrease that to get a final value. The bigger the score, the better the job is likely to be. A lower score should be a catalyst to do some deeper thinking.

Combining my terms, I got this formula:

  Value.job = 
    C × 
    Q.org × 
    Q.mgr × 
    Q.peers × 
    Q.commute × 
    Q.location × 
    M.product ×
    M.work × 
    Freq.learning

C is compensation. Start with cash salary. Add any bonuses. Add the net gain from any stock options, RSU’s, or an employee stock program. And don’t forget any other benefits — health insurance, discount on public transit, gym memberships, etc. Add those in. If you don’t have exact amounts for any of your compensation numbers — for example, your company isn’t public yet so you can only guess about stock options — just make a reasonable estimate.

The next group of other terms is for “quality” terms. Each of these terms needs a range of values that reduces or amplifies the score based on each term’s impact. I recommend a range of 0.1 to 2. A value of 0.1 maps to “poor quality” or “this part of my job sucks.” This low term should be pulling down the overall job score. Why 0.1? Because we need to talk about zeroes.

Any low term will pull your score down significantly. Be honest as you ask yourself about each term. But a zero will make the entire score zero. If your final job score is zero, it’s probably time to open up LinkedIn. A value of 0.1 serves as a nice floor that sends the right message without screaming “QUIT” and rendering the rest of the formula moot.

A 1 means things are “just OK”. A value of 1 doesn’t reduce or amplify the overall score. A 2 means that things are “GREAT”, meaning it doubles your overall score. And keeping the ceiling at 2 means that any one term does not overwhelm the rest.

Q.org- Quality of Organization
Q.mgr- Quality of Manager
Q.peers - Quality of Peers
Q.commute - Quality of Commute
Q.location - Quality of Location

The next group is for motivation scores. Motivation matters a lot to me, so I decided on a higher ceiling of 5. Finally, I want to be wokring at place were I am learning new things. This feels like a motivational item. So these three have a range of 0.1 up to 5.

M.product - Motivation from Product
M.work - Motivation from Work
Freq.learning - Frequency of Learning Opportunities

Using The Equation

Let’s say your total compensation comes out to $50,000 this year. You love the company, have a pretty good manager, fine teammates, a soul-sucking commute, a great location, you love the product, don’t love the work you’re doing, and you are learning a reasonable amount. Using my equation, we get this:

50,000 × 2 × 1.25 × 1 × 0.25 × 2 × 3 × 2 × 3 = 1,125,000

Now you have a score. What do you do with it?

Just doing the evaluation is helpful. You now have a list of items worth investigating for improvement — those things that are preventing you from getting more out of your current job. Your commute is awful, but you love the location. Can you carpool with some coworkers? Is there a public transit option? Your current work isn’t exciting, but you like the company and product. Can you bring this up with your manager? Could you transfer to another job in the company?

But if the areas for improvement are all outside of your control, maybe it’s time to look for a new job. Evaluate each term as you interview at a company and score each opportunity. Compare these scores as you go.

You won’t have an offer yet, but you can play with the compensation value. What compensation amount would make you take an offer? Haven’t met enough of the team to make a guess about the peers? Schedule a follow-up lunch. Curious about other aspects of the new job? Use your network to find folks who can help you refine your estimates. There will be lots of ambiguity. Resolve what you can and estimate the rest.

Getting to a score is a tool, not a conclusion. So get as close as is practical that helps you answer the question at hand. Do you love your job? How about your next one?

What are the components of your Drake Job Equation?

Goodbye Chris Cornell

2017-05-18T00:00:00+00:00

Originally written as a tweetstorm after Chris Cornell’s suicide. I’ve expanded a bit more here.

—dwf

I have a Soundgarden story.

July 1990, London. I was helping chaperone some high school students on a canonical “American Suburban Kid European Tour.”

It’s after 11pm. Everything is closed, so we headed back from wandering around town to sit and talk in the lobby lounge of our nice, but much older hotel.

We’ve been there for some time when the window to the street slides open. Four dudes, dressed all in black, and very drunk, crawl through and plotzed down in the chairs next to us.

It was Soundgarden.

The kids didn’t know them. I barely did. A hotel clerk walked over and closed the window, saying a very English nothing.

I talked music with Kim Thayil for a bit. “Athens. We played Athens. I love Athens. Come see us next time we’re in town.” Sadly, I never saw them live. In Athens or elsewhere.

All I remember of Chris Cornell was the hair. And the high school girls crushing just by staring into those eyes (when they were open).

After some time a different clerk came and shooed us all out of the lobby.

But I became a fan. Badmotorfinger and Superunknown are great albums. That voice. His Ave Maria is key to my Christmas playlist. I miss him even though I’ve not liked his more recent songs.

Love your friends. Especially when you know they hurt.

This is what #adulting is all about.

Goodbye Jonathan Demme

2017-04-26T00:00:00+00:00

Originally written as a tweetstorm on the death of Jonathan Demme. I’ve expanded a bit more here.

—dwf

I wasn’t much of a Talking Heads fan. I liked them fine.

I appreciated their quirky melodies & artsy videos in contrast to the rest of early MTV.

That changed when the older brother of a friend heard I had not only a stereo VCR, but an amplifier and a tape deck that connected to it.

Thus I found myself one Friday evening taping to cassette (you know, chrome-age “ripping”), the entirety of Stop Making Sense with someone I didn’t know that well. By the end of the night, we were both excited to have one of the best rock concert movies that we could play just about anywhere.

Likely watching Storefront Hitchcock tonight.

RIP and Thank You, Mr. Demme.

How One Funny Email Almost Changed My Career

2016-10-06T00:00:00+00:00

Originally published at Medium. But really originally published on Palm’s Outlook server in 2006.

–dwf

In 2006 it was time that I left my job at Palm. When you leave a job in tech, you send a goodbye email.
When you think yourself a fun and funny co-worker, you want that email to be hilarious.

Subject: Where’s DWF? as of 6/2

Thanks so much for your good wishes in the past week. Many of you have asked what I’m going to be doing after Palm. You know I can’t keep a secret from you guys for long. So here are the details.

I’ve decided to get out of the crazy world of high tech and pursue success in the low-margin, high hype world of the beverage industry. It won’t be long before my line of energy drinks, GoJ (pronounced “GO-J”), hits the shelves of your local quickie-, super- or mega-mart.

GoJ will start with good old-fashioned orange juice and finish laced with the FDA maximum concentration of caffeine. None of this high fructose corn syrup water that tastes like it might have been mixed in an orange container. Instead we’ll use only the best fresh-squeezed, flash-pasteurized juice mixed with the purest caffeine molecules our state-of-the-art laboratory can synthesize.

Our first target customers are the Slashdot crowd, selling them on the fact that one 500ml bottle not only has the equivalent caffeine to 6 cans of Jolt Cola or 4 cans of Red Bull (or just one cup of 2nd-floor Peet’s coffee), but it also meets their recommended daily allowance of fruits and vegetables. The web ad campaign, with our initial slogan “I’m on it!”, will run late summer featuring none other than Palm’s own .

Next we’ll move to a broader customer base with other flavors, including lemonade, strawberry-kiwi, and a proprietary blend of guava, apple, starfruit and pineapple we’re calling “GoJ-GASP.”

At some point we’ll need a sports celebrity spokesperson. I’m thrilled to announce that we’re in negotiations to sign Barry Bonds. Millions of kids will drink GoJ and declare under oath our new slogan “It’s the juice!”

Keep in touch. I’m on LinkedIn. And if ever want to make the leap out of tech, we’ll be looking for people to drive the delivery trucks.

–dwf

I got the laughs, kudos and high-fives I wanted. And I went on with my last days. But funny emails get forwarded.

My last few days were spent explaining the joke. And the references. And the fact that no, I didn’t have any juice you could try. Or delivery trucks. Or job applications. Or a job.

Three weeks later, I got a call from a friend-of-a-friend-of-a-coworker who was a beverage distributor in LA. I think he wanted to invest.

Eight years later, at a Handspring/Palm reunion party I was talking shop with someone who received the email directly that day. She looked at me, squinted and asked, “Wait. I thought you were making juice?”

I guess I do.

How We XP: Develop & Deploy

2015-11-01T00:00:00+00:00

Originally written (and posted to Medium) for a friend who while around software for a while, and seen some agile teams, he was just starting to manage one directly. So I wrote up this bit of a braindump.

–dwf

I’m not going to go into planning. That’s another post.

I’m going to assume that stories — the work that is to be done — are broken into small, individually-acceptable pieces of work. “Acceptable” means that a Product Manager (PM) can verify that developers did the agreed work completely. Keeping the stories small means it is easier to make progress, show progress, and validate progress.

I’m also assuming that the team is using git for version control, the developers are test-driving their code (Test-driven Development or TDD), the current test suite is green on a Continuous Integration system (CI), and that we are talking about a web app.

Development

All new development is done locally
Developers pull up to latest code and run the test suite. It should be green. If not, fix it and push back to origin before continuing.
Developers test drive the new feature.
When developers think the story is done based on the acceptance criteria agreed-upon with the PM, all tests should be green and the new functionality should be working as expected.
This is a great time for developers to find refactoring or dead code and clean those up as well. Tests should stay green.
Developers now pull up to latest code from origin in case another part of the team has pushed new code. If there are changes, merge and run tests again. If there are broken tests, fix them before continuing.
Optional: some teams like feature branches. They should be short-lived and merged/re-based with master often. The longer your changes stay away from master, the more tech debt you create. Merge early, merge often. Git-flow is a pretty good pattern here.
= Once all tests are green locally and all changes are ahead of origin master, push to origin.

Continuous Integration (CI)

Your git repo should have a post-commit hook that starts a CI build. On every push to master, no exceptions.
This is a full test run on a server that looks as close to your Production deploy configuration as possible.
This can be a long pipeline. Run unit tests, run integrations and functional tests and regression tests - whatever. Often your pipeline will have later stages that integrate multiple apps/services and cross-browser tests. Make sure you are getting feedback at all stages of the pipeline.
When CI is green all the way, you should have a CI step that deploys to Staging. This should happen several times per day. And if it’s automated, even better.

Staging (aka Acceptance)

Staging is a deployment (or environment) that looks exactly like Production, with the same versions of OS, database, etc.
Staging doesn’t need to serve the same capacity — so maybe just a primary database (no replication or secondaries ). Since there will be fewer active users, you probably only need one app server.
Optional: If your system has feature that needs acceptance that requires master/slave DB or app server failover, then by all means install/configure Staging to match that configuration.
Everyone should know when a Staging deploy is done and what new work it contains. Notify the team via email or an information radiator.
PM should verify work is done as expected on Staging. Validating that work is complete is PM’s role, Not QA.
If work is not satisfactory, Reject the work. Rejected work should be started before any new work is started.
If something is broken that used to work, PM writes a bug. PM should prioritize bugs against other work. That is, a given bug may not be as important to fix as some new feature stories.
QA should be looking at Staging to find bugs, problems, regressions, or any inconsistencies. They should be using the product as a user would and should be looking for behavior that engineering and product didn’t find. This is often called exploratory QA. You should have your QA lead read the book on Exploratory QA, Explore It! written by my good friend and colleague, Elisabeth Hendrickson

Production

Production is where the shipping product lives.
When all stories on Staging have been accepted, PM should consider pushing that version of the code to Production.
If users will have a bad experience due to known bugs or an incomplete user experience, then don’t push to Production.
Push to Production has often as possible.

Other Deploys

This list is by no means exhaustive. But small-to-medium-sized projects tend to have one or more of these deploys.

Patch Staging

A copy of Staging, meant as a short-lived deploy just to verify a hot-fix. Using a second Staging means mainline work doesn’t slow down while a patch is developed and shipped. Same rules as Staging apply, but tear this deploy down as soon as the fix is deployed to Production.

Demo

A copy of Staging, meant as a short-lived deploy for specific set of users for a short amount of time. Like a PatchGood use cases for a Demo deploy include demoing to potential investors and user research. Tear this deploy down as soon as the demo is over or when feedback is no longer required.

Integration/User Acceptance Test

A copy of Staging that can be used for more real-world testing. This usually stays in line with the normal Staging and is used for CI pipelines. Some examples are performance testing, smoke testing, and long user scenario testing.

Not Scary

Continuous Delivery

What’s described above is Continuous Delivery; insist that code always move in one direction: towards Production. Also, set up the whole team so that it’s easier to move towards Production and release often.

Production Patches

Here’s how you should deal with hot-fixes:

Developers branch from the last Production deploy version.
Developers fix the bug.
CI runs against this version of the code to ensure no regressions.
Developers deploy this version to a temporary Staging environment.
PM accepts the fix.
Developers push this branch to Production.
Developers merge the fix back to master.

Sometimes Scary

Continuous Deployment

Some teams are able to push every green build to Production. I’m not going to cover this in this post. But there is additional process and code needed to support this.

Scary

These are the things you should truly worry about.

Long duration test suites

When your test suite or CI pipeline is too long, you run the risk of completing work faster than you can verify it. If you keep locally-run suites fast - say, under 10–20 minutes - then your developers can get features to staging multiple times a day. Long CI pipelines are less awful. For example, a web app can have a multi-stage pipeline that puts an all-browser compatibility suite near the end. Main development can continue based on success of unit tests, collaboration tests, and one browser stage of tests. Then the multi-browser suite can run once or twice a day, and browser-based problems from that stage can be bugs that get prioritized against other work.

Long-lived branches

Like long test suites, this can keep your feedback loops long. In this case you have bigger risks around the rest of your app changing faster than the branch’s work can keep up. If it’s taking too long to merge something back to master, then consider that the work should be broken up into smaller bits or should be started when its dependencies are more stable.

Complex merges

This usually comes from long-lived branches. Keeping all development on master helps to minimize this.

Manual deployment

…either manual configurations and/or inconsistent deployments from Product & Staging. These are often related. You want Staging, Production, and any other deploys to have consistent configuration. Bugs due to different code interpreters, databases or OS libraries are very subtle and guaranteed time wasters. Invest in DevOps or a Platform-as-a-Service (PaaS) in order to keep your deploys consistent and repeatable.

Breaking the Stream

Code should always move towards Production. It increases the confidence in the code and keeps regressions low. When a fix is made downstream of development, then it’s harder than normal development to integrate that fix back in development.

Bose Customer Service

2011-04-21T00:00:00+00:00

I have a set of Bose QC2 noise-canceling headphones that I use every day. I bought them in late 2007. They don’t make my BART commute quiet, but they make it tolerable enough that I can listen to music and podcasts.

Within the first year the wire started to fray a little. I walked into the Bose store in San Francisco and without blinking they gave me a new cable. Very cool.

Another couple of years later and my sweaty, bald head had caused the ear cups and cushioned underside of the head band to wear out. I ordered new ear cups ($35 + shipping) from their website and made a mental note to call tech support about replacing the headband when I had a chance.

Yesterday two things happened. First, the new ear cups arrived. And then I noticed the cable fraying again in the same spot - the insulation is pulling away from the jack. So now time to call and ask what should I do. I want to order the cable and the headband.

Tyler has the rep who got my call. The cable was $15 and easy to replace. But the headband piece was not replaceable.

“So you’ve already spent the $35 for the cups. I can certainly send you a new cable, which is $15 plus shipping. But we’re certainly sorry that you’ve had these issues with our product - they really should last longer than the few years you’ve had them.”

I replied that the electronics are fine. But my glands are just outside Bose’s tolerances.

“Here’s what we can do - for $89, plus tax and shipping, and $5 to ship your old headphones back to us, I can send you our newest equivalent model. We just released it in the last year or so and they’re lots better anyway.”

Really?

“Really.”

Do I need to ship them right away? I’d like to use them on a business trip next week.

“No problem. Print the mailing label when you get it in your email - it expires in a week. But the label is good for 90 days. If we don’t receive the old set by then, we’ll cancel the whole thing. So drop it in the mail when you get back. You’re all set. I’ll transfer you to our RMA department to send back the ear cups you’ve already received. No problem.”

Damn.

I didn’t get the second rep’s name, but he finished the transaction before I could tell him the whole story.

I’m going to give Bose the benefit of the doubt that they’ll refurb what they can and dispose of the parts they can’t.

Meanwhile? Super Happy Customer.

Kids & Teachable Moments

2011-03-24T00:00:00+00:00

Back to Work’s episode Little Velvet Hands was in my head the other day, so I used it in a teachable moment with my son.

We just took a family trip to Phoenix, AZ to catch a little Giants Spring Training (Merlin, that’s cricket). When we fly, we always have the kids, 5 & 9, pack & pull their own roller bags. All the better to teach how to navigate the whole airport thing. Plus ownership & responsibility is good, right?

This trip they wanted to check their bags. BAGS FLY FREE on Southwest, so why not?

On the flight home, we land at SFO and they lost my son’s bag - he’s the 5 year old.

The boy, “WHOO-HOO! I DON’T HAVE TO UNPACK TONIGHT!”

As awesome as that is, that’s not what I came here to tell.

I stood in line, behind bunches of the seriously grumpy people, I gave my phone number, I got the receipt. And we left.

When Southwest called the next day to say the bag had been found they gave me the option to have it delivered or come pick it up and get a $50 voucher . We live close to SFO so I chose the latter.

I chose to bring the boy. It was, after all, his bag. And he needed to pick it up.

So we drove to the airport. I gave him the receipt. We walked in to the empty Southwest baggage claim area and to the baggage desk. Behind it was one woman who was clearly on a soul-sucking phone call in the middle of a soulless shift.

When that call ended, she looked at me and said, nearly perky, “Can I help you?”

And then a 5 year old boy pointed to the corner, handed her his receipt, and said, “That’s my suitcase.”

She beamed. “Well then let’s get it checked out so you can take it home.”

Clickity-clickity-clackity.

The boy, “What happened so that my bag wasn’t with everybody else’s bags?”

She, grinning, “Somebody forgot to put it on your plane. But we put it on the next one.”

The boy, “You mean from Arizona?”

She, “Yep.”

The boy, “Ok.” "Thank you."

I sent this to not only to brag. But because what we talked about all the way home is how the lady was there waiting for him and that he made her smile while she did her job because he was nice.

And that most people she dealt with during her day were mad that their luggage was missing and were NOT NICE to her. So it was important to remember that it was helpful to be nice when you can because it makes people smile.

That and he had to unpack his bag when we got home.

The boy, “Oh, MAN!”