Detached Head

Continuous Integration Values (in 2017)

Wed, 28 Jun 2017 00:00:00 +0000

Continuous Integration is one of the main practices in the Extreme Programming landscape, although through the years it has transcended beyond XP to become one of the most common cool things to have in your development team. Due to the availability of numerous and versatile automation tools (Jenkins, Travis, Go, AppVeyor, Bamboo…), continuously building and testing your codebase is just one of those things rapidly becoming a commodity more than a rare thing or a luxury good in the software development world.

But continuous integration is much more than having the latest automation gadget in place. The concept comes from the idea of constantly releasing internal versions of a whole system, with the aim to discover problems early and reveal design flaws. Kent Beck incorporated this idea, originally formulated by Grady Booch, into his collection of XP practices. The definition of continuous integration, as written in eXtreme Programming eXplained (1999) is the following:

Integrate and build the system many times a day, every time a task is completed.

While, again, this seems something common in our present day, think about what it could mean twenty years ago. Incorporate the following thought: CruiseControl, the first CI tool, was released years after the C3 project, and it probably wasn’t nearly as functional as today’s automation tools. This means that in the early days of continuous integration it wasn’t about the tools… because there were no tools (well, nothing remotely close to Jenkins or GoCD, that’s for sure).

I understand XP isn’t perfect, but simply implementing it as described in 1999 would improve so many teams.
— Steve_Hayes (@Steve_Hayes) 16 de noviembre de 2016

In its essence, CI is more about integrating ideas and obtaining rapid feedback. Overwhelmed and amazed by the shiny features of the latest automation product, we sometimes forget this. After all, Communication and Feedback are two of the four values of XP, and the big sign in your wall says people and interactions over processes and tools, right? Unfortunately, values are always harder to sell than practices, and it’s definitely much easier to sell tools than principles.

Start with Version Control and Trunk-Based Development

Good account of Continuous Integration (aka Trunk-Based Development) https://t.co/Ay1NmN3Q5q
— Martin Fowler (@martinfowler) 17 de junio de 2015

Continuous integration does not even begin with the automation provided by tools like Jenkins or Travis. It definitely starts much earlier, when we put version control systems in place and share our changelists with the rest of the team as part of our daily work.

Using version control the right way is the foundation for good continuous integration practice, no doubt about that. Consequently, there has been a lot of effort in discussing and defining the correct workflows or branching models (e.g. when working with git repositories). This is precisely motivated for the need of integrating changes the best way possible.

Anything with this picture isn't a best practise. Trunk-based development FTW // @jezhumble https://t.co/Xk2ZmKwZQw
— Dan North (@tastapod) 1 de noviembre de 2016

It is very interesting to see how the experienced XP and continuous integration practicioners advocate for trunk-based development, in which integration is not deferred and conflict is not avoided. This is, again, much harder to sell than branch-a-lot, Pull Request-based workflows, specially when all these different tools and products are packed with lots of features related to branching models.

after 10 years of giving talks on this, trunk-based development is still the idea that causes most controversy today! pic.twitter.com/5EK7LQgvkh
— Jez Humble (@jezhumble) 17 de octubre de 2016

However, while the trunk-based development stance is generally depicted as an opposition to gitflow, it is important to remark that pull requests and feature branches are perfectly compatible with trunk-based development. It’s only the frequency of integration in the mainline what matters. According to this, situations in which days, weeks (or even months) pass until feature branches are merged into master should be avoided.

The key to a good CI foundation is to embrace the commit early and often mantra, but also merging to the mainline as soon as possible, everyday if we can. Good continuous integration involves implementing toggle mechanisms and making the design abstractions of the system cope with different features or lines of work.

Build automation is a team effort

Putting the codebase management aside, it is also fundamental that each and every member in the team accepts maintaining and improving automation as part of their routine. Do not put the whole CI effort over the shoulders of one of two people in the team. Having Pipeline-As-Code mechanisms is great for that, as it is to include a Jenkinsfile or Travisfile into your repository that anyone in the team can modify. But this makes it part of the codebase now: keep it updated and keep it clean at all costs!

Also, avoid creating CI-maintenance “tasks” or “tickets”. Maintaining the continuous integration flow should be something implicit, just as writing tests or refactoring your codebase. If it’s a task, it can be deferred or procrastinated in favour of other tasks. But if keeping continuous integration up to date is part of the team’s culture, it will never be avoided.

Visualize! Communicate!

One great thing about CI tools: they provide lots of different ways of notifying a broken build. Use this. You can send emails, Slack messages and all sort of standard notifications. Go beyond this if you can: use a TV monitor to continuously visualize your build dashboard. Put a giant traffic light in your workplace (yes, there are teams doing that). Create specific Information Radiators for CI.

Optimism is an occupational hazard of programming. Feedback is the treatment. (Kent Beck)

The goal of notifying broken builds loudly is not to shame anyone in the team, but to declare a temporary state of emergency where everyone can help. There should be normal to have several of these small crises everyday, and even if they should be treated with maximum priority, integration can be just put on hold (e.g. git revert) if the situation requires it.

So, what makes a good CI? The practice of sharing our ideas and our code with the rest of our team continuously. Of course we have all sorts of tools to help us, but embracing the true spirit of continuous integration is infinitely more valuable than all the tools in the world.

DevOps Course at Telefonica Global Solutions

Thu, 01 Jun 2017 00:00:00 +0000

I taught part of a 5-day course on DevOps to the people of Telefónica Global Solutions (Madrid), including presentations and workshops on CI/CD and tools like git, GoCD, Docker, and SonarQube. The whole experience was amazing and very challenging.

Earlier this year was approached by some of the cool people from the Cloud and Infrastructure team at Gradiant. They were looking for someone to teach (or at least talk fancy) about Continuous Integration and Deployment as part of a DevOps course. Well, I do like talking, and I am keen on CI and XP in general, so I said yes.

The training was part of a bigger project involving different companies to provide Continuous Delivery tools and practices to Telefonica Global Solutions. My job was to fill in for two of the five days of the training. The course included a 2-day training on Scrum by Alberto Gutierrez and a special session for System Administrators by Roberto Salgado.

By that time I just had started reading The DevOps Handbook, and I felt compelled to teach the right ideas about DevOps, and not just a few disconnected guidelines on git, docker or GoCD. After all, as I was just finding out, DevOps is a culture (and a mindset), not a collection of tools, not a role.

- We sell tools for CI/CD, so we agree. :) But DevOps is about the culture, not the tools. ie, You don't buy tools & "do" DevOps.
— CloudBees (@CloudBees) 5 de enero de 2017

'You can't buy #DevOps, and it's not a designated team. It is the idea of people working together.' https://t.co/uKA1APxGVN
— GoCD (@goforcd) 8 de noviembre de 2016

After saying yes (maybe too fast), I just realized that preparing 16 hours of technical training is a lot of work, and an intimidating challenge, as well. The weeks before the training in Madrid turned out very stressful and exhausting. But in the end (and thanks to a lot of help and support from my colleagues), I managed to finish the slides and exercises on time for the training.

This was the initial syllabus for the two days:

The week of the course, I arrived to Madrid to meet a very motivated and passionate group of people. During the 2 days, they asked a lot of questions and showed great interest about continuous integration, version control and testing.

Of course, different problems came up. Turned out that very few of the attendees had a developer profile. Instead, most of them had a telco or systems engineering role, so I had to spend more time on version control and the basics of continuous integration than it was initially planned. This made me improvise a bit during the first day and modify the initial plan for the second session. Also, we had a lot of technical problems (specially with internet connection and all kinds of proxy-related stuff, which sucks if you want access to DockerHub).

But, guess what? It was a lot of fun and a very rewarding experience. And I learned a lot about CI and CD, and also a few important lessons:

Assume everything is going to go south, starting with technical problems.
Check the reactions of the audience and focus on the topics that are more appealing for them, even if that means leaving some stuff out of the presentation.

Fun Fact: the course started on may 8th and was planned to end by may 12th in the afternoon, but it had to end abruptly on the morning of may 12th (during Rober’s session) when they told us to turn of all the computers for security reasons.

Git status in bash prompt

Mon, 10 Oct 2016 00:00:00 +0000

Have you ever forgotten to add a particular file to your commit? Want to know your repository global status without doing a git status? You can configure your prompts to warn you, here’s how.Have you ever forgotten to add a particular file to your commit? Want to know your repository global status without doing a git status? You can configure your prompts to warn you, here’s how.

Go to your .bashrc configuration file. Locate the custom prompt variable definition (PS1), it looks like this:

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\[\033[00m\]\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi

Add the following definitions and include the __git_ps1 variable in PS1:

export GIT_PS1_SHOWDIRTYSTATE=true
export GIT_PS1_SHOWSTASHSTATE=true
export GIT_PS1_SHOWUNTRACKEDFILES=true
if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\[\033[00m\]\[\033[01;34m\]\w\[\033[00m\]\$$(__git_ps1) '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$$(__git_ps1) '
fi

You’re good to go now. Remember you’ll have to restart the terminal to apply the changes. When you are in a git project, you should see the following:

~$ cd Code/qt/gmot
~/Code/qt/gmot$ (master)
                ^^^^^^^ (current branch)
  
~/Code/qt/gmot$ (master) touch foo.txt
~/Code/qt/gmot$ (master %)
                       ^^^ (untracked content)
  
~/Code/qt/gmot$ (master %) git add foo.txt
~/Code/qt/gmot$ (master +)
                       ^^^ (staged content)
  
~/Code/qt/gmot$ (master +) git commit -m "added foo stuff"
[master 785374e] added foo stuff
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 foo.txt
 
~/Code/qt/gmot$ (master) 
                 ^^^^^^ (back to normal)

Updating to GCC 4.8 in Ubuntu 12

Mon, 08 Jun 2015 00:00:00 +0000

If you run Ubuntu 12 or any other old distro, you are probably missing most of the latest C++11 features! This is how you can update gcc to a more recent, more C++11-ish version.

First of all, check your current gcc version by doing

gcc -v

Ubuntu 12 ships with GCC v4.6 by default. Fortunately, the aptitude repos include a newer version too:

sudo apt-get install gcc-4.8
sudo apt-get install g++-4.8

Once installed, you must configure 4.8 as the default version:

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 20
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 20
sudo update-alternatives --config gcc
sudo update-alternatives --config g++

While this is the simplest way of getting a newer, cooler version of GCC in Ubuntu 12, you can follow the same procedure to update gcc in other Ubuntu versions (e.g. to gcc 4.9 in Ubuntu 14).