Showing posts with label Craftsmanship. Show all posts
Showing posts with label Craftsmanship. Show all posts

Friday, August 28, 2020

Glimmer Process (Beyond Agile)

Glimmer Process is the lightweight software development process used for building Glimmer libraries and Glimmer apps, which goes beyond Agile, rendering all Agile processes obsolete. Glimmer Process is simply made up of 7 guidelines to pick and choose as necessary until software development needs are satisfied. Not all guidelines need to be incorporated into every project, but it is still important to think through every one of them before ruling any out. Guidelines can be followed in any order. 

GPG (Glimmer Process Guidelines):

  1. Requirements Gathering: Spend no more than a few hours writing the initial requirements you know from the business owner, gathering missing requirements, and planning to elicit more requirements from users, customers, and stakeholders. Requirements are not set in stone, but serve as a good starting point in understanding what the project is about. After initial release, only document small tasks going forward.
  2. Software Architecture and Design Diagrams: Perform software architecture and design activities as necessary by analyzing requirements and creating diagrams.
  3. Initial Release Plan: This guideline's motto is "Plans are Nothing. Planning is Everything" (said by Dwight D. Eisenhower) because the initial release plan is not set in stone and might change completely, but is still invaluable in launching a new project forward. Consider including alpha releases (for internal testing) and beta releases (for customer testing).
  4. Small Releases: Develop and release in small increments. Do not release more than 3-weeks worth of work, preferring releases that are shorter than a week worth of work whenever possible. Break releases down. If you need to support multiple platforms, release for a platform at a time. If a feature includes a number of other features, then release them one by one instead of all at once. If a feature involves multiple options, release the default version first, and then release extra options later. 
  5. Usability Testing: Make sure to observe a user using your software the first few releases or when releasing critical brand new unproven features. Ask them to complete a list of goals using the software, but do not help them to complete them unless they get very stuck. Write down notes silently while observing them use your software. Once done with usability testing, you may then ask further questions about any of the notes you wrote down, and afterwards add the notes as feature enhancements and bug fixes. Implement them and then do another usability test with the improvements in place. Repeat as necessary only.
  6. Automated Tests: Cover sensitive parts of the code with automated tests at the appropriate level of testing needed. Run tests in a continuous integration server.
  7. Refactoring: Simplify code, eliminate code duplication, improve code organization, extract reusable components, and extract reusable libraries at every opportunity possible. 

See the up-to-date version of the Glimmer Process at the Glimmer (Ruby Desktop Development GUI Library) GitHub Repo:
https://github.com/AndyObtiva/glimmer/blob/master/PROCESS.md

Tuesday, October 13, 2015

CodeMentor Experts Reveal The Secrets To Staying Motivated


I've been featured in an article on CodeMentor.io, so thought I'd share:

CodeMentor Article: Experts Reveal The Secrets To Staying Motivated

“Build a personal low-risk project that tackles a new idea or puts a new twist on an old idea” – Andy Maleh

“Get training or regular mentoring by an expert in the language of choice or technology you aspire to master” – Andy Maleh

If you ever feel like solving a programming problem on a side project with the help of a mentor, expert, or pair-programming partner, hit me up at CodeMentor.io:

https://www.codementor.io/andymaleh

Saturday, April 27, 2013

Managing Rails Routes When They Get Out Of Hand

Rails developers have gotten really good over the years in breaking down logic in beefy controllers to business domain logic that lives in Models and some reusable control logic that lives in Controller Modules. Not only does this address cohesion concerns, but also the lines per file recommended limit of no more than say 200 in Ruby or 400 max.

However, what I have not seen done often enough in most of the projects I've worked on is handling of these concerns in the Rails Routes file "routes.rb". I've often seen gigantic routes files that not only take a long time to detangle and understand, but also aren't organized in any fashion as to the grouping of routes per business domain area (e.g. Feature A routes, Feature B routes, etc...), which brings us to the topic of this blog post. :)

If you simply follow the software engineering recommendation to break files into smaller ones once they've surpassed a limit of say 400 lines, then a route file (typically 500+ on bigger Rails projects and sometimes 1000+ or even 2000+) is naturally to be broken into multiple route files.

There are several techniques for doing it, but here is one easy technique that I've used on the last couple of Rails projects I was on:

1. Break routes.rb along these lines: Static Routes, Admin Routes, Account Routes (e.g. authentication with devise), and Per Feature Routes (e.g. Company Management Routes, User Personalized Info Routes, etc...)

2. Create directory "config/routes" to store smaller route files

3. Create a route file under "config/routes" for each one of the functional areas mentioned in Step 1, named "XYZRoutes" and holding a Ruby module like the following example:

module Routes
  module AccountRoutes
    def self.draw(context)
      context.instance_eval do
        devise_for :users, :controllers => { :registrations => "registrations", :sessions => 'sessions' }
        devise_scope :user do
          get "/login" => "sessions#new"
          delete '/logout' => 'sessions#destroy'
          get '/logout' => 'sessions#destroy'
          get '/register' => 'registrations#new'
        end
      end
    end
  end
end


4. Update routes.rb to look like this:


Dir.glob("#{Rails.root.to_s}/config/routes/**/*.rb").each {|route_file| load(route_file)}

SomeApp::Application.routes.draw do
  Routes::StaticRoutes.draw(self)
  Routes::AdminRoutes.draw(self)
  Routes::CompanyRoutes.draw(self)
  Routes::AccountRoutes.draw(self)
  Routes::UserRoutes.draw(self)

  mount JasmineRails::Engine => "/specs" if defined?(JasmineRails)

  root :to => "home#index"
end

This should provide you with a good blueprint for how to better organize routes files in Rails.

To summarize, break down routes.rb in Rails when it gets over 400 lines of code for the following benefits:

  • Higher cohesion per route file, improving general understanding of that area's focus as well as ease of maintainability by allowing you to find URLs for a particular domain area faster.
  • Better management of route content by not having to scroll through 100s of lines of code (sometimes 1000s) to find the URL that you want to change

I've worked in an environment once where the development team wrote an internal library to streamline modularization of routes.rb. If you know of a public open source one, please mention in comments.

2022-03-01 Update:

Rails now supports breaking up route files natively through the `draw` macro:

https://guides.rubyonrails.org/routing.html#breaking-up-very-large-route-file-into-multiple-small-ones

# config/routes.rb

Rails.application.routes.draw do
  get 'foo', to: 'foo#bar'

  draw(:admin) # Will load another route file located in `config/routes/admin.rb`
end

# config/routes/admin.rb

namespace :admin do
  resources :comments
end

This should have been mentioned a while ago, but better late than never I guess.

Thursday, July 12, 2012

Software Craftsmanship vs Software Engineering at McHenry Software Craftsmanship Meetup

Here are the slides of the long edition of my recent talk "Software Craftsmanship vs Software Engineering", which was given at McHenry Software Crafstmanship, Chicago Software Craftsmanship, and South Bend Software Craftsmanship:

Wednesday, April 25, 2012

RailsConf 2012 - My Talk Slides

Here are the slides to my talks at RailsConf 2012.

It was nice meeting all of you at the conference. For those looking for a software development opportunity with self-growth potential, hit me up by email (last slide). Cheers!


Tuesday, April 03, 2012

Software Craftsmanship vs Software Engineering at The South Bend Software Craftsmanship (Apr 2012)

I am presenting on Software Craftsmanship vs Software Engineering at The South Bend Software Craftsmanship on April 17, 2012:
http://www.meetup.com/sobesc/events/59019232/?a=ea1_grp&rv=ea1

Abstract:

The recent emergence of the Software Craftsmanship movement in the last decade has been accompanied with quite a bit of confusion on what the movement is exactly about and whether it adds any value beyond previous software development movements, such as Agile and Software Engineering.

In this talk, Andy Maleh will define Software Craftsmanship, compare and contrast to Software Engineering, and provide actual examples on how both disciplines are playing out at the Groupon software development environment. At the end of the talk, attendees will be given a chance to ask questions and share their own insights on the topic.

Bio: Andy Maleh is a Software Engineer at Groupon who specializes in user needs analysis and building transformative software that meets ongoing demands. He leads by embracing agile practices and software craftsmanship in the process of perfecting Groupon's deal experience. He joined Groupon via Obtiva, where he served as a Senior Consultant for more than five years. Andy is also the Founder and Lead Developer of the Glimmer open source project for Desktop Development with Ruby. Andy holds an M.S. in Software Engineering from DePaul University in Chicago and a B.S. in Computer Science from McGill University (Montreal). Outside of Groupon walls, Andy plays the drums in indie rock venues and travels via Longboard when the Chicago weather permits.

The event is hosted at the Greenhouse at Innovation Park (Notre Dame). 1400 E. Angela Blvd., South Bend, IN, USA.

Tuesday, February 28, 2012

Software Design Trilogy - Part III: Domain Driven Design for RoR Apps

I just gave a talk at Groupon's Geekfest titled:

Software Design Trilogy III - Domain Driven Design for RoR Apps: When Controllers and ActiveRecords Won't Cut It Anymore
You may check out Part II over here.

Here is the abstract of the talk:

Software development is not just about translating business requirements into a static software design. It is about ensuring that in the long run, the system can evolve with the business needs and facilitate communication between business people and developers around the business domain in order to quickly and accurately add or modify features. This requires superb knowledge crunching and domain modeling skills.

Domain Driven Design is an approach that Eric Evans popularized in the mid 2000s to address these concerns. It is a process that centers design of software around the business domain emphasizing the following techniques:

  • Domain Modeling and The Ubiquitous Language
  • Model Driven Design with Building Blocks
  • Refactoring Toward Deeper Insight and Supple Design
  • Large Scale Structures, Bounded Contexts, and Domain Distillation

This presentation will cover an example business domain to guide the attendees through several of the topics and ideas in the Domain Driven Design approach. Additionally, there will be a few code examples to illustrate some of the ideas as applied in a Ruby on Rails application, such as how to go beyond just a basic MVC architecture with Model Driven Design.

Domain Driven Design can help in the long run improve large scale code design inspired by the business domain, and enhance developer communication and collaboration with the business stakeholders. Attendees should walk out with a basic understanding of how abstract concepts from Eric Evans' Domain Driven Design book can be applied on an actual project in the enterprise.

And here are the slides: This concludes the Software Design Trilogy. Hope you enjoyed the learning.

Sunday, November 06, 2011

Agile Processes/Technologies in Non-Agile Hands - Part 1 of 4

One of the things that I encounter often with folks disenchanted with Agile processes/technologies is that they tried them without guidance or proper training (e.g. XP or Ruby on Rails), failed due to reasons mostly related to lack of skill, and then too quickly blamed the failure on the technology and/or Agile process. However, too often what I see when I check their codebase or environment is not Agile, yet Adhoc software development disguised as Agile and sprinkled with a few Agile practices here and there superficially applied.

To be more specific, here are the things I see in their environment:

  1. Under-engineered code design with too much coupling between all the different pieces.
  2. Difficult-to-maintain test suite.
  3. Lack of long term vision for features being developed.
  4. Extremely under-optimized code performance.

Now, let's address each of these items in their own blog post, including what the misconceptions Non-Agile folks have about them.

"Under-engineered code design with too much coupling between all the different pieces"

In the Waterfall process, it is recommended for developers to spend time designing the code up-front before diving into implementation. That prompts developers to often think of all the pieces, study coupling and cohesion, and come up with a design flexible enough to handle future requirements. Now, it's important to have software design as a skill for any developer Agile or not in order to ensure the code is maintainable. But, here is where misconceptions come into play with many inexperienced developers. Waterfall developers think that if you design for any possible future requirements, the codebase will be so easy to maintain when the future requirements come that they will be extremely productive. Thinking of it black (extremely engineered) and white (no engineering) like that results in code so over engineered that developing to meet today's business requirements comes to a crawl in terms of productivity since the code design tries to address requirements way far into the future, maybe even 6 months or a year ahead. This in my experience results in code so difficult to maintain that it discourages developers from following good design practices in the long term as they take shortcuts here and there due to business pressure whenever adding new features, resulting in a terrible code base.

People new to Agile that are not skilled with it get a different misconception as a result. They think that Agile is a license to under-architect software for the sake of simplicity, forgetting about Agile incremental evolutionary software design. That works well for the first few weeks of a project, but then very quickly deteriorates to extremely under-engineered software for business requirements, and extremely terrible software performance, either making the developers eventually get disenchanted with Agile, or making developers outside of the Agile community snicker (whenever you snicker know you're risking loss of insight about something) thinking "we are disciplined spending time on architecting for performance and future requirements from the start unlike these folks". Unfortunately, that statement also shows lack of experience with successful sustainable business software delivery as the reality of a system over-architected from the start is like I said before, extreme lack of productivity at the beginning of a project that loses business trust, and then neglect of developers to keep the code design of high quality in the future due to how over-complex it is, resulting in a code base that keeps getting worse in over-complexity till the project needs a rewrite. Though some developers accept that as a fact of software development, in my opinion and experience, that's not true beyond that they choose to write code that makes their life hell and then convince themselves that they are just accepting the "reality" that they themselves created.

Now, to best explain how Agile code design flows through a project, here is a diagram that contrasts code design complexity in Agile over time vs code design complexity flow in Waterfall and Adhoc processes (note that the diagram is used strictly for communication. It is not based on statistically collected data):

Code design complexity goes higher as more structure is added to the code. Think of 0 as an entire code blob lumped together in one file and 12 as code organized into many files along the lines of object oriented domain driven design and design patterns.

There are multiple things to note about the diagram:

  1. Complexity in the Adhoc process remains quite low over time as Adhoc developers tend to be averse to advanced design techniques such as object oriented design, inheritance, and design patterns since they consider them over-engineering.
  2. Complexity in the Waterfall process tends to go up in spikes since Waterfall developers tend to do big up-front design of software adding a lot of structure way before the need has presented itself.
  3. Complexity in the Agile process gradually climbs up following day-to-day business requirement complexity instead of preparing for future business requirements far in advance or simply ignoring complex design techniques in the name of simplification. Notice how the Agile graph overlaps the Adhoc graph in the first 2 months before it diverges later to handle increased complexity in business needs.
  4. Complexity in the Agile process tends to have dips that signal how Agile developers occasionally refactor their code to simplify when business requirements have changed and the code has become over-engineered for their current requirements.
  5. The area in the diagram between the Waterfall graph and Agile graph represents lost productivity by Waterfall developers due to having to work around over-engineered code.
  6. The area in the diagram between the Adhoc graph and Agile graph represents lost productivity by Adhoc developers due to having to work too hard with weak or non-existant abstractions (under-engineering) that makes them have to deal with difficult to detangle code (lack of separation of concerns).

Now, all of this might be common sense to a lot of developers. But, it takes a lot of practice to get disciplined at refactoring your code design regularly and boldly (with the protection of automated tests) to ensure it is neither over-engineered nor under-engineered per today's business requirements.

Still, there are quite a few other misconceptions that I would like to address, such as:

  • This all sounds good, but in "reality" developers do not have time to refactor their code regularly while still meeting goals for today's business requirements: This is like saying "All the health professional talk about food and exercise sounds good, but who has time to eat well and exercise while still being able to earn their buck for their families". Indeed, this can be a dilemma for many people, and I do not deny it. I am not the most disciplined at exercising for example. But, this is not how I think of it. The way I like to think of it is more along the lines of: "If I were to actually eat well and exercise, would I perform better in other areas of life?". Since the answer is yes, I trained myself to appreciate the taste of healthy food, thus eventually effortlessly consuming such food out of habit. I also figured out alternative ways to exercising like snowboarding, frequent longboarding, and playing the drums, keeping myself in shape, again effortlessly. Going back to our original point, "If I were to become very disciplined at refactoring code regularly, will I be better able to meet business demands in the long term?". The answer from my experience is a most certain yes. In fact, you end up meeting the goals of today's business requirements much more often in the long term if you have a code base that is neither over-engineered nor under-engineered for today's needs. So, the key thing in the short term is to keep practicing the skills of Agile software development until you effortlessly develop the habit of just-enough-design and disciplined refactoring.
  • What you describe as helpful in increased software design complexity is not really helpful for me. I never quite got the point of object oriented design, and I like to break my code into many small methods to handle complexity: While code design is certainly subjective to an extent. When working with a large code base that will be maintained by many future developers. In my experience, organizing code in an object oriented fashion around the business domain concepts helps free my mind from thinking about blobs of code to focus on abstract concepts that relate to business directly, thus better manage complexity. It takes quite the skill to think of code as abstract concepts instead of low level data shuffling with for loops and if loops, bringing us back to the idea that this misconception might be more due to lack of skill in object oriented design as opposed to a problem with the methodology itself. The same way structured procedural design is a step-up from being able to code if statements and loops in one procedure effectively, object oriented design is a step up for business application development in my opinion from structured procedural design, and it assumes you are quite skilled in structured procedural design as a pre-requisite. Of course, there are domains that benefit better from other paradigms, like the functional programming paradigm for mathematical computing and logic paradigm for boolean algebra, which is why I am recommending object oriented programming specifically for domain business model related code that has gone far in complexity beyond the manageability of structured procedural programming. If you find yourself uncomfortable with object oriented design, I strongly recommend getting more training in it, especially from people that are skilled at applying it in practical enterprise environments as opposed to class room settings.

My next blog post will focus on "Difficult-to-maintain test suite.". Stay tuned for part 2.

Wednesday, October 26, 2011

Cheaper Than The Cost of Failure

Applying certain quality practices for software development may not be cheap, but is cheaper than the cost of failure, whether it is TDD (Test-Driven Development), pair-programming, or usability testing.

Let's talk about TDD for example. Although it significantly decreases the cost of building reliable software for developers experienced with the technique, TDD requires a 3 month learning curve for a team new to it before they can start becoming more productive with it than without it from my experience. And, that's with the help of an experienced TDD coach. Now that may seem like a prohibitive cost when business development is in full swing demanding that developers deliver yesterday. That is why often the teams that hire Agile consulting folks are the ones who are already in dire trouble with their managers for being unable to deliver quality software on schedule. They are often stuck in bug fixing mode for both older and newer features, and thus are willing to pay the cost of learning better software development practices. They may resist suggested practices like TDD at first, but then when they see that software releases requiring weeks of bug fixing pretty much fail to wow customers or gain the trust of their stakeholders, that's when they realize that the cost of learning TDD may not be cheap, but is a hell of a lot cheaper than the cost of failure!

Usability testing is another example. I worked on a project several years ago where the stakeholders were exploring new user interface ideas mixing social networking with education. That seemed like the perfect situation to benefit from usability testing with paper prototypes. We tried the process for a feature or two and noticed that it was taking us about 3-4 hours per feature to come up with the paper prototypes and test them given our lack of experience with the technique. The cost of it seemed prohibitive for stakeholders, so management decided to drop the practice. Six months later, when the project was slated for release, stakeholders realized many inefficiencies in the user interface with beta-testing and wanted developers to rework certain areas completely. Unfortunately, that took a lot longer than revising user interfaces in paper prototype format (think 3-5 days vs 2-4 hours per feature) and investors had almost ran out of money by then, so they had to release the project the way it was. The released website did not end up as revolutionary as they originally intended it to be, and business had to accept the inefficient first-cut designs they had for most of the user interfaces.

Morale of the story? The next time, people try to pressure you into giving up certain practices just because of the cost associated with them, emphasize the importance of the benefits and tell them "It may not be cheap, but it's cheaper than the cost of failure!"

Saturday, October 22, 2011

Influential Books On My Career in Software Development

I was just thinking about how much influence certain books had on my career, whether concerning object-oriented design, enterprise architecture, or even business soft skills, so I decided to list them in this post.

Here are the technology books that had the most influence:
  • The Pragmatic Programmer: I still remember the fascination I had with this book after reading the story about Stone Soup in the first few chapters. It was one of those soft skill patterns that come in extremely handy when you think you have a great idea to improve effectiveness at work, but are faced with a lot of resistance to it by coworkers (especially novel ideas like Pair-Programming). In summary, the lesson from Stone Soup is to walk the walk first, and then let people talk the talk for you, instead of starting with the talk before any walk. You start applying the idea at a very small scale, and if it demonstrates benefits, people testify for you, encouraging the application of the idea on a larger scale. And if not, no harm was done given the small scale you started with. You got to try the idea at least, and can rule it out moving on to another idea instead of dwelling on people's resistance to talking about it with no action taken. I found this book a great read when I stumbled upon it several years ago as it contains many lessons in soft skills in addition to technical skills as developers need both to succeed at work after all.
  • Head First Design Patterns: This was one of the most entertaining technology book reads I have ever had, and I still find the comedy in it extremely enjoyable and educational. It is my favorite on how to learn practical application of Design Patterns, and by practical I mean, knowing when it is appropriate to apply them, not just how to apply them. 
  • Applying UML and Patterns: This book has been perhaps one of the most influential books on my object-oriented design skills as well as integrating the process with Agile software development. And, don't let the name of the book fool you. It is not focused mainly on applying UML. It just emphasizes Agile UML diagramming (mostly on paper or whiteboards) as a healthy communication mechanism between team members to better brainstorm and get ideas across. GRASP Patterns (e.g. Coupling and Cohesion) are a huge takeaway in this book as they serve as the underpinnings of successful object-oriented design, and knowing when design patterns are beneficial. 
  • Domain Driven Design: One cannot do object-oriented design well in isolation from business. Knowing how to speak the language of the business domain is crucial in achieving an easy-to-maintain-and-extend software design. 
  • Refactoring to Patterns: Design Patterns are often best applied gradually in response to new business needs as opposed to prescriptively without the business needs being present yet. For example, if an application only seems to need two ways of sorting data on a report at the moment, a simple "if" statement might get the job done well. If in the future, new business demands required more sorting options that need to be reused in multiple parts of the application, then sticking with the same code design simply expands the use of the "if" statement with more branches. That makes readability of the code less abstract and understandable at a glance, and requires duplication of the "if" statement in all parts of the app that require the same sorting options (unless procedural method reuse was done). That is an example of a need to refactor the code to the Strategy design pattern in order to replace the if statement with reliance on polymorphism (GRASP Pattern) via sort strategy objects. "Refactoring to Patterns" is a great read on this extremely useful Agile skill, and so is its predecessor Refactoring.
  • XP Explained: Reference book on the radical ideas of eXtreme Programming. A must read for any software developer who wants to go beyond the overly-controlling Waterfall process or out-of-control adhoc process in software development, even if they cannot apply all its ideas in their environment. It is eXtremely mind expanding as it pushes your thinking outside the boring box that the masses tend to constrain their thinking to.
  • Apprenticeship Patterns: Awesome book on how to become a quick learner (a must in the technology field), stay humble despite experience, and remain open to new ideas the way beginners are. Some of my favorite patterns are "Wear the White Belt", "Expose Your Ignorance", and "Breakable Toys".
Notice how I did not mention any books on Test-Driven Development or Pair-Programming. That is because from my experience, the best way to learn them is not through reading books, yet through actual daily experience via pair-programming with other developers (apprenticeship) that are quite experienced with both skills (these two skills go hand in hand). I have seen developers inexperienced with test-driven development miserably produce difficult to maintain test suites with complex code implementations that are difficult to refactor. I have also seen developers nod off or waste time tweeting while pretending to pair-program. To avoid such pitfalls, it is extremely important to pair-program with experienced pair-programmers, and to learn all different styles of test-driven development (e.g. state-based vs interaction-based, outside-in vs inside-out, integration vs unit, etc...) to understand their trade-offs and when to apply each. After all, badly done Test-Driven Development and Pair-Programming is like badly done anything. It will result in worse results, and unless the developer had enough foresight, they might then blame the technique instead of their lack of skill in it, and thus miss out on the benefits of learning completely. Just think, a non-skilled skiier might not enjoy skiing very much. But, is it the activity of skiing or their lack of skill in it that is responsible (assuming they are quite interested in it)?

Here are a few non-technology books that also had a huge influence on my success at work:
  • The 7 Habits of Highly Effective People: Helped me direct my career and sort my priorities as taught in the habits "Begin With The End In Mind" and "First Things First". Also, it had a huge influence on the way I try to come up with solutions that bring everybody in the company together, whether in development, QA, or business  (the "Think Win-Win" habit) and be open minded to collaboration that achieves much more creative solutions to problems than flying solo (the "Synergy" habit)
  • How to Win Friends and Influence People: Great aid in social skills for a techie like me, helping with public speaking skills and learning respect for non-autonomous and barely-logical entities (a.k.a non-computers, a.k.a. human beings!) 
  • The Magic of Thinking Big: This was the book that shattered all artificial and societal constraints that I had acquired over the years about how much I can achieve in life. After reading it, I stopped believing that I am constrained to my I.Q. (or maybe realized you can increase your I.Q. indefinitely) and stopped seeing any boundaries to anything I can do at any organization or in life in general for that matter. Let's just say that not only did I eventually learn to do public speaking in front of masses of people (EclipseCon, EclipseWorld, RubyConf, etc...) after a quarter century of being a shielded introverted shy person, but I also learned rudimentary drumming skills in the last 3 years and now perform in two rock bands (Gag Order and Cletus Darby) around the city of Chicago. 
What are the books that had the most positive influence on your career?

Monday, October 17, 2011

Any Good in Learning Big Up Front Design?

Since the beginning of the Agile movement, people have been renouncing big up front design in favor of incremental emerging design of software. It occurred mostly as a reaction to overly complex designs and application architectures that slowed down productivity to a crawl, especially on new projects. Anyone remembers the days of EJB 2.1? You had to configure a handful of XML file descriptors and write quite a bit of code following the EJB 2.1 conventions before you got anywhere on a new project. Now, contrast that with the Ruby on Rails architecture and how it enabled developers to get a CRUD web application up and running within minutes. Most people miss out on the other side of the story though, that is the part about why EJB's architecture was so complex to begin with. After all, development with a technology like Java's JSP alone was a lot simpler.

EJB came out as a reaction to insecure badly written Java web applications that did not take advantage of transactions and threading correctly, and mixed data persistence concerns with web flow concerns. Enterprise JavaBeans's main innovation was aspect orientation of concerns like transactions, security, threading, and caching, enabling Developers to get their benefits without coding them directly. All they had to do is configure the XML descriptors to enable them, and voila! Additionally, Entity Beans were of the earliest forms of ORM (Object Relational Mapping), Session Beans were one of the earliest forms of controllers, and Message Driven Beans were one of the earliest forms of web background workers (think Resque in the Ruby on Rails world). Also, Enterprise JavaBeans allowed nice separation between web flow and data persistance at the time (albeit with non-existent OO inheritance support), providing one of the most primitive versions of Web-MVC.

Now, if you look at many of today's web frameworks, like Rails for example, you notice that they got many of the same features that EJBs innovated, such as easy transaction support, ORM, security (authentication via Devise and authorization via CanCan for Rails), threading (automatic spawning of threads for requests in web servers like Phusion Passenger), and caching.

Sure, if you needed something much simpler, you can start a Rails app without any of the complexities of caching or other advanced features getting in the way. You may even use Sinatra and avoid reverting to Rails until your app grows large enough to warrant the shift in complexity of design.

However, if people did not spend time solving the problems plaguing web applications in the early 2000s, doing some big up front design, we might have not had any of the solutions that can help an Agile business scale up with Rails today once they have more than a handful visitors an hour.

How does this idea transfer to software design?

Developers new to object oriented programming often encounter topics such as inheritance vs composition and design patterns. If they were to bypass such topics with excuses like "inheritance produces overly complex code over procedural code reuse" or "design patterns make for overly complex designs", their blaming of the tools is sure a sign that they are not skilled with them and that they use them like a golden hammer, potentially in the wrong situations. Now, if they were to avoid learning them however, and they eventually work on a business domain that grows enough in complexity to a level where switch/giant-if statements are plaguing the code everywhere (lack of use of object oriented polymorphism/inheritance or design patterns), given that they skipped learning the techniques of big up front design, they will unfortunately fail in incremental designs as they will always stick to the primitive design techniques they started programming with (like structured decomposition of logic into methods) and thus not be able to scale up their code base to handle more complexity while remaining clear and maintainable to their fellow programmers.

That is why I strongly suggest to developers spending time learning many of the big up front design techniques out there, such as Responsibility Driven Design, Domain Driven Design, Design Patterns, Architectural Patterns, UML Modeling, and even Model Based Development (foundation behind use of DSLs) , simply for the sake of expanding their toolset for when their small simple solutions no longer provide high productivity and maintainability for the increased complexity of their project. In other words, practicing big up front design in a safe learning setting enables developers to actually become (Oh My!) more Agile when the time comes for employing the advanced design skills.

I am sure I missed a handful of useful design tools out there, so I would like others to pitch in. What design tools would you recommend to others to learn and employ in their toolset?

Friday, January 28, 2011

On Continuous Learning

So, a couple of Fridays ago I injured my right wrist snowboarding in Breckenridge, Colorado, and the doctor put a cast around my arm because I got a hairline fracture in the bone connecting to the wrist. Given that I do not have the dominant hand available for use anymore, I had to get by without it for the last two weeks, and will continue to do so for weeks to come. As a result, I had to learn how to do several things left-handed, and that got me thinking about Obtiva's culture of Continuous Learning and how it benefited the process.

Normally, I would have been discouraged and handicapped for weeks, probably using my injury as an excuse not to do some tasks and constantly complain. Given the new habits ingrained in me after being with Obtiva for 5 years now, I surprised myself by having a totally different attitude:
"Why should learning anything with my left hand be any different from learning a new language, library, or technology? Couldn't I follow the same process of gradual learning with "beginner's mind" (as mentioned in Dave Hoover's book Apprenticeship Patterns)?"

And so I did, and as a result, I learned how to do quite a few things with my left hand that I could not do with it before:

Eating with chopsticks fast enough not to starve:

Using a TrackPad comfortably:

Diagraming and writing on a whiteboard quickly enough to effectively relay ideas to my coworkers:

Drumming with one hand:

I am very grateful to Obtiva for instilling the Continuous Learning culture attitude in us, and would like to encourage everyone else to have that spirit instead of letting negativity pose as "being realistic" whenever you are learning something new, especially under difficult circumstances.

Friday, January 07, 2011

How I Learned To Apply Design Patterns

Video has been posted for my Obtiva Lunch GeekFest talk this week:

How I Learned to Apply Design Patterns Obtiva Geekfest from Obtiva on Vimeo.

Here are the slides (with style change and some revisions): How I Learned To Apply Design Patterns

I would like to emphasize a few points that came up during the presentation from the audience:

  • Corey Haines brought up the point that you can benefit from thinking about Design Patterns even if you do not implement them to the tee per the Gang of Four implementation (e.g. double dispatch for Visitor).
  • Tyler Jennings brought up the point that you may sometimes need to split model responsibilities into a separate model responsible for performing the work, such as an OrderTransfer object for an Order. It becomes responsible for transaction-ability around product transfer between two orders.
  • Corey Haines mentioned that not all developers tend to think of which Objects to assign responsibilities to up-front.

One thing that got left out of the presentation is explaining how to choose variations of a design pattern, such as State vs Strategy or Abstract Factory vs Factory Method depending on the situation. That leaves me room for a future blog post/presentation. :)

In the meantime, comments and questions are greatly encouraged.

Saturday, January 01, 2011

Why Blog?

With the new year, it is good to re-evaluate certain habits to see if they still offer value, and one habit a I would like to re-evaluate here is why I blog:
- Organize my thoughts and opinions
- Solidify my knowledge
- Bookmark my learning (virtual breadcrumb)
- Validate my ideas
- Provide others with valuable lessons
- Connect with the community

A lot of developers are intimidated by the idea of blogging due to different reasons, such as:
- Do not have enough value to contribute
- Cannot write well enough to impress (I am not Bob Martin or David Heinemeier Hansson)
- Very time consuming

Actually though, everybody brings a different perspective and in a way, it is selfish to hold back on sharing your experience when others could have benefited from the lessons you learned.

About writing well, it is a learn-able skill, and in the Internet age where expressions like BTW and IMHO are common, it really does not matter as long as you get the idea across.

Finally, regarding time spent, you can always limit the size of your blog posts or break longer blog posts into multiple parts, which as a side effect builds up more anticipation for your blog posts.

I did not start blogging by writing an amazing article on software architecture or anything like that. I simply mentioned a good science fiction read, and here is my first blog post to prove it: http://andymaleh.blogspot.com/2006/11/plunge-into-blogging-world.html

Now if you have never blogged before, go write your first blog post now! It only takes 15 minutes or less and puts on you on a great path for sharing, organizing, and expanding your knowledge.

Saturday, December 04, 2010

To Reuse or To Rewrite?

It is amazing how many developers like to rewrite functionality from scratch instead of reusing a library when most common problems such as validating a zip code or parsing currency have been solved already countless of times.

In a recent Ruby on Rails application I have been working on, I wanted to parse phone numbers in different formats, and figured why waste time re-inventing the wheel when I could use a library? I know it is easy to implement the parsing logic. I wrote such code in Java in the past. But, experience has taught me that thinking it is easy to rewrite when it is solved already is a trap. After all, so many hidden edge cases often lie in a seemingly simple problem that only reveal themselves after a long period of usage and testing. That makes it more than worthwhile to rely on a library than write things from scratch no matter how simplistic. Unfortunately, in the past, I did not have the foresight for it, and whenever I wrote things from scratch, I thought I delivered business value, and did not notice the cost sunk into fixing bugs for all the discovered edge cases over the course of using the application.

I used to like rewriting code from scratch for problems already solved with libraries for several reasons:
1- Seemed easy
2- Gave a satisfying feeling of accomplishment at the end
3- Seemed like more fun than learning the API of a library
4- Prevented me from feeling dumb while attempting to learn a library
5- Kept the code base simple

What seemed easy often ended up having so many edge cases that it became a much more complex task than anticipated, especially when some edge cases were discovered by users in embarrassing bug reports. Now of course, libraries may still have undiscovered edge cases themselves, but taking probability into account (instead of thinking black and white), it is a lower chance to find a bug in a well-used library than code I just wrote.

Regarding the feeling of accomplishment, it was a misleading reason as it could also be obtained from solving the main business problems instead of writing code for already solved lower level problems.

About the fun aspect, while it is important to do something that I enjoy in order to stay motivated, it is also important that what I find fun is most useful to clients or else somebody else more focused on the client needs will do my work faster by reusing a library, rendering me a less valuable developer who selfishly spends time on rewriting code for the fun of it.

One of my biggest fears in reusing a library was feeling dumb or overwhelmed by sheer complexity while learning it. Now, rationally speaking, if the highest goal is serving the client in the best way possible, then if it meant feeling dumb for a little while in order to implement a higher quality solution with a proven library instead of coding stuff from scratch, then so be it. Not being concerned with feeling dumb for the sake of serving a client shows both self security and humbleness. The opposite is simply selfish, insecure, and ineffective for the client. Still, if I can think of a simpler way of writing the library API, then it would be a good idea to write a facade on top of it or contribute the simpler API to the library directly.

Finally as for keeping the code base simple, this really depends on the library being used. There are cases where writing code from scratch is more efficient than using an incredibly complex library. However, I have often generalized from one bad experience with some badly designed over-complicated library, avoiding libraries for a while and losing out on the benefits of many other better and simpler libraries for other problems.

The way I think of it now is if a certain library that I am exploring seems to require me to write more complicated code than if I had written code from scratch, then I should be careful in considering the library a time waster as it may still save me time by handling edge cases I have not considered. Many times, when I wrote something from scratch for a solved problem that seemed simple, I ran into so many edge cases eventually that any productivity gained from not spending time on learning a library was offset completely with the problems encountered. And, as I stated before, I used to not notice that and think I was being productive without reusing a library, but experience taught me otherwise. My opinion now is a bit more strongly for reuse than it was a couple of years ago.

The only time I would solve a problem from scratch now is if no library out there solves it according to the quality standards required for my client. That is when it is a good opportunity to write my own library and open source it to let the community help test it and reveal edge cases quickly.

Luckily, I found a library that parses phone numbers in Ruby called phone_number. :)

Wednesday, November 24, 2010

Can Business People Read Your Code?

Yesterday, I wrote a blog post on how I refactored tests in Ruby to follow the "Data Generated Specs" pattern, making it easy to add specs and read the input and expected output values for the different cases.

The surprising effect for that is being able to review the test cases with a non-technical client, and then the client liking the format so much and requesting a copy of the specs.

I had another similar incident last week that also pleasantly surprised me.

At one point, the client asked me about a detail in the business calculation to verify that I am handling it correctly, and I indicated that I followed the math formula she gave me. Then, to confirm, I had the client come over and look at my implementation in Ruby (changed a bit for IP protection):

copay = benefit.includes_deductible? ? (value - deductible) : value
12*rate + deductible + copay

I made sure to explain the turnery operator while going over the implementation, and the client seemed to get it then cheerfully exclaim: "Great! Looks like you got it exactly the way I wrote it."

I was very happy to hear that, and I wondered if this only became possible because I followed good clean code software development techniques such as TDD and refactoring. I mean, had I not refactored my code to extract the essence of the calculation in one tiny method with less than 5 lines of code, I would have probably confused the client by showing her too much code.

How cleanly is your code written on average? Can business people read your code?

Tuesday, November 23, 2010

Testing Pattern: Data Generated Specs

Just thought of sharing this DRY Ruby testing pattern that facilitates generating a lot of similar specs with different data.

Recently, I wrote specs for a parsing engine that handled each test case with a context describing how the different attribute on a model (Benefit) would be evaluated:

context "Single: $10,000 Group: $20,000" do
describe "single_value" do
it "returns 10000 for Single: $10,000 Group: $20,000" do
benefit = Benefit.new(:value => "Single: $10,000 Group: $20,000")
benefit.single_value.should == 10000
end
end
describe "group_value" do
it "returns 10000 for Single: $10,000 Group: $20,000" do
benefit = Benefit.new(:value => "Single: $10,000 Group: $20,000")
benefit.group_value.should == 20000
end
end
describe "single_value_includes_deductible?" do
it "returns 10000 for Single: $10,000 Group: $20,000" do
benefit = Benefit.new(:value => "Single: $10,000 Group: $20,000")
benefit.single_value_includes_deductible?.should be_true
end
end
describe "group_value_includes_deductible?" do
it "returns 10000 for Single: $10,000 Group: $20,000" do
benefit = Benefit.new(:value => "Single: $10,000 Group: $20,000")
benefit.group_value_includes_deductible?.should be_true
end
end
describe "group_calculation_algorithm" do
it "returns 10000 for Single: $10,000 Group: $20,000" do
benefit = Benefit.new(:value => "Single: $10,000 Group: $20,000")
benefit.group_calculation_algorithm.should == :cap
end
end
end

But, I kept getting more and more business rules for parsing the benefit string values, and I had to add a context block like the one above for every case, so you can imagine how this got out of control very fast and had a ridiculous amount of redundancy:

context "$10,000" do
describe "single_value" do
it "returns 10000 for $10,000" do
benefit = Benefit.new(:value => "Single: $10,000")
benefit.single_value.should == 10000
end
end
describe "group_value" do
it "returns 10000 for $10,000" do
benefit = Benefit.new(:value => "Single: $10,000")
benefit.group_value.should == 10000
end
end
describe "single_value_includes_deductible?" do
it "returns 10000 for $10,000" do
benefit = Benefit.new(:value => "$10,000")
benefit.single_value_includes_deductible?.should be_true
end
end
describe "group_value_includes_deductible?" do
it "returns 10000 for $10,000" do
benefit = Benefit.new(:value => "$10,000")
benefit.group_value_includes_deductible?.should be_true
end
end
describe "group_calculation_algorithm" do
it "returns 10000 for $10,000" do
benefit = Benefit.new(:value => "$10,000")
benefit.group_calculation_algorithm.should == :as_is
end
end
end

context "Single: $10,000 per Member" do
describe "single_value" do
it "returns 10000 for Single: $10,000 per Member" do
benefit = Benefit.new(:value => "Single: $10,000 per Member")
benefit.single_value.should == 10000
end
end
describe "group_value" do
it "returns 10000 for Single: $10,000 per Member" do
benefit = Benefit.new(:value => "Single: $10,000 per Member")
benefit.group_value.should == nil
end
end
describe "single_value_includes_deductible?" do
it "returns 10000 for Single: $10,000 per Member" do
benefit = Benefit.new(:value => "Single: $10,000 per Member")
benefit.single_value_includes_deductible?.should be_true
end
end
describe "group_value_includes_deductible?" do
it "returns 10000 for Single: $10,000 per Member" do
benefit = Benefit.new(:value => "Single: $10,000 per Member")
benefit.group_value_includes_deductible?.should be_true
end
end
describe "group_calculation_algorithm" do
it "returns 10000 for Single: $10,000 per Member" do
benefit = Benefit.new(:value => "Single: $10,000 per Member")
benefit.group_calculation_algorithm.should == :per_person
end
end
end

Finally, I refactored the specs applying what I am calling the "Data Generated Specs" pattern, by writing only one spec as a prototype and feeding in the input and expected output data dynamically through a hash:

{
"Single: $10,000 Group: $20,000" => {
"single_value" => 10000,
"group_value" => 20000,
"single_value_includes_deductible?" => true,
"group_value_includes_deductible?" => true,
"group_calculation_algorithm" => :cap,
},
}.each do |input_value, expected_value|
expected_value.keys.each do |attribute|
describe attribute do
it "returns #{expected_value[attribute]} for #{input_value}" do
benefit = Benefit.new(:value => input_value)
benefit.send(attribute).should == expected_value[attribute]
end
end
end
end

Each test case became represented with a hash key/value block instead of a 30-line code context block, making it much easier to add test cases:

{
"Single: $10,000 Group: $20,000" => {
"single_value" => 10000,
"group_value" => 20000,
"single_value_includes_deductible?" => true,
"group_value_includes_deductible?" => true,
"group_calculation_algorithm" => :cap,
},
"$10,500" => {
"single_value" => 10500,
"group_value" => 10500,
"single_value_includes_deductible?" => true,
"group_value_includes_deductible?" => true,
"group_calculation_algorithm" => :as_is,
},
"Single: $10,500 per Member" => {
"single_value" => 10500,
"group_value" => nil,
"single_value_includes_deductible?" => true,
"group_value_includes_deductible?" => true,
"group_calculation_algorithm" => :per_person,
},
"Single: $10,500 per Member (Deductible not included)" => {
"single_value" => 10500,
"group_value" => nil,
"single_value_includes_deductible?" => false,
"group_value_includes_deductible?" => false,
"group_calculation_algorithm" => :per_person,
},
"See brochure for details" => {
"single_value" => nil,
"group_value" => nil,
"single_value_includes_deductible?" => false,
"group_value_includes_deductible?" => false,
"group_calculation_algorithm" => nil,
},
}.each do |input_value, expected_value|
expected_value.keys.each do |attribute|
describe attribute do
it "returns #{expected_value[attribute]} for #{input_value}" do
benefit = Benefit.new(:value => input_value)
benefit.send(attribute).should == expected_value[attribute]
end
end
end
end

Since specs are generated with their titles, when they fail, you get a nice clear description for the failure that includes the attribute name as well as the input and output values:

1) Benefit attribute single_value returns 0 for Single: $10,500 per Member
Failure/Error: benefit.send(attribute).should == expected_value[attribute]
expected: 10500,
got: 0 (using ==)
# ./spec/models/benefit_spec.rb:464

I generated about 1000+ tests with this technique. While reviewing some of the cases with a non-technical client, she liked reading the spec data so much that she requested a copy for herself to review and validate against the business rules. Specs as acceptance tests FTW!

Friday, November 19, 2010

Should You Work Hard or Smart?

About five years ago, a senior developer I met recounted to me stories about how he aced interviews:
"I always like to tell interviewers that some people work hard, and some people work smart, but I like to work HARD AND SMART!"

It was certainly a nice sounding line that had a good ring to it, but is there such a thing as working hard and smart? Or does working hard automatically imply that you're not working as smart as you could be? Wouldn't you be working less hard if you had smarter ways of accomplishing your tasks?

You are welcome to share your opinion in comments, but here is my perspective on the matter.

It certainly depends on the definition of what is "hard" and what is "smart".

If "hard" meant working 50-hour+ weeks, then on average, people get tired after say 8 or 9 hours of continuous work on a day, and their thinking capacity diminishes as a result, resulting in less "smart" work than say at the beginning of the day. So, in that case, working "hard" affects people's ability to work "smart" and the two do not quite go hand in hand.

If "hard" meant working 35-40-hour weeks with extreme concentration, taking the rest of the days off to let the brain detangle itself and get ready for the next day, then on average, this facilitates performing work that is as "smart" as possible per people's thinking capacities. So, in that case, "hard" and "smart" do go together though people who work 60-hour weeks would not consider that "hard" enough, so it ends up just being "smart", and who doesn't like that?i

Now in the software development world, if some developers find themselves working 60-hour weeks to meet a 3-month deadline for a business project, then they certainly are working "hard", but are they working as "smart" as they could be, or should they be ditching this old unproductive framework/library they are relying on and move on to a smarter technology/programming language that offers more productivity? After all, not only will that save them from over-extending themselves, yet also allow their work to be higher quality since it will be done in the majority of hours when their thinking capacity is near its fullest on average.

Doing 12+ hour days is not necessarily bad every once in a while, especially when the developer has a sudden burst of creativity and motivation. However, if done regularly, it is important to be aware of the quality of work coming out of the long hours as sometimes it may give the illusion of accomplishment when the work is actually taking a lot longer to finish with a tired less concentrated mind, and could have been done better when rested.

Wednesday, November 17, 2010

Pain Driven Development

One of the key things I learned from XP and the Agile movement in general is programming for today's requirements not tomorrow's predictions. And, every time you start writing implementation for requirements that may become valid in the future, the Agile folks would shout "YAGNI" (You Aren't Gonna Need It). Applied to code architecture and design, Ward Cunningham summarizes this philosophy nicely with his famous quote "What's the simplest thing that could possibly work?".

But, what happens when today's implementation no longer fulfills today's requirements? In other words, what happens when tomorrow becomes today and requirements grow or change? One example is when 100,000 more users are added to the system, making performance requirements much greater. Another example is when supporting one state is not enough anymore, and the business is now expanding nationally to cover all 50 states.

That is where awareness of pain comes into play. I wrote a blog post about sensitivity to pain a few years back that talks about pain and pleasure when it comes to writing and maintaining code. Developing that awareness of pain is highly important in detecting when to update today's implementation with a higher level of complexity that addresses today's requirements.

Though people have different levels of tolerance to pain, it is a gift that they can feel it as it is often what pushes them toward action. And, in the case of software development, it can point out when today's implementation no longer serves today's requirements and needs to be revised either with a higher level of complexity, or sometimes with a lower level when some requirements are no longer needed.

When I first heard of the YAGNI principle, I remember shuddering a bit and thinking "Isn't it kind of dumb to write code that I will revise in the future when I have to support more states when I could have added in multiple-state support to begin with?"

Well, unfortunately, my thinking was shallow in certain ways. While the argument is logical at one level since following flexible design practices seems to make it easier to handle some future needs, it is much less trivial if I dig a level deeper and include more variables such as whether these future needs ever materialize in the next 2 years, or how much stepping around I am doing while adding new features, mostly because of complexity in code implemented for predicted needs that are not yet valid.

And experience only confirms the concerns I raised above and shows that keeping the code as simple as possible, only addressing today's known business needs, seems to make it easiest to maintain the code and add more features as more needs come up. That is because the code always remains as simple as possible, yet adjusted in complexity only as pain is felt day-to-day.

One example of this that I recently encountered is writing a web feature that relied on data from a web service. At first, the simplest thing that could possibly work was to have it request data from the service synchronously as users hit the site. Later, as requests for data got more complex and time-consuming to fulfill from the service, the implementation became painful to deal with as far as performance, so background caching of service data was added. That is a very good example of what I like to call "Pain Driven Development" :)

Monday, November 15, 2010

What Continuous Integration Is Really About

Recently, I have been encountering a number of environments where developers work in multiple branches and do not integrate their code till the end of the iteration. They end up often spending hours fighting to merge the code in correctly, sometimes resulting in bugs or missed features.

When I see that, I cannot help but remember the pains of integrations in 6-month-long Waterfall projects. I was a junior developer at an environment in the past where developers spent 6 months implementing features in isolation of each other, and then only integrating right before the project deadline. As a result, they would run into enormous integration issues and spend 3 additional months fixing all of them before finally delivering.

Now, developers who integrate at the end of the iteration often end up with a similar result. They miss the deadline sometimes by a day or more, and end up with issues bleeding into the next iteration (e.g. missing features due to bad merge).

When I encounter such environments, and hear that developers branch out at the beginning of every iteration before developing their own features, I shudder and point out that they are not following the Agile practice of Continuous Integration. They immediately shoot back saying something like "We have cruise control setup" or "We do not have the resources to setup a CI server", which only reveals ignorance about what Continuous Integration is really about. What I was actually saying is they are not integrating continuously into one common branch, and thus not resolving integration conflicts on an hourly or daily basis, yet letting them accumulate till the end of the iteration causing an integration snowball effect.

It is an unfortunate matter of human nature to be lazy at acquiring knowledge. You always want the least amount of learning to get you to where you want to go, so often people fail to dig deeper than what they hear and miss out on the deepest essence of what they are learning. For example, a lot of developers learning MVC from frameworks like Struts or earlier editions of Rails know just enough of MVC to get by, but never spent time digging into the true essence of MVC from Smalltalk Applications (or desktop development in general), and thus fail to apply it correctly. You end up with bloated controllers, instead of splitting most of the non-control behavior into Models. In the same token, a lot of developers who hear of Continuous Integration from the marketing lingo of CI servers think that is what Continuous Integration is all about.

Here is how Martin Fowler describes Continuous Integration:
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.


Notice how the primary emphasis is on members integrating their work frequently; at least daily if not multiple times a day. Also, see how having the automated build is secondary and only there to support the primary goal. So, when developers work in their own branches and do not integrate till the end of their iteration, they are not fulfilling the primary goal of resolving conflicts often before they get big and hard to resolve, and having a CI server does not make them a team that is properly doing Continuous Integration. While a CI server certainly helps them when they integrate at the end of the iteration, they still have to deal with bigger integration issues than if they were integrating daily if not hourly.

Now, working in branches certainly has its place. It is useful when doing a spike, building an experimental feature, performing big architectural changes, or even working on a separate release all together that would not go out till a few months later. Of course, in the case of a separate release, the code would probably not get merged back into master and can be thought of as a separate project (even if it branched off the original project's code base). And, in the case of big architectural changes, it is preferred if possible to have them done in small slices within iterations, and only relying on a branch as a last resort.

Local branches in source code control systems like Git and Mercurial have their place too. You can perform work in a local branch every day if you like as long as you integrate it back to the main branch at the end of the day or every few hours. Used that way, it would still be in line with the practice of Continuous Integration.

Takeaway?

Integrate early and often on the same branch (daily/hourly) and you will leverage the benefits of Continuous Integration on your Agile project by delivering more on time and avoiding big merge/conflict issues.