In my last post, I talked about how include dependencies in our code base were causing lengthy build times. Well thanks to this entry on Jeff Atwood's blog, Coding Horror, our problem was solved overnight. Okay, our problem really isn't solved. We still have include dependency issues, but they no longer affect our build times, thanks to Incredibuild.
Jeff's mention of Incredibuild has completely transformed my day at work. A build that used to take over twenty-five minutes now takes about five minutes. I no longer have to worry about switching tasks/contexts to fill twenty-five minute voids throughout my day. I no longer have second thoughts about working in certain areas of the code, knowing that any small change would require a full rebuild. Refactoring can occur much faster and adds to the wave of motivation that has swept through our team since we began using this product.
Incredibuild is by far one of the best products I've ever worked with. Not only does it work as intended, but it was incredibly easy to setup. After reading Jeff's post, I downloaded the trial version and had the software working on two of my machines in less than fifteen minutes. A quick "You have to check this out..." email to my fellow team-members resulted in us having a seven machine build network by the end of the day. At that point, we started bugging management to approve purchasing the software and we are now on our way to becoming a much more productive team.
I am still trying to eliminate some of the worst include-dependencies/coupling in our code. But I no longer have to wait twenty-five minutes to get any feedback for my effort.
Monday, October 8, 2007
Thursday, May 24, 2007
The Case of the Horrible Include Dependencies
One of my favorite programming books is Working Effectively with Legacy Code written by Michael Feathers. The book provides an arsenal of techniques to refactor legacy code and get it into a test harness.
One specific situation the author addresses is testing a C++ class that has a large number of include dependencies. Our product has many classes that suffer from this problem. Our main library consists of over 600 source (.cpp) files and changes to a single header file often result in a large number of files having to be recompiled. A full rebuild takes over thirty minutes and is a productivity and motivation killer. I've decided to attack this problem at its root and try to eliminate as many of the dependencies as possible. At the very least, I will reduce the number of times a full-rebuild is necessary and will hopefully see a noticeable reduction in compilation time in the future.
I started off by writing three Ruby scripts to collect data about the include dependencies for our product. The first script traverses our source tree, inserting the following line into each header file after the header guard:
The second script parses the compilation output, tabulating the number of times each header file is included and how many header files each .cpp file includes.
The final script is the inverse of the first and removes all of the #pragma directives.
The scripts were easy to write and they output the exact information I want, so I don't think a code analyzer tool would have saved me much time. The main downside to my approach is that I have to do a full rebuild anytime I want to get all of the current data. But since this project is something I'm doing on the side, rapid feedback isn't that important. I can just kick off the build and come back to it when I have some down time.
Since beginning this project two weeks ago, I have been able to significantly reduce the include dependencies on about half a dozen header files. For example, one header file that was being included 455 times is now included only eighteen times. That is quite a time saver the next time someone changes that header file.
It's very motivating to see the include dependencies shrink after each small improvement I make. In his book, Michael Feathers talks about situations where programmers feel the code the work on is beyond repair. He points out that little changes here and there begin to add up, and you slowly realize the situation isn't as bad as it seems. I think the small victories I've had so far point to continued success and will result in a much better code base and a much more productive team.
One specific situation the author addresses is testing a C++ class that has a large number of include dependencies. Our product has many classes that suffer from this problem. Our main library consists of over 600 source (.cpp) files and changes to a single header file often result in a large number of files having to be recompiled. A full rebuild takes over thirty minutes and is a productivity and motivation killer. I've decided to attack this problem at its root and try to eliminate as many of the dependencies as possible. At the very least, I will reduce the number of times a full-rebuild is necessary and will hopefully see a noticeable reduction in compilation time in the future.
I started off by writing three Ruby scripts to collect data about the include dependencies for our product. The first script traverses our source tree, inserting the following line into each header file after the header guard:
#pragma message("including __FILE__")The second script parses the compilation output, tabulating the number of times each header file is included and how many header files each .cpp file includes.
The final script is the inverse of the first and removes all of the #pragma directives.
The scripts were easy to write and they output the exact information I want, so I don't think a code analyzer tool would have saved me much time. The main downside to my approach is that I have to do a full rebuild anytime I want to get all of the current data. But since this project is something I'm doing on the side, rapid feedback isn't that important. I can just kick off the build and come back to it when I have some down time.
Since beginning this project two weeks ago, I have been able to significantly reduce the include dependencies on about half a dozen header files. For example, one header file that was being included 455 times is now included only eighteen times. That is quite a time saver the next time someone changes that header file.
It's very motivating to see the include dependencies shrink after each small improvement I make. In his book, Michael Feathers talks about situations where programmers feel the code the work on is beyond repair. He points out that little changes here and there begin to add up, and you slowly realize the situation isn't as bad as it seems. I think the small victories I've had so far point to continued success and will result in a much better code base and a much more productive team.
Sunday, April 22, 2007
Great Experience with Unit Testing
This weekend I planned on stopping in at work for an hour or two to clean out some simple things on my TODO list. Unfortunately, upon opening my email, I was greeted with a message that our overnight build/run system had failed. In particular, a unit test suite for a library that I had created had failed to pass. The email also indicated that our application had significant changes in output from its previous run, something that wasn't expected for any changes that had been checked in the past day.
The unit test failure struck me as quite odd since the code in that library hadn't changed in over a year. I reran the test suite in my development environment, found the test that was failing and noticed that an output string I was expecting had changed. This failure gave me an immediate clue to where the problem was introduced, and with great satisfaction, I was able to track down the problem in no time.
As it turns out, my unit tests were not the only code expecting to see that same string value. Another major library in our system, which controls output functionality, had failed due to the changes that had been made. This library is much more complicated and does not yet have test coverage.
What I found significant about this experience, was that my unit tests had caught an error in functionality outside of the library they were testing. The library I wrote is pretty trivial, and I actually wrote the tests after writing the code (tut, tut) to try and introduce unit testing to my team. So an exercise that I felt was pretty academic at the time, had a significant practical payoff. Had I never written those tests, I would have spent much more time tracking down the change that had caused the problem.
Lesson learned: Unit tests not only test the primary functionality being tested. They can also test secondary functionality that the primary functionality depends on. Even if a test seems pointlessly trivial, its worth writing as it may uncover problems in other areas of the system.
The unit test failure struck me as quite odd since the code in that library hadn't changed in over a year. I reran the test suite in my development environment, found the test that was failing and noticed that an output string I was expecting had changed. This failure gave me an immediate clue to where the problem was introduced, and with great satisfaction, I was able to track down the problem in no time.
As it turns out, my unit tests were not the only code expecting to see that same string value. Another major library in our system, which controls output functionality, had failed due to the changes that had been made. This library is much more complicated and does not yet have test coverage.
What I found significant about this experience, was that my unit tests had caught an error in functionality outside of the library they were testing. The library I wrote is pretty trivial, and I actually wrote the tests after writing the code (tut, tut) to try and introduce unit testing to my team. So an exercise that I felt was pretty academic at the time, had a significant practical payoff. Had I never written those tests, I would have spent much more time tracking down the change that had caused the problem.
Lesson learned: Unit tests not only test the primary functionality being tested. They can also test secondary functionality that the primary functionality depends on. Even if a test seems pointlessly trivial, its worth writing as it may uncover problems in other areas of the system.
Subscribe to:
Posts (Atom)