“This book represents a thorough and extensive treatment of the software build process including the choices, benefits, and challenges of a well designed build process. I recommend it not only to all software build engineers but to all software developers since a well designed build process is key to an effective software development process.” —Kevin Bodie, Director Software Development, Pitney Bowes Inc.   “An excellent and detailed explanation of build systems, an important but often overlooked part of software development projects. The discussion of productivity as related to build systems is, alone, well worth the time spent reading this book.” —John M. Pantone, Objectech Corporation, VP, IT Educator and Course Developer   “Peter Smith provides an interesting and accessible look into the world of software build systems, distilling years of experience and covering virtually every type of tool in the build engineer’s toolbox. Well organized, well written, and very thorough; I would recommend this book to anyone with a build system under their responsibility.” —Jeff Overbey, Project Co-Lead, Photran   “Software Build Systems teaches how to think about building software. It surveys the tools and techniques for building software products and the ways things go wrong. This book will appeal to those new to build systems as well as experienced build system engineers.” —Monte Davidoff, Software Development Consultant, Alluvial Software, Inc.   Inadequate build systems can dramatically impact developer productivity. Bad dependencies, false compile errors, failed software images, slow compilation, and time-wasting manual processes are just some of the byproducts of a subpar build system. In Software Build Systems, software productivity expert Peter Smith shows you how to implement build systems that overcome all these problems, so you can deliver reliable software more rapidly, at lower cost.   Smith explains the core principles underlying highly efficient build systems, surveying both system features and usage scenarios. Next, he encapsulates years of experience in creating and maintaining diverse build systems–helping you make well-informed choices about tools and practices, and avoid common traps and pitfalls. Throughout, he shares a wide range of practical examples and lessons from multiple environments, including Java, C++, C, and C#. Coverage includes   • Mastering build system concepts, including source trees, build tools, and compilation tools • Comparing five leading build tools: GNU Make, Ant, SCons, CMake, and the Eclipse IDE’s integrated build features • Ensuring accurate dependency checking and efficient incremental compilation • Using metadata to assist debugging, profiling, and source code documentation • Packaging software for installation on your target machine • Best practices for managing complex version-control systems, build machines, and compilation tools   If you’re a developer, this book will illuminate the issues involved in building and maintaining the build system that’s best for your team. If you’re a manager, you’ll discover how to evaluate your team’s build system and improve its effectiveness. And if you’re a build “guru,” you’ll learn how to optimize the performance and scalability of your build system, no matter how demanding your requirements are.

Table of Contents:
PREFACE     xxi Why Do Build Systems Become Complex?     xxiii The True Cost of a Build System     xxiv The Focus of This Book     xxvii Who Should Read This Book?     xxviii How This Book Is Organized     xxix Summary     xxxi ACKNOWLEDGMENTS     xxxiii ABOUT THE AUTHOR     xxxv PART I THE BASICS     1 Chapter 1 BUILD SYSTEM OVERVIEW     3 What Is a Build System?     3 Compiled Languages     4 Interpreted Languages     6 Web-Based Applications     6 Unit Testing     7 Static Analysis     8 Documentation Generation     9 Components of a Build System     10 Version-Control Tools     10 Source and Object Trees     11 Compilation Tools and Build Tools     13 Build Machines     14 Release Packaging and Target Machines     15 The Build Process and Build Description     16 How a Build System Is Used     18 Build-Management Tools     19 Build System Quality     21 Summary     21 Chapter 2 A MAKE-BASED BUILD SYSTEM     23 Calculator Example     23 Creating a Simple Makefile     26 Simplifying the Makefile     28 Additional Build Targets     29 Using a Framework     31 Summary     33 Chapter 3 THE RUNTIME VIEW OF A PROGRAM     35 Executable Programs     36 Native Machine Code     36 Monolithic System Images     37 Full Program Interpretation     38 Interpreted Byte Codes     39 Libraries     40 Static Linking     41 Dynamic Linking     42 Configuration and Data Files     43 Distributed Programs     44 Summary     46 Chapter 4 FILE TYPES AND COMPILATION TOOLS     47 C/C++     48 Compilation Tools     49 Source Files     50 Assembly Language Files     52 Object Files     53 Executable Programs     56 Static Libraries     57 Dynamic Libraries     58 C++ Compilation     59 Java     60 Compilation Tools     61 Source Files     62 Object Files     63 Executable Programs     65 Libraries     67 C#     68 Compilation Tools     68 Source Files     69 Executable Programs     71 Libraries     74 Other File Types     76 UML-Based Code Generation     77 Graphic Images     79 XML Configuration Files     81 Internationalization and Resource Bundles     81 Summary     82 Chapter 5 SUBTARGETS AND BUILD VARIANTS     83 Building Subtargets     84 Building Different Editions of the Software     86 Specifying the Build Variant     87 Varying the Code     90 Building Different Target Architectures     94 Multiple Compilers     94 Platform-Specific Files/Functions     95 Multiple Object Trees     96 Summary     98 PART II THE BUILD TOOLS     99 Chapter 6 MAKE     107 The GNU Make Programming Language     108 Makefile Rules to Construct the Dependency Graph     109 Makefile Rule Types     110 Makefile Variables     112 Built-In Variables and Rules     114 Data Structures and Functions     116 Understanding Program Flow     119 Further Reading     122 Real-World Build System Scenarios     123 Scenario 1: Source Code in a Single Directory     123 Scenario 2(a): Source Code in Multiple Directories     125 Scenario 2(b): Recursive Make over Multiple Directories     126 Scenario 2(c): Inclusive Make over Multiple Directories     130 Scenario 3: Defining New Compilation Tools     137 Scenario 4: Building with Multiple Variants     138 Scenario 5: Cleaning a Build Tree     140 Scenario 6: Debugging Incorrect Builds     142 Praise and Criticism     144 Praise     144 Criticism     146 Evaluation     148 Similar Tools     149 Berkeley Make     149 NMake     150 ElectricAccelerator and SparkBuild     151 Summary     153 Chapter 7 ANT     155 The Ant Programming Language     156 A Little More Than “Hello World”     157 Defining and Using Targets     158 Ant’s Flow of Control     161 Defining Properties     162 Built-In and Optional Tasks     164 Selecting Multiple Files and Directories     168 Conditions     170 Extending the Ant Language     172 Further Reading     173 Real-World Build System Scenarios     174 Scenario 1: Source Code in a Single Directory     174 Scenario 2(a): Source Code in Multiple Directories     175 Scenario 2(b): Many Directories, with Multiple build.xml Files     175 Scenario 3: Defining New Compilation Tools     179 Scenario 4: Building with Multiple Variants     183 Scenario 5: Cleaning a Build Tree     188 Scenario 6: Debugging Incorrect Builds     188 Praise and Criticism     191 Praise     191 Criticism     191 Evaluation     193 Similar Tools     193 NAnt     194 MSBuild     194 Summary     196 Chapter 8 SCons     197 The SCons Programming Language     198 The Python Programming Language     199 Simple Compiling     202 Managing Build Environments     206 Program Flow and Dependency Analysis     210 Deciding When to Rebuild     212 Extending the Language     214 Other Interesting Features     218 Further Reading     219 Real-World Build System Scenarios     219 Scenario 1: Source Code in a Single Directory     219 Scenario 2(a): Source Code in Multiple Directories     219 Scenario 2(b): Multiple SConstruct Files     220 Scenario 3: Defining New Compilation Tools     222 Scenario 4: Building with Multiple Variants     224 Scenario 5: Cleaning a Build Tree     226 Scenario 6: Debugging Incorrect Builds     226 Praise and Criticism     229 Praise     230 Criticism     231 Evaluation     231 Similar Tools     232 Cons     232 Rake     233 Summary     235 Chapter 9 CMAKE     237 The CMake Programming Language     238 CMake Language Basics     239 Building Executable Programs and Libraries     240 Control Flow     243 Cross-Platform Support     246 Generating a Native Build System     248 Other Interesting Features and Further Reading     254 Real-World Build System Scenarios     255 Scenario 1: Source Code in a Single Directory     255 Scenario 2: Source Code in Multiple Directories     256 Scenario 3: Defining New Compilation Tools     257 Scenario 4: Building with Multiple Variants     259 Scenario 5: Cleaning a Build Tree     260 Scenario 6: Debugging Incorrect Builds     260 Praise and Criticism     261 Praise     261 Criticism     262 Evaluation     262 Similar Build Tools     263 Automake     263 Qmake     264 Summary     264 Chapter 10 ECLIPSE     267 The Eclipse Concepts and GUI     268 Creating Projects     269 Building a Project     276 Running a Project     282 Using the Internal Project Model     285 Other Build Features     286 Further Reading     288 Real-World Build System Scenarios     288 Scenario 1: Source Code in a Single Directory     288 Scenario 2: Source Code in Multiple Directories     290 Scenario 3: Defining New Compilation Tools     291 Scenario 4: Building with Multiple Variants     292 Scenario 5: Cleaning a Build Tree     295 Scenario 6: Debugging Incorrect Builds     296 Praise and Criticism     296 Praise     297 Criticism     297 Evaluation     298 Similar Build Tools     299 CDT for Eclipse, C/C++ Development Tooling     299 Summary     301 PART III ADVANCED TOPICS     303 Chapter 11 DEPENDENCIES     305 The Dependency Graph     307 Incremental Compilation     307 Full, Incremental, and Subtarget Builds     308 The Problem with Bad Dependencies     310 Problem: Missing Dependencies Causing a Runtime Error     310 Problem: Missing Dependencies Causing a Compile Error     311 Problem: Unwanted Dependencies Causing Excess Rebuilding     312 Problem: Unwanted Dependencies Causing Failed Dependency Analysis     312 Problem: Circular Dependencies     313 Problem: Implicit Sequencing As a Substitute for Dependencies     314 Problem: The Clean Target Doesn’t Clean Everything     315 Step 1: Computing the Dependency Graph     315 Gathering Exact Dependencies     316 Caching the Dependency Graph     319 Updating the Cached Dependency Graph     320 Step 2: Determining Which Files Are Out-of-Date     324 Time Stamp-Based Methods     324 Checksum-Based Methods     326 Flag Comparison     328 Advanced Methods     329 Step 3: Sequencing the Compilation Steps     330 Summary     333 Chapter 12 BUILDING WITH METADATA     335 Debugging Support     336 Profiling Support     338 Coverage Support     340 Source Code Documentation     341 Unit Testing     344 Static Analysis     348 Adding Metadata to a Build System     349 Summary     350 Chapter 13 SOFTWARE PACKAGING AND INSTALLATION     351 Archive Files     352 Packaging Scripts     353 Other Archive Formats     356 Improvements     356 Package-Management Tools     359 The RPM Package Manager Format     360 The rpmbuild Process     361 An Example RPM Spec File     363 Creating the RPM File from the Spec File     369 Installing the RPM Example     371 Custom-Built GUI Installation Tools     373 The Nullsoft Scriptable Install System (NSIS)     374 The Installer Script     376 Defining the Pages     379 The License Page     380 Directory Selection     381 The Main Component     381 The Optional Components     383 Defining a Custom Page     385 The Installation Page and the Uninstaller     387 Summary     388 Chapter 14 VERSION MANAGEMENT     391 What Should Be Version-Controlled     392 Build Description Files     393 References to Tools     395 Large Binary Files     400 Source Tree Configurations     401 What Should Not Be in the Source Tree     402 Generated Files in the Source Tree     402 Generated Files Under Version Control     404 Build-Management Scripts     405 Version Numbering     406 Version-Numbering Systems     406 Coordinating and Updating the Version Number     407 Storing and Retrieving the Version Number     410 Summary     411 Chapter 15 BUILD MACHINES     413 Native and Cross-Compilation     414 Native Compilation     414 Cross-Compilation     415 Hybrid Environments     416 Centralized Development Environments     416 Why Build Machines Differ     418 Managing Multiple Build Machines     421 Open-Source Development Environments     424 GNU Autoconf     428 The High-Level Workflow     428 An Autoconf Example     430 Running autoheader and autoconf     434 Running the configure Script on the Build Machine     435 Using the Configuration Information     437 Summary     438 Chapter 16 TOOL MANAGEMENT     441 Rules for Managing Tools     442 Tool Rule #1: Take Notes     442 Tool Rule #2: Use Version Control for the Source Code     443 Tool Rule #3: Periodically Upgrade Tools     444 Tool Rule #4: Use Version Control for the Tool Binaries     445 Breaking the Rules     448 Writing Your Own Compilation Tools     449 Custom-Written Tools with Lex and Yacc     450 Summary     453 PART IV SCALING UP     455 Chapter 17 REDUCING COMPLEXITY FOR END USERS     457 Build Frameworks     458 Developer-Facing Portion of the Build Description     459 Framework Portion of the Build Description     460 Convention over Configuration     461 Maven: An Example Build Tool     462 Reasons to Avoid Supporting Multiple Variants     463 You’ll Have More Variants to Test     463 Source Code Becomes Messy     465 Build Times Can Increase     465 Higher Disk Space Requirements     466 Various Ways to Reduce Complexity     466 Use a Modern Build Tool     466 Automatically Detect Dependencies     467 Keep Generated Files out of the Source Tree     467 Ensure That Cleaning a Build Tree Works Correctly     468 Abort the Build After the First Error     468 Provide Meaningful Error Messages     470 Validate Input Parameters     470 Don’t Overengineer Build Scripts     471 Avoid Using Cryptic Language Features     471 Don’t Use Environment Variables to Control the Build Process     472 Ensure That Release and Debug Builds Are Similar     473 Display the Exact Command Being Executed     474 Version-Control References to Tools     475 Version-Control the Build Instructions     475 Automatically Detect Changes in Compilation Flags     475 Don’t Invoke the Version-Control Tool from the Build System     476 Use Continuous Integration as Often as Possible     476 Standardize on a Single Type of Build Machine     477 Standardize on a Single Compiler     477 Avoid Littering Code with #ifdefs     477 Use Meaningful Symbol Names     478 Remove Stale Code     478 Don’t Duplicate Source Files     479 Use a Consistent Build System     480 Scheduling and Staffing Build System Changes     480 Summary     482 Chapter 18 MANAGING BUILD SIZE     485 The Problem with Monolithic Builds     486 Component-Based Software     488 Advantages of Using Components     491 What Exactly Is a Component?     493 Integrating Components into a Single Product     498 People and Process Management     502 Development Team Structure     503 Component Line-Up Management     505 Managing the Component Cache     507 Coordinating New Software Features     509 Apache Ivy     512 Chapter 19 FASTER BUILDS     515 Measuring Build System Performance     516 Measuring Performance in the Start-Up Phase     516 Measuring Performance in the Compilation Phase     526 Performance-Measurement Tools     531 Fixing the Problem: Improving Performance     534 Build Avoidance: Eliminating Unnecessary Rebuilds     535 Object File Caching     536 Smart Dependencies     539 Other Build-Avoidance Techniques     544 Parallelism     545 Build Clusters/Clouds     546 Parallel Build Tools     546 Limitations of Scalability     547 Reducing Disk Usage     548 Summary     551 REFERENCES     553 INDEX     559