Build Management
Diomidis Spinellis
Department of Management Science and Technology
Athens University of Economics and Business
Athens, Greece
dds@aueb.gr
The Build Process
Typical Project Dependencies
Example: Apache Project Dependencies
Example: Mozilla Dependencies
Example: Internet Explorer Dependencies
Tool Types
- Low level: Make, Rake, Ant
- Abstraction based: CMake, Autotools
- Framework-driven: Maven
Makefiles
- Tools like ant and make allow the automatic processing of dependencies
- Dependencies are specified in a special file
- Elements of a Makefile may include
- Variable definitions
- Rules for suffixes
- Dependencies
- Target build instructions
- Pseudo-targets
- Auto-generated dependencies
- Procedural specifications
Part of an Apache Makefile
OBJS= \
modules.o \
$(MODULES) \
main/libmain.a \
$(OSDIR)/libos.a \
ap/libap.a
.c.o:
$(CC) -c $(INCLUDES) $(CFLAGS) $<
.SUFFIXES: .def
.def.a:
emximp -o $@ $<
$(TARGET): $(EXTRA_DEPS) $(SUBTARGET)
lib$(TARGET).$(SHLIB_SUFFIX_NAME): subdirs modules.o
$(CC) -c $(INCLUDES) $(CFLAGS) buildmark.c
[...]
target_static: subdirs modules.o
$(CC) -c $(INCLUDES) $(CFLAGS) buildmark.c
$(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_SHLIB_EXPORT) \
-o $(TARGET) buildmark.o $(OBJS) $(REGLIB) $(EXPATLIB) $(LIBS)
clean:
-rm -f $(TARGET) lib$(TARGET).* *.o
@for i in $(SUBDIRS); do \
echo "===> $(SDP)$$i"; \
( cd $$i && $(MAKE) $(MFLAGS_STATIC) SDP='$(SDP)' $@ ) || exit 1; \
echo "<=== $(SDP)$$i"; \
done
#Dependencies
$(OBJS): Makefile subdirs
# DO NOT REMOVE
buildmark.o: buildmark.c include/ap_config.h include/ap_mmn.h \
include/ap_config_auto.h os/unix/os.h include/ap_ctype.h \
include/hsregex.h include/httpd.h include/ap_alloc.h include/buff.h \
include/ap.h include/util_uri.h
modules.o: modules.c include/httpd.h include/ap_config.h \
include/ap_mmn.h include/ap_config_auto.h os/unix/os.h \
include/ap_ctype.h include/hsregex.h include/ap_alloc.h include/buff.h \
include/ap.h include/util_uri.h include/http_config.h
Make Problems and Solutions
- Complexity
-
- Dry run with make -n
- Debug mode: print variables
- Before lunch: make -k
- Portability
-
The Ant Approach
- Build instructions written in XML
- File typically named
build.xml
- Variable definitions (named properties)
- Targets and dependencies
- By default, the target specified in the default attribute of the project tag is built
- Tasks are part of ant, typically not external programs
Some ant Tasks
Examples of ant built-in tasks:
- mkdir
- copy
- cvs
- delete
- javac
- jar
Example Ant Build File
<project name="Jasper" default="deploy" basedir=".">
<!-- ===================== Initialize Property Values =================== -->
<!-- See "build.properties.sample" in the top level directory for all -->
<!-- property values you must customize for successful building!!! -->
<property file="build.properties"/>
<property file="../build.properties"/>
<property file="${user.home}/build.properties"/>
<!-- Build Defaults -->
<property name="build.compiler" value="classic"/>
<property name="copy.crimson.jar" value="../lib/crimson.jar"/>
<!-- =================== BUILD: Create Directories ====================== -->
<target name="build-prepare">
<available classname="junit.framework.TestCase" property="junit.present" />
<mkdir dir="${jasper.build}"/>
<mkdir dir="${jasper.build}/jasper"/>
<mkdir dir="${jasper.build}/lib"/>
</target>
<!-- =================== BUILD: Copy Static Files ======================= -->
<target name="build-static" depends="build-prepare">
<!-- Executable Commands -->
<copy todir="${jasper.build}/bin">
<fileset dir="src/bin" />
</copy>
<fixcrlf srcdir="${jasper.build}/bin" includes="*.sh" eol="lf"/>
<fixcrlf srcdir="${jasper.build}/bin" includes="*.bat" eol="crlf"/>
<chmod perm="+x" file="${jasper.build}/bin/jasper.sh"/>
<chmod perm="+x" file="${jasper.build}/bin/jspc.sh"/>
<!-- Common Extensions -->
<copy todir="${jasper.build}/jasper" file="${copy.crimson.jar}"/>
<copy todir="${jasper.build}/jasper" file="${copy.jaxp.jar}"/>
</target>
<!-- ================= BUILD: Compile Server Components ================= -->
<target name="build-main" depends="build-static">
<!-- Compile internal server components -->
<javac srcdir="src/share" destdir="${jasper.build}/classes"
debug="${compile.debug}" deprecation="${compile.deprecation}"
optimize="${compile.optimize}"
excludes="**/CVS/**">
<classpath refid="jasper.classpath" />
</javac>
</project>
The Maven Approach
- Build instructions written in XML
- Project Object Model (POM) based on convention over configuration
- File typically named
pom.xml
- Will automatically fetch dependencies
- Building happens through pluggins with sensible defaults
- A standard lifecycle directs the execution of plugins
Maven's Lifecycle
- validate: validate the project is correct and all necessary information is available.
- initialize: initialize build state, e.g. set properties or create directories.
- generate-sources: generate any source code for inclusion in compilation.
- process-sources: process the source code, for example to filter any values.
- generate-resources: generate resources for inclusion in the package.
- process-resources: copy and process the resources into the destination directory, ready for packaging.
- compile: compile the source code of the project.
- process-classes: post-process the generated files from compilation, for example to do bytecode enhancement on Java classes.
- generate-test-sources: generate any test source code for inclusion in compilation.
- process-test-sources: process the test source code, for example to filter any values.
- generate-test-resources: create resources for testing.
- process-test-resources: copy and process the resources into the test destination directory.
- test-compile: compile the test source code into the test destination directory
- process-test-classes: post-process the generated files from test compilation, for example to do bytecode enhancement on Java classes. For Maven 2.0.5 and above.
- test: run tests using a suitable unit testing framework. These tests should not require the code be packaged or deployed.
- prepare-package: perform any operations necessary to prepare a package before the actual packaging. This often results in an unpacked, processed version of the package. (Maven 2.1 and above)
- package: take the compiled code and package it in its distributable format, such as a JAR.
- pre-integration-test: perform actions required before integration tests are executed. This may involve things such as setting up the required environment.
- integration-test: process and deploy the package if necessary into an environment where integration tests can be run.
- post-integration-test: perform actions required after integration tests have been executed. This may including cleaning up the environment.
- verify: run any checks to verify the package is valid and meets quality criteria.
- install: install the package into the local repository, for use as a dependency in other projects locally.
- deploy: done in an integration or release environment, copies the final package to the remote repository for sharing with other developers and projects.
(Source: Apache Maven Lifecycle Reference (http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#Lifecycle_Reference).)
Automating System Configuration
- System configuration: installed packages, settings, running services
- Specify using tools like: Chef, Puppet, cfengine
- Executed specification brings system up to date
- No changes are performed by hand
Best Practices
- Put all the project's elements under the build system
- Minimize work through dependency analysis
- Modularize the build into several targets
- Common include files
- Explore parallel builds to reduce build times
- Provide a way to build from scratch
- Create a tinderbox build
Best Practices (Variables)
Use variables for varying elements:
- Directories
- Tool options
- Web sites
- Version strings
- OS dependencies
Further Reading
- Mark Burgess and Ricky Ralston.
Distributed resource administration using Cfengine.
Softw., Pract. Exper., 27(9):1083–1101, 1997.
- Sun Microsystems, Inc., Santa
Clara, CA.
Sun Studio 12:
Distributed Make (dmake), 2007.
Part No: 819-5273. Available online
http://docs.sun.com/app/docs/doc/819-5273 (http://docs.sun.com/app/docs/doc/819-5273). Accessed 2009-03-13.
- Stuart I. Feldman.
Make—a program for maintaining computer programs.
Software: Practice & Experience, 9(4):255–265, 1979.
- S. McIntosh,
B. Adams, and A.E. Hassan.
The evolution of ANT build systems.
In Mining Software Repositories (MSR), 2010 7th IEEE Working Conference
on, pages 42–51, may 2010.
(doi:10.1109/MSR.2010.5463341 (http://dx.doi.org/10.1109/MSR.2010.5463341))
- Diomidis Spinellis.
Software
builders.
IEEE Software, 25(3):22–23, May/June 2008.
(doi:10.1109/MS.2008.74 (http://dx.doi.org/10.1109/MS.2008.74))
- Diomidis Spinellis.
Don't
install software by hand.
IEEE Software, 29(4):86–87, July/August 2012.
(doi:10.1109/MS.2012.85 (http://dx.doi.org/10.1109/MS.2012.85))
- Jesse Tilly and
Eric M. Burke.
Ant: The
Definitive Guide.
O'Reilly Media, 2002.
Last change: Tuesday, June 11, 2013 4:07 pm