Accelerating the distribution pipeline for Scaleform GFx, a high-performance rich media user interface and vector graphics engine.
Accelerating the Build and Packaging Process
May 16, 2008 – Scaleform Corporation
This article describes how IncrediBuild and the Xoreax Grid Engine (XGE) were used to accelerate the building and packaging phase of Scaleform’s distribution pipeline. Scaleform’s signature product, Scaleform GFx, is a lightweight, high-performance, rich media user interface and vector graphics engine. Using Scaleform GFx and Adobe Flash Studio, our customers have been able to change from a programmer-centric, static UI development environment to an artist-driven, dynamic Flash UI and animated texture production pipeline. Because we support all major gaming console systems as well as Windows, Mac OS, and Linux, Scaleform GFx is built across over 80 different configurations. Once built, the configurations must be compressed into zip files or installer executables.
Our distribution pipeline consists of four basic phases: Building, Packaging, Testing, and Posting. We build 86 different configurations consisting of our core GFx SDK, demos, and samples. These configurations are packaged into 10 different installers.
Running on a single 3.4 GHz machine, our Building process took approximately 2 hours and 40 minutes while the creation of the installers (Packaging phase) took just over 1 hour. The Testing phase can uncover problems or bugs that cause the Build and Packaging phases to be repeated before another round of testing. Without IncrediBuild and XGE, this rebuild process would create nearly 4 hours of down time between each Test cycle. This lengthy delay proved to be a problem in delivering quality software to our customers in a timely manner.
The main alternative to solve our problem was upgrading to better hardware. However, this solution had diminishing returns, as even the latest multicore PCs did not offer the speed benefits gained by adding another node to the IncrediBuild/XGE network. This is due to disk contention and other heavy system tasks that are not just processor dependent.
Through the use of IncrediBuild and XGE, we were able to distribute our build across our network to many multi-core machines. Using IncrediBuild also allowed linking on multiple cores on the host machine instead of just one for Visual Studio 2005, which itself allowed us to build our SDK n times faster on a machine with n cores (e.g., two times faster on a two-core machine, four times faster on a four-core machine).
By utilizing the XGE we were able to quickly and easily distribute the creation of GFx installation packages across multiple machines. Our largest installer takes only 8 minutes to create, but building all of them linearly on a single-CPU machine takes over an hour. When the installation packages are queued through XGE, the entire packaging process is reduced to 8 minutes, or the time required to build the largest package.
Integrating IncrediBuild to compile the GFx SDK was a simple task of having the IncrediBuild Agents installed across the network, then adding one line to our build script for every solution we wanted to distribute. The script also returns the result of each compilation, allowing us to have better error checking throughout our builds.
The Scaleform GFx installers are built by dynamically inserting a command line into a batch file, which will then queue the commands with XGE. The batch file is then passed to xgConsole, which quickly and efficiently handles the copying of many gigs of data across our network and the compression of that data to an installer less than 50MB in size.
The two phases affected by XGE were the Building and Packaging phases. Our typical time to compile the builds took 2 hours and 21 minutes; when using IncrediBuild, the time dropped down to 40 minutes. This was a net savings of 1 hour and 41 minutes, or a 71% reduction in compile time.
Our time spent packaging without distributing the installer creation was 1 hour and 3 minutes. When using XGE, the packaging time dropped down to 8 minutes, resulting in a net savings of 55 minutes, or an 88% reduction in packaging time.
This brought our total time down from 3 hours and 24 minutes to 48 minutes, reducing our building and packaging time by over 75%.
The XGE solutions have improved the speed of our distribution pipeline more than any other single solution. The 75% improvement resulted in quicker testing and ultimately faster distribution so that we can meet release dates without having to wait on a slow pipeline to complete. With Scaleform GFx becoming available on more platforms on a regular basis, as well as being integrated with an increasing number of major video game engines, it’s essential that our build process be streamlined and efficient. XGE plays a significant role in making this goal possible.
Scaleform Corporation is a leading provider of tools and middleware for the video game, embedded, mobile, and desktop markets. Scaleform licenses a suite of cross-platform, hardware-accelerated vector graphics engines and Flash UI toolkits, which allow users to design, develop, and deploy a variety of interactive entertainment content across a wide range of platforms and formats. For more information, please visit: www.scaleform.com.