Shipping an AAA Game Amid COVID-19 Lockdown
Proletariat is a Boston-based, award-winning independent video game developer that creates games designed to change the way communities play together. In late 2019, only a few months prior to COVID-19, the company had completed a $20 million funding round to expand development and fuel hiring to support the launch of the highly anticipated game, Spellbreak, which was released in September 2020.
Since 2018, Proletariat and Incredibuild have partnered to accelerate engineering tasks across the production team, including C++ code builds, Unreal Engine’s Shader compilation, and CI-triggered cooking. Working together allowed Proletariat to iterate more quickly and, in turn, enable engineers to create more code and graphics improvements. This resulted in a higher quality product that still met the deadlines needed to ship Spellbreak.
With a pivot to working from home, the engineering team was faced with a new challenge: home connectivity was never going to match the speed and access the team had from the office. Utilizing Incredibuild to distribute tasks across team equipment was no longer possible due to varying distances from each other and inconsistent home bandwidth. The alternative option of connecting engineers to the machines that remained in the office turned out to be ineffective due to VPN limitations and hardware saturation. Having engineers working at home and lacking the distributed grid computing they were used to ultimately resulted in slower developer builds and artist shader compilations – a daunting reality as the team faced Spellbreak’s looming launch deadline.
“Employees’ home bandwidth could not adequately handle our usual data load traveling back and forth to our in-office resources. Massive amounts of data and VPN saturation were making it very hard to improve everybody’s workload all across the board. We needed a solution that would solve this challenge to ensure we met our launch deadline.”
Engineering Lead, Proletariat
Avida Michaud, Engineering Lead, Proletariat
The Proletariat engineering team needed a cloud-based solution that would address the challenges of having their entire team working from home. The team decided to partner with Incredibuild to leverage its cloud capabilities with a goal of reducing engineering processing time by 50 percent and to provide a stable and productive home-work environment.
Incredibuild’s cloud framework was put to action under Proletariat’s AWS account with a few simple configuration steps. On the one hand, it continued to be fully integrated with the toolchain developers and artists had used from home. On the other, Incredibuild started optimizing AWS to process the workloads for home employees. As soon as a heavy process load was triggered on an engineer’s laptop or desktop, Incredibuild automatically intercepted the processing, broke it up into multiple jobs, spun up EC2 instances to execute the jobs in parallel, and finally returned results back to the engineer’s equipment. To keep AWS costs at a minimum, Incredibuild cloud had shared available EC2 instances across processes.
As cost was also a consideration, Proletariat also started using the Incredibuild Enterprise Dashboard to monitor AWS EC2 usage while adjusting the number of cores allocated to engineers to maximize build acceleration.
“Finally, we felt confident shipping this game on time – from home.”
Engineering Lead, Proletariat
The Bottom Line
The partnership between Proletariat and Incredibuild resulted in the improved results both teams were looking for.
PS4 full cook, which had taken 174 minutes in a standalone setup, was cut by nearly half using Incredibuild cloud, down to 90 minutes. Additionally, a 25-minute code build had been shortened to 16 minutes.
By enabling Proletariat to deploy spot VMs, while maintaining fast-paced builds, the studio managed to substantially save infrastructure cost without compromising performance.
Code builds16 mins25 mins