OpenStack’s Incredible Testing System is Behind Its Continued Success

Happy 3rd Birthday, OpenStack!

I want the community to know how integral the continuous integration (CI) system (TripleO), plus integration tests (Tempest), plus unit tests (per project), are to our success.  Previously I interviewed Monty Taylor on this topic and he had a ton of fabulous insight to share on how the CI system works.  However, in looking back on the last three years and trying to understand why OpenStack continues to grow and hit every milestone, I think we should “do the numbers.”

First up, notice the total number of unit and integration tests, which are well over 13,000 15,000.  And due to lack of time, I am even missing a few key projects like Swift, Ceilometer, and Heat (will try and update soon!). (UPDATED: Graphic below updated to include Heat and Swift.)

openstack-total-tests (1)

This is impressive, but perhaps most impressive is observing the trajectory of the creation of unit tests.  Just looking at Nova you can see that the community has been hard at work over the last three years adding test after test:


This is incredible velocity and it really tells us about the commitment of the OpenStack community to deliver a high quality, production-grade, Cloud OS kernel.

More importantly, the OpenStack infrastructure team’s continuous integration system is deploying and testing OpenStack over 700 times a day using the Tempest integration tests, which have doubled in the last year:

This is why we are able to move so fast and why no other Infrastructure-as-a-Service (IaaS) open source software development community will be able to catch us.

From the Cloudscaling engineering team:  thanks so much for the continuing hard work!