In Part I, we covered why our team at Amplified decided to make the switch to CI/CD for a project and what benefits CI/CD could provide to us. In Part II, we’ll explore what we used and the details of our process.
This implementation came about when we were faced with a very short turnaround time and needed to integrate new code continuously, make sure everything was smooth before it launched, and catch bugs as they occurred. Overall, the endeavor was successful.
We began by implementing the continuous development aspect of CI/CD on Github, an internet hosting provider for software development version control, which expanded in 2019 to include CI/CD functionality. Github’s open source nature also makes it easy to reuse and customize other workflows to fit one’s own project instead of building from scratch.
Through Github Actions, we were able to automate the software workflow and reduce the time needed for the developers to upgrade the app from staging to production. They could review the code, find and fix issues in an efficient manner, and manage branches effectively as well.
To be able to run the scripts with the secret files in the correct format, as well as interact with 3rd parties such as Google Cloud Platform and Firebase hosting, we had to use premade GitHub actions. For the frontend, it would deploy to Firebase Hosting; for the backend, it would deploy to Google App Engine.
Staging and Master Branches
One of the first decisions we had to make was whether we wanted staging and master to be different branches or just one. In the end, we decided to make them different branches. A push to master would automatically merge into staging and push to staging; however, we wanted them to be separate in order to stop deploying from master to staging at any time if we need a more stable staging — for example, for hotfixes.
Staging to Production Data
Another consideration was whether staging should point to production data. We decided against this, because we wanted the option to test without affecting user data.
Frequency of Pushes
The frequency of pushes was yet another question, and we considered pushing daily to staging/production. We decided to base the frequency more on the status of the work in staging. Staging is constantly pushing from when we merge to master, and we push to production whenever we are “confident” in staging.
We do want to be pushing to production often, so we might implement a max interval later. More important for now was to create a clear product process and QA process.
Testing Production Data
We also considered whether we should copy production data to test, but decided against it for now. Because staging is sending out real emails, we would need to pre-process PII (personally identifiable information). Eventually, we may make a “staging-prod.portal.com,” which stages code but points to production data for us to test.
PRs (Pull Requests)
Our goal was atomic, small PRs, so we aimed for PRs to the feature branches. PRs from forks to OT/feature-branch and OT/master would all be small and modular, easy to review. PRs from OT/feature-branch to OT/master would be large, but all code should already have been reviewed.
We also opted to approve everything, so no changes were allowed after PR approvals. There would be no PRs or merges directly to production ever.
Development Environment
We knew that eventually, we wanted to be on a single platform, either Google Cloud Platform or Amazon Web Services. However, we weren’t ready to make that move yet, so for now we set up our development process in our current environment. Eventually, we’ll move to one platform.
Within the current system, each engineer works on a separate feature branch, which they deploy to staging only when they’re sure it’s been thoroughly tested and is stable enough to reach production.
In staging, the QA team tests again. If it passes all the tests, the feature can reach production. If there are launch-blocking issues found in staging, we roll back the feature to its initial feature branch.
When do we deploy?
As we explained in Part I, we used to deploy on Fridays, which put a strain on our team to deliver and fix bugs right before the weekend. With the new process, deployments to production happen daily — except for Fridays. Each day, the QA team tests the staging branch with the new additions. If everything from staging is stable, we deploy to production.
Thanks to our successful implementation of the CI/CD process, the team got in a flow of constant and organized feature delivery after launching a client’s app. By implementing a new structure, they were able to continue the development process behind the scenes, while also providing new and stable functionality to the real end-users.
