I recently had the opportunity to look into some fundamental Continuous Integration and Deployment techniques we use to build nrfcloud.com. We rely heavily on AWS, so it was natural to look into using AWS CodeBuild as a task runner for our private repositories.
I previously had a fantastic experience working with Travis CI, but it adds a hefty fixed price to an infrastructure. On AWS CodeBuild costs occur only during the runtime of jobs (which are run in Docker containers). This, and a potential better integration with other AWS services, gave me the initial reason to look into using it as the main task runner for our TypeScript projects.
One precondition is to have tests. The benefits of tested software, especially in a fast-moving environment, are manifold and I have been practicing TDD for many years. Tests will ensure that a piece of software still works if you change a dependency.
Another invaluable helper is Greenkeeper: it will send you a pull-request if a dependency is updated, which will in turn trigger a CI build. If everything works fine, you are left with merging a PR. It's a 1-click solution to updating dependencies and it makes working with dependencies fun again.
Since npm 5 there is also a new and improved lockfile:
package-lock.json, which is a huge improvement over the old shrinkwrap. It ensures that when running
npm ci only the exact dependencies described in the lockfile will be installed. This enables reproducible builds builds of our package and we can be certain that we have the exact same executed code in production that we ran when testing.
Greenkeeper has an addon called
greenkeeper-lockfile which takes care of updating the lockfile just for the dependency after a successful test run.
A changed dependency also calls for a new release of the host package, and this in turn will trigger updates and releases to packages that depend on the host package. Managing this manually would simply be insane, or just plain monotonous. But thankfully to
semantic-release the process of release new versions can also be automated! (I have explained the whole the process in these slides in 2016).
All of the tools above require Node.js v8 andw when I started to run the first of our packages on AWS CodeBuild the official Node.js Docker image did only support Node.js v4 and v6. This has been fixed on May 2nd and CodeBuild now has an official Node.js v8.11 image.
I already had begun to add CodeBuild support to
env-ci, which Greenkeeper uses to detect if it is running in a CI environment and in which. Since the official images uses an outdated Git version (1.9) it required some back and forth until we figured it out.
The next task was to get
greenkeeper-lockfile to work in AWS CodeBuild runs which I have implemented in this PR. Also here I needed to deal with a quite old Git version and figuring out CodeBuilds way of cloning the repo. It also does not provide all the necessary details about the build in the environments so I needed to persist that during the build, since
greenkeeper-lockfile will apply changes to the repository -- it is updating the lockfile and commit this back to the PR.
buildspec.yml like this we finally can have automated library releases and dependency management on AWS CodeBuild!
version: 0.2 phases: install: commands: - echo "updating git ..."; apt-get update; apt-get install -y python-software-properties software-properties-common; add-apt-repository ppa:git-core/ppa; apt-get update; apt-get install -y git - npm install -g npm@ - npm install -g nRFCloud/greenkeeper-lockfile - npx greenkeeper-lockfile-update - npm ci --no-audit build: commands: - npm test post_build: commands: - npx greenkeeper-lockfile-upload - npx semantic-release
- we update git to the latest stable version, since
semantic-releasenow depends on >=2.0.0
- we run
npm install -g npm@to ensure we are running the latest stable npm
- we use our patched version of greenkeeper-lockfile
- we disable audit on
npm cito not slow down the install
And here is the result on a real PR: