Release Management and Environment Operations at a Federal Contract Client

I joined a previous client engagement into a loosely defined "AWS DevOps" role, reporting initially to a DevOps manager whose branch-driven Jenkins pipeline was my first real exposure to triggering one CI job from another. As the project grew toward multi-service infrastructure and Terraform, DevOps moved under a new engineering manager who expected me to own the pipelines and understand and debug the Terraform behind them — not to architect a new platform from scratch on day one. That manager taught me the workspace model this project ran on: separate Terraform workspaces for development, QA, UAT, and production, selected by a Jenkins environment variable at build time.

Release management

I became the operational release owner for the platform's regular release cycle. On a working release, that meant coordinating a slush/code-freeze period, building the required applications, tracking exactly which service versions were live in which environment, deploying the needed versions to UAT, and waiting for QA — automated and manual — before production. Developers kept working on feature branches during the freeze, and the customer, project management, and I confirmed production readiness together before anything shipped.

Production releases went out Friday evenings at 8 PM Eastern, during low traffic, behind a maintenance notice that didn't take the site down. I ran the required Jenkins deployment jobs by hand. The environment used blue-green deployment, so if one required component failed, we could roll back just the successful components to keep the release internally consistent rather than shipping a half-updated platform. Once QA smoke-tested production and confirmed it, the maintenance notice came down and developers came off freeze.

CI/CD and security checks

Over time we added Slack notifications for build completion and failure, and started requiring Jira ticket IDs in every commit message — a convention that a developer on the team later enforced locally in tooling; I didn't build that enforcement myself. A successful QA deployment moved the associated Jira ticket from Development to Ready for QA automatically; a failed one left it in Development. I don't remember the exact plugin or mechanism behind either the build-trigger or the Jira-transition behavior closely enough to describe it precisely — the value was in the outcome, not tooling I could still walk someone through today.

npm audit ran in CI and could fail a build outright. I also built and operated a SonarQube server for the project; its results were visible to developers, but by design it wasn't a deployment gate — a source of information, not a blocker.

Environment and certificate ownership

Changes to the environment repo and SSM Parameter Store — endpoints, feature and config values, credentials, and other environment-specific settings — increasingly came to me over time, requested through Slack and merge requests. I worked with Python scripts a colleague had already built for this rather than authoring that tooling myself. The team intended to move toward requiring a dedicated Jira ticket for every environment/SSM change, restricted to DevOps, but that process got backlogged during the GitLab migration and was never fully implemented while I was there.

Certificate management followed a documented manual process: generate the certificate request, send it to the federal client for renewal, receive the renewed certificates back, and install them into the appropriate development and production environments.

Two incidents, two lessons

A certificate rollback. While making an environments-repo change in UAT for a developer, I unintentionally rolled back certificates. I caught it, but the fix required a follow-on production release visible to the customer. This happened during a period where exceptions to normal UAT controls had become common because developers needed to hit deadlines. The lesson I still apply: if you're bypassing the normal path, get another DevOps engineer to review the exception before you make it — don't make a risky process exception alone.

A build-system rollback. Separately, and unrelated to this project's later Jenkins-to-GitLab migration (see Rebuilding Software Delivery on GitLab Under a Fixed Government Deadline), our DevOps manager replaced the existing Jenkins declarative/Groovy pipeline with a Python-based build system shortly before going on vacation. Problems followed almost immediately. I restored the previous known-good Jenkins/Groovy path first, then spent about an hour tracing through the new Python implementation — enough to recognize it as a substantial lift-and-shift rather than an isolated bug, not enough to safely rebuild it myself. I left the deeper investigation until he was back. Development stayed operational the whole time. The lesson: restore known-good service first, then decide whether deeper debugging is actually justified.

Schedule risk isn't only an engineering problem

Before the federal client transition, I was often the buffer when upstream developer changes ran late. One time, changes arrived hours behind schedule; I kicked off a build and went to lunch, the build failed while I was out, a colleague stepped in to cover it, and project management wasn't happy with how the day went. Since then, my approach has shifted: I'll still help absorb a compressed timeline, but I say so explicitly to the developers and project managers involved, rather than silently inheriting the risk of someone else's delay. Being willing to unblock the team is different from becoming the invisible buffer that hides upstream schedule risk from the people who need to see it.