# Operate in production

**Goal:** the runbooks you follow when something is already wrong: written to be executed under pressure, so each one is self-contained and starts from symptoms, not context.

## What you'll have when you're done

- A rollback runbook for the deploy path you actually use, rehearsed against its preconditions.
- Monitoring that works within the platform's log constraints, plus the two things only you can add (app-level telemetry, external uptime checks).
- A rotation inventory covering every credential your setup created.

## Prerequisites

- A production app launched via the [go-live checklist](/source-cms/deploy/go-live/)
- [acli](/start-here/glossary/#acli) authenticated ([CLI quickstart](/cloud-platform/cli/quickstart/)) for the API-driven steps

## Steps

### Roll back a deploy

The right runbook depends on how the bad code got there. All four assume one precondition set at go-live: you can identify the previous known-good state (a tag, an artifact name, or a commit). There is no stored-artifact list to browse on this platform (the artifact API answers "Artifacts are not available for Node.js applications hosted on Acquia Cloud Next"): the deployed ref is the artifact. A previously pushed tag is still a ref on the platform, and switching back to it redeploys with no rebuild, which is why the tag runbook comes first.

**BYO CI tag deploys.** Each tag's artifact was pushed as its own ref to the Acquia remote, so the previous release still exists on the platform:

1. Identify the previous good tag from your git history (`git tag --sort=-creatordate | head -5`).
2. Switch the production environment back to it with one command: `acli api:environments:code-switch <env-id> "tags/<previous-tag>"` (the raw API equivalent is in the [BYO guide](/source-cms/deploy/external-ci/)).
3. Verify the live URL serves the previous version (a version marker in your footer or a `meta` tag makes this check trivial; add one if you had to guess).
4. The bad tag stays in history; fix forward on a branch and cut a new tag. Never retag.

**Branch-artifact deploys (Dev/Stage, or production without tag gating).** The artifact branch (`main-build` from the BYO workflow, `pipelines-build-<branch>` from Pipelines, `<branch>-codestudio-build` from Code Studio) is overwritten by every run, so the previous build no longer exists as an artifact. Roll back through git:

1. `git revert` the bad commit (or reset the branch to the last good commit if your team allows force-pushes to that branch).
2. Push; the CI run rebuilds and redeploys the artifact branch.
3. This takes a full CI run, minutes not seconds. Faster when available: if your CI stores build artifacts (GitHub Actions retains them for the configured `retention-days`) and your deploy job is manually dispatchable, redeploy the last good run's stored artifact without rebuilding anything.
4. If production runs on branch deploys and the latency hurt, that is the argument for moving to tag gating ([CI/CD guide](/cloud-platform/ci-cd/guide/#promote-through-environments)).

**Pipelines deploys.** The artifact is the `pipelines-build-<branch>` branch on your application. A branch rollback is the revert-and-rerun above (`pipelines start` re-runs without waiting for a push); if you deploy tag artifacts, switch back to the previous tag's artifact as in the tag runbook.

**Cloud Platform Drupal deploys.** Code rollback is a tag switch, but data may have moved under it (update hooks, editors). The decision rule and the paired database restore live in the [code-workflow guide's rollback runbook](/cloud-platform/code-workflow/guide/#roll-back-a-release); it is the one path where rolling back code alone can make things worse.

**If you deploy through Code Studio**, the same tag/branch split applies with its `<tag>-codestudio-build` / `<branch>-codestudio-build` artifact names; see the [Code Studio](/start-here/glossary/#code-studio).

### Monitor the running app

What the platform provides on Front End Hosting – Advanced:

- **Streamable and downloadable environment logs** in the Cloud UI: your app's stdout/stderr, which makes structured application logging (one JSON line per request/error) the habit that pays off most on this platform.
- **`acli app:log:tail`** for a live stream in your terminal during deploys and incidents. The Node app's stdout may not reach `app:log:tail` or the downloadable logs; the Cloud Platform user interface's live log viewer is the reliable surface for application output.

What it does not provide: **no log forwarding**, no built-in metrics dashboards, no alerting. The platform will not tell you the site is down.

Two consequences to act on:

1. **External observability happens at the application layer.** The platform can't ship logs out, but your app makes outbound HTTP like any Node process: an OpenTelemetry or vendor SDK exporting from inside the app (errors, traces) works normally. That is an app dependency you own, not a platform feature; keep the exporter async so telemetry never blocks a response.
2. **Uptime checking is external by definition.** Point any uptime monitor at the production URL and a content page (not just `/`, which the readiness probe already covers). This is the "someone would notice" from the go-live checklist.

### Credential rotation inventory

The canonical rotation procedure (create new, deploy, verify, revoke old) is in the [auth guide](/source-cms/authenticate/guide/#rotate-credentials-with-zero-downtime); this table is the inventory of what your setup actually holds. Rotate on schedule and immediately on any suspected leak.

| Credential | Where it lives | Rotation path |
|---|---|---|
| Production site API client (`DRUPAL_CLIENT_ID`/`SECRET`) | Environment `Variables` + your CI's secret store | [Auth guide runbook](/source-cms/authenticate/guide/#rotate-credentials-with-zero-downtime); update both stores |
| Per-environment site API clients (Dev/Stage) | Each environment's `Variables` | Same runbook, per environment |
| Preview client + `PREVIEW_SECRET` | Frontend environment variables | Same runbook; re-share the preview link |
| Revalidation webhook secret (`REVALIDATION_SECRET`) | Frontend environment + registered webhook URL | [Deploy guide procedure](/source-cms/deploy/guide/); both halves together |
| Pipelines API token | The `pipelines` client's local config (per machine, via `pipelines configure`) | New token at cloud.acquia.com/a/profile/tokens, re-run `pipelines configure`, revoke the old token |
| Code Studio wizard API token (only if you use Code Studio) | Code Studio CI/CD variables (created by `acli codestudio:wizard`) | [Code Studio](/start-here/glossary/#code-studio) |
| Code Studio SSH key (`CODESTUDIO_`-prefixed, same condition) | Cloud Platform SSH keys + Code Studio | [Code Studio](/start-here/glossary/#code-studio) |
| External CI deploy key (`DEPLOY_SSH_KEY`) | CI secret store + Acquia profile SSH keys | Generate a new pair, add public key to the profile, update the CI secret, remove the old key |
| External CI Cloud API client (`CLIENT_ID`/`CLIENT_SECRET`) | CI secret store | Recreate per the [Cloud Platform API page](/cloud-platform/cli/platform-api/), update the secret, revoke the old client |
| DDEV global Acquia API token | Plaintext global DDEV config, per developer machine | New token at cloud.acquia.com/a/profile/tokens, re-run the `ddev config global` command, revoke the old token |

The rows are per credential, not per person: a departing teammate's tokens (their DDEV token, their profile SSH keys) are rotations too.

## When something goes wrong

**Rolled back but the symptom persists**: the problem isn't the code you rolled back. Check in order: environment variables changed around the same time (the `Variables` page shows current values, and changing one restarts the environment), a CMS-side change (content model, CORS, API client), CDN-cached responses masking the fix.

**`Switch code` shows no previous tag artifact to select**: the release was never tagged (branch-deploy path), so no per-tag artifact exists. Use the branch runbook's revert-and-rebuild; then adopt tag deploys so the next rollback has an artifact to land on.

**Rotation broke production even though the new credential is correct**: one store was updated and the other wasn't; runtime and build-time values live in different places ([deploy guide step 1](/source-cms/deploy/guide/)). Update the environment `Variables` and your CI's secret store, then redeploy so prerendered pages pick up the build-time value.

## Next steps

- [Go-live checklist](/source-cms/deploy/go-live/): the pre-launch pass that makes these runbooks rehearsed instead of improvised.
- [Code-workflow guide rollback runbook](/cloud-platform/code-workflow/guide/#roll-back-a-release): the code-plus-database case in full.
- [CI/CD guide](/cloud-platform/ci-cd/guide/): the promotion and governance setup that keeps rollbacks rare.