Skip to content
prod e051e98
Browse

2 · Roll back

Objective — when you can’t fix production fast enough, restore the last good release instead. The deployer keeps previous releases on disk, so rolling back is usually a 30-second symlink switch — no code changes, no merge.

Rolling back feels like giving up, but it’s the responsible move when a proper fix would take too long. A deploy doesn’t overwrite the old code — the deployer publishes each release into its own folder and just points a current symlink at the newest one. “Rollback” means pointing that symlink back at the previous folder. The old code is still sitting there, intact.

That’s why rollback is faster and safer than a hotfix: there’s no new code to get wrong. The one thing it does not undo is database migrations — if a migration changed the schema, switching code back won’t reverse it, and forcing it can lose data. That caveat drives the decision flow below.

flowchart TD
A{Production broken} --> B{Was it the<br/>LAST deploy?}
B -->|Yes| R[dep rollback production<br/>30 sec]
B -->|No| T{Know the last<br/>good tag?}
T -->|Yes| D[dep deploy --tag=vX.X.X]
T -->|No| RB[Rescue branch<br/>from a known-good tag]
A --> M{Is it a DATABASE<br/>migration issue?}
M -->|Yes| W[Do NOT just roll back code —<br/>see the migration caveat]

Pick the lightest option that fits. Option 1 covers most incidents — reach for 2 or 3 only when the bad release isn’t the most recent one.

  1. Option 1 — deployer rollback (fastest). Use when the last deploy caused the issue. Switches the symlink to the previous release — instant, no code changes.

    Terminal window
    dep rollback production
    # Expected: the live symlink now points at the previous release; site recovers
    • ✅ Production loads a working release within ~30 seconds; 👤 confirms in a browser.
  2. Option 2 — deploy a specific tag. Use when you know which version was last stable but it isn’t the immediately previous release.

    Terminal window
    git tag --list "v*" --sort=-v:refname | head -10
    dep deploy production --tag=v[LAST_GOOD]
    # Expected: production now runs the named known-good tag
    • ✅ The chosen tag is live and 👤 confirms the site works.
  3. Option 3 — rescue branch (full rollback). Use when you need to go back several versions or the options above don’t work.

    Terminal window
    git checkout -b rescue-from-v[LAST_GOOD] v[LAST_GOOD]
    git push origin rescue-from-v[LAST_GOOD]
    dep deploy production --branch=rescue-from-v[LAST_GOOD]
    git tag rescue-$(date +%Y-%m-%d) && git push origin rescue-$(date +%Y-%m-%d)
    # Expected: a rescue branch from the known-good tag is deployed and tagged
    • ✅ The rescue release is live, tagged, and 👤 confirms the site works.

The decision tree is simple: last deploy broke it → Option 1; you know the good tag → Option 2; you need to go back far → Option 3.

A code rollback does not undo migrations. If a migration is what broke production, switching code back will not fix it — and the wrong move here causes data loss.

  1. Check what would roll back, then choose a safe recovery. Only roll back a migration batch if you’re certain it drops nothing you need.

    Terminal window
    php artisan migrate:status
    # Option A: rollback last batch — ONLY if you're sure it loses no data
    php artisan migrate:rollback
    # Option B: restore from backup (hosting panel or your backup system)
    # Expected: migrate:status lists which migrations are in the last batch
    • ✅ Either the unsafe batch is safely reversed, or the database is restored from a known-good backup.

A rollback is temporary — it stops the bleeding but leaves the root cause unfixed. Capture what happened while it’s fresh, then fix it properly through the normal staged flow.

  1. Write a short incident note and queue the real fix. Record what broke, why, and how you restored it, then plan the permanent fix via the normal flow.

    • ✅ An incident note exists (what happened · root cause · resolution · action items), and the proper fix is queued for the staged develop → staging → production path, not another rushed hotfix.

The rollback bought you time; it didn’t buy you a fix. Return to develop, find the real root cause, and ship it the safe way via Continuous Deploy.

Do not mark this step done until every box below is checked.

  • 🤖 Code restored — a known-good release is live via deployer rollback, a tag, or a rescue branch.
  • 👤 Verified live — production loads, Sentry is clean, users can access the app and pay.
  • 🔀 Database handled — if a migration was involved, it was safely reversed or restored from backup (👤 decision).
  • 👤 Incident documented — what happened, root cause, and resolution are written down.
  • 🤖 Real fix queued — the permanent fix is planned for the normal staged flow, not another rushed hotfix.