3 · Harden, verify & sync (P3–P4)

TL;DR

Objective — lock down the installer-friendly 777 permissions the deployer leaves behind, confirm production schema matches staging, then fast-forward every branch back to a shared base and tag the release — so you have a clean, drift-free, rollback-anchored production state.

User part + done-when

👤 User part — almost none: every step here is terminal-runnable over SSH and git. A person only needs to investigate if a real schema difference surfaces, or to redo superadmin config when a fresh installer was used.

Done when production storage permissions are tightened, migrate --pretend reports nothing pending, the staging↔production schema diff is clean, every branch is fast-forwarded to production, and the release tag is pushed.  ·  ⏱ ~1 hr  ·  🤖 agent-runnable.

Background

Two short passes catch the most common day-one production failures: the deployer may leave installer-friendly 777 permissions behind, and schema drift between environments causes runtime errors and data corruption — production must match staging.

Steps

1. Lock down permissions (P3)

Tighten the installer-friendly permissions once the installer is done.

1. Reset storage permissions and re-lock the .env.

   ssh <SSH_PRODUCTION_ALIAS> "cd ~/domains/<DOMAIN>/deploy/shared/storage \
     && find . -type d -exec chmod 775 {} \; && find . -type f -exec chmod 664 {} \;"
   ssh <SSH_PRODUCTION_ALIAS> "chmod 640 ~/domains/<DOMAIN>/deploy/shared/.env"
   # Expected: storage dirs become 775, files 664, and .env is 640

   • ✅ Storage directories are 775, files 664, and the .env is 640.

2. Verify schema parity (P3)

Production must match staging — a logical schema diff is cleanest.

1. Check pending migrations — expect Nothing to migrate (or only those explicitly left pending).

   ssh <SSH_PRODUCTION_ALIAS> "cd ~/domains/<DOMAIN>/deploy/current && php artisan migrate --pretend"
   # Expected: "Nothing to migrate" (or only intentionally-pending migrations)

   • ✅ No unexpected pending migrations.

2. Compare staging vs production schema. A logical schema diff is cleanest; a filtered migrate:status/dump diff works without extra tooling. If real differences appear, stop and investigate before continuing.

   • ✅ The staging↔production schema diff is clean (or every difference is explained and resolved).

3. Sanity-check vendor migration file counts — and triage a mismatch, don’t panic. Compare the count of vendor migration files on each host; a mismatch is often expected, not a fault.

   STAGING_COUNT=$(ssh <SSH_STAGING_ALIAS> "cd ~/domains/<DOMAIN>/deploy/current \
     && find vendor -name '*.php' -path '*/database/migrations/*' 2>/dev/null | wc -l | tr -d ' '")
   PROD_COUNT=$(ssh <SSH_PRODUCTION_ALIAS> "cd ~/domains/<DOMAIN>/deploy/current \
     && find vendor -name '*.php' -path '*/database/migrations/*' 2>/dev/null | wc -l | tr -d ' '")
   echo "Staging: $STAGING_COUNT | Production: $PROD_COUNT"
   [ "$STAGING_COUNT" = "$PROD_COUNT" ] && echo "Match" || echo "Mismatch — triage below"
   # Expected: "Match", OR a mismatch you can explain via the table

   Situation	Cause	Action
   Production has more files	Stale composer cache on the server	dep clear_composer_cache production, then redeploy
   Staging has more, diff is only dev packages (debugbar, telescope, dusk)	Dev deps not deployed under --no-dev	Expected and correct — no action
   Staging has more, diff shows non-dev packages	Deploy didn’t finish cleanly	Redeploy: dep deploy production

   • ✅ Counts match, or the difference is explained by the table (dev-only packages absent under --no-dev is expected).

4. Export the production schema baseline into your project DB folder for future diffs.

   • ✅ A production schema baseline is saved for future comparison.

Dump noise ≠ schema difference

Raw SQL dumps include server-version headers, AUTO_INCREMENT values, and charset notation that vary between hosts. Filter those out before judging a diff — but never wave away a real table/column/index change as “just a newline”.

3. Capture server-side changes — ServerSync (P4.5)

The web installer and first-run writes create files on the production server that aren’t in git yet (the installed marker, generated config, vendor-published assets). Capture them back — but production capture is far more dangerous than staging, so review every change by hand.

Production ServerSync is review-only — never auto-merge

On production, a ServerSync that deletes a file can wipe live state. Never auto-merge a production ServerSync PR. Review every change individually — especially every D (deleted) entry — and re-confirm the clear_paths ↔ GIT_ONLY_PATHS symmetry (step 1) before applying.

Never accept a deletion of any of these — discard it with git checkout HEAD -- <path> if it appears:

.env.example   composer.lock   package-lock.json   CLAUDE.md
deploy.php      Admin-Local/**  .github/**
.claude/        .cursor/        .agent/

The inverse — storage cruft deletions are usually correct. If a vendor bundled files under storage/ that should never have been tracked, ServerSync proposing to delete them is a legitimate cleanup — accept those. The rule is protect git-only project files, allow storage-tracked-by-accident files to go. If you can’t justify a D entry, keep the file.

1. Audit clear_paths ↔ GIT_ONLY_PATHS symmetry first — and STOP on any gap. clear_paths in deploy.php drifts over time (AI configs, docs, templates). If the production ServerSync workflow’s GIT_ONLY_PATHS fell behind, the capture will propose deleting git-only files — and on production those deletions go live. Run the audit and do not trigger the capture until it passes.

   CLEAR_PATHS=$(awk '
     /add\(.clear_paths./ { capture=1; next }
     capture && /\]\);/ { capture=0 }
     capture { print }
   ' deploy.php | grep -oE "'[^']+'" | tr -d "'" | sort -u)
   
   parse_git_only_paths() {
     awk '
       /^[[:space:]]*GIT_ONLY_PATHS:/ { capture=1; next }
       capture && /^[[:space:]]*$/ { capture=0 }
       capture && /^[[:space:]]*#/ { next }
       capture && /^[[:space:]]*[A-Z_]+:/ { capture=0 }
       capture { print }
     ' "$1" | tr ' ' '\n' | grep -v '^[[:space:]]*$' | grep -v '^>-$' | sort -u
   }
   
   PROTECTED=$(parse_git_only_paths .github/workflows/server-sync-production.yml)
   MISSING=$(comm -23 <(echo "$CLEAR_PATHS") <(echo "$PROTECTED"))
   if [ -n "$MISSING" ]; then
     echo "STOP — these clear_paths are NOT protected by GIT_ONLY_PATHS in server-sync-production.yml:"
     echo "$MISSING" | sed 's/^/  /'
     echo "Add them to GIT_ONLY_PATHS, commit, merge to the default branch, then re-run."
   else
     echo "OK — every clear_path is protected; safe to trigger production ServerSync"
   fi
   # Expected: "OK — every clear_path is protected ..." — fix and re-run until it does

   • ✅ The audit prints OK; no clear_paths entry is unprotected (or the gap was added to GIT_ONLY_PATHS, merged, and the audit re-run until clean).

2. Run the capture against production, exactly as in Phase 5 · ServerSync capture but pointed at the production alias — it opens a PR rather than committing directly.

   • ✅ A ServerSync PR is open with the production-side file changes for review (or there were no server-side changes to capture).

3. Review the diff file-by-file. Approve additions/edits that belong in git; reject anything that would delete live state or re-pull a clear_paths file. Re-run the clear_paths ↔ GIT_ONLY_PATHS symmetry audit before merging.

   • ✅ Every D (deleted) file is individually justified, the symmetry audit passes, and only intended changes merge.

4. Sync branches & tag the release (P4)

Bring every branch back to a shared base, then stamp an immutable marker.

1. Confirm no branch has diverged, then fast-forward. Check each branch is an ancestor of production first — --ff-only will fail loudly on divergence, but the precheck tells you which branch and lets you stop before touching any.

   git fetch origin
   for B in main staging develop; do
     git merge-base --is-ancestor "$B" production \
       && echo "$B OK — fast-forwardable" \
       || echo "$B DIVERGED — investigate before syncing"
   done
   # Expected: all three print "OK — fast-forwardable"; STOP on any "DIVERGED"

   Only once all three report OK, fast-forward each and push:

   git checkout main      && git merge production --ff-only && git push origin main
   git checkout staging   && git merge production --ff-only && git push origin staging
   git checkout develop   && git merge production --ff-only && git push origin develop
   # Expected: each branch fast-forwards to production and pushes cleanly

   • ✅ The precheck shows all three branches OK, and main, staging, develop all point at production.

2. Tag the release on production and push the tag — this is your rollback and audit anchor.

   git checkout production
   git tag -a v${VERSION} -m "Release v${VERSION}"
   git push origin v${VERSION}
   # Expected: the annotated tag is created and pushed to the remote

   • ✅ v${VERSION} is tagged on production and pushed.

3. Finalize. Re-enable migrations in the deploy config for future deploys, update project status/docs to “live”, and return to develop with a clean tree.

   • ✅ Migrations re-enabled, docs marked “live”, and develop is checked out with a clean tree.

Reconfigure on a fresh install

If you used the fresh installer, the production admin panel is empty — redo the superadmin setup (branding, SMTP, plans, legal pages) on production, this time with live payment keys and production credentials. If you copied the staging DB, skip this and just spot-check that no staging URLs leaked into stored content.

5. Record & snapshot the release (P7)

Four closing tasks make the release reproducible and auditable: redeploy if code changed during hardening, record the release in the changelog, snapshot the schema as a versioned baseline, and capture the complete vendor-customization manifest.

1. Redeploy only if code changed since the P2 production deploy.

   git log production..develop --oneline -- "*.php" "*.js" "*.css" "*.blade.php" "deploy.php" "composer.json" "composer.lock"
   # Empty   → nothing changed; the branch sync in step 4 is enough.
   # Commits → deploy through the chain: develop → staging (deploy staging) → production (deploy production) → main.

   • ✅ Output is empty (no redeploy needed), or the changed code was deployed through staging → production and main re-synced.

2. Record the release. On develop, move [Unreleased] items into a new dated [vX.Y.Z] section of CHANGELOG.md (categories: Added, Changed, Fixed, Security, Configured), and set ProjectCard.md to Production: Live · Current Release: vX.Y.Z.

   • ✅ CHANGELOG.md has a dated [vX.Y.Z] section and ProjectCard.md reads “Live”.

3. Export a versioned schema baseline so future vendor-update diffs compare against a known-good production schema.

   Never pass production credentials on the CLI (-u 'mysql://user:pass@…' lands in shell history and ps). Use the same gitignored env pattern as Phase 5 · Atlas Cloud:

   ssh -L 3308:127.0.0.1:3306 <production-alias> -N &
   # .env.atlas (gitignored — see Phase 3 optional Atlas): ATLAS_PRODUCTION_URL="mysql://USER:PASS@127.0.0.1:3308/DBNAME"
   # Load without printing: direnv allow   OR   set -a && source .env.atlas && set +a
   [ -n "$ATLAS_PRODUCTION_URL" ] || { echo "ABORT: set ATLAS_PRODUCTION_URL in .env.atlas (never on the command line)" >&2; exit 1; }
   
   mkdir -p Admin-Local/1-Project/3-ProjectDB/2-Snapshots/v${VERSION}-release
   atlas schema inspect -u "$ATLAS_PRODUCTION_URL" --format '{{ sql . }}' \
     > Admin-Local/1-Project/3-ProjectDB/2-Snapshots/v${VERSION}-release/production.sql
   # Non-Atlas alternative: mysqldump --no-data via the same SSH tunnel + ~/.my.cnf (Phase 4)

   • ✅ A 2-Snapshots/v${VERSION}-release/production.sql baseline is saved (and copied into 1-Current/).

4. Capture the vendor-customization manifest — the complete list of every app file you changed from the pristine vendor. On a first full setup this is the capstone record: it’s your vendor-update playbook later (these are the files a new vendor release can conflict with) and the audit trail of your whole customization surface. vendor/ is gitignored, so this is the app-file diff against the frozen author-v${VERSION} baseline created in Phase 2.

   mkdir -p Admin-Local/3-Versions/1-v${VERSION}/2-Modifications
   git diff --name-status "author-v${VERSION}..HEAD" \
     > Admin-Local/3-Versions/1-v${VERSION}/2-Modifications/vendor-drift-v${VERSION}.txt
   # Expected: a list of A/M/D paths — the full set of changes from the vendor original
   echo "Modified vendor files — each MUST be recorded in _CUSTOMIZATIONS.md:"
   git diff --name-status "author-v${VERSION}..HEAD" \
     | awk '$1 ~ /^M/ {print $2}' \
     | grep -vE '^(packages/ZajModules/|resources/vendor-customizations/|Admin-Local/)' \
     || echo "  (none — every edit is inside a customization dir)"

   • ✅ A 2-Modifications/vendor-drift-v${VERSION}.txt manifest is saved, and every modified vendor file it lists is recorded in _CUSTOMIZATIONS.md (or wrapped in ZAJ:BEGIN/END markers).

Checklist

Do not mark this step done until every box below is checked.

• 🤖 Permissions locked — storage 775/664, .env 640 on production.
• 🤖 No pending migrations — migrate --pretend reports nothing unexpected.
• 🤖 Schema parity verified — staging↔production diff clean, production baseline exported.
• 🔀 ServerSync reviewed (P4.5) — production ServerSync PR reviewed file-by-file (no blind deletes), symmetry audit passed — or confirmed there were no server-side changes to capture.
• 🤖 Branches synced — main, staging, develop all fast-forwarded to production.
• 🤖 Release tagged — v${VERSION} annotated on production and pushed.
• 🤖 Redeploy check (P7) — git log production..develop is empty, or the changed code was redeployed through staging → production.
• 🤖 Release recorded (P7) — CHANGELOG.md has a dated [vX.Y.Z] section and ProjectCard.md reads “Live”.
• 🤖 Schema baseline snapshotted (P7) — 2-Snapshots/v${VERSION}-release/production.sql exported.
• 🤖 Vendor-drift manifest captured (P7) — vendor-drift-v${VERSION}.txt saved; every modified vendor file is recorded in _CUSTOMIZATIONS.md.

→Next step

Monitor & day-2