5 · Functional QA & debugger review

TL;DR

Objective — drive every critical journey and role (automate repetitive public and post-session smoke tests with Playwright MCP), run live security tests, triage failures by severity, then mine Telescope/Sentry/logs and check for unmerged fixes — the live counterpart to the static audit.

Steps at a glance:

1. Run functional QA & runtime testing — Drive the running app across every critical journey and role. Automate the repetitive smoke tests with the Playwright MCP (navigate, fill non-secret forms, capture console errors, screenshot at each step); keep human eyes on judgment-heavy flows.
2. Mine the debugger tools — Mine the tools you wired in earlier for bugs that QA didn’t surface.
3. Run the SHOULD suites — cross-browser, mobile, load — The MUST suites gate launch; these catch what real users hit on other browsers, on phones, and under traffic.

User part + done-when

🔀 User part — browser-driven checks are agent-owned when automation is available after any required human login/password entry. The human owns account creation, password entry, real reset-link mailbox completion, sandbox email/payment triggers, physical-device checks, and judgment-heavy acceptance calls. The agent owns public auth renders, seeded @example.com reset-request confirmation, auth-guard redirects, logout after a human-created session, post-session dashboard sweeps, screenshots, console/log checks, and non-secret provider-console evidence.

Done when every MUST suite passes, runtime security tests hold, failures are triaged by severity, and the debugger review surfaces no unresolved CRITICAL (including unmerged fixes).  ·  ⏱ ~2–3 hr  ·  🔀 mixed (👤 journeys, 🤖 Playwright + log mining).

Background

This is the live counterpart to the earlier static audits: you drive the running app instead of reading the code. Automate every browser-safe check; keep humans only at password, account-creation, real inbox/payment, physical-device, and judgment gates.

flowchart LR
  Static[Static + security audits] --> Auto[Automated smoke Playwright MCP]
  Auto --> Human[Human gates only]
  Human --> UAT[User acceptance sign-off]

Steps

1. Run functional QA & runtime testing

Drive the running app across every critical journey and role. Automate the repetitive smoke tests with the Playwright MCP (navigate, fill non-secret forms, capture console errors, screenshot at each step); keep human hands only on account creation, password/secret entry, real inbox/payment triggers, physical-device proof, and judgment-heavy flows.

Suite	Priority	Covers
Functional testing	MUST	Auth, subscriptions, CRUD, emails, error pages
Security testing (runtime)	MUST	Rate limiting, IDOR, session handling, prod config
Performance testing	MUST	Page speed, TTFB, caching, load
User acceptance testing	MUST	End-to-end user journeys, onboarding
Cross-browser	SHOULD	Chrome, Firefox, Safari, Edge
Mobile & responsive	SHOULD	375 / 768 / 1024 / 1440px, touch targets
Load testing	SHOULD	Apache Bench / k6 on the selected non-production environment

Reference test matrices (functional + runtime security)

Authentication — split by actor instead of labeling the whole suite human-only. 🤖 Agent checks login/signup page render, auth-guard redirects, seeded @example.com password-reset request → confirmation screen, wrong-password / non-existent-email generic errors when no secret entry is needed, screenshots, console, logs, and logout after a human-created session. 👤 Human performs account creation, password entry, real reset-link mailbox completion, and any live credential transaction. After each required session exists, the agent resumes and verifies remember-me, dashboard routes, logout protection, and logs.

Subscription — /pricing shows correct plans/prices/currency, checkout with test card 4242 4242 4242 4242 → success, dashboard shows active plan, invoice email received, upgrade / downgrade-at-period-end / cancel, billing history visible.

Core CRUD (per entity) — create / edit / delete (with confirm) / list / search / paginate / export / report generate / report export / custom date range.

Error handling — custom pages:

Trigger	Expected
/nonexistent-page-xyz	Custom 404 page
Forced 500	Custom 500 page
Forbidden resource	Custom 403 page
Expired session / stale CSRF	Custom 419 page
Empty / invalid-email / excessive-length / special-char inputs	Inline validation, handled gracefully

Runtime security:

Test	Expected
10+ rapid failed logins	Rate-limit lockout after threshold
Password-reset token after 1 hour	Expired
Password-reset token reused	Single-use — rejected on second use
Session after password change	Old sessions invalidated
User A opens User B’s resource by ID (IDOR)	403 or redirect
Non-admin hits admin route	403 or redirect
Unauthenticated API call	401
Session cookies	Secure=true, HttpOnly=true

Who does this

🔀 User + Agent — the agent drives repetitive public and post-session smoke tests with the Playwright MCP; 👤 User performs login/account-creation/password-entry transactions, walks judgment-heavy journeys, and confirms each email actually lands in the inbox.

1. Drive each suite, then triage every failure by severity. Use browser automation for agent-owned auth checks (login/signup render, auth guard, seeded reset request, logout after human login) and post-session dashboard checks. Use the Stripe test card 4242 4242 4242 4242 for the full payment lifecycle (subscribe → upgrade → downgrade → cancel → billing history) after the human authorizes any sandbox trigger or password step. Confirm every transactional email lands in the inbox (mail-tester.com target 9–10/10) or record the real-inbox completion as a human-owned row in Zaj-BACKLOG.md with owner, trigger, review date, and acceptance criterion.

   • ✅ Every MUST suite passes; agent-owned auth checks are evidenced; the payment lifecycle, emails (9–10/10), and custom error pages all work or the remaining real-inbox/payment triggers are explicitly tracked as human-owned follow-ups.
   • ✅ Each failure is triaged — CRITICAL blocks launch, MEDIUM has a workaround (fix within a week), LOW is cosmetic.

Runtime security ≠ static audit

Functional security testing is the live counterpart to the earlier audit. Have the human perform any failed-login/password-entry sequence needed for rate-limit proof, then let the agent record non-secret evidence. The agent can still access another user’s resource by ID after an authorized session exists (expect 403) and confirm APP_DEBUG=false with no debug toolbar in the response.

Prove prod config at runtime — not just in .env

A debug toolbar leaking in production HTML exposes routes, queries, and env. Confirm it is absent in the live response and that logging is locked to error:

# Debug toolbar must NOT appear in production HTML
curl -s "https://<YOUR_DOMAIN>" | grep -c "debugbar"
# Expected: 0

# Production logging level (verify in the production .env / runtime config)
grep -E "^LOG_LEVEL=" .env
# Expected: LOG_LEVEL=error  (not debug)

Also confirm APP_ENV=production and APP_DEBUG=false at runtime.

2. Mine the debugger tools

Mine the tools you wired in earlier for bugs that QA didn’t surface.

Respect the Phase 7 monitoring choices

Respect the Phase 7 monitoring and observability choices. Read Admin-Local/1-Project/1-Info/monitoring-state.md first — if the project did not adopt Telescope or Sentry, mark those debugger-review rows N/A with the reason. Never install Telescope (or any observability tool) mid-audit — that contradicts the earlier monitoring decision and adds a tool at the worst moment. If a new observability tool is truly needed, create a Zaj-BACKLOG.md row with source Phase 10 functional QA, severity, trigger, effort, and acceptance criterion so Phase 11 backlog intake can classify it intentionally.

Review	What to find
Telescope exceptions	Hidden exceptions (local); categorize by severity
Slow queries	> 500ms = CRITICAL, > 100ms = HIGH; fix with eager loading / indexes
Sentry (production)	Unresolved issues, prioritized by events & users affected
Log file scan	Errors, warnings, deprecations across local + production logs
Git unmerged fixes	Fixes stranded on feature branches, never merged to develop

1. Work each review row, categorizing findings by severity and confirming no fix is stranded off develop.

   git branch -a --contains <hash> | grep -E 'origin/develop|origin/main|develop|main'
   # Expected: develop or main is listed — the fix is in the release line

   Also check for dependency bumps stranded off the release line — a composer.lock update merged only to a feature branch is a silent regression (e.g. a package fix that never reached develop):

   git log --oneline --all -- composer.lock | head -10
   # For any bump not obviously on develop/main:
   #   git branch -a --contains <hash> | grep -E 'origin/develop|origin/main' || echo "STRANDED: bump not in release line"

   • ✅ Telescope/Sentry/log findings are categorized, or marked N/A with the reason from monitoring-state.md; slow queries fixed; every fix commit is on develop.

Unmerged fixes are silent regressions

A fix that lives only on a feature branch is not in production. Check git branch -a --contains <hash> for every fix commit — if neither develop nor main appears, cherry-pick or merge it before launch.

3. Run the SHOULD suites — cross-browser, mobile, load

The MUST suites gate launch; these catch what real users hit on other browsers, on phones, and under traffic. Run them on the selected non-production environment, never production.

1. Cross-browser — drive the key journeys (homepage, public login/register screens, post-session routes after human sign-in, all non-secret form types, modals/dropdowns, AJAX) in Chrome, Firefox, Safari, and Edge, watching the console on each.

   • ✅ Every journey renders and works in all four browsers with a clean console; browserslist in package.json covers > 1%, last 2 versions, not dead.

2. Mobile & responsive — test each page at 375 / 768 / 1024 / 1440px in DevTools, then on at least one real iPhone + one real Android.

   • ✅ No horizontal scroll at any breakpoint; touch targets ≥ 44×44px; passes on a real iPhone (Safari) and a real Android (Chrome).

3. Load test (selected non-production only) — a quick Apache Bench pass, then a scripted k6 ramp if you need a realistic profile.

   ab -n 500 -c 50 https://<selected-nonproduction-domain>/        # quick baseline
   # realistic ramp: brew install k6 && k6 run load-test.js

   Monitor the server during the run (separate shell on the selected non-production environment):

   top                                         # CPU — target < 80%
   free -m                                     # Memory — should stay stable, no runaway growth
   # In a MySQL client:
   #   SHOW STATUS LIKE 'Threads_connected';   # not near max_connections
   #   SHOW GLOBAL STATUS LIKE 'Slow_queries'; # should not climb sharply under load

   Read the bottleneck:

   Symptom	Diagnosis	Quick fix
   High CPU	Un-cached PHP / hot code path	Enable OPcache
   Memory grows	Leak / unbounded workers	Fix leak, limit workers
   Slow DB	Missing indexes	Add indexes, add Redis cache
   Connection errors	Connection limit reached	Raise max_connections

   • ✅ On the selected non-production environment: avg response < 500ms, 95th percentile < 1s, error rate < 0.1% (requests/sec > 100, 0 failed).

ZajModules is evidence-backed N/A when absent

If packages/ZajModSys/ does not exist, mark this row N/A — no ZajModules package present and cite the test -d packages/ZajModSys evidence. If it does exist, run php artisan zajmodsys:list and confirm every module is discovered, enabled, and configured (its .env keys and config('zajmodules.*') values resolve).

Checklist

Do not mark this step done until every box below is checked.

• 🔀 MUST suites pass — functional, runtime security, performance, and UAT all green.
• 🔀 Payments + emails verified — agent-owned checks are evidenced; human-owned sandbox trigger / real-inbox moments are completed or tracked in Zaj-BACKLOG.md; test-card lifecycle works; transactional emails land (9–10/10).
• 🤖 Failures triaged — every failure tagged CRITICAL / MEDIUM / LOW.
• 🤖 Debugger review done — Telescope/Sentry/logs mined or marked N/A with reason; slow queries fixed.
• 🤖 No unmerged fixes — every fix commit confirmed on develop.
• 🔀 SHOULD suites run (selected non-production) — cross-browser (4 browsers), mobile (4 breakpoints + real devices), and load test all pass on the selected non-production environment.
• 🤖 ZajModules verified or N/A — zajmodsys:list clean, or test -d packages/ZajModSys proves N/A — no ZajModules package present.

→Next step

Pre-launch sign-off