95a4dd2475
Two suites under tests/stress/, plus a tmux-friendly run_loop.sh runner. Both boot a fresh uvicorn on an isolated DB per cycle and log JSON line summaries to runs/. api_stress.mjs covers WS-level scenarios that the existing pytest suite does not exercise: 20-student happy path, late joiners with correct remaining_ms, mid-question disconnect, browser-sleep + wake to a different question_idx, cookie tampering and cross-session cookie reuse, duplicate student_id, bad submit (out-of-order, wrong idx, resubmit no-op), close-boundary race with auto-close, malformed JSON fuzz, and flaky reconnect. ui_stress.mjs drives the same flows in a real Chromium context via playwright: happy UI, sleep/wake by closing+reopening a context with the persisted cookie, document.cookie tampering attempt, and two browser contexts joining with the same student_id. Findings will be summarised in runs/summary.jsonl over time. One known issue surfaces from the fuzz scenario: app/room.py student_ws's receive_json call propagates JSONDecodeError out of the only try/except (which catches WebSocketDisconnect), killing that client's WS handler. Other clients are unaffected.
35 lines
2.7 KiB
Markdown
35 lines
2.7 KiB
Markdown
# Quiz portal stress harness
|
|
|
|
Adversarial frontend + API stress tests for the quiz portal. Built 2026-05-02.
|
|
|
|
## Files
|
|
|
|
- `lib.mjs` — shared helpers: server boot, cookie jar, `Student` and `Admin` WS wrappers, the fixed `STRESS_POOL`.
|
|
- `api_stress.mjs` — pure WS adversarial scenarios (no browser): happy path with 20 concurrent students, late join, mid-question disconnect, **sleep/wake to next question** (the phone-screen-sleep scenario), cookie tampering, cross-session cookie reuse, duplicate student_id, bad submits (out-of-order, wrong idx, resubmit), close-boundary race, malformed-JSON fuzz, flaky reconnect.
|
|
- `ui_stress.mjs` — Playwright/Chromium scenarios that exercise the real SPA: happy UI flow, sleep/wake by closing+reopening browser context with persisted cookie, cookie-tamper via `document.cookie`, two browsers with same student_id.
|
|
- `run_loop.sh` — bash wrapper that runs `api_stress.mjs` every cycle and `ui_stress.mjs` every `UI_EVERY` cycles (default 5), with a fresh random seed each time. Logs JSON summary lines to `runs/summary.jsonl` and full output to `runs/run-<timestamp>.jsonl`.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# One-shot
|
|
node api_stress.mjs # uses Date.now() seed
|
|
node api_stress.mjs 12345 8210 # explicit seed + port
|
|
node ui_stress.mjs # browser-based; HEADLESS=0 to watch
|
|
|
|
# Long-running loop in tmux
|
|
tmux new -d -s quiz_stress 'cd /home/ameer/RD/Projects/Apps/quiz/tests/stress && bash run_loop.sh'
|
|
tmux attach -t quiz_stress # to watch
|
|
tmux send -t quiz_stress C-c # to stop
|
|
```
|
|
|
|
Each cycle boots a fresh uvicorn on its own port and clean DB, runs scenarios, then tears down. Failures are recorded in the `failures` array of the per-cycle summary line.
|
|
|
|
## Known findings (tracked outside this dir)
|
|
|
|
- **Codex bug:** `app/room.py` `student_ws` (line ~87) and `instructor_ws` call `await websocket.receive_json()` whose JSON parsing can raise `JSONDecodeError`, but the surrounding `try/except` only catches `WebSocketDisconnect`. Result: a single malformed message kills that client's WS handler. The fuzz scenario in `api_stress.mjs` flags this consistently. Fix: wrap the receive in `try/except (JSONDecodeError, RuntimeError):` and either close cleanly or send `{"type":"error","code":"bad_message"}` and continue.
|
|
|
|
## Adding scenarios
|
|
|
|
Write an `async function name(server) { ... }` in `api_stress.mjs` (or `(server, browser)` for UI), add it to the `SCENARIOS` map / array, and re-run. Use `expect(cond, scenario, msg, extra)` for assertions and `note(scenario, msg)` for warnings that shouldn't fail the suite. **Critical pattern:** pre-register `waitFor` waiters BEFORE the action that triggers the message — `Student.waitFor(type)` only resolves on NEW messages, not cached ones, to avoid stale-state false passes.
|