apps/quiz
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tests/stress: add Node-based adversarial stress harness

Browse Source 
Two suites under tests/stress/, plus a tmux-friendly run_loop.sh
runner. Both boot a fresh uvicorn on an isolated DB per cycle and
log JSON line summaries to runs/.

api_stress.mjs covers WS-level scenarios that the existing pytest
suite does not exercise: 20-student happy path, late joiners with
correct remaining_ms, mid-question disconnect, browser-sleep + wake
to a different question_idx, cookie tampering and cross-session
cookie reuse, duplicate student_id, bad submit (out-of-order, wrong
idx, resubmit no-op), close-boundary race with auto-close, malformed
JSON fuzz, and flaky reconnect.

ui_stress.mjs drives the same flows in a real Chromium context via
playwright: happy UI, sleep/wake by closing+reopening a context with
the persisted cookie, document.cookie tampering attempt, and two
browser contexts joining with the same student_id.

Findings will be summarised in runs/summary.jsonl over time. One known
issue surfaces from the fuzz scenario: app/room.py student_ws's
receive_json call propagates JSONDecodeError out of the only
try/except (which catches WebSocketDisconnect), killing that client's
WS handler. Other clients are unaffected.
This commit is contained in:
ameer
2026-05-02 15:26:18 +08:00
parent 0f8824bd43
commit 95a4dd2475
8 changed files with 1095 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
tests/stress/run_loop.sh Executable file
View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Loop runner for the stress harness.
# Runs api_stress.mjs each cycle with a fresh random seed, and runs ui_stress.mjs
# every $UI_EVERY cycles (default 5). Logs JSON lines to runs/<timestamp>.jsonl.
# Run this in tmux: tmux new -d -s quiz_stress 'bash run_loop.sh'
set -uo pipefail
cd "$(dirname "$0")"
mkdir -p runs
UI_EVERY=${UI_EVERY:-5}
SLEEP_BETWEEN=${SLEEP_BETWEEN:-3}
LOG="runs/run-$(date -u +%Y%m%dT%H%M%SZ).jsonl"
SUM="runs/summary.jsonl"
echo "{\"event\":\"loop_start\",\"ts\":\"$(date -u +%FT%TZ)\",\"log\":\"$LOG\",\"ui_every\":$UI_EVERY}" | tee -a "$SUM"
cycle=0
total_pass=0
total_fail=0
total_warn=0
trap 'echo "{\"event\":\"loop_stop\",\"ts\":\"$(date -u +%FT%TZ)\",\"cycles\":'$cycle',\"total_pass\":'$total_pass',\"total_fail\":'$total_fail',\"total_warn\":'$total_warn'}" | tee -a "$SUM"; exit 0' INT TERM
while true; do
cycle=$((cycle + 1))
seed=$(( (RANDOM * 32768 + RANDOM) % 1000000 ))
port=$((8200 + (cycle % 50)))
printf '\n----- cycle %d (seed=%d port=%d) api -----\n' "$cycle" "$seed" "$port" | tee -a "$LOG"
out=$(timeout 120 node api_stress.mjs "$seed" "$port" 2>&1)
echo "$out" | tee -a "$LOG" >/dev/null
summary=$(echo "$out" | grep '"runner"' | grep '"summary"' | tail -1)
if [ -n "$summary" ]; then
p=$(echo "$summary" | sed -n 's/.*"pass":\([0-9]*\).*/\1/p')
f=$(echo "$summary" | sed -n 's/.*"fail":\([0-9]*\).*/\1/p')
w=$(echo "$summary" | sed -n 's/.*"warn":\([0-9]*\).*/\1/p')
total_pass=$((total_pass + ${p:-0}))
total_fail=$((total_fail + ${f:-0}))
total_warn=$((total_warn + ${w:-0}))
echo "{\"event\":\"cycle\",\"ts\":\"$(date -u +%FT%TZ)\",\"cycle\":$cycle,\"kind\":\"api\",\"seed\":$seed,\"port\":$port,\"pass\":${p:-0},\"fail\":${f:-0},\"warn\":${w:-0},\"running_pass\":$total_pass,\"running_fail\":$total_fail,\"running_warn\":$total_warn}" | tee -a "$SUM"
else
echo "{\"event\":\"cycle\",\"ts\":\"$(date -u +%FT%TZ)\",\"cycle\":$cycle,\"kind\":\"api\",\"seed\":$seed,\"port\":$port,\"status\":\"NO_SUMMARY\"}" | tee -a "$SUM"
fi
if [ $((cycle % UI_EVERY)) -eq 0 ]; then
printf '\n----- cycle %d (seed=%d port=%d) ui -----\n' "$cycle" "$seed" "$((port + 100))" | tee -a "$LOG"
out=$(timeout 180 node ui_stress.mjs "$seed" "$((port + 100))" 2>&1)
echo "$out" | tee -a "$LOG" >/dev/null
summary=$(echo "$out" | grep '"runner"' | grep '"summary"' | tail -1)
if [ -n "$summary" ]; then
p=$(echo "$summary" | sed -n 's/.*"pass":\([0-9]*\).*/\1/p')
f=$(echo "$summary" | sed -n 's/.*"fail":\([0-9]*\).*/\1/p')
w=$(echo "$summary" | sed -n 's/.*"warn":\([0-9]*\).*/\1/p')
total_pass=$((total_pass + ${p:-0}))
total_fail=$((total_fail + ${f:-0}))
total_warn=$((total_warn + ${w:-0}))
echo "{\"event\":\"cycle\",\"ts\":\"$(date -u +%FT%TZ)\",\"cycle\":$cycle,\"kind\":\"ui\",\"seed\":$seed,\"port\":$((port + 100)),\"pass\":${p:-0},\"fail\":${f:-0},\"warn\":${w:-0},\"running_pass\":$total_pass,\"running_fail\":$total_fail,\"running_warn\":$total_warn}" | tee -a "$SUM"
else
echo "{\"event\":\"cycle\",\"ts\":\"$(date -u +%FT%TZ)\",\"cycle\":$cycle,\"kind\":\"ui\",\"seed\":$seed,\"port\":$((port + 100)),\"status\":\"NO_SUMMARY\"}" | tee -a "$SUM"
fi
fi
sleep "$SLEEP_BETWEEN"
done