add live stress harness, app-level admin login rate limit

tests/stress/live_accuracy.mjs: classroom-scale accuracy + latency
test that targets the deployed server (single-session, sid=main).
Logs in as admin via /admin/login, resets the session, joins N
students serially over HTTP, opens N student WebSockets in batches
of 8 (250ms apart) plus the instructor WS, then drives every
question through the admin "next" command. Each student picks
uniformly random A-D, sends the submit, waits for the submit_ack,
and records the round-trip latency. After session_ended, the script
verifies that every student whose pick == correct got score > 0,
every other submission got score == 0, and reports p50/p95/p99
ack latency. First live run: 50 students, 100 submits, 100% acks,
100% accuracy match, p99 555ms (≈intercontinental RTT to HK).

tests/stress/live_loop.sh: tmux-friendly loop that runs the live
test every 60s and appends a JSONL summary line per cycle to
runs/live_summary.jsonl. Mirrors the morning's api_stress run_loop
shape so per-cycle aggregates are easy to scrape.

app/rate_limit.py: tiny in-memory token bucket. Capacity + refill
in tokens/minute, keyed by client IP via X-Forwarded-For (with a
fallback to request.client.host). Process-local state — admin
login is the only user.

POST /admin/login: rate-limited at 10 attempts/minute/IP. Generous
for the legit instructor (who succeeds in 1-2 tries) and prohibitive
for brute force from a single attacker IP. Student endpoints
deliberately NOT rate-limited because campus students share NAT
gateways and IP-level limits would false-positive a whole class.

The bucket is per-app-instance (instantiated inside the router
factory), so test apps each get a fresh one and tests don't poison
each other.
This commit is contained in:
ameer
2026-05-03 00:23:07 +08:00
parent 7a483ad3ee
commit 2136286275
5 changed files with 483 additions and 1 deletions

72
tests/stress/live_loop.sh Normal file
View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
# Long-running live accuracy + latency loop.
# Each cycle resets the live session, runs the full live_accuracy.mjs
# test, parses the summary, and appends a JSON line to runs/live_summary.jsonl.
#
# Run:
# ADMIN_PW=$(cat /tmp/quiz-admin-pw.txt) tmux new -d -s quiz_live \
# 'cd /home/ameer/RD/Projects/Apps/quiz/tests/stress && \
# ADMIN_PW="$ADMIN_PW" bash live_loop.sh'
# Stop:
# tmux send -t quiz_live C-c # graceful, then tmux kill-session -t quiz_live
#
# Tunables:
# BASE_URL - default https://quiz.ahkhan.me
# N - default 50 students
# GAP_S - seconds between cycles (default 60)
# ADMIN_PW - required, the live admin password
set -uo pipefail
cd "$(dirname "$0")"
BASE_URL="${BASE_URL:-https://quiz.ahkhan.me}"
N="${N:-50}"
GAP_S="${GAP_S:-60}"
if [ -z "${ADMIN_PW:-}" ]; then
echo "ADMIN_PW must be set in env" >&2
exit 2
fi
mkdir -p runs
SUM="runs/live_summary.jsonl"
LOG="runs/live-$(date -u +%Y%m%dT%H%M%SZ).log"
echo "{\"event\":\"loop_start\",\"ts\":\"$(date -u +%FT%TZ)\",\"target\":\"$BASE_URL\",\"N\":$N,\"gap_s\":$GAP_S,\"log\":\"$LOG\"}" | tee -a "$SUM"
cycle=0
total_pass=0
total_fail=0
total_acks=0
total_submits=0
trap 'echo "{\"event\":\"loop_stop\",\"ts\":\"$(date -u +%FT%TZ)\",\"cycles\":'$cycle'}" | tee -a "$SUM"; exit 0' INT TERM
while true; do
cycle=$((cycle + 1))
ts=$(date -u +%FT%TZ)
printf '\n----- live cycle %d (%s) -----\n' "$cycle" "$ts" | tee -a "$LOG"
out=$(timeout 180 node live_accuracy.mjs "$BASE_URL" "$ADMIN_PW" "$N" 2>&1)
ec=$?
echo "$out" | tee -a "$LOG" >/dev/null
pass=$(echo "$out" | sed -n 's/.*Accuracy ok *: \([0-9]*\) \/ \([0-9]*\).*/\1/p')
total=$(echo "$out" | sed -n 's/.*Accuracy ok *: \([0-9]*\) \/ \([0-9]*\).*/\2/p')
fail=$(echo "$out" | sed -n 's/.*Accuracy fail *: \([0-9]*\)/\1/p')
acks=$(echo "$out" | sed -n 's/.*Acks received *: \([0-9]*\) \/.*/\1/p')
p50=$(echo "$out" | sed -n 's/.*p50=\([0-9.]*\) .*/\1/p' | tail -1)
p95=$(echo "$out" | sed -n 's/.*p95=\([0-9.]*\) .*/\1/p' | tail -1)
p99=$(echo "$out" | sed -n 's/.*p99=\([0-9.]*\) .*/\1/p' | tail -1)
max=$(echo "$out" | sed -n 's/.*max=\([0-9.]*\)$/\1/p' | tail -1)
mean=$(echo "$out" | sed -n 's/.*mean=\([0-9.]*\) .*/\1/p' | tail -1)
total_pass=$((total_pass + ${pass:-0}))
total_fail=$((total_fail + ${fail:-0}))
total_acks=$((total_acks + ${acks:-0}))
total_submits=$((total_submits + ${total:-0}))
echo "{\"event\":\"cycle\",\"ts\":\"$ts\",\"cycle\":$cycle,\"exit\":$ec,\"submits\":${total:-0},\"acc_ok\":${pass:-0},\"acc_fail\":${fail:-0},\"acks\":${acks:-0},\"mean_ms\":${mean:-0},\"p50\":${p50:-0},\"p95\":${p95:-0},\"p99\":${p99:-0},\"max\":${max:-0},\"running_pass\":$total_pass,\"running_fail\":$total_fail}" | tee -a "$SUM"
sleep "$GAP_S"
done