nvrl/holy-flavor-rater
1
0 Fork 0
mirror of https://github.com/narrrl/holy-flavor-rater.git synced 2026-11-03 16:41:27 +01:00
Code Issues Packages Projects Releases Wiki Activity

rust fmt, updated ci, hooks and docs

Browse Source
This commit is contained in:
nvrl committed 2026-06-15 17:08:46 +02:00
1 parent 8c120ae78d
commit a4c9b5bad1
39 files changed
+691 -902
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.env.example
+11 -29
View File
@@ -2,40 +2,22 @@
# Your primary domain (e.g., example.com). Used to derive ALLOWED_HOSTS, CORS, and CSRF settings.
DOMAIN="localhost"
# Set to a long, random string in production (e.g. 50+ chars). REQUIRED in prod.
SECRET_KEY="django-insecure-replace-this-in-production"
# Signs the SimpleJWT-compatible tokens. Set to a long, random string in
# production. Keep it CONSTANT across deploys/restarts or all tokens invalidate.
SECRET_KEY="replace-this-in-production"
# Which settings module to load. Use holy_backend.settings.dev locally,
# holy_backend.settings.prod in production (Docker compose sets this).
DJANGO_SETTINGS_MODULE=holy_backend.settings.dev
# --- Background jobs (Rust in-process scheduler) ---
# Lets the backend own api_job scheduling. EXACTLY ONE running instance may set
# this true, or scheduled jobs fire twice against the shared DB.
RUST_ENABLE_SCHEDULER=true
# Set to "false" in production for security and performance
DEBUG=true
# Set `Secure` on auth cookies (true in production).
JWT_AUTH_COOKIE_SECURE=true
# Custom URL for the Django Admin interface (e.g., my-secret-admin/)
ADMIN_URL="admin"
# --- Celery / Redis ---
# Broker and result backend for background jobs. Docker compose defaults to the bundled redis service.
# DB 0 is used by Celery; DB 1 is used by Django cache (see DJANGO_CACHE_URL below).
CELERY_BROKER_URL=redis://redis:6379/0
CELERY_RESULT_BACKEND=redis://redis:6379/0
# Run tasks synchronously (useful for local dev or CI). Unset in production.
# CELERY_TASK_ALWAYS_EAGER=true
# Django cache backend URL. Shares the redis instance on a different DB number.
DJANGO_CACHE_URL=redis://redis:6379/1
# Flower (Celery monitoring UI) — exposed on :5555. Format: "user:password".
# CHANGE BEFORE DEPLOY.
FLOWER_BASIC_AUTH=admin:change-me
# Worker/beat containers set this in docker-compose.yml; do not set globally.
# SKIP_INIT=true
# tracing/log filter for the Rust backend.
# RUST_LOG=info,holy_backend_rs=info,sea_orm=warn
# --- Advanced Networking (Optional, derived from DOMAIN if unset) ---
# ALLOWED_HOSTS=example.com,another-domain.com
# CSRF_TRUSTED_ORIGINS=https://example.com
# CORS_ALLOWED_ORIGINS=https://example.com
# FRONTEND_URL=https://example.com
# VITE_API_URL=https://example.com/api/
.gitea/workflows/ci.yml
+22 -26
View File
@@ -51,39 +51,35 @@ jobs:
image: catthehacker/ubuntu:act-latest
env:
SECRET_KEY: "ci-test-key"
DEBUG: "false"
ALLOWED_HOSTS: "localhost"
DJANGO_SETTINGS_MODULE: holy_backend.settings.dev
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
with:
python-version: '3.14.3'
cache: 'pip'
cache-dependency-path: backend/requirements-dev.txt
components: rustfmt, clippy
- name: Install Dependencies
run: pip install -r requirements-dev.txt
working-directory: backend
- name: Cache cargo
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
backend-rs/target
key: cargo-${{ hashFiles('backend-rs/Cargo.lock') }}
- name: Ruff Lint
run: ruff check .
working-directory: backend
- name: Format Check
run: cargo fmt --check
working-directory: backend-rs
- name: Ruff Format Check
run: ruff format --check .
working-directory: backend
- name: Clippy
run: cargo clippy --all-targets
working-directory: backend-rs
- name: Django System Check
run: python manage.py check
working-directory: backend
- name: Check Migrations
run: python manage.py makemigrations --check --dry-run
working-directory: backend
- name: Build
run: cargo build --locked
working-directory: backend-rs
- name: Run Tests
run: pytest
working-directory: backend
run: cargo test --locked
working-directory: backend-rs
MIGRATION.md
-191
View File
@@ -1,191 +0,0 @@
# Django → Rust migration runbook (seamless, reversible)
Migrate the running Django backend to the Rust backend (`backend-rs/`) with
**zero downtime** and a **per-step rollback**. Endpoint groups cut over one at a
time behind nginx; either backend can serve any request, so rollback is just the
reverse edit and data never forks.
There is a driver script that performs every step below:
`scripts/cutover.sh` (run `scripts/cutover.sh` with no args for the menu). Do the
steps manually or let the script do them — both edit the same files.
---
## Why it's safe
Both backends share one substrate, so a request is serviceable by either:
| Shared thing | How |
|---|---|
| Database | same `backend/db.sqlite3` (WAL on), bind-mounted into both containers |
| Media | same `backend/media/`; nginx serves `/media/` straight from the mount (`:ro`) |
| Auth | same `SECRET_KEY` → SimpleJWT HS256 tokens verify on both; cookies cross-valid |
| Token blacklist | shared `token_blacklist_*` tables → logout/refresh consistent across a flip |
**The one invariant — never violate:** exactly **one** scheduler may own the
`api_job` table at a time. Django's Celery `beat` owns it until Phase 3; the Rust
in-process scheduler (`ENABLE_SCHEDULER`) stays **false** until then. Never both,
or scheduled jobs fire twice.
---
## Prerequisites
1. **`SECRET_KEY` in `.env` must equal Django's exact value.** If it differs, all
existing JWTs break the instant you cut over auth. Verify before starting.
2. Back up first: `bash scripts/backup.sh` (snapshots SQLite + media).
3. Optionally enable **live nginx reloads** so flips don't rebuild the frontend
image. Add to the `frontend` service in `docker-compose.yml`:
```yaml
volumes:
- ./frontend/nginx.conf:/etc/nginx/conf.d/default.conf
- ./frontend/proxy_params.conf:/etc/nginx/proxy_params.conf
```
then `docker compose up -d frontend`. Without this, each phase rebuilds the
frontend image (`docker compose up -d --build frontend`) — slower, but still
no API downtime. (`scripts/cutover.sh dev-mount` prints these lines.)
---
## Phase 0 — Deploy Rust in shadow (no traffic)
Bring up `holy-rust` beside Django. It only reads; nginx still sends all traffic
to Django.
```bash
docker compose up -d --build backend-rs # or: scripts/cutover.sh phase0
```
Smoke-test it directly, bypassing nginx:
```bash
docker exec holy-frontend wget -qO- http://holy-rust:8001/health/ # or: scripts/cutover.sh smoke
```
Spot-check a few endpoints against `holy-rust:8001` and compare to Django.
**Rollback:** `docker compose stop backend-rs`.
---
## Phase 1 — Reads
Lowest risk: read-only, no auth side effects.
Flip these locations in `frontend/nginx.conf` from `django_backend` →
`rust_backend`, then reload:
- `/api/categories/`
- `/api/flavors/`
- `/api/banners/`
```bash
scripts/cutover.sh phase1
```
Watch `docker logs holy-rust` and your error rate for a few minutes.
**Rollback:** `scripts/cutover.sh rollback 1`.
---
## Phase 2 — Auth + writes
JWTs and cookies are cross-valid, so logged-in sessions survive the flip.
Flip:
- `/api/auth/`
- `= /api/token/` (legacy JWT alias)
- `/api/ratings/`
- `/api/replies/`
- `/api/users/`
- `/api/notifications/`
- `/api/tickets/`
```bash
scripts/cutover.sh phase2
```
**Rollback:** `scripts/cutover.sh rollback 2`.
---
## Phase 3 — Admin + jobs (the careful one)
This is the only step that touches the scheduler invariant. Do it in order.
1. Flip `/api/admin-custom/` → `rust_backend`, reload.
2. **Stop Django's scheduler/workers** so nobody schedules:
```bash
docker compose stop beat worker flower
```
3. **Hand scheduling to Rust:** set `RUST_ENABLE_SCHEDULER=true` in `.env`, then
recreate the Rust container:
```bash
docker compose up -d backend-rs
```
4. Confirm single ownership:
```bash
docker logs holy-rust | grep "job scheduler ENABLED"
```
```bash
scripts/cutover.sh phase3 # does 1-4, prompts before the handover
```
At no point are both schedulers on → no double-run.
**Rollback:** `scripts/cutover.sh rollback 3` — sets `RUST_ENABLE_SCHEDULER=false`,
recreates `backend-rs`, restarts `beat worker flower`, flips admin back to Django.
---
## Phase 4 — Retire Django
Point the catch-all at Rust. Rust also serves OpenAPI + Swagger, so `/api/schema`
goes with it.
1. Flip `location /api/` and `/api/schema` → `rust_backend`, reload:
```bash
scripts/cutover.sh phase4
```
Django now receives only `/admin/` + `/static/` (its own admin site). `/media/`
is already served straight from nginx, independent of either backend.
2. Decide the Django admin site's fate. The app's admin UI uses
`/api/admin-custom/` (now on Rust), so Django `/admin/` is usually unneeded:
- **Keep it:** leave `backend` up for `/admin/` + `/static/`.
- **Drop it:** `docker compose stop backend`, then delete the `/admin/` and
`/static/` location blocks from `nginx.conf` and reload.
3. Once Django is fully off, stop Redis (only a Celery broker + Django cache —
Rust uses neither):
```bash
docker compose stop redis
```
4. Eventually remove `backend`, `worker`, `beat`, `flower`, `redis` from
`docker-compose.yml`.
**Rollback:** `scripts/cutover.sh rollback 4` (before removing services).
---
## Verify anytime
```bash
scripts/cutover.sh status # current routing per group + scheduler ownership
```
---
## Notes / caveats
- **Rate-limit counters reset at cutover.** Django's `django_ratelimit` lives in
Redis; Rust's throttle is in-process. Acceptable — windows are minutes/hours.
- **Reloads:** if the conf is baked into the image (default), a flip rebuilds the
frontend image. Bind-mount the conf (see Prerequisites) for instant
`nginx -s reload`. The script auto-detects which mode you're in.
- **Canonical data:** `backend/db.sqlite3` + `backend/media/` are host bind
mounts — never recreated by rebuilds. Keep them; they're the source of truth
the whole time.
- The same content lives in `backend-rs/README.md` ("Production cutover runbook")
as the in-tree reference.
README.md
+47 -76
View File
@@ -18,7 +18,7 @@ A high-density, community-driven platform for "Holy Energy" enthusiasts. Browse
## 🛠️ Setup & Installation
### Option 1: Docker (Recommended)
The easiest way to get the full stack (Backend + Frontend + DB) running behind a production-ready proxy.
The easiest way to get the full stack (Rust backend + Frontend) running behind a production-ready proxy.
1. **Clone the repository:**
```bash
@@ -28,62 +28,48 @@ The easiest way to get the full stack (Backend + Frontend + DB) running behind a
2. **Environment Configuration:**
```bash
cp .env.example .env
# Edit .env with your SMTP settings and Domain
# Edit .env: set SECRET_KEY, DOMAIN, and SMTP settings
```
3. **Start Services:**
```bash
docker compose up -d --build
```
*Note: On first startup, the backend runs `migrate` + `collectstatic`, then seeds the flavor catalog, legacy data, and banner configurations **once** (gated by a `/app/.seeded` marker so subsequent restarts don't repeat the work). Re-run individual seeds later from the admin Jobs tab.*
The stack brings up two containers: `backend-rs` (`holy-rust:8001`, the Rust API) and `frontend` (Nginx — serves the SPA, proxies `/api` to the backend, serves `/media` from the shared mount). `docker-compose.override.yml` adds the Traefik production overlay.
The stack brings up six containers: `redis` (broker + cache), `backend` (Gunicorn), `worker` (Celery worker), `beat` (Celery beat w/ `django_celery_beat.DatabaseScheduler`), `flower` (Celery monitoring UI on `:5555`), and `frontend` (Nginx).
The Rust backend reads the existing `backend/db.sqlite3` directly (no migration step). Seeding (flavor catalog, legacy data, banners) runs as background jobs — trigger them from the admin Jobs tab or let the scheduler run them.
### Option 2: Local Manual Setup (Development)
#### Backend
1. **Navigate and Install:**
```bash
cd backend
python -m venv venv
source venv/bin/activate # Linux/Mac
pip install -r requirements.txt
```
2. **Initialize DB:**
```bash
python manage.py migrate
python manage.py sync_flavors
python manage.py seed_legacy_flavors
python manage.py seed_banners
```
3. **Run Server:**
```bash
python manage.py runserver
```
*By default `holy_backend.settings.dev` is active, which sets `CELERY_TASK_ALWAYS_EAGER=true` — tasks run inline with no broker required. To exercise the full async path locally, start a Redis server and run `celery -A holy_backend worker -l info` + `celery -A holy_backend beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler` in separate terminals.*
```bash
cd backend-rs
cp .env.example .env # set SECRET_KEY; point DATABASE_URL at ../backend/db.sqlite3
cargo run # listens on 0.0.0.0:8001
```
#### Frontend
1. **Navigate and Install:**
```bash
cd frontend
npm install
```
2. **Run Dev Server:**
```bash
npm run dev
```
```bash
cd frontend
npm install
npm run dev
```
> The Rust backend uses the existing SQLite database under `backend/db.sqlite3`. The `backend/` directory is retained as the data store (db, `media/`, `banners/`) and historical reference for the retired Django implementation — it is no longer run.
---
## 🏗️ Architecture
The Holy Flavor Rater uses a modern **decoupled architecture**:
The Holy Flavor Rater uses a **decoupled architecture**:
### Backend (Django REST Framework)
- **Models:** Custom User model, Flavors (with category constraints), Ratings (with mention parsing), and Dynamic Banners.
- **Async jobs:** Celery workers backed by Redis handle Shopify syncs, duplicate cleanup, legacy seeding, banner seeding, database backups, and transactional email. Scheduling is owned by `django_celery_beat` (`PeriodicTask` + `IntervalSchedule`) and editable from the admin UI Jobs tab.
- **Caching & rate-limiting:** Django cache uses Redis (`RedisCache` on DB 1). `django_ratelimit` consumes the same cache.
- **Security:** JWT authentication (SimpleJWT) with refresh-token rotation, CSRF protection, and email-verified account activation.
- **Monitoring:** `/health/` endpoint for the backend; Flower UI on `:5555` for queue + worker state (basic-auth gated).
### Backend (Rust — Axum + SeaORM)
- **Web/DB:** Axum 0.8 over SeaORM (SQLite), utoipa for OpenAPI + Swagger UI. Lives in `backend-rs/`.
- **Models:** Custom User, Flavors (with `UniqueConstraint(name, category)`), Ratings (with `@mention` parsing), Tickets, Notifications, and Dynamic Banners.
- **Background jobs:** An in-process tokio scheduler runs Shopify syncs, duplicate cleanup, legacy seeding, banner seeding, and database backups (`src/jobs/`). Status is logged to the `api_job` table; intervals are editable from the admin UI Jobs tab. Replaces the old Celery + beat + Redis stack.
- **Security:** SimpleJWT-compatible HS256 auth with refresh-token rotation + blacklist, email-verified activation, an in-process rate limiter, constant-time code comparison, and Django-parity password validators.
- **Monitoring:** `GET /health/` (runs `SELECT 1`) for the docker healthcheck.
See **`backend-rs/README.md`** for the full endpoint matrix, parity notes, and security details.
### Frontend (React 19 + TypeScript)
- **State Management:** React Hooks and local state for high performance.
@@ -98,77 +84,62 @@ The Holy Flavor Rater uses a modern **decoupled architecture**:
### Integrity Protection
This project includes a **Pre-Push Git Hook** to ensure code quality. It automatically:
1. Attempts a full Frontend build (`tsc` + `vite build`).
2. Runs Backend system checks and validates missing migrations.
2. Runs Rust backend checks (`cargo check`).
**To install the hooks:**
```bash
bash scripts/install-hooks.sh
```
### Management Commands
Each is also exposed as a Celery task (`api.<name>`) and triggerable from the admin Jobs tab.
- `python manage.py sync_flavors`: Syncs the latest products from the official Holy Energy API.
- `python manage.py cleanup_duplicates`: Merges duplicate entries and maintains rating integrity.
- `python manage.py seed_legacy_flavors`: Loads retired flavors from `legacy/*.json`.
- `python manage.py seed_banners`: Updates procedurally generated banner configurations from `backend/banners/*.json`.
- `python manage.py backup_db` (`--full` for media too): Creates a consistent SQLite snapshot in `backend/backups/`.
### Background Jobs
Run by the scheduler on their interval, or triggered from the admin Jobs tab (`POST /api/admin-custom/{pk}/trigger_job/`):
- `sync_flavors`: Syncs the latest products from the official Holy Energy (Shopify) catalog.
- `cleanup_duplicates`: Merges duplicate entries and maintains rating integrity.
- `seed_legacy`: Loads retired flavors from `legacy/*.json`.
- `seed_banners`: Updates procedurally generated banner configurations from `backend/banners/*.json`.
- `backup_db`: Creates a consistent SQLite + media snapshot in `backend/backups/`.
#### Categories & `recategorize_flavors`
#### Categories
`sync_flavors` derives a flavor's category from the Shopify `product_type` (e.g. `42 - Syrup Bottle` → **Syrup**), stripping the numeric prefix and form-factor word. Any new drink line Holy ships **auto-creates** its category on the next sync. Packs, shakers, merch, stickers and other non-drink types funnel to **Packs and other**.
`sync_flavors` derives a flavor's category from the Shopify `product_type` (e.g. `42 - Syrup Bottle` → **Syrup**), stripping the numeric prefix and the form-factor word. Any new drink line Holy ships **auto-creates** its category on the next sync — no code change needed. Packs, shakers, merch, stickers and other non-drink types funnel to **Packs and other**.
One catch: `sync_flavors` only assigns a category when it **creates** a row. Existing flavors keep their original category, so a categorization change won't retro-apply to rows already in the DB. Use the one-off command below to fix mis-shelved rows. It is run manually (not exposed as a Celery task / Jobs-tab entry) and is **dry-run by default** — nothing is written without `--apply`.
```bash
python manage.py recategorize_flavors # dry-run: report what would move
python manage.py recategorize_flavors --apply # persist the moves
python manage.py recategorize_flavors --apply --merge-collisions # also fold name-clashes
```
It re-fetches the catalog, matches rows by `external_id`, and re-runs the exact `sync_flavors` categorization logic. Rows not in the catalog (discontinued) are never touched. When a move would collide with an existing flavor of the same name in the target category (`UniqueConstraint(name, category)`), it is skipped and reported unless `--merge-collisions` is passed, which folds the rows via `merge_flavors` (preserving ratings and replies).
`sync_flavors` only assigns a category when it **creates** a row — existing flavors keep their original category. To re-shelve already-stored rows, the retired Django app's `recategorize_flavors` management command remains available under `backend/` for occasional manual use (it is not part of the Rust job registry).
---
## 💾 Backups & Restoration
### Create a Backup
To create a full snapshot of your database and user-uploaded media:
```bash
bash scripts/backup.sh
```
This will create a `.tar.gz` file in `backend/backups/`.
Creates a `.tar.gz` (DB + media) in `backend/backups/`. WAL-safe: uses `sqlite3 .backup` when available, otherwise copies the DB plus its `-wal`/`-shm` sidecars. The backend also runs `backup_db` on its own schedule.
**Automating with Cron:**
Add this to your `crontab -e` to backup every night at 2:00 AM:
**Automating with Cron** — nightly at 2:00 AM:
```bash
0 2 * * * /path/to/Holy-Flavor-Rater/scripts/backup.sh
```
### Restore from Backup
1. Extract the archive: `tar -xzf backup_filename.tar.gz`
2. Stop the containers: `docker-compose down`
2. Stop the containers: `docker compose down`
3. Replace the current files:
```bash
cp extracted_folder/db.sqlite3 backend/db.sqlite3
cp -r extracted_folder/media/* backend/media/
```
4. Restart: `docker-compose up -d`
4. Restart: `docker compose up -d`
---
## 🌍 Production Configuration
Ensure your `.env` contains:
- `DEBUG=false`
- `DJANGO_SETTINGS_MODULE=holy_backend.settings.prod`
- `SECRET_KEY`: A long random string.
- `ALLOWED_HOSTS`: Your domain (e.g., `holy.narl.io`).
- `FRONTEND_URL`: For one-click email verification links.
- `CELERY_BROKER_URL` / `CELERY_RESULT_BACKEND`: Default `redis://redis:6379/0` works inside the compose network.
- `DJANGO_CACHE_URL`: Default `redis://redis:6379/1` (DB 1 to avoid colliding with celery).
- `FLOWER_BASIC_AUTH`: `user:password` for the Flower UI — **change from the default before exposing**.
Restrict Flower's port `5555` (or reverse-proxy it behind basic-auth + TLS) since it exposes task payloads and worker internals.
Set these in `.env` (see also `backend-rs/.env.example`):
- `SECRET_KEY`: long random string that signs JWTs — **keep it constant** or all tokens invalidate.
- `DOMAIN`: your domain (e.g. `holy.narl.io`); derives `FRONTEND_URL` / CORS defaults.
- `FRONTEND_URL`: base URL for one-click email verification links.
- `JWT_AUTH_COOKIE_SECURE=true`: set `Secure` on auth cookies.
- `RUST_ENABLE_SCHEDULER=true`: lets the backend own `api_job` scheduling. **Exactly one** running instance may have this enabled.
- `EMAIL_*`: SMTP settings for transactional email (unset `EMAIL_HOST` logs to console).
## 📄 License
MIT
backend-rs/.env.example
+14 -14
View File
@@ -1,44 +1,44 @@
# Path to the SAME sqlite file Django uses (strangler: both backends share one DB).
# Relative paths resolve from the process CWD; absolute is safest.
# Path to the sqlite database. Relative paths resolve from the process CWD;
# absolute is safest. Use mode=rw in prod so a wrong path fails loudly.
DATABASE_URL=sqlite://../backend/db.sqlite3?mode=rwc
# MUST equal Django's SECRET_KEY so SimpleJWT tokens verify across both backends.
SECRET_KEY=change-me-to-match-django
# Signs the SimpleJWT-compatible tokens. Keep this CONSTANT across deploys and
# restarts — changing it invalidates every issued access/refresh token. (Use the
# value the retired Django app used so tokens already in the wild stay valid.)
SECRET_KEY=change-me
# Where this service binds.
BIND_ADDR=0.0.0.0:8001
# Media URL prefix Django serves under (matches MEDIA_URL = "media/").
# Media URL prefix (matches the historical Django MEDIA_URL = "media/").
MEDIA_URL=media/
# Filesystem root for uploaded media (avatars). Must point at the SAME directory
# Django uses (MEDIA_ROOT) so both backends read/write the same files.
# Filesystem root for uploaded media (avatars).
MEDIA_ROOT=../backend/media
# Base URL of the frontend, used to build email links (verify-email, reset).
# Matches Django's FRONTEND_URL.
FRONTEND_URL=http://localhost:5173
# --- Email (lettre) -------------------------------------------------------
# Leave EMAIL_HOST unset to log emails to the console (dev). Set it to enable
# real SMTP. These mirror Django's EMAIL_* settings.
# real SMTP.
#EMAIL_HOST=smtp.example.com
EMAIL_PORT=587
EMAIL_USE_TLS=true
EMAIL_USE_SSL=false
#EMAIL_HOST_USER=noreply@example.com
#EMAIL_HOST_PASSWORD=change-me
# Accept self-signed SMTP certs (mirrors api.email_backend.InsecureEmailBackend).
# Accept self-signed SMTP certs.
EMAIL_SKIP_CERT_VERIFICATION=false
# From address; falls back to EMAIL_HOST_USER then noreply@localhost.
#DEFAULT_FROM_EMAIL=noreply@example.com
# Set `Secure` on auth cookies (true in prod, like Django's JWT_AUTH_COOKIE_SECURE).
# Set `Secure` on auth cookies (true in prod).
JWT_AUTH_COOKIE_SECURE=false
# In-process job scheduler (replaces Celery beat). MUST stay false while
# Django's beat container is still running — otherwise scheduled jobs fire twice
# against the shared api_job table. Set true ONLY after stopping Django beat.
# In-process job scheduler (replaces Celery beat). Exactly one running instance
# may set this true — it owns scheduling of the api_job table. Additional
# instances must leave it false or jobs fire twice.
ENABLE_SCHEDULER=false
# tracing filter
backend-rs/README.md
+21 -83
View File
@@ -185,94 +185,35 @@ due `sync_flavors` job which fetched `weareholy.com` and reported
## Run
```bash
cp .env.example .env # set SECRET_KEY to match Django; point DATABASE_URL at the shared db
cp .env.example .env # set SECRET_KEY; point DATABASE_URL at the shared db
cargo run # listens on BIND_ADDR (default 0.0.0.0:8001)
```
`SECRET_KEY` **must** equal Django's, or JWTs minted by Django won't verify here.
`SECRET_KEY` signs the SimpleJWT-compatible tokens — **keep it constant** across
deploys/restarts, or every issued access/refresh token is invalidated. (It must
match the value the retired Django app used, so tokens already in the wild stay
valid.)
## Strangler proxy (nginx)
## Deployment
The live routing is in **`frontend/nginx.conf`** (copied into the `holy-frontend`
image; shared proxy headers live in `frontend/proxy_params.conf`). Two upstreams
are declared — `django_backend` (`holy-backend:8000`) and `rust_backend`
(`holy-rust:8001`) — and each `/api/<group>/` location has its own `proxy_pass`.
Cutting a group over is a one-line edit (`django_backend` → `rust_backend`)
followed by a reload; rollback is the reverse edit. Because both backends share
one SQLite DB, one media dir, and the same `SECRET_KEY`, either can serve any
request and tokens stay valid across a flip.
The Rust backend is the production API; Django/Celery/Redis were retired after
the strangler migration completed. `docker compose` (`docker-compose.yml` +
`docker-compose.override.yml`) runs two services: `backend-rs` (`holy-rust:8001`)
and `frontend` (nginx). nginx serves the SPA, proxies `/api` to the Rust backend,
and serves `/media` from the shared `./backend/media` bind mount; see
`frontend/nginx.conf`.
`X-Forwarded-Proto`/`Host` are required: absolute URLs (image links, pagination
`next`/`previous`) are rebuilt from them, matching DRF's `build_absolute_uri`.
`X-Forwarded-For` lets `/api/users/me/` log the real client IP (it falls back to
the peer address otherwise, matching Django's `REMOTE_ADDR`).
the peer address otherwise).
## Production cutover runbook (seamless, reversible)
**Scheduler invariant:** exactly **one** process may own the `api_job` table.
Run a single `backend-rs` instance with `ENABLE_SCHEDULER=true`
(`RUST_ENABLE_SCHEDULER` in `.env`); any additional instances must leave it off.
Goal: migrate a running Django deployment to Rust with **no downtime** and a
per-step rollback. Everything hinges on the shared substrate already in place —
same `db.sqlite3` (WAL on), same media dir, same `SECRET_KEY` — so a request can
be served by either backend interchangeably.
**Invariant (do not violate):** exactly **one** scheduler may own the `api_job`
table at any time. Django's `beat` owns it until Phase 3; Rust's in-process
scheduler (`ENABLE_SCHEDULER`) is **off** until then. Never both.
### Phase 0 — Deploy Rust in shadow (no traffic)
1. Set `SECRET_KEY` in `.env` to the **exact** value Django already uses (else
existing JWTs break on cutover). Optionally set `FLOWER_BASIC_AUTH`.
2. `docker compose up -d --build backend-rs` — brings up `holy-rust` beside
Django. `ENABLE_SCHEDULER` defaults to false; it only reads the DB.
3. Smoke-test directly: `docker exec holy-frontend wget -qO- http://holy-rust:8001/health/`
and spot-check a few endpoints against it. nginx still sends all prod traffic
to Django. **Rollback:** `docker compose stop backend-rs`.
### Phase 1 — Reads
4. In `frontend/nginx.conf`, flip `categories`, `flavors`, `banners` to
`rust_backend`. Rebuild/reload the frontend (`docker compose up -d --build
frontend`, or `docker exec holy-frontend nginx -s reload` if you bind-mount the
conf). Watch `docker logs holy-rust` + error rates. **Rollback:** flip back, reload.
### Phase 2 — Auth + writes
5. Flip `auth`, `token` (legacy alias), `ratings`, `replies`, `users`,
`notifications`, `tickets`. JWTs and cookies are cross-valid, so in-flight
sessions survive. Reload. **Rollback:** flip back, reload.
### Phase 3 — Admin + jobs (the careful one)
6. Flip `admin-custom` to `rust_backend`, reload.
7. **Stop Django's scheduler/workers:** `docker compose stop beat worker flower`.
Now nobody schedules jobs.
8. Hand scheduling to Rust: set `RUST_ENABLE_SCHEDULER=true` in `.env`, then
`docker compose up -d backend-rs`. Confirm the log line
`job scheduler ENABLED`. Single owner throughout → no double-run.
**Rollback:** set `RUST_ENABLE_SCHEDULER=false` + recreate `backend-rs`, then
`docker compose start beat worker`.
### Phase 4 — Retire Django
9. Point the catch-all `location /api/` at `rust_backend` (covers `/api/schema`
too — Rust serves OpenAPI + Swagger). Reload. Django now receives only
`/admin/` + `/static/` (its admin site). `/media/` is already served straight
from nginx (the `:ro` mount), independent of either backend.
10. Decide the Django admin site's fate. The app's own admin UI uses
`/api/admin-custom/`, so the Django `/admin/` site is usually unneeded:
- Keeping it: leave the `backend` container up for `/admin/` + `/static/`.
- Dropping it: `docker compose stop backend`, delete the `/admin/` + `/static/`
locations from nginx.
11. Once Django is fully off, `redis` (only a Celery broker + Django cache — Rust
uses neither) can be stopped too: `docker compose stop redis`. Eventually
remove `backend`, `worker`, `beat`, `flower`, `redis` from compose.
### Notes / caveats
- **Rate-limit counters reset at cutover** — Django's `django_ratelimit` lives in
Redis; Rust's throttle is in-process. Acceptable (windows are minutes/hours).
- **Token blacklist is shared** (both use `token_blacklist_*` tables), so logout
/ refresh-rotation stays consistent across a flip.
- Reloads: the frontend image bakes the conf, so a conf change needs
`docker compose up -d --build frontend`. For instant reloads during cutover,
bind-mount `./frontend/nginx.conf:/etc/nginx/conf.d/default.conf` temporarily and
use `nginx -s reload`.
- Keep `backend/db.sqlite3` + `backend/media/` (host bind mounts) as the canonical
data across all of this — neither is recreated by rebuilds.
The canonical data lives in host bind mounts — `backend/db.sqlite3` (WAL on) and
`backend/media/` — and survives image rebuilds. Keep them.
## Caveats / parity TODO
@@ -291,9 +232,7 @@ scheduler (`ENABLE_SCHEDULER`) is **off** until then. Never both.
never used them.
- Admin endpoints + background jobs are ported (see the Admin + jobs slice).
The job-trigger and admin write endpoints are **not** rate-limited (those are
staff-only; `django_ratelimit` only covered the public flows below). When Rust
owns jobs in prod, disable the Celery `beat`/`worker` containers to avoid
double-execution.
staff-only; `django_ratelimit` only covered the public flows below).
## Security & hardening
@@ -338,9 +277,8 @@ in-flight requests drain. `/health/` now runs `SELECT 1` so the Docker
healthcheck fails when SQLite is unreachable.
SQLite runs in **WAL** journal mode (enabled once at startup, persistent in the
file header, so it covers the Django side too) so readers don't block behind the
writer under the shared-DB strangler; `busy_timeout` is sqlx's default 5 s,
matching Django's SQLite timeout. The default `DATABASE_URL` uses `mode=rw` (not
file header) so readers don't block behind the writer; `busy_timeout` is sqlx's
default 5 s, matching Django's SQLite timeout. The default `DATABASE_URL` uses `mode=rw` (not
`rwc`) so a wrong path fails loudly instead of silently creating an empty DB.
Confirmation codes are compared in **constant time** (`subtle::ConstantTimeEq`)
backend-rs/src/config.rs
+10 -5
View File
@@ -104,11 +104,16 @@ impl Config {
},
email: EmailConfig {
host: env::var("EMAIL_HOST").ok().filter(|s| !s.is_empty()),
port: env::var("EMAIL_PORT").ok().and_then(|s| s.parse().ok()).unwrap_or(587),
port: env::var("EMAIL_PORT")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(587),
use_tls: env_bool("EMAIL_USE_TLS", false),
use_ssl: env_bool("EMAIL_USE_SSL", false),
host_user: env::var("EMAIL_HOST_USER").ok().filter(|s| !s.is_empty()),
host_password: env::var("EMAIL_HOST_PASSWORD").ok().filter(|s| !s.is_empty()),
host_password: env::var("EMAIL_HOST_PASSWORD")
.ok()
.filter(|s| !s.is_empty()),
skip_cert_verification: env_bool("EMAIL_SKIP_CERT_VERIFICATION", false),
default_from: env::var("DEFAULT_FROM_EMAIL")
.ok()
@@ -120,9 +125,9 @@ impl Config {
.unwrap_or_else(|_| "../backend/banners".to_string()),
legacy_dir: env::var("LEGACY_DIR").unwrap_or_else(|_| "../legacy".to_string()),
backup_dir: env::var("BACKUP_DIR").unwrap_or_else(|_| "../backend/backups".to_string()),
// Default OFF: a fresh deploy runs beside Django, which still owns the
// scheduler. Operator flips ENABLE_SCHEDULER=true only after stopping
// Django beat (see README cutover steps).
// Default OFF so ad-hoc `cargo run` instances don't run jobs.
// Production (docker compose) sets ENABLE_SCHEDULER=true on the one
// instance that should own `api_job` scheduling.
enable_scheduler: env_bool("ENABLE_SCHEDULER", false),
})
}
backend-rs/src/datetime.rs
+2 -1
View File
@@ -5,7 +5,8 @@ use chrono::{NaiveDateTime, Timelike, Utc};
/// so truncate before inserting to keep the on-disk format identical.
pub fn now_micros() -> NaiveDateTime {
let now = Utc::now().naive_utc();
now.with_nanosecond(now.nanosecond() / 1000 * 1000).unwrap_or(now)
now.with_nanosecond(now.nanosecond() / 1000 * 1000)
.unwrap_or(now)
}
/// Format a naive-UTC datetime exactly as DRF serializes an aware UTC
backend-rs/src/email.rs
+5 -2
View File
@@ -42,14 +42,17 @@ pub async fn send_mail(
let mut builder = Message::builder().from(from).subject(subject);
for r in recipients {
let mbox: Mailbox = r.parse().map_err(|e| format!("invalid recipient {r}: {e}"))?;
let mbox: Mailbox = r
.parse()
.map_err(|e| format!("invalid recipient {r}: {e}"))?;
builder = builder.to(mbox);
}
let message = builder
.body(body.to_string())
.map_err(|e| format!("build email: {e}"))?;
let mut transport = AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(host).port(cfg.port);
let mut transport =
AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(host).port(cfg.port);
if cfg.use_ssl || cfg.use_tls {
let tls = TlsParameters::builder(host.to_string())
backend-rs/src/error.rs
+10 -5
View File
@@ -56,7 +56,8 @@ impl IntoResponse for ApiError {
)
.into_response();
if let Ok(hv) = axum::http::HeaderValue::from_str(&retry_after.to_string()) {
resp.headers_mut().insert(axum::http::header::RETRY_AFTER, hv);
resp.headers_mut()
.insert(axum::http::header::RETRY_AFTER, hv);
}
return resp;
}
@@ -73,11 +74,15 @@ impl IntoResponse for ApiError {
ApiError::BadRequest(m) => (StatusCode::BAD_REQUEST, m.clone()),
ApiError::Db(e) => {
tracing::error!(error = %e, "database error");
(StatusCode::INTERNAL_SERVER_ERROR, "Server error.".to_string())
}
ApiError::Internal => {
(StatusCode::INTERNAL_SERVER_ERROR, "Server error.".to_string())
(
StatusCode::INTERNAL_SERVER_ERROR,
"Server error.".to_string(),
)
}
ApiError::Internal => (
StatusCode::INTERNAL_SERVER_ERROR,
"Server error.".to_string(),
),
};
(status, Json(json!({ "detail": detail }))).into_response()
}
backend-rs/src/jobs/backup_db.rs
+8 -2
View File
@@ -67,7 +67,9 @@ impl BackgroundJob for BackupDb {
})
.await??;
log.push("Full archive created successfully.".to_string());
log.push(format!("Cleaning up backups older than {RETENTION_DAYS} days..."));
log.push(format!(
"Cleaning up backups older than {RETENTION_DAYS} days..."
));
log.extend(archive_log);
Ok(log.join("\n"))
@@ -99,7 +101,11 @@ fn prune_old(backup_dir: &str) -> anyhow::Result<Vec<String>> {
let modified = entry.metadata().and_then(|m| m.modified()).ok();
if let Some(mtime) = modified {
if mtime < cutoff {
let name = path.file_name().and_then(|f| f.to_str()).unwrap_or("").to_string();
let name = path
.file_name()
.and_then(|f| f.to_str())
.unwrap_or("")
.to_string();
match fs::remove_file(&path) {
Ok(_) => log.push(format!(" -> Deleted old backup: {name}")),
Err(e) => log.push(format!(" -> Failed to delete {name}: {e}")),
backend-rs/src/jobs/cleanup_duplicates.rs
+1 -6
View File
@@ -15,12 +15,7 @@ use super::BackgroundJob;
/// Trailing form-factor phrases stripped during name normalization. Each is a
/// sequence of whitespace-separated tokens; the match requires a preceding
/// token (Django's leading `\s+`), so a name that is *only* the phrase is kept.
const SUFFIXES: &[&[&str]] = &[
&["energy"],
&["hydration"],
&["iced", "tea"],
&["(legacy)"],
];
const SUFFIXES: &[&[&str]] = &[&["energy"], &["hydration"], &["iced", "tea"], &["(legacy)"]];
/// Lowercase, trim, and strip the known trailing suffixes (Django's
/// `normalize_name`). The result is a grouping key, not a display value.
backend-rs/src/jobs/mod.rs
+6 -2
View File
@@ -26,8 +26,8 @@ use sea_orm::ActiveValue::Set;
use sea_orm::{ActiveModelTrait, ColumnTrait, DatabaseConnection, DbErr, EntityTrait, QueryFilter};
use unicode_normalization::UnicodeNormalization;
use crate::entities::prelude::Job as JobEntity;
use crate::entities::job;
use crate::entities::prelude::Job as JobEntity;
use crate::state::AppState;
pub use scheduler::spawn_scheduler;
@@ -136,7 +136,11 @@ async fn run_inner(state: &AppState, job: &dyn BackgroundJob) -> anyhow::Result<
/// media image paths (and category slugs) identical to Django's so files and
/// rows are reused, not duplicated.
pub fn slugify(value: &str) -> String {
let ascii: String = value.nfkd().filter(|c| c.is_ascii()).collect::<String>().to_lowercase();
let ascii: String = value
.nfkd()
.filter(|c| c.is_ascii())
.collect::<String>()
.to_lowercase();
// Keep word chars, hyphen and whitespace; drop the rest.
let kept: String = ascii
.chars()
backend-rs/src/jobs/scheduler.rs
+6 -2
View File
@@ -11,8 +11,8 @@ use std::time::Duration;
use sea_orm::{ColumnTrait, EntityTrait, QueryFilter};
use crate::entities::prelude::Job as JobEntity;
use crate::entities::job;
use crate::entities::prelude::Job as JobEntity;
use crate::state::AppState;
const TICK: Duration = Duration::from_secs(60);
@@ -57,7 +57,11 @@ async fn tick_once(state: &AppState) -> anyhow::Result<()> {
if state.running_jobs.lock().await.contains(job.name()) {
continue;
}
tracing::info!(job = job.name(), interval_hours = row.interval_hours, "scheduling due job");
tracing::info!(
job = job.name(),
interval_hours = row.interval_hours,
"scheduling due job"
);
let st = state.clone();
tokio::spawn(async move { super::run_job(&st, job).await });
}
backend-rs/src/jobs/seed_banners.rs
+27 -7
View File
@@ -49,7 +49,11 @@ impl BackgroundJob for SeedBanners {
if path.extension().and_then(|e| e.to_str()) != Some("json") {
continue;
}
let filename = path.file_name().and_then(|f| f.to_str()).unwrap_or("").to_string();
let filename = path
.file_name()
.and_then(|f| f.to_str())
.unwrap_or("")
.to_string();
if let Err(e) = sync_one(db, &path, &mut log).await {
log.push(format!("Failed to sync banner from {filename}: {e}"));
}
@@ -63,7 +67,11 @@ async fn sync_one(
path: &std::path::Path,
log: &mut Vec<String>,
) -> anyhow::Result<()> {
let filename = path.file_name().and_then(|f| f.to_str()).unwrap_or("").to_string();
let filename = path
.file_name()
.and_then(|f| f.to_str())
.unwrap_or("")
.to_string();
let content = fs::read_to_string(path)?;
let data: Value = serde_json::from_str(&content)?;
@@ -78,9 +86,16 @@ async fn sync_one(
match existing {
None => {
let name = data.get("name").and_then(|v| v.as_str()).unwrap_or(slug).to_string();
let description =
data.get("description").and_then(|v| v.as_str()).unwrap_or("").to_string();
let name = data
.get("name")
.and_then(|v| v.as_str())
.unwrap_or(slug)
.to_string();
let description = data
.get("description")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let settings = data.get("settings").cloned().unwrap_or_else(|| json!({}));
// Mirror Banner.save(): become active when no other banner is active.
let no_active = active_count(db, None).await? == 0;
@@ -130,7 +145,10 @@ async fn sync_one(
// Banner.save() invariant on every save.
if was_active {
Banner::update_many()
.col_expr(banner::Column::IsActive, sea_orm::sea_query::Expr::value(false))
.col_expr(
banner::Column::IsActive,
sea_orm::sea_query::Expr::value(false),
)
.filter(banner::Column::IsActive.eq(true))
.filter(banner::Column::Id.ne(pk))
.exec(db)
@@ -139,7 +157,9 @@ async fn sync_one(
am.is_active = Set(true);
}
am.update(db).await?;
log.push(format!("Synced new settings for banner '{slug}' from {filename}"));
log.push(format!(
"Synced new settings for banner '{slug}' from {filename}"
));
}
}
}
backend-rs/src/jobs/seed_legacy.rs
+12 -7
View File
@@ -9,9 +9,7 @@ use std::path::Path;
use async_trait::async_trait;
use sea_orm::sea_query::{Expr, Func};
use sea_orm::ActiveValue::Set;
use sea_orm::{
ActiveModelTrait, ColumnTrait, DatabaseConnection, EntityTrait, QueryFilter,
};
use sea_orm::{ActiveModelTrait, ColumnTrait, DatabaseConnection, EntityTrait, QueryFilter};
use serde_json::{json, Value};
use crate::entities::prelude::{Category, Flavor};
@@ -21,8 +19,9 @@ use crate::state::AppState;
use super::download::{download_image, ext_for, hash_for, http_client};
use super::{slugify, BackgroundJob};
const PACK_KEYWORDS: &[&str] =
&["bundle", "set", "box", "probe", "sample", "taster", "shaker", "starter"];
const PACK_KEYWORDS: &[&str] = &[
"bundle", "set", "box", "probe", "sample", "taster", "shaker", "starter",
];
pub struct SeedLegacy;
@@ -61,7 +60,11 @@ impl BackgroundJob for SeedLegacy {
if path.extension().and_then(|e| e.to_str()) != Some("json") {
continue;
}
let fname = path.file_name().and_then(|f| f.to_str()).unwrap_or("").to_lowercase();
let fname = path
.file_name()
.and_then(|f| f.to_str())
.unwrap_or("")
.to_lowercase();
// Category from filename.
let file_cat: Option<&category::Model> = if fname.contains("energy") {
@@ -78,7 +81,9 @@ impl BackgroundJob for SeedLegacy {
let content = fs::read_to_string(&path)?;
let items: Value = serde_json::from_str(&content)?;
let Some(arr) = items.as_array() else { continue };
let Some(arr) = items.as_array() else {
continue;
};
for data in arr {
let Some(name) = data.get("Name").and_then(|v| v.as_str()) else {
backend-rs/src/jobs/sync_flavors.rs
+138 -39
View File
@@ -28,14 +28,44 @@ const DOWNLOAD_WORKERS: usize = 8;
const PACKS_SLUG: &str = "packs-and-other";
const PACKS_NAME: &str = "Packs and other";
const FORM_WORDS: &[&str] =
&["bundle", "sachet", "sachetbox", "bottle", "box", "pack", "set", "sachets"];
const FORM_WORDS: &[&str] = &[
"bundle",
"sachet",
"sachetbox",
"bottle",
"box",
"pack",
"set",
"sachets",
];
const NON_FLAVOR_TOKENS: &[&str] = &[
"value", "shaker", "merch", "sticker", "packaging", "material", "other", "items", "mixed",
"collector", "collector's", "free", "products", "naturalrabatt", "sample", "samplebox",
"value",
"shaker",
"merch",
"sticker",
"packaging",
"material",
"other",
"items",
"mixed",
"collector",
"collector's",
"free",
"products",
"naturalrabatt",
"sample",
"samplebox",
];
const PACK_TITLE_KEYWORDS: &[&str] = &[
"bundle", "set", "box", "probe", "sample", "taster", "starter", "collection", "probier",
"bundle",
"set",
"box",
"probe",
"sample",
"taster",
"starter",
"collection",
"probier",
];
const ACCESSORY_TITLE_KEYWORDS: &[&str] = &["bürste", "reinigung", "ersatzteil", "zubehör"];
const TAG_FAMILY: &[(&[&str], &str)] = &[
@@ -85,7 +115,11 @@ impl BackgroundJob for SyncFlavors {
}
};
let products = data.get("products").and_then(|v| v.as_array()).cloned().unwrap_or_default();
let products = data
.get("products")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
log.push(format!("Found {} products.", products.len()));
let packs = goc_category_by_slug_named(db, PACKS_SLUG, PACKS_NAME).await?;
@@ -95,31 +129,54 @@ impl BackgroundJob for SyncFlavors {
let mut synced_ids: Vec<i64> = Vec::new();
for p in &products {
let title = p.get("title").and_then(|v| v.as_str()).unwrap_or("").trim().to_string();
let title = p
.get("title")
.and_then(|v| v.as_str())
.unwrap_or("")
.trim()
.to_string();
let tags = parse_tags(p);
let tags_lower: Vec<String> = tags.iter().map(|t| t.to_lowercase()).collect();
let title_lower = title.to_lowercase();
// Syrup ships as one article whose variants are the flavors.
if is_syrup_variant_product(p) {
let (c, u) = sync_syrup_variants(db, &client, media_root, p, &mut synced_ids).await?;
let (c, u) =
sync_syrup_variants(db, &client, media_root, p, &mut synced_ids).await?;
created += c;
updated += u;
continue;
}
let Some(cat) =
resolve_category(db, p, &title_lower, &tags_lower, &packs).await?
let Some(cat) = resolve_category(db, p, &title_lower, &tags_lower, &packs).await?
else {
continue;
};
let variants = p.get("variants").and_then(|v| v.as_array()).cloned().unwrap_or_default();
let is_available = variants.iter().any(|v| v.get("available").and_then(|a| a.as_bool()).unwrap_or(false));
let images = p.get("images").and_then(|v| v.as_array()).cloned().unwrap_or_default();
let image_url = images.first().and_then(|i| i.get("src")).and_then(|s| s.as_str()).map(String::from);
let image_urls: Vec<String> =
images.iter().filter_map(|i| i.get("src").and_then(|s| s.as_str()).map(String::from)).collect();
let variants = p
.get("variants")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
let is_available = variants.iter().any(|v| {
v.get("available")
.and_then(|a| a.as_bool())
.unwrap_or(false)
});
let images = p
.get("images")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
let image_url = images
.first()
.and_then(|i| i.get("src"))
.and_then(|s| s.as_str())
.map(String::from);
let image_urls: Vec<String> = images
.iter()
.filter_map(|i| i.get("src").and_then(|s| s.as_str()).map(String::from))
.collect();
let description = clean_html(p.get("body_html").and_then(|v| v.as_str()).unwrap_or(""));
let shop_url = p.get("handle").and_then(|v| v.as_str()).map(shop_url_for);
let Some(pid) = p.get("id").and_then(|v| v.as_i64()) else {
@@ -176,7 +233,10 @@ fn shop_url_for(handle: &str) -> String {
/// Tags as a lowercase-able list; Shopify returns either an array or a CSV string.
fn parse_tags(p: &Value) -> Vec<String> {
match p.get("tags") {
Some(Value::Array(a)) => a.iter().filter_map(|t| t.as_str().map(String::from)).collect(),
Some(Value::Array(a)) => a
.iter()
.filter_map(|t| t.as_str().map(String::from))
.collect(),
Some(Value::String(s)) => s.split(',').map(|t| t.trim().to_string()).collect(),
_ => Vec::new(),
}
@@ -209,10 +269,16 @@ fn family_from_product_type(product_type: &str) -> Option<String> {
return None;
}
let mut tokens: Vec<&str> = stripped.split_whitespace().collect();
if tokens.iter().any(|t| NON_FLAVOR_TOKENS.contains(&t.to_lowercase().as_str())) {
if tokens
.iter()
.any(|t| NON_FLAVOR_TOKENS.contains(&t.to_lowercase().as_str()))
{
return None;
}
while tokens.last().is_some_and(|t| FORM_WORDS.contains(&t.to_lowercase().as_str())) {
while tokens
.last()
.is_some_and(|t| FORM_WORDS.contains(&t.to_lowercase().as_str()))
{
tokens.pop();
}
if tokens.is_empty() {
@@ -254,11 +320,20 @@ fn is_syrup_variant_product(p: &Value) -> bool {
.get("options")
.and_then(|v| v.as_array())
.map(|opts| {
opts.iter()
.any(|o| o.get("name").and_then(|n| n.as_str()).unwrap_or("").to_lowercase() == "flavor")
opts.iter().any(|o| {
o.get("name")
.and_then(|n| n.as_str())
.unwrap_or("")
.to_lowercase()
== "flavor"
})
})
.unwrap_or(false);
let variant_count = p.get("variants").and_then(|v| v.as_array()).map(|v| v.len()).unwrap_or(0);
let variant_count = p
.get("variants")
.and_then(|v| v.as_array())
.map(|v| v.len())
.unwrap_or(0);
has_flavor_option && variant_count > 1
}
@@ -273,7 +348,10 @@ async fn resolve_category(
if is_pack(title_lower, tags_lower) {
return Ok(Some(packs.clone()));
}
if ACCESSORY_TITLE_KEYWORDS.iter().any(|k| title_lower.contains(k)) {
if ACCESSORY_TITLE_KEYWORDS
.iter()
.any(|k| title_lower.contains(k))
{
return Ok(Some(packs.clone()));
}
if let Some(family) =
@@ -293,7 +371,10 @@ async fn resolve_category(
}
/// get_or_create a category from a family name, keyed by `slugify(name)`.
async fn category_for_family(db: &DatabaseConnection, name: &str) -> anyhow::Result<category::Model> {
async fn category_for_family(
db: &DatabaseConnection,
name: &str,
) -> anyhow::Result<category::Model> {
goc_category_by_slug_named(db, &slugify(name), name).await
}
@@ -333,7 +414,11 @@ async fn sync_syrup_variants(
let mut created = 0usize;
let mut updated = 0usize;
let variants = p.get("variants").and_then(|v| v.as_array()).cloned().unwrap_or_default();
let variants = p
.get("variants")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
for v in &variants {
let name = v
.get("option1")
@@ -354,7 +439,10 @@ async fn sync_syrup_variants(
.and_then(|s| s.as_str())
.map(String::from);
let image_urls: Vec<String> = image_url.clone().into_iter().collect();
let available = v.get("available").and_then(|a| a.as_bool()).unwrap_or(false);
let available = v
.get("available")
.and_then(|a| a.as_bool())
.unwrap_or(false);
let was_created = upsert_flavor(
db,
@@ -412,7 +500,9 @@ async fn upsert_flavor(
}
if existing.is_none() {
existing = Flavor::find()
.filter(Expr::expr(Func::lower(Expr::col(flavor::Column::Name))).eq(name.to_lowercase()))
.filter(
Expr::expr(Func::lower(Expr::col(flavor::Column::Name))).eq(name.to_lowercase()),
)
.filter(flavor::Column::CategoryId.eq(category_id))
.one(db)
.await?;
@@ -510,8 +600,8 @@ async fn download_gallery(
return Vec::new();
}
let succeeded: HashMap<usize, String> = futures::stream::iter(targets.clone().into_iter().map(
|(i, u, rel)| {
let succeeded: HashMap<usize, String> =
futures::stream::iter(targets.clone().into_iter().map(|(i, u, rel)| {
let client = client.clone();
let media_root = media_root.to_string();
async move {
@@ -519,22 +609,31 @@ async fn download_gallery(
let ok = download_image(&client, &u, &abs).await;
(i, ok, rel)
}
},
))
.buffer_unordered(DOWNLOAD_WORKERS)
.filter_map(|(i, ok, rel)| async move { if ok { Some((i, rel)) } else { None } })
.collect()
.await;
}))
.buffer_unordered(DOWNLOAD_WORKERS)
.filter_map(|(i, ok, rel)| async move {
if ok {
Some((i, rel))
} else {
None
}
})
.collect()
.await;
targets.iter().filter_map(|(i, _, _)| succeeded.get(i).cloned()).collect()
targets
.iter()
.filter_map(|(i, _, _)| succeeded.get(i).cloned())
.collect()
}
/// Django JSONField (list) stored as TEXT → Vec<String>.
fn json_str_list(v: &Option<Value>) -> Vec<String> {
match v {
Some(Value::Array(arr)) => {
arr.iter().filter_map(|x| x.as_str().map(String::from)).collect()
}
Some(Value::Array(arr)) => arr
.iter()
.filter_map(|x| x.as_str().map(String::from))
.collect(),
_ => Vec::new(),
}
}
backend-rs/src/main.rs
+7 -7
View File
@@ -11,8 +11,8 @@ mod entities;
mod error;
mod jobs;
mod media;
mod merge;
mod mentions;
mod merge;
mod openapi;
mod pagination;
mod password;
@@ -63,21 +63,21 @@ async fn main() -> anyhow::Result<()> {
let state = AppState {
db,
config,
running_jobs: std::sync::Arc::new(tokio::sync::Mutex::new(std::collections::HashSet::new())),
running_jobs: std::sync::Arc::new(
tokio::sync::Mutex::new(std::collections::HashSet::new()),
),
security: std::sync::Arc::new(crate::throttle::Security::new()),
};
// Background scheduler: runs jobs whose `interval_hours` has elapsed,
// replacing Celery beat. Restart-safe (last_run persists in the DB).
// Gated so it never double-schedules against a still-running Django beat
// during the strangler cutover — exactly one scheduler may own `api_job`.
// Gated by ENABLE_SCHEDULER — exactly one process may own `api_job`, so
// enable the scheduler on at most one running instance.
if state.config.enable_scheduler {
tracing::info!("job scheduler ENABLED (this process owns api_job scheduling)");
jobs::spawn_scheduler(state.clone());
} else {
tracing::warn!(
"job scheduler DISABLED (ENABLE_SCHEDULER!=true) — Django beat assumed to own jobs"
);
tracing::warn!("job scheduler DISABLED (ENABLE_SCHEDULER!=true)");
}
// Periodically sweep aged-out rate-limit / code buckets so the in-memory
backend-rs/src/merge.rs
+4 -1
View File
@@ -15,7 +15,10 @@ use crate::entities::{flavor, notification, rating, reply};
/// Rank a rating for conflict resolution: longest comment wins, newer breaks
/// ties (mirrors Python's tuple comparison `(len(comment), created_at)`).
fn quality(r: &rating::Model) -> (usize, chrono::NaiveDateTime) {
(r.comment.as_deref().unwrap_or("").chars().count(), r.created_at)
(
r.comment.as_deref().unwrap_or("").chars().count(),
r.created_at,
)
}
/// Merge `remove` into `keep`. Re-assigns all of `remove`'s ratings; when a user
backend-rs/src/pagination.rs
+1 -7
View File
@@ -47,13 +47,7 @@ fn with_page(ctx: &RequestCtx, page: u64) -> String {
}
impl<T: Serialize> Paginated<T> {
pub fn build(
ctx: &RequestCtx,
results: Vec<T>,
count: u64,
page: u64,
page_size: u64,
) -> Self {
pub fn build(ctx: &RequestCtx, results: Vec<T>, count: u64, page: u64, page_size: u64) -> Self {
let total_pages = count.div_ceil(page_size.max(1));
let next = if page < total_pages {
Some(with_page(ctx, page + 1))
backend-rs/src/password.rs
+13 -3
View File
@@ -10,8 +10,8 @@
//! (argon2/bcrypt) would need their own branch — login would fall through to a
//! rejection for those, so keep issuance on Django if the hasher ever changes.
use base64::Engine;
use base64::engine::general_purpose::STANDARD as B64;
use base64::Engine;
use sha2::Sha256;
use subtle::ConstantTimeEq;
@@ -45,7 +45,12 @@ pub fn verify_password(password: &str, encoded: &str) -> bool {
if iterations == 0 {
return false;
}
pbkdf2::pbkdf2_hmac::<Sha256>(password.as_bytes(), salt.as_bytes(), iterations, &mut derived);
pbkdf2::pbkdf2_hmac::<Sha256>(
password.as_bytes(),
salt.as_bytes(),
iterations,
&mut derived,
);
// Constant-time compare, length-checked first.
if expected.len() != derived.len() {
@@ -60,7 +65,12 @@ pub fn verify_password(password: &str, encoded: &str) -> bool {
pub fn hash_password(password: &str) -> String {
let salt = uuid::Uuid::new_v4().simple().to_string();
let mut derived = [0u8; DK_LEN];
pbkdf2::pbkdf2_hmac::<Sha256>(password.as_bytes(), salt.as_bytes(), ITERATIONS, &mut derived);
pbkdf2::pbkdf2_hmac::<Sha256>(
password.as_bytes(),
salt.as_bytes(),
ITERATIONS,
&mut derived,
);
let b64 = B64.encode(derived);
format!("{ALGORITHM}${ITERATIONS}${salt}${b64}")
}
backend-rs/src/password_policy.rs
+3 -1
View File
@@ -121,7 +121,9 @@ fn too_similar(password: &str, attrs: &UserAttrs) -> Result<(), String> {
continue;
}
if quick_ratio(&password, part) >= MAX_SIMILARITY {
return Err(format!("The password is too similar to the {verbose_name}."));
return Err(format!(
"The password is too similar to the {verbose_name}."
));
}
}
}
backend-rs/src/routes/admin.rs
+42 -13
View File
@@ -41,10 +41,15 @@ pub fn router() -> Router<AppState> {
.route("/admin-custom/send_test_email/", post(send_test_email))
.route("/admin-custom/users/", get(users_list))
.route("/admin-custom/{pk}/trigger_job/", post(trigger_job))
.route("/admin-custom/{pk}/update_job_schedule/", patch(update_job_schedule))
.route(
"/admin-custom/{pk}/update_job_schedule/",
patch(update_job_schedule),
)
.route(
"/admin-custom/{pk}/user_detail/",
get(user_detail_get).patch(user_detail_patch).delete(user_detail_delete),
get(user_detail_get)
.patch(user_detail_patch)
.delete(user_detail_delete),
)
}
@@ -135,7 +140,9 @@ fn opt_bool(data: &Value, field: &str) -> ApiResult<Option<bool>> {
match data.get(field) {
None | Some(Value::Null) => Ok(None),
Some(Value::Bool(b)) => Ok(Some(*b)),
_ => Err(ApiError::Validation(json!({ field: ["Must be a valid boolean."] }))),
_ => Err(ApiError::Validation(
json!({ field: ["Must be a valid boolean."] }),
)),
}
}
@@ -157,7 +164,9 @@ async fn config_patch(
})))
}
Some(_) => {
return Err(ApiError::Validation(json!({ "site_name": ["Not a valid string."] })))
return Err(ApiError::Validation(
json!({ "site_name": ["Not a valid string."] }),
))
}
}
if let Some(b) = opt_bool(&data, "maintenance_mode")? {
@@ -218,11 +227,14 @@ async fn trigger_job(
_admin: AdminUser,
Path(pk): Path<i32>,
) -> ApiResult<Json<Value>> {
let row = Job::find_by_id(pk).one(&state.db).await?.ok_or(ApiError::NotFound)?;
let row = Job::find_by_id(pk)
.one(&state.db)
.await?
.ok_or(ApiError::NotFound)?;
let Some(job_impl) = jobs::lookup(&row.name) else {
return Err(ApiError::JsonStatus(
StatusCode::BAD_REQUEST,
json!({ "error": format!("No celery task registered for {}", row.name) }),
json!({ "error": format!("No job registered for {}", row.name) }),
));
};
@@ -245,7 +257,10 @@ async fn update_job_schedule(
Path(pk): Path<i32>,
body: Bytes,
) -> ApiResult<Json<Value>> {
let row = Job::find_by_id(pk).one(&state.db).await?.ok_or(ApiError::NotFound)?;
let row = Job::find_by_id(pk)
.one(&state.db)
.await?
.ok_or(ApiError::NotFound)?;
let data = parse_body(&body);
let interval = match data.get("interval_hours") {
@@ -381,8 +396,10 @@ async fn users_list(State(state): State<AppState>, _admin: AdminUser) -> ApiResu
let out: Vec<Value> = users
.iter()
.map(|u| {
let ips: Vec<String> =
ip_map.get(&u.id).map(|v| v.iter().take(3).cloned().collect()).unwrap_or_default();
let ips: Vec<String> = ip_map
.get(&u.id)
.map(|v| v.iter().take(3).cloned().collect())
.unwrap_or_default();
admin_user_json(u, &ips)
})
.collect();
@@ -395,7 +412,10 @@ async fn build_user_detail(
ctx: &RequestCtx,
u: &user::Model,
) -> ApiResult<Value> {
let ips = ips_by_user(state, &[u.id]).await?.remove(&u.id).unwrap_or_default();
let ips = ips_by_user(state, &[u.id])
.await?
.remove(&u.id)
.unwrap_or_default();
let ratings = Rating::find()
.filter(rating::Column::UserId.eq(u.id))
.order_by_desc(rating::Column::CreatedAt)
@@ -413,7 +433,10 @@ async fn user_detail_get(
_admin: AdminUser,
Path(pk): Path<i32>,
) -> ApiResult<Json<Value>> {
let u = User::find_by_id(pk).one(&state.db).await?.ok_or(ApiError::NotFound)?;
let u = User::find_by_id(pk)
.one(&state.db)
.await?
.ok_or(ApiError::NotFound)?;
Ok(Json(build_user_detail(&state, &ctx, &u).await?))
}
@@ -424,7 +447,10 @@ async fn user_detail_patch(
Path(pk): Path<i32>,
body: Bytes,
) -> ApiResult<Json<Value>> {
let u = User::find_by_id(pk).one(&state.db).await?.ok_or(ApiError::NotFound)?;
let u = User::find_by_id(pk)
.one(&state.db)
.await?
.ok_or(ApiError::NotFound)?;
let data = parse_body(&body);
let u = match data.get("is_active") {
Some(Value::Bool(b)) => {
@@ -442,7 +468,10 @@ async fn user_detail_delete(
_admin: AdminUser,
Path(pk): Path<i32>,
) -> ApiResult<StatusCode> {
User::find_by_id(pk).one(&state.db).await?.ok_or(ApiError::NotFound)?;
User::find_by_id(pk)
.one(&state.db)
.await?
.ok_or(ApiError::NotFound)?;
crate::routes::users::delete_user_cascade(&state, pk).await?;
Ok(StatusCode::NO_CONTENT)
}
backend-rs/src/routes/auth.rs
+12 -5
View File
@@ -20,9 +20,7 @@ use serde::Deserialize;
use serde_json::json;
use crate::blacklist::{blacklist, is_blacklisted, record_outstanding};
use crate::cookies::{
clear_cookie, cookie_value, set_cookie, COOKIE_ACCESS, COOKIE_REFRESH,
};
use crate::cookies::{clear_cookie, cookie_value, set_cookie, COOKIE_ACCESS, COOKIE_REFRESH};
use crate::entities::prelude::*;
use crate::entities::user;
use crate::error::ApiResult;
@@ -102,7 +100,11 @@ async fn login(
)
.await?;
Ok(ok_with_cookies(state.config.jwt_cookie_secure, &access, &refresh))
Ok(ok_with_cookies(
state.config.jwt_cookie_secure,
&access,
&refresh,
))
}
#[derive(Deserialize, Default)]
@@ -225,7 +227,12 @@ fn ok_with_cookies(secure: bool, access: &Minted, refresh: &Minted) -> Response
);
append_cookie(
&mut resp,
&set_cookie(COOKIE_REFRESH, &refresh.token, REFRESH_LIFETIME_SECS, secure),
&set_cookie(
COOKIE_REFRESH,
&refresh.token,
REFRESH_LIFETIME_SECS,
secure,
),
);
resp
}
backend-rs/src/routes/flavors.rs
+9 -3
View File
@@ -142,7 +142,9 @@ async fn top(
.limit(10)
.all(&state.db)
.await?;
Ok(Json(build_flavors(&state, &ctx, models, viewer, &[]).await?))
Ok(Json(
build_flavors(&state, &ctx, models, viewer, &[]).await?,
))
}
/// GET /api/flavors/newest/ — 10 most recently created flavors.
@@ -156,7 +158,9 @@ async fn newest(
.limit(10)
.all(&state.db)
.await?;
Ok(Json(build_flavors(&state, &ctx, models, viewer, &[]).await?))
Ok(Json(
build_flavors(&state, &ctx, models, viewer, &[]).await?,
))
}
/// GET /api/flavors/followed_top/ — top 10 among flavors rated by people the
@@ -189,7 +193,9 @@ async fn followed_top(
.limit(10)
.all(&state.db)
.await?;
Ok(Json(build_flavors(&state, &ctx, models, Some(uid), &[]).await?))
Ok(Json(
build_flavors(&state, &ctx, models, Some(uid), &[]).await?,
))
}
#[derive(Deserialize)]
backend-rs/src/routes/ratings.rs
+5 -7
View File
@@ -98,9 +98,10 @@ async fn create(
body: Bytes,
) -> ApiResult<(StatusCode, Json<RatingOut>)> {
// django_ratelimit(key="user", rate="10/m") on RatingViewSet.create.
state
.security
.check_rate(&format!("rating:user:{uid}"), crate::throttle::RATING_CREATE)?;
state.security.check_rate(
&format!("rating:user:{uid}"),
crate::throttle::RATING_CREATE,
)?;
let data: Value = serde_json::from_slice(&body).unwrap_or(Value::Null);
// DRF validation, aggregated and field-keyed, in serializer field order.
@@ -398,10 +399,7 @@ async fn feed(
}
/// GET /api/ratings/recent/ — 10 newest with a non-empty comment (AllowAny).
async fn recent(
State(state): State<AppState>,
ctx: RequestCtx,
) -> ApiResult<Json<Vec<RatingOut>>> {
async fn recent(State(state): State<AppState>, ctx: RequestCtx) -> ApiResult<Json<Vec<RatingOut>>> {
let models = Rating::find()
.filter(rating::Column::Comment.is_not_null())
.filter(rating::Column::Comment.ne(""))
backend-rs/src/routes/replies.rs
+12 -7
View File
@@ -28,12 +28,10 @@ use crate::state::AppState;
use crate::web::RequestCtx;
pub fn router() -> Router<AppState> {
Router::new()
.route("/replies/", get(list))
.route(
"/replies/{id}/",
get(retrieve).put(update).patch(update).delete(destroy),
)
Router::new().route("/replies/", get(list)).route(
"/replies/{id}/",
get(retrieve).put(update).patch(update).delete(destroy),
)
}
#[derive(Deserialize)]
@@ -124,7 +122,14 @@ async fn update(
}
let model = active.update(&state.db).await?;
parse_mentions(&state.db, &model.text, uid, Some(model.rating_id), Some(model.id)).await?;
parse_mentions(
&state.db,
&model.text,
uid,
Some(model.rating_id),
Some(model.id),
)
.await?;
let mut dtos = build_replies(&state, vec![model]).await?;
dtos.pop().map(Json).ok_or(ApiError::Internal)
backend-rs/src/routes/social.rs
+6 -5
View File
@@ -6,15 +6,13 @@ use axum::extract::{Path, State};
use axum::routing::{get, post};
use axum::{Json, Router};
use sea_orm::ActiveValue::Set;
use sea_orm::{
ActiveModelTrait, ColumnTrait, EntityTrait, QueryFilter, QueryOrder,
};
use sea_orm::{ActiveModelTrait, ColumnTrait, EntityTrait, QueryFilter, QueryOrder};
use serde_json::{json, Value};
use crate::auth::AuthUser;
use crate::dto::NotificationOut;
use crate::entities::prelude::*;
use crate::entities::notification;
use crate::entities::prelude::*;
use crate::error::{ApiError, ApiResult};
use crate::service::build_notifications;
use crate::state::AppState;
@@ -64,7 +62,10 @@ async fn mark_all_read(
AuthUser(uid): AuthUser,
) -> ApiResult<Json<Value>> {
Notification::update_many()
.col_expr(notification::Column::IsRead, sea_orm::sea_query::Expr::value(true))
.col_expr(
notification::Column::IsRead,
sea_orm::sea_query::Expr::value(true),
)
.filter(notification::Column::RecipientId.eq(uid))
.filter(notification::Column::IsRead.eq(false))
.exec(&state.db)
backend-rs/src/routes/tickets.rs
+16 -6
View File
@@ -97,7 +97,12 @@ async fn retrieve(
dtos.pop().map(Json).ok_or(ApiError::NotFound)
}
fn require_str(data: &Value, field: &str, partial: bool, max: Option<usize>) -> ApiResult<Option<String>> {
fn require_str(
data: &Value,
field: &str,
partial: bool,
max: Option<usize>,
) -> ApiResult<Option<String>> {
match data.get(field) {
Some(Value::String(s)) if !s.is_empty() => {
if let Some(m) = max {
@@ -112,9 +117,9 @@ fn require_str(data: &Value, field: &str, partial: bool, max: Option<usize>) ->
Some(Value::String(_)) => Err(ApiError::Validation(
json!({ field: ["This field may not be blank."] }),
)),
Some(v) if !v.is_null() => {
Err(ApiError::Validation(json!({ field: ["Not a valid string."] })))
}
Some(v) if !v.is_null() => Err(ApiError::Validation(
json!({ field: ["Not a valid string."] }),
)),
_ if partial => Ok(None),
_ => Err(ApiError::Validation(
json!({ field: ["This field is required."] }),
@@ -168,7 +173,10 @@ async fn create(
}
let mut dtos = build_tickets(&state, &ctx, vec![t]).await?;
Ok((StatusCode::CREATED, Json(dtos.pop().ok_or(ApiError::Internal)?)))
Ok((
StatusCode::CREATED,
Json(dtos.pop().ok_or(ApiError::Internal)?),
))
}
/// PUT/PATCH /api/tickets/{id}/ — subject/description only (status, user read-only).
@@ -211,7 +219,9 @@ async fn destroy(
) -> ApiResult<StatusCode> {
let me = load_user(&state, uid).await?;
if !me.is_superuser {
return Err(ApiError::Forbidden("Only admins can delete tickets.".into()));
return Err(ApiError::Forbidden(
"Only admins can delete tickets.".into(),
));
}
// get_object is scoped; superusers see all, so a missing ticket is 404.
Ticket::find_by_id(id)
backend-rs/src/routes/users.rs
+95 -36
View File
@@ -39,9 +39,23 @@ use crate::throttle;
use crate::web::RequestCtx;
const THEMES: &[&str] = &[
"holy_light", "holy_dark", "latte", "frappe", "macchiato", "mocha", "pink_pastel",
"mint_pastel", "lavender_pastel", "dracula", "nord", "gruvbox", "oceanic", "t0p_sai",
"t0p_trench", "t0p_blurryface", "t0p_clancy",
"holy_light",
"holy_dark",
"latte",
"frappe",
"macchiato",
"mocha",
"pink_pastel",
"mint_pastel",
"lavender_pastel",
"dracula",
"nord",
"gruvbox",
"oceanic",
"t0p_sai",
"t0p_trench",
"t0p_blurryface",
"t0p_clancy",
];
const LANGUAGES: &[&str] = &["en", "de"];
const DRAWER_ANCHORS: &[&str] = &["left", "right"];
@@ -60,10 +74,22 @@ pub fn router() -> Router<AppState> {
.route("/users/change_password/", post(change_password))
.route("/users/update_profile/", patch(update_profile))
.route("/users/confirm_email/", post(confirm_email))
.route("/users/request_password_reset/", post(request_password_reset))
.route("/users/complete_password_reset/", post(complete_password_reset))
.route("/users/request_account_deletion/", post(request_account_deletion))
.route("/users/confirm_account_deletion/", post(confirm_account_deletion))
.route(
"/users/request_password_reset/",
post(request_password_reset),
)
.route(
"/users/complete_password_reset/",
post(complete_password_reset),
)
.route(
"/users/request_account_deletion/",
post(request_account_deletion),
)
.route(
"/users/confirm_account_deletion/",
post(confirm_account_deletion),
)
.route("/users/dashboard/", get(dashboard))
.route("/users/{id}/", get(retrieve))
.route("/users/{id}/follow/", post(follow))
@@ -207,7 +233,9 @@ async fn following_ids(state: &AppState, from: i32) -> ApiResult<Vec<i32>> {
[from.into()],
))
.await?;
rows.into_iter().map(|r| r.try_get("", "uid").map_err(Into::into)).collect()
rows.into_iter()
.map(|r| r.try_get("", "uid").map_err(Into::into))
.collect()
}
/// User ids that follow `to` (the `followers` set).
@@ -220,7 +248,9 @@ async fn follower_ids(state: &AppState, to: i32) -> ApiResult<Vec<i32>> {
[to.into()],
))
.await?;
rows.into_iter().map(|r| r.try_get("", "uid").map_err(Into::into)).collect()
rows.into_iter()
.map(|r| r.try_get("", "uid").map_err(Into::into))
.collect()
}
async fn users_by_ids(state: &AppState, ids: &[i32]) -> ApiResult<Vec<user::Model>> {
@@ -368,9 +398,10 @@ async fn update_preferences(
updated = true;
}
Some(v) => {
let banner_id = v.as_i64().map(|n| n as i32).or_else(|| {
v.as_str().and_then(|s| s.parse::<i32>().ok())
});
let banner_id = v
.as_i64()
.map(|n| n as i32)
.or_else(|| v.as_str().and_then(|s| s.parse::<i32>().ok()));
match banner_id {
Some(bid) => {
let banner = Banner::find_by_id(bid)
@@ -472,8 +503,14 @@ async fn change_password(
) -> ApiResult<Json<Value>> {
let u = load_user(&state, uid).await?;
let data = parse_body(&body);
let old = data.get("old_password").and_then(|v| v.as_str()).unwrap_or("");
let new = data.get("new_password").and_then(|v| v.as_str()).unwrap_or("");
let old = data
.get("old_password")
.and_then(|v| v.as_str())
.unwrap_or("");
let new = data
.get("new_password")
.and_then(|v| v.as_str())
.unwrap_or("");
if !crate::password::verify_password(old, &u.password) {
return Err(err_json(
@@ -594,7 +631,9 @@ async fn confirm_email(
active.email_confirmation_code = Set(None);
active.update(&state.db).await?;
state.security.clear_code(&code_key);
Ok(Json(json!({ "status": "Email confirmed", "email": pending })))
Ok(Json(
json!({ "status": "Email confirmed", "email": pending }),
))
}
// ---- signup / verification ----------------------------------------------
@@ -607,7 +646,9 @@ async fn signup(
body: Bytes,
) -> ApiResult<(StatusCode, Json<Value>)> {
let ip = crate::web::client_ip(&headers, peer);
state.security.check_rate(&format!("signup:ip:{ip}"), throttle::SIGNUP)?;
state
.security
.check_rate(&format!("signup:ip:{ip}"), throttle::SIGNUP)?;
let data = parse_body(&body);
let (Some(username), Some(email), Some(password)) = (
field_str(&data, "username"),
@@ -835,9 +876,10 @@ async fn request_password_reset(
body: Bytes,
) -> ApiResult<Json<Value>> {
let ip = crate::web::client_ip(&headers, peer);
state
.security
.check_rate(&format!("pwreset_req:ip:{ip}"), throttle::PASSWORD_RESET_REQUEST)?;
state.security.check_rate(
&format!("pwreset_req:ip:{ip}"),
throttle::PASSWORD_RESET_REQUEST,
)?;
let generic = json!({
"status": "If an account exists with this email, a reset code has been sent."
});
@@ -878,12 +920,16 @@ async fn complete_password_reset(
body: Bytes,
) -> ApiResult<Json<Value>> {
let ip = crate::web::client_ip(&headers, peer);
state
.security
.check_rate(&format!("pwreset_done:ip:{ip}"), throttle::PASSWORD_RESET_COMPLETE)?;
state.security.check_rate(
&format!("pwreset_done:ip:{ip}"),
throttle::PASSWORD_RESET_COMPLETE,
)?;
let data = parse_body(&body);
let email = field_str(&data, "email").unwrap_or_default();
let code = data.get("code").and_then(|v| v.as_str()).unwrap_or_default();
let code = data
.get("code")
.and_then(|v| v.as_str())
.unwrap_or_default();
let new_password = data.get("password").and_then(|v| v.as_str()).unwrap_or("");
let code_key = pwreset_code_key(&email);
@@ -935,9 +981,10 @@ async fn request_account_deletion(
AuthUser(uid): AuthUser,
_body: Bytes,
) -> ApiResult<Json<Value>> {
state
.security
.check_rate(&format!("deletion_req:user:{uid}"), throttle::ACCOUNT_DELETION_REQUEST)?;
state.security.check_rate(
&format!("deletion_req:user:{uid}"),
throttle::ACCOUNT_DELETION_REQUEST,
)?;
let user = load_user(&state, uid).await?;
let code = gen_code();
let uname = user.username.clone();
@@ -994,9 +1041,8 @@ async fn confirm_account_deletion(
/// in dependency order, inside one transaction.
pub(crate) async fn delete_user_cascade(state: &AppState, uid: i32) -> ApiResult<()> {
let txn = state.db.begin().await?;
let exec = |sql: &'static str| {
Statement::from_sql_and_values(DbBackend::Sqlite, sql, [uid.into()])
};
let exec =
|sql: &'static str| Statement::from_sql_and_values(DbBackend::Sqlite, sql, [uid.into()]);
// Notifications referencing the user directly (recipient/actor) or via the
// user's ratings/replies/tickets/profile-comments.
@@ -1015,18 +1061,21 @@ pub(crate) async fn delete_user_cascade(state: &AppState, uid: i32) -> ApiResult
OR rating_id IN (SELECT id FROM api_rating WHERE user_id = ?1)",
))
.await?;
txn.execute(exec("DELETE FROM api_rating WHERE user_id = ?")).await?;
txn.execute(exec("DELETE FROM api_rating WHERE user_id = ?"))
.await?;
txn.execute(exec(
"DELETE FROM api_ticketmessage WHERE user_id = ?1 \
OR ticket_id IN (SELECT id FROM api_ticket WHERE user_id = ?1)",
))
.await?;
txn.execute(exec("DELETE FROM api_ticket WHERE user_id = ?")).await?;
txn.execute(exec("DELETE FROM api_ticket WHERE user_id = ?"))
.await?;
txn.execute(exec(
"DELETE FROM api_profilecomment WHERE author_id = ?1 OR profile_owner_id = ?1",
))
.await?;
txn.execute(exec("DELETE FROM api_userip WHERE user_id = ?")).await?;
txn.execute(exec("DELETE FROM api_userip WHERE user_id = ?"))
.await?;
txn.execute(exec(
"DELETE FROM api_user_following WHERE from_user_id = ?1 OR to_user_id = ?1",
))
@@ -1041,7 +1090,8 @@ pub(crate) async fn delete_user_cascade(state: &AppState, uid: i32) -> ApiResult
"DELETE FROM token_blacklist_outstandingtoken WHERE user_id = ?",
))
.await?;
txn.execute(exec("DELETE FROM api_user WHERE id = ?")).await?;
txn.execute(exec("DELETE FROM api_user WHERE id = ?"))
.await?;
txn.commit().await?;
Ok(())
}
@@ -1144,7 +1194,10 @@ async fn add_comment(
}
let mut dtos = build_profile_comments(&state, &ctx, vec![comment]).await?;
Ok((StatusCode::CREATED, Json(dtos.pop().ok_or(ApiError::Internal)?)))
Ok((
StatusCode::CREATED,
Json(dtos.pop().ok_or(ApiError::Internal)?),
))
}
/// DELETE /api/users/{id}/delete_comment/{comment_id}/
@@ -1298,8 +1351,14 @@ async fn dashboard(
.map(|c| (c.id, c.name))
.collect();
all_flavors.sort_by(|a, b| {
let ca = cat_names.get(&a.category_id).map(String::as_str).unwrap_or("");
let cb = cat_names.get(&b.category_id).map(String::as_str).unwrap_or("");
let ca = cat_names
.get(&a.category_id)
.map(String::as_str)
.unwrap_or("");
let cb = cat_names
.get(&b.category_id)
.map(String::as_str)
.unwrap_or("");
ca.cmp(cb).then_with(|| a.name.cmp(&b.name))
});
backend-rs/src/service.rs
+62 -29
View File
@@ -10,11 +10,13 @@ use sea_orm::{
use serde_json::Value;
use crate::dto::{
FlavorOut, NotificationOut, ProfileCommentOut, RatingOut, ReplyOut, TicketMessageOut, TicketOut,
UserOut,
FlavorOut, NotificationOut, ProfileCommentOut, RatingOut, ReplyOut, TicketMessageOut,
TicketOut, UserOut,
};
use crate::entities::prelude::*;
use crate::entities::{category, flavor, notification, profile_comment, rating, reply, ticket, ticket_message, user};
use crate::entities::{
category, flavor, notification, profile_comment, rating, reply, ticket, ticket_message, user,
};
use crate::error::ApiResult;
use crate::media::{flavor_image_list, flavor_primary_image, image_field_url};
use crate::state::AppState;
@@ -22,7 +24,10 @@ use crate::web::RequestCtx;
/// Comma-join i32 ids for an inlined SQL `IN (...)` list. Safe: ints only.
fn id_list(ids: &[i32]) -> String {
ids.iter().map(|i| i.to_string()).collect::<Vec<_>>().join(",")
ids.iter()
.map(|i| i.to_string())
.collect::<Vec<_>>()
.join(",")
}
/// Optional avatar → absolute media URL (empty path treated as absent).
@@ -204,10 +209,8 @@ pub async fn build_flavors(
.filter(crate::entities::category::Column::Id.is_in(category_ids))
.all(&state.db)
.await?;
let cat_map: HashMap<i32, (String, String)> = cats
.into_iter()
.map(|c| (c.id, (c.name, c.slug)))
.collect();
let cat_map: HashMap<i32, (String, String)> =
cats.into_iter().map(|c| (c.id, (c.name, c.slug))).collect();
// Ratings for these flavors (RatingSerializer order: -created_at)
let ratings = Rating::find()
@@ -241,8 +244,7 @@ pub async fn build_flavors(
.all(&state.db)
.await?
};
let user_map: HashMap<i32, user::Model> =
users.into_iter().map(|u| (u.id, u)).collect();
let user_map: HashMap<i32, user::Model> = users.into_iter().map(|u| (u.id, u)).collect();
// Replies grouped by rating id
let mut replies_by_rating: HashMap<i32, Vec<ReplyOut>> = HashMap::new();
@@ -264,8 +266,7 @@ pub async fn build_flavors(
}
// Flavor lookup for per-rating flavor fields
let flavor_map: HashMap<i32, &flavor::Model> =
flavors.iter().map(|f| (f.id, f)).collect();
let flavor_map: HashMap<i32, &flavor::Model> = flavors.iter().map(|f| (f.id, f)).collect();
// Ratings grouped by flavor id
let mut ratings_by_flavor: HashMap<i32, Vec<RatingOut>> = HashMap::new();
@@ -334,9 +335,22 @@ pub async fn build_flavors(
fn_ += 1;
}
}
avg_by_flavor.insert(*fid, if n == 0 { None } else { Some(sum as f64 / n as f64) });
followed_avg_by_flavor
.insert(*fid, if fn_ == 0 { None } else { Some(fsum as f64 / fn_ as f64) });
avg_by_flavor.insert(
*fid,
if n == 0 {
None
} else {
Some(sum as f64 / n as f64)
},
);
followed_avg_by_flavor.insert(
*fid,
if fn_ == 0 {
None
} else {
Some(fsum as f64 / fn_ as f64)
},
);
dist_by_flavor.insert(*fid, dist);
}
@@ -370,11 +384,19 @@ pub async fn build_flavors(
f.image.as_deref(),
f.image_url.as_deref(),
),
image_urls: flavor_image_list(ctx, media, &local_paths, f.image.as_deref(), &image_urls),
image_urls: flavor_image_list(
ctx,
media,
&local_paths,
f.image.as_deref(),
&image_urls,
),
is_available: f.is_available,
is_legacy: f.is_legacy,
shop_url: f.shop_url.clone(),
rating_distribution: dist_by_flavor.remove(&f.id).unwrap_or_else(empty_distribution),
rating_distribution: dist_by_flavor
.remove(&f.id)
.unwrap_or_else(empty_distribution),
});
}
Ok(out)
@@ -432,7 +454,9 @@ async fn social_counts(
/// Set of user ids that `viewer` follows, restricted to `ids`.
async fn following_of(state: &AppState, viewer: Option<i32>, ids: &[i32]) -> ApiResult<Vec<i32>> {
let Some(v) = viewer else { return Ok(Vec::new()) };
let Some(v) = viewer else {
return Ok(Vec::new());
};
if ids.is_empty() {
return Ok(Vec::new());
}
@@ -447,7 +471,9 @@ async fn following_of(state: &AppState, viewer: Option<i32>, ids: &[i32]) -> Api
),
))
.await?;
rows.into_iter().map(|r| r.try_get("", "uid").map_err(Into::into)).collect()
rows.into_iter()
.map(|r| r.try_get("", "uid").map_err(Into::into))
.collect()
}
/// Build `UserOut`s (mirrors `UserSerializer`), batching social counts and the
@@ -464,8 +490,10 @@ pub async fn build_users(
}
let ids: Vec<i32> = users.iter().map(|u| u.id).collect();
let (following, followers, unread) = social_counts(state, &ids).await?;
let followed: std::collections::HashSet<i32> =
following_of(state, viewer, &ids).await?.into_iter().collect();
let followed: std::collections::HashSet<i32> = following_of(state, viewer, &ids)
.await?
.into_iter()
.collect();
let banner_ids: Vec<i32> = users.iter().filter_map(|u| u.selected_banner_id).collect();
let banner_slugs: HashMap<i32, String> = if banner_ids.is_empty() {
@@ -636,7 +664,9 @@ pub async fn build_notifications(
}
if let Some(rep_id) = n.reply_id {
let rep = replies.get(&rep_id)?;
return ratings.get(&rep.rating_id).and_then(|r| flavors.get(&r.flavor_id));
return ratings
.get(&rep.rating_id)
.and_then(|r| flavors.get(&r.flavor_id));
}
None
};
@@ -713,13 +743,16 @@ pub async fn build_tickets(
let mut msgs_by_ticket: HashMap<i32, Vec<TicketMessageOut>> = HashMap::new();
for m in messages {
let u = users.get(&m.user_id);
msgs_by_ticket.entry(m.ticket_id).or_default().push(TicketMessageOut {
id: m.id,
username: u.map(|x| x.username.clone()).unwrap_or_default(),
text: m.text,
created_at: crate::datetime::drf_iso(&m.created_at),
is_admin: u.map(|x| x.is_superuser).unwrap_or(false),
});
msgs_by_ticket
.entry(m.ticket_id)
.or_default()
.push(TicketMessageOut {
id: m.id,
username: u.map(|x| x.username.clone()).unwrap_or_default(),
text: m.text,
created_at: crate::datetime::drf_iso(&m.created_at),
is_admin: u.map(|x| x.is_superuser).unwrap_or(false),
});
}
Ok(tickets
backend-rs/src/throttle.rs
+12 -3
View File
@@ -30,10 +30,16 @@ pub struct Rate {
impl Rate {
pub const fn per_minute(limit: usize) -> Self {
Rate { limit, window: Duration::from_secs(60) }
Rate {
limit,
window: Duration::from_secs(60),
}
}
pub const fn per_hour(limit: usize) -> Self {
Rate { limit, window: Duration::from_secs(3600) }
Rate {
limit,
window: Duration::from_secs(3600),
}
}
}
@@ -106,7 +112,10 @@ impl Security {
/// Note that a confirmation code was just minted for `key`.
pub fn record_code(&self, key: &str) {
self.codes.lock().unwrap().insert(key.to_string(), Instant::now());
self.codes
.lock()
.unwrap()
.insert(key.to_string(), Instant::now());
}
/// True only when we hold a mint time for `key` that is older than `CODE_TTL`.
docker-compose.yml
+9 -14
View File
@@ -1,11 +1,9 @@
services:
# --- Rust backend (strangler) --------------------------------------------
# Runs beside Django, sharing the SAME db.sqlite3, media/, banners/ and
# legacy/ via bind mounts and the SAME SECRET_KEY (so SimpleJWT tokens verify
# across both). nginx routes endpoint groups here one at a time; see
# backend-rs/README.md for the cutover order. ENABLE_SCHEDULER stays false
# until Django's beat container is stopped — only one scheduler may own
# api_job, or jobs run twice.
# --- Rust backend --------------------------------------------------------
# The API. Owns db.sqlite3, media/, banners/ and legacy/ via bind mounts.
# An in-process scheduler runs the jobs that Celery beat used to (gated by
# ENABLE_SCHEDULER — exactly one process must own api_job). SECRET_KEY signs
# the SimpleJWT-compatible tokens.
backend-rs:
build:
context: ./backend-rs
@@ -30,8 +28,8 @@ services:
- BACKUP_DIR=/srv/backend/backups
- FRONTEND_URL=${FRONTEND_URL:-http://${DOMAIN:-localhost}}
- JWT_AUTH_COOKIE_SECURE=${JWT_AUTH_COOKIE_SECURE:-true}
# Keep false until Django beat is stopped (Phase 3 of cutover).
- ENABLE_SCHEDULER=${RUST_ENABLE_SCHEDULER:-false}
# Rust owns api_job scheduling (Celery beat retired).
- ENABLE_SCHEDULER=${RUST_ENABLE_SCHEDULER:-true}
- RUST_LOG=${RUST_LOG:-info,holy_backend_rs=info,sea_orm=warn}
volumes:
- ./backend:/srv/backend
@@ -62,8 +60,8 @@ services:
env_file:
- .env
volumes:
# Serve /media/ straight from the shared mount (read-only), decoupling it
# from either backend so Django can be retired without losing media.
# Serve /media/ straight from the shared mount (read-only) so media is
# independent of the backend being up.
- ./backend/media:/usr/share/nginx/media:ro
depends_on:
backend-rs:
@@ -72,6 +70,3 @@ services:
networks:
holy-net:
driver: bridge
volumes:
db_data:
frontend/Dockerfile
+2 -2
View File
@@ -31,8 +31,8 @@ COPY --from=build-stage /app/dist /usr/share/nginx/html
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
# Nginx config: SPA + strangler API routing (Django/Rust per-group). See
# nginx.conf for the cutover layout. proxy_params.conf holds shared headers.
# Nginx config: serves the SPA and proxies /api to the Rust backend.
# proxy_params.conf holds shared proxy headers.
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY proxy_params.conf /etc/nginx/proxy_params.conf
frontend/nginx.conf
+10 -23
View File
@@ -1,17 +1,9 @@
# =========================================================================
# Strangler routing: Django (holy-backend) and Rust (holy-rust) run side by
# side, sharing one SQLite DB + media dir + SECRET_KEY. Every endpoint group
# below can be served by EITHER backend, so cutover is per-group and instant:
# change a `proxy_pass` target, then `docker exec holy-frontend nginx -s reload`
# (or recreate the container). Rollback is the same edit in reverse.
#
# Cutover order (see backend-rs/README.md):
# Phase 1 reads -> categories, flavors, banners
# Phase 2 auth+write -> auth/token, ratings, replies, users, notifications, tickets
# Phase 3 admin+jobs -> admin-custom (THEN stop Django beat + set
# RUST_ENABLE_SCHEDULER=true so exactly one scheduler runs)
# Phase 4 retire -> point catch-all /api/ at rust; serve /media/ from
# nginx; drop Django /admin/ + /static/ if unused.
# Frontend nginx: serves the SPA, proxies /api to the Rust backend
# (holy-rust:8001), and serves /media straight from the shared bind mount.
# The per-group /api locations are kept explicit (one proxy_pass each) for
# readability; all point at rust_backend. The catch-all /api/ covers anything
# not listed.
# =========================================================================
upstream rust_backend {
@@ -28,17 +20,13 @@ server {
try_files $uri $uri/ /index.html;
}
# --- API: per-group strangler routing --------------------------------
# Longest-prefix match wins, so these specific groups override the
# catch-all /api/ below. To cut a group over, swap django_backend ->
# rust_backend on its proxy_pass line.
# Phase 1 — read-only (safe to move first):
# --- API -> Rust backend ---------------------------------------------
# Longest-prefix match wins; these explicit groups and the catch-all
# /api/ all resolve to rust_backend.
location /api/categories/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
location /api/flavors/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
location /api/banners/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
# Phase 2 — auth + writes:
location /api/auth/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
location = /api/token/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; } # legacy JWT alias
location /api/ratings/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
@@ -47,13 +35,12 @@ server {
location /api/notifications/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
location /api/tickets/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
# Phase 3 — admin + jobs (cut over, THEN flip the scheduler ownership):
location /api/admin-custom/ { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
# OpenAPI schema / Swagger UI (both backends serve these):
# OpenAPI schema / Swagger UI:
location /api/schema { proxy_pass http://rust_backend; include /etc/nginx/proxy_params.conf; }
# Catch-all: anything not explicitly cut over stays on Django.
# Catch-all for any /api path not listed above.
location /api/ {
proxy_pass http://rust_backend;
include /etc/nginx/proxy_params.conf;
frontend/package.json
+1 -1
View File
@@ -13,7 +13,7 @@
"test": "vitest run",
"test:watch": "vitest",
"preview": "vite preview",
"schema:export": "cd ../backend && ./venv/bin/python manage.py spectacular --file schema.yaml",
"schema:export": "curl -fsS http://localhost:8001/api/schema/ -o ../backend/schema.yaml",
"openapi:gen": "openapi-typescript ../backend/schema.yaml -o src/api/schema.d.ts",
"openapi:sync": "npm run schema:export && npm run openapi:gen"
},
scripts/backup.sh
+23 -8
View File
@@ -1,7 +1,11 @@
#!/bin/bash
set -euo pipefail
# This script creates a full snapshot of the database and user media.
# Best run as a daily cron job.
# Full snapshot of the SQLite database + user media. Host-side and safe to run
# while the Rust backend has the DB open (WAL mode): uses `sqlite3 .backup`
# when available, otherwise an online copy via the same backend container.
# The Rust scheduler also runs backup_db on its own interval; this is the
# manual/cron supplement. Best run as a daily cron job.
ROOT_DIR=$(git rev-parse --show-toplevel)
cd "$ROOT_DIR"
@@ -10,6 +14,7 @@ TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_NAME="full_snapshot_$TIMESTAMP"
TEMP_DIR="backend/backups/$BACKUP_NAME"
FINAL_DEST="backend/backups"
DB_SRC="backend/db.sqlite3"
echo "========================================"
echo " Starting Full Project Backup... "
@@ -17,19 +22,29 @@ echo "========================================"
mkdir -p "$TEMP_DIR"
# 1. Run the database backup command
# 1. Database — consistent snapshot.
echo "Backing up database..."
docker exec holy-backend python manage.py backup_db --output "/app/backups/$BACKUP_NAME/db.sqlite3"
if command -v sqlite3 &>/dev/null; then
# Atomic, WAL-safe even with the backend holding the DB open.
sqlite3 "$DB_SRC" ".backup '$TEMP_DIR/db.sqlite3'"
else
# No host sqlite3: copy the DB plus its WAL sidecars so an in-flight WAL
# isn't lost. Install sqlite3 for a guaranteed-atomic snapshot.
echo " host sqlite3 not found — copying db + WAL sidecars"
cp "$DB_SRC" "$TEMP_DIR/db.sqlite3"
[ -f "$DB_SRC-wal" ] && cp "$DB_SRC-wal" "$TEMP_DIR/db.sqlite3-wal"
[ -f "$DB_SRC-shm" ] && cp "$DB_SRC-shm" "$TEMP_DIR/db.sqlite3-shm"
fi
# 2. Copy the media folder
# 2. Media.
echo "Backing up media..."
cp -r backend/media "$TEMP_DIR/media"
# 3. Create compressed archive
# 3. Compress.
echo "Compressing..."
tar -czf "$FINAL_DEST/$BACKUP_NAME.tar.gz" -C "backend/backups" "$BACKUP_NAME"
tar -czf "$FINAL_DEST/$BACKUP_NAME.tar.gz" -C "$FINAL_DEST" "$BACKUP_NAME"
# 4. Cleanup temp folder
# 4. Cleanup temp folder.
rm -rf "$TEMP_DIR"
echo ""
scripts/cutover.sh
-200
View File
@@ -1,200 +0,0 @@
#!/usr/bin/env bash
# =========================================================================
# Strangler cutover driver: Django -> Rust, zero downtime, reversible.
#
# Flips nginx `proxy_pass` targets one phase-group at a time and reloads
# nginx, then (Phase 3) hands job scheduling from Django's Celery beat to
# the Rust in-process scheduler. Every phase has an inverse `rollback`.
#
# It edits frontend/nginx.conf in place. To make reloads INSTANT (no image
# rebuild), run `cutover.sh dev-mount` once first: it bind-mounts the conf
# into holy-frontend so `nginx -s reload` picks up edits live. Without that,
# each flip rebuilds the frontend image (slower, but still no API downtime).
#
# Usage:
# scripts/cutover.sh status # show current routing + scheduler
# scripts/cutover.sh phase0 # build + start holy-rust (shadow)
# scripts/cutover.sh phase1 # flip reads
# scripts/cutover.sh phase2 # flip auth + writes
# scripts/cutover.sh phase3 # flip admin, hand off scheduler
# scripts/cutover.sh phase4 # flip catch-all (retire Django path)
# scripts/cutover.sh rollback {1|2|3|4} # undo a phase
# scripts/cutover.sh smoke # hit holy-rust /health/ directly
#
# Invariant: exactly ONE scheduler owns api_job. Django beat owns it until
# phase3; Rust's ENABLE_SCHEDULER stays false until then. Never both.
# =========================================================================
set -euo pipefail
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CONF="$REPO/frontend/nginx.conf"
ENVFILE="$REPO/.env"
DC="docker compose"
# Endpoint groups per phase (nginx location prefixes, sans trailing proxy line).
READS=(/api/categories/ /api/flavors/ /api/banners/)
WRITES=(/api/auth/ "= /api/token/" /api/ratings/ /api/replies/ /api/users/ /api/notifications/ /api/tickets/)
ADMIN=(/api/admin-custom/)
CATCHALL=(/api/schema /api/)
c_red() { printf '\033[31m%s\033[0m\n' "$*"; }
c_grn() { printf '\033[32m%s\033[0m\n' "$*"; }
c_ylw() { printf '\033[33m%s\033[0m\n' "$*"; }
die() { c_red "ERROR: $*" >&2; exit 1; }
[ -f "$CONF" ] || die "nginx.conf not found at $CONF"
# Flip the upstream on the location line(s) for a given prefix.
# $1 = location token (e.g. /api/flavors/ or '= /api/token/')
# $2 = target upstream (rust_backend | django_backend)
flip_one() {
local loc="$1" target="$2"
# Match: location <loc> { proxy_pass http://<anything>; ... }
# Anchor on the exact location token to avoid hitting the catch-all /api/.
local esc; esc="$(printf '%s' "$loc" | sed 's/[.[\*^$/]/\\&/g')"
# The catch-all "/api/" lives on its own multi-line block; handle separately.
if [ "$loc" = "/api/" ]; then
# Flip only the catch-all block's proxy_pass (the one not on a one-liner).
perl -0pi -e "s{(location /api/ \{\s*\n\s*proxy_pass http://)\w+;}{\${1}${target};}g" "$CONF"
return
fi
sed -i -E "s|(location ${esc}[[:space:]]+\{[[:space:]]+proxy_pass http://)[a-z_]+;|\1${target};|" "$CONF"
}
flip_group() {
local target="$1"; shift
local g
for g in "$@"; do flip_one "$g" "$target"; done
}
reload_nginx() {
if $DC ps frontend --format '{{.Names}}' 2>/dev/null | grep -q holy-frontend \
&& docker inspect holy-frontend --format '{{range .Mounts}}{{.Destination}}{{"\n"}}{{end}}' 2>/dev/null \
| grep -q '/etc/nginx/conf.d/default.conf'; then
c_ylw "conf is bind-mounted -> live reload"
docker exec holy-frontend nginx -t && docker exec holy-frontend nginx -s reload
else
c_ylw "conf baked into image -> rebuilding frontend (no API downtime)"
$DC up -d --build frontend
fi
}
set_env() { # set_env KEY value (idempotent upsert in .env)
local k="$1" v="$2"
[ -f "$ENVFILE" ] || touch "$ENVFILE"
if grep -qE "^${k}=" "$ENVFILE"; then
sed -i -E "s|^${k}=.*|${k}=${v}|" "$ENVFILE"
else
printf '%s=%s\n' "$k" "$v" >> "$ENVFILE"
fi
}
confirm() {
read -r -p "$1 [y/N] " a; [ "$a" = y ] || [ "$a" = Y ] || die "aborted"
}
cmd_status() {
c_grn "== nginx routing (frontend/nginx.conf) =="
grep -E 'location (= )?/api/' "$CONF" | sed -E 's/\s*include.*//' \
| sed -E 's/proxy_pass http:\/\/(rust_backend)/proxy_pass -> \x1b[32m\1\x1b[0m/; s/proxy_pass http:\/\/(django_backend)/proxy_pass -> \1/'
echo
c_grn "== scheduler ownership =="
local rust_sched; rust_sched="$(grep -E '^RUST_ENABLE_SCHEDULER=' "$ENVFILE" 2>/dev/null | cut -d= -f2 || true)"
echo "RUST_ENABLE_SCHEDULER=${rust_sched:-unset (=false)}"
if $DC ps beat --format '{{.Names}} {{.State}}' 2>/dev/null | grep -q running; then
echo "Django beat: RUNNING (Django owns api_job)"
else
echo "Django beat: stopped"
fi
}
cmd_smoke() {
c_grn "Hitting holy-rust /health/ directly (bypasses nginx)..."
docker exec holy-frontend wget -qO- http://holy-rust:8001/health/ \
|| die "holy-rust not reachable — run phase0 first?"
echo; c_grn "OK"
}
cmd_phase0() {
grep -qE '^SECRET_KEY=.+' "$ENVFILE" 2>/dev/null \
|| die "SECRET_KEY must be set in .env to the SAME value Django uses (JWTs break otherwise)"
c_grn "Phase 0: building + starting holy-rust in shadow (no traffic)..."
$DC up -d --build backend-rs
c_ylw "Wait for healthy, then: scripts/cutover.sh smoke"
}
cmd_phase1() { c_grn "Phase 1: flip READS -> rust"; flip_group rust_backend "${READS[@]}"; reload_nginx; cmd_status; }
cmd_phase2() { c_grn "Phase 2: flip AUTH+WRITES -> rust"; flip_group rust_backend "${WRITES[@]}"; reload_nginx; cmd_status; }
cmd_phase3() {
c_grn "Phase 3: flip ADMIN -> rust, then hand off scheduler"
flip_group rust_backend "${ADMIN[@]}"; reload_nginx
c_ylw "About to STOP Django beat/worker/flower and ENABLE Rust scheduler."
c_ylw "This transfers api_job ownership. Ensure no job is mid-run."
confirm "Proceed with scheduler handover?"
$DC stop beat worker flower
set_env RUST_ENABLE_SCHEDULER true
$DC up -d backend-rs
c_grn "Done. Check: docker logs holy-rust | grep 'job scheduler ENABLED'"
cmd_status
}
cmd_phase4() {
c_grn "Phase 4: flip CATCH-ALL /api/ + /api/schema -> rust (Django serves only /admin/ + /static/)"
flip_group rust_backend "${CATCHALL[@]}"; reload_nginx
c_ylw "Django now handles only its admin site. To drop it entirely:"
c_ylw " $DC stop backend ; then remove /admin/ + /static/ locations from nginx.conf"
c_ylw " $DC stop redis # only a Celery broker + Django cache; Rust uses neither"
cmd_status
}
cmd_rollback() {
local p="${1:-}"
case "$p" in
1) c_grn "Rollback Phase 1"; flip_group django_backend "${READS[@]}"; reload_nginx ;;
2) c_grn "Rollback Phase 2"; flip_group django_backend "${WRITES[@]}"; reload_nginx ;;
3) c_grn "Rollback Phase 3: scheduler back to Django, admin back to Django"
set_env RUST_ENABLE_SCHEDULER false
$DC up -d backend-rs
$DC start beat worker flower
flip_group django_backend "${ADMIN[@]}"; reload_nginx ;;
4) c_grn "Rollback Phase 4"; flip_group django_backend "${CATCHALL[@]}"; reload_nginx ;;
*) die "usage: cutover.sh rollback {1|2|3|4}" ;;
esac
cmd_status
}
cmd_dev_mount() {
c_ylw "Add this under the 'frontend' service volumes in docker-compose.yml for live reloads:"
echo ' - ./frontend/nginx.conf:/etc/nginx/conf.d/default.conf'
echo ' - ./frontend/proxy_params.conf:/etc/nginx/proxy_params.conf'
c_ylw "Then: $DC up -d frontend"
}
case "${1:-}" in
status) cmd_status ;;
smoke) cmd_smoke ;;
phase0) cmd_phase0 ;;
phase1) cmd_phase1 ;;
phase2) cmd_phase2 ;;
phase3) cmd_phase3 ;;
phase4) cmd_phase4 ;;
rollback) cmd_rollback "${2:-}" ;;
dev-mount) cmd_dev_mount ;;
*) cat <<EOF
Strangler cutover driver. Phases are reversible; data never forks (shared DB).
status show routing + scheduler ownership
smoke curl holy-rust /health/ directly
phase0 build + start holy-rust (shadow, no traffic)
phase1 flip reads (categories, flavors, banners)
phase2 flip auth+writes (auth, token, ratings, replies, users, notifications, tickets)
phase3 flip admin + hand scheduler Django->Rust (CAREFUL)
phase4 flip catch-all /api/ (Django -> only /admin/ + /static/)
rollback {1|2|3|4} undo a phase
dev-mount print compose lines for live nginx reloads
See MIGRATION.md for the full runbook.
EOF
;;
esac
scripts/pre-push.sh
+7 -24
View File
@@ -3,11 +3,6 @@
# Get the root directory of the project
ROOT_DIR=$(git rev-parse --show-toplevel)
# Set environment variables for non-interactive backend checks
export SECRET_KEY="pre-push-check-key"
export DEBUG="false"
export ALLOWED_HOSTS="localhost"
echo "========================================"
echo " Running Pre-Push Integrity Checks "
echo "========================================"
@@ -23,32 +18,20 @@ if ! npm run build; then
exit 1
fi
# 2. Backend Check
# 2. Rust backend checks
echo ""
echo "Step 2: Checking Backend Integrity..."
cd "$ROOT_DIR/backend"
echo "Step 2: Checking Rust backend..."
cd "$ROOT_DIR/backend-rs"
# Detect python executable (prefer venv if exists)
if [ -f "venv/bin/python" ]; then
PYTHON_EXEC="venv/bin/python"
elif command -v python3 &>/dev/null; then
PYTHON_EXEC="python3"
else
PYTHON_EXEC="python"
fi
# Check for missing migrations
if ! $PYTHON_EXEC manage.py makemigrations --check --dry-run &>/dev/null; then
if ! cargo fmt --check; then
echo ""
echo "❌ ERROR: Missing migrations detected."
echo " Please run 'python manage.py makemigrations' locally."
echo "❌ ERROR: Rust formatting issues. Run 'cargo fmt' in backend-rs/."
exit 1
fi
# Run Django system checks
if ! $PYTHON_EXEC manage.py check; then
if ! cargo check --locked; then
echo ""
echo "❌ ERROR: Django system check failed."
echo "❌ ERROR: Rust backend does not compile."
exit 1
fi