mirror of
https://github.com/narrrl/pl8list-rewind.git
synced 2026-11-03 16:41:26 +01:00
Add pl8list infinite-scroll app with pl8list sign-in
Endless TikTok-style feed over pl8list's Remix loaders. A guest crawler builds a shared catalogue; a per-user ranker learns from likes, full listens and skips. Users sign in with pl8list's email code; only a hash of the local session token is stored. Ships as a hardened Docker image with a Traefik override for pl8.narl.io.
This commit is contained in:
commit
4a7f6903a3
20 files changed
+2192
No files matched your search
@@ -0,0 +1,9 @@
|
|||||||
|
.git
|
||||||
|
.env
|
||||||
|
*.db
|
||||||
|
*.db-*
|
||||||
|
__pycache__/
|
||||||
|
docs/
|
||||||
|
README.md
|
||||||
|
compose*.yaml
|
||||||
|
Dockerfile
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
# Optional overrides. Defaults work in Docker.
|
||||||
|
# HOST=127.0.0.1
|
||||||
|
# PORT=8790
|
||||||
|
# PL8_DB=/data/pl8list.db
|
||||||
|
# Set by compose.override.yaml when behind Traefik:
|
||||||
|
# TRUST_PROXY=1
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
.env
|
||||||
|
*.db
|
||||||
|
*.db-*
|
||||||
|
__pycache__/
|
||||||
+22
@@ -0,0 +1,22 @@
|
|||||||
|
# syntax=docker/dockerfile:1
|
||||||
|
FROM python:3.14-alpine
|
||||||
|
|
||||||
|
ENV PYTHONUNBUFFERED=1 \
|
||||||
|
PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
HOST=0.0.0.0 \
|
||||||
|
PORT=8790 \
|
||||||
|
PL8_DB=/data/pl8list.db
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY *.py ./
|
||||||
|
COPY static ./static
|
||||||
|
RUN python -m compileall -q . \
|
||||||
|
&& adduser -D -H -u 10001 app \
|
||||||
|
&& mkdir /data && chown app:app /data
|
||||||
|
|
||||||
|
USER app
|
||||||
|
VOLUME /data
|
||||||
|
EXPOSE 8790
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s \
|
||||||
|
CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8790/healthz', timeout=2)"
|
||||||
|
CMD ["python", "server.py"]
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
# pl8list ∞
|
||||||
|
|
||||||
|
An endless, TikTok-style scroll through [pl8list](https://pl8list.com), with its own
|
||||||
|
recommendation algorithm that learns from what you like, finish and skip. Sign in with
|
||||||
|
your pl8list account; each user gets their own feed and taste profile.
|
||||||
|
|
||||||
|
Not affiliated with pl8list.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
<p align="center">
|
||||||
|
<img src="docs/feed-mobile.png" width="260" alt="Feed on mobile">
|
||||||
|
<img src="docs/login.png" width="520" alt="Sign-in screen">
|
||||||
|
</p>
|
||||||
|
|
||||||
|
| Likes | Filters |
|
||||||
|
| ----- | ------- |
|
||||||
|
|  |  |
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose up -d --build # http://127.0.0.1:8790
|
||||||
|
```
|
||||||
|
|
||||||
|
or without Docker (Python 3.11+, no dependencies):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 server.py
|
||||||
|
```
|
||||||
|
|
||||||
|
`compose.override.yaml` puts it behind Traefik at `pl8.narl.io` on the external `narl`
|
||||||
|
network, the same way as the other narl.io services. Docker Compose picks it up
|
||||||
|
automatically. Delete it, or change the host name, to run somewhere else.
|
||||||
|
|
||||||
|
Data (catalogue, users, listening history) lives in the `data` volume as one SQLite file.
|
||||||
|
|
||||||
|
| variable | default | meaning |
|
||||||
|
| ------------- | ------------------ | ------------------------------------------------- |
|
||||||
|
| `HOST` | `127.0.0.1` | bind address (`0.0.0.0` in the image) |
|
||||||
|
| `PORT` | `8790` | port |
|
||||||
|
| `PL8_DB` | `./pl8list.db` | SQLite path (`/data/pl8list.db` in the image) |
|
||||||
|
| `TRUST_PROXY` | `0` | `1` behind a reverse proxy: use `X-Forwarded-For` for rate limits and `X-Forwarded-Proto` for `Secure` cookies |
|
||||||
|
|
||||||
|
## Accounts
|
||||||
|
|
||||||
|
There are no passwords here. Users sign in with the email of their pl8list account:
|
||||||
|
|
||||||
|
1. The user enters their email. The server starts pl8list's own login, and pl8list mails
|
||||||
|
them a 6-digit code.
|
||||||
|
2. The user types the code. The server passes it to pl8list. If pl8list accepts it, the
|
||||||
|
server reads the user's pl8list id and handle, creates a local account, and sets its
|
||||||
|
own session cookie (`pl8inf`, HttpOnly, 90 days).
|
||||||
|
|
||||||
|
The pl8list session from step 2 is used once, in memory, to read the user's personal pl8list
|
||||||
|
feed (artists in their library). It is never written to disk. Only a SHA-256 hash of the
|
||||||
|
local session token is stored. Logins are rate-limited to 5 per IP and 3 per email per
|
||||||
|
15 minutes, with at most 5 code attempts per login.
|
||||||
|
|
||||||
|
## Controls
|
||||||
|
|
||||||
|
| key | action |
|
||||||
|
| -------------- | ---------------- |
|
||||||
|
| ↓ / j, ↑ / k | next / previous |
|
||||||
|
| space | play / pause |
|
||||||
|
| ← / → | seek 10 s |
|
||||||
|
| l | like |
|
||||||
|
|
||||||
|
Click the waveform to seek. "filters" sets free-only and a BPM range. "likes" lists everything
|
||||||
|
you liked, with links to pl8list.
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
pl8list has no public API, but it is a Remix app: every page loader returns JSON when you add
|
||||||
|
`?_data=<route id>`, and `/api/*` routes return JSON directly. Paginated loaders also need
|
||||||
|
`pagination=true`. All of this works without an account for public data:
|
||||||
|
|
||||||
|
| endpoint | gives |
|
||||||
|
| -------------------------------------------------------------------- | ----------------------------------- |
|
||||||
|
| `/?_data=routes/_index` | best sellers, recommended, ticker |
|
||||||
|
| `/api/feed?page=N&pagination=true` | a user's feed (needs their session) |
|
||||||
|
| `/api/activity-ticker` | artists with recent downloads/sales |
|
||||||
|
| `/best-selling-artists?_data=routes/best-selling-artists` | top artists |
|
||||||
|
| `/search/users?q=…&page=N&pagination=true&_data=routes/search.users` | artist search (≥ 3 chars) |
|
||||||
|
| `/<handle>?page=N&pagination=true&_data=routes/$handle` | an artist's tracks, 10 per page |
|
||||||
|
| `/<handle>/<slug>/related?_data=routes/$handle_.$slug.related` | 3 related tracks |
|
||||||
|
| `/api/<handle>/<slug>/hls/playlist.m3u8?v=<uploadId>[&preview=true]` | HLS audio |
|
||||||
|
| `/login`, `/verify?email=…&mode=login` | email + 6-digit code login |
|
||||||
|
|
||||||
|
**Crawler** (`crawler.py`). There is no "all tracks" listing, so one shared catalogue grows by
|
||||||
|
snowballing: home sections, the ticker, best-selling artists and artist search give artists;
|
||||||
|
artist pages give tracks; related pages pull in neighbours. Artists are re-checked daily for
|
||||||
|
new uploads. Requests are throttled to about 1.4 per second while someone has used the app in
|
||||||
|
the last 30 minutes, or while the catalogue has fewer than 500 tracks. Otherwise it slows to
|
||||||
|
one request every 20 seconds. When a user likes or finishes a track, its related tracks and
|
||||||
|
its artist's catalogue are crawled first.
|
||||||
|
|
||||||
|
**Ranking** (`ranker.py`). Each track the user has not seen yet gets a score from popularity,
|
||||||
|
freshness, artist affinity (from their likes, completions and skips), closeness to the BPM and
|
||||||
|
key of tracks they liked, whether it is related to something they liked, and some random
|
||||||
|
exploration. Artists are then spread out so you don't get five tracks from one person in a
|
||||||
|
row. Tune `WEIGHTS` at the top of the file.
|
||||||
|
|
||||||
|
**Audio.** The browser only talks to this server. HLS playlists and segments are proxied from
|
||||||
|
pl8list as a guest, so users hear previews of paid tracks and full free tracks, the same as a
|
||||||
|
logged-out visitor on pl8list.
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
# pl8.narl.io behind Traefik on external "narl" network. TLS ends at Traefik; the
|
||||||
|
# container itself is not published on any host port.
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
environment:
|
||||||
|
TRUST_PROXY: "1"
|
||||||
|
ports: !reset []
|
||||||
|
networks:
|
||||||
|
- narl
|
||||||
|
labels:
|
||||||
|
- "traefik.enable=true"
|
||||||
|
- "traefik.docker.network=narl"
|
||||||
|
- "traefik.http.routers.pl8list-infinite.rule=Host(`pl8.narl.io`)"
|
||||||
|
- "traefik.http.routers.pl8list-infinite.entrypoints=websecure"
|
||||||
|
- "traefik.http.routers.pl8list-infinite.tls.certresolver=https_resolver"
|
||||||
|
- "traefik.http.services.pl8list-infinite.loadbalancer.server.port=8790"
|
||||||
|
|
||||||
|
networks:
|
||||||
|
narl:
|
||||||
|
external: true
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# Generic setup: app on 127.0.0.1:8790. Put reverse proxy (Traefik labels, networks)
|
||||||
|
# in compose.override.yaml and drop the port mapping if the proxy reaches the
|
||||||
|
# container over a Docker network.
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
build: .
|
||||||
|
image: pl8list-infinite:latest
|
||||||
|
env_file:
|
||||||
|
- path: .env
|
||||||
|
required: false
|
||||||
|
environment:
|
||||||
|
TRUST_PROXY: "0"
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:8790:8790"
|
||||||
|
volumes:
|
||||||
|
- data:/data
|
||||||
|
read_only: true
|
||||||
|
tmpfs:
|
||||||
|
- /tmp
|
||||||
|
restart: unless-stopped
|
||||||
|
cap_drop:
|
||||||
|
- ALL
|
||||||
|
security_opt:
|
||||||
|
- no-new-privileges:true
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "1m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
data:
|
||||||
+243
@@ -0,0 +1,243 @@
|
|||||||
|
"""Background discovery crawler.
|
||||||
|
|
||||||
|
There is no "all tracks" endpoint, so the pool grows by snowballing:
|
||||||
|
home page sections, the activity ticker, best-selling artists and user search
|
||||||
|
give us artists; artist pages give us their tracks; "related" pages of tracks
|
||||||
|
users liked or finished pull in neighbours. Everything goes through one priority
|
||||||
|
queue so feedback is crawled first.
|
||||||
|
|
||||||
|
The catalogue is shared by all users and crawled as a guest. The crawler runs
|
||||||
|
at full pace only while someone has used the app recently.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import heapq
|
||||||
|
import itertools
|
||||||
|
import logging
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
|
||||||
|
from pl8list import Client, normalize_item, normalize_related
|
||||||
|
|
||||||
|
log = logging.getLogger("crawler")
|
||||||
|
|
||||||
|
# Search needs at least 3 characters. Genre-ish words find lots of artist handles.
|
||||||
|
SEARCH_TERMS = [
|
||||||
|
"dub", "bass", "dnb", "jungle", "garage", "grime", "riddim", "dubstep", "trap", "techno",
|
||||||
|
"house", "beats", "music", "sound", "prod", "audio", "records", "drill", "breaks", "wave",
|
||||||
|
"deep", "dark", "the", "official", "dj", "mc", "lofi", "halftime", "neuro", "tek",
|
||||||
|
]
|
||||||
|
|
||||||
|
# Lower number = sooner.
|
||||||
|
P_FEEDBACK = 0
|
||||||
|
P_SEED = 10
|
||||||
|
P_ARTIST_FIRST = 20
|
||||||
|
P_ARTIST_MORE = 40
|
||||||
|
P_RELATED = 30
|
||||||
|
P_SEARCH = 60
|
||||||
|
|
||||||
|
ACTIVE_WINDOW = 1800 # a user seen this recently counts as active
|
||||||
|
IDLE_DELAY = 20 # extra seconds between requests when nobody is active
|
||||||
|
MIN_POOL = 500 # crawl at full pace until the catalogue has this many tracks
|
||||||
|
REFRESH_AFTER = 86400 # re-check an artist's newest page for uploads this often
|
||||||
|
|
||||||
|
|
||||||
|
class Crawler:
|
||||||
|
def __init__(self, client, db):
|
||||||
|
self.client = client
|
||||||
|
self.db = db
|
||||||
|
self.queue = []
|
||||||
|
self.queued = set()
|
||||||
|
self.seq = itertools.count()
|
||||||
|
self.cv = threading.Condition()
|
||||||
|
self.last_seed = 0.0
|
||||||
|
# Logged-in pl8list sessions, kept in memory only while that user's feed is crawled.
|
||||||
|
self.feed_sessions = {}
|
||||||
|
|
||||||
|
def push(self, priority, kind, arg):
|
||||||
|
key = (kind, arg)
|
||||||
|
with self.cv:
|
||||||
|
if key in self.queued:
|
||||||
|
return
|
||||||
|
self.queued.add(key)
|
||||||
|
heapq.heappush(self.queue, (priority, next(self.seq), kind, arg))
|
||||||
|
self.cv.notify()
|
||||||
|
|
||||||
|
def pop(self):
|
||||||
|
with self.cv:
|
||||||
|
while not self.queue:
|
||||||
|
self.cv.wait(timeout=30)
|
||||||
|
if not self.queue:
|
||||||
|
self._refill()
|
||||||
|
priority, _, kind, arg = heapq.heappop(self.queue)
|
||||||
|
self.queued.discard((kind, arg))
|
||||||
|
return priority, kind, arg
|
||||||
|
|
||||||
|
# Feedback hooks, called from the web server ----------------------------
|
||||||
|
|
||||||
|
def on_positive(self, track_id):
|
||||||
|
"""Liked or listened through: crawl its related tracks and its artist's catalogue."""
|
||||||
|
row = self.db.query("SELECT handle, slug FROM tracks WHERE id=?", (track_id,))
|
||||||
|
if not row:
|
||||||
|
return
|
||||||
|
handle, slug = row[0]["handle"], row[0]["slug"]
|
||||||
|
self.push(P_FEEDBACK, "related", (track_id, handle, slug))
|
||||||
|
self.push(P_FEEDBACK, "artist", handle)
|
||||||
|
|
||||||
|
def on_login(self, user_id, session):
|
||||||
|
"""Crawl the user's personal feed (artists in their pl8list library) with their session."""
|
||||||
|
self.feed_sessions[user_id] = session
|
||||||
|
self.push(P_FEEDBACK, "feed", (user_id, 1))
|
||||||
|
|
||||||
|
# Main loop -------------------------------------------------------------
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
threading.Thread(target=self.run, daemon=True, name="crawler").start()
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
self._seed()
|
||||||
|
while True:
|
||||||
|
_, kind, arg = self.pop()
|
||||||
|
try:
|
||||||
|
getattr(self, "do_" + kind)(arg)
|
||||||
|
except Exception:
|
||||||
|
log.exception("crawl %s %r failed", kind, arg)
|
||||||
|
if kind != "feed" and not self._busy():
|
||||||
|
time.sleep(IDLE_DELAY)
|
||||||
|
if time.time() - self.last_seed > 600:
|
||||||
|
self._seed()
|
||||||
|
|
||||||
|
def _busy(self):
|
||||||
|
"""Crawl at full pace while someone is listening or the catalogue is still small."""
|
||||||
|
if self.db.active_since(time.time() - ACTIVE_WINDOW):
|
||||||
|
return True
|
||||||
|
return self.db.query("SELECT COUNT(*) AS n FROM tracks")[0]["n"] < MIN_POOL
|
||||||
|
|
||||||
|
def _seed(self):
|
||||||
|
self.last_seed = time.time()
|
||||||
|
self.push(P_SEED, "home", None)
|
||||||
|
self.push(P_SEED, "ticker", None)
|
||||||
|
self.push(P_SEED, "best_artists", None)
|
||||||
|
for term in SEARCH_TERMS:
|
||||||
|
self.push(P_SEARCH, "search", term)
|
||||||
|
|
||||||
|
def _refill(self):
|
||||||
|
"""Queue ran dry: queue more artist pages, related lookups and stale artists."""
|
||||||
|
for r in self.db.query("""
|
||||||
|
SELECT handle FROM artists
|
||||||
|
WHERE total_pages IS NULL OR pages_fetched < total_pages
|
||||||
|
ORDER BY verified DESC, pages_fetched ASC LIMIT 20
|
||||||
|
"""):
|
||||||
|
self.push(P_ARTIST_MORE, "artist", r["handle"])
|
||||||
|
for r in self.db.query("""
|
||||||
|
SELECT id, handle, slug FROM tracks WHERE related_fetched=0
|
||||||
|
ORDER BY saves + downloads DESC LIMIT 10
|
||||||
|
"""):
|
||||||
|
self.push(P_RELATED, "related", (r["id"], r["handle"], r["slug"]))
|
||||||
|
for r in self.db.query("""
|
||||||
|
SELECT handle FROM artists
|
||||||
|
WHERE pages_fetched > 0 AND last_crawled < ?
|
||||||
|
ORDER BY last_crawled LIMIT 20
|
||||||
|
""", (time.time() - REFRESH_AFTER,)):
|
||||||
|
self.push(P_ARTIST_MORE, "artist_refresh", r["handle"])
|
||||||
|
|
||||||
|
# Jobs ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _add_artist(self, handle, source, name=None, img=None, verified=0, priority=P_ARTIST_FIRST):
|
||||||
|
known = self.db.query("SELECT pages_fetched FROM artists WHERE handle=?", (handle,))
|
||||||
|
self.db.upsert_artist(handle, name, img, verified, source)
|
||||||
|
if not known or known[0]["pages_fetched"] == 0:
|
||||||
|
self.push(priority, "artist", handle)
|
||||||
|
|
||||||
|
def do_home(self, _):
|
||||||
|
d = self.client.home() or {}
|
||||||
|
for section in ("bestSellers", "recommended"):
|
||||||
|
self.db.upsert_tracks([normalize_item(i) for i in (d.get(section) or {}).get("items", [])])
|
||||||
|
for e in d.get("tickerEvents") or []:
|
||||||
|
self._add_artist(e["artistHandle"], "ticker", e.get("artistName"))
|
||||||
|
|
||||||
|
def do_ticker(self, _):
|
||||||
|
for e in (self.client.activity_ticker() or {}).get("events", []):
|
||||||
|
self._add_artist(e["artistHandle"], "ticker", e.get("artistName"))
|
||||||
|
|
||||||
|
def do_best_artists(self, _):
|
||||||
|
d = self.client.best_selling_artists() or {}
|
||||||
|
for a in d.get("artists") or []:
|
||||||
|
self._add_artist(a["artist_handle"], "best_sellers", a.get("artist_name"),
|
||||||
|
a.get("artist_img"), a.get("artist_isVerified"), priority=P_SEED)
|
||||||
|
self.db.upsert_tracks([normalize_item(i) for i in (d.get("recommended") or {}).get("items", [])])
|
||||||
|
|
||||||
|
def do_feed(self, arg):
|
||||||
|
user_id, page = arg
|
||||||
|
session = self.feed_sessions.get(user_id)
|
||||||
|
if not session:
|
||||||
|
return
|
||||||
|
d = Client(session).feed(page) or {}
|
||||||
|
self.db.upsert_tracks([normalize_item(i) for i in d.get("tracks") or []])
|
||||||
|
for i in d.get("tracks") or []:
|
||||||
|
self.push(P_ARTIST_FIRST, "artist", i["track"]["user"]["handle"])
|
||||||
|
if page < (d.get("totalPages") or 0):
|
||||||
|
self.push(P_FEEDBACK, "feed", (user_id, page + 1))
|
||||||
|
else:
|
||||||
|
self.feed_sessions.pop(user_id, None)
|
||||||
|
|
||||||
|
def do_artist_refresh(self, handle):
|
||||||
|
"""Newest uploads are on page 1, so re-reading it catches new tracks."""
|
||||||
|
d = self.client.artist(handle, 1)
|
||||||
|
if d:
|
||||||
|
new = self.db.upsert_tracks([normalize_item(i) for i in d.get("tracks") or []])
|
||||||
|
if new:
|
||||||
|
log.info("artist %s: %d new uploads", handle, new)
|
||||||
|
self.db.execute("UPDATE artists SET last_crawled=? WHERE handle=?", (time.time(), handle))
|
||||||
|
|
||||||
|
def do_artist(self, handle):
|
||||||
|
row = self.db.query("SELECT pages_fetched, total_pages FROM artists WHERE handle=?", (handle,))
|
||||||
|
fetched = row[0]["pages_fetched"] if row else 0
|
||||||
|
total = row[0]["total_pages"] if row else None
|
||||||
|
if total is not None and fetched >= total:
|
||||||
|
return
|
||||||
|
page = fetched + 1
|
||||||
|
d = self.client.artist(handle, page)
|
||||||
|
if not d:
|
||||||
|
self.db.execute("UPDATE artists SET total_pages=0, last_crawled=? WHERE handle=?",
|
||||||
|
(time.time(), handle))
|
||||||
|
return
|
||||||
|
new = self.db.upsert_tracks([normalize_item(i) for i in d.get("tracks") or []])
|
||||||
|
total = d.get("totalPages") or 0
|
||||||
|
self.db.execute("UPDATE artists SET pages_fetched=?, total_pages=?, last_crawled=? WHERE handle=?",
|
||||||
|
(page, total, time.time(), handle))
|
||||||
|
log.info("artist %s page %d/%d: %d new tracks", handle, page, total, new)
|
||||||
|
if page < total:
|
||||||
|
self.push(P_ARTIST_MORE + page, "artist", handle)
|
||||||
|
|
||||||
|
def do_related(self, arg):
|
||||||
|
track_id, handle, slug = arg
|
||||||
|
d = self.client.related(handle, slug)
|
||||||
|
self.db.execute("UPDATE tracks SET related_fetched=1 WHERE id=?", (track_id,))
|
||||||
|
if not d:
|
||||||
|
return
|
||||||
|
rows = normalize_related(d)
|
||||||
|
self.db.upsert_tracks(rows, related_of=track_id)
|
||||||
|
for r in rows:
|
||||||
|
self._add_artist(r["handle"], "related", r["artist_name"], r["artist_img"], r["artist_verified"],
|
||||||
|
priority=P_RELATED)
|
||||||
|
|
||||||
|
def do_search(self, term):
|
||||||
|
row = self.db.query("SELECT pages_fetched, total_pages FROM search_terms WHERE term=?", (term,))
|
||||||
|
fetched = row[0]["pages_fetched"] if row else 0
|
||||||
|
total = row[0]["total_pages"] if row else None
|
||||||
|
if total is not None and fetched >= total:
|
||||||
|
return
|
||||||
|
page = fetched + 1
|
||||||
|
d = self.client.search_users(term, page) or {}
|
||||||
|
for item in d.get("items") or []:
|
||||||
|
a = item.get("artist") or {}
|
||||||
|
if a.get("handle"):
|
||||||
|
self._add_artist(a["handle"], "search", a.get("name"), a.get("img"), a.get("isVerified"),
|
||||||
|
priority=P_ARTIST_FIRST if a.get("isVerified") else P_SEARCH + 5)
|
||||||
|
total = d.get("totalPages") or 0
|
||||||
|
self.db.execute(
|
||||||
|
"INSERT INTO search_terms (term, pages_fetched, total_pages) VALUES (?,?,?) "
|
||||||
|
"ON CONFLICT(term) DO UPDATE SET pages_fetched=excluded.pages_fetched, total_pages=excluded.total_pages",
|
||||||
|
(term, page, total))
|
||||||
|
if page < total:
|
||||||
|
self.push(P_SEARCH + page, "search", term)
|
||||||
@@ -0,0 +1,196 @@
|
|||||||
|
"""SQLite store: the shared track catalogue, plus per-user accounts and listening events."""
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import secrets
|
||||||
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
|
||||||
|
SESSION_DAYS = 90
|
||||||
|
|
||||||
|
SCHEMA = """
|
||||||
|
CREATE TABLE IF NOT EXISTS artists (
|
||||||
|
handle TEXT PRIMARY KEY,
|
||||||
|
name TEXT,
|
||||||
|
img TEXT,
|
||||||
|
verified INTEGER DEFAULT 0,
|
||||||
|
source TEXT,
|
||||||
|
pages_fetched INTEGER DEFAULT 0,
|
||||||
|
total_pages INTEGER,
|
||||||
|
last_crawled REAL
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS tracks (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
handle TEXT NOT NULL,
|
||||||
|
artist_name TEXT,
|
||||||
|
artist_img TEXT,
|
||||||
|
artist_verified INTEGER DEFAULT 0,
|
||||||
|
slug TEXT NOT NULL,
|
||||||
|
title TEXT,
|
||||||
|
description TEXT,
|
||||||
|
bpm REAL,
|
||||||
|
key TEXT,
|
||||||
|
duration REAL,
|
||||||
|
preview_duration REAL,
|
||||||
|
price INTEGER DEFAULT 0,
|
||||||
|
created_at TEXT,
|
||||||
|
upload_id TEXT,
|
||||||
|
preview_upload_id TEXT,
|
||||||
|
waveform TEXT,
|
||||||
|
preview_waveform TEXT,
|
||||||
|
full_access INTEGER DEFAULT 0,
|
||||||
|
saves INTEGER DEFAULT 0,
|
||||||
|
downloads INTEGER DEFAULT 0,
|
||||||
|
purchases INTEGER DEFAULT 0,
|
||||||
|
comments INTEGER DEFAULT 0,
|
||||||
|
related_of INTEGER,
|
||||||
|
related_fetched INTEGER DEFAULT 0,
|
||||||
|
updated_at REAL
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS tracks_handle ON tracks(handle);
|
||||||
|
CREATE TABLE IF NOT EXISTS events (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
track_id INTEGER NOT NULL,
|
||||||
|
kind TEXT NOT NULL,
|
||||||
|
seconds REAL,
|
||||||
|
ts REAL NOT NULL
|
||||||
|
);
|
||||||
|
-- Users are pl8list accounts, keyed by their pl8list user id. No pl8list session is stored.
|
||||||
|
CREATE TABLE IF NOT EXISTS users (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
handle TEXT NOT NULL,
|
||||||
|
name TEXT,
|
||||||
|
img TEXT,
|
||||||
|
created_at REAL NOT NULL,
|
||||||
|
last_seen REAL
|
||||||
|
);
|
||||||
|
-- Our own login sessions. Only a hash of the cookie token is kept.
|
||||||
|
CREATE TABLE IF NOT EXISTS sessions (
|
||||||
|
token_hash TEXT PRIMARY KEY,
|
||||||
|
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
created_at REAL NOT NULL,
|
||||||
|
expires_at REAL NOT NULL
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS search_terms (
|
||||||
|
term TEXT PRIMARY KEY,
|
||||||
|
pages_fetched INTEGER DEFAULT 0,
|
||||||
|
total_pages INTEGER
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
|
||||||
|
TRACK_COLS = [
|
||||||
|
"id", "handle", "artist_name", "artist_img", "artist_verified", "slug", "title", "description",
|
||||||
|
"bpm", "key", "duration", "preview_duration", "price", "created_at", "upload_id",
|
||||||
|
"preview_upload_id", "waveform", "preview_waveform", "full_access", "saves", "downloads",
|
||||||
|
"purchases", "comments",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class DB:
|
||||||
|
def __init__(self, path):
|
||||||
|
self.conn = sqlite3.connect(path, check_same_thread=False, isolation_level=None)
|
||||||
|
self.conn.row_factory = sqlite3.Row
|
||||||
|
self.conn.execute("PRAGMA journal_mode=WAL")
|
||||||
|
self.conn.execute("PRAGMA foreign_keys=ON")
|
||||||
|
self.conn.executescript(SCHEMA)
|
||||||
|
# Databases from the single-user version have events without an owner.
|
||||||
|
if "user_id" not in [r["name"] for r in self.conn.execute("PRAGMA table_info(events)")]:
|
||||||
|
self.conn.execute("DELETE FROM events")
|
||||||
|
self.conn.execute("ALTER TABLE events ADD COLUMN user_id INTEGER")
|
||||||
|
self.conn.execute("CREATE INDEX IF NOT EXISTS events_user ON events(user_id, kind, track_id)")
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
|
||||||
|
def query(self, sql, args=()):
|
||||||
|
with self.lock:
|
||||||
|
return self.conn.execute(sql, args).fetchall()
|
||||||
|
|
||||||
|
def execute(self, sql, args=()):
|
||||||
|
with self.lock:
|
||||||
|
return self.conn.execute(sql, args)
|
||||||
|
|
||||||
|
def upsert_tracks(self, rows, related_of=None):
|
||||||
|
"""Insert or refresh tracks. Returns how many were new."""
|
||||||
|
new = 0
|
||||||
|
now = time.time()
|
||||||
|
with self.lock:
|
||||||
|
for r in rows:
|
||||||
|
if not r:
|
||||||
|
continue
|
||||||
|
exists = self.conn.execute("SELECT 1 FROM tracks WHERE id=?", (r["id"],)).fetchone()
|
||||||
|
vals = [r[c] for c in TRACK_COLS]
|
||||||
|
if exists:
|
||||||
|
sets = ",".join("%s=?" % c for c in TRACK_COLS[1:])
|
||||||
|
self.conn.execute("UPDATE tracks SET %s, updated_at=? WHERE id=?" % sets,
|
||||||
|
vals[1:] + [now, r["id"]])
|
||||||
|
else:
|
||||||
|
new += 1
|
||||||
|
self.conn.execute(
|
||||||
|
"INSERT INTO tracks (%s, related_of, updated_at) VALUES (%s)"
|
||||||
|
% (",".join(TRACK_COLS), ",".join("?" * (len(TRACK_COLS) + 2))),
|
||||||
|
vals + [related_of, now])
|
||||||
|
self.conn.execute(
|
||||||
|
"INSERT INTO artists (handle, name, img, verified, source) VALUES (?,?,?,?,?) "
|
||||||
|
"ON CONFLICT(handle) DO UPDATE SET name=excluded.name, img=excluded.img, "
|
||||||
|
"verified=excluded.verified",
|
||||||
|
(r["handle"], r["artist_name"], r["artist_img"], r["artist_verified"], "track"))
|
||||||
|
return new
|
||||||
|
|
||||||
|
def upsert_artist(self, handle, name=None, img=None, verified=0, source=None):
|
||||||
|
self.execute(
|
||||||
|
"INSERT INTO artists (handle, name, img, verified, source) VALUES (?,?,?,?,?) "
|
||||||
|
"ON CONFLICT(handle) DO UPDATE SET name=COALESCE(excluded.name, name), "
|
||||||
|
"img=COALESCE(excluded.img, img), verified=MAX(verified, excluded.verified)",
|
||||||
|
(handle, name, img, int(bool(verified)), source))
|
||||||
|
|
||||||
|
def add_event(self, user_id, track_id, kind, seconds=None):
|
||||||
|
self.execute("INSERT INTO events (user_id, track_id, kind, seconds, ts) VALUES (?,?,?,?,?)",
|
||||||
|
(user_id, track_id, kind, seconds, time.time()))
|
||||||
|
|
||||||
|
def stats(self, user_id):
|
||||||
|
row = self.query("""
|
||||||
|
SELECT (SELECT COUNT(*) FROM tracks) AS tracks,
|
||||||
|
(SELECT COUNT(*) FROM artists) AS artists,
|
||||||
|
(SELECT COUNT(*) FROM artists WHERE pages_fetched > 0) AS artists_crawled,
|
||||||
|
(SELECT COUNT(DISTINCT track_id) FROM events WHERE user_id=? AND kind='served') AS served
|
||||||
|
""", (user_id,))[0]
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
# Accounts --------------------------------------------------------------
|
||||||
|
|
||||||
|
def upsert_user(self, user):
|
||||||
|
now = time.time()
|
||||||
|
self.execute(
|
||||||
|
"INSERT INTO users (id, handle, name, img, created_at, last_seen) VALUES (?,?,?,?,?,?) "
|
||||||
|
"ON CONFLICT(id) DO UPDATE SET handle=excluded.handle, name=excluded.name, img=excluded.img, "
|
||||||
|
"last_seen=excluded.last_seen",
|
||||||
|
(user["id"], user["handle"], user.get("name"), user.get("img"), now, now))
|
||||||
|
|
||||||
|
def create_session(self, user_id):
|
||||||
|
"""Returns the raw token for the cookie; only its hash is stored."""
|
||||||
|
token = secrets.token_urlsafe(32)
|
||||||
|
now = time.time()
|
||||||
|
self.execute("INSERT INTO sessions (token_hash, user_id, created_at, expires_at) VALUES (?,?,?,?)",
|
||||||
|
(_hash(token), user_id, now, now + SESSION_DAYS * 86400))
|
||||||
|
return token
|
||||||
|
|
||||||
|
def session_user(self, token):
|
||||||
|
rows = self.query("""
|
||||||
|
SELECT u.* FROM sessions s JOIN users u ON u.id = s.user_id
|
||||||
|
WHERE s.token_hash=? AND s.expires_at > ?
|
||||||
|
""", (_hash(token), time.time()))
|
||||||
|
if not rows:
|
||||||
|
return None
|
||||||
|
user = dict(rows[0])
|
||||||
|
if (user["last_seen"] or 0) < time.time() - 60:
|
||||||
|
self.execute("UPDATE users SET last_seen=? WHERE id=?", (time.time(), user["id"]))
|
||||||
|
return user
|
||||||
|
|
||||||
|
def delete_session(self, token):
|
||||||
|
self.execute("DELETE FROM sessions WHERE token_hash=?", (_hash(token),))
|
||||||
|
|
||||||
|
def active_since(self, ts):
|
||||||
|
return self.query("SELECT COUNT(*) AS n FROM users WHERE last_seen > ?", (ts,))[0]["n"]
|
||||||
|
|
||||||
|
|
||||||
|
def _hash(token):
|
||||||
|
return hashlib.sha256(token.encode()).hexdigest()
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 676 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 182 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1007 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 513 KiB |
+244
@@ -0,0 +1,244 @@
|
|||||||
|
"""Thin client for pl8list.com.
|
||||||
|
|
||||||
|
pl8list is a Remix app, so every page loader can be read as JSON by adding
|
||||||
|
`?_data=<route id>` to the page URL. Resource routes under /api/ return JSON
|
||||||
|
directly. All clients share one throttle so we stay polite.
|
||||||
|
|
||||||
|
Public data (artist pages, related tracks, previews, free tracks) works without
|
||||||
|
an account, so the crawler runs as a guest. A user's own session is only needed
|
||||||
|
for login and for reading their personal feed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
BASE = "https://pl8list.com"
|
||||||
|
USER_AGENT = "pl8list-wrapper/0.2 (infinite-scroll client, https://pl8.narl.io)"
|
||||||
|
|
||||||
|
_throttle_lock = threading.Lock()
|
||||||
|
_throttle_last = 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def _throttle(min_interval):
|
||||||
|
global _throttle_last
|
||||||
|
with _throttle_lock:
|
||||||
|
wait = _throttle_last + min_interval - time.monotonic()
|
||||||
|
if wait > 0:
|
||||||
|
time.sleep(wait)
|
||||||
|
_throttle_last = time.monotonic()
|
||||||
|
|
||||||
|
|
||||||
|
class Client:
|
||||||
|
def __init__(self, session_cookie=None, min_interval=0.7):
|
||||||
|
self.session_cookie = session_cookie
|
||||||
|
self.min_interval = min_interval
|
||||||
|
|
||||||
|
def request(self, path, params=None, headers=None):
|
||||||
|
"""Raw request. Returns the open response; caller must close it."""
|
||||||
|
url = BASE + path
|
||||||
|
if params:
|
||||||
|
url += ("&" if "?" in url else "?") + urllib.parse.urlencode(params)
|
||||||
|
h = {"User-Agent": USER_AGENT, **(headers or {})}
|
||||||
|
if self.session_cookie:
|
||||||
|
h["Cookie"] = "_session=" + self.session_cookie
|
||||||
|
return urllib.request.urlopen(urllib.request.Request(url, headers=h), timeout=30)
|
||||||
|
|
||||||
|
def get_json(self, path, params=None, route=None):
|
||||||
|
params = dict(params or {})
|
||||||
|
if route:
|
||||||
|
params["_data"] = route
|
||||||
|
for attempt in range(4):
|
||||||
|
_throttle(self.min_interval)
|
||||||
|
try:
|
||||||
|
with self.request(path, params) as r:
|
||||||
|
body = r.read()
|
||||||
|
return json.loads(body) if body else None
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
if e.code in (429, 502, 503, 504) and attempt < 3:
|
||||||
|
time.sleep(2 ** attempt * 3)
|
||||||
|
continue
|
||||||
|
if e.code in (403, 404):
|
||||||
|
return None
|
||||||
|
raise
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Loaders -------------------------------------------------------------
|
||||||
|
|
||||||
|
def root(self):
|
||||||
|
return self.get_json("/", route="root")
|
||||||
|
|
||||||
|
def home(self):
|
||||||
|
return self.get_json("/", route="routes/_index")
|
||||||
|
|
||||||
|
# Paginated loaders ignore `page` unless `pagination=true` is also set,
|
||||||
|
# which is what the site's own useInfiniteLoading hook sends.
|
||||||
|
|
||||||
|
def feed(self, page=1):
|
||||||
|
return self.get_json("/api/feed", {"page": page, "pagination": "true"})
|
||||||
|
|
||||||
|
def artist(self, handle, page=1):
|
||||||
|
return self.get_json("/" + urllib.parse.quote(handle), {"page": page, "pagination": "true"},
|
||||||
|
route="routes/$handle")
|
||||||
|
|
||||||
|
def related(self, handle, slug):
|
||||||
|
path = "/%s/%s/related" % (urllib.parse.quote(handle), urllib.parse.quote(slug))
|
||||||
|
return self.get_json(path, route="routes/$handle_.$slug.related")
|
||||||
|
|
||||||
|
def best_selling_artists(self):
|
||||||
|
return self.get_json("/best-selling-artists", route="routes/best-selling-artists")
|
||||||
|
|
||||||
|
def activity_ticker(self):
|
||||||
|
return self.get_json("/api/activity-ticker")
|
||||||
|
|
||||||
|
def search_users(self, q, page=1):
|
||||||
|
return self.get_json("/search/users", {"q": q, "page": page, "pagination": "true"},
|
||||||
|
route="routes/search.users")
|
||||||
|
|
||||||
|
|
||||||
|
class LoginError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class LoginFlow:
|
||||||
|
"""pl8list's passwordless login: POST email, pl8list mails a 6-digit code, POST code.
|
||||||
|
|
||||||
|
The site keeps the pending login in its `_session` cookie, so one flow object
|
||||||
|
holds that cookie between the two steps. After `verify` succeeds,
|
||||||
|
`self.session` is the user's logged-in pl8list session.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, email):
|
||||||
|
self.email = email
|
||||||
|
self.session = None
|
||||||
|
|
||||||
|
def _call(self, method, path, route, fields=None):
|
||||||
|
url = BASE + path + ("&" if "?" in path else "?") + urllib.parse.urlencode({"_data": route})
|
||||||
|
headers = {"User-Agent": USER_AGENT}
|
||||||
|
if self.session:
|
||||||
|
headers["Cookie"] = "_session=" + self.session
|
||||||
|
data = None
|
||||||
|
if fields is not None:
|
||||||
|
data = urllib.parse.urlencode(fields).encode()
|
||||||
|
headers["Content-Type"] = "application/x-www-form-urlencoded"
|
||||||
|
req = urllib.request.Request(url, data=data, headers=headers, method=method)
|
||||||
|
try:
|
||||||
|
resp = urllib.request.urlopen(req, timeout=30)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
resp = e
|
||||||
|
except OSError as e:
|
||||||
|
raise LoginError("pl8list unreachable: %s" % e) from e
|
||||||
|
with resp:
|
||||||
|
for c in resp.headers.get_all("Set-Cookie") or []:
|
||||||
|
m = re.match(r"_session=([^;]+)", c)
|
||||||
|
if m:
|
||||||
|
self.session = m.group(1)
|
||||||
|
body = resp.read()
|
||||||
|
status = resp.status if hasattr(resp, "status") else resp.code
|
||||||
|
try:
|
||||||
|
payload = json.loads(body) if body else None
|
||||||
|
except ValueError:
|
||||||
|
payload = None
|
||||||
|
return status, resp.headers, payload
|
||||||
|
|
||||||
|
def _guest_id(self):
|
||||||
|
"""The guest id pl8list stores in its cookie; the login forms echo it back."""
|
||||||
|
try:
|
||||||
|
raw = urllib.parse.unquote(self.session).split(".")[0]
|
||||||
|
return json.loads(base64.b64decode(raw + "=" * (-len(raw) % 4))).get("pl8list_guest_id", "")
|
||||||
|
except (ValueError, AttributeError, TypeError):
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def _verify_path(self):
|
||||||
|
return "/verify?" + urllib.parse.urlencode({"email": self.email, "mode": "login"})
|
||||||
|
|
||||||
|
def start(self):
|
||||||
|
"""Ask pl8list to mail a login code."""
|
||||||
|
self._call("GET", "/login", "routes/login")
|
||||||
|
status, headers, payload = self._call("POST", "/login", "routes/login",
|
||||||
|
{"email": self.email, "pl8list_guest_id": self._guest_id()})
|
||||||
|
if "/verify" not in (headers.get("X-Remix-Redirect") or ""):
|
||||||
|
raise LoginError((payload or {}).get("error") or "pl8list refused the login (HTTP %d)" % status)
|
||||||
|
|
||||||
|
def resend(self):
|
||||||
|
_, _, payload = self._call("POST", self._verify_path(), "routes/verify",
|
||||||
|
{"action": "resend", "pl8list_guest_id": self._guest_id()})
|
||||||
|
if payload and payload.get("resendError"):
|
||||||
|
raise LoginError(payload["resendError"])
|
||||||
|
|
||||||
|
def verify(self, code):
|
||||||
|
"""Submit the code. Returns pl8list's currentUser for the new session."""
|
||||||
|
_, _, payload = self._call("POST", self._verify_path(), "routes/verify",
|
||||||
|
{"action": "verify", "code": code, "pl8list_guest_id": self._guest_id()})
|
||||||
|
if payload and payload.get("error"):
|
||||||
|
raise LoginError(payload["error"])
|
||||||
|
user = (Client(self.session).root() or {}).get("currentUser")
|
||||||
|
if not user:
|
||||||
|
raise LoginError("login failed, try again")
|
||||||
|
if not user.get("handle"):
|
||||||
|
raise LoginError("finish setting up your account on pl8list.com first (choose a handle)")
|
||||||
|
return user
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_item(item):
|
||||||
|
"""Flatten a loader track item ({track, hasFullAccess, saveCount, ...}) into a DB row."""
|
||||||
|
return normalize_track(item["track"], {
|
||||||
|
"full_access": item.get("hasFullAccess"),
|
||||||
|
"saves": item.get("saveCount"),
|
||||||
|
"downloads": item.get("downloadCount"),
|
||||||
|
"purchases": item.get("purchaseCount"),
|
||||||
|
"comments": item.get("commentCount"),
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_related(data):
|
||||||
|
"""The related loader returns bare tracks plus per-id maps for access and counts."""
|
||||||
|
counts = data.get("counts") or {}
|
||||||
|
access = data.get("access") or {}
|
||||||
|
rows = []
|
||||||
|
for t in data.get("tracks") or []:
|
||||||
|
k = str(t["id"])
|
||||||
|
rows.append(normalize_track(t, {
|
||||||
|
"full_access": access.get(k),
|
||||||
|
"saves": (counts.get("saveCount") or {}).get(k),
|
||||||
|
"downloads": (counts.get("downloadCount") or {}).get(k),
|
||||||
|
"purchases": (counts.get("purchaseCount") or {}).get(k),
|
||||||
|
"comments": (counts.get("commentCount") or {}).get(k),
|
||||||
|
}))
|
||||||
|
return [r for r in rows if r]
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_track(t, extra):
|
||||||
|
user = t.get("user") or {}
|
||||||
|
if not user.get("handle") or t.get("isDeleted") or not t.get("isPublic", True):
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"id": t["id"],
|
||||||
|
"handle": user["handle"],
|
||||||
|
"artist_name": user.get("name") or user["handle"],
|
||||||
|
"artist_img": user.get("img"),
|
||||||
|
"artist_verified": int(bool(user.get("isVerified"))),
|
||||||
|
"slug": t["slug"],
|
||||||
|
"title": t.get("title") or t.get("filename") or t["slug"],
|
||||||
|
"description": t.get("description"),
|
||||||
|
"bpm": t.get("bpm"),
|
||||||
|
"key": t.get("key"),
|
||||||
|
"duration": t.get("duration"),
|
||||||
|
"preview_duration": t.get("previewDuration"),
|
||||||
|
"price": t.get("price") or 0,
|
||||||
|
"created_at": t.get("createdAt"),
|
||||||
|
"upload_id": t.get("uploadId"),
|
||||||
|
"preview_upload_id": t.get("previewUploadId"),
|
||||||
|
"waveform": json.dumps(t.get("waveform")) if t.get("waveform") else None,
|
||||||
|
"preview_waveform": json.dumps(t.get("previewWaveform")) if t.get("previewWaveform") else None,
|
||||||
|
"full_access": int(bool(extra.get("full_access"))),
|
||||||
|
"saves": extra.get("saves") or 0,
|
||||||
|
"downloads": extra.get("downloads") or 0,
|
||||||
|
"purchases": extra.get("purchases") or 0,
|
||||||
|
"comments": extra.get("comments") or 0,
|
||||||
|
}
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
"""Pick the next batch of tracks for the scroll feed.
|
||||||
|
|
||||||
|
Each unserved playable track gets a score:
|
||||||
|
|
||||||
|
popularity log-scaled saves/downloads/purchases/comments, normalised to 0..1
|
||||||
|
freshness exp(-age / 21 days)
|
||||||
|
artist learned from your events on that artist's tracks (likes, full listens, skips)
|
||||||
|
tempo / key closeness to the BPM and key of tracks you liked or finished
|
||||||
|
related track came from the "related" page of something you liked
|
||||||
|
exploration Gumbel noise, so the feed never freezes into one niche
|
||||||
|
|
||||||
|
The batch is then picked greedily with a penalty for repeating an artist.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import datetime
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import random
|
||||||
|
from collections import defaultdict
|
||||||
|
|
||||||
|
WEIGHTS = {
|
||||||
|
"popularity": 1.2,
|
||||||
|
"freshness": 0.8,
|
||||||
|
"artist": 1.5,
|
||||||
|
"tempo": 1.0,
|
||||||
|
"key": 0.3,
|
||||||
|
"related": 1.0,
|
||||||
|
"unknown_artist": 0.2,
|
||||||
|
"exploration": 0.6,
|
||||||
|
}
|
||||||
|
# Per-event contributions to artist affinity.
|
||||||
|
EVENT_SIGNAL = {"like": 3.0, "unlike": -3.0, "complete": 1.5, "skip": -1.0}
|
||||||
|
REPEAT_ARTIST_PENALTY = 1.5
|
||||||
|
RECENT_ARTIST_WINDOW = 6
|
||||||
|
|
||||||
|
|
||||||
|
def _age_days(iso):
|
||||||
|
if not iso:
|
||||||
|
return 365.0
|
||||||
|
try:
|
||||||
|
t = datetime.datetime.fromisoformat(iso.replace("Z", "+00:00"))
|
||||||
|
except ValueError:
|
||||||
|
return 365.0
|
||||||
|
return max(0.0, (datetime.datetime.now(datetime.timezone.utc) - t).total_seconds() / 86400)
|
||||||
|
|
||||||
|
|
||||||
|
def _tempo_match(bpm, liked_bpms):
|
||||||
|
"""Gaussian closeness to any liked tempo, also counting half/double time as a match."""
|
||||||
|
if not bpm or not liked_bpms:
|
||||||
|
return 0.0
|
||||||
|
best = 0.0
|
||||||
|
for b in liked_bpms:
|
||||||
|
d = min(abs(bpm - b), abs(bpm * 2 - b), abs(bpm / 2 - b))
|
||||||
|
best = max(best, math.exp(-((d / 6.0) ** 2)))
|
||||||
|
return best
|
||||||
|
|
||||||
|
|
||||||
|
def taste(db, user_id):
|
||||||
|
"""Summarise a user's listening history into artist affinity, liked tempos/keys and liked track ids."""
|
||||||
|
rows = db.query("""
|
||||||
|
SELECT e.track_id, e.kind, e.seconds, t.handle, t.bpm, t.key
|
||||||
|
FROM events e JOIN tracks t ON t.id = e.track_id
|
||||||
|
WHERE e.user_id = ? AND e.kind IN ('like', 'unlike', 'complete', 'skip')
|
||||||
|
ORDER BY e.ts
|
||||||
|
""", (user_id,))
|
||||||
|
artist = defaultdict(float)
|
||||||
|
liked = {}
|
||||||
|
completed = {}
|
||||||
|
for r in rows:
|
||||||
|
artist[r["handle"]] += EVENT_SIGNAL[r["kind"]]
|
||||||
|
if r["kind"] == "like":
|
||||||
|
liked[r["track_id"]] = r
|
||||||
|
elif r["kind"] == "unlike":
|
||||||
|
liked.pop(r["track_id"], None)
|
||||||
|
elif r["kind"] == "complete":
|
||||||
|
completed[r["track_id"]] = r
|
||||||
|
positive = {**completed, **liked}
|
||||||
|
return {
|
||||||
|
"artist": {h: math.tanh(v / 4.0) for h, v in artist.items()},
|
||||||
|
"bpms": [r["bpm"] for r in positive.values() if r["bpm"]][-50:],
|
||||||
|
"keys": {r["key"] for r in positive.values() if r["key"]},
|
||||||
|
"positive_ids": set(positive),
|
||||||
|
"liked_ids": set(liked),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def pick(db, user_id, n=10, free_only=False, bpm_min=None, bpm_max=None):
|
||||||
|
where = ["(t.full_access=1 OR t.preview_upload_id IS NOT NULL)",
|
||||||
|
"NOT EXISTS (SELECT 1 FROM events e WHERE e.user_id=? AND e.kind='served' AND e.track_id=t.id)"]
|
||||||
|
args = [user_id]
|
||||||
|
if free_only:
|
||||||
|
where.append("t.price = 0")
|
||||||
|
if bpm_min is not None:
|
||||||
|
where.append("t.bpm >= ?")
|
||||||
|
args.append(bpm_min)
|
||||||
|
if bpm_max is not None:
|
||||||
|
where.append("t.bpm <= ?")
|
||||||
|
args.append(bpm_max)
|
||||||
|
cands = db.query("SELECT * FROM tracks t WHERE " + " AND ".join(where), args)
|
||||||
|
if not cands:
|
||||||
|
return []
|
||||||
|
|
||||||
|
t = taste(db, user_id)
|
||||||
|
recent = [r["handle"] for r in db.query("""
|
||||||
|
SELECT t.handle FROM events e JOIN tracks t ON t.id=e.track_id
|
||||||
|
WHERE e.user_id=? AND e.kind='served' ORDER BY e.ts DESC LIMIT ?
|
||||||
|
""", (user_id, RECENT_ARTIST_WINDOW))]
|
||||||
|
|
||||||
|
def pop_raw(r):
|
||||||
|
return (math.log1p(r["saves"]) + 0.7 * math.log1p(r["downloads"])
|
||||||
|
+ 1.0 * math.log1p(r["purchases"]) + 0.5 * math.log1p(r["comments"]))
|
||||||
|
|
||||||
|
max_pop = max(pop_raw(r) for r in cands) or 1.0
|
||||||
|
w = WEIGHTS
|
||||||
|
scored = []
|
||||||
|
for r in cands:
|
||||||
|
aff = t["artist"].get(r["handle"])
|
||||||
|
s = (w["popularity"] * pop_raw(r) / max_pop
|
||||||
|
+ w["freshness"] * math.exp(-_age_days(r["created_at"]) / 21.0)
|
||||||
|
+ w["artist"] * (aff or 0.0)
|
||||||
|
+ w["tempo"] * _tempo_match(r["bpm"], t["bpms"])
|
||||||
|
+ w["key"] * (1.0 if r["key"] in t["keys"] else 0.0)
|
||||||
|
+ w["related"] * (1.0 if r["related_of"] in t["positive_ids"] else 0.0)
|
||||||
|
+ w["unknown_artist"] * (1.0 if aff is None else 0.0)
|
||||||
|
+ w["exploration"] * -math.log(-math.log(random.random() or 1e-12)))
|
||||||
|
scored.append([s, r])
|
||||||
|
|
||||||
|
out = []
|
||||||
|
seen_artists = list(recent)
|
||||||
|
scored.sort(key=lambda x: -x[0])
|
||||||
|
pool = scored[: max(n * 8, 60)]
|
||||||
|
while pool and len(out) < n:
|
||||||
|
best_i = max(range(len(pool)),
|
||||||
|
key=lambda i: pool[i][0] - REPEAT_ARTIST_PENALTY * seen_artists.count(pool[i][1]["handle"]))
|
||||||
|
s, r = pool.pop(best_i)
|
||||||
|
out.append(serialize(r, s))
|
||||||
|
seen_artists.append(r["handle"])
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def serialize(r, score=None):
|
||||||
|
preview = not r["full_access"]
|
||||||
|
wave = r["preview_waveform"] if preview and r["preview_waveform"] else r["waveform"]
|
||||||
|
return {
|
||||||
|
"id": r["id"],
|
||||||
|
"handle": r["handle"],
|
||||||
|
"slug": r["slug"],
|
||||||
|
"title": r["title"],
|
||||||
|
"description": r["description"],
|
||||||
|
"artist_name": r["artist_name"],
|
||||||
|
"artist_img": r["artist_img"],
|
||||||
|
"artist_verified": bool(r["artist_verified"]),
|
||||||
|
"bpm": r["bpm"],
|
||||||
|
"key": r["key"],
|
||||||
|
"duration": r["preview_duration"] if preview and r["preview_duration"] else r["duration"],
|
||||||
|
"price": r["price"],
|
||||||
|
"created_at": r["created_at"],
|
||||||
|
"saves": r["saves"],
|
||||||
|
"downloads": r["downloads"],
|
||||||
|
"purchases": r["purchases"],
|
||||||
|
"comments": r["comments"],
|
||||||
|
"preview": preview,
|
||||||
|
"upload_id": r["preview_upload_id"] if preview else r["upload_id"],
|
||||||
|
"waveform": json.loads(wave) if wave else None,
|
||||||
|
"score": round(score, 3) if score is not None else None,
|
||||||
|
}
|
||||||
@@ -0,0 +1,412 @@
|
|||||||
|
"""Infinite-scroll front end for pl8list, with sign-in through pl8list accounts.
|
||||||
|
|
||||||
|
python3 server.py # then open http://127.0.0.1:8790
|
||||||
|
|
||||||
|
Users sign in with their pl8list email: pl8list mails them a code, they type it
|
||||||
|
here, and this server checks it with pl8list. Afterwards they get a session
|
||||||
|
cookie for this app only. Their pl8list session is used once to read their
|
||||||
|
personal feed and is never written to disk.
|
||||||
|
|
||||||
|
Configuration comes from the environment (or .env):
|
||||||
|
|
||||||
|
HOST bind address (default 127.0.0.1)
|
||||||
|
PORT port (default 8790)
|
||||||
|
PL8_DB SQLite path (default ./pl8list.db)
|
||||||
|
TRUST_PROXY 1 when behind a reverse proxy: trust X-Forwarded-For/-Proto
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import collections
|
||||||
|
import http.cookies
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import secrets
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import ranker
|
||||||
|
from crawler import Crawler
|
||||||
|
from db import DB, SESSION_DAYS
|
||||||
|
from pl8list import Client, LoginError, LoginFlow
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parent
|
||||||
|
STATIC = ROOT / "static"
|
||||||
|
EVENT_KINDS = {"play", "like", "unlike", "skip", "complete"}
|
||||||
|
# Only HLS playlists and segments are proxied, nothing else on pl8list.
|
||||||
|
PROXY_RE = re.compile(r"^/api/[^/]+/[^/]+/hls/(playlist\.m3u8|segment/\d+)$")
|
||||||
|
EMAIL_RE = re.compile(r"^[^@\s]{1,64}@[^@\s]+\.[^@\s]+$")
|
||||||
|
CONTENT_TYPES = {
|
||||||
|
".html": "text/html; charset=utf-8", ".js": "text/javascript", ".css": "text/css",
|
||||||
|
".png": "image/png", ".svg": "image/svg+xml",
|
||||||
|
}
|
||||||
|
SESSION_COOKIE = "pl8inf"
|
||||||
|
LOGIN_COOKIE = "pl8inf_login"
|
||||||
|
LOGIN_TTL = 900
|
||||||
|
MAX_BODY = 4096
|
||||||
|
CSP = "; ".join([
|
||||||
|
"default-src 'self'",
|
||||||
|
"script-src 'self' https://cdn.jsdelivr.net",
|
||||||
|
"style-src 'self'",
|
||||||
|
"img-src 'self' data: https://pl8list.com https://files.pl8list.com",
|
||||||
|
"media-src 'self' blob:",
|
||||||
|
"worker-src 'self' blob:",
|
||||||
|
"connect-src 'self'",
|
||||||
|
"base-uri 'none'",
|
||||||
|
"form-action 'self'",
|
||||||
|
"frame-ancestors 'none'",
|
||||||
|
])
|
||||||
|
|
||||||
|
log = logging.getLogger("server")
|
||||||
|
|
||||||
|
|
||||||
|
def load_env(path):
|
||||||
|
if not path.exists():
|
||||||
|
return
|
||||||
|
for line in path.read_text().splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if line and not line.startswith("#") and "=" in line:
|
||||||
|
k, v = line.split("=", 1)
|
||||||
|
os.environ.setdefault(k.strip(), v.strip().strip("'\""))
|
||||||
|
|
||||||
|
|
||||||
|
class RateLimiter:
|
||||||
|
"""At most `limit` hits per key within `window` seconds."""
|
||||||
|
|
||||||
|
def __init__(self, limit, window):
|
||||||
|
self.limit, self.window = limit, window
|
||||||
|
self.hits = collections.defaultdict(collections.deque)
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
|
||||||
|
def allow(self, key):
|
||||||
|
now = time.time()
|
||||||
|
with self.lock:
|
||||||
|
q = self.hits[key]
|
||||||
|
while q and q[0] < now - self.window:
|
||||||
|
q.popleft()
|
||||||
|
if len(q) >= self.limit:
|
||||||
|
return False
|
||||||
|
q.append(now)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
class PendingLogins:
|
||||||
|
"""Logins waiting for their emailed code, keyed by a random id in a short-lived cookie."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.flows = {}
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
|
||||||
|
def add(self, flow):
|
||||||
|
flow_id = secrets.token_urlsafe(24)
|
||||||
|
with self.lock:
|
||||||
|
now = time.time()
|
||||||
|
for k in [k for k, f in self.flows.items() if f.created < now - LOGIN_TTL]:
|
||||||
|
del self.flows[k]
|
||||||
|
flow.created, flow.attempts = now, 0
|
||||||
|
self.flows[flow_id] = flow
|
||||||
|
return flow_id
|
||||||
|
|
||||||
|
def get(self, flow_id):
|
||||||
|
with self.lock:
|
||||||
|
flow = self.flows.get(flow_id or "")
|
||||||
|
if flow and flow.created < time.time() - LOGIN_TTL:
|
||||||
|
del self.flows[flow_id]
|
||||||
|
return None
|
||||||
|
return flow
|
||||||
|
|
||||||
|
def drop(self, flow_id):
|
||||||
|
with self.lock:
|
||||||
|
self.flows.pop(flow_id, None)
|
||||||
|
|
||||||
|
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
server_version = "pl8inf"
|
||||||
|
sys_version = ""
|
||||||
|
db: DB
|
||||||
|
client: Client
|
||||||
|
crawler: Crawler
|
||||||
|
trust_proxy = False
|
||||||
|
logins = PendingLogins()
|
||||||
|
login_ip_limit = RateLimiter(5, 900)
|
||||||
|
login_email_limit = RateLimiter(3, 900)
|
||||||
|
|
||||||
|
def log_message(self, fmt, *args):
|
||||||
|
log.debug(fmt, *args)
|
||||||
|
|
||||||
|
def end_headers(self):
|
||||||
|
self.send_header("Content-Security-Policy", CSP)
|
||||||
|
self.send_header("X-Content-Type-Options", "nosniff")
|
||||||
|
self.send_header("Referrer-Policy", "strict-origin-when-cross-origin")
|
||||||
|
self.send_header("X-Frame-Options", "DENY")
|
||||||
|
super().end_headers()
|
||||||
|
|
||||||
|
# Request helpers ------------------------------------------------------
|
||||||
|
|
||||||
|
def client_ip(self):
|
||||||
|
if self.trust_proxy and self.headers.get("X-Forwarded-For"):
|
||||||
|
return self.headers["X-Forwarded-For"].split(",")[0].strip()
|
||||||
|
return self.client_address[0]
|
||||||
|
|
||||||
|
def is_https(self):
|
||||||
|
return self.trust_proxy and self.headers.get("X-Forwarded-Proto") == "https"
|
||||||
|
|
||||||
|
def cookie(self, name):
|
||||||
|
jar = http.cookies.SimpleCookie()
|
||||||
|
try:
|
||||||
|
jar.load(self.headers.get("Cookie") or "")
|
||||||
|
except http.cookies.CookieError:
|
||||||
|
return None
|
||||||
|
return jar[name].value if name in jar else None
|
||||||
|
|
||||||
|
def set_cookie(self, name, value, max_age):
|
||||||
|
parts = ["%s=%s" % (name, value), "Path=/", "HttpOnly", "SameSite=Lax", "Max-Age=%d" % max_age]
|
||||||
|
if self.is_https():
|
||||||
|
parts.append("Secure")
|
||||||
|
self.send_header("Set-Cookie", "; ".join(parts))
|
||||||
|
|
||||||
|
def current_user(self):
|
||||||
|
token = self.cookie(SESSION_COOKIE)
|
||||||
|
return self.db.session_user(token) if token else None
|
||||||
|
|
||||||
|
def read_json(self):
|
||||||
|
length = int(self.headers.get("Content-Length") or 0)
|
||||||
|
if length > MAX_BODY:
|
||||||
|
raise ValueError("body too large")
|
||||||
|
data = json.loads(self.rfile.read(length) or b"{}")
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise ValueError("expected an object")
|
||||||
|
return data
|
||||||
|
|
||||||
|
def same_origin(self):
|
||||||
|
"""Reject cross-site POSTs. Browsers always send Origin on POST."""
|
||||||
|
origin = self.headers.get("Origin")
|
||||||
|
return not origin or urllib.parse.urlsplit(origin).netloc == self.headers.get("Host")
|
||||||
|
|
||||||
|
def send_json(self, obj, status=200, cookies=()):
|
||||||
|
body = json.dumps(obj).encode()
|
||||||
|
self.send_response(status)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.send_header("Cache-Control", "no-store")
|
||||||
|
for c in cookies:
|
||||||
|
self.set_cookie(*c)
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def send_static(self, name):
|
||||||
|
path = (STATIC / name).resolve()
|
||||||
|
if STATIC not in path.parents or not path.is_file():
|
||||||
|
return self.send_error(404)
|
||||||
|
body = path.read_bytes()
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", CONTENT_TYPES.get(path.suffix, "application/octet-stream"))
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.send_header("Cache-Control", "no-cache")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
# Routing --------------------------------------------------------------
|
||||||
|
|
||||||
|
def do_GET(self):
|
||||||
|
url = urllib.parse.urlsplit(self.path)
|
||||||
|
if url.path == "/":
|
||||||
|
return self.send_static("index.html")
|
||||||
|
if url.path.startswith("/static/"):
|
||||||
|
return self.send_static(url.path[len("/static/"):])
|
||||||
|
if url.path == "/healthz":
|
||||||
|
return self.send_json({"ok": True})
|
||||||
|
user = self.current_user()
|
||||||
|
if url.path == "/local/me":
|
||||||
|
if not user:
|
||||||
|
return self.send_json({"error": "not signed in"}, 401)
|
||||||
|
return self.send_json({"user": {k: user[k] for k in ("handle", "name", "img")}})
|
||||||
|
if not user:
|
||||||
|
return self.send_error(401)
|
||||||
|
if url.path == "/local/feed":
|
||||||
|
return self.feed(user, urllib.parse.parse_qs(url.query))
|
||||||
|
if url.path == "/local/likes":
|
||||||
|
return self.likes(user)
|
||||||
|
if PROXY_RE.match(url.path):
|
||||||
|
return self.proxy(url)
|
||||||
|
self.send_error(404)
|
||||||
|
|
||||||
|
def do_POST(self):
|
||||||
|
if not self.same_origin():
|
||||||
|
return self.send_error(403)
|
||||||
|
try:
|
||||||
|
data = self.read_json()
|
||||||
|
except ValueError:
|
||||||
|
return self.send_error(400)
|
||||||
|
routes = {
|
||||||
|
"/auth/start": self.auth_start,
|
||||||
|
"/auth/resend": self.auth_resend,
|
||||||
|
"/auth/verify": self.auth_verify,
|
||||||
|
"/auth/logout": self.auth_logout,
|
||||||
|
"/local/event": self.event,
|
||||||
|
}
|
||||||
|
handler = routes.get(self.path)
|
||||||
|
if not handler:
|
||||||
|
return self.send_error(404)
|
||||||
|
handler(data)
|
||||||
|
|
||||||
|
# Auth -----------------------------------------------------------------
|
||||||
|
|
||||||
|
def auth_start(self, data):
|
||||||
|
email = str(data.get("email") or "").strip().lower()
|
||||||
|
if len(email) > 254 or not EMAIL_RE.match(email):
|
||||||
|
return self.send_json({"error": "enter a valid email"}, 400)
|
||||||
|
if not self.login_ip_limit.allow(self.client_ip()) or not self.login_email_limit.allow(email):
|
||||||
|
return self.send_json({"error": "too many attempts, wait a few minutes"}, 429)
|
||||||
|
flow = LoginFlow(email)
|
||||||
|
try:
|
||||||
|
flow.start()
|
||||||
|
except LoginError as e:
|
||||||
|
return self.send_json({"error": str(e)}, 400)
|
||||||
|
flow_id = self.logins.add(flow)
|
||||||
|
self.send_json({"ok": True, "email": email}, cookies=[(LOGIN_COOKIE, flow_id, LOGIN_TTL)])
|
||||||
|
|
||||||
|
def auth_resend(self, data):
|
||||||
|
flow = self.logins.get(self.cookie(LOGIN_COOKIE))
|
||||||
|
if not flow:
|
||||||
|
return self.send_json({"error": "login expired, start again"}, 400)
|
||||||
|
if not self.login_ip_limit.allow(self.client_ip()):
|
||||||
|
return self.send_json({"error": "too many attempts, wait a few minutes"}, 429)
|
||||||
|
try:
|
||||||
|
flow.resend()
|
||||||
|
except LoginError as e:
|
||||||
|
return self.send_json({"error": str(e)}, 400)
|
||||||
|
self.send_json({"ok": True})
|
||||||
|
|
||||||
|
def auth_verify(self, data):
|
||||||
|
flow_id = self.cookie(LOGIN_COOKIE)
|
||||||
|
flow = self.logins.get(flow_id)
|
||||||
|
if not flow:
|
||||||
|
return self.send_json({"error": "login expired, start again"}, 400)
|
||||||
|
code = str(data.get("code") or "").strip()
|
||||||
|
if not re.fullmatch(r"\d{6}", code):
|
||||||
|
return self.send_json({"error": "the code has 6 digits"}, 400)
|
||||||
|
flow.attempts += 1
|
||||||
|
if flow.attempts > 5:
|
||||||
|
self.logins.drop(flow_id)
|
||||||
|
return self.send_json({"error": "too many wrong codes, start again"}, 429)
|
||||||
|
try:
|
||||||
|
pl8_user = flow.verify(code)
|
||||||
|
except LoginError as e:
|
||||||
|
return self.send_json({"error": str(e)}, 400)
|
||||||
|
self.logins.drop(flow_id)
|
||||||
|
self.db.upsert_user(pl8_user)
|
||||||
|
self.crawler.on_login(pl8_user["id"], flow.session)
|
||||||
|
token = self.db.create_session(pl8_user["id"])
|
||||||
|
log.info("signed in: %s", pl8_user["handle"])
|
||||||
|
self.send_json({"user": {"handle": pl8_user["handle"], "name": pl8_user.get("name")}},
|
||||||
|
cookies=[(SESSION_COOKIE, token, SESSION_DAYS * 86400), (LOGIN_COOKIE, "", 0)])
|
||||||
|
|
||||||
|
def auth_logout(self, data):
|
||||||
|
token = self.cookie(SESSION_COOKIE)
|
||||||
|
if token:
|
||||||
|
self.db.delete_session(token)
|
||||||
|
self.send_json({"ok": True}, cookies=[(SESSION_COOKIE, "", 0)])
|
||||||
|
|
||||||
|
# App ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def event(self, data):
|
||||||
|
user = self.current_user()
|
||||||
|
if not user:
|
||||||
|
return self.send_error(401)
|
||||||
|
try:
|
||||||
|
track_id, kind = int(data["track_id"]), data["kind"]
|
||||||
|
seconds = float(data["seconds"]) if data.get("seconds") is not None else None
|
||||||
|
except (ValueError, KeyError, TypeError):
|
||||||
|
return self.send_error(400)
|
||||||
|
if kind not in EVENT_KINDS:
|
||||||
|
return self.send_error(400)
|
||||||
|
self.db.add_event(user["id"], track_id, kind, seconds)
|
||||||
|
if kind in ("like", "complete"):
|
||||||
|
self.crawler.on_positive(track_id)
|
||||||
|
self.send_json({"ok": True})
|
||||||
|
|
||||||
|
def feed(self, user, q):
|
||||||
|
def num(name):
|
||||||
|
try:
|
||||||
|
return float(q[name][0]) if q.get(name, [""])[0] else None
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
n = max(1, min(int(num("n") or 10), 30))
|
||||||
|
kwargs = dict(free_only=q.get("free", [""])[0] == "1", bpm_min=num("bpm_min"), bpm_max=num("bpm_max"))
|
||||||
|
# On a cold start the crawler needs a few seconds before there is anything to serve.
|
||||||
|
deadline = time.time() + 20
|
||||||
|
tracks = ranker.pick(self.db, user["id"], n, **kwargs)
|
||||||
|
while not tracks and time.time() < deadline:
|
||||||
|
time.sleep(1.5)
|
||||||
|
tracks = ranker.pick(self.db, user["id"], n, **kwargs)
|
||||||
|
for t in tracks:
|
||||||
|
self.db.add_event(user["id"], t["id"], "served")
|
||||||
|
stats = self.db.stats(user["id"])
|
||||||
|
stats["liked"] = len(ranker.taste(self.db, user["id"])["liked_ids"])
|
||||||
|
self.send_json({"tracks": tracks, "stats": stats})
|
||||||
|
|
||||||
|
def likes(self, user):
|
||||||
|
liked = ranker.taste(self.db, user["id"])["liked_ids"]
|
||||||
|
rows = self.db.query("SELECT * FROM tracks WHERE id IN (%s)" % ",".join("?" * len(liked)), list(liked)) \
|
||||||
|
if liked else []
|
||||||
|
self.send_json({"tracks": [ranker.serialize(r) for r in rows]})
|
||||||
|
|
||||||
|
def proxy(self, url):
|
||||||
|
# Previews and free tracks stream fine as a guest. Segments redirect to
|
||||||
|
# signed storage URLs; urllib follows that server-side.
|
||||||
|
try:
|
||||||
|
upstream = self.client.request(url.path, urllib.parse.parse_qsl(url.query))
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
return self.send_error(e.code)
|
||||||
|
except OSError:
|
||||||
|
return self.send_error(502)
|
||||||
|
with upstream:
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", upstream.headers.get("Content-Type", "application/octet-stream"))
|
||||||
|
if upstream.headers.get("Content-Length"):
|
||||||
|
self.send_header("Content-Length", upstream.headers["Content-Length"])
|
||||||
|
self.send_header("Cache-Control", "private, max-age=600")
|
||||||
|
self.end_headers()
|
||||||
|
try:
|
||||||
|
while chunk := upstream.read(64 * 1024):
|
||||||
|
self.wfile.write(chunk)
|
||||||
|
except (BrokenPipeError, ConnectionResetError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
load_env(ROOT / ".env")
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
ap.add_argument("--host", default=os.environ.get("HOST", "127.0.0.1"))
|
||||||
|
ap.add_argument("--port", type=int, default=int(os.environ.get("PORT", "8790")))
|
||||||
|
ap.add_argument("--db", default=os.environ.get("PL8_DB", str(ROOT / "pl8list.db")))
|
||||||
|
ap.add_argument("-v", "--verbose", action="store_true")
|
||||||
|
args = ap.parse_args()
|
||||||
|
logging.basicConfig(level=logging.DEBUG if args.verbose else logging.INFO,
|
||||||
|
format="%(asctime)s %(name)s %(message)s", datefmt="%H:%M:%S")
|
||||||
|
|
||||||
|
db = DB(args.db)
|
||||||
|
client = Client()
|
||||||
|
crawler = Crawler(client, db)
|
||||||
|
crawler.start()
|
||||||
|
|
||||||
|
Handler.db, Handler.client, Handler.crawler = db, client, crawler
|
||||||
|
Handler.trust_proxy = os.environ.get("TRUST_PROXY") == "1"
|
||||||
|
server = ThreadingHTTPServer((args.host, args.port), Handler)
|
||||||
|
server.daemon_threads = True
|
||||||
|
log.info("listening on http://%s:%d", args.host, args.port)
|
||||||
|
try:
|
||||||
|
server.serve_forever()
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
+381
@@ -0,0 +1,381 @@
|
|||||||
|
"use strict";
|
||||||
|
|
||||||
|
const PL8 = "https://pl8list.com";
|
||||||
|
const PREFETCH_AHEAD = 4; // fetch more when this close to the end
|
||||||
|
const SKIP_SECONDS = 8; // leaving earlier than this counts as a skip
|
||||||
|
const COMPLETE_RATIO = 0.85;
|
||||||
|
|
||||||
|
const feed = document.getElementById("feed");
|
||||||
|
const audio = document.getElementById("audio");
|
||||||
|
const tpl = document.getElementById("card-tpl");
|
||||||
|
const statsEl = document.getElementById("stats");
|
||||||
|
const filtersForm = document.getElementById("filters");
|
||||||
|
|
||||||
|
const cards = []; // {track, el, canvas, liked}
|
||||||
|
let active = -1;
|
||||||
|
let loading = false;
|
||||||
|
let started = false;
|
||||||
|
let hls = null;
|
||||||
|
let listened = 0; // seconds actually heard on the active card
|
||||||
|
let lastTime = 0;
|
||||||
|
let reported = false; // final event already sent for the active card
|
||||||
|
let filters = {};
|
||||||
|
|
||||||
|
// Server ----------------------------------------------------------------
|
||||||
|
|
||||||
|
function sendEvent(track, kind, seconds) {
|
||||||
|
const body = JSON.stringify({ track_id: track.id, kind, seconds });
|
||||||
|
if (!navigator.sendBeacon("/local/event", new Blob([body], { type: "application/json" }))) {
|
||||||
|
fetch("/local/event", { method: "POST", body, headers: { "Content-Type": "application/json" }, keepalive: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadMore() {
|
||||||
|
if (loading) return;
|
||||||
|
loading = true;
|
||||||
|
const status = document.createElement("div");
|
||||||
|
status.className = "status";
|
||||||
|
status.textContent = cards.length ? "loading…" : "crawling pl8list, first tracks in a few seconds…";
|
||||||
|
feed.append(status);
|
||||||
|
try {
|
||||||
|
const params = new URLSearchParams({ n: "10", ...filters });
|
||||||
|
const res = await fetch("/local/feed?" + params);
|
||||||
|
if (res.status === 401) {
|
||||||
|
status.remove();
|
||||||
|
loading = false;
|
||||||
|
showLogin();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const data = await res.json();
|
||||||
|
showStats(data.stats);
|
||||||
|
if (!data.tracks.length) {
|
||||||
|
status.textContent = "nothing new right now, crawler is still digging. scroll again in a bit.";
|
||||||
|
setTimeout(() => { status.remove(); loading = false; }, 8000);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
status.remove();
|
||||||
|
data.tracks.forEach(addCard);
|
||||||
|
} catch (err) {
|
||||||
|
status.textContent = "failed to load: " + err;
|
||||||
|
setTimeout(() => status.remove(), 5000);
|
||||||
|
}
|
||||||
|
loading = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
let stats = null;
|
||||||
|
|
||||||
|
function showStats(s) {
|
||||||
|
if (s) stats = s;
|
||||||
|
if (stats) statsEl.textContent = `${stats.tracks} tracks · ${stats.artists_crawled}/${stats.artists} artists crawled · ${stats.liked} liked`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Uploads are often titled after their file name.
|
||||||
|
const cleanTitle = (t) => t.replace(/\.(wav|mp3|flac|aiff?|ogg|m4a)$/i, "");
|
||||||
|
|
||||||
|
// Cards -----------------------------------------------------------------
|
||||||
|
|
||||||
|
const fmtTime = (s) => `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, "0")}`;
|
||||||
|
|
||||||
|
function addCard(track) {
|
||||||
|
const el = tpl.content.firstElementChild.cloneNode(true);
|
||||||
|
const img = track.artist_img ? `url("${PL8}/img/${track.artist_img}")` : "";
|
||||||
|
el.querySelector(".bg").style.backgroundImage = img;
|
||||||
|
const art = el.querySelector(".art");
|
||||||
|
if (img) art.style.backgroundImage = img;
|
||||||
|
else art.textContent = (track.artist_name || "?")[0].toUpperCase();
|
||||||
|
|
||||||
|
el.querySelector(".title").textContent = cleanTitle(track.title);
|
||||||
|
const artist = el.querySelector(".artist");
|
||||||
|
artist.textContent = track.artist_name + (track.artist_verified ? " ✓" : "");
|
||||||
|
artist.href = `${PL8}/${track.handle}`;
|
||||||
|
el.querySelector(".open").href = `${PL8}/${track.handle}/${track.slug}`;
|
||||||
|
el.querySelector(".desc").textContent = track.description || "";
|
||||||
|
el.querySelector(".dur").textContent = track.duration ? fmtTime(track.duration) : "";
|
||||||
|
|
||||||
|
const chips = el.querySelector(".chips");
|
||||||
|
const chip = (text, cls) => {
|
||||||
|
const s = document.createElement("span");
|
||||||
|
s.textContent = text;
|
||||||
|
if (cls) s.className = cls;
|
||||||
|
chips.append(s);
|
||||||
|
};
|
||||||
|
if (track.bpm) chip(`${Math.round(track.bpm)} bpm`);
|
||||||
|
if (track.key) chip(track.key);
|
||||||
|
chip(track.price ? `$${track.price}` : "free", track.price ? "" : "free");
|
||||||
|
if (track.preview) chip("preview", "preview");
|
||||||
|
|
||||||
|
el.querySelector(".counts").textContent =
|
||||||
|
`${track.saves} saves · ${track.downloads} downloads · ${track.comments} comments` +
|
||||||
|
(track.purchases ? ` · ${track.purchases} sold` : "");
|
||||||
|
|
||||||
|
const card = { track, el, canvas: el.querySelector(".wave"), liked: false };
|
||||||
|
const idx = cards.push(card) - 1;
|
||||||
|
|
||||||
|
el.querySelector(".like").addEventListener("click", () => toggleLike(idx));
|
||||||
|
el.querySelector(".next").addEventListener("click", () => go(idx + 1));
|
||||||
|
card.canvas.addEventListener("click", (e) => {
|
||||||
|
if (idx !== active || !audio.duration) return;
|
||||||
|
const r = card.canvas.getBoundingClientRect();
|
||||||
|
audio.currentTime = ((e.clientX - r.left) / r.width) * audio.duration;
|
||||||
|
});
|
||||||
|
|
||||||
|
feed.append(el);
|
||||||
|
observer.observe(el);
|
||||||
|
drawWave(card, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
function drawWave(card, progress) {
|
||||||
|
const { canvas, track } = card;
|
||||||
|
const w = canvas.clientWidth, h = canvas.clientHeight;
|
||||||
|
if (!w) return;
|
||||||
|
const dpr = window.devicePixelRatio || 1;
|
||||||
|
if (canvas.width !== w * dpr) { canvas.width = w * dpr; canvas.height = h * dpr; }
|
||||||
|
const ctx = canvas.getContext("2d");
|
||||||
|
ctx.setTransform(dpr, 0, 0, dpr, 0, 0);
|
||||||
|
ctx.clearRect(0, 0, w, h);
|
||||||
|
const data = track.waveform || [];
|
||||||
|
const bars = Math.max(1, Math.floor(w / 3));
|
||||||
|
const max = Math.max(1, ...data);
|
||||||
|
for (let i = 0; i < bars; i++) {
|
||||||
|
const v = data.length ? data[Math.floor((i / bars) * data.length)] / max : 0.15;
|
||||||
|
const bh = Math.max(2, v * h);
|
||||||
|
ctx.fillStyle = i / bars < progress ? "#ff4d6d" : "rgba(255,255,255,0.35)";
|
||||||
|
ctx.fillRect(i * 3, (h - bh) / 2, 2, bh);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Playback --------------------------------------------------------------
|
||||||
|
|
||||||
|
function streamUrl(t) {
|
||||||
|
const p = new URLSearchParams();
|
||||||
|
if (t.preview) p.set("preview", "true");
|
||||||
|
if (t.upload_id) p.set("v", t.upload_id);
|
||||||
|
return `/api/${encodeURIComponent(t.handle)}/${encodeURIComponent(t.slug)}/hls/playlist.m3u8?${p}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function play(track) {
|
||||||
|
const url = streamUrl(track);
|
||||||
|
if (window.Hls && Hls.isSupported()) {
|
||||||
|
if (hls) hls.destroy();
|
||||||
|
hls = new Hls();
|
||||||
|
hls.on(Hls.Events.ERROR, (_, d) => {
|
||||||
|
if (d.fatal) {
|
||||||
|
console.warn("hls fatal", d);
|
||||||
|
go(active + 1);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
hls.loadSource(url);
|
||||||
|
hls.attachMedia(audio);
|
||||||
|
} else {
|
||||||
|
audio.src = url; // Safari plays HLS natively
|
||||||
|
}
|
||||||
|
audio.play().catch(() => {});
|
||||||
|
}
|
||||||
|
|
||||||
|
function finishActive() {
|
||||||
|
if (active < 0 || reported) return;
|
||||||
|
reported = true;
|
||||||
|
// Flicking past a card before any audio played says nothing about the track.
|
||||||
|
if (listened < 1) return;
|
||||||
|
const { track } = cards[active];
|
||||||
|
const dur = audio.duration || track.duration || 0;
|
||||||
|
if (dur && listened / dur >= COMPLETE_RATIO) sendEvent(track, "complete", listened);
|
||||||
|
else if (listened < SKIP_SECONDS) sendEvent(track, "skip", listened);
|
||||||
|
else sendEvent(track, "play", listened);
|
||||||
|
}
|
||||||
|
|
||||||
|
function activate(idx) {
|
||||||
|
if (idx === active || !cards[idx]) return;
|
||||||
|
finishActive();
|
||||||
|
active = idx;
|
||||||
|
listened = 0;
|
||||||
|
lastTime = 0;
|
||||||
|
reported = false;
|
||||||
|
if (started) play(cards[idx].track);
|
||||||
|
if (idx >= cards.length - PREFETCH_AHEAD) loadMore();
|
||||||
|
}
|
||||||
|
|
||||||
|
function go(idx) {
|
||||||
|
if (cards[idx]) cards[idx].el.scrollIntoView({ behavior: "smooth" });
|
||||||
|
else loadMore();
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleLike(idx) {
|
||||||
|
const card = cards[idx];
|
||||||
|
card.liked = !card.liked;
|
||||||
|
card.el.querySelector(".like").classList.toggle("on", card.liked);
|
||||||
|
if (stats) { stats.liked += card.liked ? 1 : -1; showStats(); }
|
||||||
|
sendEvent(card.track, card.liked ? "like" : "unlike");
|
||||||
|
}
|
||||||
|
|
||||||
|
audio.addEventListener("timeupdate", () => {
|
||||||
|
const t = audio.currentTime;
|
||||||
|
const d = t - lastTime;
|
||||||
|
if (d > 0 && d < 1.5) listened += d; // ignore seeks
|
||||||
|
lastTime = t;
|
||||||
|
const card = cards[active];
|
||||||
|
if (card && audio.duration) {
|
||||||
|
drawWave(card, t / audio.duration);
|
||||||
|
card.el.querySelector(".cur").textContent = fmtTime(t);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
audio.addEventListener("ended", () => {
|
||||||
|
if (active >= 0 && !reported) {
|
||||||
|
sendEvent(cards[active].track, "complete", listened);
|
||||||
|
reported = true;
|
||||||
|
}
|
||||||
|
go(active + 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
const observer = new IntersectionObserver((entries) => {
|
||||||
|
for (const e of entries) {
|
||||||
|
if (e.isIntersecting) activate(cards.findIndex((c) => c.el === e.target));
|
||||||
|
}
|
||||||
|
}, { root: feed, threshold: 0.6 });
|
||||||
|
|
||||||
|
// UI --------------------------------------------------------------------
|
||||||
|
|
||||||
|
document.querySelector("#start button").addEventListener("click", () => {
|
||||||
|
started = true;
|
||||||
|
document.getElementById("start").hidden = true;
|
||||||
|
if (cards[active]) play(cards[active].track);
|
||||||
|
});
|
||||||
|
|
||||||
|
document.getElementById("filters-btn").addEventListener("click", () => { filtersForm.hidden = !filtersForm.hidden; });
|
||||||
|
|
||||||
|
document.getElementById("likes-btn").addEventListener("click", async () => {
|
||||||
|
const panel = document.getElementById("likes");
|
||||||
|
panel.hidden = !panel.hidden;
|
||||||
|
if (panel.hidden) return;
|
||||||
|
const list = document.getElementById("likes-list");
|
||||||
|
const { tracks } = await (await fetch("/local/likes")).json();
|
||||||
|
list.replaceChildren(...tracks.map((t) => {
|
||||||
|
const li = document.createElement("li");
|
||||||
|
const a = document.createElement("a");
|
||||||
|
a.href = `${PL8}/${t.handle}/${t.slug}`;
|
||||||
|
a.target = "_blank";
|
||||||
|
a.textContent = cleanTitle(t.title);
|
||||||
|
const by = document.createElement("span");
|
||||||
|
by.textContent = ` · ${t.artist_name}`;
|
||||||
|
li.append(a, by);
|
||||||
|
return li;
|
||||||
|
}));
|
||||||
|
if (!tracks.length) list.textContent = "nothing liked yet";
|
||||||
|
});
|
||||||
|
|
||||||
|
filtersForm.addEventListener("submit", (e) => {
|
||||||
|
e.preventDefault();
|
||||||
|
const f = new FormData(filtersForm);
|
||||||
|
filters = {};
|
||||||
|
if (f.get("free")) filters.free = "1";
|
||||||
|
if (f.get("bpm_min")) filters.bpm_min = f.get("bpm_min");
|
||||||
|
if (f.get("bpm_max")) filters.bpm_max = f.get("bpm_max");
|
||||||
|
filtersForm.hidden = true;
|
||||||
|
// Drop cards below the current one so the new filter applies right away.
|
||||||
|
for (const c of cards.splice(active + 1)) { observer.unobserve(c.el); c.el.remove(); }
|
||||||
|
loadMore();
|
||||||
|
});
|
||||||
|
|
||||||
|
document.addEventListener("keydown", (e) => {
|
||||||
|
if (e.target.tagName === "INPUT" || e.ctrlKey || e.metaKey || e.altKey) return;
|
||||||
|
const k = e.key.length === 1 ? e.key.toLowerCase() : e.key;
|
||||||
|
if (k === "ArrowDown" || k === "j") { e.preventDefault(); go(active + 1); }
|
||||||
|
else if (k === "ArrowUp" || k === "k") { e.preventDefault(); go(active - 1); }
|
||||||
|
else if (k === " ") { e.preventDefault(); audio.paused ? audio.play() : audio.pause(); }
|
||||||
|
else if (k === "ArrowRight") audio.currentTime = Math.min(audio.duration || 0, audio.currentTime + 10);
|
||||||
|
else if (k === "ArrowLeft") audio.currentTime = Math.max(0, audio.currentTime - 10);
|
||||||
|
else if (k === "l" && active >= 0) toggleLike(active);
|
||||||
|
});
|
||||||
|
|
||||||
|
window.addEventListener("resize", () => cards.forEach((c) => drawWave(c, 0)));
|
||||||
|
window.addEventListener("pagehide", finishActive);
|
||||||
|
|
||||||
|
// Sign-in -----------------------------------------------------------------
|
||||||
|
|
||||||
|
const loginEl = document.getElementById("login");
|
||||||
|
const emailForm = document.getElementById("email-form");
|
||||||
|
const codeForm = document.getElementById("code-form");
|
||||||
|
const loginError = document.getElementById("login-error");
|
||||||
|
|
||||||
|
async function postJSON(url, data) {
|
||||||
|
const res = await fetch(url, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(data || {}),
|
||||||
|
});
|
||||||
|
const body = await res.json().catch(() => ({}));
|
||||||
|
if (!res.ok) throw new Error(body.error || `request failed (${res.status})`);
|
||||||
|
return body;
|
||||||
|
}
|
||||||
|
|
||||||
|
function showLogin() {
|
||||||
|
document.getElementById("bar").hidden = true;
|
||||||
|
loginEl.hidden = false;
|
||||||
|
emailForm.hidden = false;
|
||||||
|
codeForm.hidden = true;
|
||||||
|
document.getElementById("email").focus();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function withForm(form, fn) {
|
||||||
|
const buttons = form.querySelectorAll("button");
|
||||||
|
buttons.forEach((b) => { b.disabled = true; });
|
||||||
|
loginError.textContent = "";
|
||||||
|
try {
|
||||||
|
await fn();
|
||||||
|
} catch (err) {
|
||||||
|
loginError.textContent = err.message;
|
||||||
|
}
|
||||||
|
buttons.forEach((b) => { b.disabled = false; });
|
||||||
|
}
|
||||||
|
|
||||||
|
emailForm.addEventListener("submit", (e) => {
|
||||||
|
e.preventDefault();
|
||||||
|
withForm(emailForm, async () => {
|
||||||
|
const { email } = await postJSON("/auth/start", { email: emailForm.email.value });
|
||||||
|
document.getElementById("sent-to").textContent = email;
|
||||||
|
emailForm.hidden = true;
|
||||||
|
codeForm.hidden = false;
|
||||||
|
codeForm.code.value = "";
|
||||||
|
codeForm.code.focus();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
codeForm.addEventListener("submit", (e) => {
|
||||||
|
e.preventDefault();
|
||||||
|
withForm(codeForm, async () => {
|
||||||
|
const { user } = await postJSON("/auth/verify", { code: codeForm.code.value });
|
||||||
|
signedIn(user);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
codeForm.code.addEventListener("input", () => {
|
||||||
|
if (/^\d{6}$/.test(codeForm.code.value)) codeForm.requestSubmit();
|
||||||
|
});
|
||||||
|
|
||||||
|
document.getElementById("resend").addEventListener("click", () => {
|
||||||
|
withForm(codeForm, async () => {
|
||||||
|
await postJSON("/auth/resend");
|
||||||
|
loginError.textContent = "new code sent";
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
document.getElementById("restart").addEventListener("click", showLogin);
|
||||||
|
|
||||||
|
document.getElementById("logout-btn").addEventListener("click", async () => {
|
||||||
|
await postJSON("/auth/logout");
|
||||||
|
location.reload();
|
||||||
|
});
|
||||||
|
|
||||||
|
function signedIn(user) {
|
||||||
|
loginEl.hidden = true;
|
||||||
|
document.getElementById("bar").hidden = false;
|
||||||
|
document.getElementById("logout-btn").textContent = `sign out @${user.handle}`;
|
||||||
|
if (!cards.length) loadMore();
|
||||||
|
}
|
||||||
|
|
||||||
|
(async () => {
|
||||||
|
const res = await fetch("/local/me");
|
||||||
|
if (res.ok) signedIn((await res.json()).user);
|
||||||
|
else showLogin();
|
||||||
|
})();
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
|
||||||
|
<title>pl8list ∞</title>
|
||||||
|
<link rel="stylesheet" href="/static/style.css">
|
||||||
|
<script src="https://cdn.jsdelivr.net/npm/hls.js@1.6.16/dist/hls.min.js"
|
||||||
|
integrity="sha384-5E8B0pTlZZJMabWpC0fyYf6OUpe15jJij34BqBAh4NXoHAlLNOjCPRrwtOXOQFAn"
|
||||||
|
crossorigin="anonymous"></script>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<header id="bar" hidden>
|
||||||
|
<span class="logo">pl8list <b>∞</b></span>
|
||||||
|
<span id="stats"></span>
|
||||||
|
<button id="filters-btn" type="button">filters</button>
|
||||||
|
<button id="likes-btn" type="button">likes</button>
|
||||||
|
<button id="logout-btn" type="button" title="sign out"></button>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<div id="login" hidden>
|
||||||
|
<div class="login-box">
|
||||||
|
<h1>pl8list <b>∞</b></h1>
|
||||||
|
<p class="tagline">endless scroll through pl8list, tuned to what you like</p>
|
||||||
|
<form id="email-form">
|
||||||
|
<label for="email">sign in with your pl8list email</label>
|
||||||
|
<input id="email" type="email" name="email" autocomplete="email" required placeholder="you@example.com">
|
||||||
|
<button type="submit">send code</button>
|
||||||
|
</form>
|
||||||
|
<form id="code-form" hidden>
|
||||||
|
<label for="code">pl8list sent a 6-digit code to <span id="sent-to"></span></label>
|
||||||
|
<input id="code" name="code" inputmode="numeric" autocomplete="one-time-code" pattern="[0-9]{6}"
|
||||||
|
maxlength="6" required placeholder="123456">
|
||||||
|
<button type="submit">sign in</button>
|
||||||
|
<div class="login-links">
|
||||||
|
<button type="button" id="resend" class="link">resend code</button>
|
||||||
|
<button type="button" id="restart" class="link">use another email</button>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
<p id="login-error" role="alert"></p>
|
||||||
|
<p class="fineprint">
|
||||||
|
No account? <a href="https://pl8list.com/signup" target="_blank" rel="noopener">sign up on pl8list</a>.
|
||||||
|
This site is not affiliated with pl8list. It never stores your pl8list login.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<form id="filters" hidden>
|
||||||
|
<label><input type="checkbox" name="free"> free only</label>
|
||||||
|
<label>bpm <input type="number" name="bpm_min" min="40" max="250" placeholder="min"></label>
|
||||||
|
<label>– <input type="number" name="bpm_max" min="40" max="250" placeholder="max"></label>
|
||||||
|
<button type="submit">apply</button>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
<aside id="likes" hidden>
|
||||||
|
<h2>likes</h2>
|
||||||
|
<ol id="likes-list"></ol>
|
||||||
|
</aside>
|
||||||
|
|
||||||
|
<main id="feed"></main>
|
||||||
|
|
||||||
|
<div id="start">
|
||||||
|
<button type="button">tap to start</button>
|
||||||
|
<p>↑ ↓ / j k scroll · space play/pause · ← → seek · l like</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<audio id="audio" preload="auto"></audio>
|
||||||
|
|
||||||
|
<template id="card-tpl">
|
||||||
|
<section class="card">
|
||||||
|
<div class="bg"></div>
|
||||||
|
<div class="content">
|
||||||
|
<div class="art"></div>
|
||||||
|
<h1 class="title"></h1>
|
||||||
|
<a class="artist" target="_blank" rel="noopener"></a>
|
||||||
|
<div class="chips"></div>
|
||||||
|
<p class="desc"></p>
|
||||||
|
<canvas class="wave"></canvas>
|
||||||
|
<div class="time"><span class="cur">0:00</span><span class="dur"></span></div>
|
||||||
|
<div class="counts"></div>
|
||||||
|
</div>
|
||||||
|
<nav class="actions">
|
||||||
|
<button type="button" class="like" title="like (l)">♥</button>
|
||||||
|
<a class="open" target="_blank" rel="noopener" title="open on pl8list"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7M9 7h8v8"/></svg></a>
|
||||||
|
<button type="button" class="next" title="next (j)">↓</button>
|
||||||
|
</nav>
|
||||||
|
</section>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script src="/static/app.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@@ -0,0 +1,259 @@
|
|||||||
|
:root {
|
||||||
|
--fg: #f2f2f2;
|
||||||
|
--muted: #9a9a9a;
|
||||||
|
--accent: #ff4d6d;
|
||||||
|
--panel: rgba(18, 18, 20, 0.92);
|
||||||
|
color-scheme: dark;
|
||||||
|
}
|
||||||
|
|
||||||
|
* { box-sizing: border-box; }
|
||||||
|
|
||||||
|
html, body {
|
||||||
|
margin: 0;
|
||||||
|
height: 100%;
|
||||||
|
background: #000;
|
||||||
|
color: var(--fg);
|
||||||
|
font: 15px/1.4 system-ui, -apple-system, "Segoe UI", sans-serif;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
a { color: inherit; }
|
||||||
|
|
||||||
|
button, input {
|
||||||
|
font: inherit;
|
||||||
|
color: inherit;
|
||||||
|
}
|
||||||
|
|
||||||
|
button {
|
||||||
|
cursor: pointer;
|
||||||
|
background: rgba(255, 255, 255, 0.1);
|
||||||
|
border: 1px solid rgba(255, 255, 255, 0.15);
|
||||||
|
border-radius: 999px;
|
||||||
|
padding: 0.35em 0.9em;
|
||||||
|
}
|
||||||
|
|
||||||
|
#bar {
|
||||||
|
position: fixed;
|
||||||
|
inset: 0 0 auto 0;
|
||||||
|
z-index: 10;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.75rem;
|
||||||
|
padding: 0.6rem 1rem;
|
||||||
|
background: linear-gradient(#000c, #0000);
|
||||||
|
}
|
||||||
|
|
||||||
|
.logo { font-weight: 600; margin-right: auto; }
|
||||||
|
.logo b { color: var(--accent); }
|
||||||
|
#stats { color: var(--muted); font-size: 0.8rem; }
|
||||||
|
|
||||||
|
#filters, #likes {
|
||||||
|
position: fixed;
|
||||||
|
z-index: 11;
|
||||||
|
top: 3rem;
|
||||||
|
right: 1rem;
|
||||||
|
background: var(--panel);
|
||||||
|
border: 1px solid #333;
|
||||||
|
border-radius: 12px;
|
||||||
|
padding: 0.8rem 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
#filters { display: flex; gap: 0.8rem; align-items: center; flex-wrap: wrap; }
|
||||||
|
#filters[hidden], #likes[hidden] { display: none; }
|
||||||
|
#filters input[type=number] {
|
||||||
|
width: 4.5em;
|
||||||
|
background: #222;
|
||||||
|
border: 1px solid #444;
|
||||||
|
border-radius: 6px;
|
||||||
|
padding: 0.2em 0.4em;
|
||||||
|
}
|
||||||
|
|
||||||
|
#likes { width: min(26rem, calc(100vw - 2rem)); max-height: 70vh; overflow: auto; }
|
||||||
|
#likes h2 { margin: 0 0 0.5rem; font-size: 1rem; }
|
||||||
|
#likes ol { margin: 0; padding-left: 1.4rem; }
|
||||||
|
#likes li { margin: 0.3rem 0; }
|
||||||
|
#likes li span { color: var(--muted); }
|
||||||
|
|
||||||
|
#feed {
|
||||||
|
height: 100dvh;
|
||||||
|
overflow-y: scroll;
|
||||||
|
scroll-snap-type: y mandatory;
|
||||||
|
scrollbar-width: none;
|
||||||
|
}
|
||||||
|
#feed::-webkit-scrollbar { display: none; }
|
||||||
|
|
||||||
|
.card {
|
||||||
|
position: relative;
|
||||||
|
height: 100dvh;
|
||||||
|
scroll-snap-align: start;
|
||||||
|
scroll-snap-stop: always;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bg {
|
||||||
|
position: absolute;
|
||||||
|
inset: -40px;
|
||||||
|
background: #111 center / cover no-repeat;
|
||||||
|
filter: blur(40px) brightness(0.45) saturate(1.3);
|
||||||
|
}
|
||||||
|
|
||||||
|
.content {
|
||||||
|
position: relative;
|
||||||
|
width: min(34rem, calc(100vw - 5.5rem));
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: center;
|
||||||
|
text-align: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.art {
|
||||||
|
width: min(16rem, 42vh, 70vw);
|
||||||
|
aspect-ratio: 1;
|
||||||
|
border-radius: 14px;
|
||||||
|
background: #222 center / cover no-repeat;
|
||||||
|
box-shadow: 0 20px 60px #000a;
|
||||||
|
display: grid;
|
||||||
|
place-items: center;
|
||||||
|
font-size: 4rem;
|
||||||
|
font-weight: 700;
|
||||||
|
color: #fff6;
|
||||||
|
margin-bottom: 0.6rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.title { font-size: 1.25rem; margin: 0; overflow-wrap: anywhere; }
|
||||||
|
.artist { color: var(--muted); text-decoration: none; }
|
||||||
|
.artist:hover { text-decoration: underline; }
|
||||||
|
|
||||||
|
.chips { display: flex; gap: 0.4rem; flex-wrap: wrap; justify-content: center; }
|
||||||
|
.chips span {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
padding: 0.1em 0.6em;
|
||||||
|
border-radius: 999px;
|
||||||
|
background: rgba(255, 255, 255, 0.12);
|
||||||
|
}
|
||||||
|
.chips .free { background: #1f7a4d; }
|
||||||
|
.chips .preview { background: #6a4b16; }
|
||||||
|
|
||||||
|
.desc {
|
||||||
|
margin: 0;
|
||||||
|
display: -webkit-box;
|
||||||
|
-webkit-box-orient: vertical;
|
||||||
|
-webkit-line-clamp: 2;
|
||||||
|
line-clamp: 2;
|
||||||
|
overflow: hidden;
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
color: var(--muted);
|
||||||
|
font-size: 0.85rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.wave { width: 100%; height: 64px; cursor: pointer; margin-top: 0.6rem; }
|
||||||
|
.time { width: 100%; display: flex; justify-content: space-between; color: var(--muted); font-size: 0.75rem; }
|
||||||
|
.counts { color: var(--muted); font-size: 0.8rem; }
|
||||||
|
|
||||||
|
.actions {
|
||||||
|
position: absolute;
|
||||||
|
right: 0.8rem;
|
||||||
|
bottom: 18vh;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.9rem;
|
||||||
|
}
|
||||||
|
.actions > * {
|
||||||
|
width: 3rem;
|
||||||
|
height: 3rem;
|
||||||
|
display: grid;
|
||||||
|
place-items: center;
|
||||||
|
border-radius: 50%;
|
||||||
|
background: rgba(255, 255, 255, 0.12);
|
||||||
|
border: none;
|
||||||
|
font-size: 1.3rem;
|
||||||
|
text-decoration: none;
|
||||||
|
padding: 0;
|
||||||
|
}
|
||||||
|
.actions .like.on { color: var(--accent); background: rgba(255, 77, 109, 0.2); }
|
||||||
|
|
||||||
|
.status {
|
||||||
|
height: 100dvh;
|
||||||
|
display: grid;
|
||||||
|
place-items: center;
|
||||||
|
color: var(--muted);
|
||||||
|
scroll-snap-align: start;
|
||||||
|
}
|
||||||
|
|
||||||
|
#bar[hidden] { display: none; }
|
||||||
|
#logout-btn { max-width: 14rem; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||||
|
|
||||||
|
#login {
|
||||||
|
position: fixed;
|
||||||
|
inset: 0;
|
||||||
|
z-index: 30;
|
||||||
|
display: grid;
|
||||||
|
place-items: center;
|
||||||
|
padding: 1rem;
|
||||||
|
background:
|
||||||
|
radial-gradient(60rem 40rem at 20% 10%, #ff4d6d33, transparent 60%),
|
||||||
|
radial-gradient(50rem 40rem at 90% 90%, #4d6dff2e, transparent 60%),
|
||||||
|
#050507;
|
||||||
|
}
|
||||||
|
#login[hidden] { display: none; }
|
||||||
|
|
||||||
|
.login-box {
|
||||||
|
width: min(24rem, 100%);
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.9rem;
|
||||||
|
}
|
||||||
|
.login-box h1 { margin: 0; font-size: 2.4rem; }
|
||||||
|
.login-box h1 b { color: var(--accent); }
|
||||||
|
.tagline { margin: 0 0 0.8rem; color: var(--muted); }
|
||||||
|
.login-box form { display: flex; flex-direction: column; gap: 0.6rem; }
|
||||||
|
.login-box form[hidden] { display: none; }
|
||||||
|
.login-box label { color: var(--muted); font-size: 0.9rem; }
|
||||||
|
.login-box input {
|
||||||
|
padding: 0.7em 0.9em;
|
||||||
|
border-radius: 10px;
|
||||||
|
border: 1px solid #333;
|
||||||
|
background: #111;
|
||||||
|
}
|
||||||
|
#code { font-family: ui-monospace, monospace; font-size: 1.4rem; letter-spacing: 0.5em; text-align: center; }
|
||||||
|
.login-box button[type=submit] { background: var(--accent); border: none; color: #fff; padding: 0.7em; font-weight: 600; }
|
||||||
|
.login-box button[disabled] { opacity: 0.6; cursor: wait; }
|
||||||
|
.login-links { display: flex; justify-content: space-between; }
|
||||||
|
.link { background: none; border: none; padding: 0; color: var(--muted); text-decoration: underline; font-size: 0.85rem; }
|
||||||
|
#login-error { margin: 0; min-height: 1.2em; color: var(--accent); font-size: 0.9rem; }
|
||||||
|
.fineprint { margin: 0; color: #666; font-size: 0.75rem; }
|
||||||
|
|
||||||
|
#start {
|
||||||
|
position: fixed;
|
||||||
|
inset: 0;
|
||||||
|
z-index: 20;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 1rem;
|
||||||
|
background: #000c;
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
#start button { font-size: 1.3rem; padding: 0.7em 1.6em; background: var(--accent); border: none; color: #fff; }
|
||||||
|
#start[hidden] { display: none; }
|
||||||
|
|
||||||
|
@media (max-width: 640px) {
|
||||||
|
#stats { display: none; }
|
||||||
|
#bar { gap: 0.4rem; padding: 0.5rem 0.7rem; }
|
||||||
|
#bar button { padding: 0.3em 0.7em; font-size: 0.85rem; }
|
||||||
|
#logout-btn { max-width: 6.5rem; }
|
||||||
|
.content { width: calc(100vw - 2rem); margin-bottom: 4.5rem; }
|
||||||
|
.actions {
|
||||||
|
right: auto;
|
||||||
|
left: 50%;
|
||||||
|
bottom: max(1.2rem, env(safe-area-inset-bottom));
|
||||||
|
transform: translateX(-50%);
|
||||||
|
flex-direction: row;
|
||||||
|
gap: 1.4rem;
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in new issue
Block a user