feat(library): weekly harvest on a systemd timer
Monday 07:00, Persistent=true so a week missed to downtime fires on next boot rather than leaving a silently empty library. 30-minute timeout so a wedged run cannot hold the slot until the following week. The script is deliberately thin — it calls the API and reports — so it never needs changing when the harvest does. Auth is a long-lived operator session in /etc/clawmates/library.token (root-only, 600); rotate by replacing the file. Exit status follows `healthy`, not paper count. A mature library shelves nothing most weeks and that is success; a run that errored is a failure even if it shelved something. The first manual fire caught a real bug in this script, in the opposite direction to this week's usual: the harvest genuinely shelved 15 papers and pushed them, and the reporter crashed on an escaped quote inside an f-string, so systemd marked the unit FAILED. A false failure destroys trust in the signal exactly as a false success does. The reporter now avoids backslashes entirely (it is embedded in a single-quoted shell string) and was proved against the real response shape before being trusted. Verified end to end on gw-04: run 1: 25 candidates, 10 already held, 15 shelved, pushed run 2: 25 candidates, 25 already held, 0 shelved, no branch, healthy Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
cf076bd8ea
commit
3124fd3c8f
@@ -0,0 +1,14 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Clawmates paper library harvest (arXiv -> shelf + vault catalogue)
|
||||||
|
Wants=docker.service
|
||||||
|
After=docker.service network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/usr/local/bin/clawmates-library.sh
|
||||||
|
StandardOutput=journal
|
||||||
|
StandardError=journal
|
||||||
|
Nice=10
|
||||||
|
# A harvest downloads PDFs and pushes a branch; give it room but do not
|
||||||
|
# let a wedged run hold the slot until the next week.
|
||||||
|
TimeoutStartSec=30min
|
||||||
Executable
+49
@@ -0,0 +1,49 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Weekly paper-library harvest.
|
||||||
|
#
|
||||||
|
# Deliberately thin: it calls the API and reports what came back. All the
|
||||||
|
# logic lives in the server, so this file never needs to change when the
|
||||||
|
# harvest does.
|
||||||
|
#
|
||||||
|
# The token lives in /etc/clawmates/library.token (root-only). It is a
|
||||||
|
# long-lived operator session; rotate by replacing the file.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
TOKEN_FILE=/etc/clawmates/library.token
|
||||||
|
[ -r "$TOKEN_FILE" ] || { echo "library: no token at $TOKEN_FILE"; exit 1; }
|
||||||
|
TOKEN=$(cat "$TOKEN_FILE")
|
||||||
|
|
||||||
|
RESP=$(docker run --rm --network clawmates_core curlimages/curl:latest \
|
||||||
|
-s -m 1800 -X POST \
|
||||||
|
-H "Authorization: Bearer $TOKEN" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"per_topic":5}' \
|
||||||
|
http://clawmates_server_1:8080/api/library/runs)
|
||||||
|
|
||||||
|
echo "library: $RESP" | head -c 2000
|
||||||
|
|
||||||
|
# Report health explicitly. A run that shelved nothing is normal for a
|
||||||
|
# mature library; a run that ERRORED is not, and the two look identical
|
||||||
|
# if you only count papers.
|
||||||
|
echo "$RESP" | python3 -c '
|
||||||
|
import json, sys
|
||||||
|
try:
|
||||||
|
d = json.load(sys.stdin)
|
||||||
|
except Exception as e:
|
||||||
|
print("library: unreadable response (%s)" % e)
|
||||||
|
sys.exit(1)
|
||||||
|
shelved = len(d.get("shelved", []))
|
||||||
|
healthy = d.get("healthy", False)
|
||||||
|
# Backslashes are avoided inside this program on purpose: it is embedded in a
|
||||||
|
# single-quoted shell string, and an escaped quote here does not survive the
|
||||||
|
# shell. The first version used one inside an f-string, crashed on every run,
|
||||||
|
# and systemd reported a FAILED unit for a harvest that had actually shelved
|
||||||
|
# 15 papers and pushed them. A false failure destroys trust in the signal as
|
||||||
|
# surely as a false success.
|
||||||
|
print("library: %d candidates, %d already held, %d shelved, healthy=%s, pushed=%s, branch=%s" % (
|
||||||
|
d.get("candidates", 0), d.get("already_had", 0), shelved,
|
||||||
|
healthy, d.get("pushed"), d.get("branch")))
|
||||||
|
for f in d.get("failed", []):
|
||||||
|
print("library: FAILED %s" % f)
|
||||||
|
sys.exit(0 if healthy else 1)
|
||||||
|
'
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Clawmates paper library — weekly harvest
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
# Monday 07:00 local. Weekly rather than daily because arXiv moves at
|
||||||
|
# roughly that pace for a narrow topic set, and a run that almost always
|
||||||
|
# finds nothing trains you to ignore it.
|
||||||
|
OnCalendar=Mon *-*-* 07:00:00
|
||||||
|
# Fire on next boot if the machine was down at the scheduled time — a
|
||||||
|
# missed week is a silently empty library.
|
||||||
|
Persistent=true
|
||||||
|
AccuracySec=1min
|
||||||
|
Unit=clawmates-library.service
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
Reference in New Issue
Block a user