osrs-codex/docs/data-sources.md

65 lines
3.7 KiB
Markdown

# OSRS Codex — Data Sources & Verification Rules
Every codex entry must carry `sources:` and `verified:` (date). Facts are
classified, and each class has a **required** verification path. An entry
whose facts fail verification stays `status: draft` and blocks merge.
## Fact classes
| class | examples | required source |
|-------|----------|-----------------|
| **item-ids** | item names, wiki ids | `prices.runescape.wiki/api/v1/osrs/mapping` |
| **alch values** | lowalch / highalch / value | same mapping endpoint; assert `lowalch == highalch*2//3` |
| **ge limits** | buy limit / 4h | mapping `limit` **and** wiki `Module:Exchange/<Item>` Lua field `limit` — both must agree |
| **membership** | F2P vs members | mapping `members` **and** item page `members = No/Yes` |
| **live prices** | buy/sell price | `prices.runescape.wiki/api/v1/osrs/latest` → `low` (buy); re-verify on every `status: verified` bump |
| **xp / mechanics** | XP per action, level reqs, cast times | OSRS wiki article infobox (oldschool.runescape.wiki/w/<Topic>) |
| **locations / NPC dialogue** | bank names, menu option strings | OSRS wiki + in-game observation (flag `observed: date`) |
| **best-practice setup** | equipment/technique choices (staff of fire, noted items, best prayer) | OSRS wiki page, or `researched <date>` (follow-up citation issue) |
## Authoritative endpoints (all require a descriptive User-Agent or 403)
```
mapping : https://prices.runescape.wiki/api/v1/osrs/mapping (JSON array)
latest : https://prices.runescape.wiki/api/v1/osrs/latest ({"data":{id:{low,high}}})
exchange: https://oldschool.runescape.wiki/api.php?action=query&titles=Module:Exchange/<Name>&prop=revisions&rvprop=content&format=json
itempage: https://oldschool.runescape.wiki/api.php?action=parse&page=<Name>&prop=wikitext&format=json
```
## Verification pipeline (triple-source, then assert)
1. Build `by_name` from mapping. Fetch `latest` once.
2. For each item fact: mapping → exchange module → item page. All must agree.
3. Assert `lowalch == highalch * 2 // 3` — catches copy/paste errors.
4. Record the **research date** in `verified:` and the endpoint used in `sources:`.
## Known traps (learned the hard way — do not repeat)
- **The `mapping` mirror is incomplete.** Observed 2026-08-27: 4,652 items,
dyes present but **no mantle/dye-variant entries at all**. An item missing
from mapping is NOT proof the item doesn't exist — cross-check the wiki
page before concluding anything. Never trust "not in mirror" as a fact.
- **`Module:Exchange` field is `limit`, not `buyLimit`.** Grepping the wrong
name yields false mismatches on every row.
- **Members flag vs intuition.** Items that "look F2P" can be members-only
(all bracelets, battlestaff, unstrung maple/yew/magic bows, unholy symbol,
orbs). The strung `Maple longbow` is F2P; `Maple longbow (u)` is members.
Always read the flag, never infer it.
- **Live prices move daily.** `latest` is a snapshot. Entries that embed
prices must carry `verified:` within the last 30 days to stay
`status: verified`; the validator warns (not fails) on staleness.
- **Alch economics are market-dependent.** Low alch is a *training*
activity at current prices (verified negative GP/cycle 2026-08-27).
Record the true economics; do not dress a loss as profit.
## Research tooling
- `osrs-wiki-item-data` skill (Vale) holds the working Python recipe,
the verified 20-item F2P low-alch catalog, and jar-verification notes.
- `ge-flip-scanner` skill: live margin/volume scanning (5m endpoint nulls
pitfall applies).
- OSRS wiki is reachable from this box via the Hermes web proxy
(`web_extract` / `web_search`), NOT direct DNS — direct
`services.runescape.wiki` resolves fail from WSL. `prices.runescape.wiki`
resolves fine directly.