80 lines
4.9 KiB
Markdown
80 lines
4.9 KiB
Markdown
# Recover Live Course Pages Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Inventory the previous public DokuWiki and migrate course details missing from the new Starlight wiki.
|
|
|
|
**Architecture:** A read-only recursive crawler captures canonical page IDs and page HTML from the pinned origin. A deterministic comparison classifies every page, and only in-scope course pages become supplemental Markdown destinations registered separately from the original export manifest.
|
|
|
|
**Tech Stack:** Node.js 22, native `fetch`/child process curl as required for pinned DNS and legacy TLS, linkedom, Turndown, Astro Starlight, Node test runner.
|
|
|
|
## Global Constraints
|
|
|
|
- Crawl `https://msvincognito.nl/wiki/` through origin `185.104.29.94` without changing the server.
|
|
- Include public Bachelor, Master AI, and Master DSDM course/programme details.
|
|
- Exclude DokuWiki system, sidebar, administrative, member-only, and unrelated pages.
|
|
- Preserve source meaning and warn that recovered academic information may be outdated.
|
|
- Keep the original 29-source migration manifest unchanged.
|
|
- Preserve unrelated working-tree changes and do not deploy or push.
|
|
|
|
---
|
|
|
|
### Task 1: Capture a complete live-page inventory
|
|
|
|
**Files:**
|
|
- Create: `scripts/crawl-previous-wiki.mjs`
|
|
- Create: `scripts/lib/previous-wiki.mjs`
|
|
- Create: `tests/previous-wiki.test.mjs`
|
|
- Create: `to-be-studied/previous-wiki/2026-08-03/manifest.json`
|
|
- Create: `to-be-studied/previous-wiki/2026-08-03/pages/**`
|
|
|
|
**Interfaces:**
|
|
- Produces: manifest records `{ id, url, title, localPath, status }` and explicit failure records
|
|
|
|
- [ ] **Step 1: Write fixture-based crawler tests** for nested sitemap discovery, deduplication, canonical IDs, and failed-page recording.
|
|
- [ ] **Step 2: Run `node --test tests/previous-wiki.test.mjs`** and confirm failure before implementation.
|
|
- [ ] **Step 3: Implement the crawler** with bounded requests, stable ordering, explicit source URLs, and no credentials.
|
|
- [ ] **Step 4: Run the crawler against the pinned origin** and repeat failed requests once; retain any persistent failure in the manifest.
|
|
- [ ] **Step 5: Verify** that the manifest count equals the unique discovered page-ID count and all successful capture paths exist.
|
|
|
|
### Task 2: Classify live pages against the new wiki
|
|
|
|
**Files:**
|
|
- Create: `scripts/compare-previous-wiki.mjs`
|
|
- Create: `docs/live-course-recovery.tsv`
|
|
- Create: `docs/live-course-recovery.md`
|
|
- Modify: `tests/previous-wiki.test.mjs`
|
|
|
|
**Interfaces:**
|
|
- Consumes: crawl manifest, migration manifest, and existing Starlight routes
|
|
- Produces: one classification per live page: `migrate`, `represented`, `excluded`, `empty`, or `failed`
|
|
|
|
- [ ] **Step 1: Write failing classification tests** for course namespaces, excluded system IDs, already represented pages, empty pages, and stable destination slugs.
|
|
- [ ] **Step 2: Run the focused tests** and confirm failure.
|
|
- [ ] **Step 3: Implement deterministic classification and destination mapping** for Bachelor, Master AI, and Master DSDM namespaces.
|
|
- [ ] **Step 4: Generate the TSV and human-readable report** with totals that reconcile exactly to the crawl manifest.
|
|
- [ ] **Step 5: Review all `migrate` and ambiguous records** against their captured page before publication; reclassify ambiguous non-course records as excluded with a written reason.
|
|
|
|
### Task 3: Convert and publish missing course details
|
|
|
|
**Files:**
|
|
- Create or modify: `src/content/docs/bachelor/**`
|
|
- Create or modify: `src/content/docs/master-ai/**`
|
|
- Create or modify: `src/content/docs/master-dsdm/**`
|
|
- Modify: `docs/supplemental-content.json`
|
|
- Modify: `src/config/sidebar.mjs`
|
|
- Modify: `tests/content-audit.test.mjs`
|
|
- Modify: `docs/live-course-recovery.tsv`
|
|
- Modify: `docs/live-course-recovery.md`
|
|
|
|
**Interfaces:**
|
|
- Consumes: reviewed `migrate` records and captured source pages
|
|
- Produces: one Starlight destination per migrated course page, with source provenance and archive-media links
|
|
|
|
- [ ] **Step 1: Add failing assertions** requiring every `migrate` record to have a registered destination, frontmatter, historical warning, source URL, and sidebar reachability.
|
|
- [ ] **Step 2: Run the focused tests** and confirm the missing destinations fail.
|
|
- [ ] **Step 3: Convert each reviewed page** to Markdown, removing DokuWiki controls while preserving factual content, links, lists, and tables.
|
|
- [ ] **Step 4: Register destinations and sidebar entries**, and update classifications from `migrate` to `migrated` only after the file exists.
|
|
- [ ] **Step 5: Run `npm run verify`** and resolve all audit, build, rendered-output, and internal-link failures.
|
|
- [ ] **Step 6: Reconcile the final report** so every live page has exactly one outcome and the reported migrated-page count equals the new destination count.
|
|
|