From 2419efbeafaa268f97b39ba4b45c467ed0d05488 Mon Sep 17 00:00:00 2001 From: Ulas Kalayci Date: Mon, 20 Apr 2026 23:06:44 +0200 Subject: [PATCH] docs: add reviewed ICS subscription v2 spec --- .../specs/ICS_URL_Subscription_v2.md | 259 ++++++++++++++++++ 1 file changed, 259 insertions(+) create mode 100644 docs/superpowers/specs/ICS_URL_Subscription_v2.md diff --git a/docs/superpowers/specs/ICS_URL_Subscription_v2.md b/docs/superpowers/specs/ICS_URL_Subscription_v2.md new file mode 100644 index 0000000..36cc712 --- /dev/null +++ b/docs/superpowers/specs/ICS_URL_Subscription_v2.md @@ -0,0 +1,259 @@ +# ICS-URL Subscription — Implementation Plan (v2) + +**Date:** 2026-04-20 +**Status:** Approved +**Supersedes:** ICS_URL_Subscription.md (v1) + +## Overview + +Allow all family members to subscribe to external calendars via ICS URL (e.g. public Google, Outlook, or any webcal-compatible feed). Events are fetched periodically and stored locally. Users choose whether a subscription is private or shared with the whole family. + +## Requirements + +- Any user (not just admins) can add subscriptions +- Per-subscription visibility: **private** (only creator) or **shared** (all family members) +- Custom color per subscription, chosen by the user +- Sync interval: shared with existing `SYNC_INTERVAL_MINUTES` setting +- Manual "Sync now" button per subscription +- Events from subscriptions are editable; user-modified events are not overwritten on re-sync but can be reset to upstream +- Events are deleted when their subscription is deleted +- Recurring events (RRULE) are expanded within a rolling window + +--- + +## 1. Database + +### 1.1 New table: `ics_subscriptions` + +```sql +CREATE TABLE ics_subscriptions ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT NOT NULL, + url TEXT NOT NULL, + color TEXT NOT NULL DEFAULT '#6366f1', + shared INTEGER NOT NULL DEFAULT 0, -- 0 = private, 1 = shared with all + created_by INTEGER REFERENCES users(id) ON DELETE SET NULL, + etag TEXT, -- HTTP ETag for conditional fetch + last_modified TEXT, -- HTTP Last-Modified for conditional fetch + last_sync TEXT, -- ISO 8601, always UTC + created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')) +); +``` + +**Change from v1:** `ON DELETE SET NULL` instead of `ON DELETE CASCADE` on `created_by`. When a user is deleted, shared subscriptions survive and can be managed by any admin. Orphaned private subscriptions (where `created_by IS NULL AND shared = 0`) are cleaned up by a post-deletion sweep or made visible to admins. + +### 1.2 Migrations to `calendar_events` + +Two new columns via append-only migration entries: + +1. `subscription_id INTEGER REFERENCES ics_subscriptions(id) ON DELETE CASCADE` +2. `user_modified INTEGER NOT NULL DEFAULT 0` — set to `1` on user edit; prevents sync overwrite + +**Migration ordering:** The `ics_subscriptions` CREATE TABLE entry must precede the `ALTER TABLE calendar_events ADD COLUMN subscription_id` entry in the migrations array. + +### 1.3 The `external_source` CHECK constraint + +**Do not recreate the `calendar_events` table.** Table recreation is the highest-risk migration possible — data loss, broken foreign keys, index rebuilds. Instead: + +- **Option A (recommended):** Drop the CHECK constraint entirely. Validate `external_source ∈ {'local', 'google', 'apple', 'ics'}` at the application layer (in the route handler and in `ics-subscription.js`). SQLite allows dropping a CHECK via table recreation, but the point is to *avoid* the recreation. If the existing CHECK was added inline in the original CREATE TABLE, it is already baked in. In that case, the CHECK will reject `'ics'` inserts. Verify the actual schema first: + ```sql + SELECT sql FROM sqlite_master WHERE name = 'calendar_events'; + ``` + If no CHECK exists → no migration needed, just validate in code. + If CHECK exists → the table recreation is unavoidable, but must run inside `BEGIN IMMEDIATE` / `COMMIT` with full column + index + FK reconstruction. Document every step. + +- **Option B (if CHECK must stay):** Recreate in a transaction. Copy data into temp table, drop original, create with new CHECK, copy back, recreate indexes and FKs, commit. Test with a populated database before merge. + +### 1.4 Unique constraint for upsert + +Add a unique index scoped to the subscription: + +```sql +CREATE UNIQUE INDEX idx_calendar_events_sub_extid + ON calendar_events (subscription_id, external_calendar_id) + WHERE subscription_id IS NOT NULL; +``` + +**Rationale:** ICS UIDs are only unique within a single feed, not globally. Without this scope, Feed B can overwrite Feed A's events if they share a UID. The upsert must use `ON CONFLICT(subscription_id, external_calendar_id)`. + +### 1.5 Visibility filter + +```sql +WHERE external_source != 'ics' + OR subscription_id IN ( + SELECT id FROM ics_subscriptions + WHERE shared = 1 + OR created_by = :userId + ) +``` + +Unchanged from v1. Events with `external_source = 'ics'` and `subscription_id IS NULL` (should not exist, but defensively) are filtered out. + +--- + +## 2. Backend + +### 2.1 New file: `server/services/ics-parser.js` + +Extract from `apple-calendar.js` into a shared module: + +| Export | Source | +|--------|--------| +| `parseICS(text)` | existing | +| `unfoldLines(text)` | existing | +| `formatICSDate(value)` | existing | +| `tzLocalToUTC(dateStr, tzid)` | existing | +| `applyDuration(start, duration)` | existing | +| `expandRRULE(vevent, windowStart, windowEnd)` | **new** | + +Both `apple-calendar.js` and `ics-subscription.js` import from here. The refactoring of existing functions must be a **separate commit** with no logic changes, tested independently before the ICS subscription code is added. + +**RRULE expansion:** `expandRRULE` generates occurrences within a rolling window (default: 6 months past → 12 months future). Supports `FREQ` (DAILY, WEEKLY, MONTHLY, YEARLY), `COUNT`, `UNTIL`, `INTERVAL`, `BYDAY`. `EXDATE` entries exclude specific occurrences. Each expanded occurrence gets a synthetic `external_calendar_id` of `{UID}__{ISO-date}` for stable upsert identity. Unsupported RRULE features (BYSETPOS, BYMONTHDAY with negative values, etc.) log a warning and fall back to non-expansion. + +### 2.2 New file: `server/services/ics-subscription.js` + +| Export | Description | +|--------|-------------| +| `fetchAndParse(url, etag?, lastModified?)` | Validate + fetch + parse (see §2.3) | +| `sync(subscriptionId?)` | Sync one or all subscriptions (see §2.4) | +| `getAll(userId)` | Return all subscriptions visible to userId (own + shared) | +| `create(userId, { name, url, color, shared })` | Validate, insert, trigger initial sync. Return subscription + sync result (success or error message) | +| `update(userId, id, fields)` | Update name/color/shared; only creator or admin | +| `remove(userId, id)` | Delete subscription (events cascade); only creator or admin | + +### 2.3 `fetchAndParse` — security hardening + +1. **Scheme whitelist:** Only `https://` and `webcal://` (normalized to `https://`). Reject `http://`, `file://`, `ftp://`, `data://`. +2. **DNS rebinding / SSRF protection:** After URL parsing, resolve the hostname. Reject if the resolved IP falls in private ranges: `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `::1`, `fc00::/7`, `fe80::/10`. Use `dns.resolve4` / `dns.resolve6` before passing to `fetch`. +3. **Timeout:** 15 seconds per request (`AbortController` + `setTimeout`). +4. **Response size limit:** Abort if `Content-Length > 10 MB` or if streamed body exceeds 10 MB. +5. **Content-Type hint check:** Warn (but don't block) if response Content-Type is not `text/calendar`. Some servers serve ICS as `text/plain`. +6. **Conditional fetch:** Send `If-None-Match: {etag}` and `If-Modified-Since: {lastModified}` headers. On `304 Not Modified`, skip parsing entirely and return early. On `200`, store new `etag` and `last-modified` response headers back to `ics_subscriptions`. + +### 2.4 `sync` — operational details + +1. Wrap the entire sync for one subscription in `BEGIN IMMEDIATE` / `COMMIT`. This turns N individual upserts into a single disk write. +2. **Per-subscription mutex:** Maintain an in-memory `Set` of currently-syncing subscriptions. If a sync is already running for a given subscription (e.g. manual sync while periodic sync is active), skip it and return early. The Set is process-local — sufficient for single-process Oikos. +3. **Upsert logic:** `INSERT ... ON CONFLICT(subscription_id, external_calendar_id) DO UPDATE SET ... WHERE user_modified = 0`. Events where `user_modified = 1` are untouched in a single statement — no per-row branching needed. +4. **Stale event cleanup:** After upsert, delete events belonging to this subscription whose `external_calendar_id` is not in the current feed's UID set AND whose `user_modified = 0`. User-modified events whose upstream counterpart disappeared are kept (the user explicitly edited them). +5. On fetch error: log warning, leave existing events and `last_sync` unchanged, continue to next subscription. + +### 2.5 New routes: `/api/v1/calendar/subscriptions` + +| Method | Path | Action | Auth | +|--------|------|--------|------| +| `GET` | `/` | List visible subscriptions | any user | +| `POST` | `/` | Create subscription | any user | +| `PATCH` | `/:id` | Update name/color/shared | creator or admin | +| `DELETE` | `/:id` | Delete subscription + events | creator or admin | +| `POST` | `/:id/sync` | Manual sync now | creator or admin | + +All handlers in `try/catch`. Responses follow `{ data: ... }` / `{ error, code }`. + +**Input validation on POST/PATCH:** +- `url`: required, must parse as valid URL, scheme must be `https` or `webcal` +- `name`: required, non-empty, max 100 chars +- `color`: required on POST, must match `/^#[0-9a-fA-F]{6}$/` +- `shared`: boolean-coercible integer (0 or 1) + +### 2.6 Setting `user_modified` + +When `PATCH /api/v1/calendar/events/:id` updates an event with `external_source = 'ics'`, the handler sets `user_modified = 1` automatically. + +**New:** `PATCH /api/v1/calendar/events/:id/reset` sets `user_modified = 0` on an ICS event. The next sync cycle will overwrite it with upstream data. Returns `{ data: { reset: true } }`. Only the event creator, subscription creator, or admin can call this. + +### 2.7 Sync integration + +`server/index.js` `syncAll()` calls `icsSubscription.sync()` alongside existing Google/Apple sync. ICS sync runs last (lowest priority — Google/Apple are authenticated and more critical). + +### 2.8 Orphan cleanup + +After a user is deleted (`ON DELETE SET NULL` on `created_by`), run a sweep: + +```sql +DELETE FROM ics_subscriptions WHERE created_by IS NULL AND shared = 0; +``` + +This removes private subscriptions that no one can see or manage. Shared orphans remain visible and editable by admins. + +--- + +## 3. Frontend + +### 3.1 Settings page (`public/pages/settings.js`) + +New card **"ICS-Abonnements"** in the existing "Kalender" tab, below Apple Calendar: + +- List of visible subscriptions: color dot, name, visibility badge (`Privat` / `Geteilt`), last sync timestamp (via `formatDate()` + `formatTime()`), sync error indicator if last sync failed +- **"Abonnement hinzufügen"** button reveals inline form: + - URL input (required, `type="url"`) + - Name input (required) + - Color picker (``) + - Toggle "Für alle sichtbar" (default: off) + - Submit / Cancel buttons +- Per-subscription actions: "Jetzt synchronisieren" (shows spinner during sync), "Bearbeiten" (inline), "Löschen" (confirmation via existing confirm pattern) +- Only creator or admin sees edit/delete/sync actions +- Initial sync error on create: show inline warning with error message, subscription is still created + +Rendered inline — no new Web Component. Consistent with Apple Calendar form pattern in the same tab. + +### 3.2 Calendar page (`public/pages/calendar.js`) + +- Events with `external_source = 'ics'` render with their subscription's color +- No special UI indicator for `user_modified` — keeps UX clean +- Event detail view for `user_modified = 1` events shows a subtle "Auf Original zurücksetzen" link that calls `PATCH .../reset` + +### 3.3 i18n + +All new strings in `public/locales/de.json`: + +- `settings.ics.title` — "ICS-Abonnements" +- `settings.ics.add` — "Abonnement hinzufügen" +- `settings.ics.form.*` — URL, Name, Color, Shared toggle labels +- `settings.ics.actions.*` — Sync, Edit, Delete labels +- `settings.ics.status.*` — last sync, sync error, syncing states +- `settings.ics.confirm_delete` — deletion confirmation +- `settings.ics.badges.*` — "Privat", "Geteilt" +- `calendar.ics.reset` — "Auf Original zurücksetzen" + +`de` is the reference locale. Other locales fall back gracefully via `t()`. + +--- + +## 4. Error Handling + +| Scenario | Behavior | +|----------|----------| +| ICS URL unreachable / timeout | Log warning, keep existing events, leave `last_sync` unchanged | +| 304 Not Modified | Skip parse, update `last_sync` timestamp only | +| Invalid ICS content | Log warning, skip malformed VEVENTs, continue with valid ones | +| URL returns non-ICS content (no `BEGIN:VCALENDAR`) | Log error, abort sync for this subscription | +| Response > 10 MB | Abort fetch, log error | +| SSRF attempt (private IP) | Reject with 400: "URL resolves to a private address" | +| Unsupported URL scheme | Reject with 400: "Only https and webcal URLs are supported" | +| RRULE with unsupported features | Log warning per event, fall back to single occurrence | +| Unauthorized edit/delete | 403 response | +| Duplicate URL across subscriptions | Allowed (user may want same feed with different color/name) | +| Initial sync fails on create | Subscription created, error message returned in response body | +| Concurrent sync on same subscription | Second sync skipped (in-memory mutex) | + +--- + +## 5. Commit Strategy + +| # | Scope | Description | +|---|-------|-------------| +| 1 | `refactor(calendar)` | Extract ICS parser from `apple-calendar.js` into `server/services/ics-parser.js`. No logic changes. Existing Apple Calendar tests must still pass. | +| 2 | `feat(calendar)` | Add `ics_subscriptions` table, `calendar_events` columns, unique index. Add RRULE expansion to parser. Migrations in correct order. | +| 3 | `feat(calendar)` | Add `ics-subscription.js` service, routes, sync integration, security hardening. Backend tests. | +| 4 | `feat(calendar)` | Frontend: settings card, calendar color rendering, reset flow, i18n keys. | + +--- + +## 6. Out of Scope + +- CalDAV authentication (Basic/OAuth) — ICS-URL only (public or pre-authenticated URLs) +- Per-event sync conflict resolution UI (beyond the reset button) +- Subscription import/export +- VTIMEZONE definitions beyond offset-based conversion (use system timezone as fallback) +- RRULE features beyond FREQ/COUNT/UNTIL/INTERVAL/BYDAY/EXDATE