Accept Scattermind private reading envelopes

README.md

@@ -49,7 +49,7 @@ Ranker's continuation job is narrow:
 
 Candidate items may include optional 1–10 `rankerHints` (`value`, `effort`, `confidence`, `urgency`, `revenue`, `novelty`, `risk`). Ranker blends those explicit Scattermind hints with text heuristics; `effort` is inverted into feasibility. Hints improve the defended order, but `recommendedLane: "defer"` or `"park"` remains a safety rail so dashboard-swamp items do not get promoted by flashy wording. For clean bridge handoff, Scattermind should send `sourceSection` and `evidenceNeeded` on each active next move. Scattermind can also send `targetAudience`, `constraints`, `assumptions`, and `nonGoals` / `avoid` at the top level, in `featureSet`, inside `snapshot.context` or `conceptMap.context`, or as a structured top-level `context` object with `summary`, `targetAudience`, `constraints`, `nonGoals` / `avoid`, and `assumptions`; Ranker merges these sources rather than letting a shallow wrapper context shadow deeper artifact guardrails. Ranker also accepts structured `decisionContext` / `decision_context`, `rankerContext` / `ranker_context`, `handoffContext` / `handoff_context`, and `bridgeContext` / `bridge_context` objects at the top level, inside bridge envelopes, feature sets, artifacts, Snapshots, or Concept Maps, so Scattermind can keep context in a machine-named contract field without losing guardrails. Lens-only Concept Maps may additionally send audience / constraints / assumptions / risk lens content, and Ranker will split sentence-style lens notes into readable decision context instead of leaking `[object Object]`. If Scattermind only has a flat context string or schema-light structured context summaries (`context.summary`, `snapshot.context.summary`, `conceptMap.context.summary`, etc.), Ranker extracts simple guardrails such as `Solo builder`, `Manual proof`, `Avoid ...`, `No ...`, `Non-goal: ...`, `Not yet ...`, and `Do not ...` into `input.decisionContext` / `handoff.decisionContext` so early bridge exports still protect against dashboard/auth/billing drift. Ranker returns that decision context in `input.decisionContext` and `handoff.decisionContext`, and penalizes candidates that conflict with source non-goals (for example saved workspaces/auth/billing before the continuation proof). If Scattermind sends duplicate candidate IDs, Ranker keeps the first ID, suffixes later duplicates (`preview-2`), and reports the normalization in `handoff.warnings` / `handoff.itemTrace` so downstream build-order references remain addressable. Ranker also accepts Scattermind's paid Concept Map object directly when it arrives with top-level `reference_code`, `working_name`, `ideaText`, and string-valued `lenses.channel` / `lenses.risk` fields; the reference code becomes source provenance, the working name becomes the source title, and labelled Build Order text is turned into rank-ready candidates without requiring Scattermind to wrap it first.
 
-Candidate trace note: candidate-level `sourceItemId` / `traceId`, `sourceTitle` / `lensTitle`, and `sourceExcerpt` / `sourceQuote` are preserved in ranked items, `buildOrderDetails`, and `handoff.itemTrace`. Lens-only Build Order text is also split into deterministic `concept-map.lenses.channel#N` source IDs with the original labelled sentence carried as `sourceQuote`, so pasted paid Concept Maps remain traceable even without explicit candidate objects. String items in laned Build Order arrays now also receive deterministic section-local source IDs such as `concept-map.buildOrder.validateNext#1` and carry the original string as `sourceQuote`, so simple Scattermind exports stay addressable downstream instead of becoming anonymous `feature-1` rows. Ranker also accepts the current Scattermind storage-row shape with `referenceCode`, `ideaText`, `context`, and string-valued `fullReadingJson` / `full_reading_json`; it expands the saved paid Concept Map before ranking so operators do not have to hand-copy lenses out of Appwrite rows. Stringified bridge envelopes in fields such as `rankerInput`, `rankerBridge`, `rankReady`, `bridgePayload`, or `continuationPlan` are expanded the same way, so Appwrite/string-copy handoffs do not have to be manually unwrapped before ranking. If the row only has free Snapshot data (`glimpseJson` / `glimpse_json` / `snapshotJson`), Ranker expands that Snapshot into a minimal continuation order: one manual proof plus the first evidence question, with the Snapshot reference code/title preserved for provenance. The public paste form mirrors this: a prose-wrapped/fenced Appwrite row paste stays intact for the API instead of unwrapping the stringified reading into nonsense fields. The decision `brief.quickGlance.sourceTrace` now repeats the winning item's source section/id/title/quote, and both `brief.source.originalPromptExcerpt` / `handoff.source.originalPromptExcerpt` or, when the original prompt is unavailable, `sourceSummaryExcerpt` carry the source context so a downstream Scattermind handoff can show why the build order exists without digging through `input.provenance`. Scattermind should use these when a next move came from a specific Concept Map lens sentence, so Ranker can defend not just what wins but where the judgement came from.
+Candidate trace note: candidate-level `sourceItemId` / `traceId`, `sourceTitle` / `lensTitle`, and `sourceExcerpt` / `sourceQuote` are preserved in ranked items, `buildOrderDetails`, and `handoff.itemTrace`. Lens-only Build Order text is also split into deterministic `concept-map.lenses.channel#N` source IDs with the original labelled sentence carried as `sourceQuote`, so pasted paid Concept Maps remain traceable even without explicit candidate objects. String items in laned Build Order arrays now also receive deterministic section-local source IDs such as `concept-map.buildOrder.validateNext#1` and carry the original string as `sourceQuote`, so simple Scattermind exports stay addressable downstream instead of becoming anonymous `feature-1` rows. Ranker also accepts the current Scattermind storage-row shape with `referenceCode`, `ideaText`, `context`, and string-valued `fullReadingJson` / `full_reading_json`; it expands the saved paid Concept Map before ranking so operators do not have to hand-copy lenses out of Appwrite rows. It also accepts private reading/API envelopes with object-valued `reading` / `fullReading` and `glimpse`, preserving `referenceCode` and `initialPrompt` while treating `reading` as the paid Concept Map. Stringified bridge envelopes in fields such as `rankerInput`, `rankerBridge`, `rankReady`, `bridgePayload`, or `continuationPlan` are expanded the same way, so Appwrite/string-copy handoffs do not have to be manually unwrapped before ranking. If the row only has free Snapshot data (`glimpseJson` / `glimpse_json` / `snapshotJson`), Ranker expands that Snapshot into a minimal continuation order: one manual proof plus the first evidence question, with the Snapshot reference code/title preserved for provenance. The public paste form mirrors this: a prose-wrapped/fenced Appwrite row paste stays intact for the API instead of unwrapping the stringified reading into nonsense fields. The decision `brief.quickGlance.sourceTrace` now repeats the winning item's source section/id/title/quote, and both `brief.source.originalPromptExcerpt` / `handoff.source.originalPromptExcerpt` or, when the original prompt is unavailable, `sourceSummaryExcerpt` carry the source context into the bridge handoff.
 
 Soft Scattermind labels are accepted at the bridge boundary so Scattermind does not need to use harsh verdict copy in its own product surface. Lens text can say `Continue first`, `Make tangible`, `Try next`, `Evidence next`, `Hold for later`, or `Set aside`, with either colon or reader-friendly dash separators (`Continue first: …`, `Continue first — …`, `Evidence next - …`). Build Order objects and direct bridge/envelope sections can use matching camel/snake-case keys such as `continueFirst`, `evidenceNext`, `holdForLater`, and `setAside`. Ranker maps those to `doFirst / validateNext / defer / park` while preserving the softer original label in `sourceQuote` or candidate source trace. Ranker also accepts softer continuation envelopes named `rankerBridge`, `continuation`, or `continuationPlan`, candidate arrays named `possibleNextMoves`, `suggestedNextMoves`, `recommendations`, or `opportunities`, laned `buildOrderPreview` / `build_order_preview` objects, and evidence-question fallback arrays (`evidenceQuestions` / `evidence_questions`, `decisionQuestions`, `questionsToAnswer`, `followupQuestions`). If a paid Concept Map has no labelled Build Order/action threads but does include `closing_note` / `closingNote` plus decision questions, Ranker treats the closing note as the active 48-hour Do first move and keeps the questions in Validate next. This lets Scattermind pass reader-friendly Concept Map copy without renaming everything into software-feature language.
 

scripts/check-rank-feedback.mjs

@@ -960,6 +960,45 @@ try {
   assert.ok(scattermindPaidShape.ranked.find(item => item.id === 'build-order-4').metrics.nonGoalConflicts.length >= 1);
   assert.deepEqual(scattermindPaidShape.handoff.warnings, []);
 
+  const privateReadingEnvelopeResponse = await fetch(`${base}/api/rank-feedback`, {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({
+      ok: true,
+      referenceCode: 'SM-PRIVATE1',
+      initialPrompt: 'I need to know what to build after Scattermind clarified my local workshop idea.',
+      reading: {
+        working_name: 'Local Workshop Starter',
+        opening_reflection: 'The paid Concept Map says to prove one tiny workshop manually before building signup machinery.',
+        lenses: {
+          risk: { title: 'What Can Mislead You', content: 'Avoid accounts, saved calendars, and payment dashboards until one workshop has real interest.' },
+          question: { title: 'Proof Steps', content: 'Proof steps: Ask five local makers if they would attend the first workshop. Run a manual invite and record yes/no replies.' },
+          channel: { title: 'Build Order', content: 'Build first: Manual workshop invite - ask five real makers before building signup UI. Test manually: One-page workshop promise - collect replies. Defer: Polished calendar after one committed group. Probably noise: Account dashboard with saved calendars and subscriptions.' },
+        },
+        questions_to_sit_with: ['Will five real makers commit to the first session?'],
+        closing_note: 'In the next 48 hours, send the manual workshop invite to five named makers.',
+        reference_code: 'SM-PRIVATE1',
+      },
+      glimpse: {
+        working_name: 'Local Workshop Starter Snapshot',
+        restated_idea: 'A local workshop idea that needs proof before software.',
+        questions_to_sit_with: ['Who would attend first?'],
+      },
+    }),
+  });
+  assert.equal(privateReadingEnvelopeResponse.status, 200);
+  const privateReadingEnvelope = await privateReadingEnvelopeResponse.json();
+  assert.equal(privateReadingEnvelope.input.provenance.artifactId, 'SM-PRIVATE1');
+  assert.equal(privateReadingEnvelope.input.provenance.snapshotTitle, 'Local Workshop Starter');
+  assert.match(privateReadingEnvelope.input.provenance.originalPrompt, /Scattermind clarified/);
+  assert.equal(privateReadingEnvelope.input.optionCount, 6);
+  assert.equal(privateReadingEnvelope.ranked[0].id, 'build-order-1');
+  assert.equal(privateReadingEnvelope.ranked[0].provenance.sourceSection, 'concept-map.lenses.channel');
+  assert.equal(privateReadingEnvelope.ranked.find(item => item.id === 'proof-step-1').lane.id, 'test');
+  assert.equal(privateReadingEnvelope.ranked.find(item => item.id === 'build-order-4').lane.id, 'park');
+  assert.ok(privateReadingEnvelope.input.decisionContext.nonGoals.includes('Avoid accounts, saved calendars, and payment dashboards until one workshop has real interest'));
+  assert.deepEqual(privateReadingEnvelope.handoff.warnings, []);
+
   const softLabelLensResponse = await fetch(`${base}/api/rank-feedback`, {
     method: 'POST',
     headers: { 'Content-Type': 'application/json' },

server.js

@@ -557,6 +557,10 @@ function looksLikeRankPayload(value = {}) {
     || value.snapshot
     || value.conceptMap
     || value.concept_map
+    || value.reading
+    || value.fullReading
+    || value.full_reading
+    || value.glimpse
     || value.buildOrder
     || value.build_order
     || value.lenses
@@ -782,6 +786,7 @@ function expandStoredScattermindReading(body = {}) {
     || original.full_reading_json
     || original.fullReading
     || original.full_reading
+    || original.reading
     || original.conceptMapJson
     || original.concept_map_json
     || '';
@@ -791,11 +796,13 @@ function expandStoredScattermindReading(body = {}) {
   if ((!parsedReading || !Object.keys(parsedReading).length) && (!parsedSnapshot || !Object.keys(parsedSnapshot).length)) return original;
 
   const explicitSnapshot = objectFrom(original.snapshot || original.glimpse);
+  const explicitConceptMap = objectFrom(original.conceptMap || original.concept_map || original.reading || original.fullReading || original.full_reading);
   const expanded = {
     ...parsedSnapshot,
     ...parsedReading,
     ...original,
     snapshot: Object.keys(explicitSnapshot).length ? explicitSnapshot : parsedSnapshot,
+    conceptMap: Object.keys(explicitConceptMap).length ? explicitConceptMap : parsedReading,
     lenses: original.lenses || parsedReading.lenses || parsedSnapshot.lenses,
     threads_to_hold: original.threads_to_hold || original.threadsToHold || parsedReading.threads_to_hold || parsedReading.threadsToHold || parsedSnapshot.threads_to_hold || parsedSnapshot.threadsToHold,
     questions_to_sit_with: original.questions_to_sit_with || original.questionsToSitWith || parsedReading.questions_to_sit_with || parsedReading.questionsToSitWith || parsedSnapshot.questions_to_sit_with || parsedSnapshot.questionsToSitWith,
@@ -1013,8 +1020,9 @@ function cleanContextText(value = '') {
 }
 
 function meaningfulTokens(text = '') {
-  const stop = new Set(['the', 'and', 'with', 'from', 'that', 'this', 'into', 'before', 'after', 'until', 'user', 'users', 'idea', 'ideas', 'build', 'order', 'works', 'first', 'avoid', 'without', 'more', 'full', 'proof', 'manual', 'value', 'layer', 'result', 'sense', 'copyable', 'source', 'traced', 'source-traced', 'evidence', 'thread', 'threads']);
+  const guardrailScope = String(text || '').replace(/\b(until|unless|after proof|after one|after the|once)\b[\s\S]*$/i, '');
-  return [...new Set(String(text).toLowerCase().match(/[a-z0-9][a-z0-9-]{3,}/g) || [])].filter(token => !stop.has(token)).slice(0, 8);
+  const stop = new Set(['the', 'and', 'with', 'from', 'that', 'this', 'into', 'before', 'after', 'until', 'unless', 'once', 'user', 'users', 'idea', 'ideas', 'build', 'order', 'works', 'first', 'avoid', 'without', 'more', 'full', 'proof', 'manual', 'value', 'layer', 'result', 'sense', 'copyable', 'source', 'traced', 'source-traced', 'evidence', 'thread', 'threads', 'real', 'interest']);
+  return [...new Set(guardrailScope.toLowerCase().match(/[a-z0-9][a-z0-9-]{3,}/g) || [])].filter(token => !stop.has(token)).slice(0, 8);
 }
 
 function isGuardrailAvoidanceMention(lowerText = '', token = '') {
@@ -1552,7 +1560,7 @@ function scoreOption(option, mode, context = '', decisionContext = {}) {
   const normalizedLaneHint = normalizeLaneHint(laneHint);
   const conflicts = nonGoalConflicts(`${option.title} ${option.description}`, decisionContext);
   const nonGoalPenalty = Math.min(14, conflicts.length * 7);
-  const laneBoost = /do|first|now|build/.test(laneHint) ? 1.35 : /validate|test|proof/.test(laneHint) ? 0.35 : /defer|park|cut/.test(laneHint) ? -0.75 : 0;
+  const laneBoost = /do|first|now|build/.test(laneHint) ? 3.1 : /validate|test|proof/.test(laneHint) ? 0.25 : /defer|park|cut/.test(laneHint) ? -0.75 : 0;
   const lanePenalty = normalizedLaneHint === 'park' ? 18 : normalizedLaneHint === 'defer' ? 9 : 0;
   const heuristicMetrics = {
     value: Math.min(10, 4.8 + hits(text, wordSets.value) * 0.9 + coreLoopHits * 0.8 + bridgeHits * 0.75 + proofHits * 0.2 + hits(context, wordSets.value) * 0.15),