Harden Ranker summary guardrails
This commit is contained in:
@@ -47,7 +47,7 @@ Ranker's continuation job is narrow:
|
||||
|
||||
`POST /api/rank-feedback` accepts a `prioritix-feature-set-v1`-style payload from Scattermind and returns ranked items plus `buildOrder.doFirst / validateNext / defer / park`. It accepts candidate arrays as `features`, `actions`, `nextMoves`, `candidates`, `experiments`, `validationTests`, or `proofTests` either at the top level or under `featureSet`, and it can consume nested Snapshot or Concept Map artifacts directly, so Scattermind can hand off `snapshot.nextActions / nextMoves` or `conceptMap.nextActions / nextMoves` without renaming them into fake software features. Experiment/proof-test arrays default into Validate next candidates, preserving source provenance while keeping them out of the active build-first lane. Sectioned Concept Maps may also include `validateNext`, `deferred`, and `parkingLot`; Ranker combines those sections into one build-order pass while preserving `sourceSection` and treating deferred/parked sections as lane hints. Ranker also accepts an already-laned Build Order object at top level, under `featureSet`, or as `conceptMap.buildOrder` (`doFirst` / `validateNext` / `deferred` / `parkingLot`, with aliases like `buildNow`, `testManually`, and `probablyNoise`). String items in those lanes are normalized into candidate actions, while object items can use `move`, `questionToAnswer`, or `evidenceQuestion` aliases. Normalized candidate objects may also use `lane` as a lane hint (`do-first`, `validate-next`, `defer`, `park`) without overwriting `sourceSection`, so Scattermind exports that separate recommendation from provenance remain traceable. Empty wrapper arrays are ignored rather than allowed to shadow a real nested Concept Map action set, and non-empty normalized wrappers are merged with Concept Map validation/deferred/parking sections rather than dropping that context. That keeps partially-normalized Scattermind exports rankable without losing the source lane contract. It also returns a `brief` with source, next-48-hour actions, carried-forward assumptions, source trace, and what would change the ranking, plus `handoff.copyableText`: a plain-text Build Order handoff with Do first / Validate next / Defer / Park, readiness, carried context, source trace, and the explicit rule that only the Do first item is active. This gives Scattermind (or a tired copy/paste user) a stable artifact to save without reconstructing the decision from JSON fields. Lane-level `buildOrderDetails` still carry each item's reason, next step, evidence question, success/kill signals, source section, lane source, score, and confidence, while the `handoff` object (`rank-feedback-result-v1`) carries source provenance, item trace rows, explicit next-step/success/kill signals when Scattermind provides them, copyable text, and contract warnings for missing artifact IDs, source sections, original prompt provenance, or evidence on active items. If Scattermind sends the current paid Concept Map shape as lenses rather than arrays, Ranker can parse `conceptMap.lenses.channel.content` / `buildOrder` labels (`Build first`, `Test manually`, `Defer`, `Probably noise`) into rank-ready candidates and can read guardrails from the risk lens. Keep this contract action-first; do not use it as a reason to add generic dashboard, auth, billing, or workspace layers before the bridge has proof.
|
||||
|
||||
Candidate items may include optional 1–10 `rankerHints` (`value`, `effort`, `confidence`, `urgency`, `revenue`, `novelty`, `risk`). Ranker blends those explicit Scattermind hints with text heuristics; `effort` is inverted into feasibility. Hints improve the defended order, but `recommendedLane: "defer"` or `"park"` remains a safety rail so dashboard-swamp items do not get promoted by flashy wording. For clean bridge handoff, Scattermind should send `sourceSection` and `evidenceNeeded` on each active next move. Scattermind can also send `targetAudience`, `constraints`, `assumptions`, and `nonGoals` / `avoid` at the top level, in `featureSet`, inside `snapshot.context` or `conceptMap.context`, or as a structured top-level `context` object with `summary`, `targetAudience`, `constraints`, `nonGoals` / `avoid`, and `assumptions`; Ranker merges these sources rather than letting a shallow wrapper context shadow deeper artifact guardrails. Lens-only Concept Maps may additionally send audience / constraints / assumptions / risk lens content, and Ranker will split sentence-style lens notes into readable decision context instead of leaking `[object Object]`. If Scattermind only has a flat context string, Ranker now extracts simple guardrails such as `Solo builder`, `Manual proof`, `Avoid ...`, `No ...`, and `Do not ...` into `input.decisionContext` / `handoff.decisionContext` so early bridge exports still protect against dashboard/auth/billing drift. Ranker returns that decision context in `input.decisionContext` and `handoff.decisionContext`, and penalizes candidates that conflict with source non-goals (for example saved workspaces/auth/billing before the continuation proof). If Scattermind sends duplicate candidate IDs, Ranker keeps the first ID, suffixes later duplicates (`preview-2`), and reports the normalization in `handoff.warnings` / `handoff.itemTrace` so downstream build-order references remain addressable. Ranker also accepts Scattermind's paid Concept Map object directly when it arrives with top-level `reference_code`, `working_name`, `ideaText`, and string-valued `lenses.channel` / `lenses.risk` fields; the reference code becomes source provenance, the working name becomes the source title, and labelled Build Order text is turned into rank-ready candidates without requiring Scattermind to wrap it first.
|
||||
Candidate items may include optional 1–10 `rankerHints` (`value`, `effort`, `confidence`, `urgency`, `revenue`, `novelty`, `risk`). Ranker blends those explicit Scattermind hints with text heuristics; `effort` is inverted into feasibility. Hints improve the defended order, but `recommendedLane: "defer"` or `"park"` remains a safety rail so dashboard-swamp items do not get promoted by flashy wording. For clean bridge handoff, Scattermind should send `sourceSection` and `evidenceNeeded` on each active next move. Scattermind can also send `targetAudience`, `constraints`, `assumptions`, and `nonGoals` / `avoid` at the top level, in `featureSet`, inside `snapshot.context` or `conceptMap.context`, or as a structured top-level `context` object with `summary`, `targetAudience`, `constraints`, `nonGoals` / `avoid`, and `assumptions`; Ranker merges these sources rather than letting a shallow wrapper context shadow deeper artifact guardrails. Lens-only Concept Maps may additionally send audience / constraints / assumptions / risk lens content, and Ranker will split sentence-style lens notes into readable decision context instead of leaking `[object Object]`. If Scattermind only has a flat context string or schema-light structured context summaries (`context.summary`, `snapshot.context.summary`, `conceptMap.context.summary`, etc.), Ranker now extracts simple guardrails such as `Solo builder`, `Manual proof`, `Avoid ...`, `No ...`, `Non-goal: ...`, `Not yet ...`, and `Do not ...` into `input.decisionContext` / `handoff.decisionContext` so early bridge exports still protect against dashboard/auth/billing drift. Ranker returns that decision context in `input.decisionContext` and `handoff.decisionContext`, and penalizes candidates that conflict with source non-goals (for example saved workspaces/auth/billing before the continuation proof). If Scattermind sends duplicate candidate IDs, Ranker keeps the first ID, suffixes later duplicates (`preview-2`), and reports the normalization in `handoff.warnings` / `handoff.itemTrace` so downstream build-order references remain addressable. Ranker also accepts Scattermind's paid Concept Map object directly when it arrives with top-level `reference_code`, `working_name`, `ideaText`, and string-valued `lenses.channel` / `lenses.risk` fields; the reference code becomes source provenance, the working name becomes the source title, and labelled Build Order text is turned into rank-ready candidates without requiring Scattermind to wrap it first.
|
||||
|
||||
Candidate trace note: candidate-level `sourceItemId` / `traceId`, `sourceTitle` / `lensTitle`, and `sourceExcerpt` / `sourceQuote` are preserved in ranked items, `buildOrderDetails`, and `handoff.itemTrace`. Lens-only Build Order text is also split into deterministic `concept-map.lenses.channel#N` source IDs with the original labelled sentence carried as `sourceQuote`, so pasted paid Concept Maps remain traceable even without explicit candidate objects. String items in laned Build Order arrays now also receive deterministic section-local source IDs such as `concept-map.buildOrder.validateNext#1` and carry the original string as `sourceQuote`, so simple Scattermind exports stay addressable downstream instead of becoming anonymous `feature-1` rows. The decision `brief.quickGlance.sourceTrace` now repeats the winning item's source section/id/title/quote, and both `brief.source.originalPromptExcerpt` / `handoff.source.originalPromptExcerpt` or, when the original prompt is unavailable, `sourceSummaryExcerpt` carry the source context so a downstream Scattermind handoff can show why the build order exists without digging through `input.provenance`. Scattermind should use these when a next move came from a specific Concept Map lens sentence, so Ranker can defend not just what wins but where the judgement came from.
|
||||
|
||||
|
||||
@@ -1010,7 +1010,40 @@ try {
|
||||
assert.equal(nextStepsAlias.ranked.find(item => item.id === 'saved-workspace').lane.id, 'park');
|
||||
assert.deepEqual(nextStepsAlias.handoff.warnings, []);
|
||||
|
||||
console.log(JSON.stringify({ ok: true, top: data.ranked[0].id, hintedTop: hinted.ranked[0].id, actionTop: actions.ranked[0].id, nestedConceptTop: nestedConcept.ranked[0].id, nonGoalTop: nonGoal.ranked[0].id, structuredContextTop: structuredContext.ranked[0].id, lensOnlyTop: lensOnly.ranked[0].id, scattermindPaidShapeTop: scattermindPaidShape.ranked[0].id, mergedContextTop: mergedContext.ranked[0].id, embeddedJsonTop: embeddedJson.ranked[0].id, fencedJsonTop: fencedJson.ranked[0].id, embeddedSnapshotTop: embeddedSnapshot.ranked[0].id, sourceExcerptTop: sourceExcerpt.ranked[0].id, snakeCaseBridgeTop: snakeCaseBridge.ranked[0].id, nextStepsAliasTop: nextStepsAlias.ranked[0].id, duplicateIds: duplicateIds.ranked.map(item => item.id), readiness: data.handoff.readiness.status, provenance: data.input.provenance, buildOrder: data.buildOrder }, null, 2));
|
||||
const summaryGuardrailResponse = await fetch(`${base}/api/rank-feedback`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
sourceName: 'Scattermind',
|
||||
artifactId: 'context_summary_guardrails',
|
||||
originalPrompt: 'Scattermind exported guardrails in context summaries rather than explicit nonGoals arrays.',
|
||||
mode: 'mvp',
|
||||
context: {
|
||||
summary: 'Solo builder. Manual proof first. No account dashboards or saved workspaces before the first useful result.',
|
||||
targetAudience: 'Tired non-AI-native solo builder',
|
||||
},
|
||||
conceptMap: {
|
||||
context: {
|
||||
summary: 'Avoid billing, auth, and collaboration until the bridge proves one defended build order.',
|
||||
},
|
||||
nextActions: [
|
||||
{ id: 'manual-summary-proof', action: 'Manual summary-context build order preview', why: 'Show one defended first move from a Concept Map.', evidenceNeeded: 'Can one tired user name the first move?', suggestedLane: 'do-first', rankerHints: { value: 9, effort: 2, confidence: 8, urgency: 8, risk: 2 } },
|
||||
{ id: 'workspace-summary', action: 'Saved workspace and account dashboard', why: 'Auth-backed saved projects and collaboration for every idea.', evidenceNeeded: 'No bridge proof yet.', rankerHints: { value: 10, effort: 1, confidence: 10, urgency: 10, risk: 1 } },
|
||||
{ id: 'billing-summary', action: 'Billing and subscription admin', why: 'Checkout, invoices, plans, and account controls.', evidenceNeeded: 'No buyer proof yet.', rankerHints: { value: 10, effort: 1, confidence: 10, urgency: 10, risk: 1 } },
|
||||
],
|
||||
},
|
||||
}),
|
||||
});
|
||||
assert.equal(summaryGuardrailResponse.status, 200);
|
||||
const summaryGuardrail = await summaryGuardrailResponse.json();
|
||||
assert.ok(summaryGuardrail.input.decisionContext.nonGoals.includes('No account dashboards or saved workspaces before the first useful result'));
|
||||
assert.ok(summaryGuardrail.input.decisionContext.nonGoals.includes('Avoid billing, auth, and collaboration until the bridge proves one defended build order'));
|
||||
assert.equal(summaryGuardrail.ranked[0].id, 'manual-summary-proof');
|
||||
assert.equal(summaryGuardrail.ranked.find(item => item.id === 'workspace-summary').lane.source, 'source-non-goal');
|
||||
assert.equal(summaryGuardrail.ranked.find(item => item.id === 'billing-summary').lane.source, 'source-non-goal');
|
||||
assert.deepEqual(summaryGuardrail.handoff.warnings, []);
|
||||
|
||||
console.log(JSON.stringify({ ok: true, top: data.ranked[0].id, hintedTop: hinted.ranked[0].id, actionTop: actions.ranked[0].id, nestedConceptTop: nestedConcept.ranked[0].id, nonGoalTop: nonGoal.ranked[0].id, structuredContextTop: structuredContext.ranked[0].id, lensOnlyTop: lensOnly.ranked[0].id, scattermindPaidShapeTop: scattermindPaidShape.ranked[0].id, mergedContextTop: mergedContext.ranked[0].id, embeddedJsonTop: embeddedJson.ranked[0].id, fencedJsonTop: fencedJson.ranked[0].id, embeddedSnapshotTop: embeddedSnapshot.ranked[0].id, sourceExcerptTop: sourceExcerpt.ranked[0].id, snakeCaseBridgeTop: snakeCaseBridge.ranked[0].id, nextStepsAliasTop: nextStepsAlias.ranked[0].id, summaryGuardrailTop: summaryGuardrail.ranked[0].id, duplicateIds: duplicateIds.ranked.map(item => item.id), readiness: data.handoff.readiness.status, provenance: data.input.provenance, buildOrder: data.buildOrder }, null, 2));
|
||||
} finally {
|
||||
server.kill('SIGTERM');
|
||||
}
|
||||
|
||||
@@ -181,7 +181,7 @@ function guardrailsFromContextText(value = '') {
|
||||
for (const sentence of contextSentences(value)) {
|
||||
const cleaned = cleanText(sentence, 180);
|
||||
if (!cleaned) continue;
|
||||
if (/^(avoid|no|do not|don't|dont|must not|never)\b/i.test(cleaned)) nonGoals.push(cleaned);
|
||||
if (/^(avoid|no|do not|don't|dont|must not|never|non-goal|non goal|not yet|out of scope)\b/i.test(cleaned)) nonGoals.push(cleaned.replace(/^non[- ]goal\s*:\s*/i, ''));
|
||||
else if (/\b(avoid|no auth|no account|no billing|no workspace|not a dashboard|without accounts|before proof|manual proof|solo builder|constraint)\b/i.test(cleaned)) constraints.push(cleaned);
|
||||
}
|
||||
return { nonGoals: uniqueList(nonGoals), constraints: uniqueList(constraints) };
|
||||
@@ -636,6 +636,12 @@ function firstContextText(sources = [], aliases = []) {
|
||||
return '';
|
||||
}
|
||||
|
||||
function contextGuardrailText(value = '') {
|
||||
if (!value || typeof value !== 'object' || Array.isArray(value)) return cleanMultiline(value || '', 3000);
|
||||
const obj = objectFrom(value);
|
||||
return cleanMultiline(obj.summary || obj.description || obj.notes || obj.brief || obj.text || '', 3000);
|
||||
}
|
||||
|
||||
function cleanDecisionContext(input = {}) {
|
||||
const featureSet = objectFrom(input.featureSet || input.feature_set);
|
||||
const artifact = objectFrom(input.artifact || featureSet.artifact);
|
||||
@@ -660,7 +666,11 @@ function cleanDecisionContext(input = {}) {
|
||||
artifact.context,
|
||||
];
|
||||
const textContextGuardrails = guardrailsFromContextText([
|
||||
typeof input.context === 'string' ? input.context : '',
|
||||
contextGuardrailText(input.context || ''),
|
||||
contextGuardrailText(featureSet.context || ''),
|
||||
contextGuardrailText(artifact.context || ''),
|
||||
contextGuardrailText(snapshot.context || ''),
|
||||
contextGuardrailText(conceptMap.context || ''),
|
||||
lensContent(riskLens),
|
||||
lensContent(constraintsLens),
|
||||
conceptMap.risk || conceptMap.whatNotToBuildYet || conceptMap.notYet || '',
|
||||
|
||||
Reference in New Issue
Block a user