Hermes Labs

Research document

Construct validity and problem transformation

A source record supporting Project Fiction. Read the original account alongside its scope, dates, and limitations.

Research status
active methodological direction; operational definitions and experiments remain unresolved.
Integrated
In this record

Scientific record

Source and recovery limits

The user requested integration of Fiction Research Program. Retrieved source excerpts (not included in this public selection: ../../docs/handovers/2026-10-23-fiction-research-program-source.md) preserve the available connected-chat text and message IDs; the adjacent JSON preserves the tool response data. The shared page also exposed the initial hypothesis-map discussion and a first-cycle report. Its Cycle 2 and Cycle 3 assistant report bodies displayed “This response couldn’t load”; the connector returned unresolved content references for them. The uploaded file and images were not inspected. This integration does not claim a complete conversation export or recovery of the missing memoranda.

The strongest source anchors are user message [private source identifier withheld] (Cycle 2 assignment) and [private source identifier withheld] (Cycle 2 review / Cycle 3 direction). Their text was read in both the shared page and connected chat. Record dates are integration dates; historical timestamps are retained in the source.

Accepted methodological direction

SUPPORTED BY EXISTING FICTION MATERIAL: Outcome-based innovation measurement and the mechanism producing an outcome are separate questions. The source user explicitly accepts this distinction. Define the phenomenon independently before interpreting search, analogy, recombination, restructuring, or their coordination as its explanation. These processes may constitute the capability; Fiction does not require a unique cognitive primitive.

The canonical research question and working definition remain unchanged. CWR remains one experimental branch, not Fiction's definition. Human construct/label/signal, model capability, runtime usefulness, reward suitability, and transfer retain separate evidence requirements.

Working direction: Investigate both (A) novel functional solutions to an externally supplied goal and (B) valuable transformation of the problem representation, objective, or actionable opportunity. These are measurement scopes to investigate, not established faculties or replacements for EXP-0001/2/3. They also do not rename the runtime/reward branches, which earlier conversation text called A/B.

UNRESOLVED: Definition B — Situated Functional Innovation — is explicitly not adopted as Fiction's complete operational definition. The source permits provisional fixed-goal outcome use, subject to validity testing. Its full definition, examples and rubric are in an unrecovered memorandum, so this record does not reconstruct them or make an executable measure available.

Measurement obligations

Distinguish four variables: externally supplied information; initial goal; participant/system problem representation; and permitted operations plus actual environmental constraints. Representation change does not itself introduce external facts or authorize rewriting the world's rules. Record what is held fixed, what is manipulated, and residual differences.

Keep novelty relative to demonstrations, individual prior knowledge, model pretraining, and wider knowledge separate. Supplied-evidence novelty does not establish novelty in pretraining or in the world. In finite worlds, rule-valid feasibility alone does not establish broader usefulness. Functional reproduction in AI and human-like mechanism require different evidence.

For problem transformation, justify value independently of the actor's chosen goal. Avoid rewarding easier self-selected goals, arbitrary substitution, or constraint violations. Control unequal search freedom and difficulty. A representation change with no observable behavioral consequence cannot establish benefit merely from its description. Whether delayed or currently unmeasurable benefits can be studied defensibly remains open.

Source proposals and missing work

NEW PROPOSAL, not adopted protocols: The browser-readable first-cycle report proposes human recombination feasibility, computational search baselines, and search-versus-representation comparisons. It distinguishes external, representational and computational closure. These remain source proposals; no new mechanism, numerical threshold, budget, or execution is adopted here. Its literature claims have not been independently reverified in this integration.

The Cycle 2 assignment asks for three competing definitions, examples/counterexamples, a prior-art comparison, non-circular measurement, and a small construct-validity experiment. The Cycle 3 assignment asks for problem-finding/problem-construction literature and a fixed-goal versus permitted-reformulation comparison. Geneplore, MemiC, analogical transfer, representational change, Getzels/Csikszentmihalyi, Reiter-Palmon/Mumford, and Boden are source-requested research leads, not newly verified evidence in this note. The existing baseline retains its actual reading-depth limits.

CV-001: source-reported proposal; incomplete local record. The user's review describes an original sixteen-case measurement diagnostic and requests its retention and revision. The original stimuli, complete protocol, numerical criteria and report body were not recovered. Do not equate CV-001 with Fiction Experiment 001 or EXP-0001, fabricate its sixteen cases, or treat designer categories as independent ground truth.

The requested adversarial coverage includes trivial novelty, effective exhaustive search, behaviorally inert representation changes, useless new goals, beneficial reformulations, and apparent innovation that violates constraints. Objective checks and independent human judgment must have explicit roles. These are requirements for future design/recovery, not completed stimulus validation.

Reconciliation and next decision

The early conversation centered the classifier and a construction-versus-retrieval distinction. Its later handover and the current canon supersede that ontology with retrieval as substrate and reuse versus transformation. Preserve the early formulation as history, not a revived label scheme.

The first-cycle report's phrasing about distinguishing innovation from ordinary search can suggest that a different mechanism is required. The subsequent user clarification explicitly removes that requirement. Outcome validity and mechanistic explanation must be tested separately.

The pilot's existing operational retrieval/deduction/recombination comparisons remain useful controls. They do not license excluding an otherwise qualifying outcome merely because search, deduction, or analogy produced it. Whether the pilot's measures capture more than constrained problem solving remains an unresolved construct-validity question.

Next: recover the missing Cycle 2/3 memoranda and CV-001 materials, compare their claims with this direction and existing gates, then prepare the smallest discriminating design with hypothesis, operational definition, rivals, controls, success/failure criteria and next decision. This source integration does not execute those historical assignments. No empirical result, gate passage, recruitment, external model call or paid experiment follows from adding the conversation to canon.

Decision: DEC-0007 (not included in this public selection: ../../decisions/decision-log.md#dec-0007--integrate-fiction-construct-validity-direction).

2026-10-23 — Subsequent review and execution

The new supplied review (not included in this public selection: experiments/pt-001/USER-REQUEST.txt) provisionally accepts fixed-goal versus valuable problem transformation as an experimental distinction and explicitly authorizes local feasibility execution. Superseded next-step restriction: recovery of the missing memoranda is no longer a prerequisite for this bounded authorized rehearsal; the source gaps remain. PT-001 is the new local realization, with actual software outputs and independent bounded verification. It does not reconstruct the original missing protocol. Outcome-based reframing attribution fails against objective/search alternatives; no human or AI cognitive evidence is claimed. The earlier source-integration record remains accurate about what was then retrieved.

Source History & Provenance

Publication and source

Public record
Selected public export · Scientific maturity is recorded separately.
Repository source
projects/fiction/construct-validity-direction.md
Website snapshot
2026-10-25 (UTC)
Source date
2026-10-23 (Integrated)
Version
No explicit version field is recorded in the source.

Download public Markdown

This is a publication-safe export. Private conversation and message identifiers withheld; source reference labels retained. Scientific statements and source reference labels are retained. The unmodified canonical source remains in the laboratory repository.

Byte integrity

Download SHA-256: f95e610afcc24f55811da91179110aa2c6f2bbf3d748ad0ed2753d8bd90415e7

The checksum covers the exact downloadable bytes, not the rendered HTML. It supports byte comparison; it is not a signature or an independent validation of the research.

Canonical source SHA-256: faad4d98515bb09c6d21fc821f73b8242f881e1113daf6f2a3aa69435b1a3c83

This second checksum identifies the internal, unmodified source; those bytes are not included in this public export.

Historical statements retain their original scope and dates. A source discussion or internal decision does not establish an empirical result. Unselected or restricted records remain unavailable in this export.