Jump to text
On this page
Copy and source
Published source confirmed at last check

Source changed 2026-11-18 21:11:39 UTC · snapshot created 2026-11-18 21:12:35 UTC · last check 2026-11-18 22:30:10 UTC

E.22 - Improvement-Oriented Quality Evaluation Question Framing

Status: Stable.

E.22:1 - Problem frame

Use E.22 when a request such as “review this” leaves it unclear what the recipient needs to learn about an object version. A floor check, a search for further improvement and a check of applied findings can require different answers about the same object.

First useful move: state what the recipient will use the answer for, identify the object version and choose the judgement that is still missing. Then name the criterion, the grounds the evaluator must examine and the answer to return. Section 4.0 develops this short construction; section 5 gives a filled request.

When those bindings are already settled and one direct evaluation is enough, run that evaluation. Rewriting its question as a separate frame adds no answer. Use E.23 when repeated improvement across passes is needed.

What can go wrong. A reviewer can answer a different question, count closed findings without judging the changed object, or propose improvements that damage a quality the recipient needs.

Practical gain. Requester and evaluator agree on the question before doing the evaluation. A floor question can stay small. A request for alternatives can name the additional trade-offs and proposals it needs.

Framing yields a question about the selected object version. The evaluator must obtain the quality result. The recipient must make any decision that uses it.

E.22:2 - Problem

Quality evaluations fail when the evaluator has to infer the question. The same object can be checked for floor adequacy, evaluated for possible improvement toward exceptional expression, compared across trade-offs, examined for open questions, or evaluated after finding absorption. Those purposes produce different findings.

When a question is left unframed, the evaluator must guess which object is under improvement, which evaluation supplies its values and what kind of result that evaluation may return.

E.22:3 - Forces

ForceTension
Cheap readiness vs ambitious improvementA floor evaluation should be short; exceptional improvement needs richer proposals.
Explicit purpose vs reviewer discoveryThe request names the purpose, while the reviewer can report important unasked questions.
Evaluation vs follow-up actionA useful evaluation may suggest a follow-up, but the suggestion remains a hypothesis until someone applies the pattern that defines or constrains the claim, relation or boundary.
Multi-coordinate gain vs Goodhart riskRaising one visible value can damage usability, affordability, locality, source preservation, or corpus ecology; use E.13 when the visible value or metric is being treated as the intended value.
Proposal portfolio vs selected resultSeveral candidate improvements may be useful without becoming a selected set, pool policy, front insertion, parity, or refresh result.

E.22:4 - Solution

E.22 gives requester and evaluator a compact way to declare the quality question that will guide improvement. The evaluator answers the question. A decision or work product beyond the authority of that evaluation must be obtained separately.

E.22:4.0 - Form a usable first request

Start from the result the recipient needs. The recipient may need a document to explain an operation, the operation to obtain a result, and the result to inform a further decision. Select the object and judgement needed now. A candidate’s narrower description cannot establish that the larger receiving need has been satisfied.

  1. Name the use and object version. Say who needs the answer, what work or decision it will inform, and which version is being evaluated. Include the conditions that affect that use. If the receiving need is unclear, recover it with the recipient or the applicable problem-formulation method.
  2. Choose the question and criterion. Say whether the recipient needs floor adequacy, further improvement, a comparison of trade-offs or the effect of applied findings. Use the characteristic space and predicate or comparator supplied by an adequate evaluation. State the floor or improvement aim and the qualities a proposed repair must preserve.
  3. Connect the criterion to attainable grounds. Identify the content, observations or reasoning needed to answer it. Tracing a source contribution into an instruction can establish a coherent construction. Trying the instruction on material can reveal a missing action. An observed application can support a performance claim under the conditions observed. State which answer is required, what the intended reader or performer can supply, and what the candidate must supply. If indispensable grounds are absent, ask for the exact unresolved condition rather than an unsupported pass.
  4. Specify the useful return. Name the evaluation’s result form and the conclusion it can support. A floor answer identifies sufficiency or the first needed repair; an improvement answer also describes the proposed change, expected gain, protected trade-offs and closure test. State evaluator conditions only when they change the question or admissibility of the answer.

When several contributions are needed, retain what their available evaluations establish and ask about the necessary connection still unanswered. A good explanation, a supported source claim and a correct constituent operation can all remain valid while their combined use is unestablished. Evaluate that connection for its own object and use. Combining unlike scores cannot establish quality for the whole.

Write the request in ordinary language with these bindings recoverable, as in section 5. The local names and schemas below specify their exact meanings when the question is represented as a QualityEvaluationQuestionFrame and QualityEvaluationUseDeclaration. Reuse the selected evaluation’s adequate descriptions and defaults. Add an intended evaluator’s identity or a dated-Work account only when the receiving question requires it.

Stop framing when the question and expected answer are sufficiently determined for the intended evaluation. If an existing request already supplies them, perform the direct evaluation and inspect whether its grounds support the consuming conclusion. A separate frame is useful when it resolves a missing or disputed binding. Restating settled bindings adds no answer.

E.22:4.1 - Local names and kind settlement

E.22 frames the question. The evaluator judges the object. evaluationPatternLocator identifies the FPF pattern description containing the evaluation predicate or constraint; an optional semanticEvaluationMethodRef names the separately identified U.Method used for that evaluation.

A characteristic-space specification, Q-Bundle description, rubric description, review-profile description, evidence-basis description, and result-form description constrain or describe that evaluation. The subject assertion and any semantic Method retain their independent bases.

For example, E.21, E.9.DA, or E.2.DA may supply the predicate for evaluating one FPF object, while A.19.ECS and C.25 supply supporting quality-model descriptions. E.19 supplies admission or refresh review questions and findings. Apply E.19 when that review is the requested result. When its findings support a quality evaluation, evaluationPatternLocator names the pattern that defines that evaluation, and the quality result cites the relied-on review findings as evidence.

A below-floor value, finding, improvement aim, or need for evaluation is not by itself an actual Problem. If the consuming use relies on an actual Problem, cite one current C.22.PFR ProblematicForRelation occurrence with its direct participants and temporal identity. The frame, evaluation, result and evidence may support a claim about that occurrence. None creates or splits it.

The framing episteme, evaluation method, descriptions used by that method, any question-changing evaluator condition, dated evaluation Work, actual operation application, evidence use, and result have distinct functions. QualityEvaluationUseDeclaration collects the applicable evaluation bindings. A plan, declaration or named candidate does not establish a current performer or occurrence.

The local support names ending in @Context serve only compatibility and retrieval. The suffix supplies no context entity, scope, participant, relation, or identity component. Every episteme follows C.2.1 identity, every set is identified by its stated extensional rule, and every neighboring Work, decision, evidence, viewpoint, grounding, or result relation remains under its direct governor.

Local nameKind and use in this pattern
QualityEvaluationQuestionFrameU.Episteme whose EntityOfConcern is the exact object version under evaluation; its ClaimGraph carries the requested quality-evaluation question about that version and its exact use bindings.
QualityEvaluationUseDeclarationU.Episteme whose EntityOfConcern is the same object version. It describes how evaluation of that version is intended to be performed and interpreted, referring separately to the evaluation pattern, optional semantic Method, selected characteristic space, predicate and any comparator, ClaimScope, quality-model descriptions, expected evidence basis, result form, and qualification window. It may state an eligibility, independence, capability or planned condition for the evaluator when that condition changes the question or admissibility of the result. It may name an intended evaluator only when that identity is part of the question. It contains no actual performer, assignment, or Work occurrence.
ObjectVersionUnderQualityEvaluationExact U.Entity version being evaluated, paired with its exact U.Kind.
EvaluationCharacteristicSpaceSelectionOne exact U.CharacteristicSpace selected for this evaluation use. The episteme that specifies the space has its own identity.
EvaluationCriterionSelectionThe exact by-value CharacteristicSpacePredicate, exact admitted ComparatorSpecRef, or both, required by the governing evaluation pattern and, when declared, its separately identified semantic evaluation Method. At least one is present.
EvaluationClaimScopeOne exact set-valued U.ClaimScope governing the evaluation claim. It is not a context label, selected structure, window, or evidence set.
QualityEvaluationResultConsumingUseThe specific intended-work, dated-work or decision object expected to use the result, under the rule that directly governs that object, paired with its exact kind and use description. Naming it neither authorizes nor performs that use.
QualityEvaluationPurposeSelectionRequested evaluation purpose or distinguishable combination of purposes.
DeclaredQualityFloorMinimum acceptable coordinate or status floor when the frame declares a floor claim.
DesiredImprovementAimRequested substantive change beyond the floor when improvement beyond the floor is requested.
ExpectedEvaluationEvidenceBasis@ContextU.Episteme whose EntityOfConcern is the exact object version under evaluation. It describes expected evidence-use positions and the missingness rule for the exact method, space, criterion, scope, and qualification window. It can be identified before a use declaration cites it. The description and the evidence values later found are distinct.
TradeoffProtectionSet@ContextA mathematical set of references to protected characteristics or coordinates, each paired with its kind. The declaration specifies the permitted elements. The question-frame edition determines which pairs are admissible for this use. Its members identify the set extensionally.
EvaluationQualificationWindowEdition, source-currentness, comparison-set, time, or declared-use window in which the requested result is intended to be current. The actual evaluation application later binds its exact point or interval.
ExpectedQualityEvaluationResultFormDescriptionU.Episteme describing the result-row form declared by the governing evaluation pattern. It is not an actual result.
QualityReviewFindingRowActionable evaluation finding that identifies the observed issue, affected evaluation property, correction direction, and closure test.
CandidateImprovementProposalRow@ContextE.22 proposal episteme with an exact correction target, expected substantive evaluation effect, trade-offs, kind-restoration disposition, outside-claim return when needed, and closure test.
CandidateImprovementOutsideClaimReference@ContextBounded local ClaimGraph node form inside one proposal row. It identifies the outside governed value, relation signature, or boundary description and the exact FPF pattern identity that governs the return. It belongs to its proposal row and cannot be referenced independently. It is neither an episteme nor a relation.
KindRestorationCheckConditionally present check when a changed FPF-governed expression can alter the object, kind, relation, slot or use position, claim kind, admissible use, or scope.
CandidateImprovementProposalPortfolio@ContextA mathematical set whose local element range consists of CandidateImprovementProposalRow@Context epistemes addressing one question frame. Resolve each proposal under C.2.1 and the proposal conditions in 4.4. The frame determines which proposals may belong and how the set may be used. The set is identified by its members.
ImprovementFollowUpHypothesis@ContextU.Episteme whose EntityOfConcern is the exact object version expected to change. It claims that one named next operation or method application is expected to address one finding and produce a stated evaluation effect under a stated test condition. A stop disposition, return, selected plan, performed Work, or actual Transformation is not such a hypothesis.

In each filled episteme form below, claimGraph carries its complete interpreted claims and semantic relations by value, in the C.2.1:4.1.2 range NonemptyClaimContent. referenceScheme carries the effective rule collection by value in EpistemeInterpretationScheme; those rules must interpret the form’s designations, predicates and qualifications. The remaining fields state the claims of that episteme. Range membership alone establishes neither their truth nor a completed evaluation.

QualityEvaluationUseDeclaration <: U.Episteme:
  entityOfConcernRef: U.EntityRef, referencing the exact object version under evaluation
  entityOfConcernKindRef: U.KindRef, referencing the exact kind of that object version
  claimGraph: NonemptyClaimContent by value
  referenceScheme: EpistemeInterpretationScheme by value
  editionId
  evaluatorConditionRef?: U.EpistemeRef, referencing an eligibility, independence, capability, or planned condition only when it changes the evaluation question or admissibility of the result
  intendedEvaluatorSystemRef?: U.EntityRef, referencing one admitted U.System only when that identity is part of the declared question; it asserts neither assignment nor performance
  evaluationPatternLocator: U.EntityRef, locating the exact FPF pattern description that contains the defining or constraining ClaimGraph
  semanticEvaluationMethodRef?: U.MethodRef, referencing the separately identified U.Method used for the evaluation
  selectedEvaluationCharacteristicSpaceRef: U.EntityRef, referencing one exact U.CharacteristicSpace
  selectedEvaluationPredicate?: CharacteristicSpacePredicate by value
  selectedComparatorSpecRef?: ComparatorSpecRef
  evaluationClaimScopeRef: U.EntityRef, referencing one exact U.ClaimScope
  evaluationQualificationWindowDescriptionRef: U.EpistemeRef, referencing one EvaluationQualificationWindow description
  evaluationCharacteristicSpaceSpecDescriptionRef?: U.EpistemeRef, referencing one A.19.ECS specification description
  evaluationQBundleDescriptionRef?: U.EpistemeRef, referencing one C.25 Q-Bundle description
  evaluationRubricDescriptionRef?: U.EpistemeRef, referencing one evaluation-rubric description
  evaluationReviewProfileDescriptionRef?: U.EpistemeRef, referencing one evaluation-review-profile description
  expectedEvaluationEvidenceBasisRef: U.EpistemeRef, referencing one ExpectedEvaluationEvidenceBasis@Context
  expectedEvaluationResultFormDescriptionRef: U.EpistemeRef, referencing one ExpectedQualityEvaluationResultFormDescription

ExpectedEvaluationEvidenceBasis@Context <: U.Episteme:
  entityOfConcernRef: U.EntityRef, referencing the exact object version whose evaluation needs the expected evidence
  entityOfConcernKindRef: U.KindRef, referencing the exact kind of that object version
  claimGraph: NonemptyClaimContent by value
  referenceScheme: EpistemeInterpretationScheme by value
  editionId
  evaluationPatternLocator: U.EntityRef, locating the same FPF evaluation pattern description
  selectedEvaluationCharacteristicSpaceRef: U.EntityRef, referencing the same U.CharacteristicSpace
  selectedEvaluationPredicate?: CharacteristicSpacePredicate by value
  selectedComparatorSpecRef?: ComparatorSpecRef
  evaluationClaimScopeRef: U.EntityRef, referencing the same U.ClaimScope
  expectedEvidencePositionDescriptionRefs[1..*]: U.EpistemeRef, each referencing one evidence-position description
  expectedEvidenceRelationKindRefs[1..*]: U.KindRef, each referencing one expected evidence-relation kind
  missingEvidenceDispositionRuleRef: U.EpistemeRef, referencing one exact episteme that states the missing-evidence disposition rule under its subject pattern
  qualificationWindowDescriptionRef: U.EpistemeRef, referencing one EvaluationQualificationWindow description

Every field above with a *Ref suffix stores the stated A.6.5 RefKind; resolving it yields the referent kind named after referencing. The use declaration and expected evidence basis carry the same object version, governing evaluation pattern, selected characteristic space, criterion binding, ClaimScope, and qualification window.

Construct the expected basis from those values, the expected evidence positions and relation kinds, and the missingness rule. Then construct the declaration with a reference to the completed basis. The expected basis has no back-reference to the declaration. This keeps the construction acyclic.

At least one of selectedEvaluationPredicate and selectedComparatorSpecRef is present. Both may be present. A label such as review, quality, or current context supplies neither. A.19 defines the predicate by value. For comparator admission, use A.19.CPM or the rule of the consumer that directly admits the comparator. Identify any actual comparison application separately. Selecting the predicate or comparator does not by itself establish the evaluation scope, evidence, time, Work, or result.

evaluatorConditionRef states only a condition that changes the evaluation question or admissibility of its result. intendedEvaluatorSystemRef is present only when the declared question depends on that exact intended System. Neither field establishes assignment or performance.

The actual evaluator System, every obtaining assignment, and dated evaluation Work belong to the separately identified evaluation application or result account. Keep any local evaluator system-role classification separate and route unresolved role wording through E.10.ROLE.

evaluationPatternLocator locates the pattern that defines or constrains the evaluation; it is not the Method, performer, Work, or result. Claim Method or MethodDescription identity only after A.3.1 and A.3.2 admit it. Characteristic-space, Q-Bundle, rubric, profile, evidence-basis, and result-form references identify separate descriptions.

None of these declaration fields is dated evaluation Work or an evaluation result. A pre-evaluation frame does not assert that the future assessment has occurred. It may reference independently obtaining Work as the checked object, consumer or prior basis, retaining that Work’s admission and temporal conditions. An ordinary result that asserts no actual Work needs no Work identifier.

If a compact projection does assert dated evaluation Work, recover every exact actual performer through A.13 and follow A.15.1 for independent Work admission. The performer, Method, time, containing System, Work identity, and the result relation remain recoverable. Add assignment and F.6 refs only when the projection or receiving use expressly represents precise assignment-bound attribution. Missing or failed F.6 alone leaves independently admitted Work intact. Recheck any admission premise invalidated by the same finding and its dependent conclusions under A.15.1.

Keep any durable result episteme, evidence use, provenance, currentness, viewpoint, grounding, and Work-to-result or decision-use relation under their own patterns. A frame, declaration, description, assignment, dashboard, or carrier establishes none of them.

Two carriers may publish the same edition of either episteme. A QualityEvaluationUseDeclaration changes edition when its object version, claim graph, reference scheme, question-changing evaluator condition or intended-evaluator identity, evaluation pattern, semantic Method, selected characteristic space, predicate and comparator, ClaimScope, qualification window, quality-model descriptions, expected evidence-basis edition, or result-form description changes. Replacing one qualified actual evaluator with another does not change the declaration unless the declared condition or claim changes.

An ExpectedEvaluationEvidenceBasis@Context changes edition when its object version, claim graph, reference scheme, evaluation pattern, selected space, predicate and comparator, ClaimScope, expected evidence positions or relation kinds, missingness rule, or qualification window changes. Carrier, context label, viewpoint, grounding record, or support serialization alone changes neither episteme.

The two local set names above require only their declared element ranges and ordinary extensional equality. Permuting a list or repeating an entry leaves its represented set unchanged. A changed question-frame edition changes the admissibility or use question, not the set identity when its members remain the same. A changed proposal episteme contributes a different member under C.2.1, even if its short label is retained. An episteme may describe or publish a set. Its record and carrier have their own identities. These set-valued uses require no additional public set kind.

E.22:4.2 - Quality evaluation purposes

Purpose valueUse whenExpected result
floorEvaluationThe question is whether the object reaches a declared floor.Values below floor, first repair, architecture hold, refresh, new-frame assignment, or admissible stop.
exceptionalImprovementEvaluationThe floor is reached and the requester wants non-dominated improvement toward exceptional expression.Per-coordinate proposal or no-candidate disposition.
paretoTradeoffEvaluationA candidate change may improve some values while worsening protected qualities.Trade-off account and non-dominated comparison.
candidateImprovementProposalEvaluationThe requester needs candidate-change proposals before changing the object or generating variants.Proposal row or bounded proposal portfolio with an expected effect on the later evaluation result.
openQuestionDiscoveryEvaluationThe requester wants important unasked questions surfaced.Question classified as existing-coordinate issue, candidate future coordinate, or outside-evaluation issue.
absorptionEvaluationReturned findings or suggestions have been applied or rejected.Quality-impact account over the changed object.

A request can combine purposes. The result distinguishes the answer to each one. A floor result does not answer exceptional improvement. Absorption count does not establish a changed evaluation result. A proposal is not a selected work item.

E.22:4.3 - Question frame

Use the construction in section 4.0 to settle the requested judgement before expressing these bindings. The schema preserves that question for its declared use.

An improvement aim is not a command to make every coordinate exceptional. Only the named evaluation can assign a 5, after the changed object earns it.

The frame may ask for substantive non-dominated proposals that could move named coordinates toward exceptional expression, while admitting no proposal or stay at current value when every plausible change would add apparatus, proof prose, boundary catalogues, or process evidence while damaging protected qualities. Support that no-proposal result with the review locations checked and references to its evidence basis.

QualityEvaluationQuestionFrame <: U.Episteme:
  entityOfConcernRef: U.EntityRef, referencing the exact object version under evaluation
  entityOfConcernKindRef: U.KindRef, referencing the exact kind of that object version
  claimGraph: NonemptyClaimContent by value
  referenceScheme: EpistemeInterpretationScheme by value
  editionId
  qualityEvaluationUseDeclarationRef: U.EpistemeRef, referencing one QualityEvaluationUseDeclaration about the same object version
  selectedEvaluationCharacteristicSpaceRef: U.EntityRef, referencing the same U.CharacteristicSpace
  selectedEvaluationPredicate?: CharacteristicSpacePredicate by value
  selectedComparatorSpecRef?: ComparatorSpecRef
  evaluationClaimScopeRef: U.EntityRef, referencing the same U.ClaimScope
  resultConsumingUseRef: U.EntityRef, referencing one exact directly governed intended-work, dated-work, or decision object
  resultConsumingUseKindRef: U.KindRef, referencing its exact kind
  resultConsumingUseDescriptionRef: U.EpistemeRef, describing how that work or decision will use the evaluation result
  evaluationPurposeSelection: QualityEvaluationPurposeSelectionValue
  declaredQualityFloorDescriptionRef?: U.EpistemeRef, referencing one declared-quality-floor description
  desiredImprovementAimDescriptionRef?: U.EpistemeRef, referencing one desired-improvement-aim description
  tradeoffProtectionSet: TradeoffProtectionSet@Context by value
  evaluationQualificationWindowDescriptionRef: U.EpistemeRef, referencing one EvaluationQualificationWindow description
  nonUseBoundaryDescriptionRef?: U.EpistemeRef, referencing a non-use-boundary description only when the grounded condition in CC-E22-6 holds

The frame’s exact object version, characteristic space, predicate/comparator binding, ClaimScope, and qualification window equal those of its use declaration and expected evidence basis. These bindings let another evaluator replay the question. The space, predicate, comparator, scope, method and consuming object retain their own identities. A changed binding creates a changed frame edition and requires a newly evaluated result.

resultConsumingUseRef is not a generic use placeholder. Before the intended work or decision occurs, the reference may resolve to one A.15.2 U.WorkPlan that names the intended Work, or to the decision question or decision-governing object under its directly governing pattern. It may resolve to U.Work only when that dated Work already obtains under A.15.1. The frame neither creates the consuming Work or decision nor authorizes it.

A minimal floor frame names the object version, one QualityEvaluationUseDeclaration, the exact selected characteristic space, applicable predicate and/or comparator, ClaimScope, result-consuming work or decision, purpose floorEvaluation, and the declared floor.

The declaration may cite defaults supplied by the governing evaluation pattern for its quality-model descriptions, evidence basis, result form, and qualification window, but defaults do not replace the exact selected space, criterion, scope, or consumer.

If the question depends on another edition, source state, comparison set, time window, or declared use, state that window explicitly. For one FPF pattern version under E.21, compactness never permits omitted coordinates, missing ShortRationale, absent PrecisionRestorationProfile, scope narrowing, or a blocker-only substitute result.

Connect the question, grounds and supported conclusion. State what the evaluation must establish about the object for the declared consuming use. In the expected evidence basis, identify the judgement that connects the selected criterion to the relevant content. For example, ask whether a Method’s conditions permit the proposed action, whether a computation yields the stated result, or whether a source supports the claim made from it. A direct presence or identity check suffices when the question asks about that property.

When receiving the result, read its rationale and observations for that connection. Determine whether they support the requested conclusion under the declared criterion, scope and qualification. Retain supported findings. Identify which judgement or inference is missing before claiming more. Obtain that answer through the applicable evaluation, reusing current supported parts and keeping the answer in its existing result. Apply the same connection to an already-scoped direct evaluation. It needs no new QualityEvaluationQuestionFrame.

These instructions use the frame’s existing claimGraph, the declaration’s expectedEvaluationEvidenceBasisRef and the declared consuming use. The expected evidence basis describes what must be answered; the obtained grounds belong in the evaluation result.

For landing-input, corpus-facing, Stable, release, external-review or another stated use, evaluate the object under the scope for which the floor was requested. If another use is to be evaluated, open a new QualityEvaluationQuestionFrame. An easier scope cannot make the current request pass.

The frame and declaration state the evaluation’s intended use. Naming an intended evaluator or planned condition establishes neither a current assignment nor Work. When dated evaluation Work is asserted, recover the exact evaluator through A.13 and use A.15.1 to admit the Work independently. Keep evidence use, typed result binding or direct result relation, and optional result episteme separate. Add F.6 only when the evaluation account expressly consumes precise assignment-bound attribution. An expected result-form description is not the result, and the consuming work or decision does not become current merely because the frame names it.

E.22:4.4 - Finding and proposal rows

An actionable finding identifies where an issue was observed, which exact entity would change, the affected evaluation characteristic or coordinate, the current evaluation result for that characteristic or coordinate when known, the proposed correction, and the closure test. A proposal adds a typed expected evaluation effect, protected trade-offs, and any outside claim. For each outside claim, include the subject-pattern locator needed for an independent check.

CandidateImprovementProposalRow@Context <: U.Episteme:
  entityOfConcernRef: U.EntityRef, referencing the exact object version under improvement
  entityOfConcernKindRef: U.KindRef, referencing the exact kind of that object version
  claimGraph: NonemptyClaimContent by value
  referenceScheme: EpistemeInterpretationScheme by value
  editionId
  qualityEvaluationQuestionFrameRef: U.EpistemeRef, referencing one QualityEvaluationQuestionFrame about the same object version
  evaluationClaimScopeRef: U.EntityRef, referencing that frame's U.ClaimScope
  reviewLocationDescriptionRef: U.EpistemeRef, referencing one description of the observed location in the reviewed object
  correctionTargetRef: U.EntityRef, referencing the exact entity proposed to change
  correctionTargetKindRef: U.KindRef, referencing the exact kind of the correction target
  affectedEvaluationCharacteristicOrCoordinateRef: U.EntityRef, referencing one governed characteristic or evaluation coordinate
  affectedEvaluationCharacteristicOrCoordinateKindRef: U.KindRef, referencing its exact kind
  currentAffectedEvaluationResultRef?: U.EntityRef, referencing the current result value for that characteristic or coordinate
  currentAffectedEvaluationResultKindRef?: U.KindRef, referencing the exact kind of that result value
  expectedSubstantiveEvaluationEffect: ProposalEvaluationEffectValue
  proposedCorrectionDescriptionRef: U.EpistemeRef, referencing one correction description
  kindRestorationCheckDisposition: ProposalKindRestorationCheckDispositionValue
  kindRestorationCheckRef?: U.EpistemeRef, referencing one KindRestorationCheck result
  expectedTradeoffRefs[]: U.EpistemeRef, each referencing one expected-trade-off description
  outsideClaimReferences[]?: CandidateImprovementOutsideClaimReference@Context by value
  closureTestRef: U.EpistemeRef, referencing one closure-test description

CandidateImprovementOutsideClaimReference@Context in CandidateImprovementProposalRow@Context.claimGraph:
  outsideClaimOrBoundaryDescriptionRef: U.EpistemeRef, referencing one description of the outside claim or boundary
  outsideValueRef?: U.EntityRef, referencing the exact outside governed value
  outsideValueKindRef?: U.KindRef, referencing the exact kind of that outside value
  outsideRelationSignatureRef?: U.EntityRef, referencing the exact RelationSignature declaration episteme when its reusable declaration is used
  subjectPatternLocator: U.EntityRef, locating the exact FPF subject-pattern description; the evaluation claim separately cites the defining ClaimGraph
  reconsiderationConditionDescriptionRef: U.EpistemeRef, referencing one description of the condition that activates renewed use of that subject pattern
ImprovementFollowUpHypothesis@Context <: U.Episteme:
  entityOfConcernRef: U.EntityRef, referencing the exact object version expected to change
  entityOfConcernKindRef: U.KindRef, referencing the exact kind of that object version
  claimGraph: NonemptyClaimContent by value
  referenceScheme: EpistemeInterpretationScheme by value
  editionId
  qualityEvaluationQuestionFrameRef: U.EpistemeRef, referencing one QualityEvaluationQuestionFrame about the same object version
  evaluationClaimScopeRef: U.EntityRef, referencing that frame's U.ClaimScope
  qualityReviewFindingDescriptionRef: U.EpistemeRef, referencing one episteme that describes the exact QualityReviewFindingRow
  proposedNextOperationDescriptionRef?: U.EpistemeRef, referencing one operation description
  proposedNextMethodRef?: U.MethodRef, referencing one U.Method
  expectedEvaluationEffectDescriptionRef: U.EpistemeRef, referencing one expected-evaluation-effect description
  testConditionDescriptionRef: U.EpistemeRef, referencing one test-condition description

Exactly one of proposedNextOperationDescriptionRef and proposedNextMethodRef is present. The question frame, proposal row, and follow-up hypothesis preserve the same object-version EntityOfConcern and ClaimScope unless a proposal explicitly opens a new frame for a different version or scope. QualityEvaluationQuestionFrame changes edition when the object version, use declaration, selected space, predicate/comparator binding, ClaimScope, consuming work or decision, purpose, floor or aim, trade-off set, qualification window, non-use boundary, claim graph, or reference scheme changes.

A proposal row changes edition when its frame, ClaimScope, correction target, affected evaluation coordinate, current result reference, proposed correction, expected effect, trade-offs, outside-claim nodes, closure test, claim graph, or reference scheme changes.

A follow-up hypothesis changes edition when its frame, ClaimScope, finding description, proposed operation or method, expected effect, test condition, claim graph, or reference scheme changes. Changing only a context label, carrier, viewpoint, grounding record or serialization leaves all three epistemes unchanged.

ProposalEvaluationEffectValue is the closed local value set repairFloor | raiseTowardExceptional | preventProtectedQualityLoss | classifyOutsideEvaluation | preserveCurrentValue. It identifies the broad type of substantive effect the proposal is expected to have on the evaluation. E.23 gives a separate coordinate-qualified prediction in ExpectedEvaluationResultChange@Context. Neither prediction asserts that the result has changed.

ProposalKindRestorationCheckDispositionValue is triggered | notTriggered | ordinaryProse | alreadySatisfied | blocker. The triggered and blocker states include kindRestorationCheckRef; the other values leave it absent.

Current affected-evaluation result ref and kind are both present or both absent. When present, the exact result resolves through the direct evaluation pattern’s typed result relation or A.6.1 application binding, and any durable result episteme remains separately governed. The proposal row neither produces nor reidentifies that result. The exact kind recovers whether the named evaluation returned a scale value, status, or another admitted result for that characteristic or coordinate.

Outside value ref and kind are paired. When the outside value is a relation, recover its direct predicate, participants and applicability from the subject pattern. Add outsideRelationSignatureRef only when the proposal uses that reusable declaration. The relation occurrence and declaration remain distinct under A.6.REL.

CandidateImprovementOutsideClaimReference@Context is a bounded local ClaimGraph node form, not a U-kind, episteme, relation, or relation-reference episteme. It is constructed inside one proposal row without a back-reference to that row. Its node identity is determined by the containing proposal edition and ClaimGraph position.

reviewLocationDescriptionRef describes where the issue was observed in the reviewed object. correctionTargetRef identifies the exact entity that would change. Keep the observed location distinct from the correction target even when both concern the same object. The row is a faithful typed proposal form of QualityReviewFindingRow and one possible member of a CandidateImprovementProposalPortfolio@Context set. It describes a proposal. Selection, planning, performed Work, actual Transformation, result binding and proof of improvement each need their own grounds.

For wording, naming, and precision-restoration proposals, proposedCorrectionDescriptionRef states the correction and its intended content effect. Apply F.19 for the ordinary repair and local revalidation. When the changed FPF-governed expression can alter meaning, KindRestorationCheck states the live object, kind, relation, slot or use position, claim kind, admissible use, and scope before and after the change. If the proposed repair cannot preserve those values and no accepted decision justifies changing them, the proposal row remains blocking.

E.22:4.5 - Absorption impact values

Absorption impactMeaning
coordinateImprovedA named coordinate or status has stronger content evidence after the change.
floorOnlyClosureA below-floor defect was repaired enough for the floor but not exceptional expression.
unchangedBecauseAlreadySatisfiedThe suggestion was already satisfied. Name the review locations and the evaluation property they already satisfy.
tradeoffIntroducedA repair raised one property and damaged another.
qualityLossDetectedThe applied or proposed change lowers a value or protected quality.
outsideObjectUnderImprovementEvaluationThe suggestion belongs under another exact evaluation or pattern.
notAdmissibleForDeclaredUseThe suggestion is rejected for the declared purpose and boundary.

In the absorption result, state how the object’s quality changed or why it remained the same, using the evaluation of that object. Counting accepted rows does not establish that result.

E.22:4.6 - OEE and NQD proposal portfolios

When the object is a candidate, archive or front member, selected set, parity report, refresh report, or declared transformation result, use E.22 to frame the quality question and return proposal rows. Use C.17 for candidate characteristics, C.18 for archive and front relations, C.19 for pool policy, G.5 for selected-set result declaration, G.9 for parity, and G.11 for currentness and refresh. When the question concerns availability to an audience, use E.17 for the source-backed publication face and return to source. Use E.24.PUB for the publication occurrence, form, carrier, audience, bounded use and availability.

E.22:5 - Worked slices

Floor evaluation: from “review the studio method” to a usable request. A studio practitioner needs to decide whether to use C.39 to develop the recording-preparation method in C.39:5.3. “Review the studio method” leaves three possible objects: the pattern’s explanation, the conditional preparation it describes, and an actually prepared recording. Here the practitioner needs the first judgement: whether the complete pattern provides usable development guidance. Let M1 name the immutable complete C.39 edition supplied with this illustrative request.

Evaluate the attached C.39 edition M1 for my decision whether to use its guidance to develop the studio preparation in C.39:5.3. The intended user is a practitioner familiar with FPF and studio editing who can obtain the recording engineer’s interval qualification. The pattern must explain how to construct and connect the needed operations, identify missing prerequisites and reconsider affected uses after a change.

Use the E.21 nineteen-coordinate characteristic space and its floor predicate: every required coordinate at least 4 for this declared use. Evaluate the complete pattern. Examine its instructions, conditions, worked reasoning and necessary direct suppliers; use the studio case to trace what the reader can actually obtain. Preserve the distinction between the engineer’s premises and the pattern’s contribution. A worked conditional calculation supports a claim about the explained construction; it does not establish performed editing, practitioner learning or a better movement judgement.

Return the complete E.21 result: coordinate values with adjacent-value rationales, the precision-restoration profile, evidence basis and supported status. For a failed coordinate, identify the smallest sufficient repair and how to check it. Protect the useful source contribution and independently valid viewing use. This request concerns M1, the stated reader and studio use with the case’s supplied premises; it does not ask for a general release decision or exceptional-improvement proposals.

Once the requester and evaluator agree on this question, the evaluation can begin. Its purpose is floorEvaluation; the complete E.21 result remains to be obtained.

Suppose the evaluator observes that M1 explains the justified interval [40,45] ms, qualified editing and the 42.5 ms correction with a 2.5 ms bound. That observation supports the guidance claim about a conditional preparation meeting 3 ms analysis and 8 ms viewing requirements. Evaluate each remaining E.21 coordinate on its own grounds. If the interval qualification is unavailable in the receiving studio, obtain that missing basis. Three marker readings alone do not justify the edit.

Tightening the analysis requirement to 2 ms changes the preparation’s applicability: no constant correction can meet it for the given 5 ms offset span. The pattern’s instruction to return that dependent use for development remains useful, and the unchanged 8 ms viewing use remains qualified. A request to assess an actually edited recording would need that recording version and evidence about it. Where the receiving question is already the limited calculation under the supplied interval, evaluate that calculation directly. A full E.21 pattern evaluation and another frame add no answer to that calculation.

A method recovered from a source. A recipient needs a usable way to examine a proposal and decide how to continue. A draft faithfully summarizes a source’s discussion of criticism and correctly names proposal, evidence and action, but never explains how the critique changes the next choice. A text-quality result can support the summary’s fidelity; it cannot establish the missing explanation of how to obtain the result. The frame asks whether the candidate gives the intended reader that connection under the stated preparation and available grounds. C.39:5.4 develops this use.

An adequate answer may instead return a proposal worth testing, with the exact condition that prevents immediate enactment and a justified attainable next inquiry. Whether that is sufficient depends on the recipient’s requested result: it can identify the next inquiry and leave a request for completed enactment open. The evaluation must not reject every conditional answer as defective or silently lower the receiving requirement.

Deleting the whole source proposal over one unsupported inference can remove its useful propose–criticize–revise construction. Ask the evaluation to inspect both the repaired connection and the contribution it must preserve.

When the ground for one later application changes, retain the source-fidelity judgement and any unaffected use, and reopen the dependent application claim. If a proposed change to the evaluation itself was developed on these cases, use A.19.ECS:4.5 to compare it with adequate execution of the incumbent evaluation and examine an independently grounded new case before claiming broader improvement.

Exceptional improvement. A pattern already passes the floor. The frame asks for substantive non-dominated improvements for named coordinates while protecting usability and related-pattern fit. The result gives proposal rows for content improvements such as missing worked cases, source-currentness carry-through, mature-comparator discharge, deletion of displaced apparatus, or relation cleanup, and gives checked no-candidate dispositions for coordinates where no non-dominated content move remains. A 5 on every coordinate is not the aim.

Absorption. External review returns many suggestions. The frame asks for absorptionEvaluation. The result says which changes improved coordinates, which were already satisfied, which introduced trade-offs, and which belong outside the evaluation.

Proposal portfolio. A candidate improvement campaign needs alternatives before editing. The frame asks for candidateImprovementProposalEvaluation. The result returns bounded proposal rows. For example, a proposal to reduce latency and a proposal to reduce energy consumption remain two alternatives with their own trade-offs. Listing the first twice or swapping their display order leaves the proposal set unchanged. A separate protection set can keep both latency and energy coordinates visible. Selection or generation uses its own governing pattern.

Physical-system proposal. A vibration evaluation of PumpAssembly@Prototype-3 selects the vibration CharacteristicSpace, RMS-vibration predicate and any comparator, one evaluation ClaimScope over the declared operating-point slices, and the design decision that will consume the result.

This case asserts dated test-bench evaluation Work. First recover the exact evaluator through A.13. A.15.1 independently identifies the Work, Method, time and containing System. If this test-bench account also needs the exact assignment under which the evaluator acted, add F.6 through the same obtaining A.13 assignment. Missing or failed F.6 alone leaves independently admitted Work and its independently grounded result route intact.

An exact application of the declared vibration-evaluation operation returns the finding of excessive RMS vibration at one operating point under its result declaration. The result binding connects that application to the finding. The frame, subject-pattern reference, assignment, expected evidence basis, and result-form description remain separate.

The proposal’s reviewLocationDescriptionRef points to that evaluation row. Its correctionTargetRef points to ImpellerBladeGeometryDescription@v3, the design episteme that would change. The measurement row is the finding location. The affected coordinate is RMS vibration. The coarse proposal effect is repairFloor because the proposal addresses excessive vibration. kindRestorationCheckDisposition=notTriggered; the protected trade-offs are efficiency and manufacturability.

If the proposal is selected for a repeated loop, E.23 adds a scale-qualified ExpectedEvaluationResultChange@Context. Manufacturing a new impeller remains dated Work under A.15.1 rather than an E.22 result.

E.22:6 - Bias annotation

E.22 favors naming the intended quality question and what would answer it. Without that information, reviewers often give a plausible answer to the wrong question.

The quality values, repeated improvement, publication of a selected set, decision about work and certification of a project claim each require the operation that produces them.

E.22:7 - Conformance checklist

CheckPassing condition
CC-E22-1Name the exact object version, selected CharacteristicSpace, exact predicate and/or admitted comparator, one U.ClaimScope, and the exact work or decision that will consume the result.
CC-E22-2State purpose, declared floor or improvement aim, protected trade-offs, qualification window, and expected result form.
CC-E22-3Keep the object-under-improvement evaluation as the source of values and the coordinate set to be evaluated. A description, dashboard, or frame cannot substitute for the selected space, predicate/comparator, actual evaluation, or result.
CC-E22-4Represent actionable returned work as typed finding or CandidateImprovementProposalRow@Context values with expected substantive evaluation effect, closure test, and the conditionally present KindRestorationCheck. An outside claim cites its subject pattern; E.22 frames the improvement question and does not restate that ontology.
CC-E22-5For absorption, report how the changes affected the quality of the object. Applied and not-applied dispositions alone do not establish that effect.
CC-E22-6State a compact declarative non-use boundary only when an independently grounded reading by a plausible intended reader would turn the result into a different claim or authority. Keep the result on the evaluation question and name only the outside claim and the pattern that defines or constrains it when one is needed. Use the named evaluation profile and F.19 for precision-restoration or phrase-apparatus issues; they do not need a local boundary catalogue.
CC-E22-7State what became worse when a proposed or applied improvement raises visible values.
CC-E22-8Use E.23 for repeated improvement after one framed evaluation returns findings or proposals.
CC-E22-8aDo not frame 5, all-5, or 5-defensible as the work target. Frame below-floor repair separately from optional exceptional-improvement proposals. An optional proposal must identify a substantive content change. Allow checked no proposal or stay at current value only when further change would be dominated by apparatus growth, proof theatre, or protected-quality loss.
CC-E22-9Preserve the object-under-improvement evaluation’s required scope, evidence basis, complete coordinate set, rationales, and result form. For an E.21 result, include its compact PrecisionRestorationProfile under E.21:4.3a; checked evidence and coordinate-specific payloads retain their governing requirements.
CC-E22-10Keep the question frame separate from actual evaluation. An evaluator condition appears in the use declaration only when it changes the question or result admissibility. An intended evaluator identity appears only when that identity is declared. For actual dated Work, recover the evaluator through A.13 and admit the Work independently through A.15.1. The obtaining A.13 assignment remains part of the actual performer’s admission grounds. Add separate assignment and F.6 refs when the account expressly represents precise assignment-bound attribution. Missing or failed F.6 alone does not revoke independently admitted Work. Recheck shared admission premises as specified in section 4.1. Keep the application, result relation, result episteme, pattern locator, Method, any independently admitted method description, quality model, evidence-basis and result-form descriptions, evidence use, and result-consuming work or decision distinct.
CC-E22-11A low value, finding, failed floor, or improvement aim does not establish an actual Problem. Any actual Problem relied on by the consuming use resolves to one current C.22.PFR occurrence with its direct participants and temporal identity.

E.22:8 - Common anti-patterns and repairs

Anti-patternRepair
“Review this” prompt. The evaluator infers purpose.In a QualityEvaluationQuestionFrame, name the exact object version, space, criterion, ClaimScope, consumer, purpose and boundary.
Context-labelled frame. Project, domain, dashboard, cadence, or context label supplies identity or evaluation scope.Identify the frame by its C.2.1 claim content and exact EntityOfConcern; bind the exact U.ClaimScope and other use values separately.
Floor pass sold as excellence. Readiness is mistaken for exceptional improvement.State exceptionalImprovementEvaluation if wanted.
Frame replaces result. The question frame names a purpose but returns prose, a two-column value table, or proposal rows without the named evaluation’s result form.Complete the missing evaluation judgements and return the named evaluation’s declared coordinates, evidence basis, rationales, payload fields, and typed result binding or direct result relation, reusing supported parts under E.22:4.3. If the result asserts dated Work, recover the evaluator through A.13 and apply A.15.1 independently; add F.6 only for an expressly represented precise assignment-bound attribution. When the result asserts an A.6.1 application, name the operation bindings. The frame alone establishes none of these occurrences or results.
Description performs evaluation. A method description, characteristic-space specification, expected evidence basis, assignment occurrence, or result-form description is treated as evaluation Work or its result.Keep each description and assignment separate. Identify the direct evaluation result; when dated Work is asserted, recover the evaluator System through A.13, apply A.15.1 independently and add F.6 only when the receiving account expressly consumes precise assignment-bound attribution. Keep evidence use and result under their direct patterns.
Scope laundering. The frame asks one use, but the result answers an easier, local-only, diagnostic, or evaluator-selected use.Obtain the missing judgements under the requested U.ClaimScope, reusing any supported parts that answer it under E.22:4.3. Preserve a result for another use at its actual scope; frame that other use separately if it is now wanted.
Applied-count absorption. Closure count replaces re-evaluation of the changed object.Re-evaluate the changed object and classify the effect on its quality.
Goodharted improvement. Visible values rise while protected qualities worsen, or a 5 target makes the evaluator add apparatus instead of improving content.Frame the expected evaluation effect as a substantive content change, add trade-off protection, reject dominated changes, apply E.13 when a visible value replaces the intended value, and admit no proposal only when checked locations show that no worthwhile content improvement remains.
Recommendation as decision. A follow-up hypothesis is treated as chosen work.Open the exact decision, work, publication, parity, refresh, evidence, or assurance pattern if that claim is needed.
Finding as actual Problem. A low coordinate, finding, or floor miss is treated as a Problem occurrence.Keep the evaluation result epistemic; cite C.22.PFR only when its actual-condition and criterion-applicability participants make one ProblematicFor occurrence obtain.
Lexical repair request. A finding says only “replace this word” or “avoid that wording.”State the intended content effect and apply F.19. Add the before/after KindRestorationCheck when the changed expression can alter FPF-governed meaning; block a change of meaning that lacks justification.

E.22:9 - Consequences

ConsequenceBenefitCost
Review requests become typed.Evaluators answer the intended quality question.A complete request names the object and evaluation.
Exceptional improvement becomes explicit.Reviews can propose non-dominated improvements rather than stopping at floor defects.Each proposal names its protected trade-offs.
Absorption becomes quality-aware.The follow-up evaluation identifies what improved or worsened.The evaluator must check the changed object.

E.22:10 - Rationale

A generic request leaves the evaluator to choose what counts as quality. To show which question a quality result answers, bind its object version, selected characteristic space, predicate and any comparator, one evaluation ClaimScope, consuming work or decision, evaluation pattern, any separately identified semantic Method, purpose, expected evidence basis, expected result form, and boundary.

When needed, the question frame also states a question-changing evaluator condition or intended-evaluator identity. The pattern, Method, assignment, descriptions, dated evaluation Work, evidence use, result, decision and project authority each retain their own grounds.

E.22:11 - SoTA-Echoing

E.22:11.1 - Choosing the lightest sufficient question construction

Practice question. How should a requester obtain an evaluation question that will support the recipient’s actual next use, without adding another description when direct evaluation already suffices? Retain an adequate direct question. If it is inadequate, recover its missing purpose, object, criterion, grounds or use. Express exact E.22 bindings when they are needed to preserve that question across evaluators and consuming uses.

Two serious alternatives already provide purpose-first construction. Basili et al., Linking Software Development and Business Strategy Through Measurement (2010), especially “Background” and “GQM+Strategies”, links goals, derived measurement questions and interpretation to context and assumptions. This historical method contributes an organizational alignment model when the question requires that larger linkage.

Kidder et al., CDC Program Evaluation Framework, 2024, Steps 3–5, develops evaluation purpose, intended users and use, questions, design, credible evidence and supported conclusions. It expressly accommodates available resources and revision when needed evidence is infeasible. Their software-organization and program-evaluation settings supply no FPF characteristic values.

Comparison on the same receiving question. Keep the complete M1 edition, reader, studio use, available premises and E.21 criteria from section 5 fixed. The following is a worked adaptation of the methods to that question, not a measurement of practitioner time or error rates.

Adequate approachExecution on this caseSufficient result and burden
Direct E.21 evaluation of the filled requestThe evaluator receives M1, the studio use, nineteen-coordinate floor, expected grounds and complete result form. Evaluation can start immediately.It answers the stated quality question. A separate E.22 frame supplies no additional judgement and adds a description to maintain. Retain this direct execution.
Goal–question–measurement construction using the GQM contributionStarting with “review the studio method”, distinguish the goal of obtaining usable development guidance from the goal of qualifying a recording. Derive questions about that guidance and connect the chosen E.21 observations to the consuming decision.It can construct the same sufficient question. The work is clarification, selection and interpretation. Use a full organizational goal/strategy model only when organizational alignment is also required.
Purpose-and-use construction using the CDC contributionIdentify the studio practitioner as the user, the decision about using M1 as the purpose, and the complete pattern as the object. Choose an attainable examination of its guidance and premises, then require conclusions supported by those grounds.It can also construct the same sufficient question. Use its program-design, engagement and data-collection work when the receiving evaluation requires those contributions; they are not prerequisites of this one pattern-quality question.
E.22 construction and, where needed, exact declarationResolve the same ambiguity in section 4.0. Keep the question about M1 separate from the conditional preparation, the resulting evaluation and a later recording decision. Bind the declared criterion, scope, expected grounds and return form through sections 4.1–4.3 when those links must be exchanged precisely.The ordinary request can be the same under all four approaches. Exact bindings make a changed object, criterion or consuming use inspectable, at the cost of constituting and maintaining those references. They do not supply another evaluation result.

Choice and accepted trade-off. Adopt the purpose-to-question and use-to-evidence direction shared by the alternatives; adapt it to the FPF object, criterion and result distinctions. Sections 1 and 4.0 put that construction before schemas, section 4.3 retains the direct-evaluation exit and section 5 supplies the filled request and its changed-use boundary.

The local addition is an explicit account of those bindings, useful when exchanging a question otherwise risks evaluating another object or carrying a result beyond its scope. Take on the cost of describing and maintaining those bindings only when that precision is needed. When an adequate direct request or the rival’s question already preserves the same distinctions, reuse it. Neither a newer explanation nor the presence of a frame establishes better detection, lower cost or a reason to replace a sufficient evaluation.

The comparison selects an FPF adaptation for this question. It leaves the relative merits of GQM, CDC and FPF as whole frameworks open. Reopen when a competing construction preserves the needed question at lower total burden, when an intended reader cannot recover the first request, when a changed source or use defeats these transfers, or when exact binding introduces more maintenance than its receiving use warrants.

E.22:11.2 - Conditions that change the evaluation question

Use the following contributions when the question calls for actionable feedback, protection of trade-offs or evidence about the reliability of an automated judge. Each constrains the question under its own condition.

Practice questionExact source and statusSelected payload and domain limitSource-use decision, changed E.22 locus, qualification, and reopen
A rubric-level evaluation needs its own reliability check rather than trust in one aggregate judge verdict.Tianjun Pan et al., RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following, arXiv:2603.25133 (2026), and Hongli Zhou et al., Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization, arXiv:2603.08091 (2026), are current preprints for automated LLM judging.Pan et al. show that fine-grained rubric judging can remain inaccurate and variable; Zhou et al. test a taxonomy of twelve bias types across generative and discriminative judges. These works concern LLM judges and instruction-following benchmarks; they do not validate an FPF evaluation or generalize their numeric results to physical, medical, or organizational evaluation.Adapt — reason: use rubric-level variability and the twelve-bias taxonomy only to require reliability evidence when an automated LLM judge is selected; neither payload supports a general judge verdict. Changed loci: QualityEvaluationUseDeclaration, ExpectedEvaluationEvidenceBasis@Context, and the Floor evaluation and Exceptional improvement slices. Qualification/currentness as of 2026-08-19: the cited 2026 preprints apply only to their automated-judge and instruction-following settings. Reopen: a later benchmark or replication changes either payload, or an E.22 use adds, removes, or materially changes its LLM-judge branch.
Actionable formative feedback distinguishes the desired condition, current performance, and a move that can close the gap.D. Royce Sadler, Formative assessment and the design of instructional systems, Instructional Science 18, 119-144 (1989), DOI 10.1007/BF00117714; John Hattie and Helen Timperley, The Power of Feedback, Review of Educational Research 77(1), 81-112 (2007), DOI 10.3102/003465430298487. Both are historical formative-feedback sources, used here for their standard/current-work/action distinction.Sadler supplies the comparison between a quality standard and current work plus action by the learner; Hattie and Timperley synthesize goal, current progress, and next-step feedback questions. Their classroom evidence does not establish FPF kinds, project authority, or the quality of a proposed repair.Adapt — reason: use the standard/current-gap/action and goal/progress/next-step structures to keep aim, present result, and possible repair distinct; classroom evidence does not validate FPF evaluation. Changed loci: floor and aim bindings in the question frame, proposal/no-proposal result boundaries, and the Absorption slice. Qualification/currentness: these historical sources support the stated feedback structure; they do not establish its superiority to a sufficient direct evaluation or validate it across domains. Reopen: current formative-feedback evidence overturns that structure, or E.22 stops using it to change proposal, no-proposal, or absorption action.
Multi-coordinate improvement needs set-valued alternatives and explicit trade-offs rather than one scalar winner.Xi Lin et al., Quality-Diversity Optimization as Multi-Objective Optimization, arXiv:2602.00478 (2026), current preprint; Haoxiang Qin et al., A survey on Quality-Diversity optimization: Approaches, applications, and challenges, Swarm and Evolutionary Computation 100:102240 (2026), DOI 10.1016/j.swevo.2025.102240, current survey.Lin et al. reformulate QD as a large multi-objective problem and use set-based scalarization; Qin et al. survey high-performing collections over descriptor spaces. These algorithmic results do not assign FPF archive, front, publication, or selection authority.Adapt — reason: use set-valued alternatives and explicit descriptor/coordinate trade-offs, not the QD algorithms or any implied authority, because one scalar winner can hide protected-quality loss. Changed loci: paretoTradeoffEvaluation, TradeoffProtectionSet@Context, CandidateImprovementProposalPortfolio@Context, and the Proposal portfolio and Physical-system proposal slices. Qualification/currentness as of 2026-08-19: the cited 2026 preprint and survey apply to QD/MOO optimization. Reopen: current QD/MOO evidence changes the case for set-valued trade-offs, or E.22 begins asserting archive, front, publication, or selection authority.
Optimizing a measure can damage the intended value through several different mechanisms.Charles Goodhart, Problems of Monetary Management: The U.K. Experience (1975), historical monetary-control failure evidence; Donald T. Campbell, Assessing the Impact of Planned Social Change, Occasional Paper 8 (1976), historical social-indicator failure evidence; David Manheim and Scott Garrabrant, Categorizing Variants of Goodhart’s Law, arXiv:1803.04585 (2018), later taxonomy; Jongwoon Choi, Gary Hecht, and William Tayler, Lost in Translation: The Effects of Incentive Compensation on Strategy Surrogation, The Accounting Review 87(4), 1135-1164 (2012), peer-reviewed experimental evidence.Goodhart concerns control that changes an observed regularity; Campbell concerns corruption pressure on social indicators; Manheim and Garrabrant distinguish several overoptimization mechanisms; Choi et al. show managers treating a measure as the strategic construct. None says that every metric is invalid or supplies the intended value automatically.Adapt — reason: use the distinct proxy-failure mechanisms and observed strategy surrogation to ask what worsened and protect the intended value; reject the inference that every metric is invalid. Changed loci: CC-E22-7, CC-E22-8a, the Goodharted improvement repair, and the E.13 relation. Qualification/currentness: the historical accounts, taxonomy and experiment support the named proxy-failure mechanisms, not a universal anti-measure rule. Reopen: evidence overturns a mechanism used here, or E.13’s intended-value and protected-quality test changes.
Automated-judge mitigation is model-dependent and can itself require a declared guarantee or evidence profile.Sadman Kabir Soumik, Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge Pipelines, arXiv:2604.23178 (2026), current preprint; Benjamin Feuer, Lucas Rosenblatt, and Oussama Elachqar, Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation, arXiv:2603.05485 (2026), current preprint.Soumik compares nine mitigations and reports model-dependent effects across four bias types; Feuer et al. define average bias-boundedness for specified judge settings. These results are benchmark- and model-bound and do not make any LLM judge generally unbiased.Adapt — reason: use model dependence and the bounded-guarantee form to require a declared reliability evidence profile and qualification, not to call an LLM judge generally unbiased. Changed loci: ExpectedEvaluationEvidenceBasis@Context, EvaluationQualificationWindow, CC-E22-9, and the Exceptional improvement slice. Qualification/currentness as of 2026-08-19: the cited 2026 preprints remain benchmark-, model-, and judge-setting-bound. Reopen: later evaluation establishes materially different mitigation transfer or guarantee conditions, or E.22 changes the evidence profile required for an automated judge.
OEE and NQD can use proposal-shaped quality pressure without collapsing proposal, candidate retention, and selection.Xi Lin et al., Quality-Diversity Optimization as Multi-Objective Optimization, arXiv:2602.00478 (2026), current preprint; Haoxiang Qin et al., A survey on Quality-Diversity optimization: Approaches, applications, and challenges, Swarm and Evolutionary Computation 100:102240 (2026), DOI 10.1016/j.swevo.2025.102240, current survey.The shared comparison question is how to preserve several high-performing alternatives across declared coordinates or descriptors. The sources do not say that an evaluation proposal is already a generated candidate, archive insertion, front update, or selected result.Adapt — reason: use the collection-over-coordinates payload only to keep an E.22 proposal portfolio distinct from candidate generation, retention, and selection; it supplies no OEE/NQD authority. Changed loci: CandidateImprovementProposalRow@Context, E.22:4.6, the Proposal portfolio slice, and the C.17–C.19/G.5 relation boundary. Qualification/currentness as of 2026-08-19: the same 2026 QD sources apply here only as a bounded OEE/NQD proposal-framing comparison. Reopen: current QD evidence changes the collection/selection distinction, or the direct C.17–C.19 or G.5 consumer boundary changes.

E.22:12 - Relations

PatternRelation
E.21Declares the complete pattern-quality coordinate set and evaluation whose application produces the values.
E.9.DADeclares the complete DRR decision-adequacy coordinate set and evaluation whose application produces the values.
E.2.DADeclares the evaluation used to judge FPF Pillar adequacy.
E.19Supplies admission or refresh review profiles and findings; a quality evaluation of a result still uses its own evaluation specification.
E.23Governs repeated improvement after framed evaluations return findings or proposal rows.
E.13Governs pragmatic utility and proxy-to-value alignment when framed values, visible measures, proposal counts, or all-5 posture are being used as the intended improvement value.
A.19, A.19.ECS, A.19.CPM, A.2.6Govern the selected CharacteristicSpace, its construction description, predicate/comparator semantics and actual comparison application, and exact U.ClaimScope; E.22 binds their values for one question. Their defining patterns retain those definitions.
A.13, A.15.1, A.2, A.2.1, F.6, A.6.1, C.2.1Define or constrain the exact evaluator, independently admitted evaluation Work, optional local classification and precise assignment-bound attribution, any operation application and result binding, and any durable result episteme. Assignment and F.6 refs are included only when the record or receiving use expressly represents attribution. The evaluation pattern defines its own typed result relation. E.22 introduces no generic evaluation-result or Work-result relation. An ordinary frame may stop at an intended evaluator or planned condition without asserting actual Work.
A.2.4, A.10, G.11Govern actual evidence use, provenance, and currentness separately from the expected evidence-basis description.
C.22.PFRGoverns an actual Problem occurrence when the consuming use relies on one; evaluation need, finding, or floor failure alone establishes none.
E.10, E.10.ROLE, A.6.P, C.2.P, F.18Repair load-bearing wording and names introduced by frames or findings. E.10.ROLE resolves an ambiguous source role before E.22 cites a local system-role kind, assignment occurrence, relation participant, or ordinary non-system meaning.
C.16, A.17, A.18, C.25Govern characteristics, scales, measurements, and quality bundles.
C.17, C.18, C.19, G.5, G.9Govern OEE and NQD candidate, archive and front, pool, selected-set, and parity claims; G.11 governs currentness.
C.11, C.24, A.15, A.20, A.21, A.10, B.3When a quality result is reused beyond evaluation, C.11 governs choice among options in an available OptionSet; C.24 governs call planning after the action or option is fixed; A.15 governs system-role–Method–Work alignment; A.20 governs internal-constraint validity for its stated transformation-flow subjects and cases; A.21 governs gate decisions; A.10 governs evidence use, provenance, and bounded reliance; B.3 governs an actual named assurance claim. A release claim needs the pattern that defines that claim.

E.22:End

Referenced in the corpus

108 literal mentions in other sections. Read their context to establish the relation.