HCD.18 - Select or Construct a Characterization and Evaluation Specification for Instructional Material
Type: Human Capability Development practitioner evaluation-specification construction pattern Status: Stable
HCD.18:0 - Use This When
Use this pattern when you need a specification for judging what instructional material contributes to particular learning work. It can guide repair or selection, qualify a proposed use or answer a question about the material. An attractive layout, a prepared reader’s correct answer or a teacher’s prompt can conceal a weak explanation.
State what this audience should be able to do with the material and the available support. Identify a task that would reveal a consequential defect. Check whether a specification already covers that use.
Return a compatible specification or construct the parts the evaluator needs. Specify what to inspect, how to interpret the evidence and which conclusions it supports about the material in this use. Where a decision depends on those conclusions, say what difference would affect it.
Use HCD.19 when the task includes evaluating the material under a compatible specification. For claims about a person’s performance, transfer or retention, use HCD.11, HCD.12 or HCD.13 respectively. Evaluate a professional reference for the work it supports. Evaluate its contribution to learning when it promises instruction.
HCD.18:0.1 - Working Terms
| Term | Meaning in this pattern |
|---|---|
| Instructional material | The explanations, examples, tasks, representations and usable ways to reach the sources, help or next task offered for the specified learning work. A worksheet, book or guided interface may carry this content. |
| Declared use | The audience, tasks, prerequisites, ordinary assistance, and question or decision the evaluation must support. |
| Task denominator | The work and consequential variations the material promises to support. Determine these from the work and its subject requirements. |
| Characteristic | A property whose values distinguish findings about the material or affect its use, repair, evidence needs or protection of another useful property. |
| Evaluation specification | The defined object and use, selected characteristics and scales, evidence requirements, result meanings, and conditions under which an evaluator may draw a conclusion. |
| Material value | A qualified judgement about a selected property of the material. The evidence supporting the judgement and the material’s fit to the specification remain separately interpretable. |
A short written agreement can carry the specification when it supplies the required meaning.
HCD.18:1 - Problem Frame
A time-accounting handout tells a learner to count from a task’s first start to handoff. Its example contains no interruption, so the intended active effort and the elapsed turnaround have the same value. The example looks correct. A prepared reader later separates the quantities without help, while another reader may follow the handout’s wrong general rule.
Both the heading list and the prepared reader’s correct answer can conceal the defective rule. In this example, introduce an interruption, ask for effort and turnaround, and inspect the handout’s explanation. Establish which distinctions the reader recovered from the material and which they supplied from prior knowledge, as far as the evidence allows.
The evaluator also needs to know what the material promises. A handout used with a teacher’s corrective question has a different support arrangement from a self-study unit. The question can legitimately enable correction without proving that the first explanation enabled independent recognition.
Specify these distinctions before interpreting the observations. Establish the subject relation, the audience’s accessible support, the validity of the task and what each observation warrants. Relate those findings when judging the promised use.
HCD.18:2 - Problem
How can a designer construct a small but complete evaluation specification that distinguishes action-changing material defects, supports defensible values and repairs, and exposes the evidence still needed for the declared use?
HCD.18:3 - Forces
- A reusable profile saves effort, but a fixed quality list can miss a task absent from the material itself.
- Shared observations can support several properties, while those properties require different repairs.
- A correct first answer can depend on the material, prior expertise, assistance or a combination. Distinguish the contributions when they affect the judgement.
- Added detail can explain a difficult distinction or make the next operation harder to find. Judge its contribution to the task.
- A usable ordinal judgement can guide repair without pretending to measure equal intervals or predict learning.
- A demanding task can reveal useful learning work or a missing prerequisite. Immediate ease cannot decide between them.
HCD.18:4 - Solution
Start with the work the material promises. Determine what the evaluator needs to observe and how those observations support the requested conclusion. Use A.19.ECS for the general specification requirements. The instructional-material evaluation profile offers property families and an optional scale for judging use and repair.
HCD.18:4.1 - Recover the promise and stop if an existing specification fits
Identify the edition, portion, intended audience and task prerequisites. Include the publication form when font size, navigation, resolution or interaction affects access. In the time-accounting example, readers need to be able to add time intervals.
State which material, tools, sources and assistance the use allows. Distinguish promised help, help available for an attempt, help received and help the reader used. For a future audience, describe the conditions as planned.
Identify the question or decision the evaluation must support. A teacher may be choosing between two explanations, an editor repairing a planning task, or an authority considering permission for self-study. Use the consequences of a mistaken judgement to set the scope and evidence required.
Check any available specification against the material, audience, required tasks, support, subject conditions, properties, scale meanings, evidence and intended use. Matching titles or 0–5 labels can conceal different meanings. Return a compatible specification. If an existing specification misses a required contribution or a requirement has changed, retain what fits and revise the affected parts.
HCD.18:4.2 - Derive the tasks and contrast cases outside the present table of contents
Start from the work the audience should be able to perform or practise. Recover its required distinctions, prerequisite operations, acceptable variants, and consequential errors from the problem and a qualified subject basis. Use those requirements to judge the material’s coverage.
For each promised task, ask what a successful contribution would contain and which plausible wrong contribution an apparently adequate explanation might also produce. Choose a case on which the alternatives differ. In the time example, an uninterrupted interval cannot distinguish active effort from elapsed turnaround; an interruption can.
Compare those requirements with the explanations, examples, practice and help the audience can use. Include an external contribution when the audience can reach and use it at the point where it is needed.
For a new or changed specification, use three kinds of contrast. Reuse suitable cases when available.
- an in-scope material that supplies the needed contribution;
- an in-scope material with a consequential deficiency;
- a nearby object or use for which this specification would answer the wrong question.
These contrasts test fit as well as quality. A personal capability judgement is outside a material specification even when the same learner task appears in both.
Distinguish the subject basis from manufacturing history. For a judgement about an existing book, inspect the contribution it makes in the intended use. If fidelity to a transcript, prescribed edition, or earlier rendering was promised, evaluate that source-to-material relation on its own evidence.
HCD.18:4.3 - Select properties by the decisions their values change
Use the profile’s property families as a search aid. Keep a characteristic when its values can change a requested finding or use decision, identify a distinct repair, call for different evidence, or protect a contribution that another repair might damage.
Try two material conditions for each proposed characteristic. If the values support the same finding, response and evidence requirements, refine the distinction if it could matter to the use; otherwise omit it. Separate properties that can fail independently and require different treatment. Accuracy and currentness can require different evidence for the same passage. A valid alternative answer rejected by the key reveals a criterion defect even when the explanation is sound.
Declare the characteristic’s bearer and grain. A false arrow in one diagram, a missing operation across an explanation–exercise chain, and a whole-route omission have different reach. Report the reach that the evidence establishes.
When the question concerns how much of a selected structure a reader can recover, use extractable structural information (FPF C.2.8) (another source edition). Identify the material episteme, the publication form expressing it, and the reader or observer. Specify the selected relations and correctness criterion, with the preparation, operations, help, access and budget that affect recovery. An existing reference can identify the material and its expression. If explanatory sufficiency or overall adequacy is also at issue, assess it on evidence that supports that claim. A shared observation may contribute to more than one judgement.
Select all properties needed for the declared conclusion. Two profile questions may support a bounded diagnosis. A complete material evaluation requires evidence or an explicit evidence gap for each required property. Report a missing observation under the applicable property.
HCD.18:4.4 - Bind values, evidence, and decisions without collapsing them
For each selected characteristic, define its value domain, scale type, polarity, anchors, and conditions of applicability. A numerical scale needs an interpretation: a count, a measured duration, and an ordinal repair judgement support different operations.
For structural amount, a description of recovered and missing relations can suffice. For the count or fraction of selected relations used here, fix the semantic units and select a nonempty set of relations. A result of zero can then mean that none of those relations was recovered. For another scale, state its interpretation under C.2.8. Distinguish observed recovery, a conditional design estimate and a mapped formal estimate. Report the missing basis when the selection, denominator or criterion required by the scale is unknown. Identify any relation required for use, since recovering other relations cannot supply it.
The profile’s optional 0–5 scale orders material conditions by use and repair. Its anchors distinguish absence of usable support, limited support, a connected central repair, a bounded repair, adequacy, and additional useful resilience. Use these ranks for ordinal judgements. Equal numerical spacing, arithmetic averages or an amount of recovered structure would require a different basis. Set a floor when the intended decision requires one and justify it for that use. State where each non-compensable requirement applies.
Make adjacent values distinguishable in the domain. For explanatory sufficiency, distinguish rebuilding the central explanation–task chain from repairing one missing transition. Distinguish adequacy for the selected cases from an evidenced advantage across consequential variations. Assign 5 when the evidence shows this advantage over 4.
Define what evidence supports each value. Inspection can establish a false unit relation. A claim about a learner’s experience requires evidence of that experience. When audience fit depends on reader performance, specify what contribution to observe and the assistance conditions. Report any missing evidence separately from an observed material value of 0.
For each characteristic, define how to report the value or reason it cannot be assigned, the evidence and limiting case, and the grounds for distinguishing adjacent values. Include implications for the intended use and the smallest worthwhile repair where one is needed. Refer to a shared observation wherever it supports a property and count it once as an observation.
When judgements differ, investigate the criterion that separates them. For 2 versus 3, determine whether the defect calls for rebuilding a connected central chain or for a bounded repair. If the evidence leaves that distinction open, report the disagreement. Revise a scale distinction that repeatedly fails to distinguish the findings or responses it was designed for.
HCD.18:4.5 - Make the specification usable by another evaluator
An evaluator using an A.19.ECS specification needs the following information.
| Specification content | What the evaluator must be able to recover |
|---|---|
| Object, use, and qualification | Exact material or material family, evaluation grain, audience prerequisites, tasks, assistance, subject/source state, question or decision, and window in which the basis remains valid. |
| Fit and contrasts | In-scope adequate and deficient examples, outside-kind or outside-use example, entry decision, and the disposition of a forced invocation on an incompatible object. |
| Characteristics and scales | Selected properties, their applicability, value domains and scale types, polarity, anchors, any floor or exceptional-value condition, and protected trade-offs. |
| Evidence and calibration | Coordinate-specific evidence minima, qualified subject criteria, tasks and relevant competing answers, admissible inference, missingness, known contrasts, and the needed reader or evaluator calibration. |
| Result and evidence | Value or reason for missingness for every selected property. Give the relevant material, task or observation, available and received help, grounds for distinguishing the value from its neighbours, a limiting case, any needed repair and the claim’s scope. Preserve the task, criterion, first answer and assistance used when they affect the inference. |
| Decisions and continuation | Supported-use, repair-needed, insufficient-evidence, disputed-criterion, and fit-defect meanings as needed; stop, reopen, and next-evidence conditions; the starting result for E.23 if improvement is needed. |
| Neighbouring claims and comparisons | Exact exits for human performance, transfer, retention, source fidelity, or production history. If selection among alternatives is live, the common comparison basis and E.22 use, including incomparable cases. |
A specification can be ready for use before all evidence has been collected. State what the evaluator needs and identify any known missing observation and the conclusion it leaves open. A calibrated illustration supports a judgement within the conditions it tested.
An incompatible object normally stops before invocation. If an invocation is nevertheless required, return an explicit fit defect and the blocked conclusion. If a selected coordinate becomes unevaluable during an in-scope application, return its evidence or criterion defect in the complete result. Neither case authorizes silently dropping that coordinate.
HCD.18:4.6 - Try the specification on contrasts and establish its continuation
Check how an evaluator applies the specification to its adequate, deficient and outside-use contrasts. Use existing applicable calibration evidence or conduct the missing checks. Examine the judgements and repairs: can an ambiguous example hide a central defect, does the key reject a correct alternative, or does a missing observation receive a low material score?
Where an AI reader or evaluator is used, qualify the contribution it makes using applicable evidence. Subject-grounded counterexamples can calibrate defect diagnosis; simulated novice behaviour cannot establish human learning. A fresh reader is useful when private knowledge of the intended defect or answer would defeat the selected inference.
Finish construction when another evaluator can identify the object and tasks, obtain or use the specified evidence and report each selected property with its limits. For a decision, the specification must distinguish the outcomes that affect that decision. If an unresolved criterion or evidence gap prevents a conclusion, identify what remains open.
Name the specification version when results will be compared later. A changed value meaning, task denominator, or support arrangement may invalidate numeric comparison with earlier results. Keep earlier results under their original meanings. For a cross-version conclusion, establish a comparison supported by both meanings or report that equivalence cannot be established. Reopen only the affected specification content when a new task, audience, source premise, assistance change, or recurrent calibration disagreement changes it.
HCD.18:5 - Archetypal Grounding
HCD.18:5.1 - A complete small specification for active effort and elapsed turnaround
A teacher is preparing a short introduction for readers who can read clock times, add intervals, and use a calculator. The promised work is to separate task effort from turnaround and compare consumed person-time with available person-time. The material stays available. A teacher may ask a corrective question after the first answer. Independent first recognition and supported correction are recorded separately.
Call the following specification Time-use S1. Its decision is whether the material supports this bounded introduction with the declared teacher contribution, and what to repair before relying on it. The object is the handout’s explanation, examples, task, and usable correction support under the stated teaching conditions. Qualification holds while these tasks, audience prerequisites, support, and time-accounting meanings remain unchanged.
The subject criteria are stipulated before an attempt: active effort sums the named person’s active intervals; elapsed turnaround spans first start to handoff; exclusive active intervals are not double-counted; simultaneous work by different people is added as person-time; a resource comparison uses the same population and unit on both sides.
| Task derived from the promise | Required contribution and discriminating error |
|---|---|
| Interrupted report | One person works on a report 09:00–09:30 and 11:30–12:00, with incident work 09:30–11:30. Return report effort 60 person-minutes, incident effort 120 person-minutes, report turnaround 180 clock-minutes, total effort 180 person-minutes, and incident share 120/180. Calling report effort 180 exposes the rival rule. |
| Concurrent people | A works on X 09:00–10:00; B works on Y 09:00–11:00; both are available 09:00–11:00. Return consumed effort 3 person-hours, available effort 4 person-hours, and elapsed window 2 hours. Compare 3 with 4, not 3 with 2. |
| Correction and continuation | Preserve the first report answer, then ask which intervals contain report work and which belong to the incident. Ask the reader to correct or defend the answer and apply the distinction to the concurrent-people task. A repeated answer without a reason does not show that feedback enabled correction. |
The interruption separates report effort from turnaround. The concurrent case separates person-time from elapsed duration. Both distinctions are required by the handout’s stated promise.
Time-use S1 selects six characteristics on the profile’s ordinal repair/use scale, with higher values better for this use. Its local rule requires evidence of at least 4 on every selected property for a supported-use conclusion. A high value elsewhere cannot offset a failed requirement. Missing evidence leaves supported use unestablished. Identify the supported results and the missing evidence. An existing observation or an attainable, worthwhile controlled probe may supply it.
| Selected characteristic | Evidence required for 4 in S1 | What distinguishes the neighbouring repair or resilience judgement |
|---|---|---|
| Subject validity | Inspect the explanation, example, task, key, and correction against the stipulated meanings; challenge the interruption and population denominators. | One bounded incorrect unit label with otherwise consistent uses calls for 3; a wrong central rule propagated through the explanation and use calls for 2. For 5, demonstrate an additional useful boundary treatment, such as separating waiting from other-task work without corrupting either quantity. |
| Task and dependency coverage | Trace both task families and their prerequisites to the material or accessible declared support; inspect that each needed operation is supplied. | A missing usable return for one supplied operation is bounded repair; absence of the concurrent-person operation needs a connected addition. For 5, show useful coverage of a consequential further case, such as unequal availability, while preserving the original promise. |
| Explanatory sufficiency | Inspect the grounds linking intervals, quantities, and comparison; obtain reader work under the declared prerequisite and support conditions that reveals how the reader recovers those grounds. | One missing transition differs from rebuilding the active/elapsed explanation chain. For 5, the reader can use the supplied explanation on an additional consequential variation, with its source contribution and assistance visible. |
| Example discrimination | Apply the correct and rival rules to the supplied examples. Use an example on which the rules give different answers for an interruption and one on which they differ for aggregate person-time and elapsed duration. The material or its declared accessible support may supply them. | An isolated misleading example amid a sufficient discriminating pair differs from an example set compatible throughout with the rival rule. For 5, an additional boundary contrast prevents a further identified misapplication. |
| Task and criterion validity | Independently solve the tasks; test the key against a correct alternative and the named rival errors. Equivalent units and justified exact or rounded shares are accepted. | A key’s bounded omission differs from a task–criterion chain that elicits or rewards the wrong quantity. For 5, the criterion demonstrably discriminates a further consequential error without rejecting a valid alternative. |
| Feedback and continuation support | Inspect the available feedback and observe a first answer followed by the declared question, an explained correction or justified retention, and the changed task. Record what the teacher supplied and what the reader did with it. | A missing next-task return differs from a support arrangement that cannot expose or correct the central discrepancy. For 5, an additional relevant difficulty can be handled through the declared support without supplying the next answer wholesale. |
In this use, inspect representations and language where they carry the selected explanations and criteria. The promise covers one form, a short route and a fixed subject edition. Revisit the property selection if those conditions change.
For each property, S1 returns the value or reason it cannot be assigned, the material passage and task, the evidence from inspection and any required reader work. Give the criterion, any observed response, relevant help available and used, grounds against adjacent values and the reach of any needed repair. A value of 4 or 5 requires the evidence specified for that property. An observed subject error can establish a lower value without a learner attempt.
HCD.18:5.2 - Contrasts that test what the specification can conclude
A deficient handout, M0, says that time spent doing a task runs from first touch to handoff. Its only worked example is uninterrupted. It offers some useful time-accounting content, but its central rule and example do not support the interruption distinction. S1 can locate a connected explanation–example repair even if a prepared reader independently returns all correct quantities.
A revised explanation, M1, distinguishes active intervals from start-to-handoff turnaround, separates the columns, gives the interrupted report result, and states how to sum person-time across people. This repairs the named subject relation. To become an adequate contrast for all of S1, the material-and-support arrangement must also supply the concurrent example, a criterion that accepts valid equivalents, and the specified reader/feedback evidence. “Better explanation” alone is not a complete six-coordinate result.
An adequate constructed contrast combines M1 with the concurrent worked comparison, the S1 criteria, and an available teacher question and retry. It shows what adequate content and support would contain. Until there is evidence of an appropriate reader using that arrangement, explanatory and feedback adequacy remain evidence requirements, not observed successes.
The claim “this reader will retain the distinction independently next month” concerns a person. S1 returns an outside-use finding. Assess retention under HCD.13 if it is the question.
Inspection identifies the defective rule in M0. The corrected definition in M1 resolves that defect. Explanatory and feedback adequacy under S1 need evidence of a reader working under the stated conditions. A claim that a person learned the distinction needs evidence about that person’s change.
HCD.18:6 - Bias-Annotation
The selected task denominator can inherit the designer’s narrow experience. Compare it with consequential work variants and a qualified domain account, especially when the present material omits them. Declared prerequisites can also conceal missing instruction by demanding knowledge the intended audience was never promised to have.
Prepared readers and language models may repair an explanation without noticing how much prior knowledge they used. Their diagnosis can establish a source defect, but their success does not automatically qualify material for a less-prepared audience. Keep reports of difficulties experienced by people under their respective task and assistance conditions.
An ordinal scale can make evaluators seek a number before understanding repair reach. Read the limiting case and evidence first; retain an unresolved boundary when its meaning is disputed.
HCD.18:7 - Conformance Checklist
Use these criteria to check a constructed specification.
- HCD18-CC1. The specification SHALL identify the material, grain, audience, tasks, prerequisites, assistance, subject basis, intended question or decision, and qualification conditions.
- HCD18-CC2. The constructor SHALL check the compatibility of any available specification and derive the task denominator from the promised work rather than the present headings alone.
- HCD18-CC3. The specification SHALL provide adequate, deficient, and outside-use contrasts and preserve the distinction between fit, material value, and evidence.
- HCD18-CC4. Each selected characteristic SHALL distinguish conditions that affect the requested judgement, action, repair, evidence needs or protection of another contribution. Specify its applicability, scale, anchors and evidence requirements, and how to report a value or missing evidence.
- HCD18-CC5. An ordinal numeric rendering SHALL NOT be treated as equal intervals or silently averaged. Any floor, exceptional value, comparison, or non-compensable condition SHALL be justified for the consuming use.
- HCD18-CC6. The result requirements SHALL preserve the evidence and adjacent-value grounds needed to recover the judgement, including first response and assistance received or used when they change the inference.
- HCD18-CC7. Calibration SHALL test discriminating contrasts and preserve criterion or evidence disagreements. Constructed cases SHALL be distinguished from observed reader work.
- HCD18-CC8. The specification SHALL state supported conclusions, stops, reopens, neighbouring-claim exits, and the E.23 starting result when repair is needed.
HCD.18:8 - Common Anti-Patterns and How to Avoid Them
| Misuse | Why it changes the wrong decision | Repair |
|---|---|---|
| Count every present heading as coverage | The material defines its own denominator and hides omitted work. | Recover the required task and compare its dependencies with the content. |
| Treat every profile family as a mandatory score | Inert coordinates add effort while an unlisted live property can still be missed. | Select the properties needed for the finding, action, evidence or protection in this use. |
| Award 0 for no learner observation | Ignorance becomes a demonstrated material failure. | Return missing evidence and the conclusion it blocks. |
| Infer material adequacy from an expert’s correct answer | The expert may supply the missing or corrected relation. | Inspect the material and how the reader produced the response; obtain audience-compatible evidence if needed. |
| Copy another programme’s 0–5 values | Identical numbers can encode different fit, repair, and floor meanings. | Retain the original scale and establish only a justified comparison of meanings. |
| Require construction records for every old book | A current product question is replaced by a historical claim. | Evaluate current support; open source-fidelity or history only when promised or consequential. |
HCD.18:9 - Consequences
A constructed specification makes the next evaluation executable: the evaluator knows what work matters, what a defect would change, what evidence to obtain, and the scope of the conclusion that evidence supports. Repairs can target a criterion, example, explanation, return, or assistance arrangement instead of expanding the material indiscriminately.
The cost is up-front task and contrast design. That cost is worthwhile when an intuitive score could drive consequential selection or repair. A compatible existing specification avoids repeating it. New audiences or purposes can require a changed specification; preserving the original result meanings makes that change inspectable.
HCD.18:10 - Architectural Rationale
Material evaluation sits inside human capability development because the material is one contribution to learning work. The same explanation can contribute differently to a teacher-supported activity, a self-study unit and a narrower preparatory use. Attribute a learner outcome to the text only on the evidence for that contribution.
A.19.ECS gives the general requirements for constructing a specification. For instructional material, derive the task set from the promised work, distinguish plausible learner rules, identify the assistance relevant to the evidence and locate the reach of a repair. HCD.19 uses a compatible specification to evaluate material and any reader work that the selected properties require.
Use the profile’s property meanings and comparison cautions for the tasks the material promises. For a claim that a practice described in the material is an admitted Method, establish the Method under FPF A.3.1. For a claim that the material is its MethodDescription, establish under A.3.2 that the material describes that admitted Method.
HCD.18:11 - SoTA-Echoing
Evaluate the material for its intended use. Compare rules on tasks where their answers differ, examine what the feedback enables, and calibrate the interpretation of reader work. These checks can identify the explanation, task or support to repair, or the observation still needed, where a heading count or an expert’s correct answer leaves the defect undetected. Constructing discriminating cases takes work.
| Practice choice | Adopt, adapt, or reject; effect on this pattern | Source contribution, limit, and reopen condition |
|---|---|---|
| Tie a judgement to the intended interpretation and use | Adapt use-bound validity in sections 4.1–4.2 and S1. A single context-free material score cannot answer an assistance-dependent use question. | The AERA/APA/NCME Standards (2014) supply the assessment-validity anchor, not a book-quality scale. Reopen when the audience, task, or inference changes. |
| Make an example distinguish consequentially different rules | Adapt contrastive example design in sections 4.2 and 4.6. The uninterrupted example in M0 cannot discriminate the two time rules; an interruption can at similar reading effort. | Wesenberg et al. (2025) supply failure evidence about ambiguous worked examples in brief units and two topics. The local time example is a constructed application, not their experiment. Reopen when ambiguity is deliberate preparation and later instruction resolves the alternatives. |
| Evaluate what feedback permits next | Adapt a separate feedback-and-continuation property in section 4.3 and S1. A key can mark an error without enabling correction. | Wisniewski et al. (2020) synthesize heterogeneous feedback effects; presence or quantity is an inadequate default. They establish no local passing threshold. Reopen when a different support arrangement changes the next action or required evidence. |
| Calibrate an automated judgement for the property being judged | Adopt bounded calibration in section 4.6 for the selected automated contribution. A discriminating probe can show when the evaluator supplies a missing explanation. A human-learning claim requires evidence about people. | Bavaresco et al. (2025) find task-dependent LLM/human agreement across NLP evaluations. This is counterevidence to unrestricted substitution, not validation of a learning-material rubric. Reopen for a new evaluator, property, or audience-dependent inference. |
HCD.18:12 - Relations
- A.19.ECS defines the general content of an evaluation specification and its fit, evidence, comparison, and continuation requirements. A.17, A.18, and C.16 retain the general characteristic and scale meanings used by its bindings.
- HCD.19 uses a compatible specification to evaluate instructional material, with representative reader evidence where the claim requires it. E.22 supplies comparison when selection is live; E.23 supplies improvement from the resulting defect and protected trade-offs.
- HCD.3 and HCD.6 can supply the target contribution and representative task design. Use qualified task and audience evidence from an appropriate source.
- HCD.11, HCD.12, and HCD.13 govern evidence about human performance, transfer, and retention. A material judgement contributes to those questions without replacing their evidence.
- NSTD.6 and NSTD.8 contribute current narrative-product adequacy, attachment, and progressive reconstruction when the selected material’s narrative use makes those properties live.
- E.4.DPF.DA governs the adequacy of a DPF package used as a framework. An instructional-material profile applies additionally only for an instructional promise; it does not turn every reference framework into a course.