RMP.3 - Operationalize the Research Design without Replacing the Question
Type: Research Method Practice practitioner method pattern Status: Stable
RMP.3:1 - Problem Frame
Use this method to specify or assess the operations of a selected research design. Identify the consequential choice of specimen, encounter, source, observation or computation. Ask how it would help distinguish the alternatives in the qualified research question. A suitable existing protocol may already answer that question.
For example, a pump engineer has selected a comparison of accelerated and field-representative seal degradation. An easily measured wear score can answer a wear question. To assess preservation of the failure mechanism, the score or other evidence must distinguish the relevant mechanisms under the comparison conditions. A detailed schedule alone cannot establish that connection.
The operational specification connects one bounded design to the particular actions and records needed to carry it out. Depending on the question, it may specify a measurement, an interpretive encounter, a historical source comparison or a model experiment.
The ResearchProtocolAndOperationalizationMap is the protocol and the reasons its consequential choices address the question. A short protocol can state the reasons beside the relevant procedure. For a more complex design, use qualified instrument accounts, source descriptions or established procedures directly, identifying the conditions under which they apply.
Return the specification that is usable for the stated scope or identify the action or inference it cannot yet support. Reuse an adequate compatible protocol. If a gap concerns the research question, use RMP.1 as needed. If it could overturn the selected design, use RMP.2. Developing a reusable Method, obtaining a required permission, performing the study and interpreting the results each depend on their own tasks and evidence.
RMP.3:2 - Problem
Some operational choices can change the question a study can answer. A replacement comparison site, different construct measure or convenience sample may remove the distinction that justified the design. Preserve both the intended arrangement and what actually happened when comparing the protocol with subsequent work.
Some confirmatory inquiries rely on choices fixed before the relevant outcome is seen. Qualitative, exploratory or source-critical inquiry may instead depend on justified development during the work. Specify what must be settled in advance, what may evolve and how a change will remain open to examination.
How can a practitioner make the design executable, preserve its criticism and allow appropriate adaptation without either hiding a defeated premise or imposing a universal research checklist?
RMP.3:3 - Forces
| Force | Tension |
|---|---|
| Specificity | The specification should let its user recover what to do and why. Copying a full technical manual may obscure the consequential research choice. |
| Methodological fit | Instruments and templates can aid a question when their observation and interpretation fit it. Familiarity alone leaves that fit untested. |
| Advance commitment | Prior choices can protect criticism, while some inquiry depends on learning what to ask or inspect during engagement. |
| Feasibility | Access, timing, measurement and resources may each appear sufficient but fail in combination. |
| Records and protection | Later criticism needs consequential choices preserved, while indiscriminate recording can expose protected information or add unusable volume. |
| Supplied competence | A researcher can apply their own qualified procedure or use a suitable laboratory, statistical or archival contribution supplied by another practitioner or service. |
| Continuity | Most of a design may survive a local obstacle, while work depending on a defeated premise must stop or change. |
RMP.3:4 - Solution
Keep the research question and the error the design must expose in view. Establish the reasons for consequential choices, check how they work together and identify what can be carried out under the present conditions. If a link is missing, state which action or claim depends on it.
RMP.3:4.1 - Recover the question and the selected design
Use the qualified question and its current basis from a ResearchQuestionBrief or equivalent information already available. Identify the use, population, setting, period or corpus that determines scope. State what the evidence would have to establish and when work can stop even if uncertainty remains. The inquiry may seek to revise an explanation, a model or another claim.
Use the ResearchDesignDecision or an equivalent selected design. Identify the evidence strategy, core contrast, relevant rivals, threats and appropriate criticism. Criticism here means an examination capable of exposing an important error or limiting the answer. Use an experiment or significance test only where it bears on that question.
A ResearchDesignHold identifies an unsettled choice. To proceed on that design, use evidence that resolves the consequential input or a subsequent justified selection. If neither is available, report what the hold prevents. A suitable design supplied independently can already give direct entry.
Examine a consequential operation. Explain its relation to the question in terms such as these.
Examining this material in this way could distinguish the relevant alternatives because … . The inference applies under these conditions … .
For the seal inquiry, relate the observation to the possibility that acceleration changes the degradation mechanism. For a historical inquiry, relate an authenticated date to the possibility that a practice preceded the directive. If the required connection is missing, use an available subject account or identify the premise on which selection depends.
RMP.3:4.2 - Connect the question to particular material and actions
Specify the choices on which the inference depends. The table offers questions for the parts of this protocol that matter.
| Choice to settle | What makes the specification usable |
|---|---|
| Phenomenon, construct or meaning | State what the inquiry is trying to distinguish and why the proposed observation, source or encounter bears on it. A familiar variable name or cited scale does not establish that relation in the present use. |
| Units, cases and selection | Identify what is selected, what is changed or exposed, what is measured and what units the inference concerns. Give the source and grounds for choosing the attainable sample or corpus. Repeated readings can supply information about one specimen’s measurement variation. Its independent specimen count remains one. |
| Comparison or interpretive relation | Explain what will be compared, challenged or interpreted, which rival it can expose, and what differences must be controlled, described or acknowledged. |
| Timing and context | Specify the periods, sequence, exposure or engagement conditions that make the material comparable or interpretable. Retain the conditions that could make a later observation a different case. |
| Measurement, source or engagement Method | Identify the qualified procedure, source-critical technique, field or interview approach, computation or other operation and the conditions on which its use depends. Name the edition or configuration if it affects what can be done or inferred. |
| Information and uncertainty | State why the attainable material is useful for the intended criticism, what its limits are, and which supplied domain or statistical judgement supports that expectation. |
| Execution and access | Identify the capabilities, materials, facilities, permissions and access the proposed action needs. State which are available and which remain unknown. If another party supplies a result or controls access, identify that dependency where it matters. |
For a quantitative inference, establish the applicable basis for sampling or precision, dependence among observations and the measurement qualifications the question requires. An existing qualified result may supply them. To interpret an instrument indication as a measurement result, use the relevant measurement account, including its uncertainty and conditions. FPF C.16 guides measurement and characterization.
For a historical inquiry, identify relevant source series, dates, authenticity, provenance, transcription and the interpretation of silence where they matter. For qualitative inquiry, consider the theoretical orientation, researcher and participant positions, encounter conditions and how interpretation may develop. Specify the grounds appropriate to each source or encounter.
Use an adequate domain procedure directly. Explain its connection to the question and identify changes in application conditions that matter to the result.
RMP.3:4.3 - State the intended use of the material and the records it needs
Explain how the proposed material will be used in the analysis or interpretation. Identify the applicable method and any remaining consequential choices about exclusions, missingness, stopping or adaptation. State what the protocol specifies. An account of what was actually done or inferred requires the corresponding evidence.
Decide what a later critic would need to examine those choices. A specimen identifier may connect an exposure record to a measured value. A search version may explain why a source entered a review. A dated protocol may distinguish an advance commitment from a choice made after an outcome was seen.
For prospective work, arrange to preserve a consequential distinction before it can be lost. Use RMP.4 for a trace of actual decisions and work, including work begun while the specification evolves. For an inherited study, use what the existing records establish. Identify unresolved choices and lost evidence explicitly.
Identify applicable retention and access conditions. A public summary, a controlled original and a permitted independent inspection can support different examinations. Use existing rights and arrangements when they suffice. If necessary information cannot be retained or examined, identify the effect on the design and on claims that would rely on it.
RMP.3:4.4 - Preserve useful commitments and method-appropriate flexibility
Preserve the choice, its rationale and its timing relative to data access, exposure to outcomes or interpretive engagement. A registration, protocol and analysis plan may each record different commitments. Check what the dated content actually settled.
Commit in advance when a method or use depends on a choice preceding the evidence. Where the approach permits development during the inquiry, specify the limits of change and how it remains connected to the question. Apply an institutional registration or approval when required, using a valid existing result where it covers the work.
In a reflexive inquiry into former workers’ experiences of a factory closure, an encounter may reveal a meaning that warrants refining the interview prompt. Preserve the starting prompt, the reason for the change and the encounters affected. Estimating the proportion of all former workers who share a view requires a suitable question, population and design. Revising the prompt alone cannot establish that proportion.
Other design families require their own operational distinctions:
| Inquiry | Consequential detail and return |
|---|---|
| Historical or source-critical | Specify which documents could establish chronology, or identify those actually inspected for a claim about past work. State how contrary or inaccessible material bears on the inference. If a required series cannot be examined, distinguish missing evidence from the absence of an entry in an inspected record. Use RMP.2 where this defeats the selected comparison. |
| Computational or model-based | Specify the model–target relation, input and parameter choices, transformations, environment needs and intended sensitivity or analysis Method. A successful program run cannot supply a missing model–phenomenon justification. |
| Evidence synthesis | Relate the review question to eligibility, search, screening, extraction and intended appraisal. If a restriction could change the answer, assess its effect on the selected design and the warranted scope. Synthesis findings require evidence of the material actually obtained and combined. |
| Mixed inquiry | Explain the contribution of each component, shared inputs and how results would be related. Check joint access, participation, capacity and assumptions. If the design needs independent evidence of chronology, extra interviews based on the same retrospective account may leave that gap open. |
Choose the distinctions that fit the inquiry. Other study families or combinations can require different operational details.
RMP.3:4.5 - Test the whole arrangement and choose the exact return
For a proposed arrangement, ask whether its operations together can expose the error or distinguish the rival that justified the design. Check consequential interactions among selection, access, measurement, timing, resources and records. Include jointly admissible unresolved values. For an existing protocol, use established evidence of these conditions directly.
Apply the joint-input condition in RMP.2:4.3. Every admitted combination of PostSelectionNeed values must preserve the family, core contrast, criticism, feasibility and refusal conditions. A detail may safely wait when it passes this test in the combined specification.
| What the specification reveals | Action and result |
|---|---|
| The choices and their joint effects stay within the selected scope. | Complete or assess the protocol with its reasons. State the supported work, remaining bounded needs and conditions for change. Carry out the work when that is the task and any required permission is in force. |
| A result is still needed for an operation, but every admissible value preserves selection. | Identify the input, the action it blocks, any known source or holder and what would allow that action to proceed. Keep the justified design and usable parts of the specification. |
| A new value could defeat the family, comparison, criticism, feasibility or refusal condition. | Identify which values preserve the design and which could defeat it. If a new choice is needed, return the unresolved value and its consequences to RMP.2. Preserve the earlier decision and any actual work in the trace. |
| The question, intended knowledge, population, context or useful stop changes. | Qualify the changed question through RMP.1 where needed. Use its established answer to revise the dependent design or specification when that work is required. |
| Permission or resources are withheld independently of evidence sufficiency. | Identify the affected action and the actual limit on proceeding. If a decision is needed, use the applicable authority or state the missing authority. Preserve the research question and assess evidence sufficiency on its own grounds. |
A calibration value may be needed for a measurement even if every admissible value preserves the required discrimination. If the instrument might instead fail to distinguish the mechanisms, that uncertainty reopens selection where the comparison depends on it. The timing of the answer does not determine its scientific importance.
RMP.3:4.6 - Leave a usable operational result
A ResearchProtocolAndOperationalizationMap is usable for its stated purpose when a suitably competent user can recover the following.
- the brief and selected design being elaborated;
- the consequential operational choices and their question-relative reasons;
- the supplied Methods, qualifications and practical conditions on which those choices depend;
- the planned conduct, use of material and required records; and
- the remaining limitations and conditions for an amendment, return or stop.
A small inquiry can place the reasons beside the relevant procedures in an existing protocol. For a larger inquiry, identify the source and instrument accounts by the versions and uses on which the specification depends.
An incomplete specification can answer which action or inference is unsupported. Retain the supported work and name that limit. If further specification is needed, assess whether a missing input can be obtained or a design choice reconsidered with sufficient value, feasibility and authority. A protocol whose core comparison has failed needs a new basis before it can be used for that inference.
RMP.3:5 - Archetypal Grounding
These constructed cases stipulate the domain premises. Their identifiers and counts illustrate how to relate records and repeated observations to one another. Sample size and scientific thresholds require justification for the intended study.
RMP.3:5.1 - Pump-P4: from a mechanism question to an executable comparison
The Pump-P4 question asks whether acceleration preserves the field seal-degradation mechanism within a specified material and fluid envelope. Suppose the selected design compares accelerated and field-representative conditions. The mechanism marker and measurement procedure are qualified, the conditions are safe for the proposed work and the specimen plan is adequate.
The team adds these entries to its existing protocol:
| Protocol choice | Question-relative reason and required record |
|---|---|
| Give each seal a specimen ID and retain its material batch and assigned condition. | The comparison concerns degradation under the two conditions; material and allocation must remain recoverable. |
| Measure the qualified mechanism marker at the specified exposure points. | Preserve the procedure version, calibration basis, time and uncertainty needed to interpret the marker. The observed distinction must bear on the degradation mechanism. Both mechanism references can exhibit the same total visible wear. |
| Treat three readings from one seal as repeated observations of that seal. | They may characterize measurement variation. The comparison still has one independent specimen here. |
| Preserve material deviations, missing observations and the conditions under which they occurred. | The available records should let a critic distinguish material behaviour from allocation, handling and selection effects as far as the evidence permits. |
Each choice relates an operation to the mechanism question and identifies material that could challenge the inference. The protocol specifies the proposed work. Permission to proceed and a decision about the maintenance interval require their own bases where applicable.
Suppose a qualified measurement result now shows that the marker cannot distinguish thermal degradation from the competing field mechanism under the proposed conditions. Return that result and its consequence for design selection through RMP.2 when selection remains open. An overall wear score needs a justified relation to mechanism before it can replace the marker. Keep the existing brief and specimen information where they remain applicable.
On the valid-marker branch, suppose P4-07 is damaged in handling before the observation. Consult the protocol’s stated treatment of missing material. Preserve what happened and any action taken through RMP.4. The remaining data and their conditions determine which inferences can be supported.
RMP.3:5.2 - ResidentStream-S17: a missing comparison is not a scheduling detail
The residents have a qualified question and a selected comparison of salinity observations across sites and times. Suppose the measurement procedure, calibration, uncertainty and permitted access are established and the plan fits the hydrological conditions of interest.
The specification identifies the reach and sampling point, the trigger for observation, the required time relation between reaches, the measurement result and the weather, access and calibration records needed for interpretation. The residents can use a qualified laboratory result or perform the measurement themselves if they have the required capability and access.
Suppose the reference reach becomes inaccessible during the informative thaw. Preserve the downstream observations and the reach, period and access conditions they describe. Check whether an adequate observation of the reference reach in that same event is already available. If the pair cannot be recovered, the downstream evidence can still support its own descriptive claim. Reconsider the paired design under RMP.2 if that question remains live. A reference observation in later dry weather concerns different conditions and needs its own justification.
RMP.3:6 - Bias-Annotation
Examine how operational choices affect which cases, phenomena or voices are represented. Convenience sampling, uneven survival of records, access restrictions and exclusions can limit the answer. Documenting a restriction makes it inspectable. Assess its consequences separately.
Consider the researcher’s position when it can affect the encounter or interpretation. Prompt wording, relationships, competence and access may change what can be observed. Use criteria appropriate to the approach, including coder agreement only when the interpretation relies on it.
Ask whether the finished arrangement could expose the important rival or error, even when it is inconvenient to observe. Instrument familiarity and institutional pressure may favor easier observations. In mixed inquiry, check whether components share the same cases, records or assumptions and what that means for their relationship.
RMP.3:7 - Conformance Checklist
Use the questions that apply to the operational specification. Use qualified domain procedures for the observations or actions themselves.
- The practitioner can recover a compatible question, current frame and selected design, with their criticism and useful-stop conditions.
- Each consequential operation has a recoverable question-relative reason and the units, context and timing needed to use its material.
- Identify relevant measurement, source, analysis and operational inputs with their conditions. Distinguish available results and permissions from proposals or unresolved needs.
- Distinguish planned analysis from work already performed. Identify which records are needed while the relevant information can still be preserved, or which have survived for a historical use.
- Commitment and amendment timing is recoverable relative to the information already available. Legitimate adaptation remains method-appropriate.
- Check the jointly admissible values of unresolved operational choices. Identify a blocked action even if the selected design remains justified.
- If the selection premise fails, identify its consequence for RMP.2. Qualify a changed question under RMP.1 when needed. For a decision about permission or resources, use the applicable authority or state the limit.
- State what the protocol supports. Identify any action or inference that remains unsupported and what, if anything, could make it available.
For a disputed or consequential use, follow a material observation from its specified unit and procedure through the proposed comparison to the error it could expose. Where unresolved inputs interact, check their adverse admissible combinations. A fully populated form can still leave either connection unsupported.
RMP.3:8 - Common Anti-Patterns and How to Avoid Them
| Failure | Why it fails | Repair |
|---|---|---|
| “We use the standard scale.” | The scale may have been validated for a different construct, population or administration. | Use suitable qualification for the intended conditions. If the relation remains open, state which design premise depends on it. |
| Three readings become three specimens. | The information claim silently changes its unit and dependence. | Keep observation and inference units distinct and use the applicable sampling and analysis rationale. |
| A completed reporting checklist is treated as a justified protocol. | Reporting detail cannot select or validate the research relation. | Connect each consequential choice to the question and supplied Method. |
| Every adaptation is either forbidden or hidden. | The first suppresses legitimate inquiry; the second removes criticism of selection and timing. | State justified commitments and preserve actual reasons, timing and scope of change. |
| A lost comparison is recorded but its original inference is retained. | A transparent record of the loss may still leave the inference unsupported. | Use any adequate substitute evidence from the relevant condition. Otherwise state what remains supported and return the defeated selection premise to RMP.2 if a choice is still needed. |
| Each remaining need is tested with all others favourable. | The combined arrangement may be infeasible or incapable of criticism. | Apply the joint-input test to admissible combinations. |
| The protocol is cited as permission or as a scientific result. | Preparatory choices establish what is proposed. | For a permission or a finding, use the applicable decision or suitable evidence for that finding, such as a qualified observation, measurement or analysis. Where these are unavailable, preserve the supported protocol and state the limit. |
RMP.3:9 - Consequences
The specification shows which operational choices implement the design and which could change the question it can answer. Arrange prospective records while important distinctions can still be preserved. Use qualified procedures and suitable records already available.
Explaining a consequential choice takes work. It may reveal that a study needs another design or an unavailable professional result. The effort is justified when it can prevent work that would not address the question. A sound specification still leaves study findings to the actual investigation.
RMP.3:10 - Architectural Rationale
Use RMP.2 to select an evidence strategy and RMP.3 to specify its operations. During specification, check that every jointly admissible remaining choice still preserves the selected evidence relation, criticism and feasibility. When it does not, report the consequence for selection.
A protocol specifies intended work. A trace records what happened. Both may be kept in one notebook and may develop together. If what happened in an inherited study is unclear, use RMP.4 to examine its surviving trace. Available records may already answer the question even when its original protocol is lost.
Use a suitable domain protocol and adequate records directly. Examine the connection to the research question when a consequential operation, change or limitation remains unclear. A measurement problem, interpretive inquiry and historical source comparison may each need a different explanation of that connection.
RMP.3:11 - SoTA-Echoing
Practice question. Which choices must be specified in advance to preserve the research question, and which may develop as evidence or interpretation changes?
Explain the reasons for consequential choices and make their timing and any amendments inspectable. Use an existing protocol when it already supplies them. Select advance commitments or permitted revisions according to the kind of inquiry and the threat being addressed.
| Source and scope | Operational contribution and limit |
|---|---|
| Flake and Fried’s Measurement Schmeasurement addresses the basis for choosing a measure. The 2026 measurement study by Goos et al. examines measurements used in Many Labs. | In §4.2, establish the relation between construct and measure and state consequential administration and scoring choices. A published replication and the Many Labs findings do not by themselves establish the validity of the local instrument or a general failure rate. |
| Lakens’s Sample Size Justification compares quantitative rationales for sample size in relation to inferential aims and constraints. | In §4.2, justify the attainable sample for the question it must answer. A resource limit can determine the sample obtained while leaving the desired precision open. Use a relevant statistical procedure for quantitative adequacy and suitable criteria for historical or qualitative inquiry. |
| SPIRIT 2025 gives reporting guidance for trial protocols. TOP 2025 distinguishes registration, protocol and analysis-plan practices. | In §§4.2–4.4, identify consequential choices and the information available when they were made. Trial-specific requirements apply where the inquiry uses that design. Inspect the contents and timing when judging what was committed in advance. |
| Haven et al.’s qualitative preregistration study offers a starting account for qualitative preregistration. Braun and Clarke’s RTARG argument and review challenges generic defaults for qualitative reporting and assessment. | In §4.4, make initial commitments explicit and preserve justified changes under the selected approach. The Delphi study and Braun and Clarke’s review of reporting in studies that used reflexive thematic analysis address particular methodological settings. Other qualitative approaches need their own grounds. |
| AHA’s professional standards and the Cochrane Handbook, chapter 2 address source criticism and question-led selection. | In §§4.2 and 4.4, examine provenance, contrary material and which sources the question requires. A lost archival series limits the conclusion that can be drawn from it. A historical or synthesis finding requires actual relevant material. |
These methodological sources support the specified operations within their own conditions. If a change in construct, approach, corpus or access defeats a connection to the question, reassess that choice. Consider a different professional procedure when it preserves the required inference and criticism at lower cost. Claims about an improvement in actual research practice need evidence of that use.
RMP.3:12 - Relations
For a practice-reconstruction question, RMP.10 relates candidate accounts to observations, critical-incident elicitation and log evidence. For a question about Method history, RMP.11 relates dated sequences to competing explanations and diagnostic observations. Include the operations the question requires and identify the limits of each source of evidence.
- RMP.1 helps qualify the research question and stopping conditions, including a materially changed question or use whose qualification remains open.
- RMP.2, especially §4.3, gives the selected design and the condition for jointly admissible unresolved inputs. Return a defeated premise there when its consequences for selection need to be settled.
- RMP.4 helps preserve evidence of conduct and changes as they occur, or assess what an existing trace supports. Capture may begin while the specification is being developed.
- FPF C.16 supplies measurement and characterization discipline; FPF A.10 supplies claim-bound evidence and provenance discipline. RMP.3 adds the particular research question and operational connection.
- Qualified domain, statistical, interpretive, source, ethics, safety, access and resource practices supply their own results. The Core Method-development family applies when the reusable Method itself needs design or repair.
- RMP.5 helps connect actual material and analysis to a bounded claim. RMP.6 helps assess the claim’s credibility using available evidence or a worthwhile test. A protocol supplies the relevant intentions and choices. Substantiate execution and conclusions from the actual work.