MKT.10 - Evaluate the Contribution and Cost of a Marketing Intervention
Type: Marketing Method pattern Status: Stable
MKT.10:1 - Problem frame
Use this pattern when a decision or a claim about marketing work depends on what that work contributes or consumes. Examples include choosing between invitations, deciding whether a supported trial improves customer results, or reconsidering a channel whose attributed revenue looks attractive but whose incremental contribution is unknown.
State what the evidence establishes for the intervention, subjects, quantity and period in question. If the question concerns an effect or additional cost, name the alternative used for comparison. The answer may support a report or decision, justify a smaller trial, or identify the missing evidence that prevents a conclusion.
The intended reader can state the decision or claim at issue and obtain the observations, causal analysis or financial account it needs. A simple count may be sufficient to describe completed work. Use the causal parts only when the decision or claim depends on what the intervention changes; an ordinary record of what happened does not require an experiment. Use a qualified specialist result when identification or estimation requires one.
MKT.10:2 - Problem
A dashboard can show impressions, replies, purchases, satisfaction and expenditure without explaining whether an intervention helped. People who receive more support may have harder problems; customers already more likely to achieve the outcome may select themselves into a trial; demand may rise during the campaign for another reason. Assigning revenue to the most recent contact describes a rule of attribution, not the result under an alternative contact policy.
Begin with the decision or claim at issue. For a causal claim, state which intervention and alternative you are comparing. Define observations that retain their meaning and obtain an adequate comparison when a causal conclusion is needed. For a decision about the intervention, compare the economic and operational consequences that matter, including the cost of a mistaken choice. For a report, state what the evidence supports.
MKT.10:3 - Forces
| Tension | What the evaluation must preserve |
|---|---|
| Immediate responses are easier to observe than useful results. | The distinction between exposure, response, acquisition, performance and benefit. |
| Customers and staff choose whom to contact or help. | Selection and other explanations of observed differences. |
| A decision is needed before all long-term effects are visible. | A bounded horizon, material delayed consequences and stated uncertainty. |
| More detailed data can improve a comparison while increasing burden. | The smallest adequate observation and permitted use of its information. |
| A positive effect can consume scarce support or arrive after bills are due. | Complete incremental work, dated flows and feasible delivery. |
MKT.10:4 - Solution
MKT.10:4.1 - State the decision or claim and the answer needed
For a choice, name the available alternatives. Examples include an invitation unchanged or with added help, a switch to another channel, lower volume or ending the intervention. For a claim about effect, state the intervention and the alternative against which its effect is claimed. For additional expense or work, identify the alternative that sets the baseline. In each case, name the result or quantity at issue and who will use the answer. An increase in replies can answer a question about response volume while remaining insufficient for an expansion justified by better customer performance.
Specify whose activity, outcome or expenditure is at issue, what counts and which period the answer covers. For a practical choice, include consequences that could rule out an action. Include different participants when their outcomes differ: the purchaser can save effort while the user acquires extra work. MKT.6 supplies the agreed result; MKT.5 supplies the intended experience conditions. Neither supplies an observed effect by itself.
Determine the kind of answer required. “How many trials were completed?” is descriptive. “Which customers are likely to request help?” is predictive. “How many additional useful completions would this invitation produce compared with the current invitation?” is causal. These questions can use related data but require different grounds. A descriptive answer may be enough to stop an infeasible service even when its causal benefit remains unresolved.
For a claim about effect, specify the magnitude or precision the claim requires before interpreting a favourable fluctuation. For a practical choice, judge a difference against the value of the result, the intervention’s burden and the consequences of a mistaken decision. Use a local threshold only when it is justified.
MKT.10:4.2 - Make each observation interpretable
Define the observation unit, eligible population, period, source and counting rule. For a response rate, say who had an opportunity to respond and what response counts. Distinguish unique people, organisations, opportunities and events. Several purchases by one organisation need not be several independent observations; a missing report is not necessarily a failed result.
Match the measure to the claimed outcome. A recorded form submission can establish that a request was submitted. It cannot by itself establish informed choice, successful work or a favourable experience. Obtain additional observations when the decision or a claim you need to support warrants the effort. Where a customer reports an experience, preserve the question, timing, scale and respondent; an ordinal rating does not automatically support arithmetic differences, monetary conversion or comparison with another questionnaire.
Check how the data are produced. A new reminder may change who answers a survey, a new interface may change what gets logged, and better records may increase the apparent number of problems. Account for these changes before interpreting a trend. Retain consequential uncertainty about missing observations and selective follow-up.
Agree how much observation is needed and how to obtain it without defeating the intervention or its receiving service. Use existing adequate records; seek a limited observation when its value for the decision or the claim under examination justifies the burden. MKT.12 supplies a focused inquiry where the unresolved issue is what the customer’s work or answer means.
MKT.10:4.3 - Construct the causal contrast and its credible comparison
When contribution means a causal effect, define the intervention and comparison as ways of acting, including intensity, timing, support and consequential variants. Specify the target population, outcome and horizon. An offer of help and actual use of that help are different interventions. An invitation sent to one team may also affect another team through shared staff or shared information.
Use FPF C.28.CM, Construct and Challenge a Causal Model to explain why the contrast could change the outcome and what else could produce the observations. Recover common causes, selection, intermediate results and feedback that change the inference. A journey diagram or chronological sequence is not a causal model merely because arrows connect its events.
Then obtain an identification answer through Identify an Intervention Effect from Available Data (MMP.15) when this requires specialist analysis. The request names the contrast and available data, including how participants entered them. The supplier returns which effect the data can identify, under which assumptions, or a useful bound or reason it remains unidentified. More accurate fitting cannot repair an effect the data do not identify.
Select a feasible comparison suited to the question. Random allocation can make groups comparable for the effect of the assigned intervention if allocation, follow-up and interference conditions are adequate. If participants choose whether to take up the offer, estimating the effect of uptake requires a basis that addresses their selection. Observational comparisons require defensible assumptions about relevant differences, usable variation and measurement. A before-and-after comparison requires a reason that other changes do not explain the result. Asking for “a control group” without these conditions supplies no causal answer.
Design the comparison with its operators. Avoid allocating a promise the service cannot fulfil or depriving people of an existing commitment. Arrange follow-up, exception handling and a stopping decision if the work becomes infeasible. If the credible comparison is unavailable, return the supported descriptive result and the unresolved causal question. A bounded action under uncertainty can still be chosen explicitly.
MKT.10:4.4 - Estimate the supported effect and challenge its use
Give the identified question and qualified observations to an adequate estimation method. Retain the estimated magnitude, uncertainty and consequential assumptions. Statistical precision addresses variability under the method; it does not remove selection, measurement error, missing follow-up or a wrong causal model.
Check the conclusions that matter for the decision. Could plausible missing outcomes reverse it? Does a result disappear when an unsupported comparison is removed? Does an average conceal a group for whom the intervention is impractical or harmful? A subgroup claim needs adequate grounds; dividing the same small dataset repeatedly does not create them.
Keep discovery and confirmation distinct when many messages, channels or outcomes were examined. A selected favourable result may justify a new bounded comparison rather than immediate expansion. If a decision permits repeated inspection, agree how the resulting uncertainty will be handled with the analyst instead of repeatedly stopping at the first attractive number.
Before applying the result elsewhere, compare the new population, baseline service, intervention and support conditions. A trial with spare specialist capacity may not describe a larger programme that queues every customer. Establish whether the earlier estimate applies under those conditions, or obtain a new comparison. A precise result for the wrong use remains an inadequate answer.
MKT.10:4.5 - Compare the whole contribution with its burden
Use Account for Customer and Product Economics Over Time (MA.8) when the question needs a dated account of work, costs, receipts or payments. If the question is how much was paid to the agency in November, total the corresponding payments in that period. An adequate existing record can answer it. If the question is how much additional expense the campaign caused, compare its costs with those under the relevant alternative for the same period. For a future choice, distinguish sunk costs from payments and obligations the choice can change. Include acquisition, preparation, delivery, adaptation, support and recovery where they affect the answer.
A customer’s purchase value is not automatically incremental profit. For an incremental profit claim, ask what revenue and expense would differ under the alternatives, which contributions would have happened anyway, and whether later work has been included. Avoid counting the same additional outcome under several channels or counting both its estimated monetary benefit and the same benefit inside a customer-value total.
An uncertain causal effect and an uncertain margin produce an uncertain economic conclusion. Use credible ranges when a point estimate would conceal material uncertainty. For a choice that depends on the size of the effect, show where the preferred action changes. The largest expected contribution can still be infeasible because of cash timing, scarce staff or obligations to current customers. MKT.11 supplies actual delivery conditions; its missing capacity is not repaired by a favourable average return.
Use nonmonetary consequences in their own terms where no adequate conversion exists. A client’s ability to complete important work, burden and experience can affect the decision without a fabricated price. When a choice requires a trade-off, state whose objectives it serves and what the decision maker accepts.
MKT.10:4.6 - Return the conclusion and its conditions for use
Return the answer the evidence supports and its limits, whether it concerns an observation, a dated amount or an effect. For a practical choice, identify the alternatives, the relevant burden and the decision they support. State any remaining uncertainty and how it limits the claim or affects the choice to continue, change, run a smaller comparison, defer expansion or stop.
MKT.3 can change the invitation; MKT.5 or MKT.6 can change the proposed contribution; MKT.11 can change the supply arrangement; MKT.13 can bound deployment of the commercial model. When an action is needed, identify it and who can take it.
Reopen the answer when the intervention, population, measurement, service capacity, economic conditions or a material causal assumption changes. Keep the original result with its scope; a later disappointing outcome does not justify rewriting what the earlier comparison observed.
MKT.10:5 - Archetypal Grounding
MKT.10:5.1 - A promising invitation with an uncertain useful effect
A provider wants to add guided onboarding to an invitation. The decision concerns another comparable group of 1,000 eligible customers. The useful outcome is verified completion of a specified task within 14 days, not clicking the invitation. Existing commitments remain available under both alternatives, and the provider has checked that the additional help can be delivered without reducing that service.
In a constructed trial, 2,000 eligible customers are randomly assigned equally to the new and current invitations. Assume complete follow-up, no material spillover and the same stated outcome rule. There are 120 useful completions among 1,000 assigned the new invitation and 100 among 1,000 assigned the current invitation. The estimated effect of the assigned invitation is 12% minus 10%, or 2 percentage points. Comparing only people who used the help would answer another question and would lose the original allocation’s simple comparability.
For this illustration, an approximate 95% confidence interval uses the difference plus or minus 1.96 times its standard error. With independent binary outcomes, that standard error is the square root of (0.12 × 0.88 / 1,000 + 0.10 × 0.90 / 1,000). The interval is approximately −0.7 to 4.7 percentage points. The positive point estimate is compatible with both a small loss and a worthwhile gain. It does not establish that the intervention improves outcomes. The interval also depends on the stated trial and estimation conditions; it says nothing about an omitted delayed outcome.
Suppose a qualified prospective account values each additional useful completion at 40 monetary units of contribution after its associated delivery expense, and another campaign for 1,000 customers would require 500 units of additional fixed preparation and help capacity. Assume these flows occur within the decision’s funded period. The point estimate gives 20 additional completions, a contribution of 800 and a net increment of 300. Roughly carrying the effect interval through this simplified account gives about −800 to 1,400, even before uncertainty in the value per completion. A positive expected increment alone is therefore a weak basis for an expensive irreversible expansion.
The decision maker chooses another affordable bounded comparison because the unresolved effect can change the decision. If even the favourable plausible effect could not cover the prospective burden, stopping would be more useful than collecting more data. If customers independently need the help to receive an existing promise, that obligation must be supplied regardless of this marketing experiment.
The next proposed group includes organisations with restrictive data access and would exhaust specialist capacity. It is no longer the same deployment condition. MKT.11 establishes a feasible service variant before its benefit is tested; multiplying the earlier 2% by the new audience would conceal the changed intervention.
MKT.10:5.2 - More support accompanies more failures
An observational report shows that customers receiving many support calls fail more often. Staff explain that they contact customers after detecting serious difficulties. The report may identify a high-burden group, but it does not establish that calls cause failure. The causal model retains initial difficulty and the timing of support; the analyst considers whether available observations permit an adequate contrast. If they do not, retain the supported description and state what prevents a causal conclusion.
MKT.10:6 - Bias-Annotation
Attribution systems, selective follow-up and incentives to show campaign success favour visible positive results. Customers who are easy to observe can differ from those who leave silently. The person commissioning an evaluation may prefer a precise answer even when the real uncertainty concerns identification. Preserve alternative explanations, missing outcomes and the possibility that no additional intervention is warranted.
MKT.10:7 - Conformance Checklist
- The answer serves the stated decision or claim and identifies its subject, quantity or outcome, and period. An effect or incremental claim specifies the alternative used for comparison.
- Observation rules distinguish the events and participants that affect interpretation.
- A causal claim has an explicit intervention contrast and an adequate identification basis; description or prediction is not silently promoted to effect.
- Estimation uncertainty and consequential causal or measurement limitations remain visible.
- A dated total covers the relevant entries once. An incremental economic comparison accounts for what differs between the alternatives in the period. A proposed action also accounts for the capacity it requires.
- The recipient can report or use the supported conclusion and recognise a change that would require revisiting it.
MKT.10:8 - Common Anti-Patterns and How to Avoid Them
| Anti-pattern | Correction |
|---|---|
| Crediting the last contact with every subsequent purchase. | Distinguish the attribution rule from the comparison needed to attribute an effect to that contact. |
| Calling a survey score the customer’s achieved result. | Observe the claimed outcome or narrow the conclusion to the reported experience. |
| Fitting more variables until the answer looks persuasive. | Obtain the causal model and identification grounds before interpreting estimates. |
| Declaring success from a positive point estimate alone. | Report the magnitude and uncertainty the claim requires. For a choice, also consider what a mistaken action would cost. |
| Expanding a supported pilot into an unsupported service queue. | Re-establish the intervention and capacity conditions for the proposed population. |
MKT.10:9 - Consequences
The evaluation becomes useful for the decision or claim under examination and limits claims that the data cannot support. It can reveal that a small descriptive answer is sufficient or that an expensive causal analysis would add too little for the decision or claim to justify its cost. Establishing a causal effect or applying its estimate to a new population, for example, may require additional observations, specialist work or a trial; unresolved uncertainty can legitimately limit expansion.
MKT.10:10 - Architectural Rationale
Marketing chooses the intervention question and uses its answer. Causal modeling supplies the explanatory structure, mathematical modeling supplies identification and estimation, and management accounting supplies the economic comparison. Use each contribution for the question it answers. A dashboard count, a fitted model and a revenue attribution rule each need further grounds before they can support an effect claim.
MKT.10:11 - SoTA-Echoing
For observational marketing models, adopt the distinction in Google Meridian’s causal-inference guidance between model fit and causal grounds: alternative well-fitting models can imply different returns, and consequential assumptions need domain justification. This supports bounded interpretation rather than selection by predictive accuracy alone.
For the construction of an answer, adopt C.28.CM and MMP.15’s developed separation of causal contrast, assumptions, identification and estimation, and use the resulting estimate within its supported scope. MA.8 supplies a dated economic account and, when the question concerns a difference, a comparison of alternatives. If an economic conclusion depends on the effect, carry the estimate and its uncertainty into the account. This adds work when an action or a claim depends on an effect; an adequate ordinary description remains sufficient for a descriptive question.
MKT.10:12 - Relations
MKT.3/5/6/7/9/11/14 supply interventions and receiving decisions. MKT.8 can locate an observed difficulty but does not identify its cause. MKT.12 can qualify the customer’s situation and the meaning of an outcome. MKT.13 uses an effect estimate only within its actual deployment conditions.
C.28.CM supplies the causal model, MMP.15 the identification answer and appropriate mathematical modeling the estimate. MA.8 supplies customer and product economics over time. A missing specialist result returns as a bounded unresolved contribution, not as an invented precise number.