Data-Driven Decision Making: From Question to Action

Begin data-driven decision making with the decision
Data-driven decision making starts with a choice someone needs to make, not a dashboard somebody wants to build. Identify the decision owner, the available options and the time at which an answer is needed. Then ask which evidence could change the choice.

A useful question is more precise than “How are we performing?” For a training team, it might be whether to revise an assessment instruction, offer additional practice or change a support process. Different choices need different evidence, and collecting more data does not automatically resolve an unclear question.

Define the measure and its limits
Specify what each variable represents, how it is collected and which population or period it covers. Distinguish participation, satisfaction, learning and workplace application. These measures can inform different questions, but they should not be substituted for one another without explanation.

Record missing data, changes in definitions and differences between groups. A trend can reflect a new collection process rather than a genuine change in the underlying situation. Ask what the measure cannot observe before using it to support a strong claim.

Compare plausible explanations
When an outcome changes, examine other changes that could matter. A higher completion rate may follow a shorter course, a different audience, a changed deadline or better support. The observed association alone does not establish which factor caused the result.

Combine appropriate quantitative and qualitative evidence when both help the question. For example, completion records may locate a pattern while interviews or an observation clarify a procedural barrier. Plan how the evidence will be integrated rather than adding quotations only to decorate a numerical conclusion.

Example: investigating unfinished training
In a constructed example, a team sees many learners stop before a final task. The first response is not to assume they lack motivation. The team checks whether the task is accessible, whether instructions are clear and whether completion is recorded correctly. It reviews an authorised sample of support questions.

The evidence suggests that learners cannot locate a required resource. The team revises the navigation and plans a follow-up check. The conclusion is bounded: the change addresses the identified access problem. It is not proof that every future completion issue has the same cause or that the entire programme is effective.

Use an analysis record someone else can inspect
Keep the source version, cleaning steps, definitions and calculation method together. If a spreadsheet contains manual corrections, document them rather than leaving the final numbers unexplained. Check totals and representative records against the source before interpreting a visualisation.

Select analysis methods that match the data and study design. Report uncertainty and important exclusions. A sophisticated model does not remove measurement bias, and a simple chart is not inferior when it answers the actual question transparently.

Turn the finding into a monitored action
Write the proposed action, its evidence, the assumptions and the person responsible. Identify what would show that the action needs adjustment. Avoid ending the work with a report nobody is authorised to use.

Review the decision after an appropriate period. Did the intervention reach the intended group? Did the measure behave as expected? What new information changes the interpretation? A data-driven process is a cycle of clear questions, appropriate evidence and revisable decisions, rather than a promise that numbers remove judgement.

Worked example: choose the denominator before the headline

In this constructed example, 20 staff are assigned a module, 18 start it, 12 complete it and 10 meet the specified assessment criterion. These are teaching figures, not an Oxford programme result. Completion among all assigned staff is 12 ÷ 20 = 60%. Meeting the criterion among completers is 10 ÷ 12, approximately 83.3%. Meeting it among all assigned staff is 10 ÷ 20 = 50%.

All three calculations can be correct while answering different questions. Reporting “83.3% success” without the denominator hides the staff who did not complete the module. Reporting only 50% may hide a useful distinction between access, completion and assessment difficulties. The next decision should investigate the relevant stage rather than choosing whichever percentage looks best.

Keep the definitions beside the chart or table. If the assessment criterion changes next month, mark that change before comparing percentages across periods. A consistent label cannot make two differently defined measures equivalent.

Editorial drafting used AI assistance. Examples labelled fictional or constructed are teaching examples, not client results. The original author record is retained; no independent specialist review is claimed.