Safety Risk Matrix Builder: Score Hazards Consistently Across Departments

Interactive worksheet

Define four or five severity levels using credible operational consequences and four or five likelihood levels using an agreed exposure period. Map each intersection to a risk class, acceptance authority, response time and required treatment. The tool should record hazard, causes, existing controls, evidence, initial score, actions, owner, due date, residual score and approval. Test sample scenarios across departments and revise ambiguous definitions before release.

The output is an operational estimate, not an official determination. Record the source and effective date for every rule, threshold or benchmark, preserve the input snapshot, and require review where the result affects compliance, safety, tax, certification or a material investment decision.

A dashboard can be accurate and still be useless

The monthly safety pack arrives with report counts, audit findings, bird strikes, injuries, training completion and a grid of red, amber and green tiles. The figures may be correct. Yet the meeting still depends on the safety manager explaining what changed, whether it matters and what anyone should do next.

That is the failure. The dashboard has transferred data, but not insight.

A Safety Management System (SMS) dashboard is not a gallery of every metric the organisation can calculate. It is a decision interface. Its job is to help a defined user recognise a material change, understand the evidence, decide a response and confirm that the response worked.

This distinction is consistent with the formal purpose of safety performance management. ICAO's Safety Management Manual guidance links safety objectives, safety performance indicators (SPIs), targets and triggers, then describes monitoring as an ongoing activity that provides early warning for informed decisions. FAA AC 120-92D similarly treats an SMS as part of operational and business decision-making, not as a reporting layer added after the work.

If a tile cannot change a decision, it probably does not belong on the primary view.

The seven reasons SMS dashboards lose their audience

1. They count events without measuring exposure

“Ground damage increased from four to six” sounds worrying. But the interpretation changes if aircraft turns doubled, stayed flat or fell by half. Counts are necessary for reconciliation; rates are usually better for comparison.

Choose a denominator that represents the opportunity for the event:

• unstable approaches per 1,000 approaches;
• ramp damage events per 10,000 aircraft turns;
• maintenance errors per 1,000 work packages or labour hours;
• wildlife strikes per 10,000 movements;
• fatigue reports per 1,000 duty periods.

Display the count and the exposure beside the rate. A rate without its underlying volume can overstate movement in small samples. A count without exposure can confuse operational growth with deteriorating safety.

2. They treat fewer reports as better safety

Voluntary report volume is partly a measure of reporting behaviour. A decline may mean fewer hazards; it may also mean staff have stopped reporting because feedback is slow, the form is difficult or they fear consequences. An increase may reflect a deteriorating operation, a successful reporting campaign, or both.

Do not label report count green when it falls and red when it rises. Read it with measures such as time to acknowledge, share of reporters receiving feedback, anonymous versus named submissions, distribution by station or department, repeat reporting, and the number of reports that lead to risk review or action.

The defensible question is not “Did reports go down?” It is “What does this change tell us about exposure, hazards and reporting confidence?”

3. They mix leading activity with safety outcomes

Training completed, audits performed and risk assessments closed are activities. Loss of separation, runway excursion precursors, maintenance errors and injuries are outcomes or operational events. Both matter, but they answer different questions.

A high training-completion percentage does not prove that a risk control works. The useful chain is:

safety objective → risk control → implementation measure → control-effectiveness measure → safety outcome

For example, “100% recurrent de-icing training completed” shows implementation. An observed-practice check, fluid application deviation rate or repeat finding can provide evidence about effectiveness. A dashboard that stops at completion rewards administration rather than risk reduction.

4. They use arbitrary red, amber and green rules

A coloured tile creates the appearance of judgement. Unless the threshold has a rationale, it merely hides judgement.

Different signals need different trigger methods. One serious event may require immediate review even when its rate remains statistically “normal”. A high-volume, lower-consequence indicator may be suited to a control chart or threshold based on stable historical performance. A target may represent desired improvement, while a trigger identifies a condition that requires attention; the two are not interchangeable.

Every colour must disclose:

• the current value and measurement period;
• the baseline, target or trigger it is compared with;
• the direction and magnitude of change;
• the denominator and sample size;
• the rule that caused the status;
• the action required and its owner.

Never rely on colour alone. Use text, shape or an explicit status so the view remains accessible and printable.

5. They give every audience the same screen

The accountable executive, safety review board, station manager and safety analyst do not make the same decisions. One universal dashboard either overwhelms leaders or starves specialists.

Build views around decision rights:

Audience	Decisions the view should support	Appropriate content
Accountable executive and board	Where risk exceeds appetite; where resources or executive intervention are needed	Top operational risks, breached triggers, control effectiveness, overdue high-risk actions, material change
Safety review board	Whether objectives and SPIs remain valid; which response to authorise	Trends with exposure, targets and triggers, cross-functional hotspots, mitigation progress, assurance results
Operational owner	What to correct this shift, week or cycle	Local precursors, exceptions, repeat events, open actions, named constraints
Safety analyst	Whether the signal is real and what explains it	Event-level drill-down, classifications, distributions, confidence, data completeness and source lineage

These are not four disconnected products. They are four levels of the same governed data model. A leader should be able to move from a breached trigger to the trend, contributing locations, relevant hazards, controls and open actions without requesting a new spreadsheet.

6. They end at the signal

Many dashboards can show that something is red. Few show what happened next.

A useful alert has a lifecycle:

1. Detected: a defined rule identifies a signal.
2. Acknowledged: a named owner accepts review responsibility.
3. Assessed: the owner tests data quality, exposure and operational context.
4. Decided: the safety forum accepts, monitors, mitigates or escalates the issue.
5. Acted upon: tasks have owners, dates and evidence.
6. Verified: assurance checks whether the control was implemented and effective.
7. Closed or revised: the organisation records the rationale and updates the indicator if needed.

If the dashboard does not show acknowledgement, decision, action and verification, it is an alarm panel with no response log. Repeated red tiles then become background noise.

7. They conceal the quality of the underlying data

A polished trend can be built from incomplete exposure records, inconsistent event classification, late submissions and duplicate occurrences. The visual remains precise while its meaning decays.

Place a data-confidence marker beside each material indicator. At minimum, monitor:

• source coverage and missing exposure records;
• mandatory-field completeness;
• reporting delay and backlog;
• duplicate or unmatched records;
• classification changes after initial submission;
• unexplained “other” and “unknown” use;
• percentage of events awaiting validation;
• changes to definitions, denominators or source systems.

Do not silently restate a trend after a data correction. Record the change, effective date and impact on comparability.

Start with the decision, not the chart

Before choosing a visual, complete one sentence:

When this condition occurs, this role will make this decision within this time, using this evidence.

If the sentence cannot be completed, the metric is not ready for a dashboard.

Then build a metric contract. The contract is the small piece of governance that prevents the same number from acquiring different meanings in different meetings.

Field	Question to answer
Safety objective	What safety outcome are we trying to achieve?
Indicator	What observable measure provides evidence?
Numerator	Which events or conditions count?
Denominator	What exposure makes periods and units comparable?
Inclusion and exclusion	Which operations, fleets, stations or event states apply?
Baseline	What reference period and operating context are valid?
Target	What improvement or sustained result is sought?
Trigger	What condition requires review or action?
Owner	Who explains the signal and owns the response?
Review cadence	How often can the data support a meaningful decision?
Drill-down	Which evidence must be available to investigate?
Change control	Who approves a definition or calculation change?

ICAO's Indicator Catalogue offers a harmonised starting framework, but ICAO makes clear that catalogue inclusion does not mandate an indicator. An organisation still has to select measures that fit its hazards, objectives, data and operation.

What a decision-led safety panel contains

One panel should answer one operational question. For a material SPI, show:

1. Plain-language question: “Are unstable approaches becoming more frequent?”
2. Current status: value, count, exposure and period.
3. Trend: enough periods to distinguish variation from a single-month jump.
4. Reference: target, trigger or expected range, with its basis.
5. Context: fleet, route, runway, station, phase of flight or other relevant segmentation.
6. Confidence: completeness, validation status and small-number warning.
7. Response: owner, decision state, due date and latest evidence.

Use annotations for operational changes: a new route, revised SOP, fleet introduction, reporting campaign or source-system migration. Otherwise the reader sees a bend in a line without the context that might explain it.

Tables are often better than charts for exceptions. If a manager needs to know which five overdue high-risk mitigations need intervention, provide those five rows with owner, due date, blockage and next action. A doughnut chart of open versus closed actions makes the manager calculate the answer and then ask for the list.

Use different logic for rare and frequent events

A single dashboard rule should not be applied to every type of safety data.

For frequent indicators with reasonably stable exposure, trend methods or control limits can help distinguish routine variation from a meaningful shift. Segment before interpreting: an apparently stable organisation-wide rate may hide deterioration at one station or within one fleet.

For rare, high-consequence occurrences, historical averages may be poor comfort. A single event can justify investigation based on severity or credible worst outcome, regardless of whether a numerical trigger is breached. Track precursor conditions and the health of critical controls, while keeping the actual occurrence visible.

For small samples, show the numerator and denominator prominently and avoid dramatic percentage claims. Moving from one event to two is a 100% increase, but that statement alone says little about underlying risk. Aggregate periods only when doing so preserves the decision value and does not conceal an urgent event.

A dashboard should expose control effectiveness

Hazards do not become controlled because an action was marked complete. The SMS needs evidence that the control exists, is used and changes risk as intended.

For each top risk, distinguish:

• design assurance: is the control capable of addressing the hazard?
• implementation assurance: is it deployed where required?
• operating assurance: is it performed consistently in live operations?
• effectiveness assurance: are the precursor or outcome measures responding as expected?

This is where many executive dashboards are weakest. They report risk ratings and action closure but omit whether the barriers work. EASA's Management System Assessment Tool emphasises identifying hazards, managing risk, monitoring mitigation effectiveness, monitoring progress towards safety objectives and taking timely managerial action. It also warns that compliance does not necessarily equal safety.

Replace “96% actions closed” with a more useful set: high-risk actions overdue, controls awaiting effectiveness verification, recurring findings after closure, and risks whose residual rating depends on an unverified control.

Design for trust and reporting culture

Drill-down creates a tension. Managers need enough detail to act; reporters need confidence that safety information will be handled appropriately.

Apply access by role. Executive views rarely need reporter identities. Operational views can display pattern, location and event context while restricting personal data to authorised investigators. Keep access logs, classification history and the rationale for changes. Publish indicator definitions so staff can understand how their reports contribute to learning.

Close the feedback loop. A dashboard can show response timeliness, decisions communicated and improvements traced to reports. Those measures make the SMS visible as a learning process, not a one-way collection system.

A practical dashboard review

Take the current monthly pack and ask these questions for every panel:

• Which named role uses it?
• Which decision could change because of it?
• Is it connected to a safety objective, hazard or control?
• Are the count, exposure and time period visible?
• Is the threshold defined and appropriate to the signal?
• Can a user distinguish change in risk from change in reporting?
• Does it expose missing or unvalidated data?
• Can the user reach the contributing evidence?
• Is there a named owner and predefined response?
• Does it show whether earlier actions were effective?

Remove panels that cannot answer the first two questions. Repair metric definitions before redesigning visuals. Then test the revised view in the actual meeting: record which panels prompted a decision, which prompted a request for unavailable evidence, and which nobody used. Dashboard usage is itself an indicator.

FAQ

How many KPIs should an aviation SMS dashboard have? There is no universal number. The primary view should contain only the indicators needed for that audience's recurring decisions. A smaller executive set can link to a broader governed catalogue for safety teams and analysts.

Should all safety indicators have targets and triggers? No. A target expresses a desired result; a trigger defines a condition for attention. ICAO notes that Annex 19 does not require a trigger for every SPI. Use thresholds where they improve a decision, and document the rationale.

Are incident-report counts a leading indicator? Not by themselves. Report volume combines occurrence opportunity, operational exposure and willingness to report. Interpret it alongside reporting-process measures and the hazard content of reports.

Is red, amber and green status bad practice? Not necessarily. It becomes bad practice when colours use arbitrary thresholds, hide the underlying values or carry no defined response. Always pair status with text and evidence.

How often should an SMS dashboard update? Match cadence to the decision and the data. An urgent high-severity signal may require immediate notification. A statistically weak monthly rate may be more meaningful over a rolling period. “Real time” is not inherently better if nobody has a real-time decision to make.

Where a system helps

The hard part is not drawing charts. It is maintaining the chain from safety objective to indicator, source data, trigger, decision, action and effectiveness evidence. A connected aviation safety platform can give each role a focused view while preserving governed definitions, investigation detail, access controls and an auditable response trail.

That turns the dashboard from a monthly presentation into part of the SMS operating rhythm. Explore SafetyMS for aviation safety.

Related reading: What Is a Safety Performance Indicator in Aviation? (KB-105) and What Is a Safety Case in Aviation? (KB-106).

Sources

• ICAO Safety Management Manual, Chapter 4: Safety Performance Management
• ICAO Safety Indicator Catalogue
• FAA AC 120-92D, Safety Management Systems for Aviation Service Providers
• EASA Management System Assessment Tool