Quality Risk Management — How Much Evidence Is Enough

• 1: FMEA in Detail — Scoring, Scaling, and Where It Breaks
• 2: Fault Tree Analysis — Working Backward From a Failure
• 3: HACCP — Critical Control Points, Borrowed From Food Safety
• 4: HAZOP — Deviations From Design Intent
• 5: Risk Ranking and Filtering — Comparing Risks That Don't Share a Scale
• 6: Ishikawa / Fishbone / PHA — Structuring the First Pass

The one idea

Two principles govern quality risk management: the evaluation of risk is grounded in scientific knowledge and ultimately links to protection of the patient; and the level of effort, formality, and documentation is proportionate to the level of risk.

Every analytical decision spends a finite budget of time, money, and attention. Risk management is how you point that budget at the failures that would actually hurt a patient, and stop gold-plating the ones that wouldn’t. It is the machinery behind “scientifically justified” — the phrase that appears in almost every ICH guideline and is doing a lot of quiet work.

The ICH Q9 framework

ICH Q9(R1) — Quality Risk Management (the R1 revision, adopted 2023, added guidance on subjectivity, the hazard-versus-risk distinction, formality, and risk-based decision-making). The process is a loop, not a form:

Stage	What happens	Analytical example
Risk assessment — identification	What could go wrong?	A co-eluting degradant is not resolved from the API
Risk assessment — analysis	How likely, how severe, how detectable?	Estimate occurrence from forced-degradation data; severity from the degradant’s qualification threshold; detection from method specificity
Risk assessment — evaluation	Is that acceptable against defined criteria?	Compare against a risk threshold agreed before the assessment
Risk control — reduction	Change the design to lower likelihood or raise detection	Switch to an orthogonal column; add a peak-purity check
Risk control — acceptance	Some residual risk is accepted, explicitly and on the record	Document the residual and the justification
Risk communication	The assessment and decisions are shared with everyone who acts on them	The control strategy, the filing, the SOP
Risk review	Revisit when something changes	A new impurity at month 9 of stability reopens the assessment

Two ideas from Q9(R1) matter for the analyst:

• Hazard is not risk. A hazard is the potential to cause harm; risk combines the probability of that harm with its severity. “This solvent is toxic” is a hazard statement; “at the residual level this method can detect, the exposure is X% of the PDE” is a risk statement.
• Formality is a dial, not a switch. A one-line rationale, a risk-ranking table, and a full cross-functional FMEA are all valid quality risk management — the guideline asks you to match the formality to what is at stake, and to say why.

The toolbox

Each tool below gets its own full walkthrough — mechanics, a worked analytical example, and where it breaks down:

Tool	Best for	Notes
FMEA / FMECA	Failures of a process or method built from many steps	The workhorse in analytical development — full walkthrough →
Fault tree analysis (FTA)	Working backward from one defined failure to its contributing causes	Good for OOS root-cause work
HACCP	Identifying and controlling critical points in a process	Origin in food safety; maps well to manufacturing
HAZOP	Deviations from design intent, guided-word by guided-word	More common in process/engineering than in the QC lab
Risk ranking and filtering	Comparing many risks that don’t share a scale	Portfolio-level and site-level decisions
Ishikawa (fishbone) / PHA	Structuring a first-pass hazard identification	Often the front end of an FMEA

FMEA in action

Failure Mode and Effects Analysis decomposes a method or process into steps, and for each step asks: what could fail (failure mode), what would that do (effect), why would it happen (cause), and how would we catch it (controls). Each mode is scored:

Risk Priority Number = Severity × Occurrence × Detection

• Severity — how bad the effect is for the patient or the decision (a wrong release decision scores high; a re-run scores low).
• Occurrence — how often the cause is expected to produce the failure.
• Detection — how likely the existing controls are to catch it before it matters. High detection score = poorly detected — this scale runs backward, and it is where most FMEAs go wrong.

Modes with a high RPN, or a high severity regardless of RPN, get an action; then the mode is re-scored to show the action worked. The number is easy to game and easy to over-trust — see the full FMEA walkthrough for a worked multi-failure-mode example and the known weaknesses worth teaching so students don’t over-trust it.

From risk assessment to control strategy

A control strategy is the planned set of controls — derived from current product and process understanding — that assures performance and quality. It is the output of risk management, not a separate exercise:

• Attribute risk assessment decides which quality attributes are critical (CQAs) and therefore need a specification and a method.
• Method risk assessment (an FMEA against the analytical target profile) decides which method parameters need to be controlled, and how tightly — this is where a robustness study is a risk-control activity, not a validation checkbox.
• The specification (Q6) and the stability program (Q1) are risk decisions in numeric form.
• Under the analytical procedure lifecycle, what counts as a reportable change to a method is set by the risk it carries.

Worked example — nitrosamine risk assessments. Between 2018 and 2023 every marketing authorization holder had to assess every product for the risk of N-nitrosamine impurities (NDMA, NDEA, and drug-specific nitrosamines), triggered by the valsartan recalls. The assessment is a textbook QRM: identify the hazard (potent mutagenic carcinogens), analyze the risk (synthetic route, nitrite sources, secondary amines, recovered solvents, water; then confirmatory testing), control it (route changes, nitrite scavengers, tightened limits at ppb levels), and communicate it (to the agency, on a deadline). It also shows the analyst’s exposure directly: the risk conclusion depended entirely on whether a method existed that could see a nitrosamine at its acceptable intake — a detection problem.

---

Source note. Risk management is anchored in ICH Q9(R1), with ICH Q8(R2), Q10, and Q14. FMEA methodology follows IEC 60812 and the AIAG-VDA FMEA handbook. The nitrosamine case follows the EMA/FDA guidance and Article 5(3) referral outcomes. (Instructor: confirm the Q9(R1) adoption date and current EMA nitrosamine guidance revision.)

1 - FMEA in Detail — Scoring, Scaling, and Where It Breaks

The one idea

RPN is a prioritization tool, not a measurement. It tells you which failure mode to look at first — it does not tell you how much worse one failure mode is than another, and treating it like it does is the single most common way an FMEA goes wrong.

Mechanics

Failure Mode and Effects Analysis decomposes a method or process into steps, and for each step asks: what could fail (failure mode), what would that do (effect), why would it happen (cause), and how would we catch it (controls)? Each mode is scored on three independent 1–10 scales and multiplied:

Risk Priority Number = Severity × Occurrence × Detection

• Severity — how bad the effect is for the patient or the decision. A wrong release decision (a failing batch shipped, or a good batch scrapped) scores high; a re-run that costs a day scores low.
• Occurrence — how often the cause is expected to produce the failure, from historical data or, absent that, engineering judgment.
• Detection — how likely the existing controls are to catch the failure before it matters. High detection score = poorly detected — this scale runs backward from the other two, and it is where most FMEAs go wrong: a “10” means “we would almost certainly miss this,” not “we’d definitely catch it.”

Modes with a high RPN, or a high severity regardless of RPN, get a corrective action; the mode is then re-scored to show the action actually moved the number, not just noted “action taken.”

Worked example — an HPLC assay method

Two things worth noticing in this table: the carryover mode (RPN 200) outranks the drifting-calibration mode (RPN 108) even though a wrong release decision from a drifting curve is arguably worse — because carryover’s detection score was so bad (8: nobody was actually looking for it). That is RPN doing its job: surfacing the blind spot, not just the scariest-sounding failure.

Why the same RPN can mean very different things

Both score 90. Mode A is a rare but severe failure that’s moderately well detected; mode B is a more frequent, less severe failure that’s poorly detected. A severity-first reviewer would act on A first regardless of the tied RPN — which is exactly the argument for not ranking a whole FMEA by RPN alone, and for flagging any mode with severity ≥ 9 for action independent of its RPN.

FMEA vs. FMECA

FMECA adds a formal criticality analysis on top of FMEA — instead of (or alongside) the RPN product, each failure mode’s criticality is assessed against a defined severity/probability matrix, often with failure-mode ratios when one cause can produce several distinct failure modes. In practice, most analytical-development FMEAs are really FMECAs in miniature: teams already flag “any severity ≥ 9 regardless of RPN” as an action trigger, which is a criticality rule, not a pure RPN rule.

Known weaknesses — worth teaching so students don’t over-trust the number

• RPN is an ordinal product treated as if it were interval data; an RPN of 100 is not “twice as bad” as 50, and — as shown above — different (S, O, D) triples give the same RPN with very different meaning.
• Detection and occurrence are often guessed. Q9(R1) explicitly flags this subjectivity and asks for it to be managed (defined scales, cross-functional scoring, documented assumptions).
• Many programs now supplement or replace RPN with a severity-first criticality matrix, or with risk ranking and filtering when comparing failure modes across unrelated processes.

When to reach for something else

FMEA decomposes one process step by step and scores every mode on the same three scales — it’s the right tool when the process is defined and you’re building or revising its control strategy. Reach for fault tree analysis instead when you’re working backward from a failure that has already happened and need to trace its root cause; reach for risk ranking and filtering when you’re comparing risks that don’t share a process or a scale at all.

2 - Fault Tree Analysis — Working Backward From a Failure

The one idea

FMEA asks, before anything has gone wrong, “what could fail in this process?” FTA asks, after something already has, “what chain of causes could have produced exactly this failure?” They run in opposite directions through the same failure space, and a mature quality system uses both.

Mechanics

A fault tree starts with a single, precisely defined top event — the failure that occurred — and branches downward through logic gates to the conditions that could produce it:

• An AND gate means every branch beneath it must be true for the event above to occur (e.g., a wrong result reaches release and the reviewer misses it).
• An OR gate means any one branch beneath it is sufficient (e.g., a degraded standard, a mis-set instrument parameter, or a transcription error could each independently cause a wrong reported value).

The tree bottoms out in basic events — causes you either confirm or rule out with data, not further decomposition. No formal Boolean notation is required to use this at the bench; the value is in the discipline of writing every “or this could have happened” branch down before deciding which one is true.

Worked example — an out-of-specification (OOS) assay result

Top event: Reported assay result outside the specification range.

Reported assay OOS
 └─ OR: Result is a true failure vs. a lab/analytical error
     ├─ OR (analytical/lab error branch)
     │    ├─ Standard was out of date or degraded
     │    │    → check standard prep date, storage conditions, prior QC data
     │    ├─ System suitability failed but was overridden or missed
     │    │    → review the suitability data logged that run
     │    ├─ Sample preparation error (dilution, weighing, transcription)
     │    │    → re-check the prep worksheet against the raw balance/pipette record
     │    └─ Instrument malfunction (detector drift, pump seal, injector carryover)
     │         → review instrument maintenance and diagnostic logs
     └─ AND (true-failure branch)
          ├─ Manufacturing process produced an out-of-spec batch
          │    → review batch record deviations, in-process controls
          └─ No analytical error found in the OOS investigation above
               → confirms the result should stand

An OOS investigation under Q7/GMP follows exactly this shape: Phase I (laboratory investigation) works the analytical-error branches first, because a confirmed lab error can invalidate the result without ever reaching the manufacturing branch; Phase II (full investigation) only proceeds down the true-failure branch once Phase I finds no assignable analytical cause.

When to reach for it vs. FMEA

FTA is reactive — it exists because a specific, already-observed failure needs a root cause, and it only makes sense once that top event is precisely defined. FMEA is prospective — it exists to find failure modes before they happen, and it doesn’t require anything to have gone wrong yet. In practice, a documented FMEA is often what an OOS investigation checks against: “was this failure mode already identified, and if so, why did the existing control not catch it?”

Known weaknesses

• FTA is only as good as the top event’s definition — a vaguely stated failure (“something went wrong with the assay”) produces an unusably broad tree.
• Trees for a complex, multi-step method can become large fast; without discipline about what counts as a “basic event,” the tree can sprawl without converging on an actionable root cause.
• FTA doesn’t score or prioritize the way RPN does — it’s a diagnostic tool for one failure, not a ranking tool across many, which is why it’s typically paired with an FMEA or risk ranking rather than used as the whole risk program.

3 - HACCP — Critical Control Points, Borrowed From Food Safety

The one idea

Instead of scoring every failure mode in a process, HACCP asks a narrower, sharper question: where in this process is a critical control point — a step where losing control means the hazard reaches the patient, with nothing downstream left to catch it?

Mechanics

HACCP originated in food safety (developed for NASA’s manned space program, to guarantee astronaut food had zero tolerance for contamination) and maps cleanly onto sterile and biologic manufacturing, which share that same “no downstream catch” property. The full method has seven principles; the ones that matter for a control-strategy discussion are:

1. Conduct a hazard analysis — what biological, chemical, or physical hazards could occur at each process step?
2. Identify critical control points (CCPs) — of all the steps, which ones are the point where the hazard can still be prevented, eliminated, or reduced to an acceptable level? A step downstream of the true control point is not itself a CCP, even if a hazard could theoretically show up there.
3. Establish critical limits — a measurable threshold for each CCP (a temperature, a pressure differential, a bioburden count) that separates “in control” from “out of control.”
4. Establish monitoring — how and how often the critical limit is checked, and by whom.
5. Establish corrective action — what happens, specifically, the moment a critical limit is exceeded.

(The remaining two principles — verification and record-keeping — are the documentation backbone that makes the first five auditable, and aren’t specific to any one CCP.)

Worked example — sterile fill/finish bioburden control

The filtration step is the CCP because it is the last point where the hazard (a non-sterile product) can still be prevented — everything upstream can be caught or corrected later in the process, and everything downstream has no way to remove an organism that already got through. That is the test for “is this a CCP,” not “could something go wrong here.”

When to reach for it vs. FMEA

FMEA decomposes an entire process into every failure mode and scores each one — useful when you want comprehensive coverage of a method or process. HACCP deliberately does the opposite: it narrows attention to the small number of points where losing control is unrecoverable, which is exactly right for manufacturing and process risk (sterility assurance, allergen control, cross-contamination) but a poor fit for analytical method risk, where FMEA’s step-by-step, fully-scored decomposition is what regulators and most labs actually expect.

Known weaknesses

• Works best when there really are a small number of make-or-break points; forcing a HACCP structure onto a process with many, roughly-equally-important risks just reproduces an FMEA with extra steps.
• Identifying the true CCP takes real process understanding — misidentifying a downstream inspection point as a CCP gives false confidence, since it doesn’t actually prevent the hazard, only detects it after the fact.
• Less natural for analytical-method risk (where FMEA dominates) than for manufacturing/process risk, where it originated and still fits best.

4 - HAZOP — Deviations From Design Intent

The one idea

HAZOP doesn’t start from a list of known failure modes the way FMEA does — it starts from the process’s own design intent and systematically asks what happens if reality deviates from it, one guide word at a time, parameter by parameter.

Mechanics

For each parameter at each step of a process (flow rate, temperature, pressure, pH, concentration, time), a HAZOP team applies a fixed set of guide words and asks what a deviation of that kind would actually cause:

Unlike FMEA, HAZOP doesn’t score every deviation on Severity/Occurrence/Detection — the output is a qualitative list of credible deviations, their causes, consequences, and existing safeguards, with follow-up actions where the safeguards look thin.

Worked example — HPLC flow rate and a bioreactor’s temperature

Notice the bioreactor row: a MORE temperature deviation doesn’t just risk an obvious failure (dead cells) — it can silently shift a critical quality attribute (glycosylation) while the culture still looks healthy, which is exactly the kind of consequence a guide-word walk-through is designed to surface deliberately, rather than relying on someone to have already thought of it.

When to reach for it vs. FMEA

HAZOP and FMEA overlap heavily in outcome — both end up identifying deviations and their consequences — but HAZOP is organized around the process’s design intent, parameter by parameter, which makes it a natural fit for engineering and process-design teams examining a new unit operation (a reactor, a filtration skid, a chromatography skid) before it’s ever run. Most QC labs default to FMEA for method risk because the “steps” of a method are already well defined; HAZOP earns its keep more in process/engineering contexts where the parameters, not discrete process steps, are the natural unit of analysis.

Known weaknesses

• Applying every guide word to every parameter at every step can be slow and exhaustive for a complex process — teams often scope it to the parameters most likely to matter, which reintroduces some of the same judgment calls HAZOP is meant to avoid.
• Without a scoring step, prioritizing which deviations to act on first is a separate, later exercise — HAZOP tells you what could deviate, not which deviation matters most.
• The overlap with FMEA means running both on the same process is often redundant; most sites pick one as the primary tool for a given risk type (HAZOP for process design, FMEA for methods) rather than running both routinely.

5 - Risk Ranking and Filtering — Comparing Risks That Don't Share a Scale

The one idea

FMEA scores risks within one process on one shared scale. Risk ranking and filtering compares risks across processes, products, or sites that have no natural shared scale at all, by explicitly defining and weighting the criteria that make one risk matter more than another.

Mechanics

1. Define criteria that matter across every risk being compared — typically patient impact, regulatory exposure, likelihood, and detectability, though a portfolio-level exercise might add business impact or timeline pressure.
2. Weight the criteria to reflect what actually matters most in this decision (patient impact usually carries the most weight; timeline pressure usually carries the least, if it’s included at all).
3. Score each risk against every criterion, using whatever scale is practical (often 1–5, sometimes qualitative bands converted to numbers).
4. Compute a weighted score and rank — then filter: set a threshold or a headcount/budget cutoff and act on what clears it, explicitly documenting why anything below the line is being deferred.

The “filtering” half is as important as the ranking half — the exercise exists to produce a short, defensible action list, not just a long sorted table nobody acts on.

Worked example — a site quality council’s quarterly resourcing decision

Five unrelated findings are competing for the same limited investigation and remediation budget this quarter:

Ranked and filtered against a “fund the top three this quarter” cutoff: the stability OOS trend (26) and the documentation deviation (23) fund first regardless of tiebreaks; the method-transfer gap and the inspection commitment tie at 19 and need a secondary criterion (e.g., regulatory due date) to break the tie for the third slot. The aging-fleet risk (13) is explicitly deferred — not ignored, documented as deferred, with the reasoning on record for the next review cycle.

When to reach for it vs. FMEA

Use risk ranking and filtering when the decision spans multiple unrelated risks competing for the same finite resource — funding, staffing, audit time — not when you’re working through the failure modes of a single process or method, which is FMEA’s job. It’s the tool for “which of these five different problems do we fix first,” not “what could go wrong in this one method.”

Known weaknesses

• The weighting scheme is itself a subjective judgment call — this is the same criticism Q9(R1) raises about FMEA’s Severity/Occurrence/Detection scoring; risk ranking and filtering doesn’t remove that subjectivity, it just moves it up a level, from scoring individual failure modes to weighting the criteria that compare them.
• Different stakeholders (quality, manufacturing, regulatory affairs) often disagree on the weights themselves — reaching agreement on the weighting is frequently the harder part of the exercise, not the scoring.
• A weighted score can create false precision — a 26 vs. a 23 looks decisive, but both numbers rest on the same soft inputs as any other risk score, and the ranking should be sanity-checked qualitatively before being treated as a tiebreaker.

6 - Ishikawa / Fishbone / PHA — Structuring the First Pass

The one idea

An FMEA is only as complete as its failure-mode list, and that list has to come from somewhere. Ishikawa (fishbone) diagrams and Preliminary Hazard Analysis are how you brainstorm it systematically, category by category, instead of relying on whoever’s in the room to remember everything from experience.

Mechanics

An Ishikawa diagram starts from a defined effect (an observed or feared problem) and branches into standard categories of contributing cause. Adapted for an analytical lab, the categories are usually:

• Method — the procedure itself: parameters, steps, order of operations
• Materials — reagents, standards, reference materials, columns, consumables
• Machine — instrumentation: hardware, software, calibration state
• Manpower — analyst training, technique, fatigue, handoffs
• Environment — temperature, humidity, lighting, vibration, power quality

Preliminary Hazard Analysis (PHA) is a lighter, earlier-stage cousin — a first-pass brainstorm of what could possibly go wrong before a process even exists in detail, often just a simple hazard/cause/effect table, used to scope what a later, more formal risk assessment needs to cover.

Worked example — “unexpected peak in a stability sample”

This is deliberately a long, unfiltered list — the point of the fishbone pass is coverage, not judgment. Three or four of these branches then become the failure-mode column of a follow-on FMEA: “contaminated mobile phase” becomes a scoreable failure mode with its own severity, occurrence, and detection; “detector lamp aging” becomes another. The fishbone did the brainstorming; the FMEA does the prioritizing.

When to reach for it vs. FMEA directly

Skip straight to FMEA when the failure modes are already well understood from experience — a mature, well-characterized method rarely needs a fresh fishbone pass. Reach for Ishikawa or PHA first when the process or method is new or unfamiliar, or when a cross-functional team is starting from very different mental models of what could go wrong and needs a shared, structured brainstorm before anyone starts scoring anything.

Known weaknesses

• Purely qualitative — a fishbone diagram or PHA table has no scoring or prioritization built in; it can surface a long list of contributing factors without telling you which ones actually matter.
• Coverage depends heavily on who’s in the room; the category headings help structure the brainstorm, but they don’t guarantee completeness the way a systematic top-down decomposition (like FMEA’s step-by-step structure) does.
• It is not, on its own, a complete quality risk management record — it’s the front end that typically feeds into an FMEA or risk ranking and filtering exercise, not a substitute for either.