Knowledge Management — From Data to Wisdom, and the Recipe That Ties It Together

The lab automation section ended with the analyst owning a model instead of running a sample. Automation gave the laboratory horsepower — hardware and software that generate results faster than any person can read them. This section is about the other half of that bargain: once you can make data that quickly, what turns it into something worth keeping?

The one idea

Automation is very good at producing data. It is completely indifferent to whether that data ever becomes knowledge. Left alone, a modern lab generates terabytes that are never looked at twice.

Your job as a scientist is to publish the right data, in the right form, with the right context, at the right time — so that it can be transformed into information, and then into durable knowledge about your product.

Knowledge management is the discipline that makes that transformation deliberate instead of accidental. It is not an IT function bolted on at the end; it is a design choice made every time someone decides what to measure, how to record it, and how to describe it.

The DIKW pyramid

The useful mental model is the data → information → knowledge → wisdom hierarchy (often attributed to Russell Ackoff’s 1989 “From Data to Wisdom”). Each level is built from the one below it by adding context, then pattern, then judgement.

Two things fall out of the table. First, the jump from data to information is where most knowledge is lost — a number with no method, no units, and no provenance can never be promoted. Second, knowledge is what lets you explain the past and wisdom is what lets you predict the future; both are the output of a model, which is why the rest of this section is largely about models.

Every scientist is a data scientist

There is a stronger claim behind the pyramid:

I have never met a scientist who is not a data scientist. I have met data scientists who are not scientists.

The skills that used to belong to a specialist — structuring data so it can be queried, keeping an analysis reproducible, versioning a method, sharing it so someone else can run it — are now part of doing pharmaceutical analysis at all. The Turing Way is a good, free handbook for exactly this: reproducible, ethical, collaborative data science. Its commitments line up point-for-point with the definition of science from Section 1 — open exchange of data and procedures, and results that hold up when someone else re-runs them.

The practical version for this course: a result you cannot reproduce, or a method you cannot hand to another lab, has not finished being science — the same lesson as method transfer and deployment in Quality Control, seen from the data side.

Formal and informal models

A model is something a community creates, amends, and interprets in order to refine its shared knowledge of some area of interest. Models explain past phenomena (knowledge) and predict future ones (wisdom). They come in two kinds, and the difference matters enormously.

Legislation is the cleanest example of an informal model: natural language, read in context, with loopholes that are easy to create and hard to test. Great science runs the other way. It is built on formal models because they can be tested cheaply and communicated without ambiguity.

The goal of knowledge management in a pharmaceutical organisation is therefore not just to store knowledge but to push it from informal toward formal — from “the operators know the dryer runs hot in summer” to a moisture model with a control limit. That progression is the same thing earlier sections called demonstration of control: moving from we think this is unlikely to here is the model that says so.

What is a database, really?

A database is, at its plainest, an organised collection of tabulated data. The history is worth a minute because each era solved a problem the last one exposed:

Follow that last row to its conclusion and you arrive at self-describing data: a record that carries its own schema, units, and context, so it means the same thing to whoever opens it, whenever they open it. Which sounds a great deal like a good laboratory notebook — a self-contained, self-explaining account of what was done and what was found. Achieving that at scale, digitally, is genuinely hard, and it is why the industry keeps building bigger containers for data.

Databases, warehouses, lakes, meshes

Each step widens what you are willing to integrate:

Underneath all of them is a joke with a point in it: most data integration is still homo sapiens integration — a scientist opening three systems in three browser tabs and reconciling them by hand. Warehouses and lakes are attempts to do that same reconciliation automatically and at scale. The value of a warehouse is not storage; it is that you can pull diverse data back out and build mathematical models across it — models of your product that no single source could support.

Automation is the engine that makes this run, at three points at once:

1. Generate the data — instruments, PAT probes, automated workflows.
2. Organise the data — capture it with its context, into a structure that can be queried.
3. Analyse and report the data — multivariate analysis, models, and the reports that go to reviewers and regulators.

Automate only step 1 and you have made the integration problem worse. The goal across all three is the same as before: move from informal to formal models, and use them to demonstrate scientific understanding of the product.

The recipe as the informational framework — S88/S95

Everything above needs a skeleton to hang on — a shared way of saying what step of what process produced this number, on what equipment, for what material. The ISA-88 / ISA-95 standards (also IEC 61512 and IEC 62264) provide exactly that.

• S88 is a model for batch processes: a recipe hierarchy (general → site → master → control recipe) built from a procedural model (procedure → unit procedure → operation → phase) that runs on a physical model (site → area → process cell → unit → equipment / control module).
• S95 connects that plant-floor model up to the enterprise — scheduling, materials, quality, the business systems.

The insight worth carrying out of this course: at the end of the day you are running a recipe. A synthesis step is a recipe. A granulation is a recipe. And an analytical method is also a recipe — prepare the mobile phase, condition the column, inject, integrate, calculate, compare to specification. Describing the QC method in the same recipe framework as the process it tests means the data from both can be linked, queried, and modelled together, instead of living in separate worlds that a human has to reconcile.

This is the subject of a body of work on the “MadSciGuys” site (content on GitHub), including a recipe-based approach to data aggregation (Poulsen & Fermier) and a paper with Rutgers on linking laboratory and plant data (Fermier & Higgins, 2018), plus a “paper on glass” treatment of digital process data capture aligned to the S88/S95 ontology — digitising the operator’s familiar form while, underneath it, capturing semantic meaning, relationships, and process context rather than just values.

The future: linked data and semantics

The direction of travel is toward linked data — data that carries not just its structure but its meaning, expressed in a shared vocabulary so that machines can combine datasets they were never designed to combine. In practice that means RDF and ontologies (the Semantic Web idea), the FAIR principles (Findable, Accessible, Interoperable, Reusable), and, in analytical chemistry specifically, standards like AnIML and the Allotrope framework for instrument data, and SiLA 2 for instrument control.

Why it matters here:

• Method sharing. A method expressed semantically can be understood — and re-run — by a lab that has never spoken to yours.
• Future-proofing. Data described in a shared, open vocabulary is still interpretable when the instrument, the software, and the people who generated it are all gone. Data locked in a vendor format is not.
• Model building at scale. Linked data is what lets you build models across the whole product lifecycle instead of one study at a time.

Where the analyst sits

The Q2 lesson was a measurement is a claim that must earn trust. The Quality Control lesson was quality is assured by accumulated knowledge, not the final test. The automation lesson was when the measurement moves onto the line, the analyst moves with it. Knowledge management is where those converge:

The analyst’s job now includes owning the knowledge, not just the number — deciding what context a result must carry to be worth keeping, structuring data so it can feed a model, pushing the group’s understanding from informal toward formal, and keeping methods described well enough to share and to survive. Section 1 called a method a hypothesis about a molecule; knowledge management is how a laboratory accumulates the answers to all those hypotheses into something coherent enough to call understanding of a product. That is the A in STEAM — judgement, interpretation, and communication — operating on the whole data estate, not one chromatogram.

For discussion

• Take a result you produced recently. Write down every piece of context it needs — units, method, sample, equipment, analyst, date, raw data location — to be promoted from data to information. How much of that is captured automatically today, and how much by a person?
• Give an example from your own work of an informal model the group relies on. What would it take to make it a formal one, and what would you gain?
• “An analytical method is a recipe.” Take a method you run and sketch it in S88 terms — procedure, operations, phases. Does anything about the method look different when you write it that way?
• Legislation is offered here as the archetypal informal model, full of loopholes and hard to test. Where in a control strategy or a specification have you seen the same failure mode — language that different readers interpret differently?
• Your lab has a data lake with ten years of raw instrument files in it. What has to be true about how those files were captured for the lake to actually support model building — and what happens if it isn’t?
• Pick one: RDF/ontologies, FAIR, AnIML/Allotrope, SiLA. What problem does it solve that a well-designed relational database does not?
• Automation lets you generate, organise, and analyse. If a program automated only the “generate” step, would its knowledge management get better or worse? Why?

Source note. This section is a teaching framing, not a single guideline. Its anchors: the DIKW / “data–information–knowledge–wisdom” hierarchy commonly traced to Russell Ackoff’s 1989 address “From Data to Wisdom” (the attribution and the exact ladder are both debated — present it as a lens, not a law); E. F. Codd, “A Relational Model of Data for Large Shared Data Banks” (Communications of the ACM, 1970), and Codd’s later OLAP paper (1993); The Turing Way handbook for reproducible data science; the FAIR data principles (Wilkinson et al., Scientific Data, 2016); ANSI/ISA-88 and ISA-95 (IEC 61512 / IEC 62264) for the recipe and enterprise-integration models; and the analytical-data standards AnIML, Allotrope, and SiLA 2. The MadSciGuys references — a recipe-based aggregation approach (Poulsen & Fermier), a lab-to-plant data paper with Rutgers (Fermier & Higgins, 2018), and an S88/S95 “paper on glass” treatment — are at github.com/AdamFermier/madsciguys; the PDFs (poulsen-fermier-1.pdf and the Rutgers paper) are in that repository. (Instructor: produce the section infographic as image.png; confirm the current ISA-88/95 edition numbers and the DIKW attribution before lecture; decide whether to demo AnIML or Allotrope live.)

• ←Lab Automation & PAT
• The Cost of Development→