This may be hard to believe, but much of quality assurance (or, QA) involves reading and writing. That's because improving quality can involve analysis of behavior -- something that's neither numerically quantifiable nor consistent. Although control charts are critical to SPC, they're not the whole story in a quality assurance program. (NOTE: some programs don't take quantifiable data into account at all. Beware!)

A QA system will ask the following questions:

1. How do we do things now?
2. How do we SAY we do things now? What requirements do we have in place?
3. Where do the first two answers deviate?
4. How do we fix the deviations?

...and perform the following actions:

5. Fix the deviations.
6. Document the way things are done according to standards.
7. Get "buy-in" from everyone involved.
8. Adopt a policy of continuous improvement which includes auditing these processes.

ISO 900x is an international standard that helps companies align the way they do things with the way they say they do things. A QA system that conforms to ISO 900x has procedures for reporting and correcting problems. This doesn't guarantee product quality, but it provides a framework for conscientious companies to pursue continuous improvement.

Event Reporting

As part of a QA system, you'll analyze a report of an event. Maybe a control chart reveals a process has fallen out of statistical control. Maybe there's been a failure in a part or machine. Maybe a new module has been written for a legacy software program. These are events, and witnesses or participants will be called on to contribute to the report. If you have to prepare the report, you may need to interview those involved. Even if you don't have to prepare the report, you certainly must read it to become familiar with the event.

When you've become familiar with the report, then you can analyze the event and all behavior leading up to it. You have to be able to visualize what each person involved (or, "actor") did. This means you may have to

• mark up the report
• clarify the identities of multiple actors
• match up each action with a single actor
• infer actions that aren't stated explicitly, such as
  • when the narrative is written in passive voice
  • when there's been a change in process state
  • when some action is assumed in the narration (e.g. estimated data -- but whose estimate?)
• place the actions in context
• organize the actions in such a way as to avoid creating conclusions (e.g. with "did" format instead of "did not")

With all the actions in context, you can make value judgments on each. The judgments can resolve

• whether more data is needed by the actor
• whether measurements or estimates taken at that moment could be considered reliable
• whether the narration was at that moment reliable

Anything that's left after those judgments are made is probably a deviation in the process -- one that you alone can't fix. But that's what you're there to identify!

Software QA

Software QA is analogous to the process above in that you want to know what each method does, when it does it, and on what variables it does it. A computer program must supply an answer for all possible instances within its scope, and those answers must be correct and reasonable, before its results can be released to a customer. Then you "lock down" the program, give it a version number, and provide documentation as needed.

The QA role is to keep a program from being updated and used without everyone concerned knowing what the update IS and therefore not using it the same way. It's a function that sometimes must be performed pretty much concurrently with the actual collection of results for the customer, unless you're selling the software as a consumer product. Customer requirements dictate how the program is to be tested.

As is the case with any consumer product, there is a difference in "generation quality." That difference can be significant. Software will (hopefully) improve with successive generations, as will most consumer products. But a "bootleg" product (for instance) will deteriorate with successive generations.

Sample Quality Assurance Review

I have completed a review of [company name's] QA document, and as a result feel comfortable with collecting information that we need to prepare a similar document of our own. I understand the need for such a document: we want our customers to feel that we can build repeatedly to their specs. (Or, if they don't give us specs, to specs that we generate from their requirements.)

Here I will summarize what I think is directly adaptable to our needs from the [company name] document, and what I think will make our document even better. It may not be necessary to have a better document, but I think we have to take a different approach than theirs for the following reasons:

They don't really define "quality," and I think this is because the term changes meaning depending on the product and the customer. I believe we can and should define the term, and with enough meat that our customers know we have direction.

There are certain scientific aspects of quality for which everyone on the team should have a basic knowledge. This is because in so knowing we'd all be on the same page, and everyone on the team bears some responsibility for the quality of the product. This is necessarily different from what [company name] is doing: [company name] is a big company and has a dedicated QA department, along with other functional departments that we just can't have for a couple growing years or so. [company name] does train "all personnel performing activities affecting quality," and tests them, but my experience tells me that much of what these folks are learning is either (a) disconnected from the job they actually do, or (b) too simple to change their approach to their work, or both. We can't afford the same approach.

Because (I assume) [company name] has a more diversified product line, or at least more customers, than we do, they give few specifics about how their QA system is applied to a job. (Where they do give specific instructions, those instructions are generally excellent -- but they're buried in the document more often than not.)

[company name]'s entire QA program is centered around inspection. (In fairness, this may be because inspection is the only way to judge compliance on certain items. But for whatever reason, the document reflects this focus.) We can't do this, because you cannot inspect quality into a product. All inspection will do is reject noncomforming parts -- it doesn't improve the processes or those who carry them out. [company name] also gives the inspection responsibilities to separate individuals (which IMPO is good) who have no direct ownership of the process (which IMPO is dubious) and who have no direct powers to change it (which IMPO is bad).

Individual roles are unclear. You will have individuals whose only role in the process is to receive a document, date-stamp it, and route it to someone else. You will have other individuals who have heavy responsibility but where they come from is unidentified. And the document is about responsibility: who takes the fall if a product fails. We need to be about responsibility in a different way: who really *needs* to support each aspect of the process. Since quality is the responsibility of each of us, we must all take the fall in event of a failure.

VERY INTERESTING MATERIAL

Here's what [company name] considers important considerations in inspection of a given component:

1. Consequences of failure.
2. Uniqueness & complexity of design or fabrication.
3. Special control or observation of processes or equipment.
4. Ability to demonstrate compliance via inspection or test.
5. Quality history and degree of standardization.
6. Difficulty in repair or replacement.

Throughout this document, [company name] refers to reviews. Preliminary and Critical Design Reviews, from which formal changes in design can be formally requested. This is a common way of doing business, which if adopted shows we can speak the same language.

Throughout this document, [company name] refers to identification and traceability. We've not been particularly good at keeping records up to this point, primarily because we have so much crap that there's no time. But it's a good idea to adopt specs on these things, like [company name] does:

1. Which records really need to be maintained.
2. Part numbers and characteristics.
3. Restrictions on where and how documents and parts can be marked.
4. Traceability from parts to associated documentation.
5. Review and approval cycles.
6. Storage, handling, and shipping.
7. How to handle noncompliance.

They actually have a "Quality Plan," though they bury it under "Inspection Planning." Which is another way of saying they are trying to inspect in quality, and again I say unto you: thou shalt not. But the plan is interesting, because all this stuff is identified:

1. Relevant drawings, part lists, and specs.
2. Technical drawings and documents requiring review and approval, and by whom.
3. Inspections needed, and by whom, and by what methods, and at what points in each process.
4. Specific identification and traceability requirements.
5. Detailed requirements for regular process control and observation, equipment condition, and personnel qualifications and training.
6. Required sampling procedures (if sampling is needed).
7. Documentation as evidence of quality compliance.

Their argument is that all this stuff must be in place before a part is shipped, and we are going to have to head in this direction as well.

DEFINITION OF QUALITY

Here is what I think will pass as a start for a definition of quality for our purposes:

The extent to which a product meets requirements agreed upon by ourselves and our customers.

Nothing this simple is stated in the [company name] document, yet if you ask around in industry you will find that a common problem in quality initiatives is having such terms undefined.

Once we define this term (whether by the above definition or some other), we have some idea where to go next. The above definition suggests that we reach an agreement with a customer over requirements. Then we work on component specs to fit into the requirements. These things are pretty much what we are doing now. But the component specs then suggest what we must do to verify compliance with specs. As you know, there are five ways to verify compliance:

• analysis
• demonstration
• inspection
• similarity
• test

We have to date relied on demonstration and test. This is not surprising because we are trying to sell a system, not its components, and because our systems have tended to be one-of-a-kind. We are now moving away from one-of-a-kind systems, and that opens up to us the other three means of verification. Even so, we are not headed toward a situation in which inspection should be our most common means of verification. We are, however, headed toward a situation where we must start inspecting, which means we have to agree in advance on the how, why, and who.

We also have to agree on what constitutes compliance, and what constitutes satisfactory test or inspection results. That's something we would probably do as a management team early in the job, but we wouldn't document the meeting.

WHAT SHOULD WE ALL KNOW ABOUT QUALITY

There are three scientific aspects of quality which I recommend we learn -- not in detail, but conceptually only will do fine. They go to what data gets collected, what analysis gets done, what conclusions can be drawn from the analysis, and to what degree of certainty. I have them listed in order of what I see as their importance to the company, and coincidentally that's also the order of how well I know them myself.

• Statistical Process Control
• Design of Experiments
• Theory of Constraints

There's a brief introduction to each of these in the sci.engr.* FAQ on Engineers and Quality.

References

Starline Software Ltd. "CATALYST EventAnalyzer."
Supply Chain Solutions. "SCS Case Studies."
whatis.com. Definitions -- quality assurance.
JTapes.com, quality of "puroresu" videos.
OSHA 1910.119, "Process Safety of Highly Hazardous Chemicals."