Measuring the Success of Training and Competency

Measuring the Success of Training and Competency

Konyika Nealy, Vice President of Quality Assurance & Validation, Pilgrim Quality Solutions

According to 21 CFR Part 211 Subpart B, Section 211.25 (Personnel Qualifications),

Each person engaged in the manufacture, processing, packing, or holding of a drug product shall have education, training, and experience, or any combination thereof, to enable that person to perform the assigned functions. Training shall be in the particular operations that the employee performs and in current good manufacturing practice…as they relate to the employee’s functions.

But what does an effective training program look like?  How do you measure success?

A Tiered Approach

A robust training program uses various teaching methodologies based on the complexity and criticality of the material being presented.  Undoubtedly the majority of training activities fall into a level one or “read and understand” category.  An employee is considered trained based on self-study alone.  This type of training generally applies to simple concepts and restrictions are based on the amount of information an employee can consume in a defined period.

Classroom training, level two, is deployed via an experienced instructor.  These interactive sessions facilitate the comprehension of more complex concepts.

The previous two levels alone are not predictive of employee behavior as there is no measurement how much of the material the trainee retained.  Levels three and four include competency elements in the form of tests and projects respectively.  Minimally, a competency exam can be structured as multiple choice questions.  This type of test is arguably only challenging the trainees recall abilities.  It is preferable to engage the employee with scenario-based questions that challenge his ability to apply what he has learned.  The use of real-life examples solicited from peers make the questions more relevant and memorable.

Finally, at level four, a trainee must actually perform a task congruent with the training material.  The output should be evaluated against a standard with carefully defined pass/fail criteria.

What is the Value of a Tiered System?

Consider the investigation of a product or process failure where the contributing cause is identified as employee error.  As a corrective action, it is common to include retraining as an action item. Generally the same level of training is assigned for an isolated incident.  Therefore this first error can be attributed to a lapse in judgment.

But what if that error is subsequently repeated?  Now it is a pattern of behavior.  Apparently we have a bigger issue since the training quality is now called into question.  Increasing the training level for the employee in that area removes the indictment to the system and gives the trainee a second chance at proving competency.

Measuring Success

So, how can you measure training effectiveness?  Reporting on contact hours and courses offered is not an adequate answer.  It is important to determine if your message is getting through and that can only be accomplished empirically.  Have you observed a decline in errors?  Has there been an improvement in output consistency across personnel?  The responses to these and similar questions are better indicators of employee competency and a solid training program.

Simply stated, training is a fundamental element of any quality system.  There is not a one-size-fits-all option.  Allow for different delivery methods to enrich your program.  The result will be a skilled and more proficient staff.

Konyika Nealy

Vice President of Quality Assurance & Validation, Pilgrim Quality Solutions

No Comments

Comments Closed