This post will provide information relating to Test Method Validation (TMV) in the Medical Device Industry, addressing the following topics:
What is a TMV?
A TMV evaluates test methods used to manufacture medical devices, ensuring they are fit for purpose. It is designed to challenge the chosen test method and allow a company to have confidence that it works as it should. Without having a TMV completed, a company may not have sufficient proof that its inspections are robust enough to prevent bad parts from entering the field.
The Medical Device Industry is highly regulated, meaning companies must meet certain criteria set by regulators to sell their devices on the market. Over the past few years, regulators have focused increasingly on TMV in medical device manufacturing and development.
To start, let's discuss the test methods themselves. A test method is the procedure you follow to generate test results. This procedure encompasses the setup of the test equipment, the preparation of the samples for testing, and how the test itself is completed. Test methods come in a huge variety, but some common examples are visual inspections of parts, GO/NOGO gauge inspections, tensile tests, and dimensional inspections.
Before completing a TMV, there would be a phase of test method development. During this phase, engineers will work to identify appropriate methods to test the desired output. This development work could look different depending on what inspection is required. It may involve reviewing current test methods to identify something to leverage, looking at new equipment, or designing GO/NOGO gauges.
Once you are happy with the test method in place, the next phase is to write the validation protocol. The specific requirements for writing a validation protocol would be outlined in a company's procedures. The TMV procedure would provide guidance around how many samples are required to run the validation, how many operators are needed, and what the requirements for passing are.
What Types of TMVs are there?
Mentioned above were common test methods of visual inspection and dimensional inspection. These inspections fall into two different categories of TMV. The first is called an Attribute Test Method Validation (ATMV), which is where visual inspection falls. An ATMV deals with binary data (pass/fail in this case). If a person is visually inspecting a part for defects, it can only either pass the test or fail it; no other result is possible.
The second TMV type is a Variable Test Method Validation (VTMV), which is where dimensional inspection falls if a measurement value is being recorded. A VTMV deals with variable data, meaning the data gathered can have various results. If a person measures the length of 3 braids with calipers, they might get 50.10, 50.05, and 49.95mm.
The generation of measurement values makes variable data more powerful than attribute data, providing more information to engineers when interpreting it.
Approaches for TMVs
The most common approach for validating a test method is running a Gage Repeatability and Reproducibility (Gage R&R) study. This test will help you assess how much variation in your measurement result is due to the test equipment (repeatability) and how much variation is due to operator variability (reproducibility). A standard test plan for a Gage R&R may involve three operators testing the same ten parts three times each. This is called a crossed Gage R&R. A simplified version is shown below for a crossed study with three parts.
Each part will be individually traceable so that when engineering completes the analysis, they can compare the results operators 1, 2, and 3 got for the same unit. When evaluating the results, Minitab will give the engineer a breakdown of the % variation attributed to different sources. These different sources will be repeatability (can I measure the same part and get the same result), reproducibility (can someone else measure the same part as me and we get the same results), and part-to-part (can we detect the differences between different parts). For a measurement system that is fit for purpose, the majority of the variance should be coming from the part-to-part variation, and we would be looking for the % Contribution to be less than 1% (see image below).
Other important information for determining whether a measurement system is fit for purpose is the % study variation and the % tolerance from the gage evaluation. The % Study Variation determines the % of the variation due to the gage compared to the total variation of measurements taken. The % tolerance is the % variation that is due to the gage compared to a process tolerance or specification.
The acceptance criteria for these outputs are provided in company TMV procedures, but generally, the following applies if you are looking at % contribution or % study variation and % tolerance variation.
If the Total Gage R&R contribution in the %Study Var column (% Tolerance, %Process) is:
Less than 10% - the measurement system is acceptable.
Between 10% and 30% - the measurement system may be acceptable depending on the application, and a justification is needed.
Greater than 30% - the measurement system is unacceptable and should be improved.
If you are looking at the %Contribution column, the corresponding standards are:
Less than 1% - the measurement system is acceptable.
Between 1% and 9% - the measurement system may be acceptable depending on the application, and a justification is needed.
Greater than 9% - the measurement system is unacceptable and should be improved.
An attribute TMV requires a different approach, an Attribute Agreement Analysis (AAA). This type of study typically requires more samples than a standard Gage R&R, as more data is required to provide confidence in the test method.
The study starts by selecting representative pass and fail parts from the process. These are then given individual unit traceability and are assigned a pass or fail result for the inspection by an SME or a group consensus on the part. Once the standard result is obtained for each unit, the appraisers can then begin their evaluation in a randomized order of the parts. The AAA assesses how much an operator agrees with themselves, how much they agree with someone else, and how much they agree with the standard result. These values will be given as % from Minitab for interpretation. Another important point to note, particularly as this is an AAA in the medical device industry, is that failing a pass part is okay. Passing a unit deemed a failure is not and will result in a failed TMV.
That concludes a brief introduction to TMV in the medical device industry. If you want to learn more about TMV, click the link below and contact us to join our waitlist for TMV training and receive a 20% discount on the course.
Comments