Important Questions and Answers concerning the Audit Trail Review
Actually, the topic "data integrity" has moved into the focus of many national and international inspections. The American FDA has sent many Warning Letters concerning comprehensive deviations observed. Therefore the regulatory authorities request not only an assurance of the product quality but they also demand that each company has a clear strategy how the integrity of critical data can be assured over its complete life cycle. Here, the review of the audit trail plays a decisive role. Despite the numerous guidelines published on this topic since 2015 it often remains unclear which requirements exist concerning the audit trail review and how they can be implemented in practice.
For this reason the Webinar "Audit Trail Review" was carried out in February 2017. Its aim was to address the main elements of data integrity and audit trail reviews:
Regulatory overview with emphasis on the requirements of MHRA and Annex 11
Classification of data - which are critical data?
Classification of systems - which systems are relevant?
What Audit Trails are important?
What shall I do with legacy systems without audit trail?
Who shall review Audit Trails?
How is the review documented?
Process and documentation in case of deviations?
Dr. Wolfgang Schumacher has been the speaker. He looks back on more than 30 years experience in the pharmaceutical industry.
He has received more than 50 questions concerning the webinar. Naturally, he could not reply to all them immediately after the webinar. Thankfully, Dr. Schuhmacher has answered all questions in writing. Hereinafter you find the first part of the questions and answers. A further selection is planned for a later time.
1. Are the analytical data arising in the course of the validation of analytical procedures critical data?
Answer: No, since these data are released with the completion of the validation process and it only presents the basis for future analyses.
Therefore there is only an indirect criticality; no audit trail review is required.
2. The calibration of equipment influences the correctness of data. Does this mean that the calibration is critical?
Answer: Since the calibration influences the patient safety only indirectly no audit trail review is required.
3a.: Practical example: What about a reactor control from production that has neither user management nor a password or an audit trail etc.? Is it allowed to continue using this control?
Answer: It is definitively recommended to plan a replacement. The time of replacement depends on the product manufactured with the reactor. This defines the criticality.
3b. Practical example: What about a reactor control from production that has neither user management nor a password or an audit trail etc.? (The control concerned is the control of a reactor which is used for the manufacture of intermediates and APIs but not of final products). Is it allowed to continue using this control or does it have to be withdrawn from use?
Answer: Thank you for clarifying the question. I do not think it is necessary to replace the reactor control immediately if it is used for the production of an intermediate. Recommendation: Assess all systems and equipment and arrange them according to priority classes (for instance by using a rather simple FMEA). This is used to define when to deal with the priority classes, for example Prio 1= until the end of 2017, Prio 2= until the end of 2018 etc…
4. Is the audit trail review prior to the release of each batch a regulatory requirement or is it only recommended so far?
Answer: Annex 11 requires the audit trail review. Inspectors consider the release of batches to be the most critical process of all.
5. Hybrid systems: No audit trail retrofitting possible. What can be done?
Answer: First of all I would clarify all other criteria (access, user/admin profile, safety). How critical is this data? Plan a replacement according to the classification of the criticality (= priority) if the data is critical.
6. Role concept: Is a user allowed to carry out a reintegration? (GC-system)
Answer: As far as I can see, the answer to this is 'no'. This should first be authorised by the Head of Laboratory stating the reason.
7. It is difficult to carry out the separation between the persons of administrator and user in the CDS. Is it possible that the admin and the user being the same person?
Answer: This is possible but it needs to be described in a SOP.
8. Process validation data are category 3 data - therefore no audit trail review is required. But isn't it required to carry out a review for the generation of validation measuring data?
Answer: As you have stated correctly, validation data have the criticality of category 3.
It is very unlikely that the user will carry out changes since there is no reason for falsification - in contrast to the batch release.
9. Hello, is is useful to review the "windows eventviewer" for GxP devices?
Answer: This is a good idea but it will probably not be realistic as too many events would be reported. Its use would anyway only be worth considering for systems which save data locally, and thank God this isn't done by many modern systems anymore.
10a.: Must the report for a riew of the audit trail be generated by the system itself? Or is it possible to use another system which is better suited for carrying out analyses?
Answer: Strictly speaking, the audit trail must be generated by the system itself ("system generated audit trail"). Which other system do you have in mind?
10b. I think, I did not express myself very clearly when asking the question: Is it only reviewed whether the process is still functioning in the course of the periodic reviews - hence, if the audit trail data is still recorded? Meaning: Functionality is documented by IT with a few samples.
Answer: Very correct!
11. May the Head of Laboratory have admin rights for example to carry out the audit trail review?
Answer: Yes, as long as she/he does not operationally process analyses herself/himself.
12. Is it possible for a user to have two system accounts? One as a user and one as administrator or reviewer?
Answer: Yes, in small organisational units. Guarantee by means of an SOP that the admin account will NOT be used operationally.
13. Do you know a system that fulfils all requirements for the audit trail function of a CDS?
Answer: So far there are hardly any systems on the market which allow to carry out the review with a minimum effort. Unfortunately, this is also true for modern systems as Empower 3.
14. File based data retention: Deletion is possible outside the software. How is it possible to "build in" safety?
Answer: Yes, in systems that can store locally on the HD of the user it is possible to set up two local user profiles. Then, for instance, the system stores data only on the profile the user cannot access.
15. What are metadata?
Answer: Data concerning data, such as the timestamp.
16. What are manufacturing execution systems?
Answer: Process control systems.
17. What do you do with legacy equipment? If no audit trail is available or if "user login" is not possible?
Answer: The systems have to be replaced in the short or in the long run - according to the risks.
18. Equipment often has a standard audit trail function. A lot of data is recorded (on/off), but only a small part of it is critical or relevant for a review. What is the best way to proceed when carrying out a review?
Answer: The best way is to configure a report that only shows the critical data but leaves out all the unnecessary elements (such as login/logout). But this has to be programmed and is expensive.
19. What is the way to proceed in the case of facilities in the production area such as AP production, mixers, filling lines with variable parameters as concerns classification of data/systems, extent and intervals of review?
Answer: It is best to classify systems and facilities according to ISA95. Then you will notice that at the bottom levels (field level, PLC, SCADA) no interactions of the user are permitted. Hence no audit trail review is required. Basis is the risk analysis of the systems and data.
20. Whose job is it to carry out the audit trial review in the laboratory?
Answer: This can be a colleague. The FDA requires "Quality Unit" (=QA) in the Draft Guideline. All other Draft Guidelines allow for a peer review.
21. Must all electronic data be stored and archived in the case of a process in the sterile area or is the batch record sufficient?
Answer: You must define what the raw data are. These have to be stored in addition to the batch record.
22. Would you consider changes of the parameters / formulations at production facilities as dynamic data? Do I have to consider them at every batch?
Answer: This data is governed by the change control and is not part of the data for a review.
23. Many of our production processes are documented by means of a batch record on paper. What should we do as concerns the review of the audit trail? Should we review the good documentation practice in the batch record by means of an audit trail review?
Answer: As long as you only document on paper the audit trail review is not relevant for you, but the 4-eyes-principle for all critical steps. The following is stated in the FDA Guidelines: "Critical process steps have to be verified by a second individual".
24 Dynamic data <-> static data. A small filtration facility (pressure measurement/flow measurement/temperature) with a locally installed SCADA system would then rather deliver static data?
Answer: That's correct. Static data as the user is not able to change the data.
25. Batch records are considered to be carriers of raw data for the documentation of processes. What about trend data that have been printed and enclosed to the batch record?
Answer: I do not consider it necessary to always review trend data.
26. Slide 9: What do you include in systems and equipment?
Answer: Everything that has to be validated and qualified?
27. Is it necessary to perform an audit trail review (ATR) after each measurement/analysis carried out at the equipment in the laboratory? Who carries out the ATR - laboratory personnel (operator) or the Head of the Laboratory?
Answer: A SOP is required for the ATR for each system that describes the details. The review after completion of the analysis is important.
We'd like to thank Dr. Schumacher for his comprehensive answers to all questions.
Of course, this webinar is also available as a recording.
Please also read part 2 of the Question and Answers concerning Audit Trail Review.
28. How do you justify the reintegration of chromatograms versus the method specification in the audit trail?
Answer: A reintegration should only be necessary if the analysis wasn't carried out for one reason or another. The reason has to be specified and must be signed by the management. The laboratory employee is not allowed to take the decision himself.
29. What is important for machine manufacturers in the area of audit trails especially as concerns the continuous production? What has to be taken into consideration? What must be included?
Answer: This question is difficult to answer as concerns a machine manufacturer since the answer depends on the product manufactured with the equipment (for instance a critical, highly effective product or a non toxic excipient). The pharmaceutical producer must carry out the risk analysis.
30. Page 7: How has data to be classified that concerns products which are manufactured from blood components and that is part of the donation process (donor management)? I consider them as critical data with influence on SQuiPP.
Answer: Just the way I see it.
31. Audit trail review only for critical data: Does this mean that it has to be carried out only for systems with data which influence the product directly? And not for systems which only
have an indirect influence?
Answer: You decide this by yourself by means of the risk analysis. No audit trail review is required for a training system for example.
32. How do you decide which data require the 4-eyes-principle? Can this be decided by means of a classification of "direct" versus "indirect" influence?
Answer: Yes, this is the first approach, together with the criticality of the system.
33. Page 35: This means that in the course of the periodic reviews it is only reviewed whether the process is still functioning - hence, if the audit trail data is still recorded?
This means no selection = random sampling of data sets?
Answer: A review by means of examples of data should also be part of the control of functionality (SYSTEM Audit Trail). This means that random sampling is useful.
34. Page 44: Why are only global systems in the scope? Usually the most critical systems in production and QC exist on local basis.
Answer: This was only supposed to be an example. Naturally, local systems are as important as global ones.
35. How or where (guidance document) is it advisable to define raw data?
Answer: (Unfortunately) there is no uniform definition. This means you are flexible to define this for your company.
36. When are the final FDA, MHRA etc Guidances to be expected?
Answer: It is doubtful whether the MHRA Guidance will be applicable at all, in light of the Brexit. The FDA Guidance is to be expected by the end of 2017 at the earliest, but it is more likely that it will be 2018.
37. When parts of data are printed it becomes static data. But the audit trail review has to be carried out for all critical and electronically produced data in the system. It is not possible to refer to the printed version!?
Answer: Static data is data the user cannot change. It is only important if the user had the possibility to change the data PRIOR to the data being printed. I do not think that a reference to the paper record will be accepted.
38. How do you assess the situation for API manufacturers?
Answer: You have to decide yourself whether you produce critical APIs. If so, I see no chance to renounce to the audit trail review in the long run. But it won't have the same priority as in the case of a manufacturer of a finished medicinal product.
39. Can the review of a printout of the drying temperature profile be considered to be part of the audit trail review of the batch?
Answer: In my opinion this is a good example for static data for which no audit trail review has to be carried out. The reason is that the user cannot change or access this data.
40. Is analytical data (raw data, metadata) from the validation of analytical procedures classified as being non critical? Do we interpret your slide number 3 "What other data is there?" correctly? In this case the audit trail review would not be required.
Answer: The validation of analytical procedures precedes the daily analyses for the batch release and constitutes the basis for them. The validation data is released with a report. The audit trail review is not required.
41. Static data - dynamic data:
a. May we define entries in our CDS such as standard concentrations, densities, dilution factors, ..., hence parameters used for the quantification of content and purity as static data? Or are these entries also part of the dynamic data such as the (re)integration of peaks?
Answer: In my opinion this is also dynamic data the user can influence or change.
42. Static data - dynamic data:
b. Is it possible to shorten the audit trail review by taking the status of the analytical sequence and controlling the recorded parameters (such as concentration of the standards, ...) at the end of the analysis and by stating that all changes and corrections carried out in the meantime are part of the analysis and evaluation and must consequently not be commented?
Answer: In my view this is acceptable as long as a control is carried out and confirmed by a second person.
43. Is it acceptable that a user has the entitlement for reintegration at a GC system or should the laboratory manager be the only person authorised to do this? Are there any relevant requirements or recommendations?
Answer: This should be regulated internally in an SOP. Preference should be given to the laboratory manager since he has a control function. I know nothing about any requirements concerning this.
44. It was mentioned in the seminar that the audit trail review is part of the Periodic Review. What other points are included in the Periodic Review?
Answer:
Roles and Responsibilities
Service Agreements
Validation/Qualification and Project Deliverables:
System Control Procedures
Audit Trail management
Incidents, Problems
Changes, Upgrades
Security, User Access, User Administration
User Guides
Backup, BCP, Performance, Reliability Monitoring
System Access Training
Archived Data
45. In Annex 11 as well as in other guidelines is required the "regular" revision of the audit trail. But is it really defined in any guideline that this has to be performed prior to the release of the product?
Answer: This is implied in Annex 11: The authors (inspectors) cite this again and again as explanation since this is the most critical data of all.
46. Or is this rather the case since "regularly" is interpreted in this way by now?
I understand that one may reach this conclusion after a risk assessment. But it could also be different, couldn't it?
Answer: Your own analysis of the critical data may well reach another conclusion. The own result defines the procedure.
47. If I have implemented a role concept for example and trained all staff members thoroughly as concerns the topic DI won't it then be sufficient to carry out a review of the audit trail on a random basis and to carry out the review prior to the batch release only in the case of irregularities such as OOS results?
Answer: It is acceptable to carry out a review on the basis of the system's "exception" reports. These exception reports always must be checked anyway. It is not acceptable to carry out a review only on a random basis (but ultimaltey, this is defined by the risk analysis).
48. Which measures may be taken if the audit trail (as well as the actual data) can be deleted or changed at the level of Windows?
Answer: In the case of systems that can store locally on the HD of the user it is possible to set up two local user profiles. Then, for instance, the system stores data only on the profile the user cannot access.
49. Is it possible at all to have analytical data in the manufacture of active pharmaceutical ingredients that must be classified as GxP critical (= direct influence on the quality/integrity of the product) if it is known that the API and the finished product will be reviewed again in the subsequent manufacture of finished products? Or is this irrelevant and if so, why?
Answer: If you reach the conclusion in the risk assessment that none of the APIs are critical a complete exclusion from the audit trail review would be conceivable. But this will hardly be accepted in practice as there are highly potent small molecules and very critical biotechnological active pharmaceutical ingredients.
50. Or, to pose the question in another way: Do you think it is conceivable that in the course of the release process of the analytical results no audit trail review is carried out with one of the CDS systems used for the manufacture of APIs? If no, which analysis (raw material, final product, proof of identity of the API, IPC, proof of efficacy) would always have to be classified as being critical?
Answer: I recommend to carry out an audit trail review for the critical APIs at least. The analysis of the final product surely is the most important one, since here the tests will be comprehensive.
51. When assessing the criticality of data the influence of the data on the data integrity is mentioned as one criterion apart from the influence on the product quality. There seems to be a direct influence on the integrity of data almost always. How can this criterion be further substantiated in order to be able to use it in a differentiated way?
Answer: This is like going round in circles. The quality of the product always is of the highest importance. The criterion "data integrity" by itself is not meaningful but has to be specified by attributes such as "access security", "duration of storage until destruction" etc.
52. I would like to know what you think about the relevance of the audit trail and the audit trail review for an eDMS which also generates, stores and manages TMF documents (eTMF) (with Approval Workflows)?
Answer: It is important for an eTMF that no changes can be carried out. I think that an audit trail review is out of the question since the data already has the form of a report. The access security to the eTMF itself is an important element in order to ensure that no data can be "changed" after the first submission.
53. The second question points in the same direction but concerns an eDMS in which GMP documents are stored and versioned by means of Approval Workflows with e-signature (such as CMC documents: specifications, laboratory methods or batch records: product master manufacturing record; or also validation documents)
Answer: The audit trail review is senseless for an eDMS as the persons signing a document read it prior to signing. During the validation of a system it is checked that no data can be "changed" (version control).
We would like to thank Dr Schumacher for taking the time to answer all questions comprehensively.
Of course, this webinar is also available as a recording.
Reference