Opinion 08-2019

European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) (RMT.0599) ‘Rationale behind the proposed amendments to the implementing rules Draft CS, AMC & GM as well as safety promotion actions that are associated with the implementing rules Rationale behind the draft CS and AMC & GM’ Page 1 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) Table of contents Table of contents 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb ..................................................................................................................................... 3 2. Proposed amendments to CS, AMC & GM and rationale in detail.......................................... 27 Page 2 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb This chapter contains the rationale behind the proposed amendments to the implementing rules (IRs) presented in Annexes Ib and IIb to Opinion No 08/2019. To differentiate them from the proposed rules, the font colour used for the explanatory notes is blue. Annex I (Definitions) to Regulation (EU) No 965/2012 Definitions for terms used in Annexes II to VIII SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Explanatory note to Annex I (Definitions) to Regulation (EU) No965/2012 competency The definition proposed is transposed from ICAO Doc 9868 ‘PANS-TRG’ TRG Amendment 5. competency-based training The definition proposed is transposed from Doc 9995: ‘Competency-based training. Training and assessment that are characterized by a performance orientation, emphasis on standards of performance and their measurement and the development of training to the specified performance standards.’ Competency-based training and EBT — use of the wording ‘assessment and training’ The proposed provision uses the wording ‘assessment and training’ instead of ‘training and assessment’ because it reflects better the model used in EBT. Currently, EBT is used for airline pilots, who are current on type. Therefore, the phases of EBT focus first on assessment, to then develop the competencies in the subsequent phases (training). The traditional use of the sentence ‘training and assessment’ is appropriate for initial type ratings and initial issues of licences where the pilots are not yet proficient, and they need to acquire a new type rating. In these cases, the sequence of ‘training’ and then ‘assessment’ is appropriate. competency framework The term ‘identified competencies’ is used to refer to the competencies the operator must choose to develop a competency framework (e.g. the 9 competencies of EASA that include the 8 competencies of Doc 99951 plus ‘Application of Knowledge’). These competencies are also called ‘core competencies’. ‘unforeseen threats and errors’ is used to link it to resilience, as the concept of resilience is very important to aviation safety. The definition is based on the Doc 9995 definition of ‘core competencies’: 1 ICAO Doc 9995 AN/497 ‘Manual of Evidence-based Training’ first edition 2013. Page 3 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb ‘Core competencies. A group of related behaviours, based on job requirements, which describe how to effectively perform a job and what proficient performance looks like. They include the name of the competency, a description, and a list of behavioural indicators’. competency A competency is manifested and observed through behaviours that mobilise the relevant knowledge, skills and attitudes to carry out activities or tasks under specified conditions. Trainees successfully demonstrate a competency by meeting the associated competency standard. The definition proposed in the Opinion is created based on: — Amendment 175 to ICAO Annex 1 ‘Personal licensing’; and — Doc 9995. The Doc 9995 references used were: — ‘7.8.5.1 To be competent in any job, a person requires a certain amount of knowledge, an adequate level of skills, and a particular set of attitudes’. — ‘7.8.5.4 To be competent, a pilot requires capabilities across a range of knowledge, skills and attitudes (KSA)’. equivalency of malfunctions The definition has been created to clarify the rules of equivalency of malfunctions. It is a new definition, which is not included in Doc 9995 AN/497 ‘Manual of Evidence-based training’ first edition 2013. evaluation phase The evaluation phase is the first assessment of competencies to identify individual training needs. On completion of the evaluation phase, any areas that do not meet the minimum competency standard will become the focus of the subsequent training. evidence-based training The definition is transposed from Doc 9995. in-seat instruction Effective monitoring and error detection are increasingly important when operating highly reliable, automated aircraft. Multiple data sources illustrate substantial rates of undetected error. Error management is reported as a very significant countermeasure in current operations with one accident study espousing that it is the most significant tool available to pilots for the prevention of accidents. Furthermore, multiple data sources show that there is a high level of intentional non-compliance and so any error management strategy must include greatly reducing its incidence. Error management skills are subject to decay. Error management currently does not form part of any strategy developed through the regulation of flight crew training; consequently, it is lacking in most training programmes. It is a key topic and needs to be incorporated into training strategies in order to raise flight crew situation awareness and further develop the professional capabilities of pilots. When in training, flight crews are usually highly vigilant, and therefore the performance observed may not be representative of performance in normal routine operations. After extensive discussion, the Page 4 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb worldwide international subject matter experts (SMEs) group that developed material for Doc 9995 concluded that an effective means to provide reliable exposure in FSTD training is to use a method called in-seat instruction (ISI). This is also an effective means to provide the recovery element of UPRT; data from loss of control – in flight (LOC-I) events regularly indicate a cognitive impairment of the pilot flying (PF) with the pilot monitoring (PM) often demonstrating a higher level of situation awareness (SA). When the PF does not immediately respond to and act on monitoring calls, the PM takes control and recovers the aircraft. This approach is supported by both Airbus and Boeing in their guidance in recovery FSTD training, and has been integrated within the EBT programme. instructor concordance The definition is transposed from the Doc 9995 definition of inter-rater reliability. Inter-rater reliability is a term not easily translated into all the languages of the European Union; therefore, a synonym for inter-rater reliability was used: ‘concordance’. In statistics, inter-rater reliability, inter-rater agreement, or concordance, is the degree of agreement among raters. manoeuvres training phase This is not a real-time training but allows crews the time to practise and improve performance in largely psychomotor skill-based exercises. Repositioning of the flight simulation in order to focus training on the intended manoeuvres will be a commonly used FSTD feature for this phase. mixed EBT programme The definition proposed is inspired by ICAO Doc 9995 Chapter 4.2, paragraph 4.2.1, point (b). ‘(b) Mixed implementation. Implementation of a mixed EBT programme means that some portion of a recurrent assessment and training is dedicated to the application of EBT. This is a means of achieving a phased implementation where, for example, the CAA regulations or rules permit such a programme as part of the operator’s specific training and assessment, but preclude such a programme for the revalidation or renewal of pilot licences. This phased implementation recognizes the potential for such an EBT programme to be developed and implemented in advance of any future enabling regulatory changes, which may then permit total implementation.’ Scenario-based training phase The definition for SBT was based on the following ideas: — Wherever possible, consideration should be given towards variations in the types of scenario, times of occurrences and types of occurrences, so that the pilots do not become overly familiar with repetition of the same scenarios. — Variations should be the focus of EBT programme design, but not left to the discretion of individual instructors in order to preserve programme integrity and fairness. The definition was transposed from Doc 9995 Chapter 3.8: ‘c) Scenario-based training phase. This phase forms the largest phase in the EBT programme, and is designed to focus on the development of competencies, whilst training to mitigate the most critical risks identified for the aircraft generation. The phase will include the management of specific threats and errors in a real-time line orientated environment. The scenarios will include Page 5 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb critical external and environmental threats, in addition to building effective crew interaction to identify and correct manage errors. A portion of the phase will also be directed towards the management of critical system malfunctions. For this programme to be fully effective, it is important to recognise that these predetermined scenarios are simply a means to develop competency, and not an end or ‘tick box’ exercise in themselves’. special condition The definition is proposed follwong the same approach as in ARO.GEN.121 and ORO.GEN.121. The definition is transposed on the basis of: (1) ARO.GEN.121 point (b) ‘(a) Where an equivalent level of safety to that achieved by the application of the CS can be achieved by another specification, a special condition to the CS issued by the Agency may be used by the certified aircraft operator to establish compliance with Regulation (EU) 2018/1139 and its the delegated and implementing acts. This special condition may be proposed by either the operator or the authority.’; and (2) the airworthiness definition proposed in EASA Management Board Decision 12/2007 dated 11 September 2007, Article 3 (2.) which states: ‘2. Deviations from the applicable airworthiness codes, environmental protection certification specifications (...)’. The wording ‘certified operator’ is used because only operators with an air operator certificate (AOC) are within the scope of the CS. The wording ‘certified operator’ is already in use in Part-ORO (e.g. ORO.GEN.125). Annex II (Part-ARO) to Regulation (EU) No 965/2012 Explanatory note to Annex II (Part-ARO) to Regulation (EU) No 965/2012 ARO.GEN.121 Certification specifications SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Explanatory note to ARO.GEN.121 Considering: — that there are already CS related to the domain of flight time limitation of the Air OPS Regulation; — that EASA received a positive majority position during the several consultations with the advisory bodies: AirOPS TeB of 22 and 23 May 2019, Member State Advisory Body (MAB) 16 October 2019, Stakeholders Advisory Body (SAB) 29 and 30 October 2019, to create CS for other domains of the Air OPS Regulation (e.g. EBT) and only for safety-critical elements or provisions related to changing technologies; and — the Basic Regulation, this Opinion proposes a new implementing rule to manage the necessary deviation from the CS, i.e. the so-called special conditions to the CS developed by EASA. Page 6 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb Having in mind that: 1- pilot training is a safety-critical element, 2- EBT is a technologically dependant concept, because EBT relies on:  providing a specific training for each generation of aircrafts  each generation of aircrafts are technology driven. The Implementing rule related to the certification and approval of the EBT programme is located in Part-ARO (Authority requirements), subpart ‘operations’, section II ‘approvals’. EASA and the review group discussed the possibility of having the initial certification of the EBT programme in section I ‘certification of commercial air transport operators’, as EBT proposes the use of certification specification. This option was discarded for two reasons amongst others: 1- Splitting the authority provision into “initial certification of the EBT programme” and “recurrent oversight – approvals” may be confusing and impractical for the reader, as it is usually more practical to have all the requirements in a single place. 2- Flight time limitation also uses ‘certification specifications’, however the provision ARO.OPS.235 ‘Approval of individual flight time specification schemes’ is also located in section II ‘approvals’. The wording proposed in ARO.GEN.121 is consistent with that used for the ‘alternative means of compliance’ under the current ARO.GEN.120. However, the competent authority may only notify the organisation that the special condition has been approved after having received a positive assessment from EASA. The CS will be used in the context of a certified environment (AOC). This approach is the same as the one used for flight time limitation. Approvals of individual flight time specification schemes are based on Book 1 of CS FTL.1 and the authority requirements are located in ARO.OPS.235 ‘Approval of individual flight time specification schemes’. The same approach is used for the EBT authority requirements, which are proposed in ARO.OPS.226. The wording in point (b) ‘certified operator’ is used because only operators with an AOC are within the scope of the CS. The wording ‘certified operator’ is already in use in Part-ORO (e.g. ORO.GEN.125). The wording in point (e) is consistent with the wording used in Article 71 paragraph (2) of the Basic Regulation. ARO.OPS.226 Approval and oversight of evidence-based training programmes SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Explanatory note to ARO.OPS.226 This IR contains the approval and oversight provisions to ensure a safe EBT programme. The provisions follow the concept already described in: Page 7 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb — point (a)(2) of ARO.GEN.200 regarding the training and qualification of the inspectors; — the associated AMC2 ARO.GEN.200(a)(2) point (a) as regards the initial training programme for the instructors; and — AMC4 ARO.GEN.200(a)(2)) concerning inspector qualification for CAT operations. The requirements on training in ARO.OPS.226 are further developed in AMC1 ARO.OPS.226(a); As regards the general structure of the rule, ARO.OPS.226 is based on the new proposed rule ARO.OPS.225 as proposed in NPA 2016-06 (A) on fuel schemes. ARO.OPS.226 point (c)(1) Due to the complexity of the EBT programme and the necessary maturity that the operator needs to demonstrate to ensure a good implementation of EBT, EASA decided to require the resolution of level 1 findings before approving full EBT. This is in line with the proposal of the RMG which agreed with the text ‘resolution of significant findings’. ARO.GEN.350 provides a definition of level 1 finding. ‘ARO.GEN.350 (a) (…) (b) A level 1 finding shall be issued by the competent authority when any significant non- compliance is detected with the applicable requirements of Regulation (EC) No 216/2008 and its Implementing Rules, with the organisation’s procedures and manuals or with the terms of an approval, certificate, specialised operation authorisation or with the content of a declaration which lowers safety or seriously hazards flight safety. The level 1 findings shall include: (1) failure to give the competent authority access to the facilities of the organisation in accordance with point ORO.GEN.140 of Annex III (Part-ORO) to this Regulation, or for balloons operators in accordance with points BOP.ADD.015 and BOP.ADD.035 of Annex II (Part-BOP) to Regulation (EU) 2018/395, during normal operating hours and after two written requests; (2) obtaining or maintaining the validity of the organisation certificate or specialised operations authorisation by falsification of submitted documentary evidence; (3) evidence of malpractice or fraudulent use of the organisation certificate or specialised operations authorisation; and (4) the lack of an accountable manager.(…)’ ARO.OPS.226 point (c)(2)(iii) EBT programmes require extensive use of data and suitable records systems. This is already required in the operator’s requirements ORO.GEN.220 and ORO.MLR.115; therefore, it has probably been overseen in the past. However, for the initial approval, the competent authority should verify that the operator is compliant as EBT will increase the workload and usability of the record-keeping system; therefore, this may be a first indication of an operator’s maturity to implement EBT. Page 8 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb The wording used ‘the adequacy of the operator’s record-keeping system, in particular with regard to flight crew training, checking and qualifications records’ refers to ORO.MLR.115 points (c) and (d) and the related AMC1 ORO.MLR.115, GM1 ORO.MLR.115(c), and GM1 ORO.MLR.115(d). ARO.OPS.226 point (c)(2)(iv) This provision allows the competent authority to access pilots grading results. This already applies today and EBT will not change the current situation. The competent authority is allowed to access the pilot records (ORO.GEN.140 ‘Access’) to verify ‘the suitability of the operator’s grading and assessment scheme’. Furthermore, the access to records and grading data for the verification of the grading system is also recognised at ICAO level (see Doc 9379 ‘Manual of Procedures for Establishment and Management of a State's Personnel Licensing System’ (Part I: General principles and organization Chapter 2 - The Licensing Authority, paragraph 2.8 Record-keeping)). ARO.OPS.226 point (d) The periodic oversight plan follows the following principles: — A performance-based safety objective is provided in the IR. — A more detailed criterion is then provided in the associated AMC1 ARO.OPS.226(d) ‘Approval and oversight of EBT programmes OVERSIGHT PLAN — PERIODIC ASSESSMENT TO VERIFY COMPLIANCE OF THE EBT PROGRAMME’ — Then, GM addressing an important criterion that competent authority should oversee is developed — GM1 to AMC1 ARO.OPS.226(d) ‘EFFECTIVENESS OF THE OPERATOR’S EBT PROGRAMME’. The provision is linked to another IR (ARO.GEN.350) that provides a reference when continuing compliance is not ensured. ‘ARO.GEN.350 (1) In the case of level 1 findings the competent authority shall take immediate and appropriate action to prohibit or limit activities, and if appropriate, it shall take action to revoke the certificate, specialised operations authorisation or specific approval or to limit or suspend it in whole or in part, depending upon the extent of the level 1 finding, until successful corrective action has been taken by the organisation.’ The intent of this rule also includes the need for the competent authority to have periodic observations of the training session; however, this requirement was not included as AMC2 ARO.GEN.305(b) already provides for such a requirement: ‘AMC2 ARO.GEN.305(b) Oversight programme PROCEDURES FOR OVERSIGHT OF OPERATIONS (…) (b) Audits and inspections, on a scale and frequency appropriate to the operation, should cover at least: (1) infrastructure, Page 9 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb (2) manuals, (3) training, (…) (c) The following types of inspections should be envisaged, as part of the oversight programme: (1) flight inspection, (2) ground inspection (e.g. documents and records), (3) training inspection (e.g. ground, aircraft/FSTD, (…)’ Point (b) normally means a documentation exercised, and point (c) normally means visit/inspection; therefore, observation of the training session. ARO.OPS.226 point (d) wording ‘EBT programme’ The term ‘EBT programme’ referred to in the rule is also contained in ORO.FC.231 point (a) ‘EBT programme’. While the table of assessment and training topics is a generic programme for an aircraft generation, the ‘EBT programme’ is specific to a particular operator and it encompasses all the requirements contained in ORO.FC.231 from point (a) to point (i). The ‘EBT programme’ is an approved programme for CAT aircraft. The reason for this approval is the existing provision ORO.FC.145 point (c); thus, ‘EBT programme’ encompasses an approved process by the competent authority. Annex III (Part-ORO) to Regulation (EU) No 965/2012 ORO.GEN.121 Certification specifications SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Explanatory note to ORO.GEN.121 The rule proposed follows the principles already established in ORO.GEN.120 ‘Means of compliance’. Point (a) is transposed from ORO.FTL.125 ‘Flight time specification schemes’ point (c) ‘(c) To demonstrate compliance with Regulation (EC) No 216/2008 and this Subpart, the operator shall apply the applicable certification specifications adopted by the Agency. (…)’ ORO.FC.146 Personnel providing training, checking and assessment SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Explanatory note to ORO.FC.146 ORO.FC.146(c) EBT is a paradigm shift and instructors play a key role in the delivery of the programme. The RMG found necessary to add an EBT course on top of the qualification required in the Aircrew Regulation. Doc 9995 requires this training as well: Page 10 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb ‘6.3.2 Instructors should undergo suitable training in order to adapt to the needs of training within an EBT programme. Training should provide the framework for existing instructors to develop their competence to undertake EBT assessment and training’. ORO.FC.146(c) wording ‘for an EBT programme’ This wording ‘for an EBT programme’ is used instead of ‘operator holding an approval for EBT’ or other wordings that could be possibly used in order to allow: — contracted activities under ORO.GEN.205; and — that other aspects of the training programme which are not linked to the EBT programme itself could be delivered by other personnel which are not EBT instructors. ORO.FC.146(c) wording ‘hold an Annex I (Part-FCL) instructor or examiner certificate’ The proposed rule is restricting the possibility of instructors that hold a certificate issued by a third country to become EBT instructors. By using the wording ‘hold an Annex I (Part-FCL) instructor or examiner certificate’, only instructors or examiners that hold a certificate issued in accordance with the EU regulatory framework can deliver EBT. The reasons for such a provision according to the RMG are the following: — The EBT programme based on competencies does not have the same prescriptive components as a task-based checking under Appendix 9 to Part-FCL. Therefore, the RMG, in an effort to ensure standardisation and integrity of the licence revalidation under EBT, wanted to put into place some level of control of instructor qualification. — To ensure alignment between Part-ORO of the Air OPS Regulation and Part-FCL of the Aircrew Regulation, the requirement of FCL.900 point (c) must be reproduced in Part-ORO. Therefore, only holders of European instructors’ certificates (with a European pilot licence or with a pilot licence issued by a third country but subject to FCL.900 (c)) are allowed to provide training to European licence holders. — The instructor qualification is anchored in Part-FCL and additional training is provided in Part-ORO. Therefore, the EBT system relies on the prerequisite of instructor qualification and standardisation in Part-FCL. Foreign certificates may or may not provide the same level of qualification and standardisation provided in Part-FCL; therefore, EU instructor certificates were required. — The level of complexity of the oversight will increase due to the different standards for instructor certificates in the non-EU countries. Furthermore, the national authority performs the oversight of the EBT programme, while EASA performs the oversight of the third-country ATOs. Allowing third-country instructors will overcomplicate the oversight for the national authority. — Furthermore, the situation where an instructor that holds a pilot licence issued by a third country provides training, only occurs when the operator has subcontracted its training to an ATO under ORO.GEN.205. In this situation, the efforts of standardisation are already big. Considering that a small number of non-standardised data introduced in the EBT system can Page 11 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb have big implications in the results of the programme, then only Part-FCL certificate holders should be allowed to provide EBT as they are standardised in EBT by the ATO. — The RMG was also concerned with the delivery of the EBT programme, as they believe that the quality of the delivery of the operator’s EBT programme could be compromised; since Europe is the first region delivering full EBT, Part-FCL certified instructors may better guarantee the consistency and philosophy of EBT. This is particularly important as at a later stage, in the context of the activities of RMT.0599, initial type rating courses may be subject to EBT. Note: Individual European certified trainers with a European pilot licence are allowed to provide EBT even if they are not the operator or ATO staff members. This is allowed under ORO.FC.205 on contracted activities. ORO.FC.146(c) wording ‘the operator’s EBT instructor standardisation’ The wording in point (a) of AMC1 ORO.FC.146(c) for the instructor’s standardisation is using ‘EBT’ for each of the two parts, ‘EBT instructor training’ and ‘EBT assessment of competence’, to ensure they are both specific for EBT. The use of ‘EBT assessment of competence’ is to ensure that the EBT instructor is allowed to revalidate the instructor certificate when the EBT assessment of competence and the assessment of competence for the revalidation of the instructor are combined. The RMG was reluctant to allow the EBT instructor to revalidate the EBT instructor certificate under an ATO not belonging to an airline, and therefore the requirements for the assessment are contained in the operators’ requirements. Hence, the revalidation of the EBT instructor certificate requires an operator. Following the concept already described in Subparts J and K of Part-FCL, the instructors should complete a course to become EBT instructors. This standardisation is composed of a training course and the assessment of competence, which follows the logic of Part-FCL. For example, FCL.930 ‘Training course’, FCL.935 ‘Assessment of competence’ and FCL.940.TRI TRI ‘Revalidation and renewal’ illustrate the situation for instructor courses and assessment: ‘FCL.940.TRI TRI — Revalidation and renewal (a) Revalidation (1) Aeroplanes. For revalidation of a TRI(A) certificate, the applicant shall, within the last 12 months preceding the expiry date of the certificate, fulfil one of the following 3 requirements: (i) conduct one of the following parts of a complete type rating training course: simulator session of at least 3hours or one air exercise of at least 1 hour comprising a minimum of 2 take-offs and landings; (ii) receive instructor refresher training as a TRI at an ATO; (iii) pass the assessment of competence in accordance with FCL.935. […]’ The RMG believes that it must be an operator EBT instructor training. Therefore, the instructor course is operator-specific. However, credits are foreseen in point (d) of AMC1 ORO.FC.146(c) when an instructor has experience in EBT, allowing for a shorter training course. Page 12 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb ORO.FC.146(c) The sentence ‘Completion of the operator’s EBT standardisation will qualify the instructor to perform EBT practical assessment’ was introduced because in the Aircrew Regulation the instructors do not have the privilege to perform EBT practical assessment. For example, the current FCL.905.TRI.TRI only provides a privilege to ‘instruct for’: ‘FCL.905.TRI TRI — Privileges and conditions The privileges of a TRI are to instruct for […]’ This provision introduces the link to Part-FCL for the EBT proficiency check in accordance with Appendix 10 (EBT practical assessment), and the wording ‘EBT practical assessment’ provides the link to Appendix 10 point 6 ‘The EBT practical assessment must be conducted in accordance with the operator’s EBT programme’. The use of ‘completion’ means also that the instructor successfully passed the instructor standardisation. In ORO.FC.231 (a)(3), this concept is already covered for the module; completion of an EBT module means to complete the programme (syllabi) and reach an acceptable level of performance. The same concept should be used for the instructor standardisation course: 1- the instructor has completed the syllabi for the EBT course, 2- an acceptable level of performance is reached (assessment of competence). For info, ORO.FC.231(a)(3)(i) (i) completes a minimum of two modules within the validity period of the type rating, separated by a period of not less than 3 months. The module is completed when: (…) ORO.FC.146(c)(2) wording ‘EBT practical assessment’ This wording is a transposition of the ICAO wording ‘practical assessment’ contained in Doc 9868 ‘PANS-TRG’ paragraph 4.4.1.2.2. Furthermore, practical assessment is defined in the new GM to definitions in Subpart ORO.FC. ORO.FC.146(c)(2) The use of a suitably qualified commander, as in AMC1 ORO.FC.230 (3)(v), has been retained under EBT. ORO.FC.231 Evidence-based training SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Explanatory note to ORO.FC.231 The EBT programme and philosophy are intended to be applied as the means of assessing and training key areas of flight crew performance in a recurrent training system. This is referred to in ICAO Annex 6, Operation of Aircraft, Part I, International commercial Air Transport — Aeroplanes, SARP 9.3, Flight crew member training programmes, and 9.4.4, Pilot proficiency checks. In addition, it is also referred to in ICAO Annex 1, Personnel Licensing, 1.2.5, Validity of licenses. The EBT programme considers the differences between aeroplane generations by tailoring the recurrent training programme to the aeroplane generation. The paradigm shift proposed under the EBT programme is not simply to replace a set of critical events with a new set, but to use the events Page 13 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb as a vehicle for assessing and developing crew performance across a range of competencies. In addition, EBT refocuses the instructor population onto analysis of the root causes to correct inappropriate actions, rather than simply asking a flight crew member to repeat a manoeuvre with no real understanding as to why it was not successfully flown in the first instance. Finally, it is acknowledged that in today’s high-fidelity simulator environment, very sophisticated training tools exist that are often not used effectively, as regulation focuses much more on checking. EBT seeks to redress the imbalance between training and checking. It recognises that an assessment of competence is necessary, but once completed, pilots learn more effectively when being trained by competent instructors to perform tasks and manage events measured according to a given set of observable behaviours (OBs), while not under test conditions. The data analyses undertaken to support the EBT programme illustrate inadequacies in the perpetuation of historical airline flight training regimes and identify areas in which major change is necessary. They strongly support the implementation of such change in both the regulation and development of recurrent airline pilot assessment and training. Finally, they identify the areas for improvement, providing the prioritisation of relevant training topics to guide in the construction of suitable EBT programmes. ORO.FC.231 point (a)(1) wording ‘a suitable EBT programme’ AMC1 ORO.FC.231(a) provides a more detailed presentation of the suitability of an operator’s EBT programme. The term ‘EBT programme’ referred to in the AMC is contained in ORO.FC.231 ‘EBT programme’. While the table of assessment and training topics is a generic programme in an aircraft generation, the ‘EBT programme’ is specific to a particular operator and it encompasses all the requirements contained in ORO.FC.231 from point (a) to point (i). The ‘EBT programme’ is an approved programme for CAT aircraft. The reason for this approval is the existing provision ORO.FC.145 point (c), thus ‘EBT programme’ encompasses an approved process by the competent authority. ORO.FC.231 point (a)(1) wording ‘demonstrate its capability to support the implementation’ The EBT training programme is intended to be implemented by phases, from a legacy training or other alternate training programmes such as the alternative training and qualification programme (ATQP) to a full EBT programme in accordance with ORO.FC.231. Mixed EBT or ATQPs are intended to provide (or have provided) enough experience for an operator to be ready to implement an EBT programme in accordance with ORO.FC.231. Also, this period should provide the competent authority with enough information on the resources needed to perform oversight of operators implementing an EBT programme in accordance with ORO.FC.231. This assures a robust and standardised EBT implementation in accordance with ORO.FC.231 across the spectrum of airlines with different levels of experience in and resources for this kind of programmes. Page 14 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb ORO.FC.231 point (a)(1) wording ‘equivalent level of safety’ The wording was transposed from the IR on ATQP (ORO.FC.A.245). The wording ‘equivalent level of safety’ is also used in other provisions across the Air OPS Regulation (e.g. minimum cabin crew, alternative means of compliance, etc.). ORO.FC.231point (a)(2) wording ‘3 year programme’ ‘3-year programme’ instead of ‘3-year cycle’, as provided in Doc 9995. It is used because: (a) the European rules generally use ‘programme’ instead of cycle (see Part-ORO); and (b) this Appendix to the Opinion proposes the definition of ‘cycle’ that expresses the notion of a 1- year period. Therefore, if ‘3-year cycle’ is used, it may be confusing. ORO.FC.231 point (a)(2)(iv) ‘evaluation’ The evaluation phase should consist of a line-orientated flight scenario during which there are one or more occurrences for evaluating one or more key elements of the required competencies. The root cause/contributing factor should be identified rather than the symptoms of any deficiency. This is not intended to be a comprehensive assessment of all areas of competency, nor a demonstration of all critical flight manoeuvres. During the evaluation phase, for any competency observed below minimum: — specific training needs should be determined; and — the subsequent SBT includes remediation and the flight crew member is not released to line flying until an acceptable level of performance is reached. ORO.FC.231 point (a)(2)(iv) ‘training’ The intent of the regulator is to complete the training phase after the evaluation phase, while the phases included in the training phase (MT and SBT) can be performed in any order. (A) An evaluation phase, comprising a line-orientated flight scenario (or scenarios) to assess competencies and identify individual training needs; and (B) A training phase, comprising: • manoeuvres training phase, comprising training to proficiency in certain defined manoeuvres; and • scenario-based training phase, comprising line-orientated flight scenario(s) to develop competencies and address individual training needs. ORO.FC.231 point (a)(3)(i) wording ‘type rating’ The use of the term ‘type rating’ clarifies the expiry date, as the validity of the type rating is up to the end of the month. Therefore, the intention of the RMG is to ensure two modules a year (each module composed of two simulator sessions). ORO.FC.231 point (a)(3)(i) wording ‘by a period of not less than 3 months’ The RMG discussed ICAO Annex 6 Part I Chapter 9 SARP 9.4.4 ‘Pilot proficiency checks’ where two checks a year are required, performed at least 4 months apart: Page 15 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb ‘9.4.4 Pilot proficiency checks 9.4.4.1 The operator shall ensure that piloting technique and the ability to execute emergency procedures is checked in such a way as to demonstrate the pilot’s competence on each type or variant of a type of aeroplane. Where the operation may be conducted under instrument flight rules, the operator shall ensure that the pilot’s competence to comply with such rules is demonstrated to either a check pilot of the operator or to a representative of the State of the Operator. Such checks shall be performed twice within any period of 1 year. Any two such checks which are similar and which occur within a period of 4 consecutive months shall not alone satisfy this requirement.’ The RMG considered that these checks are not similar, as they are not repetitive training tasks or events, but evaluations in different scenarios. Therefore, a 3-month period is consistent with the European regulatory framework where the OPC in ATQP (ORO.FC.A.245) has a validity period of 6 months with the possibility to do it 3 months in advance. Furthermore, according to ICAO Doc 9995, this document is a means of compliance with the Annex 6 SARP 9.4.4. ‘This manual is intended to provide guidance to Civil Aviation Authorities, operators and approved training organizations in the recurrent assessment and training of pilots referred to in Annex 6 to the Convention on International Civil Aviation, Operation of Aircraft, Part I, International Commercial Air Transport — Aeroplanes, paragraphs 9.3, Flight crew member training programmes, and 9.4.4, Pilot proficiency checks.’ ORO.FC.231 point (a)(4) ‘instructor concordance’ It is imperative that instructor concordance is regulated as a core aspect of an EBT programme, and should be held to high standards, as it is one of the most critical drivers of data quality in an EBT programme. Concordance should be required to prevent drift in instructor quality over time, especially in the non-technical competencies. ORO.FC.231 point (a)(5) wording ‘line operations’ The use of the term ‘line operations’ allows for training flights. At the same time, it restricts line flying when a minimum performance is not achieved. EBT is an FSTD programme; therefore, the recommendation is to provide such remedial training in the FSTD. However, the operator is allowed to conduct training flights and the pilot should be permitted to be trained in flight, assuming the minimum performance for line operations was achieved, for example, when a pilot obtains a grade two in application of procedures (PRO). This is especially relevant in small aircraft models, and although most of those models are not yet permitted in EBT, EASA has plans to incorporate them in the future. The term ‘line operations’ is used in the Air OPS Regulation and although no definition is provided, its meaning is obvious. ORO.FC.231 point (a)(5) If a low performance is observed and there is no immediate opportunity for remedial training (e.g. unforeseen circumstances, sessions separated by several days apart with flight duty in the middle, etc.), the pilot should be removed from line operations until an acceptable level of performance can be achieved. Page 16 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb ORO.FC.231(b) Why is there a need to require a competency framework? Mastering a defined number of competencies should allow a pilot to manage most of the situations in flight. The main benefit of a competency-based approach to training is its potential to encourage and enable individual aviation professionals to reach their highest level of operational capability while ensuring a basic level of competence as a minimum standard. This approach is supported by the study of MAN4GEN. Legacy training and checking, and ATQP v EBT The major difference between ATQP and EBT lies in the approach taken to identify the KSA for the successful performance in the job. ATQP and traditional training (Appendix 9) focus on a task-based approach of the pilot role by identifying the job-related tasks (and subtasks), which are then used to identify a list of KSA required for successful pilot performance. On the other side, the EBT approach starts with the performance indicators/observable behaviours of exemplary pilots to define an official list of observable behaviours (see list of OBs in the EBT competency framework) to then group them in competencies (see list of the EASA EBT competency framework — 9 competencies). Through this process, the 9 EBT competencies are related to effective or superior performance. Therefore, the question is not which KSA are required to perform the tasks of an airline pilot (ATQP approach) but which KSA do superior performers airline pilots possess and use (EBT approach). PRINCIPLES OF A COMPETENCY FRAMEWORK — The purpose of competency-based assessment and training is to assess and train the capacity of an individual to perform at the standard expected in an organisational workplace. — There is an explicit link between competencies and training, required performance on the job, and assessment. — Competencies are formulated in a way that ensures they can be developed, observed and assessed consistently in a wide variety of work contexts for a given aviation profession or role. — Each stakeholder in the process (including the trainee, instructor, training organisation, operator and regulator) has a common understanding of the competency requirements. — Clear performance criteria are established for assessing competence. — Evidence of competent performance is valid and reliable. — Instructors’ and assessors’ judgments are calibrated to achieve a high degree of inter-rater reliability. — The assessment of competencies is based on multiple observations across multiple contexts. — A relevant competency framework is clearly defined for a particular role. — To be considered competent, an individual demonstrates an integrated performance of all the required competencies to a specified standard. ASSUMPTIONS — All tasks performed by aviation professionals require the application of a relevant set of competencies. Page 17 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb — Aviation professionals apply the same set of competencies in a given role throughout their career but with different degrees of performance. ORO.FC.231(c) This requirement is transposed from Doc 9995 paragraphs 3.6.6 and 3.6.7 with the necessary amendments into the European regulatory system. ‘3.6.6 Quality management. The training system performance should be measured and evaluated in respect of the organizational objectives. Monitoring should include a feedback system to identify trends and ensure corrective action where necessary. The quality system of the operator or training organization, as defined in Doc 9841, the Manual on the Approval of Training Organizations, should monitor alignment with the EBT assessment and training guidelines recommended in this manual. 3.6.7 Feedback system. For the purpose of collecting data from an EBT programme, and making adjustments and continuous improvement to the training system, an operator should implement a performance feedback system utilising defined metrics (see paragraph 5.3)’. ORO.FC.231(c) point (1)(ii) The requirement is transposed from ICAO Doc 9995 paragraph 3.6.6 ‘(…) should monitor alignment with the EBT assessment and training guidelines recommended in this manual.(…)’. The interpretation of this paragraph was the following: as one of the main objectives of the EBT programme is to develop pilot competencies, the sentence in 3.6.6 was transformed to ‘develops pilot competencies’. ORO.FC.231(c) point (2) ‘ORO.GEN.200 Management system (a) The operator shall establish, implement and maintain a management system that includes: (1) (…) (4) maintaining personnel trained and competent to perform their tasks;’ ORO.FC.231(d) The paradigm shift from legacy training and checking programmes is a move away from checking the execution of predefined manoeuvres and tasks, based on the quality of execution. Remediation in these cases often leads to simple task repetition without an understanding of the underlying causes of ineffective performance. To be consistent with the central philosophy of EBT, the assessment should be completed at key points during the module, and the performance should be evaluated against each of the defined competencies, using the most relevant OBs to the performance observed. The instructor should take an overview of everything observed during the phase, and using a methodology similar to that published, award grades in each competency only. The grading system should be used for crew assessment, in addition to providing quantifiable data for the measurement of the training system performance. It can range from a simple ‘acceptable/unacceptable’ grading performance system to a gradual relative measurement system. ORO.FC.231(d) wording ‘a grading system to assess’ Page 18 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb The provision is transposed from Doc 9995 paragraph 3.6.3: ‘3.6.3 Assessment and grading system. A full description of the competencies is provided in Appendix 1 to Part II. It is essential to note that an operator intending to use this framework should in addition develop a clear assessment and grading system for expected crew performance. Competencies are a fundamental component of the grading system. It is not the intention of this document to fully describe a grading system, but a grading system should be used for crew assessment, in addition to providing quantifiable data for the measurement of the training system performance. It can range from a simple ‘acceptable/unacceptable’ grading performance system to a graduated relative measurement system.’ ORO.FC.231(d) point (1)(iii) Data integrity is the maintenance of, and the assurance of the accuracy and consistency of, data over its entire life-cycle and is a critical aspect of the design, implementation and usage of any system which stores, processes, or retrieves data. Any unintended changes to data as the result of a storage, retrieval or processing operation, including malicious intent, unexpected hardware failure, and human error, is failure of data integrity. ORO.FC.231(d) point (2) Why do we need a verification of the grading system? The EBT grading system provides a norm-referenced system, although it contains some characteristics of a criterion-referenced system. Glasser (1963) formalised the concept of criterion-referenced testing (CRT). The development of a CRT entails, firstly, a statement of behavioural objectives and then a systematic generation of test items designed to unambiguously ascertain to what degree these objectives have been met. Standards of performance are set using minimal levels of competence before the test is applied. The elements of the development of a CRT (e.g. to unambiguously ascertain) are difficult to achieve in the EBT system for certain OBs and grading, especially as regards non-technical skills, associated OBs and their grading. For example, a grade 3 (‘The pilot communicated adequately, by regularly demonstrating most of the OBs when required, which resulted in a safe operation’) in communication will require that all OBs are clearly and unambiguously defined. As an example, the OB ‘Uses eye contact, body movement and gestures that are consistent with and support verbal messages’ would require further criteria in the context of a particular scenario to reach the ‘unambiguously ascertain to what degree the objective has been met’ explained by Glasser (1963). These criteria could be: at least 20 seconds of eye contact along with a body movement of three gestures (e.g. indicating with the arm the side of the aircraft affected) that support the verbal message of the explanation of an engine problem to the cabin crew. Today, the revalidation of licences is based on a criterion-referenced system for the conduct of the training, tests and checks of Appendix 9 with regard to technical competencies (see FLIGHT TEST TOLERANCE, Appendix 9 to Part-FCL of the Aircrew Regulation. For the non-technical competencies, a norm-referenced system may be provided (see ORO.FC.115 &215 of the Air OPS Regulation). Today, the European aviation system uses a criterion-referenced system for revalidation of pilot licences to ensure a level playing field (one of the aims of the Basic Regulation — see Article 1). EBT Page 19 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb proposes a norm-referenced system. In order to combine both methods, a feedback process is proposed. This process is recommended in different scientific works. From all the scientific works, the RMG provided a reference to the book ‘Criterion-referenced and norm-referenced assessments: compatibility and complementarity’ author: Beatrice Lok, Carmel McNaught & Kenneth Young. An extract is provided to support the need for the verification of the grading system in EBT. The book proposed a yearly verification of the grading system; however, the RMG opposed this proposal and instead EASA proposed a one-time feedback every 3 years. ‘Feedback process: There is no need to choose between norm referencing and criterion referencing. They are both present. — Not only are they both present, but with the caveat about minor adjustments from year to year, they are consistent. Thus, it is possible both to define rubrics (criterion referencing) and to prescribe grade-distribution guidelines (norm referencing), provided the latter contains a degree of flexibility. — The presence of norm referencing and criterion referencing in a loop enables the generation of both useful feedback to learners and useful summative information to external stakeholders. — The use of criteria allows meaningful reference to higher-order learning outcomes. While these are inevitably ambiguous and even unknown to external stakeholders, the simultaneous use of norm referencing allows the interpretation of these criteria to be supported by norm comparisons, and to guard against grade inflation. — Since these steps are all in a loop, there is no need to argue which one comes first. — The entire approach is coherent with modern quality-assurance and fitness-for purpose concepts.’ ORO.FC.231(e) point (1) EASA is introducing certification specification (CS) for EBT. This approach allows to match the detailed provisions with regard to the FSTD required to deliver the EBT programme with the requirements to certify the FSTD which are contained in the CS-FSTD - see https://www.easa.europa.eu/regulations. Page 20 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb Therefore, the IR is providing the safety objective and remains technology agnostic to allow a proper evaluation of the regulatory framework. ORO.FC.231(h) wording ‘competence’ The heading of the rule is ‘line evaluation of competence’. The word ‘competence’ was selected instead of ‘competency’, because the RMG wanted to reflect that an assessment of the competencies must be made and the pilot has to reach a certain level of performance: ‘competence’. ORO.FC.231(h)(1) The safety objective is stated in the IR. The sentence ‘undertake a line evaluation in an aircraft in flight to demonstrate the safe, effective and efficient conduct’ was transposed from Doc 9995, FOREWORD and in Part I, paragraph 1.6: ‘The aim of this programme is to develop and evaluate the identified competencies required to operate safely, effectively and efficiently in a commercial air transport environment’ ‘Normal line operations’ is used because ORO.FC.230 point (c)(1) uses the same wording: ‘(1) Each flight crew member shall complete a line check on the aircraft to demonstrate competence in carrying out normal line operations described in the operations manual.’ The provision of the line evaluation of competence intends to have the same scope as the line check currently has. Obviously, this implies successful demonstration of competence in the management of any abnormal or emergency situations that may occur during the flight. Therefore, the use of ‘normal operation’ is not referring to the malfunctions; it is referring to a normal flight (not test flight, not maintenance flight, etc.). ORO.FC.231(h)(1) wording ‘in an aircraft’ The wording ‘in an aircraft’ is used in this IR to remove any ambiguity as to where the line evaluation may be undertaken. The RMG noted that in GM1 ORO.FC.230 point (c) there is a mention of ‘line check and proficiency training and checking’ in an FSTD. This will not be transferred into GM1 ORO.FC.231. ORO.FC.231(h)(3)(i) The intent of this rule is to continue to permit those operators who had been conducting ATQPs for more than 24 months and can, therefore, continue to apply a 24-month line evaluation (check under ATQP) periodicity when they transition to an EBT programme. It is worthy of note that this does not apply under the mixed EBT implementation phase. ORO.FC.230 & 245 remain applicable. Under this IR, it is left to the discretion of the competent authority whether it will grant a 24-month validity period for line checks to those operators who had not previously conducted an ATQP. However, the competent authority shall ensure that the operator is fully conversant with a competency-based evaluation system prior to applying this rule. Further guidance will be issued in a safety promotion document – EASA EBT manual. The reason behind allowing extensions of validity periods in the line evaluation of competence (line check) is the following: — Legacy training requires one line check per year. — ATQP provides an alleviation of one line check every 2 years because it requires a line- orientated evaluation per year. That means that two line orientated evaluations (LOEs) substitute one line check. Page 21 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb — EBT provides more opportunities than the ATQP for LOE, because in the evaluation and in the scenario-based training both scenarios are line-orientated flights and required twice per year (EBT requires two modules a year). ORO.FC.231(i)(1) The provision was drafted as follows: The RMG: (a) transposed the existing ORO.FC.230 of the Air OPS Regulation: ‘(…) (d) Emergency and safety equipment training and checking Each flight crew member shall complete training and checking on the location and use of all emergency and safety equipment carried. The validity period of an emergency and safety equipment check shall be 12 calendar months. (…) (f) Each flight crew member shall undergo ground training and flight training in an FSTD or an aircraft, or a combination of FSTD and aircraft training, at least every 12 calendar months. (…)’; (b) combined the 2 points; (c) removed the word ‘check’ because in EBT the concept of checking is removed. Also, in the industry, training and checking are combined; therefore, the text is intended to reflect the industry’s practice; and (d) finalised the provision by adjusting the text to the EBT regulation. ORO.FC.231(i)(2) The provision is transposed from ORO.FC.A.245 of the Air OPS Regulation and reworded as appropriate. The alleviation is consistent with the existing alleviation provided for the ATQP. ORO.FC.232 EBT programme assessment and training topics SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib ORO.FC.232(b)(1) The provision follows the principles of the ICAO Doc 9995. In fact, the definition of EBT in Doc 9995 highlights this principle. ‘Evidence-based training (EBT). Training and assessment based on operational data that is characterized by developing and assessing the overall capability of a trainee across a range of core competencies rather than by measuring the performance in individual events or manoeuvres.’ EASA has additionally introduced the wording ‘scientific principles’ because the wording use in Doc.9995 is not covered in Regulation (EU) 965/2012. Scientific principles is already introduced in ORO.FTL and therefore EASA decided to align the wording. The table defines also the frequency of training those topics. The programme is described at CS level. This means that special conditions can be also used to demonstrate compliance with the IR (in accordance with ORO.GEN.121 of the Air OPS Regulation). However, in order to seek for an approval, Page 22 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb the operator should demonstrate that this change of the programme is subject to a proper study of the operational risks. Such a large study was conducted by a collaborative group (industry and the regulator) in the IATA data report for EBT. If operators would like to modify the ‘table of assessment and training topics’, a similar work must be carried out. Annex I (Part-FCL) to Regulation (EU) No 1178/2011 Concept of revalidation within an EBT programme Background of licence revalidation — The current revalidation process has four components: (a) the applicant; (b) the examiner; (c) the technical assessment carried out in the simulator or the aircraft; and (d) the administrative procedure that includes the completion of Appendix 9, and the rest of administrative procedures in Part-FCL FCL.1030 points (b), (c) and (d) that include the licence endorsements. This process is carried out by the same person (examiner) who performs the technical assessment and the administrative procedure, at the same ‘location’ (simulator or aircraft) and at the same time (the date and time of the proficiency check). Note: Although most of the licence proficiency checks (LPCs) are carried out by a single examiner, the possibility of having several examiners for the same check already exists. — The EBT philosophy should provide a different approach, where training is maximised and therefore checks disappear (assessment is introduced) and the pilot is trained in a NON- jeopardy environment. Furthermore, the continuous training evidence of the pilot (data) should provide a better assessment of the competence of the pilot. Therefore: (a) the EBT technical assessment has several events (simulator sessions) instead of one; (b) there are several assessors of pilot performance (EBT instructors) instead of just one (examiner); however, the nominated person for crew training is an examiner designated to provide a final assessment of the data collected; and (c) the administrative procedure should be maintained; however, due to the several people being involved in the technical assessment, the administrative procedures involve the nominated person for crew training who carries the responsibility of the licence revalidation and a designated person who will endorse the licence. Concept of licence revalidation in the context of an operator’s EBT programme The revalidation process proposed has the following components: (a) the applicant; (b) the people involved in the revalidation of the pilot licence: (1) the nominated person for crew training (or the deputy(ies)) who is an examiner responsible for the operator’s EBT programme, (ensuring that the manoeuvres assessed Page 23 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb are of a good training value and that the applicant completed those manoeuvres). The nominated person will be mostly responsible for the completion of Appendix 10. This person (or the deputy(ies)) also has the overall picture of the pilot training data for the period of validity (as shown by the evidence provided by the EBT programme); (2) the designated person who has the signature delegation from the nominated person to endorse the licence and complete Appendix 10; and (3) the EBT instructors who delivered each of the technical assessments that provide data to the EBT grading system and the training system performance; (c) the several technical assessments carried out in the simulators which provide the necessary evidence to ensure that the pilot has an acceptable level of performance; and (d) the administrative procedure which includes the completion of Appendix 10 and the rest of administrative procedures provided in FCL.1030. Explanatory note to Annex I (Part-FCL) to Regulation (EU) No 1178/2011 FCL.015 Application and issue, revalidation and renewal of licences, ratings and certificates SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib FCL.625 IR — Validity, revalidation and renewal SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib FCL.625.A IR(A) — Revalidation SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib FCL.740 — Validity and renewal of class and type ratings SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib FCL.740.A — Revalidation of class and type ratings — aeroplanes SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib FCL.905.TRI TRI — Privileges and conditions SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib FCL.905.SFI SFI — Privileges and conditions SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib FCL.1025 Validity, revalidation and renewal of examiner certificates SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Appendix 10 to Annex I (Part-FCL) to Regulation (EU) No 1178/2011 SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Explanatory note to Appendix 10 Page 24 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb Under the existing Part-FCL Appendix 9, proficiency check has two components: (1) the technical assessment in the FSTD or aircraft; and (2) the administrative action. This is based on a single event taking place. Unlike that, the EBT philosophy dictates that a candidate is continuously assessed throughout the programme. This is achieved through an EBT practical assessment within each module. Appendix 10 is therefore created to cater for the needs (multiple events are taking place) of licence revalidation within an EBT programme. The completion of Appendix 10 proficiency check is based on multiple data obtained through the EBT programme regarding an enrolled flight crew member. It is therefore not based on a single event. Instead, Appendix 10 requires a continuous assessment and training of the pilot where each competency is demonstrated at or above the minimum acceptable level of performance. This may take place in a simulated environment. Appendix 10 wording ‘EBT practical assessment’ This wording is a transposition of the ICAO wording ‘practical assessment’ contained in Doc 9868 ‘PANS-TRG’ paragraph 4.4.1.2.2. Furthermore, practical assessment is defined in the new GM to definitions in Subpart ORO.FC. The wording was different in the NPA, and EASA finally decided to propose a wording closer to the one used in ICAO. Appendix 10 paragraphs 1 and 5 An EBT practical assessment within an EBT programme is equivalent to a proficiency check. However, to legally complete a proficiency check and revalidate the pilot’s licence, paragraph 5 details the requirements, including several EBT practical assessments. Appendix 10 paragraph 2 point (a)(2) This provision intends to ensure that for each type where the EBT programme is applied, there is a nominated person for crew training (or a deputy (or deputies)) that ensures the relevance of the EBT system. For airlines with several types, it is intended that there is a deputy for each fleet (type) that is responsible for the correct delivery of the EBT programme and for ensuring that the EBT system is properly working. Appendix 10 paragraph 4 point (b) Safety promotion material — completion of the operator’s EBT programme EASA has planned safety promotion task (SPT).012 to support the implementation of EBT. The following material has been developed: SPT.012 — safety promotion task 012 — safety material for EBT — COMPLETION OF THE OPERATOR’S EBT PROGRAMME WITHIN THE PERIOD OF VALIDITY (SEE FCL.1030 (b)(3)(ii) AND APPENDIX 10 PARAGRAPH 4(b). Page 25 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 1. Rationale behind the proposed amendments to the implementing rules presented in Annexes Ib and IIb APPLICANTS USING APPENDIX 10 SHALL, WITHIN THE PERIOD OF VALIDITY, COMPLETE THE OPERATOR’S EBT PROGRAMME (a) The applicant completes the operator’s EBT programme applicable to the period of validity. Normally, the rating validity is 1 year; therefore, it refers to the modules and training planned for that period of time. (b) When the applicant is enrolled part-way through the period of validity of the rating (e.g. when pilots join a new airline, or they change aircraft types), the applicant is only required to complete the elements of the operator’s EBT programme for the remaining period of validity. To ‘complete the operator’s EBT programme’ means to complete the EBT modules and any other additional training (ground, FSTD, aeroplane) or evaluation in the programme (e.g. line evaluation of competence, etc.). However, only the modules will be considered for the purpose of the EBT PRACTICAL ASSESSMENT to revalidate the licence in accordance with Appendix 10. Appendix 10 paragraph 5 point (c)(1) — delegation of signature It should be noted that the intent of the RMG and EASA for the delegation of signature proposed in this Opinion, is that the delegation of the signature should not render the person actually signing the licence accountable. The accountability remains with the nominated person (or the deputy(ies)). Appendix 10 paragraph 5 point (c)(1) — ‘specifically authorised for that purpose by the competent authority responsible for the applicant’s licence’ The provision was transposed from FCL.1030 ‘Conduct of skill test, proficiency checks and assessments of competence’ point (b)(2). Page 26 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail 2. Proposed amendments to CS, AMC & GM and rationale in detail This chapter contains the associated AMC & GM and CS (including also the associated safety promotion actions) as well as the rationale behind the proposed change. To differentiate them from the proposed rules, the font colour used for the explanatory notes is blue. The text of the amendment is arranged to show deleted text, new or amended text as shown below: — deleted text is marked with strike through; — new or amended text is highlighted in grey; — an ellipsis ‘[…]’ indicates that the rest of the text is unchanged. GM to Annex I (Definitions) to Regulation (EU) No 965/2012 GM2 Annex I Definitions ABBREVIATIONS AND ACRONYMS […] APP approach CLB climb COM communication (EBT competency) CRZ cruise DES descent EBT evidence-based training EVAL evaluation phase FPA flight path management — automation (EBT competency) FPM flight path management — manual control (EBT competency) GND ground ISI in-seat instruction KNO application of knowledge (EBT competency) LDG landing LOC-I loss of control in-flight LTW leadership and teamwork (EBT competency) MT manoeuvres training phase OB observable behaviour PRO application of procedures (EBT competency) PSD problem-solving & decision-making (EBT competency) SAW situation awareness (EBT competency) Page 27 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail SBT scenario-based training TO take-off UPRT upset prevention and recovery training WLM workload management (EBT competency) GM1X Annex I Definitions EVIDENCE-BASED TRAINING ‘Behaviour’ refers to the way a person responds, either overtly or covertly, to a specific set of conditions, and which is capable of being measured. ‘Instructor concordance’ is also called ‘inter-rater reliability’. ‘Conditions’ refers to anything that may qualify a specific environment in which performance will be demonstrated. ‘Cycle’ refers to the combination of two modules where Cycle 1 comprises Modules 1 and 2, Cycle 2 comprises Modules 3 and 4, and Cycle 3 comprises Modules 5 and 6 of the 3-year EBT programme. ‘Evaluation phase (EVAL)’ refers to the phase where a first assessment of competencies is performed in order to identify individual training needs. On completion of the evaluation phase, any areas that do not meet the minimum competency standard will become the focus of the subsequent training. The evaluation phase comprises a complete mission as a crew. ‘Facilitation technique’ refers to an active training method, which uses effective questioning, listening and a non-judgmental approach, and is particularly effective in developing skills and attitudes, assisting trainees in developing insight and their own solutions, resulting in better understanding, retention and commitment. ‘Line-orientated flight scenario’ refers to the assessment and training involving a realistic, ‘real-time’, complete mission simulation of scenarios that are representative of line operations. ‘Monitoring’ refers to a cognitive process to compare an actual to an expected state. It requires knowledge, skills and attitudes to create a mental model and to take appropriate action when deviations are recognised. ‘Observable behaviour (OB)’ refers to a single role-related behaviour that can be observed. The instructor may or may not be able to measure it. ‘Performance criteria’ refers to statements used to assess whether the required levels of performance have been achieved for a competency. A performance criterion consists of an OB, a condition (or conditions) and a competency standard. ‘Practical assessment (or EBT practical assessment)’ refers to the primary method for assessing performance and should serve to verify the integrated performance of competencies. It takes place in either a simulated or an operational environment. An EBT assessment is equivalent to a proficiency check and is performed under the instructor privilege in the context of proficiency check in accordance with Appendix 10 to Part-FCL. More information can be found in ICAO Doc 9868 ‘PANS-TRG’. Page 28 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ‘Line-orientated flight scenario(s)’ are comprised of scenario elements derived from the table of assessment and training topics. ‘Scenario-based training phase (SBT)’ refers to the largest phase in the EBT programme. It is designed to maximise crew’s exposure to a variety of situations that develop and sustain a high level of competency and resilience. The scenario for this phase should include critical external and environmental threats, to build effective crew interaction to identify and manage errors. A portion of the phase will also be directed towards the management of critical system malfunctions. Scenario elements address the training topic and detail the threat and/or error that the crew are exposed to. ‘Train-to-proficiency’ refers to approved training designed to achieve end-state performance objectives, providing sufficient assurance that the trained individual is capable of consistently carrying out specific tasks safely and effectively. Note: In the context of this definition, ‘train-to-proficiency’ can be replaced by ‘training-to-proficiency’. Behaviour This term appears in the definition of performance criteria. It is transposed from Doc 9995. It is important to highlight the wording of ‘capable of being measured’; this does not mean that the observer may be able to measure it, as the observer has obvious limitations (technical or human limitations) that may prevent the measurement of the behaviour. Conditions This definition was introduced because it is frequently used in the context of competencies and observable behaviours. The definition is transposed from the working paper to ICAO Doc 9868 ‘PANS- TRG’, AN-WP/9237 Appendix A page A3 ‘preliminary review of proposed amendments to Annex 1 and the Doc 9868 ‘PANS-TRG’ consequential to amendment 5 to the Doc 9868 ‘PANS-TRG’ with additional proposals developed by the ‘competency-based training and assessment task force’’. Evaluation phase Further guidance is provided in this GM to complement the definition provided in the IR. Facilitation technique Primary technique that should be used for EBT and competency-based training. Monitoring Monitoring is the fundament of threat and error management. Monitoring is embedded in the competency framework provided in ORO.FC.231, and its behaviour indicators are spread out in different competencies. Observable behaviour The definition is transposed from the ICAO Doc 9841 definition. However, it has been slightly amended to express the idea that although the observable behaviour is ‘capable of being measured’ as per the definition of ‘behaviour’, the instructor may be unable to measure it. This limitation (being unable to measure a behaviour) occurs due to the obvious technical or human limitations of the instructor. In Page 29 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail other words, it is very difficult to observe and measure ‘all’ and ‘every single’ behaviour that occur in an aircraft or in a simulator of aircraft for a long period of time (e.g. 8 hours of a module). ‘Performance criteria’ The definition is transposed from ICAO working papers for Doc 9868 ‘PANS-TRG’. Train-to-proficiency This text is based on GM15 Annex 1 definitions on UPRT and is referred to within the context of EBT. Practical assessment Although ICAO Doc 9995 follows an approach where summative assessment is performed at the end of the evaluation as follows: ‘3.6.2 The evaluation phase of each module will periodically be the focus of licence renewal or revalidation and may ultimately be the means by which Licensing Authorities continue to ensure that competence is maintained to hold a professional licence and type rating as applicable.’ The draft ICAO Doc 9868 ‘PANS-TRG’ to be soon published contains paragraph 4.4.1.2.2 which is moving the summative assessment, that otherwise would be made in the evaluation phase, to the end of the module. This is ensuring that no pilot is allowed to fly if found NOT competent. Below an extract of ICAO Doc 9868 ‘PANS-TRG’ ‘4.4.1.2 Assessment methods 4.4.1.2.1 The primary method for assessing performance is the conduct of practical assessments, which should serve to verify the integrated performance of competencies. It may be necessary to supplement practical assessments with other forms of evaluation. The supplemental evaluations may be included as a result of regulatory requirements and/or a decision that these methods are necessary to confirm that competence has been achieved. 4.4.1.2.2 Practical assessments take place in either a simulated or operational environment. There are two types of practical assessment: formative assessments and summative assessments. Formative and summative assessments are conducted based on 4.6.6 and 4.6.7. 4.4.1.2.2.1 Formative assessments 4.4.1.2.2.1.1 Formative assessments are a part of the learning process. Instructors provide feedback to the trainee on how they are progressing toward the interim or final competency standard. This type of assessment enables the trainee to progressively build on competencies already acquired and should aid learning by identifying gaps as learning opportunities. If trainees receive feedback or are assessed only at the end of the training, they will have no opportunity to use that information to improve their performance. The frequency and number of formative assessments may vary depending on the duration of the training and the syllabus structure and its assessment plan (see 4.6). 4.4.1.2.2.1.2 Formative assessments should serve to: a) motivate trainees; b) identify strengths and weaknesses; and c) promote learning. Page 30 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail 4.4.1.2.2.2 Summative assessments 4.4.1.2.2.2.1 Summative assessments provide a method that enables the instructor/assessor to work with a trainee to collect evidence of the competencies and performance criteria to be demonstrated with respect to the interim or final competency standard(s). Summative assessments are carried out at defined points during the training and/or at the end of training. During summative assessments, the decision is either ‘competent’ or ‘not competent’ with respect to the interim or final competency standard(s). However, this can be further developed into a more refined grading system with a scale of judgements to improve feedback for the trainee and training personnel. 4.4.4.1.2.2.2 Summative assessments that are conducted during the course to evaluate the progress of the trainee are typically carried out by the instructing team. It may be advantageous if the instructors conducting these assessments were different from the instructors who routinely work with the trainee. Summative assessments conducted at the end of training and that lead to the issue of a licence and/or rating have both legal and safety implications. Therefore, the personnel carrying out these assessments should have the necessary competencies to assess objectively and meet the authority’s requirements. Such personnel should be provided with the tools necessary to collect evidence in a systematic and reliable manner in order to ensure inter-rater reliability. 4.4.1.2.3 The list of methods below that supplement practical assessments is not intended to be restrictive. Any suitable supplemental method for assessing competence may be used. Other methods may include projects and group assignments.’ Train-to-proficiency This text is based on GM15 Annex I Definitions on UPRT and is referred to within the context of EBT. Annex II (Part-ARO) to Regulation (EU) No 965/2012 SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib AMC1 ARO.OPS.226(a) Approval and oversight of evidence-based training programmes QUALIFICATION AND TRAINING — INSPECTORS (a) For the initial approval and oversight of an operator’s EBT programme, the inspector of the competent authority should undertake EBT training as part of their required technical training (see AMC2 ARO.GEN.200(a)(2)). At the conclusion of the inspector training, the inspector should: (1) know the principles of EBT, including the following underlying principles: (i) competency-based training; (ii) learning from positive performance; (iii) building resilience; and; (iv) data-driven training; (2) know the structure of an EBT module; (3) know the method of training delivery for each phase of an EBT module; Page 31 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (4) know the principles of adult learning and how they relate to EBT; (5) recognise effective observations based on a competency framework, and document evidence of observed performance; (6) recognise and relate specific performance observations of competencies; (7) recognise trainee performance to determine competency-based training needs and recognise strengths; (8) understand methods for the evaluation of performance using a competency-based grading system; (9) recognise appropriate teaching styles during simulator training to accommodate trainee learning needs; (10) recognise facilitated trainee learning, focusing on specific competency-based training needs; and (11) understand how to conduct a debrief using facilitation techniques. (b) The objective of such training is to: (1) attain the adequate level of knowledge in the principles of approval and oversight of the EBT programmes; and (2) acquire the ability to recognise the EBT programme suitability. AMC1 ARO.OPS.226(a) The intention of the RMG and EASA is not to substitute the existing requirements in AMC4 ARO.GEN.200(a)(2) regarding the qualification of the inspector for the EBT programmes. On the contrary, the EBT training requirements are additional to those contained in AMC4 ARO.GEN.200(a)(2). Only the EBT training course is required for inspectors and not the EBT assessment of competence (demonstration). This approach follows the concept already introduced in the regulation for the Fatigue Risk Management System course. Since the EBT paradigm is mainly under the supervision of the operator, including licencing issue, the inspector needs to have an acute understanding of the principles, philosophy and application of EBT concepts, in order to understand the performance of the operator The demonstration of the acceptable level of knowledge of the inspector can be achieved through an on-the-job training. AMC1 ARO.OPS.226(a) wording ‘technical training’ The use of the term ‘technical training’ is referring to AMC2 ARO.GEN.200(a)(2) point (a)(11). ‘AMC2 ARO.GEN.200(a)(2) Management system QUALIFICATION AND TRAINING — INSPECTORS (a) Initial training programme: Page 32 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The initial training programme for inspectors should include, as appropriate to their role, current knowledge, experience and skills in at least all of the following: (…) (11) technical training, including training on aircraft-specific subjects, appropriate to the role and tasks of the inspector, in particular for those areas requiring approvals.’ GM1 to AMC1 ARO.OPS.226(a) Approval and oversight of evidence-based training programmes QUALIFICATION AND TRAINING — PRINCIPLES OF EBT — DATA-DRIVEN TRAINING EBT is a data-driven programme and proper oversight requires the inspector to have a good understanding of all features where data plays an important role in the EBT programme: (a) Flight crew training data: (1) data related to grading of competencies (level 1), data related to OBs (level 2) and how it can be used to drive the design of the operator’s EBT programme. Other training data (level 3) and how it is used in the contextualisation of an example scenario element. (2) individual flight crew training data: understand how it is used: (i) in regard to licence revalidation and renewal; and (ii) to provide tailored training and additional FSTD training. (b) Data from the management system: understand how it may be used for the selection of the example scenario element and the contextualisation of the example scenario element. (c) Instructor standardisation and concordance data: (1) how the EBT data is used to standardise the instructor and how, at the same time, the operator ensures the necessary just culture and a non-jeopardy environment for the instructors (referred to in the instructor concordance assurance programme). (2) understand the importance of quality in the data – the feedback loop of the EBT programme. GM1 to AMC1 ARO.OPS.226(a) Approval and oversight of evidence-based training programmes QUALIFICATION AND TRAINING — OPERATOR’S EBT PROGRAMME SUITABILITY To recognise and evaluate the suitability of an operator’s EBT programme, the inspector’s training programme may include those features as training objectives. AMC1 ORO.FC.231(a) provides the list of features of a suitable EBT programme. GM1 to AMC1 ARO.OPS.226(a) The GM provides further details on the learning objective number (b)(2) ‘acquire the ability to recognised the EBT programme suitability’ contained in AMC1 ARO.OPS.226(a). Page 33 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC1 ARO.OPS.226(c) Approval and oversight of evidence-based training programmes INITIAL APPROVAL — VERIFICATION OF COMPLIANCE When approving an EBT programme, the competent authority should ensure that the operator fulfils all the applicable criteria of ORO.FC.231 and its associated AMC. In particular, it should recognise the suitability of the operator’s EBT programme (AMC1 ORO.FC.231(a)). AMC1 ARO.OPS.226(c) This provision was introduced to guide the competent authority on the main characteristics of an EBT programme. This AMC may be used by the competent authority to develop checklists for audits. AMC2 ARO.OPS.226(c) Approval and oversight of evidence-based training programmes EBT PROGRAMME SUITABILITY As regards the suitability of the EBT programme, please refer to AMC1 ORO.FC.231(a). AMC1 ARO.OPS.226(d) Approval and oversight of evidence-based training programmes OVERSIGHT PLAN — PERIODIC ASSESSMENT TO VERIFY COMPLIANCE OF THE EBT PROGRAMME (a) After issuing the approval of the operator’s EBT programme, the competent authority should have a process to verify the operator’s continuing compliance. (b) Each organisation to which an EBT approval has been issued should have an inspector (or inspectors) assigned to it who is (are) trained and qualified for EBT (see AMC1 ARO.OPS.226(a)). (c) Audits and inspections, on a scale and frequency appropriate to the operation, should cover at least: (1) management supervision of the EBT programme; (2) ongoing identification of operational risk and inclusion into the operator’s EBT programme; (3) relevance of the operator’s EBT programme to address its operational and training needs; (4) effectiveness of the operator’s EBT programme to improve pilot competencies. When there is an ineffective programme, the competent authority should examine the operator processes which identify the lack of effective results; (5) compliance with all requirements of ORO.FC.231; (6) delivery of instructor training in accordance with AMC1 ORO.FC.146(c), including inspections of the training delivery; (7) conduct of assessments of competence for EBT instructors, including periodic inspections of FSTD training; (8) maintenance of crew records; (9) administration of programme enrolment and compliance with the requirements of Annex I (Part-FCL) for licence revalidation and renewal; Page 34 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (10) continuing standardisation of EBT instructors; and (11) inspection of the training delivery. AMC1 ARO.OPS.226(d) The list was derived by a read-through of ORO.FC.231 and all ARO.GEN, ARO.OPS requirements for approval and oversight. This list may be supported by a checklist similar to that developed by EASA for mixed EBT implementation. AMC1 ARO.OPS.226(d) points (b) and (c) The intent of the RMG when drafting points (b) and (c) was to ensure that the competent authority has sufficiently qualified inspectors to oversee the EBT programme. The RMG also provided the items that should be reviewed in the periodic oversight plan so appropriate resources are planned. AMC1 ARO.OPS.226d(d) point (c)(1) This means that the operator should demonstrate to the competent authority that it has a method to collect, analyse and act upon the data from the EBT programme. It is expected that this would normally be discussed in regular meetings (the training standards meetings or similar format). Minutes of the meetings should be kept. In the training standards meetings, the operator would review the data and revise the programme as necessary. This is also provided in ORO.GEN.200(a)(5) of the Air OPS Regulation. AMC1 ARO.OPS.226(d) point (c)(3) wording ‘relevance of the operator’s EBT programme’ Relevance means that an EBT programme both includes the features contained in AMC1 ORO.FC.231(a) and continuously identifies the operator’s operational risks to feed the operator’s EBT programme. There was a discussion in EASA and the RMG, whether clarifying ‘EBT effectiveness’ and ‘EBT relevance’ was necessary. These are important elements of the EBT programme (verifying performance output). AMC1 ARO.OPS.226(d) point (c)(8) The competent authority should verify compliance with the provision of record-keeping under ORO.GEN.220 and ORO.MLR.115. Data collection and record-keeping are a key part of the EBT system. AMC1 ARO.OPS.226(d) point (c)(10) wording ‘continuing standardisation of EBT instructors’ This provision includes training and concordance assurance of the instructor. Verifying concordance should be preferably a data-driven process. SPT.012 ARO.OPS.226(d) — safety promotion task 012 — safety material for EBT — EBT INSTRUCTOR STANDARDISATION CONTINUING STANDARDISATION OF EBT INSTRUCTORS Generally speaking, a good standardisation of the EBT instructors is normally based on three main areas: Page 35 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (a) Training (b) Concordance assurance programme. The programme should be functional. In practical terms, this may include the identification from a data point of view of the four types of instructors that may require standardisation: the instructors that grade very high, the instructors that grade very low, the instructors that grade always the standard (e.g. 3), and the instructors whose grading is either very high or very low and with hardly any standard grades. (c) Guidance of the operator on how to grade GM1 to AMC1 ARO.OPS.226(d) Approval and oversight of evidence-based training programmes EFFECTIVENESS OF THE OPERATOR’S EBT PROGRAMME (a) The effectiveness of the operator’s EBT programme can be determined by periodically reviewing pilot competencies across several domains, such as role, fleet (e.g. CPT/FO, A320, B737) and airline so that the continuing improvement of the EBT programme is linked to an improvement of the pilot competencies. (b) The analysis of the pilot competencies across the domains should also take into account the operator’s experience in the EBT programme and the level of difficulty contained within the scenario elements of the programme, which may result in variations of the grading results and those variations may be acceptable. GM1 to AMC1 ARO.OPS.226(d) point (b) This point is introduced to guide the competent authority to verify the results of the competencies. These grading results may have variations, and those variations are acceptable. These variations occur for several reasons, for example, due to variations in the difficulty of the EBT programme. Therefore, the effectiveness of the EBT programme should be considered from a holistic view. For instance, a temporary decrease of pilot grading in core competencies does not necessarily mean a lack of effectiveness. Operators designing modules with numerous difficult events could end up in a decrease in the grading results of some competencies and vice versa. GM2 to AMC1 ARO.OPS.226(d) Approval and oversight of evidence-based training programmes STANDARDISATION OF EBT INSTRUCTORS — ACCEPTABLE INSTRUCTOR CONCORDANCE The authority may require a minimum acceptable level of concordance. This may be a non-exaustive list: (a) Set a minimum acceptable level of concordance per aircraft fleet or by group of instructors. (b) Set a minimum acceptable level of concordance per competency. (c) Set a minimum acceptable level of concordance for all operators under its overisight, or a minimum acceptable level of concordance per operator (or type of operator) based on the risk of the operator. Page 36 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Annex III (Part-ORO) to Regulation (EU) No 965/2012 GM1 ORO.GEN.130(b) Changes related to an AOC holder CHANGES REQUIRING PRIOR APPROVAL The following list GM is a non-exhaustive checklist of items that require prior approval from the competent authority as specified in the applicable Implementing Rules: (a) alternative means of compliance and special conditions; (b) procedures regarding items to be notified to the competent authority; (…) AMC1 ORO.FC.115 Crew resource management (CRM) training CRM TRAINING — MULTI-PILOT OPERATIONS (a) General (1) Training environment CRM training should be conducted in the non-operational environment (classroom and computer-based) and in the operational environment (flight simulation training device (FSTD) including alternative training devices described in CS-FSTD and aircraft). Tools such as group discussions, team task analysis, team task simulation and feedback should be used. (2) Classroom training Whenever possible, classroom training should be conducted in a group session away from the pressures of the usual working environment, so that the opportunity is provided for flight crew members to interact and communicate in an environment conducive to learning. (3) Computer-based training (CBT) Computer-based training should not be conducted as a stand-alone training method, but may be conducted as a complementary training method. Note to complementary training method: advanced CBT such as virtual reality, chatbots, interactive scenario trainers, etc. may serve as the principal method to deliver training on a CRM element. Classroom training or training in the operational environment may be the complementary method. (…) ORO.FC.146 Personnel providing training, checking and assessment SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib AMC1 ORO.FC.146(c) Personnel providing training, checking and assessment EBT INSTRUCTOR — INITIAL STANDARDISATION PROGRAMME (a) Before delivering the operator’s EBT programme, the instructor should complete an EBT Page 37 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail instructor initial standardisation programme composed of: (1) EBT instructor training; and (2) EBT assessment of competence. EBT INSTRUCTOR TRAINING (b) The EBT instructor training course should be delivered by at least one pilot who is or has been an EBT instructor, and who has demonstrated proficiency to train the elements specified in point (c) below. (c) The EBT instructor training course should comprise theoretical and practical training. At the completion of EBT instructor training, the instructor should: (1) have knowledge of EBT, including the following underlying principles: (i) competency-based training; (ii) learning from positive performance; (iii) building resilience; and (iv) data-driven training; (2) demonstrate knowledge of the structure of an EBT module; (3) demonstrate knowledge of the method of training delivery for each phase of an EBT module; (4) demonstrate knowledge of the principles of adult learning and how they relate to EBT; (5) conduct objective observations based on a competency framework, and document evidence of observed performance; (6) relate specific performance observations of competencies; (7) analyse trainee performance to determine competency-based training needs and recognise strengths; (8) evaluate performance using a competency-based grading system; (9) apply appropriate teaching styles during simulator training to accommodate trainee learning needs; (10) facilitate trainee learning, focusing on specific competency-based training needs; and (11) conduct a debrief using facilitation techniques. (d) An instructor may be given credits for parts of point (c) if the instructor has previously demonstrated competencies in those topics. EBT ASSESSMENT OF COMPETENCE (e) Prior to conducting assessment and training within an EBT programme, the EBT instructor should complete an EBT assessment of competence where the EBT instructor delivers: (1) an evaluation phase and a manoeuvres training phase; or Page 38 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (2) a scenario-based training phase. (f) The assessment of competence has a validity period of 3 years counted from the end of the month the assessment of competence was conducted. (g) The EBT assessment of competence should be conducted by a person nominated by the operator, who: (1) is qualified in accordance with Annex I (Part-FCL) to Regulation (EU) No 1178/2011 to conduct an assessment of competence; and (2) has completed the EBT instructor standardisation. (h) The EBT assessment of competence may be combined with the assessment of competence required in Annex I (Part-FCL) to Regulation (EU) No 1178/2011. AMC2 ORO.FC.146(c) Personnel providing training, checking and assessment EBT INSTRUCTOR — RECURRENT STANDARDISATION PROGRAMME The EBT instructor should: (a) conduct an EVAL or SBT phase of an EBT module every 12 months. The 12-month period should be counted from the end of the month the module was taken. If this has not been fulfilled, the EBT instructor should complete an EBT assessment of competence. When the module is undertaken within the last 3 months of the period, the new period should be counted from the original expiry date; (b) receive annual recurrent standardisation. The recurrent standardisation should include: (1) refresher EBT training; and (2) concordance training; and (c) complete an assessment of competence every 3 years. When the assessment of competence is conducted within the 12 months preceding the expiry date, the next assessment of competence should be completed within 36 calendar months of the original expiry date of the previous assessment of competence. AMC1 ORO.FC.146(c) point (a) The review group decided to introduce this provision for the following reasons: The requirement proposed in point (a) is similar to the existing requirement that must be fulfilled by the examiners. As the EBT instructor is performing ‘assessments’, there was a consensus in the group to align both requirements due to the social implications. According to point (b)(1) of FCL.1025, the examiner must conduct two proficiency checks every year. As a module is equivalent to a proficiency check, the EBT instructor is required to follow the same approach. Although legacy instructors are required to revalidate only one session in the preceding the 12 months, the review group believes that this approach is incorrect due to the challenges associated Page 39 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail with EBT. Additionally the review group believes that more training of the instructors should improve safety. Finally, the EBT instructor is required to perform a refresher training every year. Therefore, it is reasonable to complete one EBT module every year to ensure practical training (on-the-job training). AMC1 ORO.FC.146(c) point (c) The syllabus for the EBT instructor training course has been taken from Doc 9995 and the IATA/ICAO/IFALPA Evidence-Based Training Implementation Guide. The volume of training of the EBT instructor initial standardisation course is addressed in the new GM1 ORO.FC.146(c). AMC1 ORO.FC.146(c) point (b) Point (b) provides the prerequisite for the pilot who delivers the EBT instructor training (ground course). The only prerequisite is that this pilot has completed the EBT instructor training. This pilot does not need to be a qualified instructor under Part-FCL of the Aircrew Regulation. This requirement is less stringent than the one for the person who delivers the assessment of competence (see requirement AMC1 ORO.FC.146(c) point (g)). The person who delivers the assessment of competence needs to receive an EBT instructor training and be a qualified examiner in accordance with Part-FCL of the Aircrew Regulation. AMC1 ORO.FC.146(c) wording ‘EBT assessment of competence’ The wording ‘assessment of competence’ is used for consistency purposes between Part-ORO of the Air OPS Regulation and Part-FCL of the Aircrew Regulation and to allow a combined assessment of the revalidation of the EBT instructor in Part-ORO and the revalidation of instructor’s certificate in accordance with FCL.935. AMC1 ORO.FC.146 (c) point (e) wording ‘complete an EBT assessment of competence’ versus ‘passed an EBT assessment of competence’ The word ‘passed’ is usually used in Part-FCL — for example, in FCL.905.FI.FI, FCL.915 and FCL.940.SFI.SFI. On the contrary, in Part-ORO, ‘complete’ is the one usually used — for example in ORO.FC.105, ORO.FC.120, and ORO.FC.130. As the rules on EBT are contained in Part-ORO, EASA decided to use ‘complete’. The assessment of competence was introduced as an AMC to be consistent with the CRM provision, for which also the assessment of competence is at AMC level. Furthermore, in order to ensure an equivalent level of safety in the case of an application of an alternative means of compliance (AltMoC), ORO.GEN.200 ensures the competence of the personnel. AMC1 ORO.FC.146(c) point (f) ‘validity period of 3 years’ This provision is transposed from the Aircrew Regulation as regards what applies in relation to instructors’ and examiners’ validity period. Furthermore, this provision is also included Doc 9995 paragraph 6.3.5: ‘All instructors should receive annual refresher training, and be re-assessed in the competencies specified in 6.3.3 every three years.’ Page 40 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The 12-month transition period to complete the assessment of competence is transposed from FCL.940.TRI TRI — ‘Revalidation and renewal’ of Regulation (EU) No 1178/2011. AMC1 ORO.FC.146(c) point (g)(1) The requirement proposed was originally transposed from the existing Doc 9995 and the Explanatory Note to ED Decision 2015/027/R on mixed EBT. ‘A person nominated (refer to ICAO Doc 9995 AN/497 ‘Manual of Evidence-based training’ first edition 2013 paragraph 6.3.4) by the operator for the conduct of competency assessments of EBT examiners and instructors should be a person who holds a certificate equivalent to that being assessed, provided that he or she has completed the training and assessment indicated in ICAO Doc 9995 paragraph 6.3’. That’s why point (g) of this AMC uses ‘conducted by a person nominated by the operator.’ However, to be consistent with Part-FCL of the Aircrew Regulation and ORO.FC.146(c), a further clarification was introduced as the ICAO provision does not provide details. Therefore, the Opinion requires an assessment of competence performed in the FSTD before receiving the EBT instructor qualification. To maintain legal consistency between the assessment of competence referred to in Part- FCL and the EBT assessment of competence proposed in this provision (OPS), EASA decided that only personnel holding a certificate with privileges to perform assessment of competence are allowed to perform such ‘test’. When revalidating an instructor or examiner licence in accordance with the Aircrew Regulation, the provision is already there. Therefore, this requirement does not add an extra burden to the operators. The only exception to such statement (no extra burden to the operators) would be during the transition period from legacy training to EBT, where the instructor/examiner revalidation may not match with the assessment of competence of EBT required during the initial EBT course. After the transition phase, the operator will roster the instructor/examiner revalidation in combination with an EBT assessment of competence when required. AMC1 ORO.FC.146(c) point (g)(1) wording ‘is qualified in accordance with Annex I (Part-FCL) to Regulation (EU) No 1178/2011 to conduct an assessment of competence’ This provision is introduced to ensure an examiner will perform the EBT assessment of competence. This provision is already required in the assessment of competence for the instructor in Subpart J of Part-FCL of the Aircrew Regulation. The intention of the RMG is that the assessment of competence for EBT and the assessment of competence to revalidate instructor certificate will be combined, thus this provision should not add any further requirement or cost. There was a discussion in the RMG whether this person should be a current examiner or not. If not, then the word ‘held’ could be used in the provision. The proposal to allow NON-current examiners would allow more flexibility. This is important when the EBT is introduced for the first time in the airline. This option is deviating from the concept of instructor course in Part-FCL of the Aircrew Regulation. EASA expects that most of the courses for EBT will be combined with Part-FCL instructor courses. Therefore, at the end, the assessment of competence is expected to be performed by a current qualified examiner anyway. Thus, the option of NON-current examiner was discarded and in order to Page 41 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail be consistent between Part-FCL and Part-ORO, EASA decided that only current examiners will conduct the EBT assessment of competence. AMC2 ORO.FC.146(c) The wording used is based on the requirements pertaining to the revalidation for instructors and examiners in accordance with Subparts J and K of Part-FCL of the Aircrew Regulation. The example is provided below: ‘FCL.940.TRI TRI — Revalidation and renewal (a) Revalidation (1) Aeroplanes. For revalidation of a TRI(A) certificate, the applicant shall, within the last 12 months preceding the expiry date of the certificate, fulfil one of the following 3 requirements: (i) conduct one of the following parts of a complete type rating training course: simulator session of at least 3 hours or one air exercise of at least 1 hour comprising a minimum of 2 take-offs and landings; (ii) receive instructor refresher training as a TRI at an ATO; (iii) pass the assessment of competence in accordance with FCL.935.’ Following the concept above, the revalidation for EBT instructor will be based on the completion of one full simulator session of EBT every 12 months; for that reason the AMC requires an EVAL or SBT. EASA is aware that this would mean in reality the combination of evaluation plus manoeuvres training/validation (mixed EBT) and a scenario-based training. Additionally, an assessment of competence every 3 years is required. Point (b) of AMC2 ORO.FC.146(c)) provides the requirement for EBT refresher training. This training may satisfy the requirement of FCL.940.TRI(a)(1)(ii) concerning instructor refresher training, if accepted by the competent authority. This requirement is proposed to ensure standardisation of the instructors. Furthermore, the need for concordance assurance was introduced considering Doc 9995 Attachment to Chapter 1 step 9. 9** Instructor training and 4.1.1 and Instructor EBT programme standardisation, which should be a standardization. formalized approach to ensure a consistent and standardised 6.3 of Part I approach to the EBT programme prior to implementation, including practical training reinforcing application of the assessment and grading system and maximising inter-rater reliability. GM1 ORO.FC.146(c) Personnel providing training, checking and assessment EBT INSTRUCTOR — INITIAL STANDARDISATION (a) The intent of the practical training is to ensure that EBT instructors have exposure to assessment of performance and root cause identification within an EBT programme. (b) EBT instructors receive practical assistance and guidance during standardisation in order to apply the learning from EBT instructor training. In particular, the focus should be on assessment Page 42 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail of performance and the determination of root cause for remediation, plus facilitated debriefing based on root cause as a learning objective. (c) The pilot delivering the training may be supported by a subject matter expert (or experts). The personnel providing the EBT training is selected by the operator to assess the instructor capability in delivering EBT and provide effective feedback in order that instructor practice meets the expectations of the operator. (d) Practical EBT training includes the learning objective ‘Evaluate performance using a competency-based grading system’. This may be done with videos and other multimedia. It means that EBT instructors are exposed to: (1) different levels of pilot performance. This enables EBT instructors to distinguish between pilots performing lower than the minimum acceptable level of performance (e.g. grade 1) and those whose performance is at an acceptable level in all competencies (e.g. grade 2). This EBT training may also include other performance examples (e.g. 3, 4 and 5); and (2) different scenarios (e.g. complex to less complex) so that the instructor has exposure to assessments of competency in varying EBT scenarios. (e) The EBT instructor training course may be a minimum of 14 hours (EBT instructor training alone) and the recommended length is between 21 to 24 hours (EBT instructor training plus assessment of competence). GM1 ORO.FC.146(c) EBT programme The transition to EBT involves a paradigm shift in the focus of training. To maximise the safety benefits of the programme, EBT instructors should be mentored to ensure practice develops according to the expectations of the operator. EBT instructors delivering a standardisation course should be carefully selected and trained so that the standardisation activity provides the maximum benefit. The syllabus described in the AMC1 ORO.FC.146(c) provides enough granularity for a performance- based rule. Therefore, the RMG decided not to include a prescriptive requirement regarding the number of hours needed to deliver the EBT instructor course; instead, the information is provided in point (e) of this GM. However, the RMG acknowledged the novelty of the EBT programme where the instructor training course is a fundamental piece. Therefore, it was decided to provide GM that would include some references regarding the length of the EBT instructor course in order to promote a successful discussion between the operator and the competent authority where the focus is on the outcome of the course (provided at the level of the AMC) rather than on the prescription of 14/21/24 hours (provided at the level of GM). GM1 ORO.FC.146(c) point (c) The consultation of the NPA showed that the GM needed further amendment to clarify the additional personnel that can deliver the EBT instructor training. Other subject matter experts (SMEs) (e.g. aviation experts, psychologists, teachers, other industries experts, etc.) can provide valuable resources to enrich the instructor training. GM1 ORO.FC.146(c) point (e) Page 43 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Given the paradigm shift in the philosophy of assessment and training of competencies, a nominal value of course duration was included as a standardisation element. In order to agree on a figure, the RMG reviewed the rules relating to qualification of instructors (e.g. FCL.930 TRI.TRI was consulted). The 14 hours were commensurate with those required for initial qualification of instructors. Furthermore, the IATA Evidence-Based Training Implementation Guide recommends at least a 3-day course in Appendix B. However, this appendix provides a range of duration for the course between 3 days and 5 days. GM2 ORO.FC.146(c) Personnel providing training, checking and assessment EBT INSTRUCTOR — RECURRENT STANDARDISATION (a) Refresher EBT training The intent of this training is to provide the framework for existing instructors to develop their competence to conduct EBT. Further guidance can be found in the EASA EBT manual. (b) Concordance training This training is one of the elements to ensure concordance within the EBT instructor community. Those EBT instructors who do not demonstrate concordance may require further training. The operator’s instructor standardisation and concordance assurance programme provides insight in the areas that an instructor (or instructor population) requires concordance training. As such, concordance training varies in content and scale depending on the need for concordance improvement. Instructor concordance training may include candidates grading the same controlled content (e.g. a video or paper case) followed by: (1) a subsequent comparison of intra-group variance; and (2) alignment of root-cause analyses between instructors. GM2 ORO.FC.146(c) This GM for the annual EBT instructor standardisation has been developed to clarify the intent of the provision provided in the AMC. The GM proposed provides certain criteria on how to perform the annual instructor standardisation; however, the criteria that may be provided by the competent authority are fundamental, as training is subject to approval under OM part D and revalidations and renewal of licences are performed within an EBT programme. The authority should exercise its oversight powers to ensure operators provide the right amount of training and concordance assurance to their instructors. GM2 ORO.FC.146(c) point (b) This provision was introduced following the IATA Evidence-Based Training Implementation Guide2 Chapter 4.1 ‘The EBT instructor’. 2 IATA Evidence-Based Training Implementation Guide July 2013 1st Edition. Page 44 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ‘The development of strong inter-rater reliability and consistency in the approach to EBT is of great importance and should not be underestimated either initially or as a focus for the continuous improvement of an EBT system. Establishing robust guidelines and thorough experience strengthens inter-rater reliability, provided that suitable mechanisms are put in place. Clear and concise instructions, accurate performance indicator descriptions and peer review all increase inter-rater reliability.’ GM3 ORO.FC.146(c) Personnel providing training, checking and assessment EBT INSTRUCTOR COMPETENCY FRAMEWORK Pilot competencies1 Description: See pilot competency framework Instructor See pilot competency framework observable behaviour (iOB) 1 For ground instructors, some competencies may not apply. Management of the learning environment Description: Ensures that the instruction, assessment and evaluation are conducted in a suitable and safe environment iOB 2.1 Applies TEM in the context of instruction/evaluation iOB 2.2 Briefs on safety procedures for situations that are likely to develop during instruction/evaluation iOB 2.3 Intervenes appropriately, at the correct time and level (e.g. progresses from verbal assistance to taking over control) iOB 2.4 Resumes instruction/evaluation as practicable after any intervention iOB 2.5 Plans and prepares training media, equipment and resources iOB 2.6 Briefs on training devices or aircraft limitations that may influence training, when applicable iOB 2.7 Creates and manages conditions (e.g. airspace, ATC, weather, time, etc.) to be suitable for the training objectives iOB 2.8 Adapts to changes in the environment whilst minimising training disruptions iOB 2.9 Manages time, training media and equipment to ensure that training objectives are met Page 45 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Instruction Description: Conducts training to develop the trainee’s competencies iOB 3.1 References approved sources (operations, technical and training manuals, standards and regulations) iOB 3.2 States clearly the objectives and clarifies roles for the training iOB 3.3 Follows the approved training programme iOB 3.4 Applies instructional methods as appropriate (e.g. explanation, demonstration, learning by discovery, facilitation, in-seat instruction) iOB 3.5 Sustains operational relevance and realism iOB 3.6 Adapts the amount of instructor inputs to ensure that the training objectives are met iOB 3.7 Adapts to situations that might disrupt a planned sequence of events iOB 3.8 Continuously assesses the trainee’s competencies iOB 3.9 Encourages the trainee to self-assess iOB 3.10 Allows the trainee to self-correct in a timely manner iOB 3.11 Applies trainee-centred feedback techniques (e.g. facilitation, etc.) iOB 3.12 Provides positive reinforcement Interaction with the trainees Description: Supports the trainees’ learning and development and demonstrates exemplary behaviour (role model) iOB 4.1 Shows patience and empathy (e.g. by actively listening, reading non-verbal messages and encouraging dialogue) iOB 4.2 Manages trainees’ barriers to learning iOB 4.3 Follows the approved training programme iOB 4.4 Encourages engagement and mutual support iOB 4.5 Coaches the trainees iOB 4.6 Supports the goal and training policies of the operator/ATO and authority iOB 4.7 Shows integrity (e.g. honesty and professional principles) iOB 4.8 Demonstrates acceptable personal conduct, acceptable social practices, content expertise, a model for professional and interpersonal behaviour Page 46 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail iOB 4.9 Actively seeks and accepts feedback to improve own performance Assessment and evaluation Description: Assesses the competencies of the trainee and contributes to continuous training system improvement iOB 5.1 Complies with operator/ATO and authority requirements iOB 5.2 Ensures that the trainee understands the assessment process iOB 5.3 Applies the competency standards and conditions iOB 5.4 Performs grading iOB 5.5 Provides recommendations based on the outcome of the assessment iOB 5.6 Makes decisions based on the outcome of the summative assessment iOB 5.7 Provides clear feedback to the trainee iOB 5.8 Reports strengths and weaknesses of the training system (e.g. training environment, curriculum, assessment/evaluation) including feedback from trainees iOB 5.9 Suggests improvements for the training system iOB 5.10 Produces reports using appropriate forms and media Competency assessment Final Operators and ATOs define in their OMs the level of performance to be achieved by the competency instructor and evaluator standard Condition Ground training (including CRM) and flight training in aircraft and in FSTDs: — licensing; — type rating; — conversion; — line training; and — recurrent training. GM3 ORO.FC.146(c) This GM introduces the ICAO Pilot Instructor and Evaluator competency Framework, which is based on the work led by IATA to promote the first competency-based approach for instructors and evaluators. Page 47 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The original idea to design an instructor evaluator competency set is based on the same philosophy that served as the genesis for the pilot competency set: Mastering a defined set of pilot competencies should enable a pilot to perform the routine duties and manage unforeseen situations, which cannot be trained in advance. Similarly, mastering a set of instructor and evaluator competencies (IECs) should enable an instructor and evaluator (IE) to perform instruction and evaluation duties and manage the full spectrum of assignments, from ground instruction to evaluations in dynamic flight situations. The competency framework for instructors and evaluators has been developed based on the latest ICAO standards, EU and FAA regulations, and guidance material and best practices from the industry. The defined set of IE competencies should be applied across all types of training, from licensing to operator recurrent training, and by both operators and ATOs. Developing both pilot and instructor competencies through a globally harmonised system of competencies will contribute to improved quality of training, enhanced safety and will also increase training efficiency. This GM defines the IE competencies, their descriptions and their observable behaviours. The competency framework may be used for instructor selection, initial standardisation, recurrent standardisation and assessment of competence for EBT instructors. Additional information can be found through the following link: https://www.iata.org/whatwedo/ops- infra/training-licensing/Documents/guidance_material_for_instructor_and_evaluator_training.pdf SPT.012 ORO.FC.146(c) — safety promotion task 012 — safety material for EBT — EBT INSTRUCTOR COMPETENCY FRAMEWORK EBT INSTRUCTORS STANDARDISATION PROGRAMME DESIGN This safety promotion material describes a competency-based approach to the EBT instructors training using the five instructor and evaluator competencies (IECs). A development program for IEs should use a building block approach. The aim is to progress in a structured way, step-by-step, from the initial assignment through the complete spectrum of IE duties. For any IE assignment, an IE needs to be trained and assessed in all competencies to a solid foundational level of performance. However, specific assignments require special emphasis on specific competencies during training; the final competency standard for theses competencies should be higher than foundational. As a consequence, given that the EBT instructor is already qualified in accordance with Annex I (Part- FCL), the operator EBT standardisation should put special emphasis on the competencies ‘Instruction’, ‘Interaction with the trainees’ and ‘Assessment and evaluation’. Training objectives for EBT instructor standardisation will consequently refer to the descriptions of the relevant IECs and their OBs. The table below show a simplified matrix to train and assess (‘TA’) IEs. Depending on the IE’s assignment, the competencies requiring special emphasis during training are additionally identified with ‘SE’. Page 48 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail CBTA matrix for EBT instructor EBT instructor competencies IEC1 IEC2 IEC3 IEC4 IEC5 Management of Pilot Interaction with Assessment and the learning Instruction competencies the trainees evaluation environment TA TA TA-SE TA-SE TA-SE Note: TA: competencies trained and assessed SE: competencies requiring special emphasis during training ORO.FC.230 Recurrent training and checking GM1 ORO.FC.230(a);(b);(f) Recurrent training and checking MIXED EVIDENCE-BASED RECURRENT TRAINING AND CHECKING OF FLIGHT CREW CONDUCTED IN FLIGHT SIMULATION TRAINING DEVICES (FSTDs) ICAO developed Doc 9995 ‘Manual of Evidence-based Training’, followed by the EASA EBT manual, which is intended to provide guidance to the competent civil aviation authorities, operators and approved training organisations in the recurrent assessment and training of pilots by establishing a new methodology for the development and conduct of a recurrent assessment and training and assessment programme, titled evidence-based training (EBT). ‘Evidence-based Training(EBT)’ means training and assessment based on operational data that is characterised by developing and assessing the overall capability of a trainee across a range of core competencies rather than by measuring the performance during individual events or manoeuvres. ICAO Doc 9995 and the EASA EBT manual are is the reference documents for operators seeking to implement EBT. The purpose of this guidance material (GM) is to enable the implementation of a mixed EBT according to the principles established in ICAO Doc 9995 taking into account the European regulatory framework. In the current regulatory framework, it is possible to achieve a mixed EBT implementation of EBT. Implementation of a mixed EBT programme means that some portion of the recurrent assessment and training is dedicated to the application of EBT. This includes the Licence Proficiency Check (LPC) and the Operator Proficiency Check (OPC). As it is possible to combine LPC and OPC in ORO.FC, this GM is applicable to both checks. Therefore, the EBT training programme described in this GM refers to the recurrent training and checking of flight crew, including LPCs and OPCs. The EBT training programme takes into account the differences between aircraft of different generations and the effect of these differences on training. The operator should acquire a thorough knowledge of ICAO Doc 9995 or the EASA EBT manual before implementing this GM. For applicability, see ICAO Doc 9995 Chapter 3 or the EASA tables of applicable aeroplane/helicopter types by generation. Mixed EBT programme Page 49 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Within the current regulatory framework tThe operator may undertake an mixed implementation of the mixed EBT programme according to this GM. The ICAO table of assessment and training topics is defined in ICAO Doc 9995 Chapter 4.3.1 and in Appendices 2 to 7; the EASA EBT programme is defined in AMC2 to AMC7 to ORO.FC.231(a). The baseline mixed EBT programme provides the flexibility to adapt programmes according to specific operator risks. Elements of the enhanced EBT programme may be implemented according to the definition and process described in ICAO Doc 9995 Chapter 5. The operator should contact the competent authority in order for them to assess the application of the process described in ICAO Doc 9995 including, where applicable, the results from data analyses to support the enhanced EBT programme. Personnel providing training and checking in EBT (Refers to AMC1 ORO.FC.230(d)) ICAO Doc 9995 Chapter 6, or EASA AMC1 and AMC2 to ORO.FC.146(c), which is additional to EU regulations, contains the guidance for the assessment and training and assessment of personnel involved in the conduct of EBT. Equivalency of malfunctions/Malfunction clustering (Refers to ICAO Doc 9995 Paragraph 3.8.3) According to the concept of EASA and ICAO Doc 9995 Chapter 3.8.3, major failures reduce the capability of the aircraft or the ability of the crew to cope with operating conditions to the extent that there would be a significant reduction in functional capabilities, significant increase in crew workload or in conditions impairing crew efficiency. Clusters of major failures of aircraft systems are determined by reference to malfunction characteristics and the underlying elements of crew performance required to manage them. Malfunction clustering Equivalency of malfunctions may be used to guide the operator towards the implementation of an a mixed EBT programme according to AMC1 ORO.FC.230(a)(4)(i)(A) and ORO.FC.145(d). Conduct of Licence and Operator Proficiency Checks The EASA EBT programme described in ORO.FC.231 and the ICAO EBT programme described in ICAO Doc 9995 contains modules with three phases: the evaluation phase, the manoeuvres training phase, and the scenario-based training phase. In order to comply with the existing regulatory framework, in the mixed EBT programme the LPC and OPC requirements are fulfilled by a combination of the evaluation phase and the manoeuvres validation phase, which replaces the manoeuvres training phase described in the EASA EBT programme or ICAO Doc 9995. The manoeuvres validation phase is defined in Section 3 below. This is a form of mixed EBT implementation, which is described as follows: 1. Evaluation phase: This includes check scenarios referred to in Part-FCL Appendix 9 within an accepted approved mixed EBT programme. In order to facilitate the provision of simple and realistic scenarios in accordance with ICAO Doc 9995 Chapters 3.8 and 7.4, the evaluation phase is not intended to be a comprehensive assessment of all Part-FCL Appendix 9 items; nevertheless, the list below includes the items that should be included in the evaluation phase only. Page 50 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Part-FCL or Description Part-ORO reference A H The examiner may choose between different skill test or proficiency check scenarios containing simulated relevant operations developed and E E approved by the competent authority. Full-flight simulators and other R L training devices, when available, shall be used, as established in this Part. O I P C Part-FCL Appendix 9 L O Paragraph 6 A P N T E E S R S A The test/check should be accomplished under instrument flight rules Part-FCL Appendix 9 (IFRs), if instrument rating (IR) is included, and as far as possible be E Paragraph 16 of accomplished in a simulated commercial air transport environment. An R essential element to be checked is the ability to plan and conduct the section B O flight from routine briefing material. P Part-FCL Appendix 9 Use of checklist prior to starting engines, starting procedures, radio and L navigation equipment check, selection and setting of navigation and Item 1.4 communication frequencies. A N Part-FCL Appendix 9 Before take-off checks. E Item1.6 S Adherence to departure and arrival routes and ATC instructions. Part-FCL Appendix 9 The starred item (*) shall be flown solely by reference to instruments. If Item 3.9.1* this condition is not met during the skill test or proficiency check, the type rating will be restricted to VFR only. H 2. In case of proficiency check for an IR the applicant shall pass section 5 of the proficiency check. Failure in more than three items will require the E Part-FCL Appendix 9 applicant to take the entire section 5 again. An applicant failing not more L Paragraph 2 of than three items shall take the failed items again. Failure in any item of section C I the re-check or failure in any other items of section 5 already passed will require the applicant to take the entire check again. Page 51 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail C Part-FCL Appendix 9 Starting procedures, radio and navigation equipment check, selection and O Item 1.3. setting of navigation and communication frequencies P Part-FCL Appendix 9 Taxiing/air taxiing in compliance with air traffic control instructions or with T instructions of an instructor Item 1.4 E Part-FCL Appendix 9 R Pre-take-off procedures and checks Item 1.5 S Adherence to departure and arrival routes and ATC instructions Part-FCL Appendix 9 The starred item (*) shall be flown solely by reference to instruments. If this Item 5.2* condition is not met during the skill test or proficiency check, the type rating will be restricted to VFR only. 2. Manoeuvres validation phase: The purpose of the manoeuvres validation phase is to check the handling skills necessary to fly critical flight manoeuvres so that they are maintained to a defined level of proficiency. This replaces the manoeuvres training phase described in ICAO Doc 9995 Chapter 7.5 and ORO.FC.231(a)(2)(iii)(B). Manoeuvres in this context are not part of the line- orientated flight scenario; they are a sequence of deliberate actions to achieve a prescribed flight path or to perform a prescribed event to a prescribed outcome. All remaining items listed in Part-FCL Appendix 9, and not included in the evaluation phase, should be included here. The manoeuvres listed in Doc 9995 or the EASA table of assessment and training topics for the manoeuvres training phase that do not form part of the Part-FCL Appendix 9 mandatory items may be trained after the manoeuvres validation phase. 3. Scenario-based training phase: The purpose of the scenario-based training phase is to further develop pilot core competencies in a learning environment. This does not form part of any LPC or OPC requirement. It should be noted that if the operator is following an alternative means of compliance to ORO.FC.230 (b) Operator Proficiency Check, the equivalence of using EBT evaluation and manoeuvres validation phases may no longer exist. Conduct of CRM assessment The operator is advised to use the EBT grading system and the EBT competencies for the non-technical skills assessment. Additional guidance on mixed EBT implementation is available in the EASA checklist ‘Oversight guidance for transition to Mixed EBT Implementation’. ORO.FC.231 Evidence-based training (a) EBT PROGRAMME SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib Page 52 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.231.1 EBT programme EVALUATION AND TRAINING- PHASES OF THE EBT MODULE (a) The evaluation phase comprises a line-orientated flight scenario (or scenarios) to assess competencies and identify individual training needs. (b) The training phase comprises: (1) the manoeuvres training phase, comprising training to proficiency in certain defined manoeuvres; and (2) the scenario-based training phase, comprising a line-orientated flight scenario (o scenarios) to develop competencies and address individual training needs. (c) The training phase shall be conducted timely after the evaluation phase. (d) The contingency procedures for unforeseen circumstances that may affect the delivery of the module may include a different order of the phases of the module. (e) The validity period of a module should be 12 months. The validity period should be counted from the end of the month when the module was undertaken. When the module is undertaken within the last 6 months of the validity period, the new validity period should be counted from the original expiry date provided a minimum of 2 modules are completed within the validity period of the type rating. CS OPS.EBT.231.1 point (a) The evaluation phase is a first look to assess competencies, determined training system effectiveness and identify individual training needs. On completion of the evaluation phase, any areas that do not meet the minimum competency standards will become the focus of the subsequent training. ORO.FC.231 point (c) ‘(C) The training phase shall be conducted timely after the evaluation phase’ The intent of this provision is to clarify the need to perform the training phase after the evaluation phase. In addition, the word ‘timely’ is introduced to stress the need to define a period in which the training will be provided. AMC1 ORO.FC.231(a) Evidence-based training EBT PROGRAMME SUITABILITY An operator’s EBT programme is one in which: (a) training is focused on development of competencies, rather than repetition of tasks; (b) the development of the programme is based on data-driven EBT training topics with a link to the operator’s competency framework; (c) training needs are addressed through training based on underlying competencies; (d) the programme includes: (1) an evaluation phase to identify training needs based on competencies and collect population-based data; Page 53 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (2) a manoeuvres training phase (skill retention): to trained skill-based manoeuvres (body memory actions); and (3) a scenario-based training phase to focus on identified training needs based on competencies rather than repetition of tasks; (e) the programme includes the conduct of objective observations based on a competency framework, and documents evidence of the behaviour observed; (f) there is a customisation of syllabi: (1) The operator should describe in the operations manual the procedure to customise syllabi. It should include how to: (i) select the example scenario elements within a training topic that should be included in the EBT programme; and (ii) contextualise the example scenario elements based on the operator’s operational data (e.g. input from SMS, FDM programme, etc.) and training data. (2) This customisation should be based on evidence both internal and external to the operator; (g) performance is evaluated using a competency-based grading system; (h) instructors grade competencies based on observable behaviours (OBs); (i) instructors grade the pilot using a defined methodology — observe, record, classify and assess/evaluate (ORCA) is recommended; (j) instructors have completed the EBT instructor standardisation; (k) instructors have sufficient concordance based on defined criteria (instructor concordance assurance programme); (l) the analysis of the pilot's performance is used to determine competency-based training needs; (m) there is a range of teaching styles during simulator training to accommodate trainee learning needs; and (n) facilitation techniques in debriefing are incorporated. AMC1 ORO.FC.231(f) General background about the wording ‘customisation’ The regulation usually uses ‘customised’, ‘customisation’ in the context of the operator’s EBT syllabi. The term expresses the necessity for the adaptation of the ‘table of assessment and training topics’ that must be performed at operator level. Amongst others, the EBT programme is adapted to the operational risks of the airline, the different type ratings of the operator, the pilot work force, etc. ‘Tailored’: It is referring to the further ‘customisation’ of syllabi that is performed at the level of an individual pilot. In order to make a difference between the customisation at operator level (syllabi) and the customisation at individual pilot (individual syllabus), the regulation uses the word ‘tailored’, using mainly the wording ‘tailored training’. Tailored training is required in ORO.FC.231(d) (see the the Page 54 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail related AMC) and further described in GM3 ORO.FC.231(a) customisation of the EBT programme (Syllabi). ‘Contextualise’: The verb ‘contextualise’ is used for the example scenario elements, where the operator should provide the ‘context’ of the example scenario elements provided in the ‘table of assessment and training topics’. Amongst others, weather of the example scenario element, area, route or aerodrome,, procedures at the aerodrome (e.g. low-visibility procedures (LVP)), etc. AMC1 ORO.FC.231(a) point (j) EBT instructor standardisation refers to AMC1 ORO.FC.146(c). AMC1 ORO.FC.231(a) point (i) The wording refers back to AMC3 ORO.FC.231(d)(1) ‘CONDUCT OF THE GRADING — ORCA’, which is the preferred methodology proposed in this Appendix to the Opinion. AMC1 ORO.FC.231(a) point (k) Good concordance depends on the number of rating levels used, concordance measures and complexity/ambiguity of scenarios and behaviours. The acceptance level of concordance can be defined in coordination with experts and the NAA. It can be based on earlier results. AMC1 ORO.FC.231(a) point (n) Facilitation is a very important part of EBT and therefore the review group introduced this requirement. AMC2 ORO.FC.231(a) Evidence-based training UPSET PREVENTION AND RECOVERY TRAINING (UPRT) FOR COMPLEX MOTOR-POWERED AEROPLANES WITH A MAXIMUM OPERATIONAL PASSENGER SEATING CONFIGURATION (MOPSC) OF MORE THAN 19 Operators approved for EBT should follow the provisions for upset prevention and recovery training (UPRT) contained in AMC1 ORO.FC.220&230 ‘Operator conversion training and checking & recurrent training and checking’. AMC2 ORO.FC.231(a) Although this Opinion proposes to exempt operators implementing the EBT programme from ORO.FC.230 and its AMC, the UPRT provisions were reintroduced through this AMC2 ORO.FC.231(a) due to the importance of these provisions. Doc 9995 has not transposed the latest UPRT requirements of Doc 10011 AN/506 ‘Manual on aeroplanes upset prevention and recovery training’ first edition -2014. The first phase of RMT.0599 only addresses recurrent training and checking (ORO.FC.230); therefore, the requirements for the operator conversion course (ORO.FC.220) are not amended. AMC1 ORO.FC.220&230 is linked to both IRs ORO.FC.220 and ORO.FC.230; therefore, AMC2 ORO.FC.231(a) is just acknowledging the need to fulfil the UPRT provisions. Page 55 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Page 56 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail SPT.012 — Safety promotion to ORO.FC.231(a) EBT programme (UPRT) UPSET PREVENTION AND RECOVERY TRAINING (UPRT) FOR COMPLEX MOTOR-POWERED AEROPLANES WITH A MAXIMUM OPERATIONAL PASSENGER SEATING CONFIGURATION (MOPSC) OF MORE THAN 19 The purpose of this table is to assist the operator in cross-mapping the requirements of UPRT in AMC1 ORO.FC.220&230 and how this objective is achieved in ORO.FC.231 EBT programmes. The example table is a compilation of the tables proposed by two different operators to their authorities. AMC1 ORO.FC.220&230 Operator conversion training and checking & recurrent training and checking UPSET PREVENTION AND RECOVERY TRAINING (UPRT) FOR COMPLEX MOTOR-POWERED AEROPLANES WITH A MAXIMUM OPERATIONAL PASSENGER SEATING CONFIGURATION (MOPSC) OF MORE THAN 19 SEATS Current provision in AMC1 ORO.FC.220&230 Means of compliance in ORO.FC.232 – CS OPS.EBT.232 Rationale (a) Upset prevention training should: Ground training requirements are unchanged. FSTD requirements are Applicability for EBT is (1) consist of ground training and flight training in an FSTD or an aeroplane; included in the EBT programme determined by aircraft types and variants (2) include upset prevention elements from Table 1 for the conversion training course; and Does not apply to recurrent training and checking listed in ORO.FC.231 and only for those for (3) include upset prevention elements in Table 1 for the recurrent training programme at Equivalent to the ‘B’ level within the EBT Programme, all items to be which a suitably least every 12 calendar months, such that all the elements are covered over a period not completed within the 3-year programme and some elements of UPRT qualified FSTD is exceeding 3 years. to be included every year. available (b) Upset recovery training should: (1) consist of ground training and flight training in an FFS qualified for the training task; Included in the EBT programme as upset recovery. All exercises, but especially the ones in Table 2 RECOVERY FROM (2) be completed from each seat in which a pilot’s duties require him or her to operate; DEVELOPED UPSETS, shall not take place during the evaluation phase and and it is recommended that they should be done during the (3) include the recovery exercises in Table 2 for the recurrent training programme, such manoeuvres TRAINING. that all the exercises are covered over a period not exceeding 3 years. Page 57 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ORO.FC.232 – CS OPS.EBT.232 - Example – AMC1 ORO.FC.220&230 ‘Operator conversion training Ground FSTD Rationale suggested relevant EBT ‘training topic and and checking & recurrent training and checking’ training training description’ Table 1: Elements and respective components of upset prevention training A. Aerodynamics 1. General aerodynamic characteristics • 2. Aeroplane certification and limitations • 3. Aerodynamics (high and low altitudes) • • Element A is basically fully covered by the mentioned 4. Aeroplane performance (high and low altitudes) • • EBT training topics and the exercises required by AMC1 5. Angle of attack (AOA) and stall awareness • • ORO.FC.220&230 in the URT part. 6. Stick shaker or other stall-warning device • Aircraft handling at degraded control modes is covered Automation management activation (as applicable) by the malfunction category ‘degraded aircraft control’ Manual aircraft control and furthermore covered by component H.6. (Fly-by- Upset recovery 7. Stick pusher (as applicable) • • wire protection degradations) and shall be performed (recoveries at low and high altitude) 8. Mach effects (if applicable to the aeroplane type) • • in manual and automatic flight. 9. Aeroplane stability • • If aircraft and/or operator-related evidence (e.g. 10. Control surface fundamentals • • incidents, FDM data) indicates the need to further train 11. Use of trims • • a component, aircraft and/or operator-specific 12. Icing and contamination effects • • exercises shall be added in the upset prevention Adverse weather • • training. same as elements A 1-11 13. Propeller slipstream (as applicable) B. Causes of and contributing factors to upsets 1. Environmental • deleted from FSTD training with Annex IV to ED Decision 2. Pilot-induced • 2019/005/R 3. Mechanical (aeroplane systems) • C. Safety review of accidents and incidents relating to aeroplane upsets Safety review of accidents and incidents relating deleted from FSTD training with Annex IV to ED Decision 1. • to aeroplane upsets 2019/005/R D. G-load awareness and management 1. Positive/negative/increasing/decreasing g-loads • • Specific exercises related to this element are required Manual aircraft control 2. Lateral g-awareness (sideslip) • • in the Upset Prevention part and have to be performed Upset prevention/recovery (recoveries 3. g-load management • • as PF, as they are not fully covered by the EBT training according to OEM recommendations at low topic Manual aircraft control Page 58 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ORO.FC.232 – CS OPS.EBT.232 - Example – AMC1 ORO.FC.220&230 ‘Operator conversion training Ground FSTD Rationale suggested relevant EBT ‘training topic and and checking & recurrent training and checking’ training training description’ and high altitudes — where there are indications of g-load, they can be included) E. Energy management 1. Kinetic energy v potential energy v chemical • • Aircraft-specific UPT exercises during conversion course Manual aircraft control energy (power) only, as energy management is trained during several Automation management EBT training topics F. Flight path management 1. Relationship between pitch, power and • • Components are fully covered by the mentioned EBT Automation management performance training topics. Components 5 and 6 are represented Manual aircraft control 2. Performance and effects of differing power • • by identical EBT training topics. Automation management plants (if applicable) If aircraft and/or operator-related evidence (e.g. Manual aircraft control 3. Manual and automation inputs for guidance and • • incidents, FDM data) indicates the need to further Automation management control train an element, aircraft and/or operator-specific Manual aircraft control • • exercises shall be added in the upset prevention Automation management 4. Type-specific characteristics training. Manual aircraft control 5. Management of go-arounds from various stages • • Go-around management during the approach Automation management Manual aircraft control 6. Automation management • • Automation management Manual aircraft control 7. Proper use of rudder • • Failure of the critical engine between V1 & V2 Engine-out approach & go-around Engine failure Upset prevention/recovery (This training can be combined with the Table 2 exercises) G. Recognition 1. Type-specific examples of physiological, visual • • See example scenario elements in respective AMC for Upset prevention/recovery and instrument clues during developing and aircraft generation This training can be combined with the developed upsets Table 2 exercises 2. Pitch/power/roll/yaw • • Page 59 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ORO.FC.232 – CS OPS.EBT.232 - Example – AMC1 ORO.FC.220&230 ‘Operator conversion training Ground FSTD Rationale suggested relevant EBT ‘training topic and and checking & recurrent training and checking’ training training description’ 3. Effective scanning (effective monitoring) • • 4. Type-specific stall protection systems and cues • • 5. Criteria for identifying stalls and upsets • • H. System malfunction (including immediate handling and subsequent operational considerations, as applicable) 1. Flight control defects • • System malfunction with characteristic ‘immediacy’ Automation management and/or ‘management of consequences’ Manual aircraft control Knowledge 2. Engine failure (partial or full) • • Identical EBT training topic Engine failure 3. Instrument failures • • System malfunction combining characteristic ‘loss of Automation management instrumentation’ with ‘immediacy’ and/or Manual aircraft control ‘management of consequences’ 4. Loss of reliable airspeed • • Same as component H.1. Automation management Manual aircraft control 5. Automation failures • • Same as component H.1. Automation management Manual aircraft control 6. Fly-by-wire protection degradations • • Same as component H.1. Automation management Manual aircraft control Knowledge 7. Stall protection system failures including icing • • Same as component H.1. Automation management alerting systems Manual aircraft control Upset prevention/recovery This training can be combined with the Table 2 exercises I. Manual handling skills (no autopilot, no autothrust/autothrottle and, where possible, without flight directors) 1. Flight at different speeds, including slow flight, - • Except for components 3 and 5, components are fully Automation management and altitudes within the full normal flight covered by EBT training topics, if exercises are flown Manual aircraft control envelope without autopilot, autothrust/autothrottle and, where 2. Procedural instrument flying and manoeuvring - • possible, without flight directors. Automation management including instrument departure and arrival Manual aircraft control 3. Visual approach - • Manual aircraft control Page 60 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ORO.FC.232 – CS OPS.EBT.232 - Example – AMC1 ORO.FC.220&230 ‘Operator conversion training Ground FSTD Rationale suggested relevant EBT ‘training topic and and checking & recurrent training and checking’ training training description’ 4. Go-arounds from various stages during the - • Go-around management approach Automation management Manual aircraft control Go-around, all engines operative Engine-out approach & go around 5. Steep turns - • Manual aircraft control Table 2: Exercises for upset recovery training A. Recovery from developed upsets 1. Timely and appropriate intervention • • Strongly recommended in Manoeuvres training / ISI Upset prevention/recovery phase. 2. Recovery from stall events in the following • • Flight crew must be trained as PF and PM. Upset recovery configurations: take-off configuration, clean Due to the protections in flight envelope in configuration low altitude, clean configuration the 4th generation aircrafts, the take-off near maximum operating altitude, and landing upsets can be trained as final APP stall/ Go configuration during the approach phase. around 3. Recovery from nose high at various bank angles • • Upset recovery 4. Recovery from nose low at various bank angles • • Upset recovery 5. Consolidated summary of aeroplane recovery • • Upset recovery techniques Page 61 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC3 ORO.FC.231(a) Evidence-based training EBT PROGRAMME — SCENARIO ELEMENTS AND COMPETENCY MAPPING (a) The operator may develop scenario elements and a competency map that are more relevant to its operation. (b) When developing scenario elements, the operator should ensure that there can be no negative training when asking pilots to induce their own errors. (c) Competencies mapped are those considered critical in managing the scenario. They are determined according to the following principles: (1) those competencies considered most critical to the successful management of the defined threat or error; or (2) those competencies most likely to be linked to the root cause of poor performance in the case of unsuccessful management of a defined threat or error. (d) The competency map may indicate scenarios or combinations of scenarios for development of particular competencies. (e) The competency map indicates the most critical competencies suggested by design, but the instructor should always assess all observed competencies. AMC3 ORO.FC.231(a) and GM2 ORO.FC.231(a) The purpose of AMC3 ORO.FC.231(a) is to allow operators to develop their own scenario elements and competency map to better reflect their operational environment, while maintaining the integrity of the EBT programme. Therefore, the training topics and frequency as per the table of assessment and training topics should not be amended by the operator as they derive directly from the ‘Data report for Evidence-Based Training’3 (AMC2 to AMC7), while the example scenario elements and their associated competency map may be adapted without using the AltMoC procedure (ORO.GEN.120). This provision is a transposition of the Doc 9995 Paragraph 1.2.8. Purpose To give an indication of the most likely critical competencies required for effective management of the scenario or manoeuvre (considering the management of a threat or combination of threats). Process The nominated person for crew training or EBT manager designs one or more example scenario elements, including a description and an outcome. The competency map process is as follows: The competency map process should be undertaken by SMEs who hold or have held a type rating on the aeroplane type. Steps of the competency map process: 1. Using the description of the scenario element, SME instructors determine the competencies most likely to be required for effective management of the scenario element. Generally, about three competencies may be selected. 3 IATA Data Report for Evidence-Based Training August 2014 1st Edition. Page 62 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail 2. SME instructors determine which competencies are most likely to be the root cause(s) of poor performance. Note 1. This is done in teams of instructors, and it is for mapping purposes only. If there is a desire to be more accurate, ask two groups of instructors to review and suggest the competencies following this methodology. Each instructor should use a scale (for example, 1 to 5) for each competency for each scenario element. The following instructors should perform the same analysis independently, and then the results are compared and reconciled by a small group of SME.. Note 2. It is always easy to code (SAW) or knowledge (KNO) as underlying, but there are almost invariably other competencies, especially when there is ineffective management, so the intent should be to balance the mapping of SAW or KNO and map the other predominant competencies within the scenario. Note 3: A similar process is described in ‘equivalency of malfunctions’ (see Delphi). AMC3 ORO.FC.231(a) point (e) The intent of this provision is to highlight that the competency map should not drive the instructor’s observations; instead, the instructor should observe the simulator session with a neutral observation, without focusing on the particular competencies mapped and make neutral assessment of all competencies. For the instructors, the competency map is intended to guide them on what they should expect to observe; however, this does not mean that they should ignore useful learning points for other competencies not mapped within that example scenario. AMC4 ORO.FC.231(a) Evidence-based training PERSONNEL CONDUCTING ASSESSMENT AND PROVIDING TRAINING (a) Ground and refresher training should be provided by suitably qualified personnel. (b) For non-EBT assessment and training: flight training should be provided by a flight instructor (FI), type rating instructor (TRI) or class rating instructor (CRI) or, in the case of the FSTD content, a synthetic flight instructor (SFI). The FI, TRI, CRI or SFI should satisfy the operator's standardisation, experience and knowledge requirements. (c) Emergency and safety equipment training should be provided by suitably qualified personnel. (d) CRM training should be provided by an EBT instructor or, for the classroom CRM training, a CRM trainer. (e) Additional personnel requirements are described in ORO.FC.146 and ORO.FC.231 and in the associated AMC and GM. AMC4 ORO.FC.231(a) This provision is transposed from AMC1 ORO.FC.230 point (d) with the necessary amendments. It includes the provision to allow CRM training by EBT instructors if they have completed the EBT instructors’ standardisation. This provision must be read in conjunction with ORO.FC.146; therefore, when EBT training is delivered, instructors must be provided with an EBT standardisation course. When other training is provided Page 63 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail which is not part of EBT, then only point (b) applies (no combination with ORO.FC.146). A classic example would be ‘aerodrome qualification’ category C, where a regular instructor would provide such training unless the qualification is delivered in conjunction with the EBT programme. AMC1 ORO.FC.230 point (d) reads as follows: ‘(d) Personnel providing training and checking Training and checking should be provided by the following personnel: (1) ground and refresher training by suitably qualified personnel; (2) flight training by a flight instructor (FI), type rating instructor (TRI) or class rating instructor (CRI) or, in the case of the FSTD content, a synthetic flight instructor (SFI), providing that the FI, TRI, CRI or SFI satisfies the operator's experience and knowledge requirements sufficient to instruct on the items specified in points (a)(1)(i)(A) and (B); (3) emergency and safety equipment training by suitably qualified personnel; (4) CRM: (i) integration of CRM elements into all the phases of the recurrent training by all the personnel conducting recurrent training. The operator should ensure that all personnel conducting recurrent training are suitably qualified to integrate elements of CRM into this training; (ii) classroom CRM training by at least one CRM trainer, qualified as specified in AMC3 ORO.FC.115 who may be assisted by experts in order to address specific areas.’ GM1 ORO.FC.231(a) Evidence-based training RECURRENT CREW RESOURCE MANAGEMENT (CRM) Operators implementing EBT in accordance with ORO.FC.231 may demonstrate compliance with ORO.FC.115 by showing how the recurrent CRM requirements are integrated within the operator’s EBT programme. An example of how this may be done is provided in the safety promotion material of EASA (e.g. ‘Oversight guidance for the transition to Mixed EBT Implementation’ and EASA EBT manual). GM2 ORO.FC.231(a) Evidence-based training EBT PROGRAMME — COMPETENCY MAP PROCESS Note 1. The competency map process may be done in teams of instructors. Then the results are compared and reconciled by a small group of subject matter experts (SMEs). Note 2. It is always easy to map SAW or KNO as underlying competency, but there are almost invariably other competencies, especially when there is ineffective management, so the intent should be to balance the mapping of SAW or KNO and map the other predominant competencies within the scenario. Page 64 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail GM3 ORO.FC.231(a) Evidence-based training EBT PROGRAMME — CUSTOMISATION OF THE EBT PROGRAMME (SYLLABI) (a) Syllabi can be customised at three different steps: (1) The first step would be a syllabus for the whole pilots’ population (customisation only at type rating level and/or aircraft generation level). At this step, the operator customises the example scenario elements based on relevant operational data (safety management system, state safety plan, OSD, occurrences, manufacturer data, etc.), and the training topics within the module are the same (same syllabus). At this level, it may be necessary to have a different example scenario element for the different crews within the same module to ensure that pilots are exposed to surprise and unexpected events and thus avoid pilots knowing all the details of the simulator beforehand. (2) The second step would be a different syllabus or part of it for the different populations of pilots. For example, some parts of the syllabus are different for the first officers and the captain, or it is different for the B747 pilots, or for all pilots flying an Airbus model, etc. creating therefore different syllabi). At this step, the module or part of the module is different for each population; this may include a different example scenario element for each population (or a different training topic; however, the customisation at training topic level is more difficult to control). (3) The third step would be syllabi tailored to the individual pilot (pilot customisation — individual syllabus). This step is linked to the procures established for the tailored training and the additional training of the pilots following the VENN model. (b) The procedure to describe the customisation of syllabi must be described in the OM. Customisation is based on evidence that can be gathered on three different levels, two from the inner loop, one from the outer loop. (1) Inner loop (i) Individual evidence based on training data (e.g. grading metrics, training reports, questionnaires, etc.), analysed either for an individual pilot or a group of pilots (for example, all first officers, all B747 pilots, all pilots flying an Airbus model, etc.). (ii) Operator-specific evidence gathered from the safety management process in accordance with ORO.GEN.200. (2) Outer loop Evidence gathered from external sources such as authorities (e.g. state safety plan, etc.), OEMs (e.g. OEBs, OSD, safety documentation such as getting to grip, etc.), etc. GM4 ORO.FC.231(a) Evidence-based training EBT PROGRAMME Further guidance on the EBT programme can be found in the EASA EBT manual. Page 65 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC1 ORO.FC.231(a)(1) Evidence-based training MINIMUM EXPERIENCE TO SUBSTITUTE ORO.FC.230 (a) The operator should have a minimum experience of 3 years of a mixed EBT programme. Note: More information on a mixed EBT programme is provided in GM1 ORO.FC.230(a);(b);(f) and in GM2 ORO.FC.A.245. (b) The operator should demonstrate 2 years of an instructor concordance assurance programme. (c) The operator should demonstrate 1 year of a valid equivalency of malfunctions. (d) The operator should demonstrate 1 year of integration of the training data in the customisation of the EBT programme and SMS data for the contextualisation of the example scenario elements. (e) The operator should demonstrate that there is a verification of the grading system and feedback is provided to the training system performance and to the instructor standardisation concordance assurance. SUBSTITUTION OF THE REQUIREMENTS OF ORO.FC.230 (f) One complete EBT module substitutes one operator proficiency check (OPC). (g) The line evaluation of competence substitutes the line check. AMC1 ORO.FC.231(a)(1) point (a) EASA believes that the transition from legacy training to EBT requires experience in the use of data, competency framework, grading system and instructor concordance assurance. Furthermore, a clear baseline for the training system performance must be established before any alleviation or competency-based licence revalidation can be achieved. Finally, the competent authority must be able to transition and observe changes in the operator processes that support EBT. This requires time. AMC1 ORO.FC.231(a)(1) point (f) and (g) These provisions are introduced to ensure equivalency between traditional training and EBT. There is documentation from the regulator, manufacturers and industry that may not be updated until a later stage due to the novelty of the EBT. This issue may be especially relevant when using the operational suitability data, where credits are defined for a number of checks and training (e.g. credits are defined for line check but not yet for the line evaluation of competence). With this provision, the EBT operator is allowed to make use of such credits. AMC1 ORO.FC.231(a)(1) point (f) This provision is introduced because other parts of the Regulation refer back to ‘proficiency check’. For example, in SPA.LVO.120 the low-visibility training provisions have a frequency of ‘every operator proficiency check’. Therefore, this provision is needed to indicate that a complete OPC is substituted by a complete EBT module, while an LPC is completed by at least two EBT modules as described in Appendix 10 to Part-FCL. Page 66 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Furthermore, this provision is introduced in order to provide clarity in FCL.740 point (a)(3). ‘A pilot working for a commercial air transport operator approved in accordance with the applicable air operations requirements who has passed the operators proficiency check combined with the proficiency check for the revalidation of the class or type rating shall be exempted from complying with the requirement in (2)’. The wording ‘complete’ is to ensure alignment with the current regulation — for example: ‘ORO.FC.230 Recurrent training and checking (a) Each flight crew member shall complete recurrent training and checking relevant to the type or variant of aircraft on which they operate. (b) Operator proficiency check (1) Each flight crew member shall complete operator proficiency checks as part of the normal crew complement to demonstrate competence in carrying out normal, abnormal and emergency procedures.[…]’ AMC1 ORO.FC.231(a)(2) Evidence-based training EBT PROGRAMME AND ASSSESMENT AND TRAINING TOPICS - RESILIENCE (a) Compliance with the table of assessment and training topics ensures that crews are presented with an array of realistic changing events that allow for resilience development purposes. (b) The EBT programme should be designed observing the following principles for resilience development: (1) Resilience, surprise, and unexpected events The EBT programme should be designed in such a way that in every cycle the simulator session (or part of it) allows variations so that the pilots are not familiar with the scenarios presented in the simulator session. Variations should be the focus of EBT programme design, and not left to the discretion of individual instructors, in order to preserve programme integrity and fairness. (2) Resilience and decision-making (dilemma) The EBT programme should be designed in such a way that in every cycle the crews are exposed to a scenario where more than one possible and less than ideal solutions exist, with some unfavourable conditions attached to each solution. GM1 ORO.FC.231(a)(2) Evidence-based training EBT PROGRAMME AND ASSSESMENT AND TRAINING TOPICS - RESILIENCE (a) For resilience-development, crews should be exposed to an array of realistic changing scenarios. The strategies developed by the crews whilst coping with different causes of action will create opportunities for resilience development. (b) Resilience and surprise Page 67 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The operator may create a comprehensive list of scenarios to ensure that each crew is trained in different scenarios avoiding the same scenarios for all crews. This relates to training topic ‘surprise’ and to the customisation of the EBT programme. (c) Resilience and unexpected events Exposing crews to rear, fortuitous, events may prepare crews to deal with other unexpected events. For instance, the table of assessment and training topics offers infrequent example scenario elements such as flying over ‘no fly zone’, etc. The operator may also take infrequent examples from occurrence reporting, or SMS, or manufacturer reports, etc.). This relates to decision-making (PSD) see OB6.9 ‘Demonstrates resilience when encountering an unexpected event’. (d) Dilemma The operator may create scenarios suitable for training of threat assessment, threat management processes and option generation, leading to an optimum decision-making process. At programme design, as in real life, one ‘correct answer’ should be avoided; instead, the EBT programme should offer the crews a number of less than ideal cause of actions; some with unfavourable conditions attached. This relates to decision-making (PSD) and to the contextualisation of the example scenario element. AMC1 ORO.FC.231(a)(2) The AMC has been developed taking into account the existing GM5 ORO.FC.115 Crew resource management (CRM) training, RESILIENCE DEVELOPMENT. Resilience is the ability to prepare and plan for, absorb, recover from, and more successfully adapt to adverse events as defined by the US National Academies of science, engineering and medicine. Surprised: (adjective) caused to feel surprise, amazement or wonder, or showing an emotion due to an unexpected event. Unexpected: (adjective) not expected, anticipated or foreseen. Considered unlikely to happen, not to occur soon. Is used for events and behaviours that occur without warning. Unpredictable: Unforeseeable; cannot say ahead of time. Is used for events and behaviours that are difficult or impossible to predict or foresee. Expect the Unexpected The operator can train its pilots for the unexpected so their skills of resilience are there when they need them. Resilience can be practised by starting small and growing into a more difficult situation. AMC1 ORO.FC.231(a)(2) point (b)(2) The experts consulted by EASA did not reach a consensus on where to include the training of the ‘dilemma’; include it together with resilience or include it as a separate item related to decision- making. The fact is that there are numerous studies and articles related to: — resilience and decision-making; and — resilience and ambiguity (dilemma). Page 68 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Furthermore, ambiguity and decision-making are clearly related and there are many studies and research that also relate decision-making with resilience. While some experts believe that EASA should allocate ambiguity with ‘decision-making’ other experts believe it should be included in ‘resilience’. EASA took the decision to include it in resilience. GM2 ORO.FC.231(a)(2) Evidence-based training EBT PROGRAMME — IN-SEAT INSTRUCTION (ISI) (a) Effective monitoring and error detection are increasingly important when operating highly reliable automated aircraft. (b) In-seat instruction may be used as a valuable tool to maintain and develop the training objectives of some of the training topics, such as skills of monitoring, cross-checking, error management, and recognition of mismanaged aircraft state. GM2 ORO.FC.231(a)(2) The explanatory note about ISI is provided for the IR. This GM clarifies and complements the table of assessment and training topics in regard to ISI. The RMG and EASA agreed that some elements in the ICAO baseline programme classified as frequency A in ISI and in regard to ‘example scenarios’ and ‘competency map’ are incorrect. In addition, with regard to the training topic ‘monitoring, cross checking, error management, and mismanaged aircraft state’, Doc 9995 titles the topic as in-seat instruction (ISI). EASA believes there is an inconsistency because ISI is a means to deliver a training topic and not a training topic (see definition of ISI). Therefore, ISI is removed from the training topics. Furthermore, the IATA data report for EBT does not identify that the means and the only means to deliver such topic (monitoring, cross- checking, error management, mismanaged aircraft state) should be ISI. It also has to be noted that effective monitoring and error detection as well as error management, mismanaged aircraft state, compliance and cross-checking topics are also embedded in the observance of the behavioural indicators. This way, they are present in all of the EBT FTSD sessions, and any observance of deficiencies should be taken as a learning opportunity, identifying the root cause/contributing factor, and discussed during the subsequent ‘facilitated debriefing’. AMC1 ORO.FC.231(a)(3) Evidence-based training EBT PROGRAMME — ENROLMENT (a) Enrolment is when a flight crew member commences the first EBT module. (b) A flight crew member is considered to be de-enrolled when the operator is no longer responsible for the administrative action for the flight crew’s licence revalidation under an EBT programme. (c) Flight crew members who fail to demonstrate an acceptable level of competence and are de- enrolled from the operator’s EBT programme should not exercise the privileges of that type rating. Page 69 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC1 ORO.FC.231(a)(3) This requirement is to maintain the integrity of the EBT programme. The EBT programme will be the means to revalidate pilots’ licence: the revalidation will not be based on a single simulator event, but instead on multiple simulator events. This requires clarity as to when the pilot joined the EBT programme. Normally, this will occur in the operator conversion course where an EBT module (equivalent to an OPC) is planned. This provision also has relevance in the cases of long- term sickness or long leave of absence where the pilot discontinued the training programme. AMC1 ORO.FC.231(a)(3) point (c) The provision was transposed from FCL: FCL.740.A point (c) ‘(c) Applicants who fail to achieve a pass in all sections of a proficiency check before the expiry date of a class or type rating shall not exercise the privileges of that rating until a pass in the proficiency check has been achieved.’ The provision was moved to AMC because the same requirement was transposed in FCL Appendix 10 into an implementing rule. ‘FCL.1030 Conduct of skill tests, proficiency checks and assessments of competence (..) (b) After completion of the skill test or proficiency check, the examiner shall: (1) inform the applicant of the result of the test. In the event of a partial pass or fail, the examiner shall inform the applicant that he/she may not exercise the privileges of the rating until a full pass has been obtained. The examiner shall detail any further training requirement and explain the applicant’s right of appeal; (…)’ AMC1 ORO.FC.231(a)(4) Evidence-based training INSTRUCTOR CONCORDANCE ASSURANCE PROGRAMME (ICAP) (a) The ICAP should be able to identify areas of weak concordance to drive improvement in the quality and validity of the grading system. (b) The ICAP should be adapted to the size and complexity of the instructors’ group and the complexity of the operator’s EBT programme. (c) Complex operators must include an ICAP-specific data analysis, demonstrating: (1) instructor-group assessment homogeneity (agreement); (2) instructor assessment accuracy (alignment). (d) The operator should verify the concordance of the instructors: (1) once every cycle; (2) for a sufficient number of competency-grade combinations. Page 70 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (e) The operator should establish procedures to address those instructors who do not meet the standards required. (f) The operator should maintain a list with the EBT instructors qualified to deliver the EBT programme. AMC1 ORO.FC.231(a)(4) The instructor concordance is a tool for continuous improvement of the EBT programme. Point (a) provides a requirement from a systemic view (e.g. the programme must identify that instructors in a certain fleet have problems to grade non-technical competencies or that one competency is always graded too low. This may occur with ‘application of knowledge’ or ‘application of procedures’ where instructors identify all the time ‘knowledge’ or ‘PRO’ as the root cause for all pilot being deficient when they should not, leading thus to a low grading in ‘knowledge’ or ‘PRO’. Point (e) ensures that each individual instructor has the necessary concordance (e.g. my instructor Pepito Perez has problems to rate FPM and therefore this has to be addressed). GM1 ORO.FC.231(a)(4) Evidence-based training INSTRUCTOR CONCORDANCE ASSURANCE PROGRAMME (ICAP) (a) Instructor concordance is a tool for continuous improvement of the EBT programme as data reliability results in a more accurate and effective training. (b) The operator may have a more frequent, or even a continuous, assessment of concordance as it provides more opportunities to improve. (c) Concordance standards are normally set by the operator; however, the competent authority may recommend criteria, as licences revalidation is performed under EBT. (d) Individual instructor concordance may be verified: (1) through uniform standardisation material where at least three different levels of performance are included and for all the competencies at a frequency of 72 months; (2) by reference to the analysis of the data produced by the instructor every 12 months; normalisation may be necessary as there is no homogeneity of all EBT modules and the pilots that the instructor assessed; and (e) Instructor-group assessment homogeneity (agreement) may be inferred from instructors, which have observed the same content. (f) Instructor assessment accuracy (alignment) may be inferred from comparing instructor assessments with an ‘assessment standard’ consisting of correctly identified competency(ies) and correctly identified grade levels. Neither the competency(ies) nor the grade level(s) may be communicated in advance to the instructors. The assessment standards may be set by consensus of a standards group, in order to guard against individual biases. (g) When the operator uses a small group of instructors (e.g. 10), data-driven concordance assurance programme may be directly integrated into annual refresher training, removing the need for the above guidance. Page 71 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (h) Operators with complex group of instructors (e.g. a big rotation of instructors, subcontracted instructors, big number of instructors, many different fleets, etc.), may need to implement a more extensive concordance assessment system. GM1 ORO.FC.231(a)(4) Safety promotion material — appropriate metrics EASA has planned a safety promotion task (SPT.012) to support the implementation of EBT. The following material has been developed to explain the intent of the wording used in the implementing rule ‘appropriate methods and metrics’, and other concepts used in this regulatory proposal: SPT.012 ORO.FC.231(a)(4) — safety promotion task 012 — safety material for EBT — CONCORDANCE APPROPRIATE METRICS Concordance must be assessed independently per competency, and, if possible, segregated between different levels of competency assessment. This serves to identify whether concordance varies between competencies or between levels of assessment, providing guidance that is more accurate in order to improve concordance. Assessing concordance between instructors should make use of statistical methods, gauging both individual instructor metrics as well as group instructor metrics. Different statistics may be appropriate for different types of measurement. Individual assessments should assess to what extent an individual aligns with predefined standards for the reference material (e.g. correlation analysis) and to what extent the individual’s ability to assess is improving or deteriorating over time (e.g. compared to previous concordance assessments). Group statistics may make use of group agreement (e.g. variance assessment) and group alignment (e.g. group averages compared to standards for the reference material). A high variance implies that a large number of instructors is not rating according to the standards set, and warrants investigation. Individual instructors that exhibit a large deviation from standards, consistent positive/negative bias or poor improvement/deterioration of their concordance with standards, must be considered for focused instructor training before re-engaged in EBT assessments. However, the investigation may determine that although an individual instructor exhibits a large deviation, the reason is not that this instructor is not standardised. The reason could be that the instructor is delivering a different programme (e.g. always delivering a harder-than-usual EBT programme in preparation of command upgrades) or that the instructor is providing training to a specific group of pilots (e.g. those that require remedial training). Finally, when subcontracted instructors are used, the standardisation provided to them should be particularly considered. This group of instructors may not acquire the required concordance initially. In order to maintain the data integrity for instructor concordance, the operator should maintain data traceability for each group of instructors (airline and subcontracted) as the root cause for the good or bad performance of each group may be different given that the background and environment of each group is different. Same principles may be necessary to be applied in other groups (e.g. mature instructors v young instructors). CONTINUOUS IMPROVEMENT OF CONCORDANCE Metrics of instructor concordance must drive specific interventions in instructor training, the assessment framework used and/or the reference material developed. Instructor concordance must Page 72 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail be submitted to a process of continuous improvement in order to safeguard against standards drift and concordance degradation. For this reason, these requirements do not specify statistical thresholds of minimum variance of concordance; however, improvement in concordance metrics should indicate whether the operator’s concordance programme is effective. Over time, as concordance improves, so will the reliability of EBT data. CONCORDANCE ASSURANCE AND EBT INSTRUCTOR RECURRENT STANDARDISATION Instructor concordance may be verified by controlling the content to be assessed (standardisation reference material) such as flight recordings, scripted videos and/or case studies. This material should be of comparable complexity, ambiguity and variation to situations that the operator encounters in their EBT programme. Within each 3-year period, reference material should address every competency at a minimum of two levels per competency, such that concordance is assessed across the wide range of competency assessment that instructor must be proficient in. Reference material may not be presented to the same instructor within 3 years in order to maintain true assessment of an instructor’s ability to assess accurately. Operators should strive to include a broad diversity of flight phases, situations and behaviours when developing reference material, and preferably integrate their own operations and standard operating procedures (SOPs). Reference material should be assessed using the same assessment framework used for actual EBT training delivery, and preferably assess not only the competency observation, but also the ability to assess root causes and identify subsequent training needs. Reference material should be supplemented with ‘correct’ ratings (i.e. answer sheet), such that instructor assessment can be compared against agreed-upon standards. The answer sheet should be composed by a core group of EBT instructors; preferably, rotating members to prevent standards drift and/or lasting bias. Instructor concordance may not be inferred from actual assessment data collected from EBT sessions when these sessions are not equivalent in terms of difficulty, competency distributions, etc. because this may not guarantee equal reference material between instructors. INSTRUCTOR CONCORDANCE The development of strong instructor concordance (inter-rater reliability) is critical for the validity of the EBT data collection. In a norm-referenced system, the operator must safeguard concordance between instructors. Minimum concordance standards are normally set by the operator; however, the competent authority may recommend certain criteria, especially when the revalidation of licences is performed under EBT. Distribution of grades across the instructor community for the modules conducted should be recorded. This recording may be accessible to the instructors, normally a posteriori. Some airlines underweight the grading performed by an instructor with poor concordance to have accurate competency data. Underweight may only be needed in rare cases during mixed EBT; however, it should not happen during full EBT. However, this standard needs to be easy for the instructors allowing them to focus on the observation of the students and to provide training to them rather than cross-checking complicated criteria. Page 73 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail INSTRUCTOR CONCORDANCE SCHEME: (1) This is an example of a concordance scheme: Competencies APK FPM FPA SAW WLM LTW COM PSD KNO 1 Year 1 Year 2 Year 3 Year 1 Year 1 Year 2 Year 3 2 Year 1 Year 2 Year 3 Year 1 Year 2 Grades 3 Year 3 Year 3 Year 2 Year 1 Year 2 4 Year 3 Year 2 Year 2 Year 3 Year 1 5 Year 2 Year 1 Year 1 Year 3 Year 3 * It is possible to combine several competencies in a single assessment event. (2) An unacceptable example of a concordance scheme: APK FPM FPA SAW WLM LTW COM PSD KNO 1 Year 1/ Year 2/ Year 3 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 3 2 Year 1/ Year 2/ Year 3 Year 2 Year 2 Year 2 Year 2 Year 2 Year 2 Year 2 Year 3 3 Year 1/ Year 2/ Year 3 Year 3 Year 3 Year 3 Year 3 Year 3 Year 3 Year 3 Year 3 4 Year 1/ Year 2/ Year 3 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 1 Year 3 5 Year 1/ Year 2/ Year 3 Year 2 Year 2 Year 2 Year 2 Year 2 Year 2 Year 2 Year 3 The following table provides an overview of the usability of different data sources: Uniform reference EBT Assessment and training Data EBT-I dual-observations material (e.g. videos) (e.g. assessments of competence) Useable for Yes: all instructors Yes if data normalisation is possible Partially: although the agreement? can observe the otherwise No: as not all instructors examiner should be highly same content. have observed the same content. standardised, not all instructors have observed the same content. Useable for Yes: with assessment Yes if data normalisation is possible Yes: a crosscheck is alignment? standard attached to otherwise No: as there are no possible. the material. assessment standards to compare to. Useable for Yes: outliers (both Partially: outliers may be suspected Yes: this allows other individuals and by their rating behaviour. addressing other analysis? groups) may be instructor competencies identified. beyond ability to assess. Notes Videos should ideally If training data is used to identify It is difficult/time be of both a outliers, then outlier-generated data consuming to ensure sim/flight event as may be valued as unreliable data for covering exposing well as the facilitated concordance purposes. The gradings instructors to enough debriefing. are still valid for licence revalidation competency-grade and the grading system (e.g. tailored combinations. training). AMC1 ORO.FC.231(a)(5) Evidence-based training CONTINGENCY PROCEDURES FOR UNFORESEEN CIRCUMSTANCES THAT MAY AFFECT THE DELIVERY OF THE MODULE (a) The operator should detail in the EBT programme the contingency procedures in the event of unforeseen circumstances that may affect the delivery of the module (e.g. long-term sick pilot). Page 74 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (b) In case of unforeseen interruption of a module at any point, the missing parts of the module should be rescheduled. (1) The pilot may continue line flying until the expiry of the validity period unless the performance observed was below the minimum acceptable level. (2) If the interruption results in an instructor change, the operator must ensure that the instructor completing the module is provided with the details of the performance of the pilots. (c) In case the pilot misses modules and does not meet the requirements of recent experience (FCL.060): (1) when the pilot misses one module and has not completed two modules in the preceding 12 months, the evaluation phase of the missing module should be rescheduled before the pilot can resume line operations. The manoeuvres and scenario-based training phases of the missing module should be completed 30 days after the evaluation phase or before the expiry date, whichever occurs first; (2) when the pilot misses one module in the preceding 12 months but the pilot’s rating is expired by less than 3 months, the missing module should be rescheduled before the pilot can resume line operations; (3) when the pilot misses one module in the preceding 12 months but the pilot’s rating is expired by longer than 3 months but shorter than 1 year, the missing module should be rescheduled. The evaluation should be delivered by an EBT instructor (or instructors) with examiner privileges before the pilot can resume line operations; (4) When the pilot misses two modules and the pilot’s rating is valid: (i) one module should be rescheduled before the pilot can resume line operations using an EBT instructor (or instructors) with examiner privileges; and (ii) the training topics B and C of the other module should be rescheduled before the expiry date. In such case, the 3-month separation requirement between modules may not apply; (5) When the pilot misses two modules and the pilot’s rating is expired by less than 1 year: (i) one module should be rescheduled using an EBT instructor (or instructors) with examiner privileges; and (ii) the training topics B and C of the other module before the pilot can resume line operations. In such case, the period of 3-month separation between modules may not apply; and (6) If the amount of time elapsed since the expiry of the rating is more than 1 year, the pilot is de-enrolled. AMC1 FCL.625(c) ‘IR — Validity, revalidation and renewal’ and AMC1 FCL.740(b)‘Validity and renewal of class and type ratings’ apply. (d) In the case of other situations not covered by points (b) or (c), point (a) applies. Page 75 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC1 ORO.FC.231(a)(5) point (b)(1) wording ‘unless the performance observed was below the minimum acceptable level’ The intent of the RMG is to allow line operations as long as the observed performance of the pilot was acceptable, provided the pilot was still in the validity period of the licence. However, if an unsafe performance was observed prior to an interruption, the candidate should not continue line operations until remedial training has been performed. There is a similar provision in Part-FCL where pilots shall not exercise the privileges of their licence if the LPC was failed even if their licence is still within the validity period. Description of ‘performance observed below the minimum acceptable level’ is provided in GM1 ORO.FC.231(d)(1) on GRADING SYSTEM – VENN. GM1 ORO.FC.231(a)(5) Evidence-based training CONTINGENCY PROCEDURES — RATINGS RENEWAL (a) The renewal of ratings (e.g. type rating or instrument rating) in EBT follows the Annex I (Part- FCL) to the Aircrew Regulation provisions (IRs and AMC) and is complemented with the provisions covered in AMC1 ORO.FC.231(a)(5). The ATO or AOC will determine the amount of training following Part-FCL; however, as EBT combines assessment and training, the following guidance is applicable: (1) Expiry shorter than 3 months may not require additional training in Part-FCL. In EBT, the missing module is rescheduled with an EBT instructor. Following that, the nominated person for crew training (or the deputy(ies)) for the type rating may renew the licence without extra training, as the EBT programme is now completed (at least two modules in the last 12 months). (2) In Part-FCL, when the expiry is longer than 3 months but shorter than 1 year, there need to be two training sessions. In EBT, there are two cases: (i) One module is missing: the pilot must complete the missing module (two simulator sessions) before resuming line operations. Following that, the nominated person for crew training (or the deputy(ies)) for the type rating may renew the licence in accordance with Appendix 10 as the EBT programme is now completed (two modules in the last 12 months). (ii) Two modules are missing: the pilot must complete one module (two simulator sessions) and the training topics B and C of the other missing module (an extra simulator session) with a total of three simulator sessions. Training data is gathered in a short time period; therefore, an EBT instructor with examiner privilege is introduced to ensure the proficiency of the pilot. (b) In case of an expiry longer than 1 year, the requirements of Part-FCL will be followed and the proficiency checks will be performed in accordance with Appendix 9 as the EBT system may not have sufficient training data for the pilot. Page 76 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (1) Expiry longer than 1 year but shorter than 3 years: a minimum of three training sessions in which the most important malfunctions in the available system are covered plus a proficiency check in accordance with Appendix 9 to renew the licence. (2) Expiry longer than 3 years: the pilot should undergo the training for the initial issue of the type rating. (3) Expiry longer than 7 years: the pilot should undergo the training for the initial issue of the instrument rating. GM1 ORO.FC.231(a)(5) The GM was drafted following AMC1 ORO.FC.231(a)(5) as proposed in this Appendix to the Opinion and the existing AMC1 FCL.740(b)(1) ‘Validity and renewal of class and type ratings’. AMC1 FCL.625(c) IR — ‘Validity, revalidation and renewal’ was also considered. (b) COMPETENCY FRAMEWORK SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib CS OPS.EBT.231.2 Competency framework RECOMMENDED EBT COMPETENCIES (EASA COMPETENCY FRAMEWORK) (a) The operator should include in its EBT programme at least the following competencies: Application of knowledge (KNO) Description: Demonstrates knowledge and understanding of relevant information, operating instructions, aircraft systems and the operating environment OB 0.1 Demonstrates practical and applicable knowledge of limitations and systems and their interaction OB 0.2 Demonstrates required knowledge of published operating instructions OB 0.3 Demonstrates knowledge of the physical environment, the air traffic environment including routings, weather, airports and the operational infrastructure OB 0.4 Demonstrates appropriate knowledge of applicable legislation. OB 0.5 Knows where to source required information OB 0.6 Demonstrates a positive interest in acquiring knowledge OB 0.7 Is able to apply knowledge effectively Application of procedures and compliance with regulations (PRO) Description: Identifies and applies appropriate procedures in accordance with published operating instructions and applicable regulations Page 77 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail OB 1.1 Identifies where to find procedures and regulations OB 1.2 Applies relevant operating instructions, procedures and techniques in a timely manner OB 1.3 Follows SOPs unless a higher degree of safety dictates an appropriate deviation OB 1.4 Operates aircraft systems and associated equipment correctly OB 1.5 Monitors aircraft systems status OB 1.6 Complies with applicable regulations OB 1.7 Applies relevant procedural knowledge Communication (COM) Description: Communicates through appropriate means in the operational environment, in both normal and non-normal situations OB 2.1 Determines that the recipient is ready and able to receive information OB 2.2 Selects appropriately what, when, how and with whom to communicate OB 2.3 Conveys messages clearly, accurately and concisely OB 2.4 Confirms that the recipient demonstrates understanding of important information OB 2.5 Listens actively and demonstrates understanding when receiving information OB 2.6 Asks relevant and effective questions OB 2.7 Uses appropriate escalation in communication to resolve identified deviations OB 2.8 Uses and interprets non-verbal communication in a manner appropriate to the organisational and social culture OB 2.9 Adheres to standard radiotelephone phraseology and procedures OB 2.10 Accurately reads, interprets, constructs and responds to datalink messages in English Aeroplane flight path management — automation (FPA) Description: Controls the flight path through automation OB 3.1 Uses appropriate flight management, guidance systems and automation, as installed and applicable to the conditions OB 3.2 Monitors and detects deviations from the intended flight path and takes appropriate action Page 78 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail OB 3.3 Manages the flight path to achieve optimum operational performance OB 3.4 Maintains the intended flight path during flight using automation whilst managing other tasks and distractions OB 3.5 Selects appropriate level and mode of automation in a timely manner considering phase of flight and workload OB 3.6 Effectively monitors automation, including engagement and automatic mode transitions Aeroplane flight path management — manual control (FPM) Description: Controls the flight path through manual control OB 4.1 Controls the aircraft manually with accuracy and smoothness as appropriate to the situation OB 4.2 Monitors and detects deviations from the intended flight path and takes appropriate action OB 4.3 Manually controls the aeroplane using the relationship between aeroplane attitude, speed and thrust, and navigation signals or visual information OB 4.4 Manages the flight path to achieve optimum operational performance OB 4.5 Maintains the intended flight path during manual flight whilst managing other tasks and distractions OB 4.6 Uses appropriate flight management and guidance systems, as installed and applicable to the conditions OB 4.7 Effectively monitors flight guidance systems including engagement and automatic mode transitions Leadership & teamwork (LTW) Description: Influences others to contribute to a shared purpose. Collaborates to accomplish the goals of the team OB 5.1 Encourages team participation and open communication OB 5.2 Demonstrates initiative and provides direction when required OB 5.3 Engages others in planning OB 5.4 Considers inputs from others OB 5.5 Gives and receives feedback constructively OB 5.6 Addresses and resolves conflicts and disagreements in a constructive manner Page 79 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail OB 5.7 Exercises decisive leadership when required OB 5.8 Accepts responsibility for decisions and actions OB 5.9 Carries out instructions when directed OB 5.10 Applies effective intervention strategies to resolve identified deviations OB 5.11 Manages cultural and language challenges, as applicable Problem-solving — decision-making (PSD) Description: Identifies precursors, mitigates problems, and makes decisions OB 6.1 Identifies, assesses and manages threats and errors in a timely manner OB 6.2 Seeks accurate and adequate information from appropriate sources OB 6.3 Identifies and verifies what and why things have gone wrong, if appropriate OB 6.4 Perseveres in working through problems whilst prioritising safety OB 6.5 Identifies and considers appropriate options OB 6.6 Applies appropriate and timely decision-making techniques OB 6.7 Monitors, reviews and adapts decisions as required OB 6.8 Adapts when faced with situations where no guidance or procedure exists OB 6.9 Demonstrates resilience when encountering an unexpected event Situation awareness and management of information (SAW) Description: Perceives, comprehends and manages information and anticipates its effect on the operation OB 7.1 Monitors and assesses the state of the aeroplane and its systems OB 7.2 Monitors and assesses the aeroplane’s energy state, and its anticipated flight path OB 7.3 Monitors and assesses the general environment as it may affect the operation OB 7.4 Validates the accuracy of information and checks for gross errors OB 7.5 Maintains awareness of the people involved in or affected by the operation and their capacity to perform as expected OB 7.6 Develops effective contingency plans based upon potential risks associated with threats and errors Page 80 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail OB 7.7 Responds to indications of reduced situation awareness Workload management (WLM) Description: Maintains available workload capacity by prioritising and distributing tasks using appropriate resources OB 8.1 Exercises self-control in all situations OB 8.2 Plans, prioritises and schedules appropriate tasks effectively OB 8.3 Manages time efficiently when carrying out tasks OB 8.4 Offers and gives assistance OB 8.5 Delegates tasks OB 8.6 Seeks and accepts assistance, when appropriate OB 8.7 Monitors, reviews and cross-checks actions conscientiously OB 8.8 Verifies that tasks are completed to the expected outcome OB 8.9 Manages and recovers from interruptions, distractions, variations and failures effectively while performing tasks CS OPS.EBT.231.2 and CS OPS.EBT.231.3 Competency framework ICAO will implement a new competency model in late 2020 (see State letter 18-77 - Annex 1 and the Doc 9868 ‘PANS-TRG’ consequential to Amendment 5 to the Doc 9868 ‘PANS-TRG’). The new competency model of ICAO is based on the original competency framework published in ICAO Doc 9995ö however, it is not the same. Background of the competency framework The original competency framework has been developed by a large industry experts working group and was based upon systems tested and validated in operational use. The availability of a worldwide-harmonised framework of competencies is of great value. This competency framework can be applied to both baseline and enhanced EBT programmes. Pilot core competencies were developed to support the EBT concept adopted by ICAO in 2013. An international industry working group was established in 2007. The EBT and Instructor Qualification group began work in early 2008. The Group was mostly comprised of expert practitioners in pilot training from almost 50 organisations worldwide. The group met every 2 months from early 2008 until end of 2011. The group decided that the first and critical step in the development of EBT was to identify a complete framework of performance indicators, in the form of observable actions or behaviours, usable and relevant across the complete spectrum of pilot training for CAT operations. These competencies and Page 81 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail performance indicators combine the technical and non-technical (CRM) knowledge, skills and attitudes that have been considered essential for pilots to operate aircraft safely, efficiently and effectively. The development of pilot core competencies was considered as the first important step towards the creation of the ‘total systems approach to training’. After extensive consultation and discussion, the framework of behaviours was developed, divided into 8 core competencies, each with observable performance indicators. The competencies were published in Doc 9995. The core competencies are primarily an assessment tool, offering a different approach from the evaluation of outcomes and manoeuvres, the purpose being to understand and remediate root causes of performance difficulties, rather than addressing only the symptoms. The purpose of these performance indicators is to underpin the creation of performance expectations at all stages of training in a pilot’s career. To complete the picture, a fair and usable system of grading performance is also required, and instructors using it should be trained and assessed themselves as competent in its use. The publication of Doc 9995 limits the applicability of EBT to recurrent training conducted in a qualified FSTD, but it has been always anticipated that the example framework of core competencies agreed should be applied to all aspects of initial and recurrent pilot training for CAT operations, including pilot selection and instructor pre-selection. A number of ‘behavioural marker’ systems were considered, and the group chose the most relevant and appropriate ones, and developed them further to include technical competencies and associated performance indicators. The behavioural marker system was the one published by the UK CAA in CAP 737 in 2005, in service across a wide range of cultures since 2002. The system has been validated through operational use. By far the most significant challenge for operators using these competency frameworks is the creation of an effective performance assessment and grading system, and subsequently the need for instructor training and the assurance of instructor concordance. Finally, the competency framework of EBT provides a good process for the training needs analysis. The competencies in EBT provide a hierarchy and they are linked between them. There are some competencies that the pilot cannot reach without having first other ones. For example, in order for the pilot to have a strong competency in ‘leadership and teamwork’, it is necessary to be good at ‘communications’, and probably good at ‘workload management’. At the same time, in order to be good at ‘workload management’, being good at ‘flight path management — automation’ or ‘flight path management — manual control’, depending on the scenario, is as well necessary. An example of a possible root cause analysis is shown below. Problem-solving & Leadership & teamwork decision-making Page 82 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Workload Communication Situation awareness management Application of Flight path management — Flight path management — procedures automation manual Application of knowledge This approach to competencies and the interaction/relationship between them is supported by MAN4GEN (Manual Operation for 4th Generation Airliners). An extract of the ‘Final Report Summary’ is provided below: ‘(…) Results show that high-performing crews in this scenario were highly rated in Communication, Leadership and Teamwork, Problem Solving and Decision Making, Situation Awareness, and Workload Management. These competencies need to be paired together since some of them are a consequence of good performance in the others. For example, Communication by itself is not indicative of good performance since this competency is only a medium to propagate good behaviour in the other competencies identified here. In fact, as noticed with poor-performing crews, communication needs to be effective and clear to guarantee that the recipients understand and acknowledge what is being said. If that is not the case, it can lead to a performance decrease in the other core competencies (e.g. loss of Situation Awareness). Reflecting on the results from this analysis, poor-performing crews showed difficulties in the competencies where high-performing crews were strong, especially during high-workload situations. These poor-performing crews completely skipped the planning flight phase which had a high impact during the execution flight phase, shown by the several below average and poor performance comments. Also, the heat-map shows that these crews already have difficulties in application of procedures (PRO) during low-workload situations (flight phases 1 and 2) and in manual flight throughout the scenario. High-performing crews, on the other hand, do not show negative comments for these competencies during these flight phases, yet positive comments were not present since conducting the required procedures here is not considered as above average performance. Despite the predictive asymmetry preventing the prediction of positive performance, it can at least be premised that poor performance for the overall flight can be predicted from low workload situations. All in all the collection of observed competencies are able to draw a clear picture of the differences between high and poor performing crews. This analysis has identified the competencies that are most helpful in managing unexpected and challenging events, in addition to those competencies whose absence is most likely to lead to poor performance and unsafe outcomes. The desirable competencies identified by the analysis of crew responses to this scenario are: Leadership & teamwork, communication and problem solving & decision making.’ Page 83 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.231.2 Competency framework - recommended EBT competencies (EASA competency framework) ICAO will amend the EBT competency framework provided in Doc 9995. The new model is based on the ICAO competency framework for aeroplane pilots contained in Part II, Section 1, Chapter 1 of Doc 9868 ‘PANS-TRG’ (it may be applicable in November 2020). For this reason, EASA already proposes in the Appendix to this Opinion the core competency model of ICAO with the addition of ‘application of knowledge’. CS OPS.EBT.231.2.231 — application of knowledge EASA decided to introduce ‘application of knowledge’ as an additional competency to the ICAO core competency framework. The reason behind presenting knowledge as the first one and therefore numbering this competency with the ‘zero’ (0) is that all competencies are built on the basis of knowledge. The competency however has been named ‘application of knowledge’ to indicate that it is about what the instructor is observing — observable behaviours related to knowledge; therefore, the ‘application of knowledge’. KNO is a new competency not covered in Doc 9995. There is more information about this competency in some of the material provided by manufacturers. As an example of this, Airbus OTT 999.0012/17 provides the following reference: ‘In order to ensure that the required competencies are acquired and to perform the training on undesired aircraft state, the flight crew should be aware of the following items: - Causes and contributing factors of undesired aircraft state - Examples of incidents related to undesired aircraft state. In addition, the flight crew should review all of the following items: - The control and display systems (EFIS & ECAM): The flight crew should know the indications provided by the display units, but also their evolution over time in order to anticipate the flying conditions. - The flight controls systems, that include flight control laws and protections: The flight crew should know how to handle the aircraft. In addition, the flight crew should know how the protections work, their availability, and their limits. - The automation (Autopilot (AP), Flight Directors (FD) and Auto thrust (A/THR)): The flight crew should know how to use the automation, their availability and their limits. The flight crew should review the practices to engage the automatisms, as well as the takeover techniques and recommendations (Airbus golden rule n°4). - The energy management of the aircraft, that includes thrust settings: The flight crew should understand the acceleration and deceleration capabilities of the aircraft. - The flight envelope limitations: Page 84 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The flight crew should know the flight envelope of the aircraft, in order to keep the aircraft within the environmental and aerodynamic limits and to know when the aircraft is out of these limits. - Aircraft capability related to flight control laws: The flight crew should know the capability of the aircraft in response to the related active flight control laws (normal, alternate and direct law). - Procedures and techniques related to undesired aircraft state: The flight crew should know the procedure and techniques for nose high and nose low recovery, stall recovery and unreliable airspeed.’ CS OPS.EBT.231.2.231 — Application of procedures and compliance with regulations (PRO) EASA introduced a change in the abbreviation of ‘application of procedures and compliance with the regulations’ because of a comment received to the NPA. Additionally, the old abbreviation (APK) refers to application of procedures and knowledge. This is not appropriate for EASA due to the introduction of application of knowledge as a competency. CS OPS.EBT.231.3 Competency framework ADAPTED COMPETENCY MODEL (a) An operator seeking to develop an adapted competency model under ORO.GEN.121 should: (1) identify positive behaviours and use language that avoids ambiguity; and (2) demonstrate equivalence to the recommended EBT competencies in CS OPS.EBT.231.2. (b) In order to demonstrate equivalence, the operator should map the competencies and observable behaviours to the recommended EBT competencies. (c) When the operator is translating CS OPS.231.2 into its common language, the application of ORO.GEN.121 may not be necessary. The translation may not be literal. CS OPS.EBT.231.3 Adapted competency model EBT and competency-based training are based on the concept that competencies are transferable. In the design of a competency-based assessment and training programme, a limited number of competencies are defined. If an airline decides to add or remove a competency, there should be a clear and justifiable reason to do so. Operators may develop suitable equivalent frameworks to meet their needs. — A limited number of competencies involving knowledge, skills and attitudes should be defined. — These defined competencies should cover more than a single situation and be consistently observable across a wide variety of contexts. Short summary on how to develop an operator ‘COMPETENCY FRAMEWORK’ structure A rapid analysis of the training needs and the local environment should answer the following questions: Page 85 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail — What is to be trained? — What tasks does the trainee need to be able to perform by the end of the training? — What regulatory, technical and operational knowledge is required? — What skills are required? — What attitudes are required? — What are the specific conditions required for performance (i.e. level of complexity, specific requirements)? IATA or other operator’s competency framework used as a reference Operator analysis Consider … the specific training conditions from the reference competency Basic framework of framework, the relevant competencies, indicators Select… competencies and performance and performance criteria criteria applicable Operator’s specific the defined specific operator’s competencies and Add… competencies and performance performance criteria criteria the operator’s observable Operator’s specific underpinning indicators (KSAs) Identify… observable behaviours to determine competency conditions under which the competencies must be Operator’s standards Determine... demonstrated and definition performance criteria must be met Operator’s competency framework Page 86 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.231.3 point (c) The use of the term ‘common language’ refers to the common language used by the operator. An IR for such requirement is provided in the Air OPS Regulation Annex IV. ‘CAT.GEN.MPA.120 Common language The operator shall ensure that all crew members can communicate with each other in a common language.’ GM1 ORO.FC.231(b) Evidence-based training ADAPTED COMPETENCY MODEL/POSITIVE OBSERVABLE BEHAVIOUR (a) OBs should describe behaviours that contribute to positive pilot performance. (b) The indicators should clearly describe how a competency is expected to be demonstrated by a crew member in the context of the operational environment. (c) TRAINING SYSTEM PERFORMANCE SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib AMC1 ORO.FC.231(c) Evidence-based training TRAINING SYSTEM PERFORMANCE — FEEDBACK PROCESS (a) Feedback process is the continuous process of collecting and analysing assessment and training data from an EBT programme. (b) The feedback process should use defined metrics to collect data in order to: (1) identify trends and ensure corrective action where necessary; (2) identify collective training needs; (3) review, adjust and continuously improve the training programme; (4) further develop the training system; and (5) standardise the instructors (when the standardisation and concordance assurance programme is integrated into the training system performance). (c) Alternatively, where a system for the measurement of training system performance already exists, the operator may use it and, if necessary, adapt it to meet the demands of EBT. AMC1 ORO.FC.231(c) This requirement is transposed from Doc 9995 paragraphs 4.1.2 (d) and (e) with the necessary amendments to incorporate the ICAO proposal into the European regulatory framework. ‘4.1.2 There are various mechanisms for the implementation of EBT, which should be conducted in close consultation with the CAA and which include: a) the definition of an implementation and operations plan; b) the adaptation of the programmes defined in Appendices 2 to 7 to Part II according to the generation of aircraft (fleet) and type of operation for the operator; Page 87 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail c) the EBT programme implementation (an initial limited trial phase should be considered by the CAA); d) the review of training effectiveness upon receipt of sufficient training system data; and e) the adjustment and continuous improvement of the training programme according to the training system feedback.’ AMC1 ORO.FC.231(c) point (a) The definition is transposed from ICAO Doc 9995 Chapter 3.6. ‘3.6.7 Feedback system. For the purpose of collecting data from an EBT programme, and making adjustments and continuous improvement to the training system, an operator should implement a performance feedback system utilising defined metrics’ However, the ICAO text has been modified to accommodate the wording to the EU regulatory system. AMC1 ORO.FC.231(c) point (a) wording ‘continuous’ Using the word ‘continuous’ ensures that there is data collection throughout the year and not at a certain single point in time. AMC2 ORO.FC.231(c) Evidence-based training FEEDBACK PROCESS — DATA PROTECTION – GRADING SYSTEM (a) The objective of protecting the EBT data is to avoid inappropriate use of it in order to ensure the continued availability of such data, to maintain and improve pilot competencies. (b) The data access and security policy should restrict information access to authorised persons. (c) The data access and security policy should include the measures to ensure the security of the data (e.g. information security standard). (d) The data access and security policy (including the procedure to prevent disclosure of crew identity) should be agreed by all parties involved (airline management and flight crew member representatives nominated either by the union or the flight crew themselves). (e) The data access and security policy should be in line with the organisation safety policy in order to not make available or to not make use of the EBT data to attribute blame or liability. (f) The operator may integrate the security policy within other management systems already in place (e.g. information security management). AMC2 ORO.FC.231(c) The volume of training data will increase through EBT and some provision must be made for individual data protection. However, the Main-group RMT.0599 maintained that data protection in excess of what the GDPR offers is undesirable in a safety-critical industry as the protection of the public is of higher interest than the protection of an individual pilot. On the other hand, the representative of the pilots in the EBT subgroup RMT.0599 requested more stringent data protection requirements due to the increased volume of training data. ORO.AOC.130 ‘Flight data monitoring – aeroplanes’ already requires a system that provides such kind of protection (individual data protection) and at the same time, it provides good information to Page 88 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail operators and authorities. The details of such protection and scope are provided in AMC1 ORO.AOC.130 points (g) and (k). Point (a) of this AMC is transposed from AMC1 ORO.AOC.130 point (b) and from ICAO Doc 9859 AN/474 Safety Management Manual (SMM): ‘the sole purpose of protecting safety information from inappropriate use is to ensure its continued availability so that proper and timely preventive actions can be taken and aviation safety improved;’ Point (b) of this AMC is transposed from AMC1 ORO.FC.130 point (k); however, some of the details were transferred to GM2 ORO.FC.231(b). Point (c) of this AMC is transposed from AMC1 ORO.FC.130 point (k)(6). This provision must be read in conjunction with ORO.GEN.140 the Air OPS Regulation where the competent authority has access to all records: ‘ORO.GEN.140 Access (a) For the purpose of determining compliance with the relevant requirements of Regulation (EC) No 216/2008 and its Implementing Rules, the operator shall grant access at any time to any facility, aircraft, document, records, data, procedures or any other material relevant to its activity subject to certification, SPO authorisation or declaration, whether it is contracted or not, to any person authorised by one of the following authorities: (…)’ Point (d) of this AMC was inspired by Regulation (EU) No 376/2014 Article 15 point 2(a). For point (e), the chapter 3 appendix 3 ICAO Annex 19 was used as the principles to draft this provision. GM1 ORO.FC.231(c) Evidence-based training TRAINING SYSTEM PERFORMANCE — FEEDBACK PROCESS — METRICS (a) Training metrics within the feedback process are a valuable source of data. Typical metrics may include but are not limited to: (1) differences in success rates between training topics; (2) the trainees’ feedback (e.g. surveys), which provides a different perspective as to the quality and effectiveness of the training; (3) instructor concordance assurance: this system is important to measure the effectiveness of the instructor calibration process. It is important to remind that the purpose of this system is not to spy on instructors or to pressure individuals to change their grading; (4) level 0 grading metrics (competent metrics): data metrics providing the information whether the pilot(s) is (are) competent or not. Metrics examples: distribution of pilots not competent after the SBT, distribution of pilots not competent in the EVAL and competent after the SBT; (5) level 1 grading metrics (competency metrics): quantifiable data from the grading system — numeric grade of the competencies (e.g. 1 to 5). Metrics examples: (i) distribution of level of performance within the range of competencies; Page 89 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (ii) differences in grades between aircraft types; (6) level 2 grading metrics (observable behaviour metrics): the instructors record predetermined OBs during the session (e.g. in specific example scenario elements). Metrics example: differences in displaying OBs between ranks of pilots; (7) level 3 grading metrics (task metrics): the instructor records a predetermined specific action(s) or procedure(s). For instance, did the pilot calculate the landing distance? Or, did the pilots make a call-out in a specific manoeuvre? This level is usually linked to data collection of the SMS or EBT feedback loop (e.g. was the call-out of the TCAS manoeuvre correct? ‘TCAS I have control’). Metrics example: distribution of errors for various training scenarios and aircraft types. (8) during the simulator session, the operator may consider the level of grading metrics that the instructor needs to collect, taking into consideration the workload of the instructor. (b) Training metrics are an invaluable component in supporting an EBT programme but they must be placed in the context of operational data because only the latter can justify the importance of specific training. For this purpose, data from the line evaluation of competence is important to measure the effectiveness of the EBT programme in operations. It may include data from the process for the monitoring of line operations. (c) Complex operators may, in the context of their safety management system, establish a safety action group dedicated to training: ‘training safety action group’. This may be a best practice to meet the implementing rule. GM1 ORO.FC.231(c) This requirement is transposed from Doc 9995 paragraph 5.3.1 with the necessary amendments to incorporate the ICAO proposal into the European regulatory framework: ‘5.3.1 Training metrics. The ‘inner loop’ within the training function is a valuable source of data. Taking full advantage of such data requires robust and well-calibrated training metrics. Typical metrics include: a) differences in success rates between aircraft types and training topics; b) distribution of errors for various training scenarios and aircraft types; c) skill retention capability versus skill type; d) the trainee’s feedback, which provides a different perspective as to the quality and effectiveness of the training product; and e) instructor tracking system: this system is important to measure the effectiveness of the instructor calibration process. However, it is essential to impress that the purpose of this system is not to spy on instructors or to pressure individuals to change their grading.’ GM1 ORO.FC.231(c) point (b) This provision is a transposition from the Doc 9995 paragraph 5.3.2: Page 90 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ‘5.3.2 Training metrics are an invaluable component in supporting an EBT programme but they must be placed in the context of operational data, because only the latter can justify the importance of a specific skill within the real operation.’ Furthermore, operational data is already required in ORO.AOC.130 and ORO.GEN.200 of the Air OPS Regulation. GM2 ORO.FC.231(c) Evidence-based training FEEDBACK PROCESS — DATA PROTECTION – GRADING SYSTEM The data access and security policy may, as a minimum, define: (a) a policy for access to information only to specifically authorised persons identified by their position in order to performed their duties. The required authorised person(s) does (do) not need to be the nominated person for crew training (or the deputy(ies)), but could be the EBT programme manager or a third party mutually acceptable to unions or staff and management. The third party may also be in charge of ensuring the correct application of the data access and security policy (e.g. the third party is the one activating the system to allow access to the authorised persons); (b) the identified data retention policy and accountability; (c) the measures to ensure that the security of the data includes the information security standard (e.g. information security management systems standard e.g. ISO 2700x-ISO 27001, NIST SP 800-53, etc.); (d) the method to obtain de-identified crew feedback on those occasions that require specific follow-up; and (e) When there is a need for data protection, it is preferable to de-identify the data rather than anonymise it. GM2 ORO.FC.231(c) This GM is transposed from AMC1 ORO.AOC.130 ‘Flight data monitoring – aeroplanes’ point (k): (…) ‘This procedure should, as a minimum, define: (1) the aim of the FDM programme; (2) a data access and security policy that should restrict access to information to specifically authorised persons identified by their position; (3) the method to obtain de-identified crew feedback on those occasions that require specific flight follow-up for contextual information; where such crew contact is required the authorised person(s) need not necessarily be the programme manager or safety manager, but could be a third party (broker) mutually acceptable to unions or staff and management; (4) the data retention policy and accountability, including the measures taken to ensure the security of the data; (5) the conditions under which advisory briefing or remedial training should take place; this should always be carried out in a constructive and non-punitive manner; Page 91 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (6) the conditions under which the confidentiality may be withdrawn for reasons of gross negligence or significant continuing safety concern; (7) the participation of flight crew member representative(s) in the assessment of the data, the action and review process and the consideration of recommendations; and (8) the policy for publishing the findings resulting from FDM.’ GM2 ORO.FC.231(c) – de-identified data VS anonymised data SPT.012 — Safety promotion to ORO.FC.231(c) Data protection DE-IDENTIFIED DATA Anonymised data should be avoided in EBT, as in order to achieve the ultimate goal of the EBT system, which is a fully individual and personalised training programme for the pilot, the system needs to know the training history of the pilot. De-identification offers the possibility that NO human being would be able to ever have access to the identified data (or only the pilot themselves as the data belong to them), while at the same time the system is able to offer a personalised training programme. For information, please see below general definitions of anonymization and de-identification. ‘Anonymisation means the act of permanently and completely removing personal identifiers from data, such as converting personally identifiable information into aggregated data. Anonymised data is data that can no longer be associated with an individual in any manner.’ ‘De-identification: de-identification involves the removal of personally identifying information in order to protect personal privacy. In some definitions, de-identified data may not necessarily be anonymised data and in such cases, anonymised data is a particularised subset of de-identified data.’ (d) GRADING SYSTEM SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib AMC1 ORO.FC.231(d)(1) Evidence-based training GRADING SYSTEM (a) The grading system should provide quantifiable data for the measurement of the training system performance. (b) The grading scale should be 1 to 5, where: (1) Grade 1 — NOT COMPETENT — determines that the minimum acceptable level of performance was not achieved for the conduct of line operations. An outcome of ADDITIONAL TRAINING REQUIRED and level 2 grading metrics should be recorded. (2) Grade 2 to 5 determine an outcome of COMPETENT for the conduct of line operations. (3) Grade 2 (below the average) determines that the minimum acceptable level was achieved for the conduct of line operations. Additionally, level 2 grading metrics should be recorded. Minimum performance indicates a need for training (e.g. tailored or additional) to elevate performance. It includes: Page 92 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (i) continuous grades 2 in a competency in multiple modules, or (ii) the majority of competencies graded with 2 in a module. (4) Grade 3 is the average. (5) Grade 4 determines that the pilot is above the average. (6) Grade 5 (exemplary) determines that the pilot is above the average and the outcome is enhanced safety, effectiveness and efficiency. (c) The operator should develop further grading guidance to the above points to help the instructors determine the grade of the pilots they assess. AMC2 ORO.FC.231(d)(1) Evidence-based training GRADING SYSTEM — ALTERNATIVE SYSTEM (a) An alternative grading system should provide quantifiable data for the measurement of the training system performance. (b) The grading scale for each competency should: (1) determine the grade at which the performance is considered: (i) NOT COMPETENT for the conduct of line operations. An outcome of ADDITIONAL TRAINING REQUIRED and level 2 grading metrics should be recorded; and (ii) COMPETENT for the conduct of line operations; and (2) determine for the pilot whose performance is considered competent for the conduct of line operations: (i) if the pilot needs more training (e.g. tailored or additional training) to elevate their performance to the operator specified norm; (ii) if the pilot is at the operator specified norm; (iii) if the pilot is above the average (it can be one or more grades e.g. above the average and exemplary). (c) The operator should develop further guidance to the above points to help the instructors determine the grade of the pilots they assess. AMC1 ORO.FC.231(d)(1) This provision did not obtain a full consensus in the EBT subgroup RMT.0599. Whereas the need for guidance for the grading system was fully supported, there was disagreement regarding the regulatory level it should have —IR, AMC or GM (e.g. GM2 ORO.FC.231(d)). Airline associations represented in the EBT subgroup RMT.0599 believed that the operators should have the choice to use their own grading system without any involvement of the competent authority. Other stakeholders believed there should be a prescriptive approach in order to ensure that all pilots are assessed in the same way. Some of the arguments for such prescriptive approach are: Page 93 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail — As this Opinion allows the revalidation of licences based on the EBT system and therefore based on the rate obtained within the grading system, all pilots in Europe should be graded in the same way given that pilots with a valid type rating can join any airline in the European market. Therefore, level playing field should be considered. This argument is relevant for points 1 and 2 in the scale proposed in this Appendix to the Opinion. — A standardised grading system of airlines will allow a standardised approach to grading and therefore to forms and paperwork. This may simplify bureaucracy in the competent authorities across Europe. In addition, the potential benefits this standardised approach to the grading system would bring to the oversight functions of the competent authorities were discussed. — Data exchange: EASA consulted some data experts whether a standardised approach to the grading system could bring benefits to all stakeholders4. The conclusion is that in order to facilitate the data exchange (which is of paramount importance nowadays), it is quite important to have a common grading system. Data preparation, normalisation and standardisation can take up to 90 % of the resources, while the actual data analysis may take only 10 %. A standardised approach to grading system, competency framework and OB could reduce the data preparation and normalisation close to 100 %. It could additionally increase data exchange between stakeholders (de-identification is ensured in accordance with the data protection regulations). Furthermore, platforms like the European Data4safety or the FAA Aviation safety information analysis and sharing (ASIAS) will largely benefit from a standardised approach. Note: The initiatives described above are planned on a voluntary basis and in full compliance with the GDPR. Taking into account the above, EASA decided to locate this provision at an intermediate regulatory level: AMC. This regulatory level allows an increased flexibility compared to IRs, whereby national authorities could approve deviations in accordance with AltMoC (ORO.GEN.120 of the Air OPS Regulation). Furthermore, an alternative grading system in AMC2 ORO.FC.231(d)(1) provides further flexibility to the operators. Note: As mentioned, data exchange will be done in accordance with the data protection regulations (European and national). On the other hand, some operators believe that in order to fulfil the 1 to 5 grading requirement, they will need to change their IT tools. This may be expensive. According to the RIA, the price of this system is around EUR 100 000 (one-off expense) and the same amount is needed every year (maintenance). For that reason, to avoid this one-off expense, the possibility for an alternative grading system was provided to allow those operators that already invest in a system to continue to do so. The grading provided in the AMC follows the criteria presented in the IATA Evidence-Based Training Implementation Guide, Chapter 6.4: ‘1. Fairness and accuracy The grading system should allow the evaluation to be objective, fair, and relevant. It should be reliable, accurate, consistent and resistant to abuse, halo effects, instructor-evaluator laziness, ‘box ticking’ 4 The whole spectrum of stakeholders: airlines, competent authorities, accident and incident investigation authorities, safety analysts, etc. (there are plans to extend EBT to helicopters and business jets). Page 94 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail and bias, both positive and negative. Finally, it should ensure that pilots who are unable to fulfil competency performance expectations are not released to line service. 2. Clarity The grading system should allow assessments to be transparent, clear, complete, unambiguous, and not subject to interpretation or confusion. It must also address the occasions where pilots do not have the opportunity to demonstrate a particular competency. 3. Usability The grading system should be simple, easy to use, understandable, practical, manageable, accessible, uncomplicated, and resistant to unintentional errors. It should not dominate any debrief and should be compatible with facilitation. Finally, it should be compatible with any media to be used, electronic or otherwise. 4. Ease of compliance The grading system should comply with both operator and CAA requirements. It should meet high- level regulations, allow auditing, and be traceable, explainable and long lasting. It should also ensure that any assessment is less liable to legal action. 5. Continuous improvement The grading system should provide evidence to enable improvements in both the training system and trainee performance, for the purpose of enhancing safety. It should be meaningful, deliver useful data, identify trends, aid analysis and address existing, future or potential problems in order to improve the training system. It should enable trainees to provide feedback on their assessment in order to help improve grading consistency and the grading system. It should also enable the continuous development of the trainee’s performance. 6. Motivating The grading system should be motivating, trustworthy, respectful, and easy to ‘sell’’, so that both trainers and trainees enjoy the experience without creating fear. It should also recognize exemplary performance and promote commitment by both trainers and trainees to the assessment process. 7. Technical data management The grading system should provide a manageable quantity of good data, be media compatible, easy to record and produce electronic data, compatible with analysis and presentation tools. It should also maintain data protection and assure controlled access. 8. Adaptability The grading system should be adaptable, flexible and able to tailor to all facets of the operation, aircraft types and training objectives. 9. Implementation risk The grading system should provide robust defences against the risks of ineffective implementation. The system should be comprehensible for trainers, enable efficient trainer standardisation, strong inter-rater reliability, and facilitate the identification of trainer divergence. It should be familiar to all users, cost efficient and resistant to drift and mutation.’ Page 95 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail However, the RMG provided further guidance to expand some of the characteristics as follows: — Fairness and accuracy: identifies evaluator divergence, facilitates instructor concordance, is not repressive, is not open to abuse, avoids positive/negative bias — Usability: is acceptable to evaluators, avoids unintentional mistakes, is familiar and is not complicated — Safety improvement: is compatible with facilitation, works towards excellence, is useful, identifies trends, is acceptable to operator, not costly, does not allow incompetent pass, improves system, continuous development — Adaptability: customisable, cross-cultural. AMC1 ORO.FC.231(d)(1) point (b)(2) The wording ‘competent for the conduct of line operations’ means that the pilot is competent at an industry level, in order to ensure a level playing field. It is therefore NOT intended to be at an airline level. This does not mean that the airline may require more than a grade 2 to allow the pilot to operate in their aircraft. AMC1 ORO.FC.231(d)(1) point (b) grade 5 The preferred scale of grading is 1 to 5. The reason is to have a good granularity on the pilot performance and allow the instructor to grade the norm. Although EASA allows alternative grading systems in AMC2 ORO.FC.231(d)(1) and therefore allows 1 to 4 grading, the initial intention was to measure competence performance (grade) in the same way, meaning 1 and 2 should mean the same in both grading systems as this is a key element for level playing field. Following the discussion with the RMG, EASA decided to merge grades 4 and 5 in the alternative grading system and have only one grade: grade 4. Additionally, the equivalency of grades was extended to grade 3, and therefore in the final proposal of EASA, grades 1 to 3 mean the same in both grading systems, while grade 4 in the alternative grading scale includes 4 and 5 in the standard EASA grading scale. AMC1 ORO.FC.231(d)(1) point (b) This provision intends to ensure that the operator develops guidance for its instructors. AMC3 ORO.FC.231(d)(1) Evidence-based training CONDUCT OF THE GRADING — ORCA (a) Grading the performance of flight crew members during an EBT module should include the following steps: (1) Observe performance (behaviours) during the simulator session. (2) Record details of effective and ineffective performance (behaviours) observed during the simulator session (‘record’ in this context refers to instructors taking notes). (3) Classify observations against the OBs and allocate the OBs to each competency (or competencies), using amongst others the facilitation technique. (4) Assess and evaluate (grade): assess the performance by determining the root cause(s) according to the competency framework. Low performance would normally indicate the Page 96 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail area of performance to be remediated in subsequent phases or modules. Evaluate (grade) the performance by determining a grade for each competency using a methodology defined by the operator. (b) As a minimum, the instructor should grade all the observed competencies at: (1) the end of the evaluation phase (de-briefing) by providing at least level 1 grading metrics; (2) the end of the manoeuvres training phase (de-briefing) by providing at least level 1 grading metrics; and (3) at the end of the EBT module (de-briefing) by providing at least level 0 grading metrics (level 1 grading metrics are recommended). GM1 ORO.FC.231(d)(1) Evidence-based training CONDUCT OF THE GRADING — ORCA (a) At the end of the evaluation phase, after the facilitated de-briefing, the instructor may, as a minimum, record level 1 grading metrics. (b) The instructor may conduct the simulator session of the EVAL following the principles of a summative assessment and the facilitated de-briefing following the principles of a formative assessment. The MT and SBT simulator session may be conducted as a formative assessment. (c) At the end of each training phase, it is recommended to record level 1 grading metrics unless just culture and the necessary non-jeopardy environment during training may be compromised. In that case, level 0 grading metrics for all competencies may be recorded (exceptionally ‘not observed’ or ‘left in blank’ may be recorded) and level 1 grading metrics may be recorded and de-identified for the data collection and analysis purposes. AMC3 ORO.FC.231(d)(1) This provision was inspired by the IATA Evidence-Based Training Implementation Guide, Chapter 6.6. ‘6.6 TECHNIQUES TO BE APPLIED IN GRADING Assessment is a continuous process throughout all training phases. It is the process of observing, recording, analyzing and determining crew performance against defined expectations in the context of overall performance. It includes the concept of self-critique and feedback, which can be given during training, or in summary thereafter.’ Furthermore, this technique (observe, record, classify and assess/evaluate) is widely used in the competency-based interview in the domain of human resources. An example can be found in the book ‘assessment methods in recruitment, selection and performance’ by Robert Edenborough (2005). In this context and according to the author, the process is necessary in a competency-based assessment as ‘It identifies a stepwise process that prevents a too-rapid arrival at conclusions, which is the case if such a structure is not followed.’ Page 97 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC3 ORO.FC.231(d)(1) point (b) There is a need to ensure a level playing field. Therefore, EASA decided to have a standard approach to grading. This is supported in the original idea of EBT as evidenced in the IATA Implementation Guide Appendix D where the crew is graded on both days. The fact that the EBT instructor is grading the performance of the pilot in the EVAL and SBT does not mean that this grading is accessible to everybody: — From a ‘training system performance’ point of view, this information is needed to demographically assess the level of performance of the pilot community before the module. — From a ‘nominated person flight OPS’ point of view, the information needed is whether the pilot is competent or not competent to conduct line operations. This applies to both days; otherwise, the pilot shall not fly. AMC4 ORO.FC.231(d)(1) Evidence-based training RECOMMENDED GRADING SYSTEM METHODOLOGY — VENN MODEL (a) To grade a competency, the instructor should assess the associated OBs of each competency against the following dimensions by determining: (1) what was the outcome of the threat management, error management and undesired aircraft state management relating specifically to the competency being assessed; (2) how many OBs the flight crew member demonstrated over the phase when they were required; (3) how often the flight crew member demonstrated the OB(s) when they were required; and (4) how well the flight crew member demonstrated the OB(s) when they were required. Abbreviated word picture VENN model Grading OUTCOME (1) HOW MANY (2) HOW OFTEN (3) HOW WELL(4) 1 unsafe situation few, hardly any rarely ineffectively 2 not an unsafe situation some occasionally minimally acceptable 3 safe situation many regularly adequately 4 safe situation most, almost all regularly effectively 5 enhanced safety, effectiveness and efficiency all always in an exemplary manner (b) Grades should be determined during each EBT module as follows: (1) Evaluation phase (EVAL) — overall performance of the phase at level 1 grading metrics. (2) Manoeuvres training (MT) — overall performance of the phase at level 1 grading metrics. Note: Only a limited number of competencies may be observed and graded in this phase (e.g. PRO, FPA, FPM); the others are ‘to be left in blank’. (3) Scenario-based training phase (SBT) — overall performance of the phase at level 1 grading metrics. Unless just culture and the necessary non-jeopardy environment during training may be compromised. In that case, level 0 grading metrics. Note: In-seat instruction (ISI) should not be included in any assessment. Page 98 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (c) Where any competency is graded below the minimum acceptable level of performance (grade 1 on a 5-point scale), an outcome of additional FSTD training is required. (1) Additional level 2 grading metrics must be recorded. (2) The flight crew member may not be released to unsupervised line operations until each competency is demonstrated at or above the minimum acceptable level of performance. (d) Where all competencies are determined at or above the minimum acceptable level of performance, (grade 2 on a 5-point scale) the outcome should be COMPETENT. Consistent grading below the average (2 on a 5-point scale) may indicate a need for training to elevate the performance to the average (grade 3 on a 5-point scale) as follows: (1) Any competency graded with 2 requires level 2 grading metrics. (2) Any competency graded with 2 in two consecutive simulator sessions of different recurrent modules requires individual tailored training in the SBT of the second module. (e.g. 1st Module SBT graded with 2, 2nd Module EVAL graded with 2, thus the 2nd SBT should be an individual tailored training, or 1st Module MT graded with 2 in FPA, in the 2nd MT Module the same competency is graded with 2, the SBT should be an individual tailored training focusing on FPA). (3) Any competency graded with 2 in three consecutive modules requires individual tailored training. If at the end of the tailored training (3rd SBT) the competency continues being graded with 2, additional FSTD training is required within the next 3 months. For instance, following the example above, the SBT in the 2nd Module was an individual tailored training. In the 3rd Module during the EVAL the same competency is graded with 2 and individual tailored training is applied. The SBT is graded with 2 again. The pilot may continue line operations but should receive additional FSTD training within the next 3 months. (4) The operator should not release a flight crew member to unsupervised line operations when more than four competencies (the majority of the competencies — five competencies or above) are graded with 2 in the module. (5) Any EVAL graded with 2 in more than three competencies requires individual tailored training in the SBT. If at the end of the module more than three competencies continue being graded with 2, the pilot may continue line operations but should receive additional FSTD training within the next 3 months. (e) ‘Individual tailored training’ refers to a simulator session tailored to the pilot’s individual training needs, which may require a different programme or syllabus. Normally, it may be done during the SBT and normally there is not an increase of FSTD volume (no extra simulator session). It may require an increased volume of training such as CBT, additional briefings, etc. (f) ‘Additional FSTD training’ refers to the fact that in addition to the requirements of tailored training, there is an increase of FSTD volume (extra simulator session). It normally happens after tailored training. Page 99 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC4 ORO.FC.231(d)(1) Most of the provisions included in this AMC were transposed from the GM that referred to VENN as proposed in the NPA. The upgrade from GM to AMC was suggested in some comments and decided by the review group in June 2019. For the column related to ‘how many’ (2), there was a consensus to understand: 1- ‘few, hardly any’ as few steps above 0 %, 2- ‘many’ as a majority but closer to 50 %, 3- ‘some’ in between many and most, 4- ‘most, almost all’ as a large majority closed to 100 % but not quite, and 5- ‘all’ as a 100 %. AMC4 ORO.FC.231(d)(1) point (d) The provisions of EBT regarding grading are more detailed than those provided for legacy training in ORO.FC.230 where there is no definition of what training may be required after LPC failure or OPC failure. Remediation may include FSTD training, line flying under supervision (LIFUS), or something else depending on the circumstances (e.g. virtual-reality training). GM2 ORO.FC.231(d)(1) Evidence-based training RECOMMENDED GRADING SYSTEM METHODOLOGY — VENN MODEL (a) Grades may be determined during each EBT module as follows: (1) For each assigned grade: (i) the observed performance should be identified with one or more OBs; and (ii) the OB(s) should simply link the observed performance to the competency; they are not to be used as a checklist. (2) At the completion of the EVAL, the grade should be assigned for each competency, based on the overall assessment of the performance during the EVAL. (3) At the completion of the MT, only a limited number of competencies can be graded. The others are to be left in blank. Note: The grade of a competency as ‘not observed’ is a relevant set of data to be used in the EBT programme (e.g. may be used for instructor concordance assurance programme, programme design, etc.), while ‘competency left in blank’ is stating the obvious, which is that MT is an skill retention phase and therefore it focuses on only some of the competencies which may provide NO opportunity to observe all the competencies. (4) At the completion of the module, grades should be assigned for each competency, based on the overall assessment of training during the SBT. (5) In exceptional occasions, the instructor may have been unable to assess one or two competencies in the EVAL or SBT. A ‘not observed’ may be graded. The training system performance and concordance assurance system may use these metrics to improve instructors’ standardisation and the EBT programme design. When the operator grades the MT alone (instead of grading the MT and EVAL together), a ‘not observed’ grading may Page 100 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail be frequent. It also occurs when the instructor grades each one of the manoeuvres. (6) (b) The word pictures are standardised according to the VENN model but may be simplified once instructors become familiar with the system. Word picture VENN model Application of procedures (PRO) The pilot applied procedures in an exemplary manner, by always demonstrating all of the observable 5 behaviours to a high standard when required, which enhanced safety, effectiveness and efficiency The pilot applied procedures effectively, by regularly demonstrating all of the observable behaviours 4 when required, which resulted in a safe operation The pilot applied procedures adequately, by regularly demonstrating most of the observable behaviours 3 when required, which resulted in a safe operation The pilot applied procedures at the minimum acceptable level, by only occasionally demonstrating some 2 of the observable behaviours when required, but which did not result in an unsafe situation The pilot applied procedures incorrectly, by rarely demonstrating any of the observable behaviours 1 when required, which resulted in an unsafe situation Communication (COM) The pilot communicated in an exemplary manner, by always demonstrating all of the observable 5 behaviours to a high standard when required, which enhanced safety, effectiveness and efficiency The pilot communicated effectively, by regularly demonstrating all of the observable behaviours when 4 required, which resulted in a safe operation The pilot communicated adequately, by regularly demonstrating most of the observable behaviours 3 when required, which resulted in a safe operation The pilot communicated at the minimum acceptable level, by only occasionally demonstrating some of 2 the observable behaviours when required, but which overall did not result in an unsafe situation The pilot communicated ineffectively, by rarely demonstrating any of the observable behaviours when 1 required, which resulted in an unsafe situation Flight path management — automation (FPA) The pilot managed the automation in an exemplary manner, by always demonstrating all of the 5 observable behaviours to a high standard when required, which enhanced safety, effectiveness and efficiency The pilot managed the automation effectively, by regularly demonstrating all of the observable 4 behaviours when required, which resulted in a safe operation The pilot managed the automation adequately, by regularly demonstrating most of the observable 3 behaviours when required, which resulted in a safe operation Page 101 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The pilot managed the automation at the minimum acceptable level, by only occasionally 2 demonstrating some of the observable behaviours when required, but which did not result in an unsafe situation The pilot managed the automation ineffectively, by rarely demonstrating any of the observable 1 behaviours when required, which resulted in an unsafe situation Flight path management — manual control (FPM) The pilot controlled the aircraft in an exemplary manner, by always demonstrating all of the observable 5 behaviours to a high standard when required, which enhanced safety, effectiveness and efficiency The pilot controlled the aircraft effectively, by regularly demonstrating all of the observable behaviours 4 when required, which resulted in a safe operation The pilot controlled the aircraft adequately, by regularly demonstrating most of the observable 3 behaviours when required, which resulted in a safe operation The pilot controlled the aircraft at the minimum acceptable level, by only occasionally demonstrating 2 some of the observable behaviours when required, but which did not result in an unsafe situation The pilot controlled the aircraft ineffectively, by rarely demonstrating any of the observable behaviours 1 when required, which resulted in an unsafe situation Application of knowledge (KNO) The pilot showed exemplary knowledge, by always demonstrating all of the observable behaviours to a 5 high standard when required, which significantly safety, effectiveness and efficiency The pilot showed adequate knowledge, by regularly demonstrating all of the observable behaviours 4 when required, which resulted in a safe operation The pilot showed adequate knowledge, by regularly demonstrating most of the observable behaviours 3 when required, which resulted in a safe operation The pilot showed knowledge to a minimum acceptable level, by only occasionally demonstrating some 2 of the observable behaviours when required, but which did not result in an unsafe situation The pilot showed inadequate knowledge, by rarely demonstrating any of the observable behaviours 1 when required, which resulted in an unsafe situation Leadership & teamwork (LTW) The pilot led and worked as a team member in an exemplary manner, by always demonstrating all of 5 the observable behaviours to a high standard when required, which significantly enhanced safety, effectiveness and efficiency The pilot led and worked as a team member effectively, by regularly demonstrating all of the observable 4 behaviours when required, which resulted in a safe operation The pilot led and worked as a team member adequately, by regularly demonstrating most of the 3 observable behaviours when required, which resulted in a safe operation The pilot led and worked as a team member at the minimum acceptable level, by only occasionally 2 demonstrating some of the observable behaviours when required, but which did not result in an unsafe situation The pilot led or worked as a team member ineffectively, by rarely demonstrating any of the observable 1 behaviours when required, which resulted in an unsafe situation Page 102 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Problem-solving & decision-making (PSD) The pilot solved problems and made decisions in an exemplary manner, by always demonstrating all of 5 the observable behaviours to a high standard when required, which enhanced safety, effectiveness and efficiency The pilot solved problems and made decisions effectively, by regularly demonstrating all of the 4 observable behaviours when required, which resulted in a safe operation The pilot solved problems and made decisions adequately, by regularly demonstrating most of the 3 observable behaviours when required, which resulted in a safe operation The pilot solved problems and made decisions at the minimum acceptable level, by only occasionally 2 demonstrating some of the observable behaviours when required, but which did not result in an unsafe situation The pilot solved problems or made decisions ineffectively, by rarely demonstrating any of the 1 observable behaviours when required, which resulted in an unsafe situation Situation awareness (SAW) The pilot’s situation awareness was exemplary, by always demonstrating all of the observable 5 behaviours to a high standard when required, which enhanced safety, effectiveness and efficiency The pilot’s situation awareness was good, by regularly demonstrating all of the observable behaviours 4 when required, which resulted in a safe operation The pilot’s situation awareness was adequate, by regularly demonstrating most of the observable 3 behaviours when required, which resulted in a safe operation The pilot’s situation awareness was at the minimum acceptable level, by only occasionally 2 demonstrating some of the observable behaviours when required, but which did not result in an unsafe situation The pilot’s situation awareness was inadequate, by rarely demonstrating any of the observable 1 behaviours when required, which resulted in an unsafe situation Workload management (WLM) The pilot managed the workload in an exemplary manner, by always demonstrating all of the observable 5 behaviours to a high standard when required, which significantly enhanced safety, effectiveness and efficiency The pilot managed the workload effectively, by regularly demonstrating all of the observable behaviours 4 when required, which resulted in a safe operation The pilot managed the workload adequately, by regularly demonstrating most of the observable 3 behaviours when required, which resulted in a safe operation The pilot managed the workload at the minimum acceptable level, by only occasionally demonstrating 2 some of the observable behaviours when required, but which did not result in an unsafe situation The pilot managed the workload ineffectively, by rarely demonstrating any of the observable behaviours 1 when required, which resulted in an unsafe situation Page 103 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Safety promotion material — Grading system EASA has planned a safety promotion task (SPT.012) to support the implementation of EBT. The following material has been developed: ‘SPT.012 — Safety promotion to ORO.FC.231(d) Grading system GUIDANCE TO THE GRADING SYSTEM FOR THE INSTRUCTORS TO DETERMINE THE GRADE OF THE PILOTS Although the regulation may provide enough material to develop a grading system, the operator is required to provide the instructors with further guidance to improve grading and instructor concordance. Grading should look at the entire simulator session (the whole scenario), and not only at a particular scenario element (e.g. one manoeuvre or a small scenario within the simulator). Sometimes raters (instructors) tend to decide the grading of the simulator session based on one manoeuvre when the grading in EBT should look for the global assessment instead. An example of the kind of material the operator should develop is provided below. The example is based on the threat and error management (TEM) model. The instructor may run this model first and then proceed with the grading following the word picture. The model focuses on the determination of grading 1, 2 and what is above 2. It may not help to determine what is the actual grading of 3, 4, or 5: — Grading of a non-intentional non-compliance (an undetected error or mistake but corrected in a timely manner with a safe outcome). There are two types: — Non-intentional non-compliance without consequences — trap error For instance, there is a mistake on the altitude selection, the crew are busy with other tasks and do not recognise the mistake when it occurs, and therefore they do not call ATC to confirm the altitude clearance. However, later and before the level bust, they initiate a dialogue and realise their possible mistake, or one of the pilots due to their awareness of the route altitudes realises a possible mistake which triggers a call to ATC to confirm the altitude and fix the error. For the competency identified as the root cause, the grade will not be 1. The example provides a reference to grade 3; it may be graded 2 or 4 depending on the rest of the simulator session. The competency will not probably be graded with 5. It should be noted that this does not prevent that other competencies could be graded with 5 based on the evidence of this particular scenario element. PRO may not be graded with 5 because the pilot did not confirm with ATC when in doubt of an altitude clearance as required by the SOPs; however, they may be graded with 5 in SWA because they realised that the altitude selected did not make sense with the safe altitude for the route. — Non-intentional non-compliance with consequences (undesired aircraft state associated with a reduction in margins of safety5) but managed by the flight crew successfully (flight 5 See ICAO Doc 9868 ‘PANS-TRG’Chapter 6.7 Undesired aircraft states, point 6.7.1 ‘Undesired aircraft state are characterized by divergences from parameters normally experienced during operations (e.g. aircraft position or speed Page 104 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail crew timely switched from error management to undesired aircraft state management). Therefore, the consequences were mitigated in a timely manner (e.g. mistake in the altitude selection followed by a level bust resolved by a call of ATC or a TCAS flown to a good standard, GPWS warning followed by an escape manoeuvre performed to a good standard, etc.). For the competency identified as the root cause, the grade should not be 1 or 5. The most probably grading reference is 2 because: — The outcome of the situation was not unsafe (therefore, it cannot be 1). Additionally, the instructor should also ask themselves if the crew managed all the situations successfully in all of the events during the simulator session (to look for the big picture). If this was the case, then the instructor knows that grading with 1 is not possible (the outcome was NOT unsafe) and therefore the instructor is restricted to 4 possible gradings (2, 3, 4 or 5). Then the instructor will mentally move to the next step below. — Was there a reduction in margins of safety? Yes, as in this example the pilots reach an undesirable aircraft state (therefore, it cannot be 5). At this moment in the process, the instructor knows the grading can be neither 1 nor 5 and will move to the next step below. — How big was the reduction of the safety margin? Normally, grading 4 is unlikely. At this stage in the process, it will depend on the context of the situation (how dangerous was the situation?). Normally, entering in a dangerous undesired aircraft state means that some of the OBs were not demonstrated effectively; therefore, grading 4 may not be possible as grade 4 requires ‘almost all’ OBs to be demonstrated effectively (see VENN table). Therefore, at this point in the process, the instructor also knows that the grading cannot be 4. Then, the instructor will mentally move on to the next step below. — In this step, the instructor will ask themselves how well the flight crew managed the situation (and the other situations in the simulator sessions). Once more, the grading should look for the global picture — that is why it is relevant to assess how well the flight crew resolved the situation of the example but also other situations in the simulator session. How well was the situation resolved? Did the crew timely switch from error management to undesired aircraft state management? Did the crew perform the best possible escape manoeuvre and to a good standard? Depending on the rest of the simulator session, the instructor could grade 3 if the pilot/crew managed the other events in the simulator session in the best possible way and to a good standard. Otherwise, the grading will be 2. — Intentional non-compliance but recognised and corrected in a timely manner with a safe outcome (e.g. unestablished approach followed by a go-around well below the stabilised gate). deviations, misapplications of flight controls, or incorrect systems configurations) associated with a reduction in margins of safety’ ‘undesired aircraft states must be managed by flight crews’ Page 105 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The instructor should go through the mental process described above. A summary is provided below: For the competency identified as root cause, the probable grading (reference grading) for the simulator session will be 2 and the maximum grading may be 3. 2 is the probable grading because the situation was not unsafe as the pilot executed a go-around, but the pilot did it well below the stabilised gate (e.g. 100 feet) — meaning the reduction in the safety margins was big. Obviously, the situation cannot be considered safe because the pilot should execute the go- around no later than the stabilised gate (1 000 feet or about 500 feet as per the operator’s policy). It should never be 4 or 5. It may trigger grade 1 depending on the other exercises. Note: This guidance to grading is not to be used when there is a non-compliance because a higher degree of safety dictates otherwise. In addition to the standard examples, there may be other examples for which the operator may need to decide if a higher degree of safety dictates otherwise. For example, the captain decides to take 15 seconds to refresh quickly the go-around actions and warn the first officer to be ready below 500 feet. Another example is when the crew miss the touch down zone for a bit in a long runway and decide to land instead of going around due to weather in the go around area). — Intentional non-compliance not corrected and continued to the end state (e.g. unestablished approach and maintained until landing) In this example, the competency identified as the root cause should be graded 1 (failed), and the probable root cause is PRO. Furthermore, no other competency of the pilot can be graded with 5. Note: This guidance is not to be used when there is a non-compliance because a higher degree of safety dictates otherwise (e.g. unestablished approach maintained until landing due to uncontained fire or all engines flame out, etc.). The grading should as much as possible assess ‘what has happened (be objective) and not what would have happened.’ According to ICAO Doc 9868 ‘PANS-TRG’ (State letter 18-77e) point 6.7.3 ‘undesired states can be managed effectively, restoring margins of safety, or can induce an additional error, leading to an incident, or accident.’ Undesired aircraft states and outcomes. ‘Undesired aircraft states are transitional states between a normal operational state (i.e., a stabilised approach) and an outcome. Outcomes, on the other hand, are end states, most notably, reportable safety occurrences.’ (source: skybrary.aero) ‘SPT.012 — Safety promotion to ORO.FC.231(d) Grading system and ORO.FC.146 EASA identified that during the early implementation of mixed EBT, the competencies ‘application of procedures’ (PRO) and/or ‘application of knowledge’ (KNO) have normally the lowest grading in the airline. The EBT manager should determine whether this information is genuine or not. For that purpose, Line Operations Safety Audit (LOSA), consultancy and other tools available in the industry could guide the EBT manager in determining the accuracy (veracity) of the grading results (PRO and/or KNO lowest grading). If the airline determines its pilots have high standards in PRO and/or KNO, then there may be two possibilities to explain the lower grading in PRO and/or KNO: Page 106 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail — The instructors have a wrong understanding of the OBs and grading provisions. Normally, this is not the case, as the instructors have recently received the EBT instructor course (this text pertains to new implementations of mixed EBT). The EBT manager should verify whether the instructors clearly understand the guidelines for grading provided by the airline. — The instructors are identifying the ‘training needs’ wrongly. This mistake is common in some of the airlines starting mixed EBT. Why do the instructors tend to grade PRO or KNO lower than the rest of competencies? — Every competency is constructed with knowledge, skills and attitudes. Knowledge is, therefore, an element of every competency; this confuses instructors, and if they instructor are not careful, they will grade KNO or PRO lower than they should. Example: due to distractions, a captain forgets to put the anti-ice system several times. The instructor should normally identify the root cause as ‘workload management’ (WLM), or if the pilot was flying manually, it might be a problem of ‘flight path management — manual control’ (FPM) or both competencies (see ‘distraction’ in the OBs). However, when the instructor is new to EBT, they may give a lower grade to PRO instead. Why? Because in the traditional system, the instructor should probably give a lower grade to PRO. This is usually not correct in EBT (that is why the facilitation debriefing is so important: to understand why the pilot forgot the anti-ice system). Note: If the pilot did not know the procedure, giving a lower grade to PRO is correct. — As explained in NPA 2018-07(B) and Opinion No 08/2019, the competencies are linked. For instance, a pilot should first have FPA and/or FPM to build the competency of WLM. Then the pilot can have ‘thinking time’ and construct ‘problem-solving and decision- making’ (PSD). Therefore, the instructor always has in mind the argument of referring back to KNO or PRO as they usually are at the basis of the competency pyramid. The instructor should know where to stop the root cause analysis; otherwise, all problems would be attributed to KNO.’ AMC4 ORO.FC.231(d)(2) and GM2 ORO.FC.231(d)(1) Assessment and grading form an integral part of the learning process. As part of the development of EBT as a new approach to competency-based training, a pilot performance assessment and grading system is required to address the fundamental shift from previous systems which are ‘event-based’ and require the assessment of the quality of the outcome of a manoeuvre or the management of the event or threat. In certain previous systems, behavioural markers or competencies were used as assessment tools or reason codes for the outcome of the manoeuvre or of the management of the event or threat. The paradigm shift in EBT is to focus the attention to the underlying areas of flight crew member performance to determine training needs or focus. EBT is a system designed to determine areas of focus for all flight crew members and not just those whose performance is observed below a minimum acceptable level. The system is intended to fulfil the needs of operators and was created according to a structured design process. Rationale The assessment and grading system should meet the needs of the following stakeholders. Page 107 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail — Civil aviation authority (CAA) — performance of assessments for the revalidation and renewal of flight crew licences and/or ratings — Operator — measurement of individual, crew, fleet and operator pilot performance and identification of development needs for both individuals and the system — Flight crew member — provision of information about performance measured during training, for the purpose of continuous development and improvement The system was created considering the importance of a number of design criteria. After wide consultation, criteria were considered as follows: Fairness and accuracy, clarity, usability, ease of compliance, continuous improvement, motivation, data management, adaptability, implementation risk Following the criteria definition, the development process was segregated in the following steps, with agreed criteria being applied at each step to determine the optimum solution: — System definition (what to grade: the whole event, parts of the event, individual actions or a combination with different granularity): to be consistent with the aims of EBT, it should be the competencies at predetermined points during the module. — Grading scales (considering sensitivity and the need to identify unacceptable, minimum acceptable, norm and performance above the average): a 5-point scale is commonly used with grade 1 indicating unacceptable performance, the average being grade 3; grade 2 indicates the minimum acceptable performance, and 4 and 5 indicate performance above average. There are many arguments for and against the number of points on a scale and this should be finally determined by the operator and approved by the competent authority under the operations manual part D. — Word pictures: to assure the fulfilment of the criteria, in particular, fairness, accuracy and clarity, grades are described by standardised word pictures. They describe the VENN dimensions in a standardised way, and this facilitates inter-rater reliability. The VENN model described in this GM is based on the following measurements at predetermined points during an EBT module: A= HOW WELL (e.g. The pilot communicated ineffectively…) B= HOW OFTEN (e.g. …by rarely demonstrating…) C= HOW MANY (e.g. … any of the performance indicators when required…) D= OUTCOME (e.g. … which resulted in an unsafe situation). In order to ensure consistency, a grading system should also be employed for the line evaluation of competence, with information provided for remediation where performance is determined to be below the minimum acceptable level, which in the example system is 1 on a 5-point scale. AMC1 ORO.FC.231(d)(2) Evidence-based training VERIFICATION OF THE ACCURACY OF THE GRADING SYSTEM (a) The purpose is to provide data to assess the accuracy of the grading system. Page 108 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (b) The items defined below are based on Part-FCL Appendix 9. They should be included in the evaluation and manoeuvres training phase of the applicable module. The minimum items to be included are: rejected take-off, failure of critical engine between V1 & V2, adherence to departure and arrival, 3D approaches down to a decision height (DH) not less than 60 m (200 ft), engine-out approach & go-around, 2D approach down to the MDH/A, engine-out approach & go-around, engine-out landing. (c) Instructors should record if the exercises are flown to proficiency using Appendix 9 references (define criteria). Note: Individual pilots’ grading and assessment remains according to the EBT grading system and Appendix 10. (d) This verification should be performed once every 3 years. Page 109 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail GM1 ORO.FC.231(d)(2) Evidence-based training VERIFICATION OF THE ACCURACY OF THE GRADING SYSTEM Items that may be included in a verification of the accuracy of the grading system: Flight phase for Assessment Desired outcome activation Description (includes type of (includes performance criteria OR training outcome) and training Guidance material (GM) topic topic, being threat, error or Example scenario elements WLM COM SAW FPM KNO LTW APK PSD FPA focus) Use of checklist prior to starting Use of checklist engines, starting procedures, prior to starting GND radio and navigation equipment This element is not required Intentionally left in blank Intentionally left in blank engines (1.4 check, selection and setting of AP9) navigation and communication frequencies Before take-off GND This element is not required Intentionally left in blank Intentionally left in blank checks (1.6 AP9) APK - demonstrate adequate knowledge of the technique and procedure for accomplishing a rejected take-off after power-plant/system(s) failure/warnings, including related safety factors; - take into account, prior to beginning the take-off, operational factors which could affect the manoeuvre, such as take-off warning inhibit systems or other aeroplane characteristics, runway length, surface conditions, wind, Engine failure after the application obstructions that could affect take-off performance and could adversely affect safety; From initiation of take-off to Rejected take- - perform all required pre-take-off checks as required by the appropriate checklist items; TO of take-off thrust and before complete stop (or as applicable to x x off (2.6 AP9) FPM reaching V1 - align the aeroplane on the runway centreline; procedure) - reduce the power smoothly and promptly, if appropriate to the aeroplane, when power-plant failure is recognised. Maintain the aeroplane under control close to the runway centreline; - use spoilers, prop reverse, thrust reverse, wheel brakes, and other drag/braking devices, as appropriate, maintaining positive control in such a manner as to bring the aeroplane to a safe stop. Accomplish the appropriate power-plant failure or other procedures and/or checklists as set forth in the POH or AFM or SOP 3.8.1* Adherence to departure and CLB This element is not required Intentionally left in blank arrival routes and APP ATC instructions Page 110 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail FPM The manoeuvre is considered to - establish a bank of approximately 5°, if required, or as recommended by the manufacturer, to maintain be complete at a point when the coordinated flight, and properly trim for that condition; maintain the operating engine within acceptable operating aircraft is stabilised at normal limits; engine-out climb speed with the x x - establish the best engine inoperative airspeed as appropriate to the aircraft and condition of flight; correct pitch and lateral control, - establish and maintain the recommended flight attitude and configuration for the best performance for all Failure of the manoeuvring necessary for the phase of flight; in trim condition and, as Failure of the critical engine from critical engine - maintain desired altitude within given limits, when a constant altitude is specified and is within the capability of applicable, autopilot engagement V1 and before reaching V2 in the between V1 & V2 TO the aeroplane; lowest CAT I visibility conditions - maintain the desired airspeed and heading within given limits; (2.5.2 AP9)) The manoeuvre is considered to APK - recognise an engine failure or the need to shut down an engine as simulated by the examiner; be complete at a point when the - complete engine failure vital action checks from memory; aircraft is stabilised in a clean x x - follow the prescribed aeroplane checklist, and verify the procedures for securing the inoperative engine; configuration with engine-out - demonstrate proper engine restart or shutdown procedures (whatever appropriate) in accordance with approved procedures completed procedure/checklist or the manufacturer’s recommended procedures and pertinent checklist items; and monitor all functions of the operating engine and make necessary adjustments. Manually, with one engine APK simulated inoperative; engine - select and comply with the ILS or LPV instrument approach procedure to be performed; failure has to be simulated - prior to final approach course, maintain declared or assigned altitudes within given limits without descending during final approach before below applicable minimum altitudes and maintain headings within given limits; passing 1 000 ft above - select, tune, identify and confirm the operational status of ground and aircraft navigation equipment to be used aerodrome level until for the approach procedure; touchdown or through the COM complete missed approach - establish two-way communications with ATC using the proper communications phraseology and techniques, procedure. either personally, or, if appropriate, direct co-pilot/safety pilot to do so, as required for the phase of flight or or approach segment; - comply in a timely manner with all clearances, instructions, and procedures issued by ATC and advise accordingly if unable to comply; 3.8.3* 3D FPA/FPM: operations to - establish the appropriate aircraft configuration and airspeed/V-speed considering turbulence, wind shear or other DH/A of 200ft (60 meteorological and operating conditions; m) or to higher APP - complete the aircraft check list items appropriate to the phase of flight or approach segment, including engine out Intentionally left in blank Intentionally left in blank minima if required approach and landing checklist, as appropriate; by the approach Manually, with one engine - apply necessary adjustment to the published DH and visibility criteria for the aeroplane approach category when simulated inoperative; engine required, such as NOTAMs, inoperative aeroplane and ground navigation equipment, inoperative visual aids procedure failure has to be simulated associated with the landing environment; during final approach after - on final approach course, allow no more than ½ scale deflection of the localiser and/or glideslope indications; passing the outer marker (OM) - maintain declared approach airspeeds within given limits; within a distance of not more - maintain a stabilised descent to the DH to permit completion of the visual portion of the approach and landing than 4 NM until touchdown or with minimal manoeuvring; and through the complete missed - initiate the missed approach procedure, upon reaching the DH, when the required visual references for the approach procedure. intended runway are not obtained. 3D linear vertical deviations (e.g. RNP APCH (LNAV/VNAV) using BaroVNAV): not more than – 75 ft below the vertical profile at any time, and not more than + 75 ft above the vertical profile at or below 1 000 ft above aerodrome level. 3D (LNAV/VNAV) ‘linear’ lateral deviations: cross-track error/deviation shall normally be limited to ± ½ the RNP value associated with the procedure. Brief deviations from this standard up to a maximum of 1 time the RNP value are allowable. Page 111 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail APK: - select and comply with the PBN, VOR/ LOC/ LOC BC or NDB instrument approach procedure to be performed; - complete the aircraft check list items appropriate to the phase of flight or approach segment, including engine out approach and landing checklist, as appropriate; - prior to final approach course, maintain declared altitudes in given limits without descending below applicable minimum altitudes, and maintain headings as given; - select, tune, identify, confirm and monitor the operational status of ground and aircraft navigation equipment to be used for the approach procedure; COM: - establish two-way communications with ATC using the proper communications phraseology and techniques, either personally, or, if appropriate, directs co-pilot/safety pilot to do so, as required for the phase of flight or approach segment; comply in a timely manner with all clearances, instructions, and procedures issued by ATC and advise accordingly if unable to comply; 2D operations Non-precision approach down FPA/FPM: down to the - apply necessary adjustment to the published minimum descent altitude (MDA) and visibility criteria for the to the MDH/A Intentionally left in blank Intentionally left in blank MDH/A APP aeroplane approach category when required, such as NOTAMs, inoperative aeroplane and ground navigation equipment, inoperative visual aids associated with the landing environment; (3.8.4 AP9) - on the intermediate and final segments of the final approach course: a. maintain PBN, VOR/ LOC/ LOC BC tracking within ½ scale deflection of the course deviation indicator or within 5 degrees of the desired track in the case of an NDB approach; b. fly the approach in a stabilised manner without descending below the applicable minimum altitudes depicted on the approach chart (+as required/–0 feet); 2D (LNAV) ‘linear’ lateral deviations: cross-track error/deviation shall normally be limited to ± ½ the RNP value associated with the procedure. Brief deviations from this standard up to a maximum of 1 time the RNP value are allowable. c. descend to and accurately maintain the MDA and track to the missed approach point (MAPt) or to the recommended minimum visibility that would permit completion of the visual portion of the approach with a normal rate of descent and minimal manoeuvring; d. maintain declared approach airspeeds (+10/-5 knots); e. initiate the missed approach procedure, if the required visual references for the intended runway are not obtained at the MAP ; f. execute a normal landing from a straight-in or circling approach as required. This manoeuvre should be flown from intercept to centreline until acceleration after go-around. The manoeuvre is Manual go-around with the Engine-out considered to be complete at a point when critical engine simulated approach & go- Demonstrate manual aircraft control skills with smoothness and accuracy as appropriate to the situation the aircraft is stabilised at normal engine- x x APP inoperative after an instrument around Detect deviations through instrument scanning out climb speed with the correct pitch and approach on reaching DH, MDH (4.4* AP9) Maintain spare mental capacity during manual aircraft control lateral control, in trim condition and, as or MAPt Maintain the aircraft within the flight envelope applicable, autopilot engagement Apply knowledge of the relationship between aircraft attitude, speed and thrust (describe generally critical part of manoeuvre) Initiation in a stabilised engine-out Engine-out LDG With the critical engine configuration from not less than 3 NM x x landing (5.5 AP9) inoperative final approach, until completion of roll-out Page 112 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail GM2 ORO.FC.231(d)(2) Evidence-based training VERIFICATION OF THE ACCURACY OF THE GRADING SYSTEM — FEEDBACK PROCESS (a) The verification of the accuracy of the grading system provides valuable data for the training system performance and concordance assurance. Therefore, the verification is necessary from a systemic point of view and the intention is not to measure individual pilot against Appendix 9 criteria. Concordance agreement between instructors may be high; however, the whole community of instructors may be grading too low or too high (accuracy). The statistical result of the verification against Appendix 9 criteria can provide the operator with a criterion-referenced system to adjust the accuracy of the grading system. The verification does not require an examiner, and EBT instructors may provide the necessary data. Example 1: For the last 36 months, the operator has a rate of 3 % of pilots scoring 1 (assuming data is statistically relevant). In this example, the rate of 3 % of the pilots scoring 1 is maintained across all the technical competencies. When the operator performs a verification, the rate of failure would have been only 0,5 %. This may indicate that instructors are rating too low in EBT and therefore some of the pilots scoring 1 should have been graded with a score higher than 1. This may be economically negative for the operator. On the other hand, it could be that the operators has decided to implement higher standards. Example 2: The operator has an EBT programme with a negligible rate of pilots scoring 1 and a 1 % of pilots scoring 2 in two consecutive recurrent modules. The verification of the technical competencies against Appendix 9 criteria provides a rate of 5 % failure. The nominated person for crew training should further investigate the reason behind this mismatch between EBT and Appendix 9 in the technical competencies. There may be factors influencing this mismatch (e.g. statistical issues, the events in the EBT modules are too benign compared to the Appendix 9), which may lead to a corrective action (e.g. redesign of the EBT modules). If the difficulty of the EBT scenarios is equivalent to Appendix 9 and the concordance is high between instructors, then the discrepancy in outcomes might be because the community of instructors are grading too high in the technical competencies (they are grading with 2 when they should have graded 1). Further instructor standardisation will be needed to address this. The implementation of mixed EBT following GM1 ORO.FC.230(a);(b);(f) provides a good opportunity to fine-tune and verify the accuracy of the grading system because an Appendix 9 licence proficiency check is carried out every year. The authority may not allow full EBT unless the accuracy of the grading system is demonstrated. More information in the EASA EBT manual. AMC1 ORO.FC.231(d)(2) and ORO.FC.231(d)(2) This Appendix to the Opinion already provides explanation about this topic in the explanatory note to ORO.FC.231. Page 113 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The concept of this provision is transposed from the ‘Alternative training and qualification programme’ (ATQP). However, to adapt the concept to EBT, the requirement suffered a complete shift. Background In ATQP, it is required to have a criterion-referenced system to be able to measure the effectiveness of the training programme (see explanation of ATQP below). This criterion-referenced system is set by the operator. A criterion-referenced system is set up by the regulator in the LPC. Appendix 9 defines a set of manoeuvres (mandatory manoeuvres) and a set of targets (see Appendix 9 ‘Conduct of the proficiency check — Flight tolerances’) which form a criterion-referenced system. ATQP also benefits from this criterion-referenced system of Appendix 9 because every year the ATQP pilots are required to complete an LPC (also see point (a)(6) of AMC1 ORO.FC.A.245 below). The method for the assessment in ATQP follows: 1. A task and subtask analysis of each event; 2. Each event has one or more specific training targets/objectives, which require the performance of a specific manoeuvre; 3. For each event, the proficiency that is required to be achieved should be established; 4. The conditions pertaining to each event should also be established; 5. Each event should include a range of circumstances under which the crews’ performance is to be measured and evaluated; 6. The behaviour marker must be specified; and 7. The operator should measure and monitor the progression, and target must be achieved. ‘AMC1 ORO.FC.A.245 Alternative training and qualification programme COMPONENTS AND IMPLEMENTATION (a) Alternative training and qualification programme (ATQP) components The ATQP should comprise the following: (…) (6) A method for the assessment of flight crew during conversion and recurrent training and checking. The assessment process should include event-based assessment as part of the LOE. The assessment method should comply with ORO.FC.230. (i) The qualification and checking programmes should include at least the following elements: (A) a specified structure; (B) elements to be tested/examined; (C) targets and/or standards to be attained; Page 114 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (D) the specified technical and procedural knowledge and skills, and behavioural markers to be exhibited. (ii) An LOE event should comprise tasks and sub-tasks performed by the crew under a specified set of conditions. Each event has one or more specific training targets/objectives, which require the performance of a specific manoeuvre, the application of procedures, or the opportunity to practise cognitive, communication or other complex skills. For each event the proficiency that is required to be achieved should be established. Each event should include a range of circumstances under which the crews’ performance is to be measured and evaluated. The conditions pertaining to each event should also be established and they may include the prevailing meteorological conditions (ceiling, visibility, wind, turbulence, etc.), the operational environment (navigation aid inoperable, etc.), and the operational contingencies (non-normal operation, etc.). (iii) The markers specified under the operator’s ATQP should form one of the core elements in determining the required qualification standard. A typical set of markers is shown in the table below: EVENT MARKER 1. Monitors and reports changes in automation status Awareness of 2. Applies closed loop principle in all relevant situations aeroplane systems: 3. Uses all channels for updates 4. Is aware of remaining technical resources (iv) The topics/targets integrated into the curriculum should be measurable and progression on any training/course is only allowed if the targets are fulfilled.’ — EBT For the measurement of pilot performance, Doc 9995 does not provide a full measurement system. Doc 9995 provides a set of OBs; however, it does not provide a grading system. This was resolved by the EBT subgroup RMT.0599 that provided a grading system (VENN) — included in this Appendix to the Opinion. This allowed a full measurement system for EBT. This system is more of a norm- referenced system than a criterion-referenced system. — Why EBT needs a norm-referenced system instead of a criterion-referenced system For many decades, the industry has used the completion of manoeuvres like rejected take-off, engine failure between V1 and V2, go-around from minima with the critical engine inoperative and a clearly defined flight tolerance (e.g. – 5knots/+10 knots) as a performance measurement to demonstrate the performance of the pilot. In this context, a pilot being able to demonstrate the ability to fly these often-repetitive manoeuvres within prescribed quantitative performance measurements and indicating an acceptable level of deviation from ideal criteria is deemed to be ‘competent’. Page 115 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail EBT is based on the premise that this concept is no longer appropriate as a simple indicator, due to the complexities of modern operations and automation systems, coupled with the significant attribution of serious incidents and accidents to human factors. The paradigm shift developed by EBT is that assessments, which are necessary during all forms of training and instruction, as well as evaluation and checking, should be determined according to the performance in the defined areas of competency, and not simply by the achievement of a predetermined outcome in a specific manoeuvre. The EBT concept continues to require the completion of certain tasks, but competent flight crew members should be able to complete the tasks reasonably expected of them under achievable conditions. Tasks remain important, but only in so much as they establish a predefined norm according to the curriculum, which in the case of recurrent EBT should be achieved. The key distinction is that EBT envisages a system of competence measurement, which looks at the total performance across a wide range of activities that include some traditional tasks. Another reason why EBT needs a norm-referenced system is the way EBT evaluates pilots. In the context of the traditional training and checking, pilots are checked; EBT instead assesses pilots. EBT moves away from assessment against the execution of predefined manoeuvres and tasks based on the quality of execution (ATQP and traditional training and checking), to a use of the events as a vehicle for developing and assessing crew performance across a range of competencies. EBT also refocuses the instructor population onto analysis of the root causes to correct inappropriate actions, rather than simply asking a flight crew member to repeat a manoeuvre with no real understanding as to why it was not successfully flown in the first instance. For those reasons, the EBT subgroup RMT.0599 provided a competency-based grading system closer to a norm-referenced grading system, rather than a criterion-referenced system. In other words, although the EBT grading system provides a standardised methodology to pilot assessment, it is by definition a norm-referenced grading system (events do not have a set of conditions and the OBs linked to the events do not have a defined and unambiguous criterion). While the criterion-referenced system unambiguously ascertains to what degree the objectives of the manoeuvres have been met, using such a system would mean that instructors would need to focus on the quality of execution of the manoeuvres rather than use the events as a vehicle to develop performance across a range of competencies. Note: A norm-referenced grading system is a type of assessment which yields an estimate of the position of the tested individual in a defined population. Note2: A criterion-referenced system is a type of assessment where the behavioural objectives and the systematic generation of test items are designed to unambiguously ascertain to what degree the objectives have been met. — LICENCE REVALIDATION This Opinion provides a set of rules to revalidate pilot licence under the EBT programme. A norm-referenced system is subject to a defined population; it is thus subject to the population of pilots of a particular operator. EASA, some authorities, and the pilots’ associations were concerned whether this would create a problem of level playing field for the licence revalidation. Page 116 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Please note that today licence revalidation provides a: — criterion-referenced system for the technical skills; and — norm-referenced system for the non-technical skills (e.g. CRM assessment). To resolve the issue, EASA launched a focused consultation6 in the 4th quarter of 2017 and the 1st quarter of 2018. The consultation concluded that a verification of the norm-referenced system was needed to re-assure the level playing field. (Further explanation is provided in the explanatory notes to ORO.FC.231(d) point (2), AMC1 ORO.FC.231(d)(2) and GM2 ORO.FC.231(d)(2)). In summary: — The EBT grading system is a norm-referenced grading system. Therefore, it varies from operator to operator and it depends on several factors, e.g. company standards, the design of the programme, culture of the organisations, culture of the instructors, etc. — Within an operator, a norm-referenced grading system varies throughout time. This happens because the EBT programme varies, the culture of the organisation varies, the culture of the instructors varies, the population of pilots changes, etc. Therefore, a norm-referenced grading system may provide different grading results for the same pilot performance throughout times (for example, as pilot population performance improves, better performance is needed to obtain the same grading result). — The situation above occurs while the concordance between instructors may be high. Because all instructors are varying their grading in the same direction, the population of pilots is moving to the right or to the left in the graph below, and thus the grading results of a particular performance are shifting to the right or to the left of the graph. Conclusion: Measuring competencies (especially the non-technical ones) using a norm-referenced grading may be more appropriate; however, we also need to verify the grading system against a criterion-referenced system in order to ensure legal assurance and level playing field in the revalidation of pilot licences. 6 EASA performed a focused consultation in the 4th quarter of 2017 and the 1st quarter of 2018 with several stakeholders outside the EBT subgroup RMT.0599. This consultation included the main group of RMT.0599 and other actors of the aviation industry such as the national aerospace centre of Holland (NLR), nominated persons for crew training, inspectors and consultants. Page 117 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail SUMMARY The current system provides for the LPC of the Aircrew Regulation a criterion-referenced grading, which measures performance against a fixed set of predetermined criteria or learning standards established through the mandatory manoeuvres and criteria set in Appendix 9. It is necessary for the European aviation system to apply a criterion-referenced grading system for the rating issue and revalidation. In addition, it is necessary for the feedback on the effectiveness of training programme. Therefore, the following tables provide an example for the grading system proposed in this Appendix to the Opinion (VENN 1 to 5): — The line between 1 and 2 should have the lowest variation possible between operators by a verification against a criterion-referenced system, while above grade 2, a norm-referenced system may be followed. This means that it may vary in the course of time and therefore the same performance may not obtain the same grading results in the course of time. year 1 1 2 3 4 5 Norm-referenced system Not proficient proficient Criterion-referenced system year 2 1 2 3 4 5 Norm-referenced system Not proficient proficient Criterion-referenced system Observe how the grading system in year 2 provides a grading of 5 to a lower pilot performance than year 1 and year 3. year 3 1 2 3 4 5 Norm-referenced system Not proficient proficient Criterion-referenced system AMC1 ORO.FC.231(d) The EASA EBT checklist for mixed EBT implementation provides the criteria to complete Appendix 9 (e.g. Element 3.6 may credit the item 3.4) Page 118 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail 3.4.0 to 3.4.14 (M) Normal and abnormal operations of systems. Minimum of 3 for the crew. 3.6.1 to 3.6.9 (M) Abnormal and emergency procedures. Minimum of 3 for the crew. GM1 ORO.FC.231(d)(2) The ‘desired outcome’ in some of the elements of the table in GM1 ORO.FC.231(d)(2) ‘verification of the accuracy of the grading system’ are transpose from the Appendix 9 guidance from Austro control https://www.austrocontrol.at/jart/prj3/ac/data/dokumente/HB_LSA_PEL_002_2018-03- 29_1203646.pdf (e) SUITABLE TRAINING DEVICES AND VOLUME TO COMPLETE THE OPERATOR’S EBT PROGRAMME SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib CS OPS.EBT.231.4 Volume of FSTD training and FSTD qualification level VOLUME AND FSTD QUALIFICATION LEVEL (a) The EBT programme has been developed to include a notional exemplar of 48 FSTD hours over a 3-year programme for each flight crew member. (b) Subject to ORO.GEN.121, the operator may reduce the number of FSTD hours provided an equivalent level of safety is achieved. The programme should not be less than 36 FSTD hours. (c) Each EBT module shall be conducted in an FSTD with a qualification level adequate to complete proficiency checks; therefore, it must be conducted in a full-flight simulator (FFS) level C or D. AMC1 ORO.FC.231(e) point (a) The provision is transposed from Doc 9995 (Part II paragraph 1.1.1). ‘Appendices 2 to 7 form the basis for the construction of EBT recurrent assessment and training programmes. In order to address all assessment and training topics at the defined frequency, a training programme of 48 FSTD hours over a three-year period for each flight crew member has been assumed. This EBT recurrent assessment and training should be conducted in an FSTD qualified for the purpose’. Part I paragraph 3.6.1 ‘The EBT recurrent assessment and training of the competencies (contained in Appendix 1 to Part II) are considered over a three-year recurrent assessment and training period. For the purposes of the construction of model training programmes as listed in Appendices 2 to 7 to Part II, the programme has been developed to include a notional exemplar 48 hours for each crew member over a three-year period in a suitably qualified flight simulation training device (FSTD). The training programme is divided into modules. The three phases of a module (evaluation, manoeuvres training and scenario- based training) are described in Chapter 7 of Part I.’ AMC1 ORO.FC.231(e) point (c) EASA is currently updating the requirements for FSTD through RMT.0196 ‘Update of flight simulation training device requirements’. More information about this RMT is available under Page 119 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail https://www.easa.europa.eu/document-library/rulemaking-subjects/update-flight-simulation- training-devices-requirements. Currently, Appendix 9 to Part-FCL of the Aircrew Regulation requires the FSTDs used to revalidate a type rating in the context of CAT to meet the standards required for ‘training to proficiency’. There was a consensus in the RMG to provide a similar requirement for the EBT programmes. Τhe actual drafting of the text for this provision was agreed with EASA FSTD experts and members of RMG RMT.0196. RMG RMT.0599 did not have experts in this subject and therefore the text was simply accepted with no further discussion. The reasoning behind the text proposed is related to the EASA certificate awarded to each FSTD. Each certificate (see EASA form 145 in Appendix IV to Annex VI (Part-ARA) to the Aircrew Regulation) contains a table in paragraph ‘L’ named ‘Guidance information for training, testing and checking considerations’. The line ‘Proficiency check YES/NO’ covers this item. Below are some of the considerations of the RMG for the actual and future development of FSTDs to maximise effectiveness when used as part of an EBT programme: (a) Environmental effects: (1) Weather (2) Real-time full environment simulation without limitations and demand on the instructor to code effects, layers of clouds, etc. repetitively during a session (3) Enhancement of the availability of cumulonimbus and storms with a strong correlation to motion cues (4) Availability of multiple storms and cumulonimbus to create a more realistic and challenging weather profile (5) Greater variation in precipitation effects (6) Better-modelled ground effects; especially, variations in friction caused by water, snow and ice (7) ATC (8) To maximise realism and the benefits of EBT, the air traffic control (ATC) environment needs simulation with context-specific ATC interactions. Creating a normal, dynamic and distracting ATC environment is challenging for an instructor to achieve and is a diversion from the instructor’s primary task of observing flight crew members. (b) Aircraft effects (c) Greater accuracy in modelled engine malfunctions based on engine original equipment manufacturer (OEM) data with motion and sound effects that are more realistic Currently, EASA is working on a process to allow aviation blended learning environment (ABLE) to support FSTD training. This will optimise the use of available FSTD time. When this process is in place as an approved AMC, the requirement for FSTD training may be replaced by requirements for training in any combination of devices supporting the specific tasks. Page 120 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (f) EQUIVALENCY OF MALFUNCTIONS SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib CS OPS.EBT.231.5 Malfunction characteristics Aircraft system malfunctions that place a significant demand on a proficient crew shall be organised by reference to the following characteristics: (a) immediacy; (b) complexity; (c) degradation of aircraft control; (d) loss of instrumentation; and (e) management of consequences. AMC1 ORO.FC.231(f) Evidence-based training EQUIVALENCY OF MALFUNCTIONS — PROCESS (a) The equivalency of malfunctions process should be undertaken by subject matter experts (SMEs) who hold or have held a type rating on the aeroplane type. (b) Steps of the equivalency of malfunctions: Step 1: Look (review) at all aircraft system malfunctions provided by the OEM. For example, FCOM for Airbus, or AFM for other manufacturers, does not normally provide an exhaustive list of malfunctions. Step 2: Determine and retain in a list only malfunctions that place a significant demand on a proficient crew, in isolation from an environmental or operational context. Step 3: For each retained malfunction, determine the applicable characteristic or characteristics. Step 4: Develop the EBT FSTD programme to incorporate malfunctions at the frequency specified in the table of assessment and training topics. (c) Malfunctions included in the equivalency of malfunctions but not included in the EBT FSTD programme require review and appropriate procedural knowledge training, conducted in a less qualified but suitable alternative environment (classroom, flight procedure training device, advance computer-based training, aviation blended learning environment (ABLE), etc.). Further guidance can be found in the EASA EBT manual. (d) The operator should establish procedures to determine what malfunctions should be included in the FSTD. This may include a different malfunction difficulty between the evaluation phase and the SBT. AMC1 ORO.FC.231(f) point (b) Step 1 Look (review) at all aircraft system malfunctions provided by the OEM instead of a more prescriptive wording such as flight crew operating manual, because each manufacturer has a different title for the document which contains the malfunctions relative to the aircraft (e.g. Airbus label this FCOM, Boeing Page 121 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail FCOM and AFM – other manufacturers use AFM). The quick reference handbook (QRH) is normally not an exhaustive list of malfunctions. AMC1 ORO.FC.231(f) point (b) Steps 1 and 2 Steps 1 and 2 provide a similar concept to today’s AMC1 ORO.FC.230(a)(4)(i)(A), where the list of major system malfunctions is selected (as per industry best practices) from the list of malfunctions of the real aircraft (not from the list of malfunctions provided by the FSTD). Then the operator selects the ones that are considered ‘major’ and covers them in a 3-year training period. The EBT malfunction clustering follows a similar approach where from the list of malfunctions of the real aircraft, the operator selects the ones that put a significant demand on a proficient crew. GM1 ORO.FC.231(f) has been developed to illustrate the concept of significant demand on a proficient crew. Once the malfunction is determined as putting a significant demand on a proficient crew, this means that it will have one or more of the 5 characteristics included in GM2 ORO.FC.231(f). AMC1 ORO.FC.231(f) point (c) This point is introduced in the AMC as per Doc 9995 paragraph 3.8.2 which provides the following text: ‘3.8.2 Practical training in the management of aircraft system malfunctions. Aircraft system malfunctions to be considered for the evaluation and scenario-based training phases are those that place a significant demand on a proficient crew. All malfunctions not covered by this characteristic continue to require review and appropriate procedural knowledge training with different means than considered in the recurrent EBT training conducted in an FSTD.’ and from Table I-3-1. ‘Malfunction characteristics and crew performance’ ‘Note — This refers to the case of recurrent training and assessment conducted in an FSTD qualified by the CAA at the appropriate level for recurrent training and assessment. Other malfunctions not covered by the characteristics detailed in 3.8.2 and 3.8.3 continue to require review and appropriate procedural knowledge training conducted in a less qualified but suitable environment (classroom, flight procedures training device, etc.), as an additional component of EBT. This is intended simply as a means of offloading the need to perform such training in a highly qualified FSTD, which has much greater potential benefit in other areas’. AMC1 ORO.FC.231(f) point (c) — wording ‘malfunctions included in the equivalency of malfunctions’ The wording proposed is related to the definition of ‘malfunction clustering’ proposed in Annex I to the Air OPS Regulation. AMC1 ORO.FC.231(f) point (c) — wording ‘Equivalency of malfunctions (malfunction clustering)’ Equivalency of malfunctions contains all the malfunctions that put a significant demand on a proficient crew, regardless if they are included or not in the FSTD programme. Point (c) was transposed from Doc 9995, Paragraph 3.8.2, and table I-3-1 Note: ‘All malfunctions not covered by this characteristic continue to require review and appropriate procedural knowledge training with different means than considered in the recurrent EBT training conducted in an FSTD’ Page 122 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail The intention is to require the pilot to be trained in each of the malfunctions that put a significant demand on a proficient crew. The RMG avoided on purpose examples such as multiple-choice test or online PowerPoint presentations. Instead, it proposed advance computer-based training and ABLE to foster new training means. AMC1 ORO.FC.231(f) point (c) — wording ‘EBT FSTD programme’ This refers to the 3-year EBT FSTD programme. AMC1 ORO.FC.231(f)(3) Evidence-based training CREW EXPOSURE TO AT LEAST ONE MALFUNCTION FOR EACH CHARACTERISTIC (a) Unless specified in the OSD, each crew member should be exposed to the characteristics of degraded control and loss of instrumentation in the role of pilot flying. (b) Notwithstanding point (a), for aircraft types with a limited number of malfunctions in the characteristic of degraded control or loss of instrumentation, the operator may use an alternative means of compliance in accordance with ORO.GEN.120. AMC1 ORO.FC.231(f)(3) The RMG considered this provision an important safety objective; for this reason, originally this provision was at implementing rule level. However, in order to provide flexibility to operators when malfunction clustering has a limited amount of emergencies pertaining to degradation of aircraft control and loss of instrumentation (which varies from aircraft type to aircraft type), the RMG moved this provision to AMC level. The limitation explained before creates a burden and limits the construction of line-orientated scenarios (EVAL and SBT). This feedback derives from operators who have already implemented mixed EBT (e.g. Thomas Cook Scandinavia, Alitalia, Iberia, etc.). Having this provision at AMC level allows for AltMoC in accordance with ORO.GEN.120. GM1 ORO.FC.231(f) Evidence-based training EQUIVALENCY OF MALFUNCTIONS — SIGNIFICANT DEMAND ON A PROFICIENT CREW (a) A procedure for a malfunction (e.g. non-normal, abnormal, emergency) may be considered to place significant demand on a proficient crew member if it results in one or more of the following: (1) time criticality; (2) multiple paths within the procedure (e.g. decision trees); (3) multiple inoperative or degraded systems; (4) a high potential for undetected errors (e.g. removal of flight protections); and (5) a significant increase in workload (e.g. removal of automation). (b) When a malfunction is placing a significant demand on a proficient crew, it means it has one or more of the malfunction characteristics (see more in GM2.ORO.FC.231(f)). GM1 ORO.FC.231(f) Page 123 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Once the malfunction is determined as placing a significant demand on a proficient crew, this means that the malfunction has one or more of the malfunction characteristics determined in GM2 ORO.FC.231(f). GM2 ORO.FC.231(f) Evidence-based training EQUIVALENCY OF MALFUNCTIONS — MALFUNCTION CHARACTERISTICS The following may be considered suitable definitions for each of the characteristics: (a) ‘Immediacy’: System malfunctions that require immediate and urgent crew intervention or decision (e.g. malfunctions with memory items) (b) ‘Complexity’: System malfunctions that require recovery procedures with multiple options to analyse and/or multiple decision paths to apply (c) ‘Degradation of aircraft control’: System malfunctions that result in significant degradation of flight controls in combination with abnormal handling characteristics (d) ‘Loss of instrumentation’: System malfunctions that require monitoring and management of the flight path using degraded or alternative displays. It includes primary instrumentation to monitor and manage primary aircraft systems (e.g. FLAPS indication, loss of fuel indications, etc.). (e) ‘Management of consequences’: System malfunctions that affect significantly the flight crew standard task sharing and/or the workload management and/or the decision-making process during an extensive period Note: Equivalency of malfunctions may be undertaken in consultation with the aircraft OEM. The objective of the OEM consultation is to review the operator analysis regarding the OEM operational certification (e.g. OSD) documents and the general OEM operation and training policy. GM3 ORO.FC.231(f) Evidence-based training EQUIVALENCY OF MALFUNCTIONS — ISOLATION FROM AN ENVIRONMENTAL OR OPERATIONAL CONTEXT When considering significant demand on a proficient crew, subject matter experts may consider that there are no significant environmental and operational threats. For example, the aircraft is close to a suitable aerodrome with environmental conditions permitting all published approaches to be made, with no pre-existing malfunctions and sufficient fuel for several hours (e.g. A320 or B737 overhead Ibiza - Spain, at FL350 with visible moisture at 30 000 ft, at the aerodrome wind calm, CAVOK, ISA). GM4 ORO.FC.231(f) Evidence-based training EQUIVALENCY OF MALFUNCTIONS PROCESS — DELPHI (a) The operator reviews/looks at aircraft system malfunctions provided in the official documentation of the OEM — for example, FCOM for Airbus, or AFM for other manufacturers. (b) Before launching the equivalency of malfunctions survey and when the aircraft system malfunctions list is very long, the operator may slightly shorten the list by removing the Page 124 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail malfunctions that surely will not place a significant demand of a proficient crew (see GM on SIGNIFICANT DEMAND ON A PROFICIENT CREW) (c) A group of EBT instructors statistically relevant will be selected to perform the equivalency of malfunctions survey. 50 % of the instructors’ community will be used as a reference. In small instructors’ communities, it may be necessary to refer to 100 %. In operators with large instructors’ communities, the number of instructors statistically relevant may be less than 50 %. (d) The group of instructors selected in point (c) will rate each of the malfunctions listed in points (a) and (b) (1) Each instructor will rate each one of the 5 characteristics in each malfunction listed in point (b). (2) The rate will be 0 when the malfunction does not have the characteristic (the characteristic does not appear in the malfunction). (3) The rate will be 1 to 5 when the characteristic appears in the malfunction. Rating 1 when the characteristic is not relevant for the malfunction and rate 5 when the characteristic is very relevant. (4) The instructors will rate individually (e.g. home, classroom, etc.) to avoid exchange of opinions with other instructors. (e) An average rate of the whole instructors’ community as a result of point (d) will be calculated for each characteristic of each malfunction. (f) A second round of survey will be performed with the same instructors and the same list. This time the operator will provide the average calculated in point (e) and ask them if in light of the average they would like to change their rating. (g) When an instructor changes their rating, the old rate will be discarded. (h) A new average will be calculated for each characteristic of each malfunction at the end of the second survey. The final average will be rounded to the closest integer number. (i) The operator may select an average rate of the characteristics (e.g. rate 2 or 3) at which or above which the characteristic is considered to be present in the malfunction, thus it places a significant demand on a proficient crew. (j) The operator may use the rates of the characteristics to determine the difficulty of the malfunction. As SBT is a developing phase, the operator may select a higher difficulty of the malfunctions selected in this phase. More information can be found in the EASA EBT manual. GM4 ORO.FC.231(f) The Delphi method is a structured communication technique or method, originally developed as a systematic, interactive forecasting method that relies on a panel of experts. The experts answer questionnaires in two or more rounds. After each round, a facilitator or change agent provides a de- identified summary of the experts' forecasts from the previous round as well as the reasons they provided for their judgements. Thus, experts are encouraged to revise their earlier answers in light of the replies of other members of their panel. It is believed that during this process, the range of the answers will decrease and the group will converge towards the ‘correct’ answer. Finally, the process Page 125 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail is stopped after a predefined stop criterion (e.g. number of rounds, achievement of consensus, stability of results) and the mean or median scores of the final rounds determine the results. As a reference, the figure of 50 % of the instructor community was provided for the following reasons: — The malfunction clustering should be adapted to the level of training and culture of the company. Therefore, the number of instructors should be sufficient. — The EBT should include the instructors and examiners as much as possible and this community should participate as much as possible in the development of the programme. A high level of participation may indicate that instructors and examiners are committed to implementing EBT. — Minimisation of errors: a large community of SMEs (50 % of instructors and examiners) are more likely to provide unbiased results; personal views and biased opinions may be discarded by the average results. Safety promotion material — Equivalency of malfunctions (DELPHI) EASA has planned SPT.012 to support the implementation of EBT. The following material has been developed: ‘SPT.012 — safety promotion task 012 — safety material for EBT — EQUIVALENCY OF MALFUNCTIONS EQUIVALENCY OF MALFUNCTIONS PROCESS — DELPHI — CRITERIA ON ELABORATION OF MALFUNCTION CLUSTERING The analysis of the grouping of abnormal and emergency procedures should only be carried out by a TRI EBT/SFI EBT or TRE EBT/SFE EBT in possession of the type rating of the aeroplane to be analysed. Abnormal and emergency procedures should be considered in isolation from any environmental or operational context. However, the operator should establish a minimum standardisation guide for those instructors/examiners who are going to carry out the study, in which some guidance is provided to analyse the procedures depending on the flight phase or conditions present, because significant differences will appear at the time evaluation. For instance, an abnormal procedure AIR PACK 1+2 FAULT does not have the same consequences below FL100 or at the maximum aircraft Flight Level. Standardisation guidance — The subject matter experts that develop the malfunction clustering should consider that the abnormal/emergency condition will remain when steps to fix the malfunction are included in the malfunction procedure (e.g. the failed engine will not restart, or the fuel pump remains failed after the reset, or the electric generator is not fixed after the reset, etc.). To this end, the operator should reproduce the malfunction in the FSTD programme in the same way (no restart of the engine, or successful reset of the fuel pump or electric generator) in order to meet with the characteristics assumptions. The operator may include successful resets or restart in addition to the malfunctions considered for the characteristics. When a reset puts a significant demand on a proficient crew, then both options should be included in the malfunction clustering and therefore the same malfunction should be evaluated for both cases: for successful reset/restart and for unsuccessful reset/restart. Page 126 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail — Whenever the possibility of icing is specified in the abnormal/emergency procedure, then it is assumed that this meteorological condition is present (e.g. in case of ‘pitot heating’, it is assumed that the conditions of icing are present). This case should follow the same principle as in the previous paragraph, where the EBT FSTD programme should include the icing condition when triggering the pitot heating. — Other possibilities require proper analysis. Grading — The grading varies from 1 to 5. 1 corresponds to the lowest level of malfunction characteristic. 5 corresponds to the highest. — The abnormal/emergency procedures to be graded are for the standard malfunctions for the type of aircraft (e.g. the malfunctions of the FCOM in A320) approved for the EBT programme. — The other malfunctions of the different versions of the aircraft models in the operator’s fleet (e.g. A321 / 319 / 320B4S, etc.) will be subject to a later revision and will be included in the 3- year period within the EBT topic ‘Operation- or Type-specific’.’ Minimum criteria A minimum of guidance is established when assigning a value to each of the characteristics of abnormal/emergency procedures for the standardisation purpose of the analysis. IMMDEDIACY ‘Immediacy’: System malfunctions that require immediate and urgent crew intervention or decision (e.g. malfunctions with memory items.) — If the caution or warning displays only crew awareness: minimum rating 1. — If it contains an amber ‘land as soon as possible’ (ASAP) warning: minimum rating 2. — If it contains a red ‘land as soon as possible’ (ASAP) notice: minimum rating 3. — If it is a procedure of memory steps: minimum rating 5. COMPLEXITY ‘Complexity’: System malfunctions that require recovery procedures with multiple options to analyse and/or multiple decision paths to apply ‘. — If the caution or warning displays only crew awareness: minimum rating 1. — If the caution or warning includes steps: minimum rating 2. — If the caution or warning contains or must be followed by a computer reset: minimum rating 2 (depending on the complexity of the reset). DEGRADATION OF CONTROL ‘Degradation of aircraft control’: System malfunctions that result in significant degradation of flight control in combination with abnormal handling characteristics. Page 127 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Any condition that implies an extra difficulty to fly the plane will be taken into account for the characteristic of degradation of control (which may not be limited to the flight control system). For instance, loss of flight protections laws, loss of power plant, etc. The following guidance applies: — Single engine flying (engine failure in multi-engine aircraft): minimum rating is 3 (except for some aircraft types with automatic yaw compensation in engine failures). — Alternative law flight (direct law with landing gear down): minimum rating 3/4. LOSS OF INSTRUMENTATION ‘Loss of instrumentation’: System malfunctions that require monitoring and management of the flight path using degraded or alternative displays. The characteristic to be assessed is not solely due to loss of cockpit displays. Abnormal/emergency procedures that imply flying with loss of relevant information should also be assessed. This principle increases the number of malfunctions available for this characteristic. This allows a better design of EBT FSTD sessions. — Loss of display units: minimum rate 2. — Significant loss of primary information related to systems (speeds, flap or slat position, fuel figures, etc.): minimum rate 2. — Loss of information related to abnormal and emergency procedures (FWC 1 + 2 FAULT, SDAC 1 + 2 FAULT, etc.): minimum rate 3. — Loss of information due to single failure (1 ADR Fault, 1 IR Fault, discrepancy messages, etc.): minimum rate 2. — Loss of information due to double failures (1+2 ADR Fault, 1+2 IR Fault, disagree messages, etc.): minimum rate 3/4. — Total loss of information (ADR 1+2+3 Fault, IR 1+2+3 fault, unreliable speed indication, etc.): minimum rate 5. MANAGEMENT OF CONSEQUENCES ‘Management of consequences’: System malfunctions that affect significantly the flight crew standard task sharing and/or the workload management and/or the decision-making process during an extensive period. — Consequences in the category of approach and landing or the required CAT II/III equipment: minimum rating 2. — Consequences in the minimum navigation requirements: minimum rating 2. — APP PROCEDURE in the STS: minimum rating 3. — Single engine landing: minimum rating 3. The operator, once the malfunction clustering analysis has been completed, may reflect in its training manual the maximum and minimum difficulty values of each one of the characteristics of the equivalency of malfunctions (malfunction clustering). Depending on the difficulty value, the Page 128 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail malfunction will be included in the different phases of an evaluation session (LOE) and in a training session (LOFT) (e.g. maximum LOE value 20, while SBT accepts the maximum of 25 points). (g) EQUIVALENCY OF APPROACHES RELEVANT TO OPERATIONS SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib AMC1 ORO.FC.231(g) Evidence-based training APPROACHES THAT PLACE AN ADDITIONAL DEMAND ON A PROFICIENT CREW (a) In order to identify approaches that place an additional demand on a proficient crew, an operator should: (1) review its operational network; (2) select approaches with one or more of the following characteristics: (i) unusual design; (ii) low frequency of exposure; and (iii) degraded approach guidance; (3) select at least one approach of each type and method and include them in the EBT programme at the frequency given in the table of assessment and training topics; and (4) ensure the approaches selected in (3) cover all the characteristics at the frequency given in the table of assessment and training topics. Note: The approaches listed within Section 2 of the table of assessment and training topics should be selected in this process. (b) Any approach that is required to be flown in the PF role-specifically should be classified as ‘skills retention’ and may be trained in the manoeuvres training phase (MT). AMC2 ORO.FC.231(g) Evidence-based training EQUIVALENCY OF APPROACHES RELEVANT TO OPERATIONS — SPECIFIC APPROVAL The operator may extend the interval for recurrent training and checking of approaches that require specific approval as defined in the AMC to Part-SPA (e.g. SPA.LVO) to the frequency given in the EBT programme. GM1 ORO.FC.231(g) Evidence-based training EQUIVALENCY OF APPROACHES RELEVANT TO OPERATIONS — APPROACH CHARACTERISTICS The following may be considered suitable examples for each of the approach characteristics: (a) Design (1) Unusual approach design feature — for example, offset final approach track or steep approach, etc. (2) Unusual runway design feature — for example, non-standard lighting or marking Page 129 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (b) Frequency (1) Infrequently visited airfields — for example, alternate airfields (2) Infrequently flown approaches at commonly visited airfields — for example, circling approach, CAT 2, SA CATI. (c) Degraded guidance (1) Degraded internal guidance or aircraft equipment — for example, head-up display (HUD) failure (2) Degraded external guidance or ground equipment — for example, GPS signal failure GM2 ORO.FC.231(g) Evidence-based training SELECTED APPROACHES AT THE FREQUENCY GIVEN IN THE EBT PROGRAMME The table of assessment and training topics for each generation provides the type of approach, flight method and frequency for the crew. ORO.FC.231(g) and related AMC and GM The RMG developed a definition of the concept as follows: ‘equivalency of approaches’ refers to approaches relevant to operations determined by a defined method, leading to a reduced frequency of approaches with an increased focus on the operational relevance rather than just the conduct of an approach which is not realistic in the operational context. — Introduction Doc 9995 recommends approach clustering (‘equivalency of approach types’) as a way to avoid repetitive training on approaches that require the same actions by the pilot (‘underlying elements of flight crew performance to conduct them’). It also recommends avoiding those approaches that are typically flown during line operations (‘Frequency of training may be reduced for types of approaches that are conducted regularly in line operations.’). However, Doc 9995 does not explain how to carry out an ‘equivalency of approach types’ process. Additionally, the recurrent training requirements specified in Part-SPA do not reflect the reality of normal operations. For instance, the use of the HUD in Generation 3 and 4 aircraft types is usually mandated by the operator for all phases of flight, and therefore the requirement to carry out the approaches for recurrent training specified in Part-SPA does not reflect the EBT concept of incorporating approaches that are not conducted regularly in line operations. — Approach types The industry has moved from essentially three different approach genres (non-precision, precision and low-visibility operations (LVOs)) to a multitude of different approaches utilising satellite- and ground-based enhancements. This has given way to the curved approaches and approaches with varying gradients. While an aircraft’s acquisition of the flight path has changed, the ‘underlying performance’ for crews to perform the approaches has changed only a little, as the OEMs have made the pilot interface with the autopilot and the displays very similar to conventional approaches (i.e. ILS). The main change from a pilot’s viewpoint is the introduction of HUD and emergency vision assurance system (EVAS). Page 130 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Essentially, ICAO Annex 6 has delineated the approach types as two-dimensional (2D) and three- dimensional (3D) approaches, and Type A and B in accordance with the ‘achieved’ minima. Most Generation 3 and 4 aircraft types have the same autopilot/pilot interface and displays for all 3D approach methods, irrespective of whether or not the approach is Type A or Type B. Variations do exist for the conduct of 2D methods depending on OEM. Doc 9995 groups aircraft into generations, with the biggest groups being the Generation 4 and 3 jets. The delineation between the two generations is based upon whether or not the aircraft has fly-by-wire and flight envelope protections. While this delineation is entirely relevant and useful to derive recurrent training programmes, it does not necessarily reflect the avionics capability or the pilots interface with the autopilot. For example, the 747-8 sits in Generation 3 as it has conventional flight controls. Therefore, it would unnecessarily penalise some aircraft types by clustering them in accordance with the EBT Generation. — HUD and EVAS Generation 4 and Generation 3 aircraft types fitted with a HUD utilise it for all approaches, irrespective of whether or not they are Type A or Type B utilising 3D or 2D methods. This is the standard mode of operation. Similarly, the use of EVAS, although not currently fitted to Generation 3 and 4 aircraft types, is again the standard mode of operation and utilised for all approaches. For recurrent training, flying additional approaches to revalidate the use of the HUD is simply replicating normal line operations with a little benefit. The operator can assure themselves of pilot proficiency in the use of HUD and EVAS, if fitted, through the ‘line evaluation of competence’, when it will be used in the real operational context. — Go-around training Go-around training is not considered in this section because the go-around training frequency is defined by the table of assessment and training topics, and is in excess of that required by Part-SPA. — Approach clustering In the absence of guidance in Doc 9995, the principles used for malfunction clustering have been adopted to create a similar concept for approach clustering. Two principles in particular have been considered: — approaches that place an additional demand on a proficient crew; and — approaches should be selected according to certain characteristics. For the first principle, the emphasis has been changed from ‘significant demand’ to ‘additional demand’. This is because the approach will normally be flown at the end of a scenario within the SBT. That scenario will have included malfunctions and other training topics that have already added ‘significant demand’ on the crew. The approach chosen should therefore contain good training value and realism, without compromising the learning by adding workload on top of workload. For example, a scenario involving a significant malfunction has better value and realism if concluded with an autoland rather than a circling approach. Page 131 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail For the second principle, the concept of approach characteristics has been adopted. Doc 9995 lists eight ‘parameters’ that can be used in a clustering process; however, many seem to be types of approaches rather than characteristics. Instead, it was determined that approach characteristics can be divided into three groups, which are listed in the AMC with examples given in the GM. — Types and frequency of approach training As stated earlier, the EBT Generation delineation of aircraft types is not useful when comparing avionics and pilot interface/display information. Many Generation 3 and 4 aircraft types have a single button push for all approaches, with little or no changes in the displayed information. It would seem therefore appropriate to analyse the aircraft in these Generations to review the appropriate types to develop an ‘approach generation/group’. A focus of EBT is to remove extraneous training for which there is little safety benefit, or evidence of need, and in particular, those approaches that are regularly performed in line operations. Additionally, an operator will seek a simple system that allows for the variation in the definition of training topics throughout the semester to cater for the trainees’ needs. Mandating repetitive approaches would not be beneficial to the operator or the trainee alike. Using the frequencies defined in Doc 9995, and applying the emphasis intended by EBT, the following has been derived. Type Flight method Phase Frequency A 3D EVAL & SBT B B 3D EVAL & SBT B A 2D MT B The operator’s policy generally defines which flight method should be used on line operations to conduct this kind of approaches. These recommendations should be followed by crews during LOE. During SBT or manoeuvres validation phase, it should be considered interesting to adapt the conduct of the selected approaches in order to develop specific competencies. There is no intention to define here that a pilot has to be pilot flying (PF) for each approach; this is because it is part of the line-orientated scenarios. Any approach that is required to be flown specifically in the PF role should be classified as ‘skills retention’; therefore, it should be trained in the MT. The above approaches should be flown simulating normal operations. Enhance vision system (EVS) or Enhance flight vision system (EFVS) or Head-up display (HUD) should be utilised if required in normal operations. The allocation of the types of approaches into either the EVAL and SBT or the MT is determined by the purpose of the exercise. For the 3D approaches, these are the most commonly flown in normal operations, and would therefore be the most relevant and realistic to be included in Page 132 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail training scenarios. They will additionally be chosen to place an additional demand on a proficient crew. In contrast, a 2D approach is typically flown less frequently, and normally only if a 3D approach is unavailable due to aircraft or airport downgrade. For some modern aircraft types (e.g. A380, Boeing 787), multiple, unrealistic failures should occur before a 2D approach is required. Additionally, the flight crew procedures to fly a 2D approach typically demand more automation management skill than a 3D approach. The MT is precisely what this is for: to enable the pilot to retain the skill to fly low-probability but higher-risk manoeuvres. The principle behind this type of training is skills retention. The B frequency has been considered appropriate for two reasons. Firstly, to align with the malfunction clustering B frequency, and secondly to fit in with the requirements of EVAL and SBT. In a typical EBT programme, there will be eight to ten approaches in these phases per year. As noted earlier, it would be inappropriate to add approaches with additional workload to scenarios that already place a significant demand on a proficient crew. Therefore, mandating at least two 3D approaches of different flight methods with additional demand per year has been considered the correct number. AMC2 ORO.FC.231(g) The rationale behind this AMC is that the operator has conducted a review of the approaches. It has taken into account which of them are placing a significant demand on a proficient crew and the characteristics of each of them. Therefore, more is done in terms of approaches within an operational context than is done today. EBT offers a frequency of B for specific approvals. Currently, ATQP also offers a frequency B for specific approvals. (h) LINE EVALUATION OF COMPETENCE SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib AMC1 ORO.FC.231(h) Evidence-based training LINE EVALUATION OF COMPETENCE (a) The purpose of the line evaluation of competence is to verify the capability of the flight crew member(s) to undertake line operations, including preflight and post-flight activities as specified in the operations manual. Therefore, the line evaluation of competence should be performed in the aircraft. The route should be representative of typical sectors undertaken in normal operations. The commander, or any pilot who may be required to relieve the commander, should also demonstrate their competency in the role. (b) Each flight crew member should be assessed according to the competency framework and grading system approved for their operator’s EBT programme. (c) Flight crew members should be assessed in duties as pilot flying and pilot monitoring; they should be evaluated in each role. Therefore, they should be checked on one flight sector as pilot flying and on another flight sector as pilot monitoring. (d) The operator should maintain a list and inform the competent authority about the line evaluators suitably qualified to undertake line evaluations of competence. Page 133 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (e) The person that conducts the line evaluation of competence should occupy an observer’s seat. For aeroplanes, in the case of long-haul operations where additional operating flight crew members are carried, the person may fulfil the function of a cruise relief pilot and should not occupy either pilot’s seat during take-off, departure, initial cruise, descent, approach and landing. (f) The validity period should be counted from the end of the month when the line evaluation of competence was undertaken. When the line evaluation of competence is undertaken within the last 6 months of the validity period, the new validity period should be counted from the original expiry date. AMC1 ORO.FC.231(h) The AMC has been developed following the principles contained in AMC1 ORO.FC.230 point (b)(3) on line check. For some of the points, there is almost a direct transposition with only minor amendments. For others, the amendments are extensive. AMC1 ORO.FC.231(h) point (a) The requirement is extracted from AMC1 ORO.FC.230 (b)(3)(i) with the proper modifications: ‘The commander, or any pilot who may be required to relieve the commander, should also demonstrate his/her ability to ‘manage’ the operation and take appropriate command decisions.’ AMC1 ORO.FC.231(h) point (d) This provision is transposed from the current AMC1 ORO.FC.230 point (b)(3)(v) ‘Line checks should be conducted by a commander nominated by the operator. The operator should inform the competent authority about the persons nominated. (...)’ AMC1 ORO.FC.231(h) point (f) wording ‘validity period’ The wording for ‘validity period’ is similar to that used in ORO.FC.245(d). The revalidation window (normally 3 months in the LPC) has been increased to 6 months; this is also in line with other periods of validity that exist in Part-FCL (e.g. revalidation of a rating). AMC2 ORO.FC.231(h) Evidence-based training LINE EVALUATION OF COMPETENCE — LINE EVALUATOR (a) The line evaluator should have a valid line evaluation of competence. (b) The line evaluator should receive an acceptable training based on the EBT instructor training. The EBT assessment of competence is not required. AMC1 ORO.FC.231(h)(3) Evidence-based training LINE EVALUATION OF COMPETENCE — EXTENSION OF THE VALIDITY In order to extend the validity of the line evaluation of competence to: (a) 2 years, in every cycle, one evaluation phase for each pilot should be conducted by an EBT instructor who has a valid line evaluation of competence; Page 134 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (b) 3 years, in addition to point (a) above, the operator should have a feedback process for the monitoring of line operations which: (1) identifies threats in the airline’s operating environment; (2) identifies threats within the airline’s operations; (3) assesses the degree of transference of training to the line operations; (4) checks the quality and usability of procedures; (5) identifies design problems in the human-machine interface; (6) understands pilots’ shortcuts and workarounds; and (7) assesses safety margins. AMC1 ORO.FC.231(h)(3) The 2-3-year extension of the line evaluation of competence provides a vehicle for operators who have ATQP to continue with the credits they have under an ATQP. The safety case is that an ATQP operator needs 2 years of ATQP before being approved for an extension of the validity of the line check. This requirement is mirrored here, as the operator will need more than 3 years of mixed EBT implementation to extend the validity of the line evaluation of competence. To encourage an operator to use line operations safety data programmes because they provide further safety enhancements, the 3-year extension is offered if the safety data programme is integrated within the EBT programme. The requirement of the safety data programme has been transposed from FAA AC120-90 dated 27th April 2006 paragraph 5. AMC1 ORO.FC.231(h)(3) point (a) One of the purposes of a line check is to verify the ability of a pilot to undertake normal line operations in the real aircraft. The validity of the line evaluation of competence is extended with the condition that the pilot ability to undertake normal line operations is maintained. For that purpose, an EBT instructor with current line operations experience is required once a year. That means that the operator should have sufficient EBT instructors to provide the EBT modules who have themselves enrolled in the EBT programme and a line evaluation of competence as specified in the operations manual. The extension of the line evaluation of competence is based on the substitution of the evaluation phase in the EBT module. Operations in the context refers to normal, abnormal and emergency operations of aircraft. Therefore, the intention of the provision is to have an EBT instructor who is enrolled in the operator’s EBT programme and has a valid line evaluation of competence; however, as the line evaluation of competence requires to the instructor to be enrolled, the final text does not contain the word ‘enrolled’. AMC3 ORO.FC.231(h)(3) point (b) Page 135 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail For the purpose of a feedback process for the monitoring of line operations, the group studied the ATQP line-oriented quality evaluation, ICAO Doc 9803 Line Operations Safety Audit (LOSA) and the FAA LOSA. ‘GM1 ORO.FC.A.245 Alternative training and qualification programme TERMINOLOGY (a) (…) (b) ‘Line-oriented quality evaluation (LOQE)’ is one of the tools used to help evaluate the overall performance of an operation. LOQEs consist of line flights that are observed by appropriately qualified operator personnel to provide feedback to validate the ATQP. The LOQE should be designed to look at those elements of the operation that are unable to be monitored by FDM or Advanced FDM programmes.’ According to the RMG, the most important functions of such a feedback process are the ones mentioned under points (b)(1) to (7). Point (b)(5) ‘identifies design problems in the human-machine interface’ was introduced following the information provided in the ICAO Doc 9803 where equipment design may be a cause of normalisation of deviance and therefore should be monitored. ICAO Doc 9803 Line Operations Safety Audit (LOSA) ‘1.2.6 Second, and most important, incident reporting is vulnerable to what has been called “normalization of deviance”. Over time, operational personnel develop informal and spontaneous group practices and shortcuts to circumvent deficiencies in equipment design, clumsy procedures or policies that are incompatible with the realities of daily operations, all of which complicate operational tasks.(…)’ GM1 ORO.FC.231(h) Evidence-based training LINE EVALUATION OF COMPETENCE (a) Line evaluation of competence, route and aerodrome knowledge, and recent experience requirements are intended to verify the capability of the flight crew member(s) to operate safely, effectively and efficiently under line operating conditions, including preflight and post- flight activities as specified in the operations manual. Other EBT assessments, legacy checks and emergency and safety equipment training are primarily intended to prepare flight crew members for abnormal/emergency procedures. (b) The line evaluation of competence is considered a particularly important factor in the development, maintenance and refinement of high operating standards, and can provide the operator with a valuable indication of the usefulness of its training policy and methods. GM1 ORO.FC.231(h) This text was inspired from the current regulatory material of AMC1 ORO.FC.230 and GM1 ORO.FC.230. However, the RMG proposed some small amendments. Some of them are explained below: The RMT noted GM1 ORO.FC.230 point (c). ‘(c) Proficiency training and checking Page 136 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail When an FSTD is used, the opportunity should be taken, where possible, to use LOFT.’ The RMG decided to not transpose this provision into GM1 ORO.FC.231. GM1 ORO.FC.231(h) point (a) ‘Line operations’ is used instead of ‘normal line operation’ because ‘normal line operations’ will imply that if the crew face a failure in the aircraft, they may not be able to complete the line check. The phrase ‘including preflight and post-flight activities as specified in the operations manual’ is introduced (this wording is not present in AMC or GM to ORO.FC.230) to clarify the scope of the line evaluation of competence. The EBT subgroup RMT.0599 believes that the current regulation in regard to ‘line checks’ (ORO.FC.230) should also clarify this item. GM1 ORO.FC.231(h)(4) Evidence-based training LINE EVALUATOR (a) AMC1.ORO.FC.146(c) ‘EBT instructor training’ provides some learning objectives which may be used to qualify the commander nominated by the operator to perform line evaluation of competence. The training may be a minimum of 7 hours, where one hour may be done outside the classroom. The use of advance training environments such as advance computer-based training or ABLE may reduce further the need of classroom training. The assessment of competence may not be required. Further guidance can be found in the EASA EBT manual. (b) The line evaluator training may be included in the EBT instructor standardisation and concordance programme. This option is however limited due to the limited number of line evaluations of competence that are required (every 2 or 3 years), the difficulties in observing the whole range of performance of competencies and the lack of control of the environment during a line evaluation of competencies. Therefore, the operator may need to use EBT instructors to maintain an acceptable level of standardisation. (i) GROUND TRAINING SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib AMC1 ORO.FC.231(i) Evidence-based training PERFORMANCE-BASED CONTINUOUS TECHNICAL GROUND TRAINING (a) Technical ground training programme (1) The objective of the technical ground training programme is to ensure that pilots have adequate: (i) knowledge of: (A) the aircraft systems; and (B) the operational procedures and requirements; and (ii) awareness of: Page 137 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (A) the most significant accidents or incidents that could affect their operations following the ‘threat and error management model’ or an alternative risk model agreed with the authority; and (B) the occurrences in the airline or occurrences from other airlines that may be relevant for their operations, accident/incident and occurrence review. (2) The technical ground training should: (i) be conducted as part of a 3-year programme; (ii) allow a customisation of syllabi. The operator should describe in the operations manual the procedure to determine the customisation of syllabi. This customisation should be based on evidence both internal and external to the operator. (iii) as a minimum, allow the pilot to receive technical ground training every 12 months. The validity period should be counted from the end of the month. When this training is conducted within the last 3 months of the validity period, the new validity period should be counted from the original expiry date. (3) The technical ground training syllabi should be delivered using different methods and tools. (i) The selection of the method and tool results from a combination of the learning objectives and the target group receiving the training (WHAT needs to be trained and WHO needs to be trained). (ii) The selection of the appropriate method and tool must be driven by the desired outcome in terms of adequate knowledge. (iii) The delivery of the technical ground training syllabi should include the methods or tools to verify if the pilot has acquired the objective of the technical ground training programme. This may be achieved by means a questionnaire, assessment of application of the competency ‘knowledge’ (KNO) or other suitable methods. (4) The measurement and evaluation of the training system performance through the feedback process should include the performance of the technical ground training. (b) Emergency and safety equipment training (1) Training on the location and use of all emergency and safety equipment should be conducted in an aircraft or a suitable alternative training device. (2) Every year the emergency and safety equipment training programme should include the following: (i) actual donning of a life-jacket, where fitted; (ii) actual donning of protective breathing equipment, where fitted; (iii) actual handling of fire extinguishers of the type used; Page 138 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (iv) instruction on the location and use of all emergency and safety equipment carried on the aircraft; (v) instruction on the location and use of all types of exits; and (vi) security procedures. (3) Every 3 years the programme of training should include the following: (i) actual operation of all types of exits; (ii) demonstration of the method used to operate a slide, where fitted; (iii) actual firefighting using equipment representative of that carried on the aircraft on an actual or simulated fire except that, with Halon extinguishers, an alternative extinguisher may be used; (iv) the effects of smoke in an enclosed area and actual use of all relevant equipment in a simulated smoke-filled environment; (v) actual handling of pyrotechnics, real or simulated, where applicable; (vi) demonstration in the use of the life rafts, where fitted; and (vii) particularly in the case where no cabin crew is required, first aid appropriate to the aircraft type, the kind of operation and the crew complement. (4) The successful resolution of aircraft emergencies requires interaction between flight crew and cabin/technical crew and emphasis should be placed on the importance of effective coordination and two-way communication between all crew members in various emergency situations. (5) Emergency and safety equipment training should include joint practice in aircraft evacuations so that all who are involved are aware of the duties other crew members should perform. When such practice is not possible, combined flight crew and cabin/technical crew training should include joint discussion of emergency scenarios. (6) Emergency and safety equipment training should, as far as practicable, take place in conjunction with cabin/technical crew undergoing similar training with emphasis on coordinated procedures and two-way communication between the flight crew compartment and the cabin. (7) The emergency and safety equipment training should include a pilot’s assessment of the training received; as a minimum, by means of a questionnaire, or computer-based exercises, or other suitable methods. (8) When the emergency and safety equipment training is conducted within 3 calendar months prior to the expiry of the 12-calendar-month period, the next emergency and safety equipment training should be completed within 12 calendar months of the original expiry date of the previous training. (c) Emergency and safety equipment training — extension of period of training Page 139 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (1) The emergency and safety equipment training programme should establish and maintain at least an equivalent level of proficiency achieved by complying with the provisions of (b). The level of flight crew proficiency in the use of emergency and safety equipment shall be demonstrated prior to being granted approval to extend the period of training by the competent authority. (2) The operator applying for an approval to extend the period of emergency and safety equipment training should provide the competent authority with an implementation plan, including a description of the level of flight crew proficiency to be achieved in the use of emergency and safety equipment. The implementation plan should comprise the following: (i) A safety case which should: (A) demonstrate that the required or equivalent level of proficiency in the use of emergency and safety equipment is maintained; (B) incorporate the programme of implementation, to include controls and validity checks; (C) minimise risk during all phases of the programme’s implementation and operation; and (D) include oversight, including review and audits. (ii) The measurement and evaluation of the training system performance through the feedback process should include the performance of the emergency and safety equipment training. The feedback should be used as a tool to validate that the emergency and safety equipment training is correctly implemented; this enables substantiation of the emergency and safety equipment training and ensures that objectives have been met. (iii) Documentation that details the scope and requirements of the programme, including the following: (A) the operator’s training needs and established operational and training objectives; (B) a description of the process for designing and obtaining approval for the operator’s emergency and safety equipment training programmes. This should include quantified operational and training objectives identified by the operator’s internal monitoring programmes. External sources may also be used; and (C) a description of how the programme will develop a support and feedback process to form a self-correcting training system. (3) When the emergency and safety equipment training is conducted within 6 calendar months prior to the expiry of the 24-calendar-month period, the next emergency and safety equipment training should be completed within 24 calendar months of the original expiry date of the previous training. Page 140 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail AMC1 ORO.FC.231(i) The proposed AMC points (a) and (b) are transposed from AMC1 ORO.FC.230. However, point (a) has been substantially modified. The proposed AMC point (c) is based on the principles established in ORO.FC.A.245 and AMC1 ORO.FC.A.245 ‘Alternative training and qualification programme’. AMC1 ORO.FC.231(i) wording ‘ground training’ The word used in AMC1 ORO.FC.230 is ‘ground and refresher training’; however, this wording will be modified in order to align with the title of the AMC and therefore avoid duplication and misunderstanding. AMC1 ORO.FC.231(i) point (a) The idea behind the performance-based continuous ground training is to extend the principles of evidence-based training into the area of ground training. Ground training in this context has two objectives: (1) Ensure adequate knowledge regarding aircraft systems and operational procedures and requirements. (2) Ensure adequate awareness regarding accidents and incidents following a risk model (e.g. TEM). Knowledge is essential regarding systems, procedures and requirements in order to understand, interpret and properly apply the aircraft systems operator’s procedures. However, theoretical knowledge of incidents and accidents does not prevent reoccurrence in the future. It is foremost the analysis of the incidents and accidents using an agreed risk model, in order to identify the underlying root causes, which the pilot needs to be aware of, in order to effectively apply countermeasures in the future. A ground training element should be conducted every 12 calendar months, which should be embedded in a 3-year programme, hereby adapting the EBT period. AMC1 ORO.FC.231(i) point (a)(1)(i)(B) The provision is transposed from AMC1 ORO.FC.230 point (a)(1)(i)(B), according to which the ground training should include: ‘(B) operational procedures and requirements, including ground de-icing/anti-icing and pilot incapacitation;’ However, the reference to ‘de-icing/anti-icing and pilot incapacitation’ is deleted because it is already provided in the ‘table of assessment and training topics’ as a training topic. AMC1 ORO.FC.231(i) point (b)(3)(vi) The provision is transposed from AMC1 ORO.FC.230 point (a)(2)(iii)(F). However, the provisions for helicopters are deleted as currently EBT, in accordance with Doc 9995, is only provided for some types of aeroplanes. EASA is currently working on the development of an EBT data report for helicopters in order to allow first a mixed EBT implementation and in the future an EBT programme for helicopters. AMC1 ORO.FC.231(i) point (b)(7) Page 141 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail As explained above, AMC1 ORO.FC.231(i) points (a) and (b) are transposed from AMC1 ORO.FC.230; however, the training elements and the checking elements are scattered across point (a) and point (b) of AMC1 ORO.FC.230. As in ORO.FC.231 both elements are combined in a single point, point (b)(7) is introduced; however, the wording is modified as in EBT the word ‘checking’ is not used. AMC1 ORO.FC.231(i) point (c) The requirement is transposed from the existing ATQP provision (see ORO.FC.A.245 of the Air OPS Regulation). The reason behind using the ATQP provisions is that emergency and safety equipment is outside the scope of EBT competencies; therefore, the ATQP provision is fit for purpose for the extension of validity. The maximum validity of 24 months is also transposed from ATQP. AMC1 ORO.FC.231(i) point (c)(1) This point is transposed from ORO.FC.245.A point (b) and adapted for the purposed of the AMC regarding ground training. This requirement provides the safety objective if an extension is requested, which is to achieve and maintain the level of proficiency set out in point (b). In ATQP, ORO.FC.A.245 point (b) is then reflected in AMC1 ORO.FC.A.245 point (a)(1)(i) ‘documentation’. However, the proposal for ground training under the EBT programme does not impose these requirements. This does not mean that the competent authority is not entitled to ask for it; however, as the scope of the ground training is limited and the safety objectives of the EBT are demonstrated elsewhere, the proposed regulation tries to avoid unnecessary burden. AMC1 ORO.FC.231(i) point (c)(2)(iii) The provision is transposed from AMC1 ORO.FC.A.245.point (a)(1) ‘documentation’; however, the following adjustments have been made: Point (a)(1)(i) of AMC1 ORO.FC.A.245 is not included; Point (a)(1)(ii) of AMC1 ORO.FC.A.245 is transposed with no change; Point (a)(1)(iii) of AMC1 ORO.FC.A.245 is transposed with slight modifications; Point (a)(1)(iv) of AMC1 ORO.FC.A.245 — only the concept is transposed and provision is made to express the safety objective. Extract of AMC1 ORO.FC.A.245 ‘(1) Documentation that details the scope and requirements of the programme, including the following: (i) The programme should demonstrate that the operator is able to improve the training and qualification standards of flight crew to a level that exceeds the standards prescribed in ORO.FC and Subpart E of Annex V (SPA.LVO). (ii) The operator’s training needs and established operational and training objectives. (iii) A description of the process for designing and gaining approval for the operator’s flight crew qualification programmes. This should include quantified operational and training objectives identified by the operator’s internal monitoring programmes. External sources may also be used. Page 142 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (iv) A description of how the programme will: (A) enhance safety; (B) improve training and qualification standards of flight crew; (C) establish attainable training objectives; (D) integrate CRM in all aspects of training; (E) develop a support and feedback process to form a self-correcting training system; (F) institute a system of progressive evaluations of all training to enable consistent and uniform monitoring of the training undertaken by flight crew; (G) enable the operator to be able to respond to new aeroplane technologies and changes in the operational environment; (H) foster the use of innovative training methods and technology for flight crew instruction and the evaluation of training systems; and (I) make efficient use of training resources, specifically to match the use of training media to the training needs.’ GM1 ORO.FC.231(i) Evidence-based training PERFORMANCE-BASED CONTINUOUS GROUND TRAINING — INTERNAL AND EXTERNAL EVIDENCE (a) Operator evidence (inner loop) (1) Pilot data (individual or group); (2) Population-based data according to the training metrics determined in the training system performance; (3) Identified or recognised through the safety management process covered in ORO.GEN.200. (b) External evidence from the authority and manufacturers (external loop) (1) Revision of existing rules and regulations, updated versions of the EBT data report; (2) Training needs derived from updated OSD (if appropriate for ground training), etc. (c) The evidence drives the selection of the methods and tools. GM2 ORO.FC.231(i) Evidence-based training PERFORMANCE-BASED CONTINUOUS GROUND TRAINING — METHODS AND TOOLS This is a non-exhaustive list of methods and tools to deliver ground training: classroom, presentations, web-based training, self-learning instructions, advance CBT such as virtual reality, chatbots, interactive scenario trainers. ‘SPT.012 — safety promotion task 012 — safety material for EBT — CUSTOMISATION OF SYLLABI The syllabi can be customised at three different steps: 1. The first step would be one syllabus for the whole pilot’s population (customisation only at type rating level). Page 143 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail 2. The second step would be syllabi for different populations of pilots (for example, all first officers, all B747 pilots, all pilots flying an Airbus model, etc.). 3. The third step would be individual syllabi tailored to the needs of individual pilots (pilot customisation). The procedure to describe the customisation of syllabi must be described in the OM. Customisation is based on evidence that can be gathered on three different levels, two from the inner loop, one from the outer loop. (1) Inner loop — Individual evidence based on grading reports or questionnaires, analysed for either an individual pilot or a group of pilots (for example, all first officers, all B747 pilots, all pilots flying an Airbus model, etc.) — Operator-specific evidence gathered from the safety management process in accordance with ORO.GEN.200 (2) Outer loop: — Evidence gathered from external sources like authorities (e.g. State Safety Plan, etc.), OEMs (e.g. OEBs, OSD, safety documentation such as ‘getting to grips’, etc.), etc. SPT.012 — safety promotion task 012 — safety material for EBT — SELECTION OF THE METHOD AND TOOL — LEARNING OBJECTIVES AND TARGET GROUP RECEIVING THE TRAINING The selection of appropriate methods and tools for proper ground training delivery must be driven by answering two questions. WHO needs to be trained? WHAT needs to be trained (learning objectives)? Training topics that need further explanation or are optimally learned through discussions within a group, should be delivered by providing classroom training or web-based interactive sessions. When selecting the method and tool, operators should be driven by the desire to achieve the optimum outcome, which is maximum possible knowledge increase. An example of a matrix for each question is provided below: LEARNING OBJECTIVES – What needs to be trained? 1. [Blooms taxonomy] Knowledge – comprehension – application – analysis – synthesis – evaluation 2. [pragmatic] Knowledge/first overview – deeper understanding – competencies/able to perform 3. [pragmatic] Awareness/information – understanding/ knowledge – change of behaviour/performance. TARGET GROUP – Who needs to be trained? 1. Learning preferences 2. Learning routines 3. Learning & media competencies 4. Level of expertise/experience 5. Job role and responsibility 6. Demographic/cultural characteristics 7. Access to media/resources Page 144 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail ORO.FC.232 EBT programme assessment and training topics SEE IMPLEMENTING RULES IN THE OPINION ANNEX Ib CS OPS.EBT.232.1 Aircraft type by generations EBT PROGRAMME Generation 4 From 1988. A318/A319/A320/A321 (including — Jet) EFIS cockpit — FMS equipped neo), A330, A340-200/300, A340- FADEC 500/600, B777, A380, B787, Fly-by-wire control systems A350, Bombardier C Series Advanced flight envelope protection (A220), Embraer Integrated auto flight control system — navigation E170/E175/E190/E195 performance, and terrain avoidance systems Generation fatal accident average rate: 0,1/million flights Generation 3 From 1969 A310/A300-600, B737- — Jet EFIS cockpit — FMS equipped 300/400/500, B737-600/700/800 FADEC (NG), B737 MAX, B757, B767, Integrated auto flight control system — navigation B747-400, B747-8, B717, BAE performance, and terrain avoidance systems 146, MD11, MD80, MD90, F70, Basic flight envelope protection — stick shaker/pusher F100, Bombardier CRJ Series, Generation fatal accident average rate: 0,2/million flights Embraer ERJ 135/145 Generation 3 From 1992 ATR 42-600, ATR 72-600, — Turboprop EFIS cockpit — FMS equipped Bombardier Dash 8-400, BAE ATP, EEC/ECU or higher engine control Saab 2000 Integrated auto flight control system — navigation performance and terrain avoidance systems Basic flight envelope protection — stick shaker/pusher Generation 2 From 1964. A300 (except A300-600), BAC111, — Jet Integrated auto-flight system. B727, B737-100/200, B747- EEC/ECU or higher engine control 100/200/300, DC9, DC10, F28, Analogue/CRT instrument display L1011 Basic flight envelope protection — stick shaker/pusher Generation fatal accident average rate: 0,7/million flights Generation 2 ATR 42, ATR 72 (all series except - From 1964 — Turboprop 600), BAE J-41, Fokker F27/50, Analogue/CRT instrument display Bombardier Dash 7 and Dash 8- EEC/ECU 100/200/300 Series, Convair 580- Basic flight envelope protection — stick shaker/pusher 600 Series, Shorts 330 and 360, Integrated auto flight control system Saab 340, Embraer 120 Generation 1 From 1952 DC8, B707 — Jet First commercial jets. Manual engine control Analogue instrument display Not integrated auto flight control system Page 145 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Basic flight envelope protection — stick shaker/pusher, attitude warning Generation fatal accident average rate: 3.0/million flights CS OPS.EBT.232.1 The RMG developed this CS based on the following principles: (1) Automation and human interaction with this automation (2) Accident rate: data report for EBT as a reference for each generation (3) Technology driven, the fatal rate is qualifying This provision is transposed from ICAO Doc 9995 Appendix 1 with two differences: (1) There is a definition of each generation. (2) Embraer 120 was moved from GEN3 Turboprop to GEN2 Turboprop, the reason is as follows: — The equipment in Embraer 120 is really similar to that of ATR 42-500 (or ATR 200/300). ATR 42-500 and below are classified GEN2 Turboprop. Even though the RMG acknowledged that E120 has GPWS, they considered that this reason alone was not enough to classify the E120 as GEN3 Turboprop. — The new definitions of aircraft generation include a year for each generation. Therefore, Embraer 120 should be included in GEN2 Turboprop as it was certified in October 1985. — The RMG reviewed the number of Embraer 120 flying in Europe. Their number is low and therefore the possible impact of this change is low. AMC1 ORO.FC.232 EBT programme assessment and training topics ASSESSMENT AND TRAINING TOPICS Each table of assessment and training topics is specific to the aeroplane generation specified in the title. The component elements in the column headings of the matrix are as follows: (a) Assessment and training topic. A topic or grouping of topics derived from threats, errors or findings from data analysis, to be considered for assessment and mitigation by training. (b) Frequency. The priority of the topic to be considered in an EBT programme, according to the evidence derived from a large-scale analysis of operational data is linked to a recommended frequency. There are three levels of frequency: (1) A — assessment and training topic to be included with defined scenario elements during every EBT module; (2) B — assessment and training topic to be included with defined scenario elements during every cycle; (3) C — assessment and training topic to be included with defined scenario elements at least once in the 3-year period of the EBT programme. (c) Flight phase for activation. The flight phase for the realisation of the critical threat or error in the assessment and training scenario. Page 146 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail (d) Description (includes type of topic, being threat, error or focus). A description of the training topic. (e) Desired outcome (includes performance criteria or training outcome). Simple evaluative statements on the desired outcome. (f) Example scenario elements (guidance material). The example scenario elements address the training topic and detail the threat and/or error that the crew are exposed to. (g) Competency map. Competencies marked are those considered critical in managing the scenario. AMC1 ORO.FC.232 point (b) ‘Frequency’ The explanation provided for frequency was not transposed from Doc 9995, because the document provides two different definitions in paragraphs 1.2.3 and 1.4.2 of Part II. The Opinion provides a new definition for frequency using the new term ‘cycle’. This term is defined in Annex I (Definitions) to the Air OPS Regulation. The proposal reflects the intent of the provision of ICAO in regard to frequency. This principle is based on the yearly requirement for training topics with frequency B. GM1 ORO.FC.232 EBT programme assessment and training topics TABLE OF ASSESSMENT AND TRAINING TOPICS (a) The assessment and training topics usually have several example scenario elements. At least one example scenario element is selected (e.g. Gen 4 topic ‘Go-around’ in MT has three example scenario elements — the operator may choose one at each module (frequency A)). (b) Flight phase for activation: Abbreviation Flight phase Description ALL All Any or all phases of flight GND Flight planning, Ground phases up to when the crew increases thrust for taking-off preflight, engine start & taxi-out Taxi-in, engine From the speed that permits the aircraft to be manoeuvred by means of shut-down, taxiing for arriving at a parking area until the crew completes post-flight post-flight & and flight closing duties. flight closing TO Take-off This phase begins when the crew increases the thrust for taking-off. It ends after the speed and configuration are established at a defined manoeuvring altitude or to continue the climb for cruise. CLB Climb This phase begins when the crew establishes the aircraft at a defined speed and configuration enabling the aircraft to increase altitude for the purpose of cruise. It ends with the aircraft established at a predetermined constant initial cruise altitude at a defined speed. Page 147 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 2. Proposed amendments to CS, AMC & GM and rationale in detail Abbreviation Flight phase Description CRZ Cruise The cruise phase begins when the crew establishes the aircraft at a defined speed and predetermined constant initial cruise altitude and proceeds in the direction of a destination. It ends with the beginning of descent for an approach. DES Descent This phase begins when the crew departs the cruise altitude for an approach at a particular destination. It ends when the crew initiates changes in aircraft configuration and/or speed to facilitate a landing on a particular runway. APP Approach This phase begins when the crew initiates changes in aircraft configuration and/or speeds enabling the aircraft to manoeuvre for landing on a particular runway. It ends when the aircraft is in the landing configuration and the crew is dedicated to land on a specific runway. It also includes go- around where the crew aborts the descent to the planned landing runway during the approach phase. Go-around ends after speed and configuration are established at a defined manoeuvring altitude or to continue the climb for cruise. LDG Landing This phase begins when the aircraft is in the landing configuration and the crew is dedicated to touchdown on a specific runway. It ends when the speed permits the aircraft to be manoeuvred by means of taxiing for arrival at a parking area. GM1 ORO.FC.232 EBT The table was transposed from Table II-1-1 of ICAO Doc 9995. However, the table in the GM does not contain the column that matches each flight phase with the corresponding phase in the training criticality survey. For the sake of transparency, the information is provided below: Threats/Errors All flight phases Potential threats/errors in any or all phases of flight Pre-flight and taxi Phase 1 Pre-flight and taxi: flight preparation to completion of line-up Take-off Phase 2 From the application of take-off thrust until the completion of flap and slat retraction Climb Phase 3 From the completion of flap and slat retraction until top of climb Cruise Phase 4 From top of climb until top of descent Descent Phase 5 From top of descent until the earlier of first slat/flap extension or crossing the initial approach fix Approach Phase 6 From the earlier of first slat/flap extension or crossing the initial approach fix until 15 m (50 ft) AAL, including go-around Landing Phase 7 From 15 m (50 ft) AAL until reaching taxi speed Taxi and post-flight Phase 8 From reaching taxi speed until engine shutdown Page 148 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.232.2 Generation 4 (jet) table of assessment and training topics GENERATION 4 (JET) — TABLE OF ASSESSMENT AND TRAINING TOPICS (includes type of for activation threat, error or Flight phase Frequency Desired outcome topic, being Description Assessment and Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA (includes performance criteria OR training topic Example scenario elements training outcome) focus) Generation 4 Jet — Recurrent assessment and training matrix Competency map Section 1 — Skill retention Engine failure after the application of Rejected take-off B take-off thrust and before reaching V1 TO From initiation of take-off to complete stop (or as applicable to procedure) x x (may be in LVO or CAT I or above) Failure of the critical engine from V1 The manoeuvre is complete at a point when the aircraft is stabilised at normal engine-out Failure of the critical and before reaching V2 in the lowest climb speed with the correct pitch and lateral control, in trim condition and, as applicable, engine between V1 B TO x x CAT I visibility conditions or in LVO autopilot engagement. Only one failure of the critical engine between V1 and V2 a year and V2 conditions. may be done in LVO conditions. Failure of one engine from V1 and The manoeuvre is complete at a point when the aircraft is stabilised in a clean before reaching V2 in lowest CAT I configuration with engine-out procedures completed. Only one failure of the critical x x visibility conditions or in LVO engine between V1 and V2 a year may be done in LVO conditions. Failure of one engine conditions. B TO on take-off Failure of one engine above V2 (any Demonstrate manual aircraft control segment of the TO) in lowest CAT I skills with smoothness and accuracy as The manoeuvre is complete at a point when aircraft is stabilised in a clean configuration Manoeuvres training phase x x x visibility conditions or in LVO appropriate to the situation. with engine-out procedures completed conditions. Detect deviations through instrument Initiation of emergency descent from scanning. The manoeuvre is complete once the aircraft is stabilised in emergency descent Emergency descent C Maintain spare mental capacity during CRZ x x x normal cruise altitude configuration (and profile). manual aircraft control. Maintain the aircraft within the flight Engine-out approach With the critical engine failed, normal Initiation in a stabilised engine-out configuration from not less than 3 NM final approach, B envelope. LDG x x & landing landing until completion of roll-out Apply knowledge of the relationship between aircraft attitude, speed and With the critical engine failed, manually thrust. This manoeuvre should be flown from intercept to centreline until acceleration after go- flown normal precision approach to DA, Engine-out approach around. The manoeuvre is complete at a point when the aircraft is stabilised at normal B followed by manually flown go-around APP x x & go-around engine-out climb speed with the correct pitch and lateral control, in trim condition and, as — the whole manoeuvre to be flown applicable, autopilot engagement (describe generally critical part of manoeuvre). without visual reference High energy, initiation during the approach at 150 to 300 m (500 to 1 000 ft) below the Go-around, all engines operative x x x missed approach level-off altitude Go-around A APP Go-around, all engines operative Initiation of a go-around from DA followed by visual circuit and landing x x x Go-around, all engines operative During flare/rejected landing x x x Pilot qualification to Only for commanders whose duties operate in either B require them to operate in either pilot’s APP Complete the manoeuvres mandated in ORO.FC.235. Intentionally left in blank. pilot’s seat seat Page 149 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail for activation Flight phase Assessment and Description (includes type of topic, being Desired outcome Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA Frequency (includes performance criteria OR Example scenario elements training topic threat, error or focus) training outcome) Generation 4 Jet — Recurrent assessment and training matrix Competency map Section 2 — Equivalency of approaches relevant to operations See equivalency of approaches Approach type A or B Approach type A or B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X B additional demand on a proficient crew See equivalency of approaches MT Approach type A B Approach type A flight method 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval See equivalency of approaches Approach type A B Approach type A flight method 3D or 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew EVAL or SBT See equivalency of approaches Approach type B B Approach type B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval Page 150 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map Section 3 — Training topics with frequency (A) in alphabetical order GND Predictive wind shear warning before take-off, as applicable x x x ALL Adverse-weather scenario, e.g. thunderstorm activity, precipitation, icing x x x x TO Wind shear encounter during take-off, not predictive x x x X Evaluation or scenario-based training phases TO Predictive wind shear warning during take-off x x x x TO Crosswinds with or without strong gusts on take-off x x CRZ Turbulence that increases to severe turbulence x x x x CRZ Wind shear encounter scenario during cruise x x x x x Anticipate adverse weather. APP Thunderstorm, heavy rain, Reactive wind shear warning during approach or go-around x x x x Prepare for suspected adverse APP turbulence, ice build-up to include de- Predictive wind shear warning during approach or go-around x x x x weather. APP icing issues, as well as high- Thunderstorm encounter during approach or on missed approach x x x Recognise adverse weather. Adverse weather A APP temperature conditions. Increasing tailwind on final approach (not reported) x x x x Take appropriate action. APP The proper use of anti-ice and de- Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x Apply the appropriate procedure icing systems should be included downdrafts, gusts and crosswinds including shifting wind directions correctly. APP generally in appropriate scenarios. Non-precision approach in cold-temperature conditions, requiring altitude x x x Assure aircraft control. compensation for temperature, as applicable to type APP Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) In approach, unexpected braking action ‘good to medium’ reported by the preceding x x x x APP aircraft APP Reduced visibility even after acquiring the necessary visual reference during approach, x x x due to rain or fog CLB ACAS warning, recovery and subsequent engagement of automation x x CRZ Know how and when to use the DES flight management system(s), APP guidance and automation. ALL Demonstrate correct methods for FMS tactical programming issues, e.g. step climb, runway changes, late clearances, x x X The purpose of this topic is to engagement and disengagement of destination re-programming, executing diversion encourage and develop effective CLB the auto flight system(s). Recoveries from TAWS, management of energy state to restore automated flight x x x flight path management through CRZ Demonstrate appropriate use of proficient and appropriate use of the DES flight guidance, auto thrust and flight management system(s), APP other automation systems. guidance and automation, including CLB Maintain mode awareness of the Amendments to ATC cleared levels during altitude capture modes to force mode x x x transitions between modes, EVAL or SBT CRZ auto flight system(s), including awareness and intervention monitoring, mode awareness, Automation DES engagement and automatic A vigilance and flexibility needed to management APP transitions. change from one mode to another. TO Revert to different modes when Late ATC clearance to an altitude below acceleration altitude x x x The means of mitigating errors are appropriate. TO included in this topic. The errors are Engine-out special terrain procedures x x x Detect deviations from the desired APP described as mishandled auto flight aircraft state (flight path, speed, CRZ systems, inappropriate mode Forcing AP disconnect followed by re-engagement, recovery from low- or high-speed x x x x attitude, thrust, etc.) and take selection, flight management events in cruise appropriate action. CRZ system(s) and inappropriate autopilot Engine failure in cruise to onset of descent using automation x x CRZ usage. Emergency descent x x X Anticipate mishandled auto flight DES system. Managing high-energy descent capturing descent path from above (correlation with x x x X APP Recognise mishandled auto flight unstable approach training) APP system. No ATC clearance received prior to commencement of approach or final descent x x x APP Reactive wind shear and recovery from the consequent high-energy state x x x Page 151 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map APP Take appropriate action if Automation fail to capture the approach altitude in descent (e.g. last altitude before the x x x x necessary. FAP). Ideally, the failure occurs when the workload is high (e.g. configuration of the Restore correct auto flight state. aircraft for final approach). APP Identify and manage Non-precision or infrequently flown approaches using the maximum available level of x x X consequences. automation APP Gear malfunction during approach x x x APP ATC clearances to waypoints beyond the programmed descent point for a coded final x x x X descent point during an approach utilising a final descent that is commanded by the flight management system APP Exposure to event or sequence of GPS failure prior to commencement of approach associated with position drift and a x x x X events to allow the pilot to build terrain alert DES awareness of human factors in Cabin crew report of water noise below the forward galley indicating a possible toilet x x x Evaluation or scenario-based training phases aviation and the human limitations. pipe leak, with consequent avionics failures CRZ This includes the development of Smoke removal but combined with a diversion until landing completed. x x x x x X the following competencies: GND Communication: Apron fuel spilling x x x Demonstrate: CRZ Important water leak in an aircraft galley x x x x — effective use of language; ALL — responsiveness to feedback; A relevant number of cabin crew are wounded or incapacitated. Additionally, the cabin and crew wounded or incapacitated are the most competent (e.g. senior cabin crew x x x — capability to state the plans member). ALL and resolve ambiguities. Unruly passenger(s) x x This encapsulates the general CRM GND Leadership and teamwork: Passenger oxygen: passenger service unit open and mask falling down x x x principles and objectives. It includes: Use appropriate authority to ALL communication; leadership and Passenger with medical problems — medical emergency x x ensure focus on the task. Support teamwork; problem-solving and CRZ others in completing tasks. decision-making; situation awareness Credible threat reported to the crew. Stowaway or fugitive on board. x x x x Problem-solving and decision- GND and management of information; and No METAR or TAFOR is available for destination due to industrial action at the Competencies making: x x x x workload management. destination airport non-technical A Detect deviations from the desired (CRM) CRZ state, evaluate problems, identify Credible bomb threat reported to crew x x x x Emphasis should be placed on the risk, consider alternatives and development of leadership, shown by APP select the best course of action. ACAS warning immediately following a go-around, with a descent manoeuvre required x x x x X EBT data sources to be a highly Continuously review progress and effective competency in mitigating adjust plans. risk and improving safety through Situation awareness and pilot performance. management of information: Have an awareness of the aircraft state in its environment; project and anticipate changes. Workload management: Prioritise, delegate and receive assistance to maximise focus on the task. Continuously monitor the EVAL or SBT flight progress. Page 152 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map Compliance failure. Consequences of Recognise that a compliance failure The following are examples of potential compliance failures and are not intended to be not complying with operating has occurred. developed as scenarios as part of an EBT module: instructions (e.g. SOPs). Make a verbal announcement. This is not intended to list example Take appropriate action if 1. Requesting flap beyond limit speed scenario elements, but instructors necessary. EVAL or SBT should ensure that observed non- Restore safe flight path if necessary. 2. Flaps or slats in the wrong position for phase of flight or approach compliances are taken as learning Manage consequences. Compliance A ALL Intentionally blank opportunities throughout the 3. Omitting an action as part of a procedure programme. In all modules of the programme, the FSTD should as far as 4. Failing to initiate or complete a checklist possible be treated like an aircraft, and non-compliances should not be 5. Using the wrong checklist for the situation accepted simply for expediency. APP Any threat or error that can result in Adverse-weather scenario leading to a reactive wind shear warning during approach x x x x circumstances that require a decision to perform a go-around, in addition to Adverse-weather scenario leading to a predictive wind shear warning during approach x x x x APP the execution of the go-around. Go- or go-around around scenarios should be fully Adverse-weather scenario, e.g. thunderstorm activity, heavy precipitation or icing x x x x APP developed to encourage effective forcing decision at or close to DA/MDA leadership and teamwork, in addition DA with visual reference in heavy precipitation with doubt about runway surface braking x x x x APP to problem-solving and decision- capability APP making, plus execution using manual Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x EVAL or SBT aircraft control or the flight Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x Go-around APP A management system(s) and (not reported) management automation as applicable. Design Adverse-wind scenario including strong gusts and/or crosswind out of limits below 15 m x x x APP should include the element of (50 ft) (not reported) surprise, and scenario-based go- Lost or difficult communications resulting in no approach clearance prior to x x x APP arounds should not be predictable and commencement of approach or final descent anticipated. This topic is completely distinct from the go-around Birds: large flocks of birds below DA once visual reference has been established x x x APP manoeuvre listed in the manoeuvres training section that is intended only System malfunction, landing gear malfunction during the approach APP to practise psychomotor skills and a simple application of the procedures. CLB Desired competency outcome: Flight with unreliable airspeed, which may or may not be recoverable x x x X CRZ Demonstrates manual aircraft Evaluation or scenario-based DES control skills with smoothness and APP accuracy as appropriate to the CLB situation Alternate flight control modes according to malfunction characteristics x x x X training phases CRZ Detects deviations through Manual aircraft DES Controls the flight path through instrument scanning A control APP manual control Maintains spare mental capacity CLB during manual aircraft control ACAS RA requires the pilot to descend or ATC immediate descent x x x CRZ Maintains the aircraft within the DES normal flight envelope APP Applies knowledge of the relationship between aircraft TAWS warning when deviating from planned descent routing, requiring immediate x x x DES attitude, speed and thrust response Page 153 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map Scenario immediately after take-off which requires an immediate and overweight x x x x TO landing TO Adverse wind, crosswinds with or without strong gusts on take-off x x Adverse weather, wind shear, wind shear encounter during take-off, with or without x x x TO reactive warnings TO Engine failure during initial climb, typically 30-60 m (100-200 ft) (autopilot off) x x x x Wind shear encounter scenario during cruise, significant and rapid change in wind speed x x x x x CRZ or down/updrafts, without wind shear warning Adverse weather, wind shear, wind shear encounter with or without warning during x x x x APP approach Adverse weather, deterioration in visibility or cloud base, or adverse wind, requiring a x x x x x x x APP go-around from visual circling approach, during the visual segment APP Interception of the glide slope from above (correlation with unstable approach training) x x x APP Adverse wind, crosswinds with or without strong gusts on approach, final and landing x x x LDG (within and beyond limits) Adverse weather, adverse wind, approach and landing in demanding weather x x x APP conditions, e.g. turbulence, up and downdrafts, gusts and crosswinds including shifting LDG wind directions APP Circling approach manually flown at night in minimum in-flight visibility to ensure X X X X LDG ground reference, minimum environmental lighting and no glide slope guidance lights APP Runway incursion during approach, which can be triggered by ATC at various altitudes x x x LDG or by visual contact during the landing phase LDG Adverse wind, visibility, type-specific, special consideration for long-bodied aircraft, x x x x landing in minimum visibility for visual reference, with crosswind LDG System malfunction, auto flight failure at DA during a low-visibility approach requiring a x x x x go-around flown manually APP Approach planned with autoland, followed by a failure below 1 000 ft requiring a go- x x x x LDG around and an immediate landing due to fuel shortage. ALL The scenarios should be realistic and Recognise mismanaged aircraft Deviations from the flight path, in pitch attitude, speed, altitude, bank angle x x relevant, and should be used for the state. In-seat instruction: x x purpose of demonstration and Observe the pilot’s behaviour: how Simple automation errors (e.g. incorrect mode selection, attempted engagement ALL reinforcement of effective monitoring. the pilot is mitigating errors, without the necessary conditions, entering wrong altitude or speed, failure to execute performing cross-checking, the desired mode) culminating in a need for direct intervention from the PM, and Modules in the FSTD should be treated monitoring performance and where necessary taking control. like those in an aircraft so that trainees dealing with a mismanaged aircraft In-seat instruction: x x x x Monitoring, APP have the opportunity to develop the state, in order to ensure that Unstable approach or speed/path/vertical rate not congruent with required state for EVAL or SBT cross-checking, competency with the practice of the observed deviations, errors and given flight condition error A right techniques and attitudes related mistakes are taken as learning x x x management, to these topics through pilot opportunities throughout the mismanaged performance, and that instructors programme. aircraft state have the opportunity to assess and Monitor flight path excursions. In-seat instruction: train these topics in a realistic Detect errors and threats through Demonstration exercise — recovery from bounced landing, adverse wind, strong gusts LDG environment. As shown by the EBT proper cross-checking during landing phase, resulting in a bounce and necessitating recovery action from the data report, these topics are of key performance. PM importance to improve safety in Make appropriate interventions operations. either verbally or by taking control if applicable. Page 154 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map In addition, the operator may also use Take appropriate action if these topics to develop scripted role- necessary. playing scenarios in the form of ISI Restore desired aircraft state. training. These scenarios cater for the Identify and manage need to monitor flight path excursions consequences. from the instructor pilot (PF), detect errors and make appropriate interventions, either verbally or by taking control as applicable. Demonstration scenarios may also be used. Demonstrated role-play should contain realistic and not gross errors, leading at times to a mismanaged aircraft state, which can also be combined with upset management training. DES ATC or terrain-related environment creating a high-energy descent with the need to x x x APP Reinforce stabilised approach capture the optimum profile to complete the approach in a stabilised configuration philosophy and adherence to defined DES ATC or terrain-related environment creating a high-energy descent leading to unstable x x x parameters. Encourage go-arounds APP conditions and requiring a go-around Unstable when crews are outside these A Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x approach APP parameters. Develop and sustain downdrafts, gusts and crosswinds including shifting wind directions competencies related to the APP Increasing tailwind on final approach (not reported) x x x x management of high-energy APP situations. Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) Section 3 — Training topics with frequency (B) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases Compliance with AMC1 or AMC2 to See Table 1 of AMC1 ORO.FC.220&230: Elements and respective components of upset Intentionally blank N/A ORO.FC.220&230 prevention training. Include upset prevention elements in Demonstration of the defined normal flight envelope and any associated changes in x x x Table 1 for the recurrent training flight instruments, flight director systems, and protection systems. This should take CRZ programme at least every 12 calendar the form of an instructor-led exercise to show the crew the points beyond which an months, such that all the elements upset condition could exist. Early recognition and prevention of TO Severe wind shear or wake turbulence during take-off or approach x x x x EVAL, MT or SBT are covered over a period not upset conditions. APP exceeding 3 years. The elements are Upset prevention CRZ numbered with letters from A to I in As applicable and relevant to aircraft type, demonstration at a suitable intermediate x x x B When the differences between LHS training Table 1 of AMC1 ORO.FC.220&230. level, with turbulence as appropriate; practise steep turns and note the relationship and RHS are not significant in the Each element is made up of several between bank angle, pitch and stalling speed handling of the aircraft, UPRT may CRZ numbered components. At the maximum cruise flight level for current aircraft weight, turbulence to trigger over x x x x be conducted in either seat. Through the principles of EBT, speed conditions (if FSTD capability exists, consider use of vertical wind component to covering one component should add realism) CRZ satisfy the requirement to cover the At the maximum cruise flight level for current aircraft weight, turbulence and significant x x x X element. temperature rise to trigger low-speed conditions (if FSTD capability exists, consider use of vertical wind component to add realism) CRZ High-altitude TCAS RA. Where the RA is required to be flown in manual flight x x x x TO Take-off with different crosswind/tailwind/gust conditions x x EVAL or Adverse wind/crosswind. This Recognise adverse-wind conditions. TO Take-off with unreported tailwind x x SBT Adverse wind B includes tailwind but not ATC mis- Observe limitations. TO Crosswinds with or without strong gusts on take-off x x reporting of the actual wind. Apply appropriate procedures. APP Increasing tailwind on final approach(not reported) x x x x Page 155 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map Maintain directional control and Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x APP safe flight path. downdrafts, gusts and crosswind including shifting wind directions APP Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x APP (not reported) Adverse-wind scenario including strong gusts and/or crosswind out of limits below x x x APP 15 m (50 ft) (not reported) APP Crosswind with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) For full details, see the malfunction equivalency methodology. Unless specified Intentionally blank otherwise in the operational suitability data, at least one malfunction with each characteristic should be included every year. Combining characteristics should not reduce the number of malfunctions below seven for each year. For each crew member, the characteristics of degraded control and loss of instrumentation should be in the role of pilot flying and the others may be in the role of pilot flying or pilot monitoring. (i) System malfunctions that require immediate and urgent crew intervention or decision, e.g. fire, smoke, loss of pressurisation at high altitude, failures during take- off, brake failure during landing. ALL (ii) System malfunctions that require complex procedures, e.g. multiple hydraulic Any internal failure(s) apparent or not system failures, smoke and fumes procedures, major electrical system failure. apparent to the crew Evaluation or scenario-based training phases (iii) System malfunctions that result in significant degradation of flight controls in Recognise system malfunction. combination with abnormal handling characteristics, e.g. jammed flight controls, Any item cleared by the MEL but Take appropriate action including having an impact upon flight correct stop/go decision. certain degradation of FBW control, jammed horizontal stabiliser; flaps and/or slats operations. For instance. thrust Apply the appropriate procedure locked; other malfunctions that result in degraded flight controls. (iv) System failures that require monitoring and management of the flight path using reverser locked. correctly. degraded or alternative displays, unreliable primary flight path information, unreliable Aircraft system Maintain aircraft control. malfunctions, Malfunctions to be considered should Manage consequences. airspeed, e.g. flight with unreliable airspeed (v) System failures that require extensive management of their consequences including B have one or more of the following (independent of operation or environment), e.g. fuel leak. operations under characteristics: Apply crew operating procedures MEL TO — Immediacy where necessary. MEL items with crew operating procedures applicable during take-off x X — Complexity Respond appropriately to Response to an additional factor that is affected by a MEL item (e.g. system failure, x x x X TO — Degradation of aircraft control additional system abnormalities runway state) GND — Loss of primary instrumentation associated with MEL dispatch. Malfunction during preflight preparation and prior to departure x x x CLB — Management of consequences Malfunction after departure x x x X The operator should vary Malfunctions that require immediate attention (e.g. bleed fault during engine start, x x x ALL malfunctions for each characteristic hydraulic failure during taxi) CLB over the EBT cycle. x x x X Fuel leak (management of consequences) CRZ TO Malfunction on take-off high speed below V1 x x x TO Malfunction on take-off high speed above V1 x x During taxi to the runway, a spurious brake temperature announcement. The crew had GND X X X the correct brake temperature moments before the failure. TO Tyre failure during take-off x x x TO Malfunction on initial climb x x APP Malfunction on approach x x x APP Malfunction on go-around x x x LDG Malfunction during landing x x x x x Page 156 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map See ‘compliance’ topic above. There are no defined scenarios, but the instructor This is not considered as a stand- should focus on learning opportunities when system management non-compliances alone topic. It is linked with the manifest themselves during other scenarios. Underpinning knowledge of systems and Intentionally blank X topic ‘compliance’. Aircraft system Normal system operation according their interactions should be developed and challenged, and not merely the application B Where a system is not managed management to defined instructions of normal procedures. according to normal or defined CRZ x x x x procedures, this is determined as a Minimum fuel, caused by extended delays, weather, etc. where the crew would need APP non-compliance. to manage a minimum fuel situation. LDG APP Recognise actual conditions. Approach in poor visibility x x x x Observe aircraft and/or procedural Approach in poor visibility with deteriorations necessitating a decision to perform a x x x Evaluation or scenario-based training phases APP Approach, limitations. go-around Any situation where visibility visibility close to B Apply appropriate procedures if x x x becomes a threat minimum applicable. LDG Landing in poor visibility Maintain directional control and safe flight path. Pilots should have opportunities to practise landings in demanding situations at the defined frequency. Data indicates that landing problems have their roots in a variety of factors, Landing in demanding This topic should be combined with the adverse-weather topic, aircraft system including inappropriate decision- Landing B LDG environmental conditions, with malfunctions topic or any topic that can provide exposure to a landing in demanding Intentionally blank making, in addition to manual aircraft malfunctions as appropriate conditions. control skills if difficult environmental conditions exist. The purpose of this item is to ensure that pilots are exposed to this during the programme. GND x X Recognise hazardous runway Planned anticipated hazardous conditions with dispatch information provided to TO condition. facilitate planning and execution of appropriate procedures LDG Contamination or surface quality of Observe limitations. Runway or GND x x x B the runway, taxiway, or tarmac Take appropriate action. Unanticipated hazardous conditions, e.g. unexpected heavy rain resulting in flooded taxiway condition TO including foreign objects Apply appropriate procedures runway surface LDG correctly. TO Take-off on runway with reduced cleared width due to snow x x x x Assure aircraft control. TO Stop/go decision in hazardous conditions x x x The data analysed during the development of the EBT concept Evaluation or scenario-based indicated substantial difficulties encountered by crews when faced with a threat or error, which was a training phases surprise or an unexpected event. The Exposure to an unexpected event element of surprise should be or sequence of events at the Surprise B ALL Intentionally blank Intentionally blank distinguished from what is sometimes defined frequency in order to build referred to as the ‘startle factor’ — resilience. the latter being a physiological reaction. Wherever possible, consideration should be given towards variations in the types of scenario, times of occurrences and Page 157 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map types of occurrence, so that pilots do not become overly familiar with repetitions of the same scenarios. Variations should be the focus of EBT programme design, and not left to the discretion of individual instructors, in order to preserve programme integrity and fairness. ALL ATC clearance giving insufficient terrain clearance x x x X Anticipate terrain threats. Demonstration of terrain avoidance warning systems (TAWS) (this scenario element ALL Prepare for terrain threats. x x x may be done in an ISI.) Recognise unsafe terrain clearance. TO Take appropriate action. Engine failure where performance is marginal leading to TAWS warning x x x CLB Terrain B Alert, warning, or conflict Apply the appropriate procedures DES correctly. ATC provides a wrong QNH x x APP Maintain aircraft control. Restore safe flight path. ‘Virtual mountain’ refers to the surprise element of an unexpected warning. Care DES Manage consequences. should be exercised in creating a level of realism, so this can best be achieved by an x x x unusual and unexpected change of route during the descent. This is not considered a topic for specific attention on its own, but more as a reminder to programme developers to ensure that pilots are Manage available resources Workload, exposed to immersive training efficiently to prioritise and perform distraction, B ALL Intentionally blank Intentionally blank scenarios which expose them to tasks in a timely manner under all pressure, stress manageable high workload and circumstances distractions during the course of the EBT programme, at the defined frequency. Section 3 — Training topics frequency (C) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases N/A Compliance with AMC1 or AMC2 to Table 2 of AMC1 ORO.FC.220&230: Exercises for upset recovery training Intentionally blank ORO.FC.220&230 Recognise upset condition. A. Recovery from developed upsets Make timely and appropriate Recovery from stall events, in the following configurations; Include the recovery exercises in intervention. Manoeuvres training phase or SBT Table 2 of AMC1 ORO.FC.220&230 for Take appropriate action. — take-off configuration, CLB the recurrent training programme, Assure timely and appropriate 2. — clean configuration low altitude, x x x x DES such that all the exercises are covered intervention. (AMC1 over a period not exceeding 3 years. ORO.FC.220&230 Table 2 — clean configuration near maximum operating altitude, and Through the principles of EBT, component 1) Upset recovery C covering one component should Assure aircraft control. — landing configuration during the approach phase. CRZ satisfy the requirement to cover the Maintain or restore a safe flight 3. Recovery from nose high at various bank angles x x x x CRZ whole element of recovery from path. x x x x developed upsets. The same Assess consequential issues. 4. Recovery from nose low at various bank angles CRZ principles applies to the exercises of Manage outcomes. Demonstration at a normal cruising altitude. Set conditions and disable aircraft x x x APP components 2, 3 and 4 where one Consolidated summary of systems as necessary to enable trainee to perform stall recovery according to OEM exercise may satisfy the requirement aeroplane recovery techniques. instructions to cover the whole component. (AMC1 ORO.FC.220&230 Table 2 Demonstration at an intermediate altitude during early stages of the approach. Set x x x CLB An aeroplane upset is defined as an component 5) conditions and disable aircraft systems as necessary to enable trainee to perform stall DES undesired aeroplane state in flight recovery according to OEM instructions Page 158 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map characterised by unintentional Note: The operator should assess if divergences from parameters the exercises should be practice for normally experienced during line the either seat qualification. operations or training. An aeroplane upset may involve pitch and/or bank angle divergences as well as inappropriate airspeeds for the conditions. The example scenario elements may be done in ISI, as non-ISI or a Recovery from a wake turbulence position with high-bank angle x x x x combination of both. If done in ISI: The instructor should position the aircraft within but close to the edge of the validated training envelope before handing control to the trainee to demonstrate the restoration of normal flight. Careful consideration should be given to flying within the validated training envelope. ALL ATC role-play: the instructor provides scripted instructions, as a distraction to the crew x x x ALL ATC error. Omission, Controller error, provided by the instructor according to a defined scripted scenario x x x x Respond to communications ALL miscommunication, garbled, poor Frequency congestion, with multiple aircraft using the same frequency x appropriately. quality transmission. All of these act EVAL or SBT APP Recognise, clarify and resolve any Destination temporally closed x x x x as distractions to be managed by the CRZ ambiguities. Rescue and firefighting services (RFFS) level reduction at destination x x x ATC C crew. The scenarios should be Refuse or question unsafe Runway change before the interception of the localiser or similar navigation aid in x x x x APP combined where possible, with others instructions. azimuth of the same or higher weighting, the GND/ Use standard phraseology x x x principle reason being to create Stray dogs at the opposite threshold runway TO whenever possible. distractions. ALL Poor quality transmissions x TO Any engine failure or malfunction, Engine failure or engine malfunction on take-off low speed x x x x TO which causes loss or degradation of Engine failure or engine malfunction on take-off high speed below V1 x x x x TO thrust that affects performance. This Recognise engine failure. Engine failure or engine malfunction on take-off above V1 x x x x EVAL or SBT TO is distinct from the engine-out Take appropriate action. Engine failure or engine malfunction on initial climb x x x APP manoeuvres described in the Apply appropriate procedure Engine malfunction x x x Engine failure C CRZ manoeuvres training section above, correctly. Engine failure in cruise (with autopilot) x x x which are intended only to practise Maintain aircraft control. x psychomotor skills and reinforce Manage consequences. LDG procedures to manage engine Engine failure or engine malfunction on landing failures. GND Fire in cargo or cabin/cockpit at gate x x x x GND Recognise fire, smoke or fumes Fire during taxi x x x x X EVAL or SBT GND Take appropriate action. Fire with no cockpit indication x x x x X This includes engine, electric, Fire and smoke TO Apply appropriate procedure Take-off low speed x x x x X C pneumatic, cargo fire, smoke or management TO correctly. Fire or smoke on Take-off high speed below V1 x x x x fumes. TO Maintain aircraft control. Fire or smoke on Take-off high speed above V1 x x x TO Manage consequences. Fire or smoke on Initial climb x x x CRZ Cargo fire x x x Page 159 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map APP Engine fire in approach (extinguishable) x x APP Engine fire in approach (non-extinguishable) x x x CLB CRZ Lithium battery fire in the cockpit or cabin compartment x x x x x DES APP Flight deck or cabin fire x x x X GND Any of the example scenarios elements above ending in an evacuation x x x x GND Recognise loss of communications. Loss of communications during ground manoeuvring x x TO Lost or difficult communications. Take appropriate action. Loss of communications after take-off x x X Either through pilot mis-selection or a Execute appropriate procedure as x x x x X Loss of C failure external to the aircraft. This applicable communications APP could be for a few seconds or a total Use alternative ways to Loss of communications during approach phase, including go-around loss. communicate Manage consequences This can be a demonstrated error, in that the crew may be instructed to deliberately x x x Anticipate the potential for errors insert incorrect data — for example, to take off from an intersection with full-length ALL in load/fuel/performance data. performance information. The crew will be asked to intervene when acceleration is A calculation error by one or more Recognise inconsistencies. sensed to be lower than normal, and this may be part of the operator procedures, EVAL or SBT Managing especially when operating mixed fleets with considerable variations in MTOM. pilots, or someone involved with the Manage/avoid distractions. loading, fuel, C process, or the process itself, e.g. Make changes to Fuel ground staff on industrial action. Only limited amount of fuel available, which is x x x x performance GND incorrect information on the load paperwork/aircraft system(s) to below the calculated fuel for the flight. errors sheet eliminate error. Advice crew that there is a change of the load sheet figures during taxi to the runway. x x GND Identify and manage The crew may have limited time due to a calculated take-off time (CTOT) — ATC slot consequences. Braking action reported ‘medium’. The information is transmitted just before take-off. x x x GND The flight is subject to a calculated take-off time (CTOT) — ATC slot. x x x x External failure or a combination of external failures degrading aircraft navigation GND performance on ground Evaluation or scenario-based training phases TO x x x x CLB External failure or a combination of external failures degrading aircraft navigation APP performance in flight Recognise a NAV degradation. LDG Take appropriate action. External NAV failure. Standard initial departure change during taxi. The flight may be subject to a CTOT — GND Execute appropriate procedure as x x x Navigation C Loss of GPS satellite, ANP exceeding ATC slot. applicable. APP RNP, loss of external NAV source(s) Loss of runway lighting below decision height x x x Use alternative NAV guidance. Manage consequences. No fly zone: when the crew changes control frequency, the new ATCO informs the crew that they are flying over an unannounced ‘no fly zone’ and not included in the NOTAMs. To trigger such an event, the context can as an example be as follows: an unexpected CRZ military conflict in the territory the aircraft is flying over or the crew is forced to re-route x x x in flight and the new route flies over a city that has an important event such the Olympic games, a G20/G7 submit, or the route is flying near a space rocket launch close to the time of the launch, like Guiana Space Centre, cape Cañaveral, etc.). Operations- or C ALL Intentionally blank Intentionally blank Intentionally blank Intentionally blank type-specific Operations of The operator should comply with APP See equivalency of approaches special airport C the national qualification Intentionally blank Intentionally blank LDG relevant to operations. approval requirements published in the Page 160 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 4 Jet — Recurrent assessment and training matrix Competency map Aeronautical Information Publication. Recognise incapacitation. TO Take appropriate action including During take-off x x x x X correct stop/go decision. Pilot Consequences for the non- C Apply appropriate procedure incapacitation incapacitated pilot APP correctly. During approach x x x X Maintain aircraft control. Manage consequences. Anticipate potential loss of separation. Recognise loss of separation. CLB Traffic conflict. ACAS RA or TA, or Take appropriate action. Traffic C CRZ visual observation of conflict, which ACAS warning that requires crew intervention x x x x Apply appropriate procedure DES requires evasive manoeuvring correctly. Maintain aircraft control. Manage consequences. TO Anticipate potential for wind shear. Predictive wind shear warning during take-off x x TO Avoid known wind shear or prepare Wind shear encounter during take-off x x x TO for suspected wind shear. Wind shear encounter after rotation x x TO Recognise wind shear encounter. Predictive wind shear after rotation x x APP Take appropriate action. Predictive wind shear during approach x x x With or without warnings including Apply appropriate procedure predictive. A wind shear scenario is Wind shear correctly. C ideally combined into an adverse- recovery Assure aircraft control. weather scenario containing other Recognise out of wind shear elements. APP condition. Wind shear encounter during approach x x x Maintain or restore a safe flight path. Assess consequential issues and manage outcomes. Page 161 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.232.3 Generation 3 (jet) table of assessment and training topics GENERATION 3 (JET) — TABLE OF ASSESSMENT AND TRAINING TOPICS (includes type of for activation threat, error or Flight phase Desired outcome Frequency topic, being Description Assessment and Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA (includes performance criteria OR training topic Example scenario elements training outcome) focus) Generation 3 Jet — Recurrent assessment and training matrix Competency map Section 1 — Skill retention Engine failure after the application of Rejected take-off B take-off thrust and before reaching V1 TO From initiation of take-off to complete stop (or as applicable to procedure) x x (may be in LVO or CAT I or above) Failure of the critical engine from V1 The manoeuvre is complete at a point when the aircraft is stabilised at normal engine-out Failure of the critical and before reaching V2 in the lowest climb speed with the correct pitch and lateral control, in trim condition and, as applicable, engine between V1 A TO x x CAT I visibility conditions or in LVO autopilot engagement. Only one failure of the critical engine between V1 and V2 a year and V2 conditions. may be done in LVO conditions. Failure of one engine from V1 and The manoeuvre is complete at a point when the aircraft is stabilised in a clean before reaching V2 in lowest CAT I configuration with engine-out procedures completed. Only one failure of the critical x x visibility conditions or in LVO engine between V1 and V2 a year may be done in LVO conditions. Failure of one engine conditions. B TO on take-off Failure of one engine above V2 (any Demonstrate manual aircraft control segment of the TO) in lowest CAT I skills with smoothness and accuracy as The manoeuvre is complete at a point when aircraft is stabilised in a clean configuration Manoeuvres training phase x x x visibility conditions or in LVO appropriate to the situation. with engine-out procedures completed conditions. Detect deviations through instrument Initiation of emergency descent from scanning. The manoeuvre is complete once the aircraft is stabilised in emergency descent Emergency descent C Maintain spare mental capacity during CRZ x x x normal cruise altitude configuration (and profile). manual aircraft control. Maintain the aircraft within the flight Engine-out approach With the critical engine failed, normal Initiation in a stabilised engine-out configuration from not less than 3 NM final approach, B envelope. LDG x x & landing landing until completion of roll-out Apply knowledge of the relationship between aircraft attitude, speed and With the critical engine failed, manually thrust. This manoeuvre should be flown from intercept to centreline until acceleration after go- flown normal precision approach to DA, Engine-out approach around. The manoeuvre is complete at a point when the aircraft is stabilised at normal B followed by manually flown go-around APP x x & go-around engine-out climb speed with the correct pitch and lateral control, in trim condition and, as — the whole manoeuvre to be flown applicable, autopilot engagement (describe generally critical part of manoeuvre). without visual reference High energy, initiation during the approach at 150 to 300 m (500 to 1 000 ft) below the Go-around, all engines operative x x x missed approach level-off altitude Go-around A APP Go-around, all engines operative Initiation of a go-around from DA followed by visual circuit and landing x x x Go-around, all engines operative During flare/rejected landing x x x Pilot qualification to Only for commanders whose duties operate in either B require them to operate in either pilot’s APP Complete the manoeuvres mandated in ORO.FC.235. Intentionally left in blank. pilot’s seat seat Page 162 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail for activation Flight phase Assessment and Description (includes type of topic, being Desired outcome Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA Frequency (includes performance criteria OR Example scenario elements training topic threat, error or focus) training outcome) Generation 3 Jet — Recurrent assessment and training matrix Competency map Section 2 — Equivalency of approaches relevant to operations See equivalency of approaches Approach type A or B Approach type A or B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X B additional demand on a proficient crew See equivalency of approaches MT Approach type A B Approach type A flight method 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval See equivalency of approaches Approach type A B Approach type A flight method 3D or 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew EVAL or SBT See equivalency of approaches Approach type B B Approach type B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval Page 163 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map Section 3 — Training topics with frequency (A) in alphabetical order GND Predictive wind shear warning before take-off, as applicable x x x ALL Adverse-weather scenario, e.g. thunderstorm activity, precipitation, icing x x x x TO Wind shear encounter during take-off, not predictive x x x X Evaluation or scenario-based training phases TO Predictive wind shear warning during take-off x x x x TO Crosswinds with or without strong gusts on take-off x x CRZ Turbulence that increases to severe turbulence x x x x CRZ Wind shear encounter scenario during cruise x x x x x Anticipate adverse weather. APP Thunderstorm, heavy rain, Reactive wind shear warning during approach or go-around x x x x Prepare for suspected adverse APP turbulence, ice build-up to include de- Predictive wind shear warning during approach or go-around x x x x weather. APP icing issues, as well as high- Thunderstorm encounter during approach or on missed approach x x x Recognise adverse weather. Adverse weather A APP temperature conditions. Increasing tailwind on final approach (not reported) x x x x Take appropriate action. APP The proper use of anti-ice and de- Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x Apply the appropriate procedure icing systems should be included downdrafts, gusts and crosswinds including shifting wind directions correctly. APP generally in appropriate scenarios. Non-precision approach in cold-temperature conditions, requiring altitude x x x Assure aircraft control. compensation for temperature, as applicable to type APP Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) In approach, unexpected braking action ‘good to medium’ reported by the preceding x x x x APP aircraft APP Reduced visibility even after acquiring the necessary visual reference during approach, x x x due to rain or fog CLB ACAS warning, recovery and subsequent engagement of automation x x CRZ Know how and when to use the DES flight management system(s), APP guidance and automation. ALL Demonstrate correct methods for FMS tactical programming issues, e.g. step climb, runway changes, late clearances, x x X The purpose of this topic is to engagement and disengagement of destination re-programming, executing diversion encourage and develop effective CLB the auto flight system(s). Recoveries from TAWS, management of energy state to restore automated flight x x x flight path management through CRZ Demonstrate appropriate use of proficient and appropriate use of the DES flight guidance, auto thrust and flight management system(s), APP other automation systems. guidance and automation, including CLB Maintain mode awareness of the Amendments to ATC cleared levels during altitude capture modes to force mode x x x transitions between modes, EVAL or SBT CRZ auto flight system(s), including awareness and intervention monitoring, mode awareness, Automation DES engagement and automatic A vigilance and flexibility needed to management APP transitions. change from one mode to another. TO Revert to different modes when Late ATC clearance to an altitude below acceleration altitude x x x The means of mitigating errors are appropriate. TO included in this topic. The errors are Engine-out special terrain procedures x x x Detect deviations from the desired APP described as mishandled auto flight aircraft state (flight path, speed, CRZ systems, inappropriate mode Forcing AP disconnect followed by re-engagement, recovery from low- or high-speed x x x x attitude, thrust, etc.) and take selection, flight management events in cruise appropriate action. CRZ system(s) and inappropriate autopilot Engine failure in cruise to onset of descent using automation x x CRZ usage. Emergency descent x x X Anticipate mishandled auto flight DES system. Managing high-energy descent capturing descent path from above (correlation with x x x X APP Recognise mishandled auto flight unstable approach training) APP system. No ATC clearance received prior to commencement of approach or final descent x x x APP Reactive wind shear and recovery from the consequent high-energy state x x x Page 164 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map APP Take appropriate action if Automation fail to capture the approach altitude in descent (e.g. last altitude before the x x x x necessary. FAP). Ideally, the failure occurs when the workload is high (e.g. configuration of the Restore correct auto flight state. aircraft for final approach). APP Identify and manage Non-precision or infrequently flown approaches using the maximum available level of x x X consequences. automation APP Gear malfunction during approach x x x APP ATC clearances to waypoints beyond the programmed descent point for a coded final x x x X descent point during an approach utilising a final descent that is commanded by the flight management system APP Exposure to event or sequence of GPS failure prior to commencement of approach associated with position drift and a x x x X events to allow the pilot to build terrain alert DES awareness of human factors in Cabin crew report of water noise below the forward galley indicating a possible toilet x x x Evaluation or scenario-based training phases aviation and the human limitations. pipe leak, with consequent avionics failures CRZ This includes the development of Smoke removal but combined with a diversion until landing completed. x x x x x X the following competencies: GND Communication: Apron fuel spilling x x x Demonstrate: CRZ Important water leak in an aircraft galley x x x x — effective use of language; ALL — responsiveness to feedback; A relevant number of cabin crew are wounded or incapacitated. Additionally, the and cabin crew wounded or incapacitated are the most competent (e.g. senior cabin crew x x x — capability to state the plans member). ALL and resolve ambiguities. Unruly passenger(s) x x This encapsulates the general CRM GND Leadership and teamwork: Passenger oxygen: passenger service unit open and mask falling down x x x principles and objectives. It includes: Use appropriate authority to ALL communication; leadership and Passenger with medical problems — medical emergency x x ensure focus on the task. Support teamwork; problem-solving and CRZ others in completing tasks. decision-making; situation awareness Credible threat reported to the crew. Stowaway or fugitive on board. x x x x Problem-solving and decision- GND and management of information; and No METAR or TAFOR is available for destination due to industrial action at the Competencies making: x x x x workload management. destination airport non-technical A Detect deviations from the desired (CRM) CRZ state, evaluate problems, identify Credible bomb threat reported to crew x x x x Emphasis should be placed on the risk, consider alternatives and development of leadership, shown by APP select the best course of action. ACAS warning immediately following a go-around, with a descent manoeuvre required x x x x X EBT data sources to be a highly Continuously review progress and effective competency in mitigating adjust plans. risk and improving safety through Situation awareness and pilot performance. management of information: Have an awareness of the aircraft state in its environment; project and anticipate changes. Workload management: Prioritise, delegate and receive assistance to maximise focus on the task. Continuously monitor the EVAL or SBT flight progress. Page 165 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map Compliance failure. Consequences of Recognise that a compliance failure The following are examples of potential compliance failures and are not intended to be not complying with operating has occurred. developed as scenarios as part of an EBT module: instructions (e.g. SOPs). Make a verbal announcement. This is not intended to list example Take appropriate action if 1. Requesting flap beyond limit speed scenario elements, but instructors necessary. EVAL or SBT should ensure that observed non- Restore safe flight path if necessary. 2. Flaps or slats in the wrong position for phase of flight or approach compliances are taken as learning Manage consequences. Compliance A ALL Intentionally blank opportunities throughout the 3. Omitting an action as part of a procedure programme. In all modules of the programme, the FSTD should as far as 4. Failing to initiate or complete a checklist possible be treated like an aircraft, and non-compliances should not be 5. Using the wrong checklist for the situation accepted simply for expediency. APP Any threat or error that can result in Adverse-weather scenario leading to a reactive wind shear warning during approach x x x x circumstances that require a decision to perform a go-around, in addition to Adverse-weather scenario leading to a predictive wind shear warning during approach x x x x APP the execution of the go-around. Go- or go-around around scenarios should be fully Adverse-weather scenario, e.g. thunderstorm activity, heavy precipitation or icing x x x x APP developed to encourage effective forcing decision at or close to DA/MDA leadership and teamwork, in addition DA with visual reference in heavy precipitation with doubt about runway surface x x x x APP to problem-solving and decision- braking capability APP making, plus execution using manual Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x EVAL or SBT aircraft control or the flight Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x Go-around APP A management system(s) and (not reported) management automation as applicable. Design Adverse-wind scenario including strong gusts and/or crosswind out of limits below 15 m x x x APP should include the element of (50 ft) (not reported) surprise, and scenario-based go- Lost or difficult communications resulting in no approach clearance prior to x x x APP arounds should not be predictable and commencement of approach or final descent anticipated. This topic is completely distinct from the go-around Birds: large flocks of birds below DA once visual reference has been established x x x APP manoeuvre listed in the manoeuvres training section that is intended only System malfunction, landing gear malfunction during the approach APP to practise psychomotor skills and a simple application of the procedures. CLB Desired competency outcome: Flight with unreliable airspeed, which may or may not be recoverable x x x X CRZ Demonstrates manual aircraft Evaluation or scenario-based DES control skills with smoothness and APP accuracy as appropriate to the CLB situation Alternate flight control modes according to malfunction characteristics x x x X training phases CRZ Detects deviations through Manual aircraft DES Controls the flight path through instrument scanning A control APP manual control Maintains spare mental capacity CLB during manual aircraft control ACAS RA requires the pilot to descend or ATC immediate descent x x x CRZ Maintains the aircraft within the DES normal flight envelope APP Applies knowledge of the relationship between aircraft TAWS warning when deviating from planned descent routing, requiring immediate x x x DES attitude, speed and thrust response Page 166 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map Scenario immediately after take-off which requires an immediate and overweight x x x x TO landing TO Adverse wind, crosswinds with or without strong gusts on take-off x x Adverse weather, wind shear, wind shear encounter during take-off, with or without x x x TO reactive warnings TO Engine failure during initial climb, typically 30-60 m (100-200 ft) (autopilot off) x x x x Wind shear encounter scenario during cruise, significant and rapid change in wind speed x x x x x CRZ or down/updrafts, without wind shear warning Adverse weather, wind shear, wind shear encounter with or without warning during x x x x APP approach Adverse weather, deterioration in visibility or cloud base, or adverse wind, requiring a x x x x x x x APP go-around from visual circling approach, during the visual segment APP Interception of the glide slope from above (correlation with unstable approach training) x x x APP Adverse wind, crosswinds with or without strong gusts on approach, final and landing x x x LDG (within and beyond limits) Adverse weather, adverse wind, approach and landing in demanding weather x x x APP conditions, e.g. turbulence, up and downdrafts, gusts and crosswinds including shifting LDG wind directions APP Circling approach manually flown at night in minimum in-flight visibility to ensure X X X X LDG ground reference, minimum environmental lighting and no glide slope guidance lights APP Runway incursion during approach, which can be triggered by ATC at various altitudes x x x LDG or by visual contact during the landing phase LDG Adverse wind, visibility, type-specific, special consideration for long-bodied aircraft, x x x x landing in minimum visibility for visual reference, with crosswind LDG System malfunction, auto flight failure at DA during a low-visibility approach requiring x x x x a go-around flown manually APP Approach planned with autoland, followed by a failure below 1 000 ft requiring a go- x x x x LDG around and an immediate landing due to fuel shortage. ALL The scenarios should be realistic and Recognise mismanaged aircraft Deviations from the flight path, in pitch attitude, speed, altitude, bank angle x x relevant, and should be used for the state. In-seat instruction: x x purpose of demonstration and Observe the pilot’s behaviour: how Simple automation errors (e.g. incorrect mode selection, attempted engagement ALL reinforcement of effective monitoring. the pilot is mitigating errors, without the necessary conditions, entering wrong altitude or speed, failure to execute performing cross-checking, the desired mode) culminating in a need for direct intervention from the PM, and Modules in the FSTD should be treated monitoring performance and where necessary taking control. like those in an aircraft so that trainees dealing with a mismanaged aircraft In-seat instruction: x x x x Monitoring, APP have the opportunity to develop the state, in order to ensure that Unstable approach or speed/path/vertical rate not congruent with required state for EVAL or SBT cross-checking, competency with the practice of the observed deviations, errors and given flight condition error A right techniques and attitudes related mistakes are taken as learning x x x management, to these topics through pilot opportunities throughout the mismanaged performance, and that instructors programme. aircraft state have the opportunity to assess and Monitor flight path excursions. In-seat instruction: train these topics in a realistic Detect errors and threats through Demonstration exercise — recovery from bounced landing, adverse wind, strong gusts LDG environment. As shown by the EBT proper cross-checking during landing phase, resulting in a bounce and necessitating recovery action from the data report, these topics are of key performance. PM importance to improve safety in Make appropriate interventions operations. either verbally or by taking control if applicable. Page 167 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map In addition, the operator may also use Take appropriate action if these topics to develop scripted role- necessary. playing scenarios in the form of ISI Restore desired aircraft state. training. These scenarios cater for the Identify and manage need to monitor flight path excursions consequences. from the instructor pilot (PF), detect errors and make appropriate interventions, either verbally or by taking control as applicable. Demonstration scenarios may also be used. Demonstrated role-play should contain realistic and not gross errors, leading at times to a mismanaged aircraft state, which can also be combined with upset management training. DES ATC or terrain-related environment creating a high-energy descent with the need to x x x APP Reinforce stabilised approach capture the optimum profile to complete the approach in a stabilised configuration philosophy and adherence to defined DES ATC or terrain-related environment creating a high-energy descent leading to unstable x x x parameters. Encourage go-arounds APP conditions and requiring a go-around Unstable when crews are outside these A Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x approach APP parameters. Develop and sustain downdrafts, gusts and crosswinds including shifting wind directions competencies related to the APP Increasing tailwind on final approach (not reported) x x x x management of high-energy APP situations. Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) Section 3 — Training topics with frequency (B) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases Compliance with AMC1 or AMC2 to See Table 1 of AMC1 ORO.FC.220&230: Elements and respective components of upset N/A Intentionally blank ORO.FC.220&230 prevention training. Include upset prevention elements in Demonstration of the defined normal flight envelope and any associated changes in x x x Table 1 for the recurrent training flight instruments, flight director systems, and protection systems. This should take CRZ programme at least every 12 calendar the form of an instructor-led exercise to show the crew the points beyond which an months, such that all the elements upset condition could exist. Early recognition and prevention of TO Severe wind shear or wake turbulence during take-off or approach x x x x EVAL, MT or SBT are covered over a period not upset conditions. APP exceeding 3 years. The elements are Upset prevention CRZ numbered with letters from A to I in As applicable and relevant to aircraft type, demonstration at a suitable intermediate x x x B When the differences between LHS training Table 1 of AMC1 ORO.FC.220&230. level, with turbulence as appropriate; practise steep turns and note the relationship and RHS are not significant in the Each element is made up of several between bank angle, pitch and stalling speed handling of the aircraft, UPRT may CRZ numbered components. At the maximum cruise flight level for current aircraft weight, turbulence to trigger over x x x x be conducted in either seat. Through the principles of EBT, speed conditions (if FSTD capability exists, consider use of vertical wind component to covering one component should add realism) CRZ satisfy the requirement to cover the At the maximum cruise flight level for current aircraft weight, turbulence and significant x x x X element. temperature rise to trigger low-speed conditions (if FSTD capability exists, consider use of vertical wind component to add realism) CRZ High-altitude TCAS RA. Where the RA is required to be flown in manual flight x x x x TO Recognise adverse-wind Take-off with different crosswind/tailwind/gust conditions x x EVAL or Adverse wind/crosswind. This TO conditions. Take-off with unreported tailwind x x SBT Adverse wind B includes tailwind but not ATC mis- TO Observe limitations. Crosswinds with or without strong gusts on take-off x x reporting of the actual wind. APP Apply appropriate procedures. Increasing tailwind on final approach(not reported) x x x x Page 168 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map Maintain directional control and Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x APP safe flight path. downdrafts, gusts and crosswind including shifting wind directions APP Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x APP (not reported) Adverse-wind scenario including strong gusts and/or crosswind out of limits below x x x APP 15 m (50 ft) (not reported) APP Crosswind with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) For full details, see the malfunction equivalency methodology. Unless specified otherwise in the operational suitability data, at least one malfunction with each characteristic should be included every year. Combining characteristics should not reduce the number of malfunctions below seven for each year. For each crew member, the characteristics of degraded control and loss of instrumentation should be in the role of pilot flying and the others may be in the role of pilot flying or pilot monitoring. (i) System malfunctions that require immediate and urgent crew intervention or decision, e.g. fire, smoke, loss of pressurisation at high altitude, failures during take- off, brake failure during landing. ALL (ii) System malfunctions that require complex procedures, e.g. multiple hydraulic Intentionally blank Any internal failure(s) apparent or not system failures, smoke and fumes procedures, major electrical system failure. apparent to the crew Evaluation or scenario-based training phases (iii) System malfunctions that result in significant degradation of flight controls in Recognise system malfunction. combination with abnormal handling characteristics, e.g. jammed flight controls, Any item cleared by the MEL but Take appropriate action including having an impact upon flight correct stop/go decision. certain degradation of FBW control, jammed horizontal stabiliser; flaps and/or slats operations. For instance. thrust Apply the appropriate procedure locked; other malfunctions that result in degraded flight controls. (iv) System failures that require monitoring and management of the flight path using reverser locked. correctly. degraded or alternative displays, unreliable primary flight path information, unreliable Aircraft system Maintain aircraft control. malfunctions, Malfunctions to be considered should Manage consequences. airspeed, e.g. flight with unreliable airspeed (v) System failures that require extensive management of their consequences including B have one or more of the following (independent of operation or environment), e.g. fuel leak. operations under characteristics: Apply crew operating procedures MEL TO — Immediacy where necessary. MEL items with crew operating procedures applicable during take-off x X — Complexity Respond appropriately to Response to an additional factor that is affected by a MEL item (e.g. system failure, x x x X TO — Degradation of aircraft control additional system abnormalities runway state) GND — Loss of primary instrumentation associated with MEL dispatch. Malfunction during preflight preparation and prior to departure x x x CLB — Management of consequences Malfunction after departure x x x X The operator should vary Malfunctions that require immediate attention (e.g. bleed fault during engine start, x x x ALL malfunctions for each characteristic hydraulic failure during taxi) CLB over the EBT cycle. x x x X Fuel leak (management of consequences) CRZ TO Malfunction on take-off high speed below V1 x x x TO Malfunction on take-off high speed above V1 x x During taxi to the runway, a spurious brake temperature announcement. The crew had GND X X X the correct brake temperature moments before the failure. TO Tyre failure during take-off x x x TO Malfunction on initial climb x x APP Malfunction on approach x x x APP Malfunction on go-around x x x LDG Malfunction during landing x x x x x Page 169 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map See ‘compliance’ topic above. There are no defined scenarios, but the instructor This is not considered as a stand- should focus on learning opportunities when system management non-compliances alone topic. It is linked with the manifest themselves during other scenarios. Underpinning knowledge of systems and Intentionally blank X topic ‘compliance’. Aircraft system Normal system operation according their interactions should be developed and challenged, and not merely the application B Where a system is not managed management to defined instructions of normal procedures. according to normal or defined CRZ x x x x procedures, this is determined as a Evaluation or scenario-based training phases Minimum fuel, caused by extended delays, weather, etc. where the crew would need APP non-compliance. to manage a minimum fuel situation. LDG APP Recognise actual conditions. Approach in poor visibility x x x x Observe aircraft and/or procedural Approach in poor visibility with deteriorations necessitating a decision to perform a x x x APP Approach, limitations. go-around Any situation where visibility visibility close to B Apply appropriate procedures if x x x becomes a threat minimum applicable. LDG Landing in poor visibility Maintain directional control and safe flight path. Pilots should have opportunities to practise landings in demanding situations at the defined frequency. Data indicates that landing problems have their roots in a variety of factors, Landing in demanding This topic should be combined with the adverse-weather topic, aircraft system including inappropriate decision- Landing B LDG environmental conditions, with malfunctions topic or any topic that can provide exposure to a landing in demanding Intentionally blank making, in addition to manual aircraft malfunctions as appropriate conditions. control skills if difficult environmental conditions exist. The purpose of this item is to ensure that pilots are exposed to this during the programme. The data analysed during the development of the EBT concept indicated substantial difficulties Evaluation or scenario-based training phases encountered by crews when faced with a threat or error, which was a surprise or an unexpected event. The element of surprise should be distinguished from what is sometimes referred to as the ‘startle factor’ — the latter being a physiological Exposure to an unexpected event reaction. Wherever possible, or sequence of events at the Surprise B ALL Intentionally blank Intentionally blank consideration should be given defined frequency in order to build towards variations in the types of resilience. scenario, times of occurrences and types of occurrence, so that pilots do not become overly familiar with repetitions of the same scenarios. Variations should be the focus of EBT programme design, and not left to the discretion of individual instructors, in order to preserve programme integrity and fairness. Page 170 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map TO Anticipate potential for wind shear. Predictive wind shear warning during take-off x x Avoid known wind shear or TO Wind shear encounter during take-off x x x prepare for suspected wind shear. TO Wind shear encounter after rotation x x Recognise wind shear encounter. TO Take appropriate action. Predictive wind shear after rotation x x APP With or without warnings including Predictive wind shear during approach x x x Apply appropriate procedure predictive. A wind shear scenario is Wind shear correctly. B ideally combined into an adverse- recovery Assure aircraft control. weather scenario containing other Recognise out of wind shear elements. APP condition. Wind shear encounter during approach x x x Maintain or restore a safe flight path. Assess consequential issues and manage outcomes. This is not considered a topic for specific attention on its own, but more as a reminder to programme EVAL or SBT developers to ensure that pilots are Manage available resources Workload, exposed to immersive training efficiently to prioritise and perform distraction, B ALL Intentionally blank Intentionally blank scenarios which expose them to tasks in a timely manner under all pressure, stress manageable high workload and circumstances distractions during the course of the EBT programme, at the defined frequency. Section 3 — Training topics frequency (C) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases N/A Compliance with AMC1 or AMC2 to Table 2 of AMC1 ORO.FC.220&230: Exercises for upset recovery training Intentionally blank ORO.FC.220&230 Recognise upset condition. A. Recovery from developed upsets Make timely and appropriate Recovery from stall events, in the following configurations; Include the recovery exercises in intervention. Manoeuvres training phase or SBT Table 2 of AMC1 ORO.FC.220&230 for Take appropriate action. — take-off configuration, CLB the recurrent training programme, Assure timely and appropriate 2. — clean configuration low altitude, x x x x DES such that all the exercises are covered intervention. (AMC1 over a period not exceeding 3 years. ORO.FC.220&230 Table 2 — clean configuration near maximum operating altitude, and Through the principles of EBT, component 1) Upset recovery C covering one component should Assure aircraft control. — landing configuration during the approach phase. CRZ satisfy the requirement to cover the Maintain or restore a safe flight 3. Recovery from nose high at various bank angles x x x x CRZ whole element of recovery from path. x x x x developed upsets. The same Assess consequential issues. 4. Recovery from nose low at various bank angles CRZ principles applies to the exercises of Manage outcomes. Demonstration at a normal cruising altitude. Set conditions and disable aircraft x x x APP components 2, 3 and 4 where one Consolidated summary of systems as necessary to enable trainee to perform stall recovery according to OEM exercise may satisfy the requirement aeroplane recovery techniques. instructions to cover the whole component. (AMC1 ORO.FC.220&230 Table 2 Demonstration at an intermediate altitude during early stages of the approach. Set x x x CLB An aeroplane upset is defined as an component 5) conditions and disable aircraft systems as necessary to enable trainee to perform stall DES undesired aeroplane state in flight recovery according to OEM instructions Page 171 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map characterised by unintentional Note: The operator should assess if divergences from parameters the exercises should be practice for normally experienced during line the either seat qualification. operations or training. An aeroplane upset may involve pitch and/or bank angle divergences as well as inappropriate airspeeds for the conditions. The example scenario elements may be done in ISI, as non-ISI or a Recovery from a wake turbulence position with high-bank angle x x x x combination of both. If done in ISI: The instructor should position the aircraft within but close to the edge of the validated training envelope before handing control to the trainee to demonstrate the restoration of normal flight. Careful consideration should be given to flying within the validated training envelope. ALL ATC role-play: the instructor provides scripted instructions, as a distraction to the crew x x x ALL ATC error. Omission, Controller error, provided by the instructor according to a defined scripted scenario x x x x Respond to communications ALL miscommunication, garbled, poor Frequency congestion, with multiple aircraft using the same frequency x appropriately. quality transmission. All of these act EVAL or SBT APP Recognise, clarify and resolve any Destination temporally closed x x x x as distractions to be managed by the CRZ ambiguities. Rescue and firefighting services (RFFS) level reduction at destination x x x ATC C crew. The scenarios should be Refuse or question unsafe Runway change before the interception of the localiser or similar navigation aid in x x x x APP combined where possible, with instructions. azimuth others of the same or higher GND/ Use standard phraseology x x x weighting, the principle reason being Stray dogs at the opposite threshold runway TO whenever possible. to create distractions. ALL Poor quality transmissions x TO Any engine failure or malfunction, Engine failure or engine malfunction on take-off low speed x x x x TO which causes loss or degradation of Engine failure or engine malfunction on take-off high speed below V1 x x x x TO thrust that affects performance. This Recognise engine failure. Engine failure or engine malfunction on take-off above V1 x x x x EVAL or SBT TO is distinct from the engine-out Take appropriate action. Engine failure or engine malfunction on initial climb x x x APP manoeuvres described in the Apply appropriate procedure Engine malfunction x x x Engine failure C CRZ manoeuvres training section above, correctly. Engine failure in cruise (with autopilot) x x x which are intended only to practise Maintain aircraft control. x psychomotor skills and reinforce Manage consequences. LDG procedures to manage engine Engine failure or engine malfunction on landing failures. GND Fire in cargo or cabin/cockpit at gate x x x x GND Recognise fire, smoke or fumes Fire during taxi x x x x X EVAL or SBT GND Take appropriate action. Fire with no cockpit indication x x x x X This includes engine, electric, Fire and smoke TO Apply appropriate procedure Take-off low speed x x x x X C pneumatic, cargo fire, smoke or management TO correctly. Fire or smoke on Take-off high speed below V1 x x x x fumes. TO Maintain aircraft control. Fire or smoke on Take-off high speed above V1 x x x TO Manage consequences. Fire or smoke on Initial climb x x x CRZ Cargo fire x x x Page 172 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map APP Engine fire in approach (extinguishable) x x APP Engine fire in approach (non-extinguishable) x x x CLB CRZ Lithium battery fire in the cockpit or cabin compartment x x x x x DES APP Flight deck or cabin fire x x x X GND Any of the example scenarios elements above ending in an evacuation x x x x GND Recognise loss of communications. Loss of communications during ground manoeuvring x x TO Lost or difficult communications. Take appropriate action. Loss of communications after take-off x x X EVAL or SBT Either through pilot mis-selection or a Execute appropriate procedure as x x x x X Loss of C failure external to the aircraft. This applicable communications APP could be for a few seconds or a total Use alternative ways to Loss of communications during approach phase, including go-around loss. communicate Manage consequences This can be a demonstrated error, in that the crew may be instructed to deliberately x x x Anticipate the potential for errors insert incorrect data — for example, to take off from an intersection with full-length ALL in load/fuel/performance data. performance information. The crew will be asked to intervene when acceleration is A calculation error by one or more Recognise inconsistencies. sensed to be lower than normal, and this may be part of the operator procedures, EVAL or SBT Managing especially when operating mixed fleets with considerable variations in MTOM. pilots, or someone involved with the Manage/avoid distractions. loading, fuel, C process, or the process itself, e.g. Make changes to Fuel ground staff on industrial action. Only limited amount of fuel available, which is x x x x performance GND incorrect information on the load paperwork/aircraft system(s) to below the calculated fuel for the flight. errors sheet eliminate error. Advice crew that there is a change of the load sheet figures during taxi to the runway. x x GND Identify and manage The crew may have limited time due to a calculated take-off time (CTOT) — ATC slot consequences. Braking action reported ‘medium’. The information is transmitted just before take-off. x x x GND The flight is subject to a calculated take-off time (CTOT) — ATC slot. x x x x External failure or a combination of external failures degrading aircraft navigation GND performance on ground Evaluation or scenario-based training phases TO x x x x CLB External failure or a combination of external failures degrading aircraft navigation APP performance in flight Recognise a NAV degradation. LDG Take appropriate action. External NAV failure. Standard initial departure change during taxi. The flight may be subject to a CTOT — GND Execute appropriate procedure as x x x Navigation C Loss of GPS satellite, ANP exceeding ATC slot. applicable. APP RNP, loss of external NAV source(s) Loss of runway lighting below decision height x x x Use alternative NAV guidance. Manage consequences. No fly zone: when the crew changes control frequency, the new ATCO informs the crew that they are flying over an unannounced ‘no fly zone’ and not included in the NOTAMs. To trigger such an event, the context can as an example be as follows: an unexpected CRZ military conflict in the territory the aircraft is flying over or the crew is forced to re-route x x x in flight and the new route flies over a city that has an important event such the Olympic games, a G20/G7 submit, or the route is flying near a space rocket launch close to the time of the launch, like Guiana Space Centre, cape Cañaveral, etc.). Operations- or C ALL Intentionally blank Intentionally blank Intentionally blank Intentionally blank type-specific Operations of The operator should comply with APP See equivalency of approaches special airport C the national qualification Intentionally blank Intentionally blank LDG relevant to operations. approval requirements published in the Page 173 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Jet — Recurrent assessment and training matrix Competency map Aeronautical Information Publication. Recognise incapacitation. TO Take appropriate action including During take-off x x x x X correct stop/go decision. Pilot Consequences for the non- C Apply appropriate procedure incapacitation incapacitated pilot APP correctly. During approach x x x X Maintain aircraft control. Manage consequences. GND x X Recognise hazardous runway Planned anticipated hazardous conditions with dispatch information provided to TO condition. facilitate planning and execution of appropriate procedures LDG Contamination or surface quality of Observe limitations. Runway or GND x x x EVAL or SBT C the runway, taxiway, or tarmac Take appropriate action. Unanticipated hazardous conditions, e.g. unexpected heavy rain resulting in flooded taxiway condition TO including foreign objects Apply appropriate procedures runway surface LDG correctly. TO Take-off on runway with reduced cleared width due to snow x x x x Assure aircraft control. TO Stop/go decision in hazardous conditions x x x ALL ATC clearance giving insufficient terrain clearance x x x X Anticipate terrain threats. Demonstration of terrain avoidance warning systems (TAWS) (this scenario element ALL Prepare for terrain threats. x x x may be done in an ISI.) Recognise unsafe terrain clearance. TO Take appropriate action. Engine failure where performance is marginal leading to TAWS warning x x x CLB Terrain C Alert, warning, or conflict Apply the appropriate procedures DES correctly. ATC provides a wrong QNH x x APP Maintain aircraft control. Restore safe flight path. ‘Virtual mountain’ refers to the surprise element of an unexpected warning. Care DES Manage consequences. should be exercised in creating a level of realism, so this can best be achieved by an x x x unusual and unexpected change of route during the descent. Anticipate potential loss of separation. Recognise loss of separation. CLB Traffic conflict. ACAS RA or TA, or Take appropriate action. Traffic C CRZ visual observation of conflict, which ACAS warning that requires crew intervention x x x x Apply appropriate procedure DES requires evasive manoeuvring correctly. Maintain aircraft control. Manage consequences. Page 174 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.232.4 Generation 3 (turboprop) table of assessment and training topics GENERATION 3 (TURBOPROP) — TABLE OF ASSESSMENT AND TRAINING TOPICS (includes type of for activation threat, error or Flight phase Desired outcome Frequency topic, being Description Assessment and Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA (includes performance criteria OR training topic Example scenario elements training outcome) focus) Generation 3 Turboprop — Recurrent assessment and training matrix Competency map Section 1 — Skill retention Engine failure after the application of Rejected take-off A take-off thrust and before reaching V1 TO From initiation of take-off to complete stop (or as applicable to procedure) x x (may be in LVO or CAT I or above) Failure of the critical engine from V1 The manoeuvre is complete at a point when the aircraft is stabilised at normal engine-out Failure of the critical and before reaching V2 in the lowest climb speed with the correct pitch and lateral control, in trim condition and, as applicable, engine between V1 A TO x x CAT I visibility conditions or in LVO autopilot engagement. Only one failure of the critical engine between V1 and V2 a year and V2 conditions. may be done in LVO conditions. Failure of one engine from V1 and The manoeuvre is complete at a point when the aircraft is stabilised in a clean before reaching V2 in lowest CAT I configuration with engine-out procedures completed. Only one failure of the critical x x visibility conditions or in LVO engine between V1 and V2 a year may be done in LVO conditions. Failure of one engine conditions. B TO on take-off Failure of one engine above V2 (any Demonstrate manual aircraft control segment of the TO) in lowest CAT I skills with smoothness and accuracy as The manoeuvre is complete at a point when aircraft is stabilised in a clean configuration Manoeuvres training phase x x x visibility conditions or in LVO appropriate to the situation. with engine-out procedures completed conditions. Detect deviations through instrument Initiation of emergency descent from scanning. The manoeuvre is complete once the aircraft is stabilised in emergency descent Emergency descent C Maintain spare mental capacity during CRZ x x x normal cruise altitude configuration (and profile). manual aircraft control. Maintain the aircraft within the flight Engine-out approach With the critical engine failed, normal Initiation in a stabilised engine-out configuration from not less than 3 NM final approach, A envelope. LDG x x & landing landing until completion of roll-out Apply knowledge of the relationship between aircraft attitude, speed and With the critical engine failed, manually thrust. This manoeuvre should be flown from intercept to centreline until acceleration after go- flown normal precision approach to DA, Engine-out approach around. The manoeuvre is complete at a point when the aircraft is stabilised at normal A followed by manually flown go-around APP x x & go-around engine-out climb speed with the correct pitch and lateral control, in trim condition and, as — the whole manoeuvre to be flown applicable, autopilot engagement (describe generally critical part of manoeuvre). without visual reference High energy, initiation during the approach at 150 to 300 m (500 to 1 000 ft) below the Go-around, all engines operative x x x missed approach level-off altitude Go-around A APP Go-around, all engines operative Initiation of a go-around from DA followed by visual circuit and landing x x x Go-around, all engines operative During flare/rejected landing x x x Pilot qualification to Only for commanders whose duties operate in either B require them to operate in either pilot’s APP Complete the manoeuvres mandated in ORO.FC.235. Intentionally left in blank. pilot’s seat seat Page 175 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail for activation Flight phase Assessment and Description (includes type of topic, being Desired outcome Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA Frequency (includes performance criteria OR Example scenario elements training topic threat, error or focus) training outcome) Generation 3 Turboprop — Recurrent assessment and training matrix Competency map Section 2 — Equivalency of approaches relevant to operations See equivalency of approaches Approach type A or B Approach type A or B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X B additional demand on a proficient crew See equivalency of approaches MT Approach type A B Approach type A flight method 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval See equivalency of approaches Approach type A B Approach type A flight method 3D or 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew EVAL or SBT See equivalency of approaches Approach type B B Approach type B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval Page 176 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map Section 3 — Training topics with frequency (A) in alphabetical order GND Predictive wind shear warning before take-off, as applicable x x x ALL Adverse-weather scenario, e.g. thunderstorm activity, precipitation, icing x x x x TO Wind shear encounter during take-off, not predictive x x x X Evaluation or scenario-based training phases TO Predictive wind shear warning during take-off x x x x TO Crosswinds with or without strong gusts on take-off x x CRZ Turbulence that increases to severe turbulence x x x x CRZ Wind shear encounter scenario during cruise x x x x x Anticipate adverse weather. APP Thunderstorm, heavy rain, Reactive wind shear warning during approach or go-around x x x x Prepare for suspected adverse APP turbulence, ice build-up to include de- Predictive wind shear warning during approach or go-around x x x x weather. APP icing issues, as well as high- Thunderstorm encounter during approach or on missed approach x x x Recognise adverse weather. Adverse weather A APP temperature conditions. Increasing tailwind on final approach (not reported) x x x x Take appropriate action. APP The proper use of anti-ice and de- Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x Apply the appropriate procedure icing systems should be included downdrafts, gusts and crosswinds including shifting wind directions correctly. APP generally in appropriate scenarios. Non-precision approach in cold-temperature conditions, requiring altitude x x x Assure aircraft control. compensation for temperature, as applicable to type APP Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) In approach, unexpected braking action ‘good to medium’ reported by the preceding x x x x APP aircraft APP Reduced visibility even after acquiring the necessary visual reference during approach, x x x due to rain or fog CLB ACAS warning, recovery and subsequent engagement of automation x x CRZ Know how and when to use the DES flight management system(s), APP guidance and automation. ALL Demonstrate correct methods for FMS tactical programming issues, e.g. step climb, runway changes, late clearances, x x X The purpose of this topic is to engagement and disengagement of destination re-programming, executing diversion encourage and develop effective CLB the auto flight system(s). Recoveries from TAWS, management of energy state to restore automated flight x x x flight path management through CRZ Demonstrate appropriate use of proficient and appropriate use of the DES flight guidance, auto thrust and flight management system(s), APP other automation systems. guidance and automation, including CLB Maintain mode awareness of the Amendments to ATC cleared levels during altitude capture modes to force mode x x x transitions between modes, EVAL or SBT CRZ auto flight system(s), including awareness and intervention monitoring, mode awareness, Automation DES engagement and automatic A vigilance and flexibility needed to management APP transitions. change from one mode to another. TO Revert to different modes when Late ATC clearance to an altitude below acceleration altitude x x x The means of mitigating errors are appropriate. TO included in this topic. The errors are Engine-out special terrain procedures x x x Detect deviations from the desired APP described as mishandled auto flight aircraft state (flight path, speed, CRZ systems, inappropriate mode Forcing AP disconnect followed by re-engagement, recovery from low- or high-speed x x x x attitude, thrust, etc.) and take selection, flight management events in cruise appropriate action. CRZ system(s) and inappropriate autopilot Engine failure in cruise to onset of descent using automation x x CRZ usage. Emergency descent x x X Anticipate mishandled auto flight DES system. Managing high-energy descent capturing descent path from above (correlation with x x x X APP Recognise mishandled auto flight unstable approach training) APP system. No ATC clearance received prior to commencement of approach or final descent x x x APP Reactive wind shear and recovery from the consequent high-energy state x x x Page 177 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map APP Take appropriate action if Automation fail to capture the approach altitude in descent (e.g. last altitude before the x x x x necessary. FAP). Ideally, the failure occurs when the workload is high (e.g. configuration of the Restore correct auto flight state. aircraft for final approach). APP Identify and manage Non-precision or infrequently flown approaches using the maximum available level of x x X consequences. automation APP Gear malfunction during approach x x x APP ATC clearances to waypoints beyond the programmed descent point for a coded final x x x X descent point during an approach utilising a final descent that is commanded by the flight management system APP Exposure to event or sequence of GPS failure prior to commencement of approach associated with position drift and a x x x X events to allow the pilot to build terrain alert DES awareness of human factors in Cabin crew report of water noise below the forward galley indicating a possible toilet x x x Evaluation or scenario-based training phases aviation and the human limitations. pipe leak, with consequent avionics failures CRZ This includes the development of Smoke removal but combined with a diversion until landing completed. x x x x x X the following competencies: GND Communication: Apron fuel spilling x x x Demonstrate: CRZ Important water leak in an aircraft galley x x x x — effective use of language; ALL — responsiveness to feedback; A relevant number of cabin crew are wounded or incapacitated. Additionally, the and cabin crew wounded or incapacitated are the most competent (e.g. senior cabin crew x x x — capability to state the plans member). ALL and resolve ambiguities. Unruly passenger(s) x x This encapsulates the general CRM GND Leadership and teamwork: Passenger oxygen: passenger service unit open and mask falling down x x x principles and objectives. It includes: Use appropriate authority to ALL communication; leadership and Passenger with medical problems — medical emergency x x ensure focus on the task. Support teamwork; problem-solving and CRZ others in completing tasks. decision-making; situation awareness Credible threat reported to the crew. Stowaway or fugitive on board. x x x x Problem-solving and decision- GND and management of information; and No METAR or TAFOR is available for destination due to industrial action at the Competencies making: x x x x workload management. destination airport non-technical A Detect deviations from the desired (CRM) CRZ state, evaluate problems, identify Credible bomb threat reported to crew x x x x Emphasis should be placed on the risk, consider alternatives and development of leadership, shown by APP select the best course of action. ACAS warning immediately following a go-around, with a descent manoeuvre required x x x x X EBT data sources to be a highly Continuously review progress and effective competency in mitigating adjust plans. risk and improving safety through Situation awareness and pilot performance. management of information: Have an awareness of the aircraft state in its environment; project and anticipate changes. Workload management: Prioritise, delegate and receive assistance to maximise focus on the task. Continuously monitor the EVAL or SBT flight progress. Page 178 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map Compliance failure. Consequences of Recognise that a compliance failure The following are examples of potential compliance failures and are not intended to be Intentionally blank not complying with operating has occurred. developed as scenarios as part of an EBT module: instructions (e.g. SOPs). Make a verbal announcement. This is not intended to list example Take appropriate action if 1. Requesting flap beyond limit speed scenario elements, but instructors necessary. EVAL or SBT should ensure that observed non- Restore safe flight path if necessary. 2. Flaps or slats in the wrong position for phase of flight or approach compliances are taken as learning Manage consequences. Compliance A ALL opportunities throughout the 3. Omitting an action as part of a procedure programme. In all modules of the programme, the FSTD should as far as 4. Failing to initiate or complete a checklist possible be treated like an aircraft, and non-compliances should not be 5. Using the wrong checklist for the situation accepted simply for expediency. APP Any threat or error that can result in Adverse-weather scenario leading to a reactive wind shear warning during approach x x x x circumstances that require a decision to perform a go-around, in addition to Adverse-weather scenario leading to a predictive wind shear warning during approach x x x x APP the execution of the go-around. Go- or go-around around scenarios should be fully Adverse-weather scenario, e.g. thunderstorm activity, heavy precipitation or icing x x x x APP developed to encourage effective forcing decision at or close to DA/MDA leadership and teamwork, in addition DA with visual reference in heavy precipitation with doubt about runway surface x x x x APP to problem-solving and decision- braking capability APP making, plus execution using manual Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x EVAL or SBT aircraft control or the flight Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x Go-around APP A management system(s) and (not reported) management automation as applicable. Design Adverse-wind scenario including strong gusts and/or crosswind out of limits below 15 m x x x APP should include the element of (50 ft) (not reported) surprise, and scenario-based go- Lost or difficult communications resulting in no approach clearance prior to x x x APP arounds should not be predictable and commencement of approach or final descent anticipated. This topic is completely distinct from the go-around Birds: large flocks of birds below DA once visual reference has been established x x x APP manoeuvre listed in the manoeuvres training section that is intended only System malfunction, landing gear malfunction during the approach APP to practise psychomotor skills and a simple application of the procedures. CLB Desired competency outcome: Flight with unreliable airspeed, which may or may not be recoverable x x x X CRZ Demonstrates manual aircraft Evaluation or scenario-based DES control skills with smoothness and APP accuracy as appropriate to the CLB situation Alternate flight control modes according to malfunction characteristics x x x X training phases CRZ Detects deviations through Manual aircraft DES Controls the flight path through instrument scanning A control APP manual control Maintains spare mental capacity CLB during manual aircraft control ACAS RA requires the pilot to descend or ATC immediate descent x x x CRZ Maintains the aircraft within the DES normal flight envelope APP Applies knowledge of the relationship between aircraft TAWS warning when deviating from planned descent routing, requiring immediate x x x DES attitude, speed and thrust response Page 179 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map Scenario immediately after take-off which requires an immediate and overweight x x x x TO landing TO Adverse wind, crosswinds with or without strong gusts on take-off x x Adverse weather, wind shear, wind shear encounter during take-off, with or without x x x TO reactive warnings TO Engine failure during initial climb, typically 30-60 m (100-200 ft) (autopilot off) x x x x Wind shear encounter scenario during cruise, significant and rapid change in wind speed x x x x x CRZ or down/updrafts, without wind shear warning Adverse weather, wind shear, wind shear encounter with or without warning during x x x x APP approach Adverse weather, deterioration in visibility or cloud base, or adverse wind, requiring a x x x x x x x APP go-around from visual circling approach, during the visual segment APP Interception of the glide slope from above (correlation with unstable approach training) x x x APP Adverse wind, crosswinds with or without strong gusts on approach, final and landing x x x LDG (within and beyond limits) Adverse weather, adverse wind, approach and landing in demanding weather x x x APP conditions, e.g. turbulence, up and downdrafts, gusts and crosswinds including shifting LDG wind directions APP Circling approach manually flown at night in minimum in-flight visibility to ensure X X X X LDG ground reference, minimum environmental lighting and no glide slope guidance lights APP Runway incursion during approach, which can be triggered by ATC at various altitudes x x x LDG or by visual contact during the landing phase LDG Adverse wind, visibility, type-specific, special consideration for long-bodied aircraft, x x x x landing in minimum visibility for visual reference, with crosswind LDG System malfunction, auto flight failure at DA during a low-visibility approach requiring x x x x a go-around flown manually APP Approach planned with autoland, followed by a failure below 1 000 ft requiring a go- x x x x LDG around and an immediate landing due to fuel shortage. ALL The scenarios should be realistic and Recognise mismanaged aircraft Deviations from the flight path, in pitch attitude, speed, altitude, bank angle x x relevant, and should be used for the state. In-seat instruction: x x purpose of demonstration and Observe the pilot’s behaviour: how Simple automation errors (e.g. incorrect mode selection, attempted engagement ALL reinforcement of effective monitoring. the pilot is mitigating errors, without the necessary conditions, entering wrong altitude or speed, failure to execute performing cross-checking, the desired mode) culminating in a need for direct intervention from the PM, and Modules in the FSTD should be treated monitoring performance and where necessary taking control. like those in an aircraft so that trainees dealing with a mismanaged aircraft In-seat instruction: x x x x Monitoring, APP have the opportunity to develop the state, in order to ensure that Unstable approach or speed/path/vertical rate not congruent with required state for EVAL or SBT cross-checking, competency with the practice of the observed deviations, errors and given flight condition error A right techniques and attitudes related mistakes are taken as learning x x x management, to these topics through pilot opportunities throughout the mismanaged performance, and that instructors programme. aircraft state have the opportunity to assess and Monitor flight path excursions. In-seat instruction: train these topics in a realistic Detect errors and threats through Demonstration exercise — recovery from bounced landing, adverse wind, strong gusts LDG environment. As shown by the EBT proper cross-checking during landing phase, resulting in a bounce and necessitating recovery action from the data report, these topics are of key performance. PM importance to improve safety in Make appropriate interventions operations. either verbally or by taking control if applicable. Page 180 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map In addition, the operator may also use Take appropriate action if these topics to develop scripted role- necessary. playing scenarios in the form of ISI Restore desired aircraft state. training. These scenarios cater for the Identify and manage need to monitor flight path excursions consequences. from the instructor pilot (PF), detect errors and make appropriate interventions, either verbally or by taking control as applicable. Demonstration scenarios may also be used. Demonstrated role-play should contain realistic and not gross errors, leading at times to a mismanaged aircraft state, which can also be combined with upset management training. DES ATC or terrain-related environment creating a high-energy descent with the need to x x x APP Reinforce stabilised approach capture the optimum profile to complete the approach in a stabilised configuration philosophy and adherence to defined DES ATC or terrain-related environment creating a high-energy descent leading to unstable x x x parameters. Encourage go-arounds APP conditions and requiring a go-around Unstable when crews are outside these A Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x approach APP parameters. Develop and sustain downdrafts, gusts and crosswinds including shifting wind directions competencies related to the APP Increasing tailwind on final approach (not reported) x x x x management of high-energy APP situations. Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) Section 3 — Training topics with frequency (B) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases Compliance with AMC1 or AMC2 to See Table 1 of AMC1 ORO.FC.220&230: Elements and respective components of upset Intentionally blank N/A ORO.FC.220&230 prevention training. Include upset prevention elements in Demonstration of the defined normal flight envelope and any associated changes in x x x Table 1 for the recurrent training flight instruments, flight director systems, and protection systems. This should take CRZ programme at least every 12 calendar the form of an instructor-led exercise to show the crew the points beyond which an months, such that all the elements upset condition could exist. Early recognition and prevention of TO Severe wind shear or wake turbulence during take-off or approach x x x x EVAL, MT or SBT are covered over a period not upset conditions. APP exceeding 3 years. The elements are Upset prevention CRZ numbered with letters from A to I in As applicable and relevant to aircraft type, demonstration at a suitable intermediate x x x B When the differences between LHS training Table 1 of AMC1 ORO.FC.220&230. level, with turbulence as appropriate; practise steep turns and note the relationship and RHS are not significant in the Each element is made up of several between bank angle, pitch and stalling speed handling of the aircraft, UPRT may CRZ numbered components. At the maximum cruise flight level for current aircraft weight, turbulence to trigger over x x x x be conducted in either seat. Through the principles of EBT, speed conditions (if FSTD capability exists, consider use of vertical wind component to covering one component should add realism) CRZ satisfy the requirement to cover the At the maximum cruise flight level for current aircraft weight, turbulence and significant x x x X element. temperature rise to trigger low-speed conditions (if FSTD capability exists, consider use of vertical wind component to add realism) CRZ High-altitude TCAS RA. Where the RA is required to be flown in manual flight x x x x For full details, see the malfunction equivalency methodology. Unless specified Intentionally blank Aircraft system Any internal failure(s) apparent or not Recognise system malfunction. scenari phases tion or Evalua trainin based otherwise in the operational suitability data, at least one malfunction with each o- malfunctions, B ALL apparent to the crew Take appropriate action including g characteristic should be included every year. Combining characteristics should not including correct stop/go decision. reduce the number of malfunctions below seven for each year. For each crew Page 181 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map operations under Any item cleared by the MEL but Apply the appropriate procedure member, the characteristics of degraded control and loss of instrumentation should MEL having an impact upon flight correctly. be in the role of pilot flying and the others may be in the role of pilot flying or pilot operations. For instance. thrust Maintain aircraft control. monitoring. reverser locked. Manage consequences. (i) System malfunctions that require immediate and urgent crew intervention or decision, e.g. fire, smoke, loss of pressurisation at high altitude, failures during take- Malfunctions to be considered should Apply crew operating procedures off, brake failure during landing. have one or more of the following where necessary. (ii) System malfunctions that require complex procedures, e.g. multiple hydraulic characteristics: Respond appropriately to system failures, smoke and fumes procedures, major electrical system failure. — Immediacy additional system abnormalities (iii) System malfunctions that result in significant degradation of flight controls in — Complexity associated with MEL dispatch. combination with abnormal handling characteristics, e.g. jammed flight controls, — Degradation of aircraft control certain degradation of FBW control, jammed horizontal stabiliser; flaps and/or slats — Loss of primary instrumentation locked; other malfunctions that result in degraded flight controls. — Management of consequences (iv) System failures that require monitoring and management of the flight path using The operator should vary degraded or alternative displays, unreliable primary flight path information, unreliable malfunctions for each characteristic airspeed, e.g. flight with unreliable airspeed over the EBT cycle. (v) System failures that require extensive management of their consequences (independent of operation or environment), e.g. fuel leak. TO MEL items with crew operating procedures applicable during take-off x X Response to an additional factor that is affected by a MEL item (e.g. system failure, x x x X TO runway state) GND Malfunction during preflight preparation and prior to departure x x x CLB Malfunction after departure x x x X Malfunctions that require immediate attention (e.g. bleed fault during engine start, x x x ALL hydraulic failure during taxi) CLB x x x X Fuel leak (management of consequences) CRZ TO Malfunction on take-off high speed below V1 x x x TO Malfunction on take-off high speed above V1 x x During taxi to the runway, a spurious brake temperature announcement. The crew had GND X X X the correct brake temperature moments before the failure. TO Tyre failure during take-off x x x TO Malfunction on initial climb x x APP Malfunction on approach x x x APP Malfunction on go-around x x x LDG Malfunction during landing x x x x x See ‘compliance’ topic above. There are no defined scenarios, but the instructor This is not considered as a stand- should focus on learning opportunities when system management non-compliances Evaluation or scenario-based alone topic. It is linked with the manifest themselves during other scenarios. Underpinning knowledge of systems and Intentionally blank X topic ‘compliance’. Aircraft system Normal system operation according their interactions should be developed and challenged, and not merely the application B Where a system is not managed training phases management to defined instructions of normal procedures. according to normal or defined CRZ x x x x procedures, this is determined as a Minimum fuel, caused by extended delays, weather, etc. where the crew would need APP non-compliance. to manage a minimum fuel situation. LDG APP Recognise actual conditions. Approach in poor visibility x x x x Approach, Observe aircraft and/or procedural Approach in poor visibility with deteriorations necessitating a decision to perform a x x x APP Any situation where visibility visibility close to B limitations. go-around becomes a threat minimum Apply appropriate procedures if x x x LDG Landing in poor visibility applicable. Page 182 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map Maintain directional control and safe flight path. Pilots should have opportunities to practise landings in demanding situations at the defined frequency. Data indicates that landing problems have their roots in a variety of factors, Landing in demanding This topic should be combined with the adverse-weather topic, aircraft system including inappropriate decision- Landing B LDG environmental conditions, with malfunctions topic or any topic that can provide exposure to a landing in demanding Intentionally blank making, in addition to manual aircraft malfunctions as appropriate conditions. control skills if difficult environmental conditions exist. The purpose of this item is to ensure that pilots are exposed to this during the programme. The data analysed during the development of the EBT concept indicated substantial difficulties Evaluation or scenario-based training phases encountered by crews when faced with a threat or error, which was a surprise or an unexpected event. The element of surprise should be distinguished from what is sometimes referred to as the ‘startle factor’ — the latter being a physiological Exposure to an unexpected event reaction. Wherever possible, or sequence of events at the Surprise B ALL Intentionally blank Intentionally blank consideration should be given defined frequency in order to build towards variations in the types of resilience. scenario, times of occurrences and types of occurrence, so that pilots do not become overly familiar with repetitions of the same scenarios. Variations should be the focus of EBT programme design, and not left to the discretion of individual instructors, in order to preserve programme integrity and fairness. ALL ATC clearance giving insufficient terrain clearance x x x Anticipate terrain threats. Prepare for terrain threats. Demonstration of terrain avoidance warning systems (TAWS) (this scenario element ALL x x x Recognise unsafe terrain clearance. may be done in an ISI.) EVAL or SBT TO Take appropriate action. Engine failure where performance is marginal leading to TAWS warning x x x Terrain B CLB Alert, warning, or conflict Apply the appropriate procedures DES correctly. ATC provides a wrong QNH x x APP Maintain aircraft control. Restore safe flight path. ‘Virtual mountain’ refers to the surprise element of an unexpected warning. Care should DES Manage consequences. be exercised in creating a level of realism, so this can best be achieved by an unusual x x x and unexpected change of route during the descent. Wind shear TO With or without warnings including Predictive wind shear warning during take-off x x B Anticipate potential for wind shear. recovery TO predictive. A wind shear scenario is Wind shear encounter during take-off x x x Page 183 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map TO ideally combined into an adverse- Avoid known wind shear or Wind shear encounter after rotation x x TO weather scenario containing other prepare for suspected wind shear. Predictive wind shear after rotation x x APP elements. Recognise wind shear encounter. Predictive wind shear during approach x x x Take appropriate action. Apply appropriate procedure correctly. Assure aircraft control. Recognise out of wind shear APP condition. Wind shear encounter during approach x x x Maintain or restore a safe flight path. Assess consequential issues and manage outcomes. This is not considered a topic for Intentionally blank Intentionally blank specific attention on its own, but more as a reminder to programme EVAL or SBT developers to ensure that pilots are Manage available resources Workload, exposed to immersive training efficiently to prioritise and perform distraction, B ALL scenarios which expose them to tasks in a timely manner under all pressure, stress manageable high workload and circumstances distractions during the course of the EBT programme, at the defined frequency. Section 3 — Training topics frequency (C) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases N/A Compliance with AMC1 or AMC2 to Table 2 of AMC1 ORO.FC.220&230: Exercises for upset recovery training ORO.FC.220&230 Intentionally blank A. Recovery from developed upsets Recognise upset condition. Recovery from stall events, in the following configurations; Include the recovery exercises in Make timely and appropriate Table 2 of AMC1 ORO.FC.220&230 for — take-off configuration, intervention. CLB the recurrent training programme, Take appropriate action. — clean configuration low altitude, x x x x such that all the exercises are covered 2. Manoeuvres training phase or SBT DES Assure timely and appropriate over a period not exceeding 3 years. intervention. (AMC1 — clean configuration near maximum operating altitude, and Through the principles of EBT, ORO.FC.220&230 Table 2 covering one component should — landing configuration during the approach phase. component 1) CRZ satisfy the requirement to cover the 3. Recovery from nose high at various bank angles x x x x Assure aircraft control. whole element of recovery from CRZ Maintain or restore a safe flight x x x x Upset recovery C developed upsets. The same 4. Recovery from nose low at various bank angles CRZ path. principles applies to the exercises of Assess consequential issues. Demonstration at a normal cruising altitude. Set conditions and disable aircraft x x x components 2, 3 and 4 where one APP Manage outcomes. systems as necessary to enable trainee to perform stall recovery according to OEM exercise may satisfy the requirement Consolidated summary of instructions to cover the whole component. aeroplane recovery techniques. Demonstration at an intermediate altitude during early stages of the approach. Set x x x An aeroplane upset is defined as an (AMC1 ORO.FC.220&230 Table 2 conditions and disable aircraft systems as necessary to enable trainee to perform stall undesired aeroplane state in flight component 5) recovery according to OEM instructions characterised by unintentional CLB Note: The operator should assess if divergences from parameters DES the exercises should be practice for normally experienced during line the either seat qualification. Recovery from a wake turbulence position with high-bank angle x x x x operations or training. An aeroplane upset may involve pitch and/or bank angle divergences as well as Page 184 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map inappropriate airspeeds for the conditions. The example scenario elements may be done in ISI, as non-ISI or a combination of both. If done in ISI: The instructor should position the aircraft within but close to the edge of the validated training envelope before handing control to the trainee to demonstrate the restoration of normal flight. Careful consideration should be given to flying within the validated training envelope. TO Take-off with different crosswind/tailwind/gust conditions x x TO Take-off with unreported tailwind x x TO Crosswinds with or without strong gusts on take-off x x APP Increasing tailwind on final approach(not reported) x x x x Recognise adverse-wind Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x EVAL or SBT APP conditions. Adverse wind/crosswind. This downdrafts, gusts and crosswind including shifting wind directions Observe limitations. Adverse wind C APP includes tailwind but not ATC mis- Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x Apply appropriate procedures. reporting of the actual wind. Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x APP Maintain directional control and safe flight path. (not reported) Adverse-wind scenario including strong gusts and/or crosswind out of limits below x x x APP 15 m (50 ft) (not reported) APP Crosswind with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) ALL ATC role-play: the instructor provides scripted instructions, as a distraction to the crew x x x ALL ATC error. Omission, Controller error, provided by the instructor according to a defined scripted scenario x x x x Respond to communications ALL miscommunication, garbled, poor Frequency congestion, with multiple aircraft using the same frequency x appropriately. quality transmission. All of these act EVAL or SBT APP Recognise, clarify and resolve any Destination temporally closed x x x x as distractions to be managed by the CRZ ambiguities. Rescue and firefighting services (RFFS) level reduction at destination x x x ATC C crew. The scenarios should be Refuse or question unsafe Runway change before the interception of the localiser or similar navigation aid in x x x x APP combined where possible, with instructions. azimuth others of the same or higher GND/ Use standard phraseology x x x weighting, the principle reason being Stray dogs at the opposite threshold runway TO whenever possible. to create distractions. ALL Poor quality transmissions x TO Any engine failure or malfunction, Engine failure or engine malfunction on take-off low speed x x x x TO which causes loss or degradation of Engine failure or engine malfunction on take-off high speed below V1 x x x x TO thrust that affects performance. This Engine failure or engine malfunction on take-off above V1 x x x x Recognise engine failure. TO is distinct from the engine-out Engine failure or engine malfunction on initial climb x x x EVAL or SBT Take appropriate action. APP manoeuvres described in the Engine malfunction x x x Apply appropriate procedure Engine failure C CRZ manoeuvres training section above, Engine failure in cruise (with autopilot) x x x correctly. which are intended only to practise x Maintain aircraft control. psychomotor skills and reinforce Manage consequences. LDG procedures to manage engine Engine failure or engine malfunction on landing failures. Page 185 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map GND Fire in cargo or cabin/cockpit at gate x x x x GND Fire during taxi x x x x X GND Fire with no cockpit indication x x x x X TO Take-off low speed x x x x X TO Fire or smoke on Take-off high speed below V1 x x x x Recognise fire, smoke or fumes TO Fire or smoke on Take-off high speed above V1 x x x EVAL or SBT Take appropriate action. TO This includes engine, electric, Fire or smoke on Initial climb x x x Fire and smoke Apply appropriate procedure C CRZ pneumatic, cargo fire, smoke or Cargo fire x x x management correctly. APP fumes. Engine fire in approach (extinguishable) x x Maintain aircraft control. APP Manage consequences. Engine fire in approach (non-extinguishable) x x x CLB CRZ Lithium battery fire in the cockpit or cabin compartment x x x x x DES APP Flight deck or cabin fire x x x X GND Any of the example scenarios elements above ending in an evacuation x x x x GND Recognise loss of communications. Loss of communications during ground manoeuvring x x TO Lost or difficult communications. Take appropriate action. Loss of communications after take-off x x X EVAL or SBT Either through pilot mis-selection or a Execute appropriate procedure as x x x x X Loss of C failure external to the aircraft. This applicable communications APP could be for a few seconds or a total Use alternative ways to Loss of communications during approach phase, including go-around loss. communicate Manage consequences This can be a demonstrated error, in that the crew may be instructed to deliberately x x x Anticipate the potential for errors insert incorrect data — for example, to take off from an intersection with full-length ALL in load/fuel/performance data. performance information. The crew will be asked to intervene when acceleration is A calculation error by one or more Recognise inconsistencies. sensed to be lower than normal, and this may be part of the operator procedures, EVAL or SBT Managing especially when operating mixed fleets with considerable variations in MTOM. pilots, or someone involved with the Manage/avoid distractions. loading, fuel, C process, or the process itself, e.g. Make changes to Fuel ground staff on industrial action. Only limited amount of fuel available, which is x x x x performance GND incorrect information on the load paperwork/aircraft system(s) to below the calculated fuel for the flight. errors sheet eliminate error. Advice crew that there is a change of the load sheet figures during taxi to the runway. x x GND Identify and manage The crew may have limited time due to a calculated take-off time (CTOT) — ATC slot consequences. Braking action reported ‘medium’. The information is transmitted just before take-off. x x x GND The flight is subject to a calculated take-off time (CTOT) — ATC slot. x x x x External failure or a combination of external failures degrading aircraft navigation Evaluation or scenario-based training GND performance on ground TO x x x x CLB External failure or a combination of external failures degrading aircraft navigation Recognise a NAV degradation. APP performance in flight Take appropriate action. LDG External NAV failure. phases Execute appropriate procedure as Navigation C Loss of GPS satellite, ANP exceeding Standard initial departure change during taxi. The flight may be subject to a CTOT — GND applicable. x x x RNP, loss of external NAV source(s) ATC slot. Use alternative NAV guidance. APP Manage consequences. Loss of runway lighting below decision height x x x No fly zone: when the crew changes control frequency, the new ATCO informs the crew that they are flying over an unannounced ‘no fly zone’ and not included in the NOTAMs. CRZ To trigger such an event, the context can as an example be as follows: an unexpected x x x military conflict in the territory the aircraft is flying over or the crew is forced to re-route in flight and the new route flies over a city that has an important event such the Olympic Page 186 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 3 Turboprop — Recurrent assessment and training matrix Competency map games, a G20/G7 submit, or the route is flying near a space rocket launch close to the time of the launch, like Guiana Space Centre, cape Cañaveral, etc.). Operations- or C ALL Intentionally blank Intentionally blank Intentionally blank Intentionally blank type-specific The operator should comply with Operations of the national qualification APP See equivalency of approaches special airport C requirements published in the Intentionally blank Intentionally blank LDG relevant to operations. approval Aeronautical Information Publication. Recognise incapacitation. TO Take appropriate action including During take-off x x x x X correct stop/go decision. Pilot Consequences for the non- C Apply appropriate procedure incapacitation incapacitated pilot APP correctly. During approach x x x X Maintain aircraft control. EVAL or SBT Manage consequences. GND x X Recognise hazardous runway Planned anticipated hazardous conditions with dispatch information provided to TO condition. facilitate planning and execution of appropriate procedures LDG Contamination or surface quality of Observe limitations. Runway or GND x x x C the runway, taxiway, or tarmac Take appropriate action. Unanticipated hazardous conditions, e.g. unexpected heavy rain resulting in flooded taxiway condition TO including foreign objects Apply appropriate procedures runway surface LDG correctly. TO Take-off on runway with reduced cleared width due to snow x x x x Assure aircraft control. TO Stop/go decision in hazardous conditions x x x Anticipate potential loss of separation. EVAL or SBT Recognise loss of separation. CLB Traffic conflict. ACAS RA or TA, or Take appropriate action. Traffic C CRZ visual observation of conflict, which ACAS warning that requires crew intervention x x x x Apply appropriate procedure DES requires evasive manoeuvring correctly. Maintain aircraft control. Manage consequences. CS OPS.EBT.232.5 Generation 1 (jet) table of assessment and training topics GENERATION 2 (JET) — EBT PROGRAMME — TABLE OF ASSESSMENT AND TRAINING TOPICS Given the very small number of turbo-jet aeroplanes of the second generation in current use in commercial air transport operations, the operator must apply for a special condition to develop a table of assessment and training topics to apply EBT. Page 187 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.232.6 Generation 2 (turboprop) table of assessment and training topics GENERATION 2 (TURBOPROP) — TABLE OF ASSESSMENT AND TRAINING TOPICS (includes type of for activation threat, error or Flight phase Desired outcome Frequency topic, being Description Assessment and Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA (includes performance criteria OR training topic Example scenario elements training outcome) focus) Generation 2 Turboprop — Recurrent assessment and training matrix Competency map Section 1 — Skill retention Engine failure after the application of Rejected take-off A take-off thrust and before reaching V1 TO From initiation of take-off to complete stop (or as applicable to procedure) x x (may be in LVO or CAT I or above) Failure of the critical engine from V1 The manoeuvre is complete at a point when the aircraft is stabilised at normal engine-out Failure of the critical and before reaching V2 in the lowest climb speed with the correct pitch and lateral control, in trim condition and, as applicable, engine between V1 A TO x x CAT I visibility conditions or in LVO autopilot engagement. Only one failure of the critical engine between V1 and V2 a year and V2 conditions. may be done in LVO conditions. Failure of one engine from V1 and The manoeuvre is complete at a point when the aircraft is stabilised in a clean before reaching V2 in lowest CAT I configuration with engine-out procedures completed. Only one failure of the critical x x visibility conditions or in LVO engine between V1 and V2 a year may be done in LVO conditions. Failure of one engine conditions. B TO on take-off Failure of one engine above V2 (any Demonstrate manual aircraft control segment of the TO) in lowest CAT I skills with smoothness and accuracy as The manoeuvre is complete at a point when aircraft is stabilised in a clean configuration Manoeuvres training phase x x x visibility conditions or in LVO appropriate to the situation. with engine-out procedures completed conditions. Detect deviations through instrument Initiation of emergency descent from scanning. The manoeuvre is complete once the aircraft is stabilised in emergency descent Emergency descent C Maintain spare mental capacity during CRZ x x x normal cruise altitude configuration (and profile). manual aircraft control. Maintain the aircraft within the flight Engine-out approach With the critical engine failed, normal Initiation in a stabilised engine-out configuration from not less than 3 NM final approach, A envelope. LDG x x & landing landing until completion of roll-out Apply knowledge of the relationship between aircraft attitude, speed and With the critical engine failed, manually thrust. This manoeuvre should be flown from intercept to centreline until acceleration after go- flown normal precision approach to DA, Engine-out approach around. The manoeuvre is complete at a point when the aircraft is stabilised at normal A followed by manually flown go-around APP x x & go-around engine-out climb speed with the correct pitch and lateral control, in trim condition and, as — the whole manoeuvre to be flown applicable, autopilot engagement (describe generally critical part of manoeuvre). without visual reference High energy, initiation during the approach at 150 to 300 m (500 to 1 000 ft) below the Go-around, all engines operative x x x missed approach level-off altitude Go-around A APP Go-around, all engines operative Initiation of a go-around from DA followed by visual circuit and landing x x x Go-around, all engines operative During flare/rejected landing x x x Pilot qualification to Only for commanders whose duties operate in either B require them to operate in either pilot’s APP Complete the manoeuvres mandated in ORO.FC.235. Intentionally left in blank. pilot’s seat seat Page 188 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail for activation Flight phase Assessment and Description (includes type of topic, being Desired outcome Guidance material (GM) WLM COM SAW FPM KNO LTW PRO PSD FPA Frequency (includes performance criteria OR Example scenario elements training topic threat, error or focus) training outcome) Generation 2 Turboprop — Recurrent assessment and training matrix Competency map Section 2 — Equivalency of approaches relevant to operations See equivalency of approaches Approach type A or B Approach type A or B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X B additional demand on a proficient crew See equivalency of approaches MT Approach type A B Approach type A flight method 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval See equivalency of approaches Approach type A B Approach type A flight method 3D or 2D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew EVAL or SBT See equivalency of approaches Approach type B B Approach type B flight method 3D relevant to operations that place an APP See equivalency of approaches relevant to operations x x x x X additional demand on a proficient crew See equivalency of approaches SPA approach(es) B Approach requiring specific approval relevant to operations — specific APP Approaches flown from FAF to landing or go around x x x approval Page 189 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map Section 3 — Training topics with frequency (A) in alphabetical order GND Predictive wind shear warning before take-off, as applicable x x x ALL Adverse-weather scenario, e.g. thunderstorm activity, precipitation, icing x x x x TO Wind shear encounter during take-off, not predictive x x x X Evaluation or scenario-based training phases TO Predictive wind shear warning during take-off x x x x TO Crosswinds with or without strong gusts on take-off x x CRZ Turbulence that increases to severe turbulence x x x x CRZ Wind shear encounter scenario during cruise x x x x x Anticipate adverse weather. APP Thunderstorm, heavy rain, Reactive wind shear warning during approach or go-around x x x x Prepare for suspected adverse APP turbulence, ice build-up to include de- Predictive wind shear warning during approach or go-around x x x x weather. APP icing issues, as well as high- Thunderstorm encounter during approach or on missed approach x x x Recognise adverse weather. Adverse weather A APP temperature conditions. Increasing tailwind on final approach (not reported) x x x x Take appropriate action. APP The proper use of anti-ice and de- Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x Apply the appropriate procedure icing systems should be included downdrafts, gusts and crosswinds including shifting wind directions correctly. APP generally in appropriate scenarios. Non-precision approach in cold-temperature conditions, requiring altitude x x x Assure aircraft control. compensation for temperature, as applicable to type APP Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) In approach, unexpected braking action ‘good to medium’ reported by the preceding x x x x APP aircraft APP Reduced visibility even after acquiring the necessary visual reference during approach, x x x due to rain or fog See ‘compliance’ topic above. There are no defined scenarios, but the instructor should This is not considered as a stand- focus on learning opportunities when system management non-compliances manifest alone topic. It is linked with the themselves during other scenarios. Underpinning knowledge of systems and their Intentionally blank X topic ‘compliance’. Aircraft system Normal system operation according interactions should be developed and challenged, and not merely the application of A Where a system is not managed management to defined instructions normal procedures. according to normal or defined CRZ procedures, this is determined as a Minimum fuel, caused by extended delays, weather, etc. where the crew would need APP x x x x non-compliance. to manage a minimum fuel situation. LDG CLB The purpose of this topic is to Know how and when to use the ACAS warning, recovery and subsequent engagement of automation x x CRZ encourage and develop effective flight management system(s), DES flight path management through guidance and automation. APP proficient and appropriate use of the Demonstrate correct methods for ALL flight management system(s), engagement and disengagement of FMS tactical programming issues, e.g. step climb, runway changes, late clearances, x x X guidance and automation, including the auto flight system(s). destination re-programming, executing diversion CLB transitions between modes, Demonstrate appropriate use of Recoveries from TAWS, management of energy state to restore automated flight x x x EVAL or SBT CRZ monitoring, mode awareness, flight guidance, auto thrust and Automation A DES vigilance and flexibility needed to other automation systems. management APP change from one mode to another. Maintain mode awareness of the CLB The means of mitigating errors are auto flight system(s), including Amendments to ATC cleared levels during altitude capture modes to force mode x x x CRZ included in this topic. The errors are engagement and automatic awareness and intervention DES described as mishandled auto flight transitions. APP systems, inappropriate mode Revert to different modes when TO selection, flight management appropriate. Late ATC clearance to an altitude below acceleration altitude x x x TO system(s) and inappropriate autopilot Detect deviations from the desired Engine-out special terrain procedures x x x APP usage. aircraft state (flight path, speed, Page 190 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map CRZ attitude, thrust, etc.) and take Forcing AP disconnect followed by re-engagement, recovery from low- or high-speed x x x x appropriate action. events in cruise CRZ Engine failure in cruise to onset of descent using automation x x CRZ Anticipate mishandled auto flight Emergency descent x x X DES system. Managing high-energy descent capturing descent path from above (correlation with x x x X APP Recognise mishandled auto flight unstable approach training) APP system. No ATC clearance received prior to commencement of approach or final descent x x x APP Take appropriate action if Reactive wind shear and recovery from the consequent high-energy state x x x APP necessary. Automation fail to capture the approach altitude in descent (e.g. last altitude before the x x x x Restore correct auto flight state. FAP). Ideally, the failure occurs when the workload is high (e.g. configuration of the Identify and manage aircraft for final approach). consequences. APP Non-precision or infrequently flown approaches using the maximum available level of x x X automation APP Gear malfunction during approach x x x APP ATC clearances to waypoints beyond the programmed descent point for a coded final x x x X descent point during an approach utilising a final descent that is commanded by the flight management system APP Exposure to event or sequence of GPS failure prior to commencement of approach associated with position drift and a x x x X events to allow the pilot to build terrain alert DES awareness of human factors in Cabin crew report of water noise below the forward galley indicating a possible toilet x x x Evaluation or scenario-based training phases aviation and the human limitations. pipe leak, with consequent avionics failures CRZ This includes the development of Smoke removal but combined with a diversion until landing completed. x x x x x X the following competencies: GND Communication: Apron fuel spilling x x x Demonstrate: CRZ Important water leak in an aircraft galley x x x x — effective use of language; This encapsulates the general CRM ALL — responsiveness to feedback; A relevant number of cabin crew are wounded or incapacitated. Additionally, the principles and objectives. It includes: and cabin crew wounded or incapacitated are the most competent (e.g. senior cabin crew x x x communication; leadership and — capability to state the plans member). teamwork; problem-solving and ALL and resolve ambiguities. Unruly passenger(s) x x decision-making; situation awareness GND Leadership and teamwork: Passenger oxygen: passenger service unit open and mask falling down x x x and management of information; and Competencies Use appropriate authority to ALL workload management. Passenger with medical problems — medical emergency x x non-technical A ensure focus on the task. Support (CRM) CRZ others in completing tasks. Emphasis should be placed on the Credible threat reported to the crew. Stowaway or fugitive on board. x x x x Problem-solving and decision- GND development of leadership, shown by No METAR or TAFOR is available for destination due to industrial action at the making: x x x x EBT data sources to be a highly destination airport Detect deviations from the desired CRZ effective competency in mitigating Credible bomb threat reported to crew x x x x state, evaluate problems, identify risk and improving safety through risk, consider alternatives and pilot performance. APP select the best course of action. ACAS warning immediately following a go-around, with a descent manoeuvre required x x x x X Continuously review progress and adjust plans. Situation awareness and EVAL or SBT management of information: Have an awareness of the aircraft state in its environment; project and anticipate changes. Workload management: Page 191 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map Prioritise, delegate and receive assistance to maximise focus on the task. Continuously monitor the flight progress. Compliance failure. Consequences of Recognise that a compliance failure The following are examples of potential compliance failures and are not intended to be Intentionally blank not complying with operating has occurred. developed as scenarios as part of an EBT module: instructions (e.g. SOPs). Make a verbal announcement. This is not intended to list example Take appropriate action if 1. Requesting flap beyond limit speed scenario elements, but instructors necessary. EVAL or SBT should ensure that observed non- Restore safe flight path if necessary. 2. Flaps or slats in the wrong position for phase of flight or approach compliances are taken as learning Manage consequences. Compliance A ALL opportunities throughout the 3. Omitting an action as part of a procedure programme. In all modules of the programme, the FSTD should as far as 4. Failing to initiate or complete a checklist possible be treated like an aircraft, and non-compliances should not be 5. Using the wrong checklist for the situation accepted simply for expediency. APP Any threat or error that can result in Adverse-weather scenario leading to a reactive wind shear warning during approach x x x x circumstances that require a decision to perform a go-around, in addition to Adverse-weather scenario leading to a predictive wind shear warning during approach x x x x APP the execution of the go-around. Go- or go-around around scenarios should be fully Adverse-weather scenario, e.g. thunderstorm activity, heavy precipitation or icing x x x x APP developed to encourage effective forcing decision at or close to DA/MDA leadership and teamwork, in addition DA with visual reference in heavy precipitation with doubt about runway surface x x x x APP to problem-solving and decision- braking capability APP making, plus execution using manual Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x EVAL or SBT aircraft control or the flight Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x Go-around APP A management system(s) and (not reported) management automation as applicable. Design Adverse-wind scenario including strong gusts and/or crosswind out of limits below 15 m x x x APP should include the element of (50 ft) (not reported) surprise, and scenario-based go- Lost or difficult communications resulting in no approach clearance prior to x x x APP arounds should not be predictable and commencement of approach or final descent anticipated. This topic is completely distinct from the go-around Birds: large flocks of birds below DA once visual reference has been established x x x APP manoeuvre listed in the manoeuvres training section that is intended only System malfunction, landing gear malfunction during the approach APP to practise psychomotor skills and a simple application of the procedures. CLB Flight with unreliable airspeed, which may or may not be recoverable x x x X Desired competency outcome: or scenario- CRZ Evaluation training Demonstrates manual aircraft phases based Manual aircraft DES Controls the flight path through A control skills with smoothness and control APP manual control accuracy as appropriate to the CLB Alternate flight control modes according to malfunction characteristics x x x X situation CRZ Page 192 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map DES Detects deviations through APP instrument scanning CLB Maintains spare mental capacity ACAS RA requires the pilot to descend or ATC immediate descent x x x CRZ during manual aircraft control DES Maintains the aircraft within the APP normal flight envelope Applies knowledge of the TAWS warning when deviating from planned descent routing, requiring immediate x x x DES relationship between aircraft response attitude, speed and thrust Scenario immediately after take-off which requires an immediate and overweight x x x x TO landing TO Adverse wind, crosswinds with or without strong gusts on take-off x x Adverse weather, wind shear, wind shear encounter during take-off, with or without x x x TO reactive warnings TO Engine failure during initial climb, typically 30-60 m (100-200 ft) (autopilot off) x x x x Wind shear encounter scenario during cruise, significant and rapid change in wind speed x x x x x CRZ or down/updrafts, without wind shear warning Adverse weather, wind shear, wind shear encounter with or without warning during x x x x APP approach Adverse weather, deterioration in visibility or cloud base, or adverse wind, requiring a x x x x x x x APP go-around from visual circling approach, during the visual segment APP Interception of the glide slope from above (correlation with unstable approach training) x x x APP Adverse wind, crosswinds with or without strong gusts on approach, final and landing x x x LDG (within and beyond limits) Adverse weather, adverse wind, approach and landing in demanding weather x x x APP conditions, e.g. turbulence, up and downdrafts, gusts and crosswinds including shifting LDG wind directions APP Circling approach manually flown at night in minimum in-flight visibility to ensure X X X X LDG ground reference, minimum environmental lighting and no glide slope guidance lights APP Runway incursion during approach, which can be triggered by ATC at various altitudes x x x LDG or by visual contact during the landing phase LDG Adverse wind, visibility, type-specific, special consideration for long-bodied aircraft, x x x x landing in minimum visibility for visual reference, with crosswind LDG System malfunction, auto flight failure at DA during a low-visibility approach requiring x x x x a go-around flown manually APP Approach planned with autoland, followed by a failure below 1 000 ft requiring a go- x x x x LDG around and an immediate landing due to fuel shortage. ALL The scenarios should be realistic and Recognise mismanaged aircraft Deviations from the flight path, in pitch attitude, speed, altitude, bank angle x x relevant, and should be used for the state. In-seat instruction: x x purpose of demonstration and Observe the pilot’s behaviour: how Simple automation errors (e.g. incorrect mode selection, attempted engagement Monitoring, ALL reinforcement of effective monitoring. the pilot is mitigating errors, without the necessary conditions, entering wrong altitude or speed, failure to execute EVAL or SBT cross-checking, performing cross-checking, the desired mode) culminating in a need for direct intervention from the PM, and error Modules in the FSTD should be treated monitoring performance and where necessary taking control. A management, like those in an aircraft so that trainees dealing with a mismanaged aircraft In-seat instruction: x x x x mismanaged APP have the opportunity to develop the state, in order to ensure that Unstable approach or speed/path/vertical rate not congruent with required state for aircraft state competency with the practice of the observed deviations, errors and given flight condition right techniques and attitudes related mistakes are taken as learning x x x LDG to these topics through pilot opportunities throughout the In-seat instruction: performance, and that instructors programme. Page 193 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map have the opportunity to assess and Monitor flight path excursions. Demonstration exercise — recovery from bounced landing, adverse wind, strong gusts train these topics in a realistic Detect errors and threats through during landing phase, resulting in a bounce and necessitating recovery action from the environment. As shown by the EBT proper cross-checking PM data report, these topics are of key performance. importance to improve safety in Make appropriate interventions operations. either verbally or by taking control if applicable. In addition, the operator may also use Take appropriate action if these topics to develop scripted role- necessary. playing scenarios in the form of ISI Restore desired aircraft state. training. These scenarios cater for the Identify and manage need to monitor flight path excursions consequences. from the instructor pilot (PF), detect errors and make appropriate interventions, either verbally or by taking control as applicable. Demonstration scenarios may also be used. Demonstrated role-play should contain realistic and not gross errors, leading at times to a mismanaged aircraft state, which can also be combined with upset management training. DES ATC or terrain-related environment creating a high-energy descent with the need to x x x APP Reinforce stabilised approach capture the optimum profile to complete the approach in a stabilised configuration DES philosophy and adherence to defined ATC or terrain-related environment creating a high-energy descent leading to unstable x x x parameters. Encourage go-arounds APP conditions and requiring a go-around Unstable when crews are outside these A Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x approach APP parameters. Develop and sustain downdrafts, gusts and crosswinds including shifting wind directions competencies related to the APP management of high-energy Increasing tailwind on final approach (not reported) x x x x APP situations. Crosswinds with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) Section 3 — Training topics with frequency (B) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases Compliance with AMC1 or AMC2 to See Table 1 of AMC1 ORO.FC.220&230: Elements and respective components of upset Intentionally blank N/A ORO.FC.220&230 prevention training. Include upset prevention elements in Demonstration of the defined normal flight envelope and any associated changes in x x x Table 1 for the recurrent training flight instruments, flight director systems, and protection systems. This should take CRZ Early recognition and prevention of EVAL, MT or SBT programme at least every 12 calendar the form of an instructor-led exercise to show the crew the points beyond which an upset conditions. months, such that all the elements upset condition could exist. Upset prevention TO are covered over a period not Severe wind shear or wake turbulence during take-off or approach x x x x B When the differences between LHS training APP exceeding 3 years. The elements are and RHS are not significant in the CRZ numbered with letters from A to I in As applicable and relevant to aircraft type, demonstration at a suitable intermediate x x x handling of the aircraft, UPRT may Table 1 of AMC1 ORO.FC.220&230. level, with turbulence as appropriate; practise steep turns and note the relationship be conducted in either seat. Each element is made up of several between bank angle, pitch and stalling speed CRZ numbered components. At the maximum cruise flight level for current aircraft weight, turbulence to trigger over x x x x Through the principles of EBT, speed conditions (if FSTD capability exists, consider use of vertical wind component to covering one component should add realism) Page 194 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map CRZ satisfy the requirement to cover the At the maximum cruise flight level for current aircraft weight, turbulence and significant x x x X element. temperature rise to trigger low-speed conditions (if FSTD capability exists, consider use of vertical wind component to add realism) CRZ High-altitude TCAS RA. Where the RA is required to be flown in manual flight x x x x For full details, see the malfunction equivalency methodology. Unless specified Intentionally blank otherwise in the operational suitability data, at least one malfunction with each characteristic should be included every year. Combining characteristics should not reduce the number of malfunctions below seven for each year. For each crew member, the characteristics of degraded control and loss of instrumentation should be in the role of pilot flying and the others may be in the role of pilot flying or pilot monitoring. (i) System malfunctions that require immediate and urgent crew intervention or decision, e.g. fire, smoke, loss of pressurisation at high altitude, failures during take- off, brake failure during landing. ALL (ii) System malfunctions that require complex procedures, e.g. multiple hydraulic Any internal failure(s) apparent or not system failures, smoke and fumes procedures, major electrical system failure. apparent to the crew Evaluation or scenario-based training phases Recognise system malfunction. (iii) System malfunctions that result in significant degradation of flight controls in combination with abnormal handling characteristics, e.g. jammed flight controls, Any item cleared by the MEL but Take appropriate action including certain degradation of FBW control, jammed horizontal stabiliser; flaps and/or slats having an impact upon flight correct stop/go decision. operations. For instance. thrust Apply the appropriate procedure locked; other malfunctions that result in degraded flight controls. (iv) System failures that require monitoring and management of the flight path using reverser locked. correctly. degraded or alternative displays, unreliable primary flight path information, unreliable Aircraft system Maintain aircraft control. malfunctions, Malfunctions to be considered should Manage consequences. airspeed, e.g. flight with unreliable airspeed (v) System failures that require extensive management of their consequences including B have one or more of the following (independent of operation or environment), e.g. fuel leak. operations under characteristics: Apply crew operating procedures MEL TO — Immediacy where necessary. MEL items with crew operating procedures applicable during take-off x X — Complexity Respond appropriately to Response to an additional factor that is affected by a MEL item (e.g. system failure, x x x X TO — Degradation of aircraft control additional system abnormalities runway state) GND — Loss of primary instrumentation associated with MEL dispatch. Malfunction during preflight preparation and prior to departure x x x CLB — Management of consequences Malfunction after departure x x x X The operator should vary Malfunctions that require immediate attention (e.g. bleed fault during engine start, x x x ALL malfunctions for each characteristic hydraulic failure during taxi) CLB over the EBT cycle. x x x X Fuel leak (management of consequences) CRZ TO Malfunction on take-off high speed below V1 x x x TO Malfunction on take-off high speed above V1 x x During taxi to the runway, a spurious brake temperature announcement. The crew had GND X X X the correct brake temperature moments before the failure. TO Tyre failure during take-off x x x TO Malfunction on initial climb x x APP Malfunction on approach x x x APP Malfunction on go-around x x x LDG Malfunction during landing x x x x x TO Any engine failure or malfunction, Recognise engine failure. Engine failure or engine malfunction on take-off low speed x x x x TO which causes loss or degradation of Take appropriate action. Engine failure or engine malfunction on take-off high speed below V1 x x x x Engine failure B TO thrust that affects performance. This Apply appropriate procedure Engine failure or engine malfunction on take-off above V1 x x x x TO is distinct from the engine-out correctly. Engine failure or engine malfunction on initial climb x x x APP manoeuvres described in the Maintain aircraft control. Engine malfunction x x x Page 195 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map CRZ manoeuvres training section above, Manage consequences. Engine failure in cruise (with autopilot) x x x which are intended only to practise x psychomotor skills and reinforce LDG Engine failure or engine malfunction on landing procedures to manage engine failures. Pilots should have opportunities to practise landings in demanding situations at the defined frequency. Data indicates that landing problems EVAL or SBT have their roots in a variety of factors, Landing in demanding This topic should be combined with the adverse-weather topic, aircraft system including inappropriate decision- Landing B LDG environmental conditions, with malfunctions topic or any topic that can provide exposure to a landing in demanding Intentionally blank making, in addition to manual aircraft malfunctions as appropriate conditions. control skills if difficult environmental conditions exist. The purpose of this item is to ensure that pilots are exposed to this during the programme. The data analysed during the development of the EBT concept indicated substantial difficulties Evaluation or scenario-based training phases encountered by crews when faced with a threat or error, which was a surprise or an unexpected event. The element of surprise should be distinguished from what is sometimes referred to as the ‘startle factor’ — the latter being a physiological Exposure to an unexpected event reaction. Wherever possible, or sequence of events at the Surprise B ALL Intentionally blank Intentionally blank consideration should be given defined frequency in order to build towards variations in the types of resilience. scenario, times of occurrences and types of occurrence, so that pilots do not become overly familiar with repetitions of the same scenarios. Variations should be the focus of EBT programme design, and not left to the discretion of individual instructors, in order to preserve programme integrity and fairness. ALL ATC clearance giving insufficient terrain clearance x x x Anticipate terrain threats. Prepare for terrain threats. Demonstration of terrain avoidance warning systems (TAWS) (this scenario element ALL x x x Recognise unsafe terrain clearance. may be done in an ISI.) EVAL or SBT TO Take appropriate action. Engine failure where performance is marginal leading to TAWS warning x x x Terrain B CLB Alert, warning, or conflict Apply the appropriate procedures DES correctly. ATC provides a wrong QNH x x APP Maintain aircraft control. Restore safe flight path. ‘Virtual mountain’ refers to the surprise element of an unexpected warning. Care should DES Manage consequences. be exercised in creating a level of realism, so this can best be achieved by an unusual x x x and unexpected change of route during the descent. Page 196 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map This is not considered a topic for Intentionally blank Intentionally blank specific attention on its own, but more as a reminder to programme EVAL or SBT developers to ensure that pilots are Manage available resources Workload, exposed to immersive training efficiently to prioritise and perform distraction, B ALL scenarios which expose them to tasks in a timely manner under all pressure, stress manageable high workload and circumstances distractions during the course of the EBT programme, at the defined frequency. Section 3 — Training topics frequency (C) per phase and in alphabetical order, except for the upset prevention due to the difference in the EBT phases N/A Compliance with AMC1 or AMC2 to Table 2 of AMC1 ORO.FC.220&230: Exercises for upset recovery training ORO.FC.220&230 Intentionally blank A. Recovery from developed upsets Recovery from stall events, in the following configurations; Include the recovery exercises in Table 2 of AMC1 ORO.FC.220&230 for — take-off configuration, CLB the recurrent training programme, such that all the exercises are covered 2. — clean configuration low altitude, x x x x DES over a period not exceeding 3 years. Through the principles of EBT, — clean configuration near maximum operating altitude, and Recognise upset condition. covering one component should — landing configuration during the approach phase. Make timely and appropriate satisfy the requirement to cover the CRZ intervention. 3. Recovery from nose high at various bank angles x x x x whole element of recovery from CRZ Take appropriate action. x x x x developed upsets. The same 4. Recovery from nose low at various bank angles Manoeuvres training phase or SBT CRZ Assure timely and appropriate principles applies to the exercises of intervention. (AMC1 Demonstration at a normal cruising altitude. Set conditions and disable aircraft x x x components 2, 3 and 4 where one APP ORO.FC.220&230 Table 2 systems as necessary to enable trainee to perform stall recovery according to OEM exercise may satisfy the requirement component 1) instructions to cover the whole component. Assure aircraft control. Demonstration at an intermediate altitude during early stages of the approach. Set x x x An aeroplane upset is defined as an Maintain or restore a safe flight conditions and disable aircraft systems as necessary to enable trainee to perform stall Upset recovery C undesired aeroplane state in flight path. recovery according to OEM instructions characterised by unintentional Assess consequential issues. divergences from parameters Manage outcomes. normally experienced during line Consolidated summary of operations or training. An aeroplane aeroplane recovery techniques. upset may involve pitch and/or bank (AMC1 ORO.FC.220&230 Table 2 angle divergences as well as component 5) inappropriate airspeeds for the CLB Note: The operator should assess if conditions. DES the exercises should be practice for The example scenario elements may the either seat qualification. Recovery from a wake turbulence position with high-bank angle x x x x be done in ISI, as non-ISI or a combination of both. If done in ISI: The instructor should position the aircraft within but close to the edge of the validated training envelope before handing control to the trainee to demonstrate the restoration of normal flight. Careful consideration should be given to Page 197 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map flying within the validated training envelope. TO Take-off with different crosswind/tailwind/gust conditions x x TO Take-off with unreported tailwind x x TO Crosswinds with or without strong gusts on take-off x x APP Increasing tailwind on final approach(not reported) x x x x Recognise adverse-wind Approach and landing in demanding weather conditions, e.g. turbulence, up and x x x APP conditions. Adverse wind/crosswind. This downdrafts, gusts and crosswind including shifting wind directions Observe limitations. Adverse wind C APP includes tailwind but not ATC mis- Adverse-wind scenario resulting in increasing tailwind below DA (not reported) x x x Apply appropriate procedures. reporting of the actual wind. Adverse-wind scenario including strong gusts and/or crosswind out of limits below DA x x x APP Maintain directional control and EVAL or SBT safe flight path. (not reported) Adverse-wind scenario including strong gusts and/or crosswind out of limits below x x x APP 15 m (50 ft) (not reported) APP Crosswind with or without strong gusts on approach, final and landing (within and x x x LDG beyond limits) APP Recognise actual conditions. Approach in poor visibility x x x x Observe aircraft and/or procedural Approach in poor visibility with deteriorations necessitating a decision to perform a x x x APP Approach, limitations. go-around Any situation where visibility visibility close to C Apply appropriate procedures if x x x becomes a threat minimum applicable. LDG Landing in poor visibility Maintain directional control and safe flight path. ALL ATC role-play: the instructor provides scripted instructions, as a distraction to the crew x x x ALL ATC error. Omission, Controller error, provided by the instructor according to a defined scripted scenario x x x x Respond to communications ALL miscommunication, garbled, poor Frequency congestion, with multiple aircraft using the same frequency x appropriately. quality transmission. All of these act EVAL or SBT APP Recognise, clarify and resolve any Destination temporally closed x x x x as distractions to be managed by the CRZ ambiguities. Rescue and firefighting services (RFFS) level reduction at destination x x x ATC C crew. The scenarios should be Refuse or question unsafe Runway change before the interception of the localiser or similar navigation aid in x x x x APP combined where possible, with instructions. azimuth others of the same or higher GND/ Use standard phraseology x x x weighting, the principle reason being Stray dogs at the opposite threshold runway TO whenever possible. to create distractions. ALL Poor quality transmissions x GND Fire in cargo or cabin/cockpit at gate x x x x GND Fire during taxi x x x x X GND Recognise fire, smoke or fumes Fire with no cockpit indication x x x x X EVAL or SBT TO Take appropriate action. Take-off low speed x x x x X This includes engine, electric, Fire and smoke TO Apply appropriate procedure Fire or smoke on Take-off high speed below V1 x x x x C pneumatic, cargo fire, smoke or management TO correctly. Fire or smoke on Take-off high speed above V1 x x x fumes. TO Maintain aircraft control. Fire or smoke on Initial climb x x x CRZ Manage consequences. Cargo fire x x x APP Engine fire in approach (extinguishable) x x APP Engine fire in approach (non-extinguishable) x x x Page 198 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map CLB CRZ Lithium battery fire in the cockpit or cabin compartment x x x x x DES APP Flight deck or cabin fire x x x X GND Any of the example scenarios elements above ending in an evacuation x x x x GND Recognise loss of communications. Loss of communications during ground manoeuvring x x TO Lost or difficult communications. Take appropriate action. Loss of communications after take-off x x X EVAL or SBT Either through pilot mis-selection or a Execute appropriate procedure as x x x x X Loss of C failure external to the aircraft. This applicable communications APP could be for a few seconds or a total Use alternative ways to Loss of communications during approach phase, including go-around loss. communicate Manage consequences This can be a demonstrated error, in that the crew may be instructed to deliberately x x x Anticipate the potential for errors insert incorrect data — for example, to take off from an intersection with full-length ALL in load/fuel/performance data. performance information. The crew will be asked to intervene when acceleration is A calculation error by one or more Recognise inconsistencies. sensed to be lower than normal, and this may be part of the operator procedures, EVAL or SBT Managing especially when operating mixed fleets with considerable variations in MTOM. pilots, or someone involved with the Manage/avoid distractions. loading, fuel, C process, or the process itself, e.g. Make changes to Fuel ground staff on industrial action. Only limited amount of fuel available, which is x x x x performance GND incorrect information on the load paperwork/aircraft system(s) to below the calculated fuel for the flight. errors sheet eliminate error. Advice crew that there is a change of the load sheet figures during taxi to the runway. x x GND Identify and manage The crew may have limited time due to a calculated take-off time (CTOT) — ATC slot consequences. Braking action reported ‘medium’. The information is transmitted just before take-off. x x x GND The flight is subject to a calculated take-off time (CTOT) — ATC slot. x x x x External failure or a combination of external failures degrading aircraft navigation GND performance on ground TO x x x x Evaluation or scenario-based training phases CLB External failure or a combination of external failures degrading aircraft navigation APP performance in flight Recognise a NAV degradation. LDG Take appropriate action. External NAV failure. Standard initial departure change during taxi. The flight may be subject to a CTOT — GND Execute appropriate procedure as x x x Navigation C Loss of GPS satellite, ANP exceeding ATC slot. applicable. APP RNP, loss of external NAV source(s) Loss of runway lighting below decision height x x x Use alternative NAV guidance. Manage consequences. No fly zone: when the crew changes control frequency, the new ATCO informs the crew that they are flying over an unannounced ‘no fly zone’ and not included in the NOTAMs. To trigger such an event, the context can as an example be as follows: an unexpected CRZ military conflict in the territory the aircraft is flying over or the crew is forced to re-route x x x in flight and the new route flies over a city that has an important event such the Olympic games, a G20/G7 submit, or the route is flying near a space rocket launch close to the time of the launch, like Guiana Space Centre, cape Cañaveral, etc.). Operations- or C ALL Intentionally blank Intentionally blank Intentionally blank Intentionally blank type-specific The operator should comply with Operations of the national qualification APP See equivalency of approaches special airport C requirements published in the Intentionally blank Intentionally blank LDG relevant to operations. approval Aeronautical Information Publication. Page 199 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Assessment and Description (includes type of Desired outcome Guidance material (GM) Flight phase for activation training topic topic, being threat, error (includes performance criteria Example scenario elements or focus) OR training outcome) Frequency WLM COM SAW FPM KNO LTW PRO PSD FPA Generation 2 Turboprop — Recurrent assessment and training matrix Competency map Recognise incapacitation. TO Take appropriate action including During take-off x x x x X correct stop/go decision. Pilot Consequences for the non- C Apply appropriate procedure incapacitation incapacitated pilot APP correctly. During approach x x x X Maintain aircraft control. EVAL or SBT Manage consequences. GND x X Recognise hazardous runway Planned anticipated hazardous conditions with dispatch information provided to TO condition. facilitate planning and execution of appropriate procedures LDG Contamination or surface quality of Observe limitations. Runway or GND x x x C the runway, taxiway, or tarmac Take appropriate action. Unanticipated hazardous conditions, e.g. unexpected heavy rain resulting in flooded taxiway condition TO including foreign objects Apply appropriate procedures runway surface LDG correctly. TO Take-off on runway with reduced cleared width due to snow x x x x Assure aircraft control. TO Stop/go decision in hazardous conditions x x x Anticipate potential loss of separation. EVAL or SBT Recognise loss of separation. CLB Traffic conflict. ACAS RA or TA, or Take appropriate action. Traffic C CRZ visual observation of conflict, which ACAS warning that requires crew intervention x x x x Apply appropriate procedure DES requires evasive manoeuvring correctly. Maintain aircraft control. Manage consequences. Anticipate potential for wind shear. Avoid known wind shear or prepare for suspected wind shear. TO Recognise wind shear encounter. Predictive wind shear warning during take-off x x Take appropriate action. With or without warnings including Apply appropriate procedure predictive. A wind shear scenario is Wind shear correctly. C ideally combined into an adverse- recovery Assure aircraft control. TO weather scenario containing other Wind shear encounter during take-off x x x Recognise out of wind shear TO elements. Wind shear encounter after rotation x x condition. TO Maintain or restore a safe flight Predictive wind shear after rotation x x APP path. Predictive wind shear during approach x x x APP Assess consequential issues and Wind shear encounter during approach x x x manage outcomes. Page 200 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail CS OPS.EBT.232.7 Generation 1 (jet) table of assessment and training topics GENERATION 1 (JET) — EBT PROGRAMME — TABLE OF ASSESSMENT AND TRAINING TOPICS Given the very small number of turbo-jet aeroplanes of the first generation in current use in commercial air transport operations and the lack of appropriate FSTDs for recurrent training, it has not been deemed possible to provide a table of assessment and training topics for those aeroplanes and therefore it is not possible to apply EBT. Page 201 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Appendices 2 to 6 Summary of amendments to Appendices 2 to 6 to Doc 9995: — The competency KNO is introduced and its competency map (34 hits for GEN4). — The wording ‘Guidance material’ is introduced in the ‘example scenario element column’ to indicate that this column is GM. — The ‘rejected take-off’ manoeuvre in generation 4 and 3 Jet has moved from frequency A in Doc 9995 to frequency B. The ATQP operators in the RMG demonstrated that their pilots are equally proficient in demonstrating this manoeuvre. The amendment was agreed in June 2019. — The engine failure on take-off followed the same approach described above for ‘rejected take- off’; however, EASA did not find such general consensus for the ‘failure of critical engine between V1&V2’ for generation 3; therefore, for generation 3, only one of the two engine failures has move from frequency A in Doc 9995 to frequency B. For generation 4, both engine failures have moved from frequency A to frequency B. — A new manoeuvre ‘failure of the critical engine above V2 (any segment of the TO)’ is introduced at a frequency B. This manoeuvre complements the existing manoeuvre of ‘failure of one engine on take-off - failure of one engine from V1 and before reaching V2’; only one of them is required. The reason is to allow the pilot to cope with this failure outside the segment of V1 and V2. Data provided by the operators in the RMG shows that engine failures are more probable in another segment than V1 and V2. Therefore, it allows the operators to complement their programme with a manoeuvre that should cover better their operational risks. The amendment was agreed in June 2019. — The three go-arounds in the manoeuvres training phase are merged because it was confusing for the operators. Frequencies are also merged. That means that the operator may choose only one of the three go-arounds at a frequency A. — The introduction of either seat qualification in accordance with ORO.FC.235 with a frequency B in line with ATQP ORO.FC.A.245. — Training topic ‘adverse weather’ — example scenario element ‘adverse-weather scenario’ e.g. thunderstorm activity, precipitation, icing, flight phase activation amend from take-off (TO) to all phases of flight (ALL). — Training topic ‘automation management’ — for three example scenario elements, the flight phase activation has changed from ALL to CLB, CRZ, DES, APP, as those example scenario elements cannot be triggered on ground (e.g. recoveries from TAWS, ACAS warnings, recovery and subsequent engagement of automation). — Same as above in training topic ‘manual aircraft control’ (e.g. ACAS RA to descend or ATC […]). — Training topic ‘competencies non-technical (CRM)’ — example scenario element ‘ACAS warning immediately following a go-around, with a descent manoeuvre required’ — the activation phase is changed from CRZ to APP. — Training topic ‘manual aircraft control’ — a new example scenario element and its competency map were introduced (Approach planned with autoland, followed by a failure below 1 000 feet […]). — Training topic ‘monitoring, cross checking, error management, and mismanagement aircraft Page 202 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail state’ — the term ‘in-seat instruction’ is deleted. Feedback from operators implementing mixed EBT has highlighted that ISI is not the only means of training this operational risk; therefore, an increased flexibility in regard to the means to deliver this training topic was introduced. Furthermore, the ‘Data Report for Evidence-Based Training’7 DOES NOT make any reference to in-seat instruction. — Training topic ‘upset prevention training’ — extensive amendments are introduced. Doc 9995 was published before Doc 10011 ‘UPRT manual’, and therefore Doc 9995 does not provide the latest training exercises for UPRT. The new provision proposed in AMC8 ORO.FC.231 point (a) requires compliance with AMC1&2 ORO.FC.220&230. The new text allows training this topic in all phases of the modules providing thus more flexibility. — Training topic ‘aircraft system malfunctions, including operations under MEL’ — a new example scenario element and its competency map were introduced (fuel leak (management of consequences)). — Training topic ‘terrain’ — the example scenario element of demonstration of TAWS is amended to allow operators to train this exercise with ISI in order to avoid negative training for pilots. — Stress is added to the original training topic ‘workload, distraction, pressure’ as according to the experts consulted, it is covered in this training topic. In addition, there is alignment with the provision of CRM. — A new training topic (operations of special airport approval) was introduced with a frequency of ‘C’ in order to ensure time for airports with special approval (e.g. Funchal, Innsbruck, etc.) — Training topic ‘upset recovery training’ was extensively amended. ICAO Doc 9995 was published before the ICAO Doc 10011 ‘UPRT manual’, and therefore Doc 9995 does not provide the latest training exercises for UPRT. The new text requires compliance with AMC1&2 ORO.FC.220&230. The new text allows training this topic in the MT and SBT of the modules providing thus more flexibility. EASA excluded this training topic (recovery) from the evaluation phase. The reason agreed by the experts consulted by EASA was: in the evaluation phase, every skilled pilot will avoid in the upset prevention stage the need to go into a recovery from upset; therefore, in order to avoid negative training, the recovery part should be avoided in the evaluation phase. — Furthermore, the experts consulted by EASA found that some of the recovery example scenario elements described in Doc 9995 to be example scenario elements related to prevention; therefore, EASA transferred them to the training topic of upset prevention — frequency B. For example, the example scenario element ‘Demonstration of the defined normal flight envelope and any associated changes in flight instruments, flight director systems, and protection systems. This should take the form of an instructor-led exercise to show the crew the points beyond which an upset condition could exist’ is located in Doc 9995 in the training topic ‘upset recovery’; however, in AMC1 ORO.FC.220&230 table 1 and Doc 1001 ‘UPRT manual’, this example scenario element is located in the prevention part; therefore, the conclusion of EASA and its experts was to move it to upset prevention. — Table 2 of AMC1 ORO.FC.220&230 – Recovery- elements and components were transposed into the training topic of recovery in CS OPS.EBT.232. The competency map was agreed following the Delphi methodology. 7 IATA Data Report for Evidence-Based Training August 2014 1st Edition. Page 203 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail — Some more example scenario elements were introduced by the experts of the rulemaking group with a special emphasis on scenarios of LTW and WLM. ORO.FC.240 Operation on more than one type or variant AMC1 ORO.FC.240 Operation on more than one type or variant GENERAL (a) Aeroplanes (1) When a flight crew member operates more than one aeroplane class, type or variant, as determined by the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for class-single pilot or type-single pilot, but not within a single licence endorsement, the operator should ensure that the flight crew member does not operate more than: (i) three reciprocating engine aeroplane types or variants; (ii) three turbo-propeller aeroplane types or variants; (iii) one turbo-propeller aeroplane type or variant and one reciprocating engine aeroplane type or variant; or (iv) one turbo-propeller aeroplane type or variant and any aeroplane within a particular class. (2) When a flight crew member operates more than one aeroplane type or variant within one or more licence endorsement, as determined by the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012, the operator should ensure that: (i) the minimum flight crew complement specified in the operations manual is the same for each type or variant to be operated; (ii) the flight crew member does not operate more than two aeroplane types or variants for which a separate licence endorsement is required, unless credits related to the training, checking, and recent experience requirements are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for the relevant types or variants; and (iii) only aeroplanes within one licence endorsement are flown in any one flight duty period, unless the operator has established procedures to ensure adequate time for preparation. (3) When a flight crew member operates more than one aeroplane type or variant as determined by the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for type-single pilot and type-multi pilot, but not within a single licence endorsement, the operator should comply with points (a)(2) and (4). (4) When a flight crew member operates more than one aeroplane type or variant as determined by the operational suitability data established in accordance with Page 204 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Commission Regulation (EU) No 748/2012 for type multi-pilot, but not within a single licence endorsement, or combinations of aeroplane types or variants as determined by the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for class single-pilot and type multi-pilot, the operator should comply with the following: (i) point (a)(2); (ii) before exercising the privileges of more than one licence endorsement: (A) flight crew members should have completed two consecutive operator proficiency checks OPCs and should have: — 500 hours in the relevant crew position in CAT operations with the same operator; or — for IFR and VFR night operations with performance class B aeroplanes, 100 hours or flight sectors in the relevant crew position in CAT operations with the same operator, if at least one licence endorsement is related to a class. A check flight should be completed before the pilot is released for duties as commander; (B) in the case of a pilot having experience with an operator and exercising the privileges of more than one licence endorsement, and then being promoted to command with the same operator on one of those types, the required minimum experience as commander is 6 months and 300 hours, and the pilot should have completed two consecutive operator proficiency checks OPCs before again being eligible to exercise more than one licence endorsement; (iii) before commencing training for and operation of another type or variant, flight crew members should have completed 3 months and 150 hours flying on the base aeroplane, which should include at least one proficiency check, unless credits related to the training, checking and recent experience requirements are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for the relevant types or variants; (iv) after completion of the initial line check on the new type, 50 hours flying or 20 sectors should be achieved solely on aeroplanes of the new type rating, unless credits related to the training, checking and recent experience requirements are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for the relevant types or variants; (v) recent experience requirements established in Commission Regulation (EU) No 1178/2011 for each type operated; (vi) the period within which line flying experience is required on each type should be specified in the operations manual; Page 205 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail (vii) when credits are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for the relevant type or variant, this should be reflected in the training required in ORO.FC.230 and: (A) ORO.FC.230 (b) requires two operator proficiency checks OPCs every year. When credits are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for operator proficiency checks OPCs to alternate between the types, each operator proficiency check OPC should revalidate the operator proficiency check OPC for the other type(s). The operator proficiency check OPC may be combined with the proficiency checks for revalidation or renewal of the aeroplane type rating or the instrument rating in accordance with Commission Regulation (EU) No 1178/2011. For EBT programmes, ORO.FC.231(a)(3) requires the pilot to complete a minimum of two modules of the EBT programme, separated by a period of more than 3 months, within a 12-month period. In addition, the pilot is required to be trained according to assessment and training topics distributed across a 3-year period at the defined frequency relevant to the type or variant of aircraft. When credits are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012, EBT modules should alternate between types. The EBT modules may be combined for revalidation or renewal of the aeroplane type rating or the instrument rating in accordance with Commission Regulation (EU) No 1178/2011. When operating more than one type of different generations, the operator has to fulfil both generation table of assessment and training topics as per ORO.FC.232. (B) ORO.FC.230 (c) requires one line check every year. When credits are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for line checks to alternate between types or variants, each line check should revalidate the line check for the other type or variant. For EBT programmes, ORO.FC.231(h) requires one line evaluation of competence every year. When credits are defined in the operational suitability data established in accordance with Commission Regulation (EU) No 748/2012 for line evaluation of competence to alternate between types or variants, each line evaluation of competence should revalidate the line evaluation of competence for the other type or variant. In such case, the operator should meet the requirements to extend the validity of the line evaluation of competence to 2 years. Extension to 3 years should not be allowed. (C) Annual emergency and safety equipment training and checking should cover all requirements for each type. (b) Helicopters […] AMC1 ORO.FC.240 The RMG developed the AMC and concluded that: Page 206 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail — ORO.FC.240 is applicable to EBT and does not require modification. However, some minor modifications were needed in AMC1 ORO.FC.240. ‘ORO.FC.140 Operation on more than one type or variant (a) Flight crew members operating more than one type or variant of aircraft shall: comply with the requirements prescribed in this Subpart for each type or variant, unless credits related to the training, checking, and recent experience requirements are defined in the mandatory part of the operational suitability data established in accordance with Regulation (EU) No 748/2012 for the relevant types or variants. (b) Appropriate procedures and/or operational restrictions shall be specified in the operations manual for any operation on more than one type or variant.’ — Part-FCL of the Aircrew Regulation and AMC1 FCL.740 point (a) should specify that in case of operations on more than one type or variant, two modules shall be performed on each type or variant for revalidation. In addition, the RMG discussed whether the simulator sessions of the module should be performed in the same aircraft type or it is possible to perform the simulator sessions in different aircraft types. The conclusion was that simulator session should be performed in the same aircraft type. Finally, the RMG concluded that in case of different generations of aircraft, the operator has to fulfil both generations’ EBT programme requirements as per AMC 2,3,4,5 to 6 ORO.FC.231(a). AMC1 ORO.FC.240 point (a)(4)(vii) The RMG following the principles contained in ORO.FC.240 agreed to not allow extension of validity of the line evaluation of competence further than those allowed in ORO.FC.240. ORO.FC.240 and AMC1 ORO.FC.240 allow consecutive line checks; therefore, a check is required every year; however, this is made alternatively in each type, so a check is performed for each single type every 2 years. This can be seen as an extension of the validity period of the line evaluation of competence. Therefore, the provision proposed for the line evaluation of competence limits the extension of 3 years only to single fleet operation and therefore ensures for operations of more than one type or variant 1 line evaluation of competence every 2 years, that ensures at least 1 LEoC every 3 years programme. AMC1 ORO.FC.240 point (a)(4)(vii)(B) The RMG following the principles contained in ORO.FC.240 agreed to not allow extension of validity of the line evaluation of competence further away than those allowed in ORO.FC.240. ORO.FC.240 and AMC1 ORO.FC.240 allow consecutive line checks; therefore, a check is required every year; however, this is made alternatively in each type, so a check is performed for each single type every 2 years. This can be seen as an extension of the validity period of the line evaluation of competence. Therefore, point (f) limits the extension of 3 years only to single fleet operations and therefore ensures for operations of more than one type or variant one line evaluation of competence every 2 years, which ensures at least one line evaluation of competence every 3 years. Annex I (Part-FCL) to Regulation (EU) No 1178/2011 Concept of revalidation within an EBT programme Page 207 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Background of licence revalidation — The current revalidation process has four components: (a) the applicant; (b) the examiner; (c) the technical assessment carried out in the simulator or the aircraft; and (d) the administrative procedure that includes the completion of Appendix 9, and the rest of administrative procedures in Part-FCL FCL.1030 points (b), (c) and (d) that include the licence endorsements. This process is carried out by the same person (examiner) who performs the technical assessment and the administrative procedure at the same ‘location’ (simulator or aircraft) and at the same time (the date and time of the proficiency check). Note: Although most of the LPCs are carried out by a single examiner, the possibility of having several examiners for the same check already exists. — The EBT philosophy should provide a different approach, where training is maximised and therefore checks disappear (assessment is introduced) and the pilot is trained in NON-jeopardy environment. Furthermore, the continuous training evidence of the pilot (data) should provide a better assessment of the competence of the pilot. Therefore: (a) the EBT technical assessment has several events (simulator sessions) instead of one; (b) there are several assessors of pilot performance (EBT instructors) instead of just one (examiner); however, the nominated person for crew training, who is an examiner designated to provide a final assessment of the data collected, and the administrative procedure should be maintained. As there are several people involved in the technical assessment, the administrative procedure involves the nominated person for crew training who bears the responsibility of the licence revalidation and a designated person who will endorse the licence. Concept of licence revalidation in the context of an operator’s EBT programme The revalidation process proposed has the following components: (a) the applicant; (b) the people involved in the revalidation of the pilot licence: (1) the nominated person for crew training (or the deputy(ies)) who is an examiner responsible for the operator’s EBT programme — ensuring that the manoeuvres assessed are of a good training value and that the applicant completed those manoeuvres. The nominated person will be mostly responsible for the completion of Appendix 10. This Page 208 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail person (or the deputy(ies)) also has the overall picture of the pilot training data for the period of validity (as shown by the evidence provided by the EBT programme); (2) the designated person who has the signature delegation from the nominated person to endorse the licence and complete Appendix 10; and (3) the EBT instructors who delivered each of the technical assessments that provide data to the EBT grading system and the training system performance; (c) the several technical assessments carried out in the simulators which provide the necessary evidence to ensure the pilot has an acceptable level of performance; and (d) the administrative procedure which includes the completion of Appendix 10 and the rest of administrative procedures provided in FCL.1030. FCL.015 Application and issue, revalidation and renewal of licences, ratings and certificates SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb GM1 FCL.520.A ATPL(A) – Skill test ATPL SKILL TEST IN AN EBT MODULE The skill test in accordance with Appendix 9 may be combine with an EBT module. It may follow the same process already described in mixed EBT for the LPC. The competent authority may provide guidance. More information may be found in the EASA EBT manual. FCL.625 IR — Validity, revalidation and renewal SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb FCL.625.A IR(A) — Revalidation SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb FCL.740 — Validity and renewal of class and type ratings SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb AMC2 FCL.740(b) Validity and renewal of class and type ratings RENEWAL OF CLASS AND TYPE RATINGS: REFRESHER TRAINING AT AN AOC An AOC approved for renewal of type ratings under Part-ORO can provide refresher training only: (a) for applicants enrolled under its own EBT programme; and (b) if the rating has lapsed by no more than 1 year. If the rating has lapsed by more than 1 year, the training should be performed at an ATO and AMC1 FCL.740(b) applies. FCL.740.A — Revalidation of class and type ratings — aeroplanes SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb Page 209 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail FCL.905.TRI TRI — Privileges and conditions SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb FCL.905.SFI SFI — Privileges and conditions SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb FCL.1025 Validity, revalidation and renewal of examiner certificates SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb FCL.1030 Conduct of skill tests, proficiency checks and assessments of competence AMC1 FCL.1030(b)(3) Conduct of skill tests, proficiency checks and assessments of competence OBLIGATIONS FOR EXAMINERS APPLICATION AND REPORT FORMS […] (b) For training, skill tests or proficiency checks for ATPL, MPL or class and type ratings, in AMC1 to Appendix 9 or AMC1 to Appendix 10; GM1 FCL.1030(b)(3)(ii) Conduct of skill tests, proficiency checks and assessments of competence REVALIDATION OF CLASS AND TYPE RATINGS — AEROPLANES — REQUIRED MANOEUVRES AND EXERCISES FOR APPENDIX 10 The confirmation that all the required manoeuvres and exercises have been completed means that during the period of validity, the applicant has completed the operator’s EBT programme corresponding to that period of validity as declared by the nominated person for crew training (or the deputy(ies)). GM1 FCL.1030(b)(3)(ii) This GM is developed to clarify the responsibility of the TRE as regards ‘the required manoeuvres and exercises’. See point (b)(3)(ii) of FCL.1030 of the Aircrew Regulation below: ‘FCL.1030 (…) (ii) confirmation that all the required manoeuvres and exercises have been completed, as well as information on the verbal theoretical knowledge examination, when applicable. If an item has been failed, the examiner shall record the reasons for this assessment;’ Appendix 10 to Annex I (Part-FCL) to Regulation (EU) No 1178/2011 Appendix 10 — Proficiency check for type ratings, and proficiency check for IRs when combined with type ratings — EBT practical assessment SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb Page 210 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail AMC1 to Appendix 10 — Proficiency check for type ratings, and proficiency check for IRs when combined with type ratings — EBT practical assessment APPLICATION AND REPORT FORM — LICENCES ADMINISTRATIVE PROCEDURES (a) For the purpose of revalidation, the Examiner Differences Document applies to the nominated person for crew training or the deputy(ies). (b) Minimum information provided in the form for Appendix 10. Applicant’s last name(s): Applicant’s first name(s): Signature of applicant: State of licence issue: Type of licence held: Licence number: Type rating: FSTD (aircraft type): Session 1……..Name of the instructor:____________________________________________, Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Session 2……..Name of the instructor:____________________________________________, EBT module 1 Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Session X…...Name of the instructor:____________________________________________, Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Completion of the module: ___________________________________________date / signature (nominated person for crew training) Session 1……..Name of the instructor:____________________________________________, Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Session 2……..Name of the instructor:____________________________________________, EBT module 2 Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Session X…...Name of the instructor:____________________________________________, Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Completion of the module: ___________________________________________date / signature (nominated person for crew training) (…) Session X…..Name of the instructor:____________________________________________, Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Session Y……..Name of the instructor:____________________________________________, EBT module X Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Session Z…...Name of the instructor:____________________________________________, Type and number of licence:___________________________________________________ Location, date and time: ____________________ FSTD ID code:______________________ Completion of the module: ____________________________________________date / signature (nominated person for crew training) Completion of the operator’s EBT programme __________________________________date / signature from____(date) to ____(date) (nominated person for crew training) Page 211 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Name(s) in capital letters: Signature of examiner (nominated person for crew Type and number of licence: training or deputy) ______________________ Examiner certificate number: Date: Delegation of signature for licence endorsement (instructor) Name: Signature Position in the operator: Date: GM1 to Appendix 10 — Proficiency check for type ratings, and proficiency check for IRs when combined with type rating — EBT practical assessment REVALIDATION OF LICENCES — ADMINISTRATIVE PROCEDURES (a) The use of deputies with examiner certificate to ensure at least one examiner for each fleet is equally valid for other reasons, such as workload to manage the EBT programme, several locations of the training facilities, or bases, etc. (b) For the first licence revalidation after the transition from mixed EBT, the examiner may use one mixed EBT module in addition to the other EBT module(s) as a means to revalidate the licence. (c) In accordance with the approved procedure under Appendix 10, 5. (c) (1), the nominated person for crew training (or the deputy) may nominate the EBT instructor who completed the EBT module as the person to whom the signature of the examiner is delegated. A stamp or electronic signature may exclusively be given from the nominated person crew training (or the deputy) to the EBT instructor, in order to document the delegation in a transparent and secure manner. Following that process, EBT instructors can endorse an applicant’s licence. GM1 to Appendix 10 point (b) The GM clarifies what the examiner can do during the transition to full EBT in the case the pilot has not completed two EBT modules under full EBT. As during mixed EBT the pilot is completing an EBT module, this can be used as a means to revalidate the licence under full EBT. The GM was introduced as a consequence of the public consultation of the NPA. GM2 to Appendix 10 — Proficiency check for type ratings, and proficiency check for IRs when combined with type rating — EBT practical assessment EBT PRACTICAL ASSESSMENT — PROFICIENCY CHECK Practical assessment (or EBT practical assessment) is defined in FCL.010. More information can be found in ICAO Doc 9868 ‘PANS-TRG’. The demonstration of skills to revalidate or renew referred to in the definition of proficiency check in FCL.010 is equivalent to the EBT practical assessments conducted in the EBT programme and the final review of the examiner. In fact, one single EBT practical assessment demonstrates the necessary skills performed in legacy training; however, EBT goes one step further — to revalidate or renew, the pilot performs at least two demonstrations, corresponding to at least two EBT modules within the validity period of the type rating. GM2 to Appendix 10 Page 212 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail The GM transposed a definition from Part-ORO into Part-FCL and explains how EBT provides a demonstration of skills equivalent to the traditional proficiency check. The definition of proficiency check is already provided in FCL.010; therefore, it is not included in this GM. “‘Proficiency check’ means the demonstration of skill to revalidate or renew ratings, and including such oral examination as may be required.” In legacy training, such demonstration is performed in a single event (following Appendix 9). Although an EBT practical assessment is equivalent to a proficiency check and demonstrates the necessary skills to revalidate or renew ratings, EBT goes one step further and this demonstration is performed at least twice a year in each of the EBT modules, to complete the revalidation process. The demonstration of equivalency between Appendix 9 and the EBT module is performed at least once every 3 years as required under the several provisions (IR + AMC + GM) on ‘verification of the accuracy of the grading system’. To conclude this explanatory note, the definition of ‘competency’ (where the term ‘skills’ is included) in Annex I to the Air OPS Regulation is provided below. ‘competency’ means a dimension of human performance that is used to reliably predict successful performance on the job. A competency is manifested and observed through behaviours that mobilise the relevant knowledge, skills and attitudes to carry out activities or tasks under specified conditions; Rationale behind the equivalence between OPC, LPC and the EBT programme The EBT programme is aligned with the existing approach to OPC contained in ORO.FC.130, ORO.FC.230 and AMC1 ORO.FC.230 point (b): Authorities EASA AMC2 Doc 9995 OPS FCL Remarks comments ORO.FC.230 (a) 1.4 Use of Covered by LOE Covered by LOE Part-FCL Appendix 9 item checklist prior and SBT under and SBT under 1.4 may be assessed as (M) to starting compliance compliance crew actions during a engines, single preflight cockpit frequency A frequency A starting preparation. procedures, radio and navigation (…). The expected added value of EBT is that it assesses and develops the competency application of procedures in many events instead of only in an isolated task application. Page 213 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail 1.6 Before take- Covered by LOE Covered by LOE Part-FCL Appendix 9 item off checks and SBT under and SBT under 1.6 may be assessed as (M) compliance compliance crew actions during a single event during the frequency A frequency A before take-off procedures. The expected added value of EBT is that it assesses and develops the competency application of procedures in many events instead of only in an isolated task application. B 2.5.2 Take-off with Covered by the Covered by the The failure should be engine failure manoeuvres manoeuvres inserted between V1 and (M) between V1 training phase training phase V2 to create the need for and V2 (take- asymmetric handling. It is Failure of the Failure of the off safety possible to include critical engine critical engine speed) additional failures in order between V1 & V2 between V1 & V2 to comply with 3.6.1, frequency B two different which should be added frequencies are after item 2.5.2. requested During the manoeuvres frequency A for validation phase, this item initial control of should commence from the aircraft the initiation of the failure frequency B until until: clean (a) establishment of the configuration. final configuration; or (b) completion of the abnormal checklist. Page 214 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail A 2.6 Rejected take- Covered by the Covered by the The rejected take-off is off at a manoeuvres manoeuvres considered a crew item (M) reasonable training phase training phase and may be combined with speed before the rejected take-off for reaching V1. operators (LVOs) rejected take off rejected take off In the manoeuvres frequency A frequency A validation phase, this item should commence from the initiation of the failure until: (a) full stop and completion of the abnormal checklist initial actions; or (b) full stop and completion of abnormal checklist where items 3.6.1, 3.6.7 or 3.6.8 are combined. 3.4.0 Normal and LOE and SBT An exercise may validate abnormal several Part-FCL items to aircraft idem operations of malfunctions In order to facilitate the 3.4.14 systems. provision of simple and (M) Minimum of 3 realistic scenarios in for the crew example accordance with Doc 9995 scenario: Chapters 3.8 and 7.4, the ‘at least one evaluation phase is not malfunction for intended to be a each comprehensive characteristic assessment of all Part-FCL should be Appendix 9. included in every Pre-existing technical 12-month deviations and associated period’ operational instructions should not be taken into account as 3.4.0 to 3.4.14 items. Page 215 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail 3.6.1 Abnormal and Proposal to add emergency fire and smoke to procedures. management 3.6.9 Minimum of 3 The evacuation is (M) for the crew not prescribed in in Part-FCL 3.9.1* Adherence to No reference in The crew would be departure and table of assessed when required to (M) arrival routes assessment and follow a clearance, or and ATC training topics comply with a SID or STAR. instructions C 3.9.3.4 Manually, Manoeuvres with one training phase * (M) engine engine out simulated approach & go inoperative; around engine failure has to be simulated frequency A during final approach before passing 1 000 ft above aerodrome level until touch down or through the complete missed approach procedure. Page 216 of 221 European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail D 3.9.4 2D operations Manoeuvres No reference in This item should be down to training phase Doc 9995 but completed under * (M) MDH/A. equivalency of conditions described in the approaches applies relevant operations TYPE A 2D (refer to 3.8.4) manual. approach RNAV/GNSS approaches validate OPS – (D) item and Part-FCL 3.9.4 item. During the manoeuvres validation phase, this item should commence when intercepting the final approach and end when reaching the prescribed DA/H. E 4.3* Manual go- Manoeuvres Manoeuvres During the manoeuvres around with training phase training phase validation phase, this item (M) the critical may commence engine approaching DA and end simulated engine out engine out once the aircraft is inoperative approach & go approach & go established in a clean or after an around around defined normal instrument frequency B frequency A manoeuvring approach on configuration. reaching DH MDH or MAPt. F 5.5 Landing with Manoeuvres Manoeuvres In the manoeuvres the critical training phase training phase validation phase, this item (M) engine may start passing the final Engine out Engine out simulated approach fix (FAF) and end landing landing inoperative. when the aircraft reaches frequency B frequency A normal taxi speed. Page 217 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail Annex VI (Part-ARA) to Regulation (EU) No 1178/2011 ARA.GEN.315 Procedure for issue, revalidation, renewal or change of licences, ratings or certificates — persons AMC2 ARA.GEN.315(a) Procedure for issue, revalidation, renewal or change of licences, ratings or certificates — persons VERIFICATION OF COMPLIANCE OF THE EBT PROGRAMME In order to verify that the applicant meets the requirements for revalidation or renewal of ratings within an EBT programme, the competent authority should in addition to the requirements in AMC1 ARA.GEN.315(a) verify that: (a) the nominated person for crew training (or the deputy(ies) is a current examiner in the type rating filled in in Appendix 10; (b) when the nominated person for crew training (or the deputy(ies)) delegates their signature to endorse the licence of the applicant: (1) the delegation of signature should follow the operator’s approved procedure for such purpose; and (2) the person signing the licence should be nominated, have an instructor certificate and be indicated in Appendix 10; (3) the approved procedure for delegation of signature should include procedures to prevent the instructor from signing the licence when the performance of the applicant is below the minimum acceptable level or the EBT programme applicable to the validity period has not been completed. (c) the nominated person for crew training (or the deputy(ies)) of the operator in which the applicant is enrolled ensures that the applicant has completed the EBT programme; (d) the nominated person for crew training of the operator in which the applicant is enrolled ensures that the instructors that conducted the training to the applicant have been standardised. (e) the operator performs a verification of the grading system once every 3 years; and (f) the nominated person for crew training ensures the integrity of the pilot training data. AMC2 ARA.GEN.315(a) The EBT system integrates into a single concept the provisions for revalidation of licence in Part-FCL and those for recurrent training and checking in Part-ORO. Most of the requirements for the oversight in this Opinion are proposed in Part-ARO and then refer back to Part-FCL. See below. AMC1 ARO.OPS.226(d) Approval and oversight of evidence-based training programmes OVERSIGHT PLAN — PERIODIC ASSESSMENT TO VERIFY COMPLIANCE OF THE EBT PROGRAMME (c) Audits and inspections, on a scale and frequency appropriate to the operation, should cover at least: (…) Page 218 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail (9) administration of programme enrolment and compliance with the requirements of Annex I (Part-FCL) for licence revalidation and renewal; AMC2 ARA.GEN.315(a) further explains that the licensing competent authority should verify that the relevant requirements of Part-FCL are met. The intent of the RMG was that the verification could be completed by looking at the records. Instructors are already allowed to sign licences under FCL.945 in certain conditions: ‘FCL.945 Obligations for instructors Upon completion of the training flight for the revalidation of an SEP or TMG class rating in accordance with FCL.740.A (b)(1) and only in the event of fulfilment of all the other revalidation criteria required by FCL.740.A (b)(1) the instructor shall endorse the applicant's licence with the new expiry date of the rating or certificate, if specifically authorised for that purpose by the competent authority responsible for the applicant's licence.’ AMC2 ARA.GEN.315(a) point (b) EASA performed several rounds of consultations regarding the implementation of the delegation of signatures, one element brought up to the attention of EASA, was the need to ensure a lean process to perform the signature of licences. Although EASA has initiated the project to implement the electronic signature of licences, this project may take year to conclude. On the mean time, the simulator training centre may be a good location where after the completion of the training module the pilot and a personnel of the training department (instructor) may complete the signature of the licence. For the sake of clarity and due to some comments in the CRD regarding the process of signature delegation, EASA decided to provide clarity and level playing filed by requiring instructor to be the person where the examiner (head of training or deputy) may delegate its signature. AMC2 ARA.GEN.315(a) point (c) The intent of this provision is that the competent authority verifies the training records to ensure that the applicant has completed the EBT programme. Appendix 10 covers this item: ‘Completion of the operator’s EBT programme _______signature/date (nominated person for crew training)’ AMC2 ARA.GEN.315(a) point (d) This provision refers to AMC1 ORO.FC.146(c) and AMC2 ORO.FC.146(c). The oversight of this provision falls under the jurisdiction of the competent authority issuing the EBT approval; however, the licensing authority may at its own discretion inspect the training records of the instructors that pertain to revalidation of licences. AMC1 ORO.FC.146(c) Personnel providing training, checking and assessment EBT INSTRUCTOR — INITIAL STANDARDISATION PROGRAMME (a) Before delivering the operator’s EBT programme, the instructor should complete an EBT instructor initial standardisation programme composed of: Page 219 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail (1) EBT instructor training; and (2) EBT assessment of competence. […] AMC2 ORO.FC.146(c) Personnel providing training, checking and assessment EBT INSTRUCTOR — RECURRENT STANDARDISATION PROGRAMME The EBT instructor should: (a) conduct an EBT module every 12 months. The 12-month period should be counted from the end of the month the module was taken. If this has not been fulfilled, the EBT instructor should complete an EBT assessment of competence. When the module is undertaken within the last 3 months of the period, the new period should be counted from the original expiry date; (b) receive annual recurrent standardisation. The recurrent standardisation should include: (1) refresher EBT training; and (2) concordance training; and (c) complete an assessment of competence every 3 years. When the assessment of competence is conducted within the 12 months preceding the expiry date, the next assessment of competence should be completed within 36 calendar months of the original expiry date of the previous assessment of competence. ARA.FCL.200 Procedure for issue, revalidation or renewal of a licence, rating or certificate SEE IMPLEMENTING RULES IN THE OPINION ANNEX IIb ARA.FCL.205 Monitoring of examiners AMC2 ARA.FCL.205 Monitoring of examiners EBT PROGRAMME The competent authority responsible of the operator’s EBT programme should include in the monitoring of examiner programme the nominated person for crew training and the deputy(ies) even if they hold an examiner certificate issue by other competent authority. At the discretion of the competent authority, this may also include an inspection of training delivery within the EBT programme. GM1 to AMC2 ARA.FCL.205 Monitoring of examiners EBT PROGRAMME — INSPECTION OF TRAINING DELIVERY Where the inspection of training delivery is to be conducted, the inspector of the competent authority may meet the requirements laid down in AMC1 ARO.OPS.226(a). This inspection may be combined with the oversight required in ARO.OPS.226. AMC2 ARA.FCL.205 The purpose of this AMC is to clarify how a competent authority may conduct oversight of examiners where those examiners are revalidating licences as part of an operator's EBT programme. This is Page 220 of 221  European Union Aviation Safety Agency Appendix to Opinion No 08/2019 (A) 3. Proposed amendments to CS, AMC & GM and rationale in detail because the delivery of an EBT module is performed by instructors on behalf of the nominated person for crew training who maintains ultimate responsibility for the programme and who is an examiner. Member States provide a briefing within the Examiners Differences Document https://www.easa.europa.eu/sites/default/files/dfu/Examiner%20Differences%20Document_versio n_12-2019Q3.pdf for use by examiners with a Part-FCL examiner certificate conducting a proficiency check of a licence holder whose licence was issued by a competent authority (CA) other than their own. As an EBT practical assessment is equivalent to a proficiency check (see Appendix 10), then the procedures for the proficiency check for the purpose of the Examiner Differences Document should be followed. GM1 to AMC2 ARA.FCL.205 The vehicle to allow the licensing competent authority to inspect the training also had to be provided to be in line with existing oversight responsibilities. The principle described in this GM ‘Where this inspection of training delivery is to be conducted, the inspector of the competent authority may meet the requirements’ is transposed and adapted from the existing AMC1 ARA.FCL.205, to ensure that any oversight is preferably done by appropriately trained and qualified inspectors. Page 221 of 221 EUROPEAN COMMISSION Brussels, XXX […](2019) XXX draft ANNEX IIa TO EASA OPINION No 08/2019 COMMISSION IMPLEMENTING REGULATION (EU) …/... of XXX amending Commission Regulation (EU) No 1178/2011 as regards the requirements for flight crew competence and training methods, and as regards the reporting, analysis and follow-up of occurrences in civil aviation EN EN  COMMISSION IMPLEMENTING REGULATION (EU) …/... of XXX amending Commission Regulation (EU) No 1178/2011 as regards the requirements for flight crew competence and training methods, and as regards the reporting, analysis and follow-up of occurrences in civil aviation THE EUROPEAN COMMISSION, Having regard to the Treaty on the Functioning of the European Union, Having regard to Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (1), and in particular Articles 27 and 72 thereof, Whereas: (1) Operators and personnel that are involved in the operation of aircraft, as well as their competent authorities, shall comply with the relevant essential requirements for air operations set out in Regulation (EU) 2018/1139. (2) The principal objective of Regulation (EU) 2018/1139 is to establish and maintain a high uniform level of civil aviation safety in the Union (Article 1). (3) Commission Regulation (EU) No 1178/2011 (2) lays down the training, testing and checking requirements for pilot licensing. (4) The European Plan for Aviation Safety, pursuant to Article 6 of Regulation (EU) 2018/1139, has identified that it is of key importance for aviation personnel to have the right competencies and adapt training methods to cope with the new challenges of the aviation industry. (5) The International Civil Aviation Organization (ICAO) published in 2013 Doc 9995 AN/497 ‘Manual of evidence-based training’. (6) The objective of evidence-based training (EBT) is to improve safety and to enhance the capability of flight crews to operate the aircraft in all flight regimes by being able to recognise and manage unexpected situations. (7) The alignment of Commission Regulation (EU) No 1178/2011 (the Aircrew Regulation) with Regulation (EU) No 376/2014 (3) as regards the reporting, analysis ( 1) OJ L 212, 22.8.2018, p. 1. ( 2) Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 311, 25.11.2011, p. 1). ( 3) Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation, amending Regulation (EU) No 996/2010 of the European Parliament and of the Council and repealing Directive 2003/42/EC of the EN 1 EN  and follow-up of occurrences in civil aviation is expected to increase legal certainty, support EASA standardisation inspections in the area of Regulation (EU) No 376/2014, and support the implementation of effective occurrence-reporting systems as part of safety management. (8) Commission Regulation (EU) No 1178/2011 should therefore be amended to reflect the ICAO Doc 9995 ‘Manual of evidence-based training’ guidance with the aim of enhancing flight crew competence and improving training methods, and in order to increase legal certainty, support EASA standardisation inspections in the area of Regulation (EU) No 376/2014, and support the implementation of effective occurrence-reporting systems as part of safety management. (9) The European Union Aviation Safety Agency has prepared draft implementing rules and submitted them with Opinion No 08/2019 (4) in accordance with points (b) and (c) of Article 75(2) and with Article 76(1) of Regulation (EU) 2018/1139. (10) The measures provided for in this Regulation are in accordance with the opinion of the committee established in accordance with Article 127 of Regulation (EU) 2018/1139, HAS ADOPTED THIS REGULATION: Article 1 Amendments to Commission Regulation (EU) No 1178/2011 Commission Implementing Regulation (EU) No 1178/2011 is amended in accordance with the Annex to this Regulation. Article 2 Date of entry into force and application This Regulation shall enter into force on the twentieth day following that of its publication in the Official Journal of the European Union. This Regulation shall be binding in its entirety and directly applicable in all Member States. Done at Brussels, For the Commission The President […] European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) No 1330/2007 (OJ L 122, 24.4.2014, p. 18). ( 4) https://www.easa.europa.eu/document-library/opinions EN 2 EN EUROPEAN COMMISSION Brussels, XXX […](2019) XXX draft ANNEX Ia TO EASA OPINION No 08/2019 COMMISSION IMPLEMENTING REGULATION (EU) …/... of XXX amending Commission Regulation (EU) No 965/2012 as regards the requirements for flight crew competence and training methods EN EN  COMMISSION IMPLEMENTING REGULATION (EU) …/... of XXX amending Commission Regulation (EU) No 965/2012 as regards the requirements for flight crew competence and training methods THE EUROPEAN COMMISSION, Having regard to the Treaty on the Functioning of the European Union, Having regard to Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (1), and in particular Articles 23 and 27 thereof, Whereas: (1) Operators and personnel that are involved in the operation of aircraft, as well as their competent authorities, shall comply with the relevant essential requirements for air operations set out in Regulation (EU) 2018/1139. (2) The principal objective of Regulation (EU) 2018/1139 is to establish and maintain a high uniform level of civil aviation safety in the Union (Article 1). (3) Commission Regulation (EU) No 965/2012 (2) lays down the requirements for an aircraft operator as regards the training and checking requirements for its pilots. (4) The European Plan for Aviation Safety, pursuant to Article 6 of Regulation (EU) 2018/1139, has identified that it is of key importance for aviation personnel to have the right competencies and adapt training methods to cope with the new challenges of the aviation industry ahead. (5) The International Civil Aviation Organization (ICAO) published in 2013 Doc 9995 AN/497 ‘Manual of evidence-based training’. (6) The objective of evidence-based training (EBT) is to improve safety and to enhance the capability of flight crews to operate the aircraft in all flight regimes by being able to recognise and manage unexpected situations. (7) Commission Regulation (EU) No 965/2012 should therefore be amended to reflect the ICAO Doc 9995 ‘Manual of evidence-based training’ guidance with the aim of enhancing flight crew competence and improving training methods. (8) The European Union Aviation Safety Agency has prepared draft implementing rules and submitted them with Opinion No 08/2019 (3) in accordance with points (b) and (c) of Article 75(2) and with Article 76(1) of Regulation (EU) 2018/1139. ( 1) OJ L 212, 22.8.2019, p. 1. ( 2) Commission Regulation (EU) No 965/2012 of 5 October 2012 laying down technical requirements and administrative procedures related to air operations pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 296, 25.10.2012, p. 1). EN 1 EN  (9) The measures provided for in this Regulation are in accordance with the opinion of the committee established in accordance with Article 127 of Regulation (EU) 2018/1139, HAS ADOPTED THIS REGULATION: Article 1 Amendments to Commission Regulation (EU) No 965/2012 Commission Regulation (EU) No 965/2012 is amended in accordance with the Annex to this Regulation. Article 2 Date of entry into force and application This Regulation shall enter into force on the twentieth day following that of its publication in the Official Journal of the European Union. This Regulation shall be binding in its entirety and directly applicable in all Member States. Done at Brussels, For the Commission The President […] ( 3) https://www.easa.europa.eu/document-library/opinions EN 2 EN European Union Aviation Safety Agency Opinion No 08/2019 (A) Subpart FC ‘Flight Crew’ of Annex III (Part-ORO) to Regulation (EU) No 965/2012 Update of ORO.FC: evidence-based training (EBT) RELATED NPA/CRD 2018-07 — RMT.0599 EXECUTIVE SUMMARY This Opinion delivers the results of two rulemaking tasks (RMTs): RMT.0599 ‘Update of ORO.FC’ and RMT.0681 ‘Alignment of implementing rules and acceptable means of compliance/guidance material with Regulation (EU) No 376/2014’. The detailed structure of this Opinion is given in Section 1.1. The objective of this Opinion regarding RMT.0599 is to update the flight crew training requirements to improve pilot competencies. The proposed requirements provide additional efficiency in the field of flight crew training and achieve a smooth transition to competency-based training. The present EBT Opinion is part of a global safety initiative endorsed by ICAO whose objective is to determine the relevance of the existing pilot training according to aircraft generation. EBT intends to improve safety and to enhance the capability of flight crews to operate the aircraft in all flight regimes and to be able to recognise and manage unexpected situations. The EBT concept is designed to maximise learning and minimise formal checking. This Opinion is a second step in the European rulemaking actions to implement EBT. The first step was completed in 2015 with the publication of ED Decision 2015/027/R that provided guidance material to allow the implementation of a 'mixed EBT’ which maintains the current operator proficiency check (OPC) and licence proficiency check (LPC). This Opinion proposes further changes to the Air OPS and Air Crew Regulations to allow authorities to approve the baseline EBT, which replaces OPC and LPC. This will allow a single philosophy of recurrent training within the airline. Further work is foreseen in the context of the activities of RMT.0599 to allow expansion of EBT to the operator conversion course and initial type rating, as well as to other types of aircraft (e.g. helicopters and business jets). The impact assessment (IA) shows that the implementation of EBT by the operator on a voluntary basis is the preferred option in regulating recurrent training and checking of flight crew. The IA illustrates that the proposed rules contribute to significant improvement in safety by strengthening the competencies of flight crews while providing a cost-efficient and socially acceptable framework. Action area: Human factors and competence of personnel Affected rules: Part-DEF, Part-ARO and Part-ORO of the Air OPS Regulation, Part-FCL and Part-ARA of the Aircrew Regulation Affected stakeholders: Member States, pilots, instructors, examiners, approved training organisations and operators Driver: Safety Rulemaking group: Yes Impact assessment: Full Rulemaking Procedure: Standard 5.2.2016 27.7.2018 16.12.2019 2020/Q4 2020/Q4 TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 1 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) Table of contents Table of contents 1. About this Opinion ............................................................................................................... 3 1.1. How this Opinion was developed ..................................................................................................... 3 1.2. The next steps ................................................................................................................................... 5 2. In summary — why and what................................................................................................ 7 2.1. Why we need to change the rules — issue/rationale ....................................................................... 7 2.1.1. Why we need new rules on EBT in Europe .................................................................................... 7 2.1.2. Safety recommendations (SRs) — outcome of the EASA safety assessment ................................ 8 2.1.3. ICAO amendments ...................................................................................................................... 11 2.2. What we want to achieve — objectives ......................................................................................... 11 2.3. How we want to achieve it — overview of the proposals .............................................................. 12 2.4. What are the stakeholders’ views — outcome of the consultation ............................................... 13 2.5. What are the expected benefits and drawbacks of the proposals ................................................. 15 2.5.1. Expected benefits ....................................................................................................................... 15 2.5.2. Expected drawbacks ................................................................................................................... 16 2.5.3. Overall conclusion....................................................................................................................... 17 2.6. How we monitor and evaluate the rules ........................................................................................ 18 3. References ......................................................................................................................... 19 3.1. Affected regulations ....................................................................................................................... 19 3.2. Related decisions ............................................................................................................................ 19 3.3. Other reference documents ........................................................................................................... 19 4. Appendix ............................................................................................................................ 21 TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 2 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 1. About this Opinion 1. About this Opinion 1.1. How this Opinion was developed The European Union Aviation Safety Agency (EASA) developed this Opinion in line with Regulation (EU) 2018/11391 (the ‘Basic Regulation’) and the Rulemaking Procedure2. It contains the results of two RMTs into one consolidated document while separate explanatory notes elaborate on the background and provide detailed explanations for the different RMTs. The structure of this Opinion, including its annexes, is the following: — Opinion No 08/2019: — Opinion No 08/2019 (A) (Part A): Explanatory Note related to RMT.0599 ‘Update of ORO.FC — evidence-based training (EBT)’ — Opinion No 08/2019 (B) (Part B): Explanatory Note related to RMT.0681 ‘Alignment of implementing rules with Regulation (EU) No 376/2014’ — Annex Ia to Opinion No 08/2019: draft Cover Regulation amending Regulation (EU) No 965/2012 — Annex Ib to Opinion No 08/2019: draft Annex to draft Cover Regulation amending Regulation (EU) No 965/2012 — Annex IIa to Opinion No 08/2019: draft Cover Regulation amending Regulation (EU) No 1178/2011 — Annex IIb to Opinion No 08/2019: draft Annex to draft Cover Regulation amending Regulation (EU) No 1178/2011 For the purpose of this document, references to ‘this Opinion’ or ‘this rulemaking activity’ shall be understood as referring to the scope of Opinion No 08/2019 (A). The related rulemaking activity is included in the European Plan for Aviation Safety (EPAS) 2019-2023 under RMT.0599. The scope and timescales of the task were defined in the related ToR3. The draft text of this Opinion has been developed by EASA based on the input of Rulemaking Group (RMG) RMT.0599. This group is divided in the: 1. Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1) (https://eur- lex.europa.eu/legal-content/EN/TXT/?qid=1535612134845&uri=CELEX:32018R1139). 2. EASA is bound to follow a structured rulemaking process as required by Article 115(1) of Regulation (EU) 2018/1139. Such a process has been adopted by the EASA Management Board (MB) and is referred to as the ‘Rulemaking Procedure’. See MB Decision No 18-2015 of 15 December 2015 replacing Decision 01/2012 concerning the procedure to be applied by EASA for the issuing of opinions, certification specifications and guidance material (http://www.easa.europa.eu/the- agency/management-board/decisions/easa-mb-decision-18-2015-rulemaking-procedure). 3 https://www.easa.europa.eu/sites/default/files/dfu/ToR%20%26%20Concept%20Paper%20RMT.0599%20Issue%201.pdf TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 3 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 1. About this Opinion (a) Main Group4, which ensures consistency across the different tasks of RMT.0599. It also deals with other updates of ORO.FC; (b) Evidence-based training (EBT) subgroup5, that is responsible for developing the EBT concept; and (c) Helicopter subgroup6 that is developing and updating the helicopter training requirements including EBT. This Opinion is primarily based on the inputs provided by the EBT subgroup. Due to the novelty of the EBT concept, EASA also consulted the Main group RMT.0599 on a regular basis, organised a workshop7 with the participation of industry representatives in February 2017 and performed four rounds of focused consultation with: (1) the Netherlands Aerospace Centre (NLR)8 with regard to instructor concordance and grading; (2) the Spanish competent authority (AESA)9 and Iberia group10 for the implementation of the EBT programme; (3) the Italian competent authority (ENAC)11 and Alitalia12 with regard to equivalency of malfunctions; and (4) CAA Denmark (Trafik)13 and Thomas Cook Scandinavia14 on the oversight and follow-up of the EBT programme. All interested parties were consulted through NPA 2018-0715. 726 comments were received from interested parties, including industry, national aviation authorities (NAAs) and social partners. 4 Chaired by Yann Renier (IATA) and Phill Adrian (AIA). Members: Enrique Monzón (AESA España), Rogier Leeflang (IACA), Ståle Rosland (CAA Norway), David Lord (GAMMA). Project management Francisco Arenas Alvariño EASA. 5 Chaired by Phil Cullen (UK CAA). Secretariat Ascanio Russo EASA. 6 Chaired by Tim Rolfe (Heli-offshore). 7 1st Workshop on the Implementation of the Evidence-based Training 8 Focal point: Frederik Mohrmann. 9 Focal point: Carlos Artiles and Enrique Monzón. 10 Focal point: Captain Ignacio Gallego Alemany. 11 Focal point Mario Tortorici and Sandro Apolloni. 12 Focal point: Massimo Giavalisco and Fabio Polloni. 13 Focal point Lise-Lotte Olsen Deigaard 14 Focal point: Henrik Lyngse 15 In accordance with Article 115 of Regulation (EU) 2018/1139 and Articles 6(3) and 7 of the Rulemaking Procedure. TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 4 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 1. About this Opinion Percentage (%) of comments received from each of the interested parties Airlines ATOs 42% 44% NAAs Aircraft manufacturers Unions 3.00% 8% 3.00% EASA has addressed and responded to the comments received on the NPA. EASA reviewed the comments received during the public consultation and during the focused consultation with the support of Review Group (RG) RMT.0599. The comments received and EASA’s responses to them are presented in Comment-Response Documents (CRDs) 2018-07(A) and 2018-07(B)16, and they are also summarised under Section 2.4 below. The final text of this Opinion and the draft regulations have been developed by EASA based on the input of RMG RMT.0599 and the focused consultation. The draft rule text proposed by EASA is published on the EASA website17. 1.2. The next steps This Opinion contains the proposed amendments to Regulation (EU) No 965/201218 (the ‘Air OPS Regulation’) and to Regulation (EU) No 1178/201119 (the ‘Aircrew Regulation’) and their potential impacts. It is submitted to the European Commission, which will use it as a technical basis in order to prepare EU regulations. 16 http://easa.europa.eu/document-library/comment-response-documents 17 http://easa.europa.eu/document-library/opinions 18 Commission Regulation (EU) No 965/2012 of 5 October 2012 laying down technical requirements and administrative procedures related to air operations pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 296, 25.10.2012, p. 1) (http://eur-lex.europa.eu/legal- content/EN/TXT/PDF/?uri=CELEX:32012R0965&rid=1). 19 Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 311, 25.11.2011, p. 1) (https://eur-lex.europa.eu/legal- content/EN/TXT/?qid=1528301490110&uri=CELEX:32011R1178). TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 5 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 1. About this Opinion The decisions that contain the related certification specifications (CSs), acceptable means of compliance (AMC) and guidance material (GM) will be published by EASA when the related regulations are adopted by the European Commission. For information, the Appendix to this Opinion presents the rationale behind the changes at implementing rule level. Further to this, the Appendix provides the draft text for the related EASA decisions that contains certification specifications (CS), acceptable means of compliance (AMC) and guidance material (GM), as well as the rationale behind that text. Following the publication of the regulations, EASA foresees to support the implementation of the Regulation with the following actions: — Operator conversion course (OCC) and type rating training for commercial air transport (CAT). This activity will ensure a single philosophy of training in the operator. An NPA pertaining to this activity is scheduled to be published in the course of the third quarter of 2021. — EBT for helicopters and non-commercial complex motor-powered aircraft (NCC). This activity will ensure a single philosophy of training across the industry. This may also allow training data exchange across the industry. An NPA pertaining to this activity is scheduled to be published in the course of the third quarter of 2021. — A dedicated safety promotion task (SPT.012) is included in EPAS. The intent of this task is to facilitate EBT implementation. TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 6 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what 2. In summary — why and what A further analysis of the rationale and objectives addressed by this proposal is provided in Notice of proposed Amendment 2018-07(A) Update of ORO.FC — evidence-based training subtask in the Impact Assessment chapter. 2.1. Why we need to change the rules — issue/rationale The complexity of the aviation system is continuously increasing; also, new technologies are emerging rapidly on the aviation market. Therefore, it is of key importance for the aviation personnel to: (a) have the right competencies through the adaptation of training methods in order to cope with new challenges. This is one of the most significant systemic issues in the EPAS20, 2016-2020, 2017-2021, 2018-2022, and 2019-2023; and (b) take advantage of the safety-enhancing opportunities presented by new technologies. (EPAS 2018-2022, Section 5.2.2 ‘Human factors and competence of personnel’ and EPAS 2019-2023, Section 5.1.2 ‘Human factors and competence of personnel’. 2.1.1. Why we need new rules on EBT in Europe Aircraft design and reliability have improved steadily and significantly over time; yet, accidents still occur, even in cases when the aircraft and systems were operating without malfunction. It is impossible to foresee all plausible accident scenarios, especially in today’s aviation system where its complexity and high reliability mean that the next accident may be something completely unexpected. In addition to this, the wealth of accident and incident reports and the provision of flight data analysis offer the possibility to identify risks encountered in actual operations and therefore offer the industry with the opportunity to tailor training programmes in order to mitigate those risks that flight crew members face in operations. EBT addresses both elements (prepare the pilot for the unexpected and mitigate operational risks) by moving from task-based training to prioritising the development and assessment of key competencies, leading to a better training outcome. The scenarios recommended in EBT are simply a vehicle and a means to assess and develop competence. Mastering a finite number of competencies should allow a pilot to manage situations in flight that are unforeseen by the aviation industry and for which the pilot has not been specifically trained. (ICAO Doc 9995 AN/497 ‘Manual of Evidence-based Training’ First edition - 2013 – Chapter Background). Furthermore, this approach is also supported by scientific studies such as the ‘Manual Operation for 4th Generation Airliners’ financed by the European Commision. In its ‘Final Report Summary’, the following is stated: ‘5.2. Training Development A possible solution identified to improve flight training would be to train competencies instead of pre- described flight maneuvers, many of those being non-technical. Such competency transfer between scenarios could create more resilient flight crews that are more prepared to handle operational events 20 https://www.easa.europa.eu/easa-and-you/safety-management/european-plan-aviation-safety TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 7 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what instead of the scripted and sometimes predictable scenarios currently used in training. A way to do this would be the implementation of Evidence-Based Training (EBT) concepts. To support this training methodology, this deliverable presents a scenario development method to generate scenarios for competency-based training.’ 2.1.2. Safety recommendations (SRs) — outcome of the EASA safety assessment The following safety recommendations (SRs) addressed to EASA from aircraft accident investigation report(s) published by the designated safety investigation authority21, have been considered during this RMT. FRAN-2013-017 The French Accident Investigation Board recommends that EASA, in coordination with manufacturers, operators and major non-European aviation authorities ensure that go- around training integrates instruction explaining the methodology for monitoring primary flight parameters, in particular, pitch, thrust then speed. Evaluation of the This Opinion addresses the SR through the transposition of Appendices 2 to 6 to Doc SR 9995 where are all the following are required at a frequency of twice per year (frequency A): — the training topics: — monitoring, cross-checking, error management, mismanaged aircraft state; and — go-around management; and — the manoeuvres training on: — go-around, all engines operative; — go-around, all engines operative followed by a visual circuit, manually flown; and — go-around, all engines operative during flare/rejected landing. FRAN-2013-018 The French Accident Investigation Board recommends that EASA, in cooperation with the national civil aviation authorities and major non-European aviation authorities, ensure that during recurrent periodic training, training organizations and operators give greater importance to the assessment and maintenance of the monitoring capabilities of public transport pilots. Evaluation of the This Opinion addresses the SR through the transposition of Appendices 2 to 6 to Doc SR 9995 where the training topic: ‘Monitoring, cross-checking, error management, mismanaged aircraft state’ is required at a frequency of twice per year (Frequency A). FRAN-2013-022 The French Accident Investigation Board recommends that EASA review regulatory requirements for initial and periodic training in order to ensure that go-arounds with all engines operating are performed sufficiently frequently during training. 21 Regulation (EU) No 996/2010 of the European Parliament and of the Council of 20 October 2010 on the investigation and prevention of accidents and incidents in civil aviation and repealing Directive 94/56/EC (OJ L 295, 12.11.2010, p. 35) (http://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1479716039678&uri=CELEX:32010R0996). TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 8 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what Evaluation of the This Opinion addresses the SR through the transposition of Appendices 2 to 6 to Doc SR 9995 where are all the following are required at a frequency of twice per year (frequency A): — the training topic ‘Go-around management’; and — the manoeuvres training on: — go-around, all engines operative: high energy, initiation during the approach at 150 to 300 m (500 to 1 000 ft) below the missed approach level off altitude; — go-around, all engines operative followed by a visual circuit, manually flown; and — go-around, all engines operative: during flare/rejected landing. FRAN-2013-033 The French Accident Investigation Board recommends that EASA, in cooperation with the national civil aviation authorities and major non-European aviation authorities, ensure that the risks associated with dispersion and/or channelized attention during the go-around, to the detriment of the primary flight parameters, be taught to crews. Evaluation of the This Opinion addresses the SR through the transposition of Appendices 2 to 6 to Doc SR 9995 where are all the following are required at a frequency of twice per year (frequency A): — the training topics: — monitoring, cross-checking, error management, mismanaged aircraft state; and — go-around management; and — the manoeuvres training on: — go-around, all engines operative: high energy, initiation during the approach at 150 to 300 m (500 to 1 000 ft) below the missed approach level off altitude; — go-around, all engines operative followed by a visual circuit, manually flown; and — go-around, all engines operative: during flare/rejected landing. FRAN-2013-035 The French Accident Investigation Board recommends that EASA, in coordination with manufacturers, operators and major non-European aviation authorities, study whether to extend these measures to other procedures requiring high workload in a short time frame. Evaluation of the This Opinion addresses the SR through the transposition of Appendices 2 to 6 to Doc SR 9995 where training topic ‘Competencies non-technical (CRM)’ and 14 other example scenarios where the competency ‘workload management’ is trained, are required at a frequency of twice per year (Frequency A), (crew resource management (CRM) includes TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 9 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what communication, leadership and teamwork, problem-solving and decision-making, situation awareness, and workload management). FRAN-2014-005 The French Accident Investigation Board recommends that EASA, in coordination with national authorities, make changes to the training requirements for pilots so as to include periodic reminders on the effects of contaminants such as ice on stall and loss of control on take-off. Evaluation of the This Opinion addresses the SR through the transposition of Appendices 2 to 6 to Doc SR 9995 where training topic ‘adverse weather’ is addressed at a frequency of twice per year (Frequency A). Furthermore, for CAT, EASA is taking benefit of this recurrent training and checking scheme to mandate recurrent flight crew upset prevention and recovery training (UPRT) (see ED Decision 2015/012/R22, published on 4 May 2015). The related AMC1 ORO.FC.220&230 identifies icing and contamination effects as key components of the upset prevention training programme, and recurrent training now covers all upset aspects over a period not exceeding 3 years. In EBT, these provisions still apply. FRAN-2015-062 [unofficial translation]: EASA should define the terms on how an operator can set up a risk-based training as described in Doc 9995. [French] [original text] - L’AESA définisse les modalités permettant à un exploitant de mettre en oeuvre la formation basée sur les risques telle que précisée dans le doc OACI 9995 de l’OACI. [Recommandation 2015-062]. Evaluation of the This Opinion addresses the SR through the transposition of Doc 9995. SR Furthermore, ED Decision 2015/027/R23, published on 16 December 2015, enables the implementation of EBT according to the principles established in Doc 9995 taking into account the European Union regulatory framework. FRAN-2015-063 [unofficial translation]: EASA promotes CAT operators to consider issues related to CRM and wind shear in the EBT scenario. [French] - L’AESA incite les exploitants de transport aérien commercial à prendre en compte des problématiques relatives au CRM et au cisaillement de vent dans la conception des scénarii EBT. [Recommandation 2015-063]. Evaluation of the This Opinion addresses the SR through the transposition of Doc 9995. SR Furthermore, ED Decision 2015/027/R, published on 16 December 2015, contains new GM to support the implementation of EBT by operators, to be conducted in flight simulation training devices (FSTDs), according to the principles established in Doc 9995. The GM is associated with the existing points (a), b) and (f) of ORO.FC.230 ‘Recurrent training and checking’ and ORO.FC.A.245 ‘Alternative training and qualification programme’ (see Organisation Requirements for Operators - Flight Crew (ORO.FC) of the Air OPS Regulation. 22 https://www.easa.europa.eu/document-library/agency-decisions/ed-decision-2015012r 23 https://www.easa.europa.eu/document-library/agency-decisions/ed-decision-2015027r TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 10 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what CRM and wind shear are specifically addressed in the recurrent assessment and training matrices in Doc 9995, to which the new GM refers. 2.1.3. ICAO amendments Following the work initiated by the Flight Crew Licensing and Training Panel (FCLTP)24, in 2006 ICAO published Doc 9868 ‘Procedures for Air Navigation Services — Training (PANS-TRG)’ — a document that contains procedures for the development and implementation of a competency-based training programme to support the Annex 125 requirements. This was followed in 2013 by an amendment of the aforementioned document for the introduction of EBT, which was accompanied by Doc 9995. The intention was to provide guidance to civil aviation authorities (CAAs), operators and ATOs on the recurrent assessment and training of pilots referred to in ICAO Annex 6 ‘Operation of Aircraft’ and ICAO Annex 1 ‘Personnel Licensing’, 1.2.5 ‘Validity of licenses’. Finally, through Amendment 2 to Doc 9868 (also issued in 2013), procedures for EBT were introduced in order to provide a means of assessing and training key areas of flight crew performance in a recurrent training system. In addition, more detailed guidance on qualifications of the instructor was provided. 2.2. What we want to achieve — objectives The overall objectives of the EASA system are defined in Article 1 of the Basic Regulation. This proposal will contribute to the achievement of the overall objectives by addressing the issues outlined in Section 2.1. The objective of this Opinion is to update the flight crew training requirements in order to improve assessment and training of human factors; in particular, the personnel competence. At the same time, it provides additional efficiency in the field of flight crew training while achieving a smooth transition to competency-based training. The specific objectives of this proposal are to: (a) maintain the high aviation safety level by: (1) ensuring that the recurrent training and checking programmes are adequate to provide pilots with the necessary knowledge, skills and attitudes (KSA) to be competent in their job — under this objective, EASA proposes in this Opinion new provisions to implement EBT as a first step towards the full implementation of competency-based training across Subpart FC of Part-ORO; and (2) addressing the SRs outlined in Section 2.1.2 ‘Safety recommendations’; (b) remain in compliance with ICAO by ensuring that the European rules align with the latest amendments outlined in Section 2.1.3 ‘ICAO amendments’, especially with regard to the EBT; and (c) contribute to the production of efficient regulations by adapting the necessary training standards and rules to ensure that the level of safety can only be positively affected by: 24 Meeting held in Montreal, from 8 to 19 December 2003. 25 International Standards and Recommended Practices ICAO — Annex 1 to the Convention on International Civil Aviation — Personnel Licensing. TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 11 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what (1) introducing performance-based regulation principles; (2) ensuring consistency of training-related rules across the applicable parts of Annex III (Part-ORO) to the Air OPS Regulation and Annex I (Part-FCL) to the Aircrew Regulation; and (3) ensuring the correct balance between implementing rules (IRs) and CS, AMC & GM on the subject issue. 2.3. How we want to achieve it — overview of the proposals ICAO Doc 9995 contains a complete competency framework (‘core competencies’) with competency descriptions and related behavioural indicators, encompassing what was previously known as both technical and non-technical knowledge, skills and attitudes (KSA). This way, the training content is aligned with the actual competencies necessary to operate safely, effectively and efficiently in a CAT environment. Following this rationale, EASA decided to contribute to the development of regulations that ensure that pilot training and checking are adequate to provide a pilot with the necessary KSA to recognise and manage unexpected and unusual situations. Traditional approaches to training development involve the decomposition of jobs into tasks. For each task, there is a related objective, an assessment and associated elements in a training plan. A limitation of this approach is that each task must be taught and assessed. In complex systems or when jobs evolve rapidly, it may not be possible to teach and assess each task. Moreover, learners may demonstrate the ability to perform tasks in isolation without being competent in their job. Competency-based assessment and training on the other hand are based on the concept that competencies are transferable26. In the design of a competency-based assessment and training programme, a limited number of competencies are defined and used across a variety of activities and contexts. As new technologies emerge and the complexity of the aviation system increases, the existence of a competency framework is of key importance in order for pilots to be trained on a complete and relevant set of competencies. This competency framework should allow pilots to operate more safely, effectively and efficiently in a CAT environment. Furthermore, it should allow the training community to adapt their training methods in order to manage unexpected events through reactive analyses. In other words, mastering these competencies should allow the pilot to manage situations that are unforeseen by the industry and for which the pilot has not been specifically trained. 26 See study MAN4GEN https://cordis.europa.eu/project/rcn/104513/factsheet/en TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 12 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what 2.4. What are the stakeholders’ views — outcome of the consultation NPA 2018-07 received about 726 comments. The percentage (%) of comments received from each type of stakeholder was the following: Airlines ATOs 42% 44% NAAs Aircraft manufacturers Unions 3.00% 8% 3.00% The percentage of comments by topic is presented below. Topic 40% 36% 35% 30% 25% 19% 20% 12% 13% 15% 10% 7% 6% 3% 4% 5% 0% TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 13 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what The percentage of comments per type of stakeholder is presented below. Stakeholder Competent authorities (NAAs) Number of comments The topic of the comments Position of the stakeholder NAAs made about 8 % to 9 % The comments addressed Part-ORO - in favour of implementing EBT of all the comments on the (42 % of the comments), Part-FCL - no comments against the concept of NPA. (22 % of the comments) and Part- revalidation in accordance with FCL.740 and ARO (22 % of the comments). Appendix 10. However, one comment raises The main contributors were An additional 9 % of the comments concerns about the delegation of signature in the authorities of France, made by the Member States were the licence revalidation allowed in EBT United Kingdom, the Czech generic and in support of the - in favour of providing more criteria or Republic and the Netherlands. proposal. increasing the regulatory level in some Other contributors were Finally, 3 % of the comments made provisionS (e.g. moving GM to AMC or moving Finland, Sweden, Malta, and by the authorities addressed the AMC to IR Germany. definitions proposed in the NPA. - a few comments request to limit some of the flexibility allowed in the proposal (e.g. alternative competency frameworks, etc.) Airlines Number of comments The topic of the comments Position of the stakeholder 44 % of all the comments The majority of the comments were on - in favour of implementing EBT made on the NPA. Part-ORO. However, there were - demanding more prescription or definitions comments on Part-ARO (3 % of the in some requirements Except for one airline comments), on Part-DEF (3 %) - demanding less prescription and more small/medium-sized and on Part-FCL (13,5 %). flexibility in some other provisions business jet operator, only - the airlines have different positions, with major airlines commented some of them requesting more flexibility on the NPA. while others are asking for a more prescriptive approach to avoid competitive disadvantages. Approved training organisations (ATOs) Number of comments The topic of the comments Position of the stakeholder About 3 % of all the The majority of the comments were on - in favour of implementing EBT comments on the NPA. Part-ORO (about 85 % of all the - allow ATOs to train EBT under their comments they made). They also privileges instead of under the operator’s Only the two biggest ATOs made comments on Part-FCL (the privileges. The proposal allows ATOs to train and simulator providers in remaining 15 %). EBT under ORO.GEN.205 on contracting the world made comments activities. on the NPA. No comments received from medium-sized or small ATOs. Employee associations Number of comments The topic of the comments Position of the stakeholder This type of stakeholder made The majority of comments made by - in favour of implementing EBT about 42 % of all the employee associations is on - however, they have serious concerns in comments on the NPA. Part-ORO (67 % of the comments regard to the use of instructors, the use of made by the unions). examiners and data protection. The comments were They also made comments on coordinated between the Part-FCL (about 18 % of the European Cockpit Association comments made by the unions). and the German and French 8% of the comments made by this unions and/or federation. type of stakeholder were on Part-ARO. TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 14 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what 4 % of the comments were on the definitions. Original aircraft manufacturers (OAMs) Number of comments The topic of the comments Position of the stakeholder This type of stakeholder The majority of comments made by this - in favour of implementing EBT made about 3 % of all stakeholder were made on Part-ORO. - generally speaking, this type of stakeholder the comments on the The other comments are on Part-ARO requires more guidance on some of the NPA. (22 % of the comments made by this type provisions. of stakeholder) and on the definitions Only one manufacturer- (about 5 % of the comments made by this provided comments. type of stakeholder) 2.5. What are the expected benefits and drawbacks of the proposals 2.5.1. Expected benefits Studies27 show that the effective implementation of EBT should bring about a significant contribution to aviation safety by strengthening the competencies of flight crews and enabling them to handle abnormal and unexpected situations safely. It is expected that the safety benefit of EBT would be demonstrated over time by continually improving a system targeted at focused learning28. The implementation of the EBT programme would ensure a level of safety at least equivalent to that provided by compliance with the existing pilot training requirements of ORO.FC.230 of Annex III (Part-ORO) to the Air OPS Regulation and Appendix 9 to Annex I (Part-FCL) to the Aircrew Regulation. Safety benefits should be expected through a qualitative approach, using competencies to develop resilience by exposure to varying and challenging situations. The overall result would be better training of the pilots involved and a lower flight-crew-related accident rate in the future. The level of training of pilots and personnel dealing with pilot training within the air operator certificate (AOC) holders would be improved. The EBT concept is designed to maximise learning and minimise formal checking. Where checking is required, it should evolve towards measuring the process of managing situations rather than only the outcome of this process. This will lead to a substantial change towards more learning opportunities for pilots, by recognising the expectation that professionals should continuously strive to learn and develop their capabilities, rather than only being focused on demonstrating performance according to minimum regulatory standards. The pilots will be assessed and their licences will be revalidated based on evidence from clearly described EBT evaluation modules and development of competencies throughout the EBT programme. The data29 shows that the remedial training for flight crew who fail in the LPCs and OPCs is reduced by half (50 %) after the implementation of EBT. Therefore, the proposal is expected to have a positive social impact on the stakeholders (pilots and organisations). Based on the improved skills and competencies, EBT might also have also a potentially positive effect on the flight crew career development. There would be a positive social impact on the type rating instructors (TRIs) and type rating examiners (TREs) as well, because they will receive competency-based training to improve their knowledge and skills. The same applies for the competent authorities’ instructors who would improve their 27 Man4Gen Study www.man4gen.eu; IATA, Data Report for Evidence-based Training, 2013; experience of operators that have implemented EBT 28 IATA, Data Report for Evidence-based Training, 2013 29 Based on the feedback by operators who implemented full EBT worldwide, 2008-2015, EASA questionnaire 2016 TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 15 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what knowledge by following EBT training and/or participating in all phases of the implementation of EBT by the operator and by overseeing the training of TRIs/TREs. As regards the economic impacts, a full cost-benefit analysis was performed for two different cases: a medium-sized/large operator (e.g. 1 000 pilots) and a small operator (e.g. 100 pilots). The objective was to understand the difference in the economic benefits and costs for the operator, depending on its size. The implementation of EBT is expected to bring economic benefits as follows: — Line check: 2 years after EBT implementation, an operator may be allowed to extend the line check, i.e. a pilot's line check requirement is reduced from 1 per year to 1 every 2 years. The benefit is that the operator is saving the costs it pays annually for the line check of all flight crew. — Ground training: A pilot's safety equipment procedure (SEP) training requirement is reduced from 1 per year to 1 every 2 years. The benefit is saving the daily wage of the flight crew. In addition, less CRM training is expected due to the integration of non-technical competencies in the EBT programme (1 day per pilot/year to 1 day per pilot/3 years). — Saving due to the decrease in the percentage of pilots who fail in OPC/LPC: Saving in daily wages of the flight crew for the time that they do not fly. — Indirect saving (flexibility): A reduction in pilot workload is expected due to the flexibility to run simulator sessions outside the peak flying months. The benefit is assumed to be circa 1 % of the annual wage of a pilot saved, multiplied by the number of the pilots who would be available to fly instead of going to the simulator. The operator could reap these benefits on the basis of its performance in implementing EBT which would be overseen and granted by the competent authority. 2.5.2. Expected drawbacks Despite the overall positive social impact on the affected stakeholders, some drawbacks regarding the work of the TREs could be expected. The workload and the volume and scope of the work performed by the TREs would be reduced as the revalidation of the licences will not be based on a single simulator session; it will be based on the evidence obtained through the EBT system. The reduced workload might affect negatively the current role, position and the number of examiners. Although the amount of training in EBT remains unchanged, the role of the trainer will be now performed under the privileges of the TRI certificate, instead of the TRE certificate. In terms of cost, the cost-benefit analysis described the following types of costs related to preparation, adoption and implementation of EBT (one-off and recurrent): — Development of EBT competency framework and EBT programme (one-off); — Training of the operator’s training manager and instructors to deliver EBT (one-off); — Purchase of an IT assessment tool to support the implementation of EBT (one-off); — Costs for maintaining licences for the IT tool (recurrent); — Update of the EBT training programme (recurrent); and TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 16 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what — Refresher training of the EBT instructors (recurrent). As regards the competent authorities, the requirement for inspectors to be competent in the approval of and the oversight over EBT programmes would result in increased competent authority’s workload in the short term (ca 200 hours one-off costs for initial training, approval of operator’s training programme). They would be offset with normalisation of the workload in the consecutive years in EBT oversight (ca 50-70 hours recurrent costs per operator). In addition, the workload and the relative costs for the competent authority are expected to decrease with time, as there might be a higher take- up of the EBT programmes by AOC holders. As EBT implementation supports performance- and risk- based oversight, the overall impact on the competent authority is considered very low negative in the first years and neutral in the consecutive years. 2.5.3. Overall conclusion Table 1: Overview of economic impacts per type of operator AOC (A) EBT benefits EBT one-off EBT recurrent Net benefit Saving per Return of operator (annual)30 costs31 EUR costs (annual) (benefits- pilot/ year investment EUR EUR recurrent EUR costs) in EUR Medium-sized ca 0.9 M ca 1 M ca 0.2 M 0.7 M ca 700 ≥ 3 years /large (1 000 pilots) Small ca 0.1 M ca 0.2 M ca 0.02 M 0.08 M ca 80032 ≥ 4 years (100 pilots) The cost-benefit analysis demonstrates that the EBT implementation in recurrent training and checking of flight crew in a medium-sized/large operator is a cost-effective solution. The profitability indicators show a return of investment shortly after 3 years of EBT implementation, considering that the operator would receive full economic benefits based on its performance and the decision of the competent authority. Similarly, to the medium-sized/large operator, a small operator has the potential to reap net economic benefits from the EBT implementation. However, a small operator may encounter difficulties in deploying the EBT concept due to the need to make additional investments in data collection and analysis of existing threats, and identification of potential weaknesses in the operator’s operational safety. These costs are not quantified in the cost-benefit model due to lack of reliable data. Overall, small airlines could not have the available resources or expertise to develop EBT and hence it might be difficult to implement the EBT concept in the short term. 30 It is assumed that the operator could reap the economic benefits 2 years after full implementation of the EBT. 31 Assuming that the operator would make these investment costs in the course of the 2 preparatory years (when the operator would run its traditional training and would make the EBT investment one-off costs). 32 The amount is a bit higher than for the medium-sized/large operator because of the assumption that the small operator employs 100 pilots. TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 17 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 2. In summary — why and what Therefore, the regulatory impact assessment recommends the implementation of EBT for recurrent training and checking on a voluntary basis by AOC holders. For the full impact assessment of alternative options, please refer to Chapter 3 ‘Impact assessment’ in NPA 2018-07(A). 2.6. How we monitor and evaluate the rules It is recommended that the rules are subject to monitoring and, in case it is necessary, to an evaluation of their relevance, impact, effectiveness and efficiency. It is recommended that the following monitoring indicators are used to review the implementation of the new provisions. Table 2: Proposed indicative list of indicators to monitor the results of the rules Monitoring indicator Description and rationale Data source Indicative frequency of of the indicator data collection % of AOC(A) holders The EBT concept would Survey 2 years after rules are in which implemented EBT be implemented on a place in EASA MSs voluntary basis and it is recommended to monitor how many AOC(A) holders would implement it. No and trend in The current RMT EASA Annual Safety Annual occurrences for AOC(A) contributes to mitigating Review holders where training is safety-related issues, ECCAIRS database a key enabler which play a role in improving safety across all aviation domains. If an ex post evaluation is needed, it is recommended to be carried out indicatively 5 years after the rules are in place. Cologne, 16 December 2019 Patrick KY Executive Director TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 18 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 3. References 3. References 3.1. Affected regulations — Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 311, 25.11.2011, p. 1) — Commission Regulation (EU) No 965/2012 of 5 October 2012 laying down technical requirements and administrative procedures related to air operations pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 296, 25.10.2012, p. 1) 3.2. Related decisions — Decision N° 2012/015/Directorate R of the Executive Director of the Agency of 24th October 2012 on Acceptable Means of Compliance and Guidance Material to Commission Regulation (EU) No 965/2012 of 5 October 2012 laying down technical requirements and administrative procedures related to air operations pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council ‘Guidance Material to Annex I – Definitions’ — Decision 2014/025/R of the Executive Director of the Agency of 28 July 2014 adopting Acceptable Means of Compliance and Guidance Material to Part-ARO of Commission Regulation (EU) No 965/2012 and repealing Decision 2014/014/R of the Executive Director of the Agency of 24 April 2014 ‘AMC and GM to Part-ARO — Issue 3’ — Decision 2014/017/R of the Executive Director of the Agency of 24 April 2014 adopting Acceptable Means of Compliance and Guidance Material to Part-ORO of Commission Regulation (EU) No 965/2012 and repealing Decision 2012/017/R of the Executive Director of the Agency of 24 October 2012 ‘AMC and GM to Part-ORO — Issue 2’ — Decision N° 2012/006/Directorate R of the Executive Director of the Agency of 19th April 2012 on Acceptable Means of Compliance and Guidance Material to Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council ‘Acceptable Means of Compliance and Guidance Material to Part-ARA’ — Decision N° 2011/016/R of the Executive Director of the European Aviation Safety Agency of 15 December 2011 on Acceptable Means of Compliance and Guidance Material to Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council ‘Acceptable Means of Compliance and Guidance Material to Part-FCL’ 3.3. Other reference documents — Decision No 2015/027/R of the Executive Director of the European Aviation Safety Agency of 16 December 2015 on guidance material to Part-ORO of Regulation (EU) No 965/2012 on the TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 19 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 3. References implementation of evidence-based training (EBT) within the European regulatory framework (Mixed EBT) — ICAO Annex 1 to the Convention on International Civil Aviation ‘Personnel Licensing’, 11th Edition, July 2011 — ICAO Annex 6 to the Convention on International Civil Aviation ‘Operation of Aircraft’, 10th Edition, July 2016 — ICAO Doc 9868 ‘Procedures for air navigation services Training’ Second Edition, 2016 — ICAO Doc 9995 AN/497 ‘Manual of Evidence-based Training’ First edition - 2013 — ICAO Doc 10011 AN/506 ‘Manual on aeroplane upset prevention and recovery training’ First edition – 2014 — ICAO Doc 9841 AN/456 ‘Manual on the Approval of Training Organizations’ Second edition – 2012 — ICAO Doc 9379 AN/916 ‘Manual of Procedures for Establishment and Management of a State’s Personnel Licensing System’ TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 20 of 21 An agency of the European Union  European Union Aviation Safety Agency Opinion No 08/2019 (A) 4. Appendix 4. Appendix Appendix to Opinion No 08/2019 (A) ‘Rationale behind the proposed amendments to the implementing rules — Draft CS, AMC & GM as well as safety promotion actions that are associated with the implementing rules — Rationale behind the draft CS and AMC & GM’ TE.RPRO.00036-008 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 21 of 21 An agency of the European Union European Aviation Safety Agency Opinion No 08/2019 (B) Alignment of implementing rules with Regulation (EU) No 376/2014 Related NPA/CRD 2016-19 — RMT.0681 EXECUTIVE SUMMARY This Opinion delivers the results of two rulemaking tasks (RMTs): RMT.0599 ‘Update of ORO.FC’ and RMT.0681 ‘Alignment of implementing rules and acceptable means of compliance/guidance material with Regulation (EU) No 376/2014’. The detailed structure of this Opinion is given in Section 1.1. The objective of this Opinion regarding RMT.0681 is to update Commission Regulation (EU) No 1178/2011 (the Aircrew Regulation) in order to be aligned with Regulation (EU) No 376/2014 as regards the reporting, analysis and follow-up of occurrences in civil aviation. The changes proposed through RMT.0681 aim at mitigating the risks of overlaps and ambiguities that exist in the current regulatory framework due to the coexistence of reporting requirements in the delegated and implementing acts of Regulation (EU) 2018/1139 (the Basic Regulation) and in Regulation (EU) No 376/2014 and its delegated and implementing acts. The proposed changes are expected to increase legal certainty, support EASA standardisation inspections in the area of Regulation (EU) No 376/2014, and support the implementation of effective occurrence-reporting systems as part of safety management. Action area: Safety management Affected rules: Part-ARA and Part-ORA of Commission Regulation (EU) No 1178/2011 (the Aircrew Regulation) Affected stakeholders: Operators, pilots, approved training organisations (ATOs), competent authorities, and Member States. Driver: Safety Rulemaking group: No Impact assessment: None Rulemaking Procedure: Standard 30.9.2015 19.12.2016 16.12.2019 2020/Q4 2020/Q4 TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 1 of 8 An agency of the European Union  European Aviation Safety Agency Opinion No 08/2019 Table of contents Table of contents 1. About this Opinion ............................................................................................................... 3 1.1. How this Opinion was developed ..................................................................................................... 3 1.2. The next steps ................................................................................................................................... 4 2. In summary — why and what................................................................................................ 5 2.1. Why we need to change the rules — issue/rationale ....................................................................... 5 2.2. What we want to achieve — objectives ........................................................................................... 5 2.3. How we want to achieve it — overview of the proposals ................................................................ 5 2.4. What are the stakeholders’ views — outcome of the consultation ................................................. 6 2.5. What are the expected benefits and drawbacks of the proposals ................................................... 6 2.6. How we monitor and evaluate the rules .......................................................................................... 6 3. References ........................................................................................................................... 7 3.1. Affected regulations ......................................................................................................................... 7 3.2. Related decisions .............................................................................................................................. 7 3.3. Other reference documents ............................................................................................................. 7 4. Related documents ............................................................................................................... 8 TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 2 of 8 An agency of the European Union  European Aviation Safety Agency Opinion No 08/2019 1. About this Opinion 1. About this Opinion 1.1. How this Opinion was developed The European Union Aviation Safety Agency (EASA) developed this Opinion in line with Regulation (EU) 2018/11391 (the ‘Basic Regulation’) and the Rulemaking Procedure2. It contains the results of two RMTs into one consolidated document while separate explanatory notes elaborate on the background and provide detailed explanations for the different RMTs. The structure of this Opinion, including its annexes, is the following: — Opinion No 08/2019: — Opinion No 08/2019 (A) (Part A): Explanatory Note related to RMT.0599 ‘Update of ORO.FC — evidence-based training (EBT)’ — Opinion No 08/2019 (B) (Part B): Explanatory Note related to RMT.0681 ‘Alignment of implementing rules with Regulation (EU) No 376/2014’ — Annex Ia to Opinion No 08/2019: draft Cover Regulation amending Regulation (EU) No 965/2012 — Annex Ib to Opinion No 08/2019: draft Annex to draft Cover Regulation amending Regulation (EU) No 965/2012 — Annex IIa to Opinion No 08/2019: draft Cover Regulation amending Regulation (EU) No 1178/2011 — Annex IIb to Opinion No 08/2019: draft Annex to draft Cover Regulation amending Regulation (EU) No 1178/2011 For the purpose of this document, references to ‘this Opinion’ or ‘this rulemaking activity’ shall be understood as referring to the scope of Opinion No 08/2019 (B).The related rulemaking activity is included in the European Plan for Aviation Safety (EPAS) 2019-2023 under RMT.0681. The scope and timescales of the task were defined in the related Terms of Reference (ToR)3. The draft text of this Opinion has been developed by EASA. All interested parties were consulted through NPA 2016-194. 355 comments were received from interested parties, including industry, national aviation authorities (NAAs), and social partners of which a total of 30 comments were related to the proposed amendments to Part-ARA and Part-ORA. 1 Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1) (https://eur- lex.europa.eu/legal-content/EN/TXT/?qid=1536149403076&uri=CELEX:32018R1139). 2 EASA is bound to follow a structured rulemaking process as required by Article 115(1) of Regulation (EU) 2018/1139. Such a process has been adopted by the EASA Management Board (MB) and is referred to as the ‘Rulemaking Procedure’. See MB Decision No 18-2015 of 15 December 2015 replacing Decision 01/2012 concerning the procedure to be applied by EASA for the issuing of opinions, certification specifications and guidance material (http://www.easa.europa.eu/the- agency/management-board/decisions/easa-mb-decision-18-2015-rulemaking-procedure). 3 https://www.easa.europa.eu/document-library/terms-of-reference-and-group-compositions/tor-rmt0681 4 In accordance with Article 115 of Regulation (EU) 2018/1139, and Articles 6(3) and 7 of the Rulemaking Procedure. TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 3 of 8 An agency of the European Union  European Aviation Safety Agency Opinion No 08/2019 1. About this Opinion EASA has addressed and responded to the comments received on the NPA. The comments received and EASA’s responses to them were presented in Comment-Response Document (CRD) 2016-195. The final text of this Opinion and the draft regulation have been developed by EASA. The draft rule text proposed by EASA is published on the EASA website6. The major milestones of this rulemaking activity are presented on the title page. 1.2. The next steps This Opinion contains the proposed amendments to the Aircrew Regulation7. It is submitted to the European Commission, which will use it as a technical basis in order to prepare an EU regulation. For information, EASA published the draft text for the related EASA decision that contains acceptable means of compliance (AMC) and guidance material (GM). The final decision that amends the AMC & GM will be published by EASA when the related regulation has been adopted by the European Commission. 5 https://www.easa.europa.eu/document-library/comment-response-documents/crd-2016-19 6 http://easa.europa.eu/document-library/opinions 7 Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 311, 25.11.2011, p. 1). TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 4 of 8 An agency of the European Union  European Aviation Safety Agency Opinion No 08/2019 2. In summary — why and what 2. In summary — why and what 2.1. Why we need to change the rules — issue/rationale This proposal aims at mitigating the risks of overlaps and ambiguities that exist in the current regulatory framework due to the coexistence of reporting requirements in the delegated and implementing acts of the EASA Basic Regulation and in the Regulation (EU) No 376/20148 and its delegated and implementing acts. The proposed changes are expected to increase legal certainty, support EASA standardisation inspections in the area of Regulation (EU) No 376/2014, and support the implementation of effective occurrence-reporting systems as part of safety management. 2.2. What we want to achieve — objectives The overall objectives of the EASA system are defined in Article 1 of the Basic Regulation. This proposal will contribute to the achievement of the overall objectives by addressing the issues outlined in Section 2.1. The specific objective of this proposal is to update the Aircrew Regulation in order to be aligned with the Occurrence Reporting Regulation. 2.3. How we want to achieve it — overview of the proposals Regulation (EU) No 376/2014 contains specific obligations for EASA, Member States’ competent authorities, individuals and approved organisations. These exist in parallel with the reporting obligations of the Basic Regulation and its implementing rules (IRs). While all the IRs are overall deemed to be consistent with one another, in practice, there are overlaps and ambiguities in particular in relation to safety management/management system requirements, applicable to competent authorities and organisations respectively. These issues can be addressed by updating the requirements in the affected IRs in order to render them fully consistent with Regulation (EU) No 376/2014. The basic principle that reporting obligations can be discharged by using one single reporting channel is maintained. The proposed changes will provide clarity on the relevant organisation requirements related to the maintenance of mandatory and voluntary reporting systems, the analysis and follow-up of occurrences or groups of occurrences, the implementation of ‘just culture’ principles as well as to the protection of the source of information. These requirements must be subject to the organisation’s internal audit/compliance monitoring function, and to the competent authority’s oversight. Regulation (EU) No 376/2014 does not apply to organisations not having their principal place of business in an EU Member State, but staff of any EASA-approved organisations not having their principal place of business in an EU Member State are covered by Regulation (EU) No 376/2014. However, taking into account the objective of that Regulation, i.e. to ensure that relevant information is provided to the relevant competent authority in order for it to address potential safety issues, and considering that this type of occurrence reporting is also covered by the Basic Regulation and 8 Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation, amending Regulation (EU) No 996/2010 of the European Parliament and of the Council and repealing Directive 2003/42/EC of the European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) No 1330/2007 (OJ L 122, 24.4.2014, p. 18) (https://eur-lex.europa.eu/legal- content/EN/TXT/?qid=1572945972125&uri=CELEX:32014R0376). TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 5 of 8 An agency of the European Union  European Aviation Safety Agency Opinion No 08/2019 2. In summary — why and what applicable Bilateral Aviation Safety Agreements (BASAs), EASA already receives the requested information through channels established in accordance with those existing rules and agreements. This proposal is only applicable to the Aircrew domain: Annex VI (Part-ARA) and Annex VII (Part-ORA) to the Aircrew Regulation. 2.4. What are the stakeholders’ views — outcome of the consultation Please refer to CRD 2016-19. 2.5. What are the expected benefits and drawbacks of the proposals No IA has been conducted by EASA as this regulatory proposal transposes requirements of Regulation (EU) No 376/2014 that have been already applicable since 15 November 20159. It is understood that competent authorities and organisations have already adapted their policies, procedures and manuals to be aligned with said Regulation. 2.6. How we monitor and evaluate the rules Occurrence reporting will be monitored and evaluated through the continued exchange of information between EASA and the Member States during EASA Advisory Body meetings as well as through the regular EASA standardisation inspections. Cologne, 16 December 2019 Patrick KY Executive Director 9 The impact assessment performed by the European Commission for Regulation (EU) No 376/2014 is available under: https://eur-lex.europa.eu/resource.html?uri=cellar:9535bf18-ec24-48fe-96ee-4cb01156a525.0001.02/DOC_1&format=PDF. TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 6 of 8 An agency of the European Union  European Aviation Safety Agency Opinion No 08/2019 3. References 3. References 3.1. Affected regulations — Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (OJ L 311, 25.11.2011, p. 1) 3.2. Related decisions — Decision No 2012/006/Directorate R of the Executive Director of the European Aviation Safety Agency of 19 April 2012 on Acceptable Means of Compliance and Guidance Material to Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (‘Acceptable Means of Compliance and Guidance Material to Part-ARA’) — Decision No 2012/007/Directorate R of the Executive Director of the European Aviation Safety Agency of 19 April 2012 on Acceptable Means of Compliance and Guidance Material to Commission Regulation (EU) No 1178/2011 of 3 November 2011 laying down technical requirements and administrative procedures related to civil aviation aircrew pursuant to Regulation (EC) No 216/2008 of the European Parliament and of the Council (‘Acceptable Means of Compliance and Guidance Material to Part-ORA’) 3.3. Other reference documents — Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1) — Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation, amending Regulation (EU) No 996/2010 of the European Parliament and of the Council and repealing Directive 2003/42/EC of the European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) No 1330/2007 (OJ L 122, 24.4.2014, p. 18) TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 7 of 8 An agency of the European Union  European Aviation Safety Agency Opinion No 08/2019 4. Related documents 4. Related documents CRD 2016-19 ‘Alignment of EASA Basic Regulation (Regulation (EU) 2018/1139) with the specific obligations stemming from Regulation (EU) No 376/2014’10 10 https://www.easa.europa.eu/document-library/comment-response-documents/crd-2016-19 TE.RPRO.00036-008 © European Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 8 of 8 An agency of the European Union Draft AMC and GM to Part-ARA The text of the amendment is arranged to show deleted, new or amended text as shown below: — deleted text is marked with strike through; — new or amended text is highlighted in blue; — an ellipsis ‘(…)’ indicates that the rest of the text is unchanged. SUBPART GEN — GENERAL REQUIREMENTS SECTION I — GENERAL AMC1 ARA.GEN.125(b) Information to the Agency EXCHANGE OF SAFETY-SIGNIFICANT INFORMATION WITH THE AGENCY Each competent authority should appoint a coordinator to act as the contact point for the exchange of safety-significant information between the competent authority and the Agency. Draft AMC and GM to Part-ORA The text of the amendment is arranged to show deleted, new or amended text as shown below: — deleted text is marked with strike through; — new or amended text is highlighted in blue; — an ellipsis ‘(…)’ indicates that the rest of the text is unchanged. SUBPART GEN — GENERAL REQUIREMENTS SECTION I — GENERAL AMC1 ORA.GEN.160 Occurrence reporting GENERAL (a) The organisation should report all occurrences defined in AMC 20-8, and as required by the applicable national rules implementing Directive 2003/43/EC on occurrence reporting in civil aviation. (b) In addition to the reports required by AMC 20-8 and Directive 2003/43/EC, the organisation should report volcanic ash clouds encountered during flight. (a) Where the organisation holds one or more additional organisation certificates within the scope of Regulation (EU) 2018/1139 and its delegated and implementing acts: (1) the organisation may establish an integrated occurrence reporting system covering all certificates held; and (2) single reports for occurrences should only be provided if the following conditions are met: (i) the report includes all relevant information from the perspective of the different organisation certificates held; (ii) the report addresses all relevant specific mandatory data fields and clearly identifies all certificate holders for which the report is made; and (iii) the competent authority for all certificates is the same and such single reporting was agreed with that competent authority. (b) The organisation should assign responsibility to one or more suitably qualified persons with clearly defined authority, for coordinating action on occurrences and for initiating any necessary further investigation and follow-up activity. (c) If more than one person are assigned such responsibility, the organisation should identify a single person to act as the main focal point for ensuring a single reporting channel to the accountable manager is established. This should in particular apply to organisations holding one or more additional organisation certificates within the scope of Regulation (EU) 2018/1139 and its delegated and implementing acts where the occurrence reporting system is fully integrated with that required under the additional certificate(s) held. GM1 ORA.GEN.160 Occurrence reporting MANDATORY REPORTING — GENERAL For organisations having their principal place of business in a Member State, Regulation (EU) 2015/1018 lays down a list classifying occurrences in civil aviation to be mandatorily reported. This list should not be understood as being an exhaustive collection of all issues that may pose a significant risk to aviation safety. Therefore, reporting should not be limited to the items listed in that Regulation and the additional items referred to in point (c) of ORA.GEN.160. AMC1 ORA.GEN.160(c) Occurrence reporting REPORTING BETWEEN ORGANISATIONS (a) In addition to reporting occurrences to the competent authority, reporting between organisations should be considered. Such reporting will depend on the type of training activities, the organisation’s interfaces with other organisations, their respective safety policies and procedures, as well as the extent of contracting in accordance with ORA.GEN.205. (b) If it can be determined that the occurrence has an impact on, or is related to, an aircraft component covered by a separate design approval/authorisation (TC, STC, or ETSO), then the holders of such approval/authorisation should be informed. If an occurrence concerns a component which is covered by a TC, STC, or ETSO, then only that TC, STC, or ETSO authorisation holder needs to be informed. Any operator reporting to the design approval holder should actively support any investigations that may be initiated by that organisation. Support should be provided by a timely response to information requests and by making available affected components, parts or appliances for the purpose of the investigation, subject to an agreement with the respective component, part or appliance owners. (c) The operator should in addition consider reporting to the continuing airworthiness management organisation managing its aircraft or to the organisation maintaining its aircraft. (d) For air traffic, aerodrome occurrences or bird/wildlife strikes, the organisation should also notify the appropriate air navigation services (ANS) provider, aerodrome operator or groundhandling services provider. (e) To ensure effective reporting between organisations, it is important that: (1) an interface is established between the organisations to ensure that there is an effective and timely exchange of information relating to occurrences; (2) any relevant safety issue is identified; and (3) it is clearly established which party is responsible for taking further action, if so required. (f) Organisations should establish procedures to be used for reporting between organisations, which should include as a minimum: (1) a description of the applicable requirements for reporting;  (2) the scope of such reporting, considering the organisation’s interfaces with other organisations, including organisations contracted in accordance with ORA.GEN.205; (3) a description of the reporting mechanism, including reporting forms, means, and deadlines; (4) safeguards to ensure confidentiality of the reporter and protection of personal data; and (5) responsibilities of the organisations and personnel involved in reporting, including for reporting to the competent authority. Such procedures should be included in the organisation’s management system documentation. GM1 ORA.GEN.160(c) Occurrence reporting REPORTING BETWEEN ORGANISATIONS Organisations may develop a customised list of occurrences to be reported between organisations, adapted to their particular aircraft or operation and the organisations they interface with. Such a customised list of occurrences to be reported between organisations is usually included or referenced in the organisation’s management system documentation. Any such lists should however not be considered to be definitive or exhaustive, and the reporter’s judgement of the degree of risk or potential hazard involved is essential. GM2 ORA.GEN.160(c) Occurrence reporting DESIGN APPROVAL HOLDER Depending on the case, the ‘design approval holder’ will be the holder of a type certificate, a restricted type certificate, a supplemental type certificate, a European Technical Standard Order (ETSO) authorisation, a major repair design approval, a major change design approval or any other relevant approval or authorisation for products, parts and appliances deemed to have been issued under Commission Regulation (EU) No 748/2012. AMC1 ORA.GEN.160(e) Occurrence reporting ORGANISATIONS REPORTING TO THE AGENCY (a) Mandatory reports and where possible, voluntary reports should include the information below: (1) when: UTC date; (2) where: State/area of occurrence — location of occurrence;  (3) aircraft-related information: aircraft identification: State of Registry, make-model-series, aircraft category, propulsion type, mass group, aircraft serial number, aircraft registration, and call sign; (4) aircraft operation and history of flight: operator, type of operation, last departure point, planned destination, flight phase; (5) weather: weather relevant; (6) where relevant, ANS-related information: ATM contribution, service affected, ATS unit name; (7) where relevant: aerodrome-related information: location indicator (ICAO airport code), location on the aerodrome; and (8) aircraft damage or personal injury-related information: severity in terms of highest damage and injury level, number and type of injuries to persons on ground and on aircraft). (b) Where the organisation identifies an actual or potential aviation safety risk as a result of their analysis of occurrences or group of occurrences reported to the Agency it should: (1) transmit the following information to the Agency no later than 30 days after the date of notification of the occurrences to the Agency: (i) the preliminary results of the risk assessment performed; and (ii) any preliminary mitigation action to be taken; and (2) where required, transmit the final results of the risk analysis to the Agency as soon as they are available and, in principle, no later than 3 months after the date of initial notification of the occurrences to the Agency. (c) The below list provides examples of what needs to be mandatorily reported in addition to ORA.GEN.160 point (c). This list should not be understood as being an exhaustive collection of all issues that may pose a significant risk to aviation safety and therefore reporting should not be limited to items listed therein. 1. AIR OPERATIONS 1.1. Flight preparation (1) Use of incorrect data or erroneous entries into equipment used for navigation or performance calculations which has or could have endangered the aircraft, its occupants or any other person (2) Carriage or attempted carriage of dangerous goods in contravention of applicable legislation including incorrect labelling, packaging and handling of dangerous goods 1.2. Aircraft preparation (1) Incorrect fuel type or contaminated fuel (2) Missing, incorrect or inadequate de-icing/anti-icing treatment 1.3. Take-off and landing (1) Taxiway or runway excursion (2) Actual or potential taxiway or runway incursion (3) Final approach and take-off area (FATO) incursion (4) Any rejected take-off (5) Inability to achieve required or expected performance during take-off, go-around or landing (6) Actual or attempted take-off, approach or landing with incorrect configuration setting (7) Tail, blade/wingtip or nacelle strike during take-off or landing (8) Approach continued against air operator stabilised approach criteria (9) Continuation of an instrument approach below published minima with inadequate visual references (10) Precautionary or forced landing (11) Short and long landing (12) Hard landing 1.4. Any phase of flight (1) Loss of control (2) Aircraft upset, exceeding normal pitch attitude, bank angle or airspeed inappropriate for the conditions (3) Level bust (4) Activation of any flight envelope protection, including stall warning, stick shaker, stick pusher and automatic protections (5) Unintentional deviation from intended or assigned track of the lowest of twice the required navigation performance or 10 nautical miles (6) Exceedance of aircraft flight manual limitation (7) Operation with incorrect altimeter setting (8) Jet blast or rotor and prop wash occurrences which have or could have endangered the aircraft, its occupants or any other person (9) Misinterpretation of automation mode or of any flight deck information provided to the flight crew which has or could have endangered the aircraft, its occupants or any other person 1.5. Other types of occurrences (1) Unintentional release of cargo or other externally carried equipment (2) Loss of situational awareness (including environmental, mode and system awareness, spatial disorientation, and time horizon) (3) Any occurrence where the human performance has directly contributed to or could have contributed to an accident or a serious incident 2. TECHNICAL OCCURRENCES 2.1. Structure and systems (1) Loss of any part of the aircraft structure in flight (2) Loss of a system (3) Loss of redundancy of a system (4) Leakage of any fluid which resulted in a fire hazard or possible hazardous contamination of aircraft structure, systems or equipment, or which has or could have endangered the aircraft, its occupants or any other person (5) Fuel system malfunctions or defects, which had an effect on fuel supply and/or distribution (6) Malfunction or defect of any indication system when this results in misleading indications to the crew (7) Abnormal functioning of flight controls such as asymmetric or stuck/jammed flight controls (for example: lift (flaps/slats), drag (spoilers), attitude control (ailerons, elevators, rudder) devices) 2.2. Propulsion (including engines, propellers and rotor systems) and auxiliary power units (APUs) (1) Failure or significant malfunction of any part or controlling of a propeller, rotor or powerplant (2) Damage to or failure of main/tail rotor or transmission and/or equivalent systems (3) Flameout, in-flight shutdown of any engine or APU when required (for example: extended range twin engine aircraft operations (ETOPS), minimum equipment list (MEL)) (4) Engine operating limitation exceedance, including overspeed or inability to control the speed of any high-speed rotating component (for example: APU, air starter, air cycle machine, air turbine motor, propeller or rotor) (5) Failure or malfunction of any part of an engine, powerplant, APU or transmission resulting in any one or more of the following: (a) thrust-reversing system failing to operate as commanded; (b) inability to control power, thrust or revolutions per minute (rpm); (c) non-containment of components/debris. 3. INTERACTION WITH AIR NAVIGATION SERVICES (ANS) AND AIR TRAFFIC MANAGEMENT (ATM) (1) Unsafe (air traffic control (ATC) clearance (2) Prolonged loss of communication with air traffic services (ATS) or ATM unit (3) Conflicting instructions from different ATS units potentially leading to a loss of separation (4) Misinterpretation of radio-communication which has or could have endangered the aircraft, its occupants or any other person (5) Intentional deviation from ATC instruction which has or could have endangered the aircraft, its occupants or any other person 4. EMERGENCIES AND OTHER CRITICAL SITUATIONS (1) Any event leading to the declaration of an emergency (‘Mayday’ or ‘PAN call’) (2) Any burning, melting, smoke, fumes, arcing, overheating, fire or explosion (3) Contaminated air in the cockpit or in the passenger compartment which has or could have endangered the aircraft, its occupants or any other person (4) Failure of the flight or cabin crew to apply the correct non-normal or emergency procedure to deal with an emergency (5) Use of any emergency equipment or non-normal procedure affecting in-flight or landing performance (6) Failure of any emergency or rescue system or equipment which has or could have endangered the aircraft, its occupants or any other person (7) Uncontrollable cabin pressure (8) Critically low fuel quantity or fuel quantity at destination below required final reserve fuel (9) Any use of crew oxygen system by the crew (10) Incapacitation of any member of the flight or cabin crew that results in the reduction below the minimum certified crew complement (11) Crew fatigue affecting or potentially affecting their ability to perform safely their flight duties 5. EXTERNAL ENVIRONMENT AND METEOROLOGY (1) A collision or a near collision on the ground or in the air, with another aircraft, terrain or obstacle (2) Airborne collision avoidance system, resolution advisory (ACAS RA). (3) Activation of genuine ground collision system such as ground proximity warning system (GPWS)/ terrain awareness and warning system (TAWS) ‘warning’ (4) Wildlife strike including bird strike (5) Foreign object damage/debris (FOD) (6) Unexpected encounter of poor runway surface conditions (7) Wake-turbulence encounters (8) Interference with the aircraft by firearms, fireworks, flying kites, laser illumination, high-powered lights, lasers, unmanned aircraft systems (UASs), model aircraft or by similar means (9) A lightning strike which resulted in damage to the aircraft or loss or malfunction of any aircraft system (10) A hail encounter which resulted in damage to the aircraft or loss or malfunction of any aircraft system (11) Severe turbulence encounter or any encounter resulting in injury of occupants or deemed to require a ‘turbulence check’ of the aircraft (12) A significant wind shear or thunderstorm encounter which has or could have endangered the aircraft, its occupants or any other person (13) Icing encounter resulting in handling difficulties, damage to the aircraft or loss or malfunction of any aircraft system (14) Volcanic ash encounter 6. SECURITY (1) Bomb threat or hijack (2) Difficulty in controlling intoxicated, violent or unruly passengers (3) Discovery of a stowaway SECTION II — MANAGEMENT AMC1 ORA.GEN.200(a)(1);(2);(3);(5) Management system NON-COMPLEX ORGANISATIONS — GENERAL (a) Safety risk management may be performed using hazard checklists or similar risk management tools or processes, which are integrated into the activities of the organisation. (b) The organisation should manage safety risks related to a change. The management of change should be a documented process to identify external and internal change that may have an adverse effect on safety. It should make use of the organisation’s existing hazard identification, risk assessment and mitigation processes. (c) The organisation should identify a person who fulfils the role of safety manager and who is responsible for coordinating the safety -management-system related processes and tasks. This person may be the accountable manager or a person with an operational role in the organisation. (d) Within the organisation, responsibilities should be identified for hazard identification, risk assessment and mitigation. (e) The safety policy should include a commitment to : (1) improve towards the highest safety standards,; (2) comply with all applicable legal requirements,; (3) meet all applicable standards,; (4) consider best practices and provide appropriate resources.; and (5) apply ‘just culture’ principles and, in particular, not to make available or use the information on occurrences: (i) to attribute blame or liability to someone for reporting something that would not have been otherwise detected; or (ii) for any purpose other than the maintenance or improvement of aviation safety. (f) The organisation should, in cooperation with other stakeholders, develop, coordinate and maintain an emergency response plan (ERP) that ensures orderly and safe transition from normal to emergency operations and return to normal operations. The ERP should provide the actions to be taken by the organisation or specified individuals in an emergency and reflect the size, nature and complexity of the activities performed by the organisation. AMC1 ORA.GEN.200(a)(2) Management system COMPLEX ORGANISATIONS — SAFETY POLICY (a) The safety policy should: (1) be endorsed by the accountable manager; (2) reflect organisational commitments regarding safety and its proactive and systematic management; (3) be communicated, with visible endorsement, throughout the organisation; and (4) include safety-reporting principles. (b) The safety policy should include a commitment to: (1) to improve towards the highest safety standards; (2) to comply with all applicable legislation, meet all applicable standards and consider best practices; (3) to provide appropriate resources; (4) to enforce safety as one primary responsibility of all managers; and (5) not to blame someone for reporting something which would not have been otherwise detected apply ‘just culture’ principles and, in particular, not to make available or use the information on occurrences: (i) to attribute blame or liability to front line personnel or other persons for actions, omissions or decisions taken by them that are commensurate with their experience and training; or (ii) for any purpose other than the maintenance or improvement of aviation safety. (c) Senior management should: (1) continually promote the safety policy to all personnel and demonstrate their commitment to it; (2) provide necessary human and financial resources for its implementation; and (3) establish safety objectives and performance standards. GM1 ORA.GEN.200(a)(2) Management system SAFETY POLICY (a) The safety policy is the means whereby the organisation states its intention to maintain and, where practicable, improve safety levels in all its activities and to minimise its contribution to the risk of an aircraft accident or serious incident as far as is reasonably practicable. It reflects the management’s commitment to safety, and should reflect the organisation’s philosophy of safety management, as well as become the foundation on which the organisation’s management system is built. It serves as a reminder of ‘how we do business here’. The creation of a positive safety culture begins with the issuance of a clear, unequivocal direction. The safety policy should state that the purpose of safety reporting and internal investigations is to improve safety, not to apportion blame to individuals. (b) The commitment to apply ‘just culture’ principles forms the basis for the organisation’s internal rules describing how ‘just culture’ principles are guaranteed and implemented. (c) For organisations having their principal place of business in a Member State, Regulation (EU) No 376/2014 defines the ‘just culture’ principles to be applied (refer in particular to Article 16(11) thereof). ANNEX Ib to EASA Opinion No 08/2019 Draft Annex to draft Commission Implementing Regulation (EU) …./… amending Commission Regulation (EU) No 965/2012 as regards the requirements for flight crew competence and training methods Annex I (Definitions for terms used in Annexes II to VIII) to Commission Regulation (EU) No 965/2012 is amended as follows: (1) the following definitions are inserted in Annex I: ‘(23a)‘competency’ means a dimension of human performance that is used to reliably predict successful performance on the job. A competency is manifested and observed through behaviours that mobilise the relevant knowledge, skills and attitudes to carry out activities or tasks under specified conditions; (23b) ‘competency-based training’ means assessment and training programmes that are characterised by a performance orientation, emphasis on standards of performance and their measurement and the development of training to the specified performance standards; (23c) ‘competency framework’ means a complete set of identified competencies that are developed, trained and assessed in the operator’s evidence-based training (EBT) programme utilising scenarios that are relevant to operations. The chosen competency framework should be wide enough to prepare the pilot for both foreseen and unforeseen threats and errors; (42a) ‘EBT module’ means a combination of sessions in a qualified flight simulation training device as part of the 3-year period of recurrent assessment and training; (47a) ‘enrolment’ means the administrative action carried out by the operator where a pilot participates in the operator’s EBT programme; (47b) ‘enrolled flight crew member’/‘enrolled pilot’ means the pilot that participates in the EBT recurrent training programme; (47c) ‘equivalency of approaches’ (approach clustering) means all the approaches that place an additional demand on a proficient crew regardless of whether they are used or not in the EBT modules; (47d) ‘equivalency of malfunctions’ (malfunction clustering) means all the malfunctions that put a significant demand on a proficient crew regardless of whether they are used or not in the EBT modules; (47e) ‘evaluation phase (EVAL)’ means one of the phases of an EBT module. The evaluation phase is a line-orientated flight scenario, representative of the operator’s environment during which there are one or more occurrences to evaluate key elements of the defined competency framework. The root cause rather than the symptoms in any deficiency should be identified, in order to determine training needs; 1 (47f) ‘evidence-based training (EBT)’ means assessment and training based on operational data that is characterised by developing and assessing the overall capability of a pilot across a range of competencies (competency framework) rather than by measuring the performance in individual events or manoeuvres; (69a) ‘in-seat instruction (ISI)’ means a technique used in the manoeuvres training phase or the scenario-based training phase, where the instructors can: (a) provide simple instructions to one pilot; or (b) perform predetermined exercises acting, in a pilot seat, as pilot flying (PF) or pilot monitoring (PM) for: (1) the demonstration of techniques; and/or (2) triggering the other pilot to intervene or interact; (69b) ‘instructor concordance’ is the consistency or stability of scores between different EBT instructors. It gives a score (or scores) of how much homogeneity, or consensus, there is in the ratings given by instructors (raters); (76a) ‘manoeuvres training phase (MT)’ means one of the phases of an EBT module. During this phase, according to aircraft generation, crews have time to practise and improve performance in largely psychomotor skill-based exercises by achieving a prescribed flight path or performing a prescribed event to a prescribed outcome. These exercises or events should place a significant demand on a proficient pilot. Flight path control may be accomplished by a variety of means including manual aircraft control and the use of auto flight systems; (76b) ‘mixed EBT programme’ means that a portion of the operator’s recurrent training and checking programme (ORO.FC.230) is dedicated to the application of EBT. This programme includes the licence proficiency check (Part-FCL Appendix 9); (98a) ‘proficient’ means the demonstration of the necessary skills, knowledge and attitudes that are required to perform any defined tasks to the prescribed standard; (105b)‘scenario-based training phase (SBT)’ means one of the phases of an EBT module. This phase is designed to focus on the development of competencies, whilst the pilot is trained to mitigate the most critical risks identified for the aircraft generation. It should include the management of specific operator’s threats and errors in a real-time line-orientated environment; (107b) ‘special condition’ means a deviation from the certification specifications issued by the Agency, where the certified operator demonstrates that it is able to: (a) establish compliance with Regulation (EU) 2018/1139 and its delegated and implementing acts, and (b) achieve an equivalent level of safety to that achieved by the application of the CS issued by the Agency;’; 2 Annex II (Part-ARO) to Commission Regulation (EU) No 965/2012 is amended as follows: (2) the following point ARO.GEN.121 is inserted: ‘ARO.GEN.121 Certification specifications (a) The Agency shall develop certification specifications (CS) that are used to establish compliance with Regulation (EU) 2018/1139 and its delegated and implementing acts for evidence-based training. (b) Where an equivalent level of safety to that achieved by the application of the CS can be achieved by another specification, a special condition to the CS issued by the Agency may be used by the certified operator to establish compliance with Regulation (EU) 2018/1139 and its delegated and implementing acts. This special condition may be proposed by either the operator or the authority. (c) The competent authority shall establish a system to consistently evaluate if the special condition complies with Regulation (EU) 2018/1139 and its delegated and implementing acts. (d) The competent authority shall evaluate all special conditions proposed by an organisation in accordance with ORO.GEN.121(b) by analysing the documentation provided and conducting an inspection of the organisation. When the competent authority finds that the special condition is in compliance with Regulation (EU) 2018/1139 and its delegated and implementing acts, it shall notify the Agency without undue delay. The competent authority shall provide the Agency with a full description of the special condition, including copies of any revisions to procedures and documentation that may be relevant. It shall include an assessment demonstrating that the requirements of Regulation (EU) 2018/1139 and its delegated and implementing acts are met. (e) The Agency shall assess whether the conditions set out in point (b) above have been met and shall inform the competent authority after its assessment. Thereafter, the competent authority shall notify the applicant whether or not the special condition may be implemented and, if applicable, amend the approval, or certificate, of the applicant accordingly.’; (3) the following point ARO.OPS.226 is inserted: ‘ARO.OPS.226 Approval and oversight of evidence-based training programmes (a) Where a competent authority grants an approval for EBT programmes, inspectors must receive qualification and training in EBT principles, application, approval processes and continuing oversight. (b) The competent authority shall assess and oversee the EBT programme, together with the processes that support the implementation of the EBT programme and its effectiveness. 3 (c) Before approving an EBT programme, the competent authority shall: (1) ensure the resolution of level 1 findings in the areas that will support the application of the EBT programme; (2) assess the capability of the operator to support the implementation of the EBT programme. The following elements shall be considered as a minimum: (i) the maturity and capability of the operator’s management system; (ii) the operator’s EBT programme suitability; (iii) the adequacy of the operator’s record-keeping system, in particular with regard to flight crew training, checking and qualifications records; (iv) the suitability of the operator’s grading and assessment scheme; (v) the competence (including experience) of the relevant personnel, fundamentally of the instructors, in the use of the processes and procedures that support the implementation of the EBT programme; and (vi) the operator’s implementation plan and a safety risk assessment supporting the EBT programme in order to demonstrate how an equivalent level of safety to that of the current training programme can be achieved. (d) The competent authority shall amend or revoke the EBT programme approval if continuing compliance is not ensured.’; Annex III (Part-ORO) to Commission Regulation (EU) No 965/2012 is amended as follows: (4) the following ORO.GEN.121 is inserted: ‘ORO.GEN.121 Certification specifications (a) The certified operator shall apply the certification specifications (CS) adopted by the Agency to establish compliance with Regulation (EU) 2018/1139 and its delegated and implementing acts for: (1) evidence-based training. (b) Special conditions to the CS adopted by the Agency may be used by the certified operator to establish compliance with Regulation (EU) 2018/1139 and its delegated and implementing acts. (c) When a certified operator wishes to use a special condition, it shall, prior to implementing it, provide the competent authority with a full description of the special condition. The description shall include any revisions to manuals or procedures that may be relevant, as well as an assessment demonstrating that the requirements of Regulation (EU) 2018/1139 and its delegated and implementing acts are met and an equivalent level of safety to that achieved by the application of the CS is achieved. 4  The operator may implement this special condition to the CS subject to prior approval by the competent authority and upon receipt of the notification as prescribed in ARO.GEN.121.’; (5) point ORO.FC.145 is replaced by the following: ‘ORO.FC.145 Provision of training, checking and assessment (a) All the training, checking and assessment required in this Subpart shall be conducted in accordance with the training programmes and syllabi established by the operator in the operations manual; (b) When establishing the training programmes and syllabi, the operator shall include the relevant elements defined in the mandatory part of the operational suitability data established in accordance with Regulation (EU) No 748/2012. (c) In the case of CAT operations, training and checking programmes, including syllabi and use of individual flight simulation training devices (FSTDs), shall be approved by the competent authority. (d) The FSTD shall replicate the aircraft used by the operator, as far as practicable. Differences between the FSTD and the aircraft shall be described and addressed through a briefing or training, as appropriate. (e) The operator shall establish a system to adequately monitor changes to the FSTD and to ensure that those changes do not affect the adequacy of the training programmes.’ (6) the following ORO.FC.146 is inserted: ‘ORO.FC.146 Personnel providing training, checking and assessment (a) All the training, checking and assessment required in this Subpart shall be conducted by appropriately qualified personnel. (b) In the case of flight and flight simulation training and checking, the personnel that provides the training and conducts the checks shall be qualified in accordance with Annex I (Part-FCL) to Regulation (EU) No 1178/2011. (c) For an EBT programme, the personnel that performs assessment and provides training shall: (1) hold an Annex I (Part-FCL) instructor or examiner certificate; and (2) complete the operator’s EBT instructor standardisation programme. This shall include an initial standardisation programme and a recurrent standardisation programme. Completion of the operator’s EBT initial standardisation will qualify the instructor to perform EBT practical assessment. 5 (d) Notwithstanding point (b) above, the line evaluation of competence shall be conducted by a suitably qualified commander nominated by the operator that is standardised in EBT concepts and the assessment of competencies (line evaluator). (7) the following point ORO.FC.231 is inserted: ‘ORO.FC.231 Evidence-based training (a) EBT PROGRAMME (1) The operator may substitute the requirements of ORO.FC.230 by establishing, implementing and maintaining a suitable EBT programme approved by the competent authority. The operator shall demonstrate its capability to support the implementation of the EBT programme and perform a safety risk assessment demonstrating how an equivalent level of safety is achieved. The operator’s capability includes a minimum experience in mixed EBT. (2) The EBT programme shall: (i) correspond to the size of the operator, and the nature and complexity of its activities, taking into account the hazards and associated risks inherent in those activities; (ii) ensure pilot competence by assessing and developing pilot competencies required for a safe, effective and efficient operation of aircraft; (iii) ensure each pilot is exposed to the assessment and training topics described in ORO.FC.232; (iv) include at least six modules distributed across a 3-year programme; each module shall consist of an evaluation and a training. (3) The operator shall ensure that each pilot enrolled in the EBT programme completes: (i) a minimum of two modules within the validity period of the type rating, separated by a period of not less than 3 months. The module is completed when: (A) the content of the EBT programme is completed for that module (exposure of the pilot to the training topics); and (B) an acceptable level of performance in all observed competencies has been demonstrated; (ii) line evaluation(s) of competence; and (iii) ground training. (4) The operator shall establish an instructor standardisation and concordance assurance programme. 6  (i) All instructors must be subject to this programme. (ii) The operator shall use appropriate methods and metrics to assess concordance. (iii) The operator shall demonstrate sufficient instructor concordance. (5) The EBT programme shall include contingency procedures for unforeseen circumstances that may affect the delivery of the modules. These procedures may include a different separation period between modules. The operator shall demonstrate the need for these procedures. These procedures shall ensure that a pilot does not continue line operations if the performance observed was below the minimum acceptable level. (b) COMPETENCY FRAMEWORK The operator shall use a competency framework for all aspects of assessment and training within an EBT programme. The competency framework shall: (1) be comprehensive, accurate, and usable; (2) include observable behaviours required for safe, effective and efficient operations; and (3) include a defined set of competencies, their descriptions and their associated observable behaviours. (c) TRAINING SYSTEM PERFORMANCE (1) The EBT system performance shall be measured and evaluated through a feedback process in order to: (i) validate and refine the operator’s EBT programme; and (ii) ascertain that the operator’s EBT programme develops pilot competencies. (2) The feedback process shall be included in the operator’s management system. (3) The operator shall develop procedures governing the protection of EBT data. (d) GRADING SYSTEM (1) The operator shall use a grading system to assess the pilot competencies. The grading system shall ensure: (i) a sufficient level of detail to enable accurate and useful measurements of individual performance; (ii) a performance criterion and a scale for each competency, with a point on the scale which determines the minimum acceptable level to be achieved for the conduct of line operations. The operator shall develop procedures to address low performance of the pilot; (iii) data integrity; and 7  (iv) data security. (2) The operator shall verify at regular intervals the accuracy of the grading system against a criterion-referenced system. (e) SUITABLE TRAINING DEVICES AND VOLUME OF HOURS TO COMPLETE THE OPERATOR’S EBT PROGRAMME (1) Each EBT module shall be conducted in an FSTD with a qualification level adequate to ensure the correct delivery of the assessment and training topics. (2) The operator shall provide a sufficient volume of hours in the suitable training device for the pilot to complete the operator’s EBT programme. (f) EQUIVALENCY OF MALFUNCTIONS (1) Each pilot shall receive assessment and training in the management of aircraft system malfunctions. (2) Aircraft system malfunctions that place a significant demand on a proficient crew shall be organised by characteristics. (3) Each pilot shall be exposed to at least one malfunction for each characteristic at the frequency determined by the table of assessment and training topics. (4) Demonstrated proficiency in the management of one malfunction is considered equivalent to demonstrated proficiency in the management of other malfunctions with the same characteristics. (g) EQUIVALENCY OF APPROACHES RELEVANT TO OPERATIONS (1) The operator shall ensure that each pilot receives regular training in the conduct of approach types and approach methods relevant to operations. (2) This training shall include approaches that place an additional demand on a proficient crew. (3) This training shall include the approaches that require specific approval. (h) LINE EVALUATION OF COMPETENCE (1) Each pilot shall periodically undertake a line evaluation of competence in an aircraft to demonstrate the safe, effective and efficient conduct of normal line operations described in the operations manual. (2) The validity period of a line evaluation of competence shall be 12 months. (3) The operator approved for EBT may, with the approval of the competent authority, extend the validity of the line evaluation of competence to: (i) 2 years; or (ii) 3 years, subject to a feedback process for the monitoring of line operations. 8  (4) For successful completion of the line evaluation of competence, the pilot shall demonstrate that each competency is at or above the minimum acceptable level of performance. (i) GROUND TRAINING (1) Every 12 calendar months, each pilot shall undergo: (i) technical ground training; and (ii) assessment and training on the location and use of all emergency and safety equipment carried on the aircraft. (2) The operator may, with the approval of the competent authority, extend the period of assessment and training on the location and use of all emergency and safety equipment carried on the aircraft to 24 months.’; (8) the following point ORO.FC.232 is inserted: ‘ORO.FC.232 EBT programme assessment and training topics (a) The operator shall ensure that each pilot is exposed to the assessment and training topics. (b) The assessment and training topics shall be: (1) derived from a large-scale analysis of operational data using scientific principles; (2) distributed across a 3-year period at a defined frequency; and (3) relevant to the type or variant of aircraft on which the pilot operates. (c) Only aircraft types for which the assessment and training topics are developed shall apply EBT.’; 9