7. Solution Implementation (7 pgs) 7-1 Solution Implementation FAA HF Coordinator (FAA HFC) Responsibilities. In support of the implementation based on the Solution ConOps and requirements, the FAA HF Coordinator is responsible for: Conducting HF activities or overseeing their completion by a contractor (see the following pages) Assisting the Program Office in demonstrating compliance to the HF items in the ISR Checklist (FAA network user only) Additional responsibilities during Solution Implementation (SI) include: P. 7-2: General FAA HF Coordinator (FAA HFC) Responsibilities P. 7-3: Transfer of Documents from Previous Stages P. 7-4: Monitoring and Coordinating Contractor and Support Team Activities P. 7-5: Participating in Ongoing FAA, Contractor and Working Group Meetings P. 7-6: Approving Plans and Results of Contractor HF Empirical Evaluations P. 7-7: Observing HF Activities and Approving Contractor's HF Deliverables 7-2 Solution Implementation General FAA HF Coordinator (HFC) Responsibilities. In the previous sections, the role of the FAA HFC prior to the award of a contract for Solution Implementation (SI) has been described. The FAA HFC continues to play an important role during SI, tailored to match the size, cost, and complexity of the solution being acquired (HF-STD-004A). Broadly speaking, during SI, the FAA HFC can be assigned several responsibilities: Develop an effective working relationship with the HFC working for the contractor responsible for Solution Implementation (SI), establishing a supportive pattern of communication, coordination and collaboration with well-defined expectations (Note that the contractor is responsible for the actual conduct of HF activities to support SI) Review and approve the Contractor IHFP (Contractor Integrated Human Factors Plan) for SI and get signature approval from the FAA HF Division Director (ANG-C1) Refine and approve the FAA IHFP plan for SI and get signature approval from the FAA HF Division Director (ANG-C1) Fulfill the responsibilities of the FAA HFC as indicated on the following pages See the FAA Lifecycle Management Process Flowchart for a list of HF questions that need to be addressed during SI.7-3 Solution Implementation Team Formation (HFC) Responsibilities. The FAA HFC plays several roles in Solution Implementation (SI): Step 1. Adding the SI Contractor HFC to the Human Factors Working Group and revising the membership of that group as needed Step 2. Providing members for the SI Computer Human Integration (CHI) Team (user team) Transfer Documents from Previous Stages. Some of these insights can be provided through the HF requirements associated with the Statement of Work (SOW) for SI itself, while others can be provided to the contractor SI team based on documents in transfer packages, as well as any storyboards and/or prototype(s), produced during Research for Service Analysis and earlier AMS stages. Such mechanisms for the transfer of knowledge from the earlier stages to the SI CHI and SE teams are important because it is unlikely that that the requirements in the SOW for SI can fully communicate all of the insights gained during the earlier stages. In addition, such a transfer of knowledge can be further supported by transitioning a limited number of members from the CHI team set up during the Research for Service Analysis to remain as members of the CHI team for SI, thus providing continuity. To achieve this, when appropriate the FAA HFC can request one or two specific individuals who were on the CHI team supporting the Research for Service Analysis remain on the CHI team for SI, keeping in mind that the bargaining unit makes the choices, and also keeping in mind that there is value in also bringing in members who have not previously been associated with this particular SI in order to include “fresh eyes”. Step 3. Ensuring the transfer of HF insights developed during the stages prior to SI to the contractor HFC (and supporting HF practitioners) and Systems Engineering (SE) team as well as the SI CHI team 7-4 Solution Implementation Monitoring and Coordinating Contractor and Support Team Activities. The FAA HFC plays several roles in SI: Step 4. Monitoring the contractor HF activities Step 5. Coordinating with the Safety and Training Groups Step 6. Coordinating with FAA Program Management and Systems Engineering (SE) on the HF Coordinator role on the project with respect to the Responsible, Accountable, Support, Consult, Inform (RASCI) matrix Step 7. Coordinating briefings by the HF specialist(s) involved in Research for Service Analysis and earlier AMS stages to the Contractor HFC and SE team about lessons learned during these earlier stages regarding design recommendations for the Graphical User Interface (GUI)/CHI and the associated software functionality, workflow and procedures, including compatibility with the broader hardware and software environment. Note that this broader environment includes: Acoustic noise and vibration Space for personnel, their movement, and their job aids and equipment Safe and efficient equipment configurations, facility design, and working environments Protection from chemical, biological, toxicological, radiological, thermal, mechanical, electrical, and electromagnetic hazards Illumination commensurate with anticipated visual tasks Note that it is important for the FAA HFC to manage the timing and nature of such briefings. The goal is to convey lessons learned relative to the design alternatives that are under consideration by the SI contractor in order to help inform decisions by the SI regarding the CHI and supporting software/hardware functions and associated procedures. The goal is not to stifle the expertise and creativity of the SI as a source of “fresh eyes” regarding the generation and evaluations of potentially effective design solutions.  7-5 Solution Implementation Participating in Ongoing FAA Contractor and Working Group Meetings. The FAA HFC plays several roles in SI: Step 8. Participating in TIMs, working groups, and relevant engineering, development, and management meetings. Reviewing and Approving Heuristic Analyses. The FAA HFC also monitors analytical (judgement based) evaluation of HF design features. Step 9. Reviewing and approving the completion of an heuristic analysis based on the guidelines specified the FAA Human Factors Design Standard (HFS-STD-001B). Note that it is important that this heuristic analysis be conducted with the full range of relevant use cases and scenarios in mind. 7-6 Solution Implementation Approving Plans and Results of Contractor HF Empirical Evaluations. The FAA HF Coordinator (HFC) ensures that appropriate HF empirical evaluations are planned and completed. Step 10. Providing input and approval for the plans developed by the Contractor HFC for cognitive walkthroughs, Early User Interface Events (EUIEs) and other appropriate methods for testing and evaluation (FAA Testing), including the data collection instruments for the evaluation of features in the GUI/CHI and associated software functionality in the context of the proposed workflow and procedures, as well as compatibility of the GUI/CHI and associated software functionality with the broader hardware and software environment Scenario Selection. The FAA HFC reviews and approves the scenario selection and design for use in testing and evaluation. Since only a limited number of scenarios can be selected for evaluation in cognitive walkthroughs and EUIEs, selection should be based on consideration of the frequencies of the embedded subtasks in actual operations, the criticality of successful completion of these subtasks, and any uncertainties regarding the most effective CHI designs and associated functionalities. A further consideration is to ensure coverage of not only nominal scenarios but also the inclusion of important off-nominal scenarios. The elements of these scenarios can be guided by the results of the task analyses and associated Critical Task Analysis Reports (see p. B-7) provided in the transfer package as well as by the contents of the operator human engineering design approach documents (HEDADs) produced by the Vendor responsible for SI (described below). It is important to note, however, that the CHI design and supporting software and hardware functionality (including automation) as developed by the SI may introduce new critical tasks not considered during the Research for Service Analysis and earlier AMS stages and can thus identify additional scenarios that require consideration. Selection of Operators for Testing. The selection of the operators to include in cognitive walkthroughs and EUIEs also is very important. The participants need to be selected to ensure adequate coverage in order to get feedback on design decisions from the full range of prospective users. Three points merit emphasis: First, although an effort is typically made to include a range of different users to serve as Subject Matter Experts (SMEs) on the CHI team for a project, it is often the case that these SMEs are among the more experienced of the relevant user population. Such representation is important, as such experts are likely to have a greater breadth and depth of experience Second, the CHI team of necessity has a limited number of user representatives, potentially making it impossible to fully represent the full range of different user groups Third, since the CHI team supports development over an extended period of time, like the staff responsible for the design and implementation of the software, the members of the CHI team may become too familiar with the design and therefore may have difficulty seeing it with “fresh eyes” The transfer package from the Research for Service Analysis should provide insights into the range of users that should be considered in testing The conclusion regarding the selection of participants for testing is therefore twofold: First, the members of the CHI team are an important subset of the users that need to be included in evaluations It is important to use the results of the user analysis (drawing upon findings from the Research for Service Analysis and any additional data collected early in the SI process) to determine what additional participants in testing should be obtained from the user community in order to reflect the full set of differences across users and to guide decisions about participant selection. And tests with these additional participants should be separate from tests with the CHI team members in order to get independent assessments In addition, it is important to recognize that it is not just the immediate user of a tool who needs to be considered. As an example, for certain tools for ATC, it is not sufficient to assume that systems for ATC can be designed without an assessment of impacts resulting from the performance of pilots. Thus, for some ATC tools, it is important to get input from the broader set of people who are indirectly affected by the introduction of a new capability, not just the immediate users.7-7 Solution Implementation Observing HF Activities and Approving Contractor’s HF Deliverables. Step 11. Observing contractor HF-related activities, including EUIEs, design reviews by the CHI team and other testing events, providing the Contractor HFC with input based on these observations as well as using those insights to support judgments in approving SI deliverables Step 12. Reviewing and approving HF deliverables produced by the Contractor HFC Of particular importance in this set of deliverables are HF Contract Data Requirement Lists (CDRLs) regarding the use of the system by each of the relevant user groups, such as the human engineering design approach document for the operator (Operator HEDAD-O) and the maintainer human engineering design approach document (HEDAD-M). The goal of these documents is to provide a thorough picture of the operator tasks, equipment and operator interfaces with that equipment. More specifically, each of these documents provides an evaluation of whether the equipment having an interface with that type of operator meets with human engineering design criteria, and describes the characteristics, layout and interface of the equipment having an operator interface. This evaluation is based on an assessment of the proposed design, including software (CHI and underlying functionality) and hardware in the context of the proposed workflow and procedures as well as in the context of the broader physical and software environment. This evaluation is based on the requirements of FAA-HFS-STD-001B. Thus, the scope of the HEDAD for each type of operator is to document the following workstation, workstation interface and operator information: Workstation equipment and layout, controls and displays, requirements for operator vision, environmental factors, workstation lighting, workstation signals, communication systems, special design features, multiple operator station requirements Description of the guiding human design rationale Analysis of operator tasks Evaluation of the consistency of the design decisions with human engineering design criteria