2. Why Integrate Human Factors? (8 pgs) 2-1 Why Integrate Human Factors? There are many reasons why the integration of Human Factors (HF) is important in FAA acquisitions. The following pages in Chapter 2 summarize this range of considerations. P. 2-2: FAA policies requiring the integration of HF in acquisitions P. 2-3: Benefits of HF integration P. 2-4: Required confirmation of HF integration In Service Review Checklist (FAA network user only) P. 2-5: Industry experience with the value of early integration of HF in design P. 2-6: HF experiences with the incorporation of advanced technologies P. 2-7: HF perspectives in defense acquisition policies P. 2-8: Important HF questions to address during acquisitions 2-2 Why Integrate Human Factors? FAA Policies Requiring the Integration of HF in Acquisitions. FAA Acquisition Management Policy Section 4.7 Human Factors states: “Human factors are a critical aspect of aviation safety and effectiveness. Service organizations must assure that planning, analysis, development, implementation, and in-service activities for equipment, software, facilities, and services include human factors engineering to ensure performance requirements and objectives are consistent with human capabilities and limitations.” Federal Aviation Administration (FAA) Order 9550.8, Human Factors Policy states: “Human factors shall be systematically integrated into the planning and execution of the functions of all FAA elements and activities associated with system acquisitions and system operations. FAA endeavors shall emphasize human factors considerations to enhance system performance and capitalize upon the relative strengths of people and machines. These considerations shall be integrated at the earliest phases of FAA projects.” 2-3 Why Integrate Human Factors? HF Benefits. To help ensure that acquisitions will be useful, usable and actually used, it is important to integrate input from human factors specialists and users, starting with the initial stages when a shortfall is being defined and an Operational Needs Assessment (ONA) is being developed and continuing this HF input throughout the Acquisition Management System (AMS) process. The FAA Human Factors Acquisition Job Aid indicates that the funding necessary to conduct a comprehensive HF engineering program for a solution has been estimated to be between 0.5% and 6% of developmental costs (depending upon the sensitivity of the solution to HF issues). The benefit from conducting a comprehensive HF program has been estimated at between 20% to 30% of total acquisition costs. Thus, early integration of HF helps focus on performance considerations in a cost effective manner to ensure a useful, usable product that will be actually be used.2-4 Why Integrate Human Factors? Required Confirmation of HF in in Service Review (ISR) Checklist (FAA network user only). The FAA In Service Review (ISR) Checklist (FAA network user only) specifically indicates that, as part of the acquisition process, human factors requirements, plans, analyses, and verifications have been determined and documented in accordance with the: FAA Human Factors Acquisition Job Aid FAA Human Factors Design Standard (HF-STD-001B) Human Factors Engineering Requirements (HF-STD-004A) FED-STD-795, Uniform Federal Accessibility Standards (UFAS) Other appropriate standards and conventions The FAA document Safety Risk Management Guidance for System Acquisitions also merits reference. The FAA ISR Checklist (FAA network user only) specifies requirements for assessments that have been conducted to identify, document, and resolve/mitigate HF issues associated with considerations such as: Information requirements, functional design and automation functions/roles Displays and controls, computer-human and human-automation interaction Usability, collaboration and teamwork, communications and workload Workspace design, anthropometrics and other physical work demands Procedures, staffing and training 2-5 Why Integrate Human Factors? Industry Experience with the Value of Early Integration of HF in Design. In addition to FAA requirements that HF considerations should be addressed as early as practical to minimize technical, programmatic, and operational risk (AMS Policy 4.7), input from experienced program managers and human factors practitioners in industry has indicated the following: “If you wait to do it at the back end, it’s super expensive to fix HF problems. You need to catch problems while it’s easy to fix them.” “You can’t redo the project when you find the HF problems too late. They put on band-aids like a tip sheet with lots of things in fine print and recommend the real changes be delayed until Version 2 or Version 3. That creates resistance to the use of a product and increases its potential for failure.” “We had one major project where the engineering lead reached out to the HF professional for feedback too late and then decided to leave the recommended changes for future revisions. The project failed because there were too many usability challenges.” “Some project managers and development staff think that everyone knows human factors: ‘I’m human so I know what people need’. Of course, they don’t really have the understanding that can be provided by a HF expert. But they may just cherry pick to decide when to ask a HF expert for input, or they may not involve the HF expert at all. Cherry picking doesn’t work well.” 2-6 Why Integrate Human Factors? HF Experiences with the Incorporation of Advanced Technologies. This need to begin integration of HF early in the design process also applies to the application of advanced technologies such as artificial intelligence and machine learning. As noted by Lee and Seppelt (2023): “A study of 3000 organizations in 29 industries found that only 11% had seen a sizable return on their investment in artificial intelligence (AI). In contrast, 73% of those organizations that blended AI and human capabilities into a team that learns reported a sizable return on their investment.” 2-7 Why Integrate Human Factors? HF Perspectives in Defense Acquisition Policies. The importance of effective integration of HF in the development process is further supported by requirements for Defense acquisitions: “HSI [Human Systems Integration] implementation during capability acquisition will have a positive, and probably large, return on investment in terms of: Reduced probability of adverse safety and health outcomes Reduced probability of acquisition program failure Improved equipment effectiveness Reduced overall costs” (Australian Department of Defence) The U.S. Department of Defense requires a “comprehensive HSI program using an appropriate strategy to ensure HSI-related and human performance requirements are achieved. The program will consist of: Risk management, engineering, analysis, and human-centered design The human engineering design approach for the operator and maintainer Task analyses Analysis of human error Use of human modelling and simulation Usability and other user testing to support and inform human and machine interface analysis under operational conditions HSI risk management maintained through: (a) Design. (b) Development. (c) Testing. (d) Production. (e) Fielding. (f) Sustainment A training strategy for leaders, operators, maintainers, and support personnel” (DODI 5000.95 Human Systems Integration in Defense Acquisition) 2-8 Why Integrate Human Factors? Important HF Questions to Address During Acquisitions. The Human Factors AMS Lifecycle Checklist identifies: HF questions that need to be considered as part of the Acquisition Management System (AMS) process The AMS stages at which these questions should be addressed FAA documents within which the HF findings based on these questions should be incorporated A sample of these HF questions includes: Are human capabilities and limitations considered as part of the total solution? Has a functional analysis been conducted to determine information flow and processing required? Has a front-end analysis adequately identified the human performance issues for test planning? Does documentation identify the specific human factors tasks and activities that must be planned and executed to support timely design, development, and implementation? Do the human factors requirements meet the appropriate specifications or standards, including the Human Factors Design Standard (HF-STD-001B)? Have product functions been properly allocated between the hardware, software, and the human? Have the human factors implications of each alternative been determined for their impact on human-system performance, cost, benefits, schedule, interfaces, and technical risks, and then used to select alternatives?