This site provides information for FAA program and engineering managers, FAA human factors coordinators and user teams for managing the effective integration of human factors in the FAA acquisition process supporting Air Traffic Control, Air Traffic Flow Management, Air Traffic Facilities and Technical Operations. The figure below indicates the stages included in the FAA’s formal Acquisition Management System (AMS) process.

(To page, use buttons at bottom of screen)

The goal of this site is to provide guidance and information regarding responsibilities for the integration of Human Factors (HF) within the FAA acquisition process for:

• FAA program and systems engineering management
• FAA Human Factors Coordinators 
  

Members of user teams, as well as program management and HF specialists working for vendors contracted to conduct Research for Service Analysis or contracted to perform the work required of stages within the Acquisition Management System (AMS) process may find this information useful as well.

FAA Program and Systems Engineering Management. By reviewing sections of this site, FAA program and systems engineering management can gain insights for supporting HF in project planning, staffing and budgeting. They also can better understand the HF requirements that should be incorporated into the contracts for vendors performing the research and development associated with an acquisition.

FAA Human Factors Coordinators (FAA HFCs). The job of the FAA HFC is to monitor contractor HF performance and approve deliverables, as well as to directly provide HF expertise and input in order to assure effective integration of HF during the lifecycle of an acquisition. FAA HFCs can use this site to better understand their responsibilities and to access specific information (such as relevant FAA forms and guidance) in order to perform their jobs.

There are many reasons why the integration of Human Factors (HF) is important in FAA acquisitions. The following pages in Chapter 2 summarize this range of considerations.

• P. 2-2: FAA policies requiring the integration of HF in acquisitions
• P. 2-3: Benefits of HF integration
• P. 2-4: Required confirmation of HF integration In Service Review Checklist (FAA network user only)
• P. 2-5: Industry experience with the value of early integration of HF in design
• P. 2-6: HF experiences with the incorporation of advanced technologies
• P. 2-7: HF perspectives in defense acquisition policies
• P. 2-8: Important HF questions to address during acquisitions

FAA Policies Requiring the Integration of HF in Acquisitions.

FAA Acquisition Management Policy Section 4.7 Human Factors states:

“Human factors are a critical aspect of aviation safety and effectiveness. Service organizations must assure that planning, analysis, development, implementation, and in-service activities for equipment, software, facilities, and services include human factors engineering to ensure performance requirements and objectives are consistent with human capabilities and limitations.”

Federal Aviation Administration (FAA) Order 9550.8, Human Factors Policy states: 

“Human factors shall be systematically integrated into the planning and execution of the functions of all FAA elements and activities associated with system acquisitions and system operations. FAA endeavors shall emphasize human factors considerations to enhance system performance and capitalize upon the relative strengths of people and machines. These considerations shall be integrated at the earliest phases of FAA projects.” 

HF Benefits. To help ensure that acquisitions will be useful, usable and actually used, it is important to integrate input from human factors specialists and users, starting with the initial stages when a shortfall is being defined and an Operational Needs Assessment (ONA) is being developed and continuing this HF input throughout the Acquisition Management System (AMS) process.

The FAA Human Factors Acquisition Job Aid indicates that the funding necessary to conduct a comprehensive HF engineering program for a solution has been estimated to be between 0.5% and 6% of developmental costs (depending upon the sensitivity of the solution to HF issues). The benefit from conducting a comprehensive HF program has been estimated at between 20% to 30% of total acquisition costs. 

Thus, early integration of HF helps focus on performance considerations in a cost effective manner to ensure a useful, usable product that will be actually be used.

Required Confirmation of HF in in Service Review (ISR) Checklist (FAA network user only). The FAA In Service Review (ISR) Checklist (FAA network user only) specifically indicates that, as part of the acquisition process, human factors requirements, plans, analyses, and verifications have been determined and documented in accordance with the:

• FAA Human Factors Acquisition Job Aid
• FAA Human Factors Design Standard (HF-STD-001B)
• Human Factors Engineering Requirements (HF-STD-004A)
• FED-STD-795, Uniform Federal Accessibility Standards (UFAS)
• Other appropriate standards and conventions 
  

The FAA document Safety Risk Management Guidance for System Acquisitions also merits reference.

The FAA ISR Checklist (FAA network user only) specifies requirements for assessments that have been conducted to identify, document, and resolve/mitigate HF issues associated with considerations such as:

• Information requirements, functional design and automation functions/roles
• Displays and controls, computer-human and human-automation interaction
• Usability, collaboration and teamwork, communications and workload
• Workspace design, anthropometrics and other physical work demands
• Procedures, staffing and training

Industry Experience with the Value of Early Integration of HF in Design. In addition to FAA requirements that HF considerations should be addressed as early as practical to minimize technical, programmatic, and operational risk (AMS Policy 4.7), input from experienced program managers and human factors practitioners in industry has indicated the following:

• “If you wait to do it at the back end, it’s super expensive to fix HF problems. You need to catch problems while it’s easy to fix them.”
• “You can’t redo the project when you find the HF problems too late. They put on band-aids like a tip sheet with lots of things in fine print and recommend the real changes be delayed until Version 2 or Version 3. That creates resistance to the use of a product and increases its potential for failure.”
• “We had one major project where the engineering lead reached out to the HF professional for feedback too late and then decided to leave the recommended changes for future revisions. The project failed because there were too many usability challenges.”
  
• “Some project managers and development staff think that everyone knows human factors: 
  ‘I’m human so I know what people need’. Of course, they don’t really have the understanding that can be provided by a HF expert. But they may just cherry pick to decide when to ask a HF expert for input, or they may not involve the HF expert at all. Cherry picking doesn’t work well.”

   

HF Experiences with the Incorporation of Advanced Technologies. This need to begin integration of HF early in the design process also applies to the application of advanced technologies such as artificial intelligence and machine learning. As noted by Lee and Seppelt (2023):

“A study of 3000 organizations in 29 industries found that only 11% had seen a sizable return on their investment in artificial intelligence (AI). In contrast, 73% of those organizations that blended AI and human capabilities into a team that learns reported a sizable return on their investment.” 

HF Perspectives in Defense Acquisition Policies. The importance of effective integration of HF in the development process is further supported by requirements for Defense acquisitions:

Important HF Questions to Address During Acquisitions. The Human Factors AMS Lifecycle Checklist identifies:

• HF questions that need to be considered as part of the Acquisition Management System (AMS) process
• The AMS stages at which these questions should be addressed
• FAA documents within which the HF findings based on these questions should be incorporated

A sample of these HF questions includes:

• Are human capabilities and limitations considered as part of the total solution?
• Has a functional analysis been conducted to determine information flow and processing required?
• Has a front-end analysis adequately identified the human performance issues for test planning?
• Does documentation identify the specific human factors tasks and activities that must be planned and executed to support timely design, development, and implementation?
• Do the human factors requirements meet the appropriate specifications or standards, including the Human Factors Design Standard (HF-STD-001B)?
• Have product functions been properly allocated between the hardware, software, and the human?
• Have the human factors implications of each alternative been determined for their impact on human-system performance, cost, benefits, schedule, interfaces, and technical risks, and then used to select alternatives?
  

Section 3 deals with the integration of HF prior to approval to initiate the formal Acquisition Management System (AMS) process and covers the topics indicated below. You can page through Chapter 3 to review these topics.

• P. 3-2: Triggers for developing proposed acquisition 
• P. 3-3: HF guidance when proposing small acquisition projects 
• P. 3-4: HF integration when proposing large acquisition projects
  • P. 3-5: Completing Requirements of In Service Review (ISR) Checklist (FAA network user only)
  • P. 3-6: Sample Requirements in ISR Checklist (FAA network user only)
  • P. 3-7: Planning the HF Effort
  • P. 3-8: Specifying HF Activities
  • P. 3-9: Monitoring and Reviewing HF Activities and Deliverables
  • P. 3-10: Specifying Contents of the Transfer Package
  • P. 3-11: Sample Contents of a Transfer Package
    
• P. 3-12: Conclusion: Research for Service Analysis
  

Triggers for Developing Proposed Acquisition. There are a number of different sources of triggers that need to be documented in order to produce a request for approval by the Joint Research Council (JRC) to start the formal Acquisition Management System (AMS) process for a project or to define needs and preliminary requirements for a project to develop an enhancement within an existing program. See Operational Needs Assessment (ONA) for an example. Such sources include but are not limited to the following:

• Internal to FAA
  • Air Traffic Organization (ATO)
  • Technology Development & Prototyping Division (ANG-C5 (FAA network user only))
• External to FAA - through a transfer package or recommendation submitted to the Program Management Organization (AJM) and Mission Support (AJV)
  • Collaborative Decision Making (CDM) Program
  • MITRE
  • NASA
  • RTCA
  • SWIFT (System Wide Information Management (SWIM) Industry-FAA Team)

HF Guidance When Proposing Small Acquisition Projects. This can be provided by the FAA Human Factors Division (ANG-C1) or the FAA Planning and Analysis Team (AJM-131 (FAA network user only)). These offices can assist with identifying HF analyses relevant to defining a shortfall and in outlining possible solutions. And they can help ensure that these analyses are tailored to be consistent with the size, cost, and complexity of the solution being acquired (HF-STD-004A).

HF Integration When Proposing Large Acquisition Project. For larger projects, significant Research for Service Analysis needs to be conducted in order to prepare an Operational Needs Assessment (ONA) or to define needs and preliminary requirements for a project that is an enhancement of an existing program. Additional Research for Service Analysis may also be completed after a preliminary ONA has been approved to move forward by the ATO Directors Forum or in response to the submission of a transfer package submitted to AJM-S by an external organization such as NASA or MITRE. 

Preliminary Concept Exploration, Development and Evaluation. For larger projects that merit significant Research for Service Analysis, concept exploration, development and evaluation is necessary. HF involvement in this research includes:

• Specifying HF considerations relevant to a preliminary shortfall analysis
• Defining users and their associated operational needs and constraints
• Defining use cases, scenarios and critical tasks
• Identifying important elements of the broader system environment
• Generating and evaluating design alternatives
  

Inclusion of HF input during this initial stage is important because production pressures during Solution Implementation tend to limit the extent to which design alternatives can be adequately considered and evaluated. This initial stage prior to initiation of the formal Acquisition Management System (AMS) process provides an opportunity to more fully define user needs and shortfalls, as well as to consider the broader use environment and evaluate design alternatives. 

Completing Requirements of In Service Review (ISR) Checklist. Another example of important HF issues that are relevant for consideration during Research for Service Analysis is provided by the In Service Review (ISR) Checklist (FAA network user only). The ISR Checklist is an FAA management tool that is used to ensure that important criteria have been met prior to fielding of an acquisition. 

For large projects, early consideration of these specified HF criteria during Research for Service Analysis can help to ensure that critical HF issues are addressed, as production pressures during Solution Implementation may result in inadequate, cursory consideration of these factors.

Sample Requirements in ISR Checklist. There are a number of requirements in the ISR Checklist (FAA network user only) that can be addressed in the Research for Service Analysis. Examples are provided below.

Item # 6.1.1  Have human factors requirements, plans, analyses, and verifications been determined and documented for the product in accordance with the FAA Human Factors Acquisition Job Aid, FAA Human Factors Design Standard (HF-STD-001B), Human Factors Engineering Requirements (HF-STD-004A), FED-STD-795, Uniform Federal Accessibility Standards (UFAS), and other appropriate standards and conventions?

6.1.2  Have usability issues and human factors risks for all user (e.g., operators, maintainers, supervisors, administrators) populations been addressed for all intended sites?

Item # 6.1.3  Have the human-system performance studies and analyses (such as those related to workload, training, functional design, computer-human interface, staffing, safety and health, special skills and tools, work space, automation functions/roles, displays and controls, information requirements, display presentation, visual/aural alerts, input/output devices, communications, procedures, anthropometrics, documentation, and environment) been conducted to identify, document, and resolve/mitigate human factors issues?

Item # 6.1.6  Have steps been taken to identify, eliminate, reduce, or mitigate the sources of human error with the highest risks (combination of likelihood, severity, detection, and recovery)?

See additional HF criteria in Items 6.1.4, 6.1.5, 6.3.1, 8.4.2 and 8.6.10 of the ISR Checklist (FAA network user only).

Planning the HF Effort. For large projects, there are several steps that are important to ensure that the organization conducting the Research for Service Analysis integrates HF into its work.

• Step 1. Has the FAA Human Factors Division (ANG-C1) or the FAA Planning and Analysis Team (AJM-131) been consulted for guidance on how to effectively integrate HF into the Research for Service Analysis?
• Step 2. Has a qualified HF specialist been added as part of the FAA Systems Engineering Management Team managing the project?
• Step 3. Has this HF specialist on the FAA Systems Engineering Management Team defined the HF activities and deliverables that need to be included as requirements in the Statement of Work (SOW) for the organization conducting the Research for Service Analysis, tailored to be consistent with the size, cost, and complexity of the solution being acquired (HF-STD-004A)?
• Step 4. If the Research for Service Analysis is competitively bid, has the HF specialist been involved in evaluating the submitted bids for the HF components? Has the organization conducting the research included a qualified HF specialist on its team and included a user group to provide input?  Has this organization specified the appropriate HF activities and deliverables as part of its proposal?
  

Specifying Human Factors Activities. A large project should include a focus on workflow, usability, procedures, equipment, workload/staffing, human error and operational suitability. This can include both material (e.g., equipment) and non-material (e.g., procedures) needs, shortfalls and concepts and inputs needed by stages of the Acquisition Management System (AMS) process. Activities also should include consideration of the requirements for the ONA and ISR Checklist (FAA network user only). Keeping in mind the need to tailor the effort to reflect the nature and magnitude of the project (HF-STD-004A), possible activities include:

• Ethnographic studies (on site observation/interviews of prospective users)
• Input from operational users and other stakeholders using questionnaires, focus groups, structured interviews and use case reviews
• Task, functional and user analyses, including development of use cases, scenarios and Critical Task Analysis Reports (see page B-7), considering the broader hardware, software and distributed work context
• HF input to a preliminary CONOPS, with a separate CONOPS for the different user groups (including operations and maintenance) as needed
• Specification of the HF aspects of the ONA and associated set of preliminary functional requirements
• Design of storyboards, user interface mockups and functional prototypes
• Completion of formative evaluations of storyboards, user interface mockups and prototypes, including evaluations of alternative designs and design features using heuristic analyses, cognitive walkthroughs, usability studies and/or Human-In-The Loop studies (HITLs) to evaluate usefulness and usability 
  

Monitoring and Reviewing HF Activities and Deliverables. For large projects, there are several steps that are important to ensure that the organization conducting the Research for Service Analysis effectively integrates HF into its work.

• Step 1. Has the HF specialist on the FAA Systems Engineering Management Team reviewed the details of proposed activities by the organization conducting the research and observed the completion of these activities?
• Step 2. Has the HF specialist on the FAA Systems Engineering Management Team reviewed the Operational Needs Assessment (ONA) and associated set of preliminary requirements based on the Research for Service Analysis?
• Step 3. Has the HF specialist on the FAA Systems Engineering Management Team reviewed the HF deliverables and the contents of the transfer package prepared by the organization conducting the Research for Service Analysis?
  

The following pages provide examples of HF components of sample transfer packages. 

Specifying Contents of the Transfer Package. For large projects, the findings from the Research for Service Analysis can provide valuable insights for the stages in the formal AMS process that are not adequately communicated in an Operational Needs Assessment (ONA), Concept of Operations CONOPS or requirements. As one FAA Program Manager indicated:

“We found that the contractor [who was implementing the solution] was revisiting things we solved 2 years ago, so we decided to let the contractor see the prototype developed during concept development. We’d spent years doing the original concept exploration to develop the CONOPS and requirements. Why throw all that expertise away? There’s a lot that they have learned that isn’t captured in the CONOPS and requirements. Let’s give the contractor a head start. They will have new ideas too. I wish I had done this earlier. It would have saved a lot of money.”

The transfer package should include both documents and any storyboards or prototypes that have been developed during the Research for Service Analysis. The next page provides examples of HF content for a range of transfer packages.

Sample Contents of a Transfer Package. The table of contents of a sample transfer package developed as part of the Research for Service Analysis for an FAA acquisition is provided below. Note that many of the included artifacts have important HF content based on the results of the Research for Service Analysis. Note, however, that the contents of the transfer package should be tailored to be consistent with the size, cost, and complexity of the solution being acquired (HF-STD-004A).

• Tech Transfer Package Overview
• Discussion Outcomes (The information in the Discussions Outcomes section was gathered from operational users and other stakeholders using Questionnaires, Focus Groups, Use Case Reviews, Human Factors Working Group (HFWG) / Working Group (WG) sessions, and Data Administration (DA) Working Group sessions. It included 25-30 briefings based on early UI mockups and associated feedback.)
  • Discussion Outcomes Intro and ConOps Overview briefing
  • Discussion Outcomes briefing for each topic
  • CSS-Wx Products
  • CSS-Wx Products Overview briefing
  • Textual Wx Products spreadsheet
  • Graphical Wx Products spreadsheet
• Use Cases
  • Use Cases Overview briefing
  • Use Cases List and Details spreadsheet
  • Use Cases [series of DOCs] (60 focusing on Air Traffic users; 32 on Maintenance users and 1 on data administration)
• Preferred Architecture
  • Preferred Architecture briefing
• Technical Analysis
  • Technical Analysis Summary briefing
  • Supporting Items
• Enterprise Information Management (EIM) Package
  • EIM Overview briefing
  • Position and Information Management Utilizing EIM document
  • Static Data List spreadsheet
  • Supporting Items
• Computer Human Interaction (CHI) Guidelines
  • CHI Guidelines Overview briefing
  • CHI Guidelines document
• Concept of Operations (There were three very different user groups, so three CONOPS were developed. One for ATC, one to support national management for standardization across facilities while allowing some customization by facility, and one for enterprise-wide system management, which had both National and local components. They were described in separate CONOPS.)
  • Solution CONOPS document
  • Data Admin CONOPS document
  • Maintenance CONOPS document
  • Screening Information Request (SIR) to Contract Award Summary of CONOPS Changes briefing
  • Maintenance CONOPS Overview briefing
• Support Personnel Synopsis
• Summary of Requirements Changes (SIR to Contract Award)
  • SIR to Contract Award Summary of Requirements Changes briefing
    
• Tech Transfer Glossary
  

Research for Service Analysis: Conclusion

Triggers. Research for Service Analysis is triggered by identification of an operational need or opportunity to improve performance. In order to help ensure that an acquisition meeting such an operational need will be useful, usable and actually used, FAA Order 9550.8, Human Factors Policy requires that HF considerations be addressed in the earliest phases, including this initial stage.

HF Guidance. The FAA Human Factors Division (ANG-C1) or the FAA Planning and Analysis Team (AJM-131) can be consulted to assist with identifying the HF analyses for defining a shortfall and outlining an operational concept for a solution. This includes advice on how to tailor the effort to be consistent with the size, cost, and complexity of the solution being acquired (HF-STD-004A).

HF Management. HF guidance can be provided for small projects to produce an ONA, as well as for larger projects that require significant research. It also can help ensure that information is generated to support later stages in the formal AMS process and address requirements of the ISR Checklist (FAA network user only). For larger projects, the HF specialist appointed to be part of the FAA Systems Engineering Management Team can provide input on the HF requirements for inclusion in the SOW for the organization conducting the Research for Service Analysis, including HF activities and components of the transfer package, as well as providing overall management of the HF components of the research.

Service Analysis and Strategic Planning (SASP) is the first stage in the formal AMS process once a proposed acquisition investment has been approved for evaluation by the FAA Joint Resources Council. 

In this stage the HF aspects of operational needs, shortfalls and concept changes are analyzed. 

SASP Activities. HF tasks completed during SASP include:

• P. 4-2: HF Involvement
• P. 4-3: HF Aspects of Operational Needs
• P. 4-4: HF Aspects of Shortfalls (see ShortfallAnalysisReportGuide.doc (live.com))
• P. 4-5: HF Aspects of Concept Changes
  

The HF analyses may include consideration of factors such as workflow, usability, procedures, equipment, workload/staffing, human errors and operational suitability. It can include consideration of both the materiel (e.g., equipment) and non-material (e.g., procedures) aspects of needs, shortfalls, and operational concepts. Some of this content may be supported by the transfer package from prior Research for Service Analysis. 

A list of the relevant HF questions to be addressed during SASP and relevant documents are contained in the HF Acquisition Management System (AMS) Lifecycle Checklist.

HF Involvement. SASP indicates: “The service organization or program office, facilitated by the Human Factors Division (ANG-C1), should address HF as early as practical to minimize technical, programmatic, and operational risk. In order to assess the appropriate level of HF involvement, ANG-C1 can assist coordination with agency HF resources such as the HF Acquisition Working Group to identify HF specialists that might provide direct support or other resources to a program. Ideally, HF specialists are involved prior to the CRD (Concept and Requirements Definition) phase and throughout the AMS lifecycle to help gather data about the service environment and participate in the preliminary shortfall analysis”.

HF input to documentation during and subsequent to SASP provides essential HF information upon which to build good requirements, supporting the preparation of cost, benefit, and risk analyses and developing plans, specifications, and a statement of work. This includes consideration of how:

• Operational concept, system architecture, procedures and human-system interface design impact user performance (efficiency and safety)
• Human-systems considerations impact human resources or performance outside the boundary of the product being acquired 
  

HF Aspects of Operational Needs. The FAA document “Guidelines for Service Analysis & Strategic Planning (SASP) and Concept & Requirements Definition (CRD)” indicates that, as part of SASP (Service Analysis and Strategic Planning): 

• “Using a service-level approach, issues within the operational, business, data and information, technology, organizational, process, and people (including human factors) areas that might affect delivery of targeted business outcomes are identified. The needs identified in the FAA operational environment usually represent service shortfalls associated with Enterprise information management and agency goals and objectives.”
• “In addition, it will be necessary to obtain information on new technologies and methodologies that might change the way services are provided in the future.”

HF Aspects of Shortfalls. There are many types of shortfalls that should be considered, such as missing functionality, bad business process, data coming from a source that is not a trusted source (authoritative or approved replicated sources), data availability or data accuracy, and security shortfalls. To characterize shortfalls, relevant human performance deficiencies and critical human performance requirements need to be defined. Preliminary HF inputs could include:

• Reviewing the results of operational analyses of fielded systems, services, facilities and other assets, as well as published research, technology transfers, and related documents from external organizations such as Eurocontrol, MIT Lincoln Labs, MITRE and NASA
• Reviewing the failure reports for operational systems and equipment
• Reviewing failure and safety issues reports (providing indications of performance deficiencies in statements such as "user failed to detect or identify")
• Soliciting feedback from users and customers of relevant FAA services
• Identifying and interviewing Subject Matter Experts (SMEs) regarding human performance issues surrounding the legacy system
• Qualitatively characterizing shortfalls and documenting where human performance contributes to shortfalls describing human performance limitations that are relevant to the system design (examples include working memory, task completion times for data entry, mental workload, vigilance fatigue, visually tracking multiple simultaneous targets, etc.)

Capturing human performance issues in a shortfall analysis lays the foundation for traceability to preliminary program requirements as well as measures of performance, effectiveness and suitability in the requirements definition process. In an air traffic control example, this could be errors detecting/acquiring/controlling aircraft entering a controller's sector.

HF Aspects of Concept Changes. This effort focuses on the specification of changes in the operational concept that encompass an explicit set of use cases and scenarios. This requires participation in meetings discussing use cases, reviewing use case outlines and developing scenarios, and documenting potential human performance risk areas that need to be evaluated during concept demonstration/validation exercises. 

The development of use cases and scenarios also requires completing a task analysis and function allocation analysis, defining the assignment of the roles and responsibilities of key participants (e.g., controllers, traffic managers and maintenance technicians). And it involves identification of the functional and information requirements necessary to support human performance.

The task and function analyses help to ensure that necessary task sequences are demonstrated from end-to-end in the scenarios and that all necessary functionality has been identified for review and analysis as part of concept demonstration and evaluation based on heuristic analyses, prototyping and Human-In-The-Loop (HITL) studies. Note that this effort may draw upon results from preceding Research for Service Analysis.

The Concept & Requirements Definition (CRD) Stage of the Acquisition Management System (AMS) process begins with:

Contacting ANG-C1 for guidance and determining whether it is appropriate to appoint a Preliminary FAA HF Coordinator (FAA HFC)

The Preliminary FAA HFC or the supporting HF participant then either completes the following activities or, for a larger projects, manages their completion by a vendor supporting CRD:

• Conduct or oversee HF activities:
  • P. 5-2: Conduct HF assessments to support refinement of operational shortfalls
    
  • P. 5-3. Conduct HF assessments to support refinement of concepts
    
  • P. 5-4. Conduct HF assessments to support development of solutions and requirements
• The Preliminary FAA HFC or supporting HF specialist also assists the Program Office in demonstrating compliance to the HF items in the In-Service Review (ISR) Checklist (FAA network user only)
  

Additional useful information is provided on the pages listed below.

• P. 5-5. HF Input to Preliminary Program Requirements Document
• P. 5-6. Preliminary FAA Integrated HF Program (FAA IHFP)

See crdguidelines.pdf (faa.gov) for additional details.

For relevant HF questions and associated documents to address during CRD, see HF AMS Lifecycle Checklist.

Conduct HF Assessments to Support Refinement of Operational Shortfalls. During the CRD phase, HF updates are provided to finalize the shortfall analysis (a qualitative assessment of any human performance gaps where they exist and quantification of the human performance shortfalls by calculating performance impacts and monetizing the shortfalls). 

The HF inputs to update the CRD shortfall analysis include:

• Identification of Existing Human Performance Deficiencies. As an Air Traffic Control example, these could be errors in detecting/acquiring/identifying aircraft entering a controller’s sector, errors in tracking multiple, simultaneous radar targets, or excessive time required in the performance of a given task 
• Identification of Human Performance Improvement Opportunities. Opportunities for improvements in human performance need to be determined. As an example, such opportunities to improve human performance can take the form of new technologies (such as DATACOM) that provide new or more robust information or involve changing the method of information delivery so that transmission and clarity is assured
• Articulation of Assumptions About the Future Environment that Contribute to a Performance Shortfall. Conditions that will lead to a future shortfall or permit a known shortfall to persist need to be specified to help justify the proposed acquisition as an improvement. This needs to consider the broader context in which new tools, workflows and procedures will be used.
  

Conduct HF Assessments to Support Refinement of Concepts. As indicated in crdguidelines.pdf (faa.gov): “The alternatives developed during the CRD phase will be high-level concepts, and thus referred to as preliminary alternative descriptions.”

These alternatives will developed into a Solution ConOps that:

• “Describes how users will employ the new capability and how it will achieve desired objectives for the proposed service need”
• “Defines the roles and responsibilities of key participants (e.g., controllers, maintenance technicians, pilots)”
• “Explains operational issues that system engineers must understand when developing requirements”
• “Identifies procedural issues that may lead to operational change”
• “Incorporates and reflects key enterprise data and information needs”
• “Establishes a basis for identifying alternative solutions and estimating their likely costs and benefits” (crdguidelines.pdf (faa.gov)).
  

The guidelines emphasize the importance of “human factors engineering to ensure performance requirements and objectives are consistent with human capabilities and limitations”. As an example, “HF implications can include the appropriateness of automation or procedures from a HF perspective, in the context of other systems and tasks. Such analysis can be performed well before establishing details such as computer human interface (CHI) requirements later in the AMS lifecycle” (crdguidelines.pdf (faa.gov)).

HF Assessments to Support Development of Solutions and Requirements. The analysis of solutions and requirements needs HF input regarding task allocations, along with completion of a function analysis focused on the performance of the operator and on interactions of the operator with supporting automation to ensure:

• Definition of the roles and responsibilities of key participants (e.g., controllers, traffic managers, maintenance technicians, pilots)
• Determination of whether the proposed allocations present HF risks
  

Scenario Development. HF task and function analyses then support the development of robust scenarios to be used in concept development and demonstration. The task analyses can be used to drive the development of scenarios that help to ensure that a given task sequence is demonstrated from end-to-end in a scenario and that all necessary functionality is present for review and analysis. Such a scenario-based approach is important as concepts demonstrated simply through disconnected or partial completion of disparate tasks that do not fully demonstrate the relevant operational scenario are not likely to reveal all of the human performance risks sufficiently early in the AMS process. 

These scenarios are developed by examining the potential uses of the future system and the roles people will play as operators and maintainers, including not only nominal scenarios but also scenarios highlighting workflows and operational and environmental conditions identified and documented in Critical Task Analysis Reports (see p. B-7). 

Verification and Validation (V&V). To minimize “to minimize technical, programmatic, and operational risk”, early HF evaluations are performed to support determination of concept feasibility and analysis of alternative solutions during SASP and CRD phases. During CRD, Consistent with AMS policy Section 4.7, HF involvement during V&V serves to ensure that performance requirements are consistent with human capabilities and limitations in the validation of the following products:

• Shortfall Analysis Report
• Preliminary Program Requirements Document (pPRD)
• Solution Concept of Operations
• Enterprise architecture products
• Initial investment analysis plan
• Preliminary information systems security assessment (crdguidelines.pdf (faa.gov))
  

HF Input to Preliminary Program Requirements Document. During CRD, the Preliminary FAA HFC will interact with the FAA Systems Engineering Management Team to create the preliminary Program Requirements Document (PRD). This includes providing HF Critical Operational Issues (HFCOIs) for this document. The human-system integration requirements indicated in the preliminary PRD need to be reviewed to supply human-system integration requirements to the FAA Systems Engineering Management Team. The human-system integration requirements provided in FAA Human Factors Design Standard (HF-STD-001B), can be considered.

crdguidelines.pdf (faa.gov) indicates:

• “Principal contributors to the pPRD include the Solution ConOps, Shortfall Analysis Report (SAR), Functional Analysis Document (FAD) (derived from the ConOps) and the Enterprise Architecture (EA) Artifacts. Functions contained in the functional analysis are transformed into functional requirements and inserted into the preliminary Program Requirements Document (pPRD).” 
• “The pPRD does not dictate a solution; it is considered the starting point for identifying the essential characteristics of a solution and estimating basic costs that will provide the desired OCs and service outcomes. The sponsoring service organization typically forms a team of experienced technical, user, and program personnel (e.g., operations, human factors, and safety disciplines, etc.) to develop and analyze preliminary program requirements.” 
• “Research or prototyping may be necessary to define an acceptable range of requirements. The pPRD establishes the basis for determining alternative solutions and estimating costs.” 

During the generation and refinement of concepts, depending on the nature and size of the project, the Preliminary FAA HFC or responsible HF support person needs to participate in meetings discussing alternative concepts, use cases and scenarios, documenting potential human performance risk areas to be evaluated during concept demonstration/validation exercises or needs to manage completion of these activities by a contractor.

There are a variety of sources of information that can provide preliminary requirements for the system under review. Examples include:

• Review of Operational and Maintenance CONOPS for the acquisition
• Review of predecessor system information (e.g., procedures, work-arounds, trouble reports, lessons learned)
• Review of research and analysis relevant to the future system
  
  • Contact WJHTC HF Division, CAMI, ANG-C1 and AJM-131 for listing of research, studies, and analyses they may have available for review
  • Request contact information for known subject matter experts from WJHTC HF Division, CAMI, ANG-C1 and AJM-131
  • Collaborate with the Systems Engineering Management Team for this acquisition and contact Subject Matter Experts to request inputs
    
• Attend meetings with Integrated Requirements Team (IRT)
• Review products from the Research for Service Analysis transfer package 
  

Preliminary FAA Integrated HF Program (FAA IHFP). One of the products of CRD produced by the preliminary HFC is a preliminary FAA IHFP. The recommended format and content is available at FAA IHFP. This document includes:

• Purpose, scope, and objectives of the IHFP
• HF organization, role, and responsibilities
• HF strategy, approach
• Solution/program description
• Program background information
  

The FAA IHFP is a living document, incorporating changes and revisions as a result of the identification of evolving HF issues and risks associated with the operation and maintenance of the solution. Note that the preliminary IHFP should include identification of documentation that will support closure of the HF items in the In Service Review (ISR) Checklist (FAA network user only).

In addition to supporting the HF inputs to the IHFP itself, the Preliminary FAA HFC also will ensure drafting of a 1-2 paragraph summary detailing how generic requirements are provided for good HF engineering practices. This summary also indicates how solution-specific HF requirements are derived from the shortfall analysis, SME interviews, task and function analysis, concept demonstrations, and additional HF analyses designed to mature specific human performance requirements.

The Investment Analysis (IA) stage of the Acquisition Management System (AMS) process consists of two sub-stages: Initial Investment Analysis and Final Investment Analysis. 

The Initial Investment Analysis begins with designating an FAA HF Coordinator (FAA HFC). For larger projects, as necessary the FAA HFC then establishes and chairs a Human Factors Working Group (HFWG).

IA Activities. HF tasks to be completed during IA include:

• P. 6-2: Creation of Human Factors Working Group (HFWG) 
• P. 6-3: HF Input for Substages of Analysis (IA)
• P. 6-4: HF Investment Analysis (IA) Inputs for AMS Lifecycle Checklist 
  

Creation of Human Factors Working Group (HFWG). The role of the HFWG is to assure that all HF issues and concerns are identified and successfully addressed during the course of solution development. The HFWG may include the FAA HFC, operator representation, Integrated Logistics Support (ILS) representation, and representation providing expertise in safety, training and testing. It may also have representation by a contractor who may be supporting the activities of the Investment Analysis stage for larger projects. 

More specifically, the responsibilities of the HFWG are to:

• Assist in integrating the HF effort with the system engineering effort
• Coordinate the development, review and execution of the HF effort
• Provide a forum for direct communications between members to identify and address HF requirements, objectives, concerns and issues
• Identify needed HF tasks and activities and review the results thereof
• Review contract deliverables for Human Factors implications
• Provide recommendations concerning human and solution performance
• Ensure unresolved issues are surfaced to appropriate decision makers  and propose the action to be taken to resolve those issues
• Maintain an audit trail of Human Factors activities and decisions
• Coordinate with appropriate HF-related entities
  

HF input for Substages of Investment Analysis. The HF input is provided for both the initial and final stages of IA.

Initial Sub-Stage of IA. The role of the FAA HFC during the initial stage of IA is to: analyze HF impacts of each alternative or to supervise this work being done by the HFC for a contractor supporting the IA stage.

Final Sub-Stage of IA. The role of the FAA HFC during the final stage of IA is to:

• Provide HF inputs to the AMS artifacts for the selected alternative
• Participate as a HF expert in technical evaluation team review of vendor proposals

With guidance from the HFWG, this includes:

• Incorporating HF into the initial and final Investment Analysis Plan (IAP)
• Incorporating HF into the initial and final Implementation Strategy and Planning Document (ISPD)
• Incorporating human requirements into the systems specification
• Including HF in source evaluation criteria
• Incorporating HF in the initial and final Business Case
• Incorporating HF in the Statement of Work (SOW)

As with every stage, the HF analyses are tailored to be consistent with the size, cost, and complexity of the solution being acquired (HF-STD-004A).

HF Investment Analysis (IA) Inputs for AMS Lifecycle Checklist. The IA tab of the HF AMS Lifecycle Checklist indicates a number of HF questions that should be addressed as part of the IA stage. Examples include:

• Have the Human Factors implications of each alternative been determined for their impact on cost, benefits, schedule, and technical risks?
• Do cost, schedule, and performance baselines reflect the detail necessary to cause the identification or resolution of human-system performance issues/risks?
• Does the Implementation Strategy and Planning Document (ISPD) reflect the recommended approach to manage the HF program for the project?
• Has the information from the HF assessment been incorporated in the Business Case and IA decision?  
  

FAA HF Coordinator (FAA HFC) Responsibilities. In support of the implementation based on the Solution ConOps and requirements, the FAA HF Coordinator is responsible for:

• Conducting HF activities or overseeing their completion by a contractor (see the following pages)
• Assisting the Program Office in demonstrating compliance to the HF items in the ISR Checklist (FAA network user only)
  

Additional responsibilities during Solution Implementation (SI) include:

• P. 7-2: General FAA HF Coordinator (FAA HFC) Responsibilities
• P. 7-3: Transfer of Documents from Previous Stages
• P. 7-4: Monitoring and Coordinating Contractor and Support Team Activities
• P. 7-5: Participating in Ongoing FAA, Contractor and Working Group Meetings
  
• P. 7-6: Approving Plans and Results of Contractor HF Empirical Evaluations
• P. 7-7: Observing HF Activities and Approving Contractor's HF Deliverables
  

General FAA HF Coordinator (HFC) Responsibilities. In the previous sections, the role of the FAA HFC prior to the award of a contract for Solution Implementation (SI) has been described. The FAA HFC continues to play an important role during SI, tailored to match the size, cost, and complexity of the solution being acquired (HF-STD-004A). Broadly speaking, during SI, the FAA HFC can be assigned several responsibilities:

• Develop an effective working relationship with the HFC working for the contractor responsible for Solution Implementation (SI), establishing a supportive pattern of communication, coordination and collaboration with well-defined expectations (Note that the contractor is responsible for the actual conduct of HF activities to support SI)
• Review and approve the Contractor IHFP (Contractor Integrated Human Factors Plan) for SI and get signature approval from the FAA HF Division Director (ANG-C1)
• Refine and approve the FAA IHFP plan for SI and get signature approval from the FAA HF Division Director (ANG-C1)
• Fulfill the responsibilities of the FAA HFC as indicated on the following pages
  

See the FAA Lifecycle Management Process Flowchart for a list of HF questions that need to be addressed during SI.

Team Formation (HFC) Responsibilities. The FAA HFC plays several roles in Solution Implementation (SI):

• Step 1. Adding the SI Contractor HFC to the Human Factors Working Group and revising the membership of that group as needed
• Step 2. Providing members for the SI Computer Human Integration (CHI) Team (user team)
  

Transfer Documents from Previous Stages. Some of these insights can be provided through the HF requirements associated with the Statement of Work (SOW) for SI itself, while others can be provided to the contractor SI team based on documents in transfer packages, as well as any storyboards and/or prototype(s), produced during Research for Service Analysis and earlier AMS stages.

Such mechanisms for the transfer of knowledge from the earlier stages to the SI CHI and SE teams are important because it is unlikely that that the requirements in the SOW for SI can fully communicate all of the insights gained during the earlier stages.

In addition, such a transfer of knowledge can be further supported by transitioning a limited number of members from the CHI team set up during the Research for Service Analysis to remain as members of the CHI team for SI, thus providing continuity. To achieve this, when appropriate the FAA HFC can request one or two specific individuals who were on the CHI team supporting the Research for Service Analysis remain on the CHI team for SI, keeping in mind that the bargaining unit makes the choices, and also keeping in mind that there is value in also bringing in members who have not previously been associated with this particular SI in order to include “fresh eyes”.

• Step 3. Ensuring the transfer of HF insights developed during the stages prior to SI to the contractor HFC (and supporting HF practitioners) and Systems Engineering (SE) team as well as the SI CHI team

Monitoring and Coordinating Contractor and Support Team Activities. The FAA HFC plays several roles in SI:

• Step 4. Monitoring the contractor HF activities
• Step 5. Coordinating with the Safety and Training Groups
• Step 6. Coordinating with FAA Program Management and Systems Engineering (SE) on the HF Coordinator role on the project with respect to the Responsible, Accountable, Support, Consult, Inform (RASCI) matrix
• Step 7. Coordinating briefings by the HF specialist(s) involved in Research for Service Analysis and earlier AMS stages to the Contractor HFC and SE team about lessons learned during these earlier stages regarding design recommendations for the Graphical User Interface (GUI)/CHI and the associated software functionality, workflow and procedures, including compatibility with the broader hardware and software environment. Note that this broader environment includes:
  
  • Acoustic noise and vibration
  • Space for personnel, their movement, and their job aids and equipment
  • Safe and efficient equipment configurations, facility design, and working environments
  • Protection from chemical, biological, toxicological, radiological, thermal, mechanical, electrical, and electromagnetic hazards
  • Illumination commensurate with anticipated visual tasks
    

Note that it is important for the FAA HFC to manage the timing and nature of such briefings. The goal is to convey lessons learned relative to the design alternatives that are under consideration by the SI contractor in order to help inform decisions by the SI regarding the CHI and supporting software/hardware functions and associated procedures. The goal is not to stifle the expertise and creativity of the SI as a source of “fresh eyes” regarding the generation and evaluations of potentially effective design solutions.  

Participating in Ongoing FAA Contractor and Working Group Meetings. The FAA HFC plays several roles in SI:

• Step 8. Participating in TIMs, working groups, and relevant engineering, development, and management meetings.

Reviewing and Approving Heuristic Analyses. The FAA HFC also monitors analytical (judgement based) evaluation of HF design features.

• Step 9. Reviewing and approving the completion of an heuristic analysis based on the guidelines specified the FAA Human Factors Design Standard (HFS-STD-001B). Note that it is important that this heuristic analysis be conducted with the full range of relevant use cases and scenarios in mind.
  

Approving Plans and Results of Contractor HF Empirical Evaluations. The FAA HF Coordinator (HFC) ensures that appropriate HF empirical evaluations are planned and completed. 

• Step 10. Providing input and approval for the plans developed by the Contractor HFC for cognitive walkthroughs, Early User Interface Events (EUIEs) and other appropriate methods for testing and evaluation (FAA Testing), including the data collection instruments for the evaluation of features in the GUI/CHI and associated software functionality in the context of the proposed workflow and procedures, as well as compatibility of the GUI/CHI and associated software functionality with the broader hardware and software environment
  

Scenario Selection. The FAA HFC reviews and approves the scenario selection and design for use in testing and evaluation. Since only a limited number of scenarios can be selected for evaluation in cognitive walkthroughs and EUIEs, selection should be based on consideration of the frequencies of the embedded subtasks in actual operations, the criticality of successful completion of these subtasks, and any uncertainties regarding the most effective CHI designs and associated functionalities. A further consideration is to ensure coverage of not only nominal scenarios but also the inclusion of important off-nominal scenarios.

The elements of these scenarios can be guided by the results of the task analyses and associated Critical Task Analysis Reports (see p.B-7) provided in the transfer package as well as by the contents of the operator human engineering design approach documents (HEDADs) produced by the Vendor responsible for SI (described below).

It is important to note, however, that the CHI design and supporting software and hardware functionality (including automation) as developed by the SI may introduce new critical tasks not considered during the Research for Service Analysis and earlier AMS stages and can thus identify additional scenarios that require consideration. 

Selection of Operators for Testing. The selection of the operators to include in cognitive walkthroughs and EUIEs also is very important. The participants need to be selected to ensure adequate coverage in order to get feedback on design decisions from the full range of prospective users. Three points merit emphasis:

• First, although an effort is typically made to include a range of different users to serve as Subject Matter Experts (SMEs) on the CHI team for a project, it is often the case that these SMEs are among the more experienced of the relevant user population. Such representation is important, as such experts are likely to have a greater breadth and depth of experience
• Second, the CHI team of necessity has a limited number of user representatives, potentially making it impossible to fully represent the full range of different user groups
• Third, since the CHI team supports development over an extended period of time, like the staff responsible for the design and implementation of the software, the members of the CHI team may become too familiar with the design and therefore may have difficulty seeing it with “fresh eyes”
• The transfer package from the Research for Service Analysis should provide insights into the range of users that should be considered in testing
  

The conclusion regarding the selection of participants for testing is therefore twofold:

• First, the members of the CHI team are an important subset of the users that need to be included in evaluations
• It is important to use the results of the user analysis (drawing upon findings from the Research for Service Analysis and any additional data collected early in the SI process) to determine what additional participants in testing should be obtained from the user community in order to reflect the full set of differences across users and to guide decisions about participant selection. And tests with these additional participants should be separate from tests with the CHI team members in order to get independent assessments
  

In addition, it is important to recognize that it is not just the immediate user of a tool who needs to be considered. As an example, for certain tools for ATC, it is not sufficient to assume that systems for ATC can be designed without an assessment of impacts resulting from the performance of pilots. Thus, for some ATC tools, it is important to get input from the broader set of people who are indirectly affected by the introduction of a new capability, not just the immediate users.

Observing HF Activities and Approving Contractor’s HF Deliverables.

• Step 11. Observing contractor HF-related activities, including EUIEs, design reviews by the CHI team and other testing events, providing the Contractor HFC with input based on these observations as well as using those insights to support judgments in approving SI deliverables
• Step 12. Reviewing and approving HF deliverables produced by the Contractor HFC
  

Of particular importance in this set of deliverables are HF Contract Data Requirement Lists (CDRLs) regarding the use of the system by each of the relevant user groups, such as the human engineering design approach document for the operator (Operator HEDAD-O) and the maintainer human engineering design approach document (HEDAD-M).

The goal of these documents is to provide a thorough picture of the operator tasks, equipment and operator interfaces with that equipment. More specifically, each of these documents provides an evaluation of whether the equipment having an interface with that type of operator meets with human engineering design criteria, and describes the characteristics, layout and interface of the equipment having an operator interface. This evaluation is based on an assessment of the proposed design, including software (CHI and underlying functionality) and hardware in the context of the proposed workflow and procedures as well as in the context of the broader physical and software environment. This evaluation is based on the requirements of FAA-HFS-STD-001B.

Thus, the scope of the HEDAD for each type of operator is to document the following workstation, workstation interface and operator information:

• Workstation equipment and layout, controls and displays, requirements for operator vision, environmental factors, workstation lighting, workstation signals, communication systems, special design features, multiple operator station requirements
• Description of the guiding human design rationale
• Analysis of operator tasks
• Evaluation of the consistency of the design decisions with human engineering design criteria
  

HF Activities for In-Service Management. There are two reasons for post-deployment assessments from an HF perspective:

• To determine whether there are performance issues that need to be addressed
• To identify opportunities for enhancements of the product in a future release
  

Following completion of the initial implementation of an acquisition, there are four primary HF responsibilities:

• Support the post implementation review (PIR) from an HF perspective
• Execute and monitor the results of the process for assessing HF in Operations
• Provide mitigation for identified HF problems
• Review reporting systems for possible HF issues
  

Additional details are provided on the following pages:

• P. 8-2: HF Post Implementation Assessments
• P. 8-3: Potential Post Implementation HF Issues
• P. 8-4: Opportunities for Future Enhancements
  

HF Post Implementation Assessments. There are a variety of complementary sources to support these HF assessments. One formal component is completion of the In-Service Review (ISR) checklist (FAA network user only). Other methods include:

• Design and use of a dashboard that provides quantitative data on the use of the product. Note that such a dashboard should be designed to not only indicate the frequency of use of the product, but should also provide data that helps to diagnose the underlying causes or contributing factors
• Input from operators and maintenance. Note that an environment should be established that encourages such input, including explicit requests for input. Note also that such input needs to include not only the operators directly using the product but also operators who tasks are affected by the introduction of this new product and associated procedures, which may include operators at other facilities
• Input from the trainers involved in the introduction and continued use of this product
• Direct observation of the use of the product
  

And since different issues may arise in different facilities, it is important that post-deployment assessments not fixate on a subset of those facilities. 

Potential Post Implementation HF Issues. There are a variety of reasons why HF issues could arise after the deployment of the solution. This includes:

• Resistance to change resulting in disuse of the product
• Problems with the software or hardware such that some function is not performed as intended (including interactions with other software or hardware systems)
• Barriers to use of the system such as facility or workstation layout or environmental conditions (such as lighting) that impact the effective use of the product
• Inadequacies in the procedures that have been defined and trained for using the product
• Use of unintended procedures that have not been trained that could result in inefficiencies or safety concerns
• Use of unintended procedures that are actually better than those that have been trained
• Inconsistencies in the training and/or procedures used at different facilities
• Weaknesses in initial or recurrent training (including insufficient training when there is turnover in the personnel involved)
• Issues with staffing
• Inadequate coordination and communication with other operators (at the same or different facility) that is necessary as part of the workflow required for the effective use of the new product
• Occurrence of unexpected scenarios that impact efficiency or safety 
  

When such issues are identified, the FAA HFC Coordinator, Systems Engineering and local facility management all need to be informed and an appropriate plan needs to be developed to respond to them at both a local and national level as appropriate. In some cases this may involve an urgent patch or immediate training. In others, the remediation may wait for the next release of the product.

Opportunities for Future Enhancements. While monitoring post-deployment performance, opportunities for future enhancements may be identified. 

Some enhancements may focus on minor improvements in the Computer Human Interaction (CHI) or supporting functionality. Others may involve significant new additions or changes to the CHI and the software functionality. 

While monitoring for performance issues associated with the current release of the product, information should be collected to provide insights into such needs in order to guide the design of future releases. And in some cases, requirements for future releases may have already been identified during Solution Implementation (SI). 

HF input is then required in prioritizing, planning and implementing these future releases.

AI    Artificial Intelligence
AMS     Acquisition Management System
ATC    Air Traffic Control
CHI    Computer Human Interaction
CRD    Contracts and Requirements Definition
FAA HFC    FAA Human Factors Coordinator        
Contractor HFC        Contractor Human Factors Coordinator 
HEDAD    Human engineering design approach document
HEDAD-M    Human engineering design approach document - Maintenance
HEDAD-O    Human engineering design approach document - Operator
HF    Human Factors
HFC     Human Factors Coordinator
HFCOI    Human Factors Critical Operations Issue
HFWG    Human Factors Working Group
HIS    Human Systems Integration
IAP    Investment Analysis Plan
IHFP     Integrated Human Factors Plan
ILS    Integrated Logistics Support
IPSD     Implementation Strategy and Planning Document
ISR    ISR Checklist    In Service Review Checklist
JRC    Joint Research Council
ONA    Operational Needs Assessment
PIR    Post Implementation Review
PPRD    Preliminary Program Requirements Document
PRD     Program Requirements Document
RASCI    matrix    Responsible, Accountable, Support, Consult, Inform matrix
SME    Subject Matter Expert
TFM    Traffic Flow Management

• Acquisition Program Baseline (APB) https://fast.faa.gov/docs/apbtemplate.docx 
• Acquisition Management Policy Section 4.7 Human Factors https://fast.faa.gov/docs/acquisitionManagementPolicy/AcquisitionManagementPolicy4.7.pdf#nameddest=policy4_7 
• AFI-1 https://www.faa.gov/about/office_org/headquarters_offices/afn/offices/finance/offices/investment_planning 
• AJM-131(accessible only on the FAA network) https://ksn2.faa.gov/ajm/pmo/about-us/ajm1/ajm13/131/hf/Pages/Home.aspx 
• AJM-S (Program Management Organization) https://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/pmo 
• AJV (Mission Support) https://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/mission_support 
  
• Acquisition Management System (AMS) https://fast.faa.gov 
• ANG-C1 https://www.hf.faa.gov/
• ANG-C5 (accessible only on the FAA network) https://my.faa.gov/org/staffoffices/ang/ang_org/directorates/angc/angc5 
• Air Traffic Organization (ATO) https://www.faa.gov/about/office_org/headquarters_offices/ato
• Australian Department of Defense http://burgess-limerick.com/download/d11.pdf
• Business Case Analysis Guidance - Business Case Template for Tech Refresh Investment https://fast.faa.gov/docs/businesscasetechnologyrefreshment.docx 
• CAMI (FAA Civil Aerospace Medical Institute) https://www.faa.gov/about/office_org/headquarters_offices/avs/offices/aam/cami
• Collaborative Decision Making program (CDM) https://cdm.fly.faa.gov/
• Critical Task Analysis Reports https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf (see p.B-7)
• DODI 5000.95 Human Systems Integration in Defense Acquisition https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500095p.PDF
• FED-STD-795, Uniform Federal Accessibility Standards (UFAS) http://everyspec.com/FED-STD/FED-STD-795_9884/
• Human Factors AMS Lifecycle Checklist https://faa.switchboxinc.com/published/ams_lifecycle_checklist.xlsx 
• FAA Human Factors Acquisition Job Aid https://fast.faa.gov/docs/HFAcqJobAid.doc 
• FAA Human Factors Design Standard (HF-STD-001B) https://hf.tc.faa.gov/publications/2016-12-human-factors-design-standard/ 
• Integrated HF Program (FAA IHFP) (accessible only on the FAA network) https://ksn2.faa.gov/ajm/pmo/about-us/ajm1/ajm13/131/hf/hfe-framework/Shared%20Documents/AMS%20Document%20Repository/IHFPP%20Template.docx 
• FAA Lifecycle Management Process Flowchart: Human Factors https://fast.faa.gov/flowcharts/grid_platform.cfm?p=hf 
• FAA Order 9550.8, Human Factors Policy https://www.faa.gov/documentLibrary/media/Order/9550.8.pdf 
• FAA Testing  https://www.faa.gov/sites/faa.gov/files/about/office_org/headquarters_offices/ang/TEHandbook.pdf  
• Functional Analysis Document (FAD) https://fast.faa.gov/docs/fadtemplate.docx 
• Handbook of Human Systems Integration https://www.wiley.com/en-us/Handbook+of+Human+Systems+Integration+-p-9780471020530
  
• Human Factors Engineering Requirements (HF-STD-004A) https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf
  
• Implementation Strategy and Planning Document (ISPD) https://fast.faa.gov/docs/ispdtemplate.doc 
• Investment Analysis Plan (IAP) https://fast.faa.gov/docs/iainitial.docx
• In Service Review (ISR) Checklist (FAA network user only) https://my.faa.gov/org/linebusiness/ato/safety/isd.html
• Investment Planning and Analysis Toolkit https://fast.faa.gov/AMSBB_Investment_Planning.cfm 
  
• Joint Research Council (JRC) (accessible only on the FAA network) https://ksn2.faa.gov/afn/jrcportal/SitePages/Home.aspx 
• Lee, J.D. & Seppelt, B.D. (2023). Design for Human-Automation and Human-Autonomous Systems. In: Nof, S.Y. (ed) Springer Handbook of Automation. Springer. https://doi.org/10.1007/978-3-030-96729-1_19
• MITRE CAASD https://www.mitre.org/our-impact/rd-centers/center-advanced-aviation-system-development 
• NASA https://www.nasa.gov/
• NextGen https://www.faa.gov/nextgen
• PMO https://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/pmo
• Program Requirements Document (PRD) Template https://fast.faa.gov/docs/programreq.docx 
• RASCI https://www.faa.gov/sites/faa.gov/files/2022-02/11MaureenKeeganVandVConference20Sept2018CVRModel.pdf 
• RTCA https://www.rtca.org/
• Service Analysis and Strategic Planning (SASP) https://fast.faa.gov/NFFCA_ServiceAnalysis_StrategicPlanning.cfm 
  
• Safety Risk Management Guidance for System Acquisitions https://www.faa.gov/air_traffic/publications/media/srmgsa.pdf
• SWIM Industry-FAA Team (SWIFT) https://www.faa.gov/air_traffic/technology/swim/swift
• WJHTC HF Division https://hf.tc.faa.gov/ 

• System Specification Document - Sample HF Content

Human Factors Data Item Descriptions (from HF-ST-D004a)