Keywords: aviation safety; risk analysis

Abstract

A Probabilistic Decision Support System (PDSS) is being developed to assist NASA program managers with evaluating new technologies that are intended to lower the fatal aircraft accident rate. This decision support system utilizes the method of case-based reasoning in which case studies of specific accidents are modeled and analyzed.

One such case study is the crash of Air Ontario Flight 1363 which resulted from a loss of control due to ice contamination on the wings. The decision to take-off without deicing was the end result of a complex interaction of causal factors including crew resource management, worsening weather conditions, and organizational issues.

In this research work, accidents such as Air Ontario 1363 are modeled and the reduction in risk from the insertion of appropriate technologies is assessed. A user-friendly interface is being developed to create a software application that is flexible and easy to use.

1.0 Introduction

There are several existing approaches for explaining accident causation, one of which is the Reason model of accident causation. The Reason model is a general model that traces the root cause(s) of an accident to organizational errors arising in the upper levels of any organization (Reason, 1995). This model is helpful in that it allows explanations of accidents to be based upon more than just individual operator performance. In large-scale technological systems such as those in the aviation industry, each situation or event is a result of multiple factors interacting within the organization. The accident causal chain as explained by the Reason model, sometimes called the Swiss Cheese Model, is depicted in Figure 1.

Figure 1. Reason's "Swiss cheese" model of human error causation

One of the disadvantages of the Reason model is that it does not account for the detailed interrelationships among causal factors. Without these distinct linkages, the results are too vague to be of significant practical use. It is for this reason that in this research, a combination of approaches and methodologies is used to create a more comprehensive method for evaluating aviation safety system risk.

1.1 Human Factors Analysis and Classification System (HFACS)

Between 70 and 80 percent of aviation accidents can be attributed, at least in part, to human error (Shappell and Wiegmann, 1996). Hence, the taxonomy described by the Human Factors Analysis and Classification System (HFACS) is used as a framework for the modeling and analysis of accidents.

HFACS uses Reason's model as a basis for describing the "holes" in the four levels of human failure, which are both individual and organizational in nature. The four levels are unsafe acts, preconditions for unsafe acts, unsafe supervision, and organizational influences. HFACS divides each level into a series of causal factors (Figure 2).

Figure 2. HFACS Taxonomy

For instance, unsafe acts are broken into two causal categories: errors (light yellow) and violations (darker yellow). Each of these categories, in turn, is subdivided. In particular, within the "error" category, the scheme distinguishes decision errors, skill-based errors, and perceptual errors; while "violations" are described either as routine or exceptional.

A complete list of the individual causal factors in HFACS and an explanation of each can be found in Shappell and Wiegmann (2000).

1.2 Aviation Safety Program (AvSP) Technologies

The safety of flying aboard U.S. airlines has significantly improved over the past century. However, the aviation industry continues to grow and the cost of aviation accidents has risen steadily in lives and dollars (Shappell and Wiegmann, 2000). The high costs associated with even a single accident place great pressure on the National Airspace System (NAS) to reduce the number of accidents even further from the current level.

It is for this reason that the Aviation Safety Program (AvSP) Office at NASA is actively involved in the development of new technologies to reduce the fatal aircraft accident rate by 80% within ten years (NASA Product Dictionary, 2002). These technologies include advanced simulation training modules, improved vision systems, and enhanced equipment for turbulence detection during adverse weather conditions. NASA's AvSP Technologies can be grouped into three main categories as shown in Figure 3.

Figure 3. AvSP Technologies

Unfortunately, many of these new technologies are so expensive that under current budget constraints, it would not be possible to install and implement each of them in every airline. In order for NASA to achieve its goal of an 80% reduction in the fatal aircraft accident rate, it is crucial to target the technologies (or combinations of technologies) that will have the greatest reduction in overall system risk.

2.0 Research Objective

Each technology has a target goal associated with it that is stated in terms of technical success such as an improvement in crash survivability or a percentage reduction in the number of weather-related accidents. Even though each new technology has an associated goal, these goals are estimations for individual technologies that in most cases do not indicate system risk reduction. In other words, the goals are not easily comparable since they are technology-specific and are not measured according to an overall reduction in the number of accidents. In addition, they do not provide an estimated reduction in system risk for combinations of technology insertions. These measurements are necessary for NASA to compare technologies and to determine which technologies to target for initial implementation.

The objective of this research project is to provide NASA with a Probabilistic Decision Support System (PDSS) to evaluate the projected impact upon system risk reduction of new technology insertions/interventions.

2.1 Analytical Modeling Approach

Our analytic modeling approach consists of the following stages:

1. Describe a case-based scenario, determining the events involved in the occurrence of the accident/incident.
2. Identify the causal factors (nodes) present in these events using the HFACS taxonomy or other additional factors outside of HFACS.
3. Construct an influence diagram depicting the interrelationships among these nodes.
4. Build a Bayesian Belief Network (BBN) propagated with conditional probabilities. These probabilities are based on a combination of statistical frequencies from the National Transportation Safety Board (NTSB) database and judgments from subject matter experts.
5. Insert the relevant technologies/interventions into the model of the particular case, attaching each to a causal factor.
6. Assess the relative risk of the particular case and the reduction in risk resulting from varying combinations of technology insertions.

We illustrate the use of the approach by applying it to the crash of Air Ontario Flight 1363.

Figure 4. Analytical Modeling Approach

3.0 Air Ontario 1363 Case Study

3.1 Accident Summary

The following summary is based on the description and analyses presented in Beyond Aviation Human Factors: Safety in High Technology Systems by Maurino et al. (1995).

On Friday, March 10, 1989, Captain George C. Morwood advanced the throttles of Air Ontario Flight 1363, initiating take- off from Dryden Airport, Ontario, Canada. Flight 1363 was the second part of the day's flying schedule which consisted of a Winnipeg to Thunder Bay round trip, with intermediate stops at Dryden.

Captain Morwood reviewed the operational status of the aircraft before departing Winnipeg and verified, among other maintenance deferred defects, that the Auxiliary Power Unit (APU) was unserviceable. This defect meant that either the engines had to be started from an external power unit or one engine had to be kept running to cross-start the other engine. If both engines were shut down at a station where no external power unit was available, the aircraft would be grounded until the APU was fixed or an external power unit was made available. There was no external power source at Dryden, and therefore, one engine would have to be kept running. The manufacturer, Fokker, and Air Ontario strictly prohibited de-icing with either engine running.

Because the original flight release from Thunder Bay to Dryden prepared by the Air Ontario Systems Operations Control (SOC) had not been updated, ten passengers were added to Flight 1363 after it had been refueled. Captain Morwood suggested off-loading the ten passengers and their baggage since the aircraft was overweight for take-off. However, the Air Ontario SOC duty manager overrode the Captain's decision and decided to reduce the weight by off-loading fuel. The de-fueling caused an additional 35-minute delay in the departure of Flight 1363 from Thunder Bay. Flight 1363 departed Thunder Bay and arrived in Dryden one hour behind schedule.

The hot refueling process started with passengers on board, which is considered to be an unsafe practice. This is unusual given that Captain Morwood was a conservative decision-maker and strictly adhered to rules and regulations. It was indicated by the Commission of Inquiry that Captain Morwood had a heated conversation with the SOC regarding the passenger load and weather conditions in Winnipeg prior to the departure from Dryden. It was visible that Captain Morwood became increasingly irritated while in the terminal, after his telephone contact with the SOC, and that as he briskly walked back to the aircraft, he showed signs of frustration.

When he returned to the aircraft, Captain Morwood asked the ground handler whether de-icing was available. He was told that it was, but he still did not request de-icing. Snow was falling heavily as the aircraft was about to leave the terminal, and the wings were covered in snow to depths varying from one-eighth to one-quarter of an inch.

The combination of the slush on the ground and the wet snow, which had frozen into ice on the wings, hindered the performance capabilities of the aircraft. After a longer than normal take-off roll, the aircraft rotated, lifted off slightly, began to shudder, and settled back onto the runway. It rotated again, lifting off at the 5,700 ft point of the 6,000 ft runway. It flew briefly, and cleared the end of the runway approximately 15 feet above the ground. It failed to gain altitude and crashed, coming to rest about one kilometer from the end of the runway.

This accident and a preliminary model were presented in "An Aviation System Risk Model (ASRM) Case Study: Air Ontario 1363" by Luxhøj and Maurino (2001). Since then, enhancements have been made to follow the HFACS taxonomy as mentioned previously and to reflect advice gathered from subject matter experts.

3.2 Influence Diagram For Air Ontario 1363

The following model (see Figure 5) depicts the causal factors and the interactions among them. A detailed description of these factors and interrelationships are described subsequently.

Figure 5. Influence Diagram for Flight 1363

3.3 Node Explanations

• Transport Canada: Transport Canada was not in a favorable position at the time of the incident because of spending cuts that had been imposed by the Canadian government. Hence, the number of people available and qualified to carry out responsibilities was very much reduced. The Commission of Inquiry identified several accident causal factors relating to Transport Canada. Among these factors was that Transport Canada:
  • did not provide clear guidance for carriers and crews regarding the need for de-icing.
  • did not require licensing or effective training of flight dispatchers.
• Organizational Climate: Air Ontario was the product of a merger between Austin Airways Limited - a northern operation - and Air Ontario Limited - a Great Lakes, scheduled service operation. Austin Airways was the 'winning party' or the buyer whereas Air Ontario was the 'loser' or acquired company. The companies' fleets, operating environments, employee groups and management styles were different. Following the merger, there was a bitter strike that left residual feelings of ill will among pilot groups. Of immediate relevance to the events of March 10 is the fact that while Captain Morwood came from Air Ontario Limited, First Officer Mills came from Austin Airways.
• Organizational Processes: There was ambiguity not only in flight deck operating procedures but also in maintenance and dispatch procedures. Ambiguities included incomplete information for crews regarding take-off with contamination on the wings and cold weather operations. There was a lack of standardized operations manuals.
• Resource Management: Air Ontario was owned by Air Canada. However, Air Canada did not provide Air Ontario with enough training or personnel. The dispatcher training and certification was entirely in the hands of the air carriers. The dispatcher responsible for the F-28 flights on that day was underqualified, prepared an incorrect flight release, did not notify the crew of the freezing rain at Dryden and failed to advise the crew that an overflight of Dryden was indicated.
• Inadequate Supervision: There were changes in two critical areas of operational management during the period from June 1987 until March 10, 1989: Vice President of Flight Operations and Director of Flight Operations. The instability and problems of supervision created by the lack of management continuity was an obstacle to the implementation of the changes required by the introduction of a new aircraft type.
• Failure To Correct a Known Problem: The Commission disclosed that Air Ontario personnel often deferred maintenance 'snags' in an unauthorized manner, and subsequently flew the aircraft without fixing them. This led, on a number of occasions, to operating the aircraft without a valid certificate of airworthiness.
• Planned Inappropriate Operations: Although both pilots of Flight 1363 had considerable experience, they were 'newcomers' to the F-28. Hence, pairing of the pilots was not appropriate. Moreover, the inquiry revealed that poor planning within the SOC and the lack of training and qualification of Air Ontario SOC personnel resulted in dispatching Flight 1363 in worsening weather conditions.
• Supervisory Violations: For the morning of the accident, the logbook of the aircraft listed six deferred "snags," including the unserviceable APU. As mentioned above, Air Ontario personnel often deferred maintenance 'snags' in an unauthorized manner, and subsequently flew the aircraft without fixing them. Moreover, senior management was prone to tolerating supervisory violations.
• Adverse Mental States: The report leaves no room for doubt that Captain Morwood was exhibiting distinct symptoms of stress when he landed in Dryden on the return trip. Stress degrades the ability of humans to process information. The unserviceable APU and other deferred maintenance items, the confusion over defueling versus deplaning passengers at Thunder Bay, an inexperienced SOC dispatcher, the absence of ground support facilities, concern over passenger connections and the ground hold for the Cessna were all reasons for crew frustrations (See below).
• Crew Resource Management (CRM): The crewmembers of Flight 1363 did not perform as an intact unit but rather as a collection of individuals. CRM training had not been provided by Air Ontario to its crews at the time of the accident.
• Decision Errors: In Captain Morwood's mind, under the existing circumstances, based on his knowledge and understanding of the situation and his perception of the surrounding events, he was making the right decision regarding the take off. Unfortunately, he was wrong.
• Perceptual Errors: A visual check of the wings from the cockpit led the pilots to believe that there was no accumulation of ice on the wings. This was a misperception of the actual hazard.
• Routine Violations: Air Ontario personnel often deferred maintenance 'snags' in an unauthorized manner, and subsequently flew the aircraft without fixing them. Another instance for a routine violation in this case is the so- called 80-knot test. During the take-off roll, each pilot would look out the window at 80 knots to see if the snow had blown off.
• Loss of Control (LOC): The chain of unfortunate events resulted in the unavoidable outcome: loss of control of the aircraft.

3.4 Causal Factor Interactions

The interactions among the causal factors of this accident are depicted through the links among the nodes in Figure 1. This section explains the reasons for making the links.

• Organizational Climate Organizational Processes

Spending cuts imposed by the Canadian government meant that the number of people available and qualified to carry out both their original and new responsibilities acquired as a result of the merger was very much reduced. As a result, workloads increased.

The report discusses Air Canada's lack of support to Air Ontario during the introduction of the jet service, and compares standards in specific areas such as operational policies for dispatch with an unserviceable APU; minimum equipment lists; manuals; aircraft defects, hot refueling policies; de-icing policies, etc. The comparisons show that Air Canada and Air Ontario had differing safety standards.

• Organizational Climate Resource Management

After the merger, Air Ontario staff members were not provided with sufficient levels of training and resources. Some of the appointments made at Air Ontario, which included naming the president's close relatives to key managerial positions, were the subject of considerable discussion at the Air Ontario committee board meetings. Some of those newly appointed Air Ontario managers were confronted by demands for which their experience may not have been adequate.

• Organizational Climate Supervisory Violations

The organization's tense atmosphere and poor delegation of authority allowed willful disregard of rules and regulations by supervisors. For example, the F-28 project manager had the responsibility to ensure that the implementation and operation of the F-28 program was properly monitored and supervised. The appointed manager, a relative of the president of the company, lacked experience in the F-28 program.

• Organizational Processes Adverse Mental States

Inadequate standard operating procedures and factors, such as operational tempo and time pressures, adversely affected the mental states of the employees.

• Organizational Climate Adverse Metal States

The report includes the contention that the working relationships among the pilots over the previous two days had probably not been cordial, with the consequent stress impact on the crew. Following the merger, there was a bitter strike that left residual feelings of ill will among pilot groups.

• Organizational Processes Inadequate Supervision

Lack of formal, written operating procedures resulted in supervisory inaction with regard to maintenance problems and frequent failures to de-ice.

• Resource Management Crew Resource Management (CRM)

CRM training aims at developing skills for the optimum utilization of the resources available to the captain as well as to other crewmembers. Air Ontario did not provide CRM training to its crew. Due to lack of training, the dispatcher failed to inform the rest of the crew of the freezing rain.

• Resource Management Planned Inappropriate Operations

The weather forecast called for freezing rain at Dryden during the time span of operation of Flight 1363. This forecast was available to the Air Ontario SOC while Flight 1363 was still on the ground at Thunder Bay. This information, which could have induced Capt. Morwood to overfly Dryden, was never transmitted to the pilots due to lack of dispatcher training.

• Inadequate Supervision Decision Errors

Some F-28 pilots used the Piedmont F-28 Operations Manual while others used the US Air F-28 Pilot's Handbook because Air Ontario did not have its own standardized F-28 operations manual. This lack created problems on the flight deck.

• Inadequate Supervision Failure to Correct a Known Problem

Air Ontario personnel often deferred maintenance 'snags' and flew the aircraft without fixing them. This practice was the result of inadequate supervision.

• Failed to Correct a Known Problem Routine Violation

Failures to de-ice and correct known maintenance problems are routine violations.

• Planned Inappropriate Operations Crew Resource Management (CRM)

Deficiencies in crew pairing resulted in miscommunication among the crew.

• Supervisory Violations Routine Violations

Failure to enforce rules and regulations and the actions of an unqualified dispatcher with respect to maintenance defects led to routine violations.

• Adverse Mental States Decision Errors

Stress due to numerous factors degrades the ability of humans to process information. Captain Morwood's final decision to take off was affected by events that caused high levels of stress.

• Adverse Mental States Perceptual Errors

A lack of situational awareness led to the pilots' disbelief in the need for de-icing.

• Adverse Mental States Routine Violations

The chief pilot's overconfidence led to ignoring maintenance defects on several occasions and to underestimate the need for de-icing.

• Adverse Mental States Crew Resource Management (CRM)

Stress due to the merger and misplaced motivation (acting to avoid flight delays rather than to ensure safety) resulted in miscommunication among the crew.

• Crew Resource Management (CRM) Decision Errors

Lack of information due to miscommunication and lack of training resulted in procedural errors.

• Decision Errorsà Loss of Control (LOC)

Poor decision to take off with contaminated wings resulted in a loss of control.

• Perceptual Errors Decision Errors

The pilot's visual inspection from the cockpit led them to believe that there was no ice accumulation on the wings. This misperception led to the poor decision of taking off.

• Routine Violations Loss of Control (LOC)

Several instances of pilots' failures to de-ice (e.g. 80-knot test) eventually resulted in LOC.

3.5 ASRM with AvSP Technology Insertions/Interventions

In addition to the enhancements made from the original Air Ontario 1363 model presented by Luxhøj and Maurino (2001), another model has been built to assess the possible risk-reducing effects of certain AvSP technologies. Upon discussions with subject matter experts and on our knowledge of the 41 proposed technologies, several AvSP technologies have been found to be appropriate for insertion into the model of Air Ontario Flight 1363. Figure 6 depicts the ASRM with relevant technology elements.

Figure 6. Influence Diagram of Flight 1363 with Technology Insertions

The following is a list of the technologies shown in the above influence diagram and the reasoning associated with each insertion. The definitions and descriptions in this section are based on the NASA Product Dictionary (2002).

• ASMM-1: Incident Reporting Enhancement Tools

This technology element involves upgrading the 24-year-old technology of the Aviation Safety Reporting System (ASRS) database to include: conversion of the ASRS legacy database to ORACLE, electronic submission of reports, and testing and evaluation of an analyst decision support system.

With the insertion of this technology, improvement in the ASRS report production and research processes is expected. Hence, this element is considered to have a potential impact on the following nodes: 'Organizational Processes', 'Decision Errors', and 'Transport Canada'.

• ASMM-2: National Aviation System Operational Monitoring System (NAOMS)

This element aims at the permanent field implementation of a National Aviation Operational Monitoring Service (NAOMS) responsible for developing and maintaining a comprehensive and coherent survey of the safety and performance of the NAS from the perspective of front line personnel NAS wide.

The objective of this element is to create a mechanism for measuring the overall safety of the NAS in a quantitative, precise, and repeatable way on an ongoing basis. Thus, insertion of this technology in the 'Transport Canada' and 'Organizational Processes' nodes is considered to have a potential impact.

• ASMM-3: Aviation Performance Measuring System (APMS) Tools

APMS is an integrated suite of tools to facilitate the implementation of routine flight-data analyses within each of the air-service providers. APMS develops and documents the software and procedures for data management and analyses of Flight Operational Quality Assurance data that enable users easily to interpret implications in safety and efficiency of flight.

Similar to ASMM-2, this technology aims to implement proactive management by monitoring and analyzing large flight- recorded databases on a continuous basis. The objective is to develop tools that convert data into information, merge databases, and create visualization capabilities by extending APMS to new applications. Therefore, insertion of this element in the 'Organizational Processes' and 'Transport Canada' nodes is expected to have an impact.

• ASMM-4: Performance Data Analysis & Reporting System (PDARS) Tools

This technology element provides the capability to collect and process Air Traffic Control (ATC) operational data; compute quantitative operational performance measures on a regular basis relating to system safety, delay, flexibility, predictability and user accessibility; conduct causal analyses and operational problem identification and analyses; archive performance statistics and basic operational data for use in research, development and planning studies.

The objective here is to monitor performance metrics continuously to enable the implementation of a policy of proactive NAS management. Extending APMS concepts and approaching to the ATC environment is proposed. Iterative evaluation by ATC users is expected to improve the measurement and tool requirements. Thus, insertion of ASMM-4 would have a potential impact upon the 'Transport Canada' node.

• AM-1: Next Generation Crash Analysis Codes

AM-1 involves the development of advanced analysis modules such as Finite Element Modeling (FEM) elements in a functionally integrated suite of dynamic crash analysis software. The purpose of this software is to make the design process more efficient, integrated and standardized and allow certification of crashworthiness as a system rather than by individual component testing.

Since AM-1 concerns crash analysis codes, it is considered to be relevant to the 'LOC' node in the influence diagram.

• AM-2: Energy Absorbing Seats, Restraints and Structures

AM-2 deals with development of designs for improved energy absorbing seats, restraints, and energy absorbing aircraft structures to improve survivability in accidents.

The objective in developing this element is to improve structural designs that help reduce crash loads and to develop human energy criteria for use in system design and ways to interpolate crash criteria data for a specific aircraft. Insertion of this technology element into the LOC node is expected to have an impact on the relative risk.

• AM-4: Next Generation Crashworthiness Design Guidelines

AM-4 involves the development of a handbook containing test results of materials tested and design guidelines for their use.

The objective of this element is to develop a Transport Crash Design Guide to target issues specific to the transport industry and to reduce the difficulty of validating crashworthiness. Similarly, AM-4 is considered to have a potential impact on the LOC node.

• WxAP-1: Cockpit Weather System Technologies for Enhanced Situational Awareness & Decision Making

WxAP-1 concerns the development of substantiated aviation weather information system guidelines for flight deck user interface and for operational use.

The objective is to develop weather presentations that are easy for the users to understand, interpret, and act on. This technology is considered to have a potential impact on the 'Decision Errors' node.

• WxAP-3: Weather Information Datalink Systems Technologies for Ground-to-Air Dissemination

WxAP-3 involves the development of datalink system and architecture guidelines supporting the transfer of weather information from ground-to-air. The targeted problem is the poor dissemination of weather information to the flight deck. This technology element is considered to have an impact on the 'Decision Errors' node.

3.6 Scenario Analysis

Combinations of new technologies could have a greater effect on risk reduction than the insertion of a single technology. Hence, it is important to evaluate several scenarios, each consisting of a risk assessment for different technologies and groups of technologies.

First , the risk is evaluated for the Baseline Scenario, in which there are no technology insertions. Then the risk is computed again, but for the opposite scenario, which involves the insertion of all relevant technologies and for others with the new technologies introduced singly. Additional calculations for each scenario with technology insertions include Absolute Percentage Risk Decrease and Relative Percentage Risk Decrease. Absolute % Risk Decrease is calculated as the difference in Risk Intensity Level from the baseline scenario, and Relative % Risk Decrease is calculated as (Absolute % Risk Decrease)/(Risk Intensity Level of baseline). The following is a summary of scenarios that have been generated based on the Air Ontario Flight 1363 case.

Figure 7. Preliminary Risk Assessments for the Ontario Flight 1363 Case

3.7 Software Interface

It is anticipated that once development of the ASRM software has been completed, it could have a wide user group. In addition to experienced NASA technicians, the software, with customizations, could be marketable to airline personnel.

In order to appeal to a larger market, however, the software should be flexible and easy to use. Hence, a user-friendly interface is being developed using the Visual Basic programming language. The software supplies the user with a multitude of results after the click of just a few buttons.

The interface will provide the user with the option of assessing risk for major categories of accidents including Maintenance, Loss of Control (LOC), and Controlled Flight into Terrain (CFIT), for example. Each of these categories will in turn have several cases built in, each of which is based on a past accident and its associated NTSB report. Figure 8 shows a sample display from which the user can choose a case to evaluate.

Figure 8. ASRM screenshot - Scenario Generator

Once the user has chosen a case to evaluate, he or she can then begin to generate scenarios by checking combinations of technologies and compiling. The form will show Risk Intensity Level of the generated scenario, Absolute % Risk Decrease, and Relative % Risk Decrease as discussed previously. This form with a sample scenario is shown in Figure 9:

Figure 9. ASRM screenshot - Evaluation Form

From this form, the user will also be able to view an Excel spreadsheet summary that captures the results from each scenario generated.

5.0 CONCLUSION

The analytical approach used in this research is a systematic method for modeling aircraft accidents and assessing risk reduction. It uses but is not confined to the HFACS taxonomy as a baseline framework to depict causal factors, and it combines statistical data with judgment of subject matter experts to obtain reasonable estimates of probabilities.

The Air Ontario 1363 case study demonstrates the importance of considering not only errors made by pilots and the cabin crew but also situational (e.g. weather), supervisory, and organizational factors.

Future work for this research involves the model-building, analysis, and probability elicitation of case studies in other accident categories such as Controlled Flight into Terrain (CFIT) and Runway Incursion.

6.0 Acknowledgments

Many parts of this research are done in collaboration among research team members. For help with research work discussed in this paper, the authors acknowledge the following Industrial and Systems Engineering graduate students: Muhammad Jalil, Erim Kardes, Ram Kuturu, and Ahmet Oztekin.

7.0 References

Maurino, D.E., J. Reason, N. Johnston and R.B. Lee (1995). Beyond Aviation Human Factors: Safety in High Technology Systems, Ashgate Publishing Limited, United Kingdom.

Luxhøj, James T. and Michele Maurino, "An Aviation System Risk Model (ASRM) Case Study: Air Ontario 1363," The Rutgers Scholar, Vol. 1 (2001), http://rutgersscholar.rutgers.edu.

NASA Product Dictionary (2002), February.

Reason, J. (1995). "A System Approach to Organizational Error," Ergonomics, 38, 1708-1721.

Shappell, S., and D. Wiegmann (1996). "U.S. Naval Aviation Mishaps 1977-92: Differences between single- and dual-piloted aircraft," Aviation, Space, and Environmental Medicine, 65-69.

Shappell, S., and D. Wiegmann (2000). "The Human Factors Analysis and Classification System (HFACS)," Office of Aviation Medicine, Federal Aviation Administration, 1-14.