The role of a pilot today is different compared to 50 years back. These days, most commercial aviation is in large part automated – an innovation that has decreased the number of accidents drastically and made aviation one of the safest ways to travel. In one million take-offs, the average number of accidents is less than one. [1]
But while automation has made commercial airline travel a much safer mode of transportation, the pilots remain ultimately responsible for the safety of hundreds of passengers and crew members. According to research, 60 to 80 percent of all aviation incidents and crashes can be attributed to human involvement. [2]
Since the main responsibility of the pilots has gone from manually steering the aircraft to monitoring its instrument, it is not a far reach to assume many of these accidents are due to an over-reliance on automation systems and insufficient monitoring. In fact, the National Transportation Safety Board (NTSB) has found that the common factor of over 80 percent of major flight-crew-involved accidents between 1978 and 1990 was inadequate monitoring. [3]
The ability to keep alert and attentive while monitoring the cockpit’s flight instruments has become one of the most important skills of a good pilot. Just a short lapse of attention could make the difference between life and death 10 000 meters up in the air.
In the last few years, eye tracking technology has made its impact on the automotive industry. Gaze scanning has gone from being an exciting idea, an area of research, to being installed in new car models all over the world and taking its place as one of the most important technologies for road safety.
Right now, a lot is pointing to a similar development in the aviation industry. Is eye tracking technology about to grow wings and move on to saving lives up in the air?
Where are we today?
We may finally be on the brink of installing eye tracking systems in airplanes, but the possibilities of using eye tracking to monitor pilots have been long known. For more than half a century, researchers have investigated ways to supervise the training and performance of pilots in order to increase aviation safety standards. From pioneer studies in the 1950’s, where early eye tracker systems were tested in flight simulators, to contemporary experiments where modern eye trackers are used in real in-air cockpits.
Since 2003, Smart Eye has been involved in research projects conducted by some of the world’s most established research institutes. Ulf Löfberg, Regional Sales Manager at Smart Eye, is well aware of the challenges of developing products designed especially for pilot monitoring.
– Much has already happened, these aren’t new things that are being discussed. Our systems have been used in the aviation industry for a long time. We have big clients on both sides of the Atlantic using eye tracking in their research. And at the moment, we have a very good technical foundation. We have eye tracking systems that can deliver data about the pilot’s gaze behavior, says Ulf Löfberg.
In terms of aviation, there are two main areas where eye tracking solutions like Smart Eye’s have the potential to make a real difference: pilot training and in-air pilot monitoring.
Pilot training
Within the next two decades, global demand for commercial airline pilots is predicted to double. By 2035, almost twice as many people will travel by plane compared to 2016, according to the International Air Transport Association, with the number of passengers leaping from 3.8 billion to 7.2 billion. [4] Especially Asia can expect a shortage of pilots as a result of increased airline travel. In China alone, 66 new airports are currently being built. Meanwhile, only about 15-25 percent of all pilot recruits come out on the other side of training as licensed commercial pilots. This means the number of pre-entry assessments passed must be about five times as high as the number of pilots required.
– Even if you need 30 000 pilots, you might need to screen hundreds of thousands recruits, says Ulf Löfberg.
This is where eye tracking becomes an important part of future pilot training. Tracking the recruits’ gaze patterns can help determine their potential, adding the required level of efficiency to the upcoming mass-recruitment of pilots.
Once suitable recruits have been identified, eye tracking can drastically improve the training of the new pilots. In a typical training scenario, the instructor sits behind the recruit, studying their behavior as they are navigating a simulation of an in-air cockpit. This normally limits the instructor to seeing the pilot’s head from behind, with no information about where they are directing their gaze. It is then up to the pilot to describe their own behavior.
– It becomes a subjective assessment, explains Ulf Löfberg. Under pressure it is hard to correctly repeat what you actually did and at what time. We have had instructors tell us that the pilots are giving incorrect descriptions.
With an eye tracking system installed in the training simulator, the instructor can easily determine which point of the cockpit the pilot is directing their gaze towards. This data can also be saved, in order for the instructor and the recruit to be able to review the pilot’s performance afterwards. By tracking gaze, both the instructor and the pilot recruit are provided with documentation to make training more accurate and more efficient.
Another process in which eye tracking can be a useful tool for instructors is during the assessment of pilots switching from one airline to another. In this situation, the technology can be applied in the same way as during pilot training. With eye tracking, the instructors can use the data to compare the skills of the pilot with the new airline’s routines for pilot performance.
In-air pilot monitoring
Today, eye tracking systems that ensure driver attentiveness are on their way to becoming an established part of the automotive industry. This has prompted speculation about whether, or when, we will see the same developments in the aviation industry.
It is clear that the possibilities of aviation appeal to many people. We enthusiastically jump on planes craving new experiences, and we can literally be on the other side of the planet in just 24 hours. Long-haul flights open a door to the world and are becoming more and more common. But for much of that time, the role of the pilot is relatively passive: monitoring the aircraft’s instruments while leaving most of the flying to the automation system.
Installed in a cockpit, an eye tracking system could prove important for the safety of passengers, crew and aircraft alike. By tracking a pilot’s gaze patterns, pupillometry and eyelids, the system could first of all determine if there is a pilot in the cockpit who is alive, awake and alert enough to fly the aircraft.
By tracking the pilot’s gaze, an advanced system could even predict which decisions the pilot is about to make and adapt to the crew’s state of mind and expected course of action. Should a pilot fail to pay attention to an important flight instrument, a gaze tracker could not only recognize this, but direct the pilot’s attention by highlighting important areas on the instrument board. Eventually, the aircraft could be able to adjust its automation settings according to the pilot’s attentional state and upcoming decision-making.
The future of eye tracking and aviation
How long will it take until eye tracking is a natural part of both commercial aviation and pilot training? According to Ulf Löfberg, that decision is largely up to the instructors and the pilots, seeing as it us up to them to accept and implement the technology.
– I think we will get to see pilot monitoring systems, that I am very sure of. Technically, it shouldn’t take long. We already have systems being installed in the automotive industry. It isn’t difficult to find a technical solution for the aviation industry as well. But convincing the industry to accept the solution and implement it is not as easy or quick, says Ulf Löfberg.
In short, he explains, on-going discussions involve how the knowledge gained from the head and eye tracking analysis will be used. It is a question of whether the data on current pilot performance can be used solely to assist the pilots, or if it will be stored and used to document pilot errors.
The reality of implementing new technological solutions in aircrafts can be a drawn-out process. First of all, all equipment that is to be applied to airborne planes needs to be certified. This means eye tracking systems need to be officially approved as safe, fireproof, non-disruptive of the aircraft’s instruments, and resistant to a wide variation of temperatures and vibrations. However, this only applies to equipment installed in in-air planes. A system installed in a training simulator does not require the same certification.
– In terms of certification, it’s probably easier to implement gaze tracking in a simulated training environment. But it might be more important to monitor in-air cockpits. Authorities and airlines might push harder for that to happen, says Ulf Löfberg.
Secondly, for an eye tracking system to be able to handle complex aviation environments, it needs to use several cameras. A single-camera system would not only be unable to track the full cockpit environment, it would also miss out on the important interaction between the pilots. As human error is statistically the number one cause of aviation accidents, being able to identify insufficient communication between pilots would be vital for an effective gaze tracking system.
– To be able to cover two pilots and an entire cockpit with all of its instruments, you need to track more surfaces and you need to track larger surfaces. Multi-camera systems are the only option, simply because you need to be able to track a large, complex environment, says Ulf Löfberg.
But he also states that this is not just a question of developing technology. It is just as much about changing the way the aviation industry views eye tracking, and making airlines, pilots and instructors alike understand how much pilot training and flight safety would benefit from eye tracking technology.
– The discussion about eye tracking needs to change, from discussing eye tracking in a research context to implementing it on a wide basis for the benefit of safety. Eventually, most airlines will demand better security systems in the aviation industry.
[1] Boeing. Statistical Summary of Commercial Jet Airplane Accidents Worldwide Operations, 1959–2014; Aviation Safety, Boeing Commercial Airplanes: Seattle, WA, USA, 2015. Available online: https://www.boeing.com/ news/techissues/pdf/statsum.pdf
[2] Shappell, S.; Detwiler, C.; Holcomb, K.; Hackworth, C.; Boquet, A.; Wiegmann, D.A. Human error and commercial aviation accidents: An analysis using the human factors analysis and classification system. Hum. Factors 2007, 49, 227–242. [CrossRef] [PubMed]
[3] National Transportation Safety Board (NTSB). A Review of Flightcrew-Involved, Major Accidents of U.S. Carriers, 1978 through 1990, Safety Study NTSB/SS-94/01; NTSB: Washington, DC, USA, 1994.
