Leading provider of products and services for system-level simulation to partner with leading manufacturer of high-end eye tracking technology to combine high-end, non-intrusive, eye tracking with best-in-class advanced automotive simulation.
Tag: Eye Tracking Applications
-
Webinar: Explaining Remote 3D Head and Eye Tracking in Aviation
Why watch?
3D tracking, remote tracking, multi-camera systems – these are terms we keep tossing around, but it’s not always clear how features like these relate to aviation research. In this webinar, we summarize the knowledge and experience we’ve gained from over 20 years in the industry in order to introduce you to the unique features of the remote 3D multi-camera eye tracking method.
In this webinar you’ll learn:
- How your specific area of research can benefit from remote eye tracking
- The ins and outs of remote 3D head and eye tracking – and how we truly believe it will shape the future of aviation
- How remote head and eye tracking is already being used in flight simulators, using real-life examples from aviation giants like NASA and ISAE Supearo
-
Join our webinar with iMotions!
Webinar: Advanced Eye Tracking with Smart Eye & iMotions
Join our free webinar this week where we will discuss and demonstrate several use cases centering on how Smart Eye’s suite of advanced eye trackers can be applied to research in usability, automotive, and aviation using the iMotions software. Register here! -
Product Launch: Say hi to the AI-X!
Smart Eye AI-X
GET YOUR VIEWERS PERSPECTIVE
READ MORE HERE -
BLOG: “2010-2020: A decade of innovation at Smart Eye”
We can conclude that 2019 has been a significant one for Smart Eye. It’s been a year full of great innovations and success, as well as the incredible 29 design wins Smart Eye has been rewarded over the course of 2019 alone. Now we look forward to the dawn of a brand-new decade with even greater innovations and success.
From crisis to serial contracts
When thinking back on the last time we were entering a new decade, we’re struck by just how much that has happened these last ten years. By the end of 2009, Smart Eye was still dealing with the aftermath of the Lehman crash. The global financial crisis of 2008 hit the car industry hard. Suddenly, car manufacturers’ main priority became surviving until the next quarter, and the incorporation of new, innovative technology had to wait. This forced Smart Eye to take a step back… read the full blog
-
2010-2020: A Decade of Innovation at Smart Eye
We can conclude that 2019 has been a significant one for Smart Eye. It’s been a year full of great innovations and success, as well as the incredible 27 design wins Smart Eye has been rewarded over the course of 2019 alone. Now we look forward to the dawn of a brand-new decade with even greater innovations and success.
From crisis to serial contracts
When thinking back on the last time we were entering a new decade, we’re struck by just how much that has happened these last ten years. By the end of 2009, Smart Eye was still dealing with the aftermath of the Lehman crash. The global financial crisis of 2008 hit the car industry hard. Suddenly, car manufacturers’ main priority became surviving until the next quarter, and the incorporation of new, innovative technology had to wait. This forced Smart Eye to take a step back from the automotive industry, and we entered 2010 as a company solely focusing on research instruments. At that point, we could not know which car manufacturers would survive the crisis or if we would even get the chance to deliver our technology to the automotive industry again.
But before long, things started to turn around. After about a year, in late 2010, requests from automotive clients began dropping in again. And by the end of 2011, the pendulum had swung, and the car industry was back on track with a renewed vigour.
After several years of hard work and occasional uncertainty, we finally received our first serial contract in 2015. And during the next few years, we were able to turn that one contract into 57 contracts – including car manufacturers and industry giants on almost every continent. From then on, our focus was largely placed on fulfilling the trust put in us and making sure we would eventually see more and more cars with Smart Eye’s technology on the roads.
Eye tracking in the 2020’s
With everything that has happened over the course of these last ten years, the natural development for Smart Eye has been the thoroughly expansive phase we are still going through. In 2009, we were less than 15 dedicated people working at Smart Eye. Today, we are approximately 150 talented women and men around the globe.
There is no denying that these past ten years have paved the way for a very exciting future. Of the 57 car models the 2010’s has landed us; many are set to go into mass production during the first two years of the 2020’s. And during the next ten years, we hope to see the idea we have worked with from the very beginning become a reality.
Furthermore, due to regulations recently approved by the European Union, there will not be a new car model launched on the European market from 2022 that does not feature advanced safety systems including driver drowsiness and attention warning, as well as advanced driver distraction warning. Meanwhile, similar regulations are being adapted in China. The Euro NCAP are even adjusting their rating system to include driver monitoring as a vital safety measurement. By 2030, we hope to be able to point to mass-produced cars all over the world where Smart Eye’s technology is saving lives.
Unknown to ubiquitous – the development of eye tracking technology
Over the last decade, some of us have had the pleasure to witness first-hand how eye tracking has gone from being a relatively unknown concept to something more and more ubiquitous. Ten years ago, very few people knew what eye tracking was. We were starting to get used to people telling us that while they understood the concept, they could not understand what they were supposed to use the technology for. That is a question we barely ever hear anymore.
These days, you can find eye tracking or facial recognition almost everywhere. Smartphones, computers, gaming equipment, disability aids, vehicles, VR – the list goes on. And pretty soon, we will most likely be able to add airplanes, buses and trains to that line-up. Today, almost everyone is realizing that a greater understanding for a user’s mind and intentions is a vital part of creating a better user experience – and therefore incredibly valuable.
Of course, this is what convinced our very first colleagues and passionate eye tracking advocates at the very beginning. We never doubted that we were dedicating our careers to something useful and valuable.
And it may have taken some time, but during these last ten years, the rest of the world seems to have caught on to the benefits of eye tracking. Now, we can feel quite certain that we have not received our last car contract, car manufacturer or design win.
Since 2009, Smart Eye has not just grown to involve ten times as many people, we have become an integrated wheel in the great machinery that is the global car industry of today. The last time we were entering a new decade, we were merely looking in, trying to catch a glimpse of what it might be like to be on the inside. This time, we have a crystal-clear view.
-
How Eye Tracking Will Take Aviation to New Heights
The role of a pilot today is different compared to 50 years back. These days, most commercial aviation is in large part automated – an innovation that has decreased the number of accidents drastically and made aviation one of the safest ways to travel. In one million take-offs, the average number of accidents is less than one. [1]
But while automation has made commercial airline travel a much safer mode of transportation, the pilots remain ultimately responsible for the safety of hundreds of passengers and crew members. According to research, 60 to 80 percent of all aviation incidents and crashes can be attributed to human involvement. [2]
Since the main responsibility of the pilots has gone from manually steering the aircraft to monitoring its instrument, it is not a far reach to assume many of these accidents are due to an over-reliance on automation systems and insufficient monitoring. In fact, the National Transportation Safety Board (NTSB) has found that the common factor of over 80 percent of major flight-crew-involved accidents between 1978 and 1990 was inadequate monitoring. [3]
The ability to keep alert and attentive while monitoring the cockpit’s flight instruments has become one of the most important skills of a good pilot. Just a short lapse of attention could make the difference between life and death 10 000 meters up in the air.
In the last few years, eye tracking technology has made its impact on the automotive industry. Gaze scanning has gone from being an exciting idea, an area of research, to being installed in new car models all over the world and taking its place as one of the most important technologies for road safety.
Right now, a lot is pointing to a similar development in the aviation industry. Is eye tracking technology about to grow wings and move on to saving lives up in the air?
Where are we today?
We may finally be on the brink of installing eye tracking systems in airplanes, but the possibilities of using eye tracking to monitor pilots have been long known. For more than half a century, researchers have investigated ways to supervise the training and performance of pilots in order to increase aviation safety standards. From pioneer studies in the 1950’s, where early eye tracker systems were tested in flight simulators, to contemporary experiments where modern eye trackers are used in real in-air cockpits.
Since 2003, Smart Eye has been involved in research projects conducted by some of the world’s most established research institutes. Ulf Löfberg, Regional Sales Manager at Smart Eye, is well aware of the challenges of developing products designed especially for pilot monitoring.
– Much has already happened, these aren’t new things that are being discussed. Our systems have been used in the aviation industry for a long time. We have big clients on both sides of the Atlantic using eye tracking in their research. And at the moment, we have a very good technical foundation. We have eye tracking systems that can deliver data about the pilot’s gaze behavior, says Ulf Löfberg.
In terms of aviation, there are two main areas where eye tracking solutions like Smart Eye’s have the potential to make a real difference: pilot training and in-air pilot monitoring.
Pilot training
Within the next two decades, global demand for commercial airline pilots is predicted to double. By 2035, almost twice as many people will travel by plane compared to 2016, according to the International Air Transport Association, with the number of passengers leaping from 3.8 billion to 7.2 billion. [4] Especially Asia can expect a shortage of pilots as a result of increased airline travel. In China alone, 66 new airports are currently being built. Meanwhile, only about 15-25 percent of all pilot recruits come out on the other side of training as licensed commercial pilots. This means the number of pre-entry assessments passed must be about five times as high as the number of pilots required.
– Even if you need 30 000 pilots, you might need to screen hundreds of thousands recruits, says Ulf Löfberg.
This is where eye tracking becomes an important part of future pilot training. Tracking the recruits’ gaze patterns can help determine their potential, adding the required level of efficiency to the upcoming mass-recruitment of pilots.
Once suitable recruits have been identified, eye tracking can drastically improve the training of the new pilots. In a typical training scenario, the instructor sits behind the recruit, studying their behavior as they are navigating a simulation of an in-air cockpit. This normally limits the instructor to seeing the pilot’s head from behind, with no information about where they are directing their gaze. It is then up to the pilot to describe their own behavior.
– It becomes a subjective assessment, explains Ulf Löfberg. Under pressure it is hard to correctly repeat what you actually did and at what time. We have had instructors tell us that the pilots are giving incorrect descriptions.
With an eye tracking system installed in the training simulator, the instructor can easily determine which point of the cockpit the pilot is directing their gaze towards. This data can also be saved, in order for the instructor and the recruit to be able to review the pilot’s performance afterwards. By tracking gaze, both the instructor and the pilot recruit are provided with documentation to make training more accurate and more efficient.
Another process in which eye tracking can be a useful tool for instructors is during the assessment of pilots switching from one airline to another. In this situation, the technology can be applied in the same way as during pilot training. With eye tracking, the instructors can use the data to compare the skills of the pilot with the new airline’s routines for pilot performance.
In-air pilot monitoring
Today, eye tracking systems that ensure driver attentiveness are on their way to becoming an established part of the automotive industry. This has prompted speculation about whether, or when, we will see the same developments in the aviation industry.
It is clear that the possibilities of aviation appeal to many people. We enthusiastically jump on planes craving new experiences, and we can literally be on the other side of the planet in just 24 hours. Long-haul flights open a door to the world and are becoming more and more common. But for much of that time, the role of the pilot is relatively passive: monitoring the aircraft’s instruments while leaving most of the flying to the automation system.
Installed in a cockpit, an eye tracking system could prove important for the safety of passengers, crew and aircraft alike. By tracking a pilot’s gaze patterns, pupillometry and eyelids, the system could first of all determine if there is a pilot in the cockpit who is alive, awake and alert enough to fly the aircraft.
By tracking the pilot’s gaze, an advanced system could even predict which decisions the pilot is about to make and adapt to the crew’s state of mind and expected course of action. Should a pilot fail to pay attention to an important flight instrument, a gaze tracker could not only recognize this, but direct the pilot’s attention by highlighting important areas on the instrument board. Eventually, the aircraft could be able to adjust its automation settings according to the pilot’s attentional state and upcoming decision-making.
The future of eye tracking and aviation
How long will it take until eye tracking is a natural part of both commercial aviation and pilot training? According to Ulf Löfberg, that decision is largely up to the instructors and the pilots, seeing as it us up to them to accept and implement the technology.
– I think we will get to see pilot monitoring systems, that I am very sure of. Technically, it shouldn’t take long. We already have systems being installed in the automotive industry. It isn’t difficult to find a technical solution for the aviation industry as well. But convincing the industry to accept the solution and implement it is not as easy or quick, says Ulf Löfberg.
In short, he explains, on-going discussions involve how the knowledge gained from the head and eye tracking analysis will be used. It is a question of whether the data on current pilot performance can be used solely to assist the pilots, or if it will be stored and used to document pilot errors.
The reality of implementing new technological solutions in aircrafts can be a drawn-out process. First of all, all equipment that is to be applied to airborne planes needs to be certified. This means eye tracking systems need to be officially approved as safe, fireproof, non-disruptive of the aircraft’s instruments, and resistant to a wide variation of temperatures and vibrations. However, this only applies to equipment installed in in-air planes. A system installed in a training simulator does not require the same certification.
– In terms of certification, it’s probably easier to implement gaze tracking in a simulated training environment. But it might be more important to monitor in-air cockpits. Authorities and airlines might push harder for that to happen, says Ulf Löfberg.
Secondly, for an eye tracking system to be able to handle complex aviation environments, it needs to use several cameras. A single-camera system would not only be unable to track the full cockpit environment, it would also miss out on the important interaction between the pilots. As human error is statistically the number one cause of aviation accidents, being able to identify insufficient communication between pilots would be vital for an effective gaze tracking system.
– To be able to cover two pilots and an entire cockpit with all of its instruments, you need to track more surfaces and you need to track larger surfaces. Multi-camera systems are the only option, simply because you need to be able to track a large, complex environment, says Ulf Löfberg.
But he also states that this is not just a question of developing technology. It is just as much about changing the way the aviation industry views eye tracking, and making airlines, pilots and instructors alike understand how much pilot training and flight safety would benefit from eye tracking technology.
– The discussion about eye tracking needs to change, from discussing eye tracking in a research context to implementing it on a wide basis for the benefit of safety. Eventually, most airlines will demand better security systems in the aviation industry.
[1] Boeing. Statistical Summary of Commercial Jet Airplane Accidents Worldwide Operations, 1959–2014; Aviation Safety, Boeing Commercial Airplanes: Seattle, WA, USA, 2015. Available online: https://www.boeing.com/ news/techissues/pdf/statsum.pdf
[2] Shappell, S.; Detwiler, C.; Holcomb, K.; Hackworth, C.; Boquet, A.; Wiegmann, D.A. Human error and commercial aviation accidents: An analysis using the human factors analysis and classification system. Hum. Factors 2007, 49, 227–242. [CrossRef] [PubMed]
[3] National Transportation Safety Board (NTSB). A Review of Flightcrew-Involved, Major Accidents of U.S. Carriers, 1978 through 1990, Safety Study NTSB/SS-94/01; NTSB: Washington, DC, USA, 1994.
[4] https://www.iata.org/pressroom/pr/Pages/2016-10-18-02.aspx
-
From Eye Movement to Neural Network: An In-Depth Look to our Technology
At Smart Eye, every eye movement we record and algorithm we write are intended to increase our understanding of the most complicated system in the world: the human being. We work at the edge of technology but we realize that sometimes it is hard to understand the complexity of our technology. Therefore we have created this guide, to give you a better understanding of how we do things and to explain the complex process of understanding human behavior.
