Application Areas that will be investigated within the Mindtooth project.


In this section, a general explanation of the application areas that will be investigated within the Mindtooth project is provided.

1.1.1 Automotive Simulators
ITCL has developed advanced car simulators, hosted at their facilities, that integrate a highly realistic simulation environment, characterized by a large variety of contexts (urban, country side, highway) and the possibility of interacting with other urban actors (pedestrians, motorbikes and bikes) virtually simulated through AI within the scenario or played through additional simulators hosted at ITCL itself. Therefore, the ITCL simulators are highly versatile; they allow the integration and customization of several features and technologies, therefore paving the way for different experimental trials.

Mindtooth will be tested and integrated within ITCL simulators with the following aim:
• To provide a forefront “research & service” tool to investigate mental causes of specific drivers’ behaviour, targeting research organisations specialising in traffic safety. Nowadays, apart from research, the main ITCL simulators business target relies on driving schools. Thanks to the Mindtooth system, ITCL aims at enhancing its simulators to improve training quality and efficiency through the integration of untapped data-driven insights into training. On secondary level, providing an innovative service to their industrial customers where the study of the brain response during training or even operating risky equipment or operations could be interesting and useful.

Because of the concurrent relevant field of autonomous driving, the final Mindtooth system would become a highly interesting tool to enable new channels of interaction between the driver (advanced Human Machine Interaction - HMI - application), i.e. by using neurometrics of his/her mental state, and the car. This aspect will be not directly investigated along Mindtooth, since it currently does not represent the core business of ITCL. However, because of its potential impact in automotive industry and market, any activity to encourage at least its final exploitation also in this domain will be taken into consideration.

Car simulator is essentially based on a typical real car setup (seat and dashboard with steering wheel, gearshift and all the common commands), while the virtual environment is reproduced through three screens, therefore without using any VR headset that would make wearing EEG sensors difficult (Figure 7). 

Figure 1. 

ITCL automotive simulator that will be employed during the Mindtooth project  

Therefore, EEG recording will not be performed during real driving conditions but only at simulators, coherently with the Mindtooth intended use by ITCL.
The desirable future employment of Mindtooth in public contexts such as research pilots and driving schools with likely high rates of usage per day, by people with different “head shapes”, implies particular attention to the adaptability and robustness of Mindtooth device. Furthermore, because of the presence of the seat headrest, any encumbrance in the posterior side of the head should be avoided. Also, concerns regarding the possibility of properly cleaning and disinfecting the device, including for example disposable electrodes, should be handled.    

1.1.2 Pilots’ training
In Aviation, the safety of people relies on the work and efficiency of one or more operators. For example, the passengers’ safety depends directly on the performance of the Pilot(s). In such a context, a human error could have serious and dramatic consequences. In general, Human Factors (HFs) have consistently been identified as one of the main factors—in a high proportion—of all workplaces accidents. In particular, it has been estimated that up to 90% of accidents in Aviation exhibits HF as principal cause.
The ambitious goal that UrbeAero has set itself is to integrate the Mindtooth system into its school services (Figure 8) to further improve the trainees assessment and training program management. In fact, Instructors usually use a very elaborate evaluation grid of a series of behaviors that can tell if that competence has been "absorbed" by the pilot. Crossing the observed behavior with a "how often" and "how well" matrix, we obtain the degree of "absorption" of that competence (Figure 9). At the moment, the Assessment and Training Matrix relies on the Instructor subjective evaluation. Also, the main and most expensive problem in economic terms for airlines is the standardization of the judgements given by Instructors. Consequently, if we could demonstrate of being able to objectively measure the acquisition of those core competencies, it would be a real breakthrough in the Aviation field. A direct foreseen business linked to this activity is related to the periodical check that professional pilots have to go through every 6 and 12 months, respectively the Operator Proficiency Check (OPC) and the Licence Proficiency Check (LPC). These two checks are mandatory for both the pilots and airline companies to keep the flight license, find out if the pilots still have the proper skills and competences to handle with specific flight conditions, and finally ensure high safety all around the World by analysis the statics coming from those checks.   

Figure 2. 

URBE Aero simulator that will be employed during the Mindtooth project  

Figure 3. 

Assessment and Training Matrix for competencies check.

1.1.3 Industrial workers

Figure 4. 

Industrial workers application.

According to the International Labor Organization , every 15 seconds, 151 workers in the world have a work-related accident. The global number of non-fatal occupational accidents reaches a staggering number of 317 million, annually. Even more concerning, 321,000 people die each year from occupational accidents. Work accidents remain a huge challenge for many industrial sectors, despite safety regulations and procedures. Accommodating the safety of humans is a key factor in ensuring industrial efficiency and effectiveness. In recent year, we see an increase in the availability of sensing, monitoring, and interaction technologies that promote not only for better management of machinery (namely traditional IoT), but also to the more adequate management of human resources, catering a variety of aspects. This may include assurance of their health and wellness conditions through the use of wearables and connected PPE, augmentation of their physical and mental limitations via the use of technologies such as voice-based personal assistants, assistive drones for better areal inspections, exoskeletons that help with the handling of heavy loads, etc. Such technologies help boosting human productivity, both physically, and mentally.
IBM research lab currently explores new industrial applications that can be boosted by emerging
human centric AI technologies such as commercial wearables, leading to innovative product offerings such as the Maximo Worker Insights (MWI) system developed to attain better worker safety.
IBM aims to integrate Mindtooth into the industrial area, testing its possible viability as an underlying instrumentation for a variety of industrial scenarios and simplifying its technical integration with existing offerings such as the MWI. The foreseen application scenarios where it could be useful this integration are:
(1) Real-time work diligence - keeping track of workers (mental) conditions by their supervisor while deployed to pursue their daily work activities.
(2) Remote operation – similar to (1) but where workers are deployed to remote sites.
(3) On the job training – in service ad hoc learning from doing, generally problem or technology oriented, with an opportunity to try new skills, apply previously acquired knowledge and learn from mistakes.
(4) Field coaching – similar to (3) but where a worker or a subordinate is assigned a more experienced colleague to have the latter intervene, give feedback, or sometimes even demonstrate and educate the worker when technical gaps arise.
(5) Communication tool (Objective tracking and communication of mental performance) – allowing supervisor to explicitly and quantitatively relate to the mental abilities and performances of their subordinates, keep logs of such abilities over time, give quantitative feedback, and set periodic goals about these abilities.

The validation effort in Mindtooth will aim at instantiating scenarios as the ones listed above. This may include an attempt to integrate the Mindtooth system with the MWI application in the context of (1 and 2) as a input tool that would be used to prevent accidents using data gathered online from wearable wireless sensors while extending its functionality to also accommodate mental conditions, and maybe even more relevant in the context of (3) and (4) where more novice workers need to be trained about the acquisition of new qualifications while in the field.

Figure 5. 

Possible integration with IBM Maximo Worker Insight - Mindtooth as an input instrument

Wearable and embedded sensors are making it possible for workers to be monitored within their surroundings to prevent injury from falls, overexertion, heavy machinery – the list of what wearables allow us to prevent is a lengthy one. These sensors can help prevent accidents by monitoring employee fatigue levels and suggesting rest periods, but could also serve as a means of communication between supervisors and their team members as articulated in (5). So far, such conditions have been monitored mainly from a physical / physiological perspective. With the Mindtooth system, a more complete protection could be achieved. Complementary to the above, it may also be useful to consider the Mindtooth system as a reference measurement instrument. For instance, fatigue determination could be inferred at various degrees of quality using a variety of surrogate measurements such as heart-rate variability or self-reporting by the workers. With the availability and EEG device, such measurements could be validated where EEG is used to provide the ground truth.

Beyond the intended use-case application (monitoring industry workers’ mental states to detect safety-related conditions such as high-level of fatigue or stress), other examples may include advancing solutions such as Watson Personal Assistant , currently serviced mainly via voice-based interaction. Mindtooth would employ advanced AI analytics to facilitate a deeper level of insights to predict for instance the level of alertness of drivers at the wheel, or aiding with the optimization of pilot’s shift planning using historical data about individual pilots neurometrics, thus minimizing the likelihood for a human error to occur. Mindtooth prototype being used in a working session, as it will be tested and integrated with the worker safety and wellness framework (IBM). Such aspects become even more prominent in the more contemporary pandemic time, with the future of the industrial work environment foreseen to become significantly more entangled with the context of remote operation, requiring workers control certain equipment remotely, sometime directly from the home environment, or in the context of dispatching worker to remote facilities where they are anticipated to execute their job independently.

Overall, IBM will aim to adapt the Mindtooth system to ensure its compatibility with Human-centric IoT market-grade solutions such as MWI and Watson AI services offering while aiming to instantiate the above scenarios.  

Create a free website - Details here