Coming soon

AI is increasingly being integrated into production systems, but often without attention to the resilience and traceability of the processes. In an effort to achieve the Industry 4.0 vision, data-driven systems are being implemented quickly without evaluating their usability in a process-accompanying manner and monitoring their operation adaptively. Specifically, the complex structure of machine learning (ML) systems and the data basis limited to existing training data cause the following challenges: ▪ ML systems are supposed to make decisions for which their collected data basis is insufficient. Failure is often untraceable and interpreted as a weakness of the method. Incoming data are not classified and data sets are not balanced, but are used immediately for learning without analysis ▪ ML systems do not communicate how they arrive at decisions. Behavior is perceived as an untraceable black box. ▪ ML systems cannot be specifically influenced. Unlike conventional systems, systems such as artificial neural networks do not have parameters that can be meaningfully adjusted by operators of a production system during the process.

In the project ExplAIn, the paradigms of XAI are to be transfe-rred to ML systems in production. For this purpose, an XAI system is being developed that analyzes and controls ML systems and their database in a process-accompanying manner. The XAI system shall be able to retrofit existing ML systems in production as well as to develop systems that are designed by-design according to XAI. The XAI system is to comprise four successive services in order to overcome the problems observed in AI applications in industrial practice: 1. Train AI safely - It analyzes and maintains the database 2. Use AI safely - It analyzes set tasks based on the database 3. Understand AI - It traces decisions made back to the decisive fragments in the input 4. Adapt AI - It enables influence on system behavior

Data space analysis

Data space analysis involves checking the input data of the AI system.

First, all existing training data is displayed in a data space. Then known and unknown areas are marked.

The aim is to determine the degree of familiarity of a data point based on the marked areas.

If the system is used with a data point in the unknown zone, the system response must also be checked, as the reliability of the statement cannot be guaranteed.

After checking the system response, the new data point can be used for retraining the system.

The image on the right shows an example of a data room analysis for a welding process. For illustration purposes, only two dimensions of the data space are shown here. It can be clearly seen that an unknown zone has developed at the top right.