Sensor development using a confusion matrix

As a supplier of complete sensor solutions, STEINEL Solutions has many years of experience in developing and producing sensors in a wide range of industries. In addition to our experience, the successful implementation is also based on the methodology of the confusion matrix. An article about sensor developments using the confusion matrix.

Definition of the use case

The detection behaviour of a sensor application is crucial to the success of the product. The different application situations, known as use cases, determine the behaviour of the sensor and describe how the sensor should react in each case. Not only the ideal case is considered, but all environmental factors.

STEINEL Solutions has brought successful sensor solutions to market in many industries. As a result, various application situations and solutions, such as in the area of person detection, are already known and implemented. If a new application deviates from existing solutions or new use cases are added, a joint development process begins between the product owner and realisation partner.

The product owner is familiar with the industry-specific conditions for use and the environmental conditions of their product. They specify the requirements and the desired behaviour, and thanks to their experience, they can also describe unfavourable installation situations, interpret test results of the first prototypes and derive the necessary steps from them.
The realisation partner draws on its broad expertise to develop the optimal solution. This is usually an embedded system with sensors and actuators or possible communication interfaces.

An iterative process involving intensive validation of early functional models and more mature prototypes brings to light sensor optimisations and product improvements. Sometimes, use cases are only redefined during this phase. Before the project moves into industrialisation, the requirements are jointly agreed with the test descriptions on the basis of a test plan and verified point by point.

Design of the sensor characteristic using a confusion matrix

The design of the sensor characteristics is supported by the confusion matrix, which has its origins in machine learning models. The confusion matrix, also known as the error matrix, is used to clearly describe the recognition behaviour and the limits of sensors. The figure shows the general representation of a confusion matrix (own representation).

Using the example of a motion sensor, such as those used in the outdoor areas of buildings, the following confusion matrix describes a passive infrared (PIR) sensor. This isnot a real recording, but illustrative examples to help understand the matrix. It is often advisable to define the frequency of occurrence of the individual use cases as percentages with the product owner and set them to an acceptable value. For example, false negatives of up to 0.5% are tolerable .

Examples of how to understand the confusion matrix in English

True POSitives: These cases are usually defined quickly and reflect the actual function of the sensor. They are the situations that the sensor is supposed to detect. A motion sensor detects people walking by at a speed of >2km/h and switches the light on.

True NEGATIVE: At first glance, this seems obvious. Is it the opposite of True POSITIVE? Simply put, the motion detector should not detect a person in its detection area who is not in the room. But what about a seated person who is not moving? In the case of a motion detector, this is actually called True NEGATIVE. In practice, a distinction is made between presence sensors (typically in office areas) and motion sensors. This explains why the motion sensor, unlike the presence sensor, does not react to a seated person. On the other hand, it has a greater range and cheaper components.

Furthermore, there are the two fields of false positives (phantom detections) and false negatives (blindness). Both cases describe a malfunction of the sensor. The degree to which such malfunctions are permissible must be agreed between the product owner and the development partner. This demarcation is very individual and can have an influence on development and product costs. For example, another sensor technology may be required to reliably detect specific special cases. Whether the application accepts the additional price of the additional principle is ultimately based on the product owner's price-performance estimation.

Interaction of many fields of expertise in the implementation

In the technical implementation, many specialised fields come together in a sensor application: analogue design, digital filtering and signal processing, EMC immunity, power management, communication links, adjustment and calibration, etc. All these areas require the appropriate expertise, infrastructure and tools to design the adaptation to customer-specific solutions based on existing technology modules.

Employee who wrote the technical article in the EVM laboratory during sensor development

STEINEL has specialised in and become a market leader in the field of PIR, high-frequency and optical sensors. The company has extensive experience and resources in the development of hardware and firmware, PCB design, mechanical engineering, application testing, manufacturing and process expertise.

A sensor usually contains an analogue circuit. This already makes the circuit board a customised component. To this end, the PCB designer works closely with the hardware developer to create the optimal layout. Furthermore, the sensor must be protected from environmental influences and has assembly requirements – due to the technology or as defined by the product owner.

These aspects are incorporated into the mechanical design and in turn place demands on production processes. The interplay between design and manufacturing is a central process here that should not be underestimated. The image shows Andreas Münger in the EMC laboratory.

In a sensor, the firmware is a very central part. Analogue sensor signals are digitally converted and then filtered and processed in a microcontroller. In autonomous sensors, the firmware processes the information from the sensor and controls an actuator, for example a valve for a washbasin tap.

In a networked application, the sensor information is made available to a higher-level system via an interface. This means that spatially distributed sensors can enable much more complex detection behaviour. With the wide range of communication technologies, sensors have developed strongly in recent years in the direction of the IoT (Internet of Things). A broad spectrum of implemented solutions allows STEINEL to select the appropriate technology for a variety of use cases.

Battery-powered sensors are designed to work autonomously over a long period of time. Low-power applications are the norm today. This may also include battery management and sophisticated power-saving tricks.

Another discipline that should not be underestimated is the standard-compliant implementation and certification of the product. For a sensor, EMC immunity must always be considered a critical area. At STEINEL Solutions, internal pre-compliance tests in our own EMC laboratory are used to optimise false POSITIVES and false NEGATIVE. Often, the solution lies in clever signal processing in the firmware, which ultimately also results in cost neutrality for the product. Close coordination between hardware and firmware development is crucial for an optimal overall solution.

"What fascinates me as a developer of sensors is the extremely wide range of specialised fields that are necessary. Many years of experience enable me to implement hardware and firmware myself and to find optimal solutions in both areas. A good understanding of the application in the market is a prerequisite for this."

Andreas Münger, former hardware and firmware developer at STEINEL Solutions AG