Privacy Preserving Workflow Detection for Manufacturing Using Neural Networks based Object Detection
Sprache des Titels:
IoT '21: 11th International Conference on the Internet of Things
In this paper, we introduce a detection system for workflow in a manufacturing line using depth images to preserve the privacy of workers. A depth camera sensor is mounted on a ceiling with a top-down angle and pointed to workers below completing a workflow. The system was deployed in a real life industrial process where workers had to work on a metal sheet by completing a sequence of bending steps. In this study, we experimented the effectiveness of using two classification approaches in order to identify the current workstep that workers are doing. The first approach was workflow detection by human activity recognition along with detecting related objects (a tool table, a computer screen and a machine) in the scene using only a depth camera sensor. Because of the similarity between the human body shape during different activities, the results were low and precision was 63.03%. The second approach was workflow detection by object classification and human localisation along with integrating depth camera sensor data with other sensor devices and results were better than the first approach with precision 85.42%. Within this approach, two classification models were created only using data from the Realsense sensor and two more were created including data from the bending machine. Each model has its own benefits in terms of precision, accuracy and performance, and we explain them along with the challenges the system had, in the discussion section. The results are also investigated in details and we present the future plans for the proposed detection system and for the sensors connected.