Matthew BoyleThe VEDLIoT project shows how effective AI can be applied in a variety of use cases.
The goal of the VEDLIoT project is to develop energy-efficient Deep Learning ( DL ) techniques for applications involving distributed artificial intelligence of things ( AIot ). Our all-inclusive approach places a strong emphasis on algorithm optimization while even addressing the security and safety concerns present in AIoT systems. This strategy uses a flexible and scalable cognitive IoT hardware platform that incorporates microserver technology at its core, enabling hardware customization to support numerous applications. We integrate a wide variety of hardware accelerators, including specialized ASICs and FPGAs for flexible computing, and use varied computing to improve performance and reduce energy consumption. In order to support the development and application of powerful, effective AIoT systems, the project contributes to trusted computing, distant attestation, and safe execution environments. In Issue 15 of The Innovation Platform, a previous article titled” Unlocking the potential of IoT systems: The role of Deep Learning and AI” provided more information on the VEDLIot approach.
Through use cases ranging from Smart Homes to electrical and professional IoT devices, we validate our architecture. The sections that follow provide a thorough analysis of these applications, including information on their common architecture, the integration of VEDLIoT technology, and the results for each use case across different domains.
electrical industry
The pedestrian automatic emergency braking ( P- AEB ) system, a crucial safety feature in the automotive industry, was the focus of the VEDLIoT project. The difficulty of identifying erratic pedestrians, who may immediately enter the road from a variety of locations, such as sidewalks or behind obstacles, is addressed by this system. The P-AEB system uses artificial intelligence and deep learning ( AI/DL ) to make quick decisions in urgent situations and relies primarily on sensor data from an RGB camera.
The system’s architecture is made up of edge processing units, a main computer unit (CCU) inside the vehicle, and native sensor processing modules. These parts are connected by a typical 5G connection. The main difficulty in this use case is striking a balance between the vehicle’s minimal processing capacity and the requirement for swift, low-latency processing. In order to address this, the project investigates the use of extra mathematical power from edge processing while taking into account the potential effects of changing communication channel conditions on data throughput.
The project used a variety of AI models for useful implementation, with EfficientNet serving as the main model for evaluation. The active partitioning of the DL model to efficiently distribute mathematical tasks between the onboard systems and edge resources is a key component of this implementation. The processing speed of the edge units, cellular communication capacity, and end-to-end overhead are all taken into account in this distribution.
A camera system created to meet particular requirements was used to gather data for training and testing the models. Fig. 1 provides an illustration of an industrial setting. The design of numerous traffic scenarios is required to properly optimize the DL model due to the dynamic environment of the vehicle. Given the importance of electrical applications and the use of available communication links, the project also places a higher value on practical safety, robustness, and security.
Quantitative analyses of mathematical loads were carried out on different hardware configurations. These configurations included everything from the sensor’s processing unit to the CCU, then expanded to include the edge unit and eventually carried out all of the computations there. The findings suggested that various processing resource combinations could lower overhead while preserving the accuracy of the AI/DL model.
The project’s main objective is to determine how much energy each configuration uses overall, including the amount needed to maintain communication links. The goal of this upcoming work is to strike the ideal balance between processing effectiveness and the strict safety standards that the P-AEB system entails.
Industrial Internet of Things
In particular, the VEDLIoT project focuses on two crucial areas: motor condition monitoring and series arc fault detection ( AFD ) in low-voltage direct current ( LVDC ) systems. It is at the forefront of integrating deep learning into Iot for industrial applications. These areas, which involve forecast maintenance and anomaly detection and each have their own unique AIoT solution requirements, represent important aspects of industrial Internet of Things.
The project emphasizes the significance of smart field devices ( SFDs ) for data collection in motor condition monitoring. These gadgets keep an eye on different aspects of motor operation, including mechanical, infrared, and functional conditions. For motor reliability and longevity, SFD data collection, which includes operating hours, cooling system status, and the condition of structural components like bearings, is essential. But, in large-scale IoT systems, network congestion and constrained bandwidth present a major challenge. In order to address this, heavy learning-based on-site data processing within SFDs is investigated. This will reduce the amount of data transmitted electronically, necessitating the integration of DL algorithms into resource-restricted systems while preserving hardware power efficiency. Fig. 2 depicts the VEDLIoT solution, in which the customized SFD integrates crucial sensors and an Artificial accelerator.
The complexity of detecting series arc faults, which are extremely challenging to identify due to their low existing draw, is the additional major focus area in LVDC systems. Compared to conventional quantitative techniques, AI’s adaptability and learning capabilities offer a clear advantage. A particularly created test bench that can generate electronic arcs and gather pertinent data for AI analysis is one of the VEDLIoT project’s efforts to develop AI-powered solutions for LVDC systems. This is shown in figure. 3. displaying the test bench created for the development and evaluation of AI systems, including the setup for an Artificial accelerator and microcontroller.
The overwhelming problem with both use cases is a lack of data, particularly for anomalies. Through test-beds for data collection and validation, the VEDLIoT project addresses this. It is also creating a detailed framework with the goal of enhancing the efficiency and explainability of AI algorithms.
The system architecture of real-time AFD is shown in Fig. 4. The colored block denotes VEDLIoT-researched technology that can hasten the creation of AI-based solutions. With the help of the provided streamline, the development process in the arc fault detection use case is sped up. To create arc faults and gather latest data, a test bench built in accordance with the U1699B standard is constructed. The hardware Nvidia Jetson Xavier NX is selected to process existing data in real-time based on the specific requirements derived from problem analysis and benchmarking results from the VEDLIoT project. To ensure model inference within the target detection time, profound learning model pruning has been used. The VEDLIoT project’s pruning software has produced impressive results, reducing model runtime by up to 70 % while maintaining an arc fault detection accuracy of 98 %.
The energy efficiency of SFDs, which are anticipated to outlast battery power for two years, is a major focus in motor monitoring. The MAX78000 microcontroller with a built-in CNN accelerator is used in the project to address this, maximizing power consumption and processing efficiency.
The VEDLIoT project is addressing specific challenges in motor condition monitoring and arc fault detection, as well as creating frameworks and tools for effective data collection, hardware selection, and software optimization. As a whole, it is making great strides in advancing AI and digital intelligence ( AI ) in industrial IOT systems.
Smart Home is a domain.
A bright mirror was created in the smart home use case for the VEDLIoT project, giving local processing priority over data protection. An Intel Realsense 3D radio camera and a PC monitor display are integrated into this mirror, which is made to replace conventional ones in spaces like bathrooms and dressing rooms. In addition to controlling various smart home devices, it can display data like weather forecasts, train times, and personal appointments.
Using the sound camera to record depth and RGB images, interaction with the mirror is simple and wireless. For user interaction, machine learning ( ML) algorithms can recognize objects, faces, and gestures. User identification is done using visual recognition and tracking, and the mirror is controlled by hand motions. Additionally, voice command control is made possible by a voice assistant that uses natural language processing ( NLP).
All AI processing takes place directly on the device using open-source software to ensure privacy. Delicate user data is not sent to outside servers using this strategy. Face recognition uses a Siamese network with RetinaFace detector output, gesture detection uses the YoloV7’s little model, and object detection is based on the Yano neural network (version 7 ). While an open-source MagicMirror framework offers an interactive HTML frontend ( see Fig. ), the Robot Operating System 2 ( ROS2 ) middleware facilitates modular software integration. 5. 5.
It’s important to strike a balance between native AI model computation and power consumption. The device maintains a 30 FPS framerate while consuming 45W of power and utilizing the Nvidia Jetson Orin AGX system-on-module and Hailo8 Artificial accelerator to achieve substantial power efficiency.
An Arduino Nano RP2040 Connect-based safe microphone is used to enable voice control. Upon hearing a spoken hotword, this microphone turns on and uses TLS WiFi to send encrypted audio data to the mirror. In order to ensure security and privacy in private smart home settings ( see Fig. ), the mirror then internally transcribs and processes voice commands. 6 ). The security and privacy of user data are upheld by this combination of encrypted communication and regional processing of image and audio data.
more applications
The project has broadened its scope to include ten more use cases from numerous application domains, including agriculture, industrial, automotive, and medical fields, in addition to the initial primary uses of VEDLIoT that were previously discussed. as depicted in Fig. 7. These projects cover a wide range of uses, such as pollen analysis, laser welding, and AI for driving schools. Different components of VEDLIoT technology have been incorporated into each of these projects ‘ unique applications. The use of the hardware platform, specific microservice components, security features, or the requirements engineering methodology created within VEDLIoT are all examples of this integration. On the VEDLIoT website, you can find more details about the various opened call projects.
Outlook
VEDLIoT emphasizes the need for power-efficient computing by addressing the challenge of integrating Deep Learning into IoE devices that have constrained processing power and require lower energy consumption. Custom hardware and more accelerators for AIoT applications, ranging from integrated systems to edge and cloud computing, are made available by the VEDLIo T AIOT hardware platform. To overcome the challenges of implementing Full Learning across its whole framework, the project combines cutting-edge requirements engineering techniques with safety and security concepts. In crucial commercial fields like electrical, automation, and wise homes, these concepts are rigorously tested. Each case study illustrates the advantages of incorporating VEDLIoT’s AI, ML, and precisely DL techniques into contemporary AIiT applications that are spread across the computing spectrum. This integration aims to improve efficiency, security, and safety while moreover improving functionality and performance.
Under grant agreement No. 957197, the European Union’s Horizon 2020 research and innovation program provided funding for the VEDLIoT project.
