Autonomous Mobile Platform

The Institute of Informatics of The Silesian University of Technology was focused on development of ADAS (Advanced Driver Assistance Systems) realised as embedded system for mobile platform control, development of  models for functional and architectural approach of Artificial Intelligence for ADAS, evaluation of AI (Artificial Intelligence) methods for energy efficiency purposes, real time communication of the sensor/actuator values between the mobile platform and CANoe/OPC UA, analysis and implementation of real-time CAN communication between devices as an examples of solution used in Automotive.

SUT take a part also in realization of D3.2 “Research Environment for Artificial Intelligence in Advanced Driver Assistance Systems Support” for the AutoUniMo project ‘’Automotive Production Engineering Unified Perspective based on Data Mining Methods and Virtual Factory Model’’ [1].

SUT prepared the laboratory for Research Environment forBir cok yabanciya ev sahipligi yapan istanbul Fatihte fatih escort kizlari aramak istiyorsaniz dogru adrestesiniz. iyi eglenceler. Artificial Intelligence in Advanced Driver Assistance Systems Support on the basis of models of hardware and software solutions described in report “The Research Environment for Artificial Intelligence in Advanced Driver Assistance Systems Support”.

First part of the report describes the functionalities of some ADAS modules in the area of communication by the use of wireless or wire Ethernet and CAN interfaces. Second part of the report describes prepared architectures of an Autonomous Mobile Platform. On the basis of prepared architectures of AMPs and methods of communication with ADAS and sensor were arranged research environment which allowed for preparation different ADAS solutions, AI and others algorithms for AMP which were described in the last part of the report.

According to the report D3.1 [2] in the first stage were finished a reviews of Advanced Driver Assistance Systems solutions for the Future Perspective of Sensor Fusion which were presented in the articles [3] [4].

First part of the report describes of functionalities of some ADAS modules in area of communication by the use of wireless or wire Ethernet and CAN interfaces for PC and embedded controllers on e.g. Raspberry Pi. At the beginning were described methods and  tools used for communication, testing, analysing  and simulation of ADAS solutions.  The main tools used for these purposes were:  CANoe from Vector, DB++ Editor, Simulation Setup, Measurement Setup, scripting language CAPL, Panel Designer, Vector VN1630A and Gateway i-7540D interfaces. General description, hardware and software requirements, operation principles and steps made during start-up were prepared for ADAS modules Camera, Radar and Lidar. According to data obtained from industrial partner, the best result of the use of ADAS modules was achieved for Lidar [5]. CANoe configuration was prepared for continuous measurement of Short Range Lidar without additional ECU components with the use of Vector VN1630A interface. In this area several methods were prepared in order to start-up and communicate with Lidar module. That solutions allowed to prepare further methods for receiving data from Lidar module with the use of:

• wireless and wire Ethernet interfaces and CAN/Ethernet i-7540D gateway [6]. The communication was realized in two ways and implemented on PC: CANoe and application in .NET.
• PiCAN module for Raspberry Pi controller [7] realised in C++,

Another solution was the use of XML format for receiving Ethernet data [8] (including ADAS and various sensors attached to small mobile platform) from Raspberry Pi controller by CANoe and PC. This solution allowed to prepare an ontology model for communicating with an Autonomous Mobile Platform [9] and had impact on preparation of other ontology models with the use of XML format [10] [11].

The next task was the preparation of the OPC UA server that serves as a gateway that shares CAN network data and engineering knowledge [12], which allowed for automatic exploring of the data owned by the CAN development system with the use of widely accessible address space of the OPC UA. This solution allowed for sharing data between AMPs and PCs. Additionally it allowed for preparation of projects of OPC UA protocol for the Vehicle-to-Vehicle communication [13] [14] and agent based architecture for the inventory operations management [15]. All described solutions allowed to prepare the laboratory stand for teaching and testing communication with ADAS modules.

In the area of the Architectures of an Autonomous Mobile Platforms (AMP) was prepared the first version of a mobile platform based on a Raspberry Pi 1. The first version of a simple AMP was built from a Pirate-4WD Mobile platform, a Romeo-All in one Controller, which was responsible for the control of the engine, and a Raspberry Pi model B + NOOBS, which was responsible for the control of the system. Communication with the PC was realised via a Wi-Fi solution that was integrated with the Raspberry Pi. The simple mobile platform was equipped with the first part of the sensors: accelerometer, magnetometer, gyroscope, encoder and ultrasound [16]. These sensors allowed for the measurements: velocity, distance, direction and acceleration.

First tests were prepared with implementation of GPS module [17]. The Raspbian Operating System was installed on a Raspberry Pi. This system allowed for communication with the sensors that are installed on the AMP via the I2C and SPI buses. The measurements from the connected sensors were stored on a Raspberry PI. First version of the algorithms for processing speed, angular speed and driving direction were prepared.

The Raspberry Pi 1 has too few low-level interfaces to connect more sensors. The first version of the architecture of a Simple Autonomous Mobile Platform with the STM32 controller and the Raspberry Pi 3 were prepared. For this solution was prepared INDUSTRUM PCB BOARD  [18] with STM32F303 which is equipped on: STM32F303VCT7 microcontroller,  power key 16 pins, inbuilt current monitors, CAN bus D9 connector and other peripherals. INDUSTRUM PCB BOARD allow for the connection for all sensors used in first version of AMP, more ultrasound sensors, Lidar module by CAN interface and other sensors by GPIO. Moreover, this version allowed on power management by switch On/Off modules [16] and current measurements on AMP. INDUSTRUM PCB BOARD and Raspberry Pi 3 are communicate by SPI interface with usage of prepared communication protocol which allow for transmitting different length of frames depend on available measurements in one period. Raspberry Pi 3 allows for obtaining and processing all measurement from INDUSTRUM PCB BOARD and allow for usage all functionality realised in Raspberry Pi 1. Additionally Raspberry Pi 3 allow for transmitting data with usage XML format to PC.

Raspberry Pi 3 and STM32 in addition allow for increasing processing of measurements data from connected sensors. In this area were prepared another projects for FPGA circuit in HDL for control engines, Soft Core Processor generated based on the machine code of the application [19]  and a frame filter IP core for RT-Ethernet monitoring [20]. Both last solutions were detailed described in [2]. The project for control engines on FPGA was realised on prototype system Zedboard. Zedboard is equipped on Xilinx circuit Zynq which allow to prepare project as a system on a chip. Prepared project  allowed to show differences between use ARM part of Zynq prepared in C language and FPGA part of Zynq arranged in Verilog language for control engines on AMP.

The first project of an advanced mobile platform equipped on ADAS modules from Continental was proposed. As a development of the Advanced Driver Assistance System task, a tower support assembly was built for the measurement devices. In this solution was used mobile platform Forbot 1.4 from Roboterwerk, an STM32F3 Discovery board and Raspberry PI. The mobile platform consisted of two brushless 3-phase BLDC 300W motors that were controlled by two SMCI36 motor controllers and these were managed by an STM32F3 Discovery development board. Communication between the STM32 and SMCI36 was realised using a clock signal – with pulse signals. During this research, the STM32 controlled the movements of the platform and sent it the data that was collected to the PC. The prepared construction allow the installation of measuring devices – radars, camera and lidar. The communication between lidar and control system is ensured by the CAN network. The methods of communication with ADAS modules were described earlier in first part of this report.

On the basis of prepared architectures of AMPs and methods of communication with ADAS and sensor were arranged research environment which allowed for preparation of different ADAS, AI and others algorithms for AMP.

The first data structure for the exchange of data between the Raspberry Pi and PC was prepared. It is assumed that the current measurements will be stored on a Raspberry Pi and all of the measurements, processed data and orders will be stored in a database on the PC. This will allow the functionality of the AMP to be improved in the future through the preparation of tests and simulations.

The tests of AMP allowed the work in the manual and remote mode were confirmed. The Raspbian Linux system was installed on the Raspberry Pi. Access to the AMP control was realised using a HTTP server and the terminal. The HTTP server allowed for preparation first monitoring tool of the measurements from sensors of the mobile platform. Additionally, WWW applications to prepare the outside and inside route for the mobile platform were prepared. Both WWW applications allowed to export the route data to .csv file. Csv file can be imported to .NET application. .NET application allowed to send the route to AMP as a tasks to do for driving AMP according to the prepared route. Additionally .NET application allow for  monitoring sensor on AMP by receiving XML data from AMP. For this solution on AMP was prepared function in C which allow share measurement data from AMP in XML format with usage Ethernet interfaces.

The first version of the simple implementation of an Emergency Brake Assist system was prepared on a Raspberry Pi as a C++ function. The system measures the distance to the obstacle using an ultrasonic sensor. When the distance becomes critical, the system runs the braking procedure to stop it before it hits the obstacle. The critical distance depends on the speed of the AMP while driving and the performance of the braking system. Additionally with using an ultrasonic sensor was implemented as a C++ function first version of avoiding obstacles by AMP [21]. Prepared solution in C++ for STM32 and Raspberry Pi allow for monitoring and controlling speed for an autonomous mobile platform based on the Hall sensor [22]. The first version of the neural network for controlling the engines on the mobile platform while driving was implemented [2].

Installed on AMP power management allowed for preparation algorithms in the area of energy efficiency on the basis of current measurement of the following ADAS modules: Lidar, radar and camera. The support predictive maintenance module [23] prepared to enhance the reliability and safety of ADAS sensors was implemented in Matlab with neural networks solutions. Monitoring the current solution that is used in a transmission via the CAN protocol was allowed to provide significant data about quality of the connection. Monitoring the current solution was allowed the recognition of ADAS module and over or under-voltages. As a result of the investigations, patent applications has been filed in the Patent Office of the Republic of Poland[24].

Additionally in the following article “The possibilities of system’s self-defence against malicious software”  [25] was described the concept of  the self-defence of systems before the possibility of the long lasting attack of malicious software. Another solution is Raspberry Pi based on lap counter for amateur car racing [26]. The Lap counter detects the finishing line with the accuracy about 5m that seems to be reasonable for amateur usage. The main purpose of last project is the concept of using embedded MEMS sensors position objects especially when the GPS signal is weak [27], e.g. in underground car parks or tunnels. Such an approach is important for controlling indoor objects or autonomous vehicles.

Obstacle avoidance

The information’s from these sensors are processed by Raspberry Pi and are sufficient to detect obstacles, determine the distance to obstacles and prepare actions to avoid the obstacles. The data obtained from sensors allow for the development of automatic obstacle avoidance functions for mobile platforms. The vehicle has to rotate around on its vertical axis and take the measurements during this rotation for measure the distance to obstacles in other directions.

The deep obstacle, algorithm A1.

 The deep obstacle, algorithm A2.

 The round obstacle, algorithm A1.

The ro und obstacle, algorithm A2.

Forboot control

The research station consisted of a Forbot 1.4A mobile platform from Roboterwerk, an STM32F3 Discovery board, a Raspberry PI and a PC. The mobile platform consisted of two brushless 3-phase BLDC motors that were controlled by two SMCI36 motor. Control of the engines was realised using the STM32. Communication between the STM32 and SMCI36 was realised using a clock signal. 

ZedBoard control

The project for control engines on FPGA was realised on prototype system Zedboard. Zedboard is equipped on Xilinx circuit Zynq which allow to prepare project as a system on a chip. Prepared project  allowed to show differences between use ARM part of Zynq prepared in C language and FPGA part of Zynq arranged in Verilog language for control engines on AMP. ZedBoard was used as a controller while STM32 X-NUCLEO-IHM04A1 act as motor driver.

Presentation on sience nights

The list of articles prepared in the AutoUniMo project for WP3:

[1] AutoUniMo, “FP7-PEOPLE-2013-IAPP AutoUniMo project ‘Automotive Production Engineering Unified Perspective based on Data Mining Methods and Virtual Factory Model’ (grant agreement no: 612207).” [Online]. Available: http://autounimo.aei.polsl.pl/.
[2] A. Ziębiński, “D3.1 Strategies for Artificial Intelligence in Advanced Driver Assistance Systems,” Silesian University of Technology, Gliwice, Poland, May 2015.
[3] A. Ziebinski, R. Cupek, H. Erdogan, and S. Waechter, “A Survey of ADAS Technologies for the Future Perspective of Sensor Fusion,” in Computational Collective Intelligence, vol. 9876, N. T. Nguyen, L. Iliadis, Y. Manolopoulos, and B. Trawiński, Eds. Cham: Springer International Publishing, 2016, pp. 135–146.
[4] A. Ziebinski, R. Cupek, D. Grzechca, and L. Chruszczyk, “Review of Advanced Driver Assistance Systems (ADAS),” presented at the 13th International Conference of Computational Methods in Sciences and Engineering (ICCMSE 2017), 2017.
[5] A. Ziębiński, R. Cupek, M. Kruk, M. Drewniak, and H. Erdogan, “Lidar technology in general purpose applications,” Studia Informatica, vol. 37, no. 4A, pp. 15–32, 2016.
[6] R. Cupek, A. Ziebinski, and M. Drewniak, “Ethernet-based test stand for a CAN network,” in 13th International Conference of Computational Methods in Sciences and Engineering (ICCMSE 2017), 2017.
[7] M. Drewniak, K. Tokarz, and M. Rędziński, “ADAS device operated on CAN bus using PiCAN module for Raspberry Pi,” in Computational Collective Intelligence, B. Trawiński, Ed. Cham: Springer International Publishing, 2017.
[8] M. Drewniak, R. Foszner, M. Kruk, and M. Fojcik, “A CANoe-based approach for receiving XML data over the Ethernet,” in INTECH 2017, 2017.
[9] R. Cupek, A. Ziebinski, and M. Fojcik, “An ontology model for communicating with an Autonomous Mobile Platform,” Kozielski, Kasprowski, Mrozek, Małysiak-Mrozek, Kostrzewa (eds.) BDAS 2017 CCIS Springer, 2017.
[10] I. Postanogov and T. Jastrząb, “Ontology Reuse as a Means for Fast Prototyping of New Concepts,” presented at the International Conference: Beyond Databases, Architectures and Structures, 2017, pp. 273–287.
[11] D. Krasnokucki, G. Kwiatkowski, and T. Jastrząb, “A New Method of XML-Based Wordnets’ Data Integration,” presented at the International Conference: Beyond Databases, Architectures and Structures, 2017, pp. 302–315.
[12] R. Cupek, A. Ziebinski, and M. Drewniak, “An OPC UA server as a gateway that shares CAN network data and engineering knowledge,” 18th IEEE International Conference on Industrial Technology, 2017.
[13] R. Cupek, A. Ziebinski, M. Drewniak, and M. Fojcik, “Feasibility Study of the Application of OPC UA Protocol for the Vehicle-to-Vehicle Communication,” in Computational Collective Intelligence: 9th International Conference, ICCCI 2017, Nicosia, Cyprus, September 27-29, 2017, Proceedings, Part II, N. T. Nguyen, G. A. Papadopoulos, P. Jędrzejowicz, B. Trawiński, and G. Vossen, Eds. Cham: Springer International Publishing, 2017, pp. 282–291.
[14] R. Cupek, A. Ziebinski, M. Drewniak, and M. Fojcik, “Application of OPC UA Protocol for the Internet of Vehicles,” in Computational Collective Intelligence: 9th International Conference, ICCCI 2017, Nicosia, Cyprus, September 27-29, 2017, Proceedings, Part II, N. T. Nguyen, G. A. Papadopoulos, P. Jędrzejowicz, B. Trawiński, and G. Vossen, Eds. Cham: Springer International Publishing, 2017, pp. 272–281.
[15] R. Cupek, A. Ziebinski, L. Huczala, D. Grossmann, and M. Bregulla, “Object-Oriented Communication Model for an Agent-Based Inventory Operations Management,” in INTELLI 2015, St. Julians, Malta, 2015, pp. 80–85.
[16] P. Rybka et al., “Power management and sensors handling on the autonomous mobile,” Studia Informatica, vol. 37, 2016.
[17] W. Czernek, W. Margas, R. Wyżgolik, S. Budzan, A. Ziębiński, and R. Cupek, “GPS and ultrasonic distance sensors for Autonomous Mobile Platform,” Studia Informatica, vol. 37, 2016.
[18] A. Ziębiński and P. Rybka, “AutoUniMo - INDUSTRUM PCB BOARD,” Silesian University of Technology, Gliwice, Poland, May 2016.
[19] A. Ziebinski and S. Swierc, “Soft Core Processor Generated Based on the Machine Code of the Application,” Journal of Circuits, Systems and Computers, vol. 25, no. 04, p. 1650029, Apr. 2016. IF 0,481
[20] A. Ziębiński, R. Cupek, P. Piękoś, and L. Huczala, “A frame filter IP core for RT-Ethernet monitoring,” Przeglad Elektrotechniczny, vol. 90, no. 10, pp. 219–225, 2014.
[21] A. Ziebinski, R. Cupek, and M. Nalepa, “Obstacle avoidance by a mobile platform using an ultrasound sensor,” in Computational Collective Intelligence, B. Trawiński, Ed. Cham: Springer International Publishing, 2017.
[22] A. Ziebinski, M. Bregulla, M. Fojcik, and Kłak, “Monitoring and controlling speed for an autonomous mobile platform based on the Hall sensor,” in Computational Collective Intelligence, B. Trawiński, Ed. Cham: Springer International Publishing, 2017.
[23] D. Grzechca, A. Ziebinski, and P. Rybka, “Enhanced reliability of ADAS sensors based on the observation of the power supply current and neural network application,” in Computational Collective Intelligence, B. Trawiński, Ed. Cham: Springer International Publishing, 2017.
[24] D. Grzechca, A. Ziębiński, R. Cupek, and P. Rybka, “P.422834 Sposób i układ do identyfikacji elektronicznych podsystem6w ADAS,” Sep-2017.
[25] M. Skrzewski and P. Rybka, “The Possibilities of System’s Self-defense Against Malicious Software,” in Computer Networks, vol. 718, P. Gaj, A. Kwiecień, and M. Sawicki, Eds. Cham: Springer International Publishing, 2017, pp. 144–153.
[26] K. Tokarz, P. Czekalski, and R. Raszka, “Raspberry Pi based lap counter for amateur car racing,” Studia Informatica, vol. 37, no. 4A, pp. 7–13, 2016.
[27] D. Grzechca, K. Tokarz, K. Paszek, and D. Poloczek, “Using MEMS sensors to enhance positioning when the GPS signal disappears,” in Computational Collective Intelligence, B. Trawiński, Ed. Cham: Springer International Publishing, 2017.

The list of master and engineering thesis prepared in the AutoUniMo project for WP3, The Silesian University of Technology, Faculty of Automatic Control, Electronics and Computer Science:

1. Kruk, “The laboratory for testing lidar module ADAS Continental,” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
2. Michna, “Software concept of architecture of the mobile platform on Rasberry Pi (PL: Koncepcja oprogramowania architektury platformy mobilnej na Raspberry Pi),” Master thesis, Silesian University of Technology, Gliwice, Poland, 2016.
3. Ł. Szczepański, “Software development and connection of acceleration sensor and optical encoder to the mobile platform based on Raspberry Pi,” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
4. Wosik, “Software development and connection of magnetometer and gyroscope to the mobile platform, based on Raspberry Pi,” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
5. Czernek, “Software development and connection of GPS sensor to the mobile platform based on Raspberry Pi,” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
6. Margas, “Software development and connection of ultrasonic sensors to the mobile platform based on Raspberry Pi,” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
7. Rybka, “Power Management modules ADAS with Raspberry PI (PL: Zarządzanie poborem energii modułów ADAS z wykorzystaniem Raspberry PI),” Master thesis, Silesian University of Technology, Gliwice, Poland, 2017.
8. Bundala, “Control of electric motors using ZedBoard,” Master thesis, Silesian University of Technology, Gliwice, Poland, 2017.
9. Kłak, “Controlling the robot Forboot 1.4 using the Raspberry Pi (PL: Sterowanie robotem Forboot 1.4 z wykorzystaniem Raspberry Pi),” Master thesis, Silesian University of Technology, Gliwice, Poland, 2016.
10. Kuryłek, “Database server on Raspberry PI (PL: Serwer bazy danych na Raspberry PI),” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
11. Rejchert, “Application to prepare the route for the mobile platform,” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
12. Nalepa, “Obstacle detection and avoiding algorithm for AutoUniMo platform (PL:Algorytm wykrywania i omijania przeszkód dla platformy AutoUniMo),” Master thesis, Silesian University of Technology, Gliwice, Poland, 2016.
13. Kordys, “Study the possibility of using artificial neural networks to control the mobile platform (PL: Badanie możliwości wykorzystania sztucznych sieci neuronowych do wspomagania sterowania platformy mobilnej),” Master thesis, Silesian University of Technology, Gliwice, Poland, 2015.
14. Raszka, “Computer for Motorsport,” Engineering thesis, Silesian University of Technology, Gliwice, Poland, 2016.
15. Piech, “Hardware concept of architecture of the mobile platform on Rasberry Pi (PL: Sprzętowa koncepcja architektury platformy mobilnej na Rasberry Pi),” Master thesis, Silesian University of Technology, Gliwice, Poland, 2017.
16. Rędziński, “Interface Ethernet/CAN for communication with Rasberry Pi (PL: Interfejs Ethernet/CAN do komunikacji z Rasberry Pi),” Master thesis, Silesian University of Technology, Gliwice, Poland, 2017.
17. Kruk, “The laboratory for testing module ADAS,” Master thesis, Silesian University of Technology, Gliwice, Poland, 2017.