Modelling, Simulation and Testing of automotive perception sensors

When developing the winter test programme, a key consideration was to make sure that we could capture data from a number of different sensors so that we can try to learn how the physics of different sensor technologies are affected by weather rather than measure the performance of a few specific devices.  If you’re not familiar with the background to this project please see the earlier posts for background, here. 

The sensor suite that we ended up with at the test site consists of 4 cameras with 3 different CMOS chips, 5 lidars covering 850nm, 905nm and 1550nm wavelengths, 5 different radars all operating within the automotive 77-79GHz band and a thermal imaging camera.  Some of these sensors are commercially available and ready to be installed in vehicles but others are specialised development kits that give us insight into the physics and behaviour of the sensor technologies.  As such, many of the sensors we decided to include in the test programme were not IP67 rated and not designed to cope with the temperatures we expected to encounter. –17°C is the lowest temperature ever recorded in Scotland and we needed to ensure our setup could survive this. 

A bespoke sensor housing has been designed to protect the sensors and data loggers to keep them warm and dry throughout the winter.  This is pictured in Figure 1 and features 4 windows on the front of the housing that are made from clear polycarbonate so that it is transparent to the cameras, radar and lidar housed inside.  The rest of the housing is constructed from mild steel, fully insulated and includes temperature and humidity controls to ensure the interior of the housing stays warm and dry. 

The sensor housing is positioned ~1.2m above the ground on a scaffolding frame as shown in Figure 1.  This accommodates 4 cameras, 5 radars and 1 lidar sensor in addition to all the data logging equipment and power supplies for the sensors.  Underneath the housing there is an array of 4 lidar sensors and the thermal imaging camera.  By placing these sensors underneath the main housing they are shielded from the rain and snow which means the sensor surfaces stay clear of rain drops and snow meaning any change in performance is related to the effect of weather in the atmosphere rather than droplets on the surface of the sensor. 

Figure: View of the sensor housing clearly showing the range of sensors included in the test programme.

The data logging has been configured so that all the sensors take a measurement at the same time which is every 5 minutes.  This means that they are all perceiving the same weather at the same time and it will help interpret the relative impact of the weather conditions on each type of sensor. 

Where possible we capture the raw output from the device, e.g. the UDP message stream, as this is what our sensor models will need to output.  For the cameras we capture 25 frames at each measurement time in RAW image formats and for the radar, lidar and thermal imaging sensors, the format changes depending on the device, but we are capturing 30s of data every 5 minutes.  The scheduling of the measurements is controlled so that they all start a measurement at 0, 5, 10, 15, 20, etc minutes past the hour.  This means when we capture a measurement all the sensors are perceiving the same weather as they take a measurement.   

Multiple data loggers are used to capture the information from the sensors and we are capturing about 800GB of data per day!  This will mean that by the end of the test campaign we will have close to 50TB of sensor data to analyse and we will be exploring ways to automate the analysis of all this information. 

Please get in touch if you have any questions or have got a topic in mind that you would like us to write about. You can submit your questions / topics via: Tech Blog Questions / Topic Suggestion