Modelling, Simulation and Testing of automotive perception sensors

The test site experienced its first snowy conditions early in January, as can be seen in Figure 1, which is exactly what we had hoped for.  Unfortunately, the data collected during this period is not as useful as we hoped because the project suffered a major setback during the transport of the equipment to the test location in Scotland.  Whilst taking a break during the long drive from the Claytex office to the test site, the van was broken into and a key part of the disdrometer was stolen, amongst other items, and this took several weeks to replace.  This meant that although the perception sensors were running during this first snowy period we do not have the weather data necessary to make good use of it. Nevertheless, the data shows some interesting effects that we hope to be able to quantify now that all the weather instrumentation is up and running at the test site. 

Figure 1: Snow covered test site.

Despite the setback, this early set of snowfall data has enabled the project partners to start analysing the data to understand what aspects of the sensor performance need more detailed analysis and to be included into the simulation models.  One of the most interesting early findings has been identified in the lidar data, an example of which is shown in Figure 2.  What we have observed is that there is a well contained range within which there is a cloud of points detected whilst snow is falling that is not present in good, clear, conditions.  This has raised many questions that need to be investigated through detailed analysis of the data and perhaps further measurement work will need to be carried out.

Figure 2: Lidar point cloud when snowing, displaying a clear cloud of points at the left hand side that are not present in clear conditions.

For the radar data, there is a less clear effect and more detailed analysis is required before drawing any conclusions.  The RCS of the corner reflectors does not appear to be significantly affected by snowfall but the noise from the rest of the environment appears to increase when it is snowing, see Figure 3.  More detailed and extensive analysis is required to draw any concrete conclusions.

Figure 3: Radar measurements under good conditions (left) and when snowing (right).

A sample image from one of the cameras is shown in Figure 4, this camera has an RCCB colour filter hence the green, yellow and blue stripes on the right of the sign are not the colours that our eyes would expect.  This image is a zoomed section from one of the camera frames but you can clearly make out that snowflakes are obscuring parts of the image but it’s not immediately apparent to the human eye that this level of blockage is a problem.  The disdrometer classified the snowfall at this time as “moderate snowfall”. As we progress with the analysis it will be interesting to see what the image quality metrics used by the partners make of the images recorded, whilst snow is falling to understand how significant these effects are for machine vision systems.

Figure 4: The camera used here has an RCCB colour filter. Hence, the green, yellow and blue stripes on the right of the sign are not the colours that our eyes would expect.

The change in performance for a thermal imaging camera is clearly shown in Figure 5.  On the left hand side is the view on a clear day and the right hand side shows the view at the same time of day whilst it is snowing.  Whilst it is snowing it’s much harder to make out the surroundings but the heaters built in to the targets are clear (the white spots).

Figure 5: Thermal imaging view of the site at the same time of day but on a clear day (left) and whilst it is snowing (right).

Please get in touch if you have any questions or have got a topic in mind that you would like us to write about. You can submit your questions / topics via: Tech Blog Questions / Topic Suggestion