It’s beginning to feel a lot like Christmas (well is was when I started this post!) and with our street getting into the Christmas lights in a big way, I saw an opportunity to tweak some of our FatigueM8 Machine Learning (ML) features. Our FatigueM8 unit has a forward facing camera that captures the road and traffic conditions that our drivers are operating in, and its’ a part of the system that has remained relatively unchanged for a long time. The process is straight forward, capture a still image at regular intervals (notionally 10 seconds), upload it to the cloud (AWS in our case) and using ML to determine the road and traffic conditions; before recording the results.
In our prototype FatigueM8 units we leverage an AWS service called Rekognition (read more about it here). Rekognition is really easy to use and quite accurate for our purposes, the image below show’s the raw image captured from the FatigueM8 and then based on what Recognition is able to detect in the image.
Raw 
Labelled
We apply the green Bounding Boxes based (as shown on the “Labelled” image) on the information returned from Rekognition. Interestingly, the white VW Amarok that was directly in front of the FatigueM8 unit wasn’t identified as a Car or Truck – we’ll come back to this later). Rekognition returns the name of the object, the confidence of the object’s type and the object’s coordinates in JSON format as is shown below.
[{"Name": "Car", "Confidence": 99.19426727294922, "Instances": [{"BoundingBox": {"Width": 0.3071940541267395, "Height": 0.16058334708213806, "Left": 0.6081644296646118, "Top": 0.0007275581592693925}, "Confidence": 99.19426727294922},So how does this relate to Christmas lights? Well with our street going all in and the menace that is SARS-COV-2 lurking about I thought it might be useful to understand the crowd numbers visiting our Christmas lights (check out the ABC Canberra’s video of our lights here).
We adapted one of our older FatigueM8 units, rehoused it into a waterproof case and used an OEM RaspberryPi camera casing. After testing the installation on the side of my house I noticed the angle of the camera wasn’t the best and so we created a stand and in the interests of time, just used cable ties to connect the stand to the casing (classy I know).
The Pedestrian Monitor unit used the same code logic as FatigueM8 capturing an image every five (5) seconds, passing it to AWS Rekognition and recording the results.
We used a simple algorithm to determine the validity and accuracy of the results returned. The first step was to filter based on the time of day, as most people would look at the lights after dark. Given each Rekognition scan has a cost ($0.0005) we didn’t waste processing effort or $$ if the probability of people being viewing the lights was low. For this reason we only sent images to Rekognition between 4pm-11pm.
Below is an example of one of the images captured around 6pm and also one from 10pm; noting the camera wasn’t a night vision camera the later images are pretty dark.
6pm-ish 
10pm-ish
It’s important, the pictures were captured from far enough away, and a resolution so not as to be able to identify individuals; and in this area is also CCTV cameras operating 24/7 365 days a year .
We also filtered the results based on the confidence of a person being present and even at > 70% confidence we saw a range of false positives (People being detected where none were). The images below show a couple of examples were the confidence was > 70% but < 80% that person was detected, and as you can see there is no person visible.
So let’s look at the finding, across the Christmas period. The graph below plots the Average, Median and Maximum number of pedestrians present (using Rekognition) for each day between 4pm and 11pm. The graphs shows a steady build up to peak on Christmas Eve and then a rapid fall away from Christmas night.
The dip on the 22nd Dec was due to system outage and therefore reduced data collection occurred – otherwise we’d estimate the numbers would have been similar to those either side of it.
The graph below shows the number of people observed, grouped by hour and as is evident, the number of people increased the darker it was. The data collection finished at 11pm – when the Christmas Lights were switched off.
One thing that isn’t clear in the above graph is that the highest number of people observed at once, was in the 8pm time slot (20 in 24HR time) and it peaked at 31 people.
Calculating the total number of pedestrians is not without its challenges – as we (purposely) didn’t capture enough detail to track/confirm an individual is present from one image to the next. Our manual inspections of the data showed a level of cross over between images (estimated to be 40%) where a person or group of people are present in successive images.
The second element was number of False Positives present is quite high while there is natural light present on the area that’s being captured (we’ll investigate why this is at a later point in time). A manual inspection of the data from the 24th Dec 2021 showed the a 70% False Positive rate through the 4pm-7pm window. Across the month of data capture during the 4pm-7pm window a raw total of 7083 people were observed, however reducing that number by the false positives (70%) brought it down to 2125 people observed.
Applying the cross over factor (40%) the number reduced to 1,275. Noting there was 26 days of data collection (we missed a couple in the middle) and between 4pm and 7pm is 3 hours, the estimate for pedestrians during this time band was 16 per hour (which based on the manual inspections .
Looking at the time period from 7pm to 11pm the raw total was 85,240 which is huge! But this drops to 51,144 when the crossover factor is applied (still rather large!). Breaking that down by Day, then Hour gives an average of 655 people per hour at peak time!
The total seems high, but as we’ll look at in the next instalment of FatigueM8 Friday, it may not be quite high enough!
Until Next time, stay safe.
