By Lance Eliot, the AI Trends Insider
(Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/)
I was driving on the freeway the other day and up ahead of me the traffic started to do all sorts of dodging and weaving.
Immediately, I suspected that there was likely some kind of debris that had caused drivers to enter into spasmatic moves of this nature.
As I strained to see what was going on, I could just barely see white objects of maybe a half foot in length that were in the air and seemingly randomly bobbing around at the height of the car hoods and windshields. It was difficult to determine from my distance away as to whether the objects were inherently dangerous or not, but in a sense no matter what they were, it was still a dangerous circumstance due to the reaction by the other drivers.
The act of the drivers swerving their cars had made the situation dicey and I was anticipating that some cars would ram into each other while playing this high-risk dodge game.
Upon reaching the point at which the objects were floating around, I was able to determine that they were actually pieces of mattress stuffing.
Apparently, a mattress had fallen down onto the freeway, cars had hit and exploded open the mattress, which then disgorged the innards and the stuffing was being carried up into the air by the motion of the cars.
Fortunately, having the mattress stuffing hit a car was not a serious matter per se, since it wouldn’t do any direct damage. The biggest danger now was that the drivers were reacting madly and trying to keep from getting hit by the mattress fluff. They could have just driven straight ahead and not worried about hitting it. Instead, they were weaving as though it was poisonous and they didn’t want to let it touch their cars.
Roadway Debris Consternation
Debris is always a potential issue for drivers as to whether to hit it or try to dodge it.
There’s the type of debris that sits on the actual roadway and driver’s need to determine whether to hit it, or try to roll over it, or try to avoid it, all of which is tied to what damage the debris might cause to the underbelly of the car and whether or not the car will be able to continue unabated after striking the debris.
If the roadway debris is someone’s wool cap that flew out their car window and landed on the freeway, it’s best to just run over it and continue along.
If the back of a truck has dropped a couch onto the freeway, hitting it straight on is likely to damage the car and most drivers are going to do whatever they can to avoid hitting it.
Besides debris on the roadway, there’s also airborne debris.
This flying debris can originate as roadway debris, including the example earlier of a mattress that fell onto the freeway and then got split open after some cars hit it. The resulting spray of the mattress stuffing became an airborne mess. Car after car was making the material become increasing airborne and it was flitting back-and-forth across the lanes of traffic. In this case, the fluff was harmless, but the drivers were reacting to it anyway as though it was something that could harm their cars.
That’s a natural reaction for drivers. If they aren’t sure of what it is, they are going to try and avoid it.
Special Attributes Of Flying Debris
In fact, I’d assert that we have a natural aversion to anything that is flying towards us menacingly.
As an example, about a month ago, I was on a highway and I saw what seemed like a large black plastic bag that was being hit repeatedly by cars ahead of me. It was flopping around and appeared to be empty.
When my turn came to interact with the plastic bag, it swerved up towards my windshield. I admit that I flinched for a brief moment, even though logically I knew that it couldn’t penetrate the windshield and would not be able to strike me. It’s just a natural reaction we all have that if something aims toward your head, you are likely to flinch in an effort to try and avoid it.
There is roadway debris that can turn into flying debris, as exemplified by the mattress and also the black plastic bag, but that’s not the only source of flying debris.
Flying debris can appear seemingly out of nowhere and originate via other means.
A few years ago, I was driving along, minding my own business, when all of a sudden a golf ball struck my windshield.
I had not even noticed beforehand that I was driving near a golf course, and so the loud smack upon my windshield was completely out-of-the-blue. The golf ball was moving so fast when it struck the windshield that I didn’t even see it mid-flight beforehand.
Once it hit, I then watched in horror as it cracked the windshield and then bounced away from my car. I was lucky it didn’t penetrate through the windshield. In spite of the shock of the whole situation, I managed to continue driving forward and didn’t swerve or otherwise react to the striking golf ball.
Some drivers might have reacted and maybe gotten themselves into deeper trouble by swerving into other cars or possibly even driving off the highway and into a telephone pole.
On another occasion, I was driving under a freeway overpass, once again minding my own business, when all of a sudden a water balloon struck the windshield of my car.
The water smeared across the windshield and for a brief heart-stopping moment I could not see out the windshield. Miraculously, my instinct took over and I turned on my windshield wipers to get the water out of my field of vision. This incident could have gone real bad, real quickly. Turns out there was some young thugs tossing water balloons onto cars, doing so from the overpass. They though it was pretty funny. You can imagine the kind of deathly car accidents that could have occurred. Not funny at all.
How do human drivers react to flying debris?
Sometimes they swerve to try and avoid it. Or, they might hit the brakes in hopes of not striking the flying debris.
Or, try to accelerate and get under it before it can land onto their cars.
There are even some drivers that they themselves duck their heads and lose control of their cars, reacting as though the flying debris is going to physically harm them. Sometimes a driver just freezes up and slams into the flying debris.
Overall, the circumstances and the nature of the driver are key factors in how the driver will react. Some drivers are cool and calm, others get flustered. Some situations involve debris that is relatively harmless, other situations involve debris that can do severe harm.
Factors And Driver Reactions
Via hindsight, you could say that the drivers that swerved to avoid the mattress fluff were wrong and perhaps “stupid” for having endangered other cars when they instead could have just driven straight into the stuffing.
This is a hard call to make, though. At the moment of seeing the debris, you often have just a split second to decide what to do.
And, your view of the debris might be different from other nearby drivers. Maybe you did not see that there was a mattress up ahead laying on the freeway, and so the white floating debris was an unknown item. You might not have been able to discern what the objects were, and so did the “safest” thing which seemed to be to try and avoid it.
There are a multitude of factors that come to play in dealing with flying debris, including:
- Actual danger
- Traffic conditions
- Car capabilities
- Driving skill
- Driver awareness
- Weather conditions
- Roadway status
According to a study that looked at debris related incidents in the United States during the time period of 2011 to 2014, there were an estimated 200,000 car crashes that were attributable to debris situations.
Of those 200,000 car crashes, there were about 39,000 people that were injured, and approximately 500 deaths.
It’s a staggering and sobering set of statistics.
Furthermore, we don’t know how many debris related incidents in total there were since these counts are only for those that led to a car crash. Suppose that for every debris related car crash there are 10x or maybe even 100x the number of incidents (but, who knows?), and so there could be possibly millions of debris related incidents that occur.
Often, the reaction to the debris is what leads to the car crashes and injuries.
Chain Reaction Can Worsen The Situation
A car that swerves can end-up hitting another car, which then hits another car, and so on.
The chain reaction that originates by the initial reaction to the debris can be the mainstay of how the car crashes arise.
It’s hard to say how many accidents could be avoided if the cars just drove through the debris and did not swerve or otherwise react. You could argue that maybe if we all agreed to always just drive straight through debris, we’d have lessened counts of car crashes that were debris related. It’s hard though to be the driver of a car and realize that maybe if you are willing to get injured that it can save the lives of others and so be willing to not swerve your car.
That’s a tough pill to swallow.
AI Autonomous Cars And Flying Debris Aspects
What does this have to do with AI self-driving driverless autonomous cars?
At the Cybernetic Self-Driving Car Institute, we are developing software to aid in the debris reaction of the AI systems.
We initially focused on debris sitting on the roadway (see https://aitrends.com/selfdrivingcars/roadway-debris-cognition-self-driving-cars/), and then have progressed into the flying debris aspects. AI self-driving cars that only deal with roadway debris are ill-prepared for flying debris.
That being said, some have said that they think an AI self-driving car should just ignore flying debris.
The logic is that for a Level 5 self-driving car, which is a car that can be driven by the automation entirely and needs no human driver, there isn’t a human driver that can get injured by flying debris so why worry about it. The viewpoint is that the AI can just keep driving forward and let the debris strike the car. This goes along with the aforementioned perspective that maybe we’d all be better off collectively if we didn’t react to roadway debris.
Those that argue to ignore flying debris would assert that the flying debris isn’t going to harm anything per se, and so just let it hit the car. Sure, it might crack the windshield, but so what, they say. This doesn’t account for circumstances though that involve flying debris that can penetrate the windshield and then strike the humans inside the car. Just because there isn’t a human driver doesn’t mean that there aren’t people inside the car.
We still need to consider how to protect the human occupants.
The counter-argument is that the number of instances of flying debris that actually gets into the car compartment is very low, and so rather than trying to avoid the flying debris, it’s safer overall to instruct all cars to just drive through it.
One must also consider though that the flying debris could strike and break some of the sensors of the self-driving car, such as cameras mounted on the hood or a LIDAR system mounted on the top of the car.
Harming the automation is certainly not the same as harming the humans, but at the same time keep in mind that if the automation gets harmed, it could mar the ability of the AI to drive the self-driving car, and a sudden lose of crucial sensors could lead to the self-driving car being unnavigable and lead to harm of humans both inside and outside of the vehicle.
The Utopian self-driving car pundits would say that via V2V (vehicle-to-vehicle) communications of self-driving cars, we can ensure that swerving cars don’t hit each other, and so it would be Okay to potentially swerve a car to avoid hitting debris.
Using the exploding mattress example, presumably the first car that spots the mattress would electronically alert the other nearby cars. The other nearby cars would have their respective AI provide clearance to allow the detecting self-driving car to swerve. These self-driving cars would all then alert any upcoming self-driving cars to get out of the lane that contains the debris. The floating stuffing would be ranked as harmless by the self-driving cars and they would communicate to the other traffic that its Okay to just run straight through the fluff.
The depicted Utopian world is pretty nifty, but we’re not going to have that world for many years to come.
For a long time, we’re going to have a mixture of both self-driving cars and human driven cars. There’s not an easy way to alert the human drivers about the situation and so the V2V isn’t going to get us the fully orchestrated and seamless dance of cars that the Utopians envision.
Flying Debris Is A Hard Problem
The flying debris problem is multi-faceted and not so easy to solve.
Is there a single piece of debris or are there multiple pieces of debris?
Does the debris appear to be harmful or relatively harmless?
What is the predicted path of the flying debris?
Is the flying debris being ricocheted off of other cars?
How are other cars reacting?
And so on.
The sensors of the AI self-driving car need to be detecting that the flying debris situations exists. Via the use of the cameras, radar, ultrasonic, LIDAR, and other sensors, the AI needs to interpret the incoming data and identify that there’s a debris situation arising. In many cases, the debris itself won’t be seen right away and instead the reaction of the other cars will be the clue that maybe debris is involved.
You might recall that in my story earlier that I had noticed other cars dodging and weaving in the mattress instance, prior to my then seeing the white floating objects.
During sensor fusion, the AI is trying to piece together the telltale clues, which can be difficult to do because maybe the cameras don’t yet see anything amiss, but perhaps the LIDAR has detected the flying debris.
Each of the sensors has its own limits and capabilities. Some of the sensors can do detection at a greater distance than the others. The AI needs to update the virtual model of the surrounding environment and be preparing for a potential debris situation. As time clicks from split second to split second, the additional data flowing into the AI via the sensors will step-wise unveil what is happening in the driving situation.
The AI needs to be crafting an action plan of what to do.
Stay in the same lane or switch lanes? Try to make a swerve or continue straight ahead? Apply the brakes? Add acceleration?
These are all considered and must be placed into a sequence that can be applied. Time is crucial. Whatever sequence is developed must include the time to undertake the actions. Furthermore, once the sequence is started, the circumstances can change rapidly and so the action plan will need to be adjusted too.
The AI emits commands to the controls of the self-driving car and must then pay attention to how the situation evolves.
Even getting the AI to do all of this for dealing with flying debris is only part of the solution to the problem.
Human Occupants In Self-Driving Cars
We also need to consider the human occupants in the car.
Some say that the AI should just proceed to do whatever it needs to do and there’s no need to alert the human occupants. They are just along for the ride.
But, this seems rather shortsighted.
One could argue that the AI should warn the passengers that there is flying debris. It might give the human occupants a moment to prepare themselves for what might happen next, such as the swerving of the car and perhaps the impact by the debris.
Others say that this is just going to scare the human occupants and since they aren’t able to drive the car then it doesn’t matter that they know in-advance what’s going to happen. If anything, they might panic and tense up, perhaps getting further harmed or even needlessly harmed by the forewarning.
This takes us into the ethics aspects of AI self-driving cars.
Do we want our self-driving cars to drive and not inform the human occupants about what is happening?
This question is notably a one-way question and misses the other possible direction – suppose the human occupants are the first to spot the debris, and they want to alert the AI self-driving car that something is amiss, shouldn’t they be able to do so? In other words, we have situations of the AI warning the occupants, and then the other situation of the occupants warning the AI.
What’s even tougher in this matter is whether to have the humans be able to advise or direct the AI of the self-driving car.
Suppose a human occupant yells out that there’s debris and tells the AI to swerve to the left. Meanwhile, let’s imagine that the AI has detected the debris and wants to drive straight through it.
Should the AI proceed as it has planned, or should it do what the human says to do?
You might say that the AI should interact with the human and reach an agreed solution, but realistically if the amount of time is split seconds then you’re unlikely to have the luxury of a measured debate among people and machine.
We also need to consider the time aspects in the sense of stages, namely (1) pre-debris, (2) debris detection, (3) debris encounter, and (4) post-debris aspects.
In the HCI (Human Computer Interface), you could have a dialogue of the AI and the humans about what to do if there are any debris situations that might arise during a driving journey (this is during the pre-debris stage).
For debris detection, the AI might alert the humans that debris has been detected and likewise might get input from the humans about the debris, depending upon the time allowed for such an interaction.
During the debris encounter, the AI might be informing the humans about what is taking place. And, post-debris the AI might let them know what damage if any has occurred to the self-driving car, and likewise the humans might alert the AI to damages such as if the humans themselves have gotten injured (including indirectly perhaps due to whiplash by a swerving action of the car).
The flying debris problem is considered by many of the auto makers and tech firms as a so-called edge problem of self-driving cars.
This means that it is not considered at the core of the self-driving car task. Little focus to-date is being put towards the flying debris issue.
Nonetheless, per driving stats, it is well-known that debris is an issue for cars on the roadway and we’ll need to ultimately figure out how to have AI self-driving cars that can properly take action in the face of debris, whether it’s on the roadway or flying, or both.
This also includes ensuring that there is appropriate interaction with the human occupants of the self-driving car.
They do indeed matter.
Copyright 2019 Dr. Lance Eliot
This content is originally posted on AI Trends.