Over the previous few days, Elon Musk has been making a number of statements claiming that autonomous driving methods that use lidar and radar sensors are extra harmful than Tesla’s camera-only laptop imaginative and prescient method as a result of the system will get confused when deciphering knowledge from completely different sensors.
It’s not solely false, Musk informed me instantly that he agreed that radar and imaginative and prescient could possibly be safer than simply imaginative and prescient, proper after he had Tesla take away the radars from its automobiles.
Tesla has taken a controversial method, utilizing solely cameras as sensors for driving inputs in its self-driving expertise. In distinction, most different corporations use cameras at the side of radar and lidar sensors.
When Tesla first introduced that every one its vehicles produced onward have the {hardware} able to “full self-driving” as much as degree 5 autonomous capability in 2016, it included a front-facing radar in its self-driving {hardware} suite.
Nevertheless, in 2021, after not having achieved something greater than a degree 2 driver help (ADAS) system with its self-driving effort, Elon Musk introduced a transfer that he known as “Tesla Imaginative and prescient”, which consists of transferring Tesla’s self-driving effort solely to make use of inputs from cameras.
Right here’s what I wrote in 2021 about Musk sharing his plan for Tesla to solely use cameras and neural nets:
CEO Elon Musk has been hyping the vision-only replace as “mind-blowing.” He insists that it’ll result in a real degree 5 autonomous driving system by the tip of the yr, however he has gotten that timeline mistaken earlier than.
By Might 2021, Tesla had begun eradicating the radar sensor from its lineup, beginning with the Mannequin 3 and Mannequin Y, and later the Mannequin S and Mannequin X in 2022.
Tesla engineers reportedly tried to persuade Musk to retain the usage of radar, however the CEO overruled them.
We are actually in 2025, and in contrast to what Musk claimed, Tesla has but to ship on its self-driving guarantees, however the CEO is doubling down on his vision-only method.
The controversial billionaire is making headlines this week for a sequence of recent statements attacking Tesla’s self-driving rivals and their use of radar and lidar sensors.
Earlier this week, Musk took a jab at Waymo and claimed that “lidar and radar cut back security”:
Lidar and radar cut back security attributable to sensor competition. If lidars/radars disagree with cameras, which one wins? This sensor ambiguity causes elevated, not decreased, danger. That’s why Waymos can’t drive on highways.We turned off radars in Teslas to extend security. Cameras ftw.
The assertion that “Waymos can’t drive on highways” is just false. Waymo has been conducting absolutely driverless worker testing on freeways in Phoenix, San Francisco, and Los Angeles for years, and it’s anticipated to make this expertise accessible to rider-only rides quickly.
Tesla is in the same scenario with its Robotaxi: they don’t drive on freeways with out an worker supervisor.
Musk later added:
LiDAR additionally doesn’t work properly in snow, rain or mud attributable to reflection scatter. That’s why Waymos cease working in any heavy precipitation. As I’ve stated many occasions, there’s a position for LiDAR in some circumstances and I personally oversaw the event of LiDAR for the SpaceX Dragon docking with House Station. I’m properly conscious of its strengths and weaknesses.
It’s not true that Waymos can’t work in “any heavy precipitation.”
Right here’s a video of a Waymo automobile driving by itself in heavy rain:
As compared, Tesla’s personal Robotaxi phrases of service point out that it “could also be restricted or unavailable in inclement climate.”
Final month, Tesla Robotaxi riders had their rides lower quick, and so they have been informed it was as a result of rain.
There’s loads of proof that Musk is mistaken and deceptive with these statements, however moreover, he himself admitted that radar sensors could make Tesla’s imaginative and prescient system safer.
‘Imaginative and prescient with high-res radar could be higher than pure imaginative and prescient’
In Might 2021, as Tesla started eradicating radar sensors from its automobile lineup and transitioning to a vision-only method, I used to be direct messaging (DMing) Musk to be taught extra concerning the stunning transfer.
Within the dialog, he was already making the declare that sensor competition is reducing security as he did this week in new feedback attacking Waymo.
He wrote on the time:
The chance of security might be increased with pure imaginative and prescient than imaginative and prescient+radar, not decrease. Imaginative and prescient has change into so good that radar really reduces sign/noise.
Nevertheless, what was extra fascinating is what he stated shortly after claiming that:
Musk admitted that “imaginative and prescient with high-resolution radar could be higher than pure imaginative and prescient”. Nevertheless, he claimed that such a radar didn’t exist.
In the identical dialog, I pointed Musk to present high-definition millimeter wave radars, however he didn’t reply.
It was nonetheless early for that expertise in 2021, however high-definition millimeter wave radars are actually generally utilized by corporations growing autonomous driving applied sciences, together with Waymo.
Waymo makes use of six high-definition radars in its system:
In brief, Musk was already involved about sensor competition in 2021, however he admitted that the issue could be value fixing with higher-definition radars, which already existed then and have gotten extra widespread now.
But, he criticizes corporations utilizing radar and lidar, which work equally to high-resolution radars however on completely different wavelengths, for even trying sensor fusion.
It’s not unimaginable as a result of Tesla can’t do it
A part of the issue right here seems to be that Musk thinks one thing doesn’t work as a result of Tesla can’t make it work, and he doesn’t need to admit that others are fixing the sensor fusion downside.
Tesla merely couldn’t resolve sensor fusion, so it targeted on attaining autonomy solely by means of digicam imaginative and prescient. Nevertheless, those that continued to work on the problem have made vital progress and are actually reaping the rewards.
Waymo and Baidu, each of which have degree 4 autonomous driving methods presently commercially working with out supervision, in contrast to Tesla, have closely invested in sensor fusion.
Amir Husain, an AI entrepreneur who sits on the Boards of Advisors for IBM Watson and the Division of Laptop Science at UT Austin, factors to developments in the usage of Kalman filters and Bayesian methods to resolve sensor noise covariance.
He commented on Musk’s assertion relating to the usage of radar and lidar sensors:
The problem isn’t a binary disagreement between two sensors. It generates a greater estimate than any particular person sensor can produce by itself. All of them have a margin of error. Fusion helps cut back this.
If Musk’s argument held, why would the human mind use eyes, ears, and contact to estimate object location? Why would plane mix radar, IRST, and different passive sensors to estimate object location? It is a basic misunderstanding of data principle. Each channel has noise. However redundancy reduces uncertainty.
Musk’s principal argument to deal with cameras and neural nets has been that the roads are designed for people to drive and people drive utilizing their eyes and mind, that are the {hardware} and software program equal of cameras (eyes) and neural nets (mind).
Now, most different corporations growing autonomous driving applied sciences are additionally specializing in this, however to surpass people and obtain higher ranges of security by means of precision and redundancy, they’re additionally including radar and lidar sensors to their methods.
Electrek’s Take
Musk painted Tesla right into a nook with its vision-only method, and now he’s making an attempt to mislead individuals into pondering that it’s the just one that may work, when there’s no substantial proof to help this declare.
Now, let me be clear, Musk is partly appropriate. When poorly fused, multi-sensor knowledge introduces noise, making it more difficult to function an autonomous driving system.
Nevertheless, who stated that that is an unsolvable downside? Others seem like fixing it, and we’re seeing the ends in Waymo’s and Baidu’s commercially accessible rider-only taxi companies.
In the event you can reap the benefits of radar’s means to detect distance and pace in addition to work by means of rain, fog, mud, and snow, why wouldn’t you employ it?
As he admitted within the DMs with me in 2021, Musk is conscious of this – therefore why he acknowledged that high-resolution radar mixed with imaginative and prescient could be safer than imaginative and prescient alone.
The issue is that Tesla hasn’t targeted on enhancing sensor fusion and radar integration within the final 4 years as a result of it has been all-in on imaginative and prescient.
Now, Tesla might probably nonetheless resolve self-driving with its imaginative and prescient system, however there’s no proof that it’s near taking place or any safer than different methods, resembling Waymo’s, which use radar and lidar sensors.
The truth is, Tesla continues to be solely working an autonomous driving system below the supervision of in-car staff with a number of dozen vehicles, whereas Waymo has been doing rider-only rides for years and operates over 1,500 autonomous automobiles within the US.
Similar to together with his “Robotaxi” with supervisors, Musk is making an attempt to create the phantasm that Tesla isn’t solely main in autonomy, however it’s the just one that may resolve it.
