while attention has uses outside of language models, it is not even debatable that it has had the largest effect in language by a mile IMO.
Sure. I agree
I am just noting that attention networks aren't exclusively used by NLP (although they were developed there).... and there are tasks within autonomous driving that could benefit from attention networks.
saying that the hype of deep learning (as it currently exists) is disproportionate to its actual results in autonomous driving is IMO, about 10 degrees too kind. I say this as a person with multiple DL models in production.
Yes and no.
Without being too emphatic, all I am trying to communicate is that deep-learning is not a silver bullet. Not necessarily to you... but to anyone who might read the post. Autonomous driving... deep learning... AI.... big data... a lot of these terms get thrown around and conflated in popular culture. I am sure you and I might agree on what is realistic and what are unfounded expectations
However, it is well worth recognising the contribution deep learning has made to computer vision at a practical level. You cant really do high-level computer vision research without stubbing your toe on deep learning. This very much includes autonomous driving. Any autonomous vehicles using machine vision will be running some sort of deep learning network to interpret the video stream. I agree... forget the hype. In some senses deep learning for perception tasks is routine and even mundane. But since it has become almost a prerequisite, I feel one has to be forced to acknowledge that its impact on autonomous driving has been enabling and significant (albeit, not the first coming of robotic jesus).
Im not 100% sure what your argument is.
I think where we differ is the role of technological 'step changes'.
autonomous driving is probably a solvable task in the long run but so are lots of problems that got worked on for decades. that's my suggestion here; without a major change, it doesnt matter how many things you try and bolt together like an erector set cars wont drive themselves
Like you say! Time is progress! I think this the stronger position.
Artificial neural networks are what? More than half a century old? Backpropagation decades old... 'Modern' stochastic gradient descent and running networks on dedicated hardware (GPU/TPUs) are about 15 years old? All these leaps can smash one problem and subsequently open many doors... but they hardly ever solve very general and difficult problems.
You dont need particularly general AI to solve autonomous driving (my previous silly remark about changing light bulbs). But the system can't be a narrow AI either. Writing a championship level chess algorithm, or finding cats in pictures... or translating literature from spanish to english are all difficult problems. But we can solve those problems with highly optimised algorithms for a dedicated task (narrow AI). Major leaps have a more profound effect on these well defined tasks. Driving is a mix of complex and interacting requirements... so the system has to reflect that. In other words, be slightly less narrow.
I think the secret sauce
will be in learning how to bolt things together. Not one great leap.
This is my point. Sure... very specific major changes will help... but I just dont imagine a major leap in one subsystem solving the global problem. Like I mentioned earlier you could crudely break down autonomous driving into perception, planning and control - indeed any robotics problem. Deep learning has excited people because it is having a profound impact on perception systems. But that doesnt necessarily solve the planning problem... you might use deep learning... you might not. You can have a really technical and complex discussion about localisation without even mentioning deep learning... and so it goes.
And this is where we are. We have really competent building blocks that make up something like 95% of an autonomous car... but that 5% is really important and it is going to be a hard slog completing. It is likely this will come through improvements in many fields. Some major. Many more incremental.