If advanced algorithms and data structures are never used in industry, then why learn them？

Here is one question from Quora. Just record it here...

I and people I know have never used any advanced algorithms or data structures in industry and some of us have 20 years of experience in industry. How do you motivate yourself to learn advanced algorithms and data structures or, for that matter, go for Programming contests?

A) The way I see it, it is not algorithms that I need in everyday life, but the ability to quickly analyze the problem and find a solution. If you can solve a TopCoder problem in 2 minutes, or several difficult ones on ACM under 5h then chances are that you will be able to understand mumbling of the product owner and create a good technical specification with nice architecture and neatly implement it, meanwhile explaining flaws of the original functional specification to the product owner in a comprehensive, basic way. I believe that these 10 years of programming competitions was what made a difference between being a programmer and being an architect.

Also, I think that understanding how a B-tree is implemented can help you understand performance issues with your SQL query. This is just an example of situations where you do not implement an algorithm or a data structure in a project, but still benefit from understanding why and how something works. I would say that knowledge about compression algorithms, randomized algorithms etc also helped me more than once at work.

Also, if you know how to implement a cool data structure or an algorithm using C++ STL primitives, chances are you will be able to reproduce it using Redis + LUA, or at least know what you could hypothetically do using them. I found that knowing lower and upper bounds, helps you (as an architect) make decisions. For example if you know, that "it could run in O(n) but we would need 2 months to build and test it" and "it could run in O(nlgn) using a simple MySQL" then you know exactly what to do. In a sense it is like "approximation algorithms" but the main resource are people, and maintainability, and your adversary is the OPT algorithm you learned at university.

B) This is like a football player asking "Why lift weights when a football weighs less than a pound?" The point is to train yourself to the point where the algorithms problems you actually run into are all trivial, and you can devote your mental resources to building good systems instead of trying to remember how quicksort works.

C) First, you need to know how algorithms and data structures work when choosing which one to use from an API. You want to store key/value pairs in a data structure? You know you can use a Hash Map which gives you O(1) for insertion/seek/delete. Oh, but then you realize you need to iterate the values in certain order. Do you know how a Hash Map works? How does it grow? How does the algorithm decide the order of the elements? Or is it completely random? So you use a Binary Tree, and you may end with an unbalanced implementation that is even slower than using just an array of tuples and iterating it. Maybe a Red Black Tree is a better option, it's balanced so it gives you O(log N) for insertion/seek/delete and you can iterate it by the natural ordering of the key. But how are you going to make an informed choice without knowing how does each algorithm and data structure works?

Second, if you are speaking about sorting algorithms and the like, yes, you may never get to implement one in real life as most APIs come with a heavily optimized implementation of basic algorithms. But try to implement text prediction without knowing how a Trie works, or solving any path/distance/proximity/networking problem without even basic Graph and BFS/DFS knowledge.

Yes: you can make a living developing CRUD applications without any knowledge of advanced algorithms and data structures, just like I could have a 56k modem at home and say I'm 'connected' to the internet. But that only speaks about the limitations with which I chose to live; it does not mean that anything beyond those limitations is useless.

Ref:

http://www.quora.com/Algorithms/If-advanced-algorithms-and-data-structures-are-never-used-in-industry-then-why-learn-them?srid=3Pug&share=1