If you hand over the management of your civilization to machines, you then lose the incentive to understand it yourself or to teach the next generation. […] Up until now, we put a lot of our civilization into books. But the books can’t run it for us. So we always have to teach the next generation. If you work it out, it’s about a trillion person years of teaching and learning in an unbroken chain that goes back tens of thousands of generations. What happens if that chain breaks?
Stuart J. Russell