It’s only the fastest under the assumption of constant speed. Of course, the world has so many factors that accelerate and decelerate us. Motivation, interest levels in our works, current events, etc.
In physics, the Brachistochrone problem was a problem formulated to find what curve would give the fastest movement of a ball from point A to point B under the acceleration of gravity. The answer doesn’t come out to be a straight line.
Stop thinking in straight paths. Going from here to there in a straight line isn’t likely to reach your goal the fastest as you think. Instead, figuring out what accelerates you, while also not making your path too long is the goal. It’s a balance. It requires learning and adjusting.