Most functional languages use linked lists as their primary immutable data structure. Why lists, and not e.g. trees? Trees can also reuse paths, and even model lists.
Because lists are simpler than trees. (You can see this trivially by the fact that a list is a degenerate tree, where every node has only a single child.)
The cons list is the simplest possible recursive data structure of arbitrary size.
Guy Steele argued during the design of the Fortress programming language that for the massively parallel computations of the future, both our data structures and our control flow should be tree-shaped with multiple branches, not linear as they are now. But for the time being, most of our core data structure libraries were designed with sequential, iterative processing (or tail recursion, it doesn't really matter, they are the same thing) in mind, not parallel processing.
Note that e.g. in Clojure, whose data structures were designed specifically for the parallel, distributed, "cloudy" world of today, even arrays (called vectors in Clojure), probably the most "linear" data structure of them all, are actually implemented as trees.
So, in short: a cons list is the simplest possible persistent recursive data structure, and there was no need to choose a more complicated "default". Others are of course available as options, e.g. Haskell has arrays, priority queues, maps, heaps, treaps, tries, and everything you could possibly imagine, but the default is the simple cons list.
Actually, those lists are trees! You have nodes with two fields,
cdr, which can contain more such nodes, or leaves. The only thing that makes those trees into lists, is the convention to interpret the
cdr link as a link to the next node in a linear list, and the
car link as the value of the current node.
That said, I guess that the prevalence of linked lists in functional programming is linked to the prevalence of recursion over iteration. When your only looping construct at hand is (tail-)recursion, you want data structures that are easy to use with that; and linked lists are perfect for that.