File systems can span multiple physical disks. You may be familiar with the concept of “partitions”, where one physical medium is split into multiple logical volumes, e.g. a hard disk in a typical Windows installation might have a
D: partition. However, this doesn't have to be a 1:n mapping and some file systems can handle n:m mappings between drives and volumes/file systems.
The main motivation for an abstraction level between logical volumes and physical disks is not the potential larger storage capacity, but more flexibility and better fault tolerance, especially in a server setting: when a hard drive fails, I want the server to continue running without interruption or downtime, even when I replace the failed disk. This fault tolerance implies that the same data is replicated on multiple hard disks (e.g. in a RAID-6 setup); the ability to hot-swap a drive requires the applications to be unaware of physical disks (i.e. only use the file system, not access the device directly).
This functionality can be provided by hardware disk controllers that present themselves to the OS as a single disk, but actually contain multiple drives. However, there are software approaches as well. On Linux, you can use the Logical Volume Manager to implement multi-drive partitions, move partitions between drives, or to hot-swap drives. The ZFS file system was created to support massively big amounts of data, and includes sophisticated management of large pools of physical drives. A ZFS file system is spread over all disks in its pool, and can support hot-swapping if in a suitable RAID configuration.
For example, I have configured a server with a ~1TB ZFS pool out of 10 × 150GB disks. The RAID level chosen can withstand two disk failures, and one of the disks is a hot spare that will be used to restore the pool until the faulted disks can be swapped out. Obviously, this could be scaled up with larger disks. E.g. with 20 2TB disks, I would configure them into a 30TB pool.
The RAID level chosen does have performance implications. Since a file is typically spread over multiple disks, reading it can use the combined bandwidth of all drives. However, write performance is reduced when the same file is written to multiple drives for fault tolerance.
With techniques such as RAID or Storage Area Networks, there is a lot you can do to expand the storage capacity of a system. However, at some point data can get too big to be managed on a single system, e.g. if you have too much throughput or need system-level rather than disk-level fault tolerance. In such a scenario, a different, distributed software architecture is needed to scale up further, though hard issues around data consistency arise once you have more than one system that are responsible for the same data.