I was wondering if are all tasks executed by the CPU (fetching, reading, writing, etc.)? Everything is controlled, manipulated or executed by the CPU?
When you really get down to it, the lines between the different parts get less useful, so asking if "X does everything" gets a bit tricky.
If you think about it, CPU stands for "Central Processing Unit." It handles all of the processing which is centralized. In some systems, that may be all you need. If your computer consists of nothing but an ATtiny CPU and a bank of memory, then by definition all of the execution is done inside the CPU. In these simple cases, everything is passive except for the CPU, which initiates all activity.
In more powerful computers, we see more decentralized activity. More work gets done outside of the CPU. For example, many computers offer DMA (Direct Memory Access) to permit peripherals to access memory without going through the CPU. This can be a tremendous performance boost, so most computers have this in some form or another. You could put the DMA engine on the die with the CPU, but it isn't always there, and it's typically thought of as separate from the CPU.
There's also the GPU (Graphics Processing Unit). Modern GPUs are becoming more and more powerful every day, and they almost qualify as their own computer!
Even a harddrive may have some processing in today's world. Many harddrives have a processing unit built into them which handles management of the position of the heads and may do some caching.