I live in Brazil and I'm planning on hosting my website in either DreamHost or Digital Ocean, which don't have any servers in my country. Will that affect the loading speed of my site?
The speed of light is 299 792 458 m/s. The circumference of Earth is 40 075 000 m. This means that if the data center is at the opposite side of the Earth, the delay will be at least 0.067 s., that is 67 ms. Noticeable for critical applications where every millisecond counts, such at the home page of Google or a financial sector app; not noticeable for a personal blog or a forum.
The closest Amazon data center, for instance, would be probably in Jacksonville, Florida, USA. The distance is approximately 6 000 km to the center of Brazil (if you're in the south of Brazil, you're out of luck), which translates into 20 ms.
You should also take in account that the time the packets are spent traveling through routers, which may increase the waiting time, and that they hardly travel in a straight line. This being said, the additional time is still probably not too much for an ordinary website.
Conclusion: if the hosting company has several data centers around the world (such as Amazon AWS), hosting a web app in geographical proximity to the customers can be nice. But for most web apps, it's not what will make a radical change in the perceived speed of your app.
Yes, it matters. The signals may travel at light speed (or a reasonable fraction) but there are many rest stops called routers along the way. Your data gets stopped, buffered, and stuffed down another pipe, competing with other traffic for bandwidth. The fewer routers in the path, and the fatter the pipes, the better your site will load.
That said, unless your site is a twitch-level shooter, or you're sending gigabytes of data, your users probably won't notice. The main thing that matters will be the bandwidth available from your actual host. Cheap connections may not provide much (you get what you pay for).