Having a dedicated QA team in the way that proprietary software often have makes less sense when you have an open development process and openly accessible bug tracker, as a good chunk of your most important users will be watching activities in your repository, testing prereleases (nightly/alpha/beta builds), filing bug reports, or even be involved in the development process, so that undesirable change in direction, bugs and regression are often discovered that way.
All in all though, many open source projects do have formal QA process with dedicated QA teams, for example Firefox, Debian, Gentoo, LibreOfice.
In general, automated testing and continuous integration and testing build system is vastly preferred in open source world, because in open source projects people and volunteers come and go, but automated tests will stay around. The efforts needed to document a manual testing processes and train dedicated QA members are often not much less than the effort needed to automate the test. And users who runs nightly/alpha/beta builds fulfills the needs that are normally filled by QA.
You may also notice that in many open source projects the role of QA may often cross with community participation activities, this is because QA is generally the easiest way for non technical users to get involved in the project.
Also, don't get too hung up with the terminologies, even in proprietary software development, development teams rarely distinguish between unit tests and higher level testing. People often just call any automated tests as unit tests, even for selenium-type UI level testing.