According to the Open Source Initiative, no. In order for a license to be considered open source, there are 10 clauses. One of them is that an open source license can not discriminate against a person or group:
The license must not discriminate against any person or group of persons.
The Free Software Foundation also provides a description of free software. It appears that such restrictions would also make the software nonfree according to them. It's not explicitly mentioned, but since nonfree software is any software license that prevents use, redistribution, or modification or requires for permission first. Since you are preventing at least one individual from using, redistributing, or modifying the software, I believe it would fall under their definition of nonfree:
Nonfree software is any software that is not free. Its use, redistribution or modification is prohibited, or requires you to ask for permission, or is restricted so much that you effectively can't do it freely.
The software license is the formal location for the terms of acquiring, using, and distributing the software, putting it in there would likely be what is invoked in any kind of legal discussion regarding the software. In addition, if you included such a restriction in your license, you may not be able to use services that are freely available to free and/or open source projects, such as Coverity Scan or hosting on Google Code.