The quicker you make your code publicly available, the quicker you can gain feedback and people to help you. If your intention is to make the project open source from the beginning, then I would recommend starting your project out as public by default.
Github is full of small and unfinished projects so your project should fit right in. The more details you put in the readme file the better as it will help other developers/consumers get up to speed on your project quickly.
At the very least, your private projects should be under some sort of version control. If you don't want to pay for a service, then I'd recommend using Dropbox to back up your private local repositories. This way you have file backup and version control on your project which will save you from hours of pain in the future. More recently, GitHub and its competitors have released free private repositories, so you can use your version control solution of choice privately without a paid subscription.