Is it ok to just use git only locally? I don't want to have to pay for a service that provides private repositories (such as Github) but I think git is a great way to organize my closed-source project.
While it's entirely reasonable and possible to use git locally, it's better to have backup.
You can arbitrarily push repos to basically anywhere. Github just happens to be easy hosting and collaboration. There are other options such as using Google Drive or Dropbox if you want remote storage.
Another great argument for using git locally (i.e. only one copy in the universe) is
git bisect, which can be used to find many a nefarious bug and has saved me many times. It allows one to narrow down exactly which commit an annoying bug was added (and thus lets you focus on a much smaller segment of problem code).
The only "drawback" of using Git locally(compared to not using any SCM at all) is the extra work of committing, branching and tagging - and that extra work is not only neglected in the grand scheme but also contributes directly to organizing your code and documenting your progress.
Also keep in mind that Git is a distributed SCM. While the common case is to use it with a central repository, it was designed to be usable without one. You can create patches(or even better - bundles - which are a pack of patches[a single patch can only contain a single commit]) and send them by mail to your colleagues or save them as backups. This is less comfortable than using a central repository(that's why people usually prefer to use one), but it allows you to use Git for collaboration without paying for a private repo or hosting one on your own server.
Think about what you need and what you're going to use it for - your requirements.
If it's code for a spike for a couple of hours, just write it may be ok.
If you get to the point where you start to (or feel the need to) make backups to 'save working versions', then a version control system can help.
Also, if you get code to a working point and want to do your own spike, it can be handy to do a branch which you can then merge back in, as shown below where I am the sole contributor and this repo has never been pushed anywhere ('uncouple methods' was a branch I did for a spike):
Of course if you are also maintaining software that has several versions and/or releases, a version control system is very helpful, especially with the ability to merge changes.