I agree with the common opinion that one team member is enough to start to need a bug tracker. I'd characterize it as mandatory after you have one or two real users, but important well before your first release.
Personally, I like fossil for both source control and bug tracking. It is a complete low ceremony distributed SCM that is well integrated to a bug tracker and a wiki. And it is a single-executable install, broadly portable, and uses an internal web application as its GUI. Its home page is actually served almost entirely by a copy of fossil.
With the tracker tightly integrated with revision control, you can easily link changes to tickets, and see ticket updates in the same time line view as revisions (and wiki page edits).