You don't have to distribute the source code at all until someone asks for it. But if you choose not to distribute it along with the executable, you do have to provide a "written offer" for the source code, which I assume would have to include instructions on how to contact you and ask for it.
From the GPL v2 preamble:
You must make sure that they, too, receive or can get the source code.
So it's fine if they don't receive the source code right away, as long as you provide some way for them to get it later if they want it.
I'm assuming you're already distributing the program in executable form, so the relevant part the main body of the license is:
3 You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.)
All emphasis mine.