Computers are not Turing Machines. They are Deterministic Finite State Machines.
Turing Machines have infinite memory, computers have finite memory. Turing Machines have arbitrarily many (though finite) states, computers can't have arbitrarily many states, the number of different states that a computer can be in is bounded by its memory (a computer with 1 KiByte of memory and 4 8-bit registers can only be in 8*21024 + 4*28 =~ 10300 distinct states).
The program itself is also stored in memory, therefore it is itself part of the state. Any program which runs long enough must at some point end up in a state in which it was once before. And since the program itself is part of the state and computers are deterministic, this means that it will do the exact same thing it did the last time it was in this state, thus it will end up in the same followup-state it ended up the last time, which means that it will now do the same thing it did … and so on. Any programs which runs long enough will at some point end up in a state it was in before and from that point on run in exactly the same way it did the last time.
This length of steps when the program starts to repeat itself is called the period. Every program has one, not just Pseudo-Random Number Generators.
Note that mostly, those periods are theoretical. Even if the 1 KiByte computer I mentioned above had a 5 THz CPU, it would still take 10270 times the current age of the universe to cycle through 10300 states.
However, most programs, and certainly most PRNGs don't use the entire memory of the computer just for their own state. (PRNGs in particular are usually embedded into much larger programs and thus need to "steal" their memory from them.) Plus, programs don't cycle through all possible states perfectly, typically, they can only be in a much smaller number of states. The period length can never be longer than the number of internal states, and is typically shorter. For a badly designed PRNG, it is way shorter.
The RANDU PRNG that was used on the IBM System/360 in the Scientific Subroutine Library and ported to other mainframes was used for decades in research, simulations, statistics and so on, and it has (among other major flaws, e.g. it can only ever generate odd integers, never an even one) a period of just 229. Which means that after about only 10 billion steps you know exactly what number is going to come next. (It takes only about 40 GiByte to store the entire sequence!) If you play online black jack, you know all the cards that are coming, if you play online roulette, you know what number is going to come next, and so on.
Where it is possible for a programmer to set the period for a (pseudo)random number generator
Generally, it isn't. The period length is a property of the algorithm. There are some algorithms where you can choose the size of the internal state, which influences the length of the period.
what defines a good (optimal or desirable) period
Long. The longer the better.
How long? Well, cynically speaking: long enough that buying enough harddisks to store it will be more expensive than attacking some other part of the system (e.g. bribing the casino's datacenters janitor to let you in and install a backdoor on the server which sends you the player's cards).
and why?
I hope that's obvious now. A PRNG with a small period (say 5) will give the same 5 numbers over and over again. That's not very "random".