Private set modifier is used to control the access to object's property value.
Public set modifier is directly accessing the object's property value.
When you have a closed Domain, as a rule of open/close principle, private set modifiers used to restrict direct access to modify the properties of the object.
Private setters were not available before .NET 2.0 Framework. This has meant that people have written separate SetXXX methods if they wanted a public getter but a more limited setter.
public class MyTest
public string SomeProperty
someProperty = value;
The basic rules are that you can't specify an access modifier for both the getter and the setter, and you have to use the "extra" modifier to make access more restrictive than the rest of the property.
Your question example comes from MSDN article Using Properties (C# Programming Guide).
Look at that again and it will clear up the things.