Whilst both strings and integers are immutable, they aren't the same. Strings are immutable reference types; integers are immutable value types.
If you have a string variable, s, then it simply holds a reference (a pointer) to the location in the heap where that string object is stored. If you change the string that s refers to, then potentially a new string object is created on the heap and the original one is then eligible for garbage collection if no other references to it remain.
If you however have an int variable, i, then it holds the integer value. If you change the number held by i, then that value changes. There is no new heap allocation.
It certainly isn't the case that if you change strings, "the GarbageCollector has to collect them all the time". The garbage collector only runs when it needs to; it's quite possible that during the life of your app, it'll never run. And even when it does run, it can be very fast and not cause performance problems. I'm not saying it never has performance problems; but I am saying that changing strings all the time does not guarantee performance problems.