When you encode a code point to code units based on UTF-8, then if the code point fits on 7 bits, the most significant bit is set to zero so that it tells you it is a character which is stored on 1 byte (or more precisely 7 bits).
If the codepoint occupies more than 7 bits, then the number of leading one bits of the first byte tell you how many code units constitute that given code point. According to the specification this sequence of one bits is always followed by a sigle zero bit which terminates it and therefore separates it from the start of the code point.
I have specific questions, please answer separately.
1) If the first byte makes it crystal clear how many bytes you should read for the codepoint, why is it that the first 2 bits of every continuation byte are set to “10”? Why are they necessary if you know exactly how many bytes are there? They seem to be wasting precious space.
2) The second question is what are the theoretical limits of UTF-8? Due to compatibility reasons, UTF-8 will always encode to a maximum of 4 code units. But others say that theoretically it is capable of encoding code points to up to 7 code units, which means that the first byte does not contain any of the code point bits. It is 7 one bits followed by the terminating zero. But if we start to make theories, then we could say UTF-8 could encode to an arbitrary amount of code units too if we did not limit the size indication to the first byte. For example the 52-bit nonexistent code point 0x8000000000000 could be stored as follows:
1111 1111 - 1100 1000 1000 0000 - 1000 0000 1000 0000 - 1000 0000 1000 0000 - 1000 0000 1000 0000 - 1000 0000
This would mean that this character is stored on 10 bytes.