Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
I'd like to use Youtube as an example: they use IDs in the form of PEckzwggd78.
Why don't they use simple integers?
Or imgur.com - they also use IDs such as 9b6tMZS for images and galleries. Not sequential integers.
Why don't they use integers (particularly sequential ones)?
In what cases is it a wise decision to use such string IDs instead of integers?
Youtube can't use sequentional IDs for two reasons:
Its databases are almost certainly distributed, making sequential numbering complicated.
It has a privacy option "Unlisted videos": those that don't show up in the search results, but are available if you know the ID.
Therefore, the video IDs should be reasonably random and unpredictable. Whether the ID is represented by digits only, or by a combination of letters and digits, is irrelevant: there is a trivial mapping from one representation to another.
2^40 items, in some architectures there are legitimate reasons for choosing a space of 2^80 or 2^120 bits. Examples of reasons are: reducing collision without technically checking for collision; using the sparseness of keys as part of making secrets hard to find (the "unlisted video"), etc.
On the form of the IDs: They're using Base64 (using the characters a-z, A-Z, 0-9, -, and _). This allows them to have 6 bits of information per character. YouTube uses 11-character video IDs, which means they can generate 26*11, or more than 7*1019 IDs. As Tom Scott put it, that's "enough for every single human on planet Earth to upload a video every minute for around 18,000 years." Base64 is also easy to work with, because 64 is a power of 2, which means every character represents an exact number of bits. We use hexadecimal (base 16) for the same reason.
On the non-sequential nature of the IDs: it means they don't need a synchronized counter between all the servers that assign IDs to videos. They can just generate a random number, check if it's already in use, and go from there. They could even assign each server a block of IDs to pick from and eliminate the duplication checking. I don't know if they're doing that, but they could.
Another reason for the non-sequential IDs is that it is what makes "unlisted" videos work. These are videos that won't show up in search results or as suggestions, but that are accessible if you've got the link. If you're using sequential counting, you can just go to a video, increase the ID by one, and the idea of unlisted videos is now broken.
Non-sequential IDs also help hide information from competitors, such as the total amount of videos, or the number of videos uploaded per timeframe.
I can highly recommend Tom Scott's video. His information is almost always both interesting and accurate.
Integers do not scale that well, a "normal" 32-bit unsigned integer will max out just over 4 billion.
They may not want you to know how many items they have on line or keep track of the rate they are growing. If your order numbers just count up it will be easy for a stranger to deduct how many orders you got in a certain period.
Letters can hold more information than digits, you need fewer letters to express the same "number". Not in binary format but in the presented format on a screen or on paper. For instance, you could have the year or an entire date as part of your id, or the type of customer.
1) Why do some websites use letters in their IDs? Are they strings?
We don't know if those websites store IDs in their database as strings. Numbers and strings are really the same to computers. A string is just a number, just shown with a different base. 'A' = 0x41 = 65 = 0b1000001, to the computer it's all the same. But if you display it, the larger the base, the shorter the representation, and shorter URLs are easier to read and share for humans. Sites like YouTube and Imgur use base 62 (letters, upper and lower case, plus digits) or larger (add a dash or other valid URL characters), which is relatively short for big numbers. What would you prefer to use, youtu.be/23489234892348234933 or youtu.be/B9k6KMrv8vh?
2) Why are non-sequential IDs used?
The answer by IMil explains it well:
Youtube can't use sequentional IDs for two reasons:
Its databases are almost certainly distributed, making sequential numbering complicated.
It has a privacy option "Unlisted videos": those that don't show up in the search results, but are available if you know the ID.
These also explain why the IDs are so large: (YouTube doesn't host 23,489,234,892,348,234,933 different videos, obviously)
When generating IDs, it's a problem if you accidentally generate the same ID twice, so you need a big ID space to prevent the birthday problem
People can just guess the URL of unlisted videos if the chance of any given valid ID being used for a video isn't very, very small.
People can just guess the URL of unlisted videos if the chance of any given valid ID being used for a video isn't very, very small. -- how do you know if an unlisted video isn't accessible for everyone except its author? even if someone else has guessed its ID
why not just integers, particularly sequential ones? And when, in what cases is it a wise decision to such string ID instead of integers?
As an aside, it's not necessarily the case that the internal representation is a string. They could very likely be encoding a numeric identifier as a alphanumeric string for the shorter url.
As you've pointed out that it would be easy to use a universally unique ID just using numbers because under the hood everything is just 0 and 1 and you could expand the number to more precision going up to 128 bit or more.
I think the main reason is that, assuming some arbitrary fixed range like uint32 (just for the sake of an example), if you use letters as well you can have a shorter ID in total.
I imagine that it's an esthetics reason for the URL. Instead of having 4,129,873,773 with letters it's much shorter Fu837t (just fictious made up by me). A user might even be able to remember the URL for giving it to a friend. Platforms like Youtube usually have longer UUIDs than 32 bit because they would run out of space quickly.
A short URL is desirable since it makes linking and sharing simpler (e.g you can share a link in a SMS, it is faster to type and so on). Services like Youtube or Imgurl want you to share URLs casually, so this is an important consideration.
Using alphanumerical ID's rather than numerical means you need fewer characters to express an ID of the same bit-size. For example 6 digits give you a million unique id's but 6 alphanumeric characters (using the base64 set) gives you 68 billion unique identifiers.
For all we know, the alphanumerical identifiers could be sequential numbers, just encoded in an alphanumeric format like base64. But often commercial services eschew sequential codes to prevent people from guessing ID's and to avoid disclosing business information like the amount of customers.
The word "hash" is not found in the existing, nice, answers, so here we go:
Often, data can be identified by its content hash instead of an independent, artificial ID. This is particularly evident in software like git or file systems like ZFS where this particular property of using content hashes not only makes stuff easier (for example de-duplication), but also has other nice properties like trivial caching, a secure history, detecting bit rot etc.
Hashes usually come as hex numbers (or an even larger letter space), so that's why you don't see integer IDs. There simply are no integers (in those cases).
Hashes are good if your data objects are immutable (like in ZFS or git); they would be great to store images, for example, on large CDNs. I do not know whether those particular IDs actually are hashes, but it would certainly make sense (and as Michael Kjörling commented, short IDs are probably not hashes for obvious reasons - as comparison, git uses SHA-1 values which are 20 bytes or 40 hex digits).
hashCode(), etc. Of course, the shorter the hash, the more likely random collisions are.
There's several reasons why you would use non-numeric ids, but also understand that not all values with alphabetic characters are really strings. YouTube has the reputation of an incredible number of videos, on the order of 300 hours of video uploaded every minute (ref). The unique integers representing those videos can get quite long, so the use something like Base64 URL encoded numbers (ref).
Types of Identifier Representations:
They all have their strengths and weaknesses. The more unique characters you can use for your identifiers the fewer characters you need to represent a number. Base 64 numbers are a pretty good compromise because there is an established variant that works for URLs and compresses the number of characters needed to represent a number 6 to 8 (i.e. 3/4th the size).
Readable strings work for blogs because they can raise the searchability, and it's a lot easier to generate unique titles when the number of records is small.
Ok one of the reasons is that the characters are sent as characters and not as integers anyhow. This is because of how an HTTP Get works.
When you say, "why not use an integer?" Well, the integer is then chopped up and every digit is sent as a character and you end up with a string of characters anyhow. So why not use all the options for a character?
There is also the human factor:
Take imgur for instance : https://imgur.com/*****/s6UqP
s6UqP,
The range for every character is: a through z capital, a through z sub-capital, and 0 to 9= 26+ 26+ 10 = 62 options for every position in the string. With five positions that's 916132832 possible combinations. If you would only use numbers, you would need 9 digits.
People can hold roughly 7 objects in memory, 9 digits is too much, 5 characters is doable.
