Files generally indicate their encoding with a file header. There are many examples here. However, even reading the header you can never be sure what encoding a file is really using.
For example, a file with the first three bytes 0xEF,0xBB,0xBF
is probably a UTF-8 encoded file. However, it might be an ISO-8859-1 file which happens to start with the characters 
. Or it might be a different file type entirely.
Notepad++ does its best to guess what encoding a file is using, and most of the time it gets it right. Sometimes it does get it wrong though - that's why that 'Encoding' menu is there, so you can override its best guess.
For the two encodings you mention:
- The "UCS-2 Little Endian" files are UTF-16 files (based on what I understand from the info here) so probably start with
0xFF,0xFE
as the first 2 bytes. From what I can tell, Notepad++ describes them as "UCS-2" since it doesn't support certain facets of UTF-16.
- The "UTF-8 without BOM" files don't have any header bytes. That's what the "without BOM" bit means.
enca
andchardet
for POSIX systems. – Janus Troelsen Nov 3 '14 at 21:00iconv
in particular is useful for this purpose. Essentially you iterate the corrupted characters strings/text through different encodings to see which one works. You win, when the characters are no longer corrupted. I'd love to answer here, with a programmatic example. But it's unfortunately a protected question. – Brandon Bertelsen Dec 7 '15 at 19:03chardet
orchardetect
is not available on your system, then you can install the package via your package manager (e.g.apt search chardet
— on ubuntu/debian the package is usually calledpython-chardet
orpython3-chardet
) or via pip withpip install chardet
(orpip install cchardet
for the faster c-optimized version). – ccpizza Mar 27 '19 at 17:12