The original intent was to remove the extension of the header "file" name (of the standard headers) in the C++ source file, for portability, and let the implementation supply whatever extension it wished to use for a default (.H, .hpp, .h++, .hxx, .h, or maybe something else). Remember, the standard headers need not be files, they could be objects in a database or a text library file. The compiler could see "#include <iostream>" and then open up some file which might be /usr/lib/include/c++/iostream.h or SYS$LIBRARY:IOSTREAM.HXX or whatever, or retrieve a "pre-compiled header" from a database. There was no intent for the standard to specify an actual file, let alone one with a name that didn't have an "extension". (Yes, it means the compiler needs to know something about the standard library implementation, but switches could modify that behavior like for the "include path".)
And for a non-standard header, as mentioned, the user could name the source file foo.cpp or foo.c++ or whatever the operating system allows, and name the file of declarations foo.hpp or foo.h++ or whatever. And of course foo.hpp could be referred to in the #include as just foo with the compiler applying its defaults to get the actual file name, just as you could tell the compiler to compile source file foo and the compiler (being a C++ compiler or being told it was to compile C++) would apply its defaults and open up foo.cpp or foo.c++ or whatever. (Or maybe the source code is kept in an IDE's database for the user's project and not a separate file.)
Personally, I like having extensions on files so that the editor knows what language the file is in. But that's separate from how the file is referred to in a #include.