There's actually a large class of machines called "computers" with different operational principles, but in this context about "programming languages" you're probably asking specifically about general-purpose, stored-program, digital computers.
The development of these first leapt during WW2, where there was a newly pressing need to execute a variety of complicated computations particularly relating to cryptanalysis and physical simulations.
The labour requirements of executing computations by hand (as had been the norm) soared as they increased in complexity, hence the desire for a mechanised solution that would be faster than an army of clerks, and more economical with labour.
The earliest digital computers did not store their program in memory, but instead were configured by physical alteration of their hardware - by physically altering wiring or adjusting electro-mechanical elements.
However this physical adjustment was itself found to be slow, labour-intensive, and error-prone, and the process of doing so took the machinery out of action in the meantime.
Very quickly it was found more effective to control the computer by implementing a set of fixed computational operations in hardware, and then controlling the application of these fixed operations through the use of "machine code".
This machine code was designed to be stored in the same fashion as the data which the computer operated with.
Arguably that is the first "programming language", if what we mean is some kind of symbolic representation that specifically controls this kind of computing machine.
All current computers still operate using machine code.
The idea of controlling machines with codes was not completely new at the timr though - the Strowger switch for example, which controlled telecoms routing by entering Morse-code like pulses, was invented a generation earlier in the late 19th century. The Jacquard loom in the textile industry, was also controlled by punch-cards, again invented in the 19th century.
And obviously, by WW2, we were already using mathematical syntaxes and natural language instructions to control computation executed by humans, and machine programming languages still rely latently on these human languages for their own design and meaning.
That is, computers must be understood as a mechanisation of computation, not as originating all the concepts of computation.
It's therefore impossible to be rigorous about exactly when "programming languages" first emerged and what each is "written in", unless we are arbitrary about exactly which machines we're talking about, because everything eventually reduces to the underlying natural languages, and many other kinds of machines have had codes and "languages" which control their operation.
Anyway, back to WW2-era computers. Once hardware was to be treated as fixed, analysis ensued on what kind of operations had to be furnished in hardware to provide a full capability for performing all kinds of computation. This is where you get ideas like the "Turing machine" and similar.
Expressing instructions in raw machine codes are not very ergonomic for human use, so assembly languages were devised which allows a programmer to specify the instructions in a mnemonic form, which are translated by a "compiler" program into raw machine code. This translation is simple enough that an assembly-language compiler can be written by hand (by a skilled programmer) in machine code.
The process from there is basically recursive, with a compiler for a new language each time being written first in some more primitive language. By the late 50s, there was already COBOL.
Later the concept of "cross-compilers" and "trans-pilers" also emerged, where you can write a compiler for completely new hardware (i.e. hardware whose machine code is different from any previous scheme), using relatively advanced languages already available for execution on existing hardware.
So we don't really need to start at the bottom anymore, implementing a programming language compiler first in machine code, so long as we continue to have more advanced facilities that have already been built from the bottom up.
Most commonly now, a compiler would be written in a high-level language (like C), and the machine code for the target architecture would only make an appearance as the output of the compiler, not as the origin language in which the compiler itself was written.