your program is just a finite machine that can execute a finite set of instructions
Stricto sensu, this is true, but not very interesting. Indeed, my desktop has only three terabytes of memory (including RAM, disk, registers, etc...). So it has "only" 2(3*72057594037927936*8) states, but viewing my desktop as such a huge finite machine is not very interesting. Observe that I can buy some more memory (e.g. add more disk).
Many physicists have a quantum-mechanical view of the entire universe, and might explain you that the universe itself is a huge finite state machine (after all, it has less that 10100 particles, each of them having a quantum state related to Planck's constant, etc...)
In practice, better look at your laptop as a Turing machine and consider that its memory is potentially "infinite".
(your question is messing different levels of abstractions)
Why do programs need such an extension that is an interpreter all by itself?
Embedding an interpreter is related to partial evaluation and Turing completeness. In practice it is very convenient (but might facilitate malware, given as "data" interpreted by your extensible application). You could embed Lua or Guile in your application (but that is a major design choice, that you need to make very early because of its architectural implications, notably related to garbage collection). You could design your application to accept plugins (see dlopen(3) and dlsym(3); and my manydl.c program shows that it could have many hundred thousands of generated plugins). It might even use JIT compilation techniques (see libgccjit, LLVM, asmjit, ...)
Read SICP, then also Scott's Programming Language Pragmatics, Queinnec's Lisp In Small Pieces, and R & A Arpaci-Dusseau's Operating Systems: Three Easy Pieces and Pitrat's blog.
Play with SBCL (a good Common Lisp implementation). It is generating (dynamically) machine code at every REPL interaction. Be aware of homoiconicity and metaprogramming and multi-stage programming. Read about Greenspun's tenth rule and Gall's law and about accidentally turing-complete things.
Since Turing (and his halting problem, see also Rice's theorem) and Gödel (and his incompleteness theorem) we are aware that code is data (and is proof) and data is code (self-reference is a related concept, see also Richard's paradox and read about Curry-Howard correspondance). Read Hofstadter's Gödel, Escher, Bach book.
BTW, your picture of the virtual address space of some Linux process is really naive (it was sometimes true in the previous century; today things are much more complex). See proc(5), elf(5), execve(2), mmap(2), ld-linux(8) and try
cat /proc/$(pidof emacs)/maps on your Linux system.
See also this answer to a related question (and perhaps this one).