I don't mean to come across as glib, but I feel that the clearest way I can communicate what I want to know is through some analogies:
- CPython, gcc, javac, ... -> compilers
- MS Windows, OS X, Ubuntu, ... -> operating systems
- PCRE, re2, ... -> regular languages and FSM
- Android Layouts, iOS xib files, HTML/CSS, -> ????
The things on the right are good theoretical basis for helping you get started on implementing the stuff on the left. Note, it's not so helpful on how to use it.
Question: I'd like to know what should go in that ???? above.
EDIT: Sorry about the confusion, hopefully I can clarify a bit.
I am not talking about UX.
Whereas UI/UX is about how human beings interact with the final apps that are created, in this question, I am interested more in the mechanics of how GUI frameworks are designed, not so much the resulting app itself.
You could claim compiler design is also a UX problem because you have to think about how real human beings will interact with the programming language. And that is true. But there's also a "mathematical" part of compiler design that isn't so preoccupied with end-users, but more with how to implement the design.
I don't care so much about what layouts look good to people per se, but about how layout engines/frameworks are designed.
To say that this question is about UX is like saying that compilers is about how to use programming languages.