Software engineering is engineering.
However, most people who call themselves software engineers are not engineers. Many of those that are not are programmers and coders that do not necessarily follow an engineering process. Nothing wrong with that since a) coding is fundamental to software engineering, and 2) is a science in its own right. But having an engineering process is fundamental for being and working as an engineering. That's one group.
The other group, unfortunately a quite large one, are not programmers/coders. They are just hacks, code monkeys who fling shit at their monitors and sell whatever sticks (quoted from somewhere, can't remember where.) A hive-mind collective of the dumb.
The influence of the later group is nefarious, poisonous and hilarious (in a black humor kind of way) which is why engineers in other disciplines (wrongly) sneer at the notion of software engineering, or think software engineering and computer science are anything but coding (and by coding, spaghetti coding.)
In summary: There is a negative perception of software engineering among many in other, older established disciplines. That is a wrong assumption.
But that wrong assumption comes from the real observation that many people that call themselves software engineers are nothing short of sad caricatures of lobotomized monkeys banging at keyboards in the hope that, by some random chance, they end up typing Knuth's TAOCP.
ps... My guess is that the Mathematician in your story is just butt-hurt. He had that crap deep in his chest for God knows how long, and you were simply the first outlet to let that go out in the open.