It depends on the area of mathematics he is interested in.
If he is more into Abstract Algebra or Combinatorics, he would probably like Haskell as the complex type system allows for direct programming of a variety of mathematics. ATS might also be an option.
If he is more into Matrix Analysis, Differential Equations or Dynamical Systems, I'd recommend Matlab, NumPy, or Octave. APL is via Dyalog might also be of interest; though I'd not start him on that. Numerical Recipes, while in C++ can be adapted to serve as a good guide to algorithms.
If he wants to do Statistics then R is the natural choice.