For closed domain, both OFF and ON points are outside domain, while for open domain, the OFF point is in inside the domain. I just do not get why is that.
This is incorrect. For both open and closed domain, one point is inside and one point is outside the domain. The difference is which point is inside (and which outside).
To test a domain boundary, you need one point on the inside, as close to the boundary as feasible, and one point on the outside, also as close the the boundary as feasible.
The point ON the boundary is by definition the closest you can get. For a closed domain, this point is defined to be the last point inside the domain and for an open domain this point is defined to be the first point that lies outside the domain.
Testing with two points ON the domain boundary does not provide you any additional information, so the second point must lie OFF the boundary.
If you combine the fact that you need both a point inside and a point outside the domain with where the point ON the boundary is considered to be (inside or outside), it follows logically where the point OFF the boundary should be located.
As for why you need points on the two sides of the boundary, If you have two tests telling you that the values 10 and 11 are both outside the domain, that doesn't tell you if the domain is (correctly implemented as) "values less than 10" or perhaps "values less than 9".