In general, it depends.
Specifically, I would say it depends on whether the dependended-on class A is encapsulated or injected.
In the first case, you are testing 'does B work?' In the second, it is more 'can B work, given a suitable A?'.
Getting this distinction wrong, in either direction, is a fairly common cause of bad tests, whether it is overly fragile mocks, test duplication/coupling, or missed integration issues.
Some would perhaps say 'never encapsulate', but if you took them seriously, you would end up mocking the likes of RegExp and BigInteger, which can't be sensible.
Of course, this is something of a chicken-and-egg situation, as how you are going to test B always has at least some influence on how it is designed.