Where I work, we have a lot of silos of knowledge. Basically each member of the development team is the expert (and sometimes the only one that knows a thing) on certain pieces of the overall software. For example, one person might be the expert on the rendering engine of a game. Another is the guy that works on the GUI. Rarely do the two intermingle. The concern is that it doesn't make sense to have the GUI guy work on Networking stuff, when there is already someone that can (in terms of efficiency) get something done better because they are already familiar with it.
This situation seems to make sprint planning difficult in SCRUM. It's quite possible that people on the team have no tasks or are seriously underutilized in a specific sprint simply because there is nothing there they are the "best fit" to work on. This makes calculating velocity unreliable as it will vary based on who actually gets more assignments during one sprint versus the other.
How would I overcome this issue? It seems like the focus should be more on the tasks and what needs to be done and less on the specific people to work on them. I think upper management will always prefer things to be done as quickly as possible. And if they see that GUI developer estimates 21 story points on a task related to the Engine, where the Engine guy estimates 5 points, they will question why we chose 21 (The goal would obviously be knowledge sharing, but this will cause the team to work inefficiently overall).