The line manager simultaneously embraces Scrum and has a hard time "letting go"; he has the Product Backlog in his head and also prioritizes it.
I'm assuming that your line manager is filling the role of Scrum Master. In this example, he's not embracing traditional Scrum roles.
One of the key aspects of any agile methodology is high visibility. That means that the work products are visible to the customer on a regular basis (as seen through frequent releases of potentially-shippable products), but also that internal status and metrics are visible to the team. The product backlog and it's current state should not be in anyone's head, but visible to the entire team (and some would even argue to the customer as well, if they desire it).
Also, the Product Owner is supposed to be the owner of the Product Backlog. It's the job of the Product Owner to write user stories, prioritize those stories in the Product Backlog, and ensure that stories are properly completed (verified and validated).
The Product Owner has been in the company for about 2 years and seems to have a clear vision of the product, but he is underpowered by our line manager (who in fact is held responsible for this and other projects).
Your Product Owner needs to have the ability to speak for the customer, otherwise, this role is pointless. Having a clear vision is important, but if he can't act on this vision and create and prioritize stories in order to allow software to achieve maximal value, then he can't fulfill the responsibilities of the Product Owner.
What do you think are disadvantages of this structure we find ourselves into? What could be improved? How would you improve?
The disdvantages are that you aren't utilizing your people. It's pointless to have a visionary or champion if they can't do anything to act on their visions or goals. It all comes back to the bottom line - what are you paying these people to do and are they able to do it? It sounds like they aren't able to fulfill their responsibilities, so something needs to be fixed.
The only way to improve is to reevaluate roles and make sure everyone knows their responsibilities to the project and product. You should also focus on product and process quality rather than following a process to-the-letter. Perhaps by-the-book Scrum isn't what you need, so tailor the process to fit how your team and organization works.
Other than this, our implementation of Scrum is “by the book”.
Following up on my last thought, any "by-the-book" process implementations worry me. Scrum is a framework for project management. There are concrete implementation described, and success stories, but those are for particular teams within particular organizations working on particular projects. Scrum is a perfectly fine framework, as long as you tailor it to meet the needs of your team within your organization working on your project.