Here is one rule of thumb (but don't follow it blindly! Understand why you must use it!) for the direction that your dependencies must follow
Always depend in the direction of abstraction and stability
What that mean is that the dependencies that everyone depend on must be dependencies that change the less. Everyone depend on them, thus they're costly to change. So design those dependencies to have little to no reason to change!
And one way to do that is to make those dependencies mostly of unimplemented interface and stupid data structure that serve only to transfer information between objects. Then, more opinionated module, like the UI and the Data Access layer, and various part of the business logic, implements those interface with big fat objects that actually do the work. This is called the Dependency inversion principle.
Now, the direction that the arrow of the layers must take depend on who your talking to: "UI must depend on business logic which depend on the data access! No! The business logic must be in the center and everyone depends on it!" Lots of opinions and religion, but its all futile in the end. What matter is the abstraction. If your dependencies is through abstract elements, then you have it. The rest is pure pointless intellectual debate.
So, to answer your question, I'm gonna ask you another one "What are the reason for your so-called entity to change?" You have the answer in your own question :
As POCO entities used in a dbContext are in fact a definition of the database
If your entity are a definition of the database, that mean they will change everytime the database change. And if they must conform to a dbContext, that mean they have a strong coupling to the Entity Framework. If Microsoft decide to change the way EF work, your entities will have to change. Your entities will not be the nice little abstractions that you need to minimize coupling.
So yes, in that case, if you want to consider them a model of the database, then you should hide them away in a persistence layer, and make sure that nobody ever see them that don't also know about the database.
In summary, if you want an application that is well decoupled and strongly cohesive, no, they should not be depended upon by everyone, because if you do, everyone will depend transitively on the database structure. So everytime the database change, the whole application will have to change and it will be a pain. The way people usually cope with that is to make the database not change, and that introduce a lot of rigidity in your data storage. At some point or the other the database will need to change, and you will have to pay a hefty price of technical debt at that point.
The above was as neutral as I can make it, the rest will be more opinionated. Feel free to ignore it if it don't fit your views.
Your entities should be the base of your application and everyone should depend on them. They should not, never, be an implementation of the database. The entity framework job is to bend around your entity, and if he can't do that, then it's not the framework for you: ditch it as soon as you can. He shouldn't have any power over your entity. YOU decide how your entities are structured. Don't buy into the promise of magic that the conceptor demos are showing you. At one point or the other you will need to do something unexpected, all the magic will disappear, and you will have to leave your nice dreamy world and write the code you really needed from the start.
Try to make your entities an abstraction of your domain model (which is much less likely to change), and at that point, everyone will be able to depend on them, and your database will be free to be implemented however you want. You will have less magic at first and you will code more, but in the long shot, you will win on all account.