To complement the existing excellent answers from Doc Brown and from Gort the Robot, another way of looking at access restrictions is as one of many ways we write modular programs.
In the simplest architecture, all variables and lines of code are accessible from all scopes. This would allow you to assemble code in any combination you wanted, and as the program grows, the number of combinations increases exponentially, making it hard to reason about what the program will do.
To improve this, we break down programs into reusable modules of various types. Each module has a contract defining the ways the program can interact with it; and an implementation which is hidden from other code.
The simplest type of module is a function: function signature (name, parameters, and return type) defines the contract; and the implementation can include local variables, and lines of code that cannot be jumped into from outside the function. The number of possible combinations then reduces from all possible interactions of variables and lines of code, to all possible interactions of functions.
Visibility modifiers simply iterate the same advantage: by grouping functions and variables into classes or packages, we can give each grouping a contract and an implementation. The number of possible combinations reduces further, to valid interactions of classes, or packages.
All of this can be implemented by convention rather than as part of the language - you could name global variables carefully rather than having local scope, and you could name functions carefully rather than having visibility. The more universal those conventions are, the more tools can support them - e.g. offline checkers can tell you if you're validating contracts, and IDEs can suggest only "visible" members.
In order to be truly universal, the convention needs to be baked into the language, and enforced by default tooling. This matters most when sharing or handing over code: if most tools read a leading underscore as meaning "private to this class", but the default compiler/runtime doesn't enforce that, you have to make a judgement whether someone else's code uses it "correctly". If you see
private in a Java program, you can make pretty strong assumptions about the contract.
Ultimately, it's about trading flexibility for the ability to reason about and maintain the code. Just as there are cases when using
goto rather than functions helps solve a problem, there are cases where accessing a private method gets a job done. If the language has not made that trade-off by what it enforces, the programmer has to make the same trade-off by how they use it.