The role of the public folder is to have those views that would be publicly accessible in the application. For example, a Login page would generally be a page that anonymous users could access to then log into the site.
Some MVC web applications may not have any public pages as they may use some internal security system that checks before accessing any page such as a SharePoint site that may use Windows credentials to identify users as an example so it can exist but doesn't have to exist.
Within Computer Science this would fall under Information Security and looking at Authorization policies. Each application may use different methods for enforcing security so this could go in a bunch of different directions potentially.
GUI being the Graphical User Interface fits as part of the architecture here. Each view is a User Interface and thus a GUI may well interact with the View portion where the public and private folder concept would apply within security behind the scenes though some sites may use locks and other symbols to denote what pages require authentication and authorization to visit.
Controllers can be lumped into a folder though some may want them spread and depending on the technologies used there may be different ways to implement this. Views would likely be in folders that again depending on the technologies could have security handled in various ways.
POST requests to the server would likely go through a controller to a view in the MVC architecture since there would likely be data to process that is the role of the controller. The view is responsible for returning to the user the data of the page requested.
In response to the comment:
Controller security could be done on a file level or on a function level where the latter may make more sense in some cases as the same controller could have different functions. For example, consider the Yahoo! home page where if you aren't signed in, then the most popular stuff may be shown while if you are signed in then the personalized page would be shown that could be going through the same base controller potentially. Thus, file level permissions would be pointless here since all the controller code would likely be run under the same server account.
While there can exist some web applications that have no public views, this depends a great deal on how one wants to implement security which can be done in various ways using various technologies, not unlike how two people communicate can be done in various formats, languages and grammar. Keep in mind that by keeping this so generic, there are likely dozens of different answers depending on what implementation you choose to use, e.g. you could take ASP.Net Webforms and implement your own MVC or you could use ASP.Net MVC or you could look into PHP open source code bases that may implement the pattern, etc.