The basic rule for what goes into a source control repository is that you store there everything related to a project that you need to be able to build, test, deploy and execute the project and which can not be generated from items already present in the repository.
In other words, if you can throw away the PACKAGE folder and its contents without affecting your ability continue working on the project (the build might take longer, but you don't have to hunt down and install anything yourself), then the folder can be safely left out of the repository.
If the folder contains stable 3rd-party packages that might take a long time to download or that might become unavailable, then that might be a reason to add them to your repository anyway.