I am working on a machine learning project at the moment which requires me to transfer the data from an old Java app(which is also the custodian of the data in current paradigm) to a python service which will do all the machine learning related stuff. So, I have data to the tune of a few GBs that needs to flow through the network.
What would be the most efficient way of transferring that data?
This information may be useful-
- The JAVA application is deployed as a 3-tier AWS instance and uses elastic search, postgres and neo4j.
- The python application will be deployed on a separate AWS instance.
- The data exists in Neo4J, is currently not encoded, but can be written to CSVs, or transformed into objects.
Help is appreciated! Thanks in advance!