2

I am currently working in a project that is to build a online judging system like codeforces. That is where someone can upload a file for a programming solution and our system will compile the code and generate the output and finally send the result that the solution is correct or not.

The front end is going to build in PHP and the compile engine will be in Java Rest api. This will take a http request, where the request containing which file to compile. The file is like A.cpp that is going to be compiled by the java api and response as a result(right/wrong).

Now my question is about the file that is send from PHP to Java api. How can the PHP send a file to Java api ? Will there be a file server needed ? Or do I need a file server at all ? If I need a file server what is the benefit I will get ? Actually I am not getting the point what is purpose of a file server.

Thank you.

7
  • You store the file and call the API with the path as parameter would be one simple solution. Or you store the file in a folder that gets scanned by the API at some time interval. Or a million other ways. Also: Do you intend to run that code on your server? Normally such systems would run it inside a VM or container for security reasons I think. (Also: A file server is a server dedicated to handle files. You would need and use one if you have a lot of files and file handling becomes a bottle neck. Unless you have a very high load on your system you won't have to worry about this)
    – thorsten müller
    Commented Jul 22, 2016 at 19:17
  • @thorstenmüller Thanks for reply. What do you actually mean by running code within a VM or container. and what type of security measure it will ensure.
    – Muztaba Hasanat
    Commented Jul 22, 2016 at 20:40
  • You would have a virtual machine, setup with everything necessary to compile and run the code. You clon that vm and run the code inside it. Now if the user has some malicious code that maybe tries to format your harddrive, delete files, blocks system resources or whatever else it will be limited to this vm which you delete after you got the output anyway.
    – thorsten müller
    Commented Jul 23, 2016 at 6:36
  • @thorstenmüller thank you for the explanation . :)
    – Muztaba Hasanat
    Commented Jul 23, 2016 at 6:52
  • It's called sandboxing. One example: github.com/edx/codejail (there are other variations on this, using LXC containers or similar instead of AppArmor)
    – thorsten müller
    Commented Jul 23, 2016 at 7:55

1 Answer 1

Reset to default
1

How can the PHP send a file to Java api?

This can be done in several ways: HTTP post, a data stream etc.

Will there be a file server needed ? Or do I need a file server at all?

It depends.

If I need a file server what is the benefit I will get ?

The benefits of a file server is sharing storage resources. It's the same as having a network printer instead of everyone having a cheap desktop printer. With one file server, the PHP program could upload the source file to the file server and them send only the name of the file in a message to the Java API. The result or the process could be stored in the file server and you can download it with another API call. This actually makes the process to be asynchronous. And asynchronous processes are easier to debug and schedule for bulk processing.

Another benefit of it is that in can be managed by a team that takes care of backups, retention times, redundancy, disaster recovery, etc.

EDIT. Why asynchronous processes are easier to debug?

  • The two layers don't have to be up and running in order to test one of them. For example for testing the PHP code that uploads the file to the file server, the Java API layer needs not to be up. To test the Java API you don't even have to mock the PHP call. You just put a file in the server and test the Java method that reads the file and compiles it.
  • Basically it's easier to debug and test a bulk process that takes a bulk of files and processes it.
  • I'll give you an example: for a phone call (a synchronous communication) to go through, there must be a person in the receiving end to pick up the call. On the other hand when you send an email (an asynchronous communication), the recipient needs not be connected to the internet, but as soon as he/she gets online he will receive a notificationo an email message in his/her inbox. You can test the sending part and the receiving part separatelly with no mocking.
  • Also, less error handling code must be written, meaning less code to test and debug.
8
  • +1 for mentioning asynchronous processes. So what would be better approach ? PHP will make a HTTP request with a file name to Java api. And Java api will find the file from file-server, compile, generate result and send back response as Json format to PHP. Or Java api upload the result to the file server. latter PHP will find that particular result. Which one will be better ? I think in first approach PHP need to wait for the Java api response.
    – Muztaba Hasanat
    Commented Jul 22, 2016 at 19:54
  • 1
    How is an asynchronous process easier to debug again? Care to elaborate?
    – Andy
    Commented Jul 22, 2016 at 19:56
  • @DavidPacker I just elaborated.
    – Tulains Córdova
    Commented Jul 22, 2016 at 22:40
  • @seal The second approach is asynchronous. Files can even be kept unprocessed for a time and the a cron job could proccess them. Also, where's the +1?
    – Tulains Córdova
    Commented Jul 22, 2016 at 22:42
  • 1
    I disagree that asynchronous processes are categorically easier to debug. Also in the case of the file server, your "asynchronous" version involves: - writing a file to the file server (needs error handling - did it write?) - sending API request to compile server (needs error handing - validate request) - reading the file from the file server (needs more error handling - does the file exist? Is the file server healthy?) Whereas just sending the file in the API request would just require the middle step - less error handling code, fewer servers required, fewer points of failure.
    – Sam Dufel
    Commented Jul 22, 2016 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.