-2

Our current production landscape is as following:

Several application servers (AS) instances and a single external content server (CS). Application processes (Background/Foreground) read and write files from and to the content server.

Writing is expensive in terms of performance: I.E. writing a 100kb file takes 1~ second for calculations inside AS and 1~ sec for uploading it to CS and receive a response.

I thought about a new landscape and I need a review:

Establishing a new local CS within application layer (DB/File system) and use it as a buffer. Instead of writing files immediately to the external CS, write them first and temporary to the local one. Then periodically (and load-balanced) flush files from the local CS to the external.

If a process request a file for reading, first check it's existence in the local CS, if exist - retrieve the file from there, else, retrieve it from the external CS.

Cons I thought about:

  • It's not a straight forward solution, therefore it's again KISS.
  • A process that uploads a file receives a successful response, whilst the whole uploading process is not really done, and might fail later on, and there won't be a way to inform the process about the real status of the file.

Replacing a central CS with a local one, causes another problem: the availability of a file across multiple AS instance should be preserved wisely, so we might: (1) Let each AS to read files from all other ASs by using a DB table with two fields: [FILE_ID,AS_INSTANCE] (DB access is not considered expensive in our landscape). (2) Synchronize all files between all AS instances until flush.

If more details are needed, I'll glad to supply.

Landscapes

2
  • As you have multiple AS instances, is there any chance 2 instances will try to write/update the same file? Also, how big of a problem is it if a user causes a file to be uploaded from, AS1 and then does a read for the same file that gets handled by AS2, before the file got pushed to the central CS? Jan 5 at 11:03
  • @BartvanIngenSchenau 1. Accessing the same file for writing is handled with a lock mechanism in the application layer. 2. It's indeed a problem, I added it to the question body.
    – Dorad
    Jan 6 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.