1

I am working with a trading application (reading data from the exchange) which generates a bucket load of data on a per second basis. We have different "log-levels" but even the minimal log-level generates so much data ! This process of log creation is quite I/O intensive.

Since I can write assembly using __asm__ , I was wondering if the operation of writing to the files is coded in assembly, will there be a speedup ? The rest of the code is C++.

Also, what are the caveats of doing such a thing ?

Improve this question

1 Answer 1

Reset to default
15

The performance overhead of writing lots of data to disk isn't the execution speed of your code, but rather the physical limitations of the actual hard drive. Doing it assembly won't give you a noticeable performance increase.

Your best bet is to either log less data (recommended, if you're logging that much stuff how useful can it be) or change the drives so they can write quicker! Consider higher RPM's, SSD's, RAID (zero I think for parallel writing) or at least a dedicated disk with a dedicated controller. But really, unless you really need all that logging data, you're wasting your time. Fix your log levels so that WARN and ERROR only log stuff when those things actually happen. (and if they are happening all the time... fix your code).

Improve this answer
3
  • You are thinking striped writing for raid 0. Raid 0 and raid 1 both write in parallel - just 1 writes the full amount on each drive while 0 writes 1/nth. Other raid levels offer varying write performance.
    – user40980
    Feb 12, 2013 at 6:36
  • 1
    Perhaps write to something else than a disk? Like a in-memory ring-buffer?
    – Bjarke Freund-Hansen
    Feb 12, 2013 at 10:13
  • An option might be to use UDP to push the log messages to a server allowing you to write the messages out asynchronously. You can also use something relatively lightweight, like ZeroMQ, if you don't want to work with straight UDP.
    – ipaul
    Mar 9, 2013 at 4:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.