3

I am comparing jpeg to jpeg in a constant 'video-stream'. I am using EMGU/OpenCV to compare each pixels at the byte level. There are 3 channels to each image (RGB). I have heard that it is common practice to store only the pixels that have changed between frames as a way of conserving memory space. But, if for instance, I say EVERY pixel has changed (Please note I am using an exaggerated example to make my point and I would normally discard such large changes.) then the resultant bytes saved is 3 times larger than the original jpeg.

How can I store such motion changes efficiently?

Improve this question
8
  • 1
    sounds like you are attempting to create your own video codec. What you described is basic approach of MPEG4, H.264, VC1... just about every modern codec out there. And a lot of time/research went into each of those. You sure you can't just use an existing video encoder for what you need?
    – DXM
    Oct 20, 2013 at 18:23
  • @DXM Hi, Thanks for taking an interest. What I have to work with are a constant flow of jpeg files. I have already optimized and saved each jpeg to 19kb. I want to upload the motion frames to a server. I need to maximise as much space as I can. It occurred to me that I only need to store the changes somewhere and 'reconstruct' the motion frame when the User wants to view it. So, I cannot use video encoders for this. At least I would not know how to. Thank u so much for replying though
    – Andrew Simpson
    Oct 21, 2013 at 8:48
  • "constant flow of jpeg files" really, really sounds like a video codec. You won't be using it in a traditional sense, but nothing is stopping you from creating an long-lived instance of a video encoder and feeding it one frame at a time as you receive those jpegs. Instead of using real time (i.e. wall clock), make up your own timestamps so that each timestamp on the video could be mapped to a unique image id. Then retrieval should be simple, just seek to last key frame, play partial frames and return the one with the matching timestamp. It would be much easier to use an existing codec...
    – DXM
    Oct 21, 2013 at 14:20
  • ... and much more efficient, then trying to reduplicate the work (i.e. coming up with your own video compression, which is where you are moving towards) that a community of PhD's has been doing for quite some time now. en.wikipedia.org/wiki/FFmpeg
    – DXM
    Oct 21, 2013 at 14:23
  • hmm OK. I understand your point. I shall look at ffmpeg to see what that can offer me. thanks
    – Andrew Simpson
    Oct 21, 2013 at 17:58

2 Answers 2

Reset to default
2

Have a bit that indicates whether you are storing changes from the last frame or a new frame and store whatever option is less memory consuming.

You can store either a jpeg (or any other format really, if possible for your purpose since jpeg is far for optimal for that) of the image that represents the changes or a simple jpeg of the next frame, and indicate using an added bit which format you used.

Obviously, you may (and according to your comments seem to) have constrains which block you from using this approach. In any case, this is more or less the approach used in the MPEG video format(s).

Look up I frames ("Intra coded frames/slices") and P frames ("Predicted frames/slices") in this Wikipedia article - Video compression picture types

Improve this answer
6
  • 1
    I cannot see how that would help me? Please elaborate? gr8 link BTW.
    – Andrew Simpson
    Oct 20, 2013 at 8:24
  • I can see that using p and B frames would help but how would that code in c#?
    – Andrew Simpson
    Oct 20, 2013 at 8:25
  • Hi, u say store a bit of the changes. I need to encode the x,y position and the rgb or hsl or.. differences. The jpeg is already compressed. By storing the 3 values (and if the changes are significant) then i run the risk of the bytes changes being bigger than just storing the jpeg. So..?
    – Andrew Simpson
    Oct 20, 2013 at 8:44
  • Just found out that jpegs do not store inter-compression between frames.
    – Andrew Simpson
    Oct 20, 2013 at 9:02
  • What I meant is that you can store either a jpeg (or any other format really, if possible for your purpose since jpeg is far for optimal for that) of the image that represents the changes or a simple jpeg of the next frame, and indicate using an added bit which format you used. Obviously, you may (and according to your comments seem to) have constrains which block you from using this approach. In any case, this is more or less the approach used in the MPEG video format(s).
    – Ofir
    Oct 21, 2013 at 6:59
1

From what I remember, the JPEG and MPEG/H264 family of motion codecs internally represent the images in YCbCr colorspace (not RGB), where Y is the luminance, and Cb with Cr are the chrominance components. While every pixel of Y component is used, the chrominance is downsampled according to one of a few possible schemes. The main compression gain in JPEG (and keyframes of motion codecs) is achieved by quantization of DCT results, which means throwing away the high frequency components of each image block. To compress the differences between frames caused by motion, the motion prediction is used in the luminance component (before DCT). Very briefly, each frame is divided into regular blocks of pixels, and each block of current frame is compared with pixels of the previous frame to find the closest match. This produces the deplacement vector (motion vector). Because the match is not ideal, the pixel value differences in each of the Y, Cb, and Cr components are passed through the normal JPEG compression (DCT, quantization, Huffman encoding) and sent along the motion vector to form a Delta Frame.

Here, you will find a few hints for your further research. I hope you'll not have to reinvent the wheel. There are some Linux/Unix image processing tools you might harness for your task.

Improve this answer
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.