Hi AEK,
our memory usage actually varies per codec, platform, resolume version and gpu architecture. I'll be explaining how it works when playing back dxv2 on windows in resolume 7 width a dedicated graphics card.
As you've already noticed, Resolume does
not load the entire file into memory. Our users tend to create gigantic decks, up to the point where having everything in ram just doesn't fit. We could in theory load as much as possible, however this has several problems:
- The first problem is that Resolume is mostly used in environments where multiple applications are used simultaneously to run the show. If we were to hog the entire memory that wouldn't leave anything for other applications. We cannot detect which applications you intend to run, so we would never know how much we should leave free.
- Another issue would be unpredictability of playback. If we were to load the first half of your deck into memory, you would get an unexplainable performance hit when playing something from the second half.
- The last issue is one where memory throughput actually becomes a bottleneck for us is some scenarios. This is related to all the memory based operations we need to do in order to get the frame to be visible on screen. It's a complex scenario to describe, but sometimes having a fast ssd in combination with ram can net us faster results than having just ram.
So rather than loading the entire file into memory we've tuned our video engine for streaming video data one frame at a time. In order to stream a single frame we need several buffers of ram.
- First we need a buffer to read the frame's cpu compressed data into. Depending on having alpha available this buffer's size is somewhere around (width*height*4 / 8 / ~1.4) for rgb and (width*height*4 / 4 / ~1.4) for rgba.
- Once we've read in the frame from file we need to apply the cpu decompression step. This buffer will remove the ~1.4 compression factor, so it'll be that much bigger
- Then it's time to upload the frame because the /4 and /8 compression factors are for gpu based compression. In order to upload texture data we need another buffer of the previous size but from the pinned memory pool. This is special driver memory that cannot be paged out to disk by the os.
At this point the video frame is present on the gpu and we'll be recycling the buffers for next frame.
Now it's time to tell you i lied
. During a trigger of a single video file we're actually doing this for two frames simultaneously. Once for the frame that has to show up immediately and once for the next frame. After the initial trigger only the 'next frame' buffers are required and the 'current frame' buffers are free to be recycled for other clips, if not for the next system...
As final step we've introduced an overarching system in v7. v7 inspects the current contents in the deck and tries to preallocate the maximum amount/size of buffers that could ever be used in that deck. For example it looks inside of each layer and determines the two largest clips that could be playing inside that layer (depending on trigger target and transition usage). It'll also look at how much memory will be used by outputs as well as do the same thing for video memory. Al of these requirements are added up and will be made ready over time if memory allows (up to 80%)
All of this is just for the video frames themselves. But as you can see it's already highly dynamic and hard to predict how much memory will be used. For example the 1.4 compression factor may be different if the frame is easier/harder to compress. The /4 and /8 factors may change depending on codec, eg dxv3 uses a different setup of buffers. The intermediate buffer is not required on macos so will be skipped altogether. The memory preallocation system is not present in v6 so then buffers are allocated when they're needed but not present (eg triggers)
On top of this you probably already know the OS also has a way of using 'unused' memory. If you want more details you could use a tool like RamMap to see how much of your memory is actually not used during a show.
Sorry i wasn't able to give you an exact answer, but i hope this gives you a general idea as to what's going on.