Over the last week I've been working intensely with patches that use Wire's Gradient, and here is a summary of my conclusions and workflows. Some of the following involves technologies outside the Resolume world. I omit screenshots here because I'm viewing on a 5K iMac with 10-bit display and my conclusions are based on that, not on what you might see on this forum posting via a web browser.
- There is no way of using Gradient with the patch set to only 8bpc that does not end up with banding that is too prominent and distracting for my purposes (not VJing). [EDIT: Using the (by default "hidden") dither inlet/variable on Wire's Gradient helps, see links to side-by-side tests below]
- Use Gradient at 16bpc int, either through the per-patch setting, or the per-node specific setting.
- Using 16bpc float or higher bpc in Gradient did not seem to help (in fact in some patches, using 16bpc float in the patch seemed to cause some strange side-effects in Video Mixers).
- Use 16bpc int across a patch is a massive CPU/GPU hit. So I prepare Wire patches using lower bpc, and then when I'm ready, I switch to 16bpc int, start Export, then do something else for a while.
- Exporting using Motion JPEG gives significantly more banding than DXV3 or ProRes422. (I was using Motion JPEG for various reasons, such as can be directly viewed in QuickTime, but you can always load DXV in Alley to just view it.) I now completely avoid Motion JPEG.
- Exporting at DXV3 Normal (8 bpc) gives slightly more banding than ProRes422 (10 bpc).
- Exporting at DXV3 High gives less banding than DXV3 Normal but just slightly more than than ProRes422.
So ProRes422 is my current "goto" for exports of content with gradients, except where I am further processing the Wire export in Resolume (Avenue), where I can sometimes get away with feeding DXV3 High into Resolume, and then clip-exporting from Resolume at ProRes422. But even then, where Resolume can handle the ProRes422 (note I'm not VJing live), it's best to feed it ProRes422 with minimal banding.
Did you know? You can upload DXV directly to Vimeo!
Some remarks on downstream processing (involves other technologies):
I use Handbrake with software x265 or VideoToolBox hardware H265 (massively faster but slightly lower quality) for MP4 creation. No matter what settings, no matter what bit rate or quality factor, I got significant, distracting gradient banding, even with sources that had barely any. This is with 8 bpc, I could not see how to change that in Handbrake, even if H265 supports 10 bpc.
Why would I want H265 MP4s of my of my shiny Resolume/Wire work? Because, although it's not a good intermediate CODEC, the otherwise wonderful ScreenFlow video editing software handles MP4 input a bit better than it handles ProRes422 (it can read it, but it's sometimes slow moving from frame to frame and on replaying). It also seems to display the ProRes422 at lower bpc, so you can "see" gradient banding again. [EDIT: Tested a Wire 16bpc moving gradient exported as ProRes422 direct into ScreenFlow then exported out from ScreenFlow at "lossless" ProRes422, I could definitely see prominent banding, it is introducing it somewhere.]
Final Cut Pro does not seem to read DXV, but does read ProRes422, and (even though most editing in ScreenFlow is in fact massively easier than Final Cut Pro) it displays nicely on my 5K iMac with 10-bit display, showing no obvious additional banding. So those nice Wire Gradient gradients that started at 16bpc and were well preserved by ProRes422 are done justice.
Who'd have thought? The dedicated intermediate codec with the bigger filesize gives the best results for high quality generated content (noting again I'm not VJing). And yes, I did have to spend last night clearing out some space on my external SSD 2TB media drive.
Also, by staying in the ProRes422 world, I avoid the need for performing any interim conversions, which saves a lot of time. And although ProRes422 takes a lot of disk space, I save a little bit of disk space by not having additional conversions to MP4 or other.
---
Update: Side-by-side driven gradient dithering tests
Links to some tests of the 'dither' parameter on Wire's gradient, with two colour gradients driven by smoothed randoms. Links are to 30s video on Vimeo. All were exported from Wire as ProRes422. Each video shows the un-dithered on the left, and dithered on the right, and indicates the dithering value and the patch bpc. At 4K and 8bpc patch colour depth I found a dithering value of about 16 is a good compromise between banding and noise/grain. At 4K and 16bpc the banding is less noticeable, but the dithering is still worthwhile. The banding is far more noticeable (as perceived) in all cases when the gradient is changing/driven.
4K 8bpc (int) dithering 32: https://vimeo.com/674821378/0dffd70c5a
4K 8bpc (int) dithering 16: https://vimeo.com/674828726/1f48bca5ca
4K 16bpc (int) dithering 32: https://vimeo.com/674834733/fbb1d39b9c
4K 16bpc (int) dithering 16: https://vimeo.com/674834471/50b75f4c28
The dither is not a cure-all; you can get some strange artefacts if, for example, you use high dither in Gradient with Type 'angular' into Bendoscope, as shown here: https://vimeo.com/675254871/0d5b49a1da