OpenShot Library | libopenshot  0.3.0
/build/libopenshot-0.3.0+dfsg1/doc/HW-ACCEL.md
Go to the documentation of this file.
1 <!--
2 © OpenShot Studios, LLC
3 
4 SPDX-License-Identifier: LGPL-3.0-or-later
5 -->
6 
7 ## Hardware Acceleration
8 
9 OpenShot now has experimental support for hardware acceleration, which uses 1 (or more)
10 graphics cards to offload some of the work for both decoding and encoding. This is
11 very new and experimental (as of May 2019), but we look forward to "accelerating"
12 our support for this in the future!
13 
14 The following table summarizes our current level of support:
15 
16 | | Linux Decode | Linux Encode | Mac Decode | Mac Encode | Windows Decode | Windows Encode | Notes |
17 |--------------------|:---------------:|:--------------:|:----------:|:--------------:|:--------------:|:--------------:|------------------|
18 | VA-API | ✔️ &nbsp; | ✔️ &nbsp; | - | - | - | - | *Linux Only* |
19 | VDPAU | ✔️ <sup>1</sup> | ✅ <sup>2</sup> | - | - | - | - | *Linux Only* |
20 | CUDA (NVDEC/NVENC) | ❌ <sup>3</sup> | ✔️ &nbsp; | - | - | - | ✔️ &nbsp; | *Cross Platform* |
21 | VideoToolBox | - | - | ✔️ &nbsp; | ❌ <sup>4</sup> | - | - | *Mac Only* |
22 | DXVA2 | - | - | - | - | ❌ <sup>3</sup> | - | *Windows Only* |
23 | D3D11VA | - | - | - | - | ❌ <sup>3</sup> | - | *Windows Only* |
24 | QSV | ❌ <sup>3</sup> | ❌ &nbsp; | ❌ &nbsp; | ❌ &nbsp; | ❌ &nbsp; | ❌ &nbsp; | *Cross Platform* |
25 
26 #### Notes
27 
28 1. VDPAU for some reason needs a card number one higher than it really is
29 2. VDPAU is a decoder only
30 3. Green frames (pixel data not correctly tranferred back to system memory)
31 4. Crashes and burns
32 
33 ## Supported FFmpeg Versions
34 
35 * HW accel is supported from FFmpeg version 3.4
36 * HW accel was removed for nVidia drivers in Ubuntu for FFmpeg 4+
37 
38 **Notice:** The FFmpeg versions of Ubuntu and PPAs for Ubuntu show the
39 same behaviour. FFmpeg 3 has working nVidia hardware acceleration while
40 FFmpeg 4+ has no support for nVidia hardware acceleration
41 included.
42 
43 ## OpenShot Settings
44 
45 The following settings are use by libopenshot to enable, disable, and control
46 the various hardware acceleration features.
47 
48 ```{cpp}
49 /// Use video codec for faster video decoding (if supported)
50 int HARDWARE_DECODER = 0;
51 
52 /* 0 - No acceleration
53  1 - Linux VA-API
54  2 - nVidia NVDEC
55  3 - Windows D3D9
56  4 - Windows D3D11
57  5 - MacOS / VideoToolBox
58  6 - Linux VDPAU
59  7 - Intel QSV */
60 
61 /// Number of threads of OpenMP
62 int OMP_THREADS = 12;
63 
64 /// Number of threads that FFmpeg uses
65 int FF_THREADS = 8;
66 
67 /// Maximum rows that hardware decode can handle
68 int DE_LIMIT_HEIGHT_MAX = 1100;
69 
70 /// Maximum columns that hardware decode can handle
71 int DE_LIMIT_WIDTH_MAX = 1950;
72 
73 /// Which GPU to use to decode (0 is the first, LINUX ONLY)
74 int HW_DE_DEVICE_SET = 0;
75 
76 /// Which GPU to use to encode (0 is the first, LINUX ONLY)
77 int HW_EN_DEVICE_SET = 0;
78 ```
79 
80 ## Libva / VA-API (Video Acceleration API)
81 
82 The correct version of libva is needed (libva in Ubuntu 16.04 or libva2
83 in Ubuntu 18.04) for the AppImage to work with hardware acceleration.
84 An AppImage that works on both systems (supporting libva and libva2),
85 might be possible when no libva is included in the AppImage.
86 
87 * vaapi is working for intel and AMD
88 * vaapi is working for decode only for nouveau
89 * nVidia driver is working for export only
90 
91 ## AMD Graphics Cards (RadeonOpenCompute/ROCm)
92 
93 Decoding and encoding on the (AMD) GPU is possible with the default drivers.
94 On systems where ROCm is installed and run a future use for GPU acceleration
95 of effects could be implemented (contributions welcome).
96 
97 ## Multiple Graphics Cards
98 
99 If the computer has multiple graphics cards installed, you can choose which
100 should be used by libopenshot. Also, you can optionally use one card for
101 decoding and the other for encoding (if both cards support acceleration).
102 This is currently only supported on Linux, due to the device name FFmpeg
103 expects (i.e. **/dev/dri/render128**). Contributions welcome if anyone can
104 determine what string format to pass for Windows and Mac.
105 
106 ## Help Us Improve Hardware Support
107 
108 This information might be wrong, and we would love to continue improving
109 our support for hardware acceleration in OpenShot. Please help us update
110 this document if you find an error or discover new and/or useful information.
111 
112 **FFmpeg 4 + nVidia** The manual at:
113 https://www.tal.org/tutorials/ffmpeg_nvidia_encode
114 works pretty well. We could compile and install a version of FFmpeg 4.1.3
115 on Mint 19.1 that supports the GPU on nVidia cards. A version of openshot
116 with hardware support using these libraries could use the nVidia GPU.
117 
118 **BUG:** Hardware supported decoding still has some bugs (as you can see from
119 the chart above). Also, the speed gains with decoding are not as great
120 as with encoding. Currently, if hardware decoding fails, there is no
121 fallback (you either get green frames or an "invalid file" error in OpenShot).
122 This needs to be improved to successfully fall-back to software decoding.
123 
124 **Needed:**
125  * A way to get options and limits of the GPU, such as
126  supported dimensions (width and height).
127  * A way to list the actual Graphic Cards available to FFmpeg (for the
128  user to choose which card for decoding and encoding, as opposed
129  to "Graphics Card X")
130 
131 **Further improvement:** Right now the frame can be decoded on the GPU, but the
132 frame is then copied to CPU memory for modifications. It is then copied back to
133 GPU memory for encoding. Using the GPU for both decoding and modifications
134 will make it possible to do away with these two copies. A possible solution would
135 be to use Vulkan compute which would be available on Linux and Windows natively
136 and on MacOS via MoltenVK.
137 
138 ## Credit
139 
140 A big thanks to Peter M (https://github.com/eisneinechse) for all his work
141 on integrating hardware acceleration into libopenshot! The community thanks
142 you for this major contribution!