You might have to bring this question over to the Debian Forum. I don’t work with servers, but as noted above this sounds like a situation where you need to install a newer version of the kernel.
You could try using the Testing ISO https://wiki.debian.org/DebianTesting
Handbrake is the tool that comes to mind. You’ve already mentioned FFmpeg which is what I always try to use.