What is correct method to identify streams containing alpha?

[Neon22]

We're trying to interate ffmpeg4 with Pyglet.
Its all going very well but we have one issue:

• If a VP9 (or VP8) file is created using ffmpeg4, from png files with alphas, we get a video file (webm) with alpha internally.

  Code: Select all

  ffmpeg -i image_seq%03d.png -qmin 0 -qmax 50 -crf 5 -b:v 1M output.webm

  • The report from this CLI indicates that an alpha has been generated.
• Sample webm videos with alpha can be found on this page: https://simpl.info/videoalpha/
• However on playing back we only get teh RGB and do not get the alpha unless we override the codec to be 'libvpx'. (same behaviour can be demonstrated in CLI version of ffplay).

Question: What is the correct way to identify (in code) that a VP8 or VP9 stream has an alpha component ? (So we can load it and access it via AVFrame() for use as an RGBA texture.) We are wrapping avutil, avcodec, etc.


More specifically:
We get the FormatContext from the file using AVFormatContext and the stream AVStream from that FormatContext.
Then from the AVStream we get the 'codecpar' param (AVCodecParameters ).
From the AVCodecParameters we can examine the codec_id (167 is VP9) and other useful params like 'bits_per_coded_sample'.

Interestingly these values seem correct for streams like H264 (codec_id=27) at 24 bits but are marked as 0 for the VP9 codec. Leading me to think this is not the correct place to find the proper values.

[Neon22]

OK the native codec is just not as good as the libvpx codec which is supplied by the Webm peeps. So if you want this alpha info then you should overide the codec on load with the libvpx codec.

How you do that - see here: https://stackoverflow.com/questions/353 ... ion-ffmpeg

Basically the third argument to 'avformat_open_input()' needs to be a 'av_find_input_format("libvpx")' which is type
'AVInputFormat'

If you're not sure if its a VP8,9 codec in the file context then you need to probe the file first, discover the codec, and then overide it in an actual load. See here for how to do that:
https://stackoverflow.com/questions/141 ... n-input-do

and lastly - if you want to use the native instead of webm codec unless its got an alpha (but why would you) then you can check to see if its alpha flag is set by checking the AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL side data.

Check AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL if the side_data starts with (uint64_t) 1 there is a transparency layer.

But the general consensus sems to be to always use the libvpx decoder.

Edit: more details
The id of the google VP8 codec is 139. and 167 is the Google VP9 codec.
To replace with the libvpx versions you need to find "libvpx-vp8" and "libvpx-vp9" using e.g. avcodec_find_decoder_by_name("libvpx-vp9") and use this instead when opening. If opoening a stream instead, you can also overide the codec with 'avcodec_open2(codec_context, codec, None)'