GStreamer VA-API

A year after

Víctor Jáquez

GStreamer Conference 2016

10-11 October (Berlin)

Video Acceleration

—

Application Programming Interface

What is VA-API?

• An API specification
  • Client side
  • Driver (backend) side
• A library implementation
  • Open Source MIT license
  • It opens and registers a backend

http://cgit.freedesktop.org/vaapi/libva/

What VA-API can do?

It depends on the driver.

• Decoding
• Encoding
• Post-processing
• Compositing & Rendering

What drivers are available?

• Intel VA driver for Intel chip-sets
• Intel hybrid driver
• Mesa's state-trackers for gallium drivers:
  • radeon, nouveau (?), freedreno, …
• Deprecated API bridges
  • vdpau—va bridge
  • powervr—va bridge
  • …

Intel's decoders

Intel's posptrocessor filters

Intel's encoders

Mesa state-tracker

$ vainfo
libva info: VA-API version 0.39.3
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nouveau_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.39 (libva 1.7.2)
vainfo: Driver version: mesa gallium vaapi
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264Baseline           : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

What is new?

~1 year

~390 commits

19 contributors

VP9 support

• decoder & encoder
  • Only for Kabylake
  • (Skylake using Intel's hybrid driver)
• Thanks Sree!

An official component of GStreamer

• Under GStreamer repository & bugzilla
• Follow the GStreamer release rules
• Removed libgstvaapi.so public library
• Removed custom video parsers
• Use Gstreamer's common tools (autotools & gkt-doc)
• Many thanks to Tim and Sebastian!

Encoders rename

Decoders split

Though, vaapidecodebin is still a thing...

Fixed VA surface mapping

• Differenciate negotiation caps (used by pads) and allocations caps (used by the bufferpools)

Encoders enhancements

• vaapih265enc: CBR support
• vaapih264enc: low power encode

Drivers white-list

• Only Intel's and Mesa drivers are supported
• Unmaintained API bridges bails out the gstreamer-vaapi registration
  • Unless the environment variable GST_VAAPI_ALL_DRIVERS=1 is exported

Improved plugin registration

• Only the available plugin features are registered
• Track dependencies to invalidate registration cache
  • /dev/dri/{card*,render*}
  • ${LIBVA_DRIVERS_PATH}/*_drv_video.so
  • GST_VAAPI_ALL_DRIVERS
  • LIBVA_DRIVER_NAME
  • DISPLAY
  • WAYLAND_DISPLAY

Plugin registration (Haswell)

$ gst-inspect-1.0 vaapi
Plugin Details:
  Name                     vaapi
  Description              VA-API based elements
  Filename                 /opt/gnome/jh/lib/gstreamer-1.0/libgstvaapi.so
  Version                  1.9.90
  License                  LGPL
  Source module            gstreamer-vaapi
  Source release date      2016-09-30
  Binary package           gstreamer-vaapi
  Origin URL               http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer

 vaapijpegdec: VA-API JPEG decoder
 vaapimpeg2dec: VA-API MPEG2 decoder
 vaapih264dec: VA-API H264 decoder
 vaapivc1dec: VA-API VC1 decoder
 vaapipostproc: VA-API video postprocessing
 vaapidecodebin: VA-API Decode Bin
 vaapisink: VA-API sink
 vaapimpeg2enc: VA-API MPEG-2 encoder
 vaapih264enc: VA-API H.264 encoder

 9 features:
 +-- 9 elements

Misc

• Improved DMA-buf importation from upstream buffers (cache EGLImages)
• Added support for reverse playback (show only I frames)
• Optimized native GL texture upload (cache internal texture objects)
  • Thanks Hyunjun!

What is next?

(still under discussion)

Misc

• Support DMA-buf exportation for downstream buffers (bug 755072)
• Improve reverse playback (bug 768692)
• Regression testing by adding gst-validate scenarios

Encoders

• Configure encoders by caps negotiation (bugs 723282, 769266, 757941, etc.)
• Provide advanced tuning and control options

decodebin3

• Add support for auto-plug hardware-accelerated video filters (vaapipostproc)
  • Thus, get rid of vaaapidecodebin
  • bug 687182

GL

• Replace of custom GL code with GstGL

GstObject

• Get rid of GstVaapiObject (when possible)
  • Thus, be able to use gst-trace
  • bug 768266

Applications share VA display

• Expose a small library for GstVaapiDisplay (bug 766704)
  • Resurrect /usr/lib/libgstvaapi.so
  • Add support for gobject-introspection
• Remove display cache (bug 747946)
• Support multiple vaapisink with differente VA displays (bug 754820)

Optimization

• Get rid of slice parsing code
  • Transfer it to video parsers
  • bug 691712 & bug 704865

Questions?

Thanks!

• Twitter @ceyusa
• Mail: vjaquez at igalia
• Blog: http://blogs.igalia.com/vjaquez