分类 OpenCV 下的文章

“OpenCV的全称是Open Source Computer Vision Library,是一个跨平台的计算机视觉库。OpenCV是由英特尔公司发起并参与开发,以BSD许可证授权发行。”

视频编解码分为硬件加速以及非硬件加速。硬件加速是指通过显卡,FPGA等硬件进行视频编解码,由于硬件有专门优化,所以性能高,能耗低,非硬件加速编解码是指通过CPU进行视频编解码,性能就没那么高(虽然有相关CPU指令优化),由于视频编解码计算量很大,所以能耗也很高。在PC平台上主流的硬件加速编解码有Intel集成显卡,Nvidia显卡。

Quick Sync Video
Intel Quick Sync Video(QSV)是Intel GPU上跟视频处理有关的一系列硬件特性的称呼。

英特尔® Quick Sync Video 技术可以快速转换便携式多媒体播放器的视频,还能提供在线共享、视频编辑及视频制作功能。

看到CPU带的集成显卡支持Quick Sync Video就表示支持硬件加速的视频编解码。

比如查看Intel J1900是否支持QSV.如下
https://ark.intel.com/content/www/cn/zh/ark/products/78867/intel-celeron-processor-j1900-2m-cache-up-to-2-42-ghz.html

在不同平台上可通过不同API使用Intel GPU的硬件加速能力。目前主要由两套API:VAAPI以及libmfx。

VAAPI (视频加速API,Video Acceleration API)包含一套开源的库(LibVA) 以及API规范, 用于硬件加速下的视频编解码以及处理,只有Linux上的驱动提供支持。
libmfx。Intel Media SDK中的API规范,支持视频编解码以及媒体处理。支持Windows以及Linux。

所以,回到标题所在的问题,就知道了vaapi应该只是qsv的一种包含关系。

VAAPI驱动
VAAPI驱动属于用户态驱动,用于支持LibVA,底层是i965/1915驱动。Intel提供了两种开源的VAAPI驱动:intel-vaapi-driver以及intel-media-driver,intel-media-driver较intel-vaapi-driver新,维护更积极,所以目前更推荐使用intel-media-driver。

FFmpeg VAAPI/QSV开发环境搭建
对于VAAPI以及Intel Media SDK,如果使用原生API开发的话比较麻烦,好在FFmpeg提供了对应的插件。我们可以通过FFmpeg间接使用这两套API。在FFmpeg中VAAPI还是叫做VAAPI,但是Intel Media SDK却叫做QSV(一脸懵逼)。

FFmpeg-vaapi插件:基于VAAPI接口
FFmpeg-qsv插件:基于Intel Media SDK

FFmpeg VAAPI/QSV开发环境搭建我就不做搬运工了,大家可参考官网教程。

Linux FFmpeg VAAPI/QSV Installation Environment:https://01.org/linuxmedia/quickstart/ffmpeg-vaapi-qsv-installation-environment

查看支持的编解码功能
apt install vainfo

vainfo
libva info: VA-API version 1.7.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.7 (libva 2.6.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 ()
vainfo: Supported profile and entrypoints

  VAProfileMPEG2Simple            :    VAEntrypointVLD
  VAProfileMPEG2Main              :    VAEntrypointVLD
  VAProfileH264Main               :    VAEntrypointVLD
  VAProfileH264Main               :    VAEntrypointEncSliceLP
  VAProfileH264High               :    VAEntrypointVLD
  VAProfileH264High               :    VAEntrypointEncSliceLP
  VAProfileJPEGBaseline           :    VAEntrypointVLD
  VAProfileJPEGBaseline           :    VAEntrypointEncPicture
  VAProfileH264ConstrainedBaseline:    VAEntrypointVLD
  VAProfileH264ConstrainedBaseline:    VAEntrypointEncSliceLP
  VAProfileVP8Version0_3          :    VAEntrypointVLD
  VAProfileHEVCMain               :    VAEntrypointVLD
  VAProfileHEVCMain10             :    VAEntrypointVLD
  VAProfileVP9Profile0            :    VAEntrypointVLD
  VAProfileVP9Profile2            :    VAEntrypointVLD

VAEntrypointVLD 指的是显卡能够解码这个格式,VAEntrypointEncSlice 指的是显卡可以编码这个格式。

form:https://blog.jianchihu.net/intel-gpu-hw-video-codec-develop.html

编译OpenCV 4.5.3出现这个错误
In file included from /opt/opencv-4.5.3/modules/videoio/src/cap_ffmpeg.cpp:50:0:
/opt/opencv-4.5.3/modules/videoio/src/cap_ffmpeg_impl.hpp:535:5: error: ‘AVBSFContext’ does not name a type

 AVBSFContext* bsfc;
 ^

/opt/opencv-4.5.3/modules/videoio/src/cap_ffmpeg_impl.hpp: In member function ‘void CvCapture_FFMPEG::init()’:
/opt/opencv-4.5.3/modules/videoio/src/cap_ffmpeg_impl.hpp:576:5: error: ‘bsfc’ was not declared in this scope

 bsfc = NULL;

add support for HW accelerated decode/encode in FFMPEG backend of VideoCapture and VideoWriter APIs under #ifdef check for FFMPEG version >= 4.0
introduce new properties CAP_PROP_HW_ACCELERATION, CAP_PROP_HW_DEVICE, VIDEOWRITER_PROP_HW_ACCELERATION, VIDEOWRITER_PROP_HW_DEVICE and enum VideoAccelerationType.
new properties supported in three HW-capable backends: ffmpeg, gstreamer, msmf
setting new properties supported only via params parameter in VideoCapture/VideoWriter constructor or open() function, not supported in setProperty() call after open() call
by default (if property not set) HW acceleration enabled in HW-capable backends for both VideoCapture and VideoWriter
if HW accelerated decoder/encoder not found or failed to initilize, VideoCapture/VideoWriter gracefully falls back to SW decoder/encoder. Video acceleration status (HW or SW codec) could be queried via getProperty(CAP_PROP_HW_ACCELERATION) / getProperty(VIDEOWRITER_PROP_HW_ACCELERATION)
new sample video_acceleration
test iterating three backends and acceleration types
HW acceleration types support matrix, in priority order:

OS Backend VideoCapture VideoWriter
Linux FFMPEG VAAPI, MFX MFX, VAAPI

 GStreamer    VAAPI (and others HW plugins)    VAAPI (and others HW plugins)

Windows FFMPEG D3D11, MFX MFX

 MSMF    D3D11

示例代码:
video_acceleration
samples/tapi/video_acceleration.cpp

#include <iostream>
#include <chrono>
#include "opencv2/core.hpp"
#include "opencv2/core/ocl.hpp"
#include "opencv2/core/utility.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/videoio.hpp"
#include "opencv2/highgui.hpp"

using namespace cv;
using namespace std;

const char* keys =
"{ i input    |        | input video file }"
"{ o output   |        | output video file, or specify 'null' to measure decoding without rendering to screen}"
"{ backend    | any    | VideoCapture and VideoWriter backend, valid values: 'any', 'ffmpeg', 'msmf', 'gstreamer' }"
"{ accel      | any    | GPU Video Acceleration, valid values: 'none', 'any', 'd3d11', 'vaapi', 'mfx' }"
"{ device     | -1     | Video Acceleration device (GPU) index (-1 means default device) }"
"{ out_w      |        | output width (resize by calling cv::resize) }"
"{ out_h      |        | output height (resize by calling cv::resize) }"
"{ bitwise_not| false  | apply simple image processing - bitwise_not pixels by calling cv::bitwise_not }"
"{ opencl     | true   | use OpenCL (inside VideoCapture/VideoWriter and for image processing) }"
"{ codec      | H264   | codec id (four characters string) of output file encoder }"
"{ h help     |        | print help message }";

struct {
    cv::VideoCaptureAPIs backend;
    const char* str;
} backend_strings[] = {
    { cv::CAP_ANY, "any" },
    { cv::CAP_FFMPEG, "ffmpeg" },
    { cv::CAP_MSMF, "msmf" },
    { cv::CAP_GSTREAMER, "gstreamer" },
};

struct {
    VideoAccelerationType acceleration;
    const char* str;
} acceleration_strings[] = {
    { VIDEO_ACCELERATION_NONE, "none" },
    { VIDEO_ACCELERATION_ANY, "any" },
    { VIDEO_ACCELERATION_D3D11, "d3d11" },
    { VIDEO_ACCELERATION_VAAPI, "vaapi" },
    { VIDEO_ACCELERATION_MFX, "mfx" },
};

class FPSCounter {
public:
    FPSCounter(double _interval) : interval(_interval) {
    }

    ~FPSCounter() {
        NewFrame(true);
    }

    void NewFrame(bool last_frame = false) {
        num_frames++;
        auto now = std::chrono::high_resolution_clock::now();
        if (!last_time.time_since_epoch().count()) {
            last_time = now;
        }

        double sec = std::chrono::duration_cast<std::chrono::duration<double>>(now - last_time).count();
        if (sec >= interval || last_frame) {
            printf("FPS(last %.2f sec) = %.2f\n", sec, num_frames / sec);
            fflush(stdout);
            num_frames = 0;
            last_time = now;
        }
    }

private:
    double interval = 1;
    std::chrono::time_point<std::chrono::high_resolution_clock> last_time;
    int num_frames = 0;
};

int main(int argc, char** argv)
{
    cv::CommandLineParser cmd(argc, argv, keys);
    if (cmd.has("help"))
    {
        cout << "Usage : video_acceleration [options]" << endl;
        cout << "Available options:" << endl;
        cmd.printMessage();
        return EXIT_SUCCESS;
    }

    string infile = cmd.get<string>("i");
    string outfile = cmd.get<string>("o");
    string codec = cmd.get<string>("codec");
    int device = cmd.get<int>("device");
    int out_w = cmd.get<int>("out_w");
    int out_h = cmd.get<int>("out_h");
    bool use_opencl = cmd.get<bool>("opencl");
    bool bitwise_not = cmd.get<bool>("bitwise_not");

    cv::VideoCaptureAPIs backend = cv::CAP_ANY;
    string backend_str = cmd.get<string>("backend");
    for (size_t i = 0; i < sizeof(backend_strings)/sizeof(backend_strings[0]); i++) {
        if (backend_str == backend_strings[i].str) {
            backend = backend_strings[i].backend;
            break;
        }
    }

    VideoAccelerationType accel = VIDEO_ACCELERATION_ANY;
    string accel_str = cmd.get<string>("accel");
    for (size_t i = 0; i < sizeof(acceleration_strings) / sizeof(acceleration_strings[0]); i++) {
        if (accel_str == acceleration_strings[i].str) {
            accel = acceleration_strings[i].acceleration;
            break;
        }
    }

    ocl::setUseOpenCL(use_opencl);

    VideoCapture capture(infile, backend, {
            CAP_PROP_HW_ACCELERATION, (int)accel,
            CAP_PROP_HW_DEVICE, device
    });
    if (!capture.isOpened()) {
        cerr << "Failed to open VideoCapture" << endl;
        return 1;
    }
    cout << "VideoCapture backend = " << capture.getBackendName() << endl;
    VideoAccelerationType actual_accel = static_cast<VideoAccelerationType>(static_cast<int>(capture.get(CAP_PROP_HW_ACCELERATION)));
    for (size_t i = 0; i < sizeof(acceleration_strings) / sizeof(acceleration_strings[0]); i++) {
        if (actual_accel == acceleration_strings[i].acceleration) {
            cout << "VideoCapture acceleration = " << acceleration_strings[i].str << endl;
            cout << "VideoCapture acceleration device = " << (int)capture.get(CAP_PROP_HW_DEVICE) << endl;
            break;
        }
    }

    VideoWriter writer;
    if (!outfile.empty() && outfile != "null") {
        const char* codec_str = codec.c_str();
        int fourcc = VideoWriter::fourcc(codec_str[0], codec_str[1], codec_str[2], codec_str[3]);
        double fps = capture.get(CAP_PROP_FPS);
        Size frameSize = { out_w, out_h };
        if (!out_w || !out_h) {
            frameSize = { (int)capture.get(CAP_PROP_FRAME_WIDTH), (int)capture.get(CAP_PROP_FRAME_HEIGHT) };
        }
        writer = VideoWriter(outfile, backend, fourcc, fps, frameSize, {
                VIDEOWRITER_PROP_HW_ACCELERATION, (int)accel,
                VIDEOWRITER_PROP_HW_DEVICE, device
        });
        if (!writer.isOpened()) {
            cerr << "Failed to open VideoWriter" << endl;
            return 1;
        }
        cout << "VideoWriter backend = " << writer.getBackendName() << endl;
        actual_accel = static_cast<VideoAccelerationType>(static_cast<int>(writer.get(VIDEOWRITER_PROP_HW_ACCELERATION)));
        for (size_t i = 0; i < sizeof(acceleration_strings) / sizeof(acceleration_strings[0]); i++) {
            if (actual_accel == acceleration_strings[i].acceleration) {
                cout << "VideoWriter acceleration = " << acceleration_strings[i].str << endl;
                cout << "VideoWriter acceleration device = " << (int)writer.get(VIDEOWRITER_PROP_HW_DEVICE) << endl;
                break;
            }
        }
    }

    cout << "\nStarting frame loop. Press ESC to exit\n";

    FPSCounter fps_counter(0.5); // print FPS every 0.5 seconds

    UMat frame, frame2, frame3;

    for (;;)
    {
        capture.read(frame);
        if (frame.empty()) {
            cout << "End of stream" << endl;
            break;
        }

        if (out_w && out_h) {
            cv::resize(frame, frame2, cv::Size(out_w, out_h));
            //cv::cvtColor(frame, outframe, COLOR_BGRA2RGBA);
        }
        else {
            frame2 = frame;
        }

        if (bitwise_not) {
            cv::bitwise_not(frame2, frame3);
        }
        else {
            frame3 = frame2;
        }

        if (writer.isOpened()) {
            writer.write(frame3);
        }

        if (outfile.empty()) {
            imshow("output", frame3);
            char key = (char) waitKey(1);
            if (key == 27)
                break;
            else if (key == 'm') {
                ocl::setUseOpenCL(!cv::ocl::useOpenCL());
                cout << "Switched to " << (ocl::useOpenCL() ? "OpenCL enabled" : "CPU") << " mode\n";
            }
        }
        fps_counter.NewFrame();
    }

    return EXIT_SUCCESS;
}

ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i rtsp://admin:admin@192.168.1.162/1/1 -vf "format=nv12,hwupload" -y -c:v h264_vaapi 1.mp4

ffmpeg version N-103896-ged65498 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.12) 20160609
  configuration: --enable-vaapi --disable-x86asm --enable-nonfree
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59.  9.101 / 59.  9.101
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  9.100 /  8.  9.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
Guessed Channel Layout for Input Stream #0.1 : mono
Input #0, rtsp, from 'rtsp://admin:admin@192.168.1.162/1/1':
  Metadata:
    title           : SDP Descrption
    comment         : SDP Description
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 2560x1440, 25 fps, 25 tbr, 90k tbn
  Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (pcm_alaw (native) -> aac (native))
Press [q] to stop, [?] for help

vainfo --display drm | grep H264.*Enc
libva info: VA-API version 1.13.0
libva info: Trying to open /usr/local/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_13
libva info: va_openDriver() returns 0

  VAProfileH264ConstrainedBaseline:    VAEntrypointEncSlice
  VAProfileH264Main               :    VAEntrypointEncSlice
  VAProfileH264High               :    VAEntrypointEncSlice

使用以下几种方法都可以硬解

ffmpeg -hwaccel vaapi -c:v h264 -i rtsp://admin:admin@192.168.1.162/1/1 -f null -
ffmpeg -hwaccel vaapi -i rtsp://admin:admin@192.168.1.162/1/1 -f null -
ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i rtsp://admin:admin@192.168.1.162/1/1 -f null -
ffmpeg -init_hw_device vaapi=vaapi:/dev/dri/renderD128 -filter_hw_device vaapi -i rtsp://admin:admin@192.168.1.162:554/1/1 -f null -
软解:
ffmpeg -i rtsp://admin:admin@192.168.1.162/1/1 -f null -

查看CPU占用情况,一下就看出来了。

代码路径在github的doc/examples/vaapi_encode.c
https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/vaapi_encode.c

#include <stdio.h>
#include <string.h>
#include <errno.h>

#include <libavcodec/avcodec.h>
#include <libavutil/pixdesc.h>
#include <libavutil/hwcontext.h>

static int width, height;
static AVBufferRef *hw_device_ctx = NULL;

static int set_hwframe_ctx(AVCodecContext *ctx, AVBufferRef *hw_device_ctx)
{
    AVBufferRef *hw_frames_ref;
    AVHWFramesContext *frames_ctx = NULL;
    int err = 0;

    if (!(hw_frames_ref = av_hwframe_ctx_alloc(hw_device_ctx))) {
        fprintf(stderr, "Failed to create VAAPI frame context.\n");
        return -1;
    }
    frames_ctx = (AVHWFramesContext *)(hw_frames_ref->data);
    frames_ctx->format    = AV_PIX_FMT_VAAPI;
    frames_ctx->sw_format = AV_PIX_FMT_NV12;
    frames_ctx->width     = width;
    frames_ctx->height    = height;
    frames_ctx->initial_pool_size = 20;
    if ((err = av_hwframe_ctx_init(hw_frames_ref)) < 0) {
        fprintf(stderr, "Failed to initialize VAAPI frame context."
                "Error code: %s\n",av_err2str(err));
        av_buffer_unref(&hw_frames_ref);
        return err;
    }
    ctx->hw_frames_ctx = av_buffer_ref(hw_frames_ref);
    if (!ctx->hw_frames_ctx)
        err = AVERROR(ENOMEM);

    av_buffer_unref(&hw_frames_ref);
    return err;
}

static int encode_write(AVCodecContext *avctx, AVFrame *frame, FILE *fout)
{
    int ret = 0;
    AVPacket *enc_pkt;

    if (!(enc_pkt = av_packet_alloc()))
        return AVERROR(ENOMEM);

    if ((ret = avcodec_send_frame(avctx, frame)) < 0) {
        fprintf(stderr, "Error code: %s\n", av_err2str(ret));
        goto end;
    }
    while (1) {
        ret = avcodec_receive_packet(avctx, enc_pkt);
        if (ret)
            break;

        enc_pkt->stream_index = 0;
        ret = fwrite(enc_pkt->data, enc_pkt->size, 1, fout);
        av_packet_unref(enc_pkt);
    }

end:
    av_packet_free(&enc_pkt);
    ret = ((ret == AVERROR(EAGAIN)) ? 0 : -1);
    return ret;
}

int main(int argc, char *argv[])
{
    int size, err;
    FILE *fin = NULL, *fout = NULL;
    AVFrame *sw_frame = NULL, *hw_frame = NULL;
    AVCodecContext *avctx = NULL;
    const AVCodec *codec = NULL;
    const char *enc_name = "h264_vaapi";

    if (argc < 5) {
        fprintf(stderr, "Usage: %s <width> <height> <input file> <output file>\n", argv[0]);
        return -1;
    }

    width  = atoi(argv[1]);
    height = atoi(argv[2]);
    size   = width * height;

    if (!(fin = fopen(argv[3], "r"))) {
        fprintf(stderr, "Fail to open input file : %s\n", strerror(errno));
        return -1;
    }
    if (!(fout = fopen(argv[4], "w+b"))) {
        fprintf(stderr, "Fail to open output file : %s\n", strerror(errno));
        err = -1;
        goto close;
    }

    err = av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_VAAPI,
                                 NULL, NULL, 0);
    if (err < 0) {
        fprintf(stderr, "Failed to create a VAAPI device. Error code: %s\n", av_err2str(err));
        goto close;
    }

    if (!(codec = avcodec_find_encoder_by_name(enc_name))) {
        fprintf(stderr, "Could not find encoder.\n");
        err = -1;
        goto close;
    }

    if (!(avctx = avcodec_alloc_context3(codec))) {
        err = AVERROR(ENOMEM);
        goto close;
    }

    avctx->width     = width;
    avctx->height    = height;
    avctx->time_base = (AVRational){1, 25};
    avctx->framerate = (AVRational){25, 1};
    avctx->sample_aspect_ratio = (AVRational){1, 1};
    avctx->pix_fmt   = AV_PIX_FMT_VAAPI;

    /* set hw_frames_ctx for encoder's AVCodecContext */
    if ((err = set_hwframe_ctx(avctx, hw_device_ctx)) < 0) {
        fprintf(stderr, "Failed to set hwframe context.\n");
        goto close;
    }

    if ((err = avcodec_open2(avctx, codec, NULL)) < 0) {
        fprintf(stderr, "Cannot open video encoder codec. Error code: %s\n", av_err2str(err));
        goto close;
    }

    while (1) {
        if (!(sw_frame = av_frame_alloc())) {
            err = AVERROR(ENOMEM);
            goto close;
        }
        /* read data into software frame, and transfer them into hw frame */
        sw_frame->width  = width;
        sw_frame->height = height;
        sw_frame->format = AV_PIX_FMT_NV12;
        if ((err = av_frame_get_buffer(sw_frame, 0)) < 0)
            goto close;
        if ((err = fread((uint8_t*)(sw_frame->data[0]), size, 1, fin)) <= 0)
            break;
        if ((err = fread((uint8_t*)(sw_frame->data[1]), size/2, 1, fin)) <= 0)
            break;

        if (!(hw_frame = av_frame_alloc())) {
            err = AVERROR(ENOMEM);
            goto close;
        }
        if ((err = av_hwframe_get_buffer(avctx->hw_frames_ctx, hw_frame, 0)) < 0) {
            fprintf(stderr, "Error code: %s.\n", av_err2str(err));
            goto close;
        }
        if (!hw_frame->hw_frames_ctx) {
            err = AVERROR(ENOMEM);
            goto close;
        }
        if ((err = av_hwframe_transfer_data(hw_frame, sw_frame, 0)) < 0) {
            fprintf(stderr, "Error while transferring frame data to surface."
                    "Error code: %s.\n", av_err2str(err));
            goto close;
        }

        if ((err = (encode_write(avctx, hw_frame, fout))) < 0) {
            fprintf(stderr, "Failed to encode.\n");
            goto close;
        }
        av_frame_free(&hw_frame);
        av_frame_free(&sw_frame);
    }

    /* flush encoder */
    err = encode_write(avctx, NULL, fout);
    if (err == AVERROR_EOF)
        err = 0;

close:
    if (fin)
        fclose(fin);
    if (fout)
        fclose(fout);
    av_frame_free(&sw_frame);
    av_frame_free(&hw_frame);
    avcodec_free_context(&avctx);
    av_buffer_unref(&hw_device_ctx);

    return err;
}

https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/vaapi_transcode.c

#include <stdio.h>
#include <errno.h>

#include <libavutil/hwcontext.h>
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

static AVFormatContext *ifmt_ctx = NULL, *ofmt_ctx = NULL;
static AVBufferRef *hw_device_ctx = NULL;
static AVCodecContext *decoder_ctx = NULL, *encoder_ctx = NULL;
static int video_stream = -1;
static AVStream *ost;
static int initialized = 0;

static enum AVPixelFormat get_vaapi_format(AVCodecContext *ctx,
                                           const enum AVPixelFormat *pix_fmts)
{
    const enum AVPixelFormat *p;

    for (p = pix_fmts; *p != AV_PIX_FMT_NONE; p++) {
        if (*p == AV_PIX_FMT_VAAPI)
            return *p;
    }

    fprintf(stderr, "Unable to decode this file using VA-API.\n");
    return AV_PIX_FMT_NONE;
}

static int open_input_file(const char *filename)
{
    int ret;
    const AVCodec *decoder = NULL;
    AVStream *video = NULL;

    if ((ret = avformat_open_input(&ifmt_ctx, filename, NULL, NULL)) < 0) {
        fprintf(stderr, "Cannot open input file '%s', Error code: %s\n",
                filename, av_err2str(ret));
        return ret;
    }

    if ((ret = avformat_find_stream_info(ifmt_ctx, NULL)) < 0) {
        fprintf(stderr, "Cannot find input stream information. Error code: %s\n",
                av_err2str(ret));
        return ret;
    }

    ret = av_find_best_stream(ifmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &decoder, 0);
    if (ret < 0) {
        fprintf(stderr, "Cannot find a video stream in the input file. "
                "Error code: %s\n", av_err2str(ret));
        return ret;
    }
    video_stream = ret;

    if (!(decoder_ctx = avcodec_alloc_context3(decoder)))
        return AVERROR(ENOMEM);

    video = ifmt_ctx->streams[video_stream];
    if ((ret = avcodec_parameters_to_context(decoder_ctx, video->codecpar)) < 0) {
        fprintf(stderr, "avcodec_parameters_to_context error. Error code: %s\n",
                av_err2str(ret));
        return ret;
    }

    decoder_ctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);
    if (!decoder_ctx->hw_device_ctx) {
        fprintf(stderr, "A hardware device reference create failed.\n");
        return AVERROR(ENOMEM);
    }
    decoder_ctx->get_format    = get_vaapi_format;

    if ((ret = avcodec_open2(decoder_ctx, decoder, NULL)) < 0)
        fprintf(stderr, "Failed to open codec for decoding. Error code: %s\n",
                av_err2str(ret));

    return ret;
}

static int encode_write(AVPacket *enc_pkt, AVFrame *frame)
{
    int ret = 0;

    av_packet_unref(enc_pkt);

    if ((ret = avcodec_send_frame(encoder_ctx, frame)) < 0) {
        fprintf(stderr, "Error during encoding. Error code: %s\n", av_err2str(ret));
        goto end;
    }
    while (1) {
        ret = avcodec_receive_packet(encoder_ctx, enc_pkt);
        if (ret)
            break;

        enc_pkt->stream_index = 0;
        av_packet_rescale_ts(enc_pkt, ifmt_ctx->streams[video_stream]->time_base,
                             ofmt_ctx->streams[0]->time_base);
        ret = av_interleaved_write_frame(ofmt_ctx, enc_pkt);
        if (ret < 0) {
            fprintf(stderr, "Error during writing data to output file. "
                    "Error code: %s\n", av_err2str(ret));
            return -1;
        }
    }

end:
    if (ret == AVERROR_EOF)
        return 0;
    ret = ((ret == AVERROR(EAGAIN)) ? 0:-1);
    return ret;
}

static int dec_enc(AVPacket *pkt, const AVCodec *enc_codec)
{
    AVFrame *frame;
    int ret = 0;

    ret = avcodec_send_packet(decoder_ctx, pkt);
    if (ret < 0) {
        fprintf(stderr, "Error during decoding. Error code: %s\n", av_err2str(ret));
        return ret;
    }

    while (ret >= 0) {
        if (!(frame = av_frame_alloc()))
            return AVERROR(ENOMEM);

        ret = avcodec_receive_frame(decoder_ctx, frame);
        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
            av_frame_free(&frame);
            return 0;
        } else if (ret < 0) {
            fprintf(stderr, "Error while decoding. Error code: %s\n", av_err2str(ret));
            goto fail;
        }

        if (!initialized) {
            /* we need to ref hw_frames_ctx of decoder to initialize encoder's codec.
               Only after we get a decoded frame, can we obtain its hw_frames_ctx */
            encoder_ctx->hw_frames_ctx = av_buffer_ref(decoder_ctx->hw_frames_ctx);
            if (!encoder_ctx->hw_frames_ctx) {
                ret = AVERROR(ENOMEM);
                goto fail;
            }
            /* set AVCodecContext Parameters for encoder, here we keep them stay
             * the same as decoder.
             * xxx: now the sample can't handle resolution change case.
             */
            encoder_ctx->time_base = av_inv_q(decoder_ctx->framerate);
            encoder_ctx->pix_fmt   = AV_PIX_FMT_VAAPI;
            encoder_ctx->width     = decoder_ctx->width;
            encoder_ctx->height    = decoder_ctx->height;

            if ((ret = avcodec_open2(encoder_ctx, enc_codec, NULL)) < 0) {
                fprintf(stderr, "Failed to open encode codec. Error code: %s\n",
                        av_err2str(ret));
                goto fail;
            }

            if (!(ost = avformat_new_stream(ofmt_ctx, enc_codec))) {
                fprintf(stderr, "Failed to allocate stream for output format.\n");
                ret = AVERROR(ENOMEM);
                goto fail;
            }

            ost->time_base = encoder_ctx->time_base;
            ret = avcodec_parameters_from_context(ost->codecpar, encoder_ctx);
            if (ret < 0) {
                fprintf(stderr, "Failed to copy the stream parameters. "
                        "Error code: %s\n", av_err2str(ret));
                goto fail;
            }

            /* write the stream header */
            if ((ret = avformat_write_header(ofmt_ctx, NULL)) < 0) {
                fprintf(stderr, "Error while writing stream header. "
                        "Error code: %s\n", av_err2str(ret));
                goto fail;
            }

            initialized = 1;
        }

        if ((ret = encode_write(pkt, frame)) < 0)
            fprintf(stderr, "Error during encoding and writing.\n");

fail:
        av_frame_free(&frame);
        if (ret < 0)
            return ret;
    }
    return 0;
}

int main(int argc, char **argv)
{
    const AVCodec *enc_codec;
    int ret = 0;
    AVPacket *dec_pkt;

    if (argc != 4) {
        fprintf(stderr, "Usage: %s <input file> <encode codec> <output file>\n"
                "The output format is guessed according to the file extension.\n"
                "\n", argv[0]);
        return -1;
    }

    ret = av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_VAAPI, NULL, NULL, 0);
    if (ret < 0) {
        fprintf(stderr, "Failed to create a VAAPI device. Error code: %s\n", av_err2str(ret));
        return -1;
    }

    dec_pkt = av_packet_alloc();
    if (!dec_pkt) {
        fprintf(stderr, "Failed to allocate decode packet\n");
        goto end;
    }

    if ((ret = open_input_file(argv[1])) < 0)
        goto end;

    if (!(enc_codec = avcodec_find_encoder_by_name(argv[2]))) {
        fprintf(stderr, "Could not find encoder '%s'\n", argv[2]);
        ret = -1;
        goto end;
    }

    if ((ret = (avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, argv[3]))) < 0) {
        fprintf(stderr, "Failed to deduce output format from file extension. Error code: "
                "%s\n", av_err2str(ret));
        goto end;
    }

    if (!(encoder_ctx = avcodec_alloc_context3(enc_codec))) {
        ret = AVERROR(ENOMEM);
        goto end;
    }

    ret = avio_open(&ofmt_ctx->pb, argv[3], AVIO_FLAG_WRITE);
    if (ret < 0) {
        fprintf(stderr, "Cannot open output file. "
                "Error code: %s\n", av_err2str(ret));
        goto end;
    }

    /* read all packets and only transcoding video */
    while (ret >= 0) {
        if ((ret = av_read_frame(ifmt_ctx, dec_pkt)) < 0)
            break;

        if (video_stream == dec_pkt->stream_index)
            ret = dec_enc(dec_pkt, enc_codec);

        av_packet_unref(dec_pkt);
    }

    /* flush decoder */
    av_packet_unref(dec_pkt);
    ret = dec_enc(dec_pkt, enc_codec);

    /* flush encoder */
    ret = encode_write(dec_pkt, NULL);

    /* write the trailer for output stream */
    av_write_trailer(ofmt_ctx);

end:
    avformat_close_input(&ifmt_ctx);
    avformat_close_input(&ofmt_ctx);
    avcodec_free_context(&decoder_ctx);
    avcodec_free_context(&encoder_ctx);
    av_buffer_unref(&hw_device_ctx);
    av_packet_free(&dec_pkt);
    return ret;
}