ffmpeg日志文件中解析数值PSNR，码率的样例

zymill

于 2021-12-30 13:57:01 发布

阅读量2.2k

点赞数 1

分类专栏： ffmpeg实例编程细节视频技术文章标签： python ffmpeg 数值解析 hybase

本文链接：https://blog.csdn.net/zymill/article/details/122235059

版权

编程细节同时被 3 个专栏收录

27 篇文章 0 订阅

订阅专栏

视频技术

21 篇文章 1 订阅

订阅专栏

ffmpeg实例

14 篇文章 3 订阅

订阅专栏

以下是ffmpeg执行转码记录到文件的日志，为了能分析转码结果，其中PSNR，平均码率等数值是比较常用的参数，为了能快速提前结果，本文提供了一个python的处理样例，请参考后续代码。

hybase@qq.com http://blog.csdn.net/zymill

================== start ==================== 
ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-11)
  configuration: --enable-shared --enable-static --enable-version3 --enable-gpl --enable-nonfree --enable-libfdk-aac --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libxvid --enable-libdavs2 --enable-libxavs2 --enable-libdav1d --enable-libuavs3d --enable-cuda --enable-libnpp --enable-cuvid --enable-ffnvcodec --enable-nvenc --enable-nvdec --enable-zlib --enable-libfribidi --enable-libfreetype --enable-libxml2 --enable-fontconfig --enable-libass --enable-frei0r --enable-opengl --enable-filter=gltransition --extra-libs='-lGLEW -lglfw3' --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mpegts, from '/home/ffmpeg-4.4.x/ai_trc_shfiles/nobody_hls/nobody_00001.ts':
  Duration: 00:00:04.67, start: 6.822000, bitrate: 4077 kb/s
  Program 1 
    Metadata:
      service_name    : hys Dec 22 2021 17:52:49_s01
      service_provider: hys
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x15ad340] --psnr used with psy on: results will be invalid!
[libx264 @ 0x15ad340] --tune psnr should be used if attempting to benchmark psnr!
[libx264 @ 0x15ad340] using SAR=1/1
[libx264 @ 0x15ad340] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
[libx264 @ 0x15ad340] profile High, level 4.0, 4:2:0, 8-bit
[mpegts @ 0x1591380] [mpegts_init] init first_pcr=22950000 (90khz:76500, ms:850), timestamp mode: copyts=-1
[mpegts @ 0x1591380] [mpegts_init] cbr mode: mux_rate=0 bps, pcr_period=33 ms pat_period=80 ms
[mpegts @ 0x1591380] [mpegts_init] ts->copyts= -1, first_pcr= 22950000, last_pcr=22950000, max_delay= 850000us, period ms: pat=80 sdt=1200 pcr=33, mpegts_flags: hys_mux= 0 no_keyframe_pcr= 0 av_same_mux_delay= 0
Output #0, mpegts, to '/home/ffmpeg-4.4.x/ai_trc_shfiles/nobody_trc/trc_nobody_00001.ts':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264, yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 90k tbn
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=    1 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
frame=   57 fps= 24 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
frame=   69 fps= 24 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
frame=   80 fps= 23 q=28.0 size=     256kB time=00:00:00.33 bitrate=6285.0kbits/s speed=0.0979x    
frame=   92 fps= 23 q=28.0 size=     512kB time=00:00:00.83 bitrate=5028.1kbits/s speed=0.211x    
frame=  103 fps= 23 q=28.0 size=     512kB time=00:00:01.29 bitrate=3243.9kbits/s speed=0.286x    
frame=  112 fps= 22 q=28.0 size=     768kB time=00:00:01.66 bitrate=3771.1kbits/s speed=0.331x    
frame=  112 fps= 17 q=-1.0 Lsize=    2577kB time=00:00:04.58 bitrate=4601.6kbits/s speed=0.678x    
video:2487kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.603857%
[libx264 @ 0x15ad340] frame I:3     Avg QP:18.16  size: 59277  PSNR Mean Y:48.36 U:52.91 V:52.56 Avg:49.38 Global:49.30
[libx264 @ 0x15ad340] frame P:51    Avg QP:22.12  size: 32512  PSNR Mean Y:46.27 U:51.56 V:52.01 Avg:47.44 Global:47.08
[libx264 @ 0x15ad340] frame B:58    Avg QP:25.41  size: 12263  PSNR Mean Y:44.93 U:50.65 V:51.14 Avg:46.16 Global:45.98
[libx264 @ 0x15ad340] consecutive B-frames:  7.1% 67.9% 10.7% 14.3%  0.0%
[libx264 @ 0x15ad340] mb I  I16..4: 34.1% 59.9%  6.0%
[libx264 @ 0x15ad340] mb P  I16..4:  7.8% 24.6%  1.8%  P16..4: 22.2%  5.5%  3.0%  0.0%  0.0%    skip:35.0%
[libx264 @ 0x15ad340] mb B  I16..4:  2.4%  4.9%  0.2%  B16..8: 26.3%  3.7%  0.6%  direct: 3.2%  skip:58.7%  L0:47.5% L1:45.0% BI: 7.5%
[libx264 @ 0x15ad340] 8x8 transform intra:69.3% inter:75.8%
[libx264 @ 0x15ad340] direct mvs  spatial:81.0% temporal:19.0%
[libx264 @ 0x15ad340] coded y,uvDC,uvAC intra: 42.2% 51.3% 8.0% inter: 11.6% 13.8% 0.1%
[libx264 @ 0x15ad340] i16 v,h,dc,p: 46% 15% 11% 28%
[libx264 @ 0x15ad340] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 13% 14%  4%  6%  8%  6%  7%  5%
[libx264 @ 0x15ad340] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 14%  5%  5% 10% 11%  8%  8%  7%
[libx264 @ 0x15ad340] i8c dc,h,v,p: 44% 18% 31%  8%
[libx264 @ 0x15ad340] Weighted P-Frames: Y:9.8% UV:3.9%
[libx264 @ 0x15ad340] ref P L0: 84.6% 11.6%  2.8%  1.0%
[libx264 @ 0x15ad340] ref B L0: 91.3%  7.2%  1.6%
[libx264 @ 0x15ad340] SSIM Mean Y:0.9919295 (20.931db)
[libx264 @ 0x15ad340] PSNR Mean Y:45.632 U:51.126 V:51.573 Avg:46.828 Global:46.513 kb/s:4362.20

解析上述日志中最后一行，获取PNSR，码率的Python方法如下，供参考

#/****************************************************************************
#* mark_trc.py: mark for x264 encode base on PSNR and bitrate
#               python 3.x
#*****************************************************************************
#* Copyright (c) 2021-2022 Hybase@qq.com
#* http://blog.csdn.net/zymill
#* http://github.com/zymill
#*****************************************************************************
#* Last update Date: 2021-12-30 19:30:25           version: 0.0.1
#*****************************************************************************
#*/
import os
import re
import sys
import logging
import linecache
from logging import handlers

################################################################################
# 常量参数
trc_target_avg_kbps_sd   = 1500    # 720x576
trc_target_avg_kbps_hd   = 2000    # 1280x720
trc_target_avg_kbps_uhd  = 3000    # 1920x1080
trc_target_avg_kbps_4k   = 10000   # 3840x2160
sd_r_list                = []
hd_r_list                = []
uhd_r_list               = []
u4k_r_list               = []
score_list               = []

################################ logger ######################################
class Logger(object):
    level_relations = {
        'debug':logging.DEBUG,
        'info':logging.INFO,
        'warning':logging.WARNING,
        'error':logging.ERROR,
        'crit':logging.CRITICAL
    }

    def __init__(self, filename, level='info', when='D', backCount=3, fmt='%(asctime)s - %(pathname)s[line:%(lineno)d] - [%(levelname)s] %(message)s'):
        self.logger = logging.getLogger(filename)
        format_str = logging.Formatter(fmt)
        self.logger.setLevel(self.level_relations.get(level))
        sh = logging.StreamHandler()
        sh.setFormatter(format_str)
        th = handlers.TimedRotatingFileHandler(filename=filename,when=when,backupCount=backCount,encoding='utf-8')
        th.setFormatter(format_str)
        self.logger.addHandler(sh)
        self.logger.addHandler(th)

################################################################################
# Methods check dir/file
def traverseDir(root_path, file_list, dir_list):
    dir_or_files = os.listdir(root_path)
    for dir_file in dir_or_files:
        dir_file_path = os.path.join(root_path, dir_file)
        if os.path.isdir(dir_file_path):
            dir_list.append(dir_file_path)
            traverseDir(dir_file_path, file_list, dir_list)
        else:
            file_list.append(dir_file_path)

#################################################################################
# check dst_path, create it if not found
def createDirIfNotFound(log, dst_path):
    if not os.path.exists(dst_path):
        os.makedirs(dst_path)
        log.logger.info('created ' + dst_path)
        return
    log.logger.info('found ' + dst_path)
    return

#################################################################################
# parse float value from line
def parseKeyValueFromLine(log, line, pattern):
    val = 0.0
    result_lst = pattern.findall(line)
    log.logger.info(result_lst)
    if (len(result_lst) > 0):
        val = float(result_lst[0])
    return val

#################################################################################
# parse video resolution
#  Stream #0:0: Video: h264, yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 90k tbn
def parsetVideoResolution(log, line):
    global video_width
    global video_height
    # default value
    video_width  = 0
    video_height = 0

    end = line.find("[SAR")
    if (end == -1):
        return

    sub_str = line[0 : end-1]
    start = sub_str.rfind(",");
    if (start == -1) :
        return

    sub_str = sub_str[start + 1 : ]
    log.logger.info('video resolution str: ' + sub_str)
    result_list = sub_str.split('x')
    if (len(result_list) == 2):
        video_width = int(result_list[0])
        video_height = int(result_list[1])
    return

#################################################################################
# parse PSNR and bitrate
# [libx264 @ 0x2503340 ] PSNR Mean Y:59.691 U:62.357 V:62.717 Avg:60.439 Global:59.903 kb/s:44056.39
def parsePSNRBitRate(log, line):
    global y_psnr
    global u_psnr
    global v_psnr
    global avg_psnr
    global global_psnr
    global avg_bitrate
    # default value
    y_psnr      = 0.0
    u_psnr      = 0.0
    v_psnr      = 0.0
    avg_psnr    = 0.0
    global_psnr = 0.0
    avg_bitrate = 0.0

    log.logger.debug('psnr_bitrate line: ' + line)
    if ((-1 == line.find("PSNR Mean")) or (-1 == line.find("kb/s:"))) :
        return

    y_psnr      = parseKeyValueFromLine(log, line, re.compile(r'(?<=Y:)\d+\.?\d*'))
    u_psnr      = parseKeyValueFromLine(log, line, re.compile(r'(?<=U:)\d+\.?\d*'))
    v_psnr      = parseKeyValueFromLine(log, line, re.compile(r'(?<=V:)\d+\.?\d*'))
    avg_psnr    = parseKeyValueFromLine(log, line, re.compile(r'(?<=Avg:)\d+\.?\d*'))
    global_psnr = parseKeyValueFromLine(log, line, re.compile(r'(?<=Global:)\d+\.?\d*'))
    avg_bitrate = parseKeyValueFromLine(log, line, re.compile(r'(?<=kb/s:)\d+\.?\d*'))
    return

#################################################################################
# [libx264 @ 0x2503340 ] SSIM Mean Y:0.9995006 (33.016db)
def parseSSIM(log, line):
    global avg_ssim
    avg_ssim  = 0
    log.logger.info('line: ' + line)
    avg_ssim = parseKeyValueFromLine(log, line, re.compile(r'(?<=SSIM Mean Y::)\d+\.?\d*'))
    return

#################################################################################
# process one log file
def parseOneLogFile(log, file_name):
    if not os.path.exists(file_name):
        log.logger.error('file not existed ' + file_name)
        return

    log.logger.info('start to parse results from file: ' + file_name)
    global video_width
    global video_height
    global y_psnr
    global u_psnr
    global v_psnr
    global avg_psnr
    global global_psnr
    global avg_bitrate
    global avg_ssim

    # default value
    video_width  = 0
    video_height = 0
    y_psnr       = 0.0
    u_psnr       = 0.0
    v_psnr       = 0.0
    avg_psnr     = 0.0
    global_psnr  = 0.0
    avg_bitrate  = 0.0

    for line_idx, line in enumerate(open(file_name, "rU")):
        if ((-1 != line.find("Stream")) and \
            (-1 != line.find("Video")) and \
            (-1 != line.find("[SAR")) ):
            parsetVideoResolution(log, line)
        if (-1 != line.find("PSNR Mean") and \
            (-1 != line.find("kb/s:"))):
            parsePSNRBitRate(log, line)
        if (-1 != line.find("SSIM Mean")):
            parseSSIM(log, line)

    log.logger.info('##########################################################################')
    log.logger.info(' video resolution: %d x %d', video_width, video_height)
    log.logger.info(' y_psnr          : %f', y_psnr)
    log.logger.info(' u_psnr          : %f', u_psnr)
    log.logger.info(' v_psnr          : %f', v_psnr)
    log.logger.info(' avg_psnr        : %f', avg_psnr)
    log.logger.info(' global_psnr     : %f', global_psnr)
    log.logger.info(' avg_bitrate     : %f', avg_bitrate)
    log.logger.info(' avg_ssim        : %f', avg_ssim)
    log.logger.info('##########################################################################')
    return

################################################################################
# 常量参数
trc_target_avg_kbps_sd   = 1500    # 720x576
trc_target_avg_kbps_hd   = 2000    # 1280x720
trc_target_avg_kbps_uhd  = 3000    # 1920x1080
trc_target_avg_kbps_4k   = 10000   # 3840x2160
trc_log_dir_windows      = "g:/ai_trc_shfiles/nobody_trc"
trc_log_dir_linux        = ""
trc_log_dir              = trc_log_dir_windows

# global variables
video_width         = 0
video_height        = 0
y_psnr              = 0.0
u_psnr              = 0.0
v_psnr              = 0.0
avg_psnr            = 0.0
global_psnr         = 0.0
avg_bitrate         = 0.0
avg_ssim            = 0.0  # 暂时未用 ssim 和 psnr 是两种独立的评估方法

#################################################################################
# main process
if __name__ == "__main__":
    log = Logger('sys_all.log',level='debug')
    Logger('sys_err.log', level='error')
    log.logger.info("========= program start ============")

    ## 获取日志文件列表
    root_path = trc_log_dir
    # 文件路径列表
    file_list = []
    log_file_list = []
    # 目录路径列表
    dir_list = []

    traverseDir(root_path, file_list, dir_list)

    ## show directory info
    log.logger.info('>>>')
    log.logger.info('found %d dir', len(dir_list))
    for dir in dir_list:
        log.logger.info(dir)

    ## filter log file BY ".log"
    for file in file_list:
        if (-1 != file.rfind(".log")): ## valid transcode log file
            log.logger.debug(file)
            log_file_list.append(file)

    # show log file info
    log.logger.info('>>>')
    log.logger.info('found %d log files (in %d files)', len(log_file_list), len(file_list))

    for file in log_file_list:
        ## 从日志中解析视频分辨率， PSNR, bitrate等值
        parseOneLogFile(log, file)

    ## end of all cases
    log.logger.info("=== end ===")
    log.logger.info("============ program exit ============")
    sys.exit(0)

################################################################################
# end
#