图像质量评价和视频质量评价(IQA/VQA)

python代码实现地址

1 IQA/VQA(image quality assessment/video quality assessment)

1.FR(全参考，Full Reference)
2.RR(半参考，Reduced Reference)
3.NR(无参考，No Reference/Blind)

datasets:LIVE/CSIQ/TIB2013 etc...

2 distortions(失真类型)

来源:capturing, compression, transmission, reconstruction, displaying etc

1.block artifacts(块效应,deblocking filter)
2.ringing effect(振铃效应)
3.mosquito noise(蚊式噪声)
4.blur(模糊)
etc...

video-compression-artifacts

3 subjective methods

1.MOS(Mean Opinion Score)
    Single Stimulus Methods
2.DMOS(Differential Mean Opinion Score)
    Double Stimulus Methods

4 objective methods

4.1 evaluation metrics

1.LCC(Linear Correlation Coefficient/Pearson Correlation Coefficient)
2.SROCC(Spearman Rank Order Correlation Coefficient )
3.KROCC(Kendall Rank Order Correlation Coefficient)
4.RMSE(Root Mean Square Error)
5.OR(Outlier ratio)

4.2 FR

1.MSE
2.PSNR
3.SSIM,MS-SSIM
4.VIF(visual information fidelity)
5.JND(Just Noticeable Difference)
6.VMAF(Visual Multimethod Assessment Fusion)
7.FSIM
8.VQM(Video qualitiy metrics)

4.3 NR(blind image quality assessment)

traditional

1.基于特定失真类型:
    1.1:图像模糊(blur)
        paper:A no-reference perceptual blur metric
    1.2:噪声(Noise)
        paper:A fast method for image noise estimation using laplacian operator and adaptive edge detection
    1.3:JPEG2k(块效应,block artifacts)
        paper:Using edge direction information for measuring blocking artifacts of images

2.BIQI
    paper:A Two-Step Framework for Constructing Blind Image Quality Indices
    ideas:
        1.estimates the presence of a set of distortions in the image
        2.evaluates the quality of the image along each of these distortions

3.DIIVINE
    paper:Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality
    ideas:
        1.2-stage framework involving distortion identification followed by
               distortion-specific quality assessment
        2.Statistical Model for Wavelet Coefficients

4.BLINDS-II:
    paper:Blind Image Quality Assessment:A Natural Scene Statistics Approach in the DCT Domain
    ideas:
        1.DCT domain:block DCT coefficients(estimate GGD parameters)
        2.a simple Bayesian inference model to predict image quality scores

5.BRISQUE
    paper:No-Reference Image Quality Assessmentin the Spatial Domain
    ideas:
        1.MSCN(mean subtracted contrast normalized coefficients)
        2.NSS(natural scene statistics):GGD(generalized Gaussian distribution),
                AGGD(asymmetric generalized Gaussian distribution)
        3.GGD,AGGD parameters estimation,concat feature vector,train SVM


6.NIQE
    paper:Making a ‘Completely Blind’ Image Quality Analyzer
    ideas:
        1.opinion unware
        2.patch selection:The variance field
        3.MGD(Multivariate Gaussian distribution):directly calculate score

7.PIQE
    paper:BLIND IMAGE QUALITY EVALUATION USING PERCEPTION BASED FEATURES
    ideas:
        1. label block as uniform or spatially active
        2. blocks are analysed for two type of distortion,namely,noticeable distortion and additive white noise
        3. quantify distortion using block variance


视频质量评价可分为像素域(pixel domain)和压缩域(compression domain)
6.VIIDEO(for video,pixel field)
    paper:A Completely Blind Video Integrity Oracle
    ideas:
        1.Spatial Domain Natural Video Statistics: analyse local statistics of frame
            differences  of videos
        2.Compute low pass filtered frame difference coefficients

7.compression domain
    paper:Research on No-Reference Video Quality Evaluation Algorithm Based on H.264

deep learning

1.Le Kang 2014
    paper:Convolutional Neural Networks for No-Reference Image Quality Assessment
    ideas:
        1.Taking image patches as input, the CNN works in the spatial domain without using
            hand-crafted features that are employed by most previous methods.

2.DIQI
paper:Deep Learning Network For Blind Image Quality Assessment
ideas:
    1.RGB2YIQ
    2.sparse autoencoder is adopted to pre-train each layer(L-BFGS)
    3.fine-tune the DNN

3.DIQA:
paper:Deep CNN-Based Blind Image Quality Predictor
ideas:
    1.in objective distortion part, a pixelwise objective error map is predicted
    using the CNN model.
    2.in HVS-related part, model further learns the human visual perception behavior.

4.DeepBIQ
    paper:On the Use of Deep Learning for Blind Image Quality Assessment
    ideas:
        1.estimates the image quality by average-pooling the scores predicted on multiple
            sub-regions of the original image
        2.fine-tuned for category-based image quality assessment.

5.RankIQA:
    paper:RankIQA: Learning from Rankings for No-reference Image Quality Assessment
    ideas:
        1.Siamese Network
        2.rank score

6.WaDIQaM-FR/NR
paper:Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment
ideas:
    1.Patch weight estimate&Patch quality estimate

7.VSFA
paper:Quality Assessment of In-the-Wild Videos
ideas:
    1.For content-dependency, extract features from a pre-trained image classification neural network. 
    2.For temporal-memory effects, long-term dependencies, especially the temporal hysteresis, are integrated into the network with a gated recurrent unit and a subjectively-inspired temporal pooling layer.

5 references

Laboratory for Image & Video Engineering
blind image quality tool box
IQA research
tensorflow2 DIQA
BRISQUE opencv3
scikit-video
IQA/VQA summary in ZHIHU
无参考视频质量方法研究--林翔宇
所有论文地址