A Fast QTMT Partition Decision Strategy for VVC Intra Prediction

• Published in: IEEE access 2020, Vol.8, p.107900-107911
• Main Authors: Fan, Yibo, Chen, Jun'An, Sun, Heming, Katto, Jiro, Jing, Ming'E
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Asymmetric block size; Codec; Coding; Coding standards; Complexity; Computational complexity; Convolution; Copper; Encoding; fast partition decision; Feature extraction; intra prediction; Machine learning; Partitions; quadtree with multi-type tree; Support vector machines; Texture; Vegetation; versatile video coding; Video coding; Video compression
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.3000565

Different from the traditional quaternary tree (QT) structure utilized in the previous generation video coding standard H.265/HEVC, a brand new partition structure named quadtree with nested multi-type tree (QTMT) is applied in the latest codec H.266/VVC. The introduction of QTMT brings in superior encoding performance at the cost of great time-consuming. Therefore, a fast intra partition algorithm based on variance and Sobel operator is proposed in this paper. The proposed method settles the novel asymmetrical partition issue in VVC by well balancing the reduction of computational complexity and the loss of encoding quality. To be more concrete, we first terminate further splitting of a coding unit (CU) when the texture of it is judged as smooth. Then, we use Sobel operator to extract gradient features to decide whether to split this CU by QT, thus terminating further MT partitions. Finally, a completely novel method to choose only one partition from five QTMT partitions is applied. Obviously, homogeneous area tends to use a larger CU as a whole to do prediction while CUs with complicated texture are prone to be divided into small sub-CUs and these sub-CUs usually have different textures from each other. We calculate the variance of variance of each sub-CU to decide which partition will distinguish the sub-textures best. Our method is embedded into the latest VVC official reference software VTM-7.0. Comparing to anchor VTM-7.0, our method saves the encoding time by 49.27% on average at the cost of only 1.63% BDBR increase. As a traditional scheme based on variance and gradient to decrease the computational complexity in VVC intra coding, our method outperforms other relative existing state-of-the-art methods, including traditional machine learning and convolution neural network methods.