Quality assessment of H.264/AVC encoded sequences subject to transmission losses

 

Methodology

The Absolute Category Rating (ACR) subjective test methodology has been used.

Each participant is presented a set of video sequences. After the presentation of each video sequence, he is asked to score its quality - see ITU-T P.910 for additional details.

 

Participants

53 individuals from both genders participated in this test. Their ages were in the range 17 to 50 (most of them were students).

11 participants declared to be familiar with video encoding processes but none of them was working in the video quality assessment field.

Participants were screened for normal visual acuity and color blindness. Their assessments were analyzed in order to detect inconsistencies.

The above procedures detected 3 blind color persons and 2 participants with inconsistent results. Therefore, a total of 48 participants were considered for computing MOS values.

 

Test room setup

The tests were performed at two different rooms which were configured in order to achieve identical illumination conditions.

Displays of the same model (ASUS ML238 LED-LCD) were calibrated to the same settings and used in all test sessions.

The main setup parameters were as listed below:

  • Height of the picture shown in the screen:  14 cm
  • Viewing distance:  approx. 65 cm
  • Background room illumination: approx. 14 lux
  • Peak luminance of the LCD screen: 132 lux
  • Ratio of luminance of inactive screen to peak luminance: 0.023
  • Ratio of luminance of background behind the display to peak of luminance: 0.05

 

Video sequences

The original sequences were encoded using the H.264/AVC JM 14.2 reference software.

The resolution of all sequences is 720x480 and their duration is 10 seconds (length of 300 frames for 30 Hz sequences; 250 frames for 25 Hz sequences)

The main encoder parameters were as follows:

  • Main profile, level 2.0
  • GOP structure: 15 frames length, with two B frames inserted between I/P frames
    IBBPBBPBBPBBPBBIBBP...
  • One IDR frame every 1 second of video
  • Maximum of 5 reference frames used for motion estimation
  • Low complexity RD optimization mode
  • Original JM rate control algorithm
  • RTP output format - one video slice per packet
  • 45 macroblocks per slice (each slice is a complete row of macroblocks)

 

Transmission losses

The encoded sequences have been corrupted by simulating the transmission of the streams over an error-prone network.

The simulation was carried out using the software associated to Rec. ITU-T G.1050 - "Network model for evaluating multimedia transmission over Internet Protocol".

The generated loss patterns correspond to packet loss rates ranging from 0.17% up to 7.90%.

 

Video streams

All bitstreams are available in the following links. In order to avoid large file sizes, they have been grouped in the following files:

All video streams can be decoded using the batch file and the H.264 decoder* provided here (place the contents of the zip file in the same directory as the streams).

Filenames format used in the streams:

  • Encoded sequences: <video sequence>_<spatio-temporal_resolution>_<bitrate>
  • Corrupted sequences: <video sequence>_<spatio-temporal_resolution>_<bitrate>_corrupted_plr_<packet loss rate>

* Note: the decoder provided in the zip file is a modified JM H.264 decoder that allows the specification of the NALU type (Annex B / RTP format) in the command line. It is the same used in the Polimi-EPFL video quality assessment database.

 

MOS

Resulting MOS values and their standard deviations can be downloaded in excel or text file format:

  • MOS (excel file)
  • MOS (text file) - each line contains the data for a test condition, organized as follows: sequence name, MOS value and MOS standard deviation, separated by tabulations.