Integration Testing and Comparing Videos

Commits:

• Video comparison module added: 68b9f299377af86acdf00b2f2ef38bfea53127bb
• Added a file sink element and frame comparison: aa91fe142b5970997db86ee542524b7414da0b7b
• video comparison on key frames: eddb97074cf113ae040e6f10a5bdec269a416f6e
• Fixed test.py: a9c7e39fbcb01e95ce68fcc68b1465526fa21a3e
• integration tests - set_compose_mode - passing: f019ff38e9e3b752e0ed99e774db21277f32a518
• integration tests - set_compose_mode - failing: 7bdeefc5b2fffb98be9331d9660ab9d7545d31c8
• Reduced intensity of tests and removed sleeps: b935c2bd55e5c15a80b6fa1e5a9bf9a41dbed49a
• Reduced intensity of tests and removed sleeps: a588256f5cb2ab385bef27bea570fdbfe0232d95
• Makefile: 8ad4ffbad029db57d8157d0cfa2ec87bd88237bb

Integration Tests:

I added an extra module which tests the set_compose_mode. This is the first one in the series of tests in which the output video has to be compared to ensure that everything goes as it is expected. i.e. If a PIP mode change has to occur, the test should ensure that it occurs.

Comparing the Output Video

For each test I do the following:

• I compare key frames of a video to pre-stored frames. eg. If I am testing for changing PIP mode, I will have a two frames - one before the change and one after it. Now if the code which generates these frames from the video is run over the test video, it should generate same frames.
• For generating key frames, I use ffmpeg. Using ffmpeg's inbuilt key frame selector produces black frames in between which makes comparison difficult. So I conduct the test for 6 seconds. First 3 seconds original video is present. At 3 second exact time from start, the change is made. So the next 3 seconds have the modified output refecting the change. So I take frames at 1.5 and 4.5 second marks. Doing this ensures that I get only those which are indeed needed.
• After I have both the frames, I use scipy to calculate the zero norm of the two images. Zero norm is difference of the number of zeroes in the images. If the images are identical, the difference of will be exactly 0. Zero norm can take values between 0.0 and 1.0, with 1.0 being the case where the images are not at all similar.

Things to do-

• Integrate this video comparison into the integration tests
• Complete tests for all other methods.
• Add running tests through Travis CI
• Coverage with coveralls
• Documentation with readthedocs