Selenium -> browser Mobile -> ??
Canvas; screen grab w/ control stream
Mozilla eideticker (monitors output of device)
Remote developer console on devices
What to do with permission dialogs / extra-DOM stuff
* qunit * tape
* each browser vendor has suite of browser tests
Orchestrated multiple browser w/ mult. participants very hard to test (WebRTC)
* Xiph: Testing for Opus * 160 core cluster w/ Jenkins * Each commit 1000's of hours of audio processed * Performance testing is harder * Multi-platform support (ARM, etc) * Lots of different chips * Consistent test results: hard * What test hooks can be added to codec implementations? * Support extensions for test hooks
Rdio: Capture HDMI output and compare to expected decoded frames
* not generalizable
Expose device hardware capabilities via API?
Performance tests / certification