==Operations -Meetings== https://public.etherpad-mozilla.org/p/FOMS-2017-Operations

Testing of Content -- focus

Ingest and ensuring quality. Experimental tools for quality

Audio Testing tools ?

Test product that have an audio output

How do you quantify audio quality ?

Frequency response, does this pass all frequency.

Can we adopt approaches similar to VMAF ? ( https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652 )

Audio is getting more complicated, we need better tools ? -- Doubly github

PEAK; ultimately fall back to people.

EBU has a good test set.

Detecting artifacts; Releasing some tools.

Crowdsourced subjective testing ?

What metrics could we come up that tells us something ?

Reference free Quality key area of focus, is possible ?

VidCheck?, ECard?, historically used in production workflows, detecting artifacts, green flashes etc.

Which sort of use cases for reference free comparisons.

Standardization of tools ?

What are the use cases?

Content partners delivering content

Decode level checks

Checking for up-sampling.

Metadata standardization on what encode settings were used.

Coming up for a list of metrics that are useful for video quality ?

Could there be programmatic contracts for video delivery.

Trying to sync metadata between sources.

Standard ways to represent encode level information in syndication metadata formats ?

Quality metadata passed down to clients; ABR against scene complexity.

Standardized representation of quality metadata used for delivery switching.

main problem, dash and HLS define stream level bitrate; adding a quality metric to streams.

VMAF to drive encoding; going beyond "constant quality"

Quality "across the entire stack", how CDNs? are performing? which parts of my stack is effecting which user ?

HTTP/2

P2P? distribution and low latency; adaptive delivery

Incentivizing p2p behavior

P2P? network for video delivery, ways to simulate large networks way to test them ?

Integration of multi-source delivery methods.

Low latency, measuring latency end to end. Dealing with protocols

Low latency HLS. in P2P? environment.

AB testing Peer to Peer on scale.

ABR for Peer to Peer ?

Mux multi-CDN approaches ?