FOMS2010-NYV

Blackboard notes from group discussion:

summary notes?

design notes?

Minutes from group 1 discussion: Zach

Pablo Jeroen Sylvia Chris Andrew

Phillip

 * Chris Blizzard said we need to sync multiple audio and video elements
 * Switch between streams using gapless playback
 * Buffering control while switching - don't want to ramp up when bandwidth is dropping

Jeroen

 * Not an issue: you have 3 chunks - 1 playing, 1 ready, one loading
 * Chunks are natural QOS checkpoints

Philip

 * Chunking is irrelevant because you just make a single file and do byte range requests
 * Need frame / audio accurate switching

Jeroen

 * Live streams
 * Cache support
 * JS manifest

Philip

 * Multiple sources

Jeroen

 * Playback queue - don't trash

Sylvia

 * Everything is a live stream

Pablo

 * How handle duration?

Sylvia

 * How know duration?

Jeroen

 * In the manifest

Zach

 * Just have a queue and manipulate it

Sylvia

 * Web developers don't want to think about when to switch

Jeroen

 * Just feed it a manifest and influence that

Zach

 * You want a consistent experience across platforms, if you let the browser decide when to switch, it becomes secret sauce

Sylvia

 * Manifest with duration
 * Allow JS function to allow JS developer to initiate switch

Jeroen

 * Manifest + statistics and you can influence the resource via metrics

Sylvia

 * Chris Double + Hixie are pro-metrics

Jeroen

 * Bandwidth
 * Frame drops (absoute or rate)
 * Size of window
 * Size of resource in the manifest

Sylvia

 * Examine Chris Double's proposal

Jeroen

 * Switching based on buffer is too late - bandwidth is a better indicator

Philip

 * Is seconds enough? Seconds of video loaded per second?

Jeroen

 * new Video(blah.ogv) and get load events?

Zach

 * Is there a problem with detecting underrun based on seconds?

Philip

 * Problem with bytes is that it may not be continuous

Jeroen

 * First thing I'm going to do go back to bitrate to figure out which one to go to

Philip

 * If less than 1 s/s, you'll underrun

Jeroen

 * How do I know which one to go to?

Philip

 * Won't you just step down one level

Pablo

 * May jump more

Sylvia

 * Other alternative

Philip

 * Architectural problem with first proposal
 * Right now, one gstreamer resource per video
 * Breaks a lot of interfaces, ie, duration

Sylvia & Jeroen

 * Manifest file is the only way to go

Sylvia

 * It all represents a single resource

Philip

 * 3 different resources, just to a byte-offset request

Sylvia

 * Resources have points with keyframes, with audio aligned
 * Get an index

Jeroen

 * Manifest is great for live stream - just reload manifest
 * Have duration in the manifest

Sylvia

 * Have how much played back in the manifest as well

Jeroen

 * Have when started
 * Have where end
 * Have playable chunks list

Philip & Chris

 * Updates duration is video is longer than reported

Philip

 * Things that are live, chunking stuff - all over the place

Jeroen

 * Manifests are a use case

Philip

 * Put the resources on the server, have a way to seamless switch between them
 * Need a way to play 2 different things in sync for aid for blind, etc
 * Need a way to express that you have several resources that are the same

Sylvia

 * Sign language is a different use case - different resources

Philip

 * Same technical issue of getting them in sync

Jeroen

 * Need index information at the beginning

Chris

 * What problem are we trying to solve?

Philip

 * Use chunks or byte range requests

Chris

 * Chunks are for ease of deployment
 * Byte range requests work now

Sylvia

 * These are the same resource from a developers point of view
 * Know that this link, this url are all the same
 * Have to set up 3 decoding piples

Chris:

 * Need to have alignment, can't just use handbrake

Philip

 * Can't you just seek to the same frame?

Chris

 * Much easier to do file based
 * Browser have to know how to prebuffer and get it ready
 * source.next(url, time)

Jeroen

 * In underrun, you have a problem. You have to start downloading a new resource

Philip

 * immediately stop hq download, start downloading new resource, immediately switch

Jeroen

 * FS you switch and lose tons of buffer

Philip

 * So you want to have a buffer control?

Jeroen

 * Yes - start here and build up 20 seconds

Philip

 * Media fragment
 * Mutable list of ranges
 * More option for preload

Jeroen

 * Large companies won't switch because 

Chris

 * Do it on the server
 * It's a pain in the butt
 * TED is an example - don't always finish it
 * No way to rate limit on 

Philip

 * Fast stop fast stop
 * Hold TCP connection

Chris

 * Destroy TCP window

Jeroen

 * Did the chunked guys try this

Chris

 * Don't get near moov
 * Want TCP consistency
 * Best to read at a consistent rate
 * You've got to widen and shrink
 * Want to say 60 seconds of buffer

Philip

 * Just keep reading one minute ahead consistently

Chris

 * Good feedback loop on time
 * If I'm behind read, otherwise hold

Jeroen

 * Need chunks of seek points
 * That chunk of data can be huge for long form content

Chris

 * Play as soon as you have play data

Sylvia

 * Index is at the beginning

Jeroen

 * Big with MP4?
 * The issue is startup time

Sylvia

 * List of JS files is a manifest

Philip

 * Have the tools for JS
 * Build a manifest
 * Switch in sync, buffer length controls

Sylvia

 * Even with byte range, still need index

Philip

 * seek 20 points and find it
 * Index has key frames

Philip

 * Gapless switching needs to be done anyway. Are we going to implement those?

Chris

 * Make it easy enough for people to just point it at a video file
 * Powerful enough to be able to build own heuristics and expose controls via JS

Use cases

 * Live DVR streaming
  * Have some meta-data
  * When start, when end, where I am, live head, live tail (DVR window)
 * Long-form content streaming (> 60 mins)
  * Slow startup
  * Bandwidth conditions change
  * Bandwidth waste
 * Ad insertion

Decisions

 * Do we chunk? Do we range request seek?
 * Build it into the browsers or completely expose it to JS? (JS frameworks build it on top of browser changes)
 * Build logic into browser or just do JS?

Chris

 * If you jump files not chunks, is there anything special that needs to happen to the file?
 * Make sure it's fast - index size for mp4, mkv, ogg - takes a long time to figure out where index / buffer is