Need help with Online Videos

[stuckless]

I'm in the process of adding Online Videos and PlayOn support to Phoenix. I have the VFS navigation working for both online videos and PlayOn (over UPnP) and now I'm ready to tack le the hard part which is playing the video in sagetv.

Previously, I had played with Online Videos in Phoenix, but I had issues in that when I would tell sagetv to Watch a file, while it was being downloaded, then sagetv would only ever Watch the portion that was downloaded, but not any data that was downloaded after the Watch() call. I'm was obviously missing something fundamental.

I have tried to read the logic in the default STV but I'm pretty lost in STV land. So, I was wondering if someone with some STV knowledge could look at the Online Videos portion of SageTV and try to explain to me how I'd go about watching a file while it's being downloaded.

Currently my logic is this...
Start the Download
Wait for a certain amount of buffering
Watch the file
Monitor the playback so see if we are watching too fast (ie, we need to pause and buffer more)

So, if you want to see online videos and/or PlayOn (or other UPnP MediaServer's) support natively in Phoenix... I need your help

[tmiranda]

Sean,

Did you write your own download code or are you using the downloader available in the Sage API? I'm pretty sure that in order to play downloading files you need to use the Sage download code.

Going form memory, there is an API that will tell you if I file can be progressively downloaded. It's something like IsProgressive(). Some video simply can't be played until it's completely downloaded. Can the files be watched while downloaded in the default STV?

I'll try to look into this in more detail tonight when I get home.

Tom

[stuckless]

Originally Posted by tmiranda

Sean,

Did you write your own download code or are you using the downloader available in the Sage API? I'm pretty sure that in order to play downloading files you need to use the Sage download code.

Going form memory, there is an API that will tell you if I file can be progressively downloaded. It's something like IsProgressive(). Some video simply can't be played until it's completely downloaded. Can the files be watched while downloaded in the default STV?

I'll try to look into this in more detail tonight when I get home.

Tom

Thanks Tom... I am using the SageTV download apis, but I wasn't checking the Progressive api, so I'll add that in. That being said, I was testing this with youtube videos (which I think are probably progressive). In my testing I'd use the same video and try it in sagetv and it would play right away (after some buffering)... and in my api it would play right away (after some buffering) but then it would only the buffered content.

[jusjoken]

I have never reviewed this part of the STV before and am no expert... but at a glance it looks like this is what it is doing...

• Cleanup of any old downloads - CancelFileDownload(),
• CloseAndWaitUntilClosed(), Fork to delete the temp download file
• Then start the download - DownloadResult = StartFileDownload(VideoURL, null, targetFile)
• Then check the Status - Status = GetFileDownloadStatus()
  • Check for part of the file downloaded - (GetFileDownloadStreamTime() > 3000) && (ForceFullDownload != true)
  • If the above is True - call Watch(targetFile) - then wait for a True result with a Wait(500) in between
  • Sets CurrMF = GetCurrentMediaFile()
  • then it seems to check the duration - NewDur = sage_media_rss_RSSEnclosure_getLength(CurrEnclosure)
  • then it increments the duration - NewDur = NewDur * 1000 (it resets the Duration at - NewDur > 8*60*60 * 1000
  • then it adds the show to the CurrMF
    • NewShow = AddShow(TitleText, false, TitleText, ShowDesc, NewDur, null, null, null, null, null, null, null, null, null, "TMP" + Time(), null, 0)
    • SetMediaFileShow(CurrMF, NewShow)

I can dig in a little more if you need more... hope this helps point you in the right direction!

k

[stuckless]

That is going to help quite a bit. I can already see how my code is behaving differently. Thanks.

[tmiranda]

Originally Posted by jusjoken

I have never reviewed this part of the STV before and am no expert... but at a glance it looks like this is what it is doing...

• Cleanup of any old downloads - CancelFileDownload(),
• CloseAndWaitUntilClosed(), Fork to delete the temp download file
• Then start the download - DownloadResult = StartFileDownload(VideoURL, null, targetFile)
• Then check the Status - Status = GetFileDownloadStatus()
  • Check for part of the file downloaded - (GetFileDownloadStreamTime() > 3000) && (ForceFullDownload != true)
  • If the above is True - call Watch(targetFile) - then wait for a True result with a Wait(500) in between
  • Sets CurrMF = GetCurrentMediaFile()
  • then it seems to check the duration - NewDur = sage_media_rss_RSSEnclosure_getLength(CurrEnclosure)
  • then it increments the duration - NewDur = NewDur * 1000 (it resets the Duration at - NewDur > 8*60*60 * 1000
  • then it adds the show to the CurrMF
    • NewShow = AddShow(TitleText, false, TitleText, ShowDesc, NewDur, null, null, null, null, null, null, null, null, null, "TMP" + Time(), null, 0)
    • SetMediaFileShow(CurrMF, NewShow)

I can dig in a little more if you need more... hope this helps point you in the right direction!

k

Beat me to it I was just getting ready to type the same information.

I'll add:
- There is a lot of logic before this point to figure out what type of online content you are trying to watch and then once that is determined finding a valid URL for the video. Different types of online content (YouTube, Channels.com, Google Video...) all have their own idiosyncrasies.

- The way the STV determines if the file download is complete is highly Studio dependent. It uses a "hook" that fires when the MediaPlayer determines the file is fully loaded. I'm not 100% certain of what's going on here but you'll have to make sure the MediaPlayer thinks the file is fully loaded before you stop the download.

[stuckless]

Not much luck tonight... I still can only play the portion of the file that was downloaded when Watch() was called

Here's my log

2012-02-21 23:20:19,722 [Fork-ZOMHW-9065855] INFO  phoenix.impl.MediaBrowserAPI - PlayMediaFile called for: DecoratedItem: [sagex.phoenix.upnp.UPnPMediaFile [Id: youtube-2f53b49c-6b1f-481f-a309-d0b76b09b8a0, Title: Mass 
Effect 3: Take Earth Back Trailer]]; Context: 00e04c77e4ee
2012-02-21 23:20:19,722 [Fork-ZOMHW-9065855] INFO  phoenix.impl.MediaBrowserAPI - PlayMediaFile called for: sagex.phoenix.upnp.UPnPMediaFile [Id: youtube-2f53b49c-6b1f-481f-a309-d0b76b09b8a0, Title: Mass Effect 3: Take E
arth Back Trailer]; Context: 00e04c77e4ee
2012-02-21 23:20:19,725 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Cached file: ./userdata/Phoenix/cache/onlinevideos/b/e/1/be157e555ee433974c352519be81ff11.mpg for url http://192.168.1.112:63478/youtube-2f53b49c-6b1f-481f-a309-d0b76b09b8a0/1-youtube-2f53b49c-6b1f-481f-a309-d0b76b09b8a0.mpg
2012-02-21 23:20:19,735 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Starting filedownload for http://192.168.1.112:63478/youtube-2f53b49c-6b1f-481f-a309-d0b76b09b8a0/1-youtube-2f53b49c-6b1f-481f-a309-d0b76b09b8a0.mpg to ./userdata/Phoenix/cache/onlinevideos/b/e/1/be157e555ee433974c352519be81ff11.mpg
2012-02-21 23:20:19,797 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Monitoring File: ./userdata/Phoenix/cache/onlinevideos/b/e/1/be157e555ee433974c352519be81ff11.mpg; cur size: 0
2012-02-21 23:20:20,802 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Monitor completed: Is Aborted: false
2012-02-21 23:20:20,847 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Playing Downloaded SageTV MediaFile: sagex.phoenix.upnp.UPnPMediaFile [Id: youtube-2f53b49c-6b1f-481f-a309-d0b76b09b8a0, Title: Mass Effect 3: Take Earth Back Trailer] on UI Context: 00e04c77e4ee; Title: Mass Effect 3: Take Earth Back Trailer; Progressive?: true
2012-02-21 23:20:20,847 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Stream Time: 23122
2012-02-21 23:20:20,853 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Is Watching Mass Effect 3: Take Earth Back Trailer? sage.e$f@1dbece
2012-02-21 23:20:21,354 [Fork-ZOMHW-9065855] INFO  sagex.phoenix.stv.OnlineVideoPlayer - Monitoring Playback
2012-02-21 23:20:21,355 [Thread-36] INFO  sagex.phoenix.stv.PlaybackMonitor - Playback Monitor Started
2012-02-21 23:20:21,415 [Thread-36] INFO  sagex.phoenix.stv.PlaybackMonitor - AddShow() called with duration 123000 for MediaFile[id=0 A[2451619,2451618,"Mass Effect 3: Take Earth Back Trailer",0@0221.23:19,0] mask=V host=seans-desktop encodedBy= format=MPEG2-PS 0:00:22 524 kbps [#0 Video[MPEG2-Video 29.97003 fps 720x404 180:101 progressive id=e0]#1 Audio[MP2 44100 Hz 2 channels 192 kbps MAIN idx=1 id=c0]] /home/sls/BETA/server/./userdata/Phoenix/cache/onlinevideos/b/e/1/be157e555ee433974c352519be81ff11.mpg, Seg0[Tue 2/21 23:19:58.778-Tue 2/21 23:20:21.000]]
2012-02-21 23:20:21,916 [Thread-36] INFO  sagex.phoenix.stv.PlaybackMonitor - MediaFile: duration: 22222; endTime: 22222; mf: MediaFile[id=0 A[2451619,2451618,"Mass Effect 3: Take Earth Back Trailer",0@0221.23:19,0] mask=V host=seans-desktop encodedBy= format=MPEG2-PS 0:00:22 524 kbps [#0 Video[MPEG2-Video 29.97003 fps 720x404 180:101 progressive id=e0]#1 Audio[MP2 44100 Hz 2 channels 192 kbps MAIN idx=1 id=c0]] /home/sls/BETA/server/./userdata/Phoenix/cache/onlinevideos/b/e/1/be157e555ee433974c352519be81ff11.mpg, Seg0[Tue 2/21 23:19:58.778-Tue 2/21 23:20:21.000]]

I've bolded a few things above... Even though I am setting the Show with a new duration, the media file just appears to ignore it. The mediafile id is 0, which I find odd, but maybe that is normal as well??

[stuckless]

For the curious... here's the code that is doing the download/watch

		if (Global.StartFileDownload(ctx, url, null,destFile)) {
			videoDownloading=true;
			log.info("File is downloading...");
				WaitFor wait = new WaitFor() {
					@Override
					public boolean isDoneWaiting() {
						log.info("Stream Time: " + Global.GetFileDownloadStreamTime(ctx));
						return Global.GetFileDownloadStreamTime(ctx) > 3000;
					}
				};
				wait.waitFor(30000, 500);
				
				Object watching = MediaPlayerAPI.Watch(ctx, destFile);
				log.info("Is Watching " + file.getTitle() + "? " + watching);

				WaitFor curFileWait = new WaitFor() {
					@Override
					public boolean isDoneWaiting() {
						log.info("Waiting for file to watch...");
						return MediaPlayerAPI.GetCurrentMediaFile(ctx) != null;
					}
				};
				curFileWait.waitFor(30000, 500);

				log.info("Watching file... Adding Show...");
				Object mf = MediaPlayerAPI.GetCurrentMediaFile(ctx);
				long duration = file.getMetadata().getDuration();
				// <Action Name="NewShow = AddShow(TitleText, false, TitleText, ShowDesc, NewDur, null, null, null, null, null, null, null, null, null, &quot;TMP&quot; + Time(), null, 0)" Sym="BASE-82840">
				Object show = ShowAPI.AddShow(ctx, file.getTitle(), false, file.getTitle(), file.getMetadata().getDescription(), duration, null, null, null, null, null, null, null, null, null, "TMP" + System.currentTimeMillis(), null, 0);
				MediaFileAPI.SetMediaFileShow(ctx, mf, show);
				log.info("AddShow() called with duration " + duration + " for " + mf);
		} else {
			log.info("Failed to download file...");
			videoDownloading = false;
		}

So far, I cannot get the playback to go beyond the buffered downloaded content

[tmiranda]

Sean,

It's a long shot: I see you are using GetFileDownloadStreamTime(), I think the STV uses GetFileDownloadStatus(). Maybe GetFileDownloadStatus() is doing something behind the scenes? I'll look more at it tonight.

Tom

[stuckless]

Originally Posted by tmiranda

Sean,

It's a long shot: I see you are using GetFileDownloadStreamTime(), I think the STV uses GetFileDownloadStatus(). Maybe GetFileDownloadStatus() is doing something behind the scenes? I'll look more at it tonight.

Tom

I reworked the code this morning to be what I posted here... I was using GetFileDownloadStatus() as well in the old code... I'll add in some calls to GetFileDownloadStatus as well to make sure that the download is running (although I know it is). There is another part of the code, that I temporarily removed until I getthis working, which is the PlaybackMonitor. It uses the GetFileDownloadStatus to check if the file is still downloading and if so, are we downloading slower than we are watching it (and buffer if we need to). For now, I've removed all that code, since until I get the playback working past the downloaded portion, then there is no point in monitoring

[stuckless]

I figured out the issue... and it's a weird one

I noticed that if passed destFile directly to the Watch() api then i'd get a black screen and no video playback. But if I passed destFile.getAbsolutePath() then I'd get video playback, but only the buffered content. Turns out that destFile is really a relative path file to the sagetv root, which is a valid file, but the Watch() api didn't like it (not sure why... I've sent logs to Jeff).

In the end, when I get the destFile I immediately call
destFile=destFile.getCanonicalFile()
to turn it into a file with a complete path (ie not relative) and then I use that file for the Download() and Watch() apis and everything works.

Phoenix will have native online video and upnp playback support in the next release (I don't know what that will be since Brian/Mike will need to do some UI work, and both are quite busy)

Thanks for trying help me figure out... I appreciate the help.

[OneOfMany]

Although I am just a Phoenix user, I thank you again for adding to our family Sage experience.

Thanks,
grant

[tmiranda]

Thanks for explaining the mystery.

[stuckless]

Originally Posted by tmiranda

Thanks for explaining the mystery.

After talking with Jeff he explained that internally the downloaded file is used as a key in a hashmap to track the downloading status, so this makes sense as to why it wasn't working.

Java's File implementation has a funny .equals() implementation. Basically two File() instances that reference the same file but with a different path are not equal() even though they are the same file.

File file1 = new File("Test.avi");
File file2 = new File("./Test.avi");

file1.equals(file2) returns false