Format Detector Plugin for SageTV

[KryptoNyte]

Ah, I see.

[peternm22]

I have certain h264 files that my HD300 extender will take a few seconds to start playback after hitting "Play".

Would this plugin speed this up when using an extender? Or is it only for non-extender SageTV clients?

[peternm22]

I decided to give this a try as it's easy enough to revert back. I'm getting some errors in my log when I forced a reindex:

The first error seems like there is a problem because I am still using 32 bit SageTV, but the new DLL's seem to be 64 bit. This error popped up on the first file it tried to reprocess:

Wed 4/29 19:11:13.047 [main@1fc0f2f] Redetecting format for TV file because it changed:MediaFile[id=30874844 A[30874850,30874849,"DC's Legends of Tomorrow",0@0429.18:24,41,T V] mask=TV host=Ryzen encodedBy= format=MATROSKA 0:41:47 384 kbps [#0 Video[H.264 23.98 fps 1280x718 640:359 progressive]#1 Audio[AC3 48000 Hz 6 channels 384 kbps idx=1]{IMDBID=tt10654528, SeriesInfoID=2186142, MediaType=TV, ScrapedBy=Phoenix, ScrapedDate=1588212366441, MediaProviderDataID=295760, MediaTitle=DC's Legends of Tomorrow, MediaProviderID=tvdb}] F:\Sage\Imports\DCs.Legends.of.Tomorrow.S05E10.720p.HDTV.x264-AVS.mkv, Seg0[Wed 4/29 18:24:15.296-Wed 4/29 19:06:02.676]]
Wed 4/29 19:11:13.055 [main@1fc0f2f] Using the format detector plugin
Wed 4/29 19:11:13.056 [main@1fc0f2f] MediaFormatParserPlugin processing: F:\Sage\Imports\DCs.Legends.of.Tomorrow.S05E10.720p.HDTV.x264-AVS.mkv
Wed 4/29 19:11:13.058 [main@1fc0f2f] There was an unhandled exception processing the file: DCs.Legends.of.Tomorrow.S05E10.720p.HDTV.x264-AVS.mkv C:\Program Files (x86)\SageTV\SageTV\avutil-56.dll: Can't load AMD 64-bit .dll on a IA 32-bit platform
Wed 4/29 19:11:13.059 [main@1fc0f2f] MediaFormatParserPlugin processing complete: F:\Sage\Imports\DCs.Legends.of.Tomorrow.S05E10.720p.HDTV.x264-AVS.mkv
Wed 4/29 19:11:13.059 [main@1fc0f2f] Now using external format detector for: F:\Sage\Imports\DCs.Legends.of.Tomorrow.S05E10.720p.HDTV.x264-AVS.mkv
Wed 4/29 19:11:13.121 [main@1fc0f2f] File Format Parsed F:\Sage\Imports\DCs.Legends.of.Tomorrow.S05E10.720p.HDTV.x264-AVS.mkv=MATROSKA 0:41:47 384 kbps [#0 Video[H.264 23.98 fps 1280x718 640:359 progressive]#1 Audio[AC3 48000 Hz 6 channels 384 kbps idx=1]]

All of the subsequent files don't mention the 64 bit problem, but I suspect it's related:

Wed 4/29 19:11:14.734 [LibraryImporter@5eaf9] Reimporting metadata for mediafile because it changed:MediaFile[id=28486188 A[28486191,28486189,"Amaze-Dolby-thedigitaltheater",0@0715.18:18,1,V] mask=V host=Ryzen encodedBy= format=Quicktime 0:01:03 5037 kbps [#0 Video[H.264 24.0 fps 1920x1080 16:9 4395 kbps progressive]#1 Audio[AC3 48000 Hz 2 channels 640 kbps idx=1 und]{SeriesInfoID=2141055}] F:\Sage\Imports\Amaze-Dolby-thedigitaltheater.mp4, Seg0[Sat 7/15 18:18:09.898-Sat 7/15 18:19:13.448]]
Wed 4/29 19:11:14.743 [LibraryImporter@5eaf9] Using the format detector plugin
Wed 4/29 19:11:14.744 [LibraryImporter@5eaf9] MediaFormatParserPlugin processing: F:\Sage\Imports\Amaze-Dolby-thedigitaltheater.mp4
Wed 4/29 19:11:14.744 [LibraryImporter@5eaf9] There was an unhandled exception processing the file: Amaze-Dolby-thedigitaltheater.mp4 Could not initialize class jvl.FFmpeg.jni.AVFormatContext
Wed 4/29 19:11:14.744 [LibraryImporter@5eaf9] MediaFormatParserPlugin processing complete: F:\Sage\Imports\Amaze-Dolby-thedigitaltheater.mp4
Wed 4/29 19:11:14.745 [LibraryImporter@5eaf9] Now using external format detector for: F:\Sage\Imports\Amaze-Dolby-thedigitaltheater.mp4

All of the files continued to play fine though. So even though the plugin seemed to fail, it didn't seem to cause any noticeable problems in SageTV.

[wayner]

What is a pretty resolution?

[jvl711]

Originally Posted by peternm22

I have certain h264 files that my HD300 extender will take a few seconds to start playback after hitting "Play".

Would this plugin speed this up when using an extender? Or is it only for non-extender SageTV clients?

This is unlikely to have an effect on this. My guess would be that it is the time it takes for the HD300 to analyze and start playing. The only way this would have an impact is if for some weird reason sage was redoing the format detection on this file when it was attempting to start playback. You could check the logs and see if that is the case.

Josh

[jvl711]

Originally Posted by wayner

What is a pretty resolution?

What sage uses when it wants to display a resolution to the user in a user friendly manner.

For instance 1920x1080 will be returned as "1080p". 3840x2160 will be returned as 2160p.

There is a little more to it than that, but that is the basic idea.

Josh

[wayner]

Originally Posted by jvl711

What sage uses when it wants to display a resolution to the user in a user friendly manner.

For instance 1920x1080 will be returned as "1080p". 3840x2160 will be returned as 2160p.

There is a little more to it than that, but that is the basic idea.

Josh

Thanks Josh!

[jvl711]

Originally Posted by peternm22

I decided to give this a try as it's easy enough to revert back. I'm getting some errors in my log when I forced a reindex:

The first error seems like there is a problem because I am still using 32 bit SageTV, but the new DLL's seem to be 64 bit. This error popped up on the first file it tried to reprocess:



All of the subsequent files don't mention the 64 bit problem, but I suspect it's related:



All of the files continued to play fine though. So even though the plugin seemed to fail, it didn't seem to cause any noticeable problems in SageTV.

Hi peternm22,

I apologize. This did not even occur to me. I started this development for personal use. Than I decided to contribute the changes that I made to the SageTV project because I felt others could make use of it.

I will need to add this as an item to look at. I am honestly not sure how much effort this will require. I am not a C developer, and I am kinda hacking my way through the C JNI portion of the code. I am passing pointers back and forth between C and Java, and I am not sure if the code can easily deal with 32 and/or 64 bit pointer. I think it might work OK, but I will need to do some investigation.

I also need to figure out how to compile the C code for linux as well. I was hoping to find someone in the community that could provide guidance, or coding assistance.

Josh

[mechling-burgh]

If this can be told to transcode for the Hd300 could it be made to use the newer ffmpeg that you are using for the detection since the sage version is so old?

[wnjj]

Originally Posted by jvl711

Hi peternm22,

I apologize. This did not even occur to me. I started this development for personal use. Than I decided to contribute the changes that I made to the SageTV project because I felt others could make use of it.

I will need to add this as an item to look at. I am honestly not sure how much effort this will require. I am not a C developer, and I am kinda hacking my way through the C JNI portion of the code. I am passing pointers back and forth between C and Java, and I am not sure if the code can easily deal with 32 and/or 64 bit pointer. I think it might work OK, but I will need to do some investigation.

I also need to figure out how to compile the C code for linux as well. I was hoping to find someone in the community that could provide guidance, or coding assistance.

Josh

Since you are using jlong to hold the 64-bit pointers they should be fine for 32-bit as is.

[jvl711]

Originally Posted by wnjj

Since you are using jlong to hold the 64-bit pointers they should be fine for 32-bit as is.

And I am assuming since I am casting my pointers from jlong to intptr_t that C will create the right pointer type for the system I am compiling too. I believe intptr_t is system independent from what I was reading?

So in theory. Add the 32bit FFmpeg library to my project. Add another build profile for 32bit. Make sure I include 32bit version of all of the libraries. Configure a 32bit tool chain (compiler and build tools). Than I should be able to just compile a 32 bit version

Josh

[jvl711]

Originally Posted by mechling-burgh

If this can be told to transcode for the Hd300 could it be made to use the newer ffmpeg that you are using for the detection since the sage version is so old?

Not exactly. This plugin only does media file format detection. Allowing sage to know about newer formats like HEVC (h.265) will open some doors in the future for developer(s) to make decisions about how to handle H.265 or other newer content.

Today Sage sees h.265 content as unknown, and is not really sure what it is.

Now, the library that I created to wrap FFmpeg using JNI could be used for future projects like a new transcoder engine. I did think there was an outside chance that maybe I would take a stab at that some day. The code is not out there, so if someone else wanted to build off what I started that could happen too.

I think properly handling 4k/h.265 playback and transcoding is important to SageTV's future relevance.

Josh

[sacrament055]

Would it be beneficial to install this on a Windows Client or would you only want to do this on the server?

[wnjj]

Originally Posted by jvl711

And I am assuming since I am casting my pointers from jlong to intptr_t that C will create the right pointer type for the system I am compiling too. I believe intptr_t is system independent from what I was reading?

So in theory. Add the 32bit FFmpeg library to my project. Add another build profile for 32bit. Make sure I include 32bit version of all of the libraries. Configure a 32bit tool chain (compiler and build tools). Than I should be able to just compile a 32 bit version

Josh

I think you can even just cast the jlong to the native pointer type like is done here:
https://github.com/google/sagetv/blo...ontrol.cpp#L63

There are also examples where (intptr_t) is in between though so probably more clear that way. It may depend upon the compiler and whatever warnings it throws. https://github.com/google/sagetv/blo...posixipc.c#L31

I think you have everything else correct about including and building. Good luck.

[jvl711]

Originally Posted by wnjj

I think you can even just cast the jlong to the native pointer type like is done here:
https://github.com/google/sagetv/blo...ontrol.cpp#L63

There are also examples where (intptr_t) is in between though so probably more clear that way. It may depend upon the compiler and whatever warnings it throws. https://github.com/google/sagetv/blo...posixipc.c#L31

I think you have everything else correct about including and building. Good luck.

Initial attempt to build 32 bit version have gone very smoothly. I need to install a 32 bit version of the server and test. I will release a new version if it passes. Hopefully in a few days.

wnjj, do you know if it is possible to compile a linux library from a windows C build environment? I would like to be able to build all versions of the C library from one Netbeans project if possible.

Thanks,
Josh

[wnjj]

Originally Posted by jvl711

Initial attempt to build 32 bit version have gone very smoothly. I need to install a 32 bit version of the server and test. I will release a new version if it passes. Hopefully in a few days.

wnjj, do you know if it is possible to compile a linux library from a windows C build environment? I would like to be able to build all versions of the C library from one Netbeans project if possible.

Thanks,
Josh

It looks like Cygwin can do it. I’ve used Cygwin before but not to compile Linux binaries. http://metamod-p.sourceforge.net/cro...for.linux.html

Msys2/Mingw-w64 may support it also. That’s the environment we use to compile Windows libraries from Linux/gcc style source code and Makefiles (i.e. the reverse of what you want). https://github.com/google/sagetv/blo...NativeCode.txt

[jvl711]

Hi all,

I posted a new release to github. This is version 0.2-alpha. I added a win32 build to this release. Please let me know if it is working OK. I am going to try and see if I can figure out cross compiling to linux next.

Thanks,
Josh

[SteveW]

I apologize if I’m missing this in some other part of the thread, but once I have followed the instructions to install, is there something in Sage that would show that it is successfully active? I don’t see it listed as a plugin...

[jvl711]

Originally Posted by SteveW

I apologize if I’m missing this in some other part of the thread, but once I have followed the instructions to install, is there something in Sage that would show that it is successfully active? I don’t see it listed as a plugin...

Hi Steve,

if you import a new media file or reimport you should see something in the log to show that it used the plug to do the detection.

MediaFormatParserPlugin processing complete: \\egypt\tv\Gotham\Season 04\Gotham - S04E21 - A Dark Knight One Bad Day.mkv

Let me know if that helps.

Josh

[UgaData]

Optional: you can redetect the format of all of your media files by changing this setting in Sage.properties
force_full_content_reindex=true

I installed this plug-in this AM. I believe all is working but I did have an unexpected issue. This isn't an issue with the plug-in, most likely everything is working as it should.

As part of the install, I initially set the "Sage.properties force_full_content_reindex=true", this resulted in SageTV hanging on the splash screen at - User Manger Intializing -. I did wait several minutes but I know it doesn't normally take this long. Not to mention none of the clients could connect while Sage was in this state of detecting.

I used task manager to stop SageTV and the Service (neither would respond). I then changed the Sage.properties
force_full_content_reindex=true to false and restarted Sage (I actually rebooted the machine). I was able to start the Client on the server as well as other machines.

I will attempt to re-enable the true setting at some other point in time when Sage has plenty of time to re-detect media files.

I do have a few TB's of just recordings not to mention movies and as stated earlier, the plug-in is doing what it is suppose to. I had some recordings coming up and didn't have the patients to wait for Sage to do a full re-detect on all the files.