SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV Github Development

Notices

SageTV Github Development Discussion related to SageTV Open Source Development. Use this forum for development topics about the Open Source versions of SageTV, hosted on Github.

Reply
 
Thread Tools Search this Thread Display Modes
  #41  
Old 05-12-2020, 01:34 AM
JREkiwi's Avatar
JREkiwi JREkiwi is offline
Sage Icon
 
Join Date: Jan 2005
Location: Auckland, New Zealand
Posts: 2,089
Quote:
Originally Posted by UgaData View Post
As part of the install, I initially set the "Sage.properties force_full_content_reindex=true", this resulted in SageTV hanging on the splash screen at - User Manger Intializing -. I did wait several minutes but I know it doesn't normally take this long. Not to mention none of the clients could connect while Sage was in this state of detecting.
That is normal. It does take some time before the GUI comes up and you can't connect clients, but I've never understood the logic of when it lets you connect. It's certainly before all the files have finished re-indexing.

John

Last edited by JREkiwi; 05-12-2020 at 11:17 PM. Reason: can't connect clients
Reply With Quote
  #42  
Old 05-12-2020, 04:22 PM
trk2 trk2 is offline
Sage Aficionado
 
Join Date: Jan 2006
Location: Maine
Posts: 388
Getting the following exception on a 32bit sage system. The exception seems to occur on all media formats.

Quote:
Tue 5/12 10:15:34.251 [LibraryImporter@b852ae] Reimporting metadata for mediafile because it changed:MediaFile[id=2579157 A[7567490,2579158,"Dirt",0@0123.00:39,3,M] mask=M host=LATK-SERVER encodedBy= format=FLAC 0:03:09 1038 kbps [#0 Audio[FLAC 44100 Hz 2 channels idx=0]] D:\Music\Alice in Chains\Dirt\02-Dam That River-Alice in Chains-Dirt.flac, Seg0[Sat 1/23 0:39:39.291-Sat 1/23 0:42:48.651]]
Tue 5/12 10:15:34.278 [LibraryImporter@b852ae] Using the format detector plugin
Tue 5/12 10:15:34.278 [LibraryImporter@b852ae] MediaFormatParserPlugin processing: D:\Music\Alice in Chains\Dirt\02-Dam That River-Alice in Chains-Dirt.flac
Tue 5/12 10:15:34.278 [LibraryImporter@b852ae] There was an unhandled exception processing the file: 02-Dam That River-Alice in Chains-Dirt.flac jvl.FFmpeg.jni.AVFormatContext.allocateContext()J
Tue 5/12 10:15:34.279 [LibraryImporter@b852ae] MediaFormatParserPlugin processing complete: D:\Music\Alice in Chains\Dirt\02-Dam That River-Alice in Chains-Dirt.flac
Tue 5/12 10:15:34.279 [LibraryImporter@b852ae] Now using external format detector for: D:\Music\Alice in Chains\Dirt\02-Dam That River-Alice in Chains-Dirt.flac
Tue 5/12 10:15:34.306 [LibraryImporter@b852ae] File Format Parsed D:\Music\Alice in Chains\Dirt\02-Dam That River-Alice in Chains-Dirt.flac=FLAC 0:03:09 1038 kbps [#0 Audio[FLAC 44100 Hz 2 channels idx=0]{Album=Dirt, Year=1992, ORGANIZATION=Columbia, Genre=Rock, Artist=Alice in Chains, AlbumArtist=Alice in Chains, Title=Dam That River, Composer=Jerry Cantrell, Track=2}]
Reply With Quote
  #43  
Old 05-14-2020, 09:16 AM
jvl711's Avatar
jvl711 jvl711 is offline
Sage Expert
 
Join Date: Jan 2004
Posts: 508
Quote:
Originally Posted by trk2 View Post
Getting the following exception on a 32bit sage system. The exception seems to occur on all media formats.
Just to confirm you are using the 32bit build of the library? And do you see any errors earlier in the log when it is constructing the media format plugin?

I may need to spin up a 32 bit version of Sage to test.

Josh
Reply With Quote
  #44  
Old 05-17-2020, 09:33 AM
trk2 trk2 is offline
Sage Aficionado
 
Join Date: Jan 2006
Location: Maine
Posts: 388
I restored my backup and downloaded/installed all the files fresh to make sure I didn't make a 32/64 bit version mistake. I'm still getting the same exception.
Attached Files
File Type: zip sagetv_0.zip (709.7 KB, 11 views)
Reply With Quote
  #45  
Old 05-18-2020, 11:22 AM
jvl711's Avatar
jvl711 jvl711 is offline
Sage Expert
 
Join Date: Jan 2004
Posts: 508
Quote:
Originally Posted by trk2 View Post
I restored my backup and downloaded/installed all the files fresh to make sure I didn't make a 32/64 bit version mistake. I'm still getting the same exception.
Thank you for the additional info. I will debug this week. I may need to install a 32 bit SageTV server to fully debug. I will let you know when I find the issue.

Josh
Reply With Quote
  #46  
Old 05-18-2020, 11:47 AM
wnjj wnjj is offline
Sage Icon
 
Join Date: Jan 2009
Posts: 1,268
Quote:
Originally Posted by jvl711 View Post
Thank you for the additional info. I will debug this week. I may need to install a 32 bit SageTV server to fully debug. I will let you know when I find the issue.

Josh
Hi Josh,

9.2.2 is released now so you can modify your instructions to allow for that OR your sage.jar.
Reply With Quote
  #47  
Old 05-18-2020, 11:48 AM
wnjj wnjj is offline
Sage Icon
 
Join Date: Jan 2009
Posts: 1,268
Quote:
Originally Posted by jvl711 View Post
Thank you for the additional info. I will debug this week. I may need to install a 32 bit SageTV server to fully debug. I will let you know when I find the issue.

Josh
Also, I looked at that exception and couldn't make any sense of it. All you are doing is calling the alloc function within the FFMPEG libs. I didn't see anything wrong in there.
Reply With Quote
  #48  
Old 05-18-2020, 11:50 AM
trk2 trk2 is offline
Sage Aficionado
 
Join Date: Jan 2006
Location: Maine
Posts: 388
I'll be happy to test if you need it. I use Sage for recording sports and as a media server, and there is no sports to record at the moment.

Also, is it expected behavior that Sage use its internal format detector for any recorded files?
Reply With Quote
  #49  
Old 05-18-2020, 02:59 PM
jvl711's Avatar
jvl711 jvl711 is offline
Sage Expert
 
Join Date: Jan 2004
Posts: 508
Quote:
Originally Posted by trk2 View Post
I'll be happy to test if you need it. I use Sage for recording sports and as a media server, and there is no sports to record at the moment.

Also, is it expected behavior that Sage use its internal format detector for any recorded files?
Hi,

Yes it is expected behavior that the internal sage parser does PS/TS streams. Not sure if that is all PS/TS or just mpeg2. There is some custom SageTV code that does that, and seems to do it well. This is should handle anything else, in theory.

I think I found the issue. Had to add a compile switch, which I did not need for the 64bit version. I am going to see if I can PM you the dll for testing. Thanks for helping! If it works well I will build a new release.

Josh
Reply With Quote
  #50  
Old 05-18-2020, 04:09 PM
wnjj wnjj is offline
Sage Icon
 
Join Date: Jan 2009
Posts: 1,268
Quote:
Originally Posted by jvl711 View Post
Hi,

Yes it is expected behavior that the internal sage parser does PS/TS streams. Not sure if that is all PS/TS or just mpeg2. There is some custom SageTV code that does that, and seems to do it well. This is should handle anything else, in theory.

I think I found the issue. Had to add a compile switch, which I did not need for the 64bit version. I am going to see if I can PM you the dll for testing. Thanks for helping! If it works well I will build a new release.

Josh
Can you share what compiler switch that was? I ran into some stuff when porting the Windows native code from 32 to 64 bit but can always use more info for future issues. Was it something like —kill-at involving DLL name decoration?

Last edited by wnjj; 05-18-2020 at 04:14 PM.
Reply With Quote
  #51  
Old 05-18-2020, 06:39 PM
trk2 trk2 is offline
Sage Aficionado
 
Join Date: Jan 2006
Location: Maine
Posts: 388
So the updated dll fixed the exception.

I did run into a couple other unusual issues. Upon restarting the service there were errors in the object database and then it hung following the scan of all recorded media but before re-scanning all imported media.

Quote:
Mon 5/18 19:25:57.467 [SageTV@1d71a44] CARNY building airing maps...
Mon 5/18 19:30:29.403 [ThreadMonitor@1239a0] Top 10 Cumulative CPU(sec) main:3 LuceneShowTransactionTask:1 LucenePersonTransactionTask:1
Mon 5/18 19:30:29.403 [ThreadMonitor@1239a0] Top 10 %CPU (last 300s) main:1%
Mon 5/18 19:35:29.407 [ThreadMonitor@1239a0] Top 10 Cumulative CPU(sec) main:3 LuceneShowTransactionTask:1 LucenePersonTransactionTask:1
Mon 5/18 19:35:29.407 [ThreadMonitor@1239a0] Top 10 %CPU (last 300s)
Mon 5/18 19:40:29.413 [ThreadMonitor@1239a0] Top 10 Cumulative CPU(sec) main:3 LuceneShowTransactionTask:1 LucenePersonTransactionTask:1
Mon 5/18 19:40:29.414 [ThreadMonitor@1239a0] Top 10 %CPU (last 300s)
Mon 5/18 19:45:29.420 [ThreadMonitor@1239a0] Top 10 Cumulative CPU(sec) main:3 LuceneShowTransactionTask:1 LucenePersonTransactionTask:1
Mon 5/18 19:45:29.420 [ThreadMonitor@1239a0] Top 10 %CPU (last 300s)
Mon 5/18 19:50:29.426 [ThreadMonitor@1239a0] Top 10 Cumulative CPU(sec) main:3 LuceneShowTransactionTask:1 LucenePersonTransactionTask:1
Mon 5/18 19:50:29.427 [ThreadMonitor@1239a0] Top 10 %CPU (last 300s)
Mon 5/18 19:54:07.193 [main@dde6e5] Sage.exit() called.
It exited gracefully and everything went fine on the second start of the service. A few flac files were detected as movies however. Here is an example of one:

Quote:
Mon 5/18 19:54:25.794 [LibraryImporter@10f9e3] Reimporting metadata for mediafile because it changed:MediaFile[id=2579366 A[7359617,2579368,"Megalithic Symphony (Advance)",0@1029.20:20,0,M] mask=M host=LATK-SERVER encodedBy= format=FLAC 0:00:57 895 kbps [#0 Audio[FLAC 44100 Hz 2 channels idx=0]] D:\Music\Awolnation\01 Awolnation - Megalithic Symphony.flac, Seg0[Mon 10/29 20:20:42.749-Mon 10/29 20:21:40.439]]
Mon 5/18 19:54:25.824 [LibraryImporter@10f9e3] Using the format detector plugin
Mon 5/18 19:54:25.824 [LibraryImporter@10f9e3] MediaFormatParserPlugin processing: D:\Music\Awolnation\01 Awolnation - Megalithic Symphony.flac
Mon 5/18 19:54:25.825 [LibraryImporter@10f9e3] Find stream info called
0
I noticed that only the flac files that had "Find stream info called 0" in the log were detected as movies. I have attached the log files. Sagetv_1 is the first startup that I killed, sagetv_0 is where the actual scans using the format detector plugin occured.
Attached Files
File Type: zip Sage Logs.zip (592.7 KB, 8 views)

Last edited by trk2; 05-18-2020 at 06:49 PM.
Reply With Quote
  #52  
Old 05-19-2020, 07:15 AM
jvl711's Avatar
jvl711 jvl711 is offline
Sage Expert
 
Join Date: Jan 2004
Posts: 508
Quote:
Originally Posted by wnjj View Post
Can you share what compiler switch that was? I ran into some stuff when porting the Windows native code from 32 to 64 bit but can always use more info for future issues. Was it something like —kill-at involving DLL name decoration?
Hi,

Of course! It was
Code:
-Wl,--add-stdcall-alias
. I also needed to add
Code:
-static-libgcc
, but that may only have been required for my project. Not sure. I am not very knowledgeable in C coding or compiling. Took me a long while to figure out. I was not sure why the 64 bit version worked without any exra compiler switches, but then the 32 bit had dependency errors when I tried to make a call to the JNI library. It loaded the dlls without issue, but would fail when making the native call. the static-libgcc cause the dll to fail on load.

Thanks,
Josh
Reply With Quote
  #53  
Old 05-19-2020, 11:47 AM
wnjj wnjj is offline
Sage Icon
 
Join Date: Jan 2009
Posts: 1,268
Quote:
Originally Posted by jvl711 View Post
Hi,

Of course! It was
Code:
-Wl,--add-stdcall-alias
. I also needed to add
Code:
-static-libgcc
, but that may only have been required for my project. Not sure. I am not very knowledgeable in C coding or compiling. Took me a long while to figure out. I was not sure why the 64 bit version worked without any exra compiler switches, but then the 32 bit had dependency errors when I tried to make a call to the JNI library. It loaded the dlls without issue, but would fail when making the native call. the static-libgcc cause the dll to fail on load.

Thanks,
Josh
You ran into the mess that is “calling convention”. The calling convention defines where and in what order function arguments are passed into a function when calling a DLL plus who cleans up the stack. For 32-bit, there were a variety of methods (stdcall being one). For 64-bit the industry got their act together and agreed on one method (plus the processors made it easier with more registers). The “kill-at” is another option that removes the trailing “@#” that 32bit DLL function names use. The important thing is that both the DLL and calling code agree.

If you don’t already have it, check out https://www.dependencywalker.com. They are lightweight (just unzip) programs that examine DLL dependencies and show the names of all exported functions. It does show some false errors in the dependency tree for some system libs that you can ignore. I relied on these heavily for sorting out function names, DLL versions, etc. when developing the 64-bit Windows version.

With libgcc, it seems you weren’t the only one: https://stackoverflow.com/questions/...t-compilations
__________________
Windows Installer

Last edited by wnjj; 05-19-2020 at 11:50 AM.
Reply With Quote
  #54  
Old 05-19-2020, 02:40 PM
jvl711's Avatar
jvl711 jvl711 is offline
Sage Expert
 
Join Date: Jan 2004
Posts: 508
Quote:
Originally Posted by wnjj View Post
You ran into the mess that is “calling convention”. The calling convention defines where and in what order function arguments are passed into a function when calling a DLL plus who cleans up the stack. For 32-bit, there were a variety of methods (stdcall being one). For 64-bit the industry got their act together and agreed on one method (plus the processors made it easier with more registers). The “kill-at” is another option that removes the trailing “@#” that 32bit DLL function names use. The important thing is that both the DLL and calling code agree.

If you don’t already have it, check out https://www.dependencywalker.com. They are lightweight (just unzip) programs that examine DLL dependencies and show the names of all exported functions. It does show some false errors in the dependency tree for some system libs that you can ignore. I relied on these heavily for sorting out function names, DLL versions, etc. when developing the 64-bit Windows version.

With libgcc, it seems you weren’t the only one: https://stackoverflow.com/questions/...t-compilations

Thank you for that info. That makes a lot of sense. When researching errors related to compiling C code a lot of times there is a solution given without a clear explanation. I had no clue that was the reason.

I was using dependency walker, but I did not think to look at the calls. I was looking for a missing dependency, not realizing the method calls were the issue. So does Java expect stdcall for 32 bit dlls?

Thanks,
Josh
Reply With Quote
  #55  
Old 05-19-2020, 02:56 PM
jvl711's Avatar
jvl711 jvl711 is offline
Sage Expert
 
Join Date: Jan 2004
Posts: 508
Quote:
Originally Posted by trk2 View Post
So the updated dll fixed the exception.

I did run into a couple other unusual issues. Upon restarting the service there were errors in the object database and then it hung following the scan of all recorded media but before re-scanning all imported media.



It exited gracefully and everything went fine on the second start of the service. A few flac files were detected as movies however. Here is an example of one:



I noticed that only the flac files that had "Find stream info called 0" in the log were detected as movies. I have attached the log files. Sagetv_1 is the first startup that I killed, sagetv_0 is where the actual scans using the format detector plugin occured.
Would you be willing to share one of those flac files with me for debugging. I think I am seeing something in the file other than audio.

Let me know if you need a way to send it to me.

Thanks,
Josh
Reply With Quote
  #56  
Old 05-19-2020, 04:12 PM
wnjj wnjj is offline
Sage Icon
 
Join Date: Jan 2009
Posts: 1,268
Quote:
Originally Posted by jvl711 View Post
Thank you for that info. That makes a lot of sense. When researching errors related to compiling C code a lot of times there is a solution given without a clear explanation. I had no clue that was the reason.

I was using dependency walker, but I did not think to look at the calls. I was looking for a missing dependency, not realizing the method calls were the issue. So does Java expect stdcall for 32 bit dlls?

Thanks,
Josh
Anyway, according to this Java assumes platform standards:

"Both the native methods and the interface APIs follow the standard library-calling convention on a given platform. For example, UNIX systems use the C calling convention, while Win32 systems use __stdcall."

https://docs.oracle.com/javase/7/doc...pec/design.htm

Further complicating things is the use of preceeding _ and trailing @# on exported functions in 32-bit which is independent of the register/stack order issues. That was likely the real difference when you added that stdcall-alias flag. I don't fully understand all of this but since GCC is primarily a Linux tool, compiling Windows DLL's requires a few of these tricks.

If you're curious, look at the exported names of the "bad" and working 32-bit DLL. I suspect they'll be different. I assume 32-bit Java expects _function@# names.

When I read further, it seems there are still different calling conventions for Linux vs Windows even on 64-bit but Windows dropped the silly _/@# stuff.
__________________
Windows Installer
Reply With Quote
  #57  
Old 05-19-2020, 05:19 PM
trk2 trk2 is offline
Sage Aficionado
 
Join Date: Jan 2006
Location: Maine
Posts: 388
Quote:
Originally Posted by jvl711 View Post
Would you be willing to share one of those flac files with me for debugging. I think I am seeing something in the file other than audio.

Let me know if you need a way to send it to me.

Thanks,
Josh
I took a look at these files today and I think it is detecting the album image that is part of the file's metadata. I also have a few mp3 files that have the album image as part of their metadata but it appears the format detector plugin is not being used for parsing mp3 files. Is that intended? I'll send you a PM with a link to one of the flac files.
Reply With Quote
  #58  
Old 05-20-2020, 11:24 AM
jvl711's Avatar
jvl711 jvl711 is offline
Sage Expert
 
Join Date: Jan 2004
Posts: 508
Quote:
Originally Posted by trk2 View Post
I took a look at these files today and I think it is detecting the album image that is part of the file's metadata. I also have a few mp3 files that have the album image as part of their metadata but it appears the format detector plugin is not being used for parsing mp3 files. Is that intended? I'll send you a PM with a link to one of the flac files.
So interestingly, the older version of ffmpeg that sage is using for the external format detection does not see the cover art at all. It thinks the flac file has one track/stream. The version that I am using for my library does see it. Below are

SageTVTranscoder.exe Output:
Code:
Input #0, flac, from '02 Awolnation - Some Sort of Creature.flac':
  Metadata:
    Album           : Megalithic Symphony (Advance)
    Artist          : Awolnation
    Genre           : Electronic
    title           : Some Sort of Creature
    DATE            : 2011
    TRACKNUMBER     : 02
    ThumbnailSize   : 58513
    ThumbnailOffset : 332
  Duration: 00:00:26.73, start: 0.000000, bitrate: 366 kb/s
    Stream #0.0, 1, 1/44100: Audio: flac, 44100 Hz, 2 channels, s16

current ffprobe.exe Output:

Code:
Input #0, flac, from '..\..\..\02 Awolnation - Some Sort of Creature.flac':
  Metadata:
    Album           : Megalithic Symphony (Advance)
    Artist          : Awolnation
    Genre           : Electronic
    title           : Some Sort of Creature
    DATE            : 2011
    track           : 02
  Duration: 00:00:26.73, start: 0.000000, bitrate: 366 kb/s
    Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
    Stream #0:1: Video: mjpeg (Progressive), yuvj444p(pc, bt470bg/unknown/unknown), 500x500, 90k tbr, 90k tbn, 90k tbc (attached pic)
    Metadata:
      comment         : Cover (front)
What is the outcome of this in Sage. Is the file playable. Does it play in a video player? Does it show in the music library? Not sure what we should be doing here. I do not tell sage how to classify the file in the plugin. I just tell it what the file contains.
Reply With Quote
  #59  
Old 05-20-2020, 12:36 PM
trk2 trk2 is offline
Sage Aficionado
 
Join Date: Jan 2006
Location: Maine
Posts: 388
Quote:
Originally Posted by jvl711 View Post
What is the outcome of this in Sage. Is the file playable. Does it play in a video player? Does it show in the music library? Not sure what we should be doing here. I do not tell sage how to classify the file in the plugin. I just tell it what the file contains.
The files show up in the imported video titles category. The files do not show up in the music library. They are playable, but the timeline function doesn't work, it just stays stuck at 0:00:00. It does seem to use the video player as Sage presents the normal video menu when trying to play the file which it doesn't do when playing a music file. However, the LAV video filter is not loaded however when playing the file. If you drop the file into a Sage import folder you should be able to reproduce the results.

The rest of the FLAC files (without an image in the metadata) are in the music library, however I noticed that Sage is not recognizing the metadata. It's simply listing them as "unknown" and sorted by file name. I didn't notice prior because the mp3 files imported fine and the "unknown" category is at the bottom of the list.
Reply With Quote
  #60  
Old 05-20-2020, 01:32 PM
SteveW's Avatar
SteveW SteveW is offline
Sage Aficionado
 
Join Date: Oct 2008
Location: Fall River, Nova Scotia, Canada
Posts: 354
I quick question for Jvl711..


With the plug in (and the addition of this to Sage 9.2.2) how do we take advantage of the forced subtitles? Is there a menu option? I saw someone mention something about maybe having a line in the .properties files for the videos you want to force the subtitles on?


Tnx!
__________________
Server: Win 10 Pro 64 Bit, Intel i5, 8 GB, Samsung EVO 850 500 GB for OS, WD Black 4 TB + WD Black 1 TB for Recordings, WD Green 2 TB x 1 and WD Red 6 TB x 2 for DVD/Bluray Rips, Music, Home Movies, etc., Sage 9.1.8.770, HDPVR x 2, BellTV 6131 Receiver x 2, USB-UIRT with 56 KHz Receiver

Clients: PC Client x 2, HD-300 x 4 (1 Using Netgear MoCA Coax Bridges), Placeshifter x 1
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Commercial Detector Plugin Disabled LJSSageUser Phoenix 5 03-09-2014 05:34 PM
Plugin: MizookLCD (Alternate SageTV LCDSmartie Plugin) cslatt SageTV Customizations 48 06-11-2012 10:44 AM
Plugin request for SageTV Windows Client: Autofrequency plugin mkanet SageTV v7 Customizations 3 11-12-2011 09:33 AM
Do I have to format my WHS to use SageTV? blackcows General Discussion 9 10-13-2009 06:51 PM
IR Code (Pronto Format) as in UIRT Helper --> SageTV Format as in Sagetv.properties krasev Hardware Support 0 09-12-2006 10:56 PM


All times are GMT -6. The time now is 01:23 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.