SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations > Batch Metadata Tools

Notices

Batch Metadata Tools This forums is for discussing the user-created Batch Metadata Tools for SageTV.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 07-12-2010, 07:44 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,541
Cross Linked Metadata Discussions (Any BMT Version)

NOTE: This should be resolved as of Sage 7.0.13 and BMT 4.5

The cross linked metadata issue is something that has been around since the 3.x version of BMT. I'd like to get a general consensus on when/how this happening, and so, I'd like people to document their experiences and theories here, so that I can have a single place to reference the issues.

Cross linked metadata refers to an anomaly whereby one sagetv media item mysteriously gets the metadata associated to another sagetv mediaitem, in such a way, they they appear "linked". When editing the metadata for one item in the web ui, it appears to update other items as well.

To date, I think this is only happening for SageTV recordings, but if you've experienced this for other items, then please share your experiences.

When sharing your experience, please let me know the SageTV version and your BMT version. Also, if you've had a past experience with a specific bmt version, then let me know about that as well.

Hopefully by grouping the experiences together in a single thread, we might be able to see a pattern emerge.

Thanks

Last edited by stuckless; 08-13-2010 at 05:30 PM.
Reply With Quote
  #2  
Old 07-12-2010, 07:50 AM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,957
I think my first comment would be... what is actually being crosslinked, is it just the fanart title, and list of fanart images? if so, do those really need to be explicitly set in the metadata? can't that be garnered from the title by phoenix on request?
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #3  
Old 07-12-2010, 08:17 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,541
Quote:
Originally Posted by Fuzzy View Post
I think my first comment would be... what is actually being crosslinked, is it just the fanart title, and list of fanart images? if so, do those really need to be explicitly set in the metadata? can't that be garnered from the title by phoenix on request?
What appears to be cross linked at this point is any metadata field. But in 4.2, I do not update the core metadta for items that have existing epg info, and as such the following custom metadata fields are set... MediaTitle (used as fanart title), SeasonNumber, EpisodeNumber, and MediaType. Those 4 fields are used by the Fanart APIs to quickly resolve fanart.

Phoenix supports resolving fanart without those fields, but it's very expensive, and disabled by default. Basically it has to, in real time, filename scrape the title to determine the MediaType and MediaTitle using a limited set of filename scrapers. Also for recordings, it becomes even more expensive, since it would need to actually go out to tvdb and resolve the season info, for season specific fanart.

So while, it is possible to do fanart lookups without the custom metadata fields, it's not recommended, and the root issue would still exist, which is why is the metadata becoming linked to other recordings in the first place? Is this a bmt issue? Core sagetv issue? etc. At this point, i'm pretty sure it's a bmt issue, but i'm not sure where to look.
Reply With Quote
  #4  
Old 07-12-2010, 01:32 PM
Tiki's Avatar
Tiki Tiki is offline
Sage Icon
 
Join Date: Feb 2005
Location: Southwest Florida, USA
Posts: 1,806
Actually, in my experience, I have never seen the cored metadata get corrupted. Version 3.x would overwrite core metadata with downloaded data and this could cause some problems (for example the episode ID would get overwritten which meant that Sage could no longer sort by ID properly and it might record the same show again, since it wouldn't recognize it as being watched), but this was not the cross-linking problem.

From what I could tell, it was only the custom metadata that was being cross-linked. In most cases, the STV does not display this custom metadata, so you wouldn't even see it unless (a) you look in the BMT web UI or (b) you have fanart enabled. Since fanart relies on these custom metadata fields it will display the wrong pictures when this cross-linking happens.
__________________
Server: Athlon II X4 630 CPU, Gigabyte MA770T-UD3P (AM3) Motherboard, VisionTek Radeon HD4670 PCIe w/ Component output to 50" CRT Rear-Projection HDTV (Mitsubishi), Win7-64Bit (Home Premium), 4GB RAM
Capture Devices (8 tuners): Colossus (x1), HD-PVR (x1), HDHR Prime (x2)
,USBUIRT (multi-zone)
Source:
Comcast Cable
Primary Client: Server Other Clients: (1) HD200, (1) HD300
Retired Equipment: MediaMVP, PVR150 (x2), PVR150MCE,
HDHR, HVR-2250
Reply With Quote
  #5  
Old 07-12-2010, 10:18 PM
JREkiwi's Avatar
JREkiwi JREkiwi is offline
Sage Icon
 
Join Date: Jan 2005
Location: Auckland, New Zealand
Posts: 2,059
On my system with BMT 2, 3 and 4 and SageTV 6.6, 7.0

I've only every seen MediaTitle, MediaType and EpisodeTitle crosslinked.
It only happens with the automatic plugin.
It only happens on recordings on my PVR-500 from my STB.
Every recording from the startup of SageTV recorded on my PVR-500 has the MediaTitle, MediaType and EpisodeTitle values of the first show recorded.

Editing the values on any single recording via SetMediaFileMetadata, changes it on every recording made on my PVR-500 since SageTV started.

Restarting SageTV fixes the crosslinking for that group of recordings and editing the values via SetMediaFileMetadata effects only the correct MediaFile. All new recordings after the restart will have the MediaTitle, MediaType and EpisodeTitle values of the first recorded MediaFile after the restart.

It NEVER happens on my digital recordings. All digital tuners are configured with encode_digital_tv_as_program_stream=false.

John
Reply With Quote
  #6  
Old 07-12-2010, 10:52 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,957
Quote:
Originally Posted by JREkiwi View Post
On my system with BMT 2, 3 and 4 and SageTV 6.6, 7.0

I've only every seen MediaTitle, MediaType and EpisodeTitle crosslinked.
It only happens with the automatic plugin.
It only happens on recordings on my PVR-500 from my STB.
Every recording from the startup of SageTV recorded on my PVR-500 has the MediaTitle, MediaType and EpisodeTitle values of the first show recorded.

Editing the values on any single recording via SetMediaFileMetadata, changes it on every recording made on my PVR-500 since SageTV started.

Restarting SageTV fixes the crosslinking for that group of recordings and editing the values via SetMediaFileMetadata effects only the correct MediaFile. All new recordings after the restart will have the MediaTitle, MediaType and EpisodeTitle values of the first recorded MediaFile after the restart.

It NEVER happens on my digital recordings. All digital tuners are configured with encode_digital_tv_as_program_stream=false.

John
Are you saying your seeing this with direct calls to setmediafilemetadata? if so, it sounds like an initialization issue in the sage api's itself. if you can reproduce this reliably, sounds like it needs to be brought up with them via a bug report. It would be a pretty significant bug.
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #7  
Old 07-12-2010, 11:49 PM
JREkiwi's Avatar
JREkiwi JREkiwi is offline
Sage Icon
 
Join Date: Jan 2005
Location: Auckland, New Zealand
Posts: 2,059
Quote:
Originally Posted by Fuzzy View Post
Are you saying your seeing this with direct calls to setmediafilemetadata?
I'm saying that after a MediaFile has it's metadata crosslinked with another file via the automatic plugin then any change to SeasonNumber, EpisodeNumber, or MediaType via SetMediaFileMetadata will change the metadata for both MediaFiles. After a SageTV restart it won't change the metadata for both files. I've only ever seen the crosslinking occur via the automatic plugin.

John
Reply With Quote
  #8  
Old 07-13-2010, 10:19 AM
Brent94Z Brent94Z is offline
Sage Fanatic
 
Join Date: Aug 2004
Posts: 863
The crosslinking just started a couple weeks ago for me and last night something happened for the fourth time that I've noticed...

The first time I noticed this was after a recording of "Monster Quest". After that recording it seemed like nearly everything was getting the fanart for "Monster Quest"... just the art, not the description. Then the same thing happened with "Leverage" where when "Leverage" was recorded, it got the correct fanart but then afterwords, all or nearly everything seemed to get "Leverage" as well. Then, on Friday, "Friday Night Lights" was recorded first and then everything after that that night got "Friday Night Lights" for the art. Now, last night, I had 4 recordings with the first one being "Lie To Me". Well, "Lie To Me" did NOT get the "Friday Night Lights" fanart and did get its own fanart but, all of my recordings following "Lie To Me" got the "Lie To Me" fanart.

So, it does seem to be something possible related to caching. One thing I'll keep an eye on now is to see what causes it to finally change and be correct. I *think* but not 100% on this that my recordings that are getting crosslinked don't have their own specific fanart and instead I'd normally see a frame from the first part of the video. So, while I need to watch this further, so far it looks like the fanart for one show is continuing for shows that don't have specific fanart UNTIL another show is recorded with its own specific fanart and then that show's fanart is used until another show is recorded with specific fanart. All the "oddball" recordings get fanart from the last show with actual fanart until another show that does have fanart is recorded... then the process starts over with the most recent show's fanart. Does that make sense to anybody besides me? LOL!
Reply With Quote
  #9  
Old 07-13-2010, 10:46 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,541
Quote:
Originally Posted by Brent94Z View Post
So, it does seem to be something possible related to caching.
I did some digging again into the caching that is used by bmt, which is very little. But, bmt does cache downloaded content from tvdb for performance reasons. Now, if somehow, the cached files were being referenced incorrectly, then we would see failed fanart, since the the XML that would be loaded would be from the wrong series. BMT, create new object instances for each search item, so there is no cached/reused object instances those cases.
Reply With Quote
  #10  
Old 07-13-2010, 11:08 AM
Brent94Z Brent94Z is offline
Sage Fanatic
 
Join Date: Aug 2004
Posts: 863
Quote:
Originally Posted by stuckless View Post
I did some digging again into the caching that is used by bmt, which is very little. But, bmt does cache downloaded content from tvdb for performance reasons. Now, if somehow, the cached files were being referenced incorrectly, then we would see failed fanart, since the the XML that would be loaded would be from the wrong series. BMT, create new object instances for each search item, so there is no cached/reused object instances those cases.
LOL! I didn't even read this before I posted in the other other thread but it appears that I may have found exactly what you are talking about. Hopefully so! But, it is weird that there appears to only be a handful of us seeing this yet if it was writing the URL incorrectly (which is what it seems?) then you'd think it would be doing this for everyone??? Weird.

Sorry for posting in multiple threads about the same thing. Just wanted to make sure we were on the same page
Reply With Quote
  #11  
Old 07-13-2010, 12:18 PM
Narflex's Avatar
Narflex Narflex is offline
Sage
 
Join Date: Feb 2003
Location: Redondo Beach, CA
Posts: 6,301
Sean sent me an email about this and after reading JREKiwi's post about it only happening on the PVR500 I did some digging and found the issue. It was due to how the core was reusing media format objects when they didn't change (because they never did w/ our code)....but plugins setting alternate metadata values would change them. So this is modified for 7.0.12 to never reuse those objects and won't be a problem after that. (7.0.12 should be out sometime this week)

Sorry for any inconvenience caused by this.
__________________
Jeffrey Kardatzke
Google
Founder of SageTV
Reply With Quote
  #12  
Old 07-13-2010, 12:23 PM
Brent94Z Brent94Z is offline
Sage Fanatic
 
Join Date: Aug 2004
Posts: 863
Excellent! Glad it got figured out. Thanks!
Reply With Quote
  #13  
Old 07-13-2010, 01:23 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,541
Quote:
Originally Posted by Narflex View Post
Sean sent me an email about this and after reading JREKiwi's post about it only happening on the PVR500 I did some digging and found the issue. It was due to how the core was reusing media format objects when they didn't change (because they never did w/ our code)....but plugins setting alternate metadata values would change them. So this is modified for 7.0.12 to never reuse those objects and won't be a problem after that. (7.0.12 should be out sometime this week)

Sorry for any inconvenience caused by this.
Thanks Jeff! I'm glad you were able to identify the issue... I was ready to strip out TV support altogether, since I couldn't find a reasonable fix within the BMT codebase
Reply With Quote
  #14  
Old 07-16-2010, 01:46 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,957
Quote:
Originally Posted by stuckless View Post
Thanks Jeff! I'm glad you were able to identify the issue... I was ready to strip out TV support altogether, since I couldn't find a reasonable fix within the BMT codebase
So now that THATS taken care of, sounds like its time to put back in the auto TV import to recordings feature.... ;-)
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #15  
Old 07-16-2010, 02:06 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,541
Quote:
Originally Posted by Fuzzy View Post
So now that THATS taken care of, sounds like its time to put back in the auto TV import to recordings feature.... ;-)
Absolutely. I've been gradually stripping out TV functionality from BMT (as it related to directly updating metadata), because of this very issue. Now that I know it will be resolved in core, I'll add back the ability (options) to auto-import tv shows as recordings... and add back the options from the UI as well.

That being said... i'll be on vacation (out of town) for about 3 weeks... so don't expect anything too soon
Reply With Quote
  #16  
Old 08-08-2010, 12:59 PM
skyeclad skyeclad is offline
Sage Aficionado
 
Join Date: Nov 2007
Location: NY DMA
Posts: 441
I just manually updated a TV show to get metadata(Caprica). I then noticed that 2 DVD's also received the metadata descriptions within Sage not artwork. When I go to BMT, I see Caprica as the name for the 2 DVD's but when I open the DVD within BMT it has their correct description metadata.

Is this the crosslinking issue and how can I re-establish the correct metadata within Sage?
Reply With Quote
  #17  
Old 08-08-2010, 08:14 PM
cmaloney cmaloney is offline
Sage User
 
Join Date: Feb 2009
Posts: 22
Unfortunately this is still a problem with 7.0.12.0. I just completed a new installation of SageTV, installed BMT and got it all configured. No matter what I seem to try I have 3 movies that all linked together so that whenever I fix one, the other two also display the information for the fixed movie. Any ideas how this problem can be fixed?

Below is the info from my status screen:
Code:
Phoenix
Version
	
2.0.6
sagex.api Version
	
7.0.12.0
Automatic Metadata/Fanart Lookups Enabled
	
true
Fanart Enabled
	
true
Fanart Folder
	
C:\Program Files\SageTV\SageTV\FanArt
SageTV
Version
	
7.0.14.236
Required Version
	
7.0.11
Java Version
	
1.6.0_17
Server
	
true
Recordings Used Diskspace
	
0.00 G
Library Import Scan in Progress
	
false
JVM Max Memory
	
1,040,512 M
Reply With Quote
  #18  
Old 08-08-2010, 11:57 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,957
Are these files you've Just imported, or were they imported previously back before the xrosslinking was fixed?
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #19  
Old 08-09-2010, 04:44 AM
cmaloney cmaloney is offline
Sage User
 
Join Date: Feb 2009
Posts: 22
It was a fresh install of 7.0.14. I had 6 previously but I moved the folder out of the way so that 7.0.14 installed in the default location without getting any previous settings. The only plugins I installed are: BMT, MyMovies, SageMC (not used currently), Phoenix fanart, IMDB search, and all the dependencies of these plugins. After manually fixing a single movie using the web UI, any additional movie fixes will use cause the additional movies to link with the first movie. They all show up as duplicates in the web UI, but with the correct picture name (but not the correct fan art).
Reply With Quote
  #20  
Old 08-09-2010, 06:26 AM
Tiki's Avatar
Tiki Tiki is offline
Sage Icon
 
Join Date: Feb 2005
Location: Southwest Florida, USA
Posts: 1,806
Right, but were the problem shows recorded or imported into the Sage database before or after installing 7.0.14?

I think once they've been corrupted they will stay corrupted even if you upgrade to a newer version of Sage. But if new recordings are still getting corrupted, that would indicate that the problem hasn't been completely solved.
__________________
Server: Athlon II X4 630 CPU, Gigabyte MA770T-UD3P (AM3) Motherboard, VisionTek Radeon HD4670 PCIe w/ Component output to 50" CRT Rear-Projection HDTV (Mitsubishi), Win7-64Bit (Home Premium), 4GB RAM
Capture Devices (8 tuners): Colossus (x1), HD-PVR (x1), HDHR Prime (x2)
,USBUIRT (multi-zone)
Source:
Comcast Cable
Primary Client: Server Other Clients: (1) HD200, (1) HD300
Retired Equipment: MediaMVP, PVR150 (x2), PVR150MCE,
HDHR, HVR-2250
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Metadata Tools 4.0 Beta Thread (aka BMT 4.0) stuckless Batch Metadata Tools 885 09-02-2013 08:03 AM
BMT: Read this if you don't see any BMT topics Opus4 SageTV Customizations 0 04-25-2010 09:17 PM
Is there a way of doing Metadata WITHOUT BMT plnewton SageTV Customizations 12 08-17-2009 06:53 PM
BMT Metadata Provider List? cowboyway SageMC Custom Interface 1 08-12-2009 05:54 AM
Multiple Sage Servers and cross-connecting Orthogonite General Discussion 3 03-12-2004 11:49 AM


All times are GMT -6. The time now is 08:37 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.