SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations > Batch Metadata Tools

Notices

Batch Metadata Tools This forums is for discussing the user-created Batch Metadata Tools for SageTV.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 11-08-2010, 05:54 AM
davidk21770 davidk21770 is offline
Sage Expert
 
Join Date: Mar 2006
Location: Maryland
Posts: 513
A few hints needed for MediaTitles.xml

I'm trying to write some match strings and I'm missing what the input and output strings SHOULD look like? What am I matching?

For example, why is there a leading and trailing test for \ or /?

Thanks!
__________________
HDHomeRun x2
Cable Box via Hauppauge HD PVR and USB-UIRT
2 SageTV's HD300

Intel Core 2 Quad Q9650 @4025 MHz, 8G G.Skill Performance
DFI Lanparty DK X48-T2RS Plus; SilverStone Decathlon DA1000 (died - back to an ancient enermax)
eVGA 7900 GT KO RoHS -- Zalman VP900CU Cooling
Synology NAS 22TB
Reply With Quote
  #2  
Old 11-08-2010, 07:15 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
Quote:
Originally Posted by davidk21770 View Post
I'm trying to write some match strings and I'm missing what the input and output strings SHOULD look like? What am I matching?

For example, why is there a leading and trailing test for \ or /?

Thanks!
David, perhaps you can post some examples, and I can help you fill in the blanks.

MediaTitles.xml accepts a complete file path as the input. Let me break down the sample...

Code:
	<match>
		<regex>[\\/]Babylon\s*5[\\/]</regex>
		<title>Babylon 5</title>
		<year>1993</year>
		<metadata type="tv" name="tvdb">7072</metadata>
	</match>
The about matcher uses a regular expression against the filename to force any files that are in a directory called Babylon 5 to have the series id 7072.

In the <regex>, [\\/] means match a forward slash or backslah in the path. In regex speak the [] is a special command that means match any of these characters. The backslash is doubled \\ because a single backslash is a regex escape char, so if you mean a literal backslash, then you use \\. Because I tend to want to support any platform then I need to use \\ and / to support both windows \\ and linux/mac /. So, in the above regex, it will match a section of the path that looks like
\Babylon 5\ or /Babylon 5/, and because of the optional whitespace (\s*) it will also match \Babylon5\ or \Babylon 5\, etc.

The <title> and <year> are not required in this case, since you are specifically providing <metadata tv="tvdb"> tag to specify the series id. I tend to put them in, just for "documentation". But if you didn't specify a series, then you could simply provide the <title> and optionally the <year> and bmt will dynamically look it up.

The MediaTitles is good for forcing an entire directory to a specific series, or something if you don't want to rename your files to include a better title/year, then you can force it the media titles xml.

Personally, I tend to name my stuff better.... ie, for Movies, I make sure that I include the year in the filename and for TV, I also try to name them how they are named in TVDB. Sometimes just renaming the files, is better than messing with the MediaTitles.xml.

Movie Title (Year).avi
Series Name S##E##.avi

If you have something not in a standard naming format, then the other option is to create your own scraper that will scrape your files correctly.
Reply With Quote
  #3  
Old 11-08-2010, 08:05 AM
davidk21770 davidk21770 is offline
Sage Expert
 
Join Date: Mar 2006
Location: Maryland
Posts: 513
Ok. I missed the "file path" part.
So, if I'm matching (say) CSI: Crime Scene Investigation, it normally comes in something like:
CSICrimeSceneInvestigation-HouseofHoarders-8688341-0.mpg
So, my match'd be
<regex>[\\/]CSICrimeSceneInvestigation-*</regex>

Edit:
Now, if I'm getting an imported file of the form Sanctuary.US.xxxx do I match to "Sanctuary US" or "Sanctuary.US"?
__________________
HDHomeRun x2
Cable Box via Hauppauge HD PVR and USB-UIRT
2 SageTV's HD300

Intel Core 2 Quad Q9650 @4025 MHz, 8G G.Skill Performance
DFI Lanparty DK X48-T2RS Plus; SilverStone Decathlon DA1000 (died - back to an ancient enermax)
eVGA 7900 GT KO RoHS -- Zalman VP900CU Cooling
Synology NAS 22TB

Last edited by davidk21770; 11-08-2010 at 08:08 AM.
Reply With Quote
  #4  
Old 11-08-2010, 09:28 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
Quote:
Originally Posted by davidk21770 View Post
Ok. I missed the "file path" part.
So, if I'm matching (say) CSI: Crime Scene Investigation, it normally comes in something like:
CSICrimeSceneInvestigation-HouseofHoarders-8688341-0.mpg
So, my match'd be
<regex>[\\/]CSICrimeSceneInvestigation-*</regex>
That looks correct.

Quote:
Edit:
Now, if I'm getting an imported file of the form Sanctuary.US.xxxx do I match to "Sanctuary US" or "Sanctuary.US"?
<regex>[\\/]Sanctuary.US.*</regex>
<title>Sanctuary</title>
Reply With Quote
  #5  
Old 11-08-2010, 09:35 AM
davidk21770 davidk21770 is offline
Sage Expert
 
Join Date: Mar 2006
Location: Maryland
Posts: 513
Thanks!
__________________
HDHomeRun x2
Cable Box via Hauppauge HD PVR and USB-UIRT
2 SageTV's HD300

Intel Core 2 Quad Q9650 @4025 MHz, 8G G.Skill Performance
DFI Lanparty DK X48-T2RS Plus; SilverStone Decathlon DA1000 (died - back to an ancient enermax)
eVGA 7900 GT KO RoHS -- Zalman VP900CU Cooling
Synology NAS 22TB
Reply With Quote
  #6  
Old 11-15-2010, 10:16 AM
ohpleaseno ohpleaseno is offline
Sage Aficionado
 
Join Date: Apr 2009
Location: South Florida
Posts: 464
So, just so I understand this correctly, any show that BMT is having difficulty matching, we can add additional lookups in this one file with the show title in the regex field and tvdb number in the metadata field.

Code:
          <match>
		<regex>[\\/]This Old House[\\/]</regex>
		<metadata type="tv" name="tvdb">WHATEVER THE TVDB ID IS HERE</metadata>
	  </match>

          <match>
		<regex>[\\/]New Yankee Workshop[\\/]</regex>
		<metadata type="tv" name="tvdb">WHATEVER THE TVDB ID IS HERE</metadata>
	</match>
__________________
Server: WHS with 11TB of storage
Clients:1) HD-200 to a Samsung 32" Slimfit CRT HDTV 2) HD-200 to a Pioneer 26" X1 LCD
Tuners: 1) HD-Homerun 2) Avermedia USB ATSC
Twitter: ohpleaseno
Reply With Quote
  #7  
Old 11-15-2010, 12:35 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
That is correct. the Regular Expression matches a portion of the filename (and path) to whatever you provide. So in your case, it will force the lookup to be tv (with the given tvdb id) if the file is in a directory that exactly matches, "This Old House".
Reply With Quote
  #8  
Old 11-15-2010, 12:42 PM
ohpleaseno ohpleaseno is offline
Sage Aficionado
 
Join Date: Apr 2009
Location: South Florida
Posts: 464
When you say "in a directory" does that mean it only works for imported? I'm looking to get the scan working for some PBS shows that don't show up properly for some reason. Will this fix that as well if the filename from recorded TV contains This Old House?
__________________
Server: WHS with 11TB of storage
Clients:1) HD-200 to a Samsung 32" Slimfit CRT HDTV 2) HD-200 to a Pioneer 26" X1 LCD
Tuners: 1) HD-Homerun 2) Avermedia USB ATSC
Twitter: ohpleaseno
Reply With Quote
  #9  
Old 11-15-2010, 03:27 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
if you are talking about recordings, then they are usally named something like,

ThisOldHouse-EpisodeTitle-0000000-0.ts

So, if you wanted to match on that, then you could use a starts with regex...

<regex>[\\/]ThisOldHouse-.*</regex>

That basically means match /ThisOldHouse- or \ThisOldHouse- as part of the filename path.

So if the full recording was

c:\tv\recordings\ThisOldHouse-SomeEpisode-000000-0.ts, then it would match. But if the recording was c:\tv\recordings\This Old House.ts then it would not match (because of the spacing and missing dash)
Reply With Quote
  #10  
Old 11-15-2010, 08:08 PM
ohpleaseno ohpleaseno is offline
Sage Aficionado
 
Join Date: Apr 2009
Location: South Florida
Posts: 464
does sage service need to be restarted for the change in this file to be recognized?
__________________
Server: WHS with 11TB of storage
Clients:1) HD-200 to a Samsung 32" Slimfit CRT HDTV 2) HD-200 to a Pioneer 26" X1 LCD
Tuners: 1) HD-Homerun 2) Avermedia USB ATSC
Twitter: ohpleaseno
Reply With Quote
  #11  
Old 11-15-2010, 08:29 PM
jusjoken jusjoken is offline
SageTVaholic
 
Join Date: Dec 2005
Location: Strathmore, AB
Posts: 2,567
Quote:
Originally Posted by ohpleaseno View Post
does sage service need to be restarted for the change in this file to be recognized?
Yes... at least in older versions it did and I haven't seen any word that it changed.

k
Reply With Quote
  #12  
Old 11-16-2010, 06:33 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
I think you can reload them from the bmt web ui. It's an option under the Configuration sections.
Reply With Quote
  #13  
Old 11-28-2010, 04:10 PM
ohpleaseno ohpleaseno is offline
Sage Aficionado
 
Join Date: Apr 2009
Location: South Florida
Posts: 464
Is there a log file that will tell me if things are properly hitting MediaTitles.xml?
__________________
Server: WHS with 11TB of storage
Clients:1) HD-200 to a Samsung 32" Slimfit CRT HDTV 2) HD-200 to a Pioneer 26" X1 LCD
Tuners: 1) HD-Homerun 2) Avermedia USB ATSC
Twitter: ohpleaseno
Reply With Quote
  #14  
Old 12-02-2010, 07:30 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
Quote:
Originally Posted by ohpleaseno View Post
Is there a log file that will tell me if things are properly hitting MediaTitles.xml?
The phoenix.log log file will tell you if it's using the MediaTitles... if you open the file and search for MediaTitles, i think you'll see some messages related to it.
Reply With Quote
  #15  
Old 12-08-2010, 01:50 PM
phelme's Avatar
phelme phelme is offline
Sage Icon
 
Join Date: Dec 2006
Posts: 1,127
Can we add season and episode elements to MediaTitles.xml to better target the query? A show like Jeopardy! fails because there is no season 1/episode 1 entry, which is the default to search.
Code:
2010-12-08 11:39:27,342 [2438065@qtp-18376755-6] INFO  sagex.phoenix.metadata.provider.tvdb.TVDBItemParser - TVDB Episode: http://www.thetvdb.com/api/5645B594A3F32D27/series/77075/default/1/1/en.xml
2010-12-08 11:39:27,451 [2438065@qtp-18376755-6] WARN  sagex.phoenix.metadata.provider.tvdb.TVDBItemParser - Failed to get season/episode specific information for 77075; Season: 1; episode: 1

java.io.FileNotFoundException: http://www.thetvdb.com/api/5645B594A3F32D27/series/77075/default/1/1/en.xml
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
But there is a 24/165 which would work.

Hmm, alternatively, maybe I'll see if I can put in the first episode in thetvdb.com...

Edit: Added entry to thetvdb.com for 1/1. We shall see if that takes care of it.
__________________
Server: AMD 9600 Phenom on XP, Gigabyte GA-MA78GPM, 2GB RAM, 320+250+500 GB SATA drives, HDHomeRun Prime, HD-PVR x.5.1, Paterson serial
Client/Encoder:AMD 3800+ X2, 512 MB RAM, ATI X1650 XT, nMediaPC case, Hauppauge HD-PVR, Cyberlink/ArcSoft decoders, USB-UIRT
Client/Encoder: AMD 3800+ X2, 512 MB RAM, 6150 graphics, nMediaPC case, ArcSoft decoders
Client: HD300, Asus Pundit P1-AH1, AMD 3800+ X2 CPU, 1 GB RAM, 6150 graphics, ArcSoft decoders
Backup: Synology
SageTV version: FINAL

Last edited by phelme; 12-08-2010 at 02:03 PM.
Reply With Quote
  #16  
Old 12-08-2010, 02:13 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
Quote:
Originally Posted by phelme View Post
Can we add season and episode elements to MediaTitles.xml to better target the query? A show like Jeopardy! fails because there is no season 1/episode 1 entry, which is the default to search.
Code:
2010-12-08 11:39:27,342 [2438065@qtp-18376755-6] INFO  sagex.phoenix.metadata.provider.tvdb.TVDBItemParser - TVDB Episode: http://www.thetvdb.com/api/5645B594A3F32D27/series/77075/default/1/1/en.xml
2010-12-08 11:39:27,451 [2438065@qtp-18376755-6] WARN  sagex.phoenix.metadata.provider.tvdb.TVDBItemParser - Failed to get season/episode specific information for 77075; Season: 1; episode: 1

java.io.FileNotFoundException: http://www.thetvdb.com/api/5645B594A3F32D27/series/77075/default/1/1/en.xml
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
But there is a 24/165 which would work.

Hmm, alternatively, maybe I'll see if I can put in the first episode in thetvdb.com...

Edit: Added entry to thetvdb.com for 1/1. We shall see if that takes care of it.

It would only search for season 1 episode 1, if it determined that the episode was, in fact, season 1 and episode 1 by scraping the file. BMT has no default season and episode values, they either exist, or the values are -1 (which means they don't exist). But in seeing a values of 1, means that the scrapers pulled that from somewhere.
Reply With Quote
  #17  
Old 12-11-2010, 11:33 AM
phelme's Avatar
phelme phelme is offline
Sage Icon
 
Join Date: Dec 2006
Posts: 1,127
So basically if there is no date match or season/episode identifier match (as in my Jeopardy! example, where there usually is no exact entry for every show), I'm out of luck at extracting any good data from thetvdb.com? Because currently I simply get metadata lookup failures every night.
__________________
Server: AMD 9600 Phenom on XP, Gigabyte GA-MA78GPM, 2GB RAM, 320+250+500 GB SATA drives, HDHomeRun Prime, HD-PVR x.5.1, Paterson serial
Client/Encoder:AMD 3800+ X2, 512 MB RAM, ATI X1650 XT, nMediaPC case, Hauppauge HD-PVR, Cyberlink/ArcSoft decoders, USB-UIRT
Client/Encoder: AMD 3800+ X2, 512 MB RAM, 6150 graphics, nMediaPC case, ArcSoft decoders
Client: HD300, Asus Pundit P1-AH1, AMD 3800+ X2 CPU, 1 GB RAM, 6150 graphics, ArcSoft decoders
Backup: Synology
SageTV version: FINAL
Reply With Quote
  #18  
Old 12-11-2010, 01:16 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
Quote:
Originally Posted by phelme View Post
So basically if there is no date match or season/episode identifier match (as in my Jeopardy! example, where there usually is no exact entry for every show), I'm out of luck at extracting any good data from thetvdb.com? Because currently I simply get metadata lookup failures every night.
Well, the data has to be in tvdb at the time your do the lookup. Let's say that you record Jeopardy at 8pm but tvdb has nothing for the episode, so at 9pm somone goes in the adds the data... you missed it because the lookup happened before there was data. I've seen this happen many times for some shows.

Recordings are looked up by aired date and episode title (it will try both). So, in the case of jeopardy, if it's not finding, then either the episode does not exist... or the aired date doesn't match, or the episode name doesn't match.

If you use the web ui, the next day (or a couple of days later) to do manually do a lookup, does it find it then? Have you confirmed that tvdb has the data and the data does in fact match your aired date, or episode title?
Reply With Quote
  #19  
Old 12-11-2010, 02:02 PM
phelme's Avatar
phelme phelme is offline
Sage Icon
 
Join Date: Dec 2006
Posts: 1,127
Quote:
Originally Posted by stuckless View Post
If you use the web ui, the next day (or a couple of days later) to do manually do a lookup, does it find it then? Have you confirmed that tvdb has the data and the data does in fact match your aired date, or episode title?
Nope, the only dates that have anything are either the celebrity matches or some tournaments that people enter manually. I get that since its a daily game show, there isn't likely to be any reason for it to be updated on a regular basis.

Hmm. Is there anyway I can tell BMT to ignore Jeopardy! so I don't get the daily failure messages?
__________________
Server: AMD 9600 Phenom on XP, Gigabyte GA-MA78GPM, 2GB RAM, 320+250+500 GB SATA drives, HDHomeRun Prime, HD-PVR x.5.1, Paterson serial
Client/Encoder:AMD 3800+ X2, 512 MB RAM, ATI X1650 XT, nMediaPC case, Hauppauge HD-PVR, Cyberlink/ArcSoft decoders, USB-UIRT
Client/Encoder: AMD 3800+ X2, 512 MB RAM, 6150 graphics, nMediaPC case, ArcSoft decoders
Client: HD300, Asus Pundit P1-AH1, AMD 3800+ X2 CPU, 1 GB RAM, 6150 graphics, ArcSoft decoders
Backup: Synology
SageTV version: FINAL
Reply With Quote
  #20  
Old 12-11-2010, 02:57 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
Quote:
Originally Posted by phelme View Post
Nope, the only dates that have anything are either the celebrity matches or some tournaments that people enter manually. I get that since its a daily game show, there isn't likely to be any reason for it to be updated on a regular basis.

Hmm. Is there anyway I can tell BMT to ignore Jeopardy! so I don't get the daily failure messages?
You can add an exclude filter, I'd try something like...
Code:
[/\\]Jeopardy-
See the Phoenix Configuration section of the BMT User Guide for more information (specifically the table entry for the exlude filter at the end of the section)
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
MediaTitles.xml failed, how do I see what went wrong? davidk21770 Batch Metadata Tools 5 09-28-2010 11:47 AM
Quick MediaTitles.xml question davidk21770 Batch Metadata Tools 3 09-24-2010 01:48 PM
Problem with MediaTitles.xml cmaloney Batch Metadata Tools 12 08-17-2010 07:44 PM
Finally Ready to Setup Sage..any hints.. akenis General Discussion 1 10-20-2005 02:29 PM
Building new system... setup hints? davin Hardware Support 4 01-24-2005 09:11 PM


All times are GMT -6. The time now is 03:24 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.