Metadata Tools 4.0 Beta Thread (aka BMT 4.0)

[PiX64]

Stuckless,

I posted a few days back regarding some strange metadata issues that were happening whith new recordings and such. you asked me to send u a list of my recordings and my properties files so you could look for something funny going on with BMT.

I did not do this because i couldn't find a quick and easy way to do this, but I did spend sometime trouble shooting on my own, and here is what i found.

*I am not using the auto plugin

The issue
-------------
New recording comes in, say Deadliest Catch. No properties file is created since i don't have BMT running auto. Deadliest catch however is give information for Merlin on syfy. Throught he WEB ui if i go into the Merlin folder, it will show Deadliest Catch as the name, some title for Merlin as the show title and the Merlin Box Art. When i go into this file and modify/search for appropriate metadata it will find it and save it. HOWEVER it will also apply this found metadata to all other shows in the sub folder.

So before i had say 5 merlin and 1 deadliest catch....if i update deadliest catch all 5 merlin and 1 deadliest catch will now show up as described int he first paragraph under issue....except they will be in the Deadliest Catch folder.

Where it gets really confusing is when you have more than 2 shows in a subfolder due to the issue described above. So say The Office, Parenthood, The Big Bang Theaory are all missing properties files and end up in the Deadliest catch folder... It quickly becomes a headache

My Findings
----------------
I found that doing the above approach obviously didn't work. So i went itno my recordings folders and wanted to see what exactly wat happening. I found that when i was creating a properties file through BMT it was no correctly populating the metadata into the file. I also noticed that i have MANY old propreties files hanging out in my sage recording folders...which corresponded to the "DEFAULT" poser art, and metadata i was seeing for newly recorded/watched shows.

What I did was stopped sage service, removed all old unused properties files, and then restarted sage. I then entered BMT and was able to see the same issue as described above...however this time i was able to correctly save properties files and get everything straightened out.

When a new recording comes in We shall see where it is mysteriously assinged, but i suspect that it will not show up correctly with the Default SAGE TV info until i am able to assign the correct info via BMT

Any thoughts how why old .properties files may be causing this issue?
Also is there any way to tell Sage to autodelete .properties files when the recordings are delete?

~Pix64

[stuckless]

@Pix64 - That is some good troubleshooting The interesting thing here is the BMT is not directly responsible for the cross contimination, since you have removed the automatic plugin. I have wondered if somehow bmt was reading the wrong properties, but I couldn't see where that was happening.

I'm still at a loss to explain why this happens, but keep an eye on those .properties in your recordings dir... I'm curious what the names of those properties were?

I'll also try to see if I replicate this by dumping lots of different .properties in my recordings dir as well.

[PiX64]

Thanks...

So another recording just came in which was previously getting assigned default metadata...and this time...all good.

It showed up by itself with the approriate poster art and such because it recognized the name. When i did a search for hte metadata it found it saved it created properties file, and place it in the correct subfolder...

Maybe the old prop files is the key....

The names of the properties files were

the SageTV recorded file name so DeadliestCatch######.mpg.properties

~Pix64

[broconne]

Stuckless -
I noticed the following in my phoneix.log when BMT was running and SageTV was not being notified of an update. Not sure if it related - but figured I would post it here just in case:

2010-04-28 21:58:11,972 [main] WARN  sagex.phoenix.h2.DatabaseManager - There was an error querying the Phoenix database.  Something is not right.
java.sql.SQLException: Table "MEDIA" not found; SQL statement:
select count(mediafileid) from media [42102-128] Query: select count(mediafileid) from media Parameters: []
	at org.apache.commons.dbutils.QueryRunner.rethrow(QueryRunner.java:542)
	at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:399)
	at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:425)
	at sagex.phoenix.h2.DatabaseManager.init(DatabaseManager.java:87)
	at sagex.phoenix.Phoenix.init(Phoenix.java:88)
	at sagex.phoenix.Phoenix.getInstance(Phoenix.java:47)
	at phoenix.PluginAPI.<init>(PluginAPI.java:21)
	at phoenix.api.<clinit>(api.java:1215)
	at bmt.BMT.init(BMT.java:26)
	at bmt.api.<clinit>(api.java:20)
	at org.jdna.metadataupdater.MetadataUpdater.logMetadataEnvironment(MetadataUpdater.java:165)
	at org.jdna.metadataupdater.MetadataUpdater.run(MetadataUpdater.java:135)
	at org.jdna.metadataupdater.MetadataUpdater.main(MetadataUpdater.java:115)
2010-04-28 21:58:11,992 [main] INFO  sagex.phoenix.util.TaskManager - Adding Scheduled Task: urlcachecleaner; Date: Wed Apr 28 21:58:11 EDT 2010; Repeat: 86400000

[stuckless]

@broconne - That's not an issue... yet Basically when "phoenix" is loaded by bmt, it doesn't have a valid phoenix database, and it logs the error. It doesn't affect anything at this point. (BTW... i accidentally clicked "edit" on your post, but I didn't actually edit anything)

[bialio]

Well, the '2' directory that has been showing up has now disappeared. I upgraded a number of my WHS discs - replaced the 500 GB with 1.5TB - and the disc that it thought '2' was on is no longer in the machine. So who knows.....

[PiX64]

Stuckless... found some more interesting info when continuing to look into the phantom metadata issue we were discussing earlier this week.

So lets say 5 shows record right. They will all have their own file name, and Episode Title, but the Show Title will all be the same. for example

Lets say here are the 5 recordings:

1) The Office
2) Parks and Recreation
3) The Mentalist
4) Fringe
5) Gangland

What will happen is the last metadata scan i did manually is where these 5 shows will show up. So if the last thing i scanned in was American Idol, all of these 5 shows would show up under the American Idol folder in BMT and each have American idol as its show name, with its own epsiode title which is actually correct so i would see American Idol - {office episode title}

Now what is really strange is when i find metadata on a single one of the above 5 shows and save, ALL 5 SHOWS will be moved to that particular new show title. So if i searched for The Office, shows 1 - 5 above will now be moved to the "The Office" folder in BMT.

It gets stranger...

If i follow these shows around to their different folders as i rename them. The metadata inside of BMT will appear as if it is incorrect...however when shutdown sage, remoave any OLD properties file which are no longer associated with valid recordings, restart sage...viola...everythig is correct, and the above 5 shows will show up with the correct metadata i fetched in previous steps, and show up in ther own folders as they should...

Hope that helps you at least a little in maybe thinking about what could be causing such a strange issue...

~Pix64

[stuckless]

Originally Posted by PiX64

Stuckless... found some more interesting info when continuing to look into the phantom metadata issue we were discussing earlier this week.

So lets say 5 shows record right. They will all have their own file name, and Episode Title, but the Show Title will all be the same. for example

Lets say here are the 5 recordings:

1) The Office
2) Parks and Recreation
3) The Mentalist
4) Fringe
5) Gangland

What will happen is the last metadata scan i did manually is where these 5 shows will show up. So if the last thing i scanned in was American Idol, all of these 5 shows would show up under the American Idol folder in BMT and each have American idol as its show name, with its own epsiode title which is actually correct so i would see American Idol - {office episode title}

Now what is really strange is when i find metadata on a single one of the above 5 shows and save, ALL 5 SHOWS will be moved to that particular new show title. So if i searched for The Office, shows 1 - 5 above will now be moved to the "The Office" folder in BMT.

It gets stranger...

If i follow these shows around to their different folders as i rename them. The metadata inside of BMT will appear as if it is incorrect...however when shutdown sage, remoave any OLD properties file which are no longer associated with valid recordings, restart sage...viola...everythig is correct, and the above 5 shows will show up with the correct metadata i fetched in previous steps, and show up in ther own folders as they should...

Hope that helps you at least a little in maybe thinking about what could be causing such a strange issue...

~Pix64

Those are good steps... I'm actually in the process of setting up a testing environment with a tuner, so hopefully I'll be able to reproduce some of thise stuff.... the key, for me, is being able to reproduce it consistently on my own hardware... if that happens, I'll be able to find the issue and fix it.

I just want to clarify... you are not running the automatic plugin for any of these cases, is that correct?

[PiX64]

100% correct. the status page says automatic plugin enabled server YES, but none of the boxes on the auto plugin configuration page are checked..

[toricred]

Is anybody else still having problems during prime time with BMT not getting metadata using the automatic plugin?

[broconne]

Not sure if this belongs in the TVE thread or here..

I am using BMT 4.0 and am not seeing Meta-Data in TVE for the "the office". I have created a "MediaTitles.xml" which is in my "C:\Program Files\SageTV\SageTV\scrapers" directory.

The file contains the following:

<titles>
	<!--
	A Match can contain a regex that will be used to match against the path.  If it matches, then it will
	then set the fields, title, year, or id, into the search query.
	-->
	<match>
		<regex>[\\/]TheOffice-</regex>
		<title>The Office (US)</title>
		<metadata type="tv" name="tvdb">73244</metadata>
</match> 
	
</titles>

I am running BMT from inside of SJQ and get the following output:

Thu Apr 29 21:13:04 EDT 2010: Starting task with priority 1/1/10
Executing command line: "C:/Program Files/SageTV/SageTV/MBT.bat" "E:\TheOffice-BodyLanguage-65400-0.mpg"
INFO - Configured Root Logger
INFO - Configured Logging for: bmt using file: bmt.log4j.properties
INFO - Configured Logging for: sagex-api using file: sagex-api.log4j.properties
Batch MetaData Tools (4.0-beta21)
WARN - Logger was already configured: bmt; Skipping.
INFO - Configured Logging for: phoenix using file: phoenix.log4j.properties
Processing Media Files...
Processing: TheOffice-BodyLanguage-65400-0.mpg
         Updated: 1 items
Failed to Update: 0 items
Notifying Sage to Refresh Imported Media

The output leads me to believe it worked and I get a .properties file that matches the mpg. Is there another step that needs to be taken for TVE to see the metadata or is it possible I missed a step in my BMT settings somewhere?

[razrsharpe]

Originally Posted by broconne

The output leads me to believe it worked and I get a .properties file that matches the mpg. Is there another step that needs to be taken for TVE to see the metadata or is it possible I missed a step in my BMT settings somewhere?

What's the BMT webui show (screenshot would be nice) and what does the .properties file say (copy it)? If the BMT webui is showing the correct information then TVE (or all of sage for that matter) should be able to see the new metadata...

[stuckless]

Originally Posted by broconne

Not sure if this belongs in the TVE thread or here..

I am using BMT 4.0 and am not seeing Meta-Data in TVE for the "the office". I have created a "MediaTitles.xml" which is in my "C:\Program Files\SageTV\SageTV\scrapers" directory.

The output leads me to believe it worked and I get a .properties file that matches the mpg. Is there another step that needs to be taken for TVE to see the metadata or is it possible I missed a step in my BMT settings somewhere?

Originally Posted by razrsharpe

What's the BMT webui show (screenshot would be nice) and what does the .properties file say (copy it)? If the BMT webui is showing the correct information then TVE (or all of sage for that matter) should be able to see the new metadata...

What is suspect is happening is that SageTV is not "noticing" the new metadata. It may be that the "Notifying SageTV" is not really doing anything. Also does the script pass the touch arg.

[heffneil]

Stuckless,

I am still getting data in the root of my movies folder and I get the background graphic and cover graphic but I am not getting description data. the .properties file in the root for that movie does have the description...

Any ideas?

Neil

[broconne]

Here is the data requested by razrsharpe and stuckless:

BMT Screenshot:


Properties File:

#Generator: Batch Metadata Tools (4.0-beta21); MediaFile: E:\TheOffice-BodyLanguage-65400-0.mpg
#Thu Apr 29 21:13:12 EDT 2010
Actor=Steve Carell -- Michael Scott;\nJohn Krasinski -- Jim Halpert;\nJenna Fischer -- Pam Beesly;\nRainn Wilson -- Dwight Schrute;\nCraig Robinson -- Darryl Philbin;\nDavid Denman -- Roy Anderson;\nPaul Lieberstein -- Toby Flenderson;\nMelora Hardin -- Jan Levinson-Gould;\nCreed Bratton -- Creed Bratton;\nMindy Kaling -- Kelly Kapoor;\nKate Flannery -- Meredith Palmer;\nEd Helms -- Andy Bernard;\nPhyllis Smith -- Phyllis Lapin;\nAngela Kinsey -- Angela Martin;\nB.J. Novak -- Ryan Howard;\nOscar Nu\u00F1ez -- Oscar Martinez;\nBrian Baumgartner -- Kevin Malone;\nLeslie David Baker -- Stanley Hudson;\nEllie Kemper -- Kelly Erin Hannon;
Description=Pam and Jim work on their first sales pitch together to Donna, the manager of a local restaurant. But Michael keeps misreading the signals she's putting out. Meanwhile, Dwight encourages Kelly to try out for the minority training program.
EpisodeNumber=22
EpisodeTitle=Body Language
Genre=Comedy
MediaProviderDataID=73244
MediaTitle=The Office (US)
MediaType=TV
OriginalAirDate=2010-04-29
Rated=TV-14
RunningTime=1800000
SeasonNumber=6
Title=The Office (US) - S06E22 - Body Language
Year=2010
x-MetadataProviderId=tvdb
x-ser-Cast=0|Steve Carell|Michael Scott|null;0|John Krasinski|Jim Halpert|null;0|Jenna Fischer|Pam Beesly|null;0|Rainn Wilson|Dwight Schrute|null;0|Craig Robinson|Darryl Philbin|null;0|David Denman|Roy Anderson|null;0|Paul Lieberstein|Toby Flenderson|null;0|Melora Hardin|Jan Levinson-Gould|null;0|Creed Bratton|Creed Bratton|null;0|Mindy Kaling|Kelly Kapoor|null;0|Kate Flannery|Meredith Palmer|null;0|Ed Helms|Andy Bernard|null;0|Phyllis Smith|Phyllis Lapin|null;0|Angela Kinsey|Angela Martin|null;0|B.J. Novak|Ryan Howard|null;0|Oscar Nu\u00F1ez|Oscar Martinez|null;0|Brian Baumgartner|Kevin Malone|null;0|Leslie David Baker|Stanley Hudson|null;0|Ellie Kemper|Kelly Erin Hannon|null
x-ser-Genres=Comedy

Script:

@echo off
cd c:\Program Files\SageTV\SageTV
java.exe -jar Metadatatool.jar --auto=true --prompt=false --touch --notifySageTV --overwrite %1

Thanks for all the help! These two pieces of software are just awesome.

[can3gxw]

Hey Sean, been a while since I've been able to get back to going through BMT... I noticed something this morning while I was rebuilding my library and adding a crap ton of shows to my lists... when I look at the WebUI and browse to the ALL TV and go down through, I notice that not all of the shows are sorted correctly. For example, my "Law and Order SVU" is out of "sorts", per se.. LOL - take a look at the screen shot attached. Looking at the data from left to right, then row 2, row 3, and so on....

Also:
I have one folder for each season - in order to make it all work and display in order in SageMC, I had to rename the folders "Season 01", "Season 02", and so on because otherwise SageMC would list them as "Season 1", "Season 10", "Season 11", "Season 2", etc...

Well - BMT is showing the season folders "out of order" - like it's stripping the "0" from within the season folder name. I've attached a screen shot of that.. not sure if there is something that I'm missing in a setting or not.

[razrsharpe]

@broconne - Sean I am sure has more insight (because i only have an observation)... but the property file and webui do not match. The webui is what sage has in its db. So for some reason it looks like bmt (or sage) is not updating the mediafile's metadata with the property file...

[stuckless]

@heffneil - that seems very odd... fanart can't actually be downloaded unless there is valid metadata. I'm not sure what is happening here.

@broconne - as razrsharpe pointed out... the metadata is not being picked up by sage. There are 2 possible reasons for this. 1 - the notify sagetv failed (if it did, then you should be able to force the pickup by using he refresh library command), or 2 - the timestamp on the mediafile is not being set for some reason (ie, read-only mediafiles??)

@can3gxw - i'm pretty sure the vfs does strip the leading 0, but I need to build a better "sorter" to sort this out The current sorter is just an alphanumeric sorter, but I need to build a "season" specific sorter that will sort the foldername by taking into account the season number.

Unfortunately, may will be a bmt free month for me. ie, no updates. I have a few other non-sage related projects that need my attention, and so I've been doing nothing on this for a couple of weeks now. Hopefully June/July I'll have some more updates and I'll probably take it out of beta at that time.

[WarlordBB]

I'm new to BMT so please forgive me if what I'm having a problem with is elementary to some of you and I have tried to answer this myself via search.

I am having a HELL of a time with the TV series - V (2009). I've tried every way I can think of to get it to identify my show my imported videos of the episodes as the 2009 V but it insists on identifying it as the 1984 TV show.

I've gone to each episode I have via the Web UI and gone to Find Metadata and searched for and selected the 2009 version. I've had it recognize that and work great. Then the next time I do a scan of any kind, it changes back to 1984 version. I even see two sets of Fanart in the folders, one for 2009 and one for 1984 but no matter what I try, it always changes back to 1984.

I'm using Sage Movie Wall so getting the cheesy 1984 background is just frustrating not to mention that episode names are wrong and such.

Next I tried renaming the example \SageTV\scrapers\MediaTitles.xml file so it would be "live" and then adding an entry:

<match>
	<regex>[\\/]V\s\-\s\(2009\)[\\/]</regex>
	<title>V</title>
	<year>2009</year>
	<metadata type="tv" name="tvdb">94971</metadata>
</match>

(I have my episodes stored like \Sage\Imported\V (2009)\episodefiles.avi)

Each episode is named like V - S01E01.avi

I'm not the best at regex so maybe that's the problem...

At this point, I have no clue

Any ideas would be appreciated and I'm using 4.0 beta 21 BTW.

[heffneil]

I am getting fanartc but I only have my properties file in the root of the directory where the movies reside but not in the same directory of the .vob file...

I do believe this is a bug. Stuckless you are the developer running the show here? Anything you want from me to get this fixed?

Thanks,

Neil