SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations > Batch Metadata Tools

Notices

Batch Metadata Tools This forums is for discussing the user-created Batch Metadata Tools for SageTV.

Closed Thread
 
Thread Tools Search this Thread Display Modes
  #1  
Old 01-01-2011, 08:38 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
BMT 4.9.6 Released (Misc Fixes)

Fixes
Quote:
262 Defect Completed Medium Release4.9.6 sean.stuckless WebUI: Scrolling handled by browser window BMTWEB BMT
263 Defect Completed Medium Release4.9.6 sean.stuckless WebUI: Notifications are handled by notification manager BMTWEB BMT
265 Defect Completed Medium Release4.9.6 sean.stuckless WebUI: Can't update Movie Title BMTWEB BMT
266 Defect Completed Medium Release4.9.6 sean.stuckless Fix SageAiring-NoEpisode scraper Phoenix
267 Defect Completed Medium Release4.9.6 sean.stuckless Add Movie.Year scraper Phoenix
269 Defect Completed Medium Release4.9.6 sean.stuckless Ability to set a default poster BMTWEB BMT
270 Defect Completed Medium Release4.9.6 sean.stuckless Some Movies showing S00E00 BMT
273 Defect Completed Medium Release4.9.6 sean.stuckless BMT: CommandLine: Throws classnotfound exception BMT
274 Defect Completed Medium Release4.9.6 sean.stuckless Refresh MediaTitles broken from the web ui BMTWEB BMT
276 Defect Completed Medium Release4.9.6 sean.stuckless First time dialog shows every time BMTWEB BMT
277 Defect Completed Medium Release4.9.6 sean.stuckless Title Scraping Tweaks BMT
The scrolling fix should make it work better on the ipad or other small screen devices.

The Set Default Poster bug requires the addition of 3 new custom metadata fields. These fields should be added automatically when bmt starts, but if they are not, then you can add them using the "Tools -> Fix Custom Metadata Fields" menu option.
  #2  
Old 01-01-2011, 08:45 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Well, Already I can see that the about box is happing all the time. This is fixed locally, but for some reason, when I installed this on my server, it's still popping up
  #3  
Old 01-01-2011, 09:16 PM
JREkiwi's Avatar
JREkiwi JREkiwi is offline
Sage Icon
 
Join Date: Jan 2005
Location: Auckland, New Zealand
Posts: 2,123
Thanks for the update

While you're tweaking things, there's a couple of typo's in the Tools dropdown.

"Import as Recoding"
"Un-import as Recoding"

John
  #4  
Old 01-02-2011, 07:36 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Quote:
Originally Posted by JREkiwi View Post
While you're tweaking things, there's a couple of typo's in the Tools dropdown.

"Import as Recoding"
"Un-import as Recoding"
Thanks. I've fixed those for the next build.
  #5  
Old 01-02-2011, 01:29 PM
ohpleaseno ohpleaseno is offline
Sage Aficionado
 
Join Date: Apr 2009
Location: South Florida
Posts: 464
Is

Code:
266	 Defect	 Completed	 Medium	 Release4.9.6	 sean.stuckless		 Fix SageAiring-NoEpisode scraper Phoenix
less aggressive now?
__________________
Server: WHS with 11TB of storage
Clients:1) HD-200 to a Samsung 32" Slimfit CRT HDTV 2) HD-200 to a Pioneer 26" X1 LCD
Tuners: 1) HD-Homerun 2) Avermedia USB ATSC
Twitter: ohpleaseno
  #6  
Old 01-02-2011, 02:07 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Quote:
Originally Posted by ohpleaseno View Post
Is

Code:
266	 Defect	 Completed	 Medium	 Release4.9.6	 sean.stuckless		 Fix SageAiring-NoEpisode scraper Phoenix
less aggressive now?
Yeah, it's now more specific. Before it was hitting on "File YYY-MM-DD" as well because the pattern was kind of similar to "Show-Episode-#####-##". Depending on the order the scrapers were called, then sometimes it would incorrectly get a hit.
  #7  
Old 01-03-2011, 09:44 AM
dstanley's Avatar
dstanley dstanley is offline
Sage Aficionado
 
Join Date: Sep 2008
Location: LaHave, Nova Scotia, Canada
Posts: 291
Works awesome on my iPad now - Thanks!
About box coming up every time for me too ...
__________________
SERVER: Intel Core i7-3770 CPU @ 3.40GHz - UnRaid Server 6.8.0/DVB LibreELEC with Docker stuckless-sagetv-server-java9 MOTHERBOARD: ASRock-Z77 Extreme4 / 32GB RAM CACHE DRIVE: SSD 1TB ARRAY: 24TB/7 DRIVES PARITY: 8TB
EXTERNAL DEVICES: 4x HD-PVR's Firewire Channel Changing
TUNERS:NO LONGER RECORDING LIVETV
CLIENTS: 2xHD300 2xHD200 SONY ANDROIDTV MINI-CLIENT
  #8  
Old 01-03-2011, 09:52 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Quote:
Originally Posted by dstanley View Post
Works awesome on my iPad now - Thanks!
About box coming up every time for me too ...
I'm going to try for a release between now and the weekend. The about box is a quick fix, but I didn't really feel like doing a full release just for that
  #9  
Old 01-03-2011, 08:50 PM
Gustovier Gustovier is offline
Sage Aficionado
 
Join Date: Nov 2010
Location: Chicago
Posts: 460
What's the exact process to update the default poster now?


I click on the poster and it brings up the popup to allow me to pick a new a default, I do it. The new poster still doesn't seem to persist though. Do I have to refresh the library afterwards? I think I'm prob missing something simple here.
  #10  
Old 01-04-2011, 08:21 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Quote:
Originally Posted by Gustovier View Post
What's the exact process to update the default poster now?


I click on the poster and it brings up the popup to allow me to pick a new a default, I do it. The new poster still doesn't seem to persist though. Do I have to refresh the library afterwards? I think I'm prob missing something simple here.
You might not see this change in the web ui right away, since things are cached there. But if you update the poster, and then close out of the browser, and then browse the folder, you should see you new poster being used there. (I do, and that's how I test it). In the STV, when you connect again, if you are using a plugin that uses phoenix fanart, then you'll see the default poster being used there as well. (I use bailio's fanart plugin, and it seems to be working from what I can tell).

If you want to see if anything is being updated, then you can check the raw metadata options for the item, and see if DefaultPoster has a value stored in it. To check the raw metadata, browse the folder, and then click the sprocket icon on the item which opens a metadata dialog. From the drop down, select custom metadata fields, and look for DefaultPoster.
  #11  
Old 01-08-2011, 09:56 AM
JerryB JerryB is offline
Sage Aficionado
 
Join Date: Aug 2008
Posts: 273
Problem permanently storing data.

I'm having a problem and hope someone knows what's causing it and can point me to a solution.

If I go into BMT and select missing metadata I get a listing of several episodes from different TV shows that I've recorded. I do a search and locate and save the metadata without any apparent problem. I know the metadata is saved since if I repeat the missing metadata search in BMT, these TV shows are no longer listed. The problem is, if I wait a couple of hours and then go back into BMT, the metadata that I previously saved is gone and the shows are again listed as missing metadata. Obviously something is deleting or over writting the metadata but I'm stumped as to what it could be.
  #12  
Old 01-08-2011, 10:09 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Jerry, another user reported a similar problem. I'm fairly certain that BMT is not the cause, since BMT will not write empty values for metadata. ie, bmt will not clear metadata fields, and this has come up as an issue as well, since sometimes people want to clear fields, but bmt doesn't allow for it.

I'm starting to suspect either another plugin... or even core issue. So, to test this, you'll need to do the following...

Remove ALL plugins except for bmtweb (and the plugins that it depends on). ie, you should have BMTweb and phoenix-core, phoenix-api, etc. I only know of a couple of other plugins that also fetch metadata and they are Ortus and SMM. So make sure at least that you remove those plugins (for now).

Once that is done, restart sagetv, go to the web ui, and then fetch metadata for the shows. Wait and see if it disappears.

If it does disappear, then you'll need to perform the same steps as above, except after fetching metadata for the shows, you'll need to then remove bmt as well... and restart. Wait and see if the metadata disappears.

If it does, then you can probably file a core sagetv issue, because it would appear that sagetv is removing the metadata for some reason.

Also check to see if you have and Wiz.bin.corrupt files, because it may be that for some reason the wiz.bin is getting corrupted and then sagetv is restoring the backup file making it look like that something is overwriting the metadata. Again, if this is happening, you can probably file a bug with SageTV and see if they can look at your .corrupt Wiz.bin to see what is corrupting it.

This may be bmt issue, but I've never seen it myself, and given there are over 1000 bmt users, I think i'd see more reports of this. Personally, I suspect another plugin is updating the metadata, but not checking if the metadata settings are blank or not.
  #13  
Old 01-08-2011, 12:01 PM
JerryB JerryB is offline
Sage Aficionado
 
Join Date: Aug 2008
Posts: 273
Stuckless, thanks for the suggestions. Just to be clear, I never suspected the problem was cause by BMT since BMT was finding and, at least trying to save the metadata. I was just looking for some guidance on what else could either be affecting BMT's ability to save the data or overwriting the data after BMT saved it. I've now solved the problem, at least with respect to the TV episodes that were giving me trouble and suspect it was caused by some sort of wiz.bin corruption.

Here is what I did that resolved the problem:
1) Moved the affect files to a different (non-sage related) directory on my hard drive.
2) Did a "scan imported media" in Sage so that it would see that the files were no longer there.
3) Went into the WebServer and did a search for the series title. The episodes for the files I moved appear in this search with a 0 byte file size. I deleted all of these 0 byte records in the WebServer interface.
4) Exited the WebServer and copied the files back to their original Sage directory.
5) Repeated "scan imported media". Immediately after completing this scan, all the episodes reappeared and this time they had all their metadata. I didn't even have to go back into BMT to relocate and resave the metadata. At least so far, this metadata has not disappeared.

Since everything appears to be fixed, I'm not sure filing a report with Sage would accomplish anything at this point. However, I will file one if the problem reappears, which would suggest ongoing corruption of specific show information in wiz.bin.

PS: Just for the record, I don't use SMM or Ortus so those plugins weren't installed and cannot be involved in this.

Last edited by JerryB; 01-08-2011 at 12:05 PM. Reason: Add infomation
  #14  
Old 01-08-2011, 10:05 PM
adone36 adone36 is offline
Sage Advanced User
 
Join Date: Nov 2007
Posts: 142
Still have 3 problems with BMT.

1) Exclude still does not work. I constantly get new files added to excluded directories with metadata. Ex: If I have an interview file named "Amy 1234", It shows up as "Amy" in Sage with genre "Comedy", rating "NR", category "Short".
Problem #2 makes this doubly bad.

2) BMT renames the title in Sage. BMT should NEVER put a title in because you have a hard time finding it. For example, If I have "The Descent II", BMT decides "The Descent" is the closest match and changes the title in Sage to "The Descent". All of the metadata is "The Descent". In you listings if you mouse over, you get the cover for "The Descent" and all the info matches. Go to Video details you get the file is "\\videos\The Descent II" which you'd never find unless you knew you didn't have "The Descent"

For TV, I now have multiple listings for "The Late Late Show with Craig Ferguson" and "The Late Late Show With Craig Ferguson".

BMT should never change the title because it is one of the ways you find errors.

3) BMT should not overwrite metadata from Sage recordings. There are numerous posts here about double episodes, etc that BMT cannot accommodate. BMT should only supply metadata for TV IMPORTS. Or at least have an option "Overwrite TV recording metadata".

I wish BMT would leave EPG recordings alone completely.
  #15  
Old 01-09-2011, 06:46 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Quote:
Originally Posted by adone36 View Post
1) Exclude still does not work. I constantly get new files added to excluded directories with metadata. Ex: If I have an interview file named "Amy 1234", It shows up as "Amy" in Sage with genre "Comedy", rating "NR", category "Short".
Problem #2 makes this doubly bad.
Are you SURE that
a. BMT is updating the file. Ie, does it appear in the phoenix-metadata.log as being updated by BMT
b. Your exclude filter is set correctly? (did you read the manual, did you test the filter, etc)

Quote:
2) BMT renames the title in Sage. BMT should NEVER put a title in because you have a hard time finding it. For example, If I have "The Descent II", BMT decides "The Descent" is the closest match and changes the title in Sage to "The Descent". All of the metadata is "The Descent". In you listings if you mouse over, you get the cover for "The Descent" and all the info matches. Go to Video details you get the file is "\\videos\The Descent II" which you'd never find unless you knew you didn't have "The Descent"
Are you talking about recordings or just imported movies? I mean the whole purpose of BMT is to adjust titles??? Did you read the manual? BMT uses fuzzy logic to attempt to get a best match. If you are not comfortable with that, then my suggestion is to turn off the automatic updating and simply use the manual method of updating. This is not going to change.

Quote:
For TV, I now have multiple listings for "The Late Late Show with Craig Ferguson" and "The Late Late Show With Craig Ferguson".

BMT should never change the title because it is one of the ways you find errors.
Again, is this a sagetv recording or something that imported??

Quote:
3) BMT should not overwrite metadata from Sage recordings. There are numerous posts here about double episodes, etc that BMT cannot accommodate. BMT should only supply metadata for TV IMPORTS. Or at least have an option "Overwrite TV recording metadata".

I wish BMT would leave EPG recordings alone completely.
Have you even read the user guide?

I have to say that you comments feel more like a frustrated attack on bmt than anything constructive. It doesn't appear that you've read the user manual, and you've provided no real information that I can use to help you out. There very well may be bugs in bmt that are affecting your experience, but I can't tell because you've ranted rather than provided any real details.

If you'd like help here, then at least provide some details around what is happening other than it just doesn't work or BMT SHOULD NOT do, etc...

For #1...
What directories have you excluded.
Can you provide filenames
Can you provide the exclude filter
DId you test the filter
Do the items show up in the missing metadata view

For #2
Well, that what bmt does??? There are ways that you can name your files so that bmt will have better results. That's in the manual as well.. but for starters, you can add the year to the filename... that way bmt will choose the best result that also includes a year.

For #3
Well, that's covered in the manual, but short answer is that bmt provides that option, and it's called, "Preserve Original Recording Metadata", and it's enabled by default. Maybe there's a bug here. If you provided examples, with show names, etc, then maybe I can help.

Here's a link to the user guide, I think it might help.

Last edited by stuckless; 01-09-2011 at 06:49 AM.
  #16  
Old 01-09-2011, 12:45 PM
adone36 adone36 is offline
Sage Advanced User
 
Join Date: Nov 2007
Posts: 142
I am not attacking BMT. I've used it for quite some time now. I thought the purpose of BMT was to automatically d/l metadata for imports because you have NOTHING from Sage. This make BMT a dramatic enhancement to Sage. I was not ranting. I think I was suggesting a significant issue

When Sage records TV shows I get elaborate metadata from the EPG from Tribune. I don't find the addition of a poster art thumbnail as opposed to the screen capture thumbnail as being worth the possible problems. If there was an "Overwrite existing Sage Recording metadata option", I wouldn't check it.

IMPORTS is a whole other story. IF I d/l "Survivors - S02E01", of course, BMT is a godsend in supplying metadata, but the one thing it should never change is the title. How would you ever know if there was a problem when BMT chooses the metadata and then changes the title to go along with it??

I think the great thing BMT does is to provide this metadata for shows I don't get from my provider, such as the British "Survivors".
Now, I think you are looking at it from the perspective of say missing a recording of the "The Big Bang Theory" and then d/l it and adding it to your recordings via Sage import directories. So "Big Bang Theory - S03E09" becomes and is added to the recordings as "The Big Bang Theory", etc, etc with all the resulting metadata. BMT thus "fixes" it for you. I would prefer to fix the title myself rather than have the choice made for me.

So maybe it's me, but I NEVER find Sage making a mistake in recordings, etc so 99.9% of the time BMT is fabulous when it supplies metadata for imports. When it changes existing metadata on recorded shows automatically, it can introduce problems.

Movie imports are all a blank slate, so if BMT chooses the metadata and then changes the displayed title to go with it, you do not have a list where you see "Big Bang Theory" and "The Big Bang Theory" shown and know to fix the first one, you just have an incorrect listing and no way to find it unless you click play on it one day.

This may be fine if someone has 20 tv shows and 10 movies, but I have thousands.

The exclude problem may stem from the exclude bug from before when I had dozens of incorrect metadata. If I move these files around, I still get metadata for a few of them (which were moved but not renamed) upon rescan. Perhaps using the same name is causing Sage to retrieve the data from somewhere. I'll have to experiment.
  #17  
Old 01-09-2011, 03:04 PM
gplasky's Avatar
gplasky gplasky is offline
SageTVaholic
 
Join Date: Jul 2003
Location: Howell, MI
Posts: 9,203
Quote:
Originally Posted by adone36 View Post
When Sage records TV shows I get elaborate metadata from the EPG from Tribune. I don't find the addition of a poster art thumbnail as opposed to the screen capture thumbnail as being worth the possible problems. If there was an "Overwrite existing Sage Recording metadata option", I wouldn't check it.
Check the manual again and see the setting "Preserve original metadata for Recordings". Check it if it isn't already.



Gerry
__________________
Big Gerr
_______
Server - WHS 2011: Sage 7.1.9 - 1 x HD Prime and 2 x HDHomeRun - Intel Atom D525 1.6 GHz, Acer Easystore, RAM 4 GB, 4 x 2TB hotswap drives, 1 x 2TB USB ext Clients: 2 x PC Clients, 1 x HD300, 2 x HD-200, 1 x HD-100 DEV Client: Win 7 Ultimate 64 bit - AMD 64 x2 6000+, Gigabyte GA-MA790GP-DS4H MB, RAM 4GB, HD OS:500GB, DATA:1 x 500GB, Pace RGN STB.
  #18  
Old 01-09-2011, 05:18 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
Quote:
Originally Posted by adone36 View Post
I am not attacking BMT. I've used it for quite some time now. I thought the purpose of BMT was to automatically d/l metadata for imports because you have NOTHING from Sage. This make BMT a dramatic enhancement to Sage. I was not ranting. I think I was suggesting a significant issue

When Sage records TV shows I get elaborate metadata from the EPG from Tribune. I don't find the addition of a poster art thumbnail as opposed to the screen capture thumbnail as being worth the possible problems. If there was an "Overwrite existing Sage Recording metadata option", I wouldn't check it.

IMPORTS is a whole other story. IF I d/l "Survivors - S02E01", of course, BMT is a godsend in supplying metadata, but the one thing it should never change is the title. How would you ever know if there was a problem when BMT chooses the metadata and then changes the title to go along with it??

I think the great thing BMT does is to provide this metadata for shows I don't get from my provider, such as the British "Survivors".
Now, I think you are looking at it from the perspective of say missing a recording of the "The Big Bang Theory" and then d/l it and adding it to your recordings via Sage import directories. So "Big Bang Theory - S03E09" becomes and is added to the recordings as "The Big Bang Theory", etc, etc with all the resulting metadata. BMT thus "fixes" it for you. I would prefer to fix the title myself rather than have the choice made for me.

So maybe it's me, but I NEVER find Sage making a mistake in recordings, etc so 99.9% of the time BMT is fabulous when it supplies metadata for imports. When it changes existing metadata on recorded shows automatically, it can introduce problems.

Movie imports are all a blank slate, so if BMT chooses the metadata and then changes the displayed title to go with it, you do not have a list where you see "Big Bang Theory" and "The Big Bang Theory" shown and know to fix the first one, you just have an incorrect listing and no way to find it unless you click play on it one day.

This may be fine if someone has 20 tv shows and 10 movies, but I have thousands.

The exclude problem may stem from the exclude bug from before when I had dozens of incorrect metadata. If I move these files around, I still get metadata for a few of them (which were moved but not renamed) upon rescan. Perhaps using the same name is causing Sage to retrieve the data from somewhere. I'll have to experiment.
Personally, based on the very specific nature of your requirements, I'd recommend turning off the automatic update, and just use the manual update method.

You can also specifically the set the Media Types (you can see it in Gerry's screen) to be a set of values that WILL NOT include TV. That way, BMT won't ever touch a TV file (or a file that it thinks is a TV file).

BMT will always update the the "title" of any file (tv or video) that you import into sagetv. Your file may be called, "Big Bang Theory S01E01.avi", and BMT will set the TITLE, Season and Episode fields accordingly. If you want it to show up as "Big Bang Theory S01E01" in the STV, then use a plugin that formats titles based on the available metadata. The title of the show is "Big Bang Theory", not "Big Bang Theory S01E01", but I can see where you can the season and episode to show up in the display. By making the full title include the season and episode in the metadata, then you can group by title, since you never have a unique title. Maybe grouping for you is not importing, but it is to me, and since I wrote the app, I get to decide.

It sounds as if you are having a very frustrating time with the decisions that bmt makes for automatic processing... I'd just turn it off and then you can make all the decisions on how you want want the metadata named, etc. But you can still have BMT do the lookups manually, but then 'fix' the titles so that they display correctly for you.

FYI, if preserve metadata is enabled, then BMT should not touch any of the core metadata. But a bug was introduced in 4.9.6 where bmt would still update the metadata for a recording even if this setting was enabled. I'm hoping to do a release later tonight, and it will have a fix for that.
  #19  
Old 01-09-2011, 07:59 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,712
This thread is closed. Post responses in the 4.9.7 thread.
Closed Thread


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
BMT 4.9.5 Released (Ratings, and other Goodies) stuckless Batch Metadata Tools 36 12-29-2010 05:18 PM
BMT 4.9.4 Released (Minor Bug Fixes) stuckless Batch Metadata Tools 17 12-15-2010 01:34 PM
BMT 4.8 / Phoenix 2.2 - The Metadata Fixes stuckless Batch Metadata Tools 46 11-13-2010 05:42 AM
BMT 4.9 / Phoenix 2.3 - Bug Fixes stuckless Batch Metadata Tools 1 10-31-2010 06:22 PM
Any fixes for the PVR-350 been released lately? dynamix Hardware Support 12 01-16-2007 06:14 PM


All times are GMT -6. The time now is 05:27 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.