BMT Walkthrough

[reggie14]

Sorry, I have a couple more questions.

First of all, now that BMT is running on my system, what's the best way to get metadata for my existing stuff? Am I suppose to go to the "Browse" section of the web interface and hit the "update metadata" button? Presumably its best to do it a bit more narrowly than that, such as show-by-show, in case it gets some things wrong.

Second, I can't get movies to tag at all. If I hit the "update metadata" button when I have movies on screen, nothing seems to happen. If I hit "find metadata" after clicking on a specific movie, I get a message saying "Unable to get Metadata". Is IMDB integration not working?

Third, I'm still not really sure what to do about tagging TV shows. I knew Firefly was likely to be a problem, and since it only lasted 14 episodes it seemed like a decent test case. When I ripped those episodes several months ago I anticipated the problem, and I thought that if I added the TV episode name that might fix it. Apparently it didn't.

For example, I named the Firefly pilot "Firefly- s01e01- Serenity.mkv" The s01e01 number matches the TV ordering, and the correct ordering for viewing the series, but not the aired ordering (thanks Fox). For the aired ordering, the episode named "The Train Job" is first.

When I had BMT get metadata for Firefly, it decided "Firefly- s01e01- Serenity.mkv" should get paired with "The Train Job" episode instead, since TVDB lists "The Train Job" as episode s01e01 for the aired ordering. It seems like if a TV show has an episode name in the filename, that should take precedent over any episode numbering in the filename, but it apparently doesn't work that way.

I'm not really sure what to do to fix the metadata now. Basically the aired ordering for Firefly is completely messed up. I either need to delete the metadata that it generated, or I need to somehow fix it manually (unless someone adds support for DVD ordering off of TVDB, hint hint ).

So, suppose I manually go through and change the fields for the episode name, description, and air date. Is that enough? I see BMT also populates some other fields, like "Airing ID", "IMDB id", "Metadata id", etc. Some of those extra fields seem to be for the show overall, but others seem to be for the episode. What's the best way to deal with these fields? Even if I wanted to correct them I'm not sure how I could go about doing that.

[stuckless]

Originally Posted by reggie14

First of all, now that BMT is running on my system, what's the best way to get metadata for my existing stuff? Am I suppose to go to the "Browse" section of the web interface and hit the "update metadata" button? Presumably its best to do it a bit more narrowly than that, such as show-by-show, in case it gets some things wrong.

Typically if you have a large collection, you can just use find metadata at the top level folder, and let it go. It will automatically update metadata (some will be wrong). If you are not that gutsy, then you can do it folder by folder, or item by item.

Second, I can't get movies to tag at all. If I hit the "update metadata" button when I have movies on screen, nothing seems to happen. If I hit "find metadata" after clicking on a specific movie, I get a message saying "Unable to get Metadata". Is IMDB integration not working?

I just noticed that as well... Tonight I need to look into this from the webui. I haven't noticed anything from the automatic plugin, but there may be something up with the web ui. (I don't use the web ui very often)

Third, I'm still not really sure what to do about tagging TV shows. I knew Firefly was likely to be a problem, and since it only lasted 14 episodes it seemed like a decent test case. When I ripped those episodes several months ago I anticipated the problem, and I thought that if I added the TV episode name that might fix it. Apparently it didn't.

For example, I named the Firefly pilot "Firefly- s01e01- Serenity.mkv" The s01e01 number matches the TV ordering, and the correct ordering for viewing the series, but not the aired ordering (thanks Fox). For the aired ordering, the episode named "The Train Job" is first.

When I had BMT get metadata for Firefly, it decided "Firefly- s01e01- Serenity.mkv" should get paired with "The Train Job" episode instead, since TVDB lists "The Train Job" as episode s01e01 for the aired ordering. It seems like if a TV show has an episode name in the filename, that should take precedent over any episode numbering in the filename, but it apparently doesn't work that way.

I'd have to totally disagree with that last statement. If a TV show has season and episode specific information then that takes precedent over any titles. It makes sense, since doing fuzzy matching on titles is less accurate than doing a numeric lookup basic on available season and epsisode numbers. If you want to use epsisode names as your lookup, then I suggest using the "Airing" naming format and not the S##E## format.

Firefly-TheTrainJob-000000-0.avi
Firefly-Serenity-000000-0.avi

I'm not really sure what to do to fix the metadata now. Basically the aired ordering for Firefly is completely messed up. I either need to delete the metadata that it generated, or I need to somehow fix it manually (unless someone adds support for DVD ordering off of TVDB, hint hint ).

The good news is that you can quickly remove all the metadata for firefly by simply moving the firefly folder out of your your imports, tell sage to refresh media, and then add it back in, and tell sage to refresh.

So, suppose I manually go through and change the fields for the episode name, description, and air date. Is that enough? I see BMT also populates some other fields, like "Airing ID", "IMDB id", "Metadata id", etc. Some of those extra fields seem to be for the show overall, but others seem to be for the episode. What's the best way to deal with these fields? Even if I wanted to correct them I'm not sure how I could go about doing that.

AiringId and MetadataId are sage internal fields. I show them, only because it helps me. To most people, they mean nothing. IMDBid is a reference field. If it has the wrong data, then that's not an issue. It's not really used by anything.

[reggie14]

Originally Posted by stuckless

I'd have to totally disagree with that last statement. If a TV show has season and episode specific information then that takes precedent over any titles. It makes sense, since doing fuzzy matching on titles is less accurate than doing a numeric lookup basic on available season and epsisode numbers.

I don't really know how good the fuzzy matching usually is, so you''re probably right. But it seems like imported TV shows are going to be really problematic with BMT because there's often no single clear numbering convention for TV shows. Like I said, there are several different things that can happen in a series to screw it up. Just looking at the numbering schemes on IMDB and theTVDB, out-of-order airing (e.g., Firefly), special episodes (e.g., West Wing) can really screw things up, and two-part episodes sometimes do. My thought is, if someone goes to the trouble to look up and insert episode names in the filenames, then those are probably right, even if the numbering is off.

If fuzzy matching is really bad, then you're right and probably the best thing to do is only to look at the s##e## numbers. If its OK, but often unreliable, then ideally the right thing to do might be to look up a show based on the s## e## numbers, but compare the episode title it finds with the title in the filename. If it doesn't match, report it as failed.

Of course, maybe that's just a big waste of time to implement. I don't see a lot of other people reporting problems with imported TV episodes and BMT. Maybe I'm a special case.

If you want to use epsisode names as your lookup, then I suggest using the "Airing" naming format and not the S##E## format.

Firefly-TheTrainJob-000000-0.avi
Firefly-Serenity-000000-0.avi

Ahh, cool! I didn't realize that existed. Is that format primarily intended for pulling up metadata for recorded TV shows?

Originally Posted by stuckless

AiringId and MetadataId are sage internal fields. I show them, only because it helps me. To most people, they mean nothing. IMDBid is a reference field. If it has the wrong data, then that's not an issue. It's not really used by anything.

OK. Thanks. It sounds like I wouldn't have to mess with any of those fields then.

One last question, what does BMT use when looking up TV shows? I thought it was theTVDB, but the IMDB reference field makes me wonder if its using IMDB, and then falls back to TVDB if necessary. I'd like to know which one BMT looks to first so when I rip and name episodes I can make sure I'm naming them consistently.

[stuckless]

Originally Posted by reggie14

If fuzzy matching is really bad, then you're right and probably the best thing to do is only to look at the s##e## numbers. If its OK, but often unreliable, then ideally the right thing to do might be to look up a show based on the s## e## numbers, but compare the episode title it finds with the title in the filename. If it doesn't match, report it as failed.

Of course, maybe that's just a big waste of time to implement. I don't see a lot of other people reporting problems with imported TV episodes and BMT. Maybe I'm a special case.

I don't know if you are a special case or not... but I've imported a few hundred tv shows, at least, probably close to 1000... And I've had very few issues (with the exception of specials).

Ahh, cool! I didn't realize that existed. Is that format primarily intended for pulling up metadata for recorded TV shows?

That was added to do lookups on sage recordings, which only include (if you are lucky), the series and episode names. You'll need to add the -000000-0 to the name, just so that the matcher will know to use the sage recording format when doing the lookup.

One last question, what does BMT use when looking up TV shows? I thought it was theTVDB, but the IMDB reference field makes me wonder if its using IMDB, and then falls back to TVDB if necessary. I'd like to know which one BMT looks to first so when I rip and name episodes I can make sure I'm naming them consistently.

BMT uses tvdb for TV lookups. It does not use IMDB at all for TV. But, tvdb contains an imdb field, and it gets popupated, for reference. Someday it might be useful, but right now any imdb fields that you see are simply reference fields.

[Fuzzy]

Originally Posted by reggie14

Ahh, cool! I didn't realize that existed. Is that format primarily intended for pulling up metadata for recorded TV shows?

That is the default format used by Sage for it's recordings, so it is what BMT is forced to handle. It's not normally the recommended method for imported shows, because it is more prone to error, but it should work in this case (not sure what it will pull up for the 'Serenity' episode though, as theTVDB contains an entry under 'specials' for the Serenity movie.

[reggie14]

Originally Posted by Fuzzy

That is the default format used by Sage for it's recordings, so it is what BMT is forced to handle. It's not normally the recommended method for imported shows, because it is more prone to error, but it should work in this case (not sure what it will pull up for the 'Serenity' episode though, as theTVDB contains an entry under 'specials' for the Serenity movie.

Right. No matter what, dealing with shows Like Firefly is going to be painful. Even if I rename the files I'll still have to go back in and fix the episode numbering on every episode, so that it uses the proper numbering for watching the series, rather than the order it was aired.

Should I also tweak the air dates for each episode so that it puts them in the right order? I'm not sure if plugins and potential future STVs will sort imported TV shows using the episode number from BMT or the original air date.

[reggie14]

Stuckless-

There's no general bug reports thread, so I guess I'll just post here.

A few observations:

1. BMT can’t find the right metadata for TV shows like “24”. For some reason it always goes to Season/Day 1. I can’t even use the “Find Metadata” button to try to semi-manually pick the right one.
2. TMDB seems to have some problems with titles, at least when you’re using the web interface. It can’t find metadata for titles with an apostrophe in them.
3. TMDB also seems to have problems with titles that begin with “The”.

[DirtClodimus]

Originally Posted by reggie14

Stuckless-

There's no general bug reports thread, so I guess I'll just post here.

A few observations:

1. BMT can’t find the right metadata for TV shows like “24”. For some reason it always goes to Season/Day 1. I can’t even use the “Find Metadata” button to try to semi-manually pick the right one.
2. TMDB seems to have some problems with titles, at least when you’re using the web interface. It can’t find metadata for titles with an apostrophe in them.
3. TMDB also seems to have problems with titles that begin with “The”.

I've noticed the 2nd and 3rd points here as well. Frustrating since IMDB isn't working and now TMDB doesn't work for several titles now. Any workarounds?

[stuckless]

Originally Posted by DirtClodimus

I've noticed the 2nd and 3rd points here as well. Frustrating since IMDB isn't working and now TMDB doesn't work for several titles now. Any workarounds?

I'm looking at trying to fix the imdb provider. Unforunately, it's not just a simple fix, I need to actually rewrite the detail provider

[DirtClodimus]

Originally Posted by stuckless

I'm looking at trying to fix the imdb provider. Unforunately, it's not just a simple fix, I need to actually rewrite the detail provider

No worries at all. Your work is very much appreciated!

Any ideas what the issue with tmdb is? I'm fine not fetching from imdb for a while, but was just curious if there way a way I could still fetch data from one of the providers for these titles with apostrophes and ones that start with "The"

Thanks!

[reggie14]

It looks like TMDB is pretty complete. Is there any reason to not to forget about IMDB and focus on support for TMDB?

[stuckless]

Originally Posted by reggie14

It looks like TMDB is pretty complete. Is there any reason to not to forget about IMDB and focus on support for TMDB?

I think my support for tmdb is pretty complete. They have a great API for searching and fetching metadata. But, if you the single quote and 'the' is causing issues, then I'm not sure how I could fix it.

Typically, I've found that imdb has been better for searching and returning valid hits, so I use the imdb provider with tmdb for fanart.

[DirtClodimus]

Originally Posted by stuckless

I think my support for tmdb is pretty complete. They have a great API for searching and fetching metadata. But, if you the single quote and 'the' is causing issues, then I'm not sure how I could fix it.

Typically, I've found that imdb has been better for searching and returning valid hits, so I use the imdb provider with tmdb for fanart.

I found that if I just do a manual update for the metadata and search with the leading "The" or the apostrophe taken out, it gets a result that I can use from tmdb. If that's all I have to do to get metadata for some titles until IMDB is fixed, I'm happy

[reggie14]

Originally Posted by stuckless

I think my support for tmdb is pretty complete. They have a great API for searching and fetching metadata. But, if you the single quote and 'the' is causing issues, then I'm not sure how I could fix it.

I saw some old posts on the TMDB forums about the single-quote vs. apostrophe issue. Apparently some movie titles use a proper apostrophe, while others use a single quote. I thing I always use a single quote in my movie titles. The weird thing is, the TMDB website doesn't seem to have a problem when you search for movie titles using either the apostrophe or the single-quote.

There might be some single workarounds for those two issues. For instance, you could get rid of leading any leadings "The"s in title names. I'm not sure what to do about the apostrophe, but it might work to just get rid of any punctuation marks in titles.

[stuckless]

Originally Posted by reggie14

I saw some old posts on the TMDB forums about the single-quote vs. apostrophe issue. Apparently some movie titles use a proper apostrophe, while others use a single quote. I thing I always use a single quote in my movie titles. The weird thing is, the TMDB website doesn't seem to have a problem when you search for movie titles using either the apostrophe or the single-quote.

There might be some single workarounds for those two issues. For instance, you could get rid of leading any leadings "The"s in title names. I'm not sure what to do about the apostrophe, but it might work to just get rid of any punctuation marks in titles.

That's interesting... Does anyone know (I can google it) what the apostrophe character is? I'll do some testing and see if I can add in some extra tests.

[reggie14]

Originally Posted by stuckless

That's interesting... Does anyone know (I can google it) what the apostrophe character is? I'll do some testing and see if I can add in some extra tests.

Here's a link to a page with the unicode characters for apostrophes, single opening/closing quotes, and backticks, etc.

I'm not really sure what the best thing to do is. I don't really really know how the searching algorithm works, but I still think an easy thing to do might be to just remove punctuation marks. I've tried it on the TVDB website and in some other applications which use the TVDB API, and it doesn't seem to change the search results if you remove punctuation marks.

[Fuzzy]

I'd also say stripping out punctuation would be a most likely solution (and stripping and 'the' prefixes as well)

EDIT: Were't the days of ASCII so much simpler to code in? Unicode, while nice for printing, sucks for parsing.

[stuckless]

Originally Posted by reggie14

Here's a link to a page with the unicode characters for apostrophes, single opening/closing quotes, and backticks, etc.

Thanks, that's a good reference.

Originally Posted by Fuzzy

I'd also say stripping out punctuation would be a most likely solution (and stripping and 'the' prefixes as well)

EDIT: Were't the days of ASCII so much simpler to code in? Unicode, while nice for printing, sucks for parsing.

I'll have to do some testing... Do you guys have any titles that I can use for testing?

[DirtClodimus]

Originally Posted by stuckless

Thanks, that's a good reference.



I'll have to do some testing... Do you guys have any titles that I can use for testing?

Apostrophe
A Bug's Life
Look Who's Talking

"The" prefix
The Incredibles
The Lake House
The Devil Wears Prada

With punctuation, but worked
Austin Powers: The Spy Who Shagged Me
Austin Powers: International Man of Mystery
Monsters, Inc.

These are the titles that failed recently as well as some with punctuation that worked. From what I can tell, apostrophes are bad. Commas, colons, and periods are ok. Still, might be safest to strip them all out on the search.

[stuckless]

Thanks... and my apologies. Apparently, I was over encoding the search queries going to themoviedb, and as a result, many titles, such as those with spaces and punctuation was getting passed incorrectly. I just tested with the list that you posted and it appears to work find now. It'll be a few days before I publish a new build... but when I do, it'll have this fix and the update to imdb as well.