Problem with MediaTitles.xml

[cmaloney]

Has anyone had success using MediaTitles.xml to force BMT to use tvdb as the lookup for tv shows given a shows name?

Here is an example of an entry in my MediaTitles.xml file:

        <match>
		<regex>[\\/]MythBusters[\\/]</regex>
		<metadata type="tv" name="tvdb">73388</metadata>
	</match>

I believe I have followed the directions and for \\fileserver\media\TV-Shows\MythBusters\YouTube Special.mpg, I would think it would match it and force a lookup from tvdb using id of 73388 which is Mythbusters. The problem is this doesn't work and I'm not sure why it isn't. The log file contains:

2010-08-16 20:44:39,466 - ERROR; \\fileserver\media\TV-Shows\MythBusters\YouTube Special.mpg; YouTube Special; 171457; Search Failed for SearchQuery; Type: TV; EPISODE_DATE:2010-08-16;ID:73388;RAW_TITLE:YouTube Special;FILE:\\fileserver\media\TV-Shows\MythBusters\YouTube Special.mpg;QUERY:YouTube Special;CLEAN_TITLE:YouTube Special;PROVIDER:tvdb;

Any ideas?

[cmaloney]

Ahh, looks like a null pointer exception:

2010-08-16 20:44:39,466 [Thread-28] WARN  sagex.phoenix.metadata.provider.tvdb.TVDBItemParser - Failed while parsing series: MediaSearchResult [extraArgs=CLEAN_TITLE: YouTube Special,ID: 73388,RAW_TITLE: YouTube Special,FILE: \\fileserver\media\TV-Shows\MythBusters\YouTube Special.mpg,PROVIDER: tvdb,EPISODE_DATE: 2010-08-16,, id=73388, imdbId=null, metadata=null, providerId=tvdb, score=1.0, title=YouTube Special, type=null, url=null, year=0]
java.lang.NullPointerException
	at sagex.phoenix.metadata.provider.tvdb.TVDBItemParser.getMetadata(TVDBItemParser.java:76)
	at sagex.phoenix.metadata.provider.tvdb.TVDBMetadataProvider.getMetaData(TVDBMetadataProvider.java:27)
	at sagex.phoenix.metadata.search.MetadataSearchUtil.searchById(MetadataSearchUtil.java:176)
	at sagex.phoenix.metadata.provider.tvdb.TVDBMetadataProvider.search(TVDBMetadataProvider.java:33)
	at sagex.phoenix.metadata.MetadataManager.search(MetadataManager.java:355)
	at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:556)
	at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:526)
	at sagex.phoenix.metadata.AutomaticMetadataVisitor.visit(AutomaticMetadataVisitor.java:24)
	at sagex.phoenix.vfs.DecoratedMediaFile.accept(DecoratedMediaFile.java:48)
	at sagex.phoenix.vfs.DecoratedMediaFolder.accept(DecoratedMediaFolder.java:61)
	at sagex.phoenix.metadata.PhoenixMetadataSupport$1.run(PhoenixMetadataSupport.java:110)
	at sagex.phoenix.progress.ProgressTrackerManager$1.run(ProgressTrackerManager.java:48)

[stuckless]

I'll investigate tonight... thx for the stack trace, that'll make it easier to find/fix.

[cmaloney]

Hmm, looking at the source I'm even more confused. TVDBItemParser.java doesn't seem to be in the trunk anymore and yet my server is using it somehow even though I used the built-in plugin downloader inside of sagetv. TVDBItemParser.java is inside of the BMT-S6 branch but I'm assuming that it is for sagetv 6 and earlier. Any idea what could be going on? Is the 4.5 release still using TVDBItemParser.java? If so, where can I get the source so I can help fix the problem?

[stuckless]

Originally Posted by cmaloney

Hmm, looking at the source I'm even more confused. TVDBItemParser.java doesn't seem to be in the trunk anymore and yet my server is using it somehow even though I used the built-in plugin downloader inside of sagetv. TVDBItemParser.java is inside of the BMT-S6 branch but I'm assuming that it is for sagetv 6 and earlier. Any idea what could be going on? Is the 4.5 release still using TVDBItemParser.java? If so, where can I get the source so I can help fix the problem?

There is very little code in the bmt project any more... it's been moved to the phoenix project. So you'd need to check out the phoenix source code to find the issue.

But, to save you some time... the NPE is comming from a line of code that should never be null So, I need to further investigate what that is happening.

[cmaloney]

Got it fixed I had 33 TV episodes and it parses all but 11 of them correctly now. Going to see if I can get the others to process correctly.

TVDBItemParser.java Line 87
Change:
String title = result.getExtra().get(SearchQuery.Field.EPISODE_TITLE.name());
to:
String title = result.getExtra().get(SearchQuery.Field.RAW_TITLE.name());

MetadataSearchUtil.java Line 172
Add:
res.setMediaType(query.getMediaType());

After:
MediaSearchResult res = new MediaSearchResult(prov.getInfo().getId(),
				id, query.get(Field.RAW_TITLE), NumberUtils.toInt(query
						.get(Field.YEAR)), 1.0f);

[cmaloney]

The problem with the other 11 was that for a couple of the shows in my MediaTitles.xml file I had included a title and year as the example showed. However, including a title overrode the title that was derived from the filename. IE given the file \\Family Guy\Road to Europe.mpg : Instead of Road to Europe being the episode name, the parser was trying to use Family Guy as the episode title. Everything parses correctly now.

[stuckless]

Originally Posted by cmaloney

Got it fixed I had 33 TV episodes and it parses all but 11 of them correctly now. Going to see if I can get the others to process correctly.

TVDBItemParser.java Line 87
Change:
String title = result.getExtra().get(SearchQuery.Field.EPISODE_TITLE.name());
to:
String title = result.getExtra().get(SearchQuery.Field.RAW_TITLE.name());

MetadataSearchUtil.java Line 172
Add:
res.setMediaType(query.getMediaType());

After:
MediaSearchResult res = new MediaSearchResult(prov.getInfo().getId(),
				id, query.get(Field.RAW_TITLE), NumberUtils.toInt(query
						.get(Field.YEAR)), 1.0f);

I noticed a few issues with the way that I had changed the code to do searches based on the IDs, and I've fixed that, so the MediaTitles.xml should work correctly (it was broken ). I've also added your second fix as well.

But, your first fix, while it may work for you, is incorrect in the way that SearchQuery objects are built and used. We should try to figure out why RAW_TITLE is holding an episode title and not the series name.

RAW_TITLE for a TV show would actually contain the Series title and not the episode title. EPISODE_TITLE is the only field that holds the Episode Title, and it's parsed using one of the functions from the tvfilename scrapers.

title is only used in searching if there isn't a season/episode # in the query, and we need to do a fuzzy logic search based on the episode title. So, putting the Series title in the title field will most likely fail in all cases.

I'll try to get a build out tonight with my latest changes. I actually had to fix several bugs in all the providers when a search by id was being used I've added new test cases for by id searching as well. If you are interested in seeing the test cases, you can view them here and here.

I appreciate the in depth code fix.. it's not often I get the assistance of another java developer when debugging/troubleshooting these types of issues

[stuckless]

Originally Posted by cmaloney

The problem with the other 11 was that for a couple of the shows in my MediaTitles.xml file I had included a title and year as the example showed. However, including a title overrode the title that was derived from the filename. IE given the file \\Family Guy\Road to Europe.mpg : Instead of Road to Europe being the episode name, the parser was trying to use Family Guy as the episode title. Everything parses correctly now.

I think I know why you are getting the Series name the RAW_TITLE fields. The tv format of

Folder/SeriesName/EpisodeName.mpg is not actually recognized as a TV filename scraper. ie, there is no way for me to know that the folder name is the series, so what ends up happing is that the file gets processed by the movie scrapers

The correct solution in your case would actually be to create a tvfilename scraper that would recognize the pattern TV/SeriesName/EpisodeName.mpg

By modifying the code, and adding the MediaTitles, you are getting what you want, but unfortunately, it's not correct

[cmaloney]

Originally Posted by stuckless

I think I know why you are getting the Series name the RAW_TITLE fields. The tv format of

Folder/SeriesName/EpisodeName.mpg is not actually recognized as a TV filename scraper. ie, there is no way for me to know that the folder name is the series, so what ends up happing is that the file gets processed by the movie scrapers

The correct solution in your case would actually be to create a tvfilename scraper that would recognize the pattern TV/SeriesName/EpisodeName.mpg

By modifying the code, and adding the MediaTitles, you are getting what you want, but unfortunately, it's not correct

Ahh, I was wondering why I was the only one having the problem. I guess I have a long ways to go before I fully understand the way all the pieces interact. I'll be sure to try creating the scraper.

BTW, One thing that was kind of irritating to me when I was processing the titles was that the matchScore was too high for a few of my matches causing it to fail. I had 3 shows like this today from my TiVo and I solved it by modifying the updateIfScored method to keep track of the highest scored episode and if it didn't find an exact match it would update the meta data to the best match. I know that this solution is not a perfect solution as some people might not want any false positives, but for TV shows the likelihood of this occurring should be fairly rare especially if you are specifying the series id and personally, I would rather fix the occasional mix up instead of the frequent near match that might be off by a couple of words. Maybe an option to keep the best match if a perfect one isn't found could be added? I could probably add it if there is an interest for such a feature and send you the diffs. Let me know if you want the code and I'll post it. Thanks for your responses

[jusjoken]

Originally Posted by stuckless

I think I know why you are getting the Series name the RAW_TITLE fields. The tv format of

Folder/SeriesName/EpisodeName.mpg is not actually recognized as a TV filename scraper. ie, there is no way for me to know that the folder name is the series, so what ends up happing is that the file gets processed by the movie scrapers

The correct solution in your case would actually be to create a tvfilename scraper that would recognize the pattern TV/SeriesName/EpisodeName.mpg

By modifying the code, and adding the MediaTitles, you are getting what you want, but unfortunately, it's not correct

As this is the same issue that I am dealing with in another post here I thought I would ask a question here...

Can I assume that the following folder/file format will work once you get the new release out...

Folder/SeriesName/Season x/SeriesName SxEyy EpisodeName.mpg

As this format will be recognized as a tv filename and processed by the tv scraper?

k

[stuckless]

Originally Posted by jusjoken

As this is the same issue that I am dealing with in another post here I thought I would ask a question here...

Can I assume that the following folder/file format will work once you get the new release out...

Folder/SeriesName/Season x/SeriesName SxEyy EpisodeName.mpg

As this format will be recognized as a tv filename and processed by the tv scraper?

k

The phoenix-core 2.0.8/2.0.9 builds should fix your issue as well. I actally used the SATC in my test cases

BTW, SeriesName SxEyy EpisodeName.mpg should already work, the fix that I did was allow the searcher to correctly use the id specified in the MediaTitles.xml

[jusjoken]

Originally Posted by stuckless

The phoenix-core 2.0.8/2.0.9 builds should fix your issue as well. I actally used the SATC in my test cases

BTW, SeriesName SxEyy EpisodeName.mpg should already work, the fix that I did was allow the searcher to correctly use the id specified in the MediaTitles.xml

The change fixed SATC... thanks

However, for some reason Heartland still is not using the correct Heartland from tvdb. It still uses 70598 rather than 82701 even with this match...

	<match>
		<regex>[\\/]Heartland</regex>
		<metadata type="tv" name="tvdb">82701</metadata>
	</match>

Thoughts?