Episode Information exported into Video Library

[Armondole]

Originally Posted by roxy99

The only thing is how up to date is the tvdb.com database? Does it get its data only from users who upload or is it always in synch with TV.com?

From what I've gathered TVDB.com provides an xml database to facilitate plugins on HTPC system to grab data and is in itself a screen scraper from TV.com. Please correct me if I am wrong.

In that sense if I piggy back my scrapper off of tvdb's, then my utility is dependent on that sites reliability. So on top of my utilities reliability the user needs to be concerned about tvdb's reliability.

A quote from Scott Zsori (TVDB creator):

"Right now we're dependant on users inputing the information. We did have a TV.com scraper and will have one again in the future, but I haven't had the time to rewrite it lately. Once we start scraping again, it'll be done once a day for new shows and as needed for "forced"
updates.

--Scott"

I don't believe that the reliability of TVDB is poor, but the rate of updates to the site will not be as quick as TV.com. Having said that, they have a very active userbase (not unlike SageTV) and they allow you to customize banners and add shows to the database that might not appear in TV.com properly.

Of course I am only suggesting this as an option. If it is not too difficult to add this db to your tool, then maybe have to option to scrape either TV.com or TVDB??? This way the users would have the latest shows from TV.com and the most options for banners/graphics from TVDB.

[dvd_maniac]

I've said it before and I'll say it again:

Why not have a place in the community to host metadata files so that Sage users can upload and download them without worrying about the different naming conventions people have with their imported videos, websites updating that cause compatibility issues or other unforseen problems?

[Armondole]

Originally Posted by dvd_maniac

I've said it before and I'll say it again:

Why not have a place in the community to host metadata files so that Sage users can upload and download them without worrying about the different naming conventions people have with their imported videos, websites updating that cause compatibility issues or other unforseen problems?

This wouldn't solve any of the problems of assigning metadata (or .my, .properties info) to files that were not recorded with SageTV though... is that not the issue we're trying to solve?

[dvd_maniac]

This wouldn't solve any of the problems of assigning metadata (or .my, .properties info) to files that were not recorded with SageTV though... is that not the issue we're trying to solve?

Yes, but even if the files are created and named the same based on what is trying to be done here, there are still other issues that will crop up.
For instance, no detailed information on the website, website updated and program no longer compatible and needs to be rewritten.

I just think it would be nice to simply store and download the metadata files and hopefully have the most complete and detailed versions in one place.
So I guess it can compliment what is being done here, not replacing it.

[roxy99]

Originally Posted by dvd_maniac

Yes, but even if the files are created and named the same based on what is trying to be done here, there are still other issues that will crop up.
For instance, no detailed information on the website, website updated and program no longer compatible and needs to be rewritten.

I just think it would be nice to simply store and download the metadata files and hopefully have the most complete and detailed versions in one place.
So I guess it can compliment what is being done here, not replacing it.

You guys got me all pumped up! I'm itching to release my TV.com scraper but there a few nagging little bugs.
The biggest challenge to overcome has been how to deal with the diferent naming conventions of peoples' video files. Now that that is done, there is no need for hosting of metadata files. All the data is already on TV.com. What's needed is a robust scraper that is intelligent enough to handle the names.

DVD maniac,
You may want to try a little experiment. Take one of your Buck Rogers episodes and rename the episode path from lets say

t:\...\Buck Rogers\01\1x12 Ardala Returns\ardala returns.mp4

to

t:\...\Buck Rogers\01\Season 1 - Episode 12 Ardala Returns\ardala returns.mp4

Then run my little test with -t 3 -s 2 and you should see that the episode # is still picked up. I've designed the utility to be flexible for that reason.


Armondole,
It could be an option to scrape from tvdb.com. Its xml and should be really easy to scrape off of (much easier than scraping TV.com). The thing is, 'why buy the milk if you can have the whole cow for free'. I don't think I'll need to. If my TV.com scraper turns out to be too difficult to maintain, then I may do a tvdb.com scraper instead.

[roxy99]

Originally Posted by Armondole

This wouldn't solve any of the problems of assigning metadata (or .my, .properties info) to files that were not recorded with SageTV though... is that not the issue we're trying to solve?

Assignment is a trivial matter of renaming the metadata file the same base name as the video file or vice versa.

[dvd_maniac]

DVD maniac,
You may want to try a little experiment. Take one of your Buck Rogers episodes and rename the episode path from lets say

Code:
t:\...\Buck Rogers\01\1x12 Ardala Returns\ardala returns.mp4

to

Code:
t:\...\Buck Rogers\01\Season 1 - Episode 12 Ardala Returns\ardala returns.mp4

The reason I changed it to the way I have it is that when I browse the Imported Videos in Sage, the loooonnnnngggggg...... name gets cut off. Now I do know that my whole collection is not uniform and that I plan to make it so but that example is too long.

How about if I did this:
t:\...\Buck Rogers\SE01EP12 - Ardala Returns.mp4
This way I'd cut out the season folders and have a more uniform naming convention that works plus your program should be able to determine that SE"##" is season and EP"##" is episode and that the parent folderr is Series?

[roxy99]

Originally Posted by dvd_maniac

The reason I changed it to the way I have it is that when I browse the Imported Videos in Sage, the loooonnnnngggggg...... name gets cut off. Now I do know that my whole collection is not uniform and that I plan to make it so but that example is too long.

How about if I did this:
t:\...\Buck Rogers\SE01EP12 - Ardala Returns.mp4
This way I'd cut out the season folders and have a more uniform naming convention that works plus your program should be able to determine that SE"##" is season and EP"##" is episode and that the parent folderr is Series?

You will first need to use -t 1 -s 0
Next I should point out that these variations will work:

S01E12 - Ardala Returns.mp4
s01e12 - Ardala Returns.mp4
01x12 - Ardala Returns.mp4
01.12 - Ardala Returns.mp4
Season 1 Epsiode 12 - Ardala Returns.mp4

But
SE01EP12 WILL NOT
I'll have to fix the regex for that. Is SE a common naming scheme? I thought S__E__ was more standard?

EDIT Okay I fixed it SE__EP__ works now. I'll differ sending the fix to you until I incorporate the episode scraping portion.

As a side note, I could've set the regex to look for ANY pair of 2 digit number regardless of prefixes. The problem with that approach is that it is too liberal. Sometimes numbers appear in an episode title which tricks the regex. That's why the regex pattern has to be tailored.

[stanger89]

Originally Posted by roxy99

t:\...\Buck Rogers\01\1x12 Ardala Returns\ardala returns.mp4

to

t:\...\Buck Rogers\01\Season 1 - Episode 12 Ardala Returns\ardala returns.mp4

You know, a single regular expression could match both of those:

(\w* *)(\d*)(x| - )(\w* *)*(\d*)(.*)

The second group would would match season number, the fifth would match episode number.

Nevermind, I see you were doing that already

[Armondole]

Originally Posted by roxy99

Armondole,
It could be an option to scrape from tvdb.com. Its xml and should be really easy to scrape off of (much easier than scraping TV.com). The thing is, 'why buy the milk if you can have the whole cow for free'. I don't think I'll need to. If my TV.com scraper turns out to be too difficult to maintain, then I may do a tvdb.com scraper instead.

Sounds great Roxy99! Since I'm useless at helping with coding, I figured I'd try to do some research to help you out... I'm looking forward to seeing this in action.

Cheers,
A

[roxy99]

Originally Posted by Armondole

Sounds great Roxy99! Since I'm useless at helping with coding, I figured I'd try to do some research to help you out... I'm looking forward to seeing this in action.

Cheers,
A

You can implement the .properties file portion for me. I'll take care of the TV.com scraping since I'm 95% complete there's not much in that aspect you can do to help. Python is really easy, especially for text manipulation.

Your mission:

1. use a valid .my file
2. Using note pad , manually produce a valid .properties file copying and pasting relevant sections from the .my file to the .properties file.
3. test the properties file in Sage 6 extensively-- (do searches and stuff)
4. You may need to rename you avi so that the date stamp changes and Sage refreshes its database with the new .properties info.
5. Now use Python to automate that whole process except instead of renaming the video use os.utime() to change timestamp of video file.
6. Then you're done and you can send me that Python script.

[dvd_maniac]

But
SE01EP12 WILL NOT
I'll have to fix the regex for that. Is SE a common naming scheme? I thought S__E__ was more standard?

Whatever works out best, I'll use.
I'm getting ready to standardize my naming conventions but will wait to see what this program utilizes first...

[roxy99]

Originally Posted by dvd_maniac

Whatever works out best, I'll use.
I'm getting ready to standardize my naming conventions but will wait to see what this program utilizes first...

Good plan

[Armondole]

Originally Posted by roxy99

You can implement the .properties file portion for me. I'll take care of the TV.com scraping since I'm 95% complete there's not much in that aspect you can do to help. Python is really easy, especially for text manipulation.

Your mission:

1. use a valid .my file
2. Using note pad , manually produce a valid .properties file copying and pasting relevant sections from the .my file to the .properties file.
3. test the properties file in Sage 6 extensively-- (do searches and stuff)
4. You may need to rename you avi so that the date stamp changes and Sage refreshes its database with the new .properties info.
5. Now use Python to automate that whole process except instead of renaming the video use os.utime() to change timestamp of video file.
6. Then you're done and you can send me that Python script.

I can do my best... it'll have to wait about a month for me to start this though cause I'm getting married this month.

[roxy99]

VIDEO METADATA CREATOR (for tv shows with information on www.tv.com)

http://forums.sagetv.com/forums/down...do=file&id=188

NOTE: External utility - non-Sage dependent

Version 1.0.0.0



DESCRIPTION:
This small command line utility scans a specified directory tree for tv video files and produces metadata files about the show, its actors, etc (with the .my extension). The utility incorporates
a screen scraper which produces a text file with the .my extension from information gathered from TV.com

Video file extensions scanned: avi,divx,mov,moov,mp4,mpg,rm,qt,wmv,vob


USAGE:
Since this is a command line utility which relies on command option swithes, it's recommended that the user create a windows short-cut a bat file that calls the utlity. In this way the user can be spared
from the cumbersome commandline prompt and be spared from repetitive typing.

Your video collection must be organized in a manner such that Season and Episode # are identified for each TV show,

IE:
S02.E12
03x13
etc

Note: This version does not support simply name based videos. For instance, If your tv show is named Heroes - String Theory.avi it will not work. You would need to rename it:

Heroes - S01xE22 - String Theory.avi


PARAMETERS:

The minimum required parameter is:

-p (<drive:\scanpath>)

eg: C:\Program Files\Video Metadata\videometadata.exe -p (C:\AVI\CSI miami)

This will scan the path and subfolders within 'C:\AVI\CSI miami'

SPACE AFTER -p and PARENTHESIS SURROUNDING PATH REQUIRED!

The best way to provide the -p parameter is to create a shortcut the the videometadata.exe. Right-click the shortcut and enter the parameter in the Target: box right after "C:\...\videometadata.exe"



OPTIONAL PARAMETERS:

-t title depth (default 0, Maximum 3)

Instructs utility to look up directory tree from location of the video file the
specified number of parent jumps to find the show's title name.

EG: C:\AVI\CSI miami\Season 2\EP05 episode title.avi
Here the show title is not present in the video file name: EP05 episode title.avi
Therefor we need the parameter '-t 2 to look CSI Miami'

The legal values are: 0, 1, 2, 3

-s season depth (default 0, Maximum 2)

Same as for title depth except for season number.
EG: C:\AVI\CSI miami\Season 2\EP05 episode title.avi
Requires -s 1 in order to fetch the season number.

Note that these parameters are optional if the entire video name has the required information but still
are recommended to improve the detection accuracy.


Remember, change shortcut under Target with:

"C:\...\videometadata.exe" -p (<drive:\scanpath>) -t # -s #

[smdear]

Hello,

I'm a new Sage user, and I see that this tool has been broken by changes at TV.com.

A fix would be appreciated, as this would really make my life easier and help me better organize my TV show library.