Looking for Title/Season/Episode naming formats to support for metadata scraper

[evilpenguin]

I'm working on a scraping tool and I'm hoping to accommodate as many naming conventions as I can. Basically, I just need to know how people's files are named so I can figure out how to capture the show title and season/episode # so I can track down the metadata on the internet.

Right now I've got 3 working...

Title-Episode-######-0.mpg - SageTV format, impossible to parse directly, but I can use the webserver to grab the real name using the airing number.

title S##E## - episode.avi

title ##x## - episode.avi

If you have one that looks different, if you use the folder structure for naming, or anything else that is a little crazy, please post it so I can do my best to support it.

Thanks!

[stuckless]

Which site(s) are you scraping? I'm currently working on tv scraping support in the batch metadata tools. Maybe we can share code/algorithms? Currently, for SxE type variants, I'm using the following regexp

".*[Ss]*0*([0-9]+)\\.*[EeXx]\\s*0*([0-9]+).*"

[dvd_maniac]

I have a folder for each show I archive and then a seson folder for each season and finally the episode.
Ex:
TV\CSI\09\S09E08 Say Uncle.mp4

[keefmyster]

Mine are in a format similar to dvd_maniac's, but I spelled out season and don't have the season number on the final file name:

TV\Lost\Season 04\01 The Beginning of the End.avi

[evilpenguin]

Originally Posted by dvd_maniac

I have a folder for each show I archive and then a seson folder for each season and finally the episode.
Ex:
TV\CSI\09\S09E08 Say Uncle.mp4

That's very do able, its what I use also. I think i'll have to restrict it to only being from a \TV\ subfolder for sanity's sake

Originally Posted by stuckless

Which site(s) are you scraping? I'm currently working on tv scraping support in the batch metadata tools. Maybe we can share code/algorithms? Currently, for SxE type variants, I'm using the following regexp

".*[Ss]*0*([0-9]+)\\.*[EeXx]\\s*0*([0-9]+).*"

Sorry, didn't mean to step on your toes! I'm more working on a tool you can use to create scrapers rather than a scraper for a specific site. I'm shooting for something like XBMC's system where all of the logic is in profile files so it can be expanded infinity without recompiling. In addition I also have the following goals extra goals...

* Be able to capture all episode data for entire series in one call into a single file (Think Fandata).
* Have the output format be controlled by profile files so it can be configured to write into any format

For starters, to prove out my system, I'm working on "theTVDB" and I've pretty much nailed it...

Show		: Lost
theTVDB ID	: 73739
Description	: After their plane, Oceanic Air flight 815, tore apart whilst thousands of miles off course, the survivors find themselves on a mysterious deserted island where they soon find out they are not alone.
First Aired	: September 22, 2004
Airs On		: Thursday
Runtime		: 45 minutes
Network		: ABC
Genre		: Action and Adventure
Actors		: Matthew Fox(http://www.theTVDB.com/banners/_cache/actors/27747.jpg)||Evangeline Lilly(http://www.theTVDB.com/banners/_cache/actors/27741.jpg)||Naveen Andrews(http://www.theTVDB.com/banners/_cache/actors/27749.jpg)||
Banners		: http://www.theTVDB.com/banners/graphical/73739-g4.jpg||http://www.theTVDB.com/banners/graphical/73739-g.jpg||http://www.theTVDB.com/banners/graphical/73739-g6.jpg||http://www.theTVDB.com/banners/graphical/24313-g2.jpg||http://www.theTVDB.com/banners/graphical/73739-g3.jpg||http://www.theTVDB.com/banners/graphical/73739-g8.jpg||http://www.theTVDB.com/banners/graphical/73739-g7.jpg||http://www.theTVDB.com/banners/graphical/73739-g5.jpg||http://www.theTVDB.com/banners/graphical/24313-g.jpg||http://www.theTVDB.com/banners/blank/73739.jpg||http://www.theTVDB.com/banners/graphical/73739-g2.jpg||http://www.theTVDB.com/banners/graphical/73739-g9.jpg||
Posters		: http://www.theTVDB.com/banners/posters/73739-2.jpg||http://www.theTVDB.com/banners/posters/73739-1.jpg||http://www.theTVDB.com/banners/posters/73739-5.jpg||http://www.theTVDB.com/banners/posters/73739-6.jpg||http://www.theTVDB.com/banners/posters/73739-4.jpg||http://www.theTVDB.com/banners/posters/73739-3.jpg||
Fanart		: http://www.theTVDB.com/banners/fanart/original/73739-27.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-27.jpg||http://www.theTVDB.com/banners/fanart/original/73739-15.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-15.jpg||http://www.theTVDB.com/banners/fanart/original/73739-26.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-26.jpg||http://www.theTVDB.com/banners/fanart/original/73739-1.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-1.jpg||http://www.theTVDB.com/banners/fanart/original/73739-22.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-22.jpg||http://www.theTVDB.com/banners/fanart/original/73739-17.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-17.jpg||http://www.theTVDB.com/banners/fanart/original/73739-19.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-19.jpg||http://www.theTVDB.com/banners/fanart/original/73739-3.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-3.jpg||http://www.theTVDB.com/banners/fanart/original/73739-12.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-12.jpg||http://www.theTVDB.com/banners/fanart/original/73739-9.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-9.jpg||http://www.theTVDB.com/banners/fanart/original/73739-16.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-16.jpg||http://www.theTVDB.com/banners/fanart/original/73739-7.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-7.jpg||http://www.theTVDB.com/banners/fanart/original/73739-2.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-2.jpg||http://www.theTVDB.com/banners/fanart/original/73739-23.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-23.jpg||http://www.theTVDB.com/banners/fanart/original/73739-4.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-4.jpg||http://www.theTVDB.com/banners/fanart/original/73739-25.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-25.jpg||http://www.theTVDB.com/banners/fanart/original/73739-14.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-14.jpg||http://www.theTVDB.com/banners/fanart/original/73739-11.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-11.jpg||http://www.theTVDB.com/banners/fanart/original/73739-6.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-6.jpg||http://www.theTVDB.com/banners/fanart/original/73739-8.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-8.jpg||http://www.theTVDB.com/banners/fanart/original/73739-10.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-10.jpg||http://www.theTVDB.com/banners/fanart/original/73739-18.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-18.jpg||http://www.theTVDB.com/banners/fanart/original/73739-5.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-5.jpg||http://www.theTVDB.com/banners/fanart/original/73739-24.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-24.jpg||http://www.theTVDB.com/banners/fanart/original/73739-29.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-29.jpg||http://www.theTVDB.com/banners/fanart/original/73739-20.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-20.jpg||http://www.theTVDB.com/banners/fanart/original/73739-28.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-28.jpg||http://www.theTVDB.com/banners/fanart/original/73739-21.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-21.jpg||http://www.theTVDB.com/banners/fanart/original/73739-30.jpg||http://www.theTVDB.com/banners/fanart/vignette/73739-30.jpg||
~~~~~~~~~~~~~~~~~~~~~~~~~
Episode		: 1 - 1
Title		: Pilot (1)
First Aired	: 2004-09-22
Guest Stars	: Greg Grunberg| John Dixon| Michelle Arthur|
Directors	: J.J. Abrams
Writers		: J.J. Abrams, Damon Lindelof
Description	: Stripped of everything, the 48 survivors scavenge what they can from the plane for their survival. Some panic. Some pin their hopes on rescue. A few find inner strength they never knew they had-like Kate who, with no medical training, suddenly finds herself suturing the doctor's wounds. The band of friends, family, enemies and strangers must work together against the cruel weather and harsh terrain. But the intense howls of  mysterious creatures stalking the jungle fill them all with fear. Fortunately, thanks to the calm leadership of quick-thinking Jack and level-headed Kate, they have hope. But even heroes have secrets, as the survivors will come to learn.
~~~~~~~~~~~~~~~~~~~~~~~~~
Episode		: 1 - 2
Title		: Pilot (2)
First Aired	: 2004-09-29
Guest Stars	: John Dixon| Michelle Arthur| George Alan|
Directors	: J.J. Abrams
...
...

So should definitely talk because I think we can work together. Care to pop into a new thread in the customization forum so we I can throw out some gory technical details?

[stuckless]

Actually you are not stepping on my toes. I was just doing it out of necessity, and I haven't started much yet along those lines. I'll probably just wait and see what you produce, since you are way futher ahead than I am

[evilpenguin]

How about some music naming conventions? I don't have any mp3's so people will have to help me out on this one

[QueOnda]

Originally Posted by evilpenguin

How about some music naming conventions? I don't have any mp3's so people will have to help me out on this one

I have a central location on my mp3s. I have my file name as follows:

Artist - Album - Track# - song name.mp3

The - and spaces are there.

Sometimes they are in their own folder: Artist folder -> Album folder

Some song I have don't have track# or Album, so then only Artist - song name.mp3

There are other name configurations I use they are minor. Most programs allow you to rename to certain naming convention. I don't know if mediamonkey monkey might.

[evilpenguin]

Originally Posted by QueOnda

Some song I have don't have track# or Album, so then only Artist - song name.mp3

This is the only one that would be a problem if I were doing it strictly by file name. Are most of your songs ID3 tagged with at least artist/album?

[TallMomof2]

My mp3s have the following format for the file name:

track#-track name

May or may not have spaces between track# "-" track name. Depends on the source. I rip mp3s without spaces and Amazon mp3s use spaces. All are ID3 tagged with:
Track Name
Artist
Album Title
Year
Track Number
Duration
Type
Bit Rate
Protected (Y/N)
Size

[popechild]

Originally Posted by evilpenguin

How about some music naming conventions? I don't have any mp3's so people will have to help me out on this one

I'd suggest looking to see how iTunes organizes and names songs when you use it to organize your music. If there's going to be a method used by more people than most, that's bound to be it, simply because so many people use iTunes to organize their music. (Though maybe iTunes let's you change the way it organizes it, I'm not sure about that.) I let iTunes do all the naming and organizing into folders, then I just point Sage at the parent folder and let it find it all.

[evilpenguin]

Is storing your songs without the artist/album in the file name pretty common? Its sounding like my best bet is to just read the ID3 tags for music .

[QueOnda]

Originally Posted by evilpenguin

This is the only one that would be a problem if I were doing it strictly by file name. Are most of your songs ID3 tagged with at least artist/album?

Yes, I have mostly all ID3 taggged. Could you read ID3 tagged info? Other issues would me other type of music files which are NOT mp3. Don't know what else sage is able to play. Does OGG and FLac and WAVs have ID3 Tag info?

The reason I have my music files labled the way I can sort and look what I have through file browser.

I confirmed MediaMonkey is able to rename filenames. Called auto-organize files (CTRL+R). Just an option if people has to name the files a particular way.

[Brent]

You best bet will be using the ID3 tags for music. There are so many different combinations of naming schemes for music files.

[evilpenguin]

Originally Posted by Brent

You best bet will be using the ID3 tags for music. There are so many different combinations of naming schemes for music files.

Yeah, its sure sounding like that. Oh well, not a big deal, Perl has a ID3 tag parser I should be able to use.

[MattHelm]

Originally Posted by QueOnda

Does OGG and FLac and WAVs have ID3 Tag info?

I know FLAC does NOT use ID3, but it's own format, but SageTV does understand whatever FLAC uses.

[Slipshod]

Originally Posted by evilpenguin

Yeah, its sure sounding like that. Oh well, not a big deal, Perl has a ID3 tag parser I should be able to use.

Yeah, definately parse ID3 tags - I do artist and album as directories, with track# and title as filenames. You'll need to think about what version to support, and what has precedence. Some of my old stuff has both v1 and v2 tagging on it, or just v1.

[Skirge01]

My ID3 tags are 99% accurate and complete, so that's definitely the best method. But, I also organize with a folder structure as follows:

Artist\Album\[Artist] - [Title].mp3

or

Genre\Artist\Album\[Artist] - [Title].mp3

Sometimes the track # is part of the filename, but never part of the folder structure. It's primarily live albums where I worry about track sequence at all.

The issue with parsing folder names versus ID3 tags is that folder names disallow some characters that are used in actual album, artist, or song names.