Utility: mediaScraper - metadata made easy!

[jaminben]

Yep, that sounds very cool

[Zorglub]

Originally Posted by evilpenguin

I like the sound of that, can you go into more detail on you would use these XML files to import the shows into SageTV? I'm not familiar with it.

Sure, you just use nielm webserver to enter it in the database:

http://localhost:8080/sage/XMLImporter

Then you checked the "Import TV files" box and checked those options:

* Overwrite Existing Show Information
* Overwrite Existing Sage TV File Information
* Filename handling:
o Keep original imported filename
* File timestamp handling
o Use imported Airing data to determine startTime
(use when importing edited files. Will change imported file timestamp)

You then upload your xml file, and VOILA ! your video file is refered in Recorded TV with all the right info (thanks to you ) and you can start it the same way you would with any other file...

Of course I am presently doing this manually, but I am looking for a way to call this via a command line, something like:

wget.exe "http://username:ypassword@localhost:8080/sage/XMLImporter&&FileName=N:\MEDIA\Videos\TV\Battlestar Galactica\Season 4\Battlestar Galactica (2003) 4x12 - A Disquiet Follows My Soul.xml"

so I can include it in a batch file and automatize the whole thing...

I am still trying to figure out how this last part can be done, but I am convince there is a way... I may need help on this one...

[Zorglub]

Update:

After some reading on curl (which seem to be able to handle POST better than wget) and a lot of fiddling around, I came up with the good syntax:

(FWI: this has been tested on Cygwin)

curl -F "impTVFiles=checked" \
-F "impShowOverwrite=checked" \
-F "impMFOverwrite=checked" \
-F "impMFRename=keep" \
-F "impMFRedate=redateFromAiring" \
-F "xmlFile=@exemple.xml" \
-F "Import=submit" \
http://username:fpasswd@locahost:808...ter?Import=yes

Now if you can add the option to output the episode information in the good xml format, I'll be in Heaven, and so will my wife !!!

[Muchacho]

This is fantastic! Great job and thanks.

After messing around with it for a while, I think that there is a bug with downloading the posters for movies. It generally works great, but I noticed that it does not d/l a poster if there is not a high res poster on themoviedb.org. It would seem that it should (or hopefully could) download the available lower resolution image or the one from imdb if a high res poster is not available.

Examples that did not work (mostly stuff my wife recorded…)
-The Last Kiss (has low res poster on themoviedb)
-Y Tu Mama Tambien (no poster on themoviedb, but there is one available on imdb)
-Capote (nothing available on themoviedb)

Am I missing something? Also, does sageTV periodically update its database and include the new posters, backgrounds, and .properties? It does not seem to be updating itself even after I refresh the imported folders.

Thanks again, Mike

[evilpenguin]

Originally Posted by Muchacho

It would seem that it should (or hopefully could) download the available lower resolution image or the one from imdb if a high res poster is not available

Well, not so much a bug as a feature I never bothered to implement. I'll put it on the ol' todo list.

[Muchacho]

Originally Posted by evilpenguin

Well, not so much a bug as a feature I never bothered to implement. I'll put it on the ol' todo list.

Fantastic!

Much appreciated.

[popechild]

Originally Posted by Muchacho

This is fantastic! Great job and thanks.
Examples that did not work (mostly stuff my wife recorded…)

Ha ha!

[evilpenguin]

Originally Posted by Zorglub

Update:

After some reading on curl (which seem to be able to handle POST better than wget) and a lot of fiddling around, I came up with the good syntax:

(FWI: this has been tested on Cygwin)

curl -F "impTVFiles=checked" \
-F "impShowOverwrite=checked" \
-F "impMFOverwrite=checked" \
-F "impMFRename=keep" \
-F "impMFRedate=redateFromAiring" \
-F "xmlFile=@exemple.xml" \
-F "Import=submit" \
http://username:fpasswd@locahost:808...ter?Import=yes

Now if you can add the option to output the episode information in the good xml format, I'll be in Heaven, and so will my wife !!!

I just tried it out and it worked great. Give me some time to play around with that and I should be able to get it completely automated.

[Zorglub]

Cool !

Let me know if you need any help to test it or anything else...

Thanks !

[cmorrow]

Excellent tool EP! I have used this to make properties and .my files for all my DVDs without a problem.

One suggestion I have would be to allow downloading episode specific images for TV series from TVDB. I find them a lot better than having to use a poster for dozens of episodes in a series.

Also, not sure if this is by design, but if a high quality poster isn't available from the download site, it appears none get downloaded at all. I ended up pulling poster images for 1/3 of my DVDs from IMDB and using the properties files that were generated by mediaScraper.

[Phydeaux]

Any word yet on getting M*A*S*H to work?

RLW

[evilpenguin]

Originally Posted by Phydeaux

Any word yet on getting M*A*S*H to work?

RLW

MASH is a pretty bad one. Its at the intersection of an illegal win32 character and a reserved PERL character. I'm going to need to put some more thought into it.

[r00st3r]

I was working with Bill over at tvmetadatafinder.com and was able to have him incorporate saving a .my file when scraping for metadata. It renamed my avi files and i have sage using .my files for metadata. With a background.jpg in the foldeer, it looks great. That in conjunction with the mediascraper tool for movies, makes getting metadata easier for me. Just wished I could find ont to do it all automagically. Just thought I would let you know if it helps anyone else. Sorry in advance if this is the wrong topic.

[Rob]

Great tool. Thanks!

[mickp]

Originally Posted by evilpenguin

Yep, your changes should be pretty straight forward, be sure and let me know if you have any trouble.

Thanks E.P.

I've mucked about and by trial and error arrived at the following downloading.scrape which has been working well for the past week or so.

# Scraper: getSingleImageTV
#
#  INPUT: %%singleImage%%       - Image to download
#       : %%downloadTo%%        - Path to download image
#

    Profile           =getSingleImage
    Encode CLI #2     =?>!PATH:%%downloadTo%%<:>"%%downloadTo_PATH%%"<?
    Encoder #2        =/exe mkdir
    Encode CLI #2     =?>!EXISTS:%%downloadTo%%<:>"%%singleImage%%" -O "%%downloadTo%%"<?
    Encoder #2        =/exe wget.exe


# Scraper: fanartFile
#
#  INPUT: %%tvdbID%%       - theTVDB show ID
#

    Profile           =fanartFile
#   Encode CLI #1     =%%SNIP:fanartPath%%\?>centralFanartPath<:>%%showTitle_WIN32%%<=>isTV||(>videoTS&&isMovie<)<:>%%inputFile_NAME%%_background<=>%%inputFile_NAME%%_background<?.%%singleImage_EXT%%
    Encode CLI #1     =?>videoTS&&isMovie<:>%%inputFile%%\background.%%singleImage_EXT%%<=>%%inputFile_PATH%%\?>isMovie<:>%%inputFile_NAME%%_background<=>%%inputFile_NAME%%_background<?.%%singleImage_EXT%%<?
    Encoder #1        =

# Scraper: fanartLoc
#
#  INPUT: %%tvdbID%%       - theTVDB show ID
#

    Profile           =fanartPath
    Encode CLI #1     =?>centralFanartPath<:>%%centralFanartPath%%\?>isTV<:>TV<=>isMovie<:>Movies<?<=>?>videoTS&&isMovie<:>%%inputFile%%<=>%%inputFile_PATH%%<?<?
    Encoder #1        =


# Scraper: posterFile
#
#  INPUT: %%tvdbID%%       - theTVDB show ID
#

    Profile           =posterFile
    Encode CLI #1     =?>videoTS&&isMovie<:>%%inputFile%%\folder.%%singleImage_EXT%%<=>%%inputFile_PATH%%\?>isMovie<:>%%inputFile_NAME%%<=>%%inputFile_NAME%%<?.%%singleImage_EXT%%<?
    Encoder #1        =

# Scraper: bannerFile
#
#  INPUT: %%tvdbID%%       - theTVDB show ID
#

    Profile           =bannerFile
    Encode CLI #1     =%%inputFile_PATH%%\banner.%%singleImage_EXT%%
    Encoder #1        =

As i've no idea what I'm doing and couldn't get my head around the perl nuances required to realy understand what was going on, I'm confident that you could tidy this up somewhat . I'm pretty sure that I've borked it for central fanart.

Mick.

[jaminben]

How's importing xml files into SageTV going?

[JimmyBalboa]

Quick newbie quesiton...

I have it working, but for TV shows I would like to add Ratings and not download the graphic because it's easier to distinguish the shows by the thumbnail rather than seeing the same graphics for each episode and having to reading the title.

Can this be customized to Add ratings and remove the graphic?


Awesome tool by the way!!!!

Thanks...

[popechild]

Originally Posted by evilpenguin

Wow, i've never seen anyone organize their TV like that before. Let me play around with that zip4test file and see if I can't work something out for you, you've got it organized so well I don't think it will be a problem.

Hey EP, just wondering if you'd had a chance to think through how I might be able to use this for all my ripped-as-VIDEO_TS tv stuff. Thanks for any suggestions!

[evilpenguin]

Originally Posted by jaminben

How's importing xml files into SageTV going?

I've *kinda* got it working, but it lead me down a pretty complicated path.

See, to make the XML file properly, I need to do some math using the length of the video and to detect the length of the video I needed to merge over some code from my encoding tool. Then since I was already merging over *some* code, I figured I might as well merge *all* the code and things kinda snowballed from there

Long story short, its almost done, but I have to a lot of testing to make sure I didn't break anything.

On the plus side the next version will also be able to encode videos/DVD's and I won't have to maintain two code bases anymore, both of which make me very happy.

Originally Posted by popechild

Hey EP, just wondering if you'd had a chance to think through how I might be able to use this for all my ripped-as-VIDEO_TS tv stuff. Thanks for any suggestions!

Same as above, I've got it in, but its buried under so many other changes that I need to test the hell out of it before I can release anything.

Originally Posted by JimmyBalboa

Can this be customized to Add ratings and remove the graphic?

I'm capturing the TV rating, but last time I checked there wasn't a place to put it. I'll double check that.

As for the graphics, I've got the tool downloading theTVDB's video thumbnails now so that should solve your problem.

[JimmyBalboa]

Originally Posted by evilpenguin

I'm capturing the TV rating, but last time I checked there wasn't a place to put it. I'll double check that.

As for the graphics, I've got the tool downloading theTVDB's video thumbnails now so that should solve your problem.

I am a novice at meta data but, Rating=TVPG, etc... should work right?



Thanks!