SageTV Community

SageTV Community (http://forums.sagetv.com/forums/index.php)
-   SageTV Customizations (http://forums.sagetv.com/forums/forumdisplay.php?f=27)
-   -   Utility: mediaScraper - metadata made easy! (http://forums.sagetv.com/forums/showthread.php?t=38386)

jaminben 01-30-2009 02:55 PM

Yep, that sounds very cool :thumb:

Zorglub 01-30-2009 03:26 PM

Quote:

Originally Posted by evilpenguin (Post 334400)
I like the sound of that, can you go into more detail on you would use these XML files to import the shows into SageTV? I'm not familiar with it.

Sure, you just use nielm webserver to enter it in the database:

http://localhost:8080/sage/XMLImporter

Then you checked the "Import TV files" box and checked those options:

* Overwrite Existing Show Information
* Overwrite Existing Sage TV File Information
* Filename handling:
o Keep original imported filename
* File timestamp handling
o Use imported Airing data to determine startTime
(use when importing edited files. Will change imported file timestamp)

You then upload your xml file, and VOILA ! your video file is refered in Recorded TV with all the right info (thanks to you ;)) and you can start it the same way you would with any other file...

Of course I am presently doing this manually, but I am looking for a way to call this via a command line, something like:

wget.exe "http://username:ypassword@localhost:8080/sage/XMLImporter&&FileName=N:\MEDIA\Videos\TV\Battlestar Galactica\Season 4\Battlestar Galactica (2003) 4x12 - A Disquiet Follows My Soul.xml"

so I can include it in a batch file and automatize the whole thing...

I am still trying to figure out how this last part can be done, but I am convince there is a way... I may need help on this one... :eek:

Zorglub 01-30-2009 08:37 PM

Update:

After some reading on curl (which seem to be able to handle POST better than wget) and a lot of fiddling around, I came up with the good syntax:

(FWI: this has been tested on Cygwin)

curl -F "impTVFiles=checked" \
-F "impShowOverwrite=checked" \
-F "impMFOverwrite=checked" \
-F "impMFRename=keep" \
-F "impMFRedate=redateFromAiring" \
-F "xmlFile=@exemple.xml" \
-F "Import=submit" \
http://username:fpasswd@locahost:808...ter?Import=yes

Now if you can add the option to output the episode information in the good xml format, I'll be in Heaven, and so will my wife !!! :D

Muchacho 01-31-2009 10:44 AM

downloading posters
 
This is fantastic! Great job and thanks.

After messing around with it for a while, I think that there is a bug with downloading the posters for movies. It generally works great, but I noticed that it does not d/l a poster if there is not a high res poster on themoviedb.org. It would seem that it should (or hopefully could) download the available lower resolution image or the one from imdb if a high res poster is not available.

Examples that did not work (mostly stuff my wife recordedů)
-The Last Kiss (has low res poster on themoviedb)
-Y Tu Mama Tambien (no poster on themoviedb, but there is one available on imdb)
-Capote (nothing available on themoviedb)

Am I missing something? Also, does sageTV periodically update its database and include the new posters, backgrounds, and .properties? It does not seem to be updating itself even after I refresh the imported folders.

Thanks again, Mike

evilpenguin 01-31-2009 12:43 PM

Quote:

Originally Posted by Muchacho (Post 334566)
It would seem that it should (or hopefully could) download the available lower resolution image or the one from imdb if a high res poster is not available

Well, not so much a bug as a feature I never bothered to implement. I'll put it on the ol' todo list.

Muchacho 01-31-2009 03:35 PM

Quote:

Originally Posted by evilpenguin (Post 334600)
Well, not so much a bug as a feature I never bothered to implement. I'll put it on the ol' todo list.

Fantastic!

Much appreciated.

popechild 01-31-2009 05:05 PM

Quote:

Originally Posted by Muchacho (Post 334566)
This is fantastic! Great job and thanks.
Examples that did not work (mostly stuff my wife recordedů)

Ha ha!

evilpenguin 01-31-2009 07:54 PM

Quote:

Originally Posted by Zorglub (Post 334480)
Update:

After some reading on curl (which seem to be able to handle POST better than wget) and a lot of fiddling around, I came up with the good syntax:

(FWI: this has been tested on Cygwin)

curl -F "impTVFiles=checked" \
-F "impShowOverwrite=checked" \
-F "impMFOverwrite=checked" \
-F "impMFRename=keep" \
-F "impMFRedate=redateFromAiring" \
-F "xmlFile=@exemple.xml" \
-F "Import=submit" \
http://username:fpasswd@locahost:808...ter?Import=yes

Now if you can add the option to output the episode information in the good xml format, I'll be in Heaven, and so will my wife !!! :D

I just tried it out and it worked great. Give me some time to play around with that and I should be able to get it completely automated.

Zorglub 01-31-2009 08:10 PM

Cool !:clap:

Let me know if you need any help to test it or anything else...

Thanks !

cmorrow 01-31-2009 08:29 PM

Possible to download Episode Images
 
Excellent tool EP! I have used this to make properties and .my files for all my DVDs without a problem.

One suggestion I have would be to allow downloading episode specific images for TV series from TVDB. I find them a lot better than having to use a poster for dozens of episodes in a series.

Also, not sure if this is by design, but if a high quality poster isn't available from the download site, it appears none get downloaded at all. I ended up pulling poster images for 1/3 of my DVDs from IMDB and using the properties files that were generated by mediaScraper.

Phydeaux 02-02-2009 12:29 PM

Any word yet on getting M*A*S*H to work?

RLW

evilpenguin 02-02-2009 01:31 PM

Quote:

Originally Posted by Phydeaux (Post 335056)
Any word yet on getting M*A*S*H to work?

RLW

MASH is a pretty bad one. Its at the intersection of an illegal win32 character and a reserved PERL character. I'm going to need to put some more thought into it.

r00st3r 02-03-2009 06:13 PM

I was working with Bill over at tvmetadatafinder.com and was able to have him incorporate saving a .my file when scraping for metadata. It renamed my avi files and i have sage using .my files for metadata. With a background.jpg in the foldeer, it looks great. That in conjunction with the mediascraper tool for movies, makes getting metadata easier for me. Just wished I could find ont to do it all automagically. Just thought I would let you know if it helps anyone else. Sorry in advance if this is the wrong topic.

Rob 02-03-2009 07:33 PM

Great tool. Thanks!

mickp 02-06-2009 04:01 AM

Quote:

Originally Posted by evilpenguin (Post 333852)
Yep, your changes should be pretty straight forward, be sure and let me know if you have any trouble.

Thanks E.P.

I've mucked about and by trial and error arrived at the following downloading.scrape which has been working well for the past week or so.

Code:

# Scraper: getSingleImageTV
#
#  INPUT: %%singleImage%%      - Image to download
#      : %%downloadTo%%        - Path to download image
#

    Profile          =getSingleImage
    Encode CLI #2    =?>!PATH:%%downloadTo%%<:>"%%downloadTo_PATH%%"<?
    Encoder #2        =/exe mkdir
    Encode CLI #2    =?>!EXISTS:%%downloadTo%%<:>"%%singleImage%%" -O "%%downloadTo%%"<?
    Encoder #2        =/exe wget.exe


# Scraper: fanartFile
#
#  INPUT: %%tvdbID%%      - theTVDB show ID
#

    Profile          =fanartFile
#  Encode CLI #1    =%%SNIP:fanartPath%%\?>centralFanartPath<:>%%showTitle_WIN32%%<=>isTV||(>videoTS&&isMovie<)<:>%%inputFile_NAME%%_background<=>%%inputFile_NAME%%_background<?.%%singleImage_EXT%%
    Encode CLI #1    =?>videoTS&&isMovie<:>%%inputFile%%\background.%%singleImage_EXT%%<=>%%inputFile_PATH%%\?>isMovie<:>%%inputFile_NAME%%_background<=>%%inputFile_NAME%%_background<?.%%singleImage_EXT%%<?
    Encoder #1        =

# Scraper: fanartLoc
#
#  INPUT: %%tvdbID%%      - theTVDB show ID
#

    Profile          =fanartPath
    Encode CLI #1    =?>centralFanartPath<:>%%centralFanartPath%%\?>isTV<:>TV<=>isMovie<:>Movies<?<=>?>videoTS&&isMovie<:>%%inputFile%%<=>%%inputFile_PATH%%<?<?
    Encoder #1        =


# Scraper: posterFile
#
#  INPUT: %%tvdbID%%      - theTVDB show ID
#

    Profile          =posterFile
    Encode CLI #1    =?>videoTS&&isMovie<:>%%inputFile%%\folder.%%singleImage_EXT%%<=>%%inputFile_PATH%%\?>isMovie<:>%%inputFile_NAME%%<=>%%inputFile_NAME%%<?.%%singleImage_EXT%%<?
    Encoder #1        =

# Scraper: bannerFile
#
#  INPUT: %%tvdbID%%      - theTVDB show ID
#

    Profile          =bannerFile
    Encode CLI #1    =%%inputFile_PATH%%\banner.%%singleImage_EXT%%
    Encoder #1        =

As i've no idea what I'm doing and couldn't get my head around the perl nuances required to realy understand what was going on, I'm confident that you could tidy this up somewhat :). I'm pretty sure that I've borked it for central fanart.

Mick.

jaminben 02-06-2009 04:56 AM

How's importing xml files into SageTV going?

JimmyBalboa 02-06-2009 09:23 AM

Quick newbie quesiton...

I have it working, but for TV shows I would like to add Ratings and not download the graphic because it's easier to distinguish the shows by the thumbnail rather than seeing the same graphics for each episode and having to reading the title.

Can this be customized to Add ratings and remove the graphic?


Awesome tool by the way!!!!

Thanks...

popechild 02-06-2009 09:30 AM

Quote:

Originally Posted by evilpenguin (Post 333814)
Wow, i've never seen anyone organize their TV like that before. Let me play around with that zip4test file and see if I can't work something out for you, you've got it organized so well I don't think it will be a problem.

Hey EP, just wondering if you'd had a chance to think through how I might be able to use this for all my ripped-as-VIDEO_TS tv stuff. Thanks for any suggestions!

evilpenguin 02-06-2009 01:32 PM

Quote:

Originally Posted by jaminben (Post 336132)
How's importing xml files into SageTV going?

I've *kinda* got it working, but it lead me down a pretty complicated path.

See, to make the XML file properly, I need to do some math using the length of the video and to detect the length of the video I needed to merge over some code from my encoding tool. Then since I was already merging over *some* code, I figured I might as well merge *all* the code and things kinda snowballed from there :)

Long story short, its almost done, but I have to a lot of testing to make sure I didn't break anything.

On the plus side the next version will also be able to encode videos/DVD's and I won't have to maintain two code bases anymore, both of which make me very happy.

Quote:

Originally Posted by popechild (Post 336167)
Hey EP, just wondering if you'd had a chance to think through how I might be able to use this for all my ripped-as-VIDEO_TS tv stuff. Thanks for any suggestions!

Same as above, I've got it in, but its buried under so many other changes that I need to test the hell out of it before I can release anything.

Quote:

Originally Posted by JimmyBalboa (Post 336163)
Can this be customized to Add ratings and remove the graphic?

I'm capturing the TV rating, but last time I checked there wasn't a place to put it. I'll double check that.

As for the graphics, I've got the tool downloading theTVDB's video thumbnails now so that should solve your problem.

JimmyBalboa 02-06-2009 01:42 PM

Quote:

Originally Posted by evilpenguin (Post 336220)
I'm capturing the TV rating, but last time I checked there wasn't a place to put it. I'll double check that.

As for the graphics, I've got the tool downloading theTVDB's video thumbnails now so that should solve your problem.

I am a novice at meta data but, Rating=TVPG, etc... should work right?



Thanks!


All times are GMT -6. The time now is 02:27 AM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.