Utility: mediaScraper - metadata made easy!

[popechild]

Originally Posted by jaminben

Thanks for the reply popechild

Is their a way to prevent the download of images and backgrounds? I don't remember seeing a post about this and I've just looked at the 1st/2nd post but found no answers.

I assume if you just don't use /downloadFanArt (or whatever that switch is) then it won't download anything. I asked earlier about specifying certain images to download and not others, but haven't heard anything about that yet.

I'm also noticing that I'm getting a .properties file and an .xml file placed in the shows folder. Is it possible to have these deleted automatically once the show .xml has been imported into Sage's DB?

Originally Posted by JimmyBalboa

No expert here, but logically thinking, the next time you refreshed your media library the meta data would be removed from those episodes if you delete those files.

I'd think if you use the xml to import through the webserver route, it's in there whether you remove the xml file or not. I've never used .properties files - which I believe are loaded into the db - so I don't know about those. I use the .my files, which aren't loaded anywhere but just get used when needed. If .properties files don't "stick" in the db though, I'm not sure I understand what advantage there is for .properties over .my. In other words, if the .properties has to be kept around for as long as the video is there, why bother loading the info into the db? Why not just use it on the fly like .my files? So I'm thinking it's must be at least somewhat more persistent than what you're saying, but I could be wrong.

[jaminben]

Originally Posted by evilpenguin

Specifically which images/backgrounds, I'm downloading so many. Right now /downloadFanArt grabs *everything* it can, but I could easily have a switch for disabling specific ones you don't want.

Without trying to sound too fussy, would it be possible to specify which images the user would want to use rather than disable the ones they dont want. So if I wanted coverart & fan art only I could just add /downloadbackground /downloadcoverart to the defaults file.

Other options could be:
/downloadbanner
/downloadseasoncoverart

Cheers

Ben

[popechild]

Originally Posted by evilpenguin

@Popechild - I still need to see logs and/or filenames for the problems your having so I can duplicate them.

Oh, missed this earlier. I'll try it again later and get something to you. Sorry about that.

[JimmyBalboa]

Originally Posted by evilpenguin

Have you tried restarting Sage on the Server, it may be an issue with image cache. It could also be an issue with absolute paths being used for recording drives rather than UNC.

I do use absolute paths... let me change that when I get home and see if that make a difference.

Thanks!

[popechild]

Originally Posted by jaminben

Without trying to sound too fussy, would it be possible to specify which images the user would want to use rather than disable the ones they dont want. So if I wanted backgrounds & fan art only I could just add /downloadbackground & downloadcoverart to the defaults file.

I guess it depends on whether most people want only one or two of the images or whether most will want all as to which will be more convenient. I'd like to only disable one of them (the screenshots, so that I can have Sage display the folder.jpg instead) and would rather not have to specify all of the others. But of course it could be done either way.

Speaking of the images, any reason why the new version downloads the cover art as poster.jpg instead of folder.jpg? At least I think it's poster.jpg now. Does Sage use poster.jpg, or was this just part of the way to make sure that the thumbnails get displayed now instead of the cover art? Maybe there could be a switch like /displayThumbnails that enabled downloading the thumbnails and naming the cover poster.jpg, and if the switch wasn't used it would just download the cover and call it folder.jpg.

Just thinking out loud.

[evilpenguin]

Originally Posted by jaminben

Without trying to sound too fussy, would it be possible to specify which images the user would want to use rather than disable the ones they dont want. So if I wanted coverart & fan art only I could just add /downloadbackground /downloadcoverart to the defaults file.

Other options could be:
/downloadbanner
/downloadseasoncoverart

Those are all good points. Once again, this in an area where I implemented stuff but didn't put much thought into usability.

How about, I add individual switches for enabling each image type as well as have one for downloading everything?

Originally Posted by popechild

I guess it depends on whether most people want only one or two of the images or whether most will want all as to which will be more convenient. I'd like to only disable one of them (the screenshots, so that I can have Sage display the folder.jpg instead) and would rather not have to specify all of the others. But of course it could be done either way.

Eh, i'm not to concerned about that since its something you only need to do once in the defaults.txt.

In regards to video thumbnail vs. folder.jpg I think i'll have it so if you have thumbnails enabled it will download the poster as poster.jpg and if you don't have thumbnails enabled it will download the poster as folder.jpg.

[jaminben]

Originally Posted by evilpenguin

How about, I add individual switches for enabling each image type as well as have one for downloading everything?

Sounds good to me. And if no switches exist it downloads nothing.

[Texas-Hansen]

Originally Posted by mickp

Sure. Appologies in advance if this is a bit basic.

Download the zip file and extract the two batch files (scrape.cmd and scrapAction.cmd) to the same directory that you have E.P's mediaScraper.exe file in.

launch a command prompt. There are various means to do this but if you're stuck press {windows key}+R and type "cmd" into the Run dialog that comes up and press enter.

At the command prompt type (assuming that you extracted it somewhere to teh C: drive)
"c:\whichever\directory\it\is\in\scrape \\ComputerName\ShareName" e.g. "c:\scraper\scrape \\myserver\mymovies"

NB: Swap the paths in the above example for your own. You could also use a local directory path rather than a share name e.g. c:\shares\mymovies rather than \\mypc\movies

The batch file will run mediaScraper.exe for each of the files in your movie share (and sub directories) which don't already have a .properties file.

During the process you will see some minimised command windows flashing on and off in the taskbar as the files are processed. A log file will be created in the root of the share that you're processing called ScrapeLog.txt

The log file will give you details of which files were processed, not processed and what metadata was saved.

The more obvious way to use the batch file would be to schedule it on a regular basis but that's another discussion. You could also drag a shortcut to the scrape.cmd batch file, edit the shortcut and add the path that you wish to process to the end of the target.

Please feel free to pm me if you have any questions or run into any problems. I'll be glad to work through the specifics with you.

Mick.

Dumb question...but I do not find the two batch files mentioned above - scrape.cmd and scrapAction.cmd - can someone point me in the right direction?

Thanks

[evilpenguin]

Originally Posted by Texas-Hansen

Dumb question...but I do not find the two batch files mentioned above - scrape.cmd and scrapAction.cmd - can someone point me in the right direction?

Thanks

You don't need to worry about that anymore, you can just drag the videos directly on top of mediaScraper.exe

[popechild]

Originally Posted by evilpenguin

I'll need to see scraper.log files for all the problems your having.

Okay, was finally able to run it again on a VIDEO_TS directory. The directory is a network directory that I setup as a network share (Z:\) to eliminate any possible issues with UNC paths. I dropped the directory Z:\TV Shows\Flight of the Conchords\Season 1 on the scraper. It ran through all 12 episodes (in folders named "01 - EpisodeName" etc.) and found random movie results for each episode as far as I can tell. It just doesn't seem to recognize that they're tv episodes of the show Flight of the Conchords, even though I have /baseFolder set to "Z:\TV Shows".

Anyway, see what you can figure out. I've attached the scraper.log.
Thanks!

[jaminben]

I've just noticed I don't get season and episode numbering when I import the .xml into Sage's database. Is their a setting I've missed? Or does it not support season/episode numbering when importing an xml?

Many Thanks

Ben

[evilpenguin]

Originally Posted by jaminben

I've just noticed I don't get season and episode numbering when I import the .xml into Sage's database. Is their a setting I've missed? Or does it not support season/episode numbering when importing an xml?

Many Thanks

Ben

I don't see anywhere in the SageAPI that they support Season/Episode numbers. Let me know if i'm missing something cause i'd love to include that.

Originally Posted by popechild

Anyway, see what you can figure out. I've attached the scraper.log.

Just use

/baseFolder "TV Shows"

Rather than the full path. Sorry, my directions were pretty unclear about that one.

[popechild]

That does it - thanks!

[jaminben]

Originally Posted by evilpenguin

Let me know if i'm missing something cause i'd love to include that.

Ah yes, My Bad. I assumed as some of my TV shows that I had recorded from TV had season/ Episode data that it was a part of the Sage Api. I see now its just part of the episode data, i.e "Cold Case season 2 epiosde 6 - Best Friends".
Thats put a spanner in the works for my new import Oh well back to the drawing board.

Cheers

Ben

edit:

I've had a thought, its going to upset a few people but would it be possible to have the season and episode text added to the shows episode name. So it would look something like: S02E08 - Double Booked rather than just Double Booked.

edit 2:

I have another favor to ask could you also grab season specific banners from the TVDB. If you do have the time and wouldn't mind doing this then it would also be handy if you could name these "season_banner.jpg"

[evilpenguin]

For the season/episode, we could just re-purpose one of the other properties fields that we're not using.

[jaminben]

Originally Posted by evilpenguin

For the season/episode, we could just re-purpose one of the other properties fields that we're not using.

I was thinking that but didn't like to say. Pick one and let me know what it is and I'll have a go at putting it into my import. If theirs any other usefull info that could be used from the scrape then add that as well

[msm]

aplogies if this is a stupid question but is there a way to get this to work with Sage recorded files? I dont have that many non-recorded files and it would be great to get the fanart for all of these.

[evilpenguin]

Originally Posted by jaminben

I was thinking that but didn't like to say. Pick one and let me know what it is and I'll have a go at putting it into my import. If theirs any other usefull info that could be used from the scrape then add that as well

How about...

GetShowCategory = Season Number
GetShowSubCategory = Episode Number

edit: nm, those are used for genre info.

How about we leave Category for one genre and use subCategory for S##E##? Then you can grab it, regex it, and sort by it. Also, if you could have it navigate like...

Show -> Season # -> Episodes

That would be the greatest thing ever. Having 86 episodes of House in a single folder is a bit of a pain.

[jaminben]

Originally Posted by evilpenguin

How about...

GetShowCategory = Season Number
GetShowSubCategory = Episode Number

GetShowCategory is currently being used.
GetShowSubCategory isn't being used.

How about:
GetShowSeriesInfo = Season Number
GetShowSubCategory = Episode Number

[jaminben]

Originally Posted by evilpenguin

How about...

GetShowCategory = Season Number
GetShowSubCategory = Episode Number

edit: nm, those are used for genre info.

How about we leave Category for one genre and use subCategory for S##E##? Then you can grab it, regex it, and sort by it. Also, if you could have it navigate like...

Show -> Season # -> Episodes

That would be the greatest thing ever. Having 86 episodes of House in a single folder is a bit of a pain.

Hmm, I was with you up until "Then you can grab it" everything else is a blur. I think your talking about filtering by Season and then Episode using category and subcategory, which I maybe able to do but I cannot promise. It would proberly be easier to start with just the basics.

Edit:

Its getting late her in the Uk so I'll have a think about it and get back to you. Feel free to add which everone you think will work and I'll have a go at fitting it in. It can't hurt to try