SageTV Community

SageTV Community (http://forums.sagetv.com/forums/index.php)
-   SageTV Customizations (http://forums.sagetv.com/forums/forumdisplay.php?f=27)
-   -   Utility: mediaScraper - metadata made easy! (http://forums.sagetv.com/forums/showthread.php?t=38386)

evilpenguin 01-09-2009 12:41 PM

Ok, we'll try two more things.

This *new* link, and PM me your email address and i'll email it to you. One of those HAS to work :goodjob:

Edit: Also, can you attach the scraper.log file that gets made to the thread?
Edit 2: You and MickP are the two who had the most trouble with the download so I wonder if its a Europe server that's the problem. That would explain why no one else is having problems.

jaminben 01-09-2009 12:52 PM

Quote:

Originally Posted by evilpenguin (Post 328894)
Ok, we'll try two more things.

This *new* link, and PM me your email address and i'll email it to you. One of those HAS to work :goodjob:

Edit: Also, can you attach the scraper.log file that gets made to the thread?

New link works great, file extracted fine :thumb:

When I drag a file over it does seem to be working but after a while it stops and says "out of memory". Not sure what that means as I'm running vista with 4Gb of ram and plenty of HDD space.

Anyway I've attached the latest log. I'm also in the middle of upgrading my server so I havent been able to give it my full attention and will have more time later to try again.

evilpenguin 01-09-2009 12:55 PM

Oh wow, *huge* bug on my part. I'll get it fixed right away.

Edit: ok updated the download, should be mediaScraperBeta3.zip. That was a bad one, I was actually trying to read in the entire video rather than just use its file name.

This should also fix those other issues people have reported.

jaminben 01-09-2009 02:23 PM

Sweet, beta3 worked first time :thumb:

This is one cool tool.

Thankyou

dzzza 01-09-2009 03:10 PM

1 Attachment(s)
excellent tool...:goodjob:..though i haven't been able to get it to work perfectly yet.

i can get this to work just dragging in one file at a time, but when i drag in a whole folder the utility doesn't seem to work (i've attached my log file of this instance). also, if i drag in a selection set of files (not a folder, but say 3 files at once) the utility will only process one of the files.

is this utility capable of processing multiple files at once? i'm using the Beta3 version.

thanks for any input...

evilpenguin 01-09-2009 03:55 PM

That's not going to work, I haven't added in support for multiple files or folders yet. I plan on getting to it pretty soon though.

MickP posted some useful batchfiles that should hold you over until then.

evilpenguin 01-09-2009 05:48 PM

Just something I'm noticing as I'm playing around: IMDB's popular matches are amazingly accurate when your title's not quite perfect.

For instance, if you have your movie labeled LOTR - Return Of the King, which I imagine is a pretty common way of naming that movie, I normally wouldn't be able to get any hits without doing a lot of questionable guess work on my side. But if I do a IMDB search the first Popular match is The Lord of the Rings: The Return of the King.

I saw S_M_S had another good example in the other metadata thread where he has a movie labeled with the US title, Kung Fu Hustle, and strict IMDB searches are failing because they returns the movie's International title which is just Kung Fu.

The same thing works for TV. My biggest issue right now is with Terminator: The Sarah Conner Chronicles because theTVDB search engine isn't smart enough to recognize it without the ":" as it would be if I were pulling the title out of the file name. But when I do an IMDB search on Terminator The Sarah Conner Chronicles (or even just The Sarah Conner Chronicles) the first match that comes up is Terminator: The Sarah Connor Chronicles. So if I capture the IMDB title, bring it back to theTVDB, and then search with it I'll be able to get an exact hit.

I'll play around with this a bit over the weekend, but if anyone has any files named casually or missing illegal windows characters, I'd be interested to see what IMDB comes up with as its first hit.

mickp 01-10-2009 02:28 AM

Quote:

Originally Posted by evilpenguin (Post 328993)
Just something I'm noticing as I'm playing around: IMDB's popular matches are amazingly accurate when your title's not quite perfect.

For instance, if you have your movie labeled LOTR - Return Of the King, which I imagine is a pretty common way of naming that movie, I normally wouldn't be able to get any hits without doing a lot of questionable guess work on my side. But if I do a IMDB search the first Popular match is The Lord of the Rings: The Return of the King.

I saw S_M_S had another good example in the other metadata thread where he has a movie labeled with the US title, Kung Fu Hustle, and strict IMDB searches are failing because they returns the movie's International title which is just Kung Fu.

The same thing works for TV. My biggest issue right now is with Terminator: The Sarah Conner Chronicles because theTVDB search engine isn't smart enough to recognize it without the ":" as it would be if I were pulling the title out of the file name. But when I do an IMDB search on Terminator The Sarah Conner Chronicles (or even just The Sarah Conner Chronicles) the first match that comes up is Terminator: The Sarah Connor Chronicles. So if I capture the IMDB title, bring it back to theTVDB, and then search with it I'll be able to get an exact hit.

I'll play around with this a bit over the weekend, but if anyone has any files named casually or missing illegal windows characters, I'd be interested to see what IMDB comes up with as its first hit.

Does this mean that there's a new version available with movie support?

PS: Re your earlier musings whether to download graphics for each show +5 from me. Could you add a switch to have all available graphics downloaded?

jaminben 01-10-2009 04:06 AM

Quote:

Originally Posted by mickp (Post 327747)
Usage is scrape [unc path]

Could you explain in a little more detail how this works for the less clever people like me please :thumb:

mickp 01-10-2009 04:44 AM

Quote:

Originally Posted by jaminben (Post 329096)
Could you explain in a little more detail how this works for the less clever people like me please :thumb:

Sure. Appologies in advance if this is a bit basic.

Download the zip file and extract the two batch files (scrape.cmd and scrapAction.cmd) to the same directory that you have E.P's mediaScraper.exe file in.

launch a command prompt. There are various means to do this but if you're stuck press {windows key}+R and type "cmd" into the Run dialog that comes up and press enter.

At the command prompt type (assuming that you extracted it somewhere to teh C: drive)
"c:\whichever\directory\it\is\in\scrape \\ComputerName\ShareName" e.g. "c:\scraper\scrape \\myserver\mymovies"

NB: Swap the paths in the above example for your own. You could also use a local directory path rather than a share name e.g. c:\shares\mymovies rather than \\mypc\movies

The batch file will run mediaScraper.exe for each of the files in your movie share (and sub directories) which don't already have a .properties file.

During the process you will see some minimised command windows flashing on and off in the taskbar as the files are processed. A log file will be created in the root of the share that you're processing called ScrapeLog.txt

The log file will give you details of which files were processed, not processed and what metadata was saved.

The more obvious way to use the batch file would be to schedule it on a regular basis but that's another discussion. You could also drag a shortcut to the scrape.cmd batch file, edit the shortcut and add the path that you wish to process to the end of the target.

Please feel free to pm me if you have any questions or run into any problems. I'll be glad to work through the specifics with you.

Mick.

NB: You don't need to use this batch file(s) any more as mediascraper.exe will now process all files in a directory.

jaminben 01-10-2009 05:09 AM

1 Attachment(s)
Quote:

Originally Posted by mickp (Post 329099)
Please feel free to pm me if you have any questions or run into any problems. I'll be glad to work through the specifics with you.

To save me anymore embarasment with such a simple task I thought it would be better to PM you :D However it doesnt support uploading of images so I guess I'm stuck with the embarasment :(

I have both files in the same folder as the scraper and when I try to run it with

Quote:

C:\Users\jaminben\Desktop\mediaScraperBeta3\scraper\scrape C:\Users\jaminben\Desktop\TV\Stargate Atlantis\Season 5
I get "The system cannot find the path specified".

I've attached a screenshot which proberly explains better what I'm doing.

Thanks for the help :thumb:

mickp 01-10-2009 05:46 AM

Hey jaminben,

From what I can see in your screenshot you probably just need to try;

C:\Users\jaminben\Desktop\mediaScraperBeta3\scrape C:\Users\jaminben\Desktop\TV\Stargate Atlantis\Season 5

The scraper directory in my example was equivalent to your "mediaScraperBeta3" directory.

Hope this helps.

Mick.

jaminben 01-10-2009 06:26 AM

Quote:

Originally Posted by mickp (Post 329103)
Hey jaminben,

From what I can see in your screenshot you probably just need to try;

C:\Users\jaminben\Desktop\mediaScraperBeta3\scrape C:\Users\jaminben\Desktop\TV\Stargate Atlantis\Season 5

The scraper directory in my example was equivalent to your "mediaScraperBeta3" directory.

Hope this helps.

Mick.

Nice one, that works :thumb:

Thankyou

mickp 01-10-2009 06:34 AM

Quote:

Originally Posted by jaminben (Post 329104)
Nice one, that works :thumb:

Thankyou

Cool! No worries.

Mick.

dzzza 01-10-2009 09:21 AM

Quote:

Originally Posted by evilpenguin (Post 328962)
That's not going to work, I haven't added in support for multiple files or folders yet. I plan on getting to it pretty soon though.

MickP posted some useful batchfiles that should hold you over until then.

thanks ep...the batch files do the trick, and good work on this utility!

evilpenguin 01-10-2009 12:33 PM

Hey, imdb is working, but pulling the title out of file names isn't yet. I'll have it out by Sunday.

Halstead 01-11-2009 12:40 AM

EP-
Thanks so much for creating this. One possible way to deal with 'difficult' TV Show names would be to do something like what the old tv.com scraper did- have a .txt. file in the root show directory. With it you could either:

- Have the show name as plain text, or
- Have the TVDB show ID, e.g 73545 for Battlestar Galactica (2003)

By having the scraper look there first, you could probably solve all sorts of ugly edge cases.

Oh, and can this be set up to work with DirMon at this point?

Best,

megamojo 01-11-2009 02:34 PM

Quote:

Originally Posted by Halstead (Post 329340)
Oh, and can this be set up to work with DirMon at this point?

Seconded. I've tried every possible way of calling this with DirMon, using mediascraper.exe directly, using the 2 wonderful batch files, using batch files that call those batch files, and anything else I can think of. I can't get anything to work with dirmon for the life of me. Is there something I'm doing wrong? Is there any way mediascraper.exe or scrape.cmd could be made to accept a couple parameters, such as -p for path or something like that?

evilpenguin 01-11-2009 07:19 PM

Dirmon2 may be a problem because when it launches the working directory is C:\system\window32 rather than where the .exe is. I'll check on that.

evilpenguin 01-12-2009 11:17 PM

Updated to Beta4: Movies!
 
I updated to Beta4 and added support for movies from imdb and theMovieDB.

I also added a pretty experimental feature where if it can't find an exact match on the title name from your file it'll just take IMDB's most popular match. I've found that this greatly increases hits on videos that are named more casually, but it will also cause thing named very poorly to be mis-hits. Let me know if anyone has any serious issues with this and I'll look into adjusting it accordingly.


All times are GMT -6. The time now is 11:18 AM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.