SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV Customizations

Notices

SageTV Customizations This forums is for discussing and sharing user-created modifications for the SageTV application created by using the SageTV Studio or through the use of external plugins. Use this forum to discuss customizations for SageTV version 6 and earlier, or for the SageTV3 UI.

Reply
 
Thread Tools Search this Thread Display Modes
  #41  
Old 01-09-2009, 12:41 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Ok, we'll try two more things.

This *new* link, and PM me your email address and i'll email it to you. One of those HAS to work

Edit: Also, can you attach the scraper.log file that gets made to the thread?
Edit 2: You and MickP are the two who had the most trouble with the download so I wonder if its a Europe server that's the problem. That would explain why no one else is having problems.

Last edited by evilpenguin; 01-09-2009 at 01:05 PM.
Reply With Quote
  #42  
Old 01-09-2009, 12:52 PM
jaminben jaminben is offline
Sage Icon
 
Join Date: Sep 2007
Location: Norwich, UK
Posts: 1,754
Send a message via MSN to jaminben
Quote:
Originally Posted by evilpenguin View Post
Ok, we'll try two more things.

This *new* link, and PM me your email address and i'll email it to you. One of those HAS to work

Edit: Also, can you attach the scraper.log file that gets made to the thread?
New link works great, file extracted fine

When I drag a file over it does seem to be working but after a while it stops and says "out of memory". Not sure what that means as I'm running vista with 4Gb of ram and plenty of HDD space.

Anyway I've attached the latest log. I'm also in the middle of upgrading my server so I havent been able to give it my full attention and will have more time later to try again.
__________________
Server - Win7 64bit, 2.4Ghz Intel Core 2 Duo, TBS 6284 PCI-E Quad DVB-T2 Tuner, 3 x HD200 & 1 x HD300 extenders
Reply With Quote
  #43  
Old 01-09-2009, 12:55 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Oh wow, *huge* bug on my part. I'll get it fixed right away.

Edit: ok updated the download, should be mediaScraperBeta3.zip. That was a bad one, I was actually trying to read in the entire video rather than just use its file name.

This should also fix those other issues people have reported.

Last edited by evilpenguin; 01-09-2009 at 01:46 PM.
Reply With Quote
  #44  
Old 01-09-2009, 02:23 PM
jaminben jaminben is offline
Sage Icon
 
Join Date: Sep 2007
Location: Norwich, UK
Posts: 1,754
Send a message via MSN to jaminben
Sweet, beta3 worked first time

This is one cool tool.

Thankyou
__________________
Server - Win7 64bit, 2.4Ghz Intel Core 2 Duo, TBS 6284 PCI-E Quad DVB-T2 Tuner, 3 x HD200 & 1 x HD300 extenders
Reply With Quote
  #45  
Old 01-09-2009, 03:10 PM
dzzza dzzza is offline
Sage User
 
Join Date: Oct 2005
Posts: 5
excellent tool.....though i haven't been able to get it to work perfectly yet.

i can get this to work just dragging in one file at a time, but when i drag in a whole folder the utility doesn't seem to work (i've attached my log file of this instance). also, if i drag in a selection set of files (not a folder, but say 3 files at once) the utility will only process one of the files.

is this utility capable of processing multiple files at once? i'm using the Beta3 version.

thanks for any input...
Attached Files
File Type: txt scraper.txt (36.4 KB, 303 views)
Reply With Quote
  #46  
Old 01-09-2009, 03:55 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
That's not going to work, I haven't added in support for multiple files or folders yet. I plan on getting to it pretty soon though.

MickP posted some useful batchfiles that should hold you over until then.
Reply With Quote
  #47  
Old 01-09-2009, 05:48 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Just something I'm noticing as I'm playing around: IMDB's popular matches are amazingly accurate when your title's not quite perfect.

For instance, if you have your movie labeled LOTR - Return Of the King, which I imagine is a pretty common way of naming that movie, I normally wouldn't be able to get any hits without doing a lot of questionable guess work on my side. But if I do a IMDB search the first Popular match is The Lord of the Rings: The Return of the King.

I saw S_M_S had another good example in the other metadata thread where he has a movie labeled with the US title, Kung Fu Hustle, and strict IMDB searches are failing because they returns the movie's International title which is just Kung Fu.

The same thing works for TV. My biggest issue right now is with Terminator: The Sarah Conner Chronicles because theTVDB search engine isn't smart enough to recognize it without the ":" as it would be if I were pulling the title out of the file name. But when I do an IMDB search on Terminator The Sarah Conner Chronicles (or even just The Sarah Conner Chronicles) the first match that comes up is Terminator: The Sarah Connor Chronicles. So if I capture the IMDB title, bring it back to theTVDB, and then search with it I'll be able to get an exact hit.

I'll play around with this a bit over the weekend, but if anyone has any files named casually or missing illegal windows characters, I'd be interested to see what IMDB comes up with as its first hit.

Last edited by evilpenguin; 01-09-2009 at 05:56 PM.
Reply With Quote
  #48  
Old 01-10-2009, 02:28 AM
mickp's Avatar
mickp mickp is offline
Sage Aficionado
 
Join Date: Oct 2006
Posts: 468
Quote:
Originally Posted by evilpenguin View Post
Just something I'm noticing as I'm playing around: IMDB's popular matches are amazingly accurate when your title's not quite perfect.

For instance, if you have your movie labeled LOTR - Return Of the King, which I imagine is a pretty common way of naming that movie, I normally wouldn't be able to get any hits without doing a lot of questionable guess work on my side. But if I do a IMDB search the first Popular match is The Lord of the Rings: The Return of the King.

I saw S_M_S had another good example in the other metadata thread where he has a movie labeled with the US title, Kung Fu Hustle, and strict IMDB searches are failing because they returns the movie's International title which is just Kung Fu.

The same thing works for TV. My biggest issue right now is with Terminator: The Sarah Conner Chronicles because theTVDB search engine isn't smart enough to recognize it without the ":" as it would be if I were pulling the title out of the file name. But when I do an IMDB search on Terminator The Sarah Conner Chronicles (or even just The Sarah Conner Chronicles) the first match that comes up is Terminator: The Sarah Connor Chronicles. So if I capture the IMDB title, bring it back to theTVDB, and then search with it I'll be able to get an exact hit.

I'll play around with this a bit over the weekend, but if anyone has any files named casually or missing illegal windows characters, I'd be interested to see what IMDB comes up with as its first hit.
Does this mean that there's a new version available with movie support?

PS: Re your earlier musings whether to download graphics for each show +5 from me. Could you add a switch to have all available graphics downloaded?
Reply With Quote
  #49  
Old 01-10-2009, 04:06 AM
jaminben jaminben is offline
Sage Icon
 
Join Date: Sep 2007
Location: Norwich, UK
Posts: 1,754
Send a message via MSN to jaminben
Quote:
Originally Posted by mickp View Post
Usage is scrape [unc path]
Could you explain in a little more detail how this works for the less clever people like me please
__________________
Server - Win7 64bit, 2.4Ghz Intel Core 2 Duo, TBS 6284 PCI-E Quad DVB-T2 Tuner, 3 x HD200 & 1 x HD300 extenders
Reply With Quote
  #50  
Old 01-10-2009, 04:44 AM
mickp's Avatar
mickp mickp is offline
Sage Aficionado
 
Join Date: Oct 2006
Posts: 468
Quote:
Originally Posted by jaminben View Post
Could you explain in a little more detail how this works for the less clever people like me please
Sure. Appologies in advance if this is a bit basic.

Download the zip file and extract the two batch files (scrape.cmd and scrapAction.cmd) to the same directory that you have E.P's mediaScraper.exe file in.

launch a command prompt. There are various means to do this but if you're stuck press {windows key}+R and type "cmd" into the Run dialog that comes up and press enter.

At the command prompt type (assuming that you extracted it somewhere to teh C: drive)
"c:\whichever\directory\it\is\in\scrape \\ComputerName\ShareName" e.g. "c:\scraper\scrape \\myserver\mymovies"

NB: Swap the paths in the above example for your own. You could also use a local directory path rather than a share name e.g. c:\shares\mymovies rather than \\mypc\movies

The batch file will run mediaScraper.exe for each of the files in your movie share (and sub directories) which don't already have a .properties file.

During the process you will see some minimised command windows flashing on and off in the taskbar as the files are processed. A log file will be created in the root of the share that you're processing called ScrapeLog.txt

The log file will give you details of which files were processed, not processed and what metadata was saved.

The more obvious way to use the batch file would be to schedule it on a regular basis but that's another discussion. You could also drag a shortcut to the scrape.cmd batch file, edit the shortcut and add the path that you wish to process to the end of the target.

Please feel free to pm me if you have any questions or run into any problems. I'll be glad to work through the specifics with you.

Mick.

NB: You don't need to use this batch file(s) any more as mediascraper.exe will now process all files in a directory.

Last edited by mickp; 02-14-2009 at 08:40 PM.
Reply With Quote
  #51  
Old 01-10-2009, 05:09 AM
jaminben jaminben is offline
Sage Icon
 
Join Date: Sep 2007
Location: Norwich, UK
Posts: 1,754
Send a message via MSN to jaminben
Quote:
Originally Posted by mickp View Post
Please feel free to pm me if you have any questions or run into any problems. I'll be glad to work through the specifics with you.
To save me anymore embarasment with such a simple task I thought it would be better to PM you However it doesnt support uploading of images so I guess I'm stuck with the embarasment

I have both files in the same folder as the scraper and when I try to run it with

Quote:
C:\Users\jaminben\Desktop\mediaScraperBeta3\scraper\scrape C:\Users\jaminben\Desktop\TV\Stargate Atlantis\Season 5
I get "The system cannot find the path specified".

I've attached a screenshot which proberly explains better what I'm doing.

Thanks for the help
Attached Images
File Type: jpg image1.jpg (161.7 KB, 341 views)
__________________
Server - Win7 64bit, 2.4Ghz Intel Core 2 Duo, TBS 6284 PCI-E Quad DVB-T2 Tuner, 3 x HD200 & 1 x HD300 extenders
Reply With Quote
  #52  
Old 01-10-2009, 05:46 AM
mickp's Avatar
mickp mickp is offline
Sage Aficionado
 
Join Date: Oct 2006
Posts: 468
Hey jaminben,

From what I can see in your screenshot you probably just need to try;

C:\Users\jaminben\Desktop\mediaScraperBeta3\scrape C:\Users\jaminben\Desktop\TV\Stargate Atlantis\Season 5

The scraper directory in my example was equivalent to your "mediaScraperBeta3" directory.

Hope this helps.

Mick.

Last edited by mickp; 01-10-2009 at 05:59 AM. Reason: Typo
Reply With Quote
  #53  
Old 01-10-2009, 06:26 AM
jaminben jaminben is offline
Sage Icon
 
Join Date: Sep 2007
Location: Norwich, UK
Posts: 1,754
Send a message via MSN to jaminben
Quote:
Originally Posted by mickp View Post
Hey jaminben,

From what I can see in your screenshot you probably just need to try;

C:\Users\jaminben\Desktop\mediaScraperBeta3\scrape C:\Users\jaminben\Desktop\TV\Stargate Atlantis\Season 5

The scraper directory in my example was equivalent to your "mediaScraperBeta3" directory.

Hope this helps.

Mick.
Nice one, that works

Thankyou
__________________
Server - Win7 64bit, 2.4Ghz Intel Core 2 Duo, TBS 6284 PCI-E Quad DVB-T2 Tuner, 3 x HD200 & 1 x HD300 extenders
Reply With Quote
  #54  
Old 01-10-2009, 06:34 AM
mickp's Avatar
mickp mickp is offline
Sage Aficionado
 
Join Date: Oct 2006
Posts: 468
Quote:
Originally Posted by jaminben View Post
Nice one, that works

Thankyou
Cool! No worries.

Mick.
Reply With Quote
  #55  
Old 01-10-2009, 09:21 AM
dzzza dzzza is offline
Sage User
 
Join Date: Oct 2005
Posts: 5
Quote:
Originally Posted by evilpenguin View Post
That's not going to work, I haven't added in support for multiple files or folders yet. I plan on getting to it pretty soon though.

MickP posted some useful batchfiles that should hold you over until then.
thanks ep...the batch files do the trick, and good work on this utility!
Reply With Quote
  #56  
Old 01-10-2009, 12:33 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Hey, imdb is working, but pulling the title out of file names isn't yet. I'll have it out by Sunday.
Reply With Quote
  #57  
Old 01-11-2009, 12:40 AM
Halstead Halstead is offline
Sage Advanced User
 
Join Date: Jul 2005
Location: Portland, Or
Posts: 183
EP-
Thanks so much for creating this. One possible way to deal with 'difficult' TV Show names would be to do something like what the old tv.com scraper did- have a .txt. file in the root show directory. With it you could either:

- Have the show name as plain text, or
- Have the TVDB show ID, e.g 73545 for Battlestar Galactica (2003)

By having the scraper look there first, you could probably solve all sorts of ugly edge cases.

Oh, and can this be set up to work with DirMon at this point?

Best,
__________________
Halstead York

HTPC/Server: A8 3850; Win 7 Home x86 + Java 1.6.0_26; Sage 7.1.9; Driving Epson 8500 pj @ 1080p | Office: Core2Duo E8500 w/ EFI-X running OS X 10.5 & Win 7 Pro x64 (dual boot) + Java 1.6.0_20; Sage 7.0.23; Driving 30" Samsung 1900x1200 monitor
Reply With Quote
  #58  
Old 01-11-2009, 02:34 PM
megamojo's Avatar
megamojo megamojo is offline
Sage Advanced User
 
Join Date: Apr 2006
Posts: 173
Quote:
Originally Posted by Halstead View Post
Oh, and can this be set up to work with DirMon at this point?
Seconded. I've tried every possible way of calling this with DirMon, using mediascraper.exe directly, using the 2 wonderful batch files, using batch files that call those batch files, and anything else I can think of. I can't get anything to work with dirmon for the life of me. Is there something I'm doing wrong? Is there any way mediascraper.exe or scrape.cmd could be made to accept a couple parameters, such as -p for path or something like that?
Reply With Quote
  #59  
Old 01-11-2009, 07:19 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Dirmon2 may be a problem because when it launches the working directory is C:\system\window32 rather than where the .exe is. I'll check on that.
Reply With Quote
  #60  
Old 01-12-2009, 11:17 PM
evilpenguin's Avatar
evilpenguin evilpenguin is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Seattle, WA
Posts: 3,696
Updated to Beta4: Movies!

I updated to Beta4 and added support for movies from imdb and theMovieDB.

I also added a pretty experimental feature where if it can't find an exact match on the title name from your file it'll just take IMDB's most popular match. I've found that this greatly increases hits on videos that are named more casually, but it will also cause thing named very poorly to be mis-hits. Let me know if anyone has any serious issues with this and I'll look into adjusting it accordingly.
Reply With Quote
Reply

Tags
fanart, imdb, metadata, themoviedb, thetvdb


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Utility: Batch Metadata Importer 3.x stuckless Batch Metadata Tools 2416 09-20-2011 08:03 PM
STV Import: Read & Write Metadata files for Default STV 5/6 (v2.2.9 18/Feb./2010) JREkiwi SageTV Customizations 249 10-26-2010 12:15 AM
manually edit video metadata loomdog32 SageMC Custom Interface 8 01-04-2009 12:34 AM
Runningtime not populated in Metadata mycorona SageMC Custom Interface 2 11-30-2008 01:08 PM
Keeping custom metadata for imports via AddShow() Opus4 SageTV Studio 9 02-20-2008 06:35 PM


All times are GMT -6. The time now is 10:27 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.