SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV Customizations

Notices

SageTV Customizations This forums is for discussing and sharing user-created modifications for the SageTV application created by using the SageTV Studio or through the use of external plugins. Use this forum to discuss customizations for SageTV version 6 and earlier, or for the SageTV3 UI.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 03-13-2007, 10:30 PM
pdavis pdavis is offline
Sage User
 
Join Date: Dec 2004
Posts: 18
TV.Com to .My File Generator (MyScraper)

I've written a small C# application for Windows that will parse the source file from tv.com and generate a .my file that you can then save. Feedback is welcome.

Simply download the My Scraper Executable, unzip it, and run the MyScraper.exe file. For instructions go to the Help->Instructions menu option.

* My Scraper Executable
* My Scraper C# .Net Source Code

Just for a little practice I converted the application to a web app...

http://pdavis.somee.com/

It does mostly the same thing as the Windows application but may be a version behind and is not nearly as easy to use... plus no instructions... but instructions can also be found in the executable folder in the Instructions.htm file.

For the latest release I have added illegal character checking and added a missing file to the Executable pack.
Reply With Quote
  #2  
Old 03-14-2007, 02:04 PM
millers_35 millers_35 is offline
Sage Advanced User
 
Join Date: Apr 2006
Location: Vonore, TN
Posts: 228
Send a message via AIM to millers_35
Very nice pdavis, I will be using this extensively. Greatly appreciated.

Miller
__________________
Server: AMD Athlon 64 X2 3600+, 2GB Corsair XMS2 PC2 6400 Ram, 1 X Hauppage PVR-150MCE, HDHomerun, 500GB HD + 1TB HD.
STX-HD100 connected to a Sammy HLT-6187S and Placeshifter.
Software: Vista Business(Server), SageTV 7.0.9, Java 1.6_20.
Reply With Quote
  #3  
Old 03-14-2007, 02:19 PM
dvd_maniac's Avatar
dvd_maniac dvd_maniac is offline
Sage Icon
 
Join Date: Mar 2004
Location: New England
Posts: 1,899
I've done about a dozen complete series in almost no time and effort at all.

This program rocks. I hope it will grow now that metedata properties files are here. Unlike .my files, when you create a properties file for an import in actually adds that info into the wiz.bin and becomes searchable.


Thanx pdavis
__________________
If this doesn't work right, Then:
"I'm going to blow up the Earth!"
Reply With Quote
  #4  
Old 03-14-2007, 02:34 PM
millers_35 millers_35 is offline
Sage Advanced User
 
Join Date: Apr 2006
Location: Vonore, TN
Posts: 228
Send a message via AIM to millers_35
Yeah its so quick and easy!!! I had a couple series I was dreading doing but now I got already got em down.

Miller
__________________
Server: AMD Athlon 64 X2 3600+, 2GB Corsair XMS2 PC2 6400 Ram, 1 X Hauppage PVR-150MCE, HDHomerun, 500GB HD + 1TB HD.
STX-HD100 connected to a Sammy HLT-6187S and Placeshifter.
Software: Vista Business(Server), SageTV 7.0.9, Java 1.6_20.
Reply With Quote
  #5  
Old 03-16-2007, 01:30 PM
roxy99 roxy99 is offline
Sage Expert
 
Join Date: Mar 2006
Posts: 516
Looks great and I'm looking forward to trying it, thank you. Does the application fetch the data and parse from TV.com automatically or do you first have to manually search and download from TV.com for tv series?
Reply With Quote
  #6  
Old 03-19-2007, 08:12 AM
roxy99 roxy99 is offline
Sage Expert
 
Join Date: Mar 2006
Posts: 516
Works like a charm. Thanks for that. All that's remaining is to incorporate this as a Sage plugin and get the plugin to find the exact URL of the episode within TV.com.
Reply With Quote
  #7  
Old 03-19-2007, 02:01 PM
dvd_maniac's Avatar
dvd_maniac dvd_maniac is offline
Sage Icon
 
Join Date: Mar 2004
Location: New England
Posts: 1,899
Quote:
Originally Posted by roxy99 View Post
Works like a charm. Thanks for that. All that's remaining is to incorporate this as a Sage plugin and get the plugin to find the exact URL of the episode within TV.com.
I disagree that all that is remaining is Sage plugin support for tv.com.
I really hope that someone can incorporate metadata .properites file support as well. To have .my files for a show with say 200+ episodes makes it easy to find out what a particular episode is about, but if you are trying to find a particular episode of a series and you know what the episode is about but do not know the episode name than you have to go through them one at a time and display "Video Details" of each until you find it. With .Properties files they merge with the wiz.bin and become searchable from Import Videos Search Screen (With Custom Search STVi). So if I wanted to search for all Star Trek episodes that deal with Klingons I could search the keyword "Klingon" and find them. With .my files we can't do that.
__________________
If this doesn't work right, Then:
"I'm going to blow up the Earth!"
Reply With Quote
  #8  
Old 03-31-2007, 11:32 AM
Belgrath Belgrath is offline
Sage Advanced User
 
Join Date: Sep 2006
Posts: 82
Quote:
Originally Posted by dvd_maniac View Post
I disagree that all that is remaining is Sage plugin support for tv.com.
I really hope that someone can incorporate metadata .properites file support as well. To have .my files for a show with say 200+ episodes makes it easy to find out what a particular episode is about, but if you are trying to find a particular episode of a series and you know what the episode is about but do not know the episode name than you have to go through them one at a time and display "Video Details" of each until you find it. With .Properties files they merge with the wiz.bin and become searchable from Import Videos Search Screen (With Custom Search STVi). So if I wanted to search for all Star Trek episodes that deal with Klingons I could search the keyword "Klingon" and find them. With .my files we can't do that.
I'd like to see the follow options if possible.
- Having the option to select which fields from the Tv.com website I'd like to place in my MY file.
- File syntax preference, so that I can name the files in the proper format. <Series> [<season>x<episode>] <Title>.my
- I would like to be able to get the data from Episode Recap which tends to be a bit more detailed, and place that into the <TV Overview> field.

Those options would really be a great added benifit. Thanks much ahead of time!!

Belgrath
Reply With Quote
  #9  
Old 03-31-2007, 12:39 PM
alon24 alon24 is offline
Sage Aficionado
 
Join Date: Jun 2004
Posts: 351
Quote:
Originally Posted by dvd_maniac View Post
I've done about a dozen complete series in almost no time and effort at all.

This program rocks. I hope it will grow now that metedata properties files are here. Unlike .my files, when you create a properties file for an import in actually adds that info into the wiz.bin and becomes searchable.


Thanx pdavis
Can you put a link to what are the metadata .properties?
__________________
Server
SageTv 6.3.5, Core2Duo 6300 ,2Gigs ,Saphire x1650, PVR250, 2*320GB + 160GB, java 1.6.1
Client
SageTV Client 6.3.5 , AMD 3000, 1024Mb, Saphire x1600Pro256HDMI, java 1.6.1

Using Nielm's Web server 2.22
Reply With Quote
  #10  
Old 03-31-2007, 01:01 PM
dvd_maniac's Avatar
dvd_maniac dvd_maniac is offline
Sage Icon
 
Join Date: Mar 2004
Location: New England
Posts: 1,899
The syntax/Format is a little different from the .my file format.
I will attach a metedata file here:
Attached Files
File Type: zip ClosetoHome-MakingAmends-6440972-0.mpg.zip (380 Bytes, 355 views)
__________________
If this doesn't work right, Then:
"I'm going to blow up the Earth!"
Reply With Quote
  #11  
Old 03-31-2007, 02:28 PM
Opus4's Avatar
Opus4 Opus4 is offline
Administrator
 
Join Date: Sep 2003
Location: NJ
Posts: 19,595
Quote:
Originally Posted by alon24 View Post
Can you put a link to what are the metadata .properties?
Recording metadata properties files were added in the v6.1.4 beta, but that was just a start. It's property support will probably be filled out more in future versions (not 6.1)

- Andy
__________________
SageTV Open Source v9 is available.
- Read the SageTV FAQ. Older PDF User's Guides mostly still apply: SageTV V7.0 & SageTV Studio v7.1.
- Hauppauge remote help: 1) Basics/Extending it 2) Replace it 3) Use it w/o needing focus
- HD Extenders: A) FAQs B) URC MX-700 remote setup
Note: This is a users' forum; see the Rules. For official tech support fill out a Support Request.
Reply With Quote
  #12  
Old 04-05-2007, 04:05 AM
justme justme is offline
Moderator
 
Join Date: Mar 2003
Location: Greater Baltimore/Washington Metropolitan area
Posts: 1,481
Much Thanx

Sorry for the late response...

But I want to thank you for this app and especially the new version, with handling of illegal characters. It's easy, pretty fast and highly usefull.

Wishlist:
1) Another vote for allowing output as a .properties file. I'd like it to be user selectable. ie create .my or .properties OR both

2) Another vote for adding optional parsing of episode recap descriptions. I personally wouldn't want it to replace the current/normal episode summaries when browsing thru my media library. Still it would be nice to add the data under another metatag. Maybe one that could be accessed via a STV mod(of course the data would have to be in the .my file first), if the user wanted the more detailed recap.

PS:I don't know if I'm providing info people already know or if it can even be put to any use but... you can simplify the URLs once you know the show number tv.com uses for the series you're interested in. Also you can get a episode list of a season by using season=X (where X=the season number) or 0 for all seasons, instead of all that dropdown junk that normally ends up at the end of the URL if you browse there by clicking links. ie

the normal link for a ep listing of StarTrek:Voyager season 3 looks like this:
http://www.tv.com/star-trek-voyager/show/274/episode_listings.html?season=3&tag=nav_bar;3
Once you know that TV.com's show number for Voyager is 274 and knowing that we don't need all the drop down junk in the URL means that you can shorten it to this:
http://www.tv.com/show/274/episode_listings.html?season=3
You could just change that 3 to a 0 to get a ep list for all seasons.

The same logic also seems to apply to an episode of a show, once you get that episode's tv.com number. For example episode 29 of ST:Voyager uses the number 10668, so the episode 29 summary link of:
http://www.tv.com/star-trek-voyager/prototype/episode/10668/summary.html?tag=ep_list;title;28
can be reduced to:
http://www.tv.com/episode/10668/summary.html
You can also access the episode 29 recap with this simple link:
http://www.tv.com/episode/10668/recap.html
It would be nice to be able to format a link to get an episode summary for a certian episode without knowing the TV.com ep number. For instance get a summary for episode 29 without knowing it's ep number is 10668. But I haven't found a formatting that does that. Maybe smarter people than me can work out if that's possible.
Reply With Quote
  #13  
Old 04-06-2007, 10:19 AM
roxy99 roxy99 is offline
Sage Expert
 
Join Date: Mar 2006
Posts: 516
The avi container can support text information so why can't a tag be inserted into the avi that gives the program info? Then a sage plugin can read the tag information directly from the avi instead of having to clutter the hard drive with all these .my files.

Dvdmaniac,

How is replacing one text file format for another text file format any different? I mean syntax of .my text files vs your metadata text file is different but so what? What is the advantage?
__________________
http://www.hydranterouge.com
Reply With Quote
  #14  
Old 04-06-2007, 12:18 PM
dvd_maniac's Avatar
dvd_maniac dvd_maniac is offline
Sage Icon
 
Join Date: Mar 2004
Location: New England
Posts: 1,899
The advantage is the Syntax and the way that Sage currently handles the two files.

The .my files are currently not searchable inside of Sage.
I have every episode of Star Trek and it is great to have the .my files for them but if I wanted to watch a particular episode with Klingons in it then how would I find it?
I would have to go into each Imported Video and bring up the Video Details and read it to find out if it was the episode I wanted.

Now the metadata .properties files import the information directly into the Wiz.bin and with the custom search stvi the info becomes searchable.
So I would just search for klingon and every video that has Klingon in the Title or Summary field will show up.
Also great for searching if you have a large Imported video folders. I have over 1500 movies and it is great to search for a particular actor and see all the movies I have with that actor in it.

The other reason is standardization in syntax. .my files can be formatted in various ways and so display the information differently. metadata .properties files(I think) are imported and syntaxed in Sage's native format and as such can only be displayed one way.
__________________
If this doesn't work right, Then:
"I'm going to blow up the Earth!"

Last edited by dvd_maniac; 04-06-2007 at 12:21 PM.
Reply With Quote
  #15  
Old 04-06-2007, 04:09 PM
roxy99 roxy99 is offline
Sage Expert
 
Join Date: Mar 2006
Posts: 516
Quote:
Originally Posted by dvd_maniac View Post
The advantage is the Syntax and the way that Sage currently handles the two files.

The .my files are currently not searchable inside of Sage.
I have every episode of Star Trek and it is great to have the .my files for them but if I wanted to watch a particular episode with Klingons in it then how would I find it?
I would have to go into each Imported Video and bring up the Video Details and read it to find out if it was the episode I wanted.

Now the metadata .properties files import the information directly into the Wiz.bin and with the custom search stvi the info becomes searchable.
So I would just search for klingon and every video that has Klingon in the Title or Summary field will show up.
Also great for searching if you have a large Imported video folders. I have over 1500 movies and it is great to search for a particular actor and see all the movies I have with that actor in it.

The other reason is standardization in syntax. .my files can be formatted in various ways and so display the information differently. metadata .properties files(I think) are imported and syntaxed in Sage's native format and as such can only be displayed one way.


That makes sense to me now. I did'nt realize that Wiz.bin incorporates the metadata .properties file. Pdavis should'nt have much problem implementing that as an additional format.
__________________
http://www.hydranterouge.com
Reply With Quote
  #16  
Old 04-22-2007, 08:59 AM
roxy99 roxy99 is offline
Sage Expert
 
Join Date: Mar 2006
Posts: 516
I have to say you're not getting the props you deserve. I'm learning Python programming and as an exercise I am trying to reverse engineer the work you did. Needless to say, I am having a bitch of a time parsing TV.com's html.

I have to give you kudos for a job well done. I'm sure you faced many challenges in parsing the data. If you can share any tips with me, feel free to pm me.

Thanks again for this great tool.
__________________
http://www.hydranterouge.com
Reply With Quote
  #17  
Old 05-01-2007, 08:25 AM
Armondole's Avatar
Armondole Armondole is offline
Sage User
 
Join Date: Mar 2006
Location: Ottawa, Canada
Posts: 67
Quote:
Originally Posted by Belgrath View Post
I'd like to see the follow options if possible.

- File syntax preference, so that I can name the files in the proper format. <Series> [<season>x<episode>] <Title>.my

Belgrath
I agree with you on this Belgrath. This would be a very handy feature if it was possible to implement.

Thanks for the great tool!
Reply With Quote
  #18  
Old 05-10-2007, 12:42 PM
Armondole's Avatar
Armondole Armondole is offline
Sage User
 
Join Date: Mar 2006
Location: Ottawa, Canada
Posts: 67
Has anyone ever used the XboxMediaCenter TV.com scraper for TV/Movies? I find that it is extremely accurate and easy to use... I really know nothing about how this stuff works, but I thought this might be helpful info for anyone trying to improve this script.

http://www.xboxmediacenter.com/wiki/...le=Scraper.xml
Reply With Quote
  #19  
Old 02-07-2008, 10:50 PM
pdavis pdavis is offline
Sage User
 
Join Date: Dec 2004
Posts: 18
Thanks

Thank you so much for all the great feedback. I will take your suggestions and incorporate those I can. I'm also planning on making an export to XML option now that we can import XML information back into Sage through the web interface. I also noticed that the episode guide page has changed on TV.com so that functionality no longer works, I will update that as well.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
MediaPlayer Plugin/STV Import: Winamp Media Player Plugin deria SageTV Customizations 447 12-11-2010 07:38 PM
STV Import: A Collection of Minor Tweaks for the Default STV for SageTV 5.0/6.0 JREkiwi SageTV Customizations 126 01-10-2009 08:45 PM
Which properties file does the client software use? reboot_this SageTV Software 4 03-02-2007 01:01 PM
Can Import my old wiz.bin file? jptaz SageTV Software 6 04-27-2006 05:01 AM
HDTV Network Encoder v0.9.0 07-08-05 - HD Support for Sage! Stealth1971 Hardware Support 292 02-23-2006 11:40 AM


All times are GMT -6. The time now is 11:38 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.