SageTV Community  

Go Back   SageTV Community > SageTV Products > SageTV EPG Service

Notices

SageTV EPG Service Discussion related to the SageTV EPG Service used within SageTV. Questions about service area coverage, channel lineups, EPG listings, XMLTV, or anything else related to the service or programming guide data for SageTV should be posted here.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 12-07-2010, 04:44 PM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
WebGrab+Plus , an advanced xmltv epg grabber

I have made an advanced /multisite/incremental EPG grabber and like to share it with SageTV users

Some background:
* I live in a rural area on the Canary Islands where the internet connections are just dialup speed (no ADSL or 3G). On occasion I can use a satellite downlink . Lets say the speed varies between 2-50kB/s.
* Under these conditions I have been using several epg grabbers, but it always took 'hours' to just the daily update.
* The above was the reason I decided to make my own epg grabber which limits downloading to an absolute minimum without loosing richness of data. With it, the daily download for 25 channels in full detail takes about 10 minutes (sat downlink) to 1 hour (in dial-up)

Some features:
* Incremental grabbing. It only grabs what is new or what needs updated because it is changed or missing.
* Can grab from multiple tvguide sites in one run. Each site is programmable through a SiteIni file (I have made a few Dutch, Belgium, Spanish, English Swedish and Brasilian siteini files so far). These siteinifiles can be made by the user with the help of debugging modes and the documentation. Of course I am prepared to help or make one if requested
* The output follows the xmltv standard.
* Most xmltv elements are supported, like start and stop times, title (+ original title), sub-title, category, rating, star-rating, credits (director, actor, presenter), (production)date and episode. All of which are just a choice of the user and programmable per channel.
* All kinds of other options, like, days to grab, one-show-update-only and site-index-page-only (less detail but super fast) and xmltv-element-value filtering and modification.
* A special option is the ability to compose the epg for combi-channels (the epg listings of different channels for different day periods or different title filters in one listing)
* Written in C# , executable less that 1MB. Tested for XP, Vista and W7. It runs in command mode, no GUI , no drivers to install. FreeWare, Absolutely non commercial, no malware or virus (I wouldn't even know how to get them in it !?)

In function it comes close to tvxb but tests show it is about 2 to 3 times faster while providing a more detailed epg.

The program is available for download on
http://www.servercare.nl/Lists/Posts/Post.aspx?ID=98
Further there is an active user group in Spain on
http://www.todohtpc.com/foro/softwar...-guia-epg.html

WG++Maker -- Jan
Reply With Quote
  #2  
Old 12-11-2010, 11:22 AM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
Latest version 1.0.2 went public today @
http://www.servercare.nl/Lists/Posts/Post.aspx?ID=98

Supported tvguide sites:

Belgium: tv.be, mijn-tv-gids.be
Brazil: hagah.com.br
Netherlands: tvgids.nl, gids.publiekeomroep.nl, tvgids.upc.nl tvtv.nl
Spain: plus.es, laguiatv.com, teleprogramma.tv
Sweden: dagenstv.com, svt.se
UK: tvguide.co.uk, radiotimes.com, tvtv.co.uk

Last edited by WG++Maker; 12-11-2010 at 11:46 AM.
Reply With Quote
  #3  
Old 12-14-2010, 03:00 AM
jamesdisco's Avatar
jamesdisco jamesdisco is offline
Sage Advanced User
 
Join Date: Jul 2008
Location: N Yorks, UK
Posts: 234
This looks interesting, I got an email from digiguide.com saying they have just been bought so who knows what might happen, they might stop us from taking xmltv from them.
__________________
Server: Win 10 64bit Core i3 8GB 20TB, TBS6985, TBS6984, Sage 9.
Clients: 5 x HD300, 2 x HD200, Placeshifter.
Reply With Quote
  #4  
Old 12-14-2010, 07:10 AM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
Hi JamesDisco,

I'm innocent (I didn't buy them to get you to use my grabber)!
But .. it's an idea, buy all the commercial guide sellers ... and ..

Have you tried WG++ ? Which tvguide site lists your channels? Any site you like to be added?
Reply With Quote
  #5  
Old 12-15-2010, 02:26 AM
jamesdisco's Avatar
jamesdisco jamesdisco is offline
Sage Advanced User
 
Join Date: Jul 2008
Location: N Yorks, UK
Posts: 234
At the moment i use Birtles grabber to get xmltv from www.digiguide.com . Its the only site that lists all channels and radio stations in the UK, it does require a yearly subscription though. I would definately like to try a free provider but I cant see radio on the 3 sites WG++ supports. Post down in the UK section i'm sure there will be people there would like to try this out.
__________________
Server: Win 10 64bit Core i3 8GB 20TB, TBS6985, TBS6984, Sage 9.
Clients: 5 x HD300, 2 x HD200, Placeshifter.
Reply With Quote
  #6  
Old 12-16-2010, 03:22 AM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
Jamesdisco

do I understand you correct that you want 'radio guide listings' (next to tv guide listings)?
If so, radiotimes.com list radiostations . To support them with WG++ a separate siteini file is needed which I can make if you want. Please tell me if that is what you want.
Reply With Quote
  #7  
Old 12-16-2010, 02:23 PM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
Quote:
Originally Posted by jamesdisco View Post
Post down in the UK section i'm sure there will be people there would like to try this out.
Can you give me the link to that uk section? Thanks

Last edited by WG++Maker; 12-16-2010 at 02:59 PM.
Reply With Quote
  #8  
Old 12-16-2010, 03:36 PM
brewston brewston is offline
Sage Expert
 
Join Date: Apr 2006
Location: Surrey
Posts: 719
Quote:
Originally Posted by WG++Maker View Post
Can you give me the link to that uk section? Thanks
Its here :

http://forums.sagetv.com/forums/forumdisplay.php?f=25
__________________
Tecra M5, 2 x HD200, 2 x HD300
2 x PCTV 290e
Win 7, Sage 7.1.9, Phoenix 2 STV
Stephane's XMLTV Importer, Digiguide,
Reply With Quote
  #9  
Old 01-03-2011, 08:59 AM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
New Version

There is a new version of the program 1.0.4 that went public a couple of days ago.
http://www.servercare.nl/Lists/Posts/Post.aspx?ID=98

Supported tvguide sites:

Belgium: tv.be, mijn-tv-gids.be, yelo.be (new)
Brazil: hagah.com.br
Netherlands: tvgids.nl, gids.publiekeomroep.nl, tvtv.nl
Spain: plus.es, laguiatv.com, teleprogramma.tv, sincroguia.tv (soon)
Sweden: dagenstv.com, svt.se
UK: tvguide.co.uk, radiotimes.com, tvtv.co.uk
Denmark: tvguide.dk (new)
USA and N/S America's : directv.com (new)
Reply With Quote
  #10  
Old 01-15-2011, 08:18 AM
ttriplett ttriplett is offline
Sage User
 
Join Date: Jun 2003
Location: MS
Posts: 21
EPG for middle east?

I am a subscriber to OSNetwork (orbit showtime network, formerly showtime arabia) in Jordan. Any way to get epg data for their HD channels (only 12 channels). I am unaware of a XMLTV feed from osnetwork but most if not all are US channels that might be available from another XMLTV listing... of course the times might be screwed up. Maybe a screen scrape from their online guide would be possible. Any thoughts?
Reply With Quote
  #11  
Old 01-15-2011, 03:11 PM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
Currently it seems that the online guide of OSNetwork is having some problems on the server side. So I cannot check its 'grabbability' . I try again tomorrow. Can you in the mean time check if any of the sites listed above carries your channels with the same program schedules (possibly timeshifted)? Directv has a lot of US channels.

WG++Maker
Reply With Quote
  #12  
Old 01-22-2011, 06:55 AM
ttriplett ttriplett is offline
Sage User
 
Join Date: Jun 2003
Location: MS
Posts: 21
On closer inspection, it looks like only a few of the channels might be available from a US XMLTV listing. However, their server seems to be working again. Maybe you could check the "grabability"? If you go to osnetwork.com and hover over "TV Guide", then click schedules, then click HD you will get a program listing.
Reply With Quote
  #13  
Old 01-24-2011, 03:26 PM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
ttriplett,

sorry to have kept you waiting, I needed some time to think about whether osnetwork is grabbable or not. The epg data is in html format, that is good. But it has a very different structure than 'normal' . It cannot be grabbed by WG++ as it is now. The main reason is that the site doesn't provide 'direct' start and stop times but uses grid coordinates instead. These have to be converted into time values which is not supported right now.
As soon as I have some time I will do some experiments and let you know .
Do you think there will be more users of osnetwork that need an xmltv grabber for their epg?
WG++Maker
Reply With Quote
  #14  
Old 01-29-2011, 01:52 PM
ttriplett ttriplett is offline
Sage User
 
Join Date: Jun 2003
Location: MS
Posts: 21
Quote:
Originally Posted by WG++Maker View Post
ttriplett,

-
Do you think there will be more users of osnetwork that need an xmltv grabber for their epg?
WG++Maker
I would think so since osnetwork covers north africa, mideast, south asia, etc. But who knows.
Reply With Quote
  #15  
Old 02-10-2011, 02:13 PM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
An update: I am still working on it, but it's a rather complicated site to connect! It needs a few new features of the program to be implemented of which I have a working test version. The first results indicate that epg grabbing can be done, but may be only on sundays!! (This is due to the fact that the site only has weekly schedules which only have a grabable structure on sunday!)
Reply With Quote
  #16  
Old 02-13-2011, 03:52 PM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
I have completed a first version of a SiteIni file (= site connection file for WebGrab+Plus) for Osnetwork. It grabs 7 days of epg data. There is a limitation though that it will only work on sunday! (See previous post)
It is not yet public, so for anyone interested , send me a private message and I will send you a test version of the setup.
Reply With Quote
  #17  
Old 02-23-2011, 03:06 PM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
Another update: The setup for OSNETWORK with WebGrab+Plus now works every day (the 'Sunday Only' limitation is gone). It grab 7 days of EPG with title, description, category, producer, actor, (production)date and rating as details. It will be public in a week or so, together with the next program update http://www.servercare.nl/Lists/Posts/Post.aspx?ID=98
People interested before that moment please let me know, as private message or here or at the link above.
Reply With Quote
  #18  
Old 03-04-2011, 01:32 AM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
There is a new version of the program 1.0.6 http://www.servercare.nl/Lists/Posts/Post.aspx?ID=98

Supported tvguide sites:

Belgium: tv.be, mijn-tv-gids.be, yelo.be
Brazil: hagah.com.br
Netherlands: tvgids.nl, gids.publiekeomroep.nl, tvtv.nl
Spain: plus.es, laguiatv.com, teleprogramma.tv, sincroguia.tv (improved)
Sweden: dagenstv.com, svt.se
Norway : sol.no (new)
UK: tvguide.co.uk, radiotimes.com, tvtv.co.uk, bfbs.com (new British Forces Broadcasting Services)
Denmark: tvguide.dk
USA and N/S America's : directv.com
Far East : osnetwork.com (new)

Just drop a message if you want support for a site that is not in the list ...

Last edited by WG++Maker; 03-04-2011 at 01:37 AM.
Reply With Quote
  #19  
Old 04-10-2011, 04:50 AM
perf perf is offline
Sage Advanced User
 
Join Date: Oct 2004
Location: Stockholm, Sweden
Posts: 120
I will start using WG++ for Sweden, with data from Dagenstv.com. However, I need the sub-title element to be equal to the episode element. I have not been able to modify the .ini file to achieve that. How should I do?

(Part of the problem is that all information is on the index page - there are no show details pages - and sub-title can not be scrubbed from the index page.)

/Per
Reply With Quote
  #20  
Old 04-10-2011, 09:26 AM
WG++Maker WG++Maker is offline
Sage User
 
Join Date: Dec 2010
Location: Canary Islands
Posts: 49
Quote:
Originally Posted by perf View Post
I will start using WG++ for Sweden, with data from Dagenstv.com. However, I need the sub-title element to be equal to the episode element. I have not been able to modify the .ini file to achieve that. How should I do?

(Part of the problem is that all information is on the index page - there are no show details pages - and sub-title can not be scrubbed from the index page.)

/Per
Hopefully I understand you correct that you want what is now in the episode-num element in the sub-title element. You can do that by adding at the end of your dagenstv.com.ini the following :
index_subtitle.modify {addstart(null)|'index_episode'}
This will leave you with the same data in both sub-title and episode-num.

If however, in fact, you don't use the episode-num element at all, you can also just change the line:
index_episode.scrub {single(separator="." include="Säsong ""Del ")|<div class="tbl_lrg_event_description|>|<|</div>}
into:
index_subtitle.scrub {single(separator="." include="Säsong ""Del ")|<div class="tbl_lrg_event_description|>|<|</div>}

After you are happy with the changes it is best to change the name of the dagenstv.com.ini into dagenstv.com.perf.ini and also change the site attribute in the config file from dagenstv.com into dagenstv.com.perf .This to avoid problems with possible future updates of the program or the siteinipack .

hopes this helps WG++Maker ... Jan
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
DVB-S setup with Digiguide EPG Grabber & XMLTV importer MCE-Refugee SageTV United Kingdom 27 09-08-2008 09:10 AM
JXMLTV: a Java XMLTV grabber for Belgium Olive SageTV Europe (Miscellaneous) 10 09-25-2006 01:58 PM
XMLTV and other EPG resources Opus4 SageTV EPG Service 0 04-24-2006 09:32 PM
XMLTV EPG Problems ukmgranger SageTV EPG Service 7 05-04-2004 04:50 PM
XMLTV grabber for Australia exists dir SageTV Australia/New Zealand 1 12-12-2003 01:10 AM


All times are GMT -6. The time now is 03:59 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.