Updated XMLTV import plugin (mostly for dutch users)

[koelie]

Well this is my first ever post on this forum .
I just upgraded to a new xmltv importplugin which completely ****ed up my epg, so I decided to just update an old one (the one written by demideus) myself. It is customized mostly towards users in the netherlands using tv_grab_nl_wolf, but it will mostly work for anyone...

features:
- category translation
(this is Demideus' work)
- support for actors, hosts etc
(also Demideus' work)
- updated people parsing
(so you don't get one actor named "Robert de niro, Steve Buscemi e.a.")
- updated descriptions parsing
(in the dutch listings there are often multiple descriptions per show, here I concatenate them instead of just picking the last one)
- ability to dismiss channels
(tv_grab_nl_wolf always grabs ALL channels, if you dismiss some you don't use you will keep your wiz.bin smaller)
- extra channels
(for which there are no listings but you still want them in SageTV)
- regional channel support
(for users in the netherlands, this also parses stations like TV West or TV Flevoland)
-updated: also supports firstrun / rerun
Installation:
-download xerces.jar and put it in your C:\program Files\Java\jre1.5.0\lib\ext\ folder (or wherever you have java installed)

-unzip the plugin into your sagetv dir.

-edit sage.properties and look for the following line:
epg/epg_import_plugin=
and change it to:
epg/epg_import_plugin=xmltv.XMLTVImportPlugin

- edit the xmltv.properties file to your liking

- hope y'all like it ....

koelie

[Methius]

Thanks, this works perfectly!!

Try using this grabber with it: http://www.cask-of-amontillado.com/f...ids_to_xml.zip (works perfect also)

(Dutch, and grabs a lot more than the Wolf XML)

[bertsmelik]

Originally Posted by Methius

Thanks, this works perfectly!!

Try using this grabber with it: http://www.cask-of-amontillado.com/f...ids_to_xml.zip (works perfect also)

(Dutch, and grabs a lot more than the Wolf XML)

Hi Methius,

Looks like a nice solution.
One question though: where do I put the files from this process sothat Sage picks up the epg information?

Bert

[koelie]

you have to put the xml file in the sagetv directory and it has to be called "epgdata.xml"...
i use a batch file for all this, which is scheduled to run daily...
it looks like this:

c:\xmltv\xmltv tv_grab_nl_wolf --output c:\xmltv\epgdata.xml
copy c:\xmltv\epgdata.xml c:\progra~1\freyte~1\sagetv\epgdata.xml

if you're using the tvgids_to_xml program, this would become:

tvgids_to_xml.exe
copy tvgids.xml c:\progra~1\freyte~1\sagetv\epgdata.xml

put this in a .bat file in your tvgids_to_xml dir and schedule it to run daily (using windows' scheduled tasks) and you'll be fine...

BTW methius, the reasons I prefer wolfs grabber over this one:
- it doesn't grab anything itself but merely downloads xml-files from wolfs website, making it lots faster
- i compared two resulting xml-files from both grabbers and tvgids2xml seems to indeed grab more & longer descriptions, but wolf is much better at grabbing actors, presenters etc...
- it doesn't grab local channel info (TV Rijnmond, TV West etc.)

i wish there was some source code available for this tvgids_to_xml program cause i do like its setup and wish i could expand it to grab the things i'm missing.

anyways for those people who do use tvgids_to_xml, i updated the plugin in the first post to also support firstrun/rerun info, which is grabbed by this program and not by wolf

[jan smit]

i wish there was some source code available for this tvgids_to_xml program cause i do like its setup and wish i could expand it to grab the things i'm missing.

Why don't you ask Amontillado?
He's a very helpful person.

[ekiwi]

great work thanks, love the facility to add channels with no schedule like the weather channel etc

question:- I have lots more fields in my xmltv data like
<star-rating>
<value>3/3</value>
</star-rating>

can all the valid fields in xmltv that have somewhere to be stored in sage be brought in

i know that sage will display star ratings i have seen it on some screen shots

[koelie]

the function that sage calls to import a show's details has lots of arguments.
the ones that are implemented in this plugin:

* title - the title of the show (use for reruns)
* primeTitle - the title of the show (use for first runs)
* episodeName - the name of the episode
* desc - a description of this show
* duration - not used, set to 0
* category - name of a category for this show
* subCategory - name of a subCategory for this show
* people - names of people/actors in this show and their roles
* year - the year it was produced, for movies

the arguments i have not implemented are:

* rated - rating of a show, i.e. PG, G, R, etc.
* expandedRatings - additional rating information, i.e. Violence, Nudity, Adult Content
* bonus - additional information about the show
* language - the language the show is in
* originalAirDate - the original airing date of this show, it's a long value from java.util.Date

as there is no equivalent of these in the xmltv data i get here...

so I don't see any special place to put the star ratings you mentioned.
Ofcourse it is quite possible to put this star rating in for instance the bonus or rated argument, Although I don't know whether and if so, where it will show up in sagetv

Another possibility is to append this rating to the description of the show...
the same goes for any other extra info you might have in your xml-data.

if you let me know which info (i.e. names of the xml-tags + short descriptions) you would like to see in which argument, I'll see what i can do for ya

[ekiwi]

Hmmm interesting,

Im not sure if there is a place in the sage database for them or if they are just tagged on the end of some other data but if if you look at these two screen shots

http://www.xmediazone.com/devtmp01/images/RecDet01.jpg
http://homepages.isunet.net/smichael...Recordings.gif

the second shot would suggest that there is a place for it to be stored as who would put it on the end of the title, that would make it real hard to have favorites.

From the official XMLTV.DTD

<!-- 'Star rating' - many listings guides award a programme a score as
a quick guide to how good it is. The value of this element should be
'N / M', for example one star out of a possible five stars would be
'1 / 5'. Zero stars is also a possible score (and not the same as
'unrated'). You should try to map whatever wacky system your listings
source uses to a number of stars: so for example if they have thumbs
up, thumbs sideways and thumbs down, you could map that to two, one or
zero stars out of two. Whitespace between the numbers and slash is
ignored.
-->


<!ELEMENT star-rating (value, icon*)>

EG

<star-rating>
<value>3/3</value>
</star-rating>


I understand the interface you have dosent have a way of sending it through. Maybe a request should be made to sage to allow the other data to be imported. now I am happy to place the request but this is all the info I have as i dont understand the import interface. I personally think that if there is a place in the sage database for some xmltv data then a plugin should be able to import it

xmltv currently supports i think

title+, sub-title*, desc*, credits?, date?, category*,
language?, orig-language?, length?, icon*, url*,
country*, episode-num*, video?, audio?, previously-shown?,
premiere?, last-chance?, new?, subtitles*, rating*,
star-rating?

it would appear that star rating is the only one missing that internally sage appears to use.

If it is not to much trouble It would be good if you could import all that you can. In NZ we are supplied with censorship ratings I also run the xmltv file through the xmltv.exe imdb thing with addes heaps of information including the star-ratings data

[marneb17]

Originally Posted by koelie

- updated people parsing
(so you don't get one actor named "Robert de niro, Steve Buscemi e.a.")

Great work, works really well. I have one problem, though.
It can't seem to seperate the actors. I do get something like this:

John Rhys-Davies (Sallah);Julian Glover (Walter Donovan);Michael Byrne (Vogel);Kevork Malikyan (Kazim);Harrison Ford (Indiana Jones);Sean Connery (Prof. Henry Jones);Denholm Elliott (Marcus Brody);Alison Doody (Elsa Schneider);River Phoenix (Young Indy);Jeffrey Boam (Drehbuch);George Lucas (Drehbuch);Menno Meyjes (Drehbuch);Douglas Slocombe (Kamera);Robert Ii Stevens (Kamera);Paul Beeson (Kamera);John Williams (Musik)

All as a single actor. So the seperator ";" is not detected. What is used in the nl grabber?
I use a German grabber by the way.

Would be great if you would help.

Marneb

[Amontillado]

Maybe a bit late reaction, but i never really noticed this thread.. and I appologize to Koelie if it seems like i'm hijacking his thread (not that i consider the different grabbers to be competitors, just alternatives). Koelie, if you prefer that i don't reply on remarks/questions about my grabber, let me know.
If someone has questions or suggestions he/she can pm me.. or start a thread (and pm me that there is one).

One question though: where do I put the files from this process sothat Sage picks up the epg information?

You can change this in the ini file (TotalEpgFileName=c:\progra~1\freyte~1\sagetv\epgdata.xml)

There doesn't seem to be much difference in the options of koelie's and my grabber, only thing i didn't find in koeli's list is the ability to reuse previous data and set the specific days you want to download. (reusing previously downloaded data can help speed things up, since you only need to download the last day. set specific days is a addition to this, so instead of just reading the last day you can also redownload this day (since data might have changed)

[koelie]

hey there marneb17,
i updated the zip, it also searches for ;'s as separators now
lemme know if it doesn't work...

and Amontillado, this plugin isn't actually a grabber at all, just an importplugin.
i used tv_grab_nl_wolf when i made this.

by now i have switched to downloading gzipped xml files from http://xmltv.assies.info

btw I'm currently working on a more elaborated importplugin, which will do lots more. It even has a gui for configuring it

[marneb17]

Thanks for the update, but the actors still appear the same way. Maybe my epg wasn't updaten or something. I will try some more.
Thank you for the quick reply.

-edit:
Sorry, yes it does work. Great thanks. It was still searching the peolpe from shows that already aired. But those shows were of course not imported again and therefor not corrected.
Great!

Marne

[1972]

Does it still work? I tried it last week, but no EPG data was seen in SageTV. A part of the xmltv.console.log shows:

rr: java.text.ParseException: Unparseable date: "20050429065900"
java.text.ParseException: Unparseable date: "20050429065900"
at java.text.DateFormat.parse(Unknown Source)
at xmltv.Show.setEnd(Show.java:225)
at xmltv.XMLTVImportPlugin.startElement(XMLTVImportPlugin.java:429)
at org.apache.xerces.parsers.SAXParser.startElement(SAXParser.java:1376)
at org.apache.xerces.validators.common.XMLValidator.callStartElement(XMLValidator.java:1284)
at org.apache.xerces.framework.XMLDocumentScanner.scanElement(XMLDocumentScanner.java:1806)
at org.apache.xerces.framework.XMLDocumentScanner$ContentDispatcher.dispatch(XMLDocumentScanner.java:1182)
at org.apache.xerces.framework.XMLDocumentScanner.parseSome(XMLDocumentScanner.java:381)
at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1098)
at xmltv.XMLTVImportPlugin.updateGuide(XMLTVImportPlugin.java:265)
at sage.af.q(Unknown Source)
at sage.bh.H(Unknown Source)
at sage.bc.iS(Unknown Source)
at sage.af.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

[Demideus]

Originally Posted by 1972

A part of the xmltv.console.log shows:

rr: java.text.ParseException: Unparseable date: "20050429065900"

Seems like you've got yourself an xmltv feed that does not provide timezones (e.g.: "20050429065900 +0200"). The xmltv format specifies that if the timezone is omitted the date has to be interpreted as UTC.

I've attached a patched version of Koeli's Show.java (and .class) that should work with your xmltv feed.

Let me know if it works...

[edit] Removed the changed Show.zip.
I'm getting different results for the same datestring during different tests (off by one hour). Since 1972 does not really seem to need it, I will not spend time trying to find the cause of this, but will simply remove this feature from future updates.

[1972]

Originally Posted by Demideus

Seems like you've got yourself an xmltv feed that does not provide timezones (e.g.: "20050429065900 +0200"). The xmltv format specifies that if the timezone is omitted the date has to be interpreted as UTC.

I've attached a patched version of Koeli's Show.java (and .class) that should work with your xmltv feed.

Let me know if it works...

Ahhhhh, silly me! I'm ussing TVgids_to_xml and I remembered the option in the ini file to add +0200 for the timezones, and it is set to +0000 by default. I am afraid that I have forgetten to set it to the right value after reinstalling Windows XP, SageTV , etc.....

But I'm gonna try your sollution also, thanks Demideus!