Now we can grab XMLTV data from Microsoft Media Center servers

[MCE-Refugee]

This might be of interest for those who aren't using Digiguide as their XMLTV source. I just discovered this new program that allows you to connect to Microsoft's servers & collect the EPG data which is then converted to XMLTV format. I had always imagined that this data was encrypted but evidently not (or the program's author has cracked the encryption:-).

I haven't even tried importing this into Sage yet but from a quick look at the data it is very comprehensive with previously shown & series data. I was hoping that someone who understands XMLTV a bit better would take a look & give an assessment.

The web site has full instructions on use but basically it's as simple just downloading the executable (Windows or Linux) from the web site & at a command prompt run it & give your post code e.g.
mc2xml -c gb -g "SW1X 7LA"
then choose your provider (Freeview, Freesat, Cable etc) from the list e.g.
15: SAT Sky Digital BBC London/ITV London
then it creates the data in xmltv.xml ready for upload into Sage

http://mc2xml.110mb.com/

[jaminben]

Looks intresting, but I have no idea how to get this into Sage maybe someone else will take a look and see if its better than Digiguide.

Nice find though

[MCE-Refugee]

In theory I should just be able to point Stephane's Importer at the XML file that it produces & have the data imported. Unfortunately that doesn't work as when try to add a new XMLTV source & browse to the XML file it gets highlighted in a red box & I cannot proceed any further. I assume that there is something about the syntax of the file that Stephane's program doesn't like even though to my untrained eye it looks like a perfectly valid XMLTV file. I will contact Stephane & see what he says. In case anyone else would like to take a look I will append an XMLTV file with just a few channels to this posting.

[jaminben]

Originally Posted by MCE-Refugee

Unfortunately that doesn't work as when try to add a new XMLTV source & browse to the XML file it gets highlighted in a red box & I cannot proceed any further.

Same thing happens for me. I guess the data isnt compatible with the importer.

[StephaneM]

Originally Posted by MCE-Refugee

Unfortunately that doesn't work as when try to add a new XMLTV source & browse to the XML file it gets highlighted in a red box & I cannot proceed any further.

Mouse over the text box and you'll see the error, in this case, this is because you are missing the DTD file (it must be in the same folder as the XMLTV file).

Regards,
Stéphane.

[MCE-Refugee]

Stephane,

Thank you for that link. I downloaded that file into the directory & now it all works beautifully. I am not sure that the programme data is any better than Digiguide but perhaps it has more information about broadcast dates & series link info?

It seems that your program does not ship with this xmltv.dtd file which is the standard data defintion file for what an XMLTV file contains. The xml files that I imported previously did not have this file defined in the header & were imported without problems. If I remove the reference to xml.dtd from the input file then it is imported OK. I am sure that there is a good technical reason why you have chosen to do it this way but wouldn it be better to either ship the xmltv.dtd file with your application or if a file contains a reference to this descriptor file then to ignore it if the file is missing?

[StephaneM]

Originally Posted by MCE-Refugee

wouldn it be better to either ship the xmltv.dtd file with your application or if a file contains a reference to this descriptor file then to ignore it if the file is missing?

Well the grabber should include the DTD file if it makes reference to it (or at least give a real URL). I don't include the DTD file with the Importer, however you can download it, and there is a FAQ entry about needing the DTD if the Importer complains about it.

Regards,
Stéphane.

[motobarsteward]

This appears to work very well! At last, what appears to be a complete channel listing and EPG for SKy! Has anyone worked out how to get the channel number to channel ID mapping? I know that WMC gets this info. If we can get this info that Sky's annoying daily channel number re-mapping PITBS will be a thing of the past!

[jaminben]

It would also be good to get this working with less fuss and hassle for people like me that dont really understand how to use command lines etc and would prefer a simple UI, not only for the UK but other countries around the world (I think I'm right in assuming it works around the world).

Wink Wink StephaneM or some other clever person.

[doc]

It certainly doesn't look bad does it.

I'm still using the Radio Times EPG, and the only channel I'm missing from what I want it Movies4Men2.

When I download the EPG for UK satellites, it lists Movies4Men2 as a channel, but there are no programs for it. There aren't for Movies4Men either.

I haven't done a comprehensive channel comparison, but I think to get complete listings for all channels, you may well still need to use either Radio Times or Digiguide aswell as this data.

[motobarsteward]

Has anyone worked out which number to choose for Sky FreeSat complete with channel numbers?

[rickgillyon]

Originally Posted by jaminben

Wink Wink StephaneM or some other clever person.

Stephane's Import tool will do this already. You need to put this mc2xml in a folder where you want to call it from, open a command window, go to the folder and type:
mc2xml -c gb -g "SW1X 7LA" (your postcode)

Then pick your lineup from the list. This saves the preferences, so in the Importer you just need to put <folder>\mc2xml.exe as your importer, nothing else, no batch file needed.

[motobarsteward]

Unfortunetaly, the Sky FreeSat linueup offered is marked as 'Unmapped'. I pulled this down and found that 'Unmapped' means that there are no channel numbers.

If you, like me, only get the Sky FreeSat Free to view channel lineup then you will need to select the full sky lineup (19, in my case in Swindon) and use the mc2xml.chl mechanism to just download the free to view channels. When I have written this file, I'll post it.

[doc]

Because I have Freeview and Freesat, I created 2 folders and put a copy of mc2xml.exe in each folder.

Then I ran it from one folder with the postcode options and chose the lineup for freeview, and it creates the dat file. I then ran it from the other folder and it created another dat file in that folder.

I then used Stephanes importer and you can specify different XML files for each of your lineups.

Like motobarsteward said earlier, the Freesat lineup doesn't have numbering, and it has loads of channels that you would never use. I created a chl file in the folder with the Freesat dat file and specified just the channels I wanted, this reduced the XML file to something more manageable.

If you tell Stephanes importer to run the grabber, you have to create a batch file that will run mc2xml from each folder in turn.

I haven't changed my live system as I have all the channels I need at present from Radio Times, except for that Movies4Men2 which has no programs listed from this grabber. (Its probably because theres nothing worth watching on it!)

If I were setting up from scratch and just using Freeview, I may well give this a go. I think it could also be very useful for people in other countries who don't have a reasonable XML grabber. I'd like to know if its any good for my friend in NZ.

[jaminben]

Originally Posted by rickgillyon

Then pick your lineup from the list. This saves the preferences, so in the Importer you just need to put <folder>\mc2xml.exe as your importer, nothing else, no batch file needed.

I assume this couldn't be done to grab two lineups (Freesat & Freeview) so you would need to create a batch file as Doc sugested above?

Originally Posted by motobarsteward

If you, like me, only get the Sky FreeSat Free to view channel lineup then you will need to select the full sky lineup (19, in my case in Swindon) and use the mc2xml.chl mechanism to just download the free to view channels. When I have written this file, I'll post it.

This would be very handy as I had a quick look at that but as you say their is massive amounts of channels to sort through.



I've made a Freeveiw lineup and so far the results look very good for first run shows not perfect but better than Digiguide & RT. An example would be Neighbours which shows as first runs for every show and Home & Away is always shown as a repeat. However is does seem better at picking up first runs for primetime shows.

One other thing it does better is currently off air channels showing when the channel starts broadcasting again, much better than the "NO Data" message.

Edit:

I've figured out that I can use Scheduled tasks to run each of the mc2xml.exe grabbers (Freeveiw & Freesat) then have the importer start at a slightly later time to import the data.

My next problem is Freesat and the use of mc2xml.chl mechanism as said by motobarsteward and the mc2xml website.
How do you create the .chl file? I've created a txt file and renamed it .chl but that didnt work and the programme still grabbed all 75mb worth of data which to say the least is a bit excesive.

[StephaneM]

Originally Posted by jaminben

I assume this couldn't be done to grab two lineups (Freesat & Freeview) so you would need to create a batch file as Doc sugested above?

No, you don't have to create a batch file. In the XMLTV Importer, each XMLTV source has its own grabber settings.

So you can very well put mc2xml in two different folders and create two XMLTV sources in the Importer, one for each produced XMLTV file (the Importer will then merge the data during the import)

[jaminben]

Originally Posted by StephaneM

No, you don't have to create a batch file. In the XMLTV Importer, each XMLTV source has its own grabber settings.

So you can very well put mc2xml in two different folders and create two XMLTV sources in the Importer, one for each produced XMLTV file (the Importer will then merge the data during the import)

Ah, Thats cool

I've just setup two scheduled tasks to do this but I'll take a look at the importer again and try to use your way of doing things.

Cheers StephaneM

On a diffrent note I've been playing around with the catagories and once I have these setup and start the import process the importer seems to hang at "Cleaning Lookup Table". It doesnt seem to cause any problems as far as I can see, however I do have to press the OK button to finish the import. Is this a bug or something I'm doing wrong? (Its not a time issue as I've left it overnight and the progress bar doesnt move)

[StephaneM]

Originally Posted by jaminben

the importer seems to hang at "Cleaning Lookup Table". It doesnt seem to cause any problems as far as I can see, however I do have to press the OK button to finish the import. Is this a bug

Surely, there must be a bug preventing the import from running to completing, review the log files (trace_xxxxx.log) in the Config & Logs folder, I'm sure you'll find an unexpected error in there (forward it to me then).

Regards,
Stéphane.

[stux]

[QUOTE=jaminben;303867]

I've figured out that I can use Scheduled tasks to run each of the mc2xml.exe grabbers (Freeveiw & Freesat) then have the importer start at a slightly later time to import the data.
QUOTE]

You don't even need to do that with Stephane's importer, add the path to the grabber and then go to scheduled update and enter a time. The impoter does a grab before converting to sage. All done in the one tool.

[jaminben]

Originally Posted by StephaneM

Surely, there must be a bug preventing the import from running to completing, review the log files (trace_xxxxx.log) in the Config & Logs folder, I'm sure you'll find an unexpected error in there (forward it to me then).

Regards,
Stéphane.

Hmmm, its stopped doing it now my pc has had a BSOD If it does it again I'll send you the error.

Originally Posted by stux

You don't even need to do that with Stephane's importer, add the path to the grabber and then go to scheduled update and enter a time. The impoter does a grab before converting to sage. All done in the one tool.

Thanks for that, I've also just realised that you can import a lineup from an .frq file and it matches the channels to an xml file automatically with about 95% accuraccy. Thats very sweet indeed


Edit:

Its done it again, its proberly something I'm doing and not related to the importer.

20080912 00:11:15.510 - Error - Unexpected error :Column 'epgid' is constrained to be unique.  Value 'EP0000080001' is already present.
                                   at System.Data.UniqueConstraint.CheckConstraint(DataRow row, DataRowAction action)
                                   at System.Data.DataTable.RaiseRowChanging(DataRowChangeEventArgs args, DataRow eRow, DataRowAction eAction, Boolean fireEvent)
                                   at System.Data.DataTable.SetNewRecordWorker(DataRow row, Int32 proposedRecord, DataRowAction action, Boolean isInMerge, Int32 position, Boolean fireEvent, Exception& deferredException)
                                   at System.Data.DataTable.InsertRow(DataRow row, Int32 proposedID, Int32 pos, Boolean fireEvent)
                                   at System.Data.DataRowCollection.Add(DataRow row)
                                   at LMGestion.XMLTVImporter.XMLTVImporter.LoadProgramsDocument(String lineupId)
                                   at LMGestion.XMLTVImporter.XMLTVImporter.DoImport()

I've also attached the log