Updating DVBS frequency files automatically

[routerunner]

I've been more and more disappointed by the fact that broadcasters are frequently changing their DVBS parameters without notice and as a consequence to miss recordings.

On a standard STB like the SkyBox the lineups are changed automatically OTA, but what about us with tuner cards?

So I came up with the idea to write a SageTV plugin that will be able to fetch data from well regarded websites like "KingOfSat" and automatically update your existing .frq files minimizing the risk of missing recordings

Sounds good?

Well, I'm in the specification phase of the project so this is your chance to provide your suggestions if you wish.

Thanks
Eddy

[routerunner]

Unfortunately on the 1st November 2012 another big lot of frequency changes has effected the UK freesat lovers!

On a bright side, I'm currently testing the HTML scaper I've written which takes the data from "KingOfSat" website and is working pretty well...so keep watching...

Few weeks and hopefully missing recordings will be a thing of the past

Eddy

[paulie]

Sounds like a great idea !

[Oddity]

I've been pondering the exact same issue, just discovered this post.

I have put together a very rough and ready process that sends an Email daily showing the changes.
Take a look at

http://ccgi.mattandden.plus.com/?page_id=60

Also looking to take it further to create the Lineup in Stephane's XML Importer automatically.

[routerunner]

Originally Posted by Oddity

I've been pondering the exact same issue, just discovered this post.

I have put together a very rough and ready process that sends an Email daily showing the changes.
Take a look at

http://ccgi.mattandden.plus.com/?page_id=60

Also looking to take it further to create the Lineup in Stephane's XML Importer automatically.

That's interesting, but I was thinking to something rather simpler. Basically the plugin will read your ".lnb" files and will automatically create/update the relevant ".frq" files as appropriate, then it will periodically look for changes and will update your ".frq" files under the hood.

The data source is the "KingOfSat" website.

The project is looking very promising indeed, as I said before, I've just completed the HTML parsing whereby I'm generating all the required information for SageTV and working on the integration to a proper plugin. I hope to have it done before Christmas as I'm kind of very busy at work and spare time very limited, but I will get there.

Eddy

[routerunner]

I was wondering whether there is potential interest on using this plugin outside Europe or not. I realised that the "KingOfSat" website provides very up-to-date and accurate data for European satellites only whereas "LyngSat" includes Asia and Americas as well, but the data are not updated so frequently and in more than one occasion I did find errors in the database.

Please let me know your thoughts as the "LyngSat" website is completely different to parse and don't want to mantain both, if you see what I mean.

Eddy

[gianpieroizzo]

take a look at DVBE4Sage, for some providers it is able to extract the lineups OTA and to export them for Sage. It is worth to try.

Gian Piero

[routerunner]

I'm looking for someone willing to provide some feedback about the future "DVB Satellite Importer" plugin I'm working on.

I made a java executable that by reading all your current LNB and FRQ files will cross-reference the data with the "KingOfSat" website and will produce a new FRQ files (don't worry the original are left intact) which will have the same channel number and order, but with the up to date parameters.

The cross reference is done by channel name matching which is case sensitive and must be the same as the channel name listed in the "KingOfSat" website. My suggestion is to give it a go first, then if you notice that some of the channels in the new FRQ files are not present is just because the channel name in the original FRQ file is not the same as listed in the "KingOfSat" website, so the only thing that you have to do is to change the channel name accordingly (don't worry SageTV completely ignores that text) and re-run the software.

This beta testing is very important to understand whether the engine is working properly and caters for different tuners configuration, but don't worry, the software doesn't interfere/change anything on your current SageTV setup. The "KingOfSat" website is providing data for European satellites only and, at the moment, the software should work with any East of South satellite position, the West of South satellite position are currently not supported.

If you're willing to help, please send me a PM with your email address and I'll send you a copy of the executable along with the simple instructions.

Many thanks
Eddy

[routerunner]

Regarding the beta testing, I just wanted to add that the software is only supporting tuners used with SageTV natively, so all network encoder solutions (DVB4Sage, DVBLogic, etc...) are currently not supported.

Thanks
Eddy

[Oddity]

routerunner, I realised that sage does not need all the parameters in the .frq file to tune. For example if you add a channel manually all you need to enter in the physical channel field in the following format as:-
onid-tsid-sid-channelnumber
for example BBC1 one would be
2-2045-6301-101

So you could default the rest of the information in the frq file ( or if you could update the channels directly we would not need the frq file at all.

You could also then epg collector as a source for this information as it provides it all. This would also mean that you could get the updates quicker than from scraping the website.

ps. on a similar subject I have made progress in that I now have a c# program taking the epg collector file and creating a lineup for XMLTV importer so that the channels get updated automatically overnight.
I could get it to produce an frq file for you?

[routerunner]

Originally Posted by Oddity

routerunner, I realised that sage does not need all the parameters in the .frq file to tune. For example if you add a channel manually all you need to enter in the physical channel field in the following format as:-
onid-tsid-sid-channelnumber
for example BBC1 one would be
2-2045-6301-101

Very good finding indeed. It may work for DVB-T as well knowing which parameters it needs. But what about the frequency value? Are you saying that sage gets this OTA? Some parameters ar vital to lock on the transponder, unless sage uses the above parameters to connect to web service somewhere that returns the remining info? If I recall BBC1 is one of the channel defined in the "Predefined.FRQ" have you tried with another channel like "The Vault"?

So you could default the rest of the information in the frq file ( or if you could update the channels directly we would not need the frq file at all.

Unless it stores this in the properties file or in the wiz.bin, I don't know how you coud do it without the FRQ file.

You could also then epg collector as a source for this information as it provides it all. This would also mean that you could get the updates quicker than from scraping the website.

ps. on a similar subject I have made progress in that I now have a c# program taking the epg collector file and creating a lineup for XMLTV importer so that the channels get updated automatically overnight.
I could get it to produce an frq file for you?

Well, if you can manage to get the FRQ, job is done, but I don't know the integration with XMLTV Importer (Stephane's?), can you please eleborate a bit more what is the relation with the importer? Does it cross reference the channels information in the EPG with the lineup?

On a side note, I've just started the SageTV 7 plugin integration, which I hope to complete pretty soon. Overrall I'm pleased with the result, is simple, fast and hopefully will make missing recordings a thing of the past...

Eddy

[Oddity]

Yes you'll need to put the transponder frequencies in the predefined.frq file before the {auto} section and this would work better for DVB-T as these frequencies are limited (although I have not tried DVB-T is should work ).

How are you planning to get guide data other than now and next into Sage?

With XMLTV Importer I define a lineup here you can specify the Physical channel number as per sage. Can provide more detail if you want but, basically I link the SID which is also the xml guide data channel reference in EPG collector so it will automaticall link the Channel to the guide data.


If you need some help, I'm willing to do some testing for you.

[routerunner]

Originally Posted by Oddity

Yes you'll need to put the transponder frequencies in the predefined.frq file before the {auto} section and this would work better for DVB-T as these frequencies are limited (although I have not tried DVB-T is should work ).

How are you planning to get guide data other than now and next into Sage?

With XMLTV Importer I define a lineup here you can specify the Physical channel number as per sage. Can provide more detail if you want but, basically I link the SID which is also the xml guide data channel reference in EPG collector so it will automaticall link the Channel to the guide data.


If you need some help, I'm willing to do some testing for you.

- I have three lineups in XMLTV Importer (DigiGuide (UK), mc2xml (Italy), mc2xml (Italy/France/Germany)).

- Channel mapping is one to one with FRQ as we like to have the multilanguage channels ordered in groups.

- I have 8 DVBS tuners (2 x TBS 6984 Quad), 4 dedicated UK, 3 dedicated Italy, 1 in DiSEqC switching for Italy, France, Germany.


I'm going to send you the executable to try along with some instructions.

Eddy

[Oddity]

Been testing the program you sent ran without errors and produced a new file. I have it scheduled to run daily to see that it works with a new channel or change is made.

One question how do you set the region correctly ? Ie I want a lineup for London ( Crystal Palace ).

[routerunner]

Originally Posted by Oddity

Been testing the program you sent ran without errors and produced a new file. I have it scheduled to run daily to see that it works with a new channel or change is made.

One question how do you set the region correctly ? Ie I want a lineup for London ( Crystal Palace ).

You need to set the channel name (after the :#) in the original .FRQ the same as it appears in the KingOfSat website, that's it.

:#BBC One London

Eddy

[Oddity]

routerunner, all working well.
Just a minor thing

Bliss and greatest hits tv changed their names last night to Blissmas and Christmas Hits.

[routerunner]

I've completed the first SageTV v7 plugin version of this software which will be announced and available to download from the "General" plugin repository within the next couple of days.

Thanks
Eddy

[routerunner]

I just released version 1.0.2 of the "DVB-S Importer" plugin, but I was wondering whether some of you might be interested to have this plugin working outside Europe as well.

Please let me know here so that I can prioritize the work accordingly...

Thanks
Eddy

[Fredo44]

great idea and well implemented. Works as expected.
thank you !!

[routerunner]

Originally Posted by Fredo44

great idea and well implemented. Works as expected.
thank you !!

Thank you for using it

Have you got any suggestion? The thing is that next year I'll move my interest into Android development and my time for SageTV stuff will be gradually fading down to zero, so if you want anything particular to be added this is the right time to ask

Eddy