New Aussie grabber [Archive] - SageTV Community

ozfiles

05-09-2006, 03:54 PM

Ian, the only reason I am using the MCE version is because I came across it on the aussie MCE forums. I think either version would work well with sage.

If you intend to use it with windows scheduler then you have to include the tkn file as well ie, c:\your program path\dugg.exe c:\your program path\dugg.tkn

I am using the Demideus xmltv plugin and it works no problem.

regards

Ian Norman

05-11-2006, 05:12 PM

Oz
Thanks for the info

I thought I'd try all the EPG progs and Plugins to see which one was the easiest to setup.

I'm having lots of fun !!! ( well sort of )

Cheers

h8dk97

06-05-2006, 06:19 PM

I've tried to use it on my PVR computer and it comes up with some weird errors, do you know if it requires some ports open on the firewall? I'm blocking pretty much everything except 80.
Thanks.

ozfiles

06-06-2006, 03:34 AM

h8dk97

06-06-2006, 07:25 PM

No, you don't need to open any ports. NineMsn in all there wisdom decided to change the encryption yet again and the grabber has needed to be updated in a hurry.

You can get the new grabber here http://dugg.ueuo.com/

regards.

What I meant was that this program works on my computer in the office but for some reason doesn't work on my home pc. In either case I've updated the PHP script I've been using previously and it seems to work now. I'll post an update shortly.

ozfiles

06-07-2006, 02:26 AM

I did have errors this morning but that was because my internet connection went down when the program was updating. I just ran it again and it updated no problem.

Good to hear about the PHP script update look forward to it.

regards

ozfiles

06-15-2006, 05:32 PM

Once again NineMsn have decided to play games and Dugg is producing errors. It should be fixed by the weekend according to the developer.

This whole lack of an official and free epg in Australia is really unacceptable and needs to be addressed by the networks. There is a big discussion over on the xpmediacentre forums http://www.xpmediacentre.com.au/community/australian-tv-guide/ for those that are interested.

regards.

h8dk97

06-16-2006, 12:11 AM

Yep, they've changed javascript once again, but it shouldn't be hard to address if you have enough time. Hopefully I'll have some time to change the php script as well.

ozfiles

06-28-2006, 03:35 AM

Dugg just got updated and is working fine, go get it guys.

Make sure you read the setup file that comes with the file there is some things you will need to do to your config file.

http://dugg.ueuo.com/files/dugg3-0.zip

PaulG

06-28-2006, 06:12 AM

OK, so we have a version that works a little better than the older one (this might be due to ninemsn screwing us over yet again :bang: ) however....

Is there anything else that needs to be done with dugg to get sagetv to use the outputted xml file :confused: ?

I used to use the xmltv plugin for sage but since tvharvest was no longer being updated & I hadn't been able to get the tvharvest proxy working I've been without a guide for sooooo long.

I've got dugg running atm but the foxtel digital guide is pretty large & its taking awhile

TIA
Paul

PaulG

06-29-2006, 04:10 PM

OK, I guess I don't need anything else to crunch the data... xmltv plugin works fine with it.

YAY I have an epg once again

Thanks for the help guys :p

ozfiles

06-29-2006, 05:03 PM

Sorry Paul was busy over the last couple of days.

I'm glad you got Dugg setup ok it would have to be the best of the bunch at the moment.

There is one that I use as a fallback. The oztivo grabber http://forums.freytechnologies.com/forums/showthread.php?p=124894&postcount=5 is a pretty good backup if NineMSN falls down again.

Channel Nine seem to be threatening even the oztivo guys with copyright infringment. I find the whole thing laughable and lets hope common sense prevails.
Kama seems to be coming back to bit Nine on the butt with all the trouble Eddie is having just keeping the sinking ship above water. There business model seems to be antiquated and Eddie needs to pull his finger out.

http://australianit.news.com.au/articles/0,7204,19248149%5E15333%5E%5Enbv%5E15306%2D15321,00.html

http://australianit.news.com.au/articles/0,7204,19545042%5E15309%5E%5Enbv%5E,00.html

ozfiles

06-30-2006, 04:31 PM

Another new update to handle new encryption added yesterday by NineMSN.

http://www.asazel.com/dugg.tkn

This one will automatically update Dugg when the source changes and Dugg needs to be updated.
You will need to put a line into your Dugg ini file for the autoupdate to work. autoupdate

Once again Nines efforts have been foiled again.

enjoy.

PaulG

06-30-2006, 05:35 PM

I guess that explains why my temp directory was blowing out to 1.4GB+

PaulG

06-30-2006, 10:11 PM

Ozfiles,
Have you tested the new tkn file ???

It looks odd to me, all I get is the dugg window coming up & whenever the mouse is over it its continually cycling from text pointer to busy & back.

ozfiles

06-30-2006, 10:20 PM

Hey Paul, yes getting the same behaviour as you are.

The older tkn would only grab the free to air and not the foxtel data so they have definitely changed the encryption 'sigh' yet again.

I will have to let the author know.

regards.

ozfiles

07-01-2006, 04:03 AM

Just got a reply from the author and this is what he said.

"This is normal...it is decrypting an index page, and it takes a long time. I have found that there is an inefficient part of the process that I can modify to speed it up quite alot and I'm working on this now.

A typical FTA 6 day grab involves downloading and decrypting about 1000 web pages. The same thing for pay-TV can involve 10,000 web pages. It has alot of work to do. The decryption is complex, and involves up to 8 steps (ie the data is decrypted, then the result decrypted again and so on-up to 8 times.) I have mentioned before to expect it to be slow, and there is at least one segment that I can improve on."

I just ran a test only grabbing a couple of days of foxtel and free to air data and it was still going after an hour and a half but had grabbed one of the days no problem so I think we just have to be patient on the first grab because of the shear amount of data Dugg has to churn thru.

I think the ftp will be up and running soon enough and that will reduce the download time to a more acceptable margin.

regards.

PaulG

07-01-2006, 04:32 AM

Thanks ozfiles,

NExt time you speaking with him, you might want to make the suggestion of adding some additional feedback into the app.

eg. getting page 000001, decrypting page 000001 etc etc etc

At least we then know its doing something.

Also the ability to define how much CPU time it uses would be handy.

ozfiles

07-01-2006, 10:36 PM

Paul just got an update to Dugg to fix the loop problem.
You will have to clear your cach to do this put http://dugg.ueuo.com/files/newtkn.txt in your browser and it will come up with garbage and then hit refresh.
Make sure you have enabled autoupdate in your config file and it will download Dugg 3.3.

Working a lot faster now.

regards.

PaulG

07-01-2006, 10:43 PM

There's an automatic update option in the config file ?!?!?!

You obviously have access to a LOt more doco on dugg then what I have, I can't find anything that is related to an automatic update

ozfiles

07-01-2006, 11:00 PM

All you have to do is just put autoupdate above mode=mp or mce what ever you are using I'm using mp "media portal" because mce will give a different output but it doesn't really matter.

autoupdate
mode=mp
service=94
service=168
grabdays=7
xmlname=1.xml
xmldir=C:\tvguide\
rating=ABA
timezone=+1000

PaulG

07-02-2006, 02:56 AM

autoupdate

Thanks ozfiles,

That was all I was missing.

the new updates work HEAPS quicker now.

Thanks for your help & also the dugg dev's help in getting this sorted out so quickly.

ozfiles

07-02-2006, 03:07 AM

That's ok Paul.

Lets see how long it all lasts before nine start their tit for tat tactics again.

regards.

maninblack_30

07-09-2006, 05:38 PM

Hello there, I have just finished setting up SageTV 5 with my dual tuner DVB-T card and it's working really well, but I cant make any sense of this XMLTV stuff, I'm in Melbourne, Australia, and could really do with some help.
Here's hoping,
Fraser

PaulG

07-09-2006, 06:12 PM

Hi Fraser,

I can sympathise with you in relation to that!

I'm having a few issues myself with it but I'll see if I can help you out a bit ;)

Do you have Dugg setup & running? & is it outputting the xml file ?

That should be the easy part.

now you need an xmltv plugin from here
http://www.sage-community.org/index.cgi/wiki/XMLTV

I can' tell you which one I'm using atm as I'm not at home so I can get that to you tonight.

follow the readme in the zip file you download (extract the class files to the sagetv\sagetv\xmltv directory & put the xmltv.properties file in teh sagetv\sagetv directory & modify if need be.

Stop SageTV (including the service if you use it) update the sage.properties file so it knows about the xmltv plugin (again see readme file in the zip file you downlaoded), start sage & reconfigure your tuner setups to use it.

That should be it pretty much.

I'll be able to give you more info once I get home tonight (I'm also in Melbourne)

HTH
PaulG

maninblack_30

07-09-2006, 06:18 PM

Ta Paul for your prompt reply, I have dugg and it seems to be working, the problem I'm having is with the part where they say to "add/modify source" I can't find anywhere to put in "00000" and "trial", I mus admit I didn't read the readme file with the download, I just followed instructions in some US posts, I'll give it another try over the next hour or two.
On another note, have you tried a mediaMVP by any chance? what's the picture quality like?
Cheers,
Fraser

PaulG

07-09-2006, 06:28 PM

Fraser,

It sounds like you don't have the XMLTV plugin installed/configured correctly.

Unless someone else jumps in before I get home, I'll be able to spend a bit more time on this this evening (damn work gets in teh road of all the fun stuff!)

maninblack_30

07-09-2006, 06:56 PM

The only problem is the times are out, I'm guessing by our GMT+10.
Any hints?
Man this is fantastic!

PaulG

07-09-2006, 07:14 PM

The only problem is the times are out, I'm guessing by our GMT+10.
Any hints?
Man this is fantastic!

Do you have a 'timezone' statement in your dugg.ini file ?

Otherwise, do you have your regional setting in windows setup correctly ?

Windows usually defaults to US regional settings unless explicitly told otherwise.

maninblack_30

07-09-2006, 07:15 PM

Just had to change the dugg.ini file to +1000!!!!!!
Now I'll go upstairs to my laptop and wath live tv there!!!!
Thankyou for your help, I was putting the plugin in the wrong directory, I was putting it in the java folder, not sage's.
Thankyou once again!
Fraser

ozfiles

07-09-2006, 07:18 PM

Welcome Fraser, lets see if we can get you going.

I'll start from the start and you can check that you have done the steps with your setup.

1. Download xerces from here.

http://www.myhtpcresource.net/downloads/xerces.jar

Extract the file to C:\Program Files\Java\jre1.5.0_06\lib\ext

2. Make sure Sagetv is closed and the sage server is stopped and go to your Sage properties which you can find in C:\Program Files\SageTV\SageTV and open with notepad and find this line,

epg/epg_import_plugin=

and change it to:

epg/epg_import_plugin=xmltv.XMLTVImportPlugin

3. Download a xmltv importer.

Now the one that I use is nielm's http://forums.sage.tv/forums/attachment.php?attachmentid=750&d=1084645698

Extract to C:\Program Files\SageTV\SageTV and it will create an xmltv folder at that location.

4. Get Dugg from http://dugg.ueuo.com/ and configure dugg to get your state epg and paytv epg (if you have foxtel) the xml file that dugg needs to download should be called 1.xml but since you already have dugg up and running you just need to rename your file that dugg has downloaded to 1.xml and place this file in C:\Program Files\SageTV\SageTV

5. Go to setup video sources choose one of your video sources and click on it to bring up the next screen.
Click on EPG Lineup
Use US, Canda, or XMLTV Guide Data with this Source
Cable or Satellite
Use zip/Postal Code and put 00000 in the box
Lineup in 1.xml

Your channels will be downloaded takes a minute or so and then on the last screen just click on Extended Basic Service but I don't think it really matters.

Then just click on the channels you want to have show in your epg (they will have a green dot beside them) and click on Done with Channel Setup and then Finish Source setup. Unless you have another source to add and your done.

You should now have a nice epg with all the shows listed.

regards.

edit: ok, you got it going, good stuff. I hope this will help others that are having trouble setting up xmltv

h8dk97

07-12-2006, 07:01 PM

Is it only me or dugg doesn't work? It runs for about a minute and produces no xml file. Anyone?

ozfiles

07-12-2006, 07:24 PM

Nine have done it to us again.

The Dugg author is looking into it.

I'm using the oztivo grabber as my backup and it is still a good alternative.

maninblack_30

07-12-2006, 07:24 PM

Last I checked (last night) it seemed the source of it's data wasn't available, or had perhaps changed so it couldn't grab the data,but I'm no expert!
hopefully someone here will let us know what's happening!
It really is about time we had a free epg we could use! God I wish the fta would transmit the data as the dvb-t specs allow!all to busy fighting for bandwidth etc, then transmitting the bear minimum!
cheers,
Fraser

maninblack_30

07-12-2006, 07:29 PM

Ahh, second time we've posted at the same time ozfiles!
Thought that might be the case!
As the owner of a Topfield pvrt5000, the epg thing is a real PIA!
but so far I'm impressed with sage, and if I can get a MVP cheap enough, I think this may replace most of it.(howmany times are you watching something pre-recorded, then get tired and want to watch the rest in bed? This will be the best part of Sage (I hope!)
Fraser

ozfiles

07-12-2006, 07:56 PM

but so far I'm impressed with sage, and if I can get a MVP cheap enough.

Totally agree.

I will be getting an MVP for my media center setup when I get it all sorted out.

Cheapest price I have found for the MVP is $228.80 + around $11.00 for delivery.

http://www.mitec.net.au/products.php?do=details&id=935

h8dk97

07-12-2006, 07:57 PM

I saw another program (don't remember what it's called) for MS Media Center, which also grabs data off msn site but they claim it's using IE engine, so there's no need to decrypt, it just gets the end result as you see it in your browser. I guess that solves the issue with constant changes in encryption. Do you know anything about it? would that work with sagetv?

Oh, and we do you get oztivo?

Thanks.

Nine have done it to us again.

The Dugg author is looking into it.

I'm using the oztivo grabber as my backup and it is still a good alternative.

ozfiles

07-12-2006, 08:41 PM

I do recall that program but I think the descriptions were very limited. For the life of me I can't remember the name of it. (as he goes off searching)

I put a setup guide for the oztivo epg here http://forums.freytechnologies.com/forums/showthread.php?p=124894&postcount=5

You will have to install python but it is worth it to have a guide that its always available.

regards.

maninblack_30

07-12-2006, 09:50 PM

Ozfiles, have you thought of buying one from the US over ebay?
I think this is the way I'll go just to try the whole thing, they run on 6 volts DCso I'll just throw away the power supply that comes with it, and buy a 6 volt plug-pack(with a high enough current rating) from Jaycar. they are about $100US (Au$131) but I need to see what the postage here will cost this guy (http://cgi.ebay.com.au/ws/eBayISAPI.dll?ViewItem&ih=016&item=260006989640&rd=1&sspagename=STRK%3AMEWA%3AIT&rd=1) is getting back to me soon with a price, if you are interested I'll let you know the cost. (perhaps we could get 2 and have them shipped together to save on postage?)
Cheers,:goodjob:
Fraser

ozfiles

07-12-2006, 10:57 PM

Sounds tempting Fraser but I think I will buy it locally.

I just prefer a local warranty when it comes to these sort of components.

Thanks for the offer though.

regards.

maninblack_30

07-12-2006, 11:15 PM

Can I ask where you got that price from Ozfiles? Perhaps I'll go local too for the first one, then buy subsequent ones fron O'seas (read kitchen, spare room, lounge, etc.....) and if the "dark picture" issue is noticable I can get a refund as faulty.
cheers,
Fraser

ozfiles

07-12-2006, 11:33 PM

Whack what you are looking for into here. It will list all the computer stores who sell it from the lowest to highest price.

http://www.staticice.com.au/index.html

From that list mitec are the cheapest but are in Queensland.
http://www.mitec.net.au/products.php?do=list&sc=3

regards

h8dk97

07-13-2006, 06:13 PM

Hi ozfiles,

What version of python are you using? I assume it's 2.4, I've donwloaded the latest one and when trying to run tv_grab_au_tuhs.py with configure option it complains that datetime module is not found:

$ ./tv_grab_au_tuhs.py --configure
Traceback (most recent call last):
File "./tv_grab_au_tuhs.py", line 61, in ?
import mx.base.datetime
ImportError: No module named datetime

Any ideas?

Thank you.

I do recall that program but I think the descriptions were very limited. For the life of me I can't remember the name of it. (as he goes off searching)

I put a setup guide for the oztivo epg here http://forums.freytechnologies.com/forums/showthread.php?p=124894&postcount=5

You will have to install python but it is worth it to have a guide that its always available.

regards.

ozfiles

07-13-2006, 07:04 PM

h8dk97

07-13-2006, 09:35 PM

Apparently they have cut down the python script and left out that module.

This is the one that I have and it worked no problem on a recent install I did.

You can get it from here http://www.sendspace.com/file/hkf5h7

No, I'm just stupid. I'm running cygwin on my pc and it was getting the cygwin python installation from the PATH. Got the config script, now trying to get it all working. Thanks for your help.

h8dk97

07-16-2006, 01:31 AM

Thanks ozfiles, I've got it all working. Got free epg again, hopefully this will last longer as it doesn't rely on msn.

ozfiles

07-16-2006, 02:19 AM

No worries h8dk97.

I've always had a backup just in case Ninemsn falls over.

Keep an eye on your favourites though oztivo might name a show something different to Ninemsn.

Only noticed on a few shows though, most should be the same title.

regards.

maninblack_30

07-17-2006, 01:40 AM

Hello again,
Any hint as to when Dugg will be working again anyone?
Ta,
Fraser

ozfiles

07-18-2006, 04:47 AM

Fraser, NineMsn keep playing musical chairs with the source. So I think the best option for now is to use the oztivo xmltv.
There a couple of grabbers working at the moment using the ninemsn source but they have limited descriptions for the programs because they are using the search pages associated with Ninemsn to scrap from.

I always had oztivo setup just in case the nine source fell over and it seems to be reliable enough for our purposes.

Check out the install guide I wrote and see how you go with oztivo. http://forums.freytechnologies.com/forums/showthread.php?p=124894&postcount=5

PaulG

08-15-2006, 09:27 PM

Do we have a working version of dugg as yet ?

ozfiles

08-15-2006, 10:53 PM

Here you go Paul.

http://www.filehosting.cc/download.php?id=992B6E96

I just got this on the weekend and it seems to be holding up to Ninemsn and their games.
You have to submit the numbers in the box halfway down the page to get the file.

The config file has changed a little, "no more services just add the channels you want," so make sure you read the instructions.

If you have any trouble let me know.

regards.