Utility: Web Radio and TV Encoder

[scanner95]

Originally Posted by michaeldjcox

Hi Yuly,


Does the file name or the URL (if we download it from wherever you are downloading it from) contain any parts representing the date - or in fact any parts that can change over time?

Michael

No, the name of the file does not change.

Yuly

[n9cqs]

Michael,
Thanks again for your terrific plug-in.

I have a question about the format of the time. When I try to specify a different format than the one that you have listed, it does the first harvest, but when I pull the station back up, it does not seem to store it, but is blank and provides only the same four formats that are "defaults". Is there a file that I can edit to add other possibilities? I have looked around and have not yet found where to do it.

Thanks SO MUCH!
Jeff

[michaeldjcox]

Originally Posted by n9cqs

Michael,
Thanks again for your terrific plug-in.

I have a question about the format of the time. When I try to specify a different format than the one that you have listed, it does the first harvest, but when I pull the station back up, it does not seem to store it, but is blank and provides only the same four formats that are "defaults". Is there a file that I can edit to add other possibilities? I have looked around and have not yet found where to do it.

Thanks SO MUCH!
Jeff

Hi,

Which timeformat field is that i.e. where on the config GUI?

Michael

[scanner95]

Hello Michael,

One small problem I just noticed is that I see VLC running and recording one channel, allthough I stopped it. Actually, it was about 4 hours ago I watched it, and now I see that VLC process is up, and it constantly pushes data into my recording directory.
Sage is up and running. I've attached encoder.log; the channel is called "Boets".

Cheers,
Yuly

[michaeldjcox]

Originally Posted by scanner95

Hello Michael,

One small problem I just noticed is that I see VLC running and recording one channel, allthough I stopped it. Actually, it was about 4 hours ago I watched it, and now I see that VLC process is up, and it constantly pushes data into my recording directory.
Sage is up and running. I've attached encoder.log; the channel is called "Boets".

Cheers,
Yuly

Stopping VLC has occasionally been a problem. I no longer have problems in my own setup but other WFE users do.

So I am contemplating killing VLC at the end of tuning in a future release.

The key thing is that in the "detune" command VLC is told - via the remote control port to stop and quit

There can be problems if that message does not get through.

The above from your log file is a typical correct sequence where at the end we are telling VLC to stop recording and quit and VLC seems to be saying OK.

08.03.2010 17:41:26 RecordingThread uk.co.mdjcox.webfeedencoder.RecordingThread stopRecording INFO: Stop recording channel=Boets file=D:\rec\Boets-2157132-0.mpg
08.03.2010 17:41:26 RecordingThread uk.co.mdjcox.webfeedencoder.RecordingThread processCommand INFO: Command: VlcCommand %streamAsVideoCommandPort% stop clear quit
08.03.2010 17:41:26 RecordingThread uk.co.mdjcox.webfeedencoder.RecordingThread processCommand INFO: Command: VlcCommand 5001 stop clear quit
08.03.2010 17:41:26 RecordingThread uk.co.mdjcox.webfeedencoder.RecordingThread vlcCommand INFO: Write to VLC port 5001 : stop
08.03.2010 17:41:26 RecordingThread uk.co.mdjcox.webfeedencoder.RecordingThread vlcCommand INFO: Write to VLC port 5001 : clear
08.03.2010 17:41:26 RecordingThread uk.co.mdjcox.webfeedencoder.RecordingThread vlcCommand INFO: Write to VLC port 5001 : quit
08.03.2010 17:41:26 Thread-218 uk.co.mdjcox.webfeedencoder.RecordingThread$1 run INFO: VLC response from port 5001: status change: ( stop state: 0 )
08.03.2010 17:41:26 Thread-218 uk.co.mdjcox.webfeedencoder.RecordingThread$1 run INFO: VLC response from port 5001: stop: returned 0 (no error)
08.03.2010 17:41:26 Thread-218 uk.co.mdjcox.webfeedencoder.RecordingThread$1 run INFO: VLC response from port 5001: clear: returned 0 (no error)
08.03.2010 17:41:26 Thread-218 uk.co.mdjcox.webfeedencoder.RecordingThread$1 run INFO: VLC response from port 5001: quit: returned 0 (no error)
08.03.2010 17:41:26 Thread-218 uk.co.mdjcox.webfeedencoder.RecordingThread$1 run INFO: VLC response from port 5001: status change: ( stop state: 0 )
08.03.2010 17:41:26 Thread-218 uk.co.mdjcox.webfeedencoder.RecordingThread$1 run INFO: VLC response from port 5001: status change: ( quit )

Theres lots of tuning and detuning going on in your log file.

So can I suggest a very controlled test.

Stop the encoder, delete the encoder.log, kill any stray VLC sessions you see, start the encoder, watch your channel *once* in the way you did before, stop watching, check to see if the VLC is still present and then send me the logs if VLC did not stop.

Michael

[scanner95]

Hello Michael,

In a controlled test it worked as it should: VLC not present after the stop was pressed. One strange thing, though, is that it complains about SageTV being dead all the time:

09.03.2010 10:20:00 ClientThread-33307780 uk.co.mdjcox.webfeedencoder.ConnectionThread answer INFO: Answer: 140410880
09.03.2010 10:20:15 ClientThread-33307780 uk.co.mdjcox.webfeedencoder.ConnectionThread run INFO: SageTV has possibly died
09.03.2010 10:20:15 TuningThread uk.co.mdjcox.webfeedencoder.TuningThread listenForConnections INFO: Accepted connection
09.03.2010 10:20:15 ClientThread-6824966 uk.co.mdjcox.webfeedencoder.ConnectionThread process INFO: Command: GET_FILE_SIZE D:\rec\NatGeoWild-2174087-0.mpg
09.03.2010 10:20:15 ClientThread-6824966 uk.co.mdjcox.webfeedencoder.ConnectionThread answer INFO: Answer: 147537920
09.03.2010 10:20:30 ClientThread-6824966 uk.co.mdjcox.webfeedencoder.ConnectionThread process INFO: Command: GET_FILE_SIZE D:\rec\NatGeoWild-2174087-0.mpg
09.03.2010 10:20:30 ClientThread-6824966 uk.co.mdjcox.webfeedencoder.ConnectionThread answer INFO: Answer: 154648576
09.03.2010 10:20:45 ClientThread-6824966 uk.co.mdjcox.webfeedencoder.ConnectionThread run INFO: SageTV has possibly died
09.03.2010 10:20:45 TuningThread uk.co.mdjcox.webfeedencoder.TuningThread listenForConnections INFO: Accepted connection
09.03.2010 10:20:45 ClientThread-25515818 uk.co.mdjcox.webfeedencoder.ConnectionThread process INFO: Command: GET_FILE_SIZE D:\rec\NatGeoWild-2174087-0.mpg

I've attached the encoder.log too.

Best regards,
Yuly

[michaeldjcox]

Originally Posted by scanner95

In a controlled test it worked as it should: VLC not present after the stop was pressed.

Ok well thanks encouraging. Let me know if you get further issues.

Originally Posted by scanner95

One strange thing, though, is that it complains about SageTV being dead all the time:

This is the encoder trying to talk to the WFE EPG plugin in the SageTV server.

It invites connections on port 1100.

Does the Sage "light" on the config tool show SageTV as up?

This may just be bad debug. At least it says SageTV has possibly died.

I get these messages too - I'll take a further look.

Still testing the EPG changes from earlier. Making a decision about whether just to release as part of 0.9.6 or if I can patch 0.9.5 with this change. Will decide tomorrow.

Michael

[scanner95]

Originally Posted by michaeldjcox

Ok well thanks encouraging. Let me know if you get further issues.



This is the encoder trying to talk to the WFE EPG plugin in the SageTV server.

It invites connections on port 1100.

Does the Sage "light" on the config tool show SageTV as up?

This may just be bad debug. At least it says SageTV has possibly died.

I get these messages too - I'll take a further look.

Still testing the EPG changes from earlier. Making a decision about whether just to release as part of 0.9.6 or if I can patch 0.9.5 with this change. Will decide tomorrow.

Michael

Sage light is always on - so I suppose it's just some timing issue in the debug.
Waiting for your decision about EPG: next big thing for me

Thank you,
Yuly

[n9cqs]

Originally Posted by michaeldjcox

Hi,

Which timeformat field is that i.e. where on the config GUI?

Michael

Michael,
Ooops... it is actually the date format that I am talking about appears as the drop-down box on the EPG harvest tab of the GUI. When selecting Radiotime, for instance, there are four that can be selected. It is the date that is placed at the end of the URL for scraping EPG from a website. I need yyyyMMdd, but when I manually type that in, save the configuration, and then go back again it is not there.

Thanks again.
Jeff

[n9cqs]

Michael,
I was playing a lot with WDR (WDR4 to be exact) last night. The website has good program information, but it does not match so well to what your EPG grabber is expecting (and the date format issue for the URL is one of those, mentioned in the previous post). I have a LAN webserver, and so wrote a PHP page that will translate the date from your EPG grabber to WDR, and also will then go through the data and put it into a format that your grabber works well with. My problem is that the times in the schedule are in MEZ (I think that 's the name of the zone... it is in Germany). When I set the harvester to read time in the "Europe/Berlin" time zone, it translates some of the times correctly and some not. Is there a log I can send you to try to figure out what is going on? The times on the schedule are all in 24 hour time hh.mm format. I have tried to capture the "0" hours and make them 12.00am and noon as 12.00pm with my PHP code... sometimes that works and sometimes not. Whatever I can post to let you figure out what the issue is I will try to do.
Thanks again for a great plug-in.
Jeff

[n9cqs]

Michael,

Sorry to bug you more... but I guess I am helping give things a good beta-test workout.

I am trying to capture live programs from WDR Fernsehen. They are sent in ASF format. When I capture them directly (using ReplayAV), they will play on Sage in that native format. When the transcoded show, recorded with SageTV is played on Sage, there is audio but no picture. When played on VLC directly, the transcoded Sage file does both picture and audio.

What can I send you to show you what is going on? I have my own website and I would be glad to upload the file to it and send you the link, but I would rather do that off-forum if possible.

Along that same line, I read with interest how to set up a "tvraw" set of commands that you mentioned to "Scanner" a few days ago, but I did not have a bit of luck with that. I tried it thinking that if the raw .asf files worked when recorded with another program, they would be okay not to have to translate in Sage. Maybe I'm not doing something right there, but did it just as you suggested.

Thanks again for your great plug-in.
Jeff

[michaeldjcox]

Originally Posted by scanner95

Sage light is always on - so I suppose it's just some timing issue in the debug.
Waiting for your decision about EPG: next big thing for me

Thank you,
Yuly

Hi

I will patch you with a new 0.9.5 soon.

0.9.6 is going to take too much work to be ready any time soon.

Michael

[michaeldjcox]

Originally Posted by n9cqs

Michael,
Ooops... it is actually the date format that I am talking about appears as the drop-down box on the EPG harvest tab of the GUI. When selecting Radiotime, for instance, there are four that can be selected. It is the date that is placed at the end of the URL for scraping EPG from a website. I need yyyyMMdd, but when I manually type that in, save the configuration, and then go back again it is not there.

Thanks again.
Jeff

Ok.

After you have typed in the format you want and pressed save.

Can you check in the channels/<youchannelcallsign>.props file and see if you change has been persisted.

It could just be that on reopening the edit dialog it is not set on the edtiable combo.

In which case it needs a fix but at least can be made to work.

Michael

[michaeldjcox]

Originally Posted by n9cqs

Michael,
I was playing a lot with WDR (WDR4 to be exact) last night. The website has good program information, but it does not match so well to what your EPG grabber is expecting (and the date format issue for the URL is one of those, mentioned in the previous post). I have a LAN webserver, and so wrote a PHP page that will translate the date from your EPG grabber to WDR, and also will then go through the data and put it into a format that your grabber works well with.

OK. I have tried to make the grabber universal. So if theres something it cannot do I want to fix it. We don't want to have any extra processing components if possible.

Originally Posted by n9cqs

My problem is that the times in the schedule are in MEZ (I think that 's the name of the zone... it is in Germany). When I set the harvester to read time in the "Europe/Berlin" time zone, it translates some of the times correctly and some not.

MEZ = Mitteleuropäische Zeit = CET Central European time = UTC + 1hr.

Europe/Berlin should be correct.

Perhaps the problem is the issue below.

Originally Posted by n9cqs

Is there a log I can send you to try to figure out what is going on?

If you are testing the grab from the configtool then send the configtool log file otherwise encoder.log contains the results of the periodic grab.

Originally Posted by n9cqs

The times on the schedule are all in 24 hour time hh.mm format. I have tried to capture the "0" hours and make them 12.00am and noon as 12.00pm with my PHP code... sometimes that works and sometimes not. Whatever I can post to let you figure out what the issue is I will try to do.
Thanks again for a great plug-in.
Jeff

The java date format string for 24 hours is "HH" and for 12 hour clock "hh". Be sure to use the right one. Some web sites list times in more than one format so be careful of that and add extra formats if you need it.

If you still have problems pls send the log file.

Also if possible send me the channel properties file.

Michael

[michaeldjcox]

Originally Posted by n9cqs

Michael,

Sorry to bug you more... but I guess I am helping give things a good beta-test workout.

I am trying to capture live programs from WDR Fernsehen. They are sent in ASF format. When I capture them directly (using ReplayAV), they will play on Sage in that native format. When the transcoded show, recorded with SageTV is played on Sage, there is audio but no picture. When played on VLC directly, the transcoded Sage file does both picture and audio.

VLC will play anything. SageTV is much more fussy - and differently fussy in different contexts. i.e. things play on MVPs but won't play on HD200, things that play on extenders won't play on PC clients, things that play on a SageTV client only context won't play if you put a SageTV server in the middle.

In my experience there is a difference between the sorts of things that can be watched live into SageTV and those that can be recorded to a file and then played.

With live watching I have only ever had success with mpeg2. I have tried mpeg4 and that didn't work.

I haven't tried any other formats.

Originally Posted by n9cqs

What can I send you to show you what is going on? I have my own website and I would be glad to upload the file to it and send you the link, but I would rather do that off-forum if possible.

This is really down to SageTV. Turning debug logging in SageTV server and/or client may help. My email address is in the contact details.

Originally Posted by n9cqs

Along that same line, I read with interest how to set up a "tvraw" set of commands that you mentioned to "Scanner" a few days ago, but I did not have a bit of luck with that. I tried it thinking that if the raw .asf files worked when recorded with another program, they would be okay not to have to translate in Sage. Maybe I'm not doing something right there, but did it just as you suggested.

Thanks again for your great plug-in.
Jeff

You probably need different params for ASF streams.

But I really doubt it will work. The TV raw works because it is MPEG2.

Michael

[michaeldjcox]

For scanner95 and n9cqs:

http://www.keepandshare.com/doc/1808...er-0-9-5a?da=y

features fixes for EPG grabbing day URL format and time zone issues plus grab from XMLTV file plus beefed up test facilities.

Shutdown webfeedencoder and config tool. Leave the webfeedencoder dir in any explorer windows or command shells. Download unzip, run the jar file and choose repair. Be careful if you have changed any of the 0.9.5 delivered command files or channel types. These will be overwritten - you may want to take a backup first.

Michael

[scanner95]

Great, thanks!

I will give you feedback later today.

Yuly

[scanner95]

Hello Michael,

Two questions:
1. What is the expected syntax for XMLTV file path? Will it accept local path or it has to be a URL for download? I've tried to enter local path (in a form of c:\...\xmltv.xml). Then, after clicking "Test EPG" and "Run" I see just a message "running EPG grabber for ..." and thats it. Nothing appears below that. Is this normal?
2. I assume the "EPG ID" is the "channel id" in the XML, is it? Does it have to match anything in the WFE (like Callsign), or is it independant from it?

Thanks,
Yuly

[michaeldjcox]

Originally Posted by scanner95

Hello Michael,

Two questions:
1. What is the expected syntax for XMLTV file path? Will it accept local path or it has to be a URL for download? I've tried to enter local path (in a form of c:\...\xmltv.xml). Then, after clicking "Test EPG" and "Run" I see just a message "running EPG grabber for ..." and thats it. Nothing appears below that. Is this normal?
2. I assume the "EPG ID" is the "channel id" in the XML, is it? Does it have to match anything in the WFE (like Callsign), or is it independant from it?

Thanks,
Yuly

Local path is what I tested with.

You might want to check thats correct.

and the EPG id is the channel id from the xmltv file e.g.

1

the config log should have some information on any failure.

Generally for me it took less than a minutes to show the shows for that channel

Michael

[scanner95]

Hello,

I've tried with two different XMLTV files. With one of them (a 12Mb file, which also contains cyrillic - the one I posted a link to (http://ymilner.myftp.org:1968/temp/xmltv.zip several posts ago) nothing works: I waited about 10 minutes and WFE unable to show any programs for a given channel.
With another XMLTV file (2MB, latin chars) I almost immediately see programs.
However, when I try to send the EPG to SageTV for lineup, I get a java heap space out of memory error (screenshot attached)