Utility: Web Radio and TV Encoder

[scanner95]

Michael,

The original null pointer error is gone, since I've removed and reinstalled WFE several times since then.
In Sagetv WFE EPG lineup is WebFeedEncoder
I've disabled and enabled the channel (there is just one now in the lineup).Then I went to WFE configtool and uploaded EPG data again. Then, from Nielm's web sever, I forced EPG update. Still, no EPG.

[scanner95]

I went ahead and again reinstalled WFE from scratch. This time I first choose basic EPG and uploaded my single channel.
From WFE all is fine again. But now it does not play at all from Sage. I noticed VLC is not run at all, and the recording file is not incrementing in size.
I've attached encoder.log

Best regards,
yuly

[michaeldjcox]

Hi,

Glad you're back to where you were.

In general there should be no need to reinstall WFE - it is unlikely to fix very much.

I can explain the reason why recording initiated from sageTV isn't working. I left some test code in as well as my line of debug. D'oh.

So now we are back to running a test and sending log file with this new version:


WebFeedEncoder 0.9.5a

With slightly more precise instructions:

1. Shutdown config tool, web feed encoder service, SageTV, leave the web feed encoder if your are viewing that dir or sub directory in a command tool, or explorer window, close any open files in those directories.
2. Unzip and run the jar file in an elevated command prompt.
3. Choose "Repair"
4. Perform all steps.

Michael

[scanner95]

Hello Michael,

Here is what happens now.
I repaired the installation as per your instructions. Currently there are 3 channels in the lineup (some day we make it work I'll ask you how to put back all my channels I've saved in a backup, before uninstalling the WFE )

Out of those 3 channels, 2 have XMLTV file -type of EPG, and one has basic EPG. From inside the configtool, I can test all channels and EPG working fine. Still, no matter what I do (remove & resend lineup to Sage, sent EPG to Sage, etc.), Sage does not see programs from XMLTV file. Basic EPG (with channel name all over the timescale) works fine, but no XMLTV file.
This is first problem; I hope we'll find out what happened there.

Now, to the fun stuff. I tried to play a channel from inside Sage; it played fine. The EPG for this channel was basic, the name of the channel was English, the recording file was English too, and Sage did not have problem with that. Since I could not make Russian program name to appear in Sage (due to 1st problem), I renamed the channel to russian (Кинохит), tried to hit play again. And here the problem was again: file size 0 (named "Кинохит-2426753-0.mpg"), black screen.

I'm attaching all WFE logs, also the one from Sage directory, and sage.properties too - maybe there is some hint as to why EPG does not import.

Best regards,
Yuly

[michaeldjcox]

Hi - online on google talk now if you want to chat - invite with you.

Just to check you are using the "repair" button on the installer to apply these patches rather than uninstall and reinstall yeah?

You can restore any channels by copying the channels/*.props files from the backup into the channels directory on the installation. If the encoder is running you can use the configtool File->reload channels option to load them.

On the XMLTV - we had this working did we not? Have my more recent debug versions broken this? Can you send me some more recent xmltv file.

On the other issue I see that SageTV is talking to WFE cp1251 encoding which includes facility for cryllic chars. But in the log they still appear as ?????. That should not happen. Do you also see question marks in the encoder log when you test the channel? (Use notepad to view the log)

Michael

Originally Posted by scanner95

Hello Michael,

Here is what happens now.
I repaired the installation as per your instructions. Currently there are 3 channels in the lineup (some day we make it work I'll ask you how to put back all my channels I've saved in a backup, before uninstalling the WFE )

Out of those 3 channels, 2 have XMLTV file -type of EPG, and one has basic EPG. From inside the configtool, I can test all channels and EPG working fine. Still, no matter what I do (remove & resend lineup to Sage, sent EPG to Sage, etc.), Sage does not see programs from XMLTV file. Basic EPG (with channel name all over the timescale) works fine, but no XMLTV file.
This is first problem; I hope we'll find out what happened there.

Now, to the fun stuff. I tried to play a channel from inside Sage; it played fine. The EPG for this channel was basic, the name of the channel was English, the recording file was English too, and Sage did not have problem with that. Since I could not make Russian program name to appear in Sage (due to 1st problem), I renamed the channel to russian (Кинохит), tried to hit play again. And here the problem was again: file size 0 (named "Кинохит-2426753-0.mpg"), black screen.

I'm attaching all WFE logs, also the one from Sage directory, and sage.properties too - maybe there is some hint as to why EPG does not import.

Best regards,
Yuly

[michaeldjcox]

Originally Posted by scanner95

Hello Michael,

Here is what happens now.
I repaired the installation as per your instructions. Currently there are 3 channels in the lineup (some day we make it work I'll ask you how to put back all my channels I've saved in a backup, before uninstalling the WFE )

Out of those 3 channels, 2 have XMLTV file -type of EPG, and one has basic EPG. From inside the configtool, I can test all channels and EPG working fine. Still, no matter what I do (remove & resend lineup to Sage, sent EPG to Sage, etc.), Sage does not see programs from XMLTV file. Basic EPG (with channel name all over the timescale) works fine, but no XMLTV file.
This is first problem; I hope we'll find out what happened there.

Now, to the fun stuff. I tried to play a channel from inside Sage; it played fine. The EPG for this channel was basic, the name of the channel was English, the recording file was English too, and Sage did not have problem with that. Since I could not make Russian program name to appear in Sage (due to 1st problem), I renamed the channel to russian (Кинохит), tried to hit play again. And here the problem was again: file size 0 (named "Кинохит-2426753-0.mpg"), black screen.

I'm attaching all WFE logs, also the one from Sage directory, and sage.properties too - maybe there is some hint as to why EPG does not import.

Best regards,
Yuly

Hi,

So in private chat we fixed the EPG issue - there was no recent EPG data to appear in the schedule.

On the other issue of the failure to record to files names with non-latin characters we established that the cryllic characters where disappearing in the SageTVs communication with the network encoder.

I submitted a support request to SageTV to ask the character encoding SageTV uses for this communication and got this response:

It's using ISO-8859 encoding; which is not Unicode compatible unfortunately.

So what you should do is set this back to the
default:

allow_unicode_characters_in_generated_filenames=false

so then it won't use them in the filenames.

So the upshot is that you must shutdown SageTV, change the above property, and restart SageTV.

I'm sure I can probably figure out some other way and I will also ask SageTV if the encoding driven by a property so we can change it.

Michael

[scanner95]

Great news! So there is workaround after all! Thanks!

I briefly tried in a small setup with this setings, and it does work.
Next I'm trying to bring back my channels and lineup, and here is where I get this java exception again:

25.03.2010 12:26:19 AWT-EventQueue-0 uk.co.mdjcox.gui.actions.BaseAction handleException SEVERE: send lineup handle exception
java.lang.NullPointerException
at uk.co.mdjcox.epgplugin.EpgPlugin.updateLineup(EpgPlugin.java:399)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
at sun.rmi.server.UnicastRef.invoke(Unknown Source)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(Unknown Source)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(Unknown Source)
at $Proxy1.updateLineup(Unknown Source)
at uk.co.mdjcox.epgplugin.EpgPluginClient.updateLineup(EpgPluginClient.java:41)
at uk.co.mdjcox.config.model.ConfigModel.sendLineupToSage(ConfigModel.java:365)
at uk.co.mdjcox.config.model.ConfigModel.sendLineupToSage(ConfigModel.java:344)
at uk.co.mdjcox.config.actions.SendLineupAction.performControllerAction(SendLineupAction.java:97)
at uk.co.mdjcox.gui.actions.BaseAction$2.doInBackground(BaseAction.java:226)
at uk.co.mdjcox.gui.actions.BaseAction$2.doInBackground(BaseAction.java:220)
at javax.swing.SwingWorker$1.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at javax.swing.SwingWorker.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

I've attached a configtool log. Of course, I can enter my channels again one by one, but this would be quite a long task...

[michaeldjcox]

Originally Posted by scanner95

Great news! So there is workaround after all! Thanks!

I briefly tried in a small setup with this setings, and it does work.
Next I'm trying to bring back my channels and lineup, and here is where I get this java exception again:

I've attached a configtool log. Of course, I can enter my channels again one by one, but this would be quite a long task...

First congratulations on your promotion to "SageTV advanced user"!

It looks like there is something odd with your lineup.

Please send me your lineup.props config file I think there could be a missing or broken entry.

Also I suggest perhaps you restart everything - SageTV, configtool and encoder.

Michael

[scanner95]

Here is my lineup attached. I indeed noticed that if I remove the lineup and start creating it from the complete restored list of channels from scratch, the error does not happen. maybe thats what I'll do after all.

Yuly

[michaeldjcox]

Originally Posted by scanner95

Here is my lineup attached. I indeed noticed that if I remove the lineup and start creating it from the complete restored list of channels from scratch, the error does not happen. maybe thats what I'll do after all.

Yuly

Lineup looks fine. Impressive in fact.

Can you check there is a properties file in the channels directory for each channel in the lineup you have. The name of the properties file must agree with the id in the lineup.

I worry about channel 38 with the callsign with "(" and ")". I suggest renaming that to get rid of the brackets. Likewise the call sign with a "+" may cause problems

Michael

[scanner95]

Originally Posted by michaeldjcox

Lineup looks fine. Impressive in fact.

Can you check there is a properties file in the channels directory for each channel in the lineup you have. The name of the properties file must agree with the id in the lineup.

I worry about channel 38 with the callsign with "(" and ")". I suggest renaming that to get rid of the brackets. Likewise the call sign with a "+" may cause problems

Michael

As updated in a private message, I've ended up populating a new lineup, so can't really tell what was the problem with the old one.
Still being hit from time to time by VLC not quiting properly; either when working as encoder for Sage, or even after testing channel from inside VLC.
Otherwise, it seems that the core functionality is there, and working quite nicely. Impressive!
Right now for me the main issues are (in the order of priority):
1. Tuning speed. I know this is listed in the release notes as an open issue, but my WAF will be very low if it's left as it is . Tuning for me can take any time from 10 to 40 seconds. And since there is no indication something is happening, you are left to guess what is going on.
2. VLC issue from above. Did not test it properly, but I wonder what will happen if VLC is left there writing (and overwriting) to disk a channel, and WFE tries to tune to a next channel.
3. Configuration. With my 100+ channels, this takes huge amount of time. This is probably very specific to me, though. Maybe there could be a way to import all channels (and, possibly, lineup) from, say, excel spreadsheet.
4. Lineup sorting. Unless I missed something, currently you move a channel up or down one by one. What would be better, in my view, is to specify destination channel number when adding or moving a channel, and the rest will be sorted accordingly.
5. Already mentioned before couple of times: multiple encoder instances (sources in Sage), so we could record several timed programs, and also watch on multiple clients/extenders.

I could probably come up with some other thoughts later on, after I had more experience with this outstanding plugin.

Thanks a lot,
Yuly

[michaeldjcox]

Originally Posted by scanner95

As updated in a private message, I've ended up populating a new lineup, so can't really tell what was the problem with the old one.
Still being hit from time to time by VLC not quiting properly; either when working as encoder for Sage, or even after testing channel from inside VLC.
Otherwise, it seems that the core functionality is there, and working quite nicely. Impressive!
Right now for me the main issues are (in the order of priority):
1. Tuning speed. I know this is listed in the release notes as an open issue, but my WAF will be very low if it's left as it is . Tuning for me can take any time from 10 to 40 seconds. And since there is no indication something is happening, you are left to guess what is going on.
2. VLC issue from above. Did not test it properly, but I wonder what will happen if VLC is left there writing (and overwriting) to disk a channel, and WFE tries to tune to a next channel.
3. Configuration. With my 100+ channels, this takes huge amount of time. This is probably very specific to me, though. Maybe there could be a way to import all channels (and, possibly, lineup) from, say, excel spreadsheet.
4. Lineup sorting. Unless I missed something, currently you move a channel up or down one by one. What would be better, in my view, is to specify destination channel number when adding or moving a channel, and the rest will be sorted accordingly.
5. Already mentioned before couple of times: multiple encoder instances (sources in Sage), so we could record several timed programs, and also watch on multiple clients/extenders.

I could probably come up with some other thoughts later on, after I had more experience with this outstanding plugin.

Thanks a lot,
Yuly

1. Tuning speed. This is a problem and when I have looked at this before the major part of the time seems to be with SageTV starting to play the file on an extender. If you watch the recording file in your recordings directory you we see it grows but some time elapses before SageTV plays it.

2. Killing VLC always a problem. I am planning to do " kill". Although in windows this may not always work as only professional version of windows seem to have tskill.

3. Bulk setup. Yes I can imagine that took time. Will have think about that one.

4. Hmmmm maybe :-)

5. Yes. Been on the jobs list a while this one.

Michael

[scanner95]

On number 1: maybe there is a point in contacting Sage? I'm not a developer, but when Sage works with a hardware tuner (I have a PVR-150), the startup time is pretty much consistent and is about 2-4 seconds. And in general it does not take almost any time to Sage to start playing an imported video. So this must lay in a way WFE interacts with Sage (I've seen a term "network encoder" somewhere; probably this is what WFE is to Sage; pardon my being lame).
On number 2: if you can't beat them, join them . Is there (and I'm again into smth I don't really understand) a point in keeping a VLC instance running, and just issue it different set of parameters each time channel is changed?
Best regards,
Yuly

[michaeldjcox]

Originally Posted by scanner95

On number 1: maybe there is a point in contacting Sage? I'm not a developer, but when Sage works with a hardware tuner (I have a PVR-150), the startup time is pretty much consistent and is about 2-4 seconds. And in general it does not take almost any time to Sage to start playing an imported video. So this must lay in a way WFE interacts with Sage (I've seen a term "network encoder" somewhere; probably this is what WFE is to Sage; pardon my being lame).
On number 2: if you can't beat them, join them . Is there (and I'm again into smth I don't really understand) a point in keeping a VLC instance running, and just issue it different set of parameters each time channel is changed?
Best regards,
Yuly

1. WFE is a network encoder to SageTV. There is something very slow about the way SageTV responds to network encoder. It says "START filename" to the encoder and then keeps asking "How big is the file now?". I have tried lying at this point and claiming the file is bigger than it really is but it did not do much good. I suspect there is a some configurable property in SageTV related to this issue.

2. Well good point but if the VLC session is refusing to listen to me telling it to "stop that and quit" it probably won't listen to me saying "stop that and now do something else"

Michael

[scanner95]

Originally Posted by michaeldjcox

1. WFE is a network encoder to SageTV. There is something very slow about the way SageTV responds to network encoder. It says "START filename" to the encoder and then keeps asking "How big is the file now?". I have tried lying at this point and claiming the file is bigger than it really is but it did not do much good. I suspect there is a some configurable property in SageTV related to this issue.

Yea, I found this thread of yours:
http://forums.sagetv.com/forums/showthread.php?t=40736

Pitty it did not work out. Any point in bugging support for it?

[Jason]

Originally Posted by michaeldjcox

2. Killing VLC always a problem. I am planning to do " kill". Although in windows this may not always work as only professional version of windows seem to have tskill.

I was under the assumption that both the Home and Professional version of Windows XP come with the tool Tskill . In addition, XP Professional has the more powerful tool Taskkill. I am using XP since it appears to be the lowet common denominator for Win systems...

If all else fails and there is no support in a specific version, why not simply add in PsKill to the package?

http://technet.microsoft.com/en-us/s.../bb896683.aspx

-Jason

[michaeldjcox]

Originally Posted by Jason

I was under the assumption that both the Home and Professional version of Windows XP come with the tool Tskill . In addition, XP Professional has the more powerful tool Taskkill. I am using XP since it appears to be the lowet common denominator for Win systems...

If all else fails and there is no support in a specific version, why not simply add in PsKill to the package?

http://technet.microsoft.com/en-us/s.../bb896683.aspx

-Jason

Hah! Yes thanks - downloaded that this morning.

Don't like the way it pops up an accept/decline T&Cs dialog on first run but hey ho.

Michael

[scanner95]

Originally Posted by michaeldjcox

1. WFE is a network encoder to SageTV. There is something very slow about the way SageTV responds to network encoder. It says "START filename" to the encoder and then keeps asking "How big is the file now?". I have tried lying at this point and claiming the file is bigger than it really is but it did not do much good. I suspect there is a some configurable property in SageTV related to this issue.

Michael

I jumped on this possibility and did some searching in the forum. Might be I've found an answer. In this thread, post 66 I discovered a property called

videoframe/network_encoding_to_playback_delay

I tried setting it to some low value (10 at first, then 0), and my initial impression is that the tuning now is instantaneous! I'm on placeshifter right now; I have to get back home to verify that. In case it works, we have to determine a safe value to use; probably some debugging is necessary to understand how exactly it works.

Here is yet another place where this, and some other parameters are mentioned.

[scanner95]

Guess what? It's in the manual on page 253:

videoframe/network_encoding_to_playback_delay
the amount of time SageTV should wait after starting recording of a file to start playing it
back if the capture device is a network encoder

[michaeldjcox]

Originally Posted by scanner95

Guess what? It's in the manual on page 253:

Top man!

Cannot wait to give that a try at home.

Michael