PrimeNetEncoder - HDHomeRun Prime Network Encoder

[nyplayer]

Originally Posted by jm9843

Is it necessary to recreate the properties file if coming form 1.2.2 (Beta)?

No I did not have to.

[jvl711]

Originally Posted by nyplayer

No I did not have to.

Correct.

Only when upgrading from 1.0.0 or 1.0.1.

[CptKirk]

Switched over to v1.2.4. Will test it out today.

[nyplayer]

I see more artficats on my recordings I did not see them with 1.0.1. Are you using a different muxer or something different before writing the files? Quality of the recordings are much better using 1.0.1 ...1.2.4 introduces artifacts and playback has some breakups in audio.

[jvl711]

Originally Posted by nyplayer

I see more artficats on my recordings than I did with 1.0.1. Are you using a different muxer or something different before writing the files? Quality of the recordings are much better using 1.0.1 ...1.2.4 introduces artifacts and playback has some breakups in audio.

I rearchitected a lot of how the transfer of data is happening. It use to look like this.

HDHomeRun(UDP) -> FFMPEG(CIFS)

This worked well, but was problematic because I could not really monitor the remuxing. Sometimes the HDHomeRun does not respond to the commands, and no data is produced.

So now the new method looks like this. I have a lot more control over the process. Now I just need to figure out where issues might be cropping up. My assumption is that maybe packets are getting dropped in the UDP transfer. I increased the thread priority for that part of the process hoping that the artifacts would get cleared up.

HDHomeRun(UDP) -> PrimeNetEncoder -> (STDIN)ffmpeg(STDOUT) -> PrimeNetEncoder(TCP) -> SageMediaServer.

Is there any specific times that you noticed more artifacts?

[nyplayer]

Originally Posted by jvl711

I rearchitected a lot of how the transfer of data is happening. It use to look like this.

HDHomeRun(UDP) -> FFMPEG(CIFS)

This worked well, but was problematic because I could not really monitor the remuxing. Sometimes the HDHomeRun does not respond to the commands, and no data is produced.

So now the new method looks like this. I have a lot more control over the process. Now I just need to figure out where issues might be cropping up. My assumption is that maybe packets are getting dropped in the UDP transfer. I increased the thread priority for that part of the process hoping that the artifacts would get cleared up.

HDHomeRun(UDP) -> PrimeNetEncoder -> (STDIN)ffmpeg(STDOUT) -> PrimeNetEncoder(TCP) -> SageMediaServer.

Is there any specific times that you noticed more artifacts?

Watching livetv mostly is when I noticed it ... throughout at different times... usually you notice it right away when livetv starts .... on the 1.0.1 when livetv starts no artifacts at all or throughout.


I started a recording using 1.0.1 it was clear no artifacts .... I stopped the service reloaded 1.2.4 continued the recording part 0.ts had no artifacts part 1.ts had artifacts.

[jvl711]

Originally Posted by nyplayer

Watching livetv mostly is when I noticed it ... throughout at different times... usually you notice it right away when livetv starts .... on the 1.0.1 when livetv starts no artifacts at all or throughout.


I started a recording using 1.0.1 it was clear no artifacts .... I stopped the service reloaded 1.2.4 continued the recording part 0.ts had no artifacts part 1.ts had artifacts.

The artifacts in the beginning may be due to me reading the stream sooner after the channel change than before.I am not 100% sure.

About how many artifacts in 30 mins not including the initial startup artifact do you think you are seeing? I am going to attempt to increase the buffer for the STDIN to ffmpeg. I wonder if I am dropping packets because ffmpeg is not ready to receive my packets.

I am going to test that change tonight and possibly push out another beta tomorrow.

[nyplayer]

Originally Posted by jvl711

The artifacts in the beginning may be due to me reading the stream sooner after the channel change than before.I am not 100% sure.

About how many artifacts in 30 mins not including the initial startup artifact do you think you are seeing? I am going to attempt to increase the buffer for the STDIN to ffmpeg. I wonder if I am dropping packets because ffmpeg is not ready to receive my packets.

I am going to test that change tonight and possibly push out another beta tomorrow.

I see it a lot throughout ivetv... especially during fast moving action after a while it is annoying especially watching fast moving sports. Watching espn highlights you really see them.

[jvl711]

Originally Posted by nyplayer

I see it a lot throughout ivetv... especially during fast moving action after a while it is annoying especially watching fast moving sports. Watching espn highlights you really see them.

So you are saying just straight up MPEG type artifacts... Like the bitrate is too low in the encoding? Not breakup of the audio/video where there is skipping/hissing/popping????

[nyplayer]

Originally Posted by jvl711

So you are saying just straight up MPEG type artifacts... Like the bitrate is too low in the encoding? Not breakup of the audio/video where there is skipping/hissing/popping????

Correct.. I am sure others will be seeing the same thing I tested on 2 machines. Same video artifacts.

[jvl711]

Originally Posted by nyplayer

Correct.. I am sure others will be seeing the same thing I tested on 2 machines. Same video artifacts.

Can you try setting ffmpeg.analyzedurration and ffmpeg.probesize to the defaults (5000000). Also can you double check and make sure the transcoding did not get enabled by accident in the properties files.

None of the changes I made should be causing those types of motion artifacts. I originally though you were talking about hiss/pop/green pixel type issues.

[nyplayer]

Originally Posted by jvl711

Can you try setting ffmpeg.analyzedurration and ffmpeg.probesize to the defaults (5000000). Also can you double check and make sure the transcoding did not get enabled by accident in the properties files.

None of the changes I made should be causing those types of motion artifacts. I originally though you were talking about hiss/pop/green pixel type issues.

I have tried different probesizes transcoding is not enabled ... I have tested this all kinds of ways. I tried different ffmpegs anything to get the quality the same as the 1.0.1 but it simply has artifacts ... like I did when I was using sagedct.

I will try and get screesnshots.

[nyplayer]

Don't go crazy on my account wait to see if others are reporting it. I am resetting everything to default rebooting my primes rebooting my server from scratch will let you know.

Sorry I set everything to default same thing even worst with defaults get breakups including sound ... I will have to backout and use 1.0.1 for now.

This is using Windows 10.

[jvl711]

Originally Posted by nyplayer

Don't go crazy on my account wait to see if others are reporting it. I am resetting everything to default rebooting my primes rebooting my server from scratch will let you know.

Sorry I set everything to default same thing even worst with defaults get breakups including sound ... I will have to backout and use 1.0.1 for now.

This is using Windows 10.

Thats why we are in beta to try and figure these things out. I will test the buffering changes tonight and send them out for testing if they seem to improve things.

[jvl711]

Originally Posted by nyplayer

Don't go crazy on my account wait to see if others are reporting it. I am resetting everything to default rebooting my primes rebooting my server from scratch will let you know.

Sorry I set everything to default same thing even worst with defaults get breakups including sound ... I will have to backout and use 1.0.1 for now.

This is using Windows 10.

nyplayer,

Would you mind PMing me your email so that I can send you one off JARs to test. I have made some minor changes to the buffering that might help you. I am not seeing the persistent pixelation on my side, so it might be a little hard to track down.

Also since it is windows 10... Are you sure the adaptive networking stuff is turned off.

[EnterNoEscape]

I'm building my own network encoder for the InfiniTV in java because I don't want it to be tied to Windows and I wanted to implement some features that may never make it into SageDCT. It might even be able to tune a Prime since it's using uPnP to talk to the device.

I'm very excited about the work you're doing on enabling stream remuxing/transcoding via the upload id.

I have all of the uPnP tuning bits working very well. I have most of the SageTV communication bits implemented. I'm now working on the RTSP handling. So I wanted to ask a few questions before I waste time going the wrong way.

If you don't mind me asking, how did you implement the RTSP protocol? I've found some java based libraries online and was wondering if you already played with anything. Do you think Java isn't going to be fast enough to keep up with the UDP traffic?

[jvl711]

Originally Posted by EnterNoEscape

I'm building my own network encoder for the InfiniTV in java because I don't want it to be tied to Windows and I wanted to implement some features that may never make it into SageDCT. It might even be able to tune a Prime since it's using uPnP to talk to the device.

I'm very excited about the work you're doing on enabling stream remuxing/transcoding via the upload id.

I have all of the uPnP tuning bits working very well. I have most of the SageTV communication bits implemented. I'm now working on the RTSP handling. So I wanted to ask a few questions before I waste time going the wrong way.

If you don't mind me asking, how did you implement the RTSP protocol? I've found some java based libraries online and was wondering if you already played with anything. Do you think Java isn't going to be fast enough to keep up with the UDP traffic?

Hi EnterNoEscape,

I am actually using straight/raw udp streaming from the HDHomeRun. I just pick up the stream and pass it on to FFmpeg for stream fix. This was very easy to implement. I did use a separate thread to handle these transfers, and set it's priority to MAX. I do believe java is fast enough. I have had over 6 streams being recorded through my app using UDP and have seen no issues.

I have seen occasional blips in data transfer (1-2 time and hour), but I am not sure where it is coming from. I have written a bunch of code to log when these issues happen, but at this point I am not sure if it is coming from Comcast, HDHomeRun, PrimeNetEncode, FFmpeg or SageTV MediaServer. The logging is not really pointing a finger anywhere specific at this time.

I hope this is helpful. I think I have the SageTV protocols down pretty well is you need help with that aspect.

Josh

[EnterNoEscape]

I guess hdhomerun_config.exe is setting the stream of UDP packets to your computer IP address. The fact that it's UDP is enough to make me think those occasional missed packets might be network issues or congestion. I have seen small glitches myself using SageDCT when moving large files to and from my primary SageTV server. I was thinking about creating a subnet just for recording devices so I can try to guarantee bandwidth for my tuners.

On the InfinitTV devices as far as I have been able to determine, you can't set the IP address for the stream via upnp. You have to either connect to rtsp://x.x.x.x:8554/cetonmpeg[0-5] (you retrieve the URL via an action on one of the services on one of the embedded devices) and tell it to stream to you and what port to stream to. The alternative is to set the IP address by using POST on the web interface which I'm avoiding since I want to do this as proper as possible. After looking at a few implementations of this protocol, I think I can just write my own. It doesn't appear to be that complicated.

As for the SageTV protocols; I was following the "blueprint" in EncodingServer.java. I want to implement auto discovery. At first I was a little puzzled with PROPERTIES. I think this was mostly because this is actually my first time programming in java (incredibly similar to C# and even easier to use in some regards). So as I understand it java.util.Properties can directly import the text in .properties files and it can export in either XML or in the linear format that SageTV uses by default. So all this is returning in the first line is the number of properties, then the next lines are literally the properties as they would appear in the .properties file. Honestly the comment in the code only helped to confuse me. Do I have that correct? Also I'm unclear on device_class and encoding_host. In the for loop, they are excluded as expected based on the comment, but I don't see where they are modified and added to the returned data. These properties are not inferred and do need to be set to "NetworkEncoder" and "ip:port" respectively correct?

I was going to just look at the code behind these other commands, but maybe you already know what they are looking for. I do want to implement channel scanning so I can do a scan that will actually detect CopyOnce channels and tell SageTV they don't have a signal. I think this could make initial setup easier for many people.
GET_START - Is this just the starting size of the recording?
AUTOTUNE - Channel scanning?
AUTOSCAN - Channel scanning?
AUTOINFOSCAN - What are normal return values for this command?

Thanks a bunch for the help! I feel like I'm on the right track here (I've only been working on this for about a week) despite being very ambitious. I really got my motivation back when SageTV was open sourced and I really want to contribute to the reason SageTV is so great.

Joe

[weeber]

I've noticed the same macroblocking artifacts, usually with an audio "pop" or other distortion occurring with it. Like NYPlayer, everything was fine with 1.01, but I started noticing errors with the newest betas. However, the artifacting for me is pretty rare, and I appreciate the fast tuning a lot, so I've just been living with it. He's right, the errors are reminiscent of my time with SageDCT, though not as bad or as frequent.

I did increase probesize slightly to 3000000 (If I remember right), but it didn't seem to do much. Since I didn't want to give up the fast tuning, I didn't bother with increasing values further.

I'm on Comcast with my Prime and HTPC plugged into the same network switch.

Originally Posted by nyplayer

I have tried different probesizes transcoding is not enabled ... I have tested this all kinds of ways. I tried different ffmpegs anything to get the quality the same as the 1.0.1 but it simply has artifacts ... like I did when I was using sagedct.

I will try and get screesnshots.

[jvl711]

Originally Posted by weeber

I've noticed the same macroblocking artifacts, usually with an audio "pop" or other distortion occurring with it. Like NYPlayer, everything was fine with 1.01, but I started noticing errors with the newest betas. However, the artifacting for me is pretty rare, and I appreciate the fast tuning a lot, so I've just been living with it. He's right, the errors are reminiscent of my time with SageDCT, though not as bad or as frequent.

I did increase probesize slightly to 3000000 (If I remember right), but it didn't seem to do much. Since I didn't want to give up the fast tuning, I didn't bother with increasing values further.

I'm on Comcast with my Prime and HTPC plugged into the same network switch.

I have a new version in testing right now that I hope will help. I added some more buffering on the streams to try and level out any minor delays. I expect to have is out in a few days.

Josh