Network encoder for DBOX 2 DVB set top boxes...

[potpiejimmy]

Hmm yeah, I have never been all happy with the ggrab solution myself. I've also had ggrab stop streaming here and there, especially for long shows. Biggest problem is with shows that have a high data rate of > 9MBit. TCP streaming just wastes too much of the bandwidth and UDP streaming is not reliable enough with ggrab. That's why I have picture distortions for either UDP and TCP for those shows.

Udrec on the other hand records everything just fine, that's why I'm going to include udrec with my grabrecorder. It will use udrec when using the -udp switch and ggrab when not using -udp - that way people can choose what works best for them.

I'm currently working out the problems with the udrec TS files with a Sage developer... we're almost there. Just a few more problems to solve...

I don't have Sage playback errors, though. It's probably related to your unc mapped drive, like you said.

[Alfiegerner]

That's great news mate, thanks for keeping us informed.

[Alfiegerner]

Hey potpiejimmy, my dreambox has arrived and its pretty cool The problem is though that so far on the neutrino images I've tried with it neither ggrab or udrec work with it I will do some more investigation to see why udrec doesn't like it. Seeing as it uses a neutrino image I'm hoping I should be able to get it to work with some tweaking here and there.

One thing that does seem to work well with it and with a dbox too is wingrab (wingrabz for neutrino images and wingrabe for enigma images). You may already be familiar with it and the site i'm about to link to - but just in case ...

http://www.dbox2.info/download.php?catid=5#2.

So far has worked very well in my trials, although I haven't yet found a way to be able to control it through command line - I haven't found any docs or code for it yet . On that page there also other software (ngrab etc) that might be worth looking at.

[Alfiegerner]

Hi m8,

I've found some switches to control video playback on the dbox. Apologies if you are already aware of them. I have found turning off the video playback whilst recording improves streaming for me. Could be useful ...

control/zapto?startplayback
control/zapto?stopplayback
control/zapto?statusplayback

[sinin]

Originally Posted by potpiejimmy

Actually udrec was my first choice. It would be great to use for streaming to TS files. I had it running and the result was Sage hanging using 99% cpu time. I took a while for me to find out that it's the missing PMT information in the TS files that Sage (and other media players, VLC for instance) didn't like. I haven't found a solution for it yet, and since udrec also doesn't multiplex mpegs, I had to switch to ggrab...

Hi Potpie.

I'm new on this board and was lead here in my quest to find a single media centre style application that can act a a central point for all media duties including watching and recording TV, AND using my DBox as a source for this job (although I would like to get a DVB-C card as well in the future). So far I'm looking at GBPVR, SageTV or MythTV, but I'm still along way for even setting up the home network to do this so this is still very much a "look-see" time for me.

Although I had the DBox for a while I'm only now just getting into recording and streaming. I experienced one of the issues I think you describe above after direct recording a program from the DBox to the PC that produces a TS file. VLC won’t play the file due to some incorrect header information.

However, on one of the other forums I regularly visit, it seems to me that this issue may now have been fixed and the fixes have been applied to the DBOX image CVS file so that new images create will not have this problem.

I don’t know if this is the same issue as you are experiencing so forgive my ignorance if it is not.

Originally Posted by PT-1 (from unlockers)

Problem playing TS Files with VLC

--------------------------------------------------------------------------------

Are you having Problems playing TS Files with VLC ?

Try this:
http://lvempeg.sourceforge.net/test/genpsi.exe or attached rar file to "repair" your TS files that you already have.

From now on it should work with all new Images (>CVS 16.01.2006).

The Issue was that metallica included neutrino (stream2file.cpp)
about 6 months ago to fix this issue but there was still belows fault in the routine so gmo18t fixed this issue today....

The guys at the Tuxboard included a new patch in the cvs (today) as the


Quote:
//-- PMT packet for at least 1 video and 1 audio stream --
//--------------------------------------------------------
static uint8_t pkt_pmt[] =
{
0x47, 0x4F, 0xFF, 0x10, 0x00, // HEADER-1
0x02, 0xB0, 0x17, // HEADER-2
0x04, 0x37, 0xE9, 0x00, 0x00, // HEADER-3
0xE0, 0x00, 0xF0, 0x00, // PMT-DATA
0x02, 0xE0, 0x00, 0xF0, 0x00, // (video stream 1)
0x03, 0xE0, 0x00, 0xF0, 0x00 // (audio stream 1)
};



should be


Code:
...
0x6D, 0x66, 0xE9, 0x00, 0x00, // HEADER-3
...


USAGE FOR GENPSI.EXE

Dos Window

C:\temp>genpsi.exe

usage: genpsi <tsfile-to-patch>

EDIT:
Please read here some interesting Add On for DirectDraw Players

[potpiejimmy]

Hi everybody,

sorry, I've been kinda busy in the last couple of weeks...

I just uploaded version 0.7 of grabrecorder (see my original posting). The good news: TS-recording with udrec works now (with Sage 4.1.7). The bad news: Sage doesn't like all programs/channels and pops up a playback error or hangs forever. There's also a problem with Sage when having the PAT/PMT included in the recorded files, that's why I added the -patpmt command line option to switch PAT/PMT on and off.

This leads us to three different types of recording, each of them having their pros and cons:

1) Recording without -udp (that is using ggrab)

+ works fine with all channels
+ dbox doesn't have to be in SPTS mode
+ playback starts almost immediately without problems for all recordings
- recording is not reliable, stops from time to time in the middle of the show
- sometimes ggrab hangs at 99% cpu
- glitches in the picture/problems with high data rates because TCP is used

2) Recording with -udp (udrec) AND -patpmt

+ most reliable recording using UDP and TS
+ no video glitches, no problems with high data rates
+ files are complete with PAT/PMT, playback in VideoLan player works great, even automatically detects the aspect ratio, the resulting files are just perfect
- dbox must be switched to SPTS mode
- Sage doesn't play most of the files and shows playback error because it doesn't like the PAT/PMT

3) Recording with -udp (udrec) but without -patpmt

+ most reliable recording using UDP and TS
+ no video glitches, no problems with high data rates
+ Sage plays most of the files, but takes its time to scan the files
- dbox must be switched to SPTS mode
- the resulting files lack PAT and PMT which makes them kind of incomplete, can't play them in VideoLan player, WMP works, however, most of the time
- playback doesn't start immediately and Sage hangs for minutes if the file is not long enough yet.



@sinin
Hi, thanks for your post. The trick was to pass the PAT (=0) and PMT pids to udrec via -ap. I'm not sure whether this is due to the patch you mentioned (I have the latest snapshot image installed) or not, but the files play perfectly in VLC now. Sage has problems, however, see above.


@alfiegerner
Sorry, almost forgot about your startplayback/stopplayback idea, could implement a command line option that invokes stopplayback when recording starts. On the other hand, stopping the playback is basically the same as hitting the standby-button on your dbox remote and since I usually have my dbox in standby while recording I didn't have the need for such an option. Don't you have your dbox in standby when recording?

Oh, any news on getting your dreambox to work with grabrecorder??

[Alfiegerner]

Brilliant, thanks for the release!

I didn't realise the dbox could record when in standby. That's a good tip.

I haven't yet found a way to use the dreambox with grabrecorder but I've been too busy lately to really play around.

I'll try this out later and let you know how I get on.

Is there anything I can help out with code-wise? Let me know if i can assist ...

[Alfiegerner]

Hey potpie, I've just had an idea regarding direct file recording. This is all the product idle speculation so don't worry if its a crap idea .

If you remember when we discussed direct file recording last we ran into the stumbling block of getting the network encoder to record to a particlar file name for sage to play:

Originally Posted by potpiejimmy

1) With direct file recording, the file is opened by the dbox, but the recorded must be redirected to a file with a specific name. The file names are handled by Sage and are passed on to the network encoder which then must record to the given file name. Gotta find a way to assign a file name to each direct recording command...

Okay so I was thinking about ways we might use sym-links or shortcuts and couldn't think of a way to make it work on windows machines.

Then I remembered avisynth. Apologies if I'm telling you something you already know about but in case you don't, here goes. Avisynth is scripting engine used for video editing. Once installed you can create a filename called, say, sage_network_encoder.avs and have in it the folowing line:

DirectShowSource("c:\folder\myclip.mpg")

The file 'sage_network_encoder.avs' will then redirect you to your .mpg - this works for all the media players I've tried so I guess it would work for sage too ...? (might need to tweak properties to include '.avs' files ?)

If this _did_ work some sort of clean up process would be needed - as sage would only delete the .avs file we would need to implement a way of deleting the .mpg file. Maybe some sort of rsync scripting?

mmm .. just an idea ...

[potpiejimmy]

Hi alfie,

the problem is sage wants *.mpg files and I don't think it can handle .avs files or the DirectShowSource command - and there are other problems to solve like the cleanup as you mentioned. The question is: Why do you worry?? Now that udrec TS recording works, there is absolutely no difference between direct file recording and recording with udrec. The resulting data is the same and it should also be equally reliable. There's much more control over the recording process with udrec than with direct file recording.

For me it's -udp and -patpmt. This way I get perfectly recorded files with all the original stream infos and everything, much better than the multiplexed MPG-PS streams. Only need to find out why Sage shows a playback error for most of the files, but as VLC playback with the latest Neutrino image works I'm very positive that it will eventually work with Sage, too. And until then I just watch the recorded files with any other media player.

[Alfiegerner]

Hey potpie, yeah your right, too much fretting on my part

[Alfiegerner]

hey potpie, just trying it out now and I've ran into a snag. Sorry to hassle you with this ...

Basically I'm getting GET FILE SIZE 0 as udrec_mod isn't streaming.

Okay I took this output from the console:

Starting process udrec_mod -ts -host 192.168.1.125 -now -vp 258 -ap 259 -ap 1041 -ap 0 -o D:\TV Recordings\QuestionTime-1755735-0.mpg

If I pass these paramaters myself on the command line to udrec_mod it behaves as if I have given it a wrong paramter, i.e. the help list of switch statements comes up.

The problem is with the space in the directory path D:\TV Recordings\QuestionTime-1755735-0.mpg, so "D:\TV Recordings\QuestionTime-1755735-0.mpg" work fine.

eg

udrec_mod -ts -host 192.168.1.125 -now -vp 258 -ap 259 -ap 1041 -ap 0 -o "D:\TV Recordings\QuestionTime-1755735-0.mpg"

[potpiejimmy]

Ooopsy... my fault Should have put those quotation marks around the file name... fixed it in 0.7.1

[dave™]

Hi, was really looking forward to using this plugin as it is a feature i have looked forward to for a while, i have installed everything and added a few channels, although when i goto the EPG it is blank for every channel, if you try and watch the channel, it changes the channel on my dbox and plays it fine on the tv, but in SageTV all i get is NO SIGNAL, please help as i have been up for the last few hours goin mad trying to get this to work lol

I have tried all the basics.. e.g

SPTS mode, new neutrino image, even watching through the LIVE plugin for VLC on the yweb is working, so i don't get how SageTV cant pick it up.

Thanks for all replies in advance, hope to hear from you soon.

Keep up the good work on this project

[Alfiegerner]

Can you grab a bit of debug for us?

We need to have look at the command to the grabbing program. In the grabrecorder console window see if you can isolate the line with 'udrec_mod' or 'grab_mod' in it. The best way to do this is to switch to a live tv channel and then press stop almost immediately. If you scroll up you should be able to see the command. You might have to be quick on the stop button though so it doesn't scroll too far.

Can you post the command to the grabbing program here?

edit: I forgot to say, the problem with the epg being blank is a seperate one. To fill the epg you need to manually import xmltv data. Have a read about and you should be fine.

[potpiejimmy]

Hi Dave,

it seems you're having two problems there, the missing EPG data and „no signal“. The retrieval/display of the EPG data is not related to the underlying encoder, it works independently from the actual video source. What kind of EPG source do you use (tvtv?) (what country are you in, btw?). What do you mean the EPG is blank, is that „No data“-programs or is the whole row in the EPG blank, because then you probably just have to wait a day or so until it gets populated with the shows.

Anyhow, if you can already change the channels thru grabrecorder then you're probably almost there. Just a few things to keep in mind:

1) You must enable SPTS mode ONLY when using the -udp option! When not using -udp, you must disable SPTS! (The next version of grabrecorder will automatically switch SPTS on and off in the DBOX, but until then, you have to set it up manually).

2) It takes a while until the first data is written to the files by the grabbing programs and until then, the „No signal“ box appears. Just wait a little longer, and the playback begins as soon as enough data is available. When not using -udp, it is much faster, playback starts after a few seconds, the „No signal“ box may not even appear at all, but with -udp (udrec) it takes much much longer (like 20 or 30 secs), so -udp is not really useful for live channel flipping (at the moment, will have a look at it).

3) For now, if you use -udp you should also use -patpmt to get valid files with PAT and PMT. The current Sage versions pops up an ERROR -4 when trying to play these files (most of them), but this should be fixed in the next Sage version (4.1.10, i guess).

Hope that helps
/potpiejimmy


P.S.: Oh! Alfiegerner was faster, hi Alfie yeah the command line would be helpful, too, if the problem remains

[dave™]

lol thanks for the help, i decided to do the xmltv bit first to see if i could get an epg, now when i finnished that, i clicked on watch now on a channel, and it worked!!!

Thanks guys, this is just what ive been looking for! (BTW i upgraded to the latest beta version as well)

[dave™]

did you ever find out what was causing ggrabmod to hang at 99% cpu, working perfectly except sometimes i come in to find it has crashed and not recorded anything lol

Keep up the good work

[dave™]

Let me start off by saying a big thank you for this encoder.

I have just bought sagetv and a remote just for this dbox network encoder


For some reason udp doesn't work at all with me, with PATPMT or not, and with SPTS mode enabled or not

I am happilly using tcp mode, (with SPTS mode on or off, i can't find the difference, in tcp mode anyway) although when it crashes at 99% cpu it is really annoying as it takes a awhile to end the task and get it restarted.

I have setup grabrecorder.exe as a windows service, which starts at the logon screen

But if i could just get some software that monitors the cpu usage and ends ggrab_mod if it reaches 95% or so, then the next time a channel is changed or something is starts to record, the grabrecorder.exe service will restart ggrab_mod.exe.......... but i have yet to find this software lol, so for now, it's 5 mins each time goin into the task manager


Any ideas??

Dave

[potpiejimmy]

Hi Dave,

sorry I kept you waiting so long for an answer... I still don't know what causes the ggrab process to hang at 99% cpu sorry. The only thing I could do for you was to add a command line switch -np that starts the ggrab process in "normal priority class" instead of above-normal priority which should make killing the process much easier when it hangs. Yeah, I know, it's not really a satisfying solution...

If you don't use Sage to watch Live-TV, you should really try to get -udp to work: Do you have a recent neutrino image on your box? With the new version 0.9 you should disable SPTS by default in your dbox settings. grabrecorder will activate SPTS when recording starts and deactivate when recording stops, which is more reliable than having your box in SPTS mode all the time. You should also use the -sr option.

You still can't use -patpmt, though, if you want to watch the recorded shows in Sage

[marneb17]

I tried the encoder again. First of all, the quality is, of cource, much better than the method I use right now, recording through the svideo of my pvr150.
Recording with udrec works great. I think the ggrab support is broken in version .9. I can see in the command window that the command that starts ggrab ends after the -o parameter.
But...
Since the "mpg"/ts files that are recorded aren't played back by sage very well, and don't work on the MVP extender, I looked at fixing them. After trying a lot, I found that projectx can demux the file and fixes the problems. But then i end up with two demuxed files. I muxed them in tmpg and the file played in sage and on the mvp!

So I set out to automate the process, and it was quite some work, but now it works.
The only problem I have now is LiveTV. I want Sage to use the Svideo Input when I specify nothing, but use thenetwork encoder when I record something in the future.
Here is what I thought of but didn't get to work: I thought setting the network encoder to the quality "digital" would help me. Then I could have the default recording quality set to "Best" and since the network encoder can't supply the quality "Best" it would record through the pvr150 but when I select "digital" as the recording quality then it would need to use the network encoder. But this does not work.
Does anybody have an idea to get this working?

Well, now I will explain how to demux and mux the file automatically:

I included my batch file but you have to edit it to make it fit your directories!!!!
These tools are needed:
ProjectX
mplex (attached to this post)
and touch (included in coreutils)

I do not take responsibility if this batch will delete your harddrive...

First install projectx and coreutils.
Put mpex.exe in a directory

Edit the demuxmux.bat

Now you can test it with drag-and-dropping a recorded mpeg file on the bat file !!!MAKE A BACKUP OF THE FILE, BEFOREHAND!!! The original file will get deleted! After the process is done, the file should play in Sage.
If this works for you, then you can set up the automation.
If you have other recording devices than the dbox2 then setup an extra recording directory for the dbox. Through the line "forced_video_storage_path_prefix"

If this is set then it will only write to video storage directories that begins with the string this property is set to.

You have to set this for every capture device. So that they write to different locations.
Next configure dirmon2 so that it will scan the dbox directory and process "mpg" files, "must not exist" -> "txt", and "wait until the target file is finished" must be checked.
And well that should be it.