SageTVTranscoder Plugin

[jvl711]

Originally Posted by phelme

I tried an H.265 MKV the other day through my HD300 and while it definitely was being transcoded through the plugin (x64) , the quality wasn't great and was very pixelated. The server is a 4-core AMD so should be able to handle it.

Is there something I maybe need to configure that I haven't?

I'm not sure which version I'm using so I'll have to double check that.

I think the Server handles the transcoding for the HD300. I think it will dynamically adjust the rate based on bandwidth. I have not used the HD300 in some time since moving to 4k/h.265. Maybe someone else would have an better idea.

It is awesome to hear it is actually working. I put a lot of work trying to get to that point . I never actually tested it because I am using all android clients.

[KryptoNyte]

Hmmmm. Maybe I'm going to have to start testing the HD300 at my other place, remotely, and see how it goes. I've never actually been able to get an HD300 connected remotely, so this might be interesting.

[pjpjpjpj]

I've been very sporadic in following this thread (life, sigh...) and I see some stuff on the prior page but not *exactly* my question...

I record OTA via HDHR (raw MPEG-2). I know this has historically caused all sorts of issues with transcoding and remote clients, and I believe somewhere I read that a solution is using OpenDCT, but I haven't followed that enough to jump into it. I believe some say it requires IJK (maybe?) but ExoPlayer is what everyone "wants" to use. I've been very busy and not able to spend a lot of time so I'd love it if this were the old-school Sage plugin compatibility of "install the plugin and it just works".

Long story short, will this plugin allow me to transcode an OTA recording from a HDHR in real time to MKV and send it to a remote client?

(to be clear, if the answer is "no" or "not yet", that's cool, just asking. )

[Malfunction]

Is there a command line option to reduce the resolution of something you're compressing with Matroska? I record a number of old shows that would look fine in 16:9 480p, but are recording in 1080p because they're on cable channels such as RFD-TV. I know this can be done with AVI.

[CptKirk]

I've got the transcoding and detector plugins installed and have nplayer's mkv formats added. I use the MKV Fast setting to change most tv recordings from .ts to .mkv.

I looked at the Video Conversions a bit ago and notice a few Converted, a whole bunch of failed and another whole bunch of waiting.

I've got the following loaded up in Notepad++: Sage.properties, SageTVTranscoder.log, sagetv_0.txt, sagetv_1.txt, sagetv_1.txt, sagetv_1.txt

If I search for Converted files and Failed files I cannot really see anything in the SageTVTranscoder.log file that shows/tells why one converted and one failed.

Right now it has a whole bunch sitting in the Waiting status with the one on top showing "Converting - 0%".

If I look at the Task Manager, SageTVTranscoder.exe *32 is at a constant 30% CPU utilization and 179,664K memory usage.

Is there a way to try and debug why I am seeing so many failures?

[jvl711]

Originally Posted by CptKirk

I've got the transcoding and detector plugins installed and have nplayer's mkv formats added. I use the MKV Fast setting to change most tv recordings from .ts to .mkv.

I looked at the Video Conversions a bit ago and notice a few Converted, a whole bunch of failed and another whole bunch of waiting.

I've got the following loaded up in Notepad++: Sage.properties, SageTVTranscoder.log, sagetv_0.txt, sagetv_1.txt, sagetv_1.txt, sagetv_1.txt

If I search for Converted files and Failed files I cannot really see anything in the SageTVTranscoder.log file that shows/tells why one converted and one failed.

Right now it has a whole bunch sitting in the Waiting status with the one on top showing "Converting - 0%".

If I look at the Task Manager, SageTVTranscoder.exe *32 is at a constant 30% CPU utilization and 179,664K memory usage.

Is there a way to try and debug why I am seeing so many failures?

I am not sure if the progress works with the new transcoder. SageTV is reading the STDOUT of the process and I think it is looking for a certain output for the percentage. This is something I need to look into at some point. So 0% progress does not really mean there was a failure. I would first look at the SageTV log files and see if there is any error from the transcode to go after. That is where I would start.

Josh

[CptKirk]

Will do. I'm away from home for a week. I'll dig into this when I get back.

[hvymetal]

Originally Posted by jvl711

Awesome! I will try and package that up this week and push it out.

Thank you for taking a look.

Josh

I rebuild my SageTV using linux docker to use OpenDCT so playback with the miniclient is better. Since I have many recording I need to fix I tried to transcode with copy but for some reason it fails. I'm using it with Java11 and SageTv Server 9.2.6.1.

I get the following in SageTVTrancoder.log:

################################ Exiting ################################
################################ SageTVTranscoder (FFmpeg) starting ################################
Arguments: /opt/sagetv/server/ffmpeg -dumpmetadata -v 2 -i /var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.mpg 
################################ SageTVTranscoder (FFmpeg) starting ################################
Arguments: /opt/sagetv/server/ffmpeg -v 3 -y -threads 2 -sn -vsync 1 -async 100 -stdinctrl -i /var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.mpg -threads 7 -f matroska -c copy -aspect 16:9 -deinterlace -map 0:1 -map 0:0 /var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.tmp 
Exiting for some other reason
################################ Exiting ################################

If I copy paste the command in shell window I get the following output:

[matroska @ 0x25ef640] Starting new cluster with timestamp 2387680 at offset 3338167749 bytes
[matroska @ 0x25ef640] Writing block of size 1536 with pts 2387680, dts 2387680, duration 32 at relative offset 11 in cluster at offset 3338167749. TrackNumber 2, keyframe 1
[matroska @ 0x25ef640] Can't write packet with unknown timestamp
av_interleaved_write_frame(): Invalid argument
No more output streams to write to, finishing.
[matroska @ 0x25ef640] Writing block of size 1536 with pts 2387712, dts 2387712, duration 32 at relative offset 1554 in cluster at offset 3338167749. TrackNumber 2, keyframe 1
[matroska @ 0x25ef640] Writing block of size 35080 with pts 2387738, dts 2387738, duration 33 at relative offset 3097 in cluster at offset 3338167749. TrackNumber 1, keyframe 0
[matroska @ 0x25ef640] Writing block of size 38398 with pts 2387772, dts 2387772, duration 33 at relative offset 38185 in cluster at offset 3338167749. TrackNumber 1, keyframe 0
[matroska @ 0x25ef640] Can't write packet with unknown timestamp
[matroska @ 0x25ef640] end duration = 2387805
[matroska @ 0x25ef640] stream 0 end duration = 2387805
[matroska @ 0x25ef640] stream 1 end duration = 2387744
Error writing trailer of /var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.tmp: Invalid argument
frame=71552 fps=1763 q=-1.0 Lsize= 3260042kB time=0.010 bitrate=11182.6kbits/s speed=58.9x
video:3147424kB audio:111925kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.021282%
Input file #0 (/var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.mpg):
  Input stream #0:0 (audio): 74616 packets read (114610902 bytes);
  Input stream #0:1 (video): 71552 packets read (3222962133 bytes);
  Input stream #0:2 (audio): 0 packets read (0 bytes);
  Total: 146168 packets (3337573035 bytes) demuxed
Output file #0 (/var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.tmp):
  Output stream #0:0 (video): 71552 packets muxed (3222962133 bytes);
  Output stream #0:1 (audio): 74616 packets muxed (114610902 bytes);
  Total: 146168 packets (3337573035 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x261b0c0] Statistics: 2 seeks, 12736 writeouts
[AVIOContext @ 0x25f34c0] Statistics: 3466383504 bytes read, 2 seeks
Conversion failed!

Here are the steps I used to install SageTVTranscoder-FFmpeg, MediaFormatParserPlugin and updated transcode profiles:

###Install jv711_SageTVTranscoder-FFmpeg....This can also be installed as SageTV Plugin which is safer
mv /opt/sagetv/server/ffmpeg /opt/sagetv/server/ffmpeg.org 
wget https://github.com/jvl711/SageTVTranscoder-FFmpeg/releases/download/SageTVTranscoder_v1.0.6/SageTVTranscoderLinux_v1.0.6.zip
unzip -o SageTVTranscoderLinux_v1.0.6.zip -d /opt/sagetv/server
rm SageTVTranscoderLinux_v1.0.6.zip

###Install New Trancoder Profiles
echo '
transcoder/formats/MKV-H264\superfast\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset superfast -c:v libx264;]
transcoder/formats/MKV-H264\veryfast\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset veryfast -c:v libx264;]
transcoder/formats/MKV-H264\faster\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset faster -c:v libx264;]
transcoder/formats/MKV-H264\fast\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset fast -c:v libx264;]
transcoder/formats/MKV-H264\Medium\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset medium -c:v libx264;]
transcoder/formats/MKV-H264\slow\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset slow -c:v libx264;]
transcoder/formats/MKV-H264\slower\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset slower -c:v libx264;]
transcoder/formats/MKV-H264\veryslow\ MKV=f\=matroska;MCompressionDetails\=-c copy -preset veryslow -c:v libx264;]
transcoder/formats/MKV-Remux\ MKV=f\=matroska;MCompressionDetails\=-c copy;]
' >>  /opt/sagetv/server/Sage.properties

###Install jv711_MediaFormatParserPlugin
wget https://github.com/jvl711/MediaFormatParserPlugin/releases/download/0.7/MediaFormatParserPlugin_v0.7.zip
unzip -o MediaFormatParserPlugin_v0.7.zip -d /opt/sagetv/server/JARs
rm MediaFormatParserPlugin_v0.7.zip
sed -i '/mediafile_mediaformat_parser_plugin/d' /opt/sagetv/server/Sage.properties
echo 'mediafile_mediaformat_parser_plugin=jvl.mediaformat.MediaFormatParserPlugin' >>  /opt/sagetv/server/Sage.properties

It starts the conversion but stops midway

12/15/2021  12:17 AM     3,338,283,324 ForeverYoungSearchingfortheFountainofYouth-45451932-0.tmp
11/28/2021  10:00 PM     5,217,087,124 ForeverYoungSearchingfortheFountainofYouth-45451932-0.mpg
11/28/2021  11:17 PM               161 ForeverYoungSearchingfortheFountainofYouth-45451932-0.txt
11/28/2021  11:17 PM               135 ForeverYoungSearchingfortheFountainofYouth-45451932-0.edl
11/28/2021  11:00 PM             9,211 ForeverYoungSearchingfortheFountainofYouth-45451932-0.logo.txt
11/28/2021  11:17 PM           402,888 ForeverYoungSearchingfortheFountainofYouth-45451932-0.log
               6 File(s)  8,555,782,843 bytes
               0 Dir(s)  6,666,951,823,360 bytes free

I can get it to work if I add the following switch -fflags +genpts. Anyone know how to add this to a profile?

/opt/sagetv/server/ffmpeg -fflags +genpts -v 3 -y -threads 2 -sn -vsync 1 -async 100 -stdinctrl -i /var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.mpg -threads 7 -f matroska -c copy -aspect 16:9 -deinterlace -map 0:1 -map 0:0 /var/media/tv/ForeverYoungSearchingfortheFountainofYouth-45451932-0.mkv

I found a workaround by using a man in the middle script but it would be better to store in the profile

mv /opt/sagetv/server/ffmpeg /opt/sagetv/server/ffmpeg.run
echo '/opt/sagetv/server/ffmpeg.run -fflags +genpts "$@"' >  /opt/sagetv/server/ffmpeg
chmod 777 /opt/sagetv/server/ffmpeg

[hvymetal]

I found an issue with the plugin(At least for Linux). When you install a new version of SageTV it overwrites the ffmpeg that is installed via the plugin.

In the mean time I use the following workaround I posted above by using a man in middle script. If the plugin gets updated you need to remember to repeat step 1.

1.  Install the plugin
2.  ###rename ffmpeg to ffmpeg.run
     mv /opt/sagetv/server/ffmpeg /opt/sagetv/server/ffmpeg.run
3.  ###When using docker and the following lines to sagetv-user-script.sh
     ######Workaround for FFPEG options -fflags +genpts  
     echo '/opt/sagetv/server/ffmpeg.run -fflags +genpts "$@"' > /opt/sagetv/server/ffmpeg
     chmod 777 /opt/sagetv/server/ffmpeg

[wayner]

A question regarding this plugin - on Linux if that matters.

My understanding is that this program will transcode content in realtime. For example it will transcode a H.265 file that you play on a HD300 extender as an HD300 does not support H.265 files. Is that correct? Does it also transcode files to resolutions that are supported - such as transcoding a 2160p file to 1080p when you try to play it on an extender?

Will it also transcode files being played on Placeshifter - such as fix the problem where 1080i files have never played properly on SageTV Placeshifter?

Is there any way to specify which file types get transcoded? And what the destination resolution is - like if you want the destination playback to be 720p or 1080p or whatever?

[phelme]

Originally Posted by wayner

A question regarding this plugin - on Linux if that matters.

My understanding is that this program will transcode content in realtime. For example it will transcode a H.265 file that you play on a HD300 extender as an HD300 does not support H.265 files. Is that correct? Does it also transcode files to resolutions that are supported - such as transcoding a 2160p file to 1080p when you try to play it on an extender??

I'm curious too. So far every time I've tried to use it with an HD300 and H.265 files, it is sort of successful, but only shows an extremely low resolution on the HD300 because of inadequate bandwidth or something. Which on a gigabit ethernet LAN doesn't make much sense.

[phantomii]

Hey, I identified an issue where Videos I was importing had their name incorrectly captured as "Surround 5.1" while having this Plugin installed on my Unraid Docker. You can look at post 15 in this thread for what I did to troubleshoot and narrow this down.

https://forums.sagetv.com/forums/showthread.php?t=66960

I have uninstalled this pluging for now but wanted you to be aware of the issue so when you do future releases you can maybe look at this.... Let me know if you need anything else from me!

John

[egeller]

Originally Posted by nyplayer

I noticed that the default SageTV settings for xcode are poor ... changing number of threads and priority to normal really helps in performance when transcoding.

xcode/allow_multithreading_for_hdextender_placeshifting=false
xcode_auto_deinterlace=true
xcode_disable_mono_audio=true
xcode_fix_broken_hdpvr_streams=false
xcode_process_num_threads=0
xcode_process_priority_offline=idle
xcode_process_priority_streaming=belownormal
xcode_reduce_process_priority=true

My Changed Settings ...

xcode/allow_multithreading_for_hdextender_placeshifting=true
xcode_auto_deinterlace=true
xcode_disable_mono_audio=true
xcode_fix_broken_hdpvr_streams=false
xcode_process_num_threads=2
xcode_process_priority_offline=normal
xcode_process_priority_streaming=normal
xcode_reduce_process_priority=false

After making the changes on my test system, I reverted xcode_process_num_threads back to 0. A value of 0 seems to allow Sage to use as many threads as it needs. With my FireTv 2 threads for transcoding gave sluggish results and a value of 0 threads allowed it to use 7. Your mileage may vary.

[jhkoenig]

I am struggling to successfully view imported videos in .mp4 files like this:

File Format: QUICKTIME[HEVC 1920x1036@24fps, AAC/224Kbps@48kHz 5.1 eng]

I get a blank screen and no audio, but no error messages either. The HD300 progress bar marches along, and I can skip forward and backward just as though everything was playing normally.

Can anyone suggest a profile string for the transcoder?

Thanks!

[jhkoenig]

Originally Posted by jhkoenig

I am struggling to successfully view imported videos in .mp4 files like this:

File Format: QUICKTIME[HEVC 1920x1036@24fps, AAC/224Kbps@48kHz 5.1 eng]

I get a blank screen and no audio, but no error messages either. The HD300 progress bar marches along, and I can skip forward and backward just as though everything was playing normally.

Can anyone suggest a profile string for the transcoder?

Thanks!

Oops, forgot to mention that these are encoded as H265.

[nyplayer]

Originally Posted by jhkoenig

Oops, forgot to mention that these are encoded as H265.

The HD300's I believe do not support H265. So you would have to re-encode them.

[jhkoenig]

Originally Posted by nyplayer

The HD300's I believe do not support H265.

I thought that the transcoder plugin would transcode the stream into something that it could handle. Am I wrong?

[nyplayer]

Originally Posted by jhkoenig

I thought that the transcoder plugin would transcode the stream into something that it could handle. Am I wrong?

It will not unless you convert it. The Android miniclient has settings to transcode on the fly the HD300 does not.

[jhkoenig]

Originally Posted by nyplayer

It will not unless you convert it. The Android miniclient has settings to transcode on the fly the HD300 does not.

So I can see in the SageTVTranscoder.log that it is attempting to transcode the movie on the fly, but it is using a ffmpeg DVD profile instead of an H265 profile, probably because there isn't an appropriate H265 profile.

That is what I'm seeking. Does anyone have one that works?

[egeller]

Originally Posted by jhkoenig

So I can see in the SageTVTranscoder.log that it is attempting to transcode the movie on the fly, but it is using a ffmpeg DVD profile instead of an H265 profile, probably because there isn't an appropriate H265 profile.

That is what I'm seeking. Does anyone have one that works?

Did you add the "transcoder/formats/MKV-H264" entries to sage.properties? They are in the first post of this thread. The new FFMPEG should recognize H265 without a H265 profile entry, but you need to give it options on what to transcode it to.

If you did add them, then I am out of ideas. I wonder if there is a dependency to the new format detector plugin. I think they both use the same new version of ffmpeg. I am on Linux so I need to wait for the Linux version of the transcoder plugin before I install the transcoder or format detector plug. I tried installing the latter only, and saw results that forced me to uninstall it.