SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV Github Development

Notices

SageTV Github Development Discussion related to SageTV Open Source Development. Use this forum for development topics about the Open Source versions of SageTV, hosted on Github.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 02-05-2016, 08:34 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
FFMPEG update log

I'm going to try to consolidate comments about my efforts to update FFMPEG here. Right now I have comments in a couple of threads, I may try to drag them into this thread for completeness.

My Current Plan:
Get code as close to the release in SageTV
Compare them into the same version
ID SageTV modifications
attempt to forward patch those mods into the most recent version of FFMPEG

Some Questions:
What custom support as added to ffmpeg for Sage? I saw something about processing STV streams directly. I also saw something about conversion of names of codecs back and forth


Applicable Posts (that I have found so far):
http://forums.sagetv.com/forums/show...59&postcount=1
http://forums.sagetv.com/forums/show...peg#post576622

The code version appears to be:
0.4.9-pre1 < sage version < 0.5.0


Files with SageTV Code in them

ffmpeg.c

libavcodec/h263.c:
libavcodec/mpegvideo_enc.c
libavcodec/ratecontrol.h
libavcodec/utils.c:

libavformat/allformats.c
libavformat/avidec.c
libavformat/flvdec.c
libavformat/mov.c
libavformat/Makefile
libavformat/mpeg.c
libavformat/mpegenc.c
libavformat/oggdec.c
libavformat/stv.c

Last edited by SageWizdom; 02-05-2016 at 09:08 PM.
Reply With Quote
  #2  
Old 02-05-2016, 08:38 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
I updated this comment to create a single merged and sorted list.

Grep for NARFLEX.
Grep for SageTV.
Grep for stv (I've included all the ones I think are right)

Are there any other names I should look for in the code for comments? Thanks

Quote:
ffmpeg.c:1149:// NARFLEX: I disabled this because it can still work even if this happens so there's no reason to bail!

libavcodec/h263.c:665-670: // NARFLEX: SageTV Fix divide by zero errors that can happen here

libavcodec/mpegvideo_enc.c:893:// NARFLEX: Disable this block and add above instead like they used to, it's causing FFMPEG to crash on placeshifter transcoding
libavcodec/mpegvideo_enc.c:1193: // NARFLEX: Disable this secondary if block, it's causing FFMPEG to crash on placeshifter transcoding
libavcodec/mpegvideo_enc.c:1282:// NARFLEX: Disable this block, it's causing FFMPEG to crash on placeshifter transcoding
libavcodec/mpegvideo_enc.c:2706:// NARFLEX: Doing dynamic adjustment of q based on bitrate has turned out to give pretty poor

libavcodec/ratecontrol.h:83: int last_frame_rc_reset; /// Narflex: for dynamic rate control

libavcodec/utils.c:1133: /* NARFLEX: We don't want DAR/PAR in the output string

libavformat/allformats.c: REGISTER_PROTOCOL (STV, stv);

libavformat/avidec.c:836: // Narflex - but they've changed this in the latest version of FFMPEG so I no longer am sure this is needed

libavformat/flvdec.c:477: // NOTE: Narflex - do not read in the duration here because it requires seeking to the end of

libavformat/Makefile:OBJS-$(CONFIG_STV_PROTOCOL) += stv.o

libavformat/mov.c:1598://{ MKTAG( 'u', 'd', 't', 'a' ), mov_read_udta }, // NARFLEX: We want to do more detailed metadata parsing then what's in here, so use the default tag handler
libavformat/mov.c:1807: // NARFLEX: Since we don't pass around the AVFormatParameters, but we need to know whether or not

libavformat/mpeg.c:90: // NARFLEX: I've seen FLAC files with audio:2, so let's set it higher for safety
libavformat/mpeg.c:134: // NARFLEX: This is in here to deal with MPEG2 files I keep running into that start with timestamps
libavformat/mpeg.c:378: // NARFLEX - I have no idea why they don't strip off the audio header junk

libavformat/mpegenc.c:1176: // NARFLEX: SageTV, force I frame alignment if we're doing IFrame only mode since the PPC MPEG2 decoder requires that for parsing timestamps

libavformat/oggdec.c:302:// NARFLEX: Add 'ap' so we can detect dump_metadata bit being set
libavformat/oggdec.c:332: // NARFLEX: This is where we extract the metadata from the OGG file and dump it
libavformat/oggdec.c:443: // NARFLEX: Add 'ap' so we can detect dump_metadata bit being set
libavformat/oggdec.c:494: // NARFLEX: SageTV, I added the '&& !i' to this conditional so it took the first stream.
libavformat/oggdec.c:530: // NARFLEX: Add 'ap' so we can detect dump_metadata bit being set
libavformat/oggdec.c:553: // NARFLEX: Add 'ap' (NULL) so we can detect dump_metadata bit being set

libavformat/stv.c:2: * SageTV Streaming protocol for ffmpeg client
libavformat/stv.c:21: * This allows you to stream files directly from a SageTV server
libavformat/stv.c:162: av_log(NULL, AV_LOG_ERROR, "Opening conn to SageTV server\n");
libavformat/stv.c:335: av_log(NULL, AV_LOG_ERROR, "Sending cmd to SageTV Server:%s", data);
libavformat/stv.c:447: av_log(NULL, AV_LOG_ERROR, "Sending2 cmd to SageTV Server:%s\n", data);
libavformat/stv.c:592: av_log(NULL, AV_LOG_ERROR, "Sending cmd to SageTV Server:%s\n", data);

Last edited by SageWizdom; 02-07-2016 at 07:08 AM.
Reply With Quote
  #3  
Old 02-05-2016, 08:41 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
(edited) Moved into the above comment

Last edited by SageWizdom; 02-07-2016 at 07:02 AM.
Reply With Quote
  #4  
Old 02-06-2016, 06:15 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,541
This is similar to what I did when I worked on the mplayer stuff... the only other thing that I did was use MELD and do a directory compare against the 2 versions... the problem is that even with the sources that Jeff provided, there were lots of changes, and not all of them were sagetv changes... but it did help once I narrowed things down.

The other thing that I would look for is STV.. I think there some cases where stv shows up instead of sagetv.

I think if look in the configure scripts you might see some sagetv defines in there (that don't have sagetv in them... I just trying to remember from the mlayer work) and then search for those defines.
Reply With Quote
  #5  
Old 02-06-2016, 12:50 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
You are definitely correct. I found a bunch of STV instances. I'll have to look at MELD, I've always used BeyondCompare which does the directory compare, then lets you drill down into the source file compare (which I then merge).

Thanks
Reply With Quote
  #6  
Old 02-06-2016, 01:46 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,541
Quote:
Originally Posted by SageWizdom View Post
You are definitely correct. I found a bunch of STV instances. I'll have to look at MELD, I've always used BeyondCompare which does the directory compare, then lets you drill down into the source file compare (which I then merge).

Thanks
I think that Beyond Compare is probably a better tool, but meld is what I have on linux
Reply With Quote
  #7  
Old 02-07-2016, 06:22 AM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
Quote:
Originally Posted by stuckless View Post
I think that Beyond Compare is probably a better tool, but meld is what I have on linux
BC is what I'm using on Linux. Back in the day it was Windows only, but it now works everywhere. If you want to try it, it has a free 30 day trial.

http://www.scootersoftware.com/download.php
Reply With Quote
  #8  
Old 02-15-2016, 02:43 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,578
Will updating FFMPEG fix the issues with Placeshifter not properly playing some files, like 1080i H.264 files.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #9  
Old 02-15-2016, 03:01 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
Quote:
Originally Posted by wayner View Post
Will updating FFMPEG fix the issues with Placeshifter not properly playing some files, like 1080i H.264 files.
Great question. I don't know. I know that on Mac, FFMPEG is used in the SageClient, but I have not dug into Placeshifter.
Reply With Quote
  #10  
Old 02-15-2016, 03:23 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,578
Here is a post from Fuzzy where he says that Sage uses an older version of FFMPEG that has known issues with interlaced H.264 content.

http://forums.sagetv.com/forums/show...7&postcount=16
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #11  
Old 02-16-2016, 08:49 AM
Monedeath Monedeath is offline
Sage Expert
 
Join Date: Sep 2009
Location: Idaho
Posts: 508
Quote:
Originally Posted by wayner View Post
Here is a post from Fuzzy where he says that Sage uses an older version of FFMPEG that has known issues with interlaced H.264 content.

http://forums.sagetv.com/forums/show...7&postcount=16
That only applies to pre SageTV7 users.
Reply With Quote
  #12  
Old 02-20-2016, 10:34 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
Jeff posted an updated version to github. I'll grab that one and do some more compares.

http://forums.sagetv.com/forums/show...83&postcount=1
Reply With Quote
  #13  
Old 02-21-2016, 02:57 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,578
I am pretty sure that I have tried Placeshifter V9 with H.264 1080i content from an HD-PVR and it doesn't work, at least not on Windows.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #14  
Old 03-14-2016, 08:46 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
Ok, I've chewed through the new code drop and compared to the old one. I found a few more items. Of specific note, there are now 4 terms I suggest searching for in the code (anyone know of any more?)

sagetv, narflex, jft (another developer) and stv

grep -nir sagetv *
grep -nir narflex *
grep -nir jft *
grep -nir stv *


ffmpeg.c:1693:// NARFLEX: I disabled this because it can still work even if this happens so there's no reason to bail!
libavcodec/avcodec.h:342: /* STV: HD audio codecs for detection purposes (not fully decoded) */
libavcodec/dca.c:1289: // JFT, so we only look at 1 frame when looking for DTS-HD/DTS-MA
libavcodec/h264_parser.c:197: // STV: at least one interlaced frame will mark the stream as interlaced
libavcodec/libx264.c:92: // NARFLEX: See if the bitrate information has changed, and if so, reconfigure the encoder
libavcodec/mpeg4video.c:150: // NARFLEX: SageTV Fix divide by zero errors that can happen here
libavcodec/mpegvideo_enc.c:885:// NARFLEX: Disable this block and add above instead like they used to, it's causing FFMPEG to crash on placeshifter transcoding
libavcodec/mpegvideo_enc.c:1185: // NARFLEX: Disable this secondary if block, it's causing FFMPEG to crash on placeshifter transcoding
libavcodec/mpegvideo_enc.c:1278:// NARFLEX: Disable this block, it's causing FFMPEG to crash on placeshifter transcoding
libavcodec/mpegvideo_enc.c:2712:// NARFLEX: Doing dynamic adjustment of q based on bitrate has turned out to give pretty poor
libavcodec/ratecontrol.h:83: int last_frame_rc_reset; /// Narflex: for dynamic rate control
libavcodec/utils.c:883: /* NARFLEX: We don't want DAR/PAR in the output string
libavcodec/utils.c:898: laced = 1; // STV: for some reason H.264 reports 2x frame rate (field rate)


libavformat/allformats.c:237: REGISTER_PROTOCOL (STV, stv); //SAGETV
libavformat/avidec.c:861: // JFT, we don't have a start point, try going where we found MOVI
libavformat/avidec.c:978: // Narflex - but they've changed this in the latest version of FFMPEG so I no longer am sure this is needed
libavformat/avidec.c:1064: // JFT, when discarding use size not packet size
libavformat/flacdec.c:98: // SageTV: report picture size and offset for external extraction
libavformat/flvdec.c:469: // NOTE: Narflex - do not read in the duration here because it requires seeking to the end of
libavformat/Makefile:298:OBJS-$(CONFIG_STV_PROTOCOL) += stv.o
libavformat/matroskadec.c:22:// JFT: Added binblock to improve streaming and remove unneeded copy
libavformat/matroskadec.c:832: // JFT: especially needed when we have no MMU
libavformat/matroskadec.c:1453: // NARFLEX - disable extraction of the individual track titles
libavformat/matroskadec.c:1535: // NARFLEX: Add metadata for the start of each chapter
libavformat/matroskadec.c:1859: // JFT: Basic version of no copy for special case...
libavformat/matroskadec.c:1991: // NARFLEX: This is a BUG where they use the index from the wrong track below in the comparison so it can end up accessing an invalid index
libavformat/mov.c:214: // SageTV: Truncate "year" so it really only reports the year instead of the full date
libavformat/mov.c:2265: // SageTV added tags
libavformat/mov.c:2390:// SageTV added tags
libavformat/mov.c:2820: // JFT verify that's right in all cases...
libavformat/mpeg.c:138: // NARFLEX: This is in here to deal with MPEG2 files I keep running into that start with timestamps
libavformat/mpeg.c:303: /* STV: special case broken PS files produced by a bug in our muxer that created 0x00000100 start codes on some audio PES packets */
libavformat/mpeg.c:414: // NARFLEX - I have no idea why they don't strip off the audio header junk
libavformat/mpeg.c:512: // JFT: If we don't know the type use other detection methods
libavformat/mpegenc.c:1184: // NARFLEX: SageTV, force I frame alignment if we're doing IFrame only mode since the PPC MPEG2 decoder requires that for parsing timestamps
libavformat/mpegts.c:575:// NARFLEX - we don't want another stream for AC3 inside of TrueHD; that just messes things up
libavformat/stv.c:2: * SageTV Streaming protocol for ffmpeg client
libavformat/stv.c:21: * This allows you to stream files directly from a SageTV server
libavformat/stv.c:176: av_log(NULL, AV_LOG_ERROR, "Opening conn to SageTV server\n");
libavformat/stv.c:326: av_log(NULL, AV_LOG_ERROR, "Sending cmd to SageTV Server:%s\n", data);
libavformat/stv.c:365: av_log(NULL, AV_LOG_ERROR, "Sending cmd to SageTV Server:%s", data);
libavformat/stv.c:477: av_log(NULL, AV_LOG_ERROR, "Sending2 cmd to SageTV Server:%s\n", data);
libavformat/stv.c:610:// Test JFT
libavformat/stv.c:619: av_log(NULL, AV_LOG_ERROR, "Sending cmd to SageTV Server:%s\n", data);
libavformat/utils.c:740: // NARFLEX - audio in DVR-MS files will fail if we don't subtract 1 since it gets a score of 25
libavformat/utils.c:756: // NARFLEX: David had set this at 1MB; but we had MPEG2PS recordings that failed unless it was 2MB

It looks like this code base is an early v.0.6. I'm going to try and merge into the last .6 build (6.6.4 or something like that).

The question to the crowd, just looking at the v0.5 code and the v0.6 code, there are lots of changes. I think I can patch them into the newest, but I'm not even sure how they were found. I'll try to compile the v.0.6 once I patch in and will let folks know how it went.
__________________
Server: Centos Server 14.04 LTS - 64Bit, VM in XenServer, 2 cores of a Intel i7, 2-4 GB Ram, 8 GB system Disk, 1.8 TB storage, SageTV V9.0.4.232, HDHR Prime x 1

Clients: PC Client x 1, HD-300 x 1, AppleTV x 2, WebClient (phone/tablet) x 3
Reply With Quote
  #15  
Old 03-15-2016, 03:36 PM
Narflex's Avatar
Narflex Narflex is offline
Sage
 
Join Date: Feb 2003
Location: Redondo Beach, CA
Posts: 6,301
Don't rely on those greps for finding everything...you will miss things. And did you see that I also posted the FFMPEG upstream version that the one in GitHub was forked from? That'll make figuring out the changes super easy. (I don't remember exactly where i posted it...just search for recent posts by me with FFMPEG in them)
__________________
Jeffrey Kardatzke
Google
Founder of SageTV
Reply With Quote
  #16  
Old 03-15-2016, 04:47 PM
dinki's Avatar
dinki dinki is offline
Sage Expert
 
Join Date: Mar 2006
Posts: 643
Quote:
Originally Posted by Narflex View Post
Don't rely on those greps for finding everything...you will miss things. And did you see that I also posted the FFMPEG upstream version that the one in GitHub was forked from? That'll make figuring out the changes super easy. (I don't remember exactly where i posted it...just search for recent posts by me with FFMPEG in them)
http://forums.sagetv.com/forums/show...95&postcount=6
__________________
Intel Q6600 Quad 2.4GHZ, 8GB RAM, Win7 Pro SP1 64bit, 1 TB of Data
SageTV 9, Silicondust HDHomeRun x2 OTA
HD200
Reply With Quote
  #17  
Old 03-15-2016, 07:04 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
Thanks. I just pulled the link and will do another round.

@NARFLEX
Did you guys really add the code for EM8622UNCACHED, and the integration of OGG, LIGFAAD and DTSHD/DTSMA?

I'm trying to think of how to integrate and port forward, and I was hoping to use all the protocols as used / supported in ffmpeg and just bring in the custom STV types.

If you remember what you all were adding and why (perhaps wax philosophically), it would be of tremendous assistance in understanding why this versus that an the goals.

Thank you
__________________
Server: Centos Server 14.04 LTS - 64Bit, VM in XenServer, 2 cores of a Intel i7, 2-4 GB Ram, 8 GB system Disk, 1.8 TB storage, SageTV V9.0.4.232, HDHR Prime x 1

Clients: PC Client x 1, HD-300 x 1, AppleTV x 2, WebClient (phone/tablet) x 3
Reply With Quote
  #18  
Old 03-15-2016, 08:18 PM
SageWizdom SageWizdom is offline
Sage Advanced User
 
Join Date: Oct 2013
Location: https://github.com/SageWizdom/SageConnect
Posts: 216
Quote:
Originally Posted by SageWizdom View Post
@NARFLEX
Did you guys really add the code for EM8622UNCACHED, and the integration of OGG, LIGFAAD and DTSHD/DTSMA?
I re-read this realize this does not read the way I intended. Which was really, in comparing things, I see the above types of things and I'm curious if this is something you guys added vs still being slightly off on the version and trying to figure out which bits are sage critical and which are just normally part of FFMPEG
__________________
Server: Centos Server 14.04 LTS - 64Bit, VM in XenServer, 2 cores of a Intel i7, 2-4 GB Ram, 8 GB system Disk, 1.8 TB storage, SageTV V9.0.4.232, HDHR Prime x 1

Clients: PC Client x 1, HD-300 x 1, AppleTV x 2, WebClient (phone/tablet) x 3
Reply With Quote
  #19  
Old 03-16-2016, 03:45 PM
Narflex's Avatar
Narflex Narflex is offline
Sage
 
Join Date: Feb 2003
Location: Redondo Beach, CA
Posts: 6,301
We did add all of the EM8622UNCACHED stuff...but go ahead and remove it all. That was for running on the HD100 media extender and that'll never be compiled again.

For OGG, LIBFAAD and DTSHD/MA; we did add those in there...BUT we pulled all of that from upstream versions of FFMPEG at the time (i.e. rather than porting all of FFMPEG to a newer version; we just grabbed those files and included them into our version...however it's possible we did fix extra issues in them).

There's really 3 types of changes you'll find in there:

1. SageTV feature additions; things like rate control for placeshifting, stv protocol handler, support for files that are growing while being accessed...basically features we needed to add to FFMPEG to make it work for SageTV in the ways we wanted to

2. Upstream feature pulls; things we wanted in the version of FFMPEG we were using...but were in newer versions, so we just pulled in those files to get those features

3. Bug fixes. Customers had issues...we had to fix them...so sometimes we had to fix bugs in FFMPEG.

Ideally, you would just be porting #1...because #2 should happen automatically with any newer version of FFMPEG you port to; and ideally they've fixed the bugs in #3 that we fixed in our version. However, I'm not sure they've fixed all of #3; but if new issues come up...the diffs there could be used as a point of reference for what the bugs were in FFMPEG that we fixed and maybe that those fixes need to be ported to a newer version as well.
__________________
Jeffrey Kardatzke
Google
Founder of SageTV
Reply With Quote
  #20  
Old 03-17-2016, 12:41 AM
JREkiwi's Avatar
JREkiwi JREkiwi is offline
Sage Icon
 
Join Date: Jan 2005
Location: Auckland, New Zealand
Posts: 2,059
Quote:
Originally Posted by Narflex View Post
2. Upstream feature pulls; things we wanted in the version of FFMPEG we were using...but were in newer versions, so we just pulled in those files to get those features
I know one of the things was the LATM support that was needed for the NZ and UK AAC DVB audio.

John
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ffmpeg development Fuzzy SageTV Github Development 22 11-04-2015 04:11 PM
SageDTC 2.7.0.2 with ffmpeg mtenboer Hardware Support 1 11-13-2013 02:47 PM
Mencoder instead of FFmpeg TechBill SageTV Beta Test Software 1 02-10-2008 11:23 PM
ffmpeg help ChePazzo SageTV Linux 4 12-04-2006 09:52 AM
ffmpeg v0.4.9 pre for Windows? jsturtevant SageTV Customizations 14 06-06-2006 05:24 PM


All times are GMT -6. The time now is 03:46 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.