![]() |
|
SageTV Github Development Discussion related to SageTV Open Source Development. Use this forum for development topics about the Open Source versions of SageTV, hosted on Github. |
![]() |
|
Thread Tools | Search this Thread | Display Modes |
#1
|
|||
|
|||
RFC: my updated build3rdparty.sh script
I've been trying to build sageTV from sources, and it's been slow
going. For now I've given up building the .jar files [I don't have the Java expertise to troubleshoot the problems] and am just trying to build the native executable stuff. The buildso.sh script worked mostly OK (I have no idea if the libraries are usable, but I was able to build them.) The build3rdparty.sh script: not so OK. That script does the following:
The first problem is that the x264 build requires ffmpeg include files and libraries. Since they haven't been built yet, the script uses the build host ffmpeg files for building x264. Then it builds ffmpeg using the four extra codecs. That's fragile. You can't expect it to work unless the build host ffmpeg libraries are the same version as the thirdparty/ffmpeg sources. That's not going to be the case on a modern Linux installation. The correct way to do a build like this is to first build a preliminary version of ffmpeg without the extra codecs. Second, build the codecs using the ffmpeg include/library files you just built. Then build ffmpeg a second time with the extra codecs that were just built. The second problem is that neither new nor old mplayer builds work on a modern Linux, but that's a topic for another day... Here's my current third-party build script: Code:
#!/bin/bash # # Copyright 2015 The SageTV Authors. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. set -o nounset set -o errexit trap 'echo "Build failure at line ${LINENO}"; exit 1' ERR Build=$PWD ThirdParty=$(readlink -f ../third_party) set -x # We need ffmpeg libraries in order to build at least one of the # codecs (x264), so build a preliminary version of ffmpeg without the # extra codecs. After we've built the codecs, we'll rebuild ffmpeg # with the extra codecs cd $ThirdParty/ffmpeg make clean ./configure --disable-{ffserver,ffplay,bzlib,devices,libfaac,libfaad,libx264,libxvid} \ --enable-{gpl,pthreads,nonfree,shared} make # Build the external codecs cd $ThirdParty/codecs/faac ./bootstrap ./configure CFLAGS=-fno-common --enable-static --disable-shared --without-mp4v2 make cd $ThirdParty/codecs/faad2 ./bootstrap ./configure CFLAGS=-fno-common --without-{xmms,mpeg4ip,drm} make || true # for some reason we ignore the make failure for this one :/ cd $ThirdParty/codecs/x264 ./configure --extra-cflags="-fasm -fno-common -D_FILE_OFFSET_BITS=64 -I$ThirdParty/ffmpeg" \ --extra-ldflags="$(echo -L$ThirdParty/ffmpeg/lib{avformat,avcodec,swscale})" make cd $ThirdParty/codecs/xvidcore/build/generic ./bootstrap.sh ./configure CFLAGS=-fno-common make # Build FFMPEG again, including the codecs we just built cd $ThirdParty/ffmpeg make clean ./configure --disable-{ffserver,ffplay,devices,bzlib,demuxer=msnwc_tcp} \ --enable-{gpl,pthreads,nonfree,libfaac,libx264,libxvid,libfaad} \ --extra-cflags="-I. $(echo -I$ThirdParty/codecs/{faad2/include,faac/include,x264,xvidcore/src})" \ --extra-ldflags="$(echo -L$ThirdParty/codecs/{faac/libfaac/.libs,faad2/libfaad/.libs,x264,xvidcore/build/generic/=build})" make cd $Build # If MPLAYER_NEW=1 is set, then the newer mplayer will be build MPLAYER_NEW=0 ./buildmplayer.sh # Copy the files to the release folder mkdir -p elf cp $ThirdParty/ffmpeg/ffmpeg elf/ cp $ThirdParty/codecs/jpeg-6b/jpegtran elf/ maintain and help locate failures. It now uses bash. When sagetv will only build on one specific version of a particular Linux distro (which definitely has bash), I don't think it's very important that the build script be compatible with a shell from v7 Unix on a PDP-11 in 1978. ![]() Changes I've made include the following:
Everything except mplayer now builds on a modern Linux distro. As with the "so" build, I don't yet know if the thirdpary binaries I've built so far work. I'll worry about that after I've got mplayer to build on a modern system. Would changes like these be acceptable to the sagetv maintainers? -- Grant |
#2
|
||||
|
||||
ffmpeg and mplayer in the code are so stale that there's not really a point in building them over and over again. That said, are you using gradle to do your build?
__________________
Buy Fuzzy a beer! (Fuzzy likes beer) unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers. Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA. Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S Other Clients: Mi Box in Master Bedroom, HD-200 in kids room |
#3
|
||||
|
||||
You can certainly submit a pull request with your changes, and that way, they can be reviewed to see if there are other side effects.
The existing scripts obviously does work, as is, since it's used create the linux builds. I typically do them using a docker environment. The failure of mplayer on a newer linux has to do with the compiler... i think other people have solved that as well, but I haven't see a pull request for it either ![]()
__________________
Batch Metadata Tools (User Guides) - SageTV App (Android) - SageTV Plex Channel - My Other Android Apps - sagex-api wrappers - Google+ - Phoenix Renamer Downloads SageTV V9 | Android MiniClient |
#4
|
|||
|
|||
Quote:
Quote:
|
#5
|
||||
|
||||
That's likely a java version issue... did you try installing OpenJDK8 or OracleJDK8? (even if you build using Java 8, the target is still java 1.7, so it won't force you to use Java 8)
__________________
Batch Metadata Tools (User Guides) - SageTV App (Android) - SageTV Plex Channel - My Other Android Apps - sagex-api wrappers - Google+ - Phoenix Renamer Downloads SageTV V9 | Android MiniClient |
#6
|
||||
|
||||
Quote:
a pull request. But, if that's the only way they get looked at, then I'll do that. Quote:
combinations. According to the forums it won't work on Ubuntu newer than 2014, and I know it doesn't work on an up-to-date Gentoo system. From what I've seen, that's due to dependencies on particular library versions. That's the first thing my script fixes: it no longer depends on the build host's ffmpeg library. [quote]I typically do them using a docker environment. Does that build a "regular" install that will run on a wide range of Linux distros? Or is it building something that must be run via docker? Quote:
into have been library versions dependencies. The old version requires old png and gif libraries. The work-around for that is to actually build the versions requited instead of expecting to find them already installed on the host. I've done that for libpng, but haven't done it for libgif. Except for the libgif version dependency, I've gotten the old mplayer to build on an up-to-date amd64 Gentoo system using gcc 4.9.4. There's always a chance that after I fix the libgif version dependency issue, the resulting mplayer binary won't [i]work[\i] due to compiler issues, but so far I haven't run into any build issues having to do with the compiler. For me, the "new" mplayer build fails because the sagetv git mplayer sources aren't compatible with the sagetv git ffmpeg sources. The ffmpeg include files define the amd64 register names as "FF_REG_a" "FF_REG_b" and so on. The mplayer sources expect them to be defined as "REG_a", "REG_b" etc. Quote:
The server install doesn't use it, and that's my primary concern. The Linux client only seems to need it for video playback, and I don't care about that: I only use the client for configuration purposes so I don't have to navigate menus and enter strings on the HD200 remote. |
#7
|
|||
|
|||
Quote:
Code:
openjdk version "1.8.0_111" OpenJDK Runtime Environment (IcedTea 3.2.0) (Gentoo icedtea-3.2.0) OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode) Quote:
![]() That's why I gave up trying to build the .jar files. I thought that the downloaded ones should be portable enough to be independent of the Linux distro being used. |
#8
|
||||
|
||||
The docker container for building SageTV is really there to solve the compiler issue (for mplayer only) when trying to build on a newer OS. The docker container uses Ubuntu 14.04, if I recall, and the resulting images (.tgz and .deb) should run on newer linux distros. ie, I run it on the absolute latest version of ubuntu.
Once SageTV is built, the native parts don't rely on the OS libraries, from what I can tell, with the exception of sound libraries in mplayer, and maybe the freetype libraries. I was pretty sure the ffmpeg build didn't really have any OS dependencies... I just ran ldd on ffmpeg... ie does have a couple of shared object dependencies, but other than faad the others look pretty standard. Code:
linux-vdso.so.1 => (0x00007ffce23e4000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd438545000) libfaad.so.2 => /usr/lib/x86_64-linux-gnu/libfaad.so.2 (0x00007fd438304000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd4380e9000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd437ecc000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd437b03000) /lib64/ld-linux-x86-64.so.2 (0x000055bdc03b3000) Code:
linux-vdso.so.1 => (0x00007ffe4f2b7000) libaudio.so.2 => /usr/lib/x86_64-linux-gnu/libaudio.so.2 (0x00007f74647ae000) libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007f7464589000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f746436e000) libasound.so.2 => /usr/lib/x86_64-linux-gnu/libasound.so.2 (0x00007f746406e000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f7463e6a000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7463b60000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7463943000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f746357a000) libXt.so.6 => /usr/lib/x86_64-linux-gnu/libXt.so.6 (0x00007f7463310000) libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f746310c000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f7462f04000) /lib64/ld-linux-x86-64.so.2 (0x0000555eea7d2000) libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6 (0x00007f7462cfb000) libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6 (0x00007f7462ae1000) libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f74627a7000) libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f74625a1000) libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f746237f000) libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f7462178000) And the Docker container to build sagetv is independent of the Docker containers that are used to run sagetv and there isn't any dependencies between them. I just use Docker for building sagetv because it quickly creates a "clean" room for me to do the build.
__________________
Batch Metadata Tools (User Guides) - SageTV App (Android) - SageTV Plex Channel - My Other Android Apps - sagex-api wrappers - Google+ - Phoenix Renamer Downloads SageTV V9 | Android MiniClient |
#9
|
||||||
|
||||||
Quote:
possibly the xvid one, I don't remember) that had problems. They required that the build host's native ffmpeg library headers were from the same version as was going to be built later in the build. That's wrong. My build script solved that problem by building a minimal ffmpeg first, and then using that to build the codecs. Then the final ffmpeg build incorporated those codecs (statically linked). Quote:
build-time dependencies. Are there customizations to ffmpeg that require it be built specially for sagetv? Quote:
build-time dependencies. The two that I ran into were libpng and libgif. My system has a .so for libpng12, but no header files. And the libgif version my system has is about 3 major versions too new. That said, I was able to resolve all of the libraries for the mplayer in the binary distro, and playback produced the proper audio and control UI, but I never managed to get any picture. Quote:
versions required are old. So old, they aren't available as packages under Gentoo. Quote:
two different Docker containers being discussed. While building things in a Docker container may be easy it doesn't give you the guaranteed library version/configuration compatibility that a Gentoo user takes for granted. All the other binaries on my system were built with the exact library version and configuration that I have. Running binaries built on a different system makes a Gentoo user feel unsafe. ![]() Quote:
in a clean room. You run binaries built in a dirty room, but it's the exact room in which they're going to be used. But, getting over that is my problem. I've been running foreign sagetv binaries for 8 years because I had no choice. Now that sagetv is open-source, I thought I might have a choice. Since I've got everything built except mplayer, I should be able to do a proper Gentoo server install (except with downloaded .jar files). I'm not brave enough to tackle the gradle failures yet. |
#10
|
||||
|
||||
I ran Gentoo for a few years, so I'm somewhat familiar with how it works. I stopped using it mainly because as I got older I become less patient, and I didn't want to wait for things to build.
Quote:
Mplayer doesn't use the ffmpeg that sagetv builds, because that version of mplayer actually embeds it's own ffmpeg libraries directly. Both ffmpeg, and mplayer contain sagetv specific modifications. And yeah, those are old, but unless someone makes the effort the find the changes, and then move them into a newer build, then likely we'll be stuck there ![]() Glad you have it working... but it does seem those challenges likely only exist in gentoo ![]()
__________________
Batch Metadata Tools (User Guides) - SageTV App (Android) - SageTV Plex Channel - My Other Android Apps - sagex-api wrappers - Google+ - Phoenix Renamer Downloads SageTV V9 | Android MiniClient |
#11
|
||||
|
||||
This is the first thread I've seen in awhile about problems building the source on Linux...so I would be somewhat hesitant to change the build scripts. However, the changes you made to the build script don't look like they'd actually cause any problems and just make it more compatible (although I'd want it to do parallel building again because it just takes too long otherwise). Feel free to submit a pull request on it, I'll have my Linux expert take a look at it.
![]()
__________________
Jeffrey Kardatzke Founder of SageTV |
![]() |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Start up Script error ?? | jbmia | SageTV Linux | 2 | 08-09-2011 06:43 AM |
Need a script to reboot MVP | deanm | SageTV Media Extender | 13 | 07-25-2008 01:32 AM |
What Is Currently Recording Script | joe123 | SageTV Customizations | 18 | 02-05-2007 09:40 PM |
init script | n8willis | SageTV Linux | 7 | 01-15-2007 03:33 PM |
XBMC Script | Coolwave | SageTV Customizations | 18 | 03-01-2006 01:44 PM |