Linux 64bit MiniClient (issues with JTux on 64bit)

[stuckless]

I did get MiniClient recompiled and I updated it with 64bit jogl libraries, but, when I start the miniclient and select a server it dies immediatly in the JTux code...

Sat 8/15 6:47:29.819 Connected to gfx server
Sat 8/15 6:47:29.921 Testing to see if server can do a pull mode streaming connection...
Sat 8/15 6:47:29.922 Loaded
Type size error: sizeof(jint) < sizeof(size_t)
Type size error: sizeof(jint) < sizeof(ssize_t)
Type size error: sizeof(jint) < sizeof(ino_t)
Type size error: sizeof(jint) < sizeof(nlink_t)
Type size error: sizeof(jint) < sizeof(blksize_t)
Type size error: sizeof(jint) < sizeof(nfds_t)
Type size error: sizeof(jint) < sizeof(msgqnum_t)
Type size error: sizeof(jint) < sizeof(msglen_t)
Type size error: sizeof(int) < sizeof(ino_t)
Jtux error: Java native types don't match POSIX/SUS types.
Sat 8/15 6:47:29.923 Server can do a pull-mode streaming connection.
Sat 8/15 6:47:29.926 Max screen size=java.awt.Dimension[width=1920,height=1200]
Sat 8/15 6:47:29.926 Error w/ Media Thread: java.net.SocketException: Socket closed
Sat 8/15 6:47:29.926 Error w/ GFX Thread: java.net.SocketException: Socket is closed
java.net.SocketException: Socket closed

In reading the JTux site, it makes reference to doing the checks...

Primitive Types

Java has only eight primitive types, whereas C has an unlimited number via the typedef facility. For a type like, say, pid_t in C, one can go two ways:

bullet Use a Java primitive type, such as int.
bullet Define a Java class named pid_t.
Jtux uses the former approach so that calling methods and dealing with fields in structures can be as much like C as possible. During start up, Jtux checks that each of the primitive types it uses as a typedef-type replacement is wide enough for the job.

So, I decided to take off the seatbelt, and I commented out the code inside of

JNIEXPORT void JNICALL Java_jtux_UUtil_check_1type_1sizes(JNIEnv *env, jclass obj)

After rebuilding, I can now launch the MiniClient on Linux in 64bit, and connect to server. It appears to work.

The code does an extensive job checking sizes, and I'm not sure just commented it out is the right approach

	if (sizeof(jlong) < sizeof(long)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(long)\n");
	}
	if (sizeof(jlong) < sizeof(void *)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(void *)\n");
	}
	if (sizeof(jint) < sizeof(int)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(int)\n");
	}
	if (sizeof(jshort) < sizeof(short)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jshort) < sizeof(short)\n");
	}
	if (sizeof(jlong) < sizeof(clock_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(clock_t)\n");
	}
	if (sizeof(jlong) < sizeof(gid_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(gid_t)\n");
	}
	if (sizeof(jlong) < sizeof(pid_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(pid_t)\n");
	}
	if (sizeof(jlong) < sizeof(uid_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(uid_t)\n");
	}
	if (sizeof(jlong) < sizeof(rlim_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(rlim_t)\n");
	}
	if (sizeof(jlong) < sizeof(time_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(time_t)\n");
	}
	if (sizeof(jlong) < sizeof(suseconds_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(suseconds_t)\n");
	}
	if (sizeof(jlong) < sizeof(useconds_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(useconds_t)\n");
	}
	if (sizeof(jlong) < sizeof(off_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(off_t)\n");
	}
	if (sizeof(jlong) < sizeof(fsfilcnt_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(fsfilcnt_t)\n");
	}
	if (sizeof(jlong) < sizeof(fsblkcnt_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(fsblkcnt_t)\n");
	}
	if (sizeof(jint) < sizeof(size_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(size_t)\n");
	}
	if (sizeof(jint) < sizeof(ssize_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(ssize_t)\n");
	}
	if (sizeof(jint) < sizeof(mode_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(mode_t)\n");
	}
	if (sizeof(jlong) < sizeof(dev_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(dev_t)\n");
	}
	if (sizeof(jint) < sizeof(ino_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(ino_t)\n");
	}
	if (sizeof(jint) < sizeof(nlink_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(nlink_t)\n");
	}
	if (sizeof(jint) < sizeof(blksize_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(blksize_t)\n");
	}
	if (sizeof(jlong) < sizeof(blkcnt_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(blkcnt_t)\n");
	}
	if (sizeof(jint) < sizeof(nfds_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(nfds_t)\n");
	}
	if (sizeof(jlong) < sizeof(key_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(key_t)\n");
	}
	if (sizeof(jint) < sizeof(msgqnum_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(msgqnum_t)\n");
	}
	if (sizeof(jint) < sizeof(msglen_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(msglen_t)\n");
	}
	if (sizeof(jlong) < sizeof(mqd_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jlong) < sizeof(mqd_t)\n");
	}
	if (sizeof(jint) < sizeof(socklen_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(socklen_t)\n");
	}
	if (sizeof(jint) < sizeof(sa_family_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(sa_family_t)\n");
	}
	if (sizeof(jshort) < sizeof(in_port_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jshort) < sizeof(in_port_t)\n");
	}
	if (sizeof(jint) < sizeof(in_addr_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(jint) < sizeof(in_addr_t)\n");
	}
	/*
		Assuming C ints (not only jints) are at least 32-bits, because of the int
		argument to field_ctoj_int.
	*/
	if (sizeof(int) < sizeof(in_addr_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(int) < sizeof(in_addr_t)\n");
	}
	if (sizeof(int) < sizeof(ino_t)) {
		ok = false;
		fprintf(stderr, "Type size error: sizeof(int) < sizeof(ino_t)\n");
	}
	if (!ok)
		setup_throw_errno(env, ENOSYS);

In digging through the Java Documentation, i think I know why this is failing, but not 100% sure how we should fix it.

Primitive Types and Native Equivalents
Java Type Native Type Description
boolean jboolean unsigned 8 bits
byte jbyte signed 8 bits
char jchar unsigned 16 bits
short jshort signed 16 bits
int jint signed 32 bits
long jlong signed 64 bits
float jfloat 32 bits
double jdouble 64 bits
void void not applicable

It appears that on 64bit systems jint 32bits and size_t is 64bits, whereas on 32bit systems they are the same size. This is odd, since the C spec states that size_t should be size of unsigned int, which it would expect to be 4 as well.

sizeof(jint): 4, sizeof(size_t): 8 
sizeof(int): 4, sizeof(uint): 4

Given that JTux is used quite a bit in SageTV, I'm not sure what the impact of this is going to be. ie, I'm not sure that just commenting out the code will be a the right choice

[jreichen]

MrD reported the same thing over on your "Build SageTV (Linux Specific) + GitHub primer" thread. He hadn't replaced any 32-bit libraries so you might double check that on your installation.

[routerunner]

Originally Posted by stuckless

This is odd, since the C spec states that size_t should be size of unsigned int, which it would expect to be 4 as well.

sizeof(jint): 4, sizeof(size_t): 8 
sizeof(int): 4, sizeof(uint): 4

The compiler should try to match the size of the integers to the maximum size of registers for that particular architecture, but unfortunately is not always the case, but regardless the architecture:

char is always 1 byte
short is always 2 bytes
long is always 4 bytes
long long is always 8 bytes

int must be at least the size of a short, so can be 2/4/8 bytes depending on compiler/architecture.

Eddy

[jm9843]

Originally Posted by stuckless

It appears that on 64bit systems jint 32bits and size_t is 64bits, whereas on 32bit systems they are the same size. This is odd, since the C spec states that size_t should be size of unsigned int, which it would expect to be 4 as well.

sizeof(jint): 4, sizeof(size_t): 8 
sizeof(int): 4, sizeof(uint): 4

I think that this offers a pretty good explanation.

Originally Posted by stuckless

I'm not sure that just commenting out the code will be a the right choice

Yeah, probably not.

[stuckless]

Again this isn't a problem with the sizeof(int) or sizeof(size_t), per se, but rather that JTux made some assumptions that sizeof(jint) == sizeof(size_t).

jint is NOT a std C type but a java type.

[stuckless]

Originally Posted by jreichen

MrD reported the same thing over on your "Build SageTV (Linux Specific) + GitHub primer" thread. He hadn't replaced any 32-bit libraries so you might double check that on your installation.

I'm running a full 64bit installation with no 32bit software installed. All my .so files are elf 64bit (file *.so report all 64bit), and the java jvm is 64bit. On a 32bit install JTux would be fine, since it size_t==jint.

The good news is that on the server, JTux is not used very often, but on the Miniclient it's used quite a bit.

[Fuzzy]

Out of curiosity, what is the motivator/target use for compiling a 64 bit miniclient?

[jreichen]

From what I can tell, jtux was written about 10-12 years ago so maybe it wasn't originally 64-bit compatible. There are a couple of "forks" (the original wasn't on github) but they both have the same check for jint/size_t:

https://github.com/BanjoPicker/jtux
https://github.com/swenson/jtux

During some earlier searches I was doing, it looked like jint is always 32-bit, and size_t varies with the platform. However, jni_md.h defines jint as an int. To me that means it should also vary with the size of the platform.

[Fuzzy]

if jtux is so old, should it maybe be considered obsolete and a new solution be found? What purposes is it used for in miniclient that couldn't be done with more agnostic java library calls?

[stuckless]

Originally Posted by Fuzzy

Out of curiosity, what is the motivator/target use for compiling a 64 bit miniclient?

There isn't a need for it... just that all my systems are 64bit. (Yes, i can run a 32bit miniclient)

Originally Posted by Fuzzy

if jtux is so old, should it maybe be considered obsolete and a new solution be found? What purposes is it used for in miniclient that couldn't be done with more agnostic java library calls?

JTux is only used in the Linux pieces, and from what I can it is handling some socket stuff, file io, etc, which could possibly be done using standard java, so I'm not 100% sure what it's needed, other than maybe it's "faster".

JTux is also used on the server code, but doesn't appear to be used quite as extensively.

Maybe Jeff will weigh in

[Fuzzy]

I'm betting it was used as a stop gap to have better socket control than what java 1.4 offered. I'm guessing 1.6 has some better options available, and certainly better performance. I would think in miniclient, which by its function, is likely to be the most portable of the bits of code, we should stick to as much core java api's as possible, and avoid OS specific calls, if possible.

[jreichen]

stuckless, I was looking at the JOGL upgrade and they changed a bunch of package and class names. For instance javax.media.opengl is now com.jogamp.opengl, and the GLPbuffer object no longer exists. That's as far as I got

So far I've been working on the Linux builds and haven't needed a Java IDE. Looks like it's time to work on that.

Jeff, any thoughts on the JOGL API changes?

[stuckless]

Yeah, if you look at my build.gradle, you'll see I've been playing around with pulling in the updated JOGL... and yeah, there are lots of changes. It appears that there are more than just package name changes as well

I tried all the 2.0x versions and they all have issues. So I think the version that sage is using is either a very old 2x RC or a 1x version.

[Narflex]

Originally Posted by stuckless

JTux is only used in the Linux pieces, and from what I can it is handling some socket stuff, file io, etc, which could possibly be done using standard java, so I'm not 100% sure what it's needed, other than maybe it's "faster".

JTux is also used on the server code, but doesn't appear to be used quite as extensively.

Maybe Jeff will weigh in

In the SageTV core...there's only one thing I still see JTUX being used for...and that's IOUtils.getUSBHWID (which itself, is never used anymore...it was for HW identification for the SageTV on a stick version)

So it should be fine to remove the jtux dependency altogether from the SageTV core.

In the miniclient code, it looks like there's 2 things it was doing....both are because it needed to use Unix domain sockets rather than IP sockets. One of them is for LIRC integration...the other looks like its for shared memory access for video rendering.

Originally Posted by jreichen

stuckless, I was looking at the JOGL upgrade and they changed a bunch of package and class names. For instance javax.media.opengl is now com.jogamp.opengl, and the GLPbuffer object no longer exists. That's as far as I got

So far I've been working on the Linux builds and haven't needed a Java IDE. Looks like it's time to work on that.

Jeff, any thoughts on the JOGL API changes?

I haven't looked at JOGL in a long time....feel free to update it to a more recent version, as long as you fix all the things that it breaks at the same time. I have no idea how complex that update is though.

[stuckless]

Originally Posted by Narflex

I haven't looked at JOGL in a long time....feel free to update it to a more recent version, as long as you fix all the things that it breaks at the same time. I have no idea how complex that update is though.

Jeff, I noticed on my system that I was able to run the MiniClient without JOGL... so I'm guessing that it is selectively used? Is there some advantage to using JOGL over standard java implementation, which you've done as well?

[Narflex]

Originally Posted by stuckless

Jeff, I noticed on my system that I was able to run the MiniClient without JOGL... so I'm guessing that it is selectively used? Is there some advantage to using JOGL over standard java implementation, which you've done as well?

If you turn off 3D acceleration, then it will not use OpenGL....which of course means that rendering performance is not as good as one would expect. I believe that also then requires support for a video overlay surface in the graphics card...and those went out of style quite awhile ago, especially with Liux drivers....which is why we added OpenGL rendering to the MiniClient so we could support modern graphics cards on Linux.

[stuckless]

Originally Posted by Narflex

If you turn off 3D acceleration, then it will not use OpenGL....which of course means that rendering performance is not as good as one would expect. I believe that also then requires support for a video overlay surface in the graphics card...and those went out of style quite awhile ago, especially with Liux drivers....which is why we added OpenGL rendering to the MiniClient so we could support modern graphics cards on Linux.

And I assume that 3d acceleration is off by default? I'll check it on and see what happens The UI feels pretty snappy without JOGL, but I can't actually get video playback working yet, so can't really say how that works.

[Narflex]

Originally Posted by stuckless

And I assume that 3d acceleration is off by default? I'll check it on and see what happens The UI feels pretty snappy without JOGL, but I can't actually get video playback working yet, so can't really say how that works.

I *thought* it was on by default...the source code will tell you for sure.

[Narflex]

Yeah, it defaults to 3D acceleration on by default:

    else if ("true".equals(MiniClient.myProperties.getProperty("opengl", "true")))
    {
      if (MiniClient.WINDOWS_OS)
      {
        myGfx = new DirectX9GFXCMD(this);
      }
      else
      {
        try {
          Class[] params = {this.getClass()};
          Object[] args = {this};
          java.lang.reflect.Constructor ctor = Class.forName("sage.miniclient.OpenGLGFXCMD").getConstructor(params);
          myGfx = (GFXCMD2) ctor.newInstance(args);
        }
        catch (Throwable t) {
          System.out.println("Error loading OpenGLGFXCMD class, reverting to default rendering:" + t);
          t.printStackTrace();
          myGfx = new GFXCMD2(this);
        }
      }
    }

But if JOGL fails to load...it'll just fall back to Java.

[stuckless]

Originally Posted by Narflex

Yeah, it defaults to 3D acceleration on by default:


But if JOGL fails to load...it'll just fall back to Java.

Well, that makes sense... I'm testing on 64bit machine without 64bit jogl at the moment.


Side question related to jtux and 64bit. I'm considering pulling JTux because of the 64bit issues that I found. I could probably spend the time trying to see if there really is a reason why JTux is very explicit about its checks, and I have done some work in that direction, but I now leaning towards just using

https://github.com/kohlschutter/junixsocket

They support linux/mac/arm and they have 32bit and 64bit packages. The other nice thing is that you only need to include the jar and the natives-jar in the classpath and it will auto-resolve/load the system dependant .so files.

Any thoughts, objections?

This doesn't cover the usb portion, but you said that wasn't used, so I was going to remove it... or find a recent libusb for java.