Plugin: Sage Job Queue (SJQ) v3

[Slugger]

Originally Posted by HuMan321

Thanks!

eric over at comskip had me drop a h.264 file onto comskip.exe to see if it generated an edl file. It did and then was self deleted in about 10 minutes or so. I have the SJQ file cleaner totally empty and I cannot figure out why my h.264 log and edl files are being deleted automatically. I have now stopped sjq altogether and running another test to see if that yields a clue.
Any other thoughts on what could be autodeleteting edl files on my system for only h.264 files? The edl files are not deleted on mpg files...

If comskip is failing/crashing then it may delete its output files as a cleanup mechanism. If SJQ is the one deleting those files then the sjq logs will clearly tell you that (all file deletions from the SJQ file cleaner are logged explicitly).

[ChaOConnor]

Originally Posted by Slugger

In the dir where the task client was installed should be a subdir called logs. In there should be a file called wrapper.log. That file should give a better clue as to why the task client is failing (typically it's because it can't find java or it can't read/write the sjqc.properties file).

This is what it's showing, that's where I got the 403 error from. When I put the URL in question in my browser, I get the "NotAuthorized".

STATUS | wrapper  | 2010/04/17 20:41:07 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/04/17 20:41:07 | Java Service Wrapper Community Edition 3.3.1
STATUS | wrapper  | 2010/04/17 20:41:07 |   Copyright (C) 1999-2008 Tanuki Software, Inc.  All Rights Reserved.
STATUS | wrapper  | 2010/04/17 20:41:07 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/04/17 20:41:07 | 
STATUS | wrapper  | 2010/04/17 20:41:08 | Launching a JVM...
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperManager: Initializing...
INFO   | jvm 1    | 2010/04/17 20:41:08 | SJQ Task Client v3.0.2.771
INFO   | jvm 1    | 2010/04/17 20:41:08 | Using 'C:/sjqc' as location for configuration file...
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: Encountered an error running main:
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: java.lang.RuntimeException: java.io.IOException: Server returned HTTP response code: 403 for URL: http://rebel-hq.home:8080/sjq/SJQ?proto=4&cmd=writeLog
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at com.google.code.sagetvaddons.sjqc.TaskClient.log(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at com.google.code.sagetvaddons.sjqc.TaskClient.log(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at com.google.code.sagetvaddons.sjqc.TaskClient.pop(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at com.google.code.sagetvaddons.sjqc.TaskClient.run(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at com.google.code.sagetvaddons.sjqc.TaskClient.main(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at java.lang.reflect.Method.invoke(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:236)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at java.lang.Thread.run(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://rebel-hq.home:8080/sjq/SJQ?proto=4&cmd=writeLog
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
INFO   | jvm 1    | 2010/04/17 20:41:08 | WrapperSimpleApp: 	... 11 more
STATUS | wrapper  | 2010/04/17 20:41:10 | <-- Wrapper Stopped

[Slugger]

HTTP 403 means the SJQ server is rejecting the task client IP address. Check your "task client restrictions" settings in the SJQ GUI to ensure that the IP address of the this task client is being allowed to connect.

[davephan]

I tried installing the SJQC service on a second computer. SageTV is not installed on the second computer. The reasoning is to try to offload the Comskip processing to a second computer.

When I try to start the SJQ Task Client for Windows service, the service fails to start. The error message is "Could not start the SJQ Task Client for Windows service on local computer error 1067. The process terminated unexpectedly.

I searched Google for "Windows service startup error 1067", which basically says it could be a MySQL problem.

To run the SJQC, is the Jetty a requirement to be on the same computer? For the Jetty to be installed, is SageTV a requirement?

I am trying to determine if it is possible to install only the SJQ client on the second computer so the Comskip processing can be done on the second computer, to remove the load from the SageTV computer. Some load would still be imposed on the SageTV server since the Comskip processing would be done by the second computer, but the Comskip processing would be done with a mapped drive to the SageTV computer, which would cause some loading to the SageTV computer, but I suspect less loading.

Dave

[ChaOConnor]

Originally Posted by Slugger

HTTP 403 means the SJQ server is rejecting the task client IP address. Check your "task client restrictions" settings in the SJQ GUI to ensure that the IP address of the this task client is being allowed to connect.

You are the man, thank you! I don't know how, but my static routing table for my PCs got changed on the FiOS router. I wonder if it was reset somehow while I was on business. Thanks again!

[Slugger]

Originally Posted by davephan

I tried installing the SJQC service on a second computer. SageTV is not installed on the second computer. The reasoning is to try to offload the Comskip processing to a second computer.

When I try to start the SJQ Task Client for Windows service, the service fails to start. The error message is "Could not start the SJQ Task Client for Windows service on local computer error 1067. The process terminated unexpectedly.

I searched Google for "Windows service startup error 1067", which basically says it could be a MySQL problem.

To run the SJQC, is the Jetty a requirement to be on the same computer? For the Jetty to be installed, is SageTV a requirement?

I am trying to determine if it is possible to install only the SJQ client on the second computer so the Comskip processing can be done on the second computer, to remove the load from the SageTV computer. Some load would still be imposed on the SageTV server since the Comskip processing would be done by the second computer, but the Comskip processing would be done with a mapped drive to the SageTV computer, which would cause some loading to the SageTV computer, but I suspect less loading.

Dave

Yes, the task client can be installed standalone on other computers. It does not require SageTV nor Jetty be installed, but it does, of course, require Java. The task client is a standalone app.

If it's failing to start then go to the dir where you installed the task client. In there should be a subdir called logs. In there should be a file called wrapper.log. That file will give you the exact details as to why the service failed to start.

[davephan]

The log wrapper log file said that the computer could start Java. The computer didn't have Java installed, so I installed Java, and it got farther. I tried starting the SJQC service after installing Java, and the SJQC service started. However, the client did not connect and it seems that the connection was refused according to the last line of the log file. There is a client on the SageTV computer that does connect with the local address 127.0.0.1. Is there somewhere where I would enter the IP address for the second computer so that the SJQ would know the IP of the second client?

Dave

STATUS | wrapper  | 2010/04/18 04:10:25 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/04/18 04:10:25 | Java Service Wrapper Community Edition 3.3.1
STATUS | wrapper  | 2010/04/18 04:10:25 |   Copyright (C) 1999-2008 Tanuki Software, Inc.  All Rights Reserved.
STATUS | wrapper  | 2010/04/18 04:10:25 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/04/18 04:10:25 | 
STATUS | wrapper  | 2010/04/18 04:10:25 | Launching a JVM...
FATAL  | wrapper  | 2010/04/18 04:10:25 | Unable to execute Java command.  The system cannot find the file specified. (0x2)
FATAL  | wrapper  | 2010/04/18 04:10:25 |     "java" -Djava.library.path="../lib" -classpath "../lib/wrappertest.jar;../lib/wrapper.jar;sjqc.jar;json.jar" -Dwrapper.key="MhsXq3Lp8WJiauH6" -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.pid=2784 -Dwrapper.version="3.3.1" -Dwrapper.native_library="wrapper" -Dwrapper.service="TRUE" -Dwrapper.cpu.timeout="10" -Dwrapper.jvmid=1 org.tanukisoftware.wrapper.WrapperSimpleApp com.google.code.sagetvaddons.sjqc.TaskClient
FATAL  | wrapper  | 2010/04/18 04:10:25 | Critical error: wait for JVM process failed
STATUS | wrapper  | 2010/04/18 04:11:29 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/04/18 04:11:29 | Java Service Wrapper Community Edition 3.3.1
STATUS | wrapper  | 2010/04/18 04:11:29 |   Copyright (C) 1999-2008 Tanuki Software, Inc.  All Rights Reserved.
STATUS | wrapper  | 2010/04/18 04:11:29 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/04/18 04:11:29 | 
STATUS | wrapper  | 2010/04/18 04:11:30 | Launching a JVM...
FATAL  | wrapper  | 2010/04/18 04:11:30 | Unable to execute Java command.  The system cannot find the file specified. (0x2)
FATAL  | wrapper  | 2010/04/18 04:11:30 |     "java" -Djava.library.path="../lib" -classpath "../lib/wrappertest.jar;../lib/wrapper.jar;sjqc.jar;json.jar" -Dwrapper.key="MCUFTSWBUJqn5jIw" -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.pid=2768 -Dwrapper.version="3.3.1" -Dwrapper.native_library="wrapper" -Dwrapper.service="TRUE" -Dwrapper.cpu.timeout="10" -Dwrapper.jvmid=1 org.tanukisoftware.wrapper.WrapperSimpleApp com.google.code.sagetvaddons.sjqc.TaskClient
FATAL  | wrapper  | 2010/04/18 04:11:30 | Critical error: wait for JVM process failed
STATUS | wrapper  | 2010/04/18 09:05:56 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/04/18 09:05:56 | Java Service Wrapper Community Edition 3.3.1
STATUS | wrapper  | 2010/04/18 09:05:56 |   Copyright (C) 1999-2008 Tanuki Software, Inc.  All Rights Reserved.
STATUS | wrapper  | 2010/04/18 09:05:56 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/04/18 09:05:56 | 
STATUS | wrapper  | 2010/04/18 09:05:57 | Launching a JVM...
INFO   | jvm 1    | 2010/04/18 09:05:57 | WrapperManager: Initializing...
INFO   | jvm 1    | 2010/04/18 09:05:57 | SJQ Task Client v3.0.1.705
INFO   | jvm 1    | 2010/04/18 09:05:57 | Using 'C:/sjqc' as location for configuration file...
INFO   | jvm 1    | 2010/04/18 09:05:58 | Connection refused: connect

[nyplayer]

Dave,

When using a remote client .... on that client... you have to change the 127.0.0.1 to the IP of the server(or server name) and the Jetty port. I do not use SJQ in service mode but I believe there should be a properties file.

[KarylFStein]

I'd check the c:\sjqc\sjqc.properties file on the second computer to make sure it's pointing at the SJQ computer, and that the SJQ computer is configured (Edit Settings on the web page) to allow connections from the second computer (Task client restrictions).

Note that many routers will rewrite the source IP of inbound Internet traffic to the router's local LAN IP. If you do something like 192.168.0.0/24;127.0.0.1/32 for task client restrictions, you may still be allowing external traffic to reach SJQ. That's why I use specific IPs there e.g. 192.168.0.123/32;127.0.0.1/32.

[Slugger]

KarylFStein has it right. Check the sjqc.properties file in c:\sjqc\ and ensure that the IP address in that file is set to the IP address where the SJQ server is running. By default, it will use localhost (127.0.0.1), but since you're running secondary clients, these clients need to point to the actual SJQ server IP address on your LAN.

[HuMan321]

Originally Posted by HuMan321

I am now running SJQ 3.0.2.790 with the hopes of fixing my issue with Comskip not generating edl files on h.264 files. I am running the latest version of "donator" comskip and been working with erik on his forum. erik so far thinks comskip is working properly. I have output_edl=1 setting (always did)

SJQ and comskip work fine on mpg files. When they run on h.264 files it appears to work as .log files are created and an .edl is created and as soon as it is done the edl and log files disappear with only a .incommercial file remaining on all these files. I have been running 2 cores on files over the past couple of days and am getting nowhere.
Does this sound familiar to anyone?
Any comments are appreciated.

Ruleset:
if [IsObjMediaFile == true && $.IsTVFile{} == true && FileExists != "%d%/%p%.edl" && $.IsNotManualOrFavorite{} == false && $.GetAiringChannelName{} !% "CSPAN.*|ACMAX.*|KAIDDT.*|STARZ.*|STZ.*|HBO.*|.*MAX.*|SHO.*|TMC.*|PPV\\d+"] {

COMSKIP
}

Client:
COMSKIP {
:RESOURCES 50 # Up to 4 comskips can be run simultaneously (every client has exactly 100 resources available)

# CPU priority to be used for this task process; one of LOW, NORMAL, HIGH (case sensitive)
:CPU "NORMAL" # Default is NORMAL if not specified

# Comskip returns 1 on success instead of the universally accepted zero; tell SJQ that
# a return code of 1 means success instead of the default value of zero
:RETURNCODE 1

"C:/Comskip/comskip.exe \"%c%\""
}

I believe I have found out my issue. I will post in case anyone else finds theirselves in a similar mess. I had an old version of Comskip Monitor that I had edited to not delete any edl, log, etc files. After doing an uninstall on this program my edl files now do not disappear. I am not sure why CM was deleting the files, but am glad to have my SJQ/Comskip running properly.

Thanks for all the help on this.

[davephan]

I get a connection timeout error in the wrapper.log on the SJQ computer.

I have the c:\sjqc\sjqc.properties:
host=<IP address for Comskip computer>

The SJQ computer, SJQ "client restrictions" <IP address for Comskip computer>

Comskip computer, I have the c:\sjqc\sjqc.properties:
host=<IP address for SJQ computer>

I can ping each computer from the other computer. I'm not sure if I still have something wrong in the configuration. I have the SJQC setup as a service in both computers. I have tried shutting down the SJQ computer's SJQC service and turning on the Comskip computer's SJQC service. SJQ processing works when I have it setup for the SJQC service on the SJQ computer. I can't seem to get past the connection timeout problem when trying to get the Comskip computer's SJQC service to connect to the SJQ computer.

Dave

STATUS | wrapper  | 2010/04/18 21:03:25 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/04/18 21:03:25 | Java Service Wrapper Community Edition 3.3.1
STATUS | wrapper  | 2010/04/18 21:03:25 |   Copyright (C) 1999-2008 Tanuki Software, Inc.  All Rights Reserved.
STATUS | wrapper  | 2010/04/18 21:03:25 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/04/18 21:03:25 | 
STATUS | wrapper  | 2010/04/18 21:03:25 | Launching a JVM...
INFO   | jvm 1    | 2010/04/18 21:03:26 | WrapperManager: Initializing...
INFO   | jvm 1    | 2010/04/18 21:03:26 | SJQ Task Client v3.0.1.705
INFO   | jvm 1    | 2010/04/18 21:03:26 | Using 'C:/sjqc' as location for configuration file...
INFO   | jvm 1    | 2010/04/18 21:03:49 | Connection timed out: connect
INFO   | jvm 1    | 2010/04/18 21:09:10 | Connection timed out: connect

[Slugger]

Check that the (Windows) firewall on your SJQ server host is allowing incoming connections on which ever port Jetty is running on. This smells of a firewall config problem.

[davephan]

The SJQ computer had the Windows firewall running and the SJQC computer had the Norton firewall running. I disabled both firewalls. I also tried to open up the port 8500 in the router. Now, the error changed from "Connection timed out: connect" to "Connection refused: connect".

I will try setting up the second network connection in each computer for static IP addresses, and run a cross over cable between the two second network connections to bypass the router. I will also try using the free troubleshooting utility PCATTCP so I can send and receive port traffic between the two computers to test the port connectivity.

In the SJQ computer web GUI, I set the parameter, "client restrictions" to the SJQC computer's IP address. Is that correct, or should I leave that parameter blank?

Dave

STATUS | wrapper  | 2010/04/18 23:44:00 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/04/18 23:44:00 | Java Service Wrapper Community Edition 3.3.1
STATUS | wrapper  | 2010/04/18 23:44:00 |   Copyright (C) 1999-2008 Tanuki Software, Inc.  All Rights Reserved.
STATUS | wrapper  | 2010/04/18 23:44:00 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/04/18 23:44:00 | 
STATUS | wrapper  | 2010/04/18 23:44:00 | Launching a JVM...
INFO   | jvm 1    | 2010/04/18 23:44:00 | WrapperManager: Initializing...
INFO   | jvm 1    | 2010/04/18 23:44:00 | SJQ Task Client v3.0.1.705
INFO   | jvm 1    | 2010/04/18 23:44:00 | Using 'C:/sjqc' as location for configuration file...
INFO   | jvm 1    | 2010/04/18 23:44:03 | Connection refused: connect

[KarylFStein]

Originally Posted by davephan

In the SJQ computer web GUI, I set the parameter, "client restrictions" to the SJQC computer's IP address. Is that correct, or should I leave that parameter blank?

I think you want it to be: IP/32 e.g. 192.168.0.123/32. But make it blank for now and try.

[Slugger]

Originally Posted by davephan

The SJQ computer had the Windows firewall running and the SJQC computer had the Norton firewall running. I disabled both firewalls. I also tried to open up the port 8500 in the router. Now, the error changed from "Connection timed out: connect" to "Connection refused: connect".

I will try setting up the second network connection in each computer for static IP addresses, and run a cross over cable between the two second network connections to bypass the router. I will also try using the free troubleshooting utility PCATTCP so I can send and receive port traffic between the two computers to test the port connectivity.

In the SJQ computer web GUI, I set the parameter, "client restrictions" to the SJQC computer's IP address. Is that correct, or should I leave that parameter blank?

Dave

STATUS | wrapper  | 2010/04/18 23:44:00 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/04/18 23:44:00 | Java Service Wrapper Community Edition 3.3.1
STATUS | wrapper  | 2010/04/18 23:44:00 |   Copyright (C) 1999-2008 Tanuki Software, Inc.  All Rights Reserved.
STATUS | wrapper  | 2010/04/18 23:44:00 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/04/18 23:44:00 | 
STATUS | wrapper  | 2010/04/18 23:44:00 | Launching a JVM...
INFO   | jvm 1    | 2010/04/18 23:44:00 | WrapperManager: Initializing...
INFO   | jvm 1    | 2010/04/18 23:44:00 | SJQ Task Client v3.0.1.705
INFO   | jvm 1    | 2010/04/18 23:44:00 | Using 'C:/sjqc' as location for configuration file...
INFO   | jvm 1    | 2010/04/18 23:44:03 | Connection refused: connect

That error occurs if you start the task client before the SJQ server. 5 mins later that error should stop and the client should connect. If not, then ensure the port is configured properly on the task client. That error means there's nothing listening on the host/port you've told the task client to connect to.

[HuMan321]

What is the recommended order for starting SJQ? I sometimes start the client .bat file first and then log onto SJQ thru IE.

[Slugger]

It doesn't really matter, but a server should always be started before clients. But like I say, the task client doesn't care too much - if the server's not available it'll just try to connect again after a 5 min wait.

[Taddeusz]

I'm attempting to switch from using dirmon to SJQ. I keep on seeing this in my SJQ Client log:

Date: Wed Apr 21 14:17:06 CDT 2010







HTTP ERROR 500


Problem accessing /sjq/SJQ. Reason:

    Unexpected ':' encountered on line 4
Caused by:

java.lang.RuntimeException: Unexpected ':' encountered on line 4
	at com.google.code.sagetvaddons.sjq.server.ClientParser.readTask(ClientParser.java:399)
	at com.google.code.sagetvaddons.sjq.server.ClientParser.<init>(ClientParser.java:244)
	at com.google.code.sagetvaddons.sjq.server.SJQServlet.doPost(SJQServlet.java:185)
	at com.google.code.sagetvaddons.sjq.server.SJQServlet.doGet(SJQServlet.java:100)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)
	at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:539)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)

Powered by Jetty://

[Slugger]

There's a misplaced ':' on line 4 of your client config. If you can't spot it then paste your client config here and I'm sure someone else will see it.