Plugin: Sage Job Queue (SJQ) v3

[personalt]

Any idea what this means?
I had this running fine on windows XP but redid my system in windows7 64 bit. the client calls comskip fine and it runs for at least 20 minutes. comskip then craps out(I can tell from the taskmgr) and I see the in the sjqc.bat window
SJQ Task Client v3.0.2.771 Using 'C:/sjqc' as location for configuration file...
Failed to write stack trace dump to '/sjqc.dump'
Exception in thread "Thread-3" java.lang.RuntimeException: java.io.IOException:
Server returned HTTP response code: 500 for URL: http://127.0.0.1:8092/sjq/SJQ? roto=4&cmd=writeLog
at com.google.code.sagetvaddons.sjqc.TaskClient.log(Unknown Source)
at com.google.code.sagetvaddons.sjqc.IOMonster.dumpBuffer(Unknown Source )
at com.google.code.sagetvaddons.sjqc.IOMonster.run(Unknown Source) Caused by: java.io.IOException: Server returned HTTP response code: 500 for URL:
http://127.0.0.1:8092/sjq/SJQ?proto=4&cmd=writeLog at sun.net. http://www.protocol.http.HttpURLConn...Stream(Unknown Source)

[Slugger]

Not sure why you can't write the stack dump, but whatever user you run the client as doesn't have permission to write to the dir where you installed the task client.

What is the value of the buffer in your sjqc.properties file. The error you're getting is usually because the value is too high. If it's higher than 65000 then lower it to 65000 (stop the task client before editing the value then restart the task client).

[personalt]

thanks.. this seemed to work.. I dropped the buffers down and set the permission on the client folder to everyone and it started working fine.

[ohpleaseno]

I recently rebuilt my WHS/Sage server. I migrated all of my recordings over from my old server (server) to my new one (whs). When I installed SJQ and configured it to run comskip, comskip attempts to call it from \\server\recorded TV... instead of the new location. All of the recordings got moved along with wiz.bin to the new machine and play fine.

Here is some of the client log.

Sun Mar 28 09:16:35 EDT 2010: Starting task with priority 5/1/10
Executing command line: C:/progs/comskip/comskip.exe "\\server\recorded tv\Seinfeld-TheCalzone-1999238-1.mpg"
Comskip 0.80.031, made using:
mpeg2dec-0.4.0 - by Michel Lespinasse <walken@zoy.org> and Aaron Holtzman
No such file or directory - could not open file \\server\recorded tv\Seinfeld-TheCalzone-1999238-1.mpg

The commandline used was:
C:\progs\comskip\comskip.exe "\\server\recorded tv\Seinfeld-TheCalzone-1999238-1.mpg"
Opening \\server\recorded tv\Seinfeld-TheCalzone-1999238-1.mpg

[Slugger]

Run the file against the SJQ media debugger and have a look at the logs. It appears that something is telling SJQ that the file is still at its old location. Perhaps you have a :MAPDIR setting on the client config pointing to the old server still?

[ohpleaseno]

media debugger only will find it using the absolute path (//server/recorded tv...) that doesn't exist anymore. Typing "Seinfeld" gets me nothing.

Here is the log it generated.

2010-03-28 06:50:53,062 WARN  [MediaFileQueueLoader]: Processing 'Seinfeld'...
2010-03-28 06:50:53,062 WARN  [RulesParser]: 	Line 3: Evaluating 'IsObjMediaFile == "true"' == true
2010-03-28 06:50:53,062 WARN  [RulesParser]: 	Line 3: Evaluating '$.IsTVFile{} == "true"' == true
2010-03-28 06:50:53,062 WARN  [RulesParser]: 	Line 3: Evaluating 'MediaAge > "58D"' == false
2010-03-28 06:50:53,062 WARN  [RulesParser]: 	Line 9: Evaluating 'IsObjMediaFile == "true"' == true
2010-03-28 06:50:53,062 WARN  [RulesParser]: 	Line 9: Evaluating '$.IsTVFile{} == "true"' == true
2010-03-28 06:50:53,062 WARN  [RulesParser]: 	Line 9: Evaluating '$.IsFileCurrentlyRecording{} == "false"' == true
2010-03-28 06:50:53,062 WARN  [RulesParser]: 	Line 9: Evaluating '$.IsWatchedCompletely{} == "false"' == true
2010-03-28 06:50:53,062 WARN  [MediaVariables]: 	MediaVar '%c%' == '\\server\recorded tv\Seinfeld-TheBottleDeposit-2003433-1.mpg'
2010-03-28 06:50:53,062 WARN  [MediaVariables]: 	MediaVar '%d%' == '\\server\recorded tv'
2010-03-28 06:50:53,062 WARN  [MediaVariables]: 	MediaVar '%f%' == 'Seinfeld-TheBottleDeposit-2003433-1.mpg'
2010-03-28 06:50:53,062 WARN  [MediaVariables]: 	MediaVar '%e%' == 'mpg'
2010-03-28 06:50:53,062 WARN  [MediaVariables]: 	MediaVar '%p%' == 'Seinfeld-TheBottleDeposit-2003433-1'
2010-03-28 06:50:53,062 WARN  [FieldFileExists]: 	Expanded '%d%/%p%.edl' to '\\server\recorded tv\Seinfeld-TheBottleDeposit-2003433-1.edl'
2010-03-28 06:50:55,546 WARN  [FieldFileExists]: Listing of '\\server\recorded tv' returned null!
2010-03-28 06:50:55,546 WARN  [RulesParser]: 	Line 9: Evaluating 'FileExists != "%d%/%p%.edl"' == true
2010-03-28 06:50:55,546 WARN  [MediaVariables]: 	MediaVar '%c%' == '\\server\recorded tv\Seinfeld-TheBottleDeposit-2003433-1.mpg'
2010-03-28 06:50:55,546 WARN  [MediaVariables]: 	MediaVar '%d%' == '\\server\recorded tv'
2010-03-28 06:50:55,546 WARN  [MediaVariables]: 	MediaVar '%f%' == 'Seinfeld-TheBottleDeposit-2003433-1.mpg'
2010-03-28 06:50:55,546 WARN  [MediaVariables]: 	MediaVar '%e%' == 'mpg'
2010-03-28 06:50:55,546 WARN  [MediaVariables]: 	MediaVar '%p%' == 'Seinfeld-TheBottleDeposit-2003433-1'
2010-03-28 06:50:55,546 WARN  [RulesParser]: 	Line 9: Evaluating 'Filename =$ ".mpg"' == true
2010-03-28 06:50:55,640 INFO  [MediaFileQueueLoader]: Run completed [2687ms]
2010-03-28 06:50:55,640 INFO  [DataStore]: Thread 'SJQ-LogCleaner' is seeking an exclusive DataStore connection!
2010-03-28 06:50:55,640 INFO  [DataStore]: Thread 'SJQ-LogCleaner' is waiting for an exclusive DataStore connection! [AVAIL=10; MAX=11; WAIT=0ms]
2010-03-28 06:50:55,640 INFO  [VacuumCleaner]: Thread finished [0ms]
2010-03-28 06:50:55,640 WARN  [FileCleaner]: Skipping because file cleaner settings are empty!
2010-03-28 06:50:55,640 INFO  [DataStore]: Thread 'SJQ-LogCleaner' has received an exclusive DataStore connection! [WAIT=0ms]
2010-03-28 06:50:55,812 INFO  [DataStore]: 'SJQ-LogCleaner' has returned its exclusive DataStore connection!
2010-03-28 06:50:55,812 INFO  [LogCleaner]: Thread finished [172ms]
2010-03-28 06:51:01,718 WARN  [SJQServlet]: All client connections are being accepted and processed, please consider setting up client restrictions in SJQ settings
2010-03-28 06:51:01,718 INFO  [SJQServlet]: Checking client '192.168.3.106': 100 resources free.
2010-03-28 06:51:01,734 INFO  [SJQServlet]: Servlet POST request completed [16ms]
2010-03-28 06:51:04,078 WARN  [SJQServlet]: All client connections are being accepted and processed, please consider setting up client restrictions in SJQ settings
2010-03-28 06:51:04,078 INFO  [ClientParser]: MAPPING: \\LOCALHOST\SAGETV => \\WHS\SageTV
2010-03-28 06:51:04,078 INFO  [SJQServlet]: Checking client '192.168.3.107': 100 resources free.
2010-03-28 06:51:04,078 INFO  [SJQServlet]: Servlet POST request completed [0ms]
2010-03-28 06:51:30,781 WARN  [SJQServlet]: All client connections are being accepted and processed, please consider setting up client restrictions in SJQ settings
2010-03-28 06:51:30,781 INFO  [SJQServlet]: Servlet POST request completed [0ms]

[ohpleaseno]

And my client and ruleset is pretty straightforward.

if [IsObjMediaFile == true && $.IsTVFile{} == true && $.IsFileCurrentlyRecording{} == false && $.IsWatchedCompletely{} == false && FileExists != "%d%/%p%.edl" && Filename =$ ".mpg"]
{
   :PRIORITY 10
   COMSKIP
}

:MAPDIR "\\\\LOCALHOST\\SAGETV" "\\\\WHS\\SageTV"

COMSKIP {
   :RESOURCES 25
   "C:/progs/comskip/comskip.exe \"%c%\""
}

[Slugger]

All the output you've provided is telling me that Sage is telling SJQ that the path is still \\server\... instead of the new one. If you're willing to work with me I can whip up a quick debug build that will confirm this. PM me if you're interested in running the debug build and gather some extra output for me.

[davephan]

I'm unable to open the SJQ web page after installing the Official 3.01 version.

I have the Nielm SageTV web server already installed and it works fine. I then installed the Jetty and SageAlert, which also works fine. I use port 8080 for the Nielm SageTV web server and port 8500 for the SageAlert web server.

I tried both port 80 and 443, but I cannot open the SJQ web page.

The instructions to copy the files necessary is straightforward, yet something is going wrong.

Dave

[Slugger]

SJQ will run on the same port as SageAlert. So if SageAlert is on port 8500 on your system then the URL for SJQ would be:

http://<server>:8500/sjq/

If that doesn't load then enable debug logging in Sage and check for errors in the logs.

[davephan]

Thanks for the quick response! I didn't try that combination. Port 8500 worked immediately.

Now I'll have to review the docs to figure out how to selectively comskip files. I've been wasting a lot of CPU cycles processing files from non commercial movie channels.

Dave

[Slugger]

Originally Posted by davephan

Thanks for the quick response! I didn't try that combination. Port 8500 worked immediately.

Now I'll have to review the docs to figure out how to selectively comskip files. I've been wasting a lot of CPU cycles processing files from non commercial movie channels.

Dave

This example from the SJQv3 User's Guide should get you going in the right direction.

[ohpleaseno]

If I'm trying to call mkv files in my imported videos, then I would use the following, correct?

($.IsVideoFile{} == true && $.IsTVFile{} == false)

[razrsharpe]

i think actually you would want this (havent tested this....)

IsObjMediaFile == true && $.IsTVFile{} == false && $.IsVideoFile{} == true && Filename =$ ".mkv"

[Slugger]

Originally Posted by ohpleaseno

If I'm trying to call mkv files in my imported videos, then I would use the following, correct?

($.IsVideoFile{} == true && $.IsTVFile{} == false)

Originally Posted by razrsharpe

i think actually you would want this (havent tested this....)

IsObjMediaFile == true && $.IsTVFile{} == false && $.IsVideoFile{} == true && Filename =$ ".mkv"

razrsharpe's is more complete and probably the better choice. Remember that by default SJQ only processes recorded tv files so you have to enable imported video scanning from the SJQ settings panel.

[ohpleaseno]

This is actually what worked (I'm trying to run SageAudioConvert against mkv files that only have a DTS audio track)

if [IsObjMediaFile == true && ($.IsVideoFile{} == true && $.IsTVFile{} == false) && Filename =$ ".mkv" && FileExists != "%d%/%p%.dtsDone"

For some reason, even though I'm using :MAPDIR on the client side, it still is pointing to localhost when it calls the file. Does :MAPDIR need to go in front of every job specified in the client side?

[davephan]

I tried using example 2 and modifying it. The result included channels what were not in the job. Originally, in my modified example two, one of the entries was "KTCADT@", I cleared the grid, changed "KTCADT@" to "KTCADT2", and saved the job, loaded the job, and ran the job. The same results occurred. The jobs listed include the local CBS, NBC, and some other channels that are not listed in the job. I don't have any clients set up yet. When I try to scroll through the 'Active Queue' the screen refreshes frequently, so I cannot scroll through the whole list. I think it would be nice if the 'Active Queue' also had a column for the channel name. However, as I mentioned before, the problem is the job is not detecting the channel names listed in the job. A screenshot of the "Active Queue" is attached.


Here's the original example 2 job:

if [IsObjMediaFile == true && $.IsTVFile{} == true && $.IsNotManualOrFavorite{} == false && ($.GetAiringChannelName{} != "HBO1" && $.GetAiringChannelName{} != "HBO2" && $.GetAiringChannelName{} != "HBO3" && $.GetAiringChannelName{} != "HBO4") && $.GetAiringChannelName{} != "WPBS" && ($.GetAiringChannelName{} != "PPV1" && $.GetAiringChannelName{} != "PPV2" && $.GetAiringChannelName{} != "PPV3")] {
COMSKIP
}

Here is the modified example 2 job:

if [IsObjMediaFile == true && $.IsTVFile{} == true && $.IsNotManualOrFavorite{} == false && ($.GetAiringChannelName{} != "HBOHD" && $.GetAiringChannelName{} != "HBO2" && $.GetAiringChannelName{} != "HBOSIG" && $.GetAiringChannelName{} != "HBOF") && $.GetAiringChannelName{} != "HBOC" && ($.GetAiringChannelName{} != "HBOZ" && $.GetAiringChannelName{} != "KTCADT" && $.GetAiringChannelName{} != "KTCADT2")] {
COMSKIP
}


Dave

[Slugger]

Originally Posted by ohpleaseno

For some reason, even though I'm using :MAPDIR on the client side, it still is pointing to localhost when it calls the file. Does :MAPDIR need to go in front of every job specified in the client side?

No, but it is case sensitive. localhost != LOCALHOST so make sure the case is correct in your client config compared to what SJQ is receiving from SageTV.

[Slugger]

Originally Posted by davephan

I tried using example 2 and modifying it. The result included channels what were not in the job. Originally, in my modified example two, one of the entries was "KTCADT@", I cleared the grid, changed "KTCADT@" to "KTCADT2", and saved the job, loaded the job, and ran the job. The same results occurred. The jobs listed include the local CBS, NBC, and some other channels that are not listed in the job. I don't have any clients set up yet. When I try to scroll through the 'Active Queue' the screen refreshes frequently, so I cannot scroll through the whole list. I think it would be nice if the 'Active Queue' also had a column for the channel name. However, as I mentioned before, the problem is the job is not detecting the channel names listed in the job. A screenshot of the "Active Queue" is attached.


Here's the original example 2 job:

if [IsObjMediaFile == true && $.IsTVFile{} == true && $.IsNotManualOrFavorite{} == false && ($.GetAiringChannelName{} != "HBO1" && $.GetAiringChannelName{} != "HBO2" && $.GetAiringChannelName{} != "HBO3" && $.GetAiringChannelName{} != "HBO4") && $.GetAiringChannelName{} != "WPBS" && ($.GetAiringChannelName{} != "PPV1" && $.GetAiringChannelName{} != "PPV2" && $.GetAiringChannelName{} != "PPV3")] {
COMSKIP
}

Here is the modified example 2 job:

if [IsObjMediaFile == true && $.IsTVFile{} == true && $.IsNotManualOrFavorite{} == false && ($.GetAiringChannelName{} != "HBOHD" && $.GetAiringChannelName{} != "HBO2" && $.GetAiringChannelName{} != "HBOSIG" && $.GetAiringChannelName{} != "HBOF") && $.GetAiringChannelName{} != "HBOC" && ($.GetAiringChannelName{} != "HBOZ" && $.GetAiringChannelName{} != "KTCADT" && $.GetAiringChannelName{} != "KTCADT2")] {
COMSKIP
}


Dave

In the tools menu, load the media debugger and select a recording that is not working properly (either is going into the queue but you don't think it should be or isn't going into the queue but you think it should). Run that recording through the media debugger, go to the server logs and examine the output. The output will tell you which test is causing the problem.

[MitchSchaft]

So nobody else really has troubles with HTTP ERROR: 503? I get this probably once every 2 weeks. Both SJQ and the alerter at the same time. I have to redeploy to get it back.