Plugin: Sage Job Queue (SJQ)

[Slugger]

Originally Posted by lovingHDTV

OK that worked.

When I try to run the client it fails to run:


C:\Documents and Settings\David\Desktop\sjqc-2.0.0.84>sjqc.bat 192.168.1.134 8088
Exception in thread "main" java.lang.NoClassDefFoundError: and
Caused by: java.lang.ClassNotFoundException: and
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)

Ideas?

Not quite sure. You definitely need to add the is_ssl option to the cmd line:

sjqc.bat 192.168.1.134 false 8088

Also make sure all the jars are in the same dir. Along with the bat file there should be two jars: json.jar and sjqc.jar; all in the same dir that you're starting the client from.

The stack trace suggests it's looking for a class named "and", which really boggles my mind. First, fix the cmd line and try again. Then we'll go from there.

[lovingHDTV]

Originally Posted by Slugger

Not quite sure. You definitely need to add the is_ssl option to the cmd line:

sjqc.bat 192.168.1.134 false 8088

Also make sure all the jars are in the same dir. Along with the bat file there should be two jars: json.jar and sjqc.jar; all in the same dir that you're starting the client from.

The stack trace suggests it's looking for a class named "and", which really boggles my mind. First, fix the cmd line and try again. Then we'll go from there.

Same results. I just extracted the client to a folder on my desktop and am running from there.

thanks

[Slugger]

Originally Posted by lovingHDTV

Same results. I just extracted the client to a folder on my desktop and am running from there.

thanks

Can you try to get it running without using the batch file?

C:\path\to\java\bin\java.exe -cp ".\json.jar;.\sjqc.jar" com.google.code.sagetvaddons.sjqc.TaskClient 192.168.1.134 false 8088

Also, which version of java are you using? Run java -version from the command line.

[lovingHDTV]

Originally Posted by Slugger

Can you try to get it running without using the batch file?

C:\path\to\java\bin\java.exe -cp ".\json.jar;.\sjqc.jar" com.google.code.sagetvaddons.sjqc.TaskClient 192.168.1.134 false 8088

Also, which version of java are you using? Run java -version from the command line.

java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode, sharing)

that worked and I see it connected.

[Slugger]

Originally Posted by lovingHDTV

java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode, sharing)

that worked and I see it connected.

Back to the drawing board on the whole batch file thing, though it does work for me (and at least a couple others). None the less, I'll see what I can do about that.

[lovingHDTV]

I'm now trying to create a couple config files:

Client:
# Client options
:MAXPROCS 2
/*:MAPDIR "D:\\tv" "\\\\sageserver\\tv" # Mapping D:\tv to UNC path \\sageserver\tv
:MAPDIR "D:\\videos" "\\\\sageserver\\videos"
:MAPDIR "D:\\music" "\\\\sageserver\\music"
*/

// TASK definition
COMSKIP{
:CPU LOW
:MAX 0
"C:\Program Files\ComSkip\comskip.exe --ini=C:\Program Files\ComSkip\comskip.ini -n %c%"
}

Server:
// Process all TV recordings that don't have a matching edl file
if [IsTV == "true" && FileExists != "%c%\%p%.edl"] {
:PRIORITY 5
COMSKIP
}

What I want to do is process all TV shows that do not have an edl file by running comskip. Do these look correct? If so how do I for SJQ to start a scan?

thanks for your help,
dave

[nyplayer]

Originally Posted by Slugger

Back to the drawing board on the whole batch file thing, though it does work for me (and at least a couple others). None the less, I'll see what I can do about that.

Slugger maybe it has something to do with the white spaces in his path name. I simply extracted mine to C:\sjqc-2.0.0.54 and it works fine. Probably if he quotes it... it will work.

white space before and after "and"
C:\Documents and Settings\David\Desktop\sjqc-2.0.0.84>sjqc.bat 192.168.1.134 8088

[lovingHDTV]

Originally Posted by nyplayer

Slugger maybe it has something to do with the white spaces in his path name. I simply extracted mine to C:\sjqc-2.0.0.54 and it works fine. Probably if he quotes it... it will work.

white space before and after "and"
C:\Documents and Settings\David\Desktop\sjqc-2.0.0.84>sjqc.bat 192.168.1.134 8088

That worked!

Here is what I changed that .bat file to:

java -cp "%~d0%~p0sjqc.jar;%~d0%~p0json.jar" "com.google.code.sagetvaddons.sjqc.TaskClient" %*

[Slugger]

Originally Posted by lovingHDTV

I'm now trying to create a couple config files:

Client:
# Client options
:MAXPROCS 2
/*:MAPDIR "D:\\tv" "\\\\sageserver\\tv" # Mapping D:\tv to UNC path \\sageserver\tv
:MAPDIR "D:\\videos" "\\\\sageserver\\videos"
:MAPDIR "D:\\music" "\\\\sageserver\\music"
*/

// TASK definition
COMSKIP{
:CPU LOW
:MAX 0
"C:\Program Files\ComSkip\comskip.exe --ini=C:\Program Files\ComSkip\comskip.ini -n %c%"
}

Server:
// Process all TV recordings that don't have a matching edl file
if [IsTV == "true" && FileExists != "%c%\%p%.edl"] {
:PRIORITY 5
COMSKIP
}

What I want to do is process all TV shows that do not have an edl file by running comskip. Do these look correct? If so how do I for SJQ to start a scan?

thanks for your help,
dave

"C:\Program Files\ComSkip\comskip.exe --ini=C:\Program Files\ComSkip\comskip.ini -n %c%"

All back slashes in all double quoted strings must be escaped. So in this line above either replace \ with \\ or flip them around (you may have to use \\ in the arguments depending if comskip can handle forward slashes in path names). Command arguments with spaces must be double quoted or they will not be passed to the command interpreter properly. So replace the above with something like this:

"C:\\Program Files\\ComSkip\\comskip.exe \"--ini=C:\\Program Files\\ComSkip\\comskip.ini\" -n \"%c%\""

I double quoted the %c% in case your record dir path contains a space, if you know it doesn't then that double quoting isn't necessary. Doesn't hurt to leave it.

And in your server conf:

FileExists != "%c%\%p%.edl"

Again, back slashes must be escaped:

FileExists != "%c%\\%p%.edl"

You may want to check if you're using the right media vars for this test. I think you might want to replace %c% with %d%.

[Slugger]

Originally Posted by lovingHDTV

That worked!

Here is what I changed that .bat file to:

java -cp "%~d0%~p0sjqc.jar;%~d0%~p0json.jar" "com.google.code.sagetvaddons.sjqc.TaskClient" %*

Of course... I was so baffled by where the "and" came from in the java stack trace that I didn't take the time to read all your output. Adding the quotes to the batch file now...

Thanks.

[lovingHDTV]

OK I got all the escaped " and \ in there. How do I tell SJQ to go try it? Also how can I enable logging so I can see the errors, I'm sure to have

thanks,

[Slugger]

Originally Posted by lovingHDTV

OK I got all the escaped " and \ in there. How do I tell SJQ to go try it? Also how can I enable logging so I can see the errors, I'm sure to have

thanks,

On the bottom tool bar, clicking the time of the next scheduled run allows you to do an immediate run.

Errors with the server conf are automatically logged. Click the "View server logs" toolbar button to view. Errors with the client configs aren't so friendly to view. Watch your client window, if it reports a 500 response from the server then that's an indication of an error in your client conf. Paste the URL into a browser for a more verbose error message. I'm working on a way to have those verbose messages viewable in the GUI.

[lovingHDTV]

For my server config. Will this rule start a comskip job if:

1. It is a TV show
2. the .mpg file exists
3. no matching .edl exists.

// Process all TV recordings that don't have a matching edl file
if [IsTV == "true" && FileExists != "%d%\\%p%.edl" && FileExists == "%c%"] {
:PRIORITY 5
COMSKIP
}

Is there a way to clear the queue and start over?

thanks,

[Slugger]

Originally Posted by lovingHDTV

For my server config. Will this rule start a comskip job if:

1. It is a TV show
2. the .mpg file exists
3. no matching .edl exists.

// Process all TV recordings that don't have a matching edl file
if [IsTV == "true" && FileExists != "%d%\\%p%.edl" && FileExists == "%c%"] {
:PRIORITY 5
COMSKIP
}

Is there a way to clear the queue and start over?

thanks,

Yes, excect the last FileExists is unnecessary, it will always be true. If you want to only process mpg files then you may want to replace it with:

Filename =$ ".mpg"

which says the file must be an mpg file (i.e. filename ends with .mpg). Your current rule will match any file that doesn't have a matching edl. You may also want to add IsActivelyRecording == "false" to ensure comskip doesn't start until after the recording is finished.

Currently, there is no way to clear the current queue other than to restart SJQ. You can do that by opening the context file in SageTV\jetty\contexts and resaving it. Updating the timestamp of that file will restart SJQ. Being able to add/remove single queue entries along with wiping the entire queue is definitely on the todo list.

The weather forecast is calling for rain tomorrow, if it does then I'll be able to tackle more of these todo list items in the morning. If it doesn't then I have yard work to be done. I should be able to find a few hours Sunday morning before football starts. Check the snapshot repository for new builds as they become available. Bugs, feature requests, etc. are best added to the project issue tracker. Just to make a note, here's the current todo list, feel free to add to it if you think something's missing:

• Add ability to add failed/completed jobs back to the queue, if desired
• Allow removal of items in the queue; allow wiping of entire active queue
• Add option to have failed jobs automatically go back to the queue
• Add back the "edl cleaner" (have SJQ auto cleanup specified files)
• Some kind of log management
• Have clients log their errors back to the server for viewing within the GUI; have client grab verbose dumps instead of just reporting "500 response received"
• Use the bottom tool bar for status messages
• Any other ideas?

[lovingHDTV]

1. We need a way to detect hung jobs. I have two jobs supposedly assigned to a client, but there is nothing running on the client. So there will be no forward progress until I restart everything.
2. Ability to adjust a job priority in the queue

[Slugger]

Originally Posted by lovingHDTV

1. We need a way to detect hung jobs. I have two jobs supposedly assigned to a client, but there is nothing running on the client. So there will be no forward progress until I restart everything.
2. Ability to adjust a job priority in the queue

Did the processes hang or did you kill the client while the process was running? In either case, a timeout definitely is need to catch either scenario. Adding it to the list...

Adjust waiting task priority... will come with the queue editing updates...

[lovingHDTV]

Originally Posted by Slugger

Did the processes hang or did you kill the client while the process was running? In either case, a timeout definitely is need to catch either scenario. Adding it to the list...

Adjust waiting task priority... will come with the queue editing updates...

In one case I killed the client. In most cases I was still trying to debug my client config rules, so comskip was dieing.

[nyplayer]

Slugger,

1.Add the ability to archive..unarchive..set manualrec status. You seem to have everything covered.
2. Ability to tell each client when they can process ... ex time of day they can process.

PS.
Is shutting the client process as easy as shutting down the batch job or does it still stay connected? I shut down the batch job on a client but it still shows connected.

[nyplayer]

It appears the FileExists != is not working.. but there is no way to tell at the moment what it's checking for and why the rule is not running.

the rule below doesn't work even though there is definitely no .working file.

if [IsTV == "true" && IsActivelyRecording == "false" && Filename =$ ".mpg" && FileExists != "%d%\\%p%.working"]{
    :PRIORITY 10
      QUICKFIX
}

[Slugger]

Originally Posted by nyplayer

It appears the FileExists != is not working.. but there is no way to tell at the moment what it's checking for and why the rule is not running.

the rule below doesn't work even though there is definitely no .working file.

if [IsTV == "true" && IsActivelyRecording == "false" && Filename =$ ".mpg" && FileExists != "%d%\\%p%.working"]{
    :PRIORITY 10
      QUICKFIX
}

Indeed it is raining, so no yard work is getting done, but SJQ bugs are getting fixed!

Snapshot 98 adds a new server setting: Debug. Set it to true to get all the debug info you can handle. I hesitated to put this in before adding some kind of log management features because of the massive amount of output it produces on each run of the queue loader, but obviously you need some kind of clue as to why your rules aren't doing what you expect. Give the server log viewer time to load when debug is true - it's a lot of output to be displayed.

So keep this setting to false unless you really need the output and then when you're done remember to turn it off!

Snapshot 98 also includes the fix for the FileExists test. Testing for the non-existence of a file did not work (!= operator simply ignored). This is fixed in 98.