Plugin: Sage Job Queue (SJQ) v3

[MitchSchaft]

I had SJQ and Sagealert set up and running fine last week. Starting yesterday, when I try to log in to the GUI, the webpages aren't there.
I've stopped/restarted the sage service, rebooted the sage server, cleared cookies/cache, tried to access via different PCs and even from different WAN locations.
Have you seen this issues before? If it means anything, I have BMT set up and running no problem using the same port as SJQ.

Also, SJQ and Sagealert are still working fine in the background. It posts to my twitter when a recording starts, commercials process and skip, etc. I just can't get to the web gui.

[Slugger]

I've seen this with one or two other users. For whatever reason, the temp folder that Jetty is expanding the war into to run from is being wiped - you are missing all the html files, etc. Redeploy the war file and that should fix the problem. Jetty should be expanding the war into your system temp folder - something (or someone) is wiping the contents of that directory. Once started, however, all the background threads will run just fine (as you're seeing).

[Tropicaljoe]

Originally Posted by Tropicaljoe

Slugger, Could SJQ be used for other external apps? Here is what I am looking to do. I want to be able to run a defrag on my drives while using your rulesets. It would be nice to be able to run the defrag only if no recordings are running, and no recordings are scheduled for the next hour or so. I believe that if I tried this now, it would probably run, but loop forever as the conditions would be true most of the time. Is something like this even possible?

Slugger, I've drawn a blank, can you point me in the right direction as to how the ruleset would accomplish this

[razrsharpe]

Originally Posted by Tropicaljoe

Slugger, Could SJQ be used for other external apps? Here is what I am looking to do. I want to be able to run a defrag on my drives while using your rulesets. It would be nice to be able to run the defrag only if no recordings are running, and no recordings are scheduled for the next hour or so. I believe that if I tried this now, it would probably run, but loop forever as the conditions would be true most of the time. Is something like this even possible?

Originally Posted by Tropicaljoe

Slugger, I've drawn a blank, can you point me in the right direction as to how the ruleset would accomplish this

ill take a stab...

write a batch file that would
-delete a temp DEFRAG.COMPLETE file (empty text file)
-make a temp DEFRAG.RUNNING file
-run the defrag
-delete the temp DEFRAG.RUNNING file
-make a temp DEFRAG.COMPLETE file

then write your sjq ruleset would check the timestamp on the DEFRAG.COMPLETE file ( older then x hrs) and the DEFRAG.RUNNING does not exist and all your other conditions are met... then the defrag task is run (which is the bat file i just described)

edit... i do not know if its possible to check the timestamp on the file with sjq but i bet there is a java function that can do it....

[MitchSchaft]

Originally Posted by Slugger

I've seen this with one or two other users. For whatever reason, the temp folder that Jetty is expanding the war into to run from is being wiped - you are missing all the html files, etc. Redeploy the war file and that should fix the problem. Jetty should be expanding the war into your system temp folder - something (or someone) is wiping the contents of that directory. Once started, however, all the background threads will run just fine (as you're seeing).

Excellent, that fixed both of 'em. Now, on to the question I had on the MAPDIR ruleset.

:MAPDIR "D:\\recorded tv" "\\\\sagehtpc\\recorded tv"

COMSKIP {
   :RETURNCODE 1
   "C:/SJQc/comskip80_025/comskip.exe \"--ini=C:/SJQc/comskip80_025/comskip.ini\" \"%c%\""
}

I cannot get this client to accept the MAPDIR rule.

Sat Jan 30 05:48:51 CST 2010: Starting task with priority 5/1/10
Executing command line: C:/SJQc/comskip80_025/comskip.exe "D:\Recorded TV\WWIIinHD-DayofDays-375177-0.mpg"
The commandline used was:
C:\SJQc\comskip80_025\comskip.exe "D:\Recorded TV\WWIIinHD-DayofDays-375177-0.mpg"
Opening D:\Recorded TV\WWIIinHD-DayofDays-375177-0.mpg

Comskip 0.80.025, made using:
mpeg2dec-0.4.0 - by Michel Lespinasse <walken@zoy.org> and Aaron Holtzman
No such file or directory - could not open file D:\Recorded TV\WWIIinHD-DayofDays-375177-0.mpg

It errors out saying it can't find the file. But the client I'm running on the server will pick it up. I'm not using a MAPDIR rule in that client's config.

Here's part of the log after I turn on the client on the network:
[quote]
2010-02-01 15:19:28,671 INFO [ClientParser]: MAPPING: D:\recorded tv => \\sagehtpc\recorded tv
2010-02-01 15:19:28,671 INFO [SJQServlet]: Checking client '192.168.2.11': 100 resources free.
2010-02-01 15:19:28,671 INFO [SJQServlet]: Servlet POST request completed [0ms]
2010-02-01 15:19:28,702 INFO [SJQServlet]: Servlet POST request completed [15ms]
2010-02-01 15:19:38,687 INFO [SJQServlet]: Servlet POST request completed [0ms]
2010-02-01 15:19:44,827 INFO [SJQServlet]: Servlet POST request completed [0ms]
2010-02-01 15:19:48,687 INFO [SJQServlet]: Servlet POST request completed [0ms]
2010-02-01 15:19:58,687 INFO [SJQServlet]: Servlet POST request completed [0ms]
[/qutoe]

It looks like it's mapped properly.

[Fuzzy]

That looks like it may be a case issue. SJQ is telling the client to process the files in "D:\Recorded TV". The client checks it's list of remaps first, to see it "D:\Recorded TV" is there, and it isn't (only "d:\recorded tv" is), so it uses it directly, and of course, doens't find the file.

[MitchSchaft]

Oh, you're awesome. That did the trick. I didn't think this stuff was case sensative, but it is!

[Tropicaljoe]

Originally Posted by razrsharpe

ill take a stab...

write a batch file that would
-delete a temp DEFRAG.COMPLETE file (empty text file)
-make a temp DEFRAG.RUNNING file
-run the defrag
-delete the temp DEFRAG.RUNNING file
-make a temp DEFRAG.COMPLETE file

then write your sjq ruleset would check the timestamp on the DEFRAG.COMPLETE file ( older then x hrs) and the DEFRAG.RUNNING does not exist and all your other conditions are met... then the defrag task is run (which is the bat file i just described)

edit... i do not know if its possible to check the timestamp on the file with sjq but i bet there is a java function that can do it....

When I first tried this, my sjq rule checked for the existence of the defrag.complete file, ( this worked )it however put in a task for each file that I had in my tv directory, so it qued up 4 instances of the task defrag. Is there anyway to possibly have it que up only once? I have been trying to think of some type of trigger that will only spawn this once, not on all shows in the directory. I used a simple rule ( if fileExists == defrag.complete ) as the only trigger

[Slugger]

Originally Posted by Tropicaljoe

Slugger, I've drawn a blank, can you point me in the right direction as to how the ruleset would accomplish this

Originally Posted by razrsharpe

ill take a stab...

write a batch file that would
-delete a temp DEFRAG.COMPLETE file (empty text file)
-make a temp DEFRAG.RUNNING file
-run the defrag
-delete the temp DEFRAG.RUNNING file
-make a temp DEFRAG.COMPLETE file

then write your sjq ruleset would check the timestamp on the DEFRAG.COMPLETE file ( older then x hrs) and the DEFRAG.RUNNING does not exist and all your other conditions are met... then the defrag task is run (which is the bat file i just described)

edit... i do not know if its possible to check the timestamp on the file with sjq but i bet there is a java function that can do it....

The only thing you have to be careful of is to ensure that the job is only queued once by SJQ, which is not intuitive. SJQ was designed to process Sage objects and perform tasks on them. To perform some external task just once based on some set of conditions requires you to pick a media object that you know will ALWAYS be available and key your ruleset off of it like this:

if [IsObjMediaFile == true && $.GetShowTitle{} == "Something I've Archived and Will Always Be Around, Ideally a Movie or Something With Only 1 Episode" && <Whatever conditions you care about>] {
  MYTASK
}

If you don't include the title check on some movie or something then SJQ will queue up your task once for each object it processes. Alternatively, you could handle multiple invocations in the actual script you're calling, but that results in a rather cluttered queue.

[Fuzzy]

You may want to look at a full-time background defragger. Most are pretty non-intrusive, and monitor drive/cpu usage to only work when idle. I use Diskkeeper, and it seems to work great.

I think, in the end, fragmentation should be of little concern to a sage system. Fragmented or not, a drive head is likely to be jumping around quite often when doing things like recording and playing back at the same time, no matter how much/little fragmentation you have on the drive. This is the key reason for sage recommending 64kb cluster sizes, to minimize the impact of fragments.

[voorhees]

I hope this is an easy one, and I hope I am just not sure where to look. I had to physically move my server do to some work in our house, and my router has graciously assigned it a new IP address (192.168.1.4 -> 192.168.1.6). I noticed comskip was not processing my files, so I went in SJQ and saw that my connected client is now 192.168.1.6, but when I click on Configure my task client is gone. Is there any way to find the task client I had in there when it was 192.168.1.4, so I do not have to reinvent the wheel? Does that make sense?

Thanks so much for your help in advance.

[Slugger]

Originally Posted by voorhees

I hope this is an easy one, and I hope I am just not sure where to look. I had to physically move my server do to some work in our house, and my router has graciously assigned it a new IP address (192.168.1.4 -> 192.168.1.6). I noticed comskip was not processing my files, so I went in SJQ and saw that my connected client is now 192.168.1.6, but when I click on Configure my task client is gone. Is there any way to find the task client I had in there when it was 192.168.1.4, so I do not have to reinvent the wheel? Does that make sense?

Thanks so much for your help in advance.

Good news is that your old ruleset is still in the SJQ db, but the bad news is that there's no easy way to transfer it to the new IP address. The easiest thing to do would be to assign a static IP to your client so that it uses the old one.

The other option is to download the SQLite CLP and manually switch the IP address for the client inside the database. If you choose this route, I can give you the SQL query you'll need to run in order to switch the IP address for the client.

[bikesquid]

I've seen a view posts with similar errors, but can't for the life of me track down where the problem is or what line of what file I should be looking at....
Any suggestions/advice would be HUGELY helpful.

Running latest versions of everything.... nt version of client on vista 32.

sjq is giving me this error from the client log.:

Problem accessing /sjq/SJQ. Reason:

Unexpected '?' encountered on line 11

Caused by:

java.lang.RuntimeException: Unexpected '?' encountered on line 11
at com.google.code.sagetvaddons.sjq.server.ClientParser.readTask(ClientParser.java:399)
at com.google.code.sagetvaddons.sjq.server.ClientParser.<init>(ClientParser.java:244)
at com.google.code.sagetvaddons.sjq.server.SJQServlet.doPost(SJQServlet.java:185)
at com.google.code.sagetvaddons.sjq.server.SJQServlet.doGet(SJQServlet.java:100)
....

[Slugger]

I assume your client config is only 11 lines? If so, then that error usually means you're missing a closing bracket somewhere. Post your client config if you can't find it.

[bikesquid]

Originally Posted by Slugger

I assume your client config is only 11 lines? If so, then that error usually means you're missing a closing bracket somewhere. Post your client config if you can't find it.

Thanks! I think I found it, see final just added '}' at the bottom. Is there a way to force a 'refresh' so I can see if it fixed the problem via the log?

COMSKIP {
:RESOURCES 25 # Up to 4 comskips can be run simultaneously (every client has exactly 100 resources available)

# CPU priority to be used for this task process; one of LOW, NORMAL, HIGH (case sensitive)
:CPU "NORMAL" # Default is NORMAL if not specified

# Comskip returns 1 on success instead of the universally accepted zero; tell SJQ that
# a return code of 1 means success instead of the default value of zero
:RETURNCODE 1

"C:\Program Files\ComskipMonitor\comskip80_031\comskip.exe \"%c%\""
}

[Slugger]

Just wait 5 mins or so and then check the logs.

[bikesquid]

Thanks so much for the help!

I've got a syntax question regarding spaces, the command in the client config seems to be choking on the directory structure with

"C:\Program Files\ComskipMonitor\comskip80_031\comskip.exe \"%c%\""

in reading the docs on client command syntax I'm understanding that double quotes should be ok with spaces??

[Slugger]

Choking how? Paste in the task output. Looks like your command line should be like this instead:

"\"C:\\Program Files\\ComskipMonitor\\comskip80_031\\comskip.exe\" \"%c%\""

But I'd need to see the errors from the task output to confirm.

[bikesquid]

Originally Posted by Slugger

Choking how? Paste in the task output. Looks like your command line should be like this instead:

"\"C:\\Program Files\\ComskipMonitor\\comskip80_031\\comskip.exe\" \"%c%\""

But I'd need to see the errors from the task output to confirm.

Here's what comes up under 'failed tasks, view log ' for one of the files.

Tue Feb 02 13:27:17 PST 2010: Starting task with priority 5/1/10
Executing command line: C:Program FilesComskipMonitorcomskip80_031comskip.exe "J:\Recorded TV\LawOrderCriminalIntent-AMurdererAmongUs-300804-0.ts"
Cannot run program "C:Program": CreateProcess error=2, The system cannot find the file specified
Tue Feb 02 13:42:18 PST 2010: Starting task with priority 5/1/10
Executing command line: C:Program FilesComskipMonitorcomskip80_031comskip.exe "J:\Recorded TV\LawOrderCriminalIntent-AMurdererAmongUs-300804-0.ts"
Cannot run program "C:Program": CreateProcess error=2, The system cannot find the file specified
Tue Feb 02 13:47:35 PST 2010: Starting task with priority 5/1/10
Executing command line: C:Program FilesComskipMonitorcomskip80_031comskip.exe "J:\Recorded TV\LawOrderCriminalIntent-AMurdererAmongUs-300804-0.ts"
Cannot run program "C:Program": CreateProcess error=2, The system cannot find the file specified

[Slugger]

Yeah, try the command line I posted, should fix the problem.