Plugin: Sage Job Queue (SJQ) v3

[personalt]

Nplayer - if you dont mind me asking what do these two tasks do?

Are you actually cutting out commericals here? I use comskip to mark commercials and that works fine on my server. but for the few shows that I always copy to my cell phone I would like to have it use the comksip markers and just cut the files.

I like the text file switches you have in there also. The idea of having a check for a stoprunning text file switch is so simple and smart at the same time.

Originally Posted by nyplayer

Slugger,

No lockups no high CPU usage. Rules not looking for existence of a file run fine.

2009-12-10 21:01:34,593 INFO  [SystemMessageQueueLoader]: Run completed [93ms]
2009-12-10 21:02:12,671 WARN  [FieldFileExists]: Adding '\\192.168.2.155\j$\sageuncj' to the cache!
2009-12-10 21:02:13,718 WARN  [FieldFileExists]: Done reading '\\192.168.2.155\j$\sageuncj'!
2009-12-10 21:02:13,718 WARN  [FieldFileExists]: Adding 'c:\controls' to the cache!
2009-12-10 21:02:13,718 WARN  [FieldFileExists]: Done reading 'c:\controls'!
2009-12-10 21:02:13,765 WARN  [FieldFileExists]: Adding '\\NYPLAYER\sageunct$' to the cache!
2009-12-10 21:02:19,390 WARN  [FieldFileExists]: Done reading '\\NYPLAYER\sageunct$'!
2009-12-10 21:02:24,500 WARN  [FieldFileExists]: Adding 'M:\importuncm' to the cache!
2009-12-10 21:02:24,515 WARN  [FieldFileExists]: Done reading 'M:\importuncm'!
2009-12-10 21:02:25,421 WARN  [FieldFileExists]: Adding '\\192.168.2.155\j$\importuncj' to the cache!
2009-12-10 21:02:25,437 WARN  [FieldFileExists]: Done reading '\\192.168.2.155\j$\importuncj'!
2009-12-10 21:02:26,328 WARN  [FieldFileExists]: Adding '\\192.168.2.155\importuncj' to the cache!
2009-12-10 21:02:26,328 WARN  [FieldFileExists]: Done reading '\\192.168.2.155\importuncj'!
2009-12-10 21:02:28,015 WARN  [FieldFileExists]: Adding 'M:\sagem' to the cache!
2009-12-10 21:02:28,031 WARN  [FieldFileExists]: Done reading 'M:\sagem'!
2009-12-10 21:02:29,062 WARN  [FieldFileExists]: Adding 'N:\sagen' to the cache!
2009-12-10 21:02:29,078 WARN  [FieldFileExists]: Done reading 'N:\sagen'!

It appears that if it looks for existence of a file it will not run the rule..... even though the files do exist.


this rule runs

//QFIX ALL TV
if [IsObjMediaFile == true && FileExists != "%d%/%p%.qfix" && FileExists != "c:\\controls\\oktoreboot.txt" && FileExists != "c:\\controls\\Stoprunning.txt" && $.GetAiringChannelNumber{} !="35" && $.GetAiringChannelNumber{} !="36" && Filename =$ ".mpg" && $.IsTVFile{} == "true" && $.IsNotManualOrFavorite{} == "false" && $.IsFileCurrentlyRecording{} == "false" && $.GetNumberOfSegments{} == "1" && $.GetMediaTitle{} != "Not Recording"] {
   :RUNQLOADER "true"    
    :PRIORITY 998
     SJQFIXShow
}

This rule doesn't run but if I backoff to 692 it runs ok. qfix ... vprj file exists and also my recordings are manual.

//VRDCUT NO Arhive
if [IsObjMediaFile == true && FileExists == "%d%/%p%.qfix" && FileExists == "%d%/%p%.vprj" && FileExists != "%d%/%p%.donecut" && FileExists != "c:\\controls\\oktoreboot.txt" && FileExists != "c:\\controls\\Stoprunning.txt" && Filename =$ ".mpg" && $.IsTVFile{} == "true" && $.IsManualRecord{} == "true" && $.IsLibraryFile{} == "false" && $.IsFileCurrentlyRecording{} == "false" && $.GetNumberOfSegments{} == "1"] {
  :RUNQLOADER "true"
    :PRIORITY 905
  SJQVRDCUT
}

[Slugger]

Originally Posted by nyplayer

Slugger,


Ding Ding Ding .. your test appears to be case sensitive now.
Task not being added to que, It finds the first file qfix but does not find the 2nd vprj.

Nice catch! Never thought of that. When I moved to the in-memory cache I just got the dir listing, stored it in memory then compared the file name you search for against the array of strings. I guess I'll have to detect the OS and if it's Windows, do a non-case sensitive search.

[Slugger]

Originally Posted by Slugger

Nice catch! Never thought of that. When I moved to the in-memory cache I just got the dir listing, stored it in memory then compared the file name you search for against the array of strings. I guess I'll have to detect the OS and if it's Windows, do a non-case sensitive search.

Fixed in snapshot 697.

[nyplayer]

Originally Posted by Slugger

Fixed in snapshot 697.

Thanks,

I use Videoredo to run a QFIX stream and also cut commercials... I found out that if I run my videos through QFIX then showanalyzer(comskip) that acurracy is way better.

The stoprunning.txt is so much easier to stop SJQ from running new jobs... This way the jobs running complete but nothing new is scheduled. No tampering with rules etc... I simply rename the file and SJQ carries on like nothing happened

[evilpenguin]

Originally Posted by evilpenguin

This works great in 696

I'm running some encodes tonight so i'll report back on the log changes tomorrow.

Logs spacing is also preserved nicely.

That's pretty much all I've got so SJQ v3.0 gets my thumbs up for release, great work Slugger

[voorhees]

Slugger, I am trying SJQ for the first time after keeping up with it for quite a while. What a great add on (as if I had any doubt based on my experience with SRE). The documentation was fantastic for getting it setup and running.

One quick question though. I changed the Jetty maxFormContentSize as recommended, however do I need to bump up the client buffer from 199000 to avoid the issue evilpenguin noted when live comskiping a long program (i.e. football game)?

Thanks again for another great add on.

[evilpenguin]

Nope, either of those will fix the issue on their own and having both is fairly harmless also.

[Slugger]

Originally Posted by awhenry

On the topic of logs, three other quick questions. First, when viewing the server logs, clicking "Clear Log" does not clear the log (the banner message does appear saying the logs were cleared, but all the log entries remain). Is this supposed to work in the current version?

Originally Posted by Slugger

Is it suppose to? Sure. Does it? No. Do I know about it? Yes. Will it be fixed before beta ends for SJQv3. Yes.

Turns out I spoke too soon. It seems that the log4j package (unsurprisingly) keeps the active log file open and locked while it's writing to it, meaning I can't delete it with a call to java.io.File.delete() (the call just always fails) so I will not be able to support this feature. I have, however, updated the GUI to display an appropriate response when you attempt to clear the server log (it will be in the next available build).

[aaronb]

My ruleset is not working, I'm getting this error in the log:

2009-12-13 18:48:18,316 ERROR [SystemMessageQueueLoader]: Unexpected exception
java.lang.RuntimeException: Unexpected '"' on line 4
	at com.google.code.sagetvaddons.sjq.server.RulesParser.readJob(RulesParser.java:502)
	at com.google.code.sagetvaddons.sjq.server.RulesParser.postShellChecker(RulesParser.java:328)
	at com.google.code.sagetvaddons.sjq.server.RulesParser.loadQ(RulesParser.java:266)
	at com.google.code.sagetvaddons.sjq.server.RulesParser.(RulesParser.java:239)
	at com.google.code.sagetvaddons.sjq.server.SystemMessageQueueLoader.run(SystemMessageQueueLoader.java:61)
	at java.lang.Thread.run(Unknown Source)

Ruleset:

# HDPVR Encoding - Law and Order
if [IsObjMediaFile == true && $.GetShowTitle{} == "Law & Order" && FileExists != "%d%/%p%.mkv"] 
{
   "C:/mediashrink/mediaShrink.exe \"%c%\" /inPlace /batch /highProfile /cliBitrate 2500"
}

I copied it almost directly from evilpenguin's example here, but it appears it does not like the syntax.

Any ideas? I can post more of the log if needed.

[Slugger]

Originally Posted by aaronb

My ruleset is not working, I'm getting this error in the log:

2009-12-13 18:48:18,316 ERROR [SystemMessageQueueLoader]: Unexpected exception
java.lang.RuntimeException: Unexpected '"' on line 4

Ruleset:

# HDPVR Encoding - Law and Order
if [IsObjMediaFile == true && $.GetShowTitle{} == "Law & Order" && FileExists != "%d%/%p%.mkv"] 
{
   "C:/mediashrink/mediaShrink.exe \"%c%\" /inPlace /batch /highProfile /cliBitrate 2500"
}

I copied it almost directly from evilpenguin's example here, but it appears it does not like the syntax.

Any ideas? I can post more of the log if needed.

You don't put the actual command line in the ruleset, rulesets queue up task ids and clients run the commands. So in your ruleset replace that command line with a task id (call it MEDIASHRINK let's say). Then in your client config define MEDIASHRINK to run the command line above (just as EP did it in the post you reference).

[ohpleaseno]

I'm having some trouble starting a client machine as a service. I get an Error 1069 when I attempt to start it.

The login I'm giving it should be from the client machine, not the server, correct?

[ohpleaseno]

Okay, as soon as I post the problem, it resolves itself. I created a new user on the machine and it took.

I think it's a conspiracy by my machine to make me look stupid in this thread. That's the second time that has happened.

[aaronb]

Originally Posted by Slugger

You don't put the actual command line in the ruleset, rulesets queue up task ids and clients run the commands. So in your ruleset replace that command line with a task id (call it MEDIASHRINK let's say). Then in your client config define MEDIASHRINK to run the command line above (just as EP did it in the post you reference).

Ahh, I see. So what you're saying is I need to read the entire user guide, not just the first 2/3rds of it.

[Slugger]

I'm assuming that no recent messages is good news? This is your last chance to bring up any issues with server build 3.0.0.697 and/or task client build 3.0.0.694. As far as I'm concerned, SJQv3 is done and complete with those builds (actually the final server will be 698 that contains a small fix for the error message when trying to clear server logs). I plan on marking these builds as "official" SJQv3 on Thu and will then be taking a break from all things SageTV (except for the actual use of it) until sometime later in January 2010.

Thanks to all who have helped with the testing of SJQv3, it's been a long, but productive three months to get SJQv3 to where it is today.

[loonsailor]

I just upgraded from sjq v2 to v3. It seems to work - at least I can open the interface via jetty and talk to it, and the tab is entitled "SJQ: Sage Job Queue 3.x". However, I thought that the rules would look different, and would have been automatically updated. I only have one rule, and editor show it as follows:

if [IsTV == true && IsScheduledRecording == true && ChannelName !% "KQED.*|KTEH.*|KRCB.*|WORLD|CSPAN.*|CALCHAN.*|EDAC.*|KVIE.*|STARZ.*|STZ.*|HBO.*|.*MAX.*|SHO.*"] {
    COMSKIP
}

That's a v2 rule, yes? Am I supposed to do something to update it automatically, or should I do it by hand?

I'm on linux, if that matters.

[evilpenguin]

You'll need to update it manually, but its very easy.

if [IsObjMediaFile == true && $.IsTVFile{} == true && $.IsNotManualOrFavorite{} == false && $.GetAiringChannelName{} !% "KQED.*|KTEH.*|KRCB.*|WORLD|CSPAN.*|CALCHAN.*|EDAC.*|KVIE.*|STARZ.*|STZ.*|HBO.*|.*MAX.*|SHO.*"] {
   COMSKIP
}

There are a lot of changes/improvements in SJQ v3 so you should prolly take some time and read through the updated user manual in the first post.

[MitchSchaft]

My issue was never addressed from a few pages back .

[Slugger]

Originally Posted by MitchSchaft

My issue was never addressed from a few pages back .

And what issue was that? (I hate trying to find old posts in the same thread.)

[doc]

Originally Posted by MitchSchaft

My issue was never addressed from a few pages back .

I've just searched for your old posts so I may have missed one.

Your job runs the comskip.exe direct, and comskip returns a 1 when a job successfully finishes. Every other program in the world returns a 0 (zero), hence SJQ thinks its failed.

If you do a search on here for failing jobs and comskip, there will be an example of a batch file you can use that will return a 0 instead of a 1.

If I've got the wrong post then accept my apologies.

[MitchSchaft]

No, that was the post