Plugin: SJQ v1.1.0RC1 Available - Testers Needed

[Slugger]

Sage Job Queue is a daemon that allows you to process your media files based on a set of rules you define. For example, you can choose to run your commercial flagging software against very specific recordings while ignoring others. All rules are based on not only the name of the files, but the contents of the file as stored in the SageTV database.

After many Sunday morning coding sessions, I've now got the first release candidate for SJQ v1.1.0 ready for testing.

I've done a lot of testing myself, but I can't possibly cover it all so I'm hoping that any users of SJQ out there that are interested in having a configuration servlet will help me squash any remaining bugs in the servlet code.

The only change in v1.1.0RC1 compared to the latest stable version (1.0.1) is the addition of the web plugin compatible servlet. Features of the servlet include:

• Configure all plugin options without needing to stop/restart SageTV
• Create and edit your ruleset file via dynamic web form - no longed need to know ruleset syntax for creating rules!
• View plugin log files via web
• View complete history of all jobs completed by SJQ

Before upgrading to RC1 please backup your existing SJQ data. You should backup the following two files:

• rules.sjq: Your current SJQ ruleset file
• sjq.sqlite: The SJQ data store; stores the entire job history as completed by SJQ - you don't want to lose this to corruption

I've tested the db upgrade code along with writing of the ruleset file by the servlet and I haven't had any issues, but it doesn't mean they don't exist so please, please backup these data files before upgrading to any RC build - you've been warned.

With that out of the way, some notes about this RC build:

• The rule editor is a bit ugly, I'm still working on some CSS changes for things like the 'Remove' buttons, etc. But I don't know how to fix things like the differing sizes of drop downs, which really makes things ugly so I'm open to suggestions from any web design gurus out there
• Please see the online docs for the meaning of the various items in a rule

The RC build is only available from here - I didn't want to upload RC builds to the forums.

The online docs, including installation instructions for the servlet, are available here.

Please report any bugs in my bugzilla database or post a message to this thread.

Please post any feedback at all to this thread, including feature requests, etc.

[LJSSageUser]

Originally Posted by Slugger

After many Sunday morning coding sessions, I've now got the first release candidate for SJQ v1.1.0 ready for testing.

Clearly intended only for the elite. This is a great piece of work, but I had to go all the way to the wiki to find out what it does.

Thanks for the work.

[JonTom]

Yeah, you might want to mention what this plugin actually does in your OP. Near the top...

[Slugger]

Originally Posted by JonTom

Yeah, you might want to mention what this plugin actually does in your OP. Near the top...

Description added to OP... thanks.

[TechBill]

I have a question on this

Add slugger.sagetv.sjq.SageJobQueue as a loadable plugin in the appropriate .properties file by adding the class to the load_at_startup_runnable_classes list.


I do have this in my properties file

load_at_startup_runnable_classes=net.sf.sageplugins.webserver.StartServer

do I add "slugger.sagetv.sjq.SageJobQueue" to the same line with space or seperating with common or just create a new line underneath like this

load_at_startup_runnable_classes=net.sf.sageplugins.webserver.StartServer
load_at_startup_runnable_classes=slugger.sagetv.sjq.SageJobQueue

Or something like this?
load_at_startup_runnable_classes=net.sf.sageplugins.webserver.StartServer, slugger.sagetv.sjq.SageJobQueue

I tried both way and I am not able to get SJQ loaded, each time I go to the webserver it stated SJQ is not started or installed etc.



Thanks
Bill

[Slugger]

Originally Posted by TechBill

I have a question on this

Add slugger.sagetv.sjq.SageJobQueue as a loadable plugin in the appropriate .properties file by adding the class to the load_at_startup_runnable_classes list.


I do have this in my properties file

load_at_startup_runnable_classes=net.sf.sageplugins.webserver.StartServer

do I add "slugger.sagetv.sjq.SageJobQueue" to the same line with space or seperating with common or just create a new line underneath like this

load_at_startup_runnable_classes=net.sf.sageplugins.webserver.StartServer
load_at_startup_runnable_classes=slugger.sagetv.sjq.SageJobQueue

Or something like this?
load_at_startup_runnable_classes=net.sf.sageplugins.webserver.StartServer, slugger.sagetv.sjq.SageJobQueue

I tried both way and I am not able to get SJQ loaded, each time I go to the webserver it stated SJQ is not started or installed etc.



Thanks
Bill

Put it on the same line separated by a semicolon. Be sure there are no extra spaces at the end of the line or between the semicolon, etc. as this will cause problems with SageTV loading the classes.

[TechBill]

Originally Posted by Slugger

Put it on the same line separated by a semicolon. Be sure there are no extra spaces at the end of the line or between the semicolon, etc. as this will cause problems with SageTV loading the classes.

Thank you for the guide!

SJQ is up and running for me now with the webserver after changing common to semicolon with no spaces.


Bill

[TechBill]

I can't seem to create any rules (cllicking on new rules in Ruleset Editor does nothing)

SO I looked at the plugin log and here a c/p from it

SJQ v1.1.0RC1
Started: Wed Feb 13 04:23:27 CST 2008

Reading ruleset from 'C:\Program Files\SageTV\SageTV\rules.sjq'
java.io.FileNotFoundException: rules.sjq (The system cannot find the file specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(Unknown Source)
at slugger.sagetv.sjq.RulesParser.(Unknown Source)
at slugger.sagetv.sjq.SageJobQueue.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
*** Sleeping until Wed Feb 13 07:03:00 CST 2008 ***


<< < > >> 102550100500
ID Type Program Completed Filename
No records found.

<< < > >> 102550100500



Did I missed a step installing it?

Thank you
Bill

[Slugger]

Originally Posted by TechBill

I can't seem to create any rules (cllicking on new rules in Ruleset Editor does nothing)

SO I looked at the plugin log and here a c/p from it

SJQ v1.1.0RC1
Started: Wed Feb 13 04:23:27 CST 2008

Reading ruleset from 'C:\Program Files\SageTV\SageTV\rules.sjq'
java.io.FileNotFoundException: rules.sjq (The system cannot find the file specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(Unknown Source)
at slugger.sagetv.sjq.RulesParser.(Unknown Source)
at slugger.sagetv.sjq.SageJobQueue.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
*** Sleeping until Wed Feb 13 07:03:00 CST 2008 ***


<< < > >> 102550100500
ID Type Program Completed Filename
No records found.

<< < > >> 102550100500



Did I missed a step installing it?

Thank you
Bill

No, I just think it's a bug where I'm assuming that a previous ruleset file already exists at startup.

Stop sage server and client(s). Create an empty file:
C:\Program Files\SageTV\SageTV\rules.sjq

Restart Sage. Does that fix it?

[TechBill]

Originally Posted by Slugger

No, I just think it's a bug where I'm assuming that a previous ruleset file already exists at startup.

Stop sage server and client(s). Create an empty file:
C:\Program Files\SageTV\SageTV\rules.sjq

Restart Sage. Does that fix it?

Yes it fixes it. Thanks

Now I am trying to create a rule for SA to be excuted when it start recording but I am wondering what about the ones already recorded which haven't been processed by the SA yet?



Thanks!

Bill

[Slugger]

Originally Posted by TechBill

Yes it fixes it. Thanks

Now I am trying to create a rule for SA to be excuted when it start recording but I am wondering what about the ones already recorded which haven't been processed by the SA yet?



Thanks!

Bill

Do you mean what to do about the ones that have already been processed by SA?

In the jar file there is a class called slugger.sagetv.sjq.SageCommCheck - this program will mark recordings that already have an edl or txt file created as processed so instead of using the SA executable as your action, call this program instead for the first run of SJQ:

java -cp "C:\Program Files\...\JARs\slugger_sage-job-queue.jar" slugger.sagetv.sjq.SageCommCheck %f%

Make sure you use the same action type as you will when calling SA. After this runs once then switch the executable to SA. Now SJQ will only run SA against recordings that don't have an edl or txt file.

If you indeed meant preventing SJQ from running SA against old recordings that don't already have an edl or txt file then you're going to have to be creative in your ruleset definition.

The easiest might be to set SJQ to run 2 mins after the start of each recording and add a test:

IsActivelyRecording equals true

This will ensure SA only ever runs on programs that are actively recording. However, if you have more than one tuner then it'll only ever process one recording at a time and miss the other(s).

A more robust solution would be to create a rule such that it excludes those older programs you don't want to be processed by SA. For example, are all of these recordings known to be commercial free? PPV/HBO, etc.? If so then just create a test that checks the channel the program was recorded from and exclude those recordings that come from channels you don't want SA to process.

[Slugger]

v1.1.0RC2 now available. Again, the RC builds are only available from my server and will disappear once v1.1.0 is released. All stable versions are available from the SageTV forums download area.

Fixed in this build:

* Bug 44: Handle case where there is no ruleset file available at plugin startup

[bluenote]

This is a great addon. I've been wanting something like this for ages. The only downside is having to carry webserver to run it (which, while great, I don't have a current use for atm.)

One thing that I'd love to be able to do (and maybe I already can?) is wait until files are X days old. This is going to be used for transcoding, and there's a lot of shows that I watch right away, and so don't make sense to be converted. Is there a way to do that?

Cory

[Slugger]

Originally Posted by cnovak

This is a great addon. I've been wanting something like this for ages. The only downside is having to carry webserver to run it (which, while great, I don't have a current use for atm.)

One thing that I'd love to be able to do (and maybe I already can?) is wait until files are X days old. This is going to be used for transcoding, and there's a lot of shows that I watch right away, and so don't make sense to be converted. Is there a way to do that?

Cory

You do not need the web server to run the plugin. Without the servlet, you simply have to write the ruleset file yourself. See the online docs for details on how to do that.

I've added a test to check the age of a recording to my todo list; look for it in the next build.

[Slugger]

Originally Posted by Slugger

You do not need the web server to run the plugin. Without the servlet, you simply have to write the ruleset file yourself. See the online docs for details on how to do that.

I've added a test to check the age of a recording to my todo list; look for it in the next build.

v1.1.0RC3 is now available.

Changes in this build:

* Bug 45: Add new test 'Age' that allows testing against the age of media files

The numbers you specify for the age of a media file must look like this:

nx where n is a positive integer (digits only) and x is one of the following letters:

S: seconds
M: minutes
H: hours
D: days
O: mOnths
Y: years

So to create a test that ensures the media object is more than 3 days old use the following:

> Age: 3D

To ensure the media object is at least 5 months old:

!< Age: 5O

* Not less than 5 months is equivalent to greater than or equal to 5 months

If using the servlet to setup your rules just be sure to follow the format for the numbers. Failure to follow the format will cause an exception, which in turn means the test will always return false and therefore your rule will always return false.

[bluenote]

Thanks slugger, I'll be trying this out tomorrow!

[bluenote]

Hey slugger

I didn't get to configure this as quick as I would have liked, but I'm working on it now. I'm having trouble with the EDL check class that you refer to above. I guess maybe I'm doing something wrong with the quotes, maybe you can help. I tried this from the webbrowser on the console thinking it might be path related but that didn't seem to help either. thanks for any pointers.


Here's my executable from the web interface:
java -cp "C:\Program Files\SageTV\SageTV\JARs\slugger_sage-job-queue.jar" slugger.sagetv.sjq.SageCommCheck "%f%"

(which gives me an error along the lines of 'the server thinks this is an invalid action, did you quote properly' etc.

EDIT: I thought I could check for the existence of a %targetfilenamewithoutextension%.EDL with the fileexists check, but I can't. Im hoping to use this to trigger commercial cutting, and i could use it as a backup for the java command above ^^. Ah! I just checked the docs again and it seems there is the facility (I thought so!) but from what I can see, I cannot make that entry through the web interface? I'll see if I can modify the rules file after the fact with that addition.

Thanks

Cory

[Slugger]

Originally Posted by cnovak

Hey slugger

I didn't get to configure this as quick as I would have liked, but I'm working on it now. I'm having trouble with the EDL check class that you refer to above. I guess maybe I'm doing something wrong with the quotes, maybe you can help. I tried this from the webbrowser on the console thinking it might be path related but that didn't seem to help either. thanks for any pointers.


Here's my executable from the web interface:
java -cp "C:\Program Files\SageTV\SageTV\JARs\slugger_sage-job-queue.jar" slugger.sagetv.sjq.SageCommCheck "%f%"

(which gives me an error along the lines of 'the server thinks this is an invalid action, did you quote properly' etc.

EDIT: I thought I could check for the existence of a %targetfilenamewithoutextension%.EDL with the fileexists check, but I can't. Im hoping to use this to trigger commercial cutting, and i could use it as a backup for the java command above ^^. Ah! I just checked the docs again and it seems there is the facility (I thought so!) but from what I can see, I cannot make that entry through the web interface? I'll see if I can modify the rules file after the fact with that addition.

Thanks

Cory

The servlet actually checks the exe before accepting it to ensure that it exists and is executable. (Currently) the servlet does not check the system path for exes and requires that you specify the full path, including the exe's file extension. To setup a rule from the servlet, I had to use this line as the executable name:

"C:\Program Files\Java\jre1.6.0_03\bin\java.exe" -cp "C:\Program Files\SageTV\SageTV\JARs\slugger_sage-job-queue.jar" "%f%"

Be sure to include all the quotes as I've done. Or, as you say, you can just add the action line to the ruleset file manually, but if you do that be sure to run the command line on your own to ensure it's valid and does something. Even if doing it yourself, the JRE seems to require the full file name of the exe (including extension) to execute a process.

[Slugger]

Originally Posted by cnovak

Hey slugger
EDIT: I thought I could check for the existence of a %targetfilenamewithoutextension%.EDL with the fileexists check, but I can't. Im hoping to use this to trigger commercial cutting, and i could use it as a backup for the java command above ^^. Ah! I just checked the docs again and it seems there is the facility (I thought so!) but from what I can see, I cannot make that entry through the web interface? I'll see if I can modify the rules file after the fact with that addition.

Thanks

Cory

A bug with the servlet input. I'm using the default input for boolean fields (true/false) instead of presenting a text input. That will definitely be fixed with the next build. If that isn't tonight then it may not be until the weekend - I've got a busy week ahead of me.

[Slugger]

The RC4 build is now available.

Changes in this build:

* Bug 46: FileExists test should provide text input in servlet instead of boolean input
* Bug 47: Move all logging to database

Please read the changelog closely for this build before applying. RC4 introduces my database logging code, which may not be completely stable, though I've been having success running this particular build.