Plugin: SJQv4

[davephan]

I just upgraded from SJQ version 3 to version 4. I installed both 'Sage Job Queue version 4.0.0.1380' and 'Sage Job Queue Agent (Task Client) version 4.0.0.1379' plug-ins. I then entered my e-mail address in the plug-in and it gave me a 'licensed' status.

A SJQ4 Client service was not created. I don't see SJQ on the SageTV 'setup' menu.

How do I get SJQ4 installed?

Dave

[Slugger]

Install the SJQ UI plugin as well (under UI mods).

[davephan]

Originally Posted by Slugger

Install the SJQ UI plugin as well (under UI mods).

I installed the SJQ UI plug-in, and still don't see SJQ under the SageTV 'setup' menu. However, when I browse to a recording, then I see 'Assign SJQ Tasks' as an option. I drill down into that menu, and I see 'My Tasks'. I can check or uncheck 'My Tasks', but can't see anything in 'My Tasks'.

Dave

[Slugger]

Let's make sure the SJQ engine and agent are running...

Is the SageTV\plugins\sjq\ dir on your server? Is there a file in there named sjq.log? Is it being written to? If so, we know the engine is installed and running.

Same thing for SageTV\plugins\sjq-agent\logs\. Is there a file in there named sjqagent.log? Is it being written to? If so, we know the agent is installed and running.

If the agent and engine are running properly then it seems to be just an issue with the UI plugin and I have to defer those issues to tmiranda. Best bet is to open a ticket and he'll address it as soon as he can.

If the agent nor engine are running properly then that's my department and I'll be happy to help you track down those problems.

[davephan]

Originally Posted by Slugger

Let's make sure the SJQ engine and agent are running...

Is the SageTV\plugins\sjq\ dir on your server? Is there a file in there named sjq.log? Is it being written to? If so, we know the engine is installed and running.

Same thing for SageTV\plugins\sjq-agent\logs\. Is there a file in there named sjqagent.log? Is it being written to? If so, we know the agent is installed and running.

If the agent and engine are running properly then it seems to be just an issue with the UI plugin and I have to defer those issues to tmiranda. Best bet is to open a ticket and he'll address it as soon as he can.

If the agent nor engine are running properly then that's my department and I'll be happy to help you track down those problems.

There is a service called 'SJQ Task Client for Windows', but it doesn't say anything about version 4. The SQJ log and SJQ-Agent logs are being created. There is something about 'My Task_precheck.groovy does not exist or is not readable. Test considered FAILED! Can you tell what the problem is from the log files?

Dave

[pkasin]

Can someone tell me what SQJv4 (or SQJ) does?

[Slugger]

The task client is a Sage plugin as you've installed it, not a Windows service (though there is also a Windows service task client available for SJQv4). The plugin task client runs inside of the Sage JVM and not as a Windows service so it will not show up in your Windows services list. That Windows Service you see now is the old v3 task client and is of no use to you. You can uninstall it (using the bat file available in the v3 task client installation dir).

The logs mean the engine and task client are running properly, which is good. Were you editing the sjqagent.properties file? Did you create a task? The errors you show in the logs just mean the script it was told to run for that configured task can't be found as specified so it failed the precheck. That's normal (in that it can't find the script so it bailed, as expected). To remove those errors, just remove that task def'n from the sjqagent.properties file in sjq-agent\conf\

Basically, SJQ engine and task client are up and running, there just seems to be an issue with the UI. Did you restart the UI after installing the UI plugin? You have to do that before the SJQ entries will appear on the menus. If so, open a ticket unless tmiranda sees these posts and replies.

[Slugger]

Originally Posted by pkasin

Can someone tell me what SQJv4 (or SQJ) does?

It's a multi-threaded, distributed task queue and scripting environment for SageTV. Use it to automate tasks (comskip, file moves, transcoding, etc.).

Online User Guide (with more details).

[Slugger]

@davephan I just looked at your server logs and it seems you still have SJQv3 installed and running alongside SJQv4. This is not good. I'm assuming you never installed SJQv3 as a Sage 7 plugin, but just carried it over from your Sage 6 to 7 upgrade? If so, you need to manually remove SJQv3 by deleting the sjq.context.xml file from SageTV\jetty\contexts\ (you can also remove the sjq.war file from SageTV\jetty\webapps\ AFTER removing the xml file).

[tmiranda]

@davephan - I suspect another Plugin has added an item to the Setup menu which causes the SJQ4 menu item to get hidden off the bottom of the screen. Cursor down one item past the last item, then cursor right, and the menus should appear.

Installing the "Scrolling Setup" UI Mod Plugin will make the Setup menu scrollable.

Tom

[davephan]

Originally Posted by Slugger

@davephan I just looked at your server logs and it seems you still have SJQv3 installed and running alongside SJQv4. This is not good. I'm assuming you never installed SJQv3 as a Sage 7 plugin, but just carried it over from your Sage 6 to 7 upgrade? If so, you need to manually remove SJQv3 by deleting the sjq.context.xml file from SageTV\jetty\contexts\ (you can also remove the sjq.war file from SageTV\jetty\webapps\ AFTER removing the xml file).

I used to use SJQ3 with Sage6. I guess I assumed that SJQ4 would replace any SJQ3 files. I have both sjq.context.xml and sjq.war files deleted now.

Originally Posted by tmiranda

@davephan - I suspect another Plugin has added an item to the Setup menu which causes the SJQ4 menu item to get hidden off the bottom of the screen. Cursor down one item past the last item, then cursor right, and the menus should appear.

Installing the "Scrolling Setup" UI Mod Plugin will make the Setup menu scrollable.

Tom

I found it! I'm amazed I didn't see it before!

Thanks for the help!


Dave

[Slugger]

Originally Posted by davephan

I used to use SJQ3 with Sage6. I guess I assumed that SJQ4 would replace any SJQ3 files. I have both sjq.context.xml and sjq.war files deleted now.

It would replace the v3 files if you removed the Sage v6 version of SJQ and replaced it with the version from the plugin manager when you upgraded from Sage v6 to v7. If you didn't then the SJQv4 plugin would not be aware that a Sage v6 plugin was installed. But as long as you removed the old version then you're fine.

[SteveW]

So, I've read every post in this thread trying to find my answer to my question (and I think my head is going to explode by the way )...

I also am seeking to implement the aforementioned "seek and destroy" method of firing comskips with SJQv4 to account not only for scheduled recordings, but also for when someone decides to throw in a manual record here and there, and also ignore comskipping of movie channels (ie The Movie Network).

I see a few mentions in here of people wanting to do the same thing. The scripting in this version is way over my head... I guess my questions are:

a) Is the capability to do that "seek and destroy but ignoring ones from these channels" currently possible using the current version of SJQv4?

b) If someone has already implemented this, could they share a sample of their scripts? If I could get my hands on a working sample I'd be able to work my way through it to set it up for my needs...

Tnx!

-S

[Slugger]

Originally Posted by SteveW

So, I've read every post in this thread trying to find my answer to my question (and I think my head is going to explode by the way )...

I also am seeking to implement the aforementioned "seek and destroy" method of firing comskips with SJQv4 to account not only for scheduled recordings, but also for when someone decides to throw in a manual record here and there, and also ignore comskipping of movie channels (ie The Movie Network).

I see a few mentions in here of people wanting to do the same thing. The scripting in this version is way over my head... I guess my questions are:

a) Is the capability to do that "seek and destroy but ignoring ones from these channels" currently possible using the current version of SJQv4?

Is it possible? Of course. With a full, complete scripting language, anything you can imagine is possible. Has anyone implemented it? I don't think so. I don't like being the gate keeper of scripting for two reasons:

1) I moved SJQ to the Groovy platform to get out of the script maintenance business. It's my hope that other users will contribute to the available scripts.

2) I got out of the scripting business because I wanted to narrow the scope of my involvement to that of providing the environment. I provide the task queue, the multi-threaded and distributed task execution environment and I leave the details of what tasks you do to each user/the community.

With that said, I don't mind whipping up some of these common/popular use case examples, but be forewarned:

1) If I don't plan on using it, the Groovy script(s) I produce for these scenarios will be lightly tested. Depending on what it does, you'll want to be careful.

2) I won't rush to do these example scripts. Again, if I'm not using it then creating/fixing/maintaining them won't be a priority.

3) I'm not picking on you specifically here, but it's my hope that users, where possible, start to explore Groovy on their own (kind of the "feed a man a fish vs. teach him to fish" idea). I think with the use of Groovy, I'm starting to build a rather powerful scripting platform for SageTV here and I really hope others beside myself start to explore and harness its power and share their solutions with the community.

b) If someone has already implemented this, could they share a sample of their scripts? If I could get my hands on a working sample I'd be able to work my way through it to set it up for my needs...

Tnx!

-S

My morning meetings were cancelled and watching the Oilers play is so depressing anymore that I just end up skimming through the games in about 20 minutes now so I might just find an hour or so to whip up a starting point for you this evening. But you have to define your requirements a little more clearly for me.

Do you want to periodically inspect every media file on your system and determine if it needs to be comskipped or do you want to handle the RecordingStarted event for any recording and then determine if that recording needs to be comskipped?

You decide and I'll whip up an example for your choice, leaving the other as an exercise for the reader.

[korben_dallas]

Originally Posted by pkasin

Can someone tell me what SQJv4 (or SQJ) does?

It is very powerful, but not for the faint-hearted. Advanced scripting skills are highly recommended for usage.

Also, it is somewhat limited without a paid license.

[Slugger]

@SteveW I didn't hear back from you so I went ahead and whipped up the periodic scan and queue example (as opposed to the event based handler). This script simulates the old SJQv3 way of doing things. Basically, configure this script to run via the SJQv4 crontab once every 10 mins, an hour, etc. and it will scan and process each media file, queuing up tasks for matching files. Read the script carefully and configure it to your needs before letting it loose on your media collection!

The script is available here.

The event based scan and queue script is, therefore, left as an exercise for the reader.

Happy scripting!

[davephan]

The SJQ4 GUI interface does not clearly display the video recording names in the queue, completed, or failed like the old SJQ3 did. Is there any way to display that information?

SJQ4 is working, but when it is running, my impression that it is running blind, processing the files with comskip. There isn't a nice GUI to track which files are being processed, which files are in the queue, and which files failed.

SJQ4 has advantages over SJQ3 for many things, but the lack of a good GUI to monitor the status seems like a step backwards, unless there is a way to do it and I am missing something.

Dave

[SteveW]

Originally Posted by Slugger

@SteveW I didn't hear back from you so I went ahead and whipped up the periodic scan and queue example (as opposed to the event based handler). This script simulates the old SJQv3 way of doing things. Basically, configure this script to run via the SJQv4 crontab once every 10 mins, an hour, etc. and it will scan and process each media file, queuing up tasks for matching files. Read the script carefully and configure it to your needs before letting it loose on your media collection!

The script is available here.

The event based scan and queue script is, therefore, left as an exercise for the reader.

Happy scripting!

Sorry Slugger... Spent most of my time after posting that trying to un-disaster our house while my wife and our new baby were away visiting the inlaws/new grandparents.

Thanks for that! I'll give it a spin. As long as there is a framework there, I should be able to tweak it up.

-S

[tmiranda]

Originally Posted by davephan

The SJQ4 GUI interface does not clearly display the video recording names in the queue, completed, or failed like the old SJQ3 did. Is there any way to display that information?

SJQ4 is working, but when it is running, my impression that it is running blind, processing the files with comskip. There isn't a nice GUI to track which files are being processed, which files are in the queue, and which files failed.

SJQ4 has advantages over SJQ3 for many things, but the lack of a good GUI to monitor the status seems like a step backwards, unless there is a way to do it and I am missing something.

Dave,

Open a ticket and I'll look into providing more information about what is happening. I generally agree that I need to show more status and some of that is in the works.

(Slugger - Have you looked at the update I sent to you?)

Tom

[Slugger]

Originally Posted by tmiranda

(Slugger - Have you looked at the update I sent to you?)

That email got lost in the clutter that is my inbox. Anyway, just installed it this morning and tested it out. Looks good except for one issue on your end and the UI update also discovered a bug in the task client on my end, which is now fixed in the repository. See your inbox for more details.