Plugin: SJQv4

[Slugger]

Since it's comskip, what's probably happened is that the recording for this task was deleted between the time it was queued and when it was first assigned to a client. This was a bug in the comskip scripts, but was fixed long ago.

Go back to the db and run this:

SELECT * FROM queue_metadata WHERE ID = 2721

Does the recording being referred to still exist in Sage? This is probably the problem. If it doesn't then you might be running older versions of the groovy scripts - newer versions should detect missing recordings and just drop the task from the queue.

You don't need to restart anything after disabling the plugin - it will just stop.

[jbuszkie]

I had already stopped the bad task before I got your message about accessing the DB. So that job I posted was a legit one. I looked at the metadata before
I killed it and it only had a time and ID thing IIRC.. Is a stopped task still in the DB?

BTW.. My comskip groovy file was from

Last Updated: 29 Oct 2010

Guess it's time to update!

Jim

[jbuszkie]

I updated my groovy file and I ran the task manually and it "SKIPPED".
I went back and ran the task manually queued with the old script and I got the same behavior. (NumberFormatException) So my guess it I must have queued up the task manually some time in the past. (or the file was deleted before the script was run)

Thanks for the help!

Jim

[Slugger]

Yeah... stop Sage, cleanup the .lobs.db directory as described here, update the groovy scripts, restart Sage. Everything should be good to go after that.

This is for everyone... it is very important to keep groovy scripts you install from others up to date. This is true of the ones I've written and supplied plus any others you may get from other sources. Though there are problems with H2 LOB column caching, it's my experience that the issues are only relevant when there's a buggy and/or misconfigured task in a user's SJQ setup. If the groovy scripts are up to date and tasks are configured to only run as required then I've never seen the .lobs.db issue chew up a user's hard drive. This issue of H2 eating up the OS drive with LOB cache files only seems to surface when tasks start running wild and that, in my experience, only happens when you're either using older (i.e. buggy) versions of groovy scripts or you've misconfigured the frequency with which your tasks are queued.

[peternm22]

I'm been trying to set this up today to run Showanalyzer on my recordings, but I can't get it to work.

I bought the licence key, followed the tutorial and installed all the necessary plugins, but my task always comes up as "failed". I've attached the relevant screenshots, but I can't figure out how to view the configuration for a task after it has been created so I don't have a screenshot of that. I did manage to find this in a config file:

#Generated by SJQv4 agent
#Mon Jul 18 12:27:51 PDT 2011
task.showanalyzer.maxtime=86400
task.showanalyzer.args="$SJQ4_PATH\\$SJQ4_LAST_SEGMENT"
task.showanalyzer.test=
agent.mapdir=
task.showanalyzer.exe=C\:/Program Files (x86)/Dragon Global/ShowAnalyzer/ShowAnalyzer/exe
task.showanalyzer.rcmax=0
agent.schedule=* * * * *
task.showanalyzer.maxprocs=3
agent.resources=100
task.showanalyzer.schedule=ON
agent.port=23344
task.showanalyzer.maxtimeratio=1.0
task.showanalyzer.resources=33
task.showanalyzer.rcmin=0
task.showanalyzer.testargs=

The path to ShowAnalyzer looks a little odd, but I followed the instructions in the tutorial. Any ideas?

[Slugger]

Is that config file copy/pasted verbatim?

If so, then it's incorrect:

task.showanalyzer.exe=C\:/Program Files (x86)/Dragon Global/ShowAnalyzer/ShowAnalyzer/exe

Should probably be ".exe" at the end instead of "/exe"

If you look at the task client log file, it will probably tell you the exe (as defined) doesn't exist. The task client log is in \SageTV\plugins\sjq-agent\logs\sjqagent.log - if you're running the plugin version of the task client. If you're running the standalone version then it'll be in the logs subdir of where ever you installed it.

EDIT: You say the exe value was used as seen in the tutorial. Can you point me to where you see that so I can correct it? I couldn't find any reference to the exe path as shown above, which is incorrect.

[peternm22]

Hi Slugger,

Thanks I got it working now.

This is the tutorial I am referring to: http://code.google.com/p/sagetv-addo...i/Sjq4Comskip1

It's the first image. All of the slashes in the executable path are "/", so that's what I entered: C:/Program Files (x86)/Dragon Global/ShowAnalyzer/ShowAnalyzer.exe, and then it was all butchered in the properties file. Manually editing the file fixed it.

Could just be user error on my part though

[peternm22]

More questions:

1) Is it possible to only launch a task only on favourites and manual recordings? I know I can run a task on specific favourites, but I don't see a way of excluding livetv. I seem to recall you could customize this in earlier versions via the web interface.

2) I've been trying to use SAD to delete old EDL files, but it doesn't seem to always work. Specifically on live tv (if #1 is fixed, this won't be a problem). Also if a recording has multiple parts (-0, -1, etc), any edl files that aren't -0 don't get deleted. Is there anyway to run SAD on a schedule as well to scan for all orphaned EDL files? I know I can do this with Dirmon2, but I was hoping to avoid running a separate program for this task (it's the reason I wanted to use SJQ in the first place).

Thanks.

[graywolf]

Originally Posted by peternm22

More questions:

1) Is it possible to only launch a task only on favourites and manual recordings? I know I can run a task on specific favourites, but I don't see a way of excluding livetv. I seem to recall you could customize this in earlier versions via the web interface.

2) I've been trying to use SAD to delete old EDL files, but it doesn't seem to always work. Specifically on live tv (if #1 is fixed, this won't be a problem). Also if a recording has multiple parts (-0, -1, etc), any edl files that aren't -0 don't get deleted. Is there anyway to run SAD on a schedule as well to scan for all orphaned EDL files? I know I can do this with Dirmon2, but I was hoping to avoid running a separate program for this task (it's the reason I wanted to use SJQ in the first place).

Thanks.

You could put the following in your Test Script, or your Exe Script if you don't have a Test Script

String id   = SJQ4_METADATA.get("SJQ4_ID");
Object mf = MediaFileAPI.GetMediaFileForID(id.toInteger());

if ( AiringAPI.IsNotManualOrFavorite(mf) ) {
    println("${FileName} is not a Favorite or Manual Recording - skipped")
    return 2
}

[Slugger]

Originally Posted by peternm22

More questions:

1) Is it possible to only launch a task only on favourites and manual recordings? I know I can run a task on specific favourites, but I don't see a way of excluding livetv. I seem to recall you could customize this in earlier versions via the web interface.

2) I've been trying to use SAD to delete old EDL files, but it doesn't seem to always work. Specifically on live tv (if #1 is fixed, this won't be a problem). Also if a recording has multiple parts (-0, -1, etc), any edl files that aren't -0 don't get deleted. Is there anyway to run SAD on a schedule as well to scan for all orphaned EDL files? I know I can do this with Dirmon2, but I was hoping to avoid running a separate program for this task (it's the reason I wanted to use SJQ in the first place).

Thanks.

Originally Posted by graywolf

You could put the following in your Test Script, or your Exe Script if you don't have a Test Script

String id   = SJQ4_METADATA.get("SJQ4_ID");
Object mf = MediaFileAPI.GetMediaFileForID(id.toInteger());

if ( AiringAPI.IsNotManualOrFavorite(mf) ) {
    println("${FileName} is not a Favorite or Manual Recording - skipped")
    return 2
}

And instead of SAD, use this script. Setup this script to run via the SJQv4 crontab periodically and it'll take care of deleting all the artifacts, even preexisting orphaned ones.

And if you're using this script as your comskip pretest then it already has a check for live tv and it's basically the same code as graywolf has provided above.

[peternm22]

Thanks, I'll try those out tonight when I get a chance.

SJQ is certainly a powerful little beast isn't it It seems quite simple on the surface, but there is a LOT you can do with it.

[graywolf]

Where do I find the log files for Tasks?

Trying to track down an issue I have with the move script and it would be easier for me to cut/paste from a log file than trying to copy it from the screen in order for me to continue the research.

[Slugger]

The logs are in the SJQ database. If you want to get access to them, you'll have to go through a manual connection to the db.

[graywolf]

Thanks. Helpful.

OK. So it appears that for some reason the move failed. Possibly Comskip was running? And therefore unable to delete the source filename.mpg.tmp.

Since the move failed, I "assume" it deletes any remnant on the destination location as I didn't find any files there.

And it appears that it left the source file as filename.mpg.tmp, most likely once again due to Comskip.

Guess I'll need to look into my timing of events. Maybe uninstall Comercial Detector and just run SJQ tasks for Comskip.

Question: Is there a "sleep" command in groovy that I could add onto my script? Reason is I have the move script triggered by recording end.

LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties
----- stdout -----

Parm passed = \\tower\unarchived\br\The O'Reilly Factor
New File = \\tower\unarchived\br\The O'Reilly Factor/TheOReillyFactor-3106285-0.mpg
Moving [C:\SageRecordings\TheOReillyFactor-3106285-0.mpg] to destination: \\tower\unarchived\br\The O'Reilly Factor
Failed to move file to destination!
------------------


----- stderr -----

java.io.IOException: Failed to delete original file 'C:\SageRecordings\TheOReillyFactor-3106285-0.mpg' after copy to '\\tower\unarchived\br\The O'Reilly Factor\TheOReillyFactor-3106285-0.mpg'
at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:1821)
at org.apache.commons.io.FileUtils.moveFileToDirectory(FileUtils.java:1856)
at org.apache.commons.io.FileUtils$moveFileToDirectory.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:132)
at Script1.run(Script1.groovy:70)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:315)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:111)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:103)
at com.google.code.sagetvaddons.sjq.agent.ScriptRunner.exec(ScriptRunner.java:111)
at com.google.code.sagetvaddons.sjq.agent.ScriptRunner.main(ScriptRunner.java:164)
------------------



==============================
===== Thu Jul 21 21:01:08 EDT 2011 =====

LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties
----- stdout -----

Test Started
This is a TV show
Modified OutputPath = \\tower\unarchived\br\The O'Reilly Factor
Would Start EXE with argument \\tower\unarchived\br\The O'Reilly Factor
Test Ended
------------------




==============================

[Slugger]

sleep x

Where x is a positive integer and represents the number of milliseconds the script should sleep.

And timing is everything... if something else is using the file then the delete will fail and that will trigger everything to be rolled back. Control comskip via SJQ and ensure that comskip uses as many or more resources than your move task, and then have your move task check for an edl before moving. This will ensure that comskip completes before a move is attempted. Of course, you have to handle the edge cases where some recordings may not be comskipped, etc. Point being, you definitely have to be aware of all external processes using a file before performing actions such as deleting them on it.

[graywolf]

Thanks for the confirmation and letting me know that sleep is in milliseconds.

I'll probably uninstall/disable Commercial Detector for the time being and see about having SJQ run Comskip after the file gets moved.

Need to put in some rules to exclude certain channels but that should be easy with the SJQ4_METADATA info.

[graywolf]

Guess I need to do some more reading.

Thinking about how to add to Move script a method to create a System Message (so it is more noticable) or maybe even e-mail capability on Move Failures. At least until I get everything running smoothly.

[graywolf]

Originally Posted by graywolf

Guess I need to do some more reading.

Thinking about how to add to Move script a method to create a System Message (so it is more noticable) or maybe even e-mail capability on Move Failures. At least until I get everything running smoothly.

Did some poking around and I think the following should fit the bill, in case anyone else was thinking about it and missed it first, second, third....reading of the docs.

NEW: Generate System Message on Task Failure

Starting with engine version 4.0.0.1392, you can configure your tasks to generate a SageTV system message when they fail. Simply add the sysmsgonfail property to your tasks:

task.mytask.sysmsgonfail=true

Any task with this setting will generate a warning level system message in SageTV when the task's exe fails. Nothing is generated when the test script fails.

NOTE: A future version of the SJQ UI may include the ability to modify this setting. Until then, you must directly edit the sjqagent.properties file to set this option.

[peternm22]

I've got all of the groovy scripts running now. One question about the comskip script.

I don't want it run Showanalyzer on live tv (and I have this configured), but say if I start watching a show as live tv but then decide to record it as a manual recording. In this scenario, I would want Showanalyzer to launch when it sees the recording status has been changed from live to manual. Currently, it doesn't seem to do this. Once it has been "skipped", a status change to that recording doesn't seem to make it re-enter the queue.

I can manually add the task to the recording from the options menu, but is there anyway to do this automatically?

Thanks.

[KarylFStein]

Originally Posted by peternm22

I've got all of the groovy scripts running now. One question about the comskip script.

I don't want it run Showanalyzer on live tv (and I have this configured), but say if I start watching a show as live tv but then decide to record it as a manual recording. In this scenario, I would want Showanalyzer to launch when it sees the recording status has been changed from live to manual. Currently, it doesn't seem to do this. Once it has been "skipped", a status change to that recording doesn't seem to make it re-enter the queue.

I can manually add the task to the recording from the options menu, but is there anyway to do this automatically?

Thanks.

I run the media_file_scanner script every few minutes to check for things that need comskip. Not what you were asking, but I believe will catch what you describe.