Plugin: SJQv4

[Slugger]

Originally Posted by KarylFStein

I think you want your crontab entry to have a minute e.g. 0 4 * * *.

Indeed you definitely want a minute value other than * otherwise it will queue up 60 restart jobs every night and your server will be restarted 60 times each night. Pick a specific minute of the 4 o'clock hour you want the restart to happen.

[TechBill]

Originally Posted by Slugger

Indeed you definitely want a minute value other than * otherwise it will queue up 60 restart jobs every night and your server will be restarted 60 times each night. Pick a specific minute of the 4 o'clock hour you want the restart to happen.

I fixed the cron job setting.

It's executing the script every morning at 4am but in the SageTV web showing that the Task Queue is failing every night even it restarting SageTV service without a problem.

I am getting a fail message every morning but when I look at SageTV system message uptime, it shows me that service have been restarted each day without a problem.

Since it working, I will leave it alone unless there a way to fix it so it would show "Complete" instead of "Failed"?

Thank you

Bill

[Slugger]

Other than being annoying, the failed status really doesn't hurt anything. But to fix it, just increase the sleep time as needed for your system. For example, the script I use to do the restart looks like this:

Utility.ServerRestart()
sleep 300000
return 0

That sleep means the client waits 5 minutes before trying to tell the server it completed successfully. If that's not enough time for your Sage server to restart so it can accept the response from the client then increase it as needed.

Also, this only works if you're using the command line or Windows service version of the task client. If you're using the Sage plugin version of the task client then it will always report a failure because the task client is restarted along with the Sage server in that case and therefore will never be able to report the completion of the task. (Well, I could offer you some alternative code to actually make the plugin client report success, but it would just blindly report success, which is about as useful/meaningful as it always reporting failed.)

[TechBill]

Originally Posted by Slugger

Other than being annoying, the failed status really doesn't hurt anything. But to fix it, just increase the sleep time as needed for your system. For example, the script I use to do the restart looks like this:

Utility.ServerRestart()
sleep 300000
return 0

That sleep means the client waits 5 minutes before trying to tell the server it completed successfully. If that's not enough time for your Sage server to restart so it can accept the response from the client then increase it as needed.

Also, this only works if you're using the command line or Windows service version of the task client. If you're using the Sage plugin version of the task client then it will always report a failure because the task client is restarted along with the Sage server in that case and therefore will never be able to report the completion of the task. (Well, I could offer you some alternative code to actually make the plugin client report success, but it would just blindly report success, which is about as useful/meaningful as it always reporting failed.)

It what I thought, since it working and I will just ignore it.

Thank you for clearing it up.

Bill

[Slugger]

As of tonight, I have unsubscribed from this thread. What does this mean? It means I am no longer actively developing or supporting this plugin. All my plugins have actually been EOL'd since about 2012-2013, but tonight I unsubscribe from these threads which means I no longer get instant email notifications when people post. I'm not leaving, I'm still a user, I'm still around these forums, but I no longer support these out of date plugins and since I don't sub to these threads any longer, I may or may not notice your questions and therefore may or may not answer them -- assume that I won't, certainly not in a timely manner.

This also means I will never release another version of this plugin to the repository. I remain the owner of the plugin in the repository, but will approve a transfer if requested. No one has taken me up on that offer since my original announcement many weeks back. Unless someone does, there will be no more releases of this plugin.

[peternm22]

I've decided to try out this plugin again, but I've gotten stuck setting it up once again.

I've installed SJQ, the SJQ UI, and the SJQ Task client on my SageTV server.

I've setup a Comskip task. I've assigned this task to the RecordingStarted event.

I start a recording, and it shows up in the Queued Tasks screen. Comskip never launches. Whenever I go to the Queued Tasks screen, the Started/Completed times are updated to the exact time I enter the screen. The log files show no information.

How can I see what's going wrong? Also, how can I edit the Comskip task I setup to make sure I set it up correctly? I can't find anywhere to edit/view the task I created.

Thanks.

[KarylFStein]

Originally Posted by peternm22

How can I see what's going wrong? Also, how can I edit the Comskip task I setup to make sure I set it up correctly? I can't find anywhere to edit/view the task I created.

Thanks.

Go to manage clients and select the task from the drop down next to a client. My comskip task is a Groovy script. If yours is make sure you have "script:" in front of the executable name. Are you looking at the log files in the web GUI? If so maybe the log files on the server have more information in them (in SageTV/plugins/sjq and sjq-agent on the clients).

[peternm22]

Thank you! I found the problem, a misspelled path.

My Comskip task is just pointed to the comskip.exe file, and I'm using the Groovy comskip_test.groovy script as my test script (this is one of the sample scripts available from the setup guide).

In the past I've used Showanalyzer, but I'm giving Comskip a go to see how it works. In the Showanalyzer setup you could specify channels not to process, I don't think Comskip offers the same functionality so I'll have to use SJQ for this.

Is Groovy the best (only) way to exclude certain channels from processing in SJQ?

Sorry for asking all of these obvious questions, but this thread is 79 pages long, and a Google search isn't show me much relevant information.

[Denali]

Originally Posted by peternm22

In the Showanalyzer setup you could specify channels not to process, I don't think Comskip offers the same functionality so I'll have to use SJQ for this.

Have a look at Setup/Detailed Setup/Customize/ComSkip Playback Options/CommercialDetector/Do Not Run comskip on These Channels/

[peternm22]

I'm trying to do a more advanced Comskip task so that I can load different .ini files depending on the channel. Instead of using the executable and test script, I want to put everything inside of a single script. I then call this script as the executable like so: script:C:\Sage\Tools\comskip.groovy

My initial testing has been working so far, however I can't seem to find a way to mark a task as skipped.

if(ignoreLiveTv && AiringAPI.IsNotManualOrFavorite(mf)) {
   println "Recording is live TV or IR recording, skipping comskip!"
   return 2
}

The above code works in a test script (and marks the task as skipped). If I run the script as the primary executable though, the task is marked as failed.

It seems that SJQ only understands "return 2" in the context of a test script, and not an executable. Any way around this?

As a workaround, I could make two scripts. One as the test script (which first verifies whether or not the task should be skipped or not), and then an executable script which decides what .ini file to use, and the launches Comskip with that .ini file. Is this the best way to do this?

Is it possible to have a test script return a string value, and then have that string value passed as an argument to the exe? This way I would only need one groovy script.

[peternm22]

I'm going ahead with the two script method (test and executable), since I can't find a way to do everything in one script.

I have the basic logic working now. You can define different ini files by channel number (might change it to channel name later on), by simply adding this line: ruleList.push('first.ini>400|300|291') This should be self explanatory.

If the channel number isn't found in any of the ruleList's, then it just uses the default ini file. See code below:

ComskipPath = "C:/Program Files (x86)/Comskip/comskip.exe"
iniDirectory = "C:/Program Files (x86)/Comskip/"
DefaultIniFile = "comskip.ini"

def ruleList = []
ruleList.push('first.ini>400|300|291') //each line defines which channels to use a particular .ini with
ruleList.push('second.ini>255|652')

/****************************************************************************************/
/****************************************************************************************/
/****************************************************************************************/

Object mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA.get("SJQ4_ID").toInteger()) 
def channel = AiringAPI.GetAiringChannelNumber(mf)

file = SJQ4_METADATA.get("SJQ4_PATH") + "\\" + SJQ4_METADATA.get("SJQ4_LAST_SEGMENT")
fileType = SJQ4_METADATA.get("SJQ4_LAST_SEGMENT_TYPE") 

def path = new File(SJQ4_METADATA.get("SJQ4_PATH"))
def fileName = SJQ4_METADATA.get("SJQ4_LAST_SEGMENT")



def channelFound = false

ruleList.each { contents ->
	if(!channelFound) {
		def (inifile, channels) = contents.tokenize('>')
		channels = "/${channels}/"
		if(channel =~ channels) {
			channelFound = true
			println "Use '${inifile}' for channel ${channel}"
			runComskip(inifile)
			return 0
		}
	}
}

if(!channelFound) {
	println "Use '${DefaultIniFile}' for default processing on channel ${channel}"
	runComskip(DefaultIniFile)
}


def runComskip(String iniConfig) {
	arg = "--ini=${iniDirectory}${iniConfig}"

	def command = [ComskipPath, arg, file].execute()
    def p = command.waitFor() 
    p.consumeProcessOutput(System.out, System.err)
    def rc = p.waitFor()
    if(rc == 0 || rc == 1)
       return 0
    else
       return 1 

}

Everything works until it gets to the point where it executes the process. Comskip is launched and starts writing the .edl file, but Comskip just stops before it has finished processing. In Windows task manager, comskip.exe is just sitting there with zero CPU usage. I had to end the process.

This is what popped up in the log file afterwards:

javax.script.ScriptException: javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: java.lang.Integer.consumeProcessOutput() is applicable for argument types: (java.io.PrintStream, java.io.PrintStream) values: [java.io.PrintStream@f41f34, java.io.PrintStream@12cb871]
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:138)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:124)
	at com.google.code.sagetvaddons.sjq.agent.ScriptRunner.exec(ScriptRunner.java:114)
	at com.google.code.sagetvaddons.sjq.agent.ScriptRunner.main(ScriptRunner.java:175)
Caused by: javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: java.lang.Integer.consumeProcessOutput() is applicable for argument types: (java.io.PrintStream, java.io.PrintStream) values: [java.io.PrintStream@f41f34, java.io.PrintStream@12cb871]
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:335)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:132)
	... 3 more
Caused by: groovy.lang.MissingMethodException: No signature of method: java.lang.Integer.consumeProcessOutput() is applicable for argument types: (java.io.PrintStream, java.io.PrintStream) values: [java.io.PrintStream@f41f34, java.io.PrintStream@12cb871]
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)
	at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
	at Script1.runComskip(Script1.groovy:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1076)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:906)
	at groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:149)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(GroovyScriptEngineImpl.java:316)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
	at Script1.run(Script1.groovy:36)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:332)
	... 4 more

Any idea what is going wrong? It seems like it is having trouble with consumeProcessOutput().

I saw in some other examples that certain things (libraries?) need to be imported. Do I need those imports with this script? If so, which ones would i need?

Thanks.

[KarylFStein]

I use a two script method, but don't think it's necessary. I used the scripts here as templates:

https://sagetv-addons.googlecode.com...nk/SJQScripts/

Not knowing much about Java or Groovy, I think you don't want that extra def p and just use the methods from command. Here's my comskip task, (again just ripped from the samples):

def cmd = "C:\\comskip\\comskip.exe"
def failed = false

for(int i = 0; i <= SJQ4_METADATA["SJQ4_MAX_SEGMENT_INDEX"].toInteger(); ++i) {
   def segment = SJQ4_METADATA["SJQ4_PATH"] + "/" + SJQ4_METADATA["SJQ4_SEGMENT_${i}"]
   def exe = [cmd, segment]
   def edlFile = "${segment}.edl"
   if (Utility.IsFilePath(edlFile))
      continue;

   def p = exe.execute()
   p.consumeProcessOutput(System.out, System.err)
   p.waitFor()
   def rc = p.exitValue()
   if(rc < 0 || rc > 1) // Task failed, but run against the rest of the segments anyways...
      failed = true
}
return failed ? 1 : 0

[peternm22]

Originally Posted by KarylFStein

Here's my comskip task, (again just ripped from the samples):

What same did you use for this code? I looked at all of them, and the only comskip related samples were for just the test script. I couldn't see any for actually running comskip.exe themselves.

Thanks.

[KarylFStein]

Huh, beats me . Maybe it was posted in this thread somewhere. I know I didn't write it from scratch.

[peternm22]

Hot dang, it worked!

Thanks for the code. Time to clean things up a little and I'll post the finished code when I'm done.

[peternm22]

A new problem just showed up.

One of my encoders is a network encoder (an R5000), and on certain channels it can take 30 seconds before it starts writing data to the recording file. I'm also trying to do commercial detection while the show is being recorded (as I have always done with Showanalyzer)

Comskip doesn't like this and produces a few "Can not open file" in the log before giving up.

Is there anyway for SJQ to wait until the recording actually has data before calling Comskip.exe?

[KarylFStein]

Originally Posted by peternm22

A new problem just showed up.

One of my encoders is a network encoder (an R5000), and on certain channels it can take 30 seconds before it starts writing data to the recording file. I'm also trying to do commercial detection while the show is being recorded (as I have always done with Showanalyzer)

Comskip doesn't like this and produces a few "Can not open file" in the log before giving up.

Is there anyway for SJQ to wait until the recording actually has data before calling Comskip.exe?

What I do is set my comskip trigger task to run from crontab at 5 and 35 minutes after the hour. It also skips things that are not "manual or favorite" recordings. Maybe you could just add a file size check in the Groovy script and sleep 30 seconds a couple times before giving up trying to run comskip (?).

[peternm22]

Both good ideas. I have my Comskip task currently attached to the RecordingStarted event.

I'm using a slightly modified version of the comskip_test.groovy as my test. I have it set to only run Comskip on Manual and Favorite recordings.

I also added an ignore channel setting, where I enter in all the channels I don't want scanned (premium channels that have no commercials). These trigger a "return 2", which tells SJQ to make these tasks as skipped.

I'm just working on tweaking my main executable script. I still need to add logic to deal with recordings that have multiple segments (I see your code does that already). I'm also going to look into adding a filesize check like you suggested, and not fire comskip.exe until the file actually has data.

Thanks again.

[peternm22]

I think I've got all of the bugs worked out now. See below for the groovy script.

A few notes:

• At the top of the script you need to define some variables. The path to comskip.exe, the directory in which you store your .ini files (usually the same place as comskip.exe, but I wanted to give the script some flexibility). Also set the default .ini file use if the channel isn't matched in any of our ruleList's.
• You can add as many .ini ruleList's as you like. The format is: ruleList.push('amc.ini>AMCCHD|CIIIDT') where amc.ini is you ini file, and then a list of channels that should use this ini file. Make sure to keep the ">" character in there, it's what separates the ini file from the channel list. Each channel is separated with a "|" character.
• ruleList's are parsed in the same order as they are listed. So if you define the same channel name in two or more ruleList's, only the first one would be use.
• This script is designed to be placed in the executable when setting up a task in SJQ. For example: script:C:\Sage\Tools\sjqscript\multicomskip.groovy
• You still need to define a test file for the task. The test file can check if an edl file already exists, set comskip to not run on livetv, etc. I use a test file based on this one: http://sagetv-addons.googlecode.com/...ip_test.groovy
• I have some logic in the script to make sure that comskip.exe isn't launched until the recording actually has data in it. I did this because one of my network encoders can be a bit slow starting to write the file. The script checks to make sure the file isn't empty, if it is, it waits 30 seconds and tries again. It retries 3 times, so in total it will wait 90 seconds before giving up

I hope this script can help out people out. I know a little bit of programming so that helped, but groovy was completely new to me.

ComskipPath = "C:/Program Files (x86)/Comskip/comskip.exe"
iniDirectory = "C:/Program Files (x86)/Comskip/"
DefaultIniFile = "comskip.ini"

def ruleList = []

//each line defines which channels to use a particular .ini with
//just add another line with the same format to define additional files
ruleList.push('amc.ini>AMCCHD|CIIIDT') 
ruleList.push('second.ini>WDIVDT|ECAN')
ruleList.push('third.ini>FXCHD|WUHFDT|MCHD')

/****************************************************************************************/
/* No changes needed below this line                                                   **/
/****************************************************************************************/

Object mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA.get("SJQ4_ID").toInteger()) 
def channel = AiringAPI.GetAiringChannelName(mf)

def IniFileToUse = DefaultIniFile

for (String item : ruleList) {
	def (inifile, channels) = item.tokenize('>')
	channels = /${channels}/
	if(channel =~ channels) {
		println "Use '${inifile}' for channel ${channel}"
		IniFileToUse = inifile
		break
	}
}

if (IniFileToUse == DefaultIniFile) {
		println "Channel didn't match any rules. Use  default '${DefaultIniFile}' for channel ${channel}"
}

def failed = false

for(int i = 0; i <= SJQ4_METADATA["SJQ4_MAX_SEGMENT_INDEX"].toInteger(); ++i) {
   def arg = "--ini=${iniDirectory}${IniFileToUse}"
   def segment = SJQ4_METADATA["SJQ4_PATH"] + "/" + SJQ4_METADATA["SJQ4_SEGMENT_${i}"]
   
   File fileseg = new File(segment)

	for (i = 0; i < 3; i++) {
		if (fileseg.length() == 0) {
			println "File is empty. Sleeping 30 seconds."
			sleep(30000)
			if (i == 2) {
				println "File was empty, could not process."
				return 1
			}
		} else {
			break
		}
	}
   
   def exe = [ComskipPath, arg, segment]
   def edlFile = "${segment}.edl"
   if (Utility.IsFilePath(edlFile))
      continue;

   def p = exe.execute()
   p.consumeProcessOutput(System.out, System.err)
   p.waitFor()
   def rc = p.exitValue()
   if(rc < 0 || rc > 1) // Task failed, but run against the rest of the segments anyways...
      failed = true
}
return failed ? 1 : 0

[wayner]

Does anyone have SJQ v4 working under Sage V9? I have installed V9 and installed three plugins: SJQ, SJQ UI and SJQ Agent (Task Client). But I don't get a SJQ entry in my Setup menu as I do under my V7 server.

In addition the SQJ Web UI says "Not compatible" I wonder if that is really the case or if it just doesn't like V9 as the version number?