Plugin: SJQv4

[KarylFStein]

Originally Posted by phareous

Thanks I will try changing the slash.

On an unrelated note, does anyone have a script or tutorial on how to setup SQJ4 to clean out old .edl and .txt files?

Use sjqv3_file_cleaner.groovy from here.

[Slugger]

Originally Posted by phareous

Thanks I will try changing the slash.

On an unrelated note, does anyone have a script or tutorial on how to setup SQJ4 to clean out old .edl and .txt files?

http://sagetv-addons.googlecode.com/...nk/SJQScripts/

You're looking for the sjqv3_file_cleaner.groovy script in that directory.

[phareous]

Originally Posted by Slugger

My guess is you're passing "$SJQ4_PATH/$SJQ4_LAST_SEGMENT" as the exe args for the task? If so, change that slash to a backslash and that should fix you up. If your exe is a native Windows app then the slashes must always be backslashes in file paths, but if your exe is, for example, a Java app then the slash types (usually) won't matter. comskip is a Windows app and needs the slashes set correctly in this case.

I changed to a backslash. It now works ok on the local machine, but my two remote clients still give the same error (albeit with the backslash shown now). They still say permission denied - could not open file. I don't understand why its doing that

[Slugger]

Originally Posted by phareous

I changed to a backslash. It now works ok on the local machine, but my two remote clients still give the same error (albeit with the backslash shown now). They still say permission denied - could not open file. I don't understand why its doing that

Are the remote clients running the Windows service version of the task client? If so, did you change the user the service runs as from LocalSystem to an actual account? LocalSystem account cannot access network resources and so you must change the user the service runs as if it requires access to non local resources.

[phareous]

Originally Posted by Slugger

http://sagetv-addons.googlecode.com/...nk/SJQScripts/

You're looking for the sjqv3_file_cleaner.groovy script in that directory.

So dumb question - how do I call this? I tried putting it as the executable and left test script blank... but when it runs it just says failed and the logs are empty

If I try to do it as a test script, then I can't proceed since it requires something for the executable argument

[Slugger]

Originally Posted by phareous

So dumb question - how do I call this? I tried putting it as the executable and left test script blank... but when it runs it just says failed and the logs are empty

If I try to do it as a test script, then I can't proceed since it requires something for the executable argument

If it failed and the logs are empty in the UI then it's a client config issue (usually). Go to your sjqagent.log file and see what it has to say when it tried to run the task. Typical error is it can't read the groovy script.

(FYI: You should set it up as the exe with no test then config a cronjob to run it periodically for you.)

[phareous]

Originally Posted by Slugger

If it failed and the logs are empty in the UI then it's a client config issue (usually). Go to your sjqagent.log file and see what it has to say when it tried to run the task. Typical error is it can't read the groovy script.

(FYI: You should set it up as the exe with no test then config a cronjob to run it periodically for you.)

2011-03-22 22:22:18,054 ERROR [192_168_1_150-23347-264]: IOError
java.io.IOException: Cannot run program "c:\sage_data\fileclean.groovy": CreateProcess error=193, %1 is not a valid Win32 application
at java.lang.ProcessBuilder.start(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58)
at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:246)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:302)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:149)
at com.google.code.sagetvaddons.sjq.agent.ProcessRunner.runExternalExe(ProcessRunner.java:317)
at com.google.code.sagetvaddons.sjq.agent.ProcessRunner.runExe(ProcessRunner.java:206)
at com.google.code.sagetvaddons.sjq.agent.ProcessRunner.run(ProcessRunner.java:168)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: CreateProcess error=193, %1 is not a valid Win32 application
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 10 more
2011-03-22 22:22:18,069 INFO[ListenerClient]: Disconnected from 192.168.1.150:23347
2011-03-22 22:22:18,070 WARN [ProcessRunner]: Removing task from active list: SJQ4Task-192.168.1.150-23347-264

[Slugger]

Paste in a few more lines above that where the task is assigned to the client (I want to see the Task[...] data structure). But it looks like you are trying to run a groovy script as an exe, but to do that you must prefix it with script: (i.e. script:C:\path\to\file.groovy)

[phareous]

Originally Posted by Slugger

Paste in a few more lines above that where the task is assigned to the client (I want to see the Task[...] data structure). But it looks like you are trying to run a groovy script as an exe, but to do that you must prefix it with script: (i.e. script:C:\path\to\file.groovy)

Adding "script:" did it - thanks for your help

[phareous]

Originally Posted by Slugger

Are the remote clients running the Windows service version of the task client? If so, did you change the user the service runs as from LocalSystem to an actual account? LocalSystem account cannot access network resources and so you must change the user the service runs as if it requires access to non local resources.

I changed the user the services run as and now its working...thanks again for the help

[tmiranda]

Slugger,

Just an FYI. I've been using the backup scripts for a while now and they work great. I made two small tweaks to customize it for my situation and have not had any problems at all. I was using a batch file and the Windows scheduler but the SJQ solution is a lot better.

Thanks,

Tom

[rsagetv99]

I am getting the following error in the logs, does it mean that my db is corrupted?

2011-03-29 11:54:52,205 INFO  [TaskQueue]: Running queue processor now!
2011-03-29 11:54:52,236 INFO  [TaskQueue]: Assigned task 6114 of type 'SCANMEDIA' to Client[host=htpc:23344,state=ONLINE,lastUpdate=Tue Mar 29 11:54:52 EDT 2011]
2011-03-29 11:54:52,236 INFO[ListenerClient]: Disconnected from htpc:23344
2011-03-29 11:54:52,236 INFO[Listener]: Received connection from: /192.168.1.4:3560
2011-03-29 11:54:52,236 INFO  [Handler]: CMD: GETARGS :: PEER: /192.168.1.4:3560
2011-03-29 11:54:52,236 WARN  [TaskQueue]: Returning 'null' for 6114
2011-03-29 11:54:52,236 INFO  [Handler]: CMD: QUIT :: PEER: /192.168.1.4:3560
2011-03-29 11:54:53,236 INFO[Listener]: Received connection from: /192.168.1.4:3563
2011-03-29 11:54:54,642 INFO[Listener]: Received connection from: /192.168.1.4:3566
2011-03-29 11:54:55,236 INFO[ListenerClient]: Disconnected from htpc:23344
2011-03-29 11:54:55,236 INFO  [ActiveTaskManager]: Validated 1 running task(s)!
2011-03-29 11:54:55,236 INFO  [TaskQueue]: Scheduling queue processor for ~8 seconds from now!
2011-03-29 11:54:55,361 INFO  [Handler]: CMD: ADDTASK :: PEER: /192.168.1.4:3566
2011-03-29 11:54:55,408 INFO  [TaskQueue]: Scheduling queue processor for ~8 seconds from now!
2011-03-29 11:54:55,423 INFO  [Handler]: CMD: ADDTASK :: PEER: /192.168.1.4:3566
2011-03-29 11:54:55,486 INFO  [Handler]: CMD: ADDTASK :: PEER: /192.168.1.4:3566
2011-03-29 11:54:55,502 ERROR [Handler]: IOError: 192.168.1.4:3566
java.io.IOException: org.h2.jdbc.JdbcBatchUpdateException: NULL not allowed for column "VAL"; SQL statement:
INSERT INTO queue_metadata (id, var, val) VALUES (?, ?, ?) [90006-145]
	at com.google.code.sagetvaddons.sjq.server.TaskQueue.addTask(TaskQueue.java:221)
	at com.google.code.sagetvaddons.sjq.server.commands.Addtask.execute(Addtask.java:57)
	at com.google.code.sagetvaddons.sjq.listener.Handler.run(Handler.java:70)
	at java.lang.Thread.run(Unknown Source)
Caused by: org.h2.jdbc.JdbcBatchUpdateException: NULL not allowed for column "VAL"; SQL statement:
INSERT INTO queue_metadata (id, var, val) VALUES (?, ?, ?) [90006-145]
	at org.h2.jdbc.JdbcPreparedStatement.executeBatch(JdbcPreparedStatement.java:1105)
	at com.google.code.sagetvaddons.sjq.server.DataStore.setMetadata(DataStore.java:641)
	at com.google.code.sagetvaddons.sjq.server.DataStore.addTask(DataStore.java:613)
	at com.google.code.sagetvaddons.sjq.server.TaskQueue.addTask(TaskQueue.java:217)
	... 3 more
2011-03-29 11:54:55,533 ERROR [Handler]: IOError: 192.168.1.4:3563

[Slugger]

The DB isn't corrupted, the error means you're trying to insert a task into the queue with a null value for metadata. Exactly what task are you trying to insert and how?

[rsagetv99]

Originally Posted by Slugger

The DB isn't corrupted, the error means you're trying to insert a task into the queue with a null value for metadata. Exactly what task are you trying to insert and how?

Sorry, I've been having problems getting this setup the way SJQ3 was setup. I copied the media_scan script that was provided here (i think by battams?) and tried adding it to call the comskip script also provided here. I have another comskip task defined for SD which just looks for MPGs instead of TS files.

I probably have something set wrong, hopefully you can figure it out.

Here is my sjqagent.properties.

#Generated by SJQv4 agent
#Tue Mar 29 11:53:20 EDT 2011
task.comskipsd.rcmax=0
task.comskip.maxtime=86400
task.comskipsd.resources=20
task.scanmedia.rcmax=0
task.comskipsd.schedule=ON
task.scanmedia.exe=script\:u\:/sagescripts/media_file_scanner.groovy
task.comskipsd.maxprocs=2
task.comskipsd.maxtime=43200
task.comskipsd.maxtimeratio=1.0
task.comskip.rcmax=0
agent.port=23344
task.comskipsd.test=u\:/sagescripts/comskip_test.groovy
task.scanmedia.schedule=ON
task.scanmedia.maxprocs=1
task.comskipsd.testargs=mpg
task.comskip.args="$sjq4_path\\$sjq4_last_segment"
task.comskipsd.exe=c\:/program files/dragon global/showanalyzersuite/showanalyzerengine.exe
task.comskip.resources=60
task.comskipsd.rcmin=0
task.comskip.maxprocs=1
task.scanmedia.testargs=
task.comskip.schedule=ON
agent.schedule=ON
task.scanmedia.rcmin=0
task.scanmedia.args=
task.comskip.rcmin=0
task.comskip.test=u\:/sagescripts/comskip_test.groovy
task.comskip.testargs=ts
task.comskip.maxtimeratio=1.0
task.comskipsd.args="$sjq4_path\\$sjq4_last_segment"
task.scanmedia.maxtime=3600
agent.mapdir=
agent.resources=100
task.scanmedia.maxtimeratio=1.0
task.comskip.exe=c\:/program files/dragon global/showanalyzersuite/showanalyzerengine.exe
task.scanmedia.resources=20
task.scanmedia.test=

[Slugger]

"$sjq4_path\\$sjq4_last_segment"

That needs to be in caps in both places in the file.

Let's see media_file_scanner.groovy, there seems to be a bug in it (or at least in the version you've installed).

[rsagetv99]

It should be the latest, I just downloaded it a couple days ago.

/*
    Media File Scanner/Task Queuer
    
    Last Modified: 15 Feb 2011
           Author: Derek Battams <derek AT battams DOT ca>
    
    Use this script to periodically scan your media objects and check to see if
    any need to have tasks queued on them.  Basically, set up your media mask
    and then modify the needsProcessing() function to do the checks you want
    against each object.  If needsProcessing() returns true for an object then
    it will queue up each task listed in the taskIds list for that object.
    
    Typically, you would run this script periodically via the SJQv4 crontab.
    
    PLEASE RUN THIS SCRIPT WITH testMode = true BEFORE ALLOWING IT TO ACTUALLY
    QUEUE UP TASKS!!
*/

/***** CONFIGURE BELOW *****/

def testMode = false // If true, only print out which media files would be queued up, don't actually add the tasks to the queue
def mediaMask = "T" // What types of media should be scanned? (T = TV, M = Music, V = Imported Video, D = DVD, B = BluRay, P = Pictures; TMV = TV + Music + Imports, etc.)
def taskIds = ["COMSKIP","COMSKIPSD"] // Multiple tasks can be listed, separated by commas

/*
    Returns true if the argument needs to be queued or false if it should be skipped
    
    Modify this function to determine which media files get queued up and which don't
*/
def needsProcessing(Object mediaFile) {
    // This function could be written in a much more condensed manner, but I'm breaking it up for the sake of readability
    
    // So let's skip queuing this media file if it's live tv or an IR recording
    if(AiringAPI.IsNotManualOrFavorite(mediaFile))
        return false
    
    // Personally, I don't comskip until the recording is done, so don't queue up recordings in progress
    if(MediaFileAPI.IsFileCurrentlyRecording(mediaFile))
        return false

    // Let's also skip it if it's from a channel known not to have commercials (adjust the regex accordingly)
    if(AiringAPI.GetAiringChannelName(mediaFile) =~ /HBO.*|M(?:HD){0,1}|WPBS|.*PPV.*/)
        return false

    // Let's also skip it if there is already an edl file for the media file; adjust the extensions, if necessary        
    if(hasArtifacts(mediaFile, ["edl"])) // This function is defined at the bottom of the file
        return false
        
    // I don't like comskipping hockey and football games, I just have the skip ahead set properly on my remote so don't bother
    if(MediaFileAPI.GetMediaTitle(mediaFile) =~ /NFL Football|MLB Baseball|NHL Hockey|College Football|College Basketball/)
        return false
    
    // All our tests have passed so return true
    return true
}

/***** END CONFIG BLOCK *****/

/***** DO NOT MODIFY BELOW THIS LINE *****/

import com.google.code.sagetvaddons.sjq.network.ServerClient
import com.google.code.sagetvaddons.metadata.Factory
import org.apache.commons.io.FilenameUtils

def sc = !testMode ? new ServerClient() : null
MediaFileAPI.GetMediaFiles(mediaMask).each { mf ->
    if(needsProcessing(mf)) {
        if(!testMode)
            taskIds.each { id ->
                sc.addTask(id, Factory.getMap(mf))
            }
        else
            println "Would queue up '${MediaFileAPI.GetMediaTitle(mf)}' (${MediaFileAPI.GetMediaFileID(mf)}); skipped because test mode is TRUE"
    }
}
if(sc != null)
    sc.close()
return 0

// Returns true if any segment of the given media file has at least one artifact of any of the given artifact extensions; false otherwise
def hasArtifacts(Object mf, List exts) {
    for(def it : MediaFileAPI.GetSegmentFiles(mf)) {
        def absPath = it.getAbsolutePath()
        def dir = FilenameUtils.getFullPath(absPath)
        def base = FilenameUtils.getBaseName(absPath)
        for(def ext : exts) {
            def artifact = "${dir}${base}.$ext"
            if(Utility.IsFilePath(artifact))
                return true
        }
    }
    return false
}

[Slugger]

Ok, switch testMode to true and then run it again. It appears one of your media files is returning null for a value that is unexpected, which is causing the attempt to set null in the database. Run it in test mode, look at the output in the UI and report back the list of media files and ids it's attempting to queue up.

[bikesquid]

Got what I hope is a quick syntax question -
While working on a rename script I'm trying to rename all the associated files (wildcard the extensions) using this:

Trying to use SJQ4_SEGMENT_0_NAME to assign filename w/o ext to a variable.
Using the following:
def FileName = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA.get(SJQ4_SEGMENT_0_NAME))

and getting a missing method exception: No signature of method error in the log. Though the 'values' it shows in the log error is the correct file name. Any suggestions?

[Slugger]

MediaFileAPI.GetMediaFileForID() grabs the actual Sage object and expects an integer as the argument. You just want to grab the value from the env map:

def FileName = SJQ4_METADATA["SJQ4_SEGMENT_0_NAME"]

SJQ4_METADATA.get("SJQ4_SEGMENT_0_NAME") is also valid, I prefer for the former.

[bikesquid]

I'm not sure if this is progress, but the lines print ok, but getting an error with the moveFile command

def parentDir = MediaFileAPI.GetParentDirectory(mf)
def OldFileNameTS = (SJQ4_METADATA["SJQ4_PATH"] + "\\" + SJQ4_METADATA["SJQ4_SEGMENT_0_NAME"] + ".ts")
def OldFileNameVPrj = (SJQ4_METADATA["SJQ4_PATH"] + "\\" + SJQ4_METADATA["SJQ4_SEGMENT_0_NAME"] + ".VPrj")

FileUtils.moveFile(OldFileNameTS, new File(parentDir, targetFileNameTS))
FileUtils.moveFile(OldFileNameVPrj, new File(parentDir, targetFileNameVPrj))

error screenshot attached.

What I'd really like to do is just rename everything with filename.* to targetfilename.*, this is what I can figure out without a grasp of the lingo. It's crude but will work for my needs if I can get past birthing...

rename script log.png