Plugin: SJQv4

[Slugger]

Originally Posted by bikesquid

Ummm, mostly because when I tried and it bombed I didn't want to ask yet another question... so I ran away.... After-all, if ain't "completely" broke, don't fix it. I assumed it would be as simple as "(path/scriptxxx.groovy).execute" but then I couldn't figure out how to pass variables back and forth between scripts.

Groovy scripts aren't executable directly within the OS. Actually, they aren't executable at all. You have to do a lot of extra work to try and do something like this. Instead, what I was suggesting was to rename your files and then add a separate comskip job to the task queue to perform the comskip. Such an approach does two things:

1) Simplifies this script by reducing the amount of "stuff" it's trying to do

2) Allows SJQ to better manage resources by properly distributing your comskip tasks

This isn't necessary, of course, but if you want to see how it's done take a look at my media_file_scanner.groovy script (in my scripts repository). Lines 65-75 in that script show how to add tasks to the SJQ task queue from Groovy (don't forget to include the import statements as well).

I do have a follow-up "under the hood" SJQ question though. Is there a way to inject a job into the queue from outside sage? I.E. once comskip is done I'd like to review the edits and then put the job back in to run a few more steps, such as cutting the comskip output from the file, handbrake etc. But as another process(s) and as the object file isn't a sagetv recording any longer it's difficult to find from within sagetv.

Yes, you can do it in Groovy (see above) or Java using the sjq-common.jar and sjq-listener.jar files. The code example referenced above, with a few Java-specific mods, will also work in Java. Using the SJQ API is the "approved" way to add tasks to the queue outside of Sage, but there are other ways. Namely, you could just directly connect to the SJQ database and insert tasks that way, but I discourage that approach for many reasons, the least of which is that messing around with the database directly will almost guarantee you corrupt the database, rendering SJQv4 useless. However, if you really know what you're doing and feel comfortable enough with doing it then it's a possibility. I won't lend much support in explaining this approach (because I won't support trying to fix it if something goes wrong).

[Slugger]

Here's a quick script for sending email from Groovy.

Notes:

• This example requires the latest task client version (v4.0.0.1482+); older versions will not be able to run this example unless you manually install the various required jars; IMPORTANT: standalone (only) task client upgrade readme
• If using SageGroovy, you must upgrade to v4.0.0.1482.0+ or manually install the required jars; IMPORTANT: upgrade readme
• For users using the Sage plugin version task client, v4.0.0.1482 is built and ready to be deployed, but is delayed while I work out some dependency jar issues with another developer; until the new version is available, this example will not work with the plugin version of the task client

More examples of how to send email via Groovy are here. In particular, the last page of that article uses the Groovy AntBuilder object to run the ant task for sending email. This is by far the easiest/most convenient way to send email from Groovy, but since under the hood the ant task uses the JavaMail API, you're also free to directly use the JavaMail API for sending email from Groovy (and the article linked to above shows an example of doing that).

private class EmailSettings {
    static def host = 'smtp.gmail.com'
    static def user = 'user@gmail.com'
    static def pwd  = 'password'
    static def port = 465
    static def subj = 'Email Subject'
    static def from = user // To send via Gmail, from addr must equal user addr
    static def to   = 'john@doe.com'
}

   def ant = new AntBuilder()
   ant.mail(mailhost: EmailSettings.host, mailport: "${EmailSettings.port}", subject: EmailSettings.subj, user: EmailSettings.user, password: EmailSettings.pwd, ssl: "true") {
       from(address: EmailSettings.from)
       to(address: EmailSettings.to)
       message('My message body')   
   }

This example shows how to send email via Gmail's server. It comes from a much larger script of mine and was stripped down to the bare minimum in order to show email working. You don't, for example, need to use a private class of constants for the settings; you could just use the values directly in the ant.mail() call.

Happy scripting!

[bikesquid]

I've built a script to do the commerical cutting using videoredo (which uses windows COM interface for automagic output) and though it works fine from sagegroovy it's failing from within sagetv. I'm guessing it's a permissions issue? I have set the COM+ service to an admin id for startup user with no success. Here's the script:

// def SJQ4_METADATA = ["SJQ4_ID":"4598125", "SJQ4_TYPE":"MediaFile"]

// remove the above line before using the scirpt in SJQv4.

import org.apache.commons.io.FilenameUtils
import org.apache.commons.io.FileUtils

private class Settings {
    static public final boolean TEST_MODE = false
}

def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())

if(mf == null) {  //make sure it's a valid file id
    println "Invalid media file id! [${SJQ4_METADATA['SJQ4_ID']}]"
    return 1
}

def title = ShowAPI.GetShowTitle(mf)
def Prefix = FilenameUtils.getBaseName(MediaFileAPI.GetFileForSegment(mf, 0).getAbsolutePath())
def Origfile = new File(Prefix + ".ts")
def Outputfile = new File(Prefix + ".mkv")
def Origdir = ((MediaFileAPI.GetParentDirectory(mf).toString()) + "\\")
def base = FilenameUtils.getBaseName(Origfile.getName())
def OrigExt = ("." + FilenameUtils.getExtension(Origfile.getName()))
def CutExt = ".ts"
def CutDir = "\\\\Sage-pc\\Mirror_Set\\Process_TV\\com_free\\"
def FullInputFile = Origdir + Origfile
def FullCutFile = CutDir + Origfile
println FullCutFile

// Cutting file using VideoRedo
def i = 1
def file = new File(FullCutFile)
while(file.exists())
    file = new File(CutDir, "$base-${i++}$CutExt")
println "New file is: $file"
FullCutFile = file

println "Cutting files that look like '${FullInputFile}' to '${FullCutFile}'."
def command = ('cscript.exe //nologo "C:\\Program Files (x86)\\VideoReDoTVSuite4\\vp.vbs " "' + FullInputFile + '" "' + FullCutFile + '" "/p:H.264 Transport Stream" /q /e')
println "full cut command will be '$command'"

if(!Settings.TEST_MODE) { // Are we running test mode, if not do this
        def proc = command.execute()
        def initialSize = 4096
        def outStream = new ByteArrayOutputStream(initialSize)
        def errStream = new ByteArrayOutputStream(initialSize)

            proc.consumeProcessOutput(outStream, errStream)
            proc.waitFor()
            println 'out:\n' + outStream
            println 'err:\n' + errStream

            } else {  // if we are in test mode do this
                println "Would run '$command' if test mode were disabled!"
            }

//relink ExternalID to new file

if(!Settings.TEST_MODE) { // Are we running test mode, if not do this
    def src = (FullCutFile.toString())
    println "fullpathfile is=$src"

    def sid = ShowAPI.GetShowExternalID(mf)
    println "ExternalID is ${sid}"

    if(!Utility.IsFilePath(src)) {
       println "${src} does not exist!"
       return 1
    }

    if(MediaFileAPI.GetMediaFileForFilePath(new File(src)) != null) {
       println "${src} is already a registered SageTV media file!"
       return 1
    }

    def show = ShowAPI.GetShowForExternalID(sid)
    if(show == null) {
       println "ShowEID ${sid} is invalid!"
       return 1
    }

    def mfadd = MediaFileAPI.AddMediaFile(new File(src), null)
    if(mfadd == null) {
       println "Failed to add media file!"
       return 1
    }
    
    if(!MediaFileAPI.SetMediaFileShow(mfadd, show)) {
       println "Failed to link show metadata to media file!"
       return 1
    }

    MediaFileAPI.MoveTVFileOutOfLibrary(mfadd)
    println "Imported '${src}' and linked it to ShowEID '${sid}'!"
} else {  // if we are in test mode do this
       println "Would have relinked the showID '${sid}' to the new file '${src}' if test mode were disabled!"
}
    

def wildBase = (Origdir + base +".*")
if(!Settings.TEST_MODE) { // Are we running test mode, if not do this
    FileUtils.deleteFile(wildBase)
} else {  // if we are in test mode do this
    println "Would delete '$wildBase' if test mode were disabled!"
}

The error log from the cscript command is "WScript.CreateObject: Could not create object named "VideoReDo.VideoReDoSilent".....

Any suggestions?

[Slugger]

If using the Sage plugin version of the task client, is the Sage service running as a user with sufficient permission to access the COM service?

If using the Windows service version of the task client, is that service running as a user with sufficient permissions?

The error means nothing to me, but if you think it's a permissions issue then I'd start by checking the permissions of the task client process.

[bikesquid]

Originally Posted by Slugger

If using the Sage plugin version of the task client, is the Sage service running as a user with sufficient permission to access the COM service?

That was it! Changing the Sage service to running as a user solved the premissions issue.

Thx again.

[bikesquid]

Two questions:

1) I'm having issues with cut/paste & copy/paste within sagegroovy. Sometimes it works, sometimes it insists on pasting a prior cut/copy. Seems if I use the windows global cut/paste it overrides a internal one... i.e. if I use CTRL+C/CTRL+V that works more often than internal menu option... but not always. Just an update as it may be me but may be related to the 64bit?

2) I'm trying to link the metadata with the new file. First step, as I understand it is to get the externalID, so I'm using:

def SJQ4_METADATA = ["SJQ4_ID":"4598527", "SJQ4_TYPE":"MediaFile"]
def sid = ShowAPI.GetShowExternalID("SJQ4_ID")
println "ExternalID is ${sid}"

but I get nothing back. Is theShowAPI.GetShowExternalID not using the SJQ4_ID?

[bikesquid]

Nevermind, I got it.... one of those copy/paste issues as I was building....

Code works as:

def SJQ4_METADATA = ["SJQ4_ID":"4598527", "SJQ4_TYPE":"MediaFile"]
def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())
def sid = ShowAPI.GetShowExternalID(mf)
println "ExternalID is ${sid}"

[bikesquid]

I'm lost... stuck with the delete phase of things. I know it's a problem with the argument type being passed to the deleteFile but have no idea what to do about it.... Any suggestions?


I'm trying to do a (I thought) simple equiv. to "del filename.*" using the groovy script process and can't figure out a way past the following error.

\\Sage-pc\Mirror_Set\Recorded TV\7Newsat5-4519451-0
Exception thrown

groovy.lang.MissingMethodException: No signature of method: static org.apache.commons.io.FileUtils.deleteFile() is applicable for argument types: (java.lang.String) values: [\\Sage-pc\Mirror_Set\Recorded TV\7Newsat5-4519451-0.ts]

    at cut$_run_closure1.doCall(cut.groovy:36)

    at cut.run(cut.groovy:32)

The offending code looks like this:

def SJQ4_METADATA = ["SJQ4_ID":"4606425", "SJQ4_TYPE":"MediaFile"]

// remove the above line before using the scirpt in SJQv4.

import org.apache.commons.io.FilenameUtils
import org.apache.commons.io.FileUtils

private class Settings {
    static public final boolean TEST_MODE = true
}

def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())

if(mf == null) {  //make sure it's a valid file id
    println "Invalid media file id! [${SJQ4_METADATA['SJQ4_ID']}]"
    return 1
}

def title = ShowAPI.GetShowTitle(mf)
def Prefix = FilenameUtils.getBaseName(MediaFileAPI.GetFileForSegment(mf, 0).getAbsolutePath())
def Origfile = new File(Prefix + ".ts")
def VideoReDoFile = new File(Prefix + ".VPrj")
def Outputfile = new File(Prefix + ".mkv")
def Origdir = (MediaFileAPI.GetParentDirectory(mf))
def base = FilenameUtils.getBaseName(Origfile.getName())
def OrigExt = ("." + FilenameUtils.getExtension(Origfile.getName()))
def CutExt = ".ts"
def CutDir = "\\\\Sage-pc\\Mirror_Set\\Process_TV\\com_free\\"
def wildBase = new File(Origdir, base)// + ".*")
println wildBase

Utility.DirectoryListing(Origdir).each {
    if(FilenameUtils.wildcardMatchOnSystem(it.getName(), "${Prefix}.*")) {
        def fileName = new File(Origdir, base + (it.getName().substring(it.getName().indexOf('.'))))
            try {
                FileUtils.deleteFile(fileName)
            }  catch(IOException e) {
               e.printStackTrace()
            }
    }
}

[Slugger]

It's telling you the FileUtils.deleteFile() call is invalid and offers no suggestions. So you head to the javadocs for FileUtils:

http://commons.apache.org/io/apidocs...FileUtils.html

You look for the deleteFile() method... ah, it doesn't exist. So you're calling a non-existent method.

As it turns out, java.io.File objects have their own delete() method. So this should work:

fileName.delete()

That call returns true or false so you may want to check the return value and handle failures accordingly.

[bikesquid]

Originally Posted by Slugger

It's telling you the FileUtils.deleteFile() call is invalid and offers no suggestions. So you head to the javadocs for FileUtils:

http://commons.apache.org/io/apidocs...FileUtils.html

You look for the deleteFile() method... ah, it doesn't exist. So you're calling a non-existent method.

As it turns out, java.io.File objects have their own delete() method. So this should work:

fileName.delete()

That call returns true or false so you may want to check the return value and handle failures accordingly.

Sorry, that seems so obvious when you say it. I'm having a hell of a time figuring out the error messages and what to look at to analyze. When it's a no signature of meathod it seems that usually I've screwed up the string/integer/file type so I'm conditioned to seek that problem/solution. I had been trying to use forceDelete with equal lack of success when I couldn't figure out .delete(). I think in that case I was banging away with .toString(), new File(fileName) trying to get a more understandable error message out of it. Too bad there's no 'clear and obvious error messages for noobs' plugin to sagegroovy's gui!

As it turns out I started this extravaganza with fileName.delete() but kept putting the fileName in the () and getting all sorts of non specific messages. I'm very unclear on how to interpret "public boolean delete()" I keep trying to stick the object of the command in the brackets....

[Slugger]

Originally Posted by bikesquid

As it turns out I started this extravaganza with fileName.delete() but kept putting the fileName in the () and getting all sorts of non specific messages. I'm very unclear on how to interpret "public boolean delete()" I keep trying to stick the object of the command in the brackets....

If you want to continue your learning, Google static methods vs. instance methods (or something like that). The Sage API is implemented as a series of static methods, which is why you have to pass the object as a parameter to the method call. There are reasons I won't bother getting into as to why it's like that.

But most methods in most APIs will be instance methods, which means you have an object, say of type File, and you call methods against that object (i.e. filename.delete()). If you read up on the differences between static methods and instance methods you should be able to see the differences.

Like I said, you'll become a Java programmer yet!

[bikesquid]

Originally Posted by Slugger

If you want to continue your learning, Google static methods vs. instance methods (or something like that). The Sage API is implemented as a series of static methods, which is why you have to pass the object as a parameter to the method call. There are reasons I won't bother getting into as to why it's like that.

But most methods in most APIs will be instance methods, which means you have an object, say of type File, and you call methods against that object (i.e. filename.delete()). If you read up on the differences between static methods and instance methods you should be able to see the differences.

That's helpful, thanks. I've started reviewing the java tutorials.... ... reluctantly....

I'm looking at the example media_file_scanner script to break up the mega_script and have that working more-or-less but for error testing, etc it would be helpful if there is a test I could run to see if a given task for a given ID was running, completed successfully, in queue, etc. Is there such a routine that I can use at various points? I'm using the scanner to scan for a file placed after I've reviewed proposed cuts and if the file exists put the cut process in the queue, which is working, but if the scanner comes around again before executed it's dropping another task for the same thing in the queue which is buggering things up. So I'd like to check if that task for that mediafile is already done, in process, queue'd etc.

Got a couple other uses for such a test lined up as well if it exists!

[Slugger]

You cannot query SJQv4 (reliably) about the execution and subsequent result of previous tasks. This is because SJQv4 purges old tasks from the queue periodically.

However, there is a "Slugger approved" way of doing what you want, but the burden is on the user. So as not to get lost in the maze of info buried throughout this thread, I've written up a wiki doc explaining what you need to do. That should point you in the right direction.

[bikesquid]

Originally Posted by Slugger

However, there is a "Slugger approved" way of doing what you want, but the burden is on the user. So as not to get lost in the maze of info buried throughout this thread, I've written up a wiki doc explaining what you need to do. That should point you in the right direction.

Brilliant! So in theory I could establish a custom metadata object at the end of media scanning as queue'd, change it to running at the start of execution of task and again to complete at the finale. So, again - in theory, I can have one scanning process reviewing the status of all current recordings and know where they are in the scheme of things.... Love it! (Doesn't mean I'll succeed at doing it, but I'll give it a go)

[Slugger]

Originally Posted by bikesquid

Brilliant! So in theory I could establish a custom metadata object at the end of media scanning as queue'd, change it to running at the start of execution of task and again to complete at the finale. So, again - in theory, I can have one scanning process reviewing the status of all current recordings and know where they are in the scheme of things.... Love it! (Doesn't mean I'll succeed at doing it, but I'll give it a go)

Yeah, so instead of using '1' as the value, you can use multiple values for various states. Set it to 'queued' when you queue, then when the actual task script starts, change it to 'running', then when it's done change it to 'success' or 'failed' based on the results of the task. Then you can query that metadata value to see what happened wherever you need to check.

The same type of strategy can be used as a (sketchy) form of IPC between OS process, SJQ tasks, etc. I say "sketchy" because metadata properties aren't really the best way to communicate between processes, but it's rather simple to implement and better than nothing. If reliability and fault tolerance are crucial then a more robust form of IPC should be used. However, for most use cases, metadata should be enough.

[bikesquid]

I'm trying to use of MediaFileAIP.GetMediaFiles("T") to get the same list that is generated by the sage ui for 'current recordings' and what it's returning seems to be everything, including archive. I'd 'like' to be getting the list of 'current recordings' (from sage ui) Anybody have a clue what the sage ui is polling to get the 'current recordings' list?

[Slugger]

Originally Posted by bikesquid

I'm trying to use of MediaFileAIP.GetMediaFiles("T") to get the same list that is generated by the sage ui for 'current recordings' and what it's returning seems to be everything, including archive. I'd 'like' to be getting the list of 'current recordings' (from sage ui) Anybody have a clue what the sage ui is polling to get the 'current recordings' list?

Have a look at MediaFileAPI.IsLibraryFile()

A TV recording is archived if MediaFileAPI.IsTVFile() == true && MediaFileAPI.IsLibraryFile() == true

You don't need to check IsTVFile() because you're already implicitly doing that check by passing the 'T' media mask to GetMediaFiles().

[bikesquid]

Originally Posted by Slugger

Have a look at MediaFileAPI.IsLibraryFile()

A TV recording is archived if MediaFileAPI.IsTVFile() == true && MediaFileAPI.IsLibraryFile() == true

You don't need to check IsTVFile() because you're already implicitly doing that check by passing the 'T' media mask to GetMediaFiles().

Yes, understood, but what I'm getting when doing the GetMediaFiles is a MUCH larger list (all the files are tv media, yes, but not just current recordings (and I've no idea if it's all tv or just part... the list seems to cut off at about 1000.)

Maybe there's something set incorrectly with the media files, but somehow the sage ui is getting the listing correct. So I guess the real question is: what's being considered 'current recordings' by the sage ui?

[Slugger]

GetMediaFiles('T') returns all tv files, always. If you only want "current" ones then you need to further filter the returned list yourself using the API calls above.

[tmiranda]

Slugger,

I want to change the backup scripts so that if it is not run by a certain time of day it will not be run at all on that day. Right now I have the script set to do a backup at 4:00 AM and I want to make sure that if it's not run by 8:00 AM it's not run at all.

Do you suggest I do this by making the agent only available between 4 AM and 8 AM, or do you suggest I check the time in the test script and return 2 if it's past 8 AM?

Tom