Plugin: SJQv4

[Slugger]

I don't see any reason why it's happening. The only thing I can think of is that you're queuing two tasks ["CUT", "HANDBRAKE"] and the "CUT" task is also running while "HANDBRAKE" is running and deleted the original MediaFile object from Sage such that when handbrake script tries to reinit it, it can't.

Alternatively, print out the value of SJQ4_METADATA['SJQ4_ID'] right before you reinit the object. If that number still looks right then instead of reinitializing by calling GetMediaFileForID(), try GetMediaFileForFilePath() and pass it the original file. If that also returns null then something is deleting the media file object in Sage (or there's a bug in the sagex-apis, maybe on the remote side). The sagex-api bug theory can be put to rest by loading up Studio and trying to load the object from Studio. If it returns null from there then something's deleting it, if not then it's probably a sagex-api bug (assuing the SJQ4_ID var is still right in your script at the time of reinit).

[Korny]

Slugger, I'll submit this to a ticket once I figure out how to use it
But here are the two scripts(sh and groovy"

  // Assume the first argument of the test script is the path, as given by the SageTV core
def mediaFile = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA['SJQ4_ID'].toInteger())
def fileName = "${SJQ4_METADATA['SJQ4_PATH']}/${SJQ4_METADATA['SJQ4_LAST_SEGMENT']}"
def showTitle = MediaFileAPI.GetMediaTitle(mediaFile)
def episode = ShowAPI.GetShowEpisode(mediaFile)

    if(!Tools.setExeArgs("\"$fileName\" \"$showTitle\" \"$episode\"" ))
       return 1 // If the call fails then return the task to the queue and try again later...
     return 0 // The exe args have been modified, return success and fire off the exe!!

And

#!/bin/bash
filename=$1
showtitle=$2
episode=$3
cp $filename "/media/Array1/needsprocess/$showtitle.$episode.mkv"

My actual bash file is much bigger but only needs the 3 above arguments to do what I need
Its still outputting a file by the name of "Family Guy"."Lois 911".mkv
ideally it should be Family Guy. Lois 911.mkv I'm not worred about spaces at this time, I'm using another program to rename files after this process but the quotation marks throw it off.
What exactly do you want me to put on the ticket?

Originally Posted by Slugger

Double quote everything... that's the safest way.

if(!Tools.setExeArgs("\"$fileName\" \"$showTitle\" \"$episode\""))

If that's not working then open a ticket and I'll have to do some testing on Linux to try and reproduce you're results and then fix it.

[Slugger]

@Korny Just open a ticket and link to the post above.

[tmiranda]

Here is an interim release of the UI that adds a "Task Overview" menu which allows you to see the state of Tasks. Clicking on one of the tabs across the top row filters by Task state. Clicking on the row just below that sorts by that field. I'm too lazy to add a screen shot but once you look at the menu it should be obvious.

To install: Download the attached .zip file and place the extracted .xml in the install directory of your Sage server. Then go to the plugin manager and update the SJQ4 UI Mod.

I believe there is a bug in this release, but I don't remember what it is

Please post feedback here.

Tom

[Slugger]

Originally Posted by Korny

Slugger, I'll submit this to a ticket once I figure out how to use it
But here are the two scripts(sh and groovy"

  // Assume the first argument of the test script is the path, as given by the SageTV core
def mediaFile = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA['SJQ4_ID'].toInteger())
def fileName = "${SJQ4_METADATA['SJQ4_PATH']}/${SJQ4_METADATA['SJQ4_LAST_SEGMENT']}"
def showTitle = MediaFileAPI.GetMediaTitle(mediaFile)
def episode = ShowAPI.GetShowEpisode(mediaFile)

    if(!Tools.setExeArgs("\"$fileName\" \"$showTitle\" \"$episode\"" ))
       return 1 // If the call fails then return the task to the queue and try again later...
     return 0 // The exe args have been modified, return success and fire off the exe!!

And

#!/bin/bash
filename=$1
showtitle=$2
episode=$3
cp $filename "/media/Array1/needsprocess/$showtitle.$episode.mkv"

My actual bash file is much bigger but only needs the 3 above arguments to do what I need
Its still outputting a file by the name of "Family Guy"."Lois 911".mkv
ideally it should be Family Guy. Lois 911.mkv I'm not worred about spaces at this time, I'm using another program to rename files after this process but the quotation marks throw it off.
What exactly do you want me to put on the ticket?

For others hitting this issue, the bug has been fixed in svn, but an updated build will be delayed until approximately next weekend as there are other fixes in svn that require some coordination with another plugin dev before I can release the build.

EDIT: Standalone task client builds are available now with this fix in it (4.0.0.1509). The plugin version of this build will be added to the plugin repository asap (hopefully by next weekend).

[bikesquid]

Originally Posted by Slugger

I don't see any reason why it's happening. The only thing I can think of is that you're queuing two tasks ["CUT", "HANDBRAKE"] and the "CUT" task is also running while "HANDBRAKE" is running and deleted the original MediaFile object from Sage such that when handbrake script tries to reinit it, it can't.

The tasks are run sequentially in that the handbrake.test.groovy is waiting for the custom metadata field being set to 'cut_done'. The cut.groovy should be complete before it begins, but it is reassigning the external ID to the new file while the handbrake.groovy script is 'waiting'... and maybe this is screwing things up? When looking at the sageui, there are times when the show is shown twice, though the original has been deleted... till sage rescans... This may be the issue? The results are inconsistent, but I'm not sure there's really a pattern.....

[Slugger]

The fact you get the expected results when running from SageGroovy, but not when running from SJQ means there's some variable in the mix causing the reinit to fail. So you either need to find out what's causing the reinit to fail (I'm fairly certain it's a bug in your process) or work around it by capturing all the relevant data you need from the object at the beginning of the script and not having to reinit the Sage object after the handbrake process finishes.

[bikesquid]

Originally Posted by Slugger

The fact you get the expected results when running from SageGroovy, but not when running from SJQ means there's some variable in the mix causing the reinit to fail.

But,.. But,... But... when I'm running from SageGroovy I'm starting each task manually and passing it the ID rather than starting both from a 'seed' script and having one wait, so it's not truly and apples to apples comparison.

I understand what you're saying about there being something wrong with my process.... I suspect the reassignment of metadata to the new file within the cut script while the handbrake script waits.... i'd like to bypass that. I wonder if it's possible to do,... and if so what the syntax would be, to queue a task from within a task and pass it the new file variable?

Though this isn't working, something along the lines of:

ServerClient.addTask("//Sage-tv/c/Program Files (x86)/SageTV/scripts/Hand_Brake.groovy", mf)?

[Slugger]

Originally Posted by bikesquid

Though this isn't working, something along the lines of:

ServerClient.addTask("//Sage-tv/c/Program Files (x86)/SageTV/scripts/Hand_Brake.groovy", mf)?

addTask(TASK_ID, Factory.get(mf))

Where TASK_ID is an SJQ configured task name capable of running Hand_Brake.groovy and you have to import com.google.code.sagetvaddons.metadata.Factory into your groovy script so the call to Factory.get() works. So you don't add tasks to the queue by exe, you add tasks by configured task id using the task ids configured in your task client(s).

[davephan]

Originally Posted by tmiranda

Here is an interim release of the UI that adds a "Task Overview" menu which allows you to see the state of Tasks. Clicking on one of the tabs across the top row filters by Task state. Clicking on the row just below that sorts by that field. I'm too lazy to add a screen shot but once you look at the menu it should be obvious.

To install: Download the attached .zip file and place the extracted .xml in the install directory of your Sage server. Then go to the plugin manager and update the SJQ4 UI Mod.

I believe there is a bug in this release, but I don't remember what it is

Please post feedback here.

Tom

I just installed SJQ4 again. I then put the .xml file in C:\program files (x86)\SageTV\SageTV. I'm not sure where the plugin manager needs to be updated, that isn't clear to me. I don't understand how I can see the SJQ4 UI.


Edit: I found the SJQ4 UI plug-in that was missing. I'll have to go through the docs and the very long SJQ4 thread to get SJQ4 configured.

Dave

[tmiranda]

Originally Posted by davephan

Edit: I found the SJQ4 UI plug-in that was missing. I'll have to go through the docs and the very long SJQ4 thread to get SJQ4 configured.

There is a learning curve, but it's not as bad as you might think.

[bikesquid]

Originally Posted by Slugger

addTask(TASK_ID, Factory.get(mf))

Where TASK_ID is an SJQ configured task name capable of running Hand_Brake.groovy and you have to import com.google.code.sagetvaddons.metadata.Factory into your groovy script so the call to Factory.get() works. So you don't add tasks to the queue by exe, you add tasks by configured task id using the task ids configured in your task client(s).

Having some syntax problem with this I think. I've tried this with various versions of quotes etc. The TASK_ID, if I'm understanding you, is the same task as called before by the media scanning script....

def SJQ4_METADATA = ["SJQ4_ID":"4695793", "SJQ4_TYPE":"MediaFile"]
import com.google.code.sagetvaddons.metadata.Factory
def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())
addTask(HANDBRAKE, Factory.getMap(mf))

no success. I'm getting this:

groovy.lang.MissingMethodException: No signature of method: cut.addTask() is applicable for argument types: (java.lang.String, java.util.HashMap) values: [HANDBRAKE, [SJQ4_LAST_SEGMENT_TYPE:ts, SJQ4_SEGMENT_0_NAME:Dual Survival - Split Up, SJQ4_SEGMENT_0_TYPE:ts, SJQ4_MAX_SEGMENT_INDEX:0, SJQ4_PATH:\\Sage-pc\Mirror_Set\Process_TV\com_free, SJQ4_SEGMENT_0:Dual Survival - Split Up.ts, SJQ4_LAST_SEGMENT:Dual Survival - Split Up.ts, SJQ4_TYPE:MediaFile, SJQ4_LAST_SEGMENT_NAME:Dual Survival - Split Up, SJQ4_ID:4695793]]

[Slugger]

sc.addTask(TASK_ID, Factory.get(mf))

Where sc is ServerClient instance:

import com.google.code.sagetvaddons.sjq.network.ServerClient
import com.google.code.sagetvaddons.metadata.Factory

def sc = new ServerClient()
def taskId = 'MY_TASK'
def metadata = Factory.get(mf) // mf is a MediaFile object initialized somewhere above
sc.addTask(taskId, metadata)
sc.close()

Have a look at my media file scanner script as a full example of how to add tasks into the queue from Groovy.

[bikesquid]

Originally Posted by Slugger

sc.addTask(TASK_ID, Factory.get(mf))

Where sc is ServerClient instance:

import com.google.code.sagetvaddons.sjq.network.ServerClient
import com.google.code.sagetvaddons.metadata.Factory

def sc = new ServerClient()
def taskId = 'MY_TASK'
def metadata = Factory.get(mf) // mf is a MediaFile object initialized somewhere above
sc.addTask(taskId, metadata)
sc.close()

Have a look at my media file scanner script as a full example of how to add tasks into the queue from Groovy.

.

Yup, was doing that at first,... was getting the same error. The solution turns out to be: Factory.getMap(), not Factory.get(mf), that added to the new ServerClient is working! Thanks again Slugger!

So if anyone else is trying to get this working the whole thing looks like:

def SJQ4_METADATA = ["SJQ4_ID":"4695793", "SJQ4_TYPE":"MediaFile"]

def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())
import com.google.code.sagetvaddons.sjq.network.ServerClient
import com.google.code.sagetvaddons.metadata.Factory

def sc = new ServerClient()
def taskId = 'HANDBRAKE'
def metadata = Factory.getMap(mf)
sc.addTask(taskId, metadata)
sc.close()

[Slugger]

Good catch! You'd think I'd remember my own API calls that I've written.

[bikesquid]

Unfortunately it didn't resolve the underlying problem as I was hoping.

I'm getting the attached invalid object reference error, which I assume has to do with the 'double' recording references after the SetMediaFileMetadata. What seems to happen is the media object exists in the sagetv db in both locations, original and Cut process output when the handbrake task is starting, but only showing one when it's done... I'm guessing the object in question is linked to the original file, which has been deleted by the end of the Cut process.... I'm guessing the SetMediaFileMetadata isn't replacing the object reference, but adding a new one, one that's not being passed to the new process.... I've no idea how to get around/through/under this.

I have reinitialized the mf definition post handbrake command without success. All the println statements of definitions do seem to show correctly just prior to that reinit.

So close....

[bikesquid]

Originally Posted by Slugger

instead of reinitializing by calling GetMediaFileForID(), try GetMediaFileForFilePath() and pass it the original file. If that also returns null then something is deleting the media file object in Sage (or there's a bug in the sagex-apis, maybe on the remote side).

Here's something I don't understand... ... well one 'something' anyway.

def SJQ4_METADATA = ["SJQ4_ID":"4689691", "SJQ4_TYPE":"MediaFile"]
def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())
def SourceFile = MediaFileAPI.GetFileForSegment(mf, 0)

println SourceFile
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)

When run in SageGroovy get's me a result of:

\\Sage-pc\Mirror_Set\Recorded TV\7Newsat5-4679037-0.ts
RemoteObjectRef[26944424];
RemoteObjectRef[10712894];
RemoteObjectRef[5143008];
RemoteObjectRef[25877097];

Shouldn't the object reference be the same with each polling on the same file?

[Slugger]

Originally Posted by bikesquid

Here's something I don't understand... ... well one 'something' anyway.

def SJQ4_METADATA = ["SJQ4_ID":"4689691", "SJQ4_TYPE":"MediaFile"]
def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())
def SourceFile = MediaFileAPI.GetFileForSegment(mf, 0)

println SourceFile
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)
println MediaFileAPI.GetMediaFileForFilePath(SourceFile)

When run in SageGroovy get's me a result of:

\\Sage-pc\Mirror_Set\Recorded TV\7Newsat5-4679037-0.ts
RemoteObjectRef[26944424];
RemoteObjectRef[10712894];
RemoteObjectRef[5143008];
RemoteObjectRef[25877097];

Shouldn't the object reference be the same with each polling on the same file?

No. Each of those objects will refer to the same MediaFile object on the SageTV server, but each call generates a new reference pointing to the same object. That's expected, that's normal. What isn't normal is the call returning null. Well, it might be normal depending on when/how the original MediaFile object is disappearing. Unfortunately, I've studied your scripts and I don't see why it's happening. To go any further with my debugging would require me to actually install and run these tasks and I just don't have the environment to set that up in.

What you need to figure out is what's happening between the time you start handbrake and the time it finishes. Something is deleting the MediaFile object such that another call to GetMediaFileForID() is returning null, presumably because the MediaFile referred to by the SJQ4_ID no longer exists. You need to figure out why that's happening and what's causing it to happen and prevent it from deleting the object in wiz.bin too early. If that's not possible then you need to capture all the metadata you need from that object before you start handbrake such that you don't need to reinit that object after handbrake finishes.

[bikesquid]

Originally Posted by Slugger

You need to figure out why that's happening and what's causing it to happen and prevent it from deleting the object in wiz.bin too early.

The delete is happening at the end of the CUT script, AFTER the metadata is relinked to the NEW file, so I believe the process is 'kosher', but sage does show two of that recording for a bit until it refreshes its dataset... I suspect that's the issue, but have no idea how to 'force' the correct object to be read by the handbrake script. I was hoping the GetMediaFileForFilePath would provide a way to say "No, not that one, this one" to the handbrake script.... Technically I am done after the handbrake, minus a move to final archive location, so I was hoping not to 'loose' touch with the file.

[Slugger]

So am I understanding this correctly?

1. You run the scanner script, it finds recordings to be processed and for each one it queues a CUT and HANDBRAKE task in the queue. The metadata assigned to the SJQ4_* vars is based on this MediaFile object found by the scanner.

2. CUT always runs before HANDBRAKE and HANDBRAKE waits for CUT to finish before it will start.

3. The CUT task does whatever it does and deletes the original MediaFile object from wiz.bin and creates a new one using MediaFileAPI.AddMediaFile().

4. HANDBRAKE starts after CUT has completed. By this time, the original media file used to seed the metadata for the task is deleted and has been removed from wiz.bin. HANDBRAKE cannot start until CUT has finished.

If I'm understanding your process properly then what you want to do is not queue both CUT and HANDBRAKE from the scanner in step #1. Instead, just queue the CUT task. The CUT task will then queue the HANDBRAKE task using the new MediaFile object you created in the CUT script as the seed for metadata of the HANDBRAKE task.. This way when the HANDBRAKE task loads its MediaFile object, it'll be loading the new one created in the CUT task, which is the one you want to run handbrake against anyway, right?