Plugin: SJQv4

[bikesquid]

Originally Posted by Slugger

So am I understanding this correctly?

I've really got 3 files in the process, original source file (#1), the output of the CUT script (#2), and the output from HANDBRAKE (#3). The current version of things does the following:
1) MEDIASCAN find files and queues cut script.
2) CUT script takes #1 and outputs a cut variant in a different directory (#2),
3) CUT script then links the #1 metadata to #2.
4) last step in the CUT script deletes #1.
5) HANDBRAKE script takes #2 and outputs #3,
6) then (and this is where it all goes pear-shaped) links #3 to the same metadata yet again,
7) and lastly deletes #2.

So I believe it's as you suggest it should be. However, mf STILL references #1 not #2 for some reason....

I now realize, It's only because in earlier (and not working) testing that I hard coded the directory for #2 in the HANDBRAKE script... likely masking the problem?? Regardless there seem to be (at least for a while) two instances of the recording in the sagetv db screwing things up.

Two ideas; I could do a MediaFileAPI.DeleteFile AND the AddMediaFile at the end of the CUT script, ..... It may be a cleaner way to handle the file manipulations I want to do anyway, but I don't know the implications. I'll play with that idea later if I've got the nerve...

In the meantime, It may make sense to just use ".GetMediaFileForFilePath(SourceFile)", I can't think of a clean way to do this, but this may point the HANDBRAKE script at the correct file:

def SJQ4_METADATA = ["SJQ4_ID":"4695707", "SJQ4_TYPE":"MediaFile"]

import org.apache.commons.io.FilenameUtils
def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())
def SourceFile = (FilenameUtils.getBaseName(MediaFileAPI.GetFileForSegment(mf, 0).getAbsolutePath()) + ".ts")
def CUTOutputDir = "\\\\Sage-pc\\Mirror_Set\\Recorded TV\\"
SourceFile = new File(CUTOutputDir + SourceFile)
mf = MediaFileAPI.GetMediaFileForFilePath(SourceFile)
SourceFile = MediaFileAPI.GetFileForSegment(mf, 0)
println SourceFile

Hopefully that'll get me the right object info associated with the right file to proceed. Too bad there's not a MoveObjectToFile API call I could use!

[Slugger]

If you're doing it as I suggested above, then the SJQ4_ID value must be different in the HANDBRAKE task because it should be pointing to the new MediaFile object you created in the CUT task. If the SJQ4_ID is the same in both tasks then that's the problem.

[bikesquid]

Slugger, thanks a million for the patient leading

OK, I've got to say this SJQ4 octopus is actually pretty damn cool once you start getting the upper hand!

I've still got some error checking to put in the code, but it all seems to be playing nicely now and cleaning up as it goes. I've now got processes running together that rename the recording to "Series - SxxExx" or "movie title (year)" format as appropriate, running comskip, waiting for me to review proposed edits, then cutting commercials with VideoReDo, and finally compressing and outputting an .mkv file when wanted with Handbrake, all the while maintaining metadata correctly. SWEET!

Again, thanks for all the help.

[wayner]

@slugger - I see that you have added a groovy skip for keep stb alive. Any ideas on how to determine whether a recording is just a " black screen" because your stb was powered down? In the past I thought of trying to schedule a test recording and then run that test recording through Comskip. You could then parse the Comskip logs to see if the recording looked bad. If it was bad then you could try sending a power on and try again.

I occasionally have failed recordings due to my Rogers SA3200 box being powered down. Unlike the Rogers HD boxes this box does not have a setting that will power on when you send a channel change. This means that if the box is powered down for some reason, such as due to a software update or power outage, then I will miss recordings. This box is just hooked up to an analog capture device so it creates regular SD MPEG-2 files which are pretty easy to Comskip.

Any other ideas on how to check for a valid recording?

[Slugger]

Originally Posted by wayner

@slugger - I see that you have added a groovy skip for keep stb alive. Any ideas on how to determine whether a recording is just a " black screen" because your stb was powered down? In the past I thought of trying to schedule a test recording and then run that test recording through Comskip. You could then parse the Comskip logs to see if the recording looked bad. If it was bad then you could try sending a power on and try again.

I occasionally have failed recordings due to my Rogers SA3200 box being powered down. Unlike the Rogers HD boxes this box does not have a setting that will power on when you send a channel change. This means that if the box is powered down for some reason, such as due to a software update or power outage, then I will miss recordings. This box is just hooked up to an analog capture device so it creates regular SD MPEG-2 files which are pretty easy to Comskip.

Any other ideas on how to check for a valid recording?

Video processing is not my strength nor does it interest me so I have no way to assist you on that task. You find a program to analyze the video and determine if you need to send the power command to your stb and SJQ can do the rest.

[wayner]

Fair enough. Anyone else have any ideas on what to use for the video processing?

[bikesquid]

Originally Posted by tmiranda

Here is an interim release of the UI that adds a "Task Overview" menu which allows you to see the state of Tasks. Clicking on one of the tabs across the top row filters by Task state. Clicking on the row just below that sorts by that field. I'm too lazy to add a screen shot but once you look at the menu it should be obvious.

To install: Download the attached .zip file and place the extracted .xml in the install directory of your Sage server. Then go to the plugin manager and update the SJQ4 UI Mod.

I believe there is a bug in this release, but I don't remember what it is

Please post feedback here.

Tom

Like the new screen Tom! Sortable columns is really a nice touch. Two observations for you:

First, many of the 'titles' show blank even though when I click on them I get the right info in the pop-up... I think it has to do with the underlying files being deleted? The data is still there and would be nice to see as a 'history'. Maybe there could be leading * if the file is mia?

Second, any way of adding an action if a 'state' is clicked on? I imagined it would pop up a screen giving options to cancel, etc.

[tmiranda]

Originally Posted by bikesquid

Like the new screen Tom! Sortable columns is really a nice touch. Two observations for you:

First, many of the 'titles' show blank even though when I click on them I get the right info in the pop-up... I think it has to do with the underlying files being deleted? The data is still there and would be nice to see as a 'history'. Maybe there could be leading * if the file is mia?

Second, any way of adding an action if a 'state' is clicked on? I imagined it would pop up a screen giving options to cancel, etc.

I've not seen the first issue. I'll have to see why the pop-up is different form the main screen.

I thought about being able to change the state, but I was against duplicating functionality already available in another menu. I'll think about it some more.

[phareous]

SQJ4 has been working fine for weeks and all of a sudden it stopped. All the tasks show as failed. I have two remote clients (running the service) and one running on the server as the task plugin. For all three of them, I can view the test script logs but the executable logs won't show (says they aren't available).

Any ideas on how I should start troubleshooting this?

[Slugger]

Originally Posted by phareous

SQJ4 has been working fine for weeks and all of a sudden it stopped. All the tasks show as failed. I have two remote clients (running the service) and one running on the server as the task plugin. For all three of them, I can view the test script logs but the executable logs won't show (says they aren't available).

Any ideas on how I should start troubleshooting this?

Look in sjqagent.log for each task client.

[Slugger]

IMPORTANT: A task client plugin update will be released at some point over the weekend (probably Sunday afternoon). If you also have the Jetty plugin installed then read the following carefully.

If you don't have Jetty installed then you can ignore this message. If you have the web interface, mobile interface, SageAlert, SRE or possibly other plugins installed then you also have Jetty installed and this applies to you.

Before upgrading the task client to the new release, you MUST upgrade Jetty to its latest version first. It is safe to upgrade the SJQv4 task client plugin only after you've upgraded Jetty. The required Jetty update is already available in the plugin manager and can be installed immediately.

If you upgrade the SJQv4 task client without upgrading Jetty first you will end up with duplicate jars in your SageTV\JARs folder and things aren't likely going to work as desired.

Unfortunately, the SageTV plugin manager has no way for us to automate this or prevent the conflict from occurring if you don't follow these instructions. If you don't update Jetty first then you will end up with duplicate jars, which will cause you problems.

Again, the task client update is scheduled for release by Sunday afternoon.

[davephan]

I just started trying to use SJQ4, and I am not having any success so far...

I am trying to follow the SJQ4 Comskip 1 example. In the "Executable", "Executable Arguments", and "Test Script" forward slashes are used instead of back slashes. In Windows paths have back slashes, not forward slashes. Forward slashes are only used for switches.

When typing in the data, there are spaces between each character. This is disconcerting, but the data entered does not have the extra spaces between the characters after completing each line.

The complexity seems to be about ten times greater than SJQ3, so I think the learning curve will be very steep.

I tried committing the SDComskip task and had about 70 "Returned" items. I suspect that was because I did not have the compskip_test.groovy script in place. Deleting all 70 of the items one by one is a pain. There should be a way to mark all of them to delete them all at once.

I can struggle more with SJQ4. First, are the forward slashes in the Comskip 1 example correct? I think the documentation is a bit weak to get SQJ4 working. Maybe there are steps that I am missing...

Dave

[Slugger]

Originally Posted by davephan

I just started trying to use SJQ4, and I am not having any success so far...

I am trying to follow the SJQ4 Comskip 1 example. In the "Executable", "Executable Arguments", and "Test Script" forward slashes are used instead of back slashes. In Windows paths have back slashes, not forward slashes. Forward slashes are only used for switches.

When typing in the data, there are spaces between each character. This is disconcerting, but the data entered does not have the extra spaces between the characters after completing each line.

The complexity seems to be about ten times greater than SJQ3, so I think the learning curve will be very steep.

I tried committing the SDComskip task and had about 70 "Returned" items. I suspect that was because I did not have the compskip_test.groovy script in place. Deleting all 70 of the items one by one is a pain. There should be a way to mark all of them to delete them all at once.

I can struggle more with SJQ4. First, are the forward slashes in the Comskip 1 example correct? I think the documentation is a bit weak to get SQJ4 working. Maybe there are steps that I am missing...

Dave

For file paths in the test script and exe, the slashes can be either way since Java can handle them either way. If you're passing file paths in the arguments then the slashes would probably have to be correct for the OS you're running on (unless the application you're passing the args to can also handle the slashes either way).

[davephan]

I have "Returned" for status for each file that SJQ4 ran. I would like to try deleting the tasks and start over. How do I delete all the "Returned" tasks in the "Task Overview" section? I can delete one at a time, but there are 70 of them. It takes quite awhile to delete one at a time.

To create a task, the path is "Setup", "Sage Job Queue", "Show Clients", "Create New Task".

There does not seem to be a way to delete tasks, only create new tasks. How do I delete tasks?

Dave

[Slugger]

To delete multiple tasks at once:

1) Connect directly to the SJQv4 database by following this document

2) Run ONE of the following queries:

To delete everything in the queue: DELETE FROM queue
(This will mess up the sync between task clients and the server if there are any tasks in running or started state when you run this query.)

To delete everything except running tasks: DELETE FROM queue WHERE state NOT IN ('RUNNING','STARTED')

To delete tasks that are waiting: DELETE FROM queue WHERE state = 'WAITING'

If these kinds of operations are missing from the UI then open a ticket with tmiranda and he may add the features to the UI.

To delete a configured task: Load a client, click on supported tasks, select the task you want to delete, on the next screen you should be given the option to edit the task or delete it. Choose delete.

[davephan]

I've made some progress with the SJQ4 installation, but it still isn't working yet.

The SJQ4 "Task Overview" has jobs that show states "Skipped", "Failed", and "Complete". No files are actually processed with Comskip.exe.

I'm confused by the sjqagent.properties file located in the c:\sjqagent\config folder.

The sample is:

task.mytask.exe=C:/bin/mytask.exe
# task.mytask.args=/f /g /t
task.mytask.test=C:/scripts/mytask_precheck.groovy
# task.mytask.testargs=
# task.mytask.schedule=* * * * *
# task.mytask.resources=100
# task.mytask.maxprocs=1
# task.mytask.rcmin=0
# task.mytask.rcmax=0

Only two lines appear to be relevant in the sample are:
task.mytask.exe=C:/bin/mytask.exe
task.mytask.test=C:/scripts/mytask_precheck.groovy

The comskip_test.groovy file is located in my c:\comskip folder. I don't have a C:\bin folder.

I could put the comskip_test.groovy path in the second line, but I'm not sure what to do with the first line:
task.mytask.exe=C:/bin/mytask.exe

I did try to install the SJQ4 Task Client for Windows (v4.0.0.1509). The service is set to "automatic", but it will not start. I assume that the reason that the SJQ4 Task Client is because of a problem with the sjqagent.properties file. When I try to start the SJQ4 Task Client service, I get the following error:

Windows could not start the SJQ4 task client agent for Windows (v4.0.0.1509) on the local computer. For more information, review the system event log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 1.

Dave

[Slugger]

Originally Posted by davephan

I've made some progress with the SJQ4 installation, but it still isn't working yet.

The SJQ4 "Task Overview" has jobs that show states "Skipped", "Failed", and "Complete". No files are actually processed with Comskip.exe.

View the task output from the UI. Does it provide any clue as to what's happening?

I'm confused by the sjqagent.properties file located in the c:\sjqagent\config folder.

The sample is:

task.mytask.exe=C:/bin/mytask.exe
# task.mytask.args=/f /g /t
task.mytask.test=C:/scripts/mytask_precheck.groovy
# task.mytask.testargs=
# task.mytask.schedule=* * * * *
# task.mytask.resources=100
# task.mytask.maxprocs=1
# task.mytask.rcmin=0
# task.mytask.rcmax=0

Only two lines appear to be relevant in the sample are:
task.mytask.exe=C:/bin/mytask.exe
task.mytask.test=C:/scripts/mytask_precheck.groovy

The comskip_test.groovy file is located in my c:\comskip folder. I don't have a C:\bin folder.

I could put the comskip_test.groovy path in the second line, but I'm not sure what to do with the first line:
task.mytask.exe=C:/bin/mytask.exe

All of those lines are relevant and control various aspects of the task itself.

http://code.google.com/p/sagetv-addo...ent.properties

The test script determines if your exe should run or not. The exe is the actual program you want to run against your object file. Nothing has to go in C:\bin\, that was just the example location. Put files where ever you want.

I did try to install the SJQ4 Task Client for Windows (v4.0.0.1509). The service is set to "automatic", but it will not start. I assume that the reason that the SJQ4 Task Client is because of a problem with the sjqagent.properties file. When I try to start the SJQ4 Task Client service, I get the following error:

Windows could not start the SJQ4 task client agent for Windows (v4.0.0.1509) on the local computer. For more information, review the system event log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 1.

Go to the directory where you installed the task client. In there is a logs dir. In there is a wrapper.log. That file should contain the details of why the service failed to start. Typical reason is it was unable to find java.exe on the system path.

[Korny]

I know I'm missing something, but I'm reading the sage Mediafile API and the Getmedaifilemetadata. I really wish I had a coding back ground, but I'm slowly learning. In the end all I'm really trying to do is output filepath showtitle and seasonxepisode(example 3x04)
http://download.sage.tv/api/sage/api...lang.String%29
I'm just trying to grab the episode number, I'm using

def mediaFile = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA['SJQ4_ID'].toInteger())
def fileName = "${SJQ4_METADATA['SJQ4_PATH']}/${SJQ4_METADATA['SJQ4_LAST_SEGMENT']}"
def showTitle = MediaFileAPI.GetMediaTitle(mediaFile)
def episode = MediaFileAPI.GetMediaFileMetadata(mediafile, EpisodeNumber)
def season = MediaFileAPI.GetMediaFileMetadata(mediafile SeasonNumber)
    if(!Tools.setExeArgs("\"$fileName\" \"$showTitle\" \"$seasonx$episode\"" ))
       return 1 // If the call fails then return the task to the queue and try again later...
     return 0 // The exe args have been modified, return success and fire off the exe!

I'm getting groovy.lang.MissingPropertyException: No such Property: mediafile for class

[razrsharpe]

Originally Posted by Korny

I know I'm missing something, but I'm reading the sage Mediafile API and the Getmedaifilemetadata. I really wish I had a coding back ground, but I'm slowly learning. In the end all I'm really trying to do is output filepath showtitle and seasonxepisode(example 3x04)
http://download.sage.tv/api/sage/api...lang.String%29
I'm just trying to grab the episode number...

I think you want this:

def episode = MediaFileAPI.GetMediaFileMetadata(mediafile, "EpisodeNumber")
def season = MediaFileAPI.GetMediaFileMetadata(mediafile, "SeasonNumber")

notice the comma after mediafile in the season line and the " around episode and season number

[bikesquid]

Originally Posted by Korny

I know I'm missing something, but I'm reading the sage Mediafile API and the Getmedaifilemetadata. I really wish I had a coding back ground, but I'm slowly learning. In the end all I'm really trying to do is output filepath showtitle and seasonxepisode(example 3x04)
http://download.sage.tv/api/sage/api...lang.String%29
I'm just trying to grab the episode number, I'm using
def mediaFile = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA['SJQ4_ID'].toInteger())

Originally Posted by razrsharpe

I think you want this:

def episode = MediaFileAPI.GetMediaFileMetadata(mediafile, "EpisodeNumber")
def season = MediaFileAPI.GetMediaFileMetadata(mediafile, "SeasonNumber")

notice the comma after mediafile in the season line and the " around episode and season number

You'll also need to change mediafile to mediaFile as variables are case sensitive... and your def statement above uses the capital version....