SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations

Notices

SageTV v7 Customizations This forums is for discussing and sharing user-created modifications for the SageTV version 7 application created by using the SageTV Studio or through the use of external plugins. Use this forum to discuss plugins for SageTV version 7 and newer.

Reply
 
Thread Tools Search this Thread Display Modes
  #1161  
Old 02-08-2012, 09:06 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Coppit - I am not sure if this is exactly the same problems you were having but I was having issues with SJQ/Groovy choking on exes that had verbose output - and HandbrakeCLI is an exact example of this as is comskip even when you have the verbosity set very low.

But my code was 100% groovy which is different from you. Let me know if you want to see my code.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #1162  
Old 02-08-2012, 09:15 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Quote:
Originally Posted by Slugger View Post
In summary, can SJQv4 do this kind of thing? Absolutely. But it will require you to write some decent amount of scripts to do it - or ask wayner nicely for his solution.
I never finished adapting Slugger's code to do this but it is almost there. But the code is all in Groovy so you have to graduate to SJQ4.

If you look back a couple of pages you will see a script that Slugger posted that contains the code to rename all of your Recorded TV files. But this doesn't actually rename the files as he didn't want it to screw up your file names. It just posts the old and new file names. It doesn't rename the folder structure but that should be a minor revision.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server

Last edited by wayner; 02-09-2012 at 07:43 AM.
Reply With Quote
  #1163  
Old 02-08-2012, 11:52 PM
GKusnick's Avatar
GKusnick GKusnick is offline
SageTVaholic
 
Join Date: Dec 2005
Posts: 5,083
I'm trying to automate some transcoding tasks and I think SJQ might be the right tool for the job. In the past I've always just coded my own solution to this sort of problem but this time around I'm particularly attracted to SJQ's ability to offload task execution from the Sage server by farming it out to other machines on my LAN. That's something I'd rather not reinvent if I don't have to.

I've read through the docs and how-tos (but not this entire thread, so apologies if I've missed something). I think I see how to set up tasks and link them to events so that new recordings automatically get queued for transcoding. The part I'm not clear on, however, is what to do with my existing backlog of recordings, which I'd also like transcoded.

I see how to trigger tasks manually or periodically, but if I'm not mistaken, those sorts of tasks do not get passed a MediaFile argument; only the ones triggered by recording events get that. What I'd like to be able to do is to select a task and say, Queue an instance of this task for each existing recording (or for each existing MediaFile that meets some specified criteria, although I suppose I can postpone that check to my task's test script).

Alternatively, if there's a way that an executing task can queue additional tasks, that would also serve my purpose. Then I could just write a one-shot manual task to scan the existing recordings and queue transcoding tasks as appropriate. But I'm not really seeing a way to do that either.

Yet another way might be for me to raise fake RecordingStopped events for the existing recordings, thereby fooling SJQ into queuing tasks for them. But that seems a bit heavy-handed and may have undesirable side-effects.

Has anybody else dealt with this issue of applying a newly defined task to a large set of existing recordings? Any suggestions on the best approach to take?

Thanks.
__________________
-- Greg
Reply With Quote
  #1164  
Old 02-09-2012, 07:09 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,191
You can use/modify Slugger's Media File Scanner/Task Queuer script to determine the logic of scanning your library and to queue a task for the files/recordings that match the criteria

An example is attached. changed .goovy to .txt in order to get past the upload filter
Attached Files
File Type: txt media_scanner.txt (4.1 KB, 69 views)
Reply With Quote
  #1165  
Old 02-09-2012, 07:43 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Manual tasks do get passed a MediaFile argument. When you select a recording in Sage (assuming you have SJQ installed) one of the options in the menu is to run an SJQ task which will then be performed on that specific MediaFile. The problem is that you don't want to have to do this hundreds of times to run the task on all of your recordings.

If what you want to transcode is mainly TV shows another way to do this would be to manually get SJQ to launch tasks for all episodes of a given show or all episodes in a given season. That would probably lead to a manageable number of tasks in the queue at any given time. You could then have separate shows being transcoded on separate PCs. I don't know what the limit would be but I don't know that you want to have too many transcode jobs going at once if the original files are all on the same hard drive since that drive will be getting hammered, particularly if you have multiple recordings also occurring at the same time.

You might have to change your Sage options so that you get a menu to pop up when you select a TV Show title in Sage - from that menu you can then select a task which, I am pretty sure, will then get executed on all instances of the show.

I don't sort by Season but I believe you can get Sage to sort your shows by seasons, can you not? If so then you can do the same thing I described above but on a season by season basis for each show.

@GKusnick - which UI are you using? I don't know that it matters too much but the stuff I am describing is for Diamond but I am pretty sure that it is the same for the default UI. I don't know about Phoenix.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #1166  
Old 02-09-2012, 07:56 AM
Slugger Slugger is offline
SageTVaholic
 
Join Date: Mar 2007
Location: Kingston, ON
Posts: 4,008
@GKusnick

The media scanner example script is probably what you want to process existing recordings. Run it once from the crontab (or manually). You can queue other tasks from a script.

Lines 65-75 in that script show how you can queue other tasks from a script (i.e. the ServerClient() object).
__________________
Twitter: @ddb_db
Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive
Capture: 2 x Colossus
STB Controller: 1 x USB-UIRT
Software:Java 1.7.0_71; SageTV 7.1.9
Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter
Plugins: Too many to list now...
Reply With Quote
  #1167  
Old 02-09-2012, 12:22 PM
GKusnick's Avatar
GKusnick GKusnick is offline
SageTVaholic
 
Join Date: Dec 2005
Posts: 5,083
Thanks; that helps a lot.

Still not 100% sure if I'm going to go this route. I had a brainstorm in bed last night about how to modify my existing Jetty webapp to work as a task dispatcher. So I may experiment with that a bit before jumping into SJQ. But at least I have a clearer picture now of how to do it either way.

Wayner, I have my own custom UI that I've been using for years. But I do revert to the stock UI for maintenance and configuration chores that my UI doesn't implement.
__________________
-- Greg
Reply With Quote
  #1168  
Old 02-10-2012, 03:33 AM
coppit coppit is offline
Sage Advanced User
 
Join Date: Mar 2003
Posts: 175
Quote:
Originally Posted by wayner View Post
I am not sure if this is exactly the same problems you were having but I was having issues with SJQ/Groovy choking on exes that had verbose output
It's possible, but:
  • I'm echoing all the output from HandBrake now and don't seem to have problems
  • It seems like the movies HandBrake gets stuck on have signal noise

It feels more like HandBrake going off the rails when there's lots of corruption in the MPEG stream.
Reply With Quote
  #1169  
Old 02-10-2012, 03:44 AM
GKusnick's Avatar
GKusnick GKusnick is offline
SageTVaholic
 
Join Date: Dec 2005
Posts: 5,083
Quote:
Originally Posted by coppit View Post
It feels more like HandBrake going off the rails when there's lots of corruption in the MPEG stream.
The released version of Handbrake (0.9.5) crashes routinely on my Colossus recordings. From my reading of the Handbrake forums, there seems to be a known memory-management bug in this version that shows up when running on 32-bit Windows. The bug has since been fixed but hasn't yet made it into a release build. If you're adventurous, you could try one of the nightly builds from here to see if it solves your problem (without causing any new problems).
__________________
-- Greg
Reply With Quote
  #1170  
Old 02-10-2012, 10:23 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Quote:
Originally Posted by tmiranda View Post
Java Heap Monitor is correct. It put's a nice little bar graph in the header and lets you set trigger levels for messages.
Thanks for the heads-up - just installed this and it looks great.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #1171  
Old 02-11-2012, 11:24 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Handbrake Script

Here is the script that I use to call Handbrake to reencode. This script uses the preset iPad settings but it should be pretty easy to change the HBParams variable to change the parameters. I have written everything in groovy and call groovy from the executable line so there is no test script.

To configure in SJQ put in your Task ID name and put "script:c:/MyFolder/HB_ipad.groovy" (replancing your folder and your actual groovy file name) in the Executable line. Nothing else has to be changed.

This is for Win7 342 bit but it should be easy to adapt to other OSes. You may have to change your Handbrake path, particularly if you are running a 64 bit version of Windows or not running Windows.

The one tricky thing with this script is that you have to process the output from Handrake since it is so verbose.

Code:
Object mf = MediaFileAPI.GetMediaFileForID(Integer.parseInt(SJQ4_METADATA.get("SJQ4_ID")))
String FileName =  MediaFileAPI.GetFileForSegment(mf,0)
FilePrefix = FileName.substring(0, FileName.lastIndexOf('.'))
FileType= FileName.substring(FileName.lastIndexOf('.')+1, FileName.size())
InputVidFile=FileName
OutputVidFile=FilePrefix+".mp4"
println FileType

if (FileType != "mpg" ) {
println("Recording is not mpg")
   return 1
}

HBCLIPath='"C:\\Program Files\\Handbrake\\HandBrakeCLI.exe"'
HBParams=" --preset iPad -v 0"

HBProc=HBCLIPath+" -i "+InputVidFile+" -o "+OutputVidFile+HBParams
println (HBProc)

def initialSize = 8192
def outStream = new ByteArrayOutputStream(initialSize)
def errStream = new ByteArrayOutputStream(initialSize)

def command=HBProc
def proc=command.execute()
proc.consumeProcessOutput(outStream, errStream)
proc.waitFor()

println "return code: ${ proc.exitValue()}"
println "stderr: ${proc.err.text}"
println "stdout: ${proc.in.text}" // *out* from the external program is *in* for groovy
println 'out:\n' + outStream
println 'err:\n' + errStream
return 0
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #1172  
Old 02-20-2012, 01:20 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Return codes for SJQ?

Slugger and/or Tmiranda -

What is the range of return codes for SJQv4? And what colours do these correspond to in the UI?

Does it make sense to have a "failed" return code for a job that isn't necessary - like when you go to run comskip but there is already a valid .edl file for the recording? What return code should I use for such events?
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #1173  
Old 02-20-2012, 03:37 PM
Slugger Slugger is offline
SageTVaholic
 
Join Date: Mar 2007
Location: Kingston, ON
Posts: 4,008
Quote:
Originally Posted by wayner View Post
Slugger and/or Tmiranda -

What is the range of return codes for SJQv4? And what colours do these correspond to in the UI?
Not sure what the STVi will show, but there are four categories of return codes recognized by the SJQ engine:
  • The exe's return code fell between the defined min rc and max rc settings for the defined task. In this case, the task is marked as COMPLETED.
  • The exe's rc falls outside of the defined min/max settings; task is marked FAILED
  • The test script's rc = 1 or rc > 2; the task is set back to WAITING and will be retried later
  • The test script's rc = 2; the task is set to SKIPPED and will not be executed or retried

Quote:
Does it make sense to have a "failed" return code for a job that isn't necessary - like when you go to run comskip but there is already a valid .edl file for the recording? What return code should I use for such events?
For this case, you should have a test script that finds the edl and then returns rc=2, which will mark the task as SKIPPED. Marking it FAILED is also acceptable as it achieves the same end goal, just depends if you care that tasks are marked as failed.
__________________
Twitter: @ddb_db
Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive
Capture: 2 x Colossus
STB Controller: 1 x USB-UIRT
Software:Java 1.7.0_71; SageTV 7.1.9
Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter
Plugins: Too many to list now...
Reply With Quote
  #1174  
Old 02-20-2012, 06:31 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Thanks Slugger - one other question. I have a task that runs when all recordings are complete that checks for bad video files that are caused by a STB being off.

I also run tasks depending on the show being recorded - for kids' shows I move to another folder and encode using handbrake. This is a destructive process in that it deletes the original TV file which will cause the other process to not run correctly.

Is it possible to determine in which order SJQ runs these tasks - in other words how do I know which one gets run first? Is there any way of specifying a priority which will cause a certain task to run first?
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #1175  
Old 02-20-2012, 06:35 PM
Slugger Slugger is offline
SageTVaholic
 
Join Date: Mar 2007
Location: Kingston, ON
Posts: 4,008
Quote:
Originally Posted by wayner View Post
Is it possible to determine in which order SJQ runs these tasks - in other words how do I know which one gets run first? Is there any way of specifying a priority which will cause a certain task to run first?
No, all of the things you mention are non-deterministic. If you have two or more tasks that depend on each other you must either synchronize the tasks externally (through lock files or other mechanisms) or (and my preferred/recommended approach) you must combine the tasks into a single task exe script using Groovy. In this case, your groovy script is likely just a light weight wrapper that calls each individual exe, but does so in the proper/required order, etc.
__________________
Twitter: @ddb_db
Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive
Capture: 2 x Colossus
STB Controller: 1 x USB-UIRT
Software:Java 1.7.0_71; SageTV 7.1.9
Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter
Plugins: Too many to list now...
Reply With Quote
  #1176  
Old 02-20-2012, 07:09 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,613
Thanks Slugger, that's what I expected. I will either combine into one "mother" task, or I will run a cron job in the middle of the night that processes all files that were recorded yesterday.

My scripts do basically just do some processing to call EXEs like comskip and handbrake, plus Java/Groovy I/O commands, but you have convinced me to use Groovy as much as possible so I don't even use batch files anymore, everything is called from a Groovy script.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #1177  
Old 02-21-2012, 02:54 PM
phelme's Avatar
phelme phelme is offline
Sage Icon
 
Join Date: Dec 2006
Posts: 1,027
Crap. I'm suddenly getting failures with tasks. An example is the following:

Code:
===== Tue Feb 21 12:39:48 PST 2012 =====

Remote SageAPI is not functional.  Can't use server: rmi://x.x.x.x:1098
----- stderr -----

java.lang.RuntimeException: Remote SageAPI is not functional.  Can't use server: rmi://x.x.x.x:1098
	at sagex.remote.rmi.RMISageAPI.(RMISageAPI.java:28)
	at com.google.code.sagetvaddons.sjq.agent.ScriptRunner.main(ScriptRunner.java:154)
Caused by: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: 
	java.io.EOFException
	at sun.rmi.transport.tcp.TCPChannel.createConnection(Unknown Source)
	at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source)
	at sun.rmi.server.UnicastRef.newCall(Unknown Source)
	at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
	at sagex.remote.rmi.RMISageAPI.callService(RMISageAPI.java:36)
	at sagex.remote.rmi.RMISageAPI.callService(RMISageAPI.java:52)
	at sagex.remote.rmi.RMISageAPI.(RMISageAPI.java:25)
	... 1 more
Caused by: java.io.EOFException
	at java.io.DataInputStream.readByte(Unknown Source)
	... 8 more
Tried turning off the firewall and that's not the cause. Restarted the SJQ task client service. Don't think I changed anything else... though I had been messing with the NIC driver settings for SageDCT. Hmmm. What else should I look at?

Last edited by phelme; 02-21-2012 at 03:11 PM.
Reply With Quote
  #1178  
Old 02-21-2012, 03:15 PM
Slugger Slugger is offline
SageTVaholic
 
Join Date: Mar 2007
Location: Kingston, ON
Posts: 4,008
sagex-services plugins is no longer functional. Restart/fix it or restart the Sage server.
__________________
Twitter: @ddb_db
Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive
Capture: 2 x Colossus
STB Controller: 1 x USB-UIRT
Software:Java 1.7.0_71; SageTV 7.1.9
Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter
Plugins: Too many to list now...
Reply With Quote
  #1179  
Old 02-21-2012, 03:57 PM
phelme's Avatar
phelme phelme is offline
Sage Icon
 
Join Date: Dec 2006
Posts: 1,027
Quote:
Originally Posted by Slugger View Post
sagex-services plugins is no longer functional. Restart/fix it or restart the Sage server.
I've tried both restarting the SageTV service and resetting the sagex-services plugin and it's still failing. Do I need to uninstall the plugin itself? That will be a little bit of a pain because of the dependencies it has.

The jetty stuff is still functioning if that's any help.
Reply With Quote
  #1180  
Old 02-21-2012, 04:32 PM
Slugger Slugger is offline
SageTVaholic
 
Join Date: Mar 2007
Location: Kingston, ON
Posts: 4,008
Not sure why the RMI service would suddenly stop working. You might want to track down stuckless and see if he has any thoughts (sagex-services is his plugin). My initial guess is that something else is latching onto port 1098? Only a guess though.
__________________
Twitter: @ddb_db
Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive
Capture: 2 x Colossus
STB Controller: 1 x USB-UIRT
Software:Java 1.7.0_71; SageTV 7.1.9
Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter
Plugins: Too many to list now...
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Plugin: MizookLCD (Alternate SageTV LCDSmartie Plugin) cslatt SageTV Customizations 48 06-11-2012 10:44 AM
SJQv4: Technology Preview Slugger SageTV v7 Customizations 39 12-17-2010 01:17 PM
SageTV Plugin Developers: Any way to see stats for your plugin? mkanet SageTV Software 4 12-12-2010 10:33 PM
MediaPlayer Plugin/STV Import: Winamp Media Player Plugin deria SageTV Customizations 447 12-11-2010 07:38 PM
SJQv4: Design Discussion Slugger SageTV v7 Customizations 26 10-18-2010 08:22 AM


All times are GMT -6. The time now is 06:34 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.