Plugin: SJQv4

[Slugger]

Originally Posted by davephan

It would also be nice if the job priority for selected jobs could be increased. Maybe multiple jobs could be selected to increase or decrease the priority so the priority would not have to be changed one job at a time.

There is no priority associated with tasks in SJQv4. Not sure if I'll be bringing the priority queue back; don't think there's a real benefit to it.

CPU priority might be another item. Although the CPU priority could be done by going to the Task Manager, select the task, and hover the mouse over the 'change priority' option in Task Manager. Maybe CPU priority would not be needed once it was determined that all the jobs were set to 'Low' CPU priority. If the 'comskip.exe' process isn't set to 'Low' priority, it can kill the computer's overall performance.

CPU settings also is not ported, but this probably will be now that I think it about. I do see value in this feature. I'd suggest you open a ticket on the proj site for this one before I forget about it again.

The video file type, mpeg2, TS would be nice to know. The TS files take much longer to process with Comskip than mpeg2 files. The user will then have a better idea why some jobs are taking longer to process than others in the status screen.

You want the video type in the status screen? The problem here, which Tom has addressed through some API change requests for me to implement, is that not every task is associated with a media file. So to generically say, "I'd like to see the video type in the status page" is not really valid because some tasks won't have this info because they aren't associated with a media file object (or any object, possibly). I suppose media file tasks could include this info, but as you can see in the screenshot, space is limited so you'll have to choose wisely as to which info is on the screen.

[tmiranda]

Slugger beat me to the punchline regarding Queue Prioroty and CPU Priority.

The number of items in the queue is already there in the lower right hand corner. The first number is the number of items passing the filter and the second number is the total.

Adjustable refresh rate should not be a problem.

Cramming more information on the screen will be a problem. As it is I had to shrink the font size to get things displayed properly and letting the user choose what items to display will be a lot of work, that I'm not up for at the moment. As Slugger said, we have to choose carefully what information to display.

[razrsharpe]

Originally Posted by Slugger

There is no priority associated with tasks in SJQv4. Not sure if I'll be bringing the priority queue back; don't think there's a real benefit to it.

The only benefit i see for it is when you are processing a backlog of tasks on your existing media collection but want to positively ensure that newly recorded content gets processed immediately before the remaining backlog.

[Slugger]

Originally Posted by razrsharpe

The only benefit i see for it is when you are processing a backlog of tasks on your existing media collection but want to positively ensure that newly recorded content gets processed immediately before the remaining backlog.

True, but, presumably, those new content tasks are being queued by the same mechanism as the backlog tasks so wouldn't the priority of all the tasks be the same anyway? If you have different loaders for different types of tasks that do the same thing (i.e. load new content via events and old content via a scanner cronjob) then, yes, a priority would be nice. Is that how you're doing it?

My problem with the priority queue is that it's never really fully honoured for various reasons. Using your example, backlog tasks may get inserted first and assigned to clients just before new content tasks are added to the queue so your priority isn't being honoured. I would never add/allow logic to have actively running tasks with a lower priority be halted and sent back to the queue to allow a newly inserted higher priority task to run (just a nightmare waiting to happen if I tried to implement it) so in reality the whole idea of priority based tasks are, at best, fuzzy. This is why I silently dropped it in SJQv4. My reasoning is that eventually everything gets done. I do understand the use case you present, but do people really get a backlog of hundreds or even dozens of tasks where a priority queue would actually be desperately required (other than perhaps an initial install of SJQ)?

[razrsharpe]

Originally Posted by Slugger

True, but, presumably, those new content tasks are being queued by the same mechanism as the backlog tasks so wouldn't the priority of all the tasks be the same anyway? If you have different loaders for different types of tasks that do the same thing (i.e. load new content via events and old content via a scanner cronjob) then, yes, a priority would be nice. Is that how you're doing it?

Personally I'm not doing it at all because i dont have a backlog of tasks right now... but if i did then yes i would have a separate task for loading the backlog via cron and new tasks via events... or say you wanted to prioritize one type of task over another (comskipping live vs. encoding)

My problem with the priority queue is that it's never really fully honoured for various reasons. Using your example, backlog tasks may get inserted first and assigned to clients just before new content tasks are added to the queue so your priority isn't being honoured. I would never add/allow logic to have actively running tasks with a lower priority be halted and sent back to the queue to allow a newly inserted higher priority task to run (just a nightmare waiting to happen if I tried to implement it) so in reality the whole idea of priority based tasks are, at best, fuzzy. This is why I silently dropped it in SJQv4. My reasoning is that eventually everything gets done. I do understand the use case you present, but do people really get a backlog of hundreds or even dozens of tasks where a priority queue would actually be desperately required (other than perhaps an initial install of SJQ)?

Certainly the priority queue can and should only be honored at the time the tasks are picked up by a client... I agree that anything else would be crazy to consider and crazier to support.

Use case: say for example I decide i want to transcode all my dvds into something smaller or transcode all my tv older then a month... right now those tasks would take over the queue and prevent comskipping of new recordings... I guess you could limit the transcoding tasks to not take place during peak recording times which would solve most problems... but there are times when recordings happen at off peak recording times...

One place where i use the priority queue on a nightly basis is for hdpvr recordings. I prioritize recordings from the HDHR because those complete faster and i have a higher probability of watching those recordings before the ones from the hdpvr...

[phareous]

For some reason when I go to view the completed tasks, it freezes the entire sagetv service. Any ideas? Is it a bug?

[Slugger]

Originally Posted by phareous

For some reason when I go to view the completed tasks, it freezes the entire sagetv service. Any ideas? Is it a bug?

I've never seen this happen. You'll need to enable debug logging, reproduce, generate the logs for tmiranda and then open a bug report on the project site.

[phareous]

I restarted the service and it stopped freezing. I do notice that sometimes when I go to view the log on a remote client, it doesn't return anything. Not a huge deal though

[tmiranda]

Originally Posted by phareous

I restarted the service and it stopped freezing. I do notice that sometimes when I go to view the log on a remote client, it doesn't return anything. Not a huge deal though

Random freezes are not a good thing, if it happens again please let me know.

[kchase56]

Recently I've moved from SJQ v3 to SJQ v4. I've installed the Windows Service SJQ Task Client, SJQ v4 Plugin and UI. I can execute test scripts such as SCANFORMEDIA and it works just fine. I'm having problems executing tasks that run .exe's such as ShowAnalyzer.exe directly. They always fail. I've looked at the various logs (sjq.log, sjqagent.log and wrapper.log), but there isn't anything that really clues me in as to what is going on. I've tried enabling DEBUG logging, but that doesn't seem to be working (ie. I still only get WARN and INFO log information). I've attached my logs and sjqagent.properties files.

Any information that someone could provide regarding my situation would be greatly appreciated.

@Slugger: Thanks for all your hard work on all of your excellent tools and plugins!

Thanks

[KarylFStein]

Originally Posted by kchase56

Recently I've moved from SJQ v3 to SJQ v4. I've installed the Windows Service SJQ Task Client, SJQ v4 Plugin and UI. I can execute test scripts such as SCANFORMEDIA and it works just fine. I'm having problems executing tasks that run .exe's such as ShowAnalyzer.exe directly. They always fail. I've looked at the various logs (sjq.log, sjqagent.log and wrapper.log), but there isn't anything that really clues me in as to what is going on. I've tried enabling DEBUG logging, but that doesn't seem to be working (ie. I still only get WARN and INFO log information). I've attached my logs and sjqagent.properties files.

Any information that someone could provide regarding my situation would be greatly appreciated.

@Slugger: Thanks for all your hard work on all of your excellent tools and plugins!

Thanks

One thing that fist sticks out: change $SJQ_LAST_SEGMENT to $SJQ4_LAST_SEGMENT in your exe args.

[Slugger]

Originally Posted by kchase56

Recently I've moved from SJQ v3 to SJQ v4. I've installed the Windows Service SJQ Task Client, SJQ v4 Plugin and UI. I can execute test scripts such as SCANFORMEDIA and it works just fine. I'm having problems executing tasks that run .exe's such as ShowAnalyzer.exe directly. They always fail. I've looked at the various logs (sjq.log, sjqagent.log and wrapper.log), but there isn't anything that really clues me in as to what is going on. I've tried enabling DEBUG logging, but that doesn't seem to be working (ie. I still only get WARN and INFO log information). I've attached my logs and sjqagent.properties files.

Any information that someone could provide regarding my situation would be greatly appreciated.

@Slugger: Thanks for all your hard work on all of your excellent tools and plugins!

Thanks

From your sjqagent.log:

2011-03-04 10:16:53,372 INFO  [ProcessRunner]: Converted '"$SJQ4_PATH\$SJQ_LAST_SEGMENT"' to '"\\HT-PC\SageTV Recordings\$SJQ_LAST_SEGMENT"'

It should be "$SJQ4_PATH\$SJQ4_LAST_SEGMENT" as the exe argument. As the log shows, it's not expanding the vars properly because of the typo so presumably ShowAnalyzer is failing because the file name doesn't exist. ShowAnalyzer doesn't seem to dump output to stdout but instead to its own log file located where the video file is. If you look in that log it will probably give a "file not found" type error.

Either way, fix the args in SJQ then the task should run properly.

[wayner]

Tonight I upgraded to Sage 7.12 and the new Diamond UI. I no longer appear to have SJQ in my setup menu - I am not 100% sure that it was these actions that caused the issue or something else. Has anyone else seen this?

Edit - never mind as the UI was disabled - perhaps since it needed an upgrade?

[madas]

Hello,

I'm running the sample comskip script that is shown on the wiki.

Unfortunately it seems to be trying to comskip every live-tv show that I watch as soon as the program finishes. The old SJQ used to have a function "IsScheduled" which was true or false depending on if it was a scheduled recording or just something that was being watched.

Any ideas?

Thanks

[KarylFStein]

Originally Posted by madas

Hello,

I'm running the sample comskip script that is shown on the wiki.

Unfortunately it seems to be trying to comskip every live-tv show that I watch as soon as the program finishes. The old SJQ used to have a function "IsScheduled" which was true or false depending on if it was a scheduled recording or just something that was being watched.

Any ideas?

Thanks

I switched to using media_file_scanner.groovy as a crontab entry because of this, (didn't want comskip on SlimPlayer recordings plus I had some non-comskipped recordings that I wanted automatically processed). But, I *think* you can add an entry like this to your test to get what you want:

if (AiringAPI.IsNotManualOrFavorite(mf)) return 2

But I may be wrong . That's something in the media scanner and works for me to aviod live recordings.

[Slugger]

Originally Posted by madas

Hello,

I'm running the sample comskip script that is shown on the wiki.

Unfortunately it seems to be trying to comskip every live-tv show that I watch as soon as the program finishes. The old SJQ used to have a function "IsScheduled" which was true or false depending on if it was a scheduled recording or just something that was being watched.

Any ideas?

Thanks

Did you also install the comskip_test.groovy script as the test for the task? Did you config that test script to ensure the comskipLive = false is set?

Originally Posted by KarylFStein

I switched to using media_file_scanner.groovy as a crontab entry because of this, (didn't want comskip on SlimPlayer recordings plus I had some non-comskipped recordings that I wanted automatically processed). But, I *think* you can add an entry like this to your test to get what you want:

if (AiringAPI.IsNotManualOrFavorite(mf)) return 2

But I may be wrong . That's something in the media scanner and works for me to aviod live recordings.

That would work.

[kchase56]

Originally Posted by Slugger

From your sjqagent.log:

2011-03-04 10:16:53,372 INFO  [ProcessRunner]: Converted '"$SJQ4_PATH\$SJQ_LAST_SEGMENT"' to '"\\HT-PC\SageTV Recordings\$SJQ_LAST_SEGMENT"'

It should be "$SJQ4_PATH\$SJQ4_LAST_SEGMENT" as the exe argument. As the log shows, it's not expanding the vars properly because of the typo so presumably ShowAnalyzer is failing because the file name doesn't exist. ShowAnalyzer doesn't seem to dump output to stdout but instead to its own log file located where the video file is. If you look in that log it will probably give a "file not found" type error.

Either way, fix the args in SJQ then the task should run properly.

Thanks Slugger that was the problem and everything is working properly now.

Again thanks for all of your hard work.

[bikesquid]

New problem... Had to rebuild the sage box for other reasons and now when I try to edit a task for a client in the UI (via a client) I get a "Internal error updating client". Completed tasks just shows Loading....

Any ideas?

sjq.log shows the following during that timeframe:

2011-03-06 14:39:18,364 ERROR [Handler]: IOError
java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(Unknown Source)
at java.io.ObjectInputStream$BlockDataInputStream.readUnsignedShort(Unknown Source)
at java.io.ObjectInputStream$BlockDataInputStream.readUTF(Unknown Source)
at java.io.ObjectInputStream.readUTF(Unknown Source)
at com.google.code.sagetvaddons.sjq.listener.Handler.run(Handler.java:62)
at java.lang.Thread.run(Unknown Source)
2011-03-06 14:39:18,574 INFO [Handler]: CMD: QUIT :: PEER: /192.168.1.119
2011-03-06 14:39:18,784 INFO [Handler]: CMD: QUIT :: PEER: /192.168.1.119
2011-03-06 14:40:04,584 INFO [AgentManager]: Pinging Client[host=192.168.1.119:23344,state=ONLINE,lastUpdate=2011-03-06 14:38:16.863]
2011-03-06 14:40:05,194 INFO [ListenerClient]: Disconnected from 192.168.1.119:23344
2011-03-06 14:40:05,194 INFO [AgentManager]: Pinging Client[host=Master-PC:23344,state=ONLINE,lastUpdate=2011-03-06 14:38:42.034]
2011-03-06 14:40:05,204 INFO [ListenerClient]: Disconnected from Master-PC:23344
2011-03-06 14:40:05,364 INFO [TaskQueue]: Scheduling queue processor for ~8 seconds from now!
2011-03-06 14:40:13,364 INFO [TaskQueue]: Running queue processor now!

[Slugger]

Nothing wrong there (on the server side), the IOException is normal(ish).

You will need to get the SageClient log file with debug enabled and open a ticket on the project site for Tom to investigate. You should see errors and or exceptions related to SJQ in that log file (sageclient_0.txt). That's what Tom will need to see.

[tmiranda]

Originally Posted by Slugger

You will need to get the SageClient log file with debug enabled and open a ticket on the project site for Tom to investigate. You should see errors and or exceptions related to SJQ in that log file (sageclient_0.txt). That's what Tom will need to see.

Yes, logfile needed please. Did you change anything when you rebuilt the box?

Slugger - I'm assuming the STV is getting an error from one of the APIs that access the H2 database. Any chance his database is corrupt and/or in some funky state that it's not supposed to be in?