Plugin: Sage Job Queue (SJQ)

[madas]

Hi Slugger,

I have my rules back in (Comskip then Transcode) and all seems good with the latest beta!

One thing i've noticed with the latest beta is that is doesn't seem to clear up files. It I do a "delete now" it doesn't clear up the .txt files for .mpg's that are no longer there. Here are my settings

Delete files with extensions matching: edl,log,txt
When it does not have...: avi,mpeg,mpg
Auto Delete Daily: ticked

its leaving .txt files in my Sage media directory even after the mpg file has been removed.


Thanks!

[Slugger]

Originally Posted by madas

Hi Slugger,

I have my rules back in (Comskip then Transcode) and all seems good with the latest beta!

One thing i've noticed with the latest beta is that is doesn't seem to clear up files. It I do a "delete now" it doesn't clear up the .txt files for .mpg's that are no longer there. Here are my settings

Delete files with extensions matching: edl,log,txt
When it does not have...: avi,mpeg,mpg
Auto Delete Daily: ticked

its leaving .txt files in my Sage media directory even after the mpg file has been removed.


Thanks!

Great... when I have a free hour some evening this week I'll package it up and upload 1.5.0 to SourceForge.

As for the file cleanup, the inputs must be regular expressions. So by using commas you're actually telling SJQ to look for files with the literal extension "edl,log,txt". Instead, use this:

Delete files matching: edl|log|txt
When it does not have: avi|mpeg|mpg

[madas]

Hi Slugger,

I put in pipes | between the extensions like you suggested, after I hit "update" it changes them to dots .

Is this normal? it still doesn't seem to be clearing up the directory.

Thanks

[Slugger]

Originally Posted by madas

Hi Slugger,

I put in pipes | between the extensions like you suggested, after I hit "update" it changes them to dots .

Is this normal? it still doesn't seem to be clearing up the directory.

Thanks

Definitely not normal. When do they get changed to periods? In the browser or when written to the ruleset file? This is definitely weird.

[madas]

slugger,

It gets changed in the browser when i hit "update". I don't see those settings being written to rules.sjq...should they?

[Slugger]

Originally Posted by madas

slugger,

It gets changed in the browser when i hit "update". I don't see those settings being written to rules.sjq...should they?

Yeah, they shouldn't be in the ruleset file. I'm almost 100% positive I do no transformation of that input at all, but it's been so long since I've actually modified those settings in my installation that I'm going to have to go and test it myself tonight after work. Which browser(s) do you use? Which one(s) is/are reproducing these results?

[madas]

IE7 and Firefox 3

[Slugger]

Originally Posted by madas

IE7 and Firefox 3

I cannot reproduce the behavior in IE nor Firefox. Mind you I use IE6 and Firefox 2, but I highly doubt that's the issue. Do you happen to have any plugins installed in the browsers that might fiddle with form data?

To determine if it's the browser or SJQ, please manually load and run this URL in a browser:

http://your_server/Sage/SJQ?sub=rmUpdate&del_regex=edl%7Clog%7Ctxt&if_regex=avi%7Cmpeg%7Cmpg&auto_del=true

Your browser will probably prompt you to save a file, go ahead and save it somewhere then open it (it's just plain text). The contents should be just "{}". If not, then please reply with the contents. Once this is done reload the servlet and the boxes should be filled with "edl|txt|log" and "avi|mpg|mpeg" with the auto box checked, as expected. If that's the case then we know it's your browser modding the values before submitting the form. If it still loads dots then we have some more investigating to do. Let me know how it goes.

[Slugger]

SJQ v1.5.0 is now available.

It is recommended that all users of SJQ upgrade to this version as it includes an updated JDBC driver that addresses memory leak issues with SJQ.

v1.4.0 -> v1.5.0
================

* 1994751: Add new test: IsArchived
* 2010817: JDBC fixes; upgrade SQLite JDBC driver

[madas]

Hi Slugger,

I used your link and now the field is populated correctly. The file that the server sent contained just {} as you said.

I don't have any plugins/form fillers that would fiddle with the data, and i'm doing it over a LAN, no proxies etc. Very strange!

[Slugger]

Originally Posted by madas

Hi Slugger,

I used your link and now the field is populated correctly. The file that the server sent contained just {} as you said.

I don't have any plugins/form fillers that would fiddle with the data, and i'm doing it over a LAN, no proxies etc. Very strange!

The results of this test point to your browser as being the one changing the value before SJQ even processes it. So we're down to two possibilities here:

1. Something specific to your browser is causing those values to be changed prior to the form submission

2. The SJQ JavaScript code is modifying the values.

I can't reproduce the scenario you describe with IE nor Firefox and I've reviewed the JS code that handles the form submission and I do nothing in my code but read what's typed into those boxes so I'm about as sure as I'll ever be that it's not SJQ modifying the values (I'd say 100% sure, but I've been in the software industry long enough to catch myself before typing such a sentence).

If you happen to be a coder yourself, then feel free to dive into the JavaScript code in SageTV\webserver\sage\sjqweb\sjq.js and prove me wrong - you wouldn't be the first person to find bugs in my code where I was sure they didn't exist. Search for this line in that file (it's towards the bottom):

YAHOO.util.Event.addListener('sjqAutoRm', 'click', function(e)

That's where the event listener is defined that handles the update button click for that section of the form. Unfortunately, unless something new is brought up, I'm stumped as to the cause of this one. Again, I'm leaning more towards #1 as the cause, but am more than willing to address the issue in SJQ should you be able to find something that points to it as the cause.

[madas]

So long as we have a workaround! other than that everything is going great...so good work on this...SJQ is an extremely powerful tool

[perfessor101]

Hello,
I was curious where are the rules set and logs stored?

I'm just setting up a new server :
Installed Windows XP MCE 2005 with SP3

sorry, rant and warning to others to follow
(microsoft doesn't have sp3 specifically ready for Windows XP MCE 2005 ... but they let you install anyway, nice people ;-) (and if you slipstream SP3 in or use the SP3 redistributable you're really borked, I'm in Love)
rant finished and thank you for your patience

Installed latest Java from Sun (1.6.0_06)
Installed SageTV latest Full version (6.3.10.166)
Installed SageMC and Video Tools (latest)
Installed Nielms Webserver (2.22)
Installed SageJobQueue (1.50) through windows installer version
Installed VideoRedo Plus (2.5.6.512)

The rule I have set up doesn't seem to run ... and when I restart the server the Ruleset Editor has no rules listed. I've clicked save a few times but the ruleset doesn't seem to be loading after a restart.

Also it seems to be repeatedly asking for my login\password and periodically it's yelping about AJAX something.

I'm going to read through the manual install instructions just in case I or the installer may have missed something. I would have yesterday but your website and Nielm's wasn't loading yesterday.

Thanks for your time,
Bobby

[Slugger]

Originally Posted by perfessor101

Hello,
I was curious where are the rules set and logs stored?

The ruleset file should be located at \SageTV\rules.sjq and the logs are stored internally in the SJQ database - viewable via the web servlet (there is an open feature ticket in the project to create a program capable of extracting the log to a plain text file, but haven't got around to it yet).

Anytime you click the save button you should get a dialog back telling you the save succeeded. If you're getting AJAX failure dialogs then the browser is unable to connect to the web server (possible firewall issues?).

[perfessor101]

The website (http://www.battams.ca/sjq/doc/) is still giving me a "The page you have requested could not be found. (404) " error.

(although I went to into the Google Cache and loaded one from 28 Jun 2008 19:21:28 GMT and printed it)

I can't find a rules.sjq file in my c:\program files\SageTV\SageTV directory or anywhere under it ...

ok found this line missing in the servlets.properties file servlet./sage/SJQ.code=net.sourceforge.sagetv.sjq.SJQServlet after the windows install on my machine. (copied it in from the webpage)

Is there a place where I can find a complete listing of what should be in the servlets.properties file? (I downloaded SJQ-1.5.0.zip and I'll check in there)

There seems to be a week or two difference on some of the file-dates from the installer as compared to those in the zip file (1.5.0). I'm not sure if that's a compressing versus installer creation difference.

I tried dragging all the files in the SJQ-1.5.0.zip and dropped them in the SageTV directory.
Shut it down and restarted it a few times.
The webpage says the rules are saved each time I click on Save Rules, but when I check the SageTV folder I can't find the rules.SJQ file.
When I type in values on the configuration page they are there the next time I restart my browser ... but they're all gone if I restart the SageTV Service.

and now the AJAX warning only seems to come up if I leave the WebPage up when I shutdown the sagetvservice.

What should the name of the sql file for the database be? (to check if it's being created)

I guess checking the sagetv_*.txt logs is next? What would I be looking for in there?

Thanks for the help,
Bobby

[nyplayer]

Try this website.

http://sagetv.battams.ca/?d=sjq#tests

[perfessor101]

Hello,
Here's a grep sjq of the sagetv*.txt logs ... nothing jumps out to me ...

Is there any way for me to see the 'dumped messages' ?

I'm not sure where I got that other web address from ... but thanks for the right one.

Bobby

[Slugger]

Originally Posted by perfessor101

The website (http://www.battams.ca/sjq/doc/) is still giving me a "The page you have requested could not be found. (404) " error.

(although I went to into the Google Cache and loaded one from 28 Jun 2008 19:21:28 GMT and printed it)

Bad/old URL. Where did you find that one (obviously I've missed some that need to be updated). http://sagetv.battams.ca (for now, eventually all of the content on that site will move to sourceforge).

Originally Posted by perfessor101

I can't find a rules.sjq file in my c:\program files\SageTV\SageTV directory or anywhere under it ...

C:\Program Files\SageTV\SageTV>dir rules.sjq
 Volume in drive C has no label.
 Volume Serial Number is 9075-AEE2

 Directory of C:\Program Files\SageTV\SageTV

2008-06-18  15:22               313 rules.sjq
               1 File(s)            313 bytes
               0 Dir(s)  67,998,367,744 bytes free

C:\Program Files\SageTV\SageTV>dir sjq.sqlite
 Volume in drive C has no label.
 Volume Serial Number is 9075-AEE2

 Directory of C:\Program Files\SageTV\SageTV

2008-07-12  09:05         1,598,464 sjq.sqlite
               1 File(s)      1,598,464 bytes
               0 Dir(s)  67,998,367,744 bytes free

sjq.sqlite being the database file used by SJQ. It's a basic SQLite3 database file accessible with any recent SQLite3 client binary.

Originally Posted by perfessor101

ok found this line missing in the servlets.properties file servlet./sage/SJQ.code=net.sourceforge.sagetv.sjq.SJQServlet after the windows install on my machine. (copied it in from the webpage)

Is there a place where I can find a complete listing of what should be in the servlets.properties file? (I downloaded SJQ-1.5.0.zip and I'll check in there)

That class is the only thing needed for SJQ, I'm quite surprised the installer didn't handle that automatically.

Originally Posted by perfessor101

There seems to be a week or two difference on some of the file-dates from the installer as compared to those in the zip file (1.5.0). I'm not sure if that's a compressing versus installer creation difference.

Never really looked into it too closely, but I know the ZIP is constructed to maintain the timestamps of external libs I copy over. The installer builder may or may not maintain those timestamps - certainly file sizes should be identical. I always run both before uploading so I'm fairly certain both packages are good.

Originally Posted by perfessor101

I tried dragging all the files in the SJQ-1.5.0.zip and dropped them in the SageTV directory.
Shut it down and restarted it a few times.
The webpage says the rules are saved each time I click on Save Rules, but when I check the SageTV folder I can't find the rules.SJQ file.
When I type in values on the configuration page they are there the next time I restart my browser ... but they're all gone if I restart the SageTV Service.

This behaviour means that SJQ is unable to write the sjq.rules file. When you click save the browser updates it's internal data structure then attempts to dump that to the file on disk. So for some reason, the attempt to write to disk is returning a success code to the browser, but not writing the file for some reason. When you leave the web page the internal copy of the ruleset is lost and the next time you go back it first attempts to read it from the server, but if the file doesn't exist then you are back to an empty ruleset.

Originally Posted by perfessor101

and now the AJAX warning only seems to come up if I leave the WebPage up when I shutdown the sagetvservice.

This is expected. The browser queries the server every 30 seconds for the live, up to the second status of what SJQ is doing. If SageTV is stopped then the web server is stopped so the browser can't communicate for status info, which gives the AJAX errors.

Originally Posted by perfessor101

What should the name of the sql file for the database be? (to check if it's being created)

sjq.sqlite (as shown above)

Originally Posted by perfessor101

I guess checking the sagetv_*.txt logs is next? What would I be looking for in there?

Thanks for the help,
Bobby

Reading between the lines here, I get the feeling that maybe you have more than one SageTV instance (a separate server and a separate client) and that you're trying to split the SJQ install between them? Running SJQ on your server maybe and your web server runs from the client?

There's not much in sagetv logs that's going to help. First, you have to find out where the database file is going (the ruleset file being created by the servlet should be in the same spot). The very first thing SJQ does when started is initialize the database and if that fails then SJQ dies off quickly. Assuming that's not happening then somewhere sjq.sqlite must exist.

If you are running the web server on a SageTV client then it's in the client's SageTV directory where the ruleset and database files will live. Never really thought about it before, but the web server must exist on the same physical machine as the one running SJQ (if using the web servlet to config SJQ).

[nyplayer]

Firefox seems to be the problem I was just updating with FireFox nothing took... I used IE7 and it saves to the rules file. I could not even create a new rules file with FireFox. I started using Firefox yesterday.

[Slugger]

Originally Posted by nyplayer

Firefox seems to be the problem I was just updating with FireFox nothing took... I used IE7 and it save to the rules file. I could not even create a new rules file with FireFox.

Hmm... I only use FF. Though this laptop still has FF2 (work requirement), are you using FF3 by any chance? Though I thought I tried FF3 once with SJQ on my desktop, but may not have.