Plugin: SJQv4

[lenkost]

Fixed task.comskip.exe=script: ...
Installed agent 1350
Still no luck

Log for executable in SJQ UI shows:
The commandline used was: C:\comskip\comskip.exe E:\SageTV2\EarlyEddition ... .ts
No such file or directory - could not open file

It seems like mapping did not work.

sjqagent.properties:

#Generated by SJQv4 agent
#Fri Jan 28 10:44:49 EST 2011
agent.mapdir=D:\\Sagetv1\\,\\\\LenTV\\Sagetv1\\;E:\\Sagetv2\\,\\\\LenTV\\Sagetv2\\
task.comskip.exe=script\:\\\\LenTV\\SJQScripts\\comskip.groovy
task.comskip.schedule=ON
task.comskip.args="$SJQ4_PATH\\$SJQ4_LAST_SEGMENT"
task.comskip.test=\\\\LenTV\\SJQScripts\\comskip_test.groovy
agent.schedule=ON
task.comskip.testargs=ts
task.comskip.rcmin=0
task.comskip.maxprocs=4
task.comskip.maxtimeratio=1.0
agent.resources=100
task.comskip.maxtime=14400
task.comskip.rcmax=1
agent.port=23344
task.comskip.resources=25

sjqagent.log

2011-01-28 18:11:39,649 INFO  [Exe]: Received task 1683 of type 'COMSKIP' from 192.168.0.110:23347...
2011-01-28 18:11:39,649 INFO  [192_168_0_110-23347-1683]: Starting process runner for: Task[id=COMSKIP, reqRes=25, maxInst=4, sched=ON, maxTime=14400, maxTimeRatio=1.0, rc=0-1, exe=script:\\LenTV\SJQScripts\comskip.groovy, exeArgs="$SJQ4_PATH\$SJQ4_LAST_SEGMENT", test=\\LenTV\SJQScripts\comskip_test.groovy, testArgs=ts]
2011-01-28 18:11:39,649 INFO  [ProcessRunner]: Converted '"$SJQ4_PATH\$SJQ4_LAST_SEGMENT"' to '"E:\SageTV2\EarlyEdition-TheWedding-13612897-0.ts"'
2011-01-28 18:11:41,462 INFO[ListenerClient]: Disconnected from 192.168.0.110:23347
2011-01-28 18:11:42,368 INFO  [Config]: Watching 'C:\sjqagent\bin\..\conf'
2011-01-28 18:11:43,868 INFO[ListenerClient]: Disconnected from 192.168.0.110:23347
2011-01-28 18:11:44,884 INFO[ListenerClient]: Disconnected from 192.168.0.110:23347
2011-01-28 18:11:44,884 WARN  [ProcessRunner]: Removing task from active list: SJQ4Task-192.168.0.110-23347-1683

Wrapper.log

STATUS | wrapper  | 2011/01/28 17:57:22 | Launching a JVM...
INFO   | jvm 1    | 2011/01/28 17:57:22 | WrapperManager: Initializing...
INFO   | jvm 1    | 2011/01/28 17:57:22 | LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties
INFO   | jvm 1    | 2011/01/28 17:57:22 | Embedded SageAPI is not functional.  We are most likely running remotely.
INFO   | jvm 1    | 2011/01/28 17:57:22 | Adding Remote Server: lentv
INFO   | jvm 1    | 2011/01/28 17:57:22 | SJQv4 Agent (Task Client) v4.0.0.1350
INFO   | jvm 1    | 2011/01/28 17:57:22 | 
INFO   | jvm 1    | 2011/01/28 17:57:22 | The following scripting engines are available in this task client:
INFO   | jvm 1    | 2011/01/28 17:57:22 | 	Groovy Scripting Engine/2.0 [groovy]
INFO   | jvm 1    | 2011/01/28 17:57:22 | 	Mozilla Rhino/1.6 release 2 [js]
INFO   | jvm 1    | 2011/01/28 17:57:22 |

[Slugger]

It's a bug with the mapDir() function. Currently, it's case sensitive on Windows when it shouldn't be. I've fixed the code and the next task client build will fix it, but in the mean time, just edit your agent.mapdir property to match the case of the paths being returned by Sage:

E:\\SageTV2 instead of E:\\Sagetv2 in your agent.mapdir setting should fix the problem.

[lenkost]

This helped, but there is another strange phenomena:

When task is started, I see a comskip.exe process in task manager. I may have up to 4 of them and they usually consume 20-22% CPU each. But here via script they all consume 0. As soon as I kill task in SJQ UI, corresponding comskip.exe remains and CPU usage for this specific process grows to normal.
If I kill all tasks, all of them go to normal CPU usage.
And this answers my previous question: Comskip.exe process is not removed if task is killed.

Also current directory for each comskip task is reported by task manager as C:\sjqagent\bin\. I am not sure whether comskip.exe should run in its own directory.

I see a line in the log:

[ProcessRunner]: Converted '"$SJQ4_PATH\$SJQ4_LAST_SEGMENT"' to '"E:\SageTV2\GreenHornet-EatDrinkandBeDead-13682922-0.ts"'

Would it be possible to implement MapDir so ProcessRunner convers this path using user mapping? Then Comskip.exe could be run without script?

Thank you.

[Slugger]

Originally Posted by lenkost

This helped, but there is another strange phenomena:

When task is started, I see a comskip.exe process in task manager. I may have up to 4 of them and they usually consume 20-22% CPU each. But here via script they all consume 0. As soon as I kill task in SJQ UI, corresponding comskip.exe remains and CPU usage for this specific process grows to normal.
If I kill all tasks, all of them go to normal CPU usage.
And this answers my previous question: Comskip.exe process is not removed if task is killed.

Replace the groovy script for comskip with this (making the same adjustments as necessary):

def exe = "c:/comskip/comskip.exe"
// I'm assuming the first cmd line arg is the file name to be comskipped,
// adjust accordingly
def fileName = Tools.mapDir(SJQ4_ARGS[0])
println "Using '${fileName}' as argument..."

def cmdLine = [exe, fileName]
println "Executing ${cmdLine}..."
Process p = cmdLine.execute()
// Let's properly chew up the in/out/err streams to avoid deadlocking
// (thought Groovy did this automatically, but seems it doesn't)
p.consumeProcessOutput(System.out, System.err)
p.waitFor()
return p.exitValue()

Also current directory for each comskip task is reported by task manager as C:\sjqagent\bin\. I am not sure whether comskip.exe should run in its own directory.

You can set the working directory of the call to comskip by changing the execute() call from cmdLine.execute() to cmdLine.execute(null, new File("C:/my_dir")), which will change the working directory of the exe to start in C:/my_dir (of course, adjust accordingly).

With that said, I don't think that's your problem. The actual problem should be fixed with the script edit mentioned above.

I see a line in the log:

[ProcessRunner]: Converted '"$SJQ4_PATH\$SJQ4_LAST_SEGMENT"' to '"E:\SageTV2\GreenHornet-EatDrinkandBeDead-13682922-0.ts"'

Would it be possible to implement MapDir so ProcessRunner convers this path using user mapping? Then Comskip.exe could be run without script?

Thank you.

Is it possible? Yes. Is it going to happen? Unfortunately, no. To do this would require I add some kind of code parser/executer to the cmd args interpreter, which is just a big mess that I've aimed to avoid in SJQv4. Any special processing, etc. that one needs to do will need to be done in Groovy (or any other language you add to your task client).

(Lots of programmer speak here, skip to bold paragraph if you don't care. )With a lot of black magic and hacking, I could enhance this wrapper to track and kill the launched exe on Linux/Unix, but on Windows, it just can't happen. When you tell Java to kill a subprocess it sends a SIGTERM signal and when a JVM receives a SIGTERM is just forcibly kills the process, ignoring any registered shutdown hooks, which I could use to elegantly kill forked processes. On Linux/Unix, there are known workarounds where I can actually discover the process id of the processes being spawned by the JVM and then whip up some magic to kill those. On Windows, it isn't possible. And since, by design, the JVM creates processes outside its own process group, killing the JVM lets those forked processes continue to run, that's by design in the JVM. All of that added up together means exes launched in Groovy won't be killed when the JVM running the Groovy script is killed (on any OS).

Now, when I eventually add support for the test script to modify the exe args then you can get what you want: be able to directly call comskip.exe with the mapdir'd version of the file path. Until then, you'll have to use the groovy wrapper, knowing you can't kill the comskip processes.

[lenkost]

The last script change fixed CPU usage issues.

Now, when I eventually add support for the test script to modify the exe args then you can get what you want: be able to directly call comskip.exe with the mapdir'd version of the file path. Until then, you'll have to use the groovy wrapper, knowing you can't kill the comskip processes.

Glad to hear you are working on this. Not to be pushy, any estimate on when?
Is there a ticket so I can get progress updates?

Thank you.

[Slugger]

Originally Posted by lenkost

Glad to here you are working on this. Not to be pushy, any estimate on when?
Is there a ticket so I can get progress updates?

Thank you.

Star this ticket. Currently no ETA. Might be today if I don't head out anywhere. If not today, then earliest I'd look at it again would be next weekend.

[Slugger]

Here's a preview of what's coming up in the next SJQv4 build:

Server changes:

• Fix 274: Support task output larger than 64KB
• Fix 287: Allow dynamic modification of exe arguments from test script
• Fix 288: Allow dynamic modificaiton of a task's resource usage at runtime

Task client changes:

• Fix bug with mapDir() where file names were not being treated case insensitive on Windows
• Fix 274: Support task output larger than 64KB
• Fix 287: Allow dynamic modification of exe arguments from test script
• Fix 288: Allow dynamic modificaiton of a task's resource usage at runtime

Details on how to use the methods to modify exe args and task resources are available here.

This build has not been released yet - I'm still running some testing on it, but should be uploaded in the next few days.

[Spectrum]

Looking forward to it! I've already set up my new comskip script to use the exe args changes and modify the resources. Should reduce the number of tasks I'm queueing by 2/3 once I make sure it runs

[Slugger]

Originally Posted by Spectrum

Looking forward to it! I've already set up my new comskip script to use the exe args changes and modify the resources. Should reduce the number of tasks I'm queueing by 2/3 once I make sure it runs

Hopefully you read the javadocs for the resource modification function? Specifically about the race condition that this feature introduces.

The initial release will allow total resource usage to exceed a client's max resource settings, as described in the javadocs, but I'm on the fence about whether I should reject a resource change from a script if it will exceed the client's max resources? I don't suspect most users will properly deal with a failed resource change and will be more confused when nothing happens so I decided to allow the max value to be exceeded under the conditions described in the javadocs. I'd be curious to hear other perspectives.

[Spectrum]

Yeah I saw the docs regarding the race condition after your commit/comment on that issue. I don't think it will be a problem for me; if it happens the system just gets a little busy and things will go back to normal as things finish up.

As for opinions on future releases, I guess it depends on what kind of error handling would be available if a resource re-allocation would be > max resources. From a test script instance it would make sense to re-allocate and then re-queue the job with the new resource level. Things get more complicated when it's in the middle of a script as a job though. Is it safe to stop and requeue? If there is a test script attached to the job and should it be re-run? Can it just stall in its current state and wait for more resources? If stalled and waiting does it wait at the front of the queue or the back of the queue? Should the script just create an error and let the user sort it out? Of course re-queuing may not even be possible, I'm just speculating but there certainly is a can of worms here and I don't have the answer. I'm just happy you got that in and am looking forward to it!

[Slugger]

Had a thought on this... if you initially assign the task the max resources it'll ever use and only ever decrease the task's resources at runtime then you'll never exceed the client's total resources. Doing it this way may mean some tasks will be delayed (by up to 3 mins), but you'll never exceed the 100 resources.

[lenkost]

Could you post test script implementing MapDir with the direct call to Comskip.exe?

Thank you.

[Slugger]

Originally Posted by lenkost

Could you post test script implementing MapDir with the direct call to Comskip.exe?

Thank you.

I haven't checked this into subversion yet (i.e. it could be buggy still), but this is my new comskip test script that I'm using. I've now been able to combine my HDCOMSKIP and SDCOMSKIP tasks into a single task, that I've named COMSKIP.

Create COMSKIP just like you would HDCOMSKIP. Make comskip.exe (or showanalyzerengine.exe) the exe for the task and make the script below the test. Set the required resources to the max you'd use for H.264 processing (for me that's 75). Then you'll see this script does the following:

1) Checks all the preconditions as usual (recording isn't active, edl doesn't already exist, etc.)

2) If the file name is NOT a .ts (i.e. it's not H.264) then decrease the resources being used by this task from 75 to 25.

3) mapDir() the directory of the recording and then reset the exe's arguments

/*
           Title: Comskip Tester for SJQv4
          Author: Derek Battams <derek AT battams DOT ca>
    Last Updated: 02 Feb 2011

    Use this groovy script as the test for your SJQv4 comskip tasks.  This version of the script allows you
    to skip running comskip if:

        * The recording file extension does not match the first command line argument passed to this script
        * If an edl (or equivalent) already exists in the same directory as the recording

    You can also configure this script to delay starting of comskip until the recording is completed.

    All of these settings are configured in the block below.  Do not edit below the line specified unless you
    know what you're doing (and if you do then send me your bug fix patches, please).
*/

/*************** CONFIGURE VALUES IN THIS BLOCK ONLY! ***************/

boolean checkRecordingType = true // Check the recording is of the type specified on the command line (i.e. only process mpg recordings)
boolean checkForEdl = true        // If true, mark the task as skipped if an edl already exists
String edlType = "edl"            // The edl extension (i.e. some people use .txt comskip files instead of .edl files)
boolean comskipLive = false       // If false, this test will push the task back to the queue while the recording is in progress
int nonTsRes = 25                 // Number of resources required for non-H.264 file processing
String h264Ext = "ts"             // File extension for H.264 content (usually 'ts')
/********************************************************************/

/************************* DO NOT EDIT BELOW!! **********************/

import org.apache.commons.io.FilenameUtils

// But if you do edit below to fix a bug, then email me the patch! ;)

Object mf = MediaFileAPI.GetMediaFileForID(Integer.parseInt(SJQ4_METADATA.get("SJQ4_ID")))
File path = Tools.mapDir(new File(SJQ4_METADATA.get("SJQ4_PATH"))) // Remap it based on client settings
String fileName = SJQ4_METADATA.get("SJQ4_LAST_SEGMENT")
String fileType = FilenameUtils.getExtension(fileName)

if(mf == null || path == null || path.getAbsolutePath().length() == 0 || fileName == null || fileName.length() == 0) {
   println("ERROR: Invalid environment data passed to scritpt!");
   return 2;
}

String edl = FilenameUtils.removeExtension(fileName) + "." + edlType;
if(new File(path, edl).exists()) {
   println("An edl already exists for this recording, skipping comskip!");
   println("Found '" + new File(path, edl).getAbsolutePath() + "'");
   return 2;
}

if(checkRecordingType && fileType != null && fileType.length() > 0 && !fileName.endsWith("." + fileType)) {
   println("Recording is not of specified type '" + fileType + "', skipping comskip!");
   return 2;
}

if(!comskipLive && MediaFileAPI.IsFileCurrentlyRecording(mf)) {
   println("Recording in progress, waiting...");
   return 1;
}

// Adjust the task's resources based on the file extension, etc.
if(!fileType.equals(h264Ext))
   Tools.setTaskResources(nonTsRes) // Lower the req'd resources if we aren't processing an H.264 recording; default already has it set to high for H.264

// Reset the arg to comskip/SA based on the mapDir()'d directory
Tools.setExeArgs("${path}\\\$SJQ4_LAST_SEGMENT")

return 0;

[lenkost]

Did not convert to network path.
Using test script without modifications.
Resources set to 25 because I only use ts and prefer to run 4x25

#Generated by SJQv4 agent
#Wed Feb 02 22:25:08 EST 2011
task.comskip.exe=C:\\comskip\\comskip.exe
task.comskip.schedule=ON
agent.mapdir=E:\\SageTV2\\,\\\\LenTV\\Sagetv2\\;D:\\SageTV1\\,\\\\LenTV\\Sagetv1\\
task.comskip.args="$SJQ4_PATH\\$SJQ4_LAST_SEGMENT"
task.comskip.test=\\\\LenTV\\SJQScripts\\comskip_test.groovy
agent.schedule=ON
task.comskip.testargs=ts
task.comskip.rcmin=0
task.comskip.maxprocs=4
task.comskip.maxtimeratio=1.0
agent.resources=100
task.comskip.maxtime=14400
task.comskip.rcmax=1
agent.port=23344
task.comskip.resources=25

2011-02-02 22:30:24,555 INFO  [Config]: Props file update detected, reloading properties from disk!
2011-02-02 22:30:57,618 INFO  [Exe]: Received task 1876 of type 'COMSKIP' from 192.168.0.110:23347...
2011-02-02 22:30:57,618 INFO  [192_168_0_110-23347-1876]: Starting process runner for: Task[id=COMSKIP, reqRes=25, maxInst=4, sched=ON, maxTime=14400, maxTimeRatio=1.0, rc=0-1, exe=C:\comskip\comskip.exe, exeArgs="$SJQ4_PATH\$SJQ4_LAST_SEGMENT", test=\\LenTV\SJQScripts\comskip_test.groovy, testArgs=ts]
2011-02-02 22:30:58,602 INFO  [Config]: Watching 'C:\sjqagent\bin\..\conf'
2011-02-02 22:30:59,727 INFO[ListenerClient]: Disconnected from lentv:23347
2011-02-02 22:31:01,243 INFO[ListenerClient]: Disconnected from 192.168.0.110:23347
2011-02-02 22:31:02,212 INFO  [ProcessRunner]: Converted 'E:\SageTV2\$SJQ4_LAST_SEGMENT' to 'E:\SageTV2\Hannity-13700821-0.ts'
2011-02-02 22:31:02,352 INFO[ListenerClient]: Disconnected from 192.168.0.110:23347
2011-02-02 22:31:03,352 INFO[ListenerClient]: Disconnected from 192.168.0.110:23347
2011-02-02 22:31:04,259 INFO[ListenerClient]: Disconnected from 192.168.0.110:23347
2011-02-02 22:31:04,259 WARN  [ProcessRunner]: Removing task from active list: SJQ4Task-192.168.0.110-23347-1876

[Slugger]

Change line 35 of the test script from this:

File path = Tools.mapDir(new File(SJQ4_METADATA.get("SJQ4_PATH")))

To this:

File path = Tools.mapDir(new File(SJQ4_METADATA.get("SJQ4_PATH") + File.separator))

Should fix it up.

[Slugger]

Fixed a few other bugs, replace the test script with this one (it includes the mapDir() fix from above):

/*
           Title: Comskip Tester for SJQv4
          Author: Derek Battams <derek AT battams DOT ca>
    Last Updated: 03 Feb 2011

    Use this groovy script as the test for your SJQv4 comskip tasks.  This version of the script allows you
    to skip running comskip if:

        * If an edl (or equivalent) already exists in the same directory as the recording
        * If the recording is in progress

    This script will also decrease the amount of resources it's using if it detects that it's not processing an H.264 file (optional, on by default).

    All of these settings are configured in the block below.  Do not edit below the line specified unless you
    know what you're doing (and if you do then send me your bug fix patches, please).
*/

/*************** CONFIGURE VALUES IN THIS BLOCK ONLY! ***************/

boolean checkForEdl = true        // If true, mark the task as skipped if an edl already exists
String edlType = "edl"            // The edl extension (i.e. some people use .txt comskip files instead of .edl files)
boolean comskipLive = false       // If false, this test will push the task back to the queue while the recording is in progress
boolean modResForH264 = true      // Decrease the used resources for this task if it's not processing an H.264 file?
int nonTsRes = 25                 // Number of resources required for non-H.264 file processing
String h264Ext = "ts"             // File extension for H.264 content (usually 'ts')
boolean adjustExeArgs = true      // Should the mapDir()'d file directory be applied to the exe args?

/********************************************************************/

/************************* DO NOT EDIT BELOW!! **********************/

// But if you do edit below to fix a bug, then email me the patch! ;)

import org.apache.commons.io.FilenameUtils

Object mf = MediaFileAPI.GetMediaFileForID(Integer.parseInt(SJQ4_METADATA.get("SJQ4_ID")))
File path = Tools.mapDir(new File(SJQ4_METADATA.get("SJQ4_PATH") + File.separator)) // Remap it based on client settings
String fileName = SJQ4_METADATA.get("SJQ4_LAST_SEGMENT")
String fileType = FilenameUtils.getExtension(fileName)

if(mf == null || path == null || path.getAbsolutePath().length() == 0 || fileName == null || fileName.length() == 0) {
   println("ERROR: Invalid environment data passed to scritpt!");
   return 2;
}

String edl = FilenameUtils.removeExtension(fileName) + "." + edlType;
if(checkForEdl && new File(path, edl).exists()) {
   println("An edl already exists for this recording, skipping comskip!");
   println("Found '" + new File(path, edl).getAbsolutePath() + "'");
   return 2;
}

if(!comskipLive && MediaFileAPI.IsFileCurrentlyRecording(mf)) {
   println("Recording in progress, waiting...");
   return 1;
}

// Adjust the task's resources based on the file extension, etc.
if(modResForH264 && !fileType.equals(h264Ext))
   Tools.setTaskResources(nonTsRes) // Lower the req'd resources if we aren't processing an H.264 recording; default already has it set to high for H.264

// Adjust the command line arguments
if(adjustExeArgs)
   Tools.setExeArgs("${path}${File.separator}\$SJQ4_LAST_SEGMENT")

return 0;

[Spectrum]

Originally Posted by Slugger

Fixed a few other bugs, replace the test script with this one (it includes the mapDir() fix from above):

Bah I come in to post that I got my new single comskip script up and running fine and find out you've made more updates to your example script; now I have to compare these updates I ditched the one I had pre-written in favor of your example as it was a bit more readable. I'm liking the ability to modify resources and command line args so thanks again for implementing that

[Spectrum]

Slugger, I've been waiting for new recordings to test my mover script in action and have hit a snag. I have a task setup that will get kicked off by cron called QUEUEMOVIES2MOVE that scans for movies that need to be moved to my NAS. That script queues the MVMEDIA script when it finds a movie that matches the appropriate criteria. The problem I found is that the MVMEDIA tasks don't run when queued by the QUEUEMOVIES2MOVE script, they just hang in the queue and are never assigned to a client. If I manually assign the MVMEDIA script to a recording it begins moving and transfers fine.

In the case I've captured below there was 1 task running (#679) that was using 67 resources so the agent had 33 resources free. The queue script ran and queued up the mvmedia script on 1 recording. The mvmedia script sat in the queue with the server (sjq.log) showing there were no clients available to accept the task.

The task configs are below and the groovy scripts and snippits from the logs are attached. Do you see anything that could be causing the task to get hung in the queue like that?


MVMEDIA config

C:\dev\sagegroovy>grep -i mvmedia "\\htpc\ddrive\Media Applications\SageTV\SageTV\plugins\sjq-agent\conf\sjqagent.properties"
task.mvmediafile.maxtimeratio=1.0
task.mvmediafile.maxprocs=1
task.mvmediafile.schedule=ON
task.mvmediafile.test=
task.mvmediafile.rcmin=0
task.mvmediafile.maxtime=86400
task.mvmediafile.testargs=
task.mvmediafile.exe=script\:D\:/Media Applications/SageTV/GroovyScripts/mv_media_file.groovy
task.mvmediafile.resources=5
task.mvmediafile.args=\\\\nesoi\\Movies\\
task.mvmediafile.rcmax=0

QUEUEMOVIES2MOVE config

C:\dev\sagegroovy>grep -i queuemovies "\\htpc\ddrive\Media Applications\SageTV\SageTV\plugins\sjq-agent\conf\sjqagent.properties"
task.queuemovies2move.args=
task.queuemovies2move.rcmax=0
task.queuemovies2move.maxprocs=1
task.queuemovies2move.schedule=ON
task.queuemovies2move.exe=script\:D\:/Media Applications/SageTV/GroovyScripts/queue_movies_to_move.groovy
task.queuemovies2move.testargs=
task.queuemovies2move.test=
task.queuemovies2move.rcmin=0
task.queuemovies2move.maxtimeratio=1.0
task.queuemovies2move.resources=1
task.queuemovies2move.maxtime=86400

[Slugger]

You're queuing up tasks named 'MVMEDIA' from the queue up script, but your sjqagent.properties file references a task named 'MVMEDIAFILE'. As far as I can tell, you have no task client capable of running tasks named 'MVMEDIA'. I'd probably change the queue up script to queue up 'MVMEDIAFILE' tasks instead of 'MVMEDIA' tasks.

[Slugger]

For those interested, the repository URL has changed:

http://sagetv-addons.googlecode.com/...nk/SJQScripts/

It's no longer an Eclipse project - I edit from the SageGroovy IDE and it was getting to be a pain to edit there then go to Eclipse to commit. Anyway, I think I've updated most of the links in the wiki docs, but if you come across any I missed please let me know.