Can anyone host a mirror of SageTV hosted plugins?

[Slugger]

I've written the script to mirror all plugins hosted on download.sage.tv and it also auto edits the SageTVPlugins.xml file to reference the new links. It's working, I'm running the mirror on my personal server, but I cannot host it for all, I simply do not have the bandwidth to even pretend I could.

The script is a Groovy script, which probably doesn't help most, but I could probably port it to PHP or Perl or whatever with little effort. Alternatively, I'd be willing to run the script myself and rsync the repository with a public server, if someone has one and are willing and able to host a repository mirror.

PM me if you're willing and able to host this repository. I'll probably just keep mirroring it locally for now and if the need arises then we'd need to copy the mirror somewhere and provide the new SageTVPlugins.xml file.

[tmiranda]

Originally Posted by Slugger

I've written the script to mirror all plugins hosted on download.sage.tv and it also auto edits the SageTVPlugins.xml file to reference the new links. It's working, I'm running the mirror on my personal server, but I cannot host it for all, I simply do not have the bandwidth to even pretend I could.

The script is a Groovy script, which probably doesn't help most, but I could probably port it to PHP or Perl or whatever with little effort. Alternatively, I'd be willing to run the script myself and rsync the repository with a public server, if someone has one and are willing and able to host a repository mirror.

PM me if you're willing and able to host this repository. I'll probably just keep mirroring it locally for now and if the need arises then we'd need to copy the mirror somewhere and provide the new SageTVPlugins.xml file.

Why not keep it on google code? I would love the irony of that. I'm sure they allow enough discspace to accomodate it.

[Slugger]

Originally Posted by tmiranda

Why not keep it on google code? I would love the irony of that. I'm sure they allow enough discspace to accomodate it.

If the src for a binary isn't available, you can't store the binary on GC.

[tmiranda]

Originally Posted by Slugger

If the src for a binary isn't available, you can't store the binary on GC.

How about something like DropBox or whatever Microsoft is calling their free web storage solution these days?

[Slugger]

Originally Posted by tmiranda

How about something like DropBox or whatever Microsoft is calling their free web storage solution these days?

Preferably, someone's got a good old fashioned server that they can run a script from cron to sync with download.sage.tv and then make the links available via HTTP. These free storage solutions usually don't have any way to FTP files (or any automated way) and certainly won't allow cronjobs to sync the files up periodically.

[Slugger]

A few people have inquired about the local mirroring script I created, so here it is:

http://sagetv-addons.googlecode.com/...or_repo.groovy

You'll need to configure the variables at the top as required. Along with downloading the files from Sage's server, it also generates an updated XML file with the links adjusted. In case of disaster, simply replace your current SageTVPlugins.xml file with the repo.xml file this script generates. Once replaced, all the Sage hosted plugins will be downloaded from your local HTTP server.

I'm running this script daily from SJQv4. It is safe to run as frequent as you like - it will only download files if they don't exist in your local mirror or if the MD5 checksum has changed (i.e. a new version of a file has been submitted to SageTV).

[graywolf]

Originally Posted by tmiranda

How about something like DropBox or whatever Microsoft is calling their free web storage solution these days?

FYI - dropbox recently screwed up their access rules so that for a period (couple hours?) anyone was able to access anyones account.

http://nakedsecurity.sophos.com/2011...-in-as-anyone/

[will]

Originally Posted by Slugger

PM me if you're willing and able to host this repository. I'll probably just keep mirroring it locally for now and if the need arises then we'd need to copy the mirror somewhere and provide the new SageTVPlugins.xml file.

How much storage space is your mirroring taking up and how much monthly bandwidth do you anticipate using (best guess)? I have a few websites (hosted professionally) that may have enough excess bandwidth.

[graywolf]

Looks to be under 40 MB? At least for the Sage hosted plugins. Some are hosted elsewhere (ie. Diamond and others)


Bandwidth?? Depends upon how many people hit it.
Would need to hit it to install plugins, upgrade plugins, etc.

A grain of sand isn't much...but a whole bunch of grains of sand could be. Sahara Desert?

[whurlston]

Send me an email to this username at oshinetworks.com

I have the storage space and bandwidth to accommodate you.

[null0byte]

Hey, I've been a lurker here for a couple years but haven't posted much till now. I've got a host with about 1TB each of bandwidth and disk space just itching to have a use ( it's nearly empty right now, just hosting a small site for a friend). Do you need a second mirror? Unfortunately I'm not all that versed in how to set things up but I'm willing to learn and want to help out where I can. PM me if interested.

[Naylia]

I could repurposed a GoDaddy account that I have paid the next 3 years on if needed. Just PM me...

[SHS]

Look people there no need worry about this rigth now just wait a few months

[darcilicious]

Better safe than sorry.

Until a few days ago, I thought I had plenty of time to buy an HD300 too.

[jhkoenig]

If the earlier offers fall through, PM me.

[Naylia]

Originally Posted by SHS

Look people there no need worry about this rigth now just wait a few months

I think with the lack of information available, people feel better doing something/anything to prepare for the worst case. If you learn a lil about XMLTV and some folks put in place a backup plugin server, everyone will feel like there's a safety net. And if we're just gonna waste time reading Internet forums, then it's not like we can't spare the time

[hugenbdd]

Originally Posted by Slugger

A few people have inquired about the local mirroring script I created, so here it is:

http://sagetv-addons.googlecode.com/...or_repo.groovy

You'll need to configure the variables at the top as required. Along with downloading the files from Sage's server, it also generates an updated XML file with the links adjusted. In case of disaster, simply replace your current SageTVPlugins.xml file with the repo.xml file this script generates. Once replaced, all the Sage hosted plugins will be downloaded from your local HTTP server.

I'm running this script daily from SJQv4. It is safe to run as frequent as you like - it will only download files if they don't exist in your local mirror or if the MD5 checksum has changed (i.e. a new version of a file has been submitted to SageTV).

For those wanting to run this on windows, here are a few tips to get you up and started faster.

1.) Download groovy 1.8.0 http://groovy.codehaus.org/Download
2.) Make sure you set your java_home variable.
3.) Add the file "commons-codec-1.5.jar" to your groovy "libary" generally located here: C:\Program Files\Groovy\Groovy-1.8.0\lib
The file can be found here: http://commons.apache.org/codec/download_codec.cgi
4.) Change the location around line 25.
To: def localOutput = new File('YOURLOCATION.')
Example: def localOutput = new File('C:\\Users\\Dave\\Desktop\\SagePlugins')
*Notice the double \
5.) Change the "URL" to your own webserver. In this case loop back. 127.0.0.1
def targetUrl = 'http://127.0.0.1/repository'
6.) Run from the command line or groovy console.
C:\Users\Dave\Desktop>groovy mirror_repo.groovy


Hopefully this will be hosted in the near future and no one else will have to worry about downloading the plug-ins on their own.

Thanks Slugger for a great script!


Dave

[nyplayer]

Shouldn't the contents of this def localOutput = new File('C:\\Users\\Dave\\Desktop\\SagePlugins') be copied over to the webserver targetUrl = 'http://127.0.0.1/repository'

In My case I downloaded all the plugins to C:\nyplayer\SagePlugins... so I should copy them over to targetUrl = 'http://127.0.0.1/repository' ??????

Edit

Better yet where Do I create the repository folder in my Jetty webserver???? So that it can be accessed locally?

[KJake]

If this actually becomes necessary, I have a linux server with plenty of bandwidth and space. Notice the upload speed (aka how fast people can download from me)...

[Slugger]

Originally Posted by nyplayer

Shouldn't the contents of this def localOutput = new File('C:\\Users\\Dave\\Desktop\\SagePlugins') be copied over to the webserver targetUrl = 'http://127.0.0.1/repository'

In My case I downloaded all the plugins to C:\nyplayer\SagePlugins... so I should copy them over to targetUrl = 'http://127.0.0.1/repository' ??????

Edit

Better yet where Do I create the repository folder in my Jetty webserver???? So that it can be accessed locally?

The script assumes the localOutput is the web server folder (i.e. the web server is running on the host you're downloading from). For Jetty, change localOutput to:

def localOutput = new File('C:/Program Files/SageTV/SageTV/jetty/static/repository')

and

def targetUrl = 'http://192.168.2.5:8080/repository'

Change the IP/port in the URL as necessary for your Jetty installation.