Library: Remote APIs for SageTV

[stuckless]

Wiki | Download | Project Site | Http/Rest Guide

For another project I am continuing to work on, I ended up creating this Sage API Wrapper that also provides Remote Execution of Sage services (RPC) using either Java or Xml capabilities. You can download it here.

Features
* Remote/Local Java Calls to the Sage Services
* REST Services for all SageTV commands (Xml over Http)
* Returns complete sage objects and lists in either Xml or Json format
* Can return results in the SageXmlWriter format the Niel's Web Server users
* Publish your own remote services using server side javascript
* Access Media artifacts, such as posters, backgrounds, etc, (requires phoenix apis)

What can you do with this Api?
Using the Java API, you can write java code that runs outside the main sage server. ie, if you wanted to write a script that finds all matched tv shows and then deletes them, you can do that without needing to run that code inside SageTV

Using the Xml API, you can make requests from a browser and execute sage calls, or get back server information, using nothing more than javascript in a browser.

Documentation
You can read more about from the download site or the project site.

Use this Thread to discuss the library, etc.

[Slugger]

Just saw this before calling it a night so I haven't looked at the details/code yet, but my first question, before I forget, is, do the API calls return strongly typed objects/use strongly typed arguments much like Greg's API?

Your RPC example uses an Object[] where I might expect to see MediaFile[] or MediaFile.List (if using Greg's API as an example).

[stuckless]

The fundamental difference between this API and Greg's API is that this API does not return Strongly Typed objects. It doesn't create any wrappers or additional objects. It's simply a collection of Static Classes and Methods, as the original Javadoc portrays. If it did provide a Stongly typed set of object, then I'd really see no point in creating it, Since Greg's API does that already.

The only way to do Strongly Typed objects for Sage's API, is to do as Greg's API did, which is create an Object instance for each Sage object, because Sage does not export a usable type for their objects (they obfuscate). While typed objects makes for a better object structure, it does lead some object overhead (in this case), which I was trying to avoid for the purposes of RPC. Typed objects (in this case) would also require marhaling and unmarhalling even when used in embedded mode. The RPC api does mashall and unmarshall in RPC mode, but in embedded mode, it does not require it, which should make it slightly more efficient.

So, instead of using something like...

media.getTitle()

You'll do as the Sage Javadoc says....

MediaFileAPI.GetTitle(media)

[jreichen]

I got a chance to look over your source code. I like your use of the sagex.* package naming. I'm converting over to that myself.

Have you seen the Jetty plugin I created? Any number of HTTP-based applications can be deployed in there, so you might take a look at that as an option for the server side. It can provide security as well as allow multiple apps running on the same port. That said, I like your mini httpd for its simplicity and some people may want the option of using it if that's the only HTTP app they're using.

Another benefit of Jetty is it should also provide each app with its own class loader, although I haven't tested it. There have been problems in the past with different plugins using different versions of the same jar file. This should reduce those conflicts for apps deployed in Jetty.

Anyway, nice work on the RPC API

[jreichen]

There is a related RMI plugin, but it doesn't look it's gone anywhere in a few years and I don't even see a download. I thought it was worth linking to it from here, though.

[stuckless]

There is a related RMI plugin, but it doesn't look it's gone anywhere...

I thought about RMI... for about 10 seconds.... I've used it in the past, and I really don't like the complexity that it provides. As well, the RMI would have required me to create a ton of interfaces and object that represented the sage objects, which was what I was trying to avoid in the first place RMI is a good choice for enterprise applications, especially if you have a large development team and professional services

Have you seen the Jetty plugin I created? Any number of HTTP-based applications can be deployed in there...

I don't need to be sold on the virtues of Jetty I use it all the time, it's a great embedded web server. I did consider releasing it as a war file, but in the end, I resurrected an old micro server that I wrote many years ago. For this particular instance it works fine. The one challenge that I would have in hosting it inside a jetty container, is that I need to find the server and port the server is running on. For my micro server, that was easy, and I'm sure I can find a way to do it in jetty as well. What I may do, is provide a jetty adapter so that people could choose to host the RPC server inside a Jetty container. And, the microserver is not meant to be a replacement for a real server, it's just a simple http connection handler. I would think that most people would go the jetty route, just to take advantage of the real servlet capabilities.

I did seriously look as using jetty, and I also looked at the tiny web server that nielm uses... but in the end, both were overkill in terms of dependencies, and configuration. And to be honest, the reason I created the API was so that I could start embedding less into Sage and more into an external server process

Thanks for your feedback.

[cslatt]

I see on the project page that your end goal is to export an XML web service - just wanted to mark myself down as looking forward to that. As a .NET programmer who has just never managed to find the time to get into Java, I'd love to finally have an easy way to access the Sage API.

[stuckless]

I see on the project page that your end goal is to export an XML web service

I may actually have a release this weekend. I've finished the API, but there are some minor things that i need to work out, mainly related to argument passing and signature matching.... The Java RPC api handles those things pretty easily, but when every argument that comes in the url is a string, it's a little harder to do signature matching

[stuckless]

I just released the Xml over Http api for SageTV... See the Wiki for more information on how to make Sage Http calls. Checkout the Xml Rpc Guide as well.

The Xml Rpc API has not been well tested. So if you encounter issues, then please add bugs to the project bug page, or post a note in this thread.

The xml rpc is basically a set of rest service that return xml responses. The complete Sage API is exposed as Rest services, but not all aspects of the API lends itself well to rpc services, so don't expect to be able to do everything over http. But, if all you need is to do things things like refresh your media library, so see what tuners you have installed, then you can do calls like this...

http://sageserver:8080/sagex/rpcXml/CaptureDeviceAPI/GetCaptureDevices

and you'll back a response like...

<response>
  <header>
  </header>
  <body>
    <array size="2">
      <value index="0">adapter0</value>
      <value index="1">video0</value>
    </array>
  </body>
</response>

Please read ALL the wiki before using the API, since crafting an incorrect url will just lead to confusion and frustration...

Good luck, and if you have issue, then I'll try my best to help.

[stuckless]

I just released a new version of the Sage RPC API. This relase now uses Nielm's webserver instead of it's own micro server. Please refere to the wiki for installation instructions.

I decided to follow jreichen's advice and use an existing server. Nielm's Web Server was easy to integrate with, just add an entry to the servlets.properties. I'm currently looking at providing an integration with Jreichen's Jetty Plugin as well.

This new release also adds thumbnail support via a servlet call. ie, you can request the thumbnail, as an image url, for a given media file entry.

[jreichen]

On the Jetty plugin wiki, I briefly mention the BeanShell servlet. It accepts BeanShell code (basically Java) and interprets it, so you could write a script to do several operations at once and submit it to the supplied servlet, or a custom servlet, possibly via RPC wrappers. It may also be possible to do the same with other JVM-compatible scripting languages (e.g. Groovy).

That would be one more possibility for remotely executing SageTV code.

I attached an example zip that can be extracted to the SageTV folder if you already have the Jetty plugin installed.

[bcjenkins]

I am thinking about writing an iPhone application leveraging the REST services aspect. I have a few questions about the servlet available.

1) Does it support authentication?
2) Does it support SSL?
3) Is it a capable streaming server?
4) Can it be proxied through a server (Apache/Lighttpd) to enable support of the above? (When proxying nielm's webserver through Apache, IIRC there are issues with streaming files esp. when SSL is used)

Thanks,

B

[stuckless]

Originally Posted by bcjenkins

I am thinking about writing an iPhone application leveraging the REST services aspect. I have a few questions about the servlet available.

That's interesting... because I'll probably be doing something similar in next couple of months. I don't have an iphone, but I ordered a couple of ipod touch units for my kids. My goal is to selectively take some recorded shows and movies, and transcode them for the ipod. My thinking was that I'd run a nightly cron to transcode certain recorded shows.

1) Does it support authentication?

The rest APIs should support whatever authentication is in the running container. I haven't done anything special to handle authentication.

2) Does it support SSL?

Again, similar to question 1, the rest servlet is simply a servlet, so if you run it within a ssl container, then you should be fine.

3) Is it a capable streaming server?

It's capable of streaming, but it's not a capable streaming server The /sagex/mediafile/ service simply finds the media file and sends back its contents as a stream. It does not perform any on the fly transcoding, etc, which is why, my intention would be to transcode selected files on a nightly basis, using a cron. My current server can't transcode HD content on the fly, so I didn't invest much time into the streaming portion.

Just a note about a streaming and ssl. If you are thinking of using ssl, I would still NOT stream media over ssl. Encrypting a mediafile would be intense, and I can't immagine that you'd need/want to do it. So I would use ssl for authentication, get, but stream all media over http. (personal oppinion)

4) Can it be proxied through a server (Apache/Lighttpd) to enable support of the above? (When proxying nielm's webserver through Apache, IIRC there are issues with streaming files esp. when SSL is used)

I would think that if you used jreichen's jetty plugin, you could enable the ajp connector (I think that jetty supports this), and if so, then you can definately use apache as a frontend.

If you are simply looking to do url forwarding/proxing, then I would think it would work ok for most calls, but the streaming apis may not work... I haven't tested this for myself.... since in my testing, I'm just running inside nielm's server and accessing it directly.

[jreichen]

Originally Posted by stuckless

Again, similar to question 1, the rest servlet is simply a servlet, so if you run it within a ssl container, then you should be fine.

Some devices may require SSL connections to use port 443, and may not allow unsigned certificates. I think it's probably the exception rather than the rule and those restrictions will disappear as devices become more mature.

Originally Posted by stuckless

Just a note about a streaming and ssl. If you are thinking of using ssl, I would still NOT stream media over ssl. Encrypting a mediafile would be intense, and I can't immagine that you'd need/want to do it. So I would use ssl for authentication, get, but stream all media over http. (personal oppinion)

One of the advantages of Jetty is that it can serve SSL and non-SSL connections at the same time. The UI could connect to the SSL port for web pages and RPC calls, and to the non-SSL port for streaming.

Originally Posted by stuckless

I would think that if you used jreichen's jetty plugin, you could enable the ajp connector (I think that jetty supports this), and if so, then you can definately use apache as a frontend.

If you are simply looking to do url forwarding/proxing, then I would think it would work ok for most calls, but the streaming apis may not work... I haven't tested this for myself.... since in my testing, I'm just running inside nielm's server and accessing it directly.

Jetty has built-in AJP connectors but I haven't tried them[1][2]. It also has support for modifying HTTP headers to support reverse proxies.

There is also a thread about reverse proxy and the web servers. I think jhh had pretty good luck, but his Apache was on the same machine as Sage. I tried it with Apache on a different machine and couldn't quite get it working.

[jreichen]

Originally Posted by bcjenkins

I am thinking about writing an iPhone application leveraging the REST services aspect....

One of the speakers at the Denver Java User's Group in November is presenting how to write iPhone apps that work with Java web services. I talked to him today and it sounds interesting. I'll be happy to report back anything you might find helpful.

[shadeblue.com]

Hi Sean,

I have been playing with your SageTVApi library and testing a few concepts that I would like to develop further.

I ran into one problem, and I'm not sure if it is a problem in the library or if I am not using the API correctly. Basically any methods I call attempting to get a Widget object using the WidgetApi all fail.

Here is one simple example:

System.out.println("Widget: " + WidgetAPI.GetCurrentSTVFile());

Calling this method returns:

Widget: null

Any ideas or pointers? Ultimately I am trying to get a menu widget and execute it to display the menu on screen. i.e.:

Object widget = sagex.api.WidgetAPI.GetWidgetSymbol("CDSWT-46001");
sagex.api.WidgetAPI.ExecuteWidgetChain(widget);

PS.. methods in other parts of the API are working from my client ... such as:
sagex.api.Global.GetUIContextNames()
I am using your API ver: "sagex.api-6.4.8-1.jar" in under the remoting option (running from separate JVM on same computer as Sage)



Thanks!

[stuckless]

I have put some thought into how to transparently handle the "context" problem. Initially, I didn't need to use the context, so it got put on the back burner, but I'll spend some time and try to come up with something.

My initial thoughts are along these lines....
1. for each generated method, generate a second method that also can accept a context parameter, so WidgetAPI would include a method ExecuteWIdgetChain(widget) and ExecuteWidgetChain(widget, context).

2. add a theadlocal variable to generated class, so that you can set the context before executing the method. So, when you call ExecuteWidgetChain, you'd have to set the context first, so something like this...

sagex.api.WidgetAPI.SetContext(context);
sagex.api.WidgetAPI.ExecuteWidgetChain(widget);
sagex.api.WidgetAPI.SetContext(null);

The issue I see with the ThreadLocal approach is that I don't know how Sage handles it's UI code on the server. ie, if you were to run this code within sage, does sage execute each ui in it's own thread, or do all threads share the same thread....

3. Create a static list of "context" enabled fields and modify the generator to ONLY create context aware methods for methods that can accept a context...

Any thoughts?

[GKusnick]

This is where wrapper objects come in handy, because you can wrap the context up once when you instantiate the API object and not have to worry about it again. I understand your reasons for wanting to stick with static methods only, but the price you pay for that is pushing the burden of keeping track of instance data (like UI context) back onto the caller.

And by the way, shadeblue, you want FindWidgetBySymbol instead of GetWidgetSymbol, and LaunchMenuWidget instead of ExecuteWidgetChain.

[stuckless]

This is where wrapper objects come in handy, because you can wrap the context up once when you instantiate the API object and not have to worry about it again

I totally agree.... wrapper objects are definately better when you need to store instance data.... (and I encourage everyone to use your API if their intention is to embed their code inside of sage tv process)

So I guess I could generate a set of wrapper objects to fix the problem

[shadeblue.com]

Originally Posted by GKusnick

This is where wrapper objects come in handy, because you can wrap the context up once when you instantiate the API object and not have to worry about it again. I understand your reasons for wanting to stick with static methods only, but the price you pay for that is pushing the burden of keeping track of instance data (like UI context) back onto the caller.

And by the way, shadeblue, you want FindWidgetBySymbol instead of GetWidgetSymbol, and LaunchMenuWidget instead of ExecuteWidgetChain.

Hi GKusnick,

Thank you for the correction. I knew the 'FindWidgetBySymbol' was correct, I must have copied it wrong when I was creating the forum post. I must have missed 'LaunchMenuWidget', I did not know that was the correct call.

Again, Thank You!
shadeBlue