Library: Remote APIs for SageTV

[stuckless]

Wiki | Download | Project Site | Http/Rest Guide

For another project I am continuing to work on, I ended up creating this Sage API Wrapper that also provides Remote Execution of Sage services (RPC) using either Java or Xml capabilities. You can download it here.

Features
* Remote/Local Java Calls to the Sage Services
* REST Services for all SageTV commands (Xml over Http)
* Returns complete sage objects and lists in either Xml or Json format
* Can return results in the SageXmlWriter format the Niel's Web Server users
* Publish your own remote services using server side javascript
* Access Media artifacts, such as posters, backgrounds, etc, (requires phoenix apis)

What can you do with this Api?
Using the Java API, you can write java code that runs outside the main sage server. ie, if you wanted to write a script that finds all matched tv shows and then deletes them, you can do that without needing to run that code inside SageTV

Using the Xml API, you can make requests from a browser and execute sage calls, or get back server information, using nothing more than javascript in a browser.

Documentation
You can read more about from the download site or the project site.

Use this Thread to discuss the library, etc.

[Slugger]

Just saw this before calling it a night so I haven't looked at the details/code yet, but my first question, before I forget, is, do the API calls return strongly typed objects/use strongly typed arguments much like Greg's API?

Your RPC example uses an Object[] where I might expect to see MediaFile[] or MediaFile.List (if using Greg's API as an example).

[stuckless]

The fundamental difference between this API and Greg's API is that this API does not return Strongly Typed objects. It doesn't create any wrappers or additional objects. It's simply a collection of Static Classes and Methods, as the original Javadoc portrays. If it did provide a Stongly typed set of object, then I'd really see no point in creating it, Since Greg's API does that already.

The only way to do Strongly Typed objects for Sage's API, is to do as Greg's API did, which is create an Object instance for each Sage object, because Sage does not export a usable type for their objects (they obfuscate). While typed objects makes for a better object structure, it does lead some object overhead (in this case), which I was trying to avoid for the purposes of RPC. Typed objects (in this case) would also require marhaling and unmarhalling even when used in embedded mode. The RPC api does mashall and unmarshall in RPC mode, but in embedded mode, it does not require it, which should make it slightly more efficient.

So, instead of using something like...

media.getTitle()

You'll do as the Sage Javadoc says....

MediaFileAPI.GetTitle(media)

[jreichen]

I got a chance to look over your source code. I like your use of the sagex.* package naming. I'm converting over to that myself.

Have you seen the Jetty plugin I created? Any number of HTTP-based applications can be deployed in there, so you might take a look at that as an option for the server side. It can provide security as well as allow multiple apps running on the same port. That said, I like your mini httpd for its simplicity and some people may want the option of using it if that's the only HTTP app they're using.

Another benefit of Jetty is it should also provide each app with its own class loader, although I haven't tested it. There have been problems in the past with different plugins using different versions of the same jar file. This should reduce those conflicts for apps deployed in Jetty.

Anyway, nice work on the RPC API

[jreichen]

There is a related RMI plugin, but it doesn't look it's gone anywhere in a few years and I don't even see a download. I thought it was worth linking to it from here, though.

[stuckless]

There is a related RMI plugin, but it doesn't look it's gone anywhere...

I thought about RMI... for about 10 seconds.... I've used it in the past, and I really don't like the complexity that it provides. As well, the RMI would have required me to create a ton of interfaces and object that represented the sage objects, which was what I was trying to avoid in the first place RMI is a good choice for enterprise applications, especially if you have a large development team and professional services

Have you seen the Jetty plugin I created? Any number of HTTP-based applications can be deployed in there...

I don't need to be sold on the virtues of Jetty I use it all the time, it's a great embedded web server. I did consider releasing it as a war file, but in the end, I resurrected an old micro server that I wrote many years ago. For this particular instance it works fine. The one challenge that I would have in hosting it inside a jetty container, is that I need to find the server and port the server is running on. For my micro server, that was easy, and I'm sure I can find a way to do it in jetty as well. What I may do, is provide a jetty adapter so that people could choose to host the RPC server inside a Jetty container. And, the microserver is not meant to be a replacement for a real server, it's just a simple http connection handler. I would think that most people would go the jetty route, just to take advantage of the real servlet capabilities.

I did seriously look as using jetty, and I also looked at the tiny web server that nielm uses... but in the end, both were overkill in terms of dependencies, and configuration. And to be honest, the reason I created the API was so that I could start embedding less into Sage and more into an external server process

Thanks for your feedback.

[babgvant]

Does the service expose a WSDL? Or something like it that could use to autogenerate a proxy class?

I'd like to integrate with a c# application, and it would save a ton of time...

[stuckless]

Originally Posted by babgvant

Does the service expose a WSDL? Or something like it that could use to autogenerate a proxy class?

I'd like to integrate with a c# application, and it would save a ton of time...

Short answer is no. The Rest API is not a web services api.

That being said, much the of the code in the api is autogenerated from the SageTV javadocs. The code currently parses the javadoc and creates a metadata for the SageAPI, from that, java apis, jsp tags, and http apis are generated. So, you could create a generator, and generate .cs files that proxy the the http calls.

[babgvant]

I'm having a hard time getting this installed using the Jetty method. I have Jetty running for the mobile interface, so it seems to be configured properly.

I have sagex.xml in C:\Program Files\SageTV\SageTV\jetty\contexts & sagex.api-6.4.8-11.jar in C:\Program Files\SageTV\SageTV\JARs

but http://localhost:8181/sagex/rpcXml/Global/GetOS gives me a 404.

Any suggestions?

Thanks.

[stuckless]

If you've previously installed the remote apis under nielm's server, then remove the servlet entry from his servlets.properties file before installing it into jetty.

Could you check your sagetv_0.txt log file (in the sagetv home directory) and see if it
contains the following entries.

Wed 12/24 15:39:50.885 Remote API Servlet initializing.
Wed 12/24 15:39:50.887 Sage Provider Implementation: sagex.remote.EmbeddedSageAPIProvider; sage://embedded
Wed 12/24 15:39:50.907 Sage Xml RPC Servlet Created.
Wed 12/24 15:39:50.908 Sage Java RPC Servlet Created.
Wed 12/24 15:39:50.909 Sage Json RPC Servlet Created.
Wed 12/24 15:39:50.911 Media Servlet Handler Created.
Wed 12/24 15:39:50.913 Sage Xbox Xml RPC Servlet Created.
Wed 12/24 15:39:50.914 Registered Handlers.
Wed 12/24 15:39:50.914 Configuring Remote Broadcast Services...
Wed 12/24 15:39:50.915 Properties not found: /opt/sagetv/server/sagex-api.properties; using defaults.
Wed 12/24 15:39:50.917 Sage Remote Api Info: Server: mediaserver; Rmi Port: 1098; Http Port: 8081
Wed 12/24 15:39:50.958 Starting Sage RMI Server...
Wed 12/24 15:39:51.069 Sage Java RMI Server ready
Wed 12/24 15:39:51.104 Annoucing Remote Sage API Server availability....
Wed 12/24 15:39:51.105 2008-12-24 15:39:51.625::INFO:  Deploy /opt/sagetv/server/jetty/contexts/sagex.xml -> org.mortbay.jetty.servlet.Context@324ff{/,null}

And I apologize in advance for the next question.... but I assume that since you are using "localhost", then are running the browser on the same machine on which the sagetv server is running?

Also, make sure that in your JARs directory, that you only have a 1 copy of the sagex.api jar file. If there multiple copies, with diferrent versions, then delete all other copies, except the latest version.

What's your java version?

[babgvant]

Closest thing I have in the sagetx_0.log is

Fri 12/26 22:27:20.355 Loading startup runnable:sagex.jetty.starter.Main
Fri 12/26 22:27:20.358 Loaded startup runnable:sagex.jetty.starter.Main

localhost was on the sage server.

only 1 of the sagex.api files is present (I had previously deleted *-9)

Version 6 Update 11.

Thanks

[bradvido]

Stuckless, is there any way for me to disable logging for your API completely. Using BMT or otherwise?

[stuckless]

Originally Posted by bradvido

Stuckless, is there any way for me to disable logging for your API completely. Using BMT or otherwise?

In bmt goto configuration -> logging -> log4j and select sagex-api, and then change the log level to "off" instead of "info", click save.

[stuckless]

Given that 7.1.8 has some new security apis... I thought I'd be proactive and create a new build BEFORE someone asks me

[bradvido]

Originally Posted by stuckless

In bmt goto configuration -> logging -> log4j and select sagex-api, and then change the log level to "off" instead of "info", click save.

Worked great, thanks

[Xsabre]

How can I disable authentication @ http://myserver:8080/sagex/api

I can access it but it requires the default authentication. How do you disable this?

Commenting out in realm.properties doesn't work.

SageTV 6 with latest sagex-api-7.1.9.2-sage6.zip

[Slugger]

Originally Posted by Xsabre

How can I disable authentication @ http://myserver:8080/sagex/api

I can access it but it requires the default authentication. How do you disable this?

Commenting out in realm.properties doesn't work.

SageTV 6 with latest sagex-api-7.1.9.2-sage6.zip

You need to comment out the authentication setup in the context file for sagex in \SageTV\jetty\contexts\

Note, however, that doing so opens your sagex app up to anyone to connect to. If this server is accessible via the internet then anyone who knows your IP/hostname and how the Sage API works can easily destroy your Sage system. Any script kiddie with a free 10 mins could write a shell script with wget to simply delete every recording off your system and there's no turning back unless you have backups of all your content. I really, really, really discourage people from disabling the authentication in sagex, unless the server is not accessible from the internet and you trust your firewall.

[HokiePerogi]

I've been running the following command for weeks on a nightly basis to build an external file for myself. However, it seems to have started to fail sometime between tonight and a few weeks ago. I am unable to get the full result set from the following sagex request:

http://sagepc:8009/sagex/api?c=GetMediaFiles&1=VDBL&start=0&size=50

Google Chrome responds with:

This page contains the following errors:

error on line 180 at column 16: Specification mandate value for attribute Voice
Below is a rendering of the page up to the first error.

Full XML attached as 2.txt


And IE 8 responds with:

- <Result size="50">
- <MediaFile>
- <MediaFileFormatDescription>
- <![CDATA[ MPEG2-TS[H.264 16:9 720p@60fps, Dolby Digital/384Kbps@48kHz Stereo]
  ]]> 
  </MediaFileFormatDescription>
  <IsLibraryFile>true</IsLibraryFile> 
  <IsCompleteRecording>true</IsCompleteRecording> 
  <IsPictureFile>false</IsPictureFile> 
- <SegmentFiles size="1">
- <File>
- <![CDATA[ E:\StrangerThanFiction-1844288-0.ts
  ]]> 
  </File>
  </SegmentFiles>
- <MediaTitle>
- <![CDATA[ Stranger Than Fiction
  ]]> 
  </MediaTitle>
- <ParentDirectory>
- <![CDATA[ E:\
  ]]> 
  </ParentDirectory>
  <IsThumbnailLoaded>false</IsThumbnailLoaded> 
  <FileDuration>8981301</FileDuration> 
  <FileStartTime>1300406422464</FileStartTime> 
  <FileEndTime>1300415403765</FileEndTime> 
  <NumberOfSegments>1</NumberOfSegments> 
  <IsMediaFileObject>true</IsMediaFileObject> 
  <AlbumForFile /> 
- <MediaFileEncoding>
- <![CDATA[ 
  ]]> 
  </MediaFileEncoding>
- <Airing>
- <RecordingName>
- <![CDATA[ 
  ]]> 
  </RecordingName>
- <AiringChannelName>
- <![CDATA[ 
  ]]> 
  </AiringChannelName>
  <AiringDuration>9000000</AiringDuration> 
  <AiringStartTime>1300406400000</AiringStartTime> 
  <AiringEndTime>1300415400000</AiringEndTime> 
- <AiringRatings size="5">
- <Item>
- <![CDATA[ TVPG
  ]]> 
  </Item>
- <Item>
- <![CDATA[ PG-13
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Nudity
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Language
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Adult Situations
  ]]> 
  </Item>
  </AiringRatings>
  <ScheduleStartTime>1300406400000</ScheduleStartTime> 
  <ScheduleEndTime>1300415400000</ScheduleEndTime> 
  <ScheduleDuration>9000000</ScheduleDuration> 
- <AiringTitle>
- <![CDATA[ Stranger Than Fiction
  ]]> 
  </AiringTitle>
  <WatchedDuration>0</WatchedDuration> 
  <WatchedStartTime>0</WatchedStartTime> 
  <WatchedEndTime>0</WatchedEndTime> 
  <LatestWatchedTime>1300406400000</LatestWatchedTime> 
  <IsWatchedCompletely>false</IsWatchedCompletely> 
  <IsManualRecord>false</IsManualRecord> 
  <TrackNumber>0</TrackNumber> 
- <RecordingQuality>
- <![CDATA[ 
  ]]> 
  </RecordingQuality>
  <IsAiringObject>true</IsAiringObject> 
- <ParentalRating>
- <![CDATA[ TVPG
  ]]> 
  </ParentalRating>
  <AiringPartNumber>1</AiringPartNumber> 
  <AiringTotalParts>1</AiringTotalParts> 
- <ScheduleRecordingRecurrence>
- <![CDATA[ 
  ]]> 
  </ScheduleRecordingRecurrence>
- <Channel>
- <ChannelName>
- <![CDATA[ 
  ]]> 
  </ChannelName>
  <ChannelNetwork /> 
- <ChannelNumber>
- <![CDATA[ 
  ]]> 
  </ChannelNumber>
  <IsChannelViewable>false</IsChannelViewable> 
  <IsChannelObject>true</IsChannelObject> 
  <ChannelLogoCount>0</ChannelLogoCount> 
  <StationID>0</StationID> 
- <ChannelDescription>
- <![CDATA[ 
  ]]> 
  </ChannelDescription>
  </Channel>
- <Show>
- <ShowCategory>
- <![CDATA[ Movie
  ]]> 
  </ShowCategory>
  <IsShowEPGDataUnique>true</IsShowEPGDataUnique> 
- <ShowSubCategory>
- <![CDATA[ Comedy
  ]]> 
  </ShowSubCategory>
- <ShowDescription>
- <![CDATA[ A mentally unstable IRS auditor (Will Ferrell) hears an author's (Emma Thompson) voice in his head and discovers that he is the ill-fated protagonist of her latest work. While a book-company employee (Queen Latifah) tries to cure the author's case of writer's block, the auditor and a professor (Dustin Hoffman) set out to find the woman and make her change her story.
  ]]> 
  </ShowDescription>
- <ShowEpisode>
- <![CDATA[ 
  ]]> 
  </ShowEpisode>
  <ShowDuration>6300000</ShowDuration> 
- <ShowExternalID>
- <![CDATA[ MV1775540000
  ]]> 
  </ShowExternalID>
- <PeopleInShow>
- <![CDATA[ Will Ferrell, Maggie Gyllenhaal, Dustin Hoffman, Queen Latifah, Emma Thompson, Tony Hale, Tom Hulce, Linda Hunt, Eli Goodman, Denise Hughes, Jason Burke, Linara Washington, Marc Forster, Zach Helm, Lindsay Doran, Joseph Drake, Nathan Kahane, Eric Kopeloff
  ]]> 
  </PeopleInShow>
- <PeopleListInShow size="18">
- <Item>
- <![CDATA[ Will Ferrell
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Maggie Gyllenhaal
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Dustin Hoffman
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Queen Latifah
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Emma Thompson
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Tony Hale
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Tom Hulce
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Linda Hunt
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Eli Goodman
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Denise Hughes
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Jason Burke
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Linara Washington
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Marc Forster
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Zach Helm
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Lindsay Doran
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Joseph Drake
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Nathan Kahane
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Eric Kopeloff
  ]]> 
  </Item>
  </PeopleListInShow>
- <RolesInShow size="18">
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Actor
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Director
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Writer
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Producer
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Executive Producer
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Executive Producer
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Executive Producer
  ]]> 
  </Item>
  </RolesInShow>
- <ShowLanguage>
- <![CDATA[ English
  ]]> 
  </ShowLanguage>
  <ShowSeasonNumber>0</ShowSeasonNumber> 
  <ShowEpisodeNumber>0</ShowEpisodeNumber> 
- <PeopleAndCharacterListInShow size="18">
- <Item>
- <![CDATA[ Will Ferrell
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Maggie Gyllenhaal
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Dustin Hoffman
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Queen Latifah
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Emma Thompson
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Tony Hale
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Tom Hulce
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Linda Hunt
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Eli Goodman
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Denise Hughes
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Jason Burke
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Linara Washington
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Marc Forster
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Zach Helm
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Lindsay Doran
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Joseph Drake
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Nathan Kahane
  ]]> 
  </Item>
- <Item>
- <![CDATA[ Eric Kopeloff
  ]]> 
  </Item>
  </PeopleAndCharacterListInShow>
- <ShowCategoriesString>
- <![CDATA[ Movie / Comedy / Fantasy
  ]]> 
  </<The XML page cannot be displayed 
Cannot view XML input using XSL style sheet. Please correct the error and then click the Refresh button, or try again later. 


--------------------------------------------------------------------------------

A name contained an invalid character. Error processing resource 'http://sagepc:8009/sagex/api?c=GetMediaFiles&1=VDBL&start...
 
    <Guest Voice><![CDATA[]]></Guest Voice>
----------------^
/SPAN>ShowCategoriesString>
- <ShowCategoriesList size="3">
<DIV

When I run my vbscript code to get the results of this query and then parse the dom (as a MSXML2.DOMDocument), the parsing is failing to find the elements:

ShowEpisode
MediaFileID
MediaTitle

I'm thinking because there is something wrong with returned XML, or some recent windows update changed the behavior somehow.

Any thoughts or ideas what could be wrong?


Thanks!

[stuckless]

@HokiePerogi - 7.1.9.11 should fix your issue.

One a side note, I noticed something in the Sagex rest apis, that I didn't realize that I had implemented, which is filtering of fields in a request. Originally this filtering code existed in the JSON serializer but not in the xml serializer, so now it's there in both.

The /sagex/api help page has been updated to reflect the 'filter' parameter.

I think several people have asked if they could filter the fields being returns, and I've always replied 'no'...

A use case for this, is that when you call, GetMediaFiles, you might only want to, initially, send back the MediaFileID, and Title, and not all off the 100s of other fields. So, to do that, simply append,

filter=MediaFileID|MediaTitle

to the query and ONLY fields matching those names will be returned.

[HokiePerogi]

Originally Posted by stuckless

@HokiePerogi - 7.1.9.11 should fix your issue.

One a side note, I noticed something in the Sagex rest apis, that I didn't realize that I had implemented, which is filtering of fields in a request. Originally this filtering code existed in the JSON serializer but not in the xml serializer, so now it's there in both.

The /sagex/api help page has been updated to reflect the 'filter' parameter.

I think several people have asked if they could filter the fields being returns, and I've always replied 'no'...

A use case for this, is that when you call, GetMediaFiles, you might only want to, initially, send back the MediaFileID, and Title, and not all off the 100s of other fields. So, to do that, simply append,

filter=MediaFileID|MediaTitle

to the query and ONLY fields matching those names will be returned.

Awesome. Thanks for the quick turnaround on this fix! I can confirm that my issue is resolved.

I also tried out the new filter parameter which looks very promising, especially for my needs. Using your example worked perfectly. However, for some reason, when I add ShowEpisode to the filter list, or just use ShowEpisode by itself in the filter list, I do not get my expected results:

http://sagepc:8009/sagex/api?c=GetMediaFiles&1=TVDBL&start=0&size=10&filter=ShowEpisode

results in:

<Result size="10">
<MediaFile/>
<MediaFile/>
<MediaFile/>
<MediaFile/>
<MediaFile/>
<MediaFile/>
<MediaFile/>
<MediaFile/>
<MediaFile/>
<MediaFile/>
</Result>

Is the filter functionality available on all metadata, or only select fields?

Thanks again @stuckless for the continued work on everything SageTV.