Utility: mediaScraper - metadata made easy!

[evilpenguin]

Just by looking at the file name, its pretty much impossible for mediaScraper to track down the metadata for Sage Recordings. Sage just throws away too much important formatting information for the tool to be able to track down the metadata.

The only way to do it is to cross check the Sage Recording against the info in from webserver plug-in to pull out the actual Show title and original airing date. Using that I should be able to scrape the proper information, but I haven't bothered to implement it yet.

[wayner]

Couldn't you use calls to the Sage API to return the Show name, episode name, etc?

I don't know if you are familiar with SJQ (written by Slugger) but it sets up Tests based on characteristics of Media Objects such as: Title, Episode, Description, OrigAirDate, Category, Person, etc. See here for details: http://code.google.com/p/sagetv-addo...i/SJQ2TestList

[evilpenguin]

Originally Posted by wayner

Couldn't you use calls to the Sage API to return the Show name, episode name, etc?

I don't know if you are familiar with SJQ (written by Slugger) but it sets up Tests based on characteristics of Media Objects such as: Title, Episode, Description, OrigAirDate, Category, Person, etc. See here for details: http://code.google.com/p/sagetv-addo...i/SJQ2TestList

SJQ is writtin in Java so it can use Sage's API pretty easily. I use Perl getting it working would be much more complicated, but I can take a look to see if its doable.

[stuckless]

EP... The Remote APIs that I released, has an Xml and Json component. I export every Sage API Call over Http, so you should be able to deal with that fairly easily in any language.

You can take a quick look at the Http guide to see if it meets your needs.

[evilpenguin]

Originally Posted by stuckless

EP... The Remote APIs that I released, has an Xml and Json component. I export every Sage API Call over Http, so you should be able to deal with that fairly easily in any language.

You can take a quick look at the Http guide to see if it meets your needs.

Yeah, that's one of the things I was going to look into.

But I think I might have found a native perl method for calling java straight out of JAR files, so I'll give that a shot first.

[stuckless]

FYI, Even if you can call Java directly from perl, you'll still need to have your perl vm in the same java vm as the SageTV process or else it won't communicate. (ie, you java calls will be like starting up a separate instance of SageTV, which may be problematic, if it works at all)

[Peter_h]

Hey EP,

Great tool!

I have a problem though. I've ran this on about 20 files. 3 of them will not import into the XML database. I tried manually importing through the webserver and it said that "Airing duration "" is not valid".

So I checked the XML files for all 3 and sure enough, there is nothing in the airing duration field.

I checked the scraper log and i see this.

 + Getting Video Info: (\\Deepthought\SageTV\TV\Heroes\Season 3\Heroes S03E02 - The Butterfly Effect.avi)(358393.736328125)
    ! Error in ffmpeg log: (Unknown format)

I don't know if that is relevant.

I am attaching the scraper log as well as the .xml file. Can you please take a look?

Thanks,
Peter

[evilpenguin]

Yep, that ffmpeg line is the issue. I use that to determine the duration of the video and if it fails I can't do fill in the duration line in the XML file.

[Peter_h]

Originally Posted by evilpenguin

Yep, that ffmpeg line is the issue. I use that to determine the duration of the video and if it fails I can't do fill in the duration line in the XML file.

Is there a fix for this? Or is there a way I can manually determine the duration. I noticed that it is just in seconds.

[Peter_h]

Originally Posted by Peter_h

Is there a fix for this? Or is there a way I can manually determine the duration. I noticed that it is just in seconds.

Alright problem solved.

The files don't even play so they must be corrupt. I tried to play them in MPC and it says can not render file. That would explain why the ffmpeg line failed.

Sorry about that.

[evilpenguin]

Originally Posted by Peter_h

Alright problem solved.

The files don't even play so they must be corrupt. I tried to play them in MPC and it says can not render file. That would explain why the ffmpeg line failed.

Sorry about that.

Yeah, that's what I figured. 99.9% of the time when ffmpeg can't detect the format the video is just bad.

[MattJackson86]

Hey. So I just gave this utility a shot and WOW! It looks like at long last I can replace the tedious task of using DVD profiler for me ripped dvds. I did notice though that when using this on my VIDEO_TS directorys it is generating a "Poster.jpg" and not a "folder.jpg" so thumbnails are not showing up. Am I missing something? It also seems the metadata from the .my files is not loading. I have changed sagemc to use them as you specified.

My folder struct is like so:

M:\DVD\300\VIDEO_TS
M:\DVD\300.my (does not load)
M:\DVD\300\background.jpg (works great)
M:\DVD\300\poster.jpg (works if I rename to folder.jpg)


Awesome job. This kind of app was long overdue.

[jaminben]

Originally Posted by MattJackson86

I have changed sagemx to use them as you specified.

M:\DVD\300.my (does not load)

Have you enabled the use of .my files in SageMC settings,

Settings > Enhancements > Use .my files.

[MattJackson86]

Yes I have.

FYI I am also using the "Movie Info Screen V1.0"

[jaminben]

Originally Posted by MattJackson86

Yes I have.

FYI I am also using the "Movie Info Screen V1.0"

Version 1.0 is old and outdated. I cannot remember coding any .my file support for it so that maybe your problem. Even the newer version doesn't support .my files fully as someone else pointed out on another thread, but my personel version does support it fully I'll update the download soon*ish* to take full advantage of .my files. If you do upgrade to the latest version then you may also need to update your version of SageMC as well.

[MattJackson86]

Sorry I should have been more specific. I am running the latest version of Sage, SageMC, Movie Screen 1.0a_36. I downloaded them all and did a clean install last night. Still no luck on the .my support though...

[jaminben]

Originally Posted by MattJackson86

Sorry I should have been more specific. I am running the latest version of Sage, SageMC, Movie Screen 1.0a_36. I downloaded them all and did a clean install last night. Still no luck on the .my support though...

Strange, mine works ok. All my .my files are located within the same folder as the media file its associated with. Perhaps someone else who knows more about .my files and can help as I'm out of ideas.

[MattJackson86]

Yea I have been over settings multiple times and it all looks correct. I must just be missing something. Thank for your help (and the awesome movie screen changes). Not having touched my sage install in 2-3 years as its been solid, its nice to see so many cool stuff thats come out and improved it a lot.

[Peter_h]

Hey EP.

I have a feature request. Would it be possible to add a switch to automatically create thewhatEverYourFileNameIs.override file?

Here is an example I ran into yesterday where this would be needed.

I have the defaults set to get all of the backgrounds, etc, rename the file, /genxml, and import into the webserver.

I downloaded an episode of heroes. "heroes.S03.E03.xvid.hd.avi". This went out and did it's thing and was then renamed to "Heroes - One of us, One of them".

This did not import into the webserver. As soon as I saw the filename, I knew it was because of the "," so I renamed all the files and went into the xml and deleted the ",". Then I went to the webserver and manually added the xml again and it worked perfectly.

Now I know I can add a "Heroes - One of us One of them.overide" with "Heroes - One of us, One of them" inside and it would work. However, I don't ever know the episode names and they are rarely included when downloading. The only reason I knew there was a problem was because I was missing a show.

In this instance I would have to look up the file and rename before your tool runs. Or run the tool and then check, rename or add the .overide and run it again.

It would be awesome if you could add a switch that would make the scrapper automatically create a .overide file when these problem characters come up.

The people that will benefit most are the ones that use the webserver import feature.

Thanks,
Peter

Edit: Post #277 and #278 reflect the same thing.

[wayner]

It appears that MediaScraper doesn't like dvr-ms files as it didn't do anything when I dropped such a file onto MediaScraper. When I changed the extension to .mpg it processed the file properly. Any reason why this would be - do you test for file extension first?