STV Import: Read & Write Metadata files for Default STV 5/6 (v2.2.9 18/Feb./2010)

[JREkiwi]

This module provides the ability to display additional information in "view video/recording detail" for Imported videos and DVD rips (by reading and creating .my files) and for downloading IMDB coverart on the Default STV. Also it can create .properties file and save IMDB data directly to the SageTV database (wiz.bin). Using code stolen from a variety of source, this import provides function previously seen in the Germsage and Cayars STV's. Thanks to flachbar for his automatic creation and cleanup function and his import library.

Features of this version are:
- Works on version 5 & 6
- Reads .my files. Can be enabled/disabled
- Can read .my files from a global directory
- Automatic creation of .my and/or .properties files for all recordings. Can be enabled/disabled but requires restart to activate.
- Automatic cleanup of orphan .my files. Can be enabled/disabled but requires restart to activate.
- Create .my and/or .properties files for individual recording. Can be enabled/disabled
- Uses the same properties as SageMC so switching between the Default STV and SageMC will keep the same common functions enabled/disabled.
- Ability to create .my and/or .properties file from any past or future airing. Saves the files into the path specified in Options/Customize. (Defaults to recordings directory)
- Bulk .my and/or .properties file creation. Creates files for all past/future Airings for a title in the Sage database.
- Save data from IMDB to .my and/or properties file
- Optional Download of IMDB image
- Cleans up old .my Reading import if installed
- Cleans up old .my RW import if installed
Version 6 only features
- Save data from IMDB to wiz.bin
- Edit metadata in wiz.bin
- Import data from .my file into wiz.bin

Installation is a 3/4 step process.
Step 1. Unzip into SageTV maintaining the directories. (The df_sageutils.jar from the zip file needs to be in the JARS directory)
Step 2. Restart Sage so it will pick up the jar.
Step 3. Load the IMDB Search Interface import (if you want to save IMDB data)
Step 4. Load the RW My Files import

Prerequisites: For saving IMDB data Nielm's IMDB Search Interface must be installed before this import.
If using TiKi's FanArt 2.3 or higher install TiKi's FanArt first.
If using Phoenix FanArt for Default STV, install Phoenix FanArt for Default STV first.

The current version is RW Metadata 2.2.9 (18 February 2010) download from here.

_______________________________________________________________

Configuring Options
The majority of the configurable options are via
Setup -> Detailed Setup -> Customize -> Metadata Options

The Metadata Options Panel, displays the version of the import


_______________________________________________________________

Automatic Create and Delete Options Configure



Create Metadata files for all TV Airings Yes/No
When Yes, will create .my and/or .properties files for all recordings

Create Metadata File templates for Imported Files Yes/No
When Yes, will create .my and/or .properties files for all Imported Files, however as the files are created using data from the SageTV database, it's value is to create templates for imported videos.

Overwrite Existing Metadata Files Yes/No
When Yes, the Automatic Creation will overwrite any existing .my and/or .properties files
When No, it wont.

Do Full Media Scan Hourly/Daily
The standard scan for files only creates files for Recordings that have completed since the last scan. The Full Media scan will create files for all Videos that do not have files (handy when doing compression and the files change, but the recording complete time doesn't)

Delete Orphaned Metadata Files Yes/No
When Yes, will delete any .my or .properties files in the TV Recordings Directories that do not have an associated media file

Delete Orphaned Metadata Files for Imported Files Yes/No
When Yes, will delete any .my or .properties files in the Imported Videos Directories that do not have an associated media file
_______________________________________________________________

Metadata File Types Configure



Make .properties Files Yes/No
When Yes, .properties files will be created

Make .my Files Yes/No
When Yes, .my files will be created

Save Metadata from IMDB Yes/No
When Yes, enables a Save button on the IMDB show page

Download Images from IMDB Yes/No
When Yes, Cover Art will be saved when saving from IMDB
_______________________________________________________________

Metadata Save to DB Configure



Auto Save to DB from .my Files Yes/No
When Yes, will automatically import data from a .my file (if no data already exists in the DB) into the SageTV database when going into Video detail

Auto Save Overwrites DB Yes/No
When Yes, will overwirte existing data in the SageTV database when Auto Saving

Manual Save to DB from .my Files Yes/No
When Yes, will import data from a .my file (if no data already exists in the DB) into the SageTV database using the Import .my file button in Manage Metadata

Manual Save Overwrites DB Yes/No
When Yes, will overwirte existing data in the SageTV database when saving into the SageTV database using the Import .my file button in Manage Metadata

Edit Metadata Yes/No
Enables the Edit Metadata button on the Manage Metadata panel. Edit Metadata, edits the data stored in the SageTV database and saves the changes to the database and .my and/or .properties files

_______________________________________________________________

Read .my Files Yes/No
When Yes SageTV will display Metadata in Detail Video Info from a .my file if available and there is no existing data in the SageTV database for the video.
When No SageTV will not display Metadata

Always Read .my Files Yes/No
When Yes SageTV will display Metadata in Detail Video Info from a .my file if available even if there is existing data in the SageTV database for the video.
When No SageTV will not display Metadata

Bulk Creation Options Configure
When Enabled Captions will be disabled at start of playback and displayed if the volume is muted
When Disabled will use the External/Embedded Captions Enable options

Manual Metadata Files Directory
The Directory that Bulk File creation and Manual Creation of Past and Future Airings will save the .my and .properties files.
_______________________________________________________________





_______________________________________________________________

Additional Information

For anyone following progress in the A Collection of Minor Tweaks for the Default STV thread, this thread is home now and it has been removed from over there.

Background:

.my files are an old text format file from the old MYHTPC (precursor to Meedio) days. They are basically a tagged format file which a variety of applications can read.

By using these files, you can have similar information in the Video Detail for Imported Videos that you normally get in Sage Recordings. The file is created with the same name as the video file, with a .my extension.

This import provides for the creation and reading of .my files. The data can be sourced from current EPG entries or, IMDB data.

A simple example of the contents of a .my file:

title=Man, Moment, Machine
TVEpisode=Werner Von Braun and the V2 Rocket
TVOverview=In 1942, Hitler demands a silent, deadly weapon that is capable of completely destroying London in a matter of days.

_______________________________________________________________

Changes:
(1.5b2)
- Save data from IMDB to .my file
- Read more tags in .my file
(1.5b3)
- Optional Download of IMDB image
- Cleans up old .my Reading import if installed
- Fixed writing of Tagline
- Fixed non-clearing of filename attribute
(1.5)
- Fixed make .my file button not appearing in version 5
(1.5.1)
- Fixed not writing IMDB .my files correctly if there was no description (mostly TV shows, but who creates .my files with no description)
(1.5.2)
- Added IMDB Trivia and Quotes to IMDB .my file creation (Using tags TVTrivia and TVQuotes) and support for reading those tags.
- Added ability for multi-paragraph Overview (still needs to be on 1 line but ### in .my file will display as a newline)
- Added changes from SageMC for Global looped fork termination
- Changed property name for not creating automatic .my files for imported videos to the same as SageMC for ease of switching STVs.
(1.5.3)
- Fix for not working on clients. Only change from 1.5.2
(1.5.4)
- Added Search button on IMDB search results page. Allows you to try another search if the auto search doesn't get the result you want.
(1.5.5)
- Fixed writing of .my files from IMDB for ripped DVD's
(1.5.6)
- Fixed writing of .jpg files from IMDB for ripped DVD's
(1.5.7)
- Added read of tags from pdavis' TV.COM Scrapper
(1.5.8)
- Changed some attribute names to eliminate possibility of confusion in 6.1 using the same attribute names
(2.0.0)
- Name change to RW_Metadata.stvi
- Automatic creation of .my and or .properties files for current recordings. (see 6.1.4 beta for properties file info)
- Manual creation of .my and or .properties files for any or all, current, past of future airing.
- Display more than default wiz.bin show information in imported videos
- Saves IMDB data to wiz.bin, .properties files, .my files
- Only for version 6
(2.0.1)
- Fix for tags with spaces
(2.0.2)
- Fix Bulk creation not making all files
(2.0.3)
- Add highlighting of tags displayed in imported videos
(2.0.4)
- Fixes for Actor parsing (see note below)
(2.0.5)
- Rewrite of file creation buttons and options panel
- Changes to imported video detail display
- Additional options for auto file creation
- Option to always display .my file data even if other wiz.bin data exists
- Fix for not saving from IMDB when an Actor had no role
(2.0.6)
- Option to automatically import some fields from .my file to wiz.bin
- Edit Video Information in wiz.bin
- Return of the .my and properties delete buttons
(2.0.7)
- Added read of tags from pdavis' TV.COM Scrapper
(2.0.8)
- Changed some attribute names to eliminate possibility of confusion in 6.1 using the same attribute names
- Fix writing of incorrect Titles to properties files of imported videos
- Changes to Edit Video Info panel
(2.0.9)
- Added support for SageTV version 5 (unsupported functions like saving to wiz.bin should not be visible)
- Added Automatic and Manual import of Cast from .my files into wiz.bin
- More detailed cast information saved from IMDB
- Removed restriction on Automatic file creation only working when Server UI was running
(2.1.0)
- Added save of MPAA Rating from IMDB to wiz.bin, properties and my files
- Added display of Rating and Extended Rating for imported videos
- Added Automatic and Manual import of Rating and Extended Rating from .my files into wiz.bin
(2.1.1)
- Fix writing and importing of multi-paragraph properties
- Added some more tags that will import from .my files
(2.1.2)
- Fix import of .my file TVGenre missing first character
- When importing .my file it will save to wiz.bin and .properties file
- When saving from Edit Metadata it will save to wiz.bin and .properties file
- Added importing of title, TVEpisode and ProdCode from .my file
(2.1.3)
- Save from Edit Metadata will save to .my file and retain any existing additional data
- Change Auto Delete of Orphaned files to delete .my and .properties files
- Add Option for Auto Create of files to overwrite existing files
- Add Option for Bulk Creation of files to overwrite existing files
- Add Option to Auto Delete Orphaned files in import directories
- Save from IMDB now returns to originating video folder
- Fix Title and Episode saving from IMDB for TV Recordings
(2.1.4)
- If Parental Controls is enabled, prompt for password to save changed Rated or Expanded Ratings in Edit Metadata
- Fix updating .my files from Edit Metadata (Not clearing deleted fields)
(2.1.5)
- Changed delimiter of multiple entry fields in properties files to use semicolon
(2.1.6)
- Support for SageTV 6.2
(2.1.7)
- Add Option for specifying a global directory for reading .my files from.
- Add Option for using subdirectory (MyFiles) below directory video file is located in for reading .my files from.
(2.1.8)
- Support for Linux
- Changes to stvi import code
(2.1.9)
- Fix Edit Metadata not showing all fields for Archived videos
- Changed button size on Edit Metadata input panel
(2.2.0)
- Added Option to not change Video Title to IMDB Title on IMDB save
- Added Option to not download Trivia, Quotes etc. on IMDB save
- Change Trivia, Quotes etc. to save in Description field instead of Misc
- Changed IMDB rating from Numeric Rating (3.2/10) to Star Rating (***)
- Added Option for Base no. of Stars for Star Rating on IMDB save
- Fix Category saving to include Movie etc. on IMDB save
- Fix Edit Metadata to show correct fields based on video type
- Fix extra / when importing of SubCategories on Import from .my file
- Fix returning to originating video folder on IMDB save when IMDB search performed from Detailed Info panel
- Added Left (Back) listener on Edit Metadata Menu
(2.2.1)
- Added bulk import from .my files. Setup -> Detailed Setup -> Customize -> Metadata Options -> Metadata Save to DB -> Import to DB from .my Files
(2.2.2)
- Fix location and naming of .my and .properties files for DVDs
- Fix bulk import of my files not honoring overwrite setting
- Changed bulk import to always use .my file title field
- Fixed returning to originating folder when saving from IMDB and IMDB search was from Video Detailed Info
- Added Option when saving from IMDB to prompt to change Video Name to IMDB name
- Tidied up settings properties (check your settings as some will be back at default values)
(2.2.3)
- Reworked logic for the automatic generation thread to ensure the thread is killed when switching STVs and if a client disconnects ungracefully. Also removed the requirement to restart after importing or enabling/disabling. File generation is no longer global and can be enabled/disabled on a per client basis.
- Reduced the amount of logging done by default with an option to enable full debug logging.
- Added Option when saving from IMDB to either prompt to change Video Name to IMDB name or always save as the IMDB name or original Video name.
- Removed saving of Extended Rating info (ie: for Violence) from IMDB because the IMDB plugin v6.10 no longer returns a value
- Fixed saving of Star rating from IMDB because the IMDB plugin v6.10 returns it in a different format
(2.2.4)
- Fixed saving MPAA rating from IMDB (requires updated imdb.jar file)
- Fixed saving TV show info from IMDB (requires updated imdb.jar file)
- Changed scan frequency and checks to ensure only one client performs scans and cleanups
(2.2.5)
- Fixed cleanup not setting next run time correctly
- Fixed cleanup starting multiple threads
- Added additional check of file extension before deleted by cleanup
- Changed logging to always log any file deleted
- Changed checking file types for .my file cleanup to stop on first file match
(2.2.6)
- Changed display of cast details to make them more readable.
- Changed import code to still work when TiKi FanArt is installed (install after TiKi FanArt v2.3 or higher.
- Added Parental lock to importing .my files
(2.2.9)
- Changed import code to work when Phoenix FanArt for Default STV is installed (install after Phoenix FanArt for Default STV.
- Added editing of some Phoenix Metadata properties
- Added saving of some Phoenix Metadata properties to .properties files
- Added saving of Mediafile format to .my files
- Added updated IMDB JAR file to fix problems with Quotes, MPAA Ratings and TV Episode info

Note There was a bug in SageTV prior to 6.1.6 when not running in service mode that caused the wiz.bin save from IMDB and subsequent .properties file creation to not list any Cast/Crew details.

John

[dvd_maniac]

I am unable to get this to work.

It saves the .my file but when I try to display video details it will not display correctly.
It displays title and released twice but no overview or cast.
Example for 16 Blocks:
Title:
Movies\Drama\16 Blocks
Released:
2006
Title:
Movies\Drama\16 Blocks
Released:
2006
File:
[M:\Movies\Drama\16 Blocks.mp4]

Format:
[H.264 720x304@24fps, AAC@48khz Stereo eng]

Filesize: 530.09 MB

[JREkiwi]

Originally Posted by dvd_maniac

I am unable to get this to work.

It saves the .my file but when I try to display video details it will not display correctly.
It displays title and released twice but no overview or cast.

Which version of Sage and the import are you running?
Is the data in the .my file?
Can you try a clean install of just the import and see if that makes a difference.

I do need to do some work on reading of the IMDB data in files in the test version, but you shouldn't be getting duplicates.

John

[JREkiwi]

New test version RW My Files 1.5 beta 2 (2 Dec 2006) download from here

- Save data from IMDB to .my file
- Read more tags in .my file

Requires Nielm's IMDB Search Interface installed first for saving IMDB data.

John

[jkohn]

So just out of curiosity, what is a .my file?

[JREkiwi]

Originally Posted by jkohn

So just out of curiosity, what is a .my file?

.my files are an old text format file from the old MYHTPC (precursor to Meedio) days. They are basically a tagged format file which a variety of applications can read.

By using these files, you can have similar information in the Video Detail for Imported Videos that you normally get in Sage Recordings. The file is created with the same name as the video file, with a .my extension.

An example of a .my file would be:

title=Man, Moment, Machine
TVEpisode=Werner Von Braun and the V2 Rocket
TVOverview=In 1942, Hitler demands a silent, deadly weapon that is capable of completely destroying London in a matter of days.
overview=In 1942, Hitler demands a silent, deadly weapon that is capable of completely destroying London in a matter of days.
TVStartDate=30/11/06
TVStartTime=22:30
TVEndTime=23:30
TVFile=W:\My Sage\ManMomentMachine-WernerVonBraunandtheV2Rocket-524101-0.mpg
TVFirstRun=true
TVFavorite=true
TVManual=false
TVLibrary=false
TVGenre=TV Series
TVChannelNumber=53
TVChannelName=The History Channel
TVChannelDescription=The History Channel
TVExternalID=SH1IDnDh0000

This import provides for the creation and reading of .my files. The data can be sourced from current EPG entries or, in the test version, IMDB data.

I'm sure someone else could provide a better explanation.

John

[JREkiwi]

Posted a new test version: RW My Files 1.5 beta 3 (3 Dec 2006) download from here
I think this version is now complete, so any feedback would be appreciated

Features/Changes:
(1.5b2)
- Save data from IMDB to .my file
- Read more tags in .my file
(1.5b3)
- Optional Download of IMDB image
- Cleans up old .my Reading import if installed
- Fixed writing of Tagline
- Fixed non-clearing of filename attribute

Prerequisites: For saving IMDB data Nielm's IMDB Search Interface must be installed before this import.

So installation would be
Step 1. Unzip into SageTV maintaining the directories. (The df_sageutils.jar from the zip file needs to be in the JARS directory)
Step 2. Restart Sage so it will pick up the jar.
Step 3. Load the IMDB Search Interface import
Step 4. Load the RW My Files import

John

[Necro]

Does anyone know of any apps that will automagically generate .my files for this use outside of Sage? Specifically for TV Episode data...since doing those searches via IMDB is tedious at best and the information is many times lacking.

[dj7675]

Does this work in SageMC? Or does it already have this tool in it?

Thanks

Darin

[JREkiwi]

Originally Posted by dj7675

Does this work in SageMC? Or does it already have this tool in it?

No it doesn't work in SageMC.

SageMC does .my file reading
The automatic .my file creation and cleanup was stolen from SageMC.
SageMC does saving from IMDB into the Sage database, but if you use that on Imported Videos it breaks sorting in the version 6 Default STV, so I went with writing to .my files from IMDB.
The bulk creation and creation from past and future airings I don't think are in SageMC.

John

[dflachbart]

Originally Posted by JREkiwi

The bulk creation and creation from past and future airings I don't think are in SageMC.

John

Actually they are ... You can generate .my files for any group of past or future airings (on the 'Other Showings' screen). But your STVi might have some more options for this, at some point I'll have to catch up ...

Dirk

[dvd_maniac]

Originally Posted by Necro

Does anyone know of any apps that will automagically generate .my files for this use outside of Sage? Specifically for TV Episode data...since doing those searches via IMDB is tedious at best and the information is many times lacking.

I find www.tv.com to be much better for TV episodes. IMDB has a lot of series with blank synopsis or has the Series description for most episodes.
What i did to create .my files for series that were not in the Sage database was go to a show episode guide Like here and hit the print episode guide button. then edit out all the extra stuff and then Use Word's find/replace feature to change the tags to fit this stvi. Finally I cut, paste the info into a blank txt file and save it as the video filename. Star Trek TNG took me about 1/2 hour to do the whole series.

But if someone can find a way to automate this..... that would be great.
But please automate it using tv.com for TV series

[justme]

Originally Posted by dvd_maniac

But if someone can find a way to automate this..... that would be great.
But please automate it using tv.com for TV series

Ditto for me too. I'd be very greatfull for a IMDB style import, that gave access to TV.com's rich data for show's episodes. Of course I'd really want that support/compatibility(like IMDB) eventually added to this great import.

[JREkiwi]

Originally Posted by flachbar

Actually they are ... You can generate .my files for any group of past or future airings (on the 'Other Showings' screen). But your STVi might have some more options for this, at some point I'll have to catch up ...

Dirk

Dirk, I didn't know that, I could have just stolen that too, along with the rest.

John

[JREkiwi]

Originally Posted by justme

Ditto for me too. I'd be very greatfull for a IMDB style import, that gave access to TV.com's rich data for show's episodes. Of course I'd really want that support/compatibility(like IMDB) eventually added to this great import.

It's easy, as soon as flachbar writes it for SageMC, I'll steal it

John

[JREkiwi]

New version of RW My Files 1.5 (7 Dec 2006)

New Features
- Save data from IMDB to .my file
- Read more tags in .my file
- Optional Download of IMDB image
- Cleans up old .my Reading import if installed

John

[dflachbart]

Originally Posted by JREkiwi

It's easy, as soon as flachbar writes it for SageMC, I'll steal it

John

Did I ever mention I hate website parsing ?

Dirk

[dj7675]

Originally Posted by JREkiwi

New version of RW My Files 1.5 (7 Dec 2006)

New Features
- Save data from IMDB to .my file
- Read more tags in .my file
- Optional Download of IMDB image
- Cleans up old .my Reading import if installed

John

A couple of questions regarding your new version....

-Optional download of imdb impage....where do you configure that you want it to automatically download the image?
-Does it download the image to the same place as the .my file?
-Is the image given the same hame as the .my file?

This could be very cool if it gives the image the samge name as the .my file but with a .jpg extension. My Roku will display the image for the show then automatically. Is this correct?

Thanks for doing this. I was just going to as about the image from imdb as well but you beat me to it (if I am understanding how it works correctly).

Darin

[JREkiwi]

Originally Posted by dj7675

A couple of questions regarding your new version....

-Optional download of imdb impage....where do you configure that you want it to automatically download the image?
-Does it download the image to the same place as the .my file?
-Is the image given the same hame as the .my file?

This could be very cool if it gives the image the samge name as the .my file but with a .jpg extension. My Roku will display the image for the show then automatically. Is this correct?

Darin, in Setup/Detailed Setup/Customize/.my File Options there are options for enabling creating .my files from IMDB and another option for downloading the images. Both are enabled by default.

You'll end up with a video file, a .my file and a .jpg named the same as the video file and stored in the same directory.

There is basic 2 ways it creates .my files. Either from the Sage database, or from IMDB data. To save from IMDB, you need to do an IMDB Search from the Video Options panel and Make a .my file from there.

John

[dj7675]

Thanks for the reply. Got it working very nice. Not sure if it is possible, but it would be perfect if it could automatically create the image just like it automatically creates the .my file. But that would be too much to ask.

Thanks again.

Darin