Is there a way to manually edit the Sage database

[mikesm]

Folks, I have been trying a lot of things to get some programs that showed up in the imported videos database removed. The problem is they are recordings, and for some reason, Sage thinks they are imported videos as well (they show up in both menus). I can't get rid of them in the imported videos display, at least not without removing them from the system completely, at least I haven't tried that.

Is there a tool where I can edit the database manually to get rid of these items. I can't seem to get rid of them in any other way. I know the database isn't in access or some other format that has a client that can connect to it in a standard way, but was hoping some developer might have crossed this bridge already and built such a tool.

It's a problem because it clutters up the filesystem view in imported videos and makes it hard for my family to use.

Thanks,
Mike

[Opus4]

The only way to edit the database is via the API calls.

You've got the same exact file in both the imported video browser and the recordings menus? What STV is this, btw? If you aren't usnig the default STV, try it there w/o any customizations loaded.

On the detailed info screens for videos & recordings, does it show the exact same file path for the video and recording entries? If so, try: close SageTV, move the file to another recording dir, then restart SageTV.

- Andy

[mikesm]

Originally Posted by Opus4

The only way to edit the database is via the API calls.

You've got the same exact file in both the imported video browser and the recordings menus? What STV is this, btw? If you aren't usnig the default STV, try it there w/o any customizations loaded.

On the detailed info screens for videos & recordings, does it show the exact same file path for the video and recording entries? If so, try: close SageTV, move the file to another recording dir, then restart SageTV.

- Andy

This was with SageMC. And it had the same problem in the default STV.

I created a new recording directory, added it to the configuration so it's recognized as another recording directory, moved one of the files there (jarhead from HBO in HD), and the entry for it in the imported videos list went away. Good deal.

But, when I moved it back to the normal Sage recording directory, re-scanned the imports folder, the entry came back in the imported videos list. Why is this happening? How can I purge the meta-data for these mis-indexed files so they don't come back in imported videos?

I don't want to have to keep creating additional recording directories, because it mess up the shares since all these files on a NAS., even the Sage recording directories are stored on the NAS.

Thanks,
mike

[Opus4]

Originally Posted by mikesm

This was with SageMC.
...
But, when I moved it back to the normal Sage recording directory, re-scanned the imports folder, the entry came back in the imported videos list. Why is this happening? How can I purge the meta-data for these mis-indexed files so they don't come back in imported videos?

So the path on the detail info screens was exactly the same for the imported video & the recording? I don't know how it got listed as both an imported video + recording. Maybe there is some sort of side effect with something SageMC is doing about the imported video data or with the recording. If you can reproduce how it happened, that might help determine what happened & whether there is a bug in the core.

But, for this video, _maybe_ the imported metadata would get dumped during a cleanup after an EPG update, though I'm not certain if it would do that. Try leaving the file in another directory until after the EPG is updated.

Edit: Is there any sort of external SageMC metadata that is getting reassociated with the video again?

Edit 2: If the metadata being associated with the file as an imported video inlcudes a Show ID that says it could be a valid recording, then the data could be kept in the database as a way to recover the info when the file is recovered.

- Andy

[mikesm]

Interesting... I forgot to check the recordings screen to verify it still showed up as a native recording. It doesn't, it's only listed now as an imported video. Funny, given that Sage recorded it. I checked a handful of the programs that are clutterring up the imported videos screen, and none of them show up in the recordings database anymore. Only as imported videos.

I guess there is no way to get it back into the recordings database with all it's metadata?

It doesn't reside in a directory that is in the imported videos list, but in a SageTV recordings directory. How is it that it shows up at all then?

Should I actually move all these into one if the imported videos directories? Most are movies, though some kids shows that I have a ton of, so I can get rid of those I suppose.

Thanks,
Mike

[Opus4]

Originally Posted by mikesm

Interesting... I forgot to check the recordings screen to verify it still showed up as a native recording. It doesn't, it's only listed now as an imported video. Funny, given that Sage recorded it. I checked a handful of the programs that are clutterring up the imported videos screen, and none of them show up in the recordings database anymore. Only as imported videos.

I guess there is no way to get it back into the recordings database with all it's metadata?

If you are using v6.1, then as long as it has the file ID at the end of the filename that was put there when it was recorded, there should be data to connect it to... but it sounds like the data might have been customized in some way via SageMC or some metadata customization & it is now considered to be an import.

It doesn't reside in a directory that is in the imported videos list, but in a SageTV recordings directory. How is it that it shows up at all then?

Video files in a recording dir that are not recognized as recordings are automatically imported.

- Andy

[mikesm]

Originally Posted by Opus4

If you are using v6.1, then as long as it has the file ID at the end of the filename that was put there when it was recorded, there should be data to connect it to... but it sounds like the data might have been customized in some way via SageMC or some metadata customization & it is now considered to be an import.

Video files in a recording dir that are not recognized as recordings are automatically imported.

- Andy

Well, the name of the file is Jarhead-900865-0.mpg. It is sitting in the SageTV recording directory. It does not show up in the recordings guide, but does show up in imported videos. Is there a way to get this to show up as a normal recording? I have a about a dozen other such recordings which have the same issue.

What is the logic for automatically importing the file if it shows up in the recordings directory? This doesn't seem to make a lot of sense to me.

Thanks,
Mike

[GKusnick]

When did you record it? If you scroll the EPG back to that date, does that airing still show up? If you can't remember the date, try using the Search screen to locate historical airings of that show.

Once you've found an airing, make sure the numeric ID in the filename matches the AiringID (not the ShowID) for that airing. That's all that should be needed to connect the file to the database entry. I don't know if SageMC shows AiringID info in its detail screen; if not, perhaps you can get that info through the webserver plugin. At worst, it's available through API calls from Studio.

If you can't find a historical airing for that recording, then the metadata has been deleted from the database and you won't be able to get it out of the Imports section without manually reentering the metadata through the webserver or some other plugin.

I can't speak for the devs, but personally I think that listing orphaned recordings as imports is better than not listing them at all. At least this way you can still play them or delete them from within the Sage UI.

[mikesm]

Originally Posted by GKusnick

When did you record it? If you scroll the EPG back to that date, does that airing still show up? If you can't remember the date, try using the Search screen to locate historical airings of that show.

Once you've found an airing, make sure the numeric ID in the filename matches the AiringID (not the ShowID) for that airing. That's all that should be needed to connect the file to the database entry. I don't know if SageMC shows AiringID info in its detail screen; if not, perhaps you can get that info through the webserver plugin. At worst, it's available through API calls from Studio.

If you can't find a historical airing for that recording, then the metadata has been deleted from the database and you won't be able to get it out of the Imports section without manually reentering the metadata through the webserver or some other plugin.

I can't speak for the devs, but personally I think that listing orphaned recordings as imports is better than not listing them at all. At least this way you can still play them or delete them from within the Sage UI.

So what am I supposed to do to get the recording treated as a recording again if the program's info is still in the guide? Should it pick it up automatically if it's in the recordings directory?

Orphaned recordings can always be moved to the imports directory. Automatically importing them can have some issues since it confuses things with this metadata management issue.

Thanks,
Mike

[stanger89]

Originally Posted by mikesm

Orphaned recordings can always be moved to the imports directory.

What if you only use the Sage interface for management? Sage is designed so that you don't have to go outside the interface, basically for anything.

[mikesm]

Originally Posted by stanger89

What if you only use the Sage interface for management? Sage is designed so that you don't have to go outside the interface, basically for anything.

Recordings aren't supposed to be orphaned either. And I suppose properties files shouldn't need editing too... :-)

Thanks,
Mike

[Opus4]

Originally Posted by mikesm

Well, the name of the file is Jarhead-900865-0.mpg. It is sitting in the SageTV recording directory. It does not show up in the recordings guide, but does show up in imported videos. Is there a way to get this to show up as a normal recording? I have a about a dozen other such recordings which have the same issue.

I kept thinking that you've been messing with metadata in SageMC, based on your other recent posts, but if you've messed around with wiz.bin & then reverted to an older copy of that file, the recording may not even match the ID for the same show in the other wiz.bin, if it even has a listing for that show. In this case, there is no airing to match this recording to, so it can't be considered a recording any more.

What is the logic for automatically importing the file if it shows up in the recordings directory? This doesn't seem to make a lot of sense to me.

This was already answered above, but it sure keeps files in your recording dirs from getting lost among all the other recordings & then never knowing you've got unused files sitting around. Unrecognized files in the recording dirs used to be automatically deleted or moved to a subdirectory called "errant". Auto-deletion was determined to be very bad, but I think you can still set the seeker/use_errant_dir property to true. You would have to remember to check the dir, if that still works.

Originally Posted by mikesm

So what am I supposed to do to get the recording treated as a recording again if the program's info is still in the guide? Should it pick it up automatically if it's in the recordings directory?

If the airing info with that ID is there, it should have matched it already automatically. If moving the file to a new recording dir doesn't reset it & match its ID, then there is probably nothing to match to.

- Andy

[mikesm]

Originally Posted by Opus4

I kept thinking that you've been messing with metadata in SageMC, based on your other recent posts, but if you've messed around with wiz.bin & then reverted to an older copy of that file, the recording may not even match the ID for the same show in the other wiz.bin, if it even has a listing for that show. In this case, there is no airing to match this recording to, so it can't be considered a recording any more.

This was already answered above, but it sure keeps files in your recording dirs from getting lost among all the other recordings & then never knowing you've got unused files sitting around. Unrecognized files in the recording dirs used to be automatically deleted or moved to a subdirectory called "errant". Auto-deletion was determined to be very bad, but I think you can still set the seeker/use_errant_dir property to true. You would have to remember to check the dir, if that still works.

If the airing info with that ID is there, it should have matched it already automatically. If moving the file to a new recording dir doesn't reset it & match its ID, then there is probably nothing to match to.

- Andy

Thanks for the reply. I haven't been "messing" with wiz.bin - I didn't think there was any point in trying. :-)

Moving it to the 2nd sage recordings directory and triggering a rescan didn't result in it being considered a recording, so the guide info for it has probably expired.

It did show up in the imported videos again with the new location.

What is the property for setting the errant dir? That would give me more control over what happens. When the imported videos display gets cluttered, it's a problem for the WAF. Hence my focus on fixing this.

I'm working to get a chunk of time to try and take another shot at getting the DVD metadata interaction with dvdprofiler right again, but wanted to fix this issue first.

Sounds like my best bet is to move them to a new directory and use IMDB to populate the meta data for the movies.

Thanks much for being so responsive. It helps a lot to know the underlying mechanisms.

Thanks,
mike

[Opus4]

Originally Posted by mikesm

Thanks for the reply. I haven't been "messing" with wiz.bin - I didn't think there was any point in trying. :-)

This reference to "messing with wiz.bin" really just meant restoring an older copy of that file.

What is the property for setting the errant dir?

That's in my last post, above: "set the seeker/use_errant_dir property to true".

I'm working to get a chunk of time to try and take another shot at getting the DVD metadata interaction with dvdprofiler right again, but wanted to fix this issue first.

Sounds like my best bet is to move them to a new directory and use IMDB to populate the meta data for the movies.

If you never restored an older copy of wiz.bin, this was the other "messing around" that I was referring to -- anything that changes metadata in the database changes the data in wiz.bin, because that is the database. Maybe something affected those recordings, if you tried doing any customization of their data.

- Andy

[mikesm]

Originally Posted by Opus4

This reference to "messing with wiz.bin" really just meant restoring an older copy of that file.

That's in my last post, above: "set the seeker/use_errant_dir property to true".

If you never restored an older copy of wiz.bin, this was the other "messing around" that I was referring to -- anything that changes metadata in the database changes the data in wiz.bin, because that is the database. Maybe something affected those recordings, if you tried doing any customization of their data.

- Andy

By set it it true, I guess you are saying specify the directory instead of setting it to the word "true". Right?

The cause of the other problems with DVD profiler was a permission problem in the directories. The directory from the DVD rip was owned on the NAS by my personal user account, not the HTPC user acct that sage runs as. So when the importer ran, it couldn't write into the DVD rip directory. This seemed to cause the failure in importing it properly. Sage somehow thought the entry was loaded properly, but it wasn't. So that's how bad data got loaded into wiz.bin.

I had not put back an older copy of wiz.bin, because I thought would have have messed up the recordings of normal TV programs that were recording when I was doing the debugging.

Had I had your explanation of how Sage uses the tag in the name to do the lookup on program metadata at the time, I would have done exactly that and reloaded the old wiz.bin file since any of the new recordings would have been picked up from old data in the guide that would not have yet expired. This should be in an FAQ etc..., it would save you having to answer questions and users like me a lot of hair pulling.

Still, it would be nice to be able to edit the raw program data to deal with slipups in importing data. I guess I could write one using the studio API?

Thanks,
Mike

[Opus4]

Originally Posted by mikesm

By set it it true, I guess you are saying specify the directory instead of setting it to the word "true". Right?

No, set it to true:

seeker/use_errant_dir=true

Again: I haven't tried it in a long time, so I have no idea if it still works the same.

- Andy

[nielm]

Originally Posted by mikesm

Still, it would be nice to be able to edit the raw program data to deal with slipups in importing data.

My Webserver.

You have several alternatives:

• Edit show info (in the detailed info screen) for manually adding show info and putting it in the recordings...
• Import recordings info from XML (main menu)
• Import show info from XML for an existing file.

The XML can also be exported by the webserver, or by another plugin that auto-generates XML files for all recordings.

[mikesm]

Originally Posted by nielm

My Webserver.

You have several alternatives:

• Edit show info (in the detailed info screen) for manually adding show info and putting it in the recordings...
• Import recordings info from XML (main menu)
• Import show info from XML for an existing file.

The XML can also be exported by the webserver, or by another plugin that auto-generates XML files for all recordings.

This is very cool. I have your webserver installed, but never tried using it for this purpose.

The program info shown when I hit edit show info contains the original air date of the show, when it was recorded etc... I thought if this was an imported show, it should not have this info.

If it has all this info, why didn't it show up as a recording when it lived in the recordings directory? How does it know all this without a valid tag? When it was imported, where did it get this info?

Nielm, one thing missing on the webserver is the info about the thumbnail that is used to display it. I have having a lot of issues with these formerly errant recordings in that Sage refuses to display the cover .jpg file that IMDB found and loaded in the directory with the video. It just uses the video thumbnail generated by Sage and ignores the .jpg. It does this in SageMC and in the default UI. Why isn't the .jpg taking priority over the video snapshot?

The edit show info also doesn't let me change the location of the file. This would be handy to having to go through full media scan, and also make it less of an issue to have sage think it no longer exists.

Come to think of it, with your web server, can I delete all info about the file permanently, and then force a re-importation from scratch? This meta-data doesn't seem to want to go away, even if I delete the file or move it to another directory. I can't seem to get a clean import done.

PS It's show ID is: MF1372490. What does that tell you about where Sage thought the file came from?

Thanks much!
Mike

[nielm]

Originally Posted by mikesm

The program info shown when I hit edit show info contains the original air date of the show, when it was recorded etc... I thought if this was an imported show, it should not have this info.

It works it out from the file timestamp and duration.

Nielm, one thing missing on the webserver is the info about the thumbnail that is used to display it.

because it has no access to that data

The edit show info also doesn't let me change the location of the file..

it is not intended to - just to allow editing of show metadata. If you want to move it to a different folder, do it in Explorer...

PS It's show ID is: MF1372490. What does that tell you about where Sage thought the file came from?

its an auto-generated ID for an Imported Video -- it could not find a TV airing ID to match the file name...

[mikesm]

Originally Posted by nielm

It works it out from the file timestamp and duration.
because it has no access to that data

Ok, I see, if the timestamp changed during a file copy or the like, it would not have the correct airing time. Got it.

Originally Posted by nielm

it is not intended to - just to allow editing of show metadata. If you want to move it to a different folder, do it in Explorer...

So thumbnails and the like are not stored in the database? So if it wont use the cover .jpg, that's a code issue, not a database metadata problem?

If I move it to another folder, and then reimport it, the metadata seems to hang around and reattach itself to the file. Hence my desire to remove the entry completely. I want to make Sage completely forget the video was ever in it's system. Simply moving it and rescanning doesn't do that.


Thanks
Mike