Any well to tell when Sage Is importing new media?

[PLUCKYHD]

I am wondering if there is an easy way to tell when Sage is scanning for new media (the period check it does be it 2 hours or whatever the setting). I would like to fire off a scan of my own when it is finished. If it is in the API I missed it I am sure

TIA
skye

[stuckless]

Originally Posted by PLUCKYHD

I am wondering if there is an easy way to tell when Sage is scanning for new media (the period check it does be it 2 hours or whatever the setting). I would like to fire off a scan of my own when it is finished. If it is in the API I missed it I am sure

TIA
skye

PLUCKY, check out the mediafile_metadata_parser_plugins in the Sage.properties. It's what BMT uses to fetch metadata for the automatic plugin. It is called each time a new file is added to the sagetv library. It will be called when SageTV does does it's periodic refresh and it finds new files.

Be careful when you use this method because if you start calling MediaFileMetadata apis on this newly added file, then it may trigger sagetv to call your plugin again, and result in an endless loop. (bmt 3.0 had that issue)

[stanger89]

Check the stock UI, it puts up an icon when a scan is going on.

[PLUCKYHD]

Originally Posted by stuckless

PLUCKY, check out the mediafile_metadata_parser_plugins in the Sage.properties. It's what BMT uses to fetch metadata for the automatic plugin. It is called each time a new file is added to the sagetv library. It will be called when SageTV does does it's periodic refresh and it finds new files.

Be careful when you use this method because if you start calling MediaFileMetadata apis on this newly added file, then it may trigger sagetv to call your plugin again, and result in an endless loop. (bmt 3.0 had that issue)

I knew you would know because you did it with BMT.

I am not calling mediafilemetadata apis so I should be okay then.

Thanks will give this a look better than writing the route I was going and loaded a class at startup.

[PLUCKYHD]

Hey Sean,

Having a issue here. Does it want a runnable clase like the load_atstartup does? It throws this error when trying to load

Error instantiating metadata parser plugin of:java.lang.ClassCastException: smm.LoadSMMWorker cannot be cast to sage.MediaFileMetadataParser

No sure what it means by cannot be cast to sage.MediaFileMetadataParser.
The wierd thing is is does show my logging like it is working after that error.

Also after checking not sure this is the way I want to go as it calls this class on every video file it is importing correct? I more or less just need to have mine call once after importing is finished.

[stuckless]

Your class needs to implement the sage.MediaFileMetadataParser

public class LoadSMMWorker implements sage.MediaFileMetadataParser {
}

[stuckless]

Originally Posted by PLUCKYHD

Also after checking not sure this is the way I want to go as it calls this class on every video file it is importing correct? I more or less just need to have mine call once after importing is finished.

Yeah, this is called for EVERY media file (videos, recordings, pictures and music). You'd have to use the file extension to determine if you wanted to handle this file or not.

For bmt, in 4.0, I allow the user to specify which file types they want to handle in the automatic metadata plugin.

This won't help you much today... but in phoenix, there is an EventBus, where developers can subscribe to certain events. Currently there is an event that is fired each time a new MediaFile is successfully added to the wiz.bin. I'm actually using this model in the automatic plugin. ie, I register an event handler for media and the automatic plugin simple fires the event that then later gets picked up by the event handlers. This allows me to background process the requests instead of having them block, like they do in 3.x.

[PLUCKYHD]

Originally Posted by stuckless

Yeah, this is called for EVERY media file (videos, recordings, pictures and music). You'd have to use the file extension to determine if you wanted to handle this file or not.

For bmt, in 4.0, I allow the user to specify which file types they want to handle in the automatic metadata plugin.

This won't help you much today... but in phoenix, there is an EventBus, where developers can subscribe to certain events. Currently there is an event that is fired each time a new MediaFile is successfully added to the wiz.bin. I'm actually using this model in the automatic plugin. ie, I register an event handler for media and the automatic plugin simple fires the event that then later gets picked up by the event handlers. This allows me to background process the requests instead of having them block, like they do in 3.x.

Thanks for the teaser Sean . I think for now since I don't need it fired on every video file only once I will keep it running in the startup loadable classes and have it's own 2 hour (or user preferred time interval) for automatic scanning.

While I got your attention since that is the route I am going for now. It is my understanding that by using setDaemon(true) and making my thread a demon thread it will automatically be killed when the sage service stops. Also it is my understanding that it will automatically be started when the sage service starts regardless of when SageTV Ui instance starts. Is that correct? The reason I ask is because I see no logging happening when the sagetv service starts but do see the entry starting the thread when a instance of sagetv starts.

thanks again

[Opus4]

Check out the MediaFilesImported hook. It provides a list of newly imported files too.

- Andy

[PLUCKYHD]

Originally Posted by Opus4

Check out the MediaFilesImported hook. It provides a list of newly imported files too.

- Andy

Thanks Andy I think I want to stay out of the STV requirement for this thread but am I correct in my assumption above about startup runnable classes?

My simple thought there is start the thread and in run have a

while(true){

     try {
			Thread.sleep(amount of time to wait between scans);
		} catch (InterruptedException ex) {
			 System.out.println("daemonServer: sleep exception: " + ex);}
		
              
          Run my code}

Does that seem like a good simple way to achieve a rescan of my own database?

[stuckless]

Originally Posted by PLUCKYHD

Thanks Andy I think I want to stay out of the STV requirement for this thread but am I correct in my assumption above about startup runnable classes?

My simple thought there is start the thread and in run have a

while(true){

     try {
			Thread.sleep(amount of time to wait between scans);
		} catch (InterruptedException ex) {
			 System.out.println("daemonServer: sleep exception: " + ex);}
		
              
          Run my code}

Does that seem like a good simple way to achieve a rescan of my own database?

Plucky, I'd avoid creating your own threads, if at all possible, unless you fully understand how to manage threads. ie, you need to manage the shut down of your thead, etc. Also, depending on what you thread is doing, it can have an impact on overall performance.

Any time you can leverage an existing hook instead of writing your own thread and thread monitoring, you'll be better off. ie, using the medailfe metadata parser plugin hook would be a better than managing your own thread, since sage is going to configure a directory scanning intervalue, etc. You don't want too many independant threads all doing file scans.

If you do ahead with your own thread, then be sure to research how to manage the shutdown of your thread... ie, while (true) is not good. Also research the InterruptedException, since you are expected to handle it correctly if you catch it.

[PLUCKYHD]

Originally Posted by stuckless

Plucky, I'd avoid creating your own threads, if at all possible, unless you fully understand how to manage threads. ie, you need to manage the shut down of your thead, etc. Also, depending on what you thread is doing, it can have an impact on overall performance.

Any time you can leverage an existing hook instead of writing your own thread and thread monitoring, you'll be better off. ie, using the medailfe metadata parser plugin hook would be a better than managing your own thread, since sage is going to configure a directory scanning intervalue, etc. You don't want too many independant threads all doing file scans.

If you do ahead with your own thread, then be sure to research how to manage the shutdown of your thread... ie, while (true) is not good. Also research the InterruptedException, since you are expected to handle it correctly if you catch it.

I will admit I am still learning the thread thing. The problem with leveraging an existing hook is then the process is only done when the UI is open and that is not going to help me much. Also it was my understanding and I could have read wrong that with a daemon thread it was automatically shutdown when the JVM is ended (or sagetv service stopped in this case) and that is why I chose that route so I wouldn't have to worry about the thread staying open.

The problem with the mediafile parser class it that it would call my searching for every single video file when I only need it called once after finished not for every video file. (I am checking a sql database on this call nothing to do with sagedb just need it ran at a regular interval).

I am reading up on InterruptedException now seems simple to handle it.

Again my goal is also not to screw anything up and if I am still misunderstanding or going about this wrong please don't hesitate to let me know

[stuckless]

If you create threads, then you can prevent the jvm from cleanly shutting down. ie, the jvm exits when all threads are exited. Sometimes this can be as simple as adding a shutdown hook that notifies your thread to exit.

[Fuzzy]

Is there a way you can sync up your sql database one record at a time? So sage finds a new file, and it then handles creating the data in your sql?

[PLUCKYHD]

Originally Posted by stuckless

If you create threads, then you can prevent the jvm from cleanly shutting down. ie, the jvm exits when all threads are exited. Sometimes this can be as simple as adding a shutdown hook that notifies your thread to exit.

Okay that makes sense I can add a shutdown hook easily enough

Originally Posted by Fuzzy

Is there a way you can sync up your sql database one record at a time? So sage finds a new file, and it then handles creating the data in your sql?

The problem lies in that I am not just looking for new files but also updated metadata (Mymovies searches for updated metadata as well as new files) so searching them one a time wouldn't be ideal as it would leave out any changed files.) I know BMT touches the files to achieve change status but I am trying to avoid having to do that.

nothing is ever easy

[PLUCKYHD]

Also should mention that I am not actually writing to the wiz.bin in this process so I guess that is why it doesn't make sense for me to do it per file.

[jphipps]

I use a background thread running in the server that waits on a serial blocking queue, and I use the metadata plugin call to drop an object onto the queue and process getting the metadata and inserting the data into the database in that background thread, so it doesn't hold up any of the normal sage processing...

Thanks,
Jeff

[stuckless]

Plucky, something else to consider, rather than managing your own thread, is to use the java Timer and TimerTask. What you are trying to do, is basicly why these classes were added to the java standard library.

The simple use case is something like this...

TimerTask task = new TimerTask() {
    public void run() {
        /// check for new files, and add them to the database
    }
}

Timer timer = new Timer(true);
time.schedule(task, Calendar.getInstance().getTime(), (60*60*1000));

This basically will schedule your task to run every 1 hour (60*60*1000ms) until the jvm shuts down, but you don't have to manage the thread code, etc.

For more information on the Timer check out the javadoc.

[PLUCKYHD]

Originally Posted by stuckless

Plucky, something else to consider, rather than managing your own thread, is to use the java Timer and TimerTask. What you are trying to do, is basicly why these classes were added to the java standard library.

The simple use case is something like this...

TimerTask task = new TimerTask() {
    public void run() {
        /// check for new files, and add them to the database
    }
}

Timer timer = new Timer(true);
time.schedule(task, Calendar.getInstance().getTime(), (60*60*1000));

This basically will schedule your task to run every 1 hour (60*60*1000ms) until the jvm shuts down, but you don't have to manage the thread code, etc.

For more information on the Timer check out the javadoc.

Thanks Sean I will read up on that from your example it looks a better route to go. I have been running my thread all weekend and starting and stopping it without any problems but your above post about using while(true) scared me a bit even though I am not seeing issues your proposal makes me feel better/safer and I will probably go that route. I am guessing the run thread would automatically kill itself after running since there is no looping?

Thanks again Sean for all your willingness to help me out when I need it.

[PLUCKYHD]

This part of the java doc scares me a bit

"After the last live reference to a Timer object goes away and all outstanding tasks have completed execution, the timer's task execution thread terminates gracefully (and becomes subject to garbage collection). However, this can take arbitrarily long to occur. By default, the task execution thread does not run as a daemon thread, so it is capable of keeping an application from terminating. If a caller wants to terminate a timer's task execution thread rapidly, the caller should invoke the the timer's cancel method. "

I don't want my scanning to keep sage service from being able to shut down if it is in the process where as the deamon thread I am running now just gets killed regardless if it is in process or not. Of course it may be arbitrary because at max I don't see my process running more than 20seconds for a total rebuild....

Still reading