Explanation of JavaHeap and how it pertains to image caching

[PLUCKYHD]

I am a little confused on how this all works so excuse the ignorance. I am wondering why say when I have my java heap set to 1gb does it not keep images in cache when say the usage is only at or around 200 to 300mb. I am noticing on clients images are loading/reloading allot slower than on the server (to be expected as all the images are on the server) but wondering why it is not keeping more images in the java heap/memory for quicker access.

Is there a better/more efficient way in java to cache the images locally once they are called once as to make the next call quicker?

thanks for any pointers/explanations.

[jreichen]

Originally Posted by PLUCKYHD

I am a little confused on how this all works so excuse the ignorance. I am wondering why say when I have my java heap set to 1gb does it not keep images in cache when say the usage is only at or around 200 to 300mb. I am noticing on clients images are loading/reloading allot slower than on the server (to be expected as all the images are on the server) but wondering why it is not keeping more images in the java heap/memory for quicker access.

Is there a better/more efficient way in java to cache the images locally once they are called once as to make the next call quicker?

thanks for any pointers/explanations.

Check out Java's WeakHashMap and Weak/Soft/PhantomReferences. It's been a while since I've used them so I don't have details off the top of my head, but using them is a good way to cache objects. They will hold references to objects unless the JVM needs to free memory for other objects.

This page looks like a good explanation.

[PLUCKYHD]

Originally Posted by jreichen

Check out Java's WeakHashMap and Weak/Soft/PhantomReferences. It's been a while since I've used them so I don't have details off the top of my head, but using them is a good way to cache objects. They will hold references to objects unless the JVM needs to free memory for other objects.

This page looks like a good explanation.

Nice read thanks for that but curious by using a weakhashmap as it suggest to do how do I make sure that hashmap is storing the image and not just the string to the image? Do I need to be storing File in the hashmap vs string? Sorry still a little confused on how to be sure it is caching/storing the image in a weak reference and not just the string/path name.

Still wondering why when my java heap is not full it is losing image items loaded by sage I am guessing sage does cleanup of it's own causing this and hopefully by maybe using this weakreference I can avoid them being dumped until memory is full?

[jreichen]

Originally Posted by PLUCKYHD

Nice read thanks for that but curious by using a weakhashmap as it suggest to do how do I make sure that hashmap is storing the image and not just the string to the image? Do I need to be storing File in the hashmap vs string? Sorry still a little confused on how to be sure it is caching/storing the image in a weak reference and not just the string/path name.

The key of your hash map would be the string and the value would be the image. Here's some pseudocode:

Map<String, Image> imageMap = new WeakHashMap<String, Image>();

// add your images to the cache
...

// retrieve image
String imageName.....
Image image = imageMap.get(imageName);
if (image == null)
{
    // image was removed from cache,
    // retrieve image again and cache in the map again
    ...
}
// use the image
...

Since the key is a String (and may very well have strong references to it), you might want do some testing to make sure it works like you want it to. Use an infinite loop to add images to the map until you either get an OutOfMemoryError or the JVM starts removing existing items from the map to make room for new ones. If you get an OutOfMemoryError, check out the info in the WeakHashMap JavaDoc about wrapping the value in a WeakReference object.

Originally Posted by PLUCKYHD

Still wondering why when my java heap is not full it is losing image items loaded by sage I am guessing sage does cleanup of it's own causing this and hopefully by maybe using this weakreference I can avoid them being dumped until memory is full?

I'm not sure how the Sage core handles it. Stuckless may know from the work he's doing on Phoenix. You could ask him over on that thread.

[GKusnick]

Originally Posted by PLUCKYHD

Do I need to be storing File in the hashmap vs string?

File and String are just two different ways of representing a file path (i.e. the location of the file on disk). Neither one of them contains the actual file contents, i.e. the pixels of the image, and that's presumably what you want to cache. So you'd probably need to do LoadImageFile() and cache the result of that to retain the actual image in memory.

[stuckless]

Originally Posted by jreichen

I'm not sure how the Sage core handles it. Stuckless may know from the work he's doing on Phoenix. You could ask him over on that thread.

In Phoenix I do make extensive use of the weakhashmap for caching. But, that being said, I think plucky is looking for more of a "core" caching, since, he's already loaded the images, and then sage appears to throw them out of it's internal cache.

After grepping from 'cache' in the sage.properties, it returned these properties...

preload_expression_cache=false
ui/dump_font_cache=false
ui/enable_hardware_scaling_cache=false
ui/image_cache_min_num_allowance2=2
ui/system_memory_2dimage_cache_limit=20000000
ui/system_memory_2dimage_cache_size=16000000

maybe tweaking the 2dimage cache setting will help. I'm guessing that 16,000,000 is 16M.

Another thing to consider, is that once images are loaded they are no longer compressed, and as such they consume much more memory. ie, a 50K jpm may take several megs once it is loaded. I'm not sure the exact formula but i think it's something like, bytes= H * W * (Depth / 8). So a 32bit image that is 200x300 would take approx 240,000 bytes of actual memory... In other words, your memory could start filling up quickly if those images don't get out of the cache, espcially if you are loading backgrounds (1920x1080). Also, if sage does an internal image cache, it probably doesn't weight what is being put into it. ie, if you loaded 100s of thumbnails, and it needed to load a background, it would probably swap out all those thumbnails to make room for the background in the cache

I think Opus4 or Narflex could best answer how sage internally handles caching of images.

[PLUCKYHD]

Originally Posted by jreichen

The key of your hash map would be the string and the value would be the image. Here's some pseudocode:

Map<String, Image> imageMap = new WeakHashMap<String, Image>();

// add your images to the cache
...

// retrieve image
String imageName.....
Image image = imageMap.get(imageName);
if (image == null)
{
    // image was removed from cache,
    // retrieve image again and cache in the map again
    ...
}
// use the image
...

Makes more sense thanks.

Originally Posted by GKusnick

File and String are just two different ways of representing a file path (i.e. the location of the file on disk). Neither one of them contains the actual file contents, i.e. the pixels of the image, and that's presumably what you want to cache. So you'd probably need to do LoadImageFile() and cache the result of that to retain the actual image in memory.

Thanks Greg are you refering to the sage loadimage call or is there a java one. Would doing what jreichen recommended and using the java "Image"
achieve?

Originally Posted by stuckless

In Phoenix I do make extensive use of the weakhashmap for caching. But, that being said, I think plucky is looking for more of a "core" caching, since, he's already loaded the images, and then sage appears to throw them out of it's internal cache.

After grepping from 'cache' in the sage.properties, it returned these properties...

preload_expression_cache=false
ui/dump_font_cache=false
ui/enable_hardware_scaling_cache=false
ui/image_cache_min_num_allowance2=2
ui/system_memory_2dimage_cache_limit=20000000
ui/system_memory_2dimage_cache_size=16000000

You are right on what I am wanting to do will have to test out jriechen and greg's idea and see which one works for me. I am really just wanting a cache local and only really wanting it on client pc's as the problem comes in when getting images from unc locations (ie server) and it is only coverart I am trying to do this with not backgrounds. I will try changing those propertys as well and see if that has any effect I had never even noticed those before.


As always guys I really appreciate the help with the "noob"

cheers

[PLUCKYHD]

Originally Posted by stuckless

Also, if sage does an internal image cache, it probably doesn't weight what is being put into it. ie, if you loaded 100s of thumbnails, and it needed to load a background, it would probably swap out all those thumbnails to make room for the background in the cache

Also wanted to add I think this is exactly what it does which is unfortunate although I could be wrong but when loading a bunch of backgrounds by changing items an then scrolling is when the cache appears to lose the thubmnails/posters. That is way I am hoping by caching the thumbnails separately myself I can avoid this.

Hopefully Opus can chime in here as I am curious about those settings in the properties file as well and if using sage's loadimage forces those to stay in cache longer than calling a image without loading it.

cheers

[Narflex]

SageTV has limits on the amount of images it'll store in order to not chew up all the available memory. For Java buffered images; this is controlled by the ui/system_memory_2dimage_cache_size(limit) properties (if it exceeds the 'limit' it'll free images until it is below the 'size'). There's also 'raw' images which are allocated using native memory, not from the JVM's heap. That cache size is controlled by the same properties; but Java and raw images are tracked separately. Internally; SageTV will use the 'raw' images; but if you ever use an API call to get a BufferedImage; then it'll do the conversion (or if you're using the Java2D renderer; then it'll use BufferedImages internally). The 'raw' images use NIO ByteBuffer direct memory access for performance reasons (this is what the 'Native Image Loader' does if you recall the days before that existed).

So the simple explanation is just increase those property values if you want it to cache more images.

This is separate from caching that may be done in VRAM (for 3D renderers) or in the graphics memory on an extender client. There's another property for controlling the VRAM limit (ui/max_d3d_vram_usage on Windows) and the limit on extenders is not configurable. The VRAM limit will also be reduced if the graphics card indicates it has less memory than that setting. Images in these caches are not accessible by anything but the SageTV rendering engines though.

[PLUCKYHD]

Jeff,

Thanks for the explanation

ui/system_memory_2dimage_cache_limit=20000000
ui/system_memory_2dimage_cache_size=16000000

could you explain what those to defaults are is it 20mb and 16mb or is the first a number of images cached and the second 16mb?

and also the 2d or 3d limit or both are not configurable on extenders?

[Tiki]

Originally Posted by PLUCKYHD

Jeff,

Thanks for the explanation

ui/system_memory_2dimage_cache_limit=20000000
ui/system_memory_2dimage_cache_size=16000000

could you explain what those to defaults are is it 20mb and 16mb or is the first a number of images cached and the second 16mb?

and also the 2d or 3d limit or both are not configurable on extenders?

I think what Narflex meant was these two values are used together for hysteresis. If you exceed the "limit" (20,000,000), it will clear out memory until it gets back below the "size" (16,000,000).

Having two values like this avoids thrashing (otherwise once you hit the limit, it would constantly be dumping just enough to get below the limit). By dropping down a ways below the limit, it gives it some time to fill back up before it has to dump stuff again.

[PLUCKYHD]

That sounds right now onto if those values equal 16mb and 20mb..

Thanks for that tiki

[david1234]

Originally Posted by jreichen

Check out Java's WeakHashMap and Weak/Soft/PhantomReferences. It's been a while since I've used them so I don't have details off the top of my head, but using them is a good way to cache objects. They will hold references to objects unless the JVM needs to free memory for other objects.

This page looks like a good explanation.

Wow, did I learn something new today! Five years of working exclusively in Java, and I keep learning new things.

I'd never even heard of WeakRefernces, and trying to understand that led me to SoftReferences, and a really simple SoftHashMap implementation
http://java-interview-faqs.blogspot....ent-cache.html

[PLUCKYHD]

Okay so in trying to create the weakhashmap method and pass either a bufferedImage or Image back to sage I get this in the sage log

Thu 2/4 7:56:04.560 [BGLoader2-SAGETV_PROCESS_LOCAL_UI@1725455] ImageUtils creating BI copy BufferedImage@18d1b67: type = 5 ColorModel: #pixelBits = 24 numComponents = 3 color space = java.awt.color.ICC_ColorSpace@110bd74 transparency = 1 has alpha = false isAlphaPre = false ByteInterleavedRaster: width = 245 height = 343 #numDataElements 3 dataOff[0] = 2 w=245 h=343 freeMem=11065776 totalMem=113041408
Thu 2/4 7:56:04.560 [FinalRender-SAGETV_PROCESS_LOCAL_UI@1d1c3cd] Rendered new font to cache index=0 font=java.awt.Font[family=Arial,name=Arial,style=plain,size=33]

Over and Over again and nothing is every displayed I am guessing I am overloading it by trying to pass a buffered image directly and it is too much. So that route may not work. I could increase the properties files and tested but that is not really the results I am wanting.

[stuckless]

Plucky, I think that I'd need to see more of your code before I could offer help on this.

Typically, when you see log statement repeated over and over again... you've introduced some form of recusion into your pattern.

[PLUCKYHD]

Ask and you shall recieve

    private static void AddImageToHash(File ImageFile) throws IOException{
    Image current =  ImageIO.read(ImageFile);
    String location=ImageFile.toString();
    imageMap.put(location, current);}

    public static Image GetImageFromCache(File ImageFile) throws IOException{
    Image image = imageMap.get(ImageFile.toString());
    if (image == null){
    AddImageToHash(ImageFile);
    return GetImageFromCache(ImageFile);}
    else{return image;}}

That is the two test calls I made and I tried both Image and Buffered Image. Of course for the key I am just using the filelocation to a string to start out with. I logged it and it doesn't appear to be looping.

[stuckless]

Plucky... The fact that GetImageFromCache calls GetImageFromCache, means that there is potential for recursion

A couple of things...
1. WeakHashMaps are not guaranteed to have your data, even after a put... however unlikely the case, a call to put, could in theory return null on the next get.
2. Don't call yourself from within your method, unless you are really trying to implement recursion (ie, like directory scanning, etc).

So, GetImage... could be written as...

    public static Image GetImageFromCache(File ImageFile) throws IOException{
       Image image = imageMap.get(ImageFile.toString());
       if (image == null){
           Image = loadImage(ImageFile);
           imageMap.put(ImageFile.toString(), Image);
       }
       return Image;
}

btw... maps do not need to use strings as keys... so you could simply use the File object as the key if you wanted to.

[PLUCKYHD]

Originally Posted by stuckless

Plucky... The fact that GetImageFromCache calls GetImageFromCache, means that there is potential for recursion

A couple of things...
1. WeakHashMaps are not guaranteed to have your data, even after a put... however unlikely the case, a call to put, could in theory return null on the next get.
2. Don't call yourself from within your method, unless you are really trying to implement recursion (ie, like directory scanning, etc).

So, GetImage... could be written as...

    public static Image GetImageFromCache(File ImageFile) throws IOException{
       Image image = imageMap.get(ImageFile.toString());
       if (image == null){
           Image = loadImage(ImageFile);
           imageMap.put(ImageFile.toString(), Image);
       }
       return Image;
}

btw... maps do not need to use strings as keys... so you could simply use the File object as the key if you wanted to.

Yeah i knew a might have an issue with that looping problem but I didn't see any repeats in the log so I didn't worry much about it. Will try your way and see if I have better luck thanks again Sean.

[Narflex]

Also remember that WeakHashMap uses weak keys...not values....so if you're keying off a String or File and not retaining that reference anywhere else then the values in your map will get removed rather quickly. What you probably want to do instead is use a regular HashMap and then wrap the values you put in the Map with a WeakReference. But actually for your purposes; a SoftReference might be better unless you retain a strong reference to the images you're putting in the Map somewhere else. Be sure you have a good understanding of strong, soft and weak references and how the GC interacts with them; otherwise you'll probably get unexpected results when using the java.lang.ref stuff.

[stuckless]

That's a good point Jeff... and it's something that gets overlooked at times (myself included). Here's a simple overview of references in java. I think it explains the weak and soft references quite well... and don't even bother to read the section on phantom references, since you'll never use them.