Thursday, February 7, 2013

Example: Lazy Map and Least Recently Used (LRU) Map

In this post, I will demonstrate both an Apache Commons Collections Lazy Map and an Apache Commons Collections Least Recently Used (LRU) Map used in concert.

I had many ( and I mean many ) examples of a Lazy Map. The first example used Google SOAP Services for data aggregation, I wrote it almost three years ago. Sadly, the life cycle of the Google's SOAP services and my timeliness didn't mesh. By the time I was ready to post it, it no longer existed. Then I decided to use Google Base, so I spent a month writing it and decided the article was overly complicated and was more about Google Base then the Lazy Map. Frustrated, I wrote something that used HSQLDB, but when I was done it was over the top ( I'm still laughing about the complexity ). I put the whole project on the back burner for long time, and then I found inspiration again with a simple requirement while working at IBM. This was the first time I was able to use a LRU ( Least Recently Used ) map in a real world environment ( not just an example ). Enjoying the act of learning, I became rather excited to use it. This was a great opportunity to see the LRU map in action and explore the Lazy Map.

You may have heard the proverb if all you have is a hammer, everything looks like a nail. It means, single-minded people who have limited knowledge usually apply only one solution. Understanding the fundamentals of both the Java Collections Framework and the Apache Commons Framework will boh widen your skills and make you a far better asset than someone who does not. I knew of the LRU map and the Lazy Map, but I never used either in a real world scenario. I just didn't have a need for either of them. I’m sure someone without knowing about it would be able to write something in a day or worse two. Conversely, all I had to do was read the documents and I was up and using it in about an hour. If you are like me and enjoy learning new things, you can image my excitement when I dug deep into this class.

Naturally I can’t go into the details of the requirement that drove me to this class, but the objective was to keep track of values statically with keys. Arguably It could have been done with a static map or some kind of Singleton or any combination of the two. However, I had to get it done fast, the values (results of the operation) were expensive to manufacture, large, and the activity on the object would focus on a few keys at a time. Being concerned about space, I calculated the memory usage per value and set a limit to the size of the map (part of the LRU Map behavior). One of the well-designed aspects of the entire Apache Commons Framework is its use of the decorator pattern and the factory pattern. Unfortunately, the factory interface is completely un aware of the key. Making it useless for my needs. With the decorator pattern, which has access to the key, I could wrap the LRU Map with a Lazy Map. Expanding the functional characteristics a simple HashMap. The Lazy Map can "fill in" the expired values automatically and the LRU can "expire out" infrequently accessed values. Using a static Lazy Map, on the other hand, could, if unbounded, easily consume large amounts of memory.

I built an example that uses these two objects and will fetch US Stock Exchange Market Data (specifically NASDAQ) from Yahoo. Like all Maps, you need to be aware of Thread Safety  and use them with discretion. There are plenty of examples, explanations of concurrency and scalability issues with Maps, if you Google them. If you examine my example closely, you might take note of a serious short coming. I don't want to give it away, it might be fun for some to find it. Ill explain it near the end, and give some examples on how to over come it. I hope you enjoy this walk through let me know if you find it helpful.

The developers of Apache Commons Collections framework use the decorator pattern extensively, and if you do any Java Web development, you are familiar with this pattern. You can search through the code and find many references to a method called decorate. Decorating an object allows the user to customize and or chain the behavior to a specific need without impacting the original object. For example there is a decorate method on the TransformedMap object, which opens the door to many possibilities. it is worth noting that the Apache Commons Collections framework has changed recently so it is possible that something I'm writing about may have changed.

TransformedMap.decorate( )

Transformed Map is the ultimate transformer and I cover it a little more here. This class and method allows the user to manipulate the Key, Value, and or the backing object. So you could do something like this.

package com.blogspot.apachecommonstipsandtricks.lazymapexamples;
import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.map.TransformedMap;
import java.util.*;

public class DecoratingMaps {
    public static void main(String[] args) {
        List<g;tuple>  tuples = Arrays.asList( new tuple("a", "123"),
                                            new tuple("b", "456"),
                                            new tuple("c", "789"),
                                            new tuple("d", "012") );

        Map map = TransformedMap.decorate(new HashMap(), new Transformer() {
            public Object transform(Object o) {
                return ((tuple) o).getName();
            }
        }, new Transformer() {
            public Object transform(Object o) {
                return ((tuple) o).getValue();
            }
        });


        Iterator<tuple> iterator = tuples.iterator();
        while (iterator.hasNext()) {
            tuple tuple = iterator.next();
            map.put(tuple, tuple);
        }

        System.out.println(map.toString());
    }

    private static class tuple {
        String name;
        String value;

        private tuple(String name, String value) {
            this.name = name;
            this.value = value;
        }

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public String getValue() {
            return value;
        }

        public void setValue(String value) {
            this.value = value;
        }
    }
}

Which allows us to just Put an object in a map and let the map worry about what part is the key and which is the value. I've used this in the past and it allows me to ( with a factory ) build maps that know how to store the values alleviating the user from the burden of this knowledge, a paractice of seperation of concern. The code above produce a map like this

{d=012, b=456, c=789, a=123}

Now that I have shown you the transformed map class, what is a Lazy Map? A Lazy Map is simply a map that knows how to create or fetch a value based on the key. This is because the values may or may not be present in the map. So, there has to be some solid relationship between the value and key's uniqueness that it can calculate the value. The LRU or Least Recently Used map is typically a map with a static size that ages out content based on an algorithm for the purpose of keeping the object within a size limit ( or some other dimension such as time ). As I mentioned, this example uses Yahoo Stocks, so I abstracted the Yahoo commands into a Enum. The purpose of this enum is to bridge the gap between commands sent to the service and human readability. Here is the enum.


package com.blogspot.apachecommonstipsandtricks.lazymapexamples;

public enum YahooFinanaceCommands
{
    a("Ask"),
    b("Bid"),
    b4("Book Value"),
    c1("Change"),
    c8("After Hours Change (Real-time)"),
    d2("Trade Date"),
    e7("EPS Estimate Current Year"),
    f6("Float Shares"),
    j("52-week Low"),
    g3("Annualized Gain"),
    g6("Holdings Gain (Real-time)"),
    j1("Market Capitalization"),
    j5("Change From 52-week Low"),
    k2("Change Percent (Real-time)"),
    k5("Percebt Change From 52-week High"),
    l2("High Limit"),
    m2("Day's Range (Real-time)"),
    m5("Change From 200-day Moving Average"),
    m8("Percent Change From 50-day Moving Average"),
    o("Open"),
    p2("Change in Percent"),
    q("Ex-Dividend Date"),
    r2("P/E Ratio (Real-time)"),
    r7("Price/EPS Estimate Next Year"),
    s7("Short Ratio"),
    t7("Ticker Trend"),
    v1("Holdings Value"),
    w1("Day's Value Change"),
    y("Dividend Yield"),
    a2("Average Daily Volume"),
    b2("Ask (Real-time)"),
    b6("Bid Size"),
    c3("Commission"),
    d("Dividend/Share"),
    e("Earnings/Share"),
    e8("EPS Estimate Next Year"),
    g("Day's Low"),
    k("52-week High"),
    g4("Holdings Gain"),
    i("More Info"),
    j3("Market Cap (Real-time)"),
    j6("Percent Change From 52-week Low"),
    k3("Last Trade Size"),
    l("Last Trade (With Time)"),
    l3("Low Limit"),
    m3("50-day Moving Average"),
    m6("Percent Change From 200-day Moving Average"),
    n("Name"),
    p("Previous Close"),
    p5("Price/Sales"),
    r("P/E Ratio"),
    r5("PEG Ratio"),
    s("Symbol"),
    t1("Last Trade Time"),
    t8("1 yr Target Price"),
    v7("Holdings Value (Real-time)"),
    w4("Day's Value Change (Real-time)"),
    a5("Ask Size"),
    b3("Bid (Real-time)"),
    c("Change & Percent Change"),
    c6("Change (Real-time)"),
    d1("Last Trade Date"),
    e1("Error Indication (returned for symbol changed / invalid)"),
    e9("EPS Estimate Next Quarter"),
    h("Day's High"),
    g1("Holdings Gain Percent"),
    g5("Holdings Gain Percent (Real-time)"),
    i5("Order Book (Real-time)"),
    j4("EBITDA"),
    k1("Last Trade (Real-time) With Time"),
    k4("Change From 52-week High"),
    l1("Last Trade (Price Only)"),
    m("Day's Range"),
    m4("200-day Moving Average"),
    m7("Change From 50-day Moving Average"),
    n4("Notes"),
    p1("Price Paid"),
    p6("Price/Book"),
    r1("Dividend Pay Date"),
    r6("Price/EPS Estimate Current Year"),
    s1("Shares Owned"),
    t6("Trade Links"),
    v("Volume"),
    w("52-week Range"),
    x("Stock Exchange");

    private String description;

    YahooFinanaceCommands(String description)
    {
        this.description = description;
    }
}


Disclosure and credit needs to be in order here. First, I am not associated with Yahoo, NASDAQ, or NYSE in any form, so you are on your own with this and second I got all of these commands from the Website http://www.gummy-stuff.org/Yahoo-data.htm. Furthermore, Yahoo has a throttle on this service, and if you exceed it, you will be locked out for some time and even banned. You need to read the usage disclosures of Yahoo, NASDAQ, and NYSE.

So, as you can guess, I also needed to get a list of keys for testing and reference ( the Stock Symbols ). This was an daunting task, it took me hours of searching via Google before I found at least one exchange that had the symbols publicly accessible and downloadable. Again, a little disclosure, I am not associated with NASDAQ or the NYSE, please don't ask me for help. Although I got the NYSE working, I thought it easier to just demo NASDAQ and that is why there is code commented out. Before I get to deep, lets recap our objective. We need the keys for the NASDAQ, build a Lazy Map with a LRU of the stock values based on Yahoo stock service and some concurrency modification protection with the ConcurrentHashMap.

With the enum I can now assemble the URN of a URL for the purpose of pulling stocks from Yahoo and then based on the results, fill out a object called Stock. This is what the Stock object will look like.

package com.blogspot.apachecommonstipsandtricks.lazymapexamples.entity;

public class Stock implements Cloneable
{
    private String symbol;
    private String name;
    private Float fifetytwoWeekLow;
    private Float fifetytwoWeekHigh;
    private Float peRatioRealTime;
    private Float priceEPSEstimateNextYear;

    public Stock()
    {
    }

    public Stock(String symbol, String name, Float fifetytwoWeekLow, Float fifetytwoWeekHigh, Float peRatioRealTime,
                 Float priceEPSEstimateNextYear)
    {
        this.symbol = symbol;
        this.name = name;
        this.fifetytwoWeekLow = fifetytwoWeekLow;
        this.fifetytwoWeekHigh = fifetytwoWeekHigh;
        this.peRatioRealTime = peRatioRealTime;
        this.priceEPSEstimateNextYear = priceEPSEstimateNextYear;
    }

    public String getSymbol()
    {
        return symbol;
    }

    public void setSymbol(String symbol)
    {
        this.symbol = symbol;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }

    public Float getFifetytwoWeekLow()
    {
        return fifetytwoWeekLow;
    }

    public void setFifetytwoWeekLow(Float fifetytwoWeekLow)
    {
        this.fifetytwoWeekLow = fifetytwoWeekLow;
    }

    public Float getFifetytwoWeekHigh()
    {
        return fifetytwoWeekHigh;
    }

    public void setFifetytwoWeekHigh(Float fifetytwoWeekHigh)
    {
        this.fifetytwoWeekHigh = fifetytwoWeekHigh;
    }

    public Float getPeRatioRealTime()
    {
        return peRatioRealTime;
    }

    public void setPeRatioRealTime(Float peRatioRealTime)
    {
        this.peRatioRealTime = peRatioRealTime;
    }

    public Float getPriceEPSEstimateNextYear()
    {
        return priceEPSEstimateNextYear;
    }

    public void setPriceEPSEstimateNextYear(Float priceEPSEstimateNextYear)
    {
        this.priceEPSEstimateNextYear = priceEPSEstimateNextYear;
    }

    @Override
    public Object clone() // throws CloneNotSupportedException
    {
        return new Stock(this.getSymbol(), this.getName(), this.getFifetytwoWeekLow(), this.getFifetytwoWeekHigh(),
                         this.getPeRatioRealTime(), this.getPriceEPSEstimateNextYear());
    }

    @Override
    public String toString()
    {
        return "Stock{" +
               "symbol='" + symbol + '\'' +
               ", name='" + name + '\'' +
               ", fifetytwoWeekLow=" + fifetytwoWeekLow +
               ", fifetytwoWeekHigh=" + fifetytwoWeekHigh +
               ", peRatioRealTime=" + peRatioRealTime +
               ", priceEPSEstimateNextYear=" + priceEPSEstimateNextYear +
               '}';
    }
}

So I will be requesting from the Yahoo Service a 52 week high and low value, the Price to Earning ratio ( Real Time ) and the Price EPS estimated next year. I will then put the results in the Stock object and store it in my LRU map.

Are you beginning to see a problem yet? If not, that is ok, let me explain. The LRU map disposes not just the value in the map, it also disposes both the value and the KEY.. pause .. the key too ..  pause .. So, if you wanted to get all keys via map.keySet() and then iterator through all the values, forget it. If your knee jerk reaction is to yell FOUL, think about it, it makes sense. The purpose is to reduce the memory foot print, iterating through the whole map completely undermines the LRU as it will have to go get everything and then throw out the results. But in our example, we know all the keys ( even if they change when a company is sold or changes it's name ). What to do.. gluing functionality together can be a real challenge, but fun, so lets get gluing!

So thinking out loud, we cant just use the LRU and Lazy Map, the final object will have to have some functionality beyond a map. Naturally a singleton utility class, we will call it StockFetcher. Stock Fetcher will both encapsulate the LRU / Lazy Map / Concurrent Hash Map, cache the Stock Symbols and age them out,  provides a get Stock method, and provide an iterator to go over the Stock symbols it knows about. Here it is.....

package com.blogspot.apachecommonstipsandtricks.lazymapexamples;

import com.blogspot.apachecommonstipsandtricks.lazymapexamples.entity.Stock;
import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.map.LRUMap;
import org.apache.commons.collections.map.LazyMap;
import org.apache.commons.csv.CSVParser;
import org.apache.commons.csv.CSVStrategy;
import org.apache.commons.lang.StringUtils;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.StringReader;
import java.net.URL;

import java.util.concurrent.ConcurrentHashMap;
import java.util.*;

public class StockFetcher
{
    private static final int MaxCacheSize = 100;
    private static final long MilliSecondsToFlush = ( ( 1000L * 60L ) * 60L )* 24L;
    private static Timer timer = new Timer();

    // I broke these out, so you could easily modify the values without having to dig in the code.
    public static final YahooFinanaceCommands[] YahooCommands = new YahooFinanaceCommands[]{YahooFinanaceCommands.s,
            YahooFinanaceCommands.n,
            YahooFinanaceCommands.j,
            YahooFinanaceCommands.k,
            YahooFinanaceCommands.r2,
            YahooFinanaceCommands.r7};

    private static final Map<String, Stock> map = LazyMap.decorate(new ConcurrentHashMap(new LRUMap(MaxCacheSize)),
                                                                   new StockMarketFactoryTransformer());

    private static final List<String> KNOWN_STOCK_SYMBOLS = new ArrayList<String>();
    private static final String FTP_SYMBOLDIRECTORY_NASDAQ = "ftp://ftp.nasdaqtrader.com/symboldirectory/nasdaqlisted.txt";
    // private static final String FTP_SYMBOLDIRECTORY_NYSE= "http://www.nyse.com/attachment/activated_lrps.xls";
    // private static final String FTP_SYMBOLDIRECTORY_NYSE= "http://online.wsj.com/public/resources/documents/NYSE.csv";

    private static final StockFetcher ourInstance = new StockFetcher();

    /**
     * This is a singleton.
     */
    private StockFetcher()
    {
        fetchKeys();
    }

    /**
     * refer to singleton pattern
     * @return the static instance of this class.
     */
    public static StockFetcher getInstance()
    {
        return ourInstance;
    }

    /**
     * Method responsible for fetching all the NASDAQ Stocks symbols and setting up a timmer to do it again.
     */
    private static synchronized void fetchKeys()
    {
        timer.schedule(new KeyFlushTimerTask(), MilliSecondsToFlush);
        System.out.println("Fetching keys");
        int count = 0;
        try
        {
            URL url = new URL(FTP_SYMBOLDIRECTORY_NASDAQ);
            BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
            String str;
            // nice thing about doing the two in.readlines is it skips the header.
            if (((str = in.readLine()) != null))
            {
                while ((str = in.readLine()) != null)
                {
                    String[] strings = str.split("\\|");
                    if (strings.length == 6)
                    {
                        count++;
                        KNOWN_STOCK_SYMBOLS.add(strings[0]);
                    }
                }
            }
            in.close();
        }
        catch (IOException e)
        {
            throw new IllegalStateException("An exception occured while attempting to fetch the Stock quote.", e);
        }
        finally
        {
            System.out.println(" Read " + count + " symbols.");
        }
    }

    /**
     * Takes an Array of Yahoo Finanace comamnds and assembles the URL string that will be sent to Yahoo.
     * @param yfcs a vararg of YahooFinanaceCommands to send to yahoo.
     * @return a URL used to send to Yahoo.
     */
    private static String compileCommands(YahooFinanaceCommands... yfcs)
    {
        StringBuilder sb = new StringBuilder();
        if ( yfcs != null)
        {
            for (YahooFinanaceCommands yfc : yfcs)
            {
                sb.append(yfc.toString());
            }
        }
        if (sb.length() != 0)
        {
            return "&f=" + sb.toString();
        }
        return "";
    }

    /**
     * Builds a URN
     * @param symbols a var arg of symbols to fetch.
     * @return the urn
     */
    private static String query(String... symbols)
    {
        return StringUtils.join(symbols, "+");
    }

    /**
     * Get a stock based on it's nasdq symbol
     * @param symbol the nasdaq symbol
     * @return a stock object.
     */
    public Stock get(String symbol)
    {
        Stock aStock = map.get(symbol);
        return (aStock==null)?null:(Stock)aStock.clone();
    }

    /**
     * Iterates all the known stock symbols.
     * @return String interator.
     */
    public Iterator<String> iterator()
    {
        return ( new ArrayList<String>( KNOWN_STOCK_SYMBOLS ) ).iterator();
    }

    private static class StockMarketFactoryTransformer implements Transformer
    {
        public Object transform(Object o)
        {
            System.out.println("Fetching data for " + (String) o );
            Stock retStock = null;
            try
            {
                URL url = new URL("http", "finance.yahoo.com", 80,
                                  "/d/quotes.csv?s=" + query((String) o) + compileCommands(YahooCommands));
                BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
                String str;
                if (((str = in.readLine()) != null))
                {
                    do
                    {
                        CSVParser csvParser = new CSVParser(new StringReader(str),
                                                            CSVStrategy.DEFAULT_STRATEGY);

                        String[] strings = StringUtils.stripAll(csvParser.getLine());
                        // System.out.println(Arrays.toString(strings));
                        int i = 0;
                        retStock = new Stock();
                        retStock.setSymbol(strings[i++]);
                        retStock.setName(strings[i++]);
                        retStock.setFifetytwoWeekLow(safeFloatConvert(strings[i++]));
                        retStock.setFifetytwoWeekHigh(safeFloatConvert(strings[i++]));
                        retStock.setPeRatioRealTime(safeFloatConvert(strings[i++]));
                        retStock.setPriceEPSEstimateNextYear(safeFloatConvert(strings[i]));
                    }
                    while ((str = in.readLine()) != null);
                }
                in.close();
            }
            catch (IOException e)
            {
                throw new IllegalStateException("An exception occured while attempting to fetch the Stock quote.", e);
            }
            return retStock;
        }

        private Float safeFloatConvert(String string)
        {
            Float retFloat = null;
            if (!"N/A".equalsIgnoreCase(string))
            {
                try
                {
                    retFloat = Float.valueOf(string);
                }
                catch (NumberFormatException e)
                {
                    e.printStackTrace();
                }
            }
            return retFloat;
        }
    }
    
    private static class KeyFlushTimerTask extends TimerTask
    {
        public void run()
        {
            fetchKeys();
        }
    }
}

The variable MaxCacheSize allows us to set the maximum size of the LRU map ( up to 100 entities in this example ) while MilliSecondsToFlush is used by a java timer task ( see static Timer timer ) defined near the end of the Stock Fetcher class as KeyFlushTimerTask which extends TimerTask. This object will evict all the stock symbols in the KNOWN_STOCK_SYMBOLS ( not the map which you might want to do if it suits your needs ).

    private static final int MaxCacheSize = 100;
    private static final long MilliSecondsToFlush = ( ( 1000L * 60L ) * 60L )* 24L;
    private static Timer timer = new Timer();

So following those three lines is the following line..

    // I broke these out, so you could easily modify the values without having to dig in the code.
    public static final YahooFinanaceCommands[] YahooCommands = new YahooFinanaceCommands[]{YahooFinanaceCommands.s,
            YahooFinanaceCommands.n,
            YahooFinanaceCommands.j,
            YahooFinanaceCommands.k,
            YahooFinanaceCommands.r2,
            YahooFinanaceCommands.r7};

What I've done here is assembled a static array of YahooFinanaceCommands which will represent the command ( URN of the URL ) I will be using to call Yahoo with ( see the YahooFinanaceCommands enum above ).

So now onto the magic of the whole tutorial, the glue of the whole system. We will now decorate the map.

    private static final Map<String, Stock> map = LazyMap.decorate(
                                                            new ConcurrentHashMap(new LRUMap(MaxCacheSize)),
                                                                   new StockMarketFactoryTransformer());


What we have done so far is call the decorate method on the Apache Commons Lazy Map object backing it with a ConcurrentHashMap which in turn is backed by a LRU Map defined to the size of MaxCacheSize with a transformer called StockMarketFactoryTransformer. It is perfectly acceptable to Pump Your Fist and yell OH YEAH once this sinks in! So our new map will take on the characteristics of three different Map Objects ( LRUMap, ConcurrentHashMap and LRUMap ). Chaining them all together to create a whole new kind of Map. A Map, that behaves exactly as I want it to again:

Pump Fist and yell OH YEAH!

Special Note here regarding ConcurrentHashMap and maps in general. Naturally maps are not thread safe and using a ConcurrentHashMap will impact the performance, but for our purpose this is all acceptable. It might not be in your environment so give it some consideration before you copy-cut and paste this code.

Moving on, you will have seen KNOWN_STOCK_SYMBOLS, FTP_SYMBOLDIRECTORY_NASDAQ, and the commented out URLs called FTP_SYMBOLDIRECTORY_NYSE. There is no more hiding it; the problem with this code/pattern involves knowing what the keys are. In an attempt to capture the keys (which are the stock symbols), I will hold the stock symbols in FTP_SYMBOLDIRECTORY_NASDAQ and since this is a singleton, it will happen once in the private constructor when fetchKeys is fired off ( and by the way, this is when the timer task schedules itself ). I know this solution is not perfect, eventually the Stock market keys will be out of sync as companies come and go. Not to mention the problems and complexity associated with the file at FTP_SYMBOLDIRECTORY_NASDAQ, but I never made a claim that this code is perfect. In most cases, it is just good enough. So to re-iterate, the singleton is fired up, the function fetchKeys is fired which will fetch all the symbols in the file stored on the NASDAQ server and adding all of them to the array KNOWN_STOCK_SYMBOLS and scheduling the timer to fire the method again. I have exposed the keys via an iterator method further down, but as I mentioned earlier this defeats the whole purpose of the LRU Map if you iterate over them.

The function compileCommands takes a Java 1.5 varargs of YahooCommands to build part of the URN to the URL of the Yahoo Server. I did this so, you can add more values if you desire. This function will be used later when we pull the stock value. The values are parsed then stored in the Stock object and placed in the Lazy Map / ConcurrentHashMap / LRUMap ( shweeeet ). All of this is done by the inner class StockMarketFactoryTransformer. This object implements transformer and was injected in the LazyMap Decorate method.

    private static class StockMarketFactoryTransformer implements Transformer
    {
        public Object transform(Object o)
        {
            System.out.println("Fetching data");
            Stock retStock = null;
            try
            {
                URL url = new URL("http", "finance.yahoo.com", 80,
                                  "/d/quotes.csv?s=" + query((String) o) + compileCommands(YahooCommands));
                BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
                String str;
                if (((str = in.readLine()) != null))
                {
                    do
                    {

                        CSVParser csvParser = new CSVParser(new StringReader(str),
                                                            CSVStrategy.DEFAULT_STRATEGY);

                        String[] strings = StringUtils.stripAll(csvParser.getLine());
                        // System.out.println(Arrays.toString(strings));
                        int i = 0;
                        retStock = new Stock();
                        retStock.setSymbol(strings[i++]);
                        retStock.setName(strings[i++]);
                        retStock.setFifetytwoWeekLow(safeFloatConvert(strings[i++]));
                        retStock.setFifetytwoWeekHigh(safeFloatConvert(strings[i++]));
                        retStock.setPeRatioRealTime(safeFloatConvert(strings[i++]));
                        retStock.setPriceEPSEstimateNextYear(safeFloatConvert(strings[i]));
                    }
                    while ((str = in.readLine()) != null);
                }
                in.close();
            }
            catch (IOException e)
            {
                throw new IllegalStateException("An exception occured while attempting to fetch the Stock quote.", e);
            }
            return retStock;
        }

        private Float safeFloatConvert(String string)
        {
            Float retFloat = null;
            if (!"N/A".equalsIgnoreCase(string))
            {

                try
                {
                    retFloat = Float.valueOf(string);
                }
                catch (NumberFormatException e)
                {
                    e.printStackTrace();
                }
            }
            return retFloat;
        }
    }


So here is a basic example of it's usage and honestly not a good example. In this example, we are in a infinite loop, the first time we instantiate the StockFetcher via getInstance it loads all the keys ( stock symbols from nasdaq ). Then we call get AAPL and since it is not in the cache the transformer executes and pulls the stock and stores it. Further we sleep for a few seconds, and then attempt to get the stock XXIA, again it doesn't exist so it goes and gets it... while in the loop we make the same calls to AAPL and XXIA and since they are in the cache, we hit the cache. Eventually the stocks are evicted from the map if more than 100 unique hits occur on the map.

package com.blogspot.apachecommonstipsandtricks.lazymapexamples;

import com.blogspot.apachecommonstipsandtricks.lazymapexamples.entity.Stock;


public class StockTickerTest
{
    public static void main(String[] args)
    {
        do{
            try
            {
                Stock stock = StockFetcher.getInstance().get("AAPL");
                System.out.println("stock = " + stock);
                System.out.println("going to sleep.");
                Thread.sleep(10000);
                stock = StockFetcher.getInstance().get("XXIA");
                System.out.println("stock = " + stock);
                System.out.println("going to sleep.");
                Thread.sleep(10000);
            }
            catch (InterruptedException e)
            {
                e.printStackTrace();
            }

        }while(true);

    }

}


I hope you enjoyed this and found it helpful. Please feel free to send me comments or suggestions.
Author: Philip Senger