Tuesday, March 31, 2009

Examples of Lazy Lists and Factories

In this post, I'm going to examine Lazy Lists and the Apache Factory.

In accordance with most of the Apache Commons Collection utilities, a LazyList is simply a decorator (and at this point that shouldn't be a surprise). It decorates another List creating a sort-of "list on demand" list. Calling the method get(int index) when the index is greater than the size of the list will grow the list with objects from the given factory. Using the get(int index) method to get an object that the factory did not fill in will result in an IndexOutOfBoundsException.

For all of the examples, lets use the following simple factory.
The Code:

package com.blogspot.apachecommonstipsandtricks.lazymapexamples;
import java.util.*;
import org.apache.commons.collections.*;
public class LazyListFactory implements Factory
{
private static int i = 0;
private List<String> strings = new ArrayList();
public LazyListFactory(List<String> strings)
{
this.strings.addAll(strings);
}
public Object create()
{
return this.strings.get(i++);
}
}

In this example Java code we have implemented the Factory interface, which defines a Object create(), but more importantly the constructor takes as an argument a List of Strings. These strings are effectively the "feed" that the create method will use. A more practical solution would be some form of a data source, but that would result in a lot of code, and this is simple. So with this factory, let us create some lazy data!

Our first java example of a lazy list.
The Code:

package com.blogspot.apachecommonstipsandtricks.lazymapexamples;
import java.util.*;
import org.apache.commons.collections.list.*;
public class ListManager
{
public static void main(String[] args)
{
// First lets create a simple list "A" through "C"
List<String> backingList = new ArrayList<String>( Arrays.asList("A","B","C") );
// Now we create a Decorated List with homegrown class called LazyListFactory which implements Factory
List<String> list = LazyList.decorate( GrowthList.decorate( backingList ), new LazyListFactory( Arrays.asList("D","E","F","G") ) );
System.out.println("The size method on list thinks there are " + list.size() + " elements in the array list. but Im going to get 7!");
// as the more and more elements are requested, the factory creates more..
for (int i = 0; i < 7; i++)
{
Object o = list.get(i);
System.out.println( " Element [" + i + "] = " + o );
}
}
}


Results in

The size method on list thinks there are 3 elements in the array list. but Im going to get 7!
Element [0] = A
Element [1] = B
Element [2] = C
Element [3] = D
Element [4] = E
Element [5] = F
Element [6] = G


In this example, we first created a list, with Strings "A","B", and "C" than we decorated it, and attached a factory constructed with four more additional strings "D" through "G". I knew there was seven items in the list, so I forced it to loop to seven. Did you notice, the list thought the size was only 3. You might be thinking, which I am too, that’s not very helpful. How can I tell how much to loop through? Let’s look at this same example, but with some minor modifications.

The Code:

package com.blogspot.apachecommonstipsandtricks.lazymapexamples;
import java.util.*;
import org.apache.commons.collections.list.*;
public class ListManager
{
public static void main(String[] args)
{
// First lets create a simple list "A" through "C"
List<String> backingList = new ArrayList<String>( Arrays.asList("A","B","C") );
// Now we create a Decorated List with homegrown class called LazyListFactory which implements Factory
List<String> list = LazyList.decorate( GrowthList.decorate( backingList ), new LazyListFactory( Arrays.asList("D","E","F","G") ) );

// Now puit "Z" in position 15
list.set( 15, "Z" );

System.out.println("After putting \"Z\" in position 15, the size is equal to " + list.size());

System.out.println("Which allows us to us a standard for-loop with a IndexOutOfBoundsException try-catch block.");
for (int i = 0; i < list.size(); i++)
{
try
{
Object o = list.get(i);
System.out.println( " Element [" + i + "] = " + o );
} catch (IndexOutOfBoundsException e) {}
}

}
}


You might have immediately noticed I put another String, "Z" at position 15, and now, the size is 15. I find it likely that this might be a solution to the "unknown size problem". The problem statement might go something like this, create a LazyList and set the last element to a known end state. Therefore, the user could pick what they wanted to consume out of the list while knowing the end is near at 15.

The results of the program above would look like this.

After putting "Z" in position 15, the size is equal to 16
Which allows us to us a standard for-loop with a IndexOutOfBoundsException try-catch block.
Element [0] = A
Element [1] = B
Element [2] = C
Element [3] = D
Element [4] = E
Element [5] = F
Element [6] = G
Element [15] = Z

If you are hung up on swallowing the IndexOutOfBoundsException exception or simply don’t like the idea of having to put in the last element this next solution may suite you. It uses the Java 1.5 for-loop.

The Code:

package com.blogspot.apachecommonstipsandtricks.lazymapexamples;
import java.util.*;
import org.apache.commons.collections.list.*;
public class ListManager
{
public static void main(String[] args)
{
// First lets create a simple list "A" through "C"
List<String> backingList = new ArrayList<String>( Arrays.asList("A","B","C") );
// Now we create a Decorated List with homegrown class called LazyListFactory which implements Factory
List<String> list = LazyList.decorate( GrowthList.decorate( backingList ), new LazyListFactory( Arrays.asList("D","E","F","G") ) );

// Now puit "Z" in position 15
list.set( 15, "Z" );

System.out.println("With a Java 1.5 for-loop the results look like this." );
int i = 0;
for (String s : list)
{
System.out.println( " Element [" + i++ + "] = " + s );
}
}
}


Like every solution, there is a draw back to this method, when an element does not exist, the result is null. In this example, I print every element, including the nulls. Let's examine the output.

With a Java 1.5 for-loop the results look like this.
Element [0] = A
Element [1] = B
Element [2] = C
Element [3] = null
Element [4] = null
Element [5] = null
Element [6] = null
Element [7] = null
Element [8] = null
Element [9] = null
Element [10] = null
Element [11] = null
Element [12] = null
Element [13] = null
Element [14] = null
Element [15] = Z


Each solution I have covered has some form of a draw back. I hope you can find one suitable for you. I will be examining the LazyMap in my next installment; it is well suited for hierarchical data.


Thanks for reading this article, I look forward to any questions or feedback. Please drop me a line if you want me to cover something in particular or would like to point something useful out.
Author: Philip A Senger

Tuesday, February 17, 2009

Examples of Bag and MultiKeys

In this installment, I’m going to discuss Java usages tips and tricks for the Apache Commons Collection Bag interface. Honestly, the first time I saw the Apache Commons Collection Bags API, I thought "Well, that isn't very useful." The Bag interface states the intention of the class; it is to count occurrences of objects. Like all the products in the Apache Commons Collections, Predicates, Closures, and Transformers can be plugged into the bag at run time to determine the behavior. I ran some tests, which are at the end of the article. The cost of using the framework is negligible, at about 10 to 20 milliseconds. This of course is compared to a traditional method of sampling counting occurrences and stuffing into a map. You can be the judge of the test, let me know if you think the test is accurate.

So, what kind of problem justifies adding the complexity of Bags to a project? Joining the magic of a Transformer, MultiKey, and a decorator you can produce a dynamic solution for counting groups of occurrences of properties of beans in a collection. This would be suitable for a generic controller responsible for counting groups of properties. It would be very helpful in a reporting system. Let’s take a look at the sample and walk through the code.

The Code:

package com.blogspot.apachecommonstipsandtricks.bags;
import java.util.*;
import java.net.*;
import java.io.*;
import java.lang.reflect.*;
import org.apache.commons.collections.*;
import org.apache.commons.collections.comparators.*;
import org.apache.commons.collections.keyvalue.*;
import org.apache.commons.collections.bag.*;
import org.apache.commons.collections.bag.HashBag;
import org.apache.commons.lang.*;
import org.apache.commons.beanutils.*;
import com.blogspot.apachecommonstipsandtricks.*;
public class MagicBagOfTricks
{
public static void main(String[] args) throws IOException, InvocationTargetException, NoSuchMethodException, IllegalAccessException
{
List<LaborForce> list = new ArrayList<LaborForce>();
URL url = new URL("http", "1796193846474123283-a-1802744773732722657-s-sites.googlegroups.com", 80, "/site/psenger/Home/data.txt");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
if (((str = in.readLine()) != null))
{
do
{
String[] strings = str.split("\t");
strings = StringUtils.stripAll(strings);

State state = State.valueOf(StringUtils.trim(strings[0]));
Gender gender = "Male".equals(StringUtils.trim(strings[1])) ? Gender.Male : Gender.Female;
Integer year = Integer.valueOf(StringUtils.trim(strings[2]));

list.add(new LaborForce(state, gender, year));
} while ((str = in.readLine()) != null);
}
in.close();

Map map = PropertyUtils.describe(new LaborForce());
Set s = map.keySet();
s.remove("class");
System.out.println("All the possible properties on the LaborForce bean are [" + StringUtils.join(s.toArray(),",") + "]");

Bag masterBag = new HashBag();
Bag genderBag = TransformedBag.decorate(masterBag, new PropertiesMultiKeyTransformer( new String[]{ "gender" } ) );
Bag genderYearBag = TransformedBag.decorate(masterBag, new PropertiesMultiKeyTransformer( new String[]{ "gender", "year" } ) );
Bag genderYearStateBag = TransformedBag.decorate(masterBag, new PropertiesMultiKeyTransformer( new String[]{ "gender", "year" , "state" } ) );

genderBag.addAll( list );
genderYearBag.addAll( list );
genderYearStateBag.addAll( list );

Comparator comparator = ComparatorUtils.chainedComparator(new Comparator[]{new MultiKeyCompartor(0),new MultiKeyCompartor(1),new MultiKeyCompartor(2)});
Set<MultiKey> set = new TreeSet<MultiKey>(comparator);
set.addAll(masterBag.uniqueSet());
for (MultiKey multiKey : set)
{
System.out.println( "[" +StringUtils.join(multiKey.getKeys(),',') + "] = " + masterBag.getCount(multiKey));
}
}
private static class PropertiesMultiKeyTransformer implements Transformer
{
String[] methodNames;
private PropertiesMultiKeyTransformer(String[] methodNames)
{
this.methodNames = methodNames;
}
public Object transform(Object o)
{
List<Object> ooos = new ArrayList<Object>();
for (String methodName : methodNames)
{
try
{
ooos.add(PropertyUtils.getProperty(o, methodName));
}
catch (Exception e)
{
throw new FunctorException(e);
}
}
return new MultiKey(ooos.toArray(new Object[ooos.size()]));
}
}
private static class MultiKeyCompartor implements Comparator<MultiKey>
{
private int i;
private MultiKeyCompartor(int i)
{
this.i = i;
}
public int compare(MultiKey o1, MultiKey o2)
{
Object[] keys1 = o1.getKeys();
Object[] keys2 = o2.getKeys();
Object oo1 = null;
try
{
oo1 = keys1[i];
}
catch (ArrayIndexOutOfBoundsException e)
{
}
Object oo2 = null;
try
{
oo2 = keys2[i];
}
catch (ArrayIndexOutOfBoundsException e)
{
}
NullComparator nullComparator = new NullComparator(false);
return nullComparator.compare(oo1, oo2);
}
}
}


Between lines 18 and 36 of the MagicBagOfTricks class, we fetch the sample data. The data is in the format of StatetabGendertabyear. If you want to download the data, you can get it here.


List<LaborForce> list = new ArrayList<LaborForce>();
URL url = new URL("http", "1796193846474123283-a-1802744773732722657-s-sites.googlegroups.com", 80, "/site/psenger/Home/data.txt");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
if (((str = in.readLine()) != null))
{
do
{
String[] strings = str.split("\t");
strings = StringUtils.stripAll(strings);

State state = State.valueOf(StringUtils.trim(strings[0]));
Gender gender = "Male".equals(StringUtils.trim(strings[1])) ? Gender.Male : Gender.Female;
Integer year = Integer.valueOf(StringUtils.trim(strings[2]));

list.add(new LaborForce(state, gender, year));
} while ((str = in.readLine()) != null);
}
in.close();


The purpose of code on line 38 through 41 is to just print the properties of the bean, LaborForce, minus the class method. It is important to see how the PropertyUtils describes the bean, because we will be accessing it later via the same API.


Map map = PropertyUtils.describe(new LaborForce());
Set s = map.keySet();
s.remove("class");
System.out.println("All the possible properties on the LaborForce bean are [" + StringUtils.join(s.toArray(),",") + "]");


On line 43, we create a single HashBag object. I will refer to this HashBag object as the masterBag. A HashBag is an implementation of Bag with the characteristics of a HashSet. When objects are stored in the HashBag, the inserted object’s hash code is assessed, just like a HashSet, and ensures the uniqueness of the object in the Bag as it is stored. You might think the count would be 1 no mater how many times you insert the object, but in fact, the bag counts the inserted and removed occurrences. Even more confusing is the fact that there is an iterator method that returns all multiple occurrences and a uniqueSet method that pulls a set of objects. Line 44 through 46, is where the magic happens. We create three decorated Bags, with TransformedBag.decorate. All of them with variations of the PropertiesMultiKeyTransformer and backed by a single HashBag, the masterBag from line 43. When an object is inserted into one of the decorated HashBags, it is transformed by PropertiesMultiKeyTransformer and the results are physically placed in the masterBag. PropertiesMultiKeyTransformer uses the PropertyUtils to pull all the given properties off the bean to create a MultiKey object. It’s important that the we always put the same property in each object index in the array of objects of the MultiKey. If we didn’t, we would get an exception. Furthermore, we can’t transform the object into an array of objects because HashCode doesn’t exist on an Array of Objects. This becomes a problem when we insert Objects into the bag. The MultiKey is a perfect fit for this problem and that is the reason for its use. This class is effectively a wrapper class.


Bag masterBag = new HashBag();
Bag genderBag = TransformedBag.decorate(masterBag, new PropertiesMultiKeyTransformer( new String[]{ "gender" } ) );
Bag genderYearBag = TransformedBag.decorate(masterBag, new PropertiesMultiKeyTransformer( new String[]{ "gender", "year" } ) );
Bag genderYearStateBag = TransformedBag.decorate(masterBag, new PropertiesMultiKeyTransformer( new String[]{ "gender", "year" , "state" } ) );


On line 48 through 50 we add the object to the decorated transformed bags, which are all backed by the masterBag.


genderBag.addAll( list );
genderYearBag.addAll( list );
genderYearStateBag.addAll( list );


On line 52 through 58 I ask the masterBag for a set, load the results in a TreeSet that has a chained Transformer. The results are a sorted set of MultiKeys with counts of the occurrences of the vectors in the list.


Comparator comparator = ComparatorUtils.chainedComparator(new Comparator[]{new MultiKeyCompartor(0),new MultiKeyCompartor(1),new MultiKeyCompartor(2)});
Set<MultiKey> set = new TreeSet<MultiKey>(comparator);
set.addAll(masterBag.uniqueSet());
for (MultiKey multiKey : set)
{
System.out.println( "[" +StringUtils.join(multiKey.getKeys(),',') + "] = " + masterBag.getCount(multiKey));
}


The inner class PropertiesMultiKeyTransformer is a Transformer constructed with an array of Strings that represent the method names to use with PropertyUtils.getProperty on the beans in the list. It dynamically pulls the given properties from the bean and loads them into the new MultiKey. There is a limit to the number of Objects a MultiKey can accept, but for this demo, we are ok. In addition, the Objects in the MultiKey have to be consistently in the same order for each comparator. We are using a ComparatorUtils.chainedComparator of MultiKeyCompartor which will throw a ClassCastException if the objects are the wrong order.


private static class PropertiesMultiKeyTransformer implements Transformer
{
String[] methodNames;
private PropertiesMultiKeyTransformer(String[] methodNames)
{
this.methodNames = methodNames;
}
public Object transform(Object o)
{
List<Object> ooos = new ArrayList<Object>();
for (String methodName : methodNames)
{
try
{
ooos.add(PropertyUtils.getProperty(o, methodName));
}
catch (Exception e)
{
throw new FunctorException(e);
}
}
return new MultiKey(ooos.toArray(new Object[ooos.size()]));
}
}
private static class MultiKeyCompartor implements Comparator<MultiKey>
{
private int i;
private MultiKeyCompartor(int i)
{
this.i = i;
}
public int compare(MultiKey o1, MultiKey o2)
{
Object[] keys1 = o1.getKeys();
Object[] keys2 = o2.getKeys();
Object oo1 = null;
try
{
oo1 = keys1[i];
}
catch (ArrayIndexOutOfBoundsException e)
{
}
Object oo2 = null;
try
{
oo2 = keys2[i];
}
catch (ArrayIndexOutOfBoundsException e)
{
}
NullComparator nullComparator = new NullComparator(false);
return nullComparator.compare(oo1, oo2);
}
}


The code for the data transfer object used within this project is called the LaborForce object. See the following sample for the object. You can find State and Gender in some of the other projects used in this blog.

The Code:

package com.blogspot.apachecommonstipsandtricks.bags;
import com.blogspot.apachecommonstipsandtricks.*;
public class LaborForce
{
private State state;
private Gender gender;
private int year;
public LaborForce()
{
}
public LaborForce(State state, Gender gender, int year)
{
this.state = state;
this.gender = gender;
this.year = year;
}
public State getState()
{
return state;
}
public void setState(State state)
{
this.state = state;
}
public Gender getGender()
{
return gender;
}
public void setGender(Gender gender)
{
this.gender = gender;
}
public int getYear()
{
return year;
}
public void setYear(int year)
{
this.year = year;
}
}


The results:

All the possible properties on the LaborForce bean are [state,gender,year]
[Male] = 7842
[Male,1950] = 134
[Male,1950,AL] = 1
[Male,1950,AK] = 6
[Male,1950,AZ] = 2
[Male,1950,CA] = 1
[Male,1950,CO] = 4
[Male,1950,CT] = 3
[Male,1950,DE] = 3
[Male,1950,DC] = 1
[Male,1950,FL] = 2
[Male,1950,GA] = 2
[Male,1950,HI] = 4
..
..
[Female,2008,SD] = 3
[Female,2008,TN] = 2
[Female,2008,TX] = 6
[Female,2008,UT] = 2
[Female,2008,VT] = 1
[Female,2008,VA] = 2
[Female,2008,WA] = 3
[Female,2008,WI] = 3


Performance Test


The test I ran was very simple. I wrapped a timer around parts of the code that were unique to the process. And the results of using the Bag are small, compared to a traditional method.

The code:

package com.blogspot.apachecommonstipsandtricks.bags;
import java.util.*;
import java.text.*;
import java.net.*;
import java.io.*;
import org.apache.commons.collections.*;
import org.apache.commons.collections.bag.*;
import org.apache.commons.collections.bag.HashBag;
import org.apache.commons.lang.time.*;
import org.apache.commons.lang.*;
import com.blogspot.apachecommonstipsandtricks.*;
public class BagStressTest
{
private static final int NumberOfTimesToRun = 1000;
private static final int MasterListSize = 10;

public static void main(String[] args) throws IOException
{
int i = 1;
SimpleDateFormat sdf = new SimpleDateFormat("ss.SSS");
StopWatch sw;
Bag bag = null;
Map<State, Integer> map = null;
long sumOfTime = 0;

List<LaborForce> list = new ArrayList<LaborForce>();
URL url = new URL("http", "1796193846474123283-a-1802744773732722657-s-sites.googlegroups.com", 80, "/site/psenger/Home/data.txt");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
if (((str = in.readLine()) != null))
{
do
{
String[] strings = str.split("\t");
strings = StringUtils.stripAll(strings);

State state = State.valueOf(StringUtils.trim(strings[0]));
Gender gender = "Male".equals(StringUtils.trim(strings[1])) ? Gender.Male : Gender.Female;
Integer year = Integer.valueOf(StringUtils.trim(strings[2]));

list.add(new LaborForce(state, gender, year));
} while ((str = in.readLine()) != null);
}
in.close();

List<LaborForce> listOfSampleData = new ArrayList<LaborForce>();
for (int j = 0; j < MasterListSize; j++)
{
listOfSampleData.addAll( list );
}
System.out.println("The sample listOfSampleData length is " + listOfSampleData.size());
System.out.println(" ---------- Start Test ---------- ");
sumOfTime = 0;
sw = new StopWatch();
for (int j = 0; j < NumberOfTimesToRun; j++)
{
sw.start();
bag = TransformedBag.decorate(new HashBag(), new Transformer()
{
public Object transform(Object o)
{
LaborForce dto = (LaborForce) o;
return dto.getState();
}
});
bag.addAll(listOfSampleData);
sw.stop();
sumOfTime += sw.getTime();
sw.reset();
}
System.out.println("With a Bag, Average Time in seconds and milliseconds is " + sdf.format( new Date( sumOfTime / NumberOfTimesToRun ) ) );
System.out.println(" ---------- End Test ---------- ");

// Alternative Way.
System.out.println(" ---------- Start Test ---------- ");
sumOfTime = 0;
sw.reset();
for (int j = 0; j < NumberOfTimesToRun; j++)
{
sw.start();
map = new HashMap<State, Integer>();
for (LaborForce lbf : listOfSampleData)
{
Integer count = map.get(lbf.getState());
if (null == count)
{
count = 0;
}
count++;
map.put(lbf.getState(), count);
}
sw.stop();
sumOfTime += sw.getTime();
sw.reset();
}
System.out.println("Traditional counting, Average Time in seconds and milliseconds is " + sdf.format( new Date( sumOfTime / NumberOfTimesToRun ) ) );
System.out.println(" ---------- End Test ---------- ");
}
}


The results:

The sample listOfSampleData length is 156000
---------- Start Test ----------
With a Bag, Average Time in seconds and milliseconds is 00.032
---------- End Test ----------
---------- Start Test ----------
Traditional counting, Average Time in seconds and milliseconds is 00.026
---------- End Test ----------


Thanks for reading this article, I look forward to any questions or feedback.

Author: Philip A Senger

Friday, February 13, 2009

Examples of Set Theory in Java with Apache Commons Collections

Before I get started on Apache Commons Collections Unions, intersections, and sub collections let me clarify something. You should always push the activity of manipulating data to the data layer. In an N-Tier system, the term Tier implies the system has concerns separated into layers. Cluttering up a system with concerns all over the place will make a nightmare Brownfield System. However if you are unable and are forced to compose the object at your layer (which is not unusual), these techniques tend to be extensible. This type of scenario occurs if you are working with disparate systems. For example, half of the data may come from a SQL like data source and the other from a SOAP Service.

Proper Sub collections, Unions, and Intersections are terms used in a branch of mathematics referred to as Set Theory. It describes membership status of items within sets. Sets are identified generically by a letter. The term union is identified by the symbol ∪, Intersection ∩, sub sets and super sets are defined by ⊂ and ⊃ respectfully.

To illustrate a union consider the set A and B. When the two are union-ed they make the following new set called C. This is akin to the mathematical function plus.



Conversely when A intersects with B, C is created. As you can see, C is the difference of the two sets. This is very similar to the minus function.



A Sub Collection and Proper Sub Collection are a little more difficult to understand. While in this example A contains B making it a Proper Sub Collection of A. if B contained everything in A it would not be proper sub collection anymore, It would only be a sub collection.


Example : Unions in Java with CollectionUtils.union


Lets look at Unions and how this can be done in Java with the Apache Commons CollectionUtils.union.

The Code:

package com.blogspot.apachecommonstipsandtricks.examplesOfCollectionsAndSets;
import java.util.*;
import org.apache.commons.lang.*;
public abstract class AbstractCollectionExampleUtils
{
protected static List<Integer> A = Arrays.asList(1, 2, 3, 4);
protected static List<Integer> B = Arrays.asList(3, 4);
protected static List<Integer> C = Arrays.asList(5, 6);
protected static List<Integer> D = Arrays.asList(1, 2, 3, 4, 5);
protected static List<Integer> E = Arrays.asList();
/**
* Intersection = cap
*/
protected static String intersection() { return "∩"; }
/**
* Union = cup
*/
protected static String union() { return "∪"; }
/**
* Subset of
*/
protected static String subsetOf(){ return "⊂"; }
/**
* Superset of
*/
protected static String supersetOf(){ return "⊃"; }
/**
* Not a subset of
*/
protected static String notASubsetOf(){ return "⊄"; }
/**
* Subset of or equal to
*/
protected static String subSetOfOrEqualTo(){ return "⊆"; }
/**
* Superset of or equal to
*/
protected static String superSetOfOrEqualTo(){ return "⊇"; }
/**
* Print Set
*/
protected static String set(Collection<Integer> S)
{
return " {" + StringUtils.join(S.iterator(), ",") + "} ";
}
}
package com.blogspot.apachecommonstipsandtricks.examplesOfCollectionsAndSets;
import static org.apache.commons.lang.StringUtils.rightPad;
import java.util.*;
import org.apache.commons.collections.*;
public class Union extends AbstractCollectionExampleUtils
{
public static void main(String[] args)
{
System.out.println("When" + rightPad(set(A), 13) + union() + rightPad(set(B), 13) + "=" + union(A, B));
System.out.println("When" + rightPad(set(A), 13) + union() + rightPad(set(C), 13) + "=" + union(A, C));
System.out.println("When" + rightPad(set(A), 13) + union() + rightPad(set(D), 13) + "=" + union(A, D));
System.out.println("When" + rightPad(set(A), 13) + union() + rightPad(set(E), 13) + "=" + union(A, E));
}
private static String union(Collection s, Collection ss)
{
return set(CollectionUtils.union(s, ss));
}
}


The Results:

When {1,2,3,4} ∪ {3,4} = {1,2,3,4}
When {1,2,3,4} ∪ {5,6} = {1,2,3,4,5,6}
When {1,2,3,4} ∪ {1,2,3,4,5} = {1,2,3,4,5}
When {1,2,3,4} ∪ {} = {1,2,3,4}


Example : Intersection in Java with CollectionUtils.intersection


Now, lets look at intersections and how this can be done in Java with the Apache Commons CollectionUtils.intersection.

The Code:

package com.blogspot.apachecommonstipsandtricks.examplesOfCollectionsAndSets;
import static org.apache.commons.lang.StringUtils.rightPad;
import java.util.*;
import org.apache.commons.collections.*;
public class Intersection extends AbstractCollectionExampleUtils
{
public static void main(String[] args)
{
System.out.println("When" + rightPad(set(A),13) + intersection() + rightPad(set(B),13) + " " + intersection(A, B));
System.out.println("When" + rightPad(set(B),13) + intersection() + rightPad(set(A),13) + " " + intersection(B, A));
System.out.println("When" + rightPad(set(A),13) + intersection() + rightPad(set(C),13) + " " + intersection(A, C));
System.out.println("When" + rightPad(set(C),13) + intersection() + rightPad(set(A),13) + " " + intersection(C, A));
System.out.println("When" + rightPad(set(A),13) + intersection() + rightPad(set(D),13) + " " + intersection(A, D));
System.out.println("When" + rightPad(set(D),13) + intersection() + rightPad(set(A),13) + " " + intersection(D, A));
System.out.println("When" + rightPad(set(A),13) + intersection() + rightPad(set(E),13) + " " + intersection(A, E));
System.out.println("When" + rightPad(set(E),13) + intersection() + rightPad(set(A),13) + " " + intersection(E, A));
}
private static String intersection(Collection s, Collection ss)
{
return "CollectionUtils.intersection(" + rightPad(set(s), 13) + "," + rightPad(set(ss), 13) + ") = " + rightPad(set(CollectionUtils.intersection(s, ss)),11);
}
}


The Results:

When {1,2,3,4} ∩ {3,4} CollectionUtils.intersection( {1,2,3,4} , {3,4} ) = {3,4}
When {3,4} ∩ {1,2,3,4} CollectionUtils.intersection( {3,4} , {1,2,3,4} ) = {3,4}
When {1,2,3,4} ∩ {5,6} CollectionUtils.intersection( {1,2,3,4} , {5,6} ) = {}
When {5,6} ∩ {1,2,3,4} CollectionUtils.intersection( {5,6} , {1,2,3,4} ) = {}
When {1,2,3,4} ∩ {1,2,3,4,5} CollectionUtils.intersection( {1,2,3,4} , {1,2,3,4,5} ) = {1,2,3,4}
When {1,2,3,4,5} ∩ {1,2,3,4} CollectionUtils.intersection( {1,2,3,4,5} , {1,2,3,4} ) = {1,2,3,4}
When {1,2,3,4} ∩ {} CollectionUtils.intersection( {1,2,3,4} , {} ) = {}
When {} ∩ {1,2,3,4} CollectionUtils.intersection( {} , {1,2,3,4} ) = {}


Example : Sub collections and Proper Sub collections in Java with CollectionUtils.containsAny, isProperSubCollection, and isSubCollection


The first two examples were simple enough. Let’s look at Sub collection versus Proper Sub Collection. We will use the containsAny, isProperSubCollection and isSubCollection static methods off the CollectionUtils class.


package com.blogspot.apachecommonstipsandtricks.examplesOfCollectionsAndSets;
import java.util.*;
import static org.apache.commons.lang.StringUtils.rightPad;
import org.apache.commons.collections.*;
public class SubCollections extends AbstractCollectionExampleUtils
{
public static void main(String[] args)
{
System.out.println("Intersection Tests : ");
System.out.println("Is" + rightPad(set(A), 13) + intersection() + rightPad(set(B), 13) + " " + containsAny(A, B));
System.out.println("Is" + rightPad(set(B), 13) + intersection() + rightPad(set(A), 13) + " " + containsAny(B, A));
System.out.println("Is" + rightPad(set(A), 13) + intersection() + rightPad(set(C), 13) + " " + containsAny(A, C));
System.out.println("Is" + rightPad(set(C), 13) + intersection() + rightPad(set(A), 13) + " " + containsAny(C, A));
System.out.println("Is" + rightPad(set(A), 13) + intersection() + rightPad(set(D), 13) + " " + containsAny(A, D));
System.out.println("Is" + rightPad(set(D), 13) + intersection() + rightPad(set(A), 13) + " " + containsAny(D, A));
System.out.println("Is" + rightPad(set(A), 13) + intersection() + rightPad(set(E), 13) + " " + containsAny(A, E));
System.out.println("Is" + rightPad(set(E), 13) + intersection() + rightPad(set(A), 13) + " " + containsAny(E, A));
System.out.println("Is" + rightPad(set(E), 13) + intersection() + rightPad(set(E), 13) + " " + containsAny(E, E));
System.out.println("");
System.out.println("Subsets Tests: " );
System.out.println("B " + subsetOf() + " A indicates B is a subset of A, but are not equal. When B is equal to A it is usually denoted as B " + subSetOfOrEqualTo() + " A");
System.out.println("");
System.out.println("Is" + rightPad(set(A), 13) + subsetOf() + rightPad(set(A), 13) + " " + subsetOfOrEqualTo(A, A));
System.out.println("Is" + rightPad(set(B), 13) + subsetOf() + rightPad(set(A), 13) + " " + subsetOfOrEqualTo(B, A));
System.out.println("Is" + rightPad(set(C), 13) + subsetOf() + rightPad(set(A), 13) + " " + subsetOfOrEqualTo(C, A));
System.out.println("");
System.out.println("Is" + rightPad(set(A), 13) + subSetOfOrEqualTo() + rightPad(set(A), 13) + " " + superSetOfOrEqualTo(A, A));
System.out.println("Is" + rightPad(set(B), 13) + subSetOfOrEqualTo() + rightPad(set(A), 13) + " " + superSetOfOrEqualTo(B, A));
System.out.println("Is" + rightPad(set(C), 13) + subSetOfOrEqualTo() + rightPad(set(A), 13) + " " + superSetOfOrEqualTo(C, A));
}
protected static String containsAny(Collection s, Collection ss)
{
return rightPad(" CollectionUtils.containsAny(" + rightPad(set(s), 13) + "," + rightPad(set(ss), 13) + ")", 67) + " = " + String.valueOf(CollectionUtils.containsAny(s, ss));
}
protected static String subsetOfOrEqualTo(Collection s, Collection ss)
{
return rightPad(" CollectionUtils.isProperSubCollection(" + rightPad(set(s), 13) + "," + rightPad(set(ss), 13) + ")", 67) + " = " + String.valueOf(CollectionUtils.isProperSubCollection(s, ss));
}
private static String superSetOfOrEqualTo(Collection s, Collection ss)
{
return rightPad(" CollectionUtils.isSubCollection(" + rightPad(set(s), 13) + "," + rightPad(set(ss), 13) + ")", 67) + " = " + String.valueOf(CollectionUtils.isSubCollection(s,ss));
}
}


The Results:

Intersection Tests :
Is {1,2,3,4} ∩ {3,4} CollectionUtils.containsAny( {1,2,3,4} , {3,4} ) = true
Is {3,4} ∩ {1,2,3,4} CollectionUtils.containsAny( {3,4} , {1,2,3,4} ) = true
Is {1,2,3,4} ∩ {5,6} CollectionUtils.containsAny( {1,2,3,4} , {5,6} ) = false
Is {5,6} ∩ {1,2,3,4} CollectionUtils.containsAny( {5,6} , {1,2,3,4} ) = false
Is {1,2,3,4} ∩ {1,2,3,4,5} CollectionUtils.containsAny( {1,2,3,4} , {1,2,3,4,5} ) = true
Is {1,2,3,4,5} ∩ {1,2,3,4} CollectionUtils.containsAny( {1,2,3,4,5} , {1,2,3,4} ) = true
Is {1,2,3,4} ∩ {} CollectionUtils.containsAny( {1,2,3,4} , {} ) = false
Is {} ∩ {1,2,3,4} CollectionUtils.containsAny( {} , {1,2,3,4} ) = false
Is {} ∩ {} CollectionUtils.containsAny( {} , {} ) = false

Subsets Tests:
B ⊂ A indicates B is a subset of A, but are not equal. When B is equal to A it is usually denoted as B ⊆ A

Is {1,2,3,4} ⊂ {1,2,3,4} CollectionUtils.isProperSubCollection( {1,2,3,4} , {1,2,3,4} ) = false
Is {3,4} ⊂ {1,2,3,4} CollectionUtils.isProperSubCollection( {3,4} , {1,2,3,4} ) = true
Is {5,6} ⊂ {1,2,3,4} CollectionUtils.isProperSubCollection( {5,6} , {1,2,3,4} ) = false

Is {1,2,3,4} ⊆ {1,2,3,4} CollectionUtils.isSubCollection( {1,2,3,4} , {1,2,3,4} ) = true
Is {3,4} ⊆ {1,2,3,4} CollectionUtils.isSubCollection( {3,4} , {1,2,3,4} ) = true
Is {5,6} ⊆ {1,2,3,4} CollectionUtils.isSubCollection( {5,6} , {1,2,3,4} ) = false


The Apache Commons Collections has some enormous potential. I find I uses these static methods frequently. I hope this has helped you in solving your problems, please feel free to drop me a line if you have any questions.

Author: Philip A Senger

Saturday, February 7, 2009

Examples of Functors, Transformers, Predicates, and Closures in Java

One day, I found myself re-designing a procurement portal, and I kept re-writing the same for-loop over and over again (no pun intended). I had an epiphany; I could do better and I started using the Apache Commons Collection Utilities (Transformers, Predicates, and Closures). Now don’t think just because I started to use the Apache Commons Collection Utilities, the project was better. However, the result was a highly extensible framework…. Later it was dismantled by another team… but that is a different story (Grin).

Functors

I laugh every time I think of the word Functors, but that is because I’m immature, case in point, I still laugh at fart jokes. Anyway, Functors, or Function Objects, in the Apache or Jakarta Commons Collection Utilities are a set of interfaces designed specifically to be used against collections of objects. This framework embodies a balance between code reuse and behavioral specialization through composition as opposed to strict Object Oriented design. Composition is well suited for Creational patterns such as Factories, Structural patterns like Decorators, or Behavioral patterns like Strategies. The Apache Commons Collections framework defines three types of interfaces:
  • Closures are functions that can alter the object and get a reference to each object in the collection.

  • Transformers are responsible for transforming data from one format to another or from one object to another.

  • Predicates simply execute a conditional test against each item in a collection and return true or false for each item.
NOTE:
My examples sometimes use Anonymous Inner Classes and Inner Classes. Some developers have strong feelings about defining classes in this manner. There are times when doing this is appropriate and times when it is inappropriate. As with any programming solutions, this technique may or may not suit your needs or environment. So, let’s get over it and move on.

Closure

Here is a typical problem statement which maybe resolved with the use of Closures. I want to execute a specific method or change the state on every object in a collection. For example, I might want to execute the toString method. Please note that these examples do not pull out a value and transform into another collection of objects. They simply iterate over the collection and do something to it or with it. For the purpose of this first example, I will send the results of the toString to system.out. In the second example, I will alter the state of each bean and change the name of every object. Both examples use the utility method CollectionUtils.forAllDo.

Lets take some baby steps, and look at this example.

The Code:
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
import java.util.*;
import org.apache.commons.collections.*;
import org.apache.commons.lang.*;
import com.blogspot.apachecommonstipsandtricks.*;
public class SimpleClosure
{
public static void main(String[] args)
{
System.out.println("\nTest Number One Results :");
List<String> collectionOfWords = Arrays.asList("Java", "Example",
"Help", "Tips", "And",
"Tricks", "Apache",
"Commons", "Collections");
// Lets call toString on every object and print it out.
CollectionUtils.forAllDo(collectionOfWords, new Closure()
{
public void execute(Object o)
{
assert o != null;
System.out.print(o.toString() + " ");
}
});
System.out.println("\n\nTest Number Two Results :");
int i = 1;
List<DTO> collectionOfDTOs = Arrays.asList(new DTO(i++, "Java Tips and Tricks", Gender.Male, State.WI),
new DTO(i++, "Apache Commons" , Gender.Male, State.WI),
new DTO(i++, "Jakarta Commons" , Gender.Male, State.WI),
new DTO(i++, "Collections" , Gender.Male, State.WI),
new DTO(i++, "Closures" , Gender.Male, State.WI) );
CollectionUtils.forAllDo(collectionOfDTOs, new Closure()
{
public void execute(Object o)
{
DTO dto = (DTO) o;
assert dto != null;
String s = StringUtils.defaultIfEmpty(dto.getName(), "null");
dto.setName("Yoda says, " + s + " Rocks!");
}
});
CollectionUtils.forAllDo(collectionOfDTOs,PrintIt.getInstance());
}
}
The Results:
Test Number One Results :
Java Example Help Tips And Tricks Apache Commons Collections

Test Number Two Results :
com.blogspot.apachecommonstipsandtricks.DTO{id=1, name='Yoda says, Java Tips and Tricks Rocks!', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=2, name='Yoda says, Apache Commons Rocks!', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Yoda says, Jakarta Commons Rocks!', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=4, name='Yoda says, Collections Rocks!', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=5, name='Yoda says, Closures Rocks!', gender=Male, state=WI}

You might have noticed I created a class called PrintIt and it has a static method called getInstance. This class is an implementation of the singleton design pattern. In a large system where Closures (Predicates and Transformers) are being created by the hundreds, it makes sense to keep the memory foot print to a minimum. This is accomplished by using a singleton pattern. Word of caution, there is only one instance of this class per Java Virtual Machine (jvm), so using static properties could cause big problems, if you don’t understand how they work.

The Code:
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
import org.apache.commons.collections.*;
public class PrintIt implements Closure
{
// This class implements a Singleton Pattern
private static PrintIt ourInstance = new PrintIt();
/**
* Get a singleton instance of PrintIt
*/
public static PrintIt getInstance()
{
return ourInstance;
}
private PrintIt() // This is a singleton, dont change this!
{
}
public void execute(Object o)
{
System.out.println( o.toString() );
}
}
Once again, in the first example, all we did was iterate through a collection and call toString on every element in the collection. In the second example, we are actually modifying the state of the bean by rewriting the name of the DTO.. I love Yoda! Lets talk about transformers now.

Transformer

I’m a little older than the Transformer cartoons, but I cant help but think of the movie that was released not long ago. So, here is a problem statement that would best be resolved by transformers. You just have a whole bunch of string values from a Http Request object and you need to convert them to Integers.

The Code:
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
import java.util.*;
import org.apache.commons.collections.*;
public class SimpleTransformer
{
public static void main(String[] args)
{
Collection<String> stringOfNumbers = Arrays.asList("1", "2", "3", "4");
Collection<Integer> intNums = CollectionUtils.collect(stringOfNumbers, new Transformer() {
public Object transform(Object o) {
return Integer.valueOf((String) o);
}
});
CollectionUtils.forAllDo(intNums, PrintIt.getInstance() );
}
}
The Results:
1
2
3
4
Again, I used the PrintIt class to print the results, but you get the idea with this example. I converted a collection of Strings to Integers. Of course there are many ways to skin a cat, this is just an example. You might have noticed by now, that the interface is public Object transform(Object o)… It is not a Java 1.5 Generics implementation. I’m not entirely sure why the Apache team hasn’t released a Java 1.5 Generics version of these utilities, but if you are in a pinch and you have to have it, someone posted a link to an adaptation of the library that has the Generics. http://larvalabs.com/collections

Here is a more practical problem statement. You have a collection of plain old java beans and in each bean a method that returns a String, part of the String represents the id into another system, environment or sub identity. For example, the String might be prefixed with three letters, for example "PAS". You might find some legacy ERP systems that do this to the PO number. Maybe the billing department puts the initials of the sales person or team at the front of the number. In this example we will transform an Array of Strings that have ids in them into an array of ids in the form of numbers only.

The Code:
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
import java.util.*;
import org.apache.commons.collections.*;
import org.apache.commons.lang.*;
public class SimpleTransformer
{
public static void main(String[] args)
{
Collection<String> stringOfNumbers = Arrays.asList("ABC0001", "BCD0002", "CDF0003", "BFA0004");
Collection<Integer> intNums = CollectionUtils.collect(stringOfNumbers, new Transformer()
{
public Object transform(Object o)
{
String s = ((String) o);
return Integer.valueOf(s.substring(3, s.length()));
}
});
CollectionUtils.forAllDo(intNums, PrintIt.getInstance());
}
}
The Results:
1
2
3
4
Now let’s look at something a little more practical. As a problem statement, let’s say we have a billing object from the old system called OldBill, and we need to identify it in the new system, called NewBill. The ids in the old system started with “A” and a number. In the new system they will start with “Z” and the number from the old system plus 500. In the next example we will break apart the concerns of the transformers into two different transformers and glue them together with the utility class TransformerUtils.chainedTransformer. Creating a transformer in this manner allows us to plug in a different behavior or add and subtract behaviors, even on the fly.

The Code:
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
import java.util.*;
import org.apache.commons.collections.*;
public class ChainedTransformer
{
public static void main(String[] args)
{
List<OldBill> aList = Arrays.asList(new OldBill("A1"), new OldBill("A2"),
new OldBill("A3"), new OldBill("A4"));
Transformer[] chainedTransformer = new Transformer[]{
new Transformer() {
public Object transform(Object o) {
return ((OldBill )o).getId().replace('A', 'Z');
}
},
new Transformer() {
public Object transform(Object o) {
char[] c = ((String) o).toCharArray();
int x = Integer.parseInt(String.valueOf(c[1])) + 500;
return new NewBill( String.valueOf(c[0]) + x );
}
}
};
System.out.println("The aList");
CollectionUtils.forAllDo(aList, PrintIt.getInstance());
List<NewBill> bList = (List<NewBill>) CollectionUtils.collect(aList, TransformerUtils.chainedTransformer(chainedTransformer));
System.out.println("\nThe bList");
CollectionUtils.forAllDo(bList, PrintIt.getInstance());
}
}
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
public class OldBill
{
private String id;
public OldBill(String id)
{
this.id = id;
}
public String getId()
{
return id;
}
public void setId(String id)
{
this.id = id;
}
@Override public String toString()
{
return "OldBill{id='" + id + "\'}";
}
}
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
public class NewBill
{
private String id;
public NewBill(String id)
{
this.id = id;
}
public String getId()
{
return id;
}
public void setId(String id)
{
this.id = id;
}
@Override public String toString()
{
return "NewBill{id='" + id + "\'}";
}
}
The Results:
The aList
OldBill{id='A1'}
OldBill{id='A2'}
OldBill{id='A3'}
OldBill{id='A4'}

The bList
NewBill{id='Z501'}
NewBill{id='Z502'}
NewBill{id='Z503'}
NewBill{id='Z504'}
Predicate

Predicates do one thing and one thing only, they return either true or false. As a problem statement, let’s say we have a collection of Strings and we want keep out values that can not be converted to numbers.

The Code:
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
import java.util.*;
import org.apache.commons.collections.*;
public class SimplePredicate
{
public static void main(String[] args)
{
List<String> mixedup = Arrays.asList("A", "0", "B", "C", "1", "D", "F", "3");
Collection numbersOnlyList = CollectionUtils.predicatedCollection(new ArrayList(),
new Predicate() {
public boolean evaluate(Object o) {
try {
Integer.valueOf((String) o);
return true;
} catch (NumberFormatException e) {
return false;
}
}
});
for (String s : mixedup) {
try {
numbersOnlyList.add(s);
} catch (IllegalArgumentException e) {
System.out.println("I love CollectionUtils!");
}
}
System.out.println("\nResults of the predicatedCollection List:");
CollectionUtils.forAllDo(numbersOnlyList, PrintIt.getInstance() );
}
}
The Results:
I love CollectionUtils!
I love CollectionUtils!
I love CollectionUtils!
I love CollectionUtils!
I love CollectionUtils!

Results of the predicatedCollection List:
0
1
3
Ok here is something more useful. Have you ever had a need to do something like SQL but in Java? For example, you wanted to select beans from a collection based on some conditional blocks? Here are some great examples of SQL commands that can be used in Java and they behave just like SQL complex where clauses, distinct, like, and group by. In this example, we will be using the folloing utlitly classes and maps
The Code:
package com.blogspot.apachecommonstipsandtricks.transformersexamples;
import java.util.*;
import org.apache.commons.collections.*;
import org.apache.commons.collections.map.*;
import com.blogspot.apachecommonstipsandtricks.*;
public class PredicatesSQLSample
{
public static void main(String[] args)
{
List<DTO> list = Arrays.asList(new DTO(1,"Bob", Gender.Male, State.WI), new DTO(2,"Larry",Gender.Male, State.WI),
new DTO(3,"Bill", Gender.Male, State.WI), new DTO(4,"Sue", Gender.Female, State.AZ),
new DTO(3,"Bill", Gender.Male, State.WI), new DTO(4,"Sue", Gender.Female, State.AZ),
new DTO(5,"Joe", Gender.Male, State.AZ), new DTO(6,"Zoe", Gender.Female, State.MI));
Predicate sqlOrQueryPredicate = PredicateUtils.anyPredicate(new Predicate[]{
new Predicate()
{
public boolean evaluate(Object o)
{
return State.WI.equals(((DTO) o).getState());
}
}, new Predicate()
{
public boolean evaluate(Object o)
{
return Gender.Female.equals(((DTO) o).getGender());
}
}
});
Predicate sqlAndQueryPredicate = PredicateUtils.allPredicate(new Predicate[]{
new Predicate()
{
public boolean evaluate(Object o)
{
return State.AZ.equals(((DTO) o).getState());
}
}, new Predicate()
{
public boolean evaluate(Object o)
{
return Gender.Male.equals(((DTO) o).getGender());
}
}
});
Predicate likeNameStartsWithB = new Predicate(){
public boolean evaluate(Object o)
{
return ((DTO) o).getName().startsWith("B");
}
};

Collection aList = CollectionUtils.select(list, sqlOrQueryPredicate);
Collection bList = CollectionUtils.select(list, PredicateUtils.notPredicate( sqlOrQueryPredicate ));
Collection cList = CollectionUtils.select(list, sqlAndQueryPredicate);
Collection dList = CollectionUtils.select(list, PredicateUtils.allPredicate(new Predicate[]{PredicateUtils.uniquePredicate(), sqlOrQueryPredicate} ));
Collection eList = CollectionUtils.select(list, PredicateUtils.allPredicate(new Predicate[]{PredicateUtils.uniquePredicate() ,likeNameStartsWithB} ));
Collection fList = CollectionUtils.select(list, PredicateUtils.uniquePredicate() );

Map aGroupByStateMap = TransformedMap.decorate(new MultiValueMap(),new Transformer(){
public Object transform(Object o)
{
return ((DTO) o).getState();
}
}, TransformerUtils.nopTransformer() );
for (Object o : fList)
{
aGroupByStateMap.put( o, o );
}

System.out.println("\nAll the people :\nselect * from list");
CollectionUtils.forAllDo(list,PrintIt.getInstance());
System.out.println("\nAll the people in Wisconsin OR Female :\nselect * from list where ( state = WI or gender = female );");
CollectionUtils.forAllDo(aList, PrintIt.getInstance());
System.out.println("\nAll the people NOT ( Wisconsin OR Female ) :\nselect * from list where ! ( state = WI or gender = female );");
CollectionUtils.forAllDo(bList, PrintIt.getInstance());
System.out.println("\nAll the people in Arizona AND Male :\nselect * from list where ( state = AZ and gender = male );");
CollectionUtils.forAllDo(cList, PrintIt.getInstance());
System.out.println("\nAll the distinct people in Arizona AND Male :\nselect distinct * from list where ( state = WI or gender = female );");
CollectionUtils.forAllDo(dList, PrintIt.getInstance());
System.out.println("\nAll the distinc people with the name that starts with B :\nselect distinct * from list where name like \"B%\";");
CollectionUtils.forAllDo(eList, PrintIt.getInstance());
System.out.println("\nAll the distinct people grouped by state :\nselect distinct * from list group by state;");
Set states = aGroupByStateMap.keySet();
for (Object state : states)
{
System.out.println(((State)state).getFullyQualifiedName());
CollectionUtils.forAllDo((Collection) aGroupByStateMap.get(state), PrintIt.getInstance());
}
}
}
The Results:
All the people :
select * from list
com.blogspot.apachecommonstipsandtricks.DTO{id=1, name='Bob', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=2, name='Larry', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Bill', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=4, name='Sue', gender=Female, state=AZ}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Bill', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=4, name='Sue', gender=Female, state=AZ}
com.blogspot.apachecommonstipsandtricks.DTO{id=5, name='Joe', gender=Male, state=AZ}
com.blogspot.apachecommonstipsandtricks.DTO{id=6, name='Zoe', gender=Female, state=MI}

All the people in Wisconsin OR Female :
select * from list where ( state = WI or gender = female );
com.blogspot.apachecommonstipsandtricks.DTO{id=1, name='Bob', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=2, name='Larry', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Bill', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=4, name='Sue', gender=Female, state=AZ}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Bill', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=4, name='Sue', gender=Female, state=AZ}
com.blogspot.apachecommonstipsandtricks.DTO{id=6, name='Zoe', gender=Female, state=MI}

All the people NOT ( Wisconsin OR Female ) :
select * from list where ! ( state = WI or gender = female );
com.blogspot.apachecommonstipsandtricks.DTO{id=5, name='Joe', gender=Male, state=AZ}

All the people in Arizona AND Male :
select * from list where ( state = AZ and gender = male );
com.blogspot.apachecommonstipsandtricks.DTO{id=5, name='Joe', gender=Male, state=AZ}

All the distinct people in Arizona AND Male :
select distinct * from list where ( state = WI or gender = female );
com.blogspot.apachecommonstipsandtricks.DTO{id=1, name='Bob', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=2, name='Larry', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Bill', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=4, name='Sue', gender=Female, state=AZ}
com.blogspot.apachecommonstipsandtricks.DTO{id=6, name='Zoe', gender=Female, state=MI}

All the distinc people with the name that starts with B :
select distinct * from list where name like "B%";
com.blogspot.apachecommonstipsandtricks.DTO{id=1, name='Bob', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Bill', gender=Male, state=WI}

All the distinct people grouped by state :
select distinct * from list group by state;
ARIZONA
com.blogspot.apachecommonstipsandtricks.DTO{id=4, name='Sue', gender=Female, state=AZ}
com.blogspot.apachecommonstipsandtricks.DTO{id=5, name='Joe', gender=Male, state=AZ}
MICHIGAN
com.blogspot.apachecommonstipsandtricks.DTO{id=6, name='Zoe', gender=Female, state=MI}
WISCONSIN
com.blogspot.apachecommonstipsandtricks.DTO{id=1, name='Bob', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=2, name='Larry', gender=Male, state=WI}
com.blogspot.apachecommonstipsandtricks.DTO{id=3, name='Bill', gender=Male, state=WI}

Well, that wraps it up. Next week I plan on showing you some more examples of this fantastic api called Apache Commons.

Author: Philip A Senger

Sunday, January 25, 2009

TransformedMap and Transformers: Examples of Strategy, Decorator, and Factory

I mentioned some software design patterns in my last two examples one was a Factory, Decorator, and the other a Strategy. Before we get into the next example, I want to make sure you understand these patterns because, clearly the developers at Apache where thinking of these patterns when they developed the Commons Collections API. Let’s talk about a Factory first.

Factories:
Factories are a subset of what is called creational design patterns (See the wikipedia Abstract factory pattern). There are a couple of variations on the factory pattern but basically, you ask the factory for a named object, it assembles it and returns it. Just like a factory, hence the name. This is the building block of a lot of my web applications.

Decorator:
A decorator pattern is simply a way of receiving one value and decorating it into something suitable for a different view. For example, if you have a bean and it has a value of 12.599999999 you might want to decorator it to say something like $12.54. The apache team decided that the TransformedMap.decorate method was actually decorating the key not really a strategy.

Strategy:
A strategy pattern (See the wikipedia Strategy pattern) is a subset of behavioral design patterns. This pattern is analogous to a socket wrench (the socket part not the wrench part). So, as you might know, the socket can be replaced with different sizes at anytime. It simply plugs into the wrench and can be turned right or left. The concept here is we create an interface (which is a contract or in the wrench analogy the part that connects the socket to the wrench) whereby algorithms can be selected and dynamically installed at runtime.

Personally, I feel the TransformedMap.decorate is a borderline case between a strategy and a decorator…. On one hand it decorates the values going into the key, but on the other hand it changes the behavior at run time...Also, Im using a MultiValueMap for the backing map which totally changes the behavior. So, I will refer to it as a Strategy. I know some people will take issue with this, oh well. Lets declare some simple objects, like the Gender and State enum.


package com.blogspot.apachecommonstipsandtricks;
public enum Gender
{
Male, Female
}



package com.blogspot.apachecommonstipsandtricks;
/**
* Official USPS Abbreviations
*/
public enum State
{
AL("ALABAMA"), AK("ALASKA"), AS("AMERICAN SAMOA"),
AZ("ARIZONA "), AR("ARKANSAS"), CA("CALIFORNIA "),
CO("COLORADO "), CT("CONNECTICUT"), DE("DELAWARE"),
DC("DISTRICT OF COLUMBIA"), FM("FEDERATED STATES OF MICRONESIA"), FL("FLORIDA"),
GA("GEORGIA"), GU("GUAM "), HI("HAWAII"),
ID("IDAHO"), IL("ILLINOIS"), IN("INDIANA"),
IA("IOWA"), KS("KANSAS"), KY("KENTUCKY"), LA("LOUISIANA"),
ME("MAINE"), MH("MARSHALL ISLANDS"), MD("MARYLAND"),
MA("MASSACHUSETTS"), MI("MICHIGAN"), MN("MINNESOTA"),
MS("MISSISSIPPI"), MO("MISSOURI"), MT("MONTANA"),
NE("NEBRASKA"), NV("NEVADA"), NH("NEW HAMPSHIRE"),
NJ("NEW JERSEY"), NM("NEW MEXICO"), NY("NEW YORK"),
NC("NORTH CAROLINA"), ND("NORTH DAKOTA"), MP("NORTHERN MARIANA ISLANDS"),
OH("OHIO"), OK("OKLAHOMA"), OR("OREGON"),
PW("PALAU"), PA("PENNSYLVANIA"), PR("PUERTO RICO"),
RI("RHODE ISLAND"), SC("SOUTH CAROLINA"), SD("SOUTH DAKOTA"),
TN("TENNESSEE"), TX("TEXAS"), UT("UTAH"),
VT("VERMONT"), VI("VIRGIN ISLANDS"), VA("VIRGINIA "),
WA("WASHINGTON"), WV("WEST VIRGINIA"), WI("WISCONSIN"), WY("WYOMING");
private String fullyQualifiedName;
State(String fullyQualifiedName)
{
this.fullyQualifiedName = fullyQualifiedName;
}
public String getFullyQualifiedName()
{
return fullyQualifiedName;
}
}


Ok, now define our Data Transfer Object (DTO), just like the last example.


package com.blogspot.apachecommonstipsandtricks;
public class DTO
{
private int id;
private String name;
private Gender gender;
private State state;

public DTO(int id, String name, Gender gender, State state)
{
this.id = id;
this.name = name;
this.gender = gender;
this.state = state;
}
public int getId()
{
return id;
}
public void setId(int id)
{
this.id = id;
}
public String getName()
{
return name;
}
public void setName(String name)
{
this.name = name;
}
public Gender getGender()
{
return gender;
}
public void setGender(Gender gender)
{
this.gender = gender;
}
public State getState()
{
return state;
}
public void setState(State state)
{
this.state = state;
}
@Override
public String toString()
{
return "com.blogspot.apachecommonstipsandtricks.DTO{id=" + id + ", name='" + name + '\'' + ", gender=" + gender + ", state=" + state + '}';
}
}


And now, ( drum roll please ) the factory.


package com.blogspot.apachecommonstipsandtricks;
import java.util.*;
import org.apache.commons.collections.map.*;
import org.apache.commons.collections.*;
public class MapFactory
{
public static Map getMap(MapFactoryEnum whichFactory)
{
Map returnMap = null;
if (null != whichFactory)
{
Transformer transformer;
switch (whichFactory)
{
case NAME:
transformer = TransformerUtils.invokerTransformer("getName");
break;
case STATE:
transformer = TransformerUtils.invokerTransformer("getState");
break;
case GENDER:
transformer = TransformerUtils.invokerTransformer("getGender");
break;
default:
throw new IllegalArgumentException("Unknown Map");
}
returnMap = TransformedMap.decorate( new MultiValueMap(), transformer, TransformerUtils.nopTransformer());
}
return returnMap ;
}
public enum MapFactoryEnum
{
NAME, STATE, GENDER
}
}

So, when you call getMap, you request a map by a name. The factory builds a new MultiValueMap and decorates it, as in a Strategy with the transformer.

The code:

package com.blogspot.apachecommonstipsandtricks;
import java.util.*;
import org.apache.commons.collections.*;
import org.apache.commons.lang.*;
public class TestMapFactory
{
public static void main(String[] args)
{
int i = 0;
List<dto> list = Arrays.asList(new DTO(i++,"Bob", Gender.Male, State.WI), new DTO(i++,"Larry",Gender.Male, State.WI),
new DTO(i++,"Bill", Gender.Male, State.WI), new DTO(i++,"Sue", Gender.Female, State.AZ),
new DTO(i++,"Joe", Gender.Male, State.AZ), new DTO(i++,"Zoe", Gender.Female, State.WI));

List<dto> dtosFromTheMap;
Collection names;
Map map;

System.out.println("------------------------- By State -------------------------");
map = MapFactory.getMap(MapFactory.MapFactoryEnum.STATE);

for (DTO dto : list)
{
map.put( dto, dto );
}

dtosFromTheMap = (List<dto>) map.get(State.WI);
names = CollectionUtils.collect(dtosFromTheMap, TransformerUtils.invokerTransformer("getName"));
System.out.println(StringUtils.join(names.iterator(),",") + " are in " + State.WI.getFullyQualifiedName() );

dtosFromTheMap = (List<dto>) map.get(State.AZ);
names = CollectionUtils.collect(dtosFromTheMap, TransformerUtils.invokerTransformer("getName"));
System.out.println(StringUtils.join(names.iterator(),",") + " are in " + State.AZ.getFullyQualifiedName() );

System.out.println("------------------------- By Gender -------------------------");
map = MapFactory.getMap(MapFactory.MapFactoryEnum.GENDER);

for (DTO dto : list)
{
map.put( dto, dto );
}

dtosFromTheMap = (List<dto>) map.get(Gender.Male);
names = CollectionUtils.collect(dtosFromTheMap, TransformerUtils.invokerTransformer("getName"));
System.out.println(StringUtils.join(names.iterator(),",") + " are " + Gender.Male );

dtosFromTheMap = (List<dto>) map.get(Gender.Female);
names = CollectionUtils.collect(dtosFromTheMap, TransformerUtils.invokerTransformer("getName"));
System.out.println(StringUtils.join(names.iterator(),",") + " are " + Gender.Female );
}
}


This code could be very useful if you where caching items or grouping selections together inside a controller. Just a side note, maps are always non-thread safe. I will explain in my next example what that means.

The results:

------------------------- By State -------------------------
Bob,Larry,Bill,Zoe are in WISCONSIN
Sue,Joe are in ARIZONA
------------------------- By Gender -------------------------
Bob,Larry,Bill,Joe are Male
Sue,Zoe are Female

Author: Philip A Senger

Tuesday, January 20, 2009

TransformedMap and Transformers: Decorated "put" strategy

This is a long example regarding decorated maps and transformers. It’s important to understand the concepts I will cover, because I will use them in the next example, and continue to build on them.

The object TransformedMap has a static method called decorate (http://commons.apache.org/collections/api-release/org/apache/commons/collections/map/TransformedMap.html), this method allows you to load a backing map and two transformers (http://commons.apache.org/collections/api-release/org/apache/commons/collections/Transformer.html), one for the key and the other for the value. The returning object is a decorated map. So, when you put an item into this new decorated map, the key and values are transformed. Unfortunately, the transformer interface doesn’t allow you to gain access to the backing object through the interface. You could be cleaver and jam it into a constructor ( more on this later ). Be forewarned, modifying a map while in the midst of another modification method will result in a big ugly run time exception ( just in case you thought you could us this method to build your own MultiValueMap ).

Enough chatter… lets look at some code

The code:

package com;
import java.util.*;
import org.apache.commons.collections.map.*;
import org.apache.commons.collections.*;
public class MapDecorator
{
public static void main(String[] args)
{
int i = 0;
List<DTO> list = Arrays.asList(new DTO(i++,"Bob",Gender.Male,State.WI), new DTO(i++,"Larry",Gender.Male,State.WI),
new DTO(i++,"Bill", Gender.Male, State.WI), new DTO(i++,"Sue", Gender.Female, State.AZ),
new DTO(i++,"Joe", Gender.Male, State.AZ), new DTO(i++,"Zoe", Gender.Female, State.WI));
// Decorate a map where the key is the id, and the value is the object.
Map exampleOne = TransformedMap.decorate( new HashMap(),
TransformerUtils.invokerTransformer("getId"),
TransformerUtils.nopTransformer());
// Decorate a map where the key is the name and the value is the id
Map exampleTwo = TransformedMap.decorate( new HashMap(),
new Transformer() {
public Object transform(Object o)
{
return ((DTO)o).getName();
}
},
new Transformer() {
public Object transform(Object o)
{
return ((DTO)o).getId();
}
} );
// load up the maps.
for (DTO dto : list)
{
exampleOne.put(dto, dto);
exampleTwo.put(dto, dto);
}
printTheMap("exampleOne",exampleOne);
printTheMap("exampleTwo",exampleTwo);
}

private static void printTheMap(String mapName, Map exampleOne)
{
System.out.println("Map Name = " + mapName );
System.out.println("------------ Keys ------------");
for (Object key : exampleOne.keySet())
{
System.out.println("key = " + key);
}
System.out.println("------------ Values ------------");
for (Object value : exampleOne.values())
{
System.out.println("value = " + value);
}
System.out.println("------------------------------");
}
public static class DTO
{
private int id;
private String name;
private Gender gender;
private State state;
public DTO(int id, String name, Gender gender, State state)
{
this.id = id;
this.name = name;
this.gender = gender;
this.state = state;
}
public int getId()
{
return id;
}
public void setId(int id)
{
this.id = id;
}
public String getName()
{
return name;
}
public void setName(String name)
{
this.name = name;
}
public Gender getGender()
{
return gender;
}
public void setGender(Gender gender)
{
this.gender = gender;
}
public State getState()
{
return state;
}
public void setState(State state)
{
this.state = state;
}
@Override public String toString()
{
return "DTO{id=" + id + ", name='" + name + '\'' + ", gender=" + gender + ", state=" + state + '}';
}
}
public static enum Gender
{
Male, Female
}
/**
* Official USPS Abbreviations
*/
public static enum State
{
AL("ALABAMA"), AK("ALASKA"), AS("AMERICAN SAMOA"),
AZ("ARIZONA "), AR("ARKANSAS"), CA("CALIFORNIA "),
CO("COLORADO "), CT("CONNECTICUT"), DE("DELAWARE"),
DC("DISTRICT OF COLUMBIA"), FM("FEDERATED STATES OF MICRONESIA"), FL("FLORIDA"),
GA("GEORGIA"), GU("GUAM "), HI("HAWAII"),
ID("IDAHO"), IL("ILLINOIS"), IN("INDIANA"),
IA("IOWA"), KS("KANSAS"), KY("KENTUCKY"), LA("LOUISIANA"),
ME("MAINE"), MH("MARSHALL ISLANDS"), MD("MARYLAND"),
MA("MASSACHUSETTS"), MI("MICHIGAN"), MN("MINNESOTA"),
MS("MISSISSIPPI"), MO("MISSOURI"), MT("MONTANA"),
NE("NEBRASKA"), NV("NEVADA"), NH("NEW HAMPSHIRE"),
NJ("NEW JERSEY"), NM("NEW MEXICO"), NY("NEW YORK"),
NC("NORTH CAROLINA"), ND("NORTH DAKOTA"), MP("NORTHERN MARIANA ISLANDS"),
OH("OHIO"), OK("OKLAHOMA"), OR("OREGON"),
PW("PALAU"), PA("PENNSYLVANIA"), PR("PUERTO RICO"),
RI("RHODE ISLAND"), SC("SOUTH CAROLINA"), SD("SOUTH DAKOTA"),
TN("TENNESSEE"), TX("TEXAS"), UT("UTAH"),
VT("VERMONT"), VI("VIRGIN ISLANDS"), VA("VIRGINIA "),
WA("WASHINGTON"), WV("WEST VIRGINIA"), WI("WISCONSIN"), WY("WYOMING");
private String fullyQualifiedName;
State(String fullyQualifiedName)
{
this.fullyQualifiedName = fullyQualifiedName;
}
public String getFullyQualifiedName()
{
return fullyQualifiedName;
}
}
}



The results:

Map Name = exampleOne
------------ Keys ------------
key = 2
key = 4
key = 1
key = 3
key = 5
key = 0
------------ Values ------------
value = DTO{id=2, name='Bill', gender=Male, state=WI}
value = DTO{id=4, name='Joe', gender=Male, state=AZ}
value = DTO{id=1, name='Larry', gender=Male, state=WI}
value = DTO{id=3, name='Sue', gender=Female, state=AZ}
value = DTO{id=5, name='Zoe', gender=Female, state=WI}
value = DTO{id=0, name='Bob', gender=Male, state=WI}
------------------------------
Map Name = exampleTwo
------------ Keys ------------
key = Bob
key = Larry
key = Zoe
key = Joe
key = Sue
key = Bill
------------ Values ------------
value = 0
value = 1
value = 5
value = 4
value = 3
value = 2
------------------------------


The decorated Map, exampleOne, is backed by a HashMap and when put is called on the map, it calls the TransformerUtils.invokerTransformer("getId") (http://commons.apache.org/collections/api-release/org/apache/commons/collections/TransformerUtils.html#invokerTransformer(java.lang.String)) on the Object for the key. This results in the invoker using reflections to pull the id off the bean. In the same stroke, the value is transformed by TransformerUtils.nopTransformer() (http://commons.apache.org/collections/api-release/org/apache/commons/collections/TransformerUtils.html#nopTransformer()). This literally does nothing to the object used as the value; it is a kind of pass through.

// Decorate a map where the key is the id, and the value is the object.
Map exampleOne = TransformedMap.decorate( new HashMap(),
TransformerUtils.invokerTransformer("getId"),
TransformerUtils.nopTransformer());


For example

DTO dto = new DTO(100,"Bob",Gender.Male,State.WI);
exampleOne.put(dto, dto);

Results in an entry that has a key of 100 and a value of DTO(100,"Bob",Gender.Male,State.WI).

You might want to get a couple of values out and build a multi-value key or something else. The Map, exampleTwo, shows two anonymous inner implementations of the transformer interface.

Map exampleTwo = TransformedMap.decorate( new HashMap(),
new Transformer() {
public Object transform(Object o)
{
return ((DTO)o).getName();
}
},
new Transformer() {
public Object transform(Object o)
{
return ((DTO)o).getId();
}
} );


For example if you did this.

DTO dto = new DTO(100,"Bob",Gender.Male,State.WI);
exampleTwo.put(dto, dto);

The results would be a key of Bob pointing to a value of 100… are you beginning to see how awesome this api is? Are your gears grinding yet? Wait to you see what we do next time.
Author: Philip A Senger

Sunday, January 18, 2009

Transformer Invoker: For a collection of beans, collect a property

How many times have you had to collect a property from a list of beans, statically or dynamically? I use this little method a lot. For-loops are nice, but because this uses reflection to get the value, you can cook up a factory to get the data.

The code:

package com;
import java.util.*;
import org.apache.commons.collections.*;
public class TransformerExample
{
public static void main(String[] args)
{
List list = Arrays.asList(new DTO("Bob"), new DTO("Larry"), new DTO("Mo"), new DTO("Joe"));
Collection<String> names = CollectionUtils.collect(list, TransformerUtils.invokerTransformer("getName"));
for (String name : names)
{
System.out.println("name = " + name);
}
}
public static class DTO
{
private String name;
public DTO(String name)
{
this.name = name;
}
public String getName()
{
return name;
}
public void setName(String name)
{
this.name = name;
}
}
}



The results:

name = Bob
name = Larry
name = Mo
name = Joe
Author: Philip A Senger

Friday, January 16, 2009

Multi Value Map : The values are a List

NOTE:
Be aware that the method put does more than just "put" an object into the map. Therefore, if this object is the by-product of a method it would be advisable to wrap the map in an un-modifiable map. A lot of developers argue that this object violates the map interface. Frankly I think this argument is akin to splitting hairs, it works for me and I use it everyday.

The code:

MultiMap mhm = new MultiValueMap();

key = "Group One";
mhm.put(key, "Item One");
mhm.put(key, "Item Two");
mhm.put(key, "Item Three");

key = "Group Two";
mhm.put(key, "Item Four");
mhm.put(key, "Item Five");

Set keys = mhm.keySet();
for (Object k : keys) {
out.println("("+k+“ : "+mhm.get(k)+")");
}

The results:

(Group One : [Item One, Item Two, Item Three])
(Group Two : [Item Four, Item Five])
Author: Philip A Senger

Bi-Directional Map : The values can be turned into the keys.

NOTE:
This example is a Hash Map on both the key and value. Notice that last-in wins. For example X kills C when D is put twice as the value.

The code:

BidiMap aMap = new DualHashBidiMap();
aMap.put("B", "A");
aMap.put("A", "B");
aMap.put("C", "D");
aMap.put("X", "D");
MapIterator it = aMap.mapIterator();
System.out.println("Before Inverse");
while (it.hasNext()) {
key = it.next();
value = it.getValue();
out.println(key + " -> " + value);
}
aMap = aMap.inverseBidiMap();
System.out.println("After Inverse");
it = aMap.mapIterator();
while (it.hasNext()) {
key = it.next();
value = it.getValue();
out.println(key + " -> " + value);
}

The results:

Before Inverse
A -> B
B -> A
X -> D
After Inverse
D -> X
A -> B
B -> A
Author: Philip A Senger