Navigation
Recent blog posts
Support Me!
User login
Overly Complicated ORM
Why is it that in Hibernate I have to specify the mundane? Isn't a project like this supposed to save me from the mundane? Then why do I find myself having to state that the column named ProjectName in PROJECT is mapped to the methods setProjectName and getProjectName in the Project javabean? *And* Why do I have to specify this in verbose XML files? Hibernate is not alone in this regard. Top Link and others force the same mundane tasks on you, in different ways. I'm only picking on Hibernate because of its popularity.
The reason this comes up is one my developers came to me recently (now about a year ago) and asked why don't we use Hibernate or any other ORM tools? He was sick of writing mundane code to instantiate objects from a result set. And rightly so. I responded with many reasons, and while they may not be applicable to other projects, here are a few of our most poignant:
1. Our project is 4 years old. That doesn't just mean that it is running for four years, but we have been actively developing new code everday for four years. Requirements change during that length of time and what we have now is not what we started out with. Also, the java landscape has changed quite a bit since the project started. Some tools around now didn't exist back then. Some weren't as mature. But primarily, because of the volatile landscape, it is scary to adopt any toolset for fear of it being obsolete in a year. Sometimes you find yourself waiting until the better option appears. Case in point, we skipped right over Struts and went to Spring MVC. Which, as a side note, is turning out to be the ONLY framework we may ever need.
2. On top of standard javabeans, our project has non-traditional "metadata" objects that don't map well to relational tables. These objects are essentially a java object backed by a hashtable. It allows us to define the structure of business objects based on different rules at runtime, years down the road from when the object first came into being. Though we actually had a license to use Toplink based on our Oracle purchases, we never used it because of the metadata situation. Back then, it required custom code which didn't give us any savings.
3. The goal of an ORM tool (from my developer's perspective) is to reduce the mundane to nothing. Its standard boiler plate code that we write to take a database result set and turn it into an array of objects. And while this code is moving data from one object into another, it is, in a sense, describing the relationship between a database table and a java object. Every time I took another peak at the persistance frameworks, they always seemed to be relying on the developer to describe that relationship. So in effect, they were moving the work done in java code, to xml, or to GUI configuration. In my eyes, the time savings is negligble. We are still doing the work of describing the relationship. Regardless of where you put it, you still have to do it. That's just not the solution we will incorporate into our considerably large project.
4. There are complicated issues you run into with an application cluster, caching on each node, and composite objects that are updated by different modules in the application. Obviously, from this discussion, we are not using EJB but plain old java objects. What ever ORM framework we use, it must support our caching strategy. How about this: If you have composite objects that can be updated independantly of each other, how is the cache notified of the update? Hibernate attempts to solve this with Proxy objects (I believe, its been a year since I looked) but it wasn't the most elegant solution. To their credit, I don't think there is an elegant solution outside of Spring and Aspect Oriented Programming (I'll get to that in another article). This is just one example of the caching issues we faced.
5. We hadn't discovered iBATIS and its integration with Spring, yet. (This wasn't a argument I made, of course. Its just here to allude to the solution we did settle down with later on... covered in another article...)
Sure, there are benefits you gain just by using a ORM tool instead of hand coding everything. But none of the tools provided the total package I wanted.
For the sake of discussion, let's forget about point #4. That's a difficult topic to tackle. Point #3 is what really gets me and is what I'm here to talk about. Come on guys. Its SQL. Its JavaBeans. Be smart about it. We have Java Reflections. Use it. Look at what Ruby on Rails is doing. And look at how long we have had Java Reflections? Why are we dicking around when JavaBeans has been a standard/convention for years now.
To put my money where my mouth is, here is a proof of concept. As I like to call it, a Poor Man's Persistance. Keep in mind, this was just rattled off the top of my head one day (After the discussion with our depressed developer) and could be improved in many ways. Though its a proof of concept here, we actually used it in production for some time until iBATIS came along.
First, here are our algorithms for intializing, loading, and saving data to and from the database and the object:
Initialization:
1. Using ResultSetMetaData, determine all columns in the table the object matches.
2. Store the columns and column types for use later on.
3. Using reflection, determine all public methods of the object that match a specified pattern. (getXXX() and setXXX())
4. Parse each method name to find the field name it is acting on.
Look up in our column list to find the column that matches the field name.
If we find one, store the getter and setter methods, the field name, and the type of the field for use later on.
Load:
1. For a record in the table, create a new object.
2. For each column in the record's resultset, execute its corresponding setter on the new object, passing the column value as a parameter.
Store:
1. For the specified object, execute our stored getter methods on it to retrieve a map of key => value pairs that represent the object.
2. From the resulting map, generate an insert/update SQL statement and execute it.
Looking at this process, we can create a library that will only require two parameters: the object it is managing and the name of the table that represents it. A keen eye will notice that this only covers simple objects. It doesn't handle composite objects built off of joining multiple tables together. I'll get to that one in a minute.
So here is our implementation:
(In the interests of keeping this short, the descriptor objects contain public fields without proper visibilty and get/set methods.)
ObjectDescriptor.java
1:package com.phatness.persistence;
2:
3:import java.lang.reflect.Method;
4:
5:public class ObjectDescriptor {s
6: public String keyName;
7: public Method keyGetter;
8: public boolean autoinc;
9: public Class objClass;
10: public String table;
11: public HashMap fields;
12: public Method childrenGetters[];
13: public Method childrenSetters[];
14: public ObjectDescriptor children[];
15:
16: public ObjectDescriptor() {
17: objClass = Object.class;
18: table = "";
19: children = new ObjectDescriptor[0];
20: fields = new HashMap();
21: childrenGetters = new Method[0];
22: childrenSetters = new Method[0];
23: autoinc = false;
24: }
25:
26: public Method getSetMethodForField(String fieldName) {
27: FieldDescriptor field = (FieldDescriptor) fields.get(fieldName);
28: if(field != null) {
29: return field.setter;
30: }
31: return null;
32: }
33:}
34:
35:
FieldDescriptor.java
1:spackage com.phatness.persistence;
2:
3:import java.lang.reflect.Method;
4:
5:public class FieldDescriptor {
6: public String name;
7: public Class type;
8: public Method getter;
9: public Method setter;
10:
11: public FieldDescriptor() {
12: name = "";
13: type = java.lang.String.class;
14: }
15:}
ss
Now the pertinent part of Manager.java. The first part is to determine the structure of a class we are persisting:
1:protected void readClassStructure(ObjectDescriptor obj) {
2: Method methodList[] = obj.objClass.getMethods();
3: HashMap fields = new HashMap();
4: for(int counter = 0; counter < methodList.length; counter++) {
5: boolean primary = false;
6: String fieldName = null;
7: String original = null;
8: if(methodList[counter].getName().startsWith("get")) {
9: logger.info(obj.objClass.getName() + "." + methodList[counter].getName() + "()");
10: // Check to see if we have a master table field.
11: fieldName = methodList[counter].getName().substring(3);
12: original = fieldName;
13: } else if(methodList[counter].getName().startsWith("is")) {
14: logger.info(obj.objClass.getName() + "." + methodList[counter].getName() + "()");
15: // Check to see if we have a master table field.
16: fieldName = methodList[counter].getName().substring(2);
17: original = fieldName;
18: }
19: if(fieldName != null) {
20: logger.info("Field Name: " + fieldName);
21: if(fieldName.equals("OrgID")) {
22: // because we shorten this, let's fix it to what it would be in the table.
23: fieldName = "OrganizationID";
24: }
25: if(fieldName.equals("ID")) {
26: // this is our primary key
27: primary = true;
28: // hack for child tables that have keys not named ProjectPhaseID but PhaseID
29: // right now this is only ProjectPhase (It would normally be named
30: // Phase, but there is already a Phase for job.
31: // these are completely unrelated entities.
32: if(obj.table.indexOf("_") > 0) {
33: // if this is a child or relationship table, grab the name after the underscore
34: String name = obj.table.substring(obj.table.indexOf(("_")));
35: name = name.toLowerCase();
36: fieldName = name.substring(1, 2).toUpperCase() + name.substring(2) + "ID";
37: } else {
38: fieldName = obj.objClass.getName().substring(
39: obj.objClass.getPackage().getName().length() + 1) + "ID";
40: }
41: logger.warning("Primary Key Name: " + fieldName);
42: }
43: if(isTableField(obj.table, fieldName)) {
44: if(primary) {
45: obj.keyName = fieldName;
46: obj.keyGetter = methodList[counter];
47: } else {
48: Class returnType = methodList[counter].getReturnType();
49: Method setter = null;
50: try {
51: setter = obj.objClass.getMethod("set" + original,
52: new Class[] { returnType });
53: } catch(NoSuchMethodException e) {
54: // should handle this later, some exceptions would be
55: // fields that aren't set the same way. Others would be
56: // not following the contract we have stated here
57: // for the later case, we should alert the developer
58: // they have an incorrect entity.
59: logger.warning(UtilityTank.getErrorDetail("There is no " +
60: "such method: set" + original, e));
61: }
62: FieldDescriptor field = new FieldDescriptor();
63: field.name = fieldName;
64: field.type = returnType;
65: field.getter = methodList[counter];
66: field.setter = setter;
67: fields.put(fieldName, field);
68: }
69: }
70: }
71: }
72: obj.fields = fields;
73: // look up the sequence generation info (sequence_master or autoinc)
74: String query = "select " + obj.keyName + " from SEQUENCE_MASTER";
75: DBAtom atom = DBControl.getAtom();
76: try {
77: atom.execQuery(query);
78: if(atom.next()) {
79: logger.info(obj.keyName + " is found in SEQUENCE_MASTER");
80: obj.autoinc = false;
81: } else
82: obj.autoinc = true;
83: } catch (Exception e) {
84: logger.fine(UtilityTank.getErrorDetail(e));
85: obj.autoinc = true;
86: } finally {
87: DBControl.release(atom);
88: }
89:
90:
91: // For all of our stated children, load their structure, as well.
92: for(int counter = 0; counter < obj.children.length; counter++) {
93: logger.info("Looking at child class: " + obj.children[counter].objClass.getName());
94: readClassStructure(obj.children[counter]);
95: }
96:}
Once we know the structure of a class, we can easily assemble it from a generic map of data:
1:public Entity assemble(Entity entity, HashMap data, ObjectDescriptor obj) {
2: logger.info("Entity class: " + entity.getClass().getName());
3: if(!entity.getClass().equals(obj.objClass))
4: throw new IllegalArgumentException("Entity does not match declared " +
5: "object structure. Possible problem in children calls.");
6: // Setup primary key
7: entity.setID((String) data.get(obj.keyName));
8: // As a temporary fix, until we figure out how to handle mixed data, find the
9: // set(String, String) metadata method and use that to store the data as
10: // metadata on top of the normal setXXX() method.
11: // This is definitely a hack that MUST be removed shortly.
12: Method bandaid = null;
13: try {
14: bandaid = obj.objClass.getMethod("set", new Class[] { String.class, String.class });
15: logger.info("Bandaid: " + bandaid.getName());
16: } catch(Exception e) {
17: logger.info("Object was not a metadata obj.");
18: }
19: // Setup fields via setter methods
20: Iterator i = data.keySet().iterator();
21: while(i.hasNext()) {
22: String key = (String) i.next();
23: Object value = data.get(key);
24: Method method = obj.getSetMethodForField(key);
25: if(method != null) {
26: try {
27: if(data.get(key) == null)
28: continue;
29: // These should be one argument setter methods
30: Class params[] = method.getParameterTypes();
31: Object values[];
32: logger.info("For: " + method.getName() + " on " + value + " as type: " + params[0].getName());
33: if(params[0] == Date.class) {
34: if(value.getClass() == Date.class)
35: values = new Object[] { value };
36: else
37: values = new Object[] { DonerDate.stringDateToJavaDate((String) value) };
38: } else if(params[0] == boolean.class) {
39: values = new Object[] { new Boolean(value.toString()) };
40: } else if(params[0] == long.class) {
41: values = new Object[] { new Long(value.toString()) };
42: } else if(params[0] == int.class) {
43: values = new Object[] { new Integer(value.toString()) };
44: } else if(params[0] == float.class) {
45: values = new Object[] { new Float(value.toString()) };
46: } else if(params[0] == double.class) {
47: values = new Object[] { new Double(value.toString()) };
48: } else {
49: values = new Object[] { value };
50: }
51: method.invoke(entity, values);
52: if(bandaid != null && value != null)
53: bandaid.invoke(entity, new Object[] { key, value.toString() });
54: } catch(Exception e) {
55: logger.warning(UtilityTank.getErrorDetail("Exception using: " +
56: method.getName(), e));
57: }
58: } else if(!key.equals(obj.keyName)){
59: // If we are not the primary key, it never has the correct accessor methods.
60: logger.warning("Have a " + obj.objClass.getName() + " field: " + key +
61: " with no set method in obj. Data will be lost.");
62: }
63: }
64: return entity;
65:}
Disassembling an object is even easier. Here is the code to convert an object back into a Map:
1:public HashMap disassemble(Entity entity, ObjectDescriptor obj) {
2: HashMap map = new HashMap();
3:
4: if(!entity.getClass().equals(obj.objClass))
5: throw new IllegalArgumentException("Entity: " + entity.getClass().getName() +
6: " does not match initialized manager class: " + obj.objClass.getName());
7:
8: for(Iterator i = obj.fields.keySet().iterator(); i.hasNext(); ) {
9: String key = (String) i.next();
10: FieldDescriptor field = (FieldDescriptor) obj.fields.get(key);
11: logger.info("Executing: " + field.getter.getName() + " for field: " + key);
12: try {
13: Object value = field.getter.invoke(entity, null);
14: logger.config(" Returned value of: " + value);
15: map.put(key, value);
16: } catch(IllegalAccessException e) {
17: logger.warning(UtilityTank.getErrorDetail(e));
18: } catch(InvocationTargetException e) {
19: logger.warning(UtilityTank.getErrorDetail(e));
20: }
21: }
22: return map;
23:}
And anyone that has worked with databases for longer than a day knows how to take a generic map and turn it into a SQL statement for inserts and updates.
If you are further interested, send me a line. There is quite a bit of housekeeping code left off.
Caveats:
In the end, this method doesn't solve complex graphs of objects. If you are taking ORM seriously, there is only one tool you need to look at. That's iBatis at ibatis.org. They do it right from the beginning.