Refactoring large data object

Question

What are some common strategies for refactoring large \"state-only\" objects?

I am working on a specific soft-real-time decision support system which do

Answer 1

From experience working also with R&D stuff with soft real-time performance constrains (and sometimes monster fat classes), I would suggest NOT to use OR mappers. In such situations, you'll be better off dealing "touching the metal" and working directly with JDBC result sets. This is my suggestion for apps with soft real-time constrains and massive amounts of data items per package. More importantly, if the number of distinct classes (not class instances, but class definitions) that need to persisted is large, and you also have memory constrains in your specs, you will also want to avoid ORMs like Hibernate.

Going back to your original question:

What you seem to have is a typical problem of 1) mapping multiple data items into a OO model and 2) such multiple data items do not exhibit a good way of grouping or segregation (and any attempt to grouping tends simply not to feel right.) Sometimes the domain model does not lend itself for such aggregation, and coming up with an artificial way of doing so typically ends up in compromises that don't satisfy all design requirements and desires.

To make matters worse, a OO model typically requires/expects you to have all the items present in a class as class' fields. Such a class is typically without behavior, so it is just a struct-like construct, aka data envelope or data shuttle.

But such situations beg the following questions:

Does your application need to read/write all 40, 50+ data items at once, always? *Must all data items be always present?*

I do not know the specifics of your problem domain, but in general I've found that we rarely ever need to deal with all data items at once. This is where a relational model shines because you don't have to query all rows from a table at once. You only pulls those you need as projections of the table/view in question.

In a situation where we have a potentially large number of data items, but on average the number of data items being passed down the wire is less than the maximum, you'd be better off using a Properties pattern.

Instead of defining a monster envelope class holding all items :

// java pseudocode
class envelope
{
   field1, field2, field3... field_n;
   ...
   setFields(m1,m2,m3,...m_n){field1=m1; .... };
   ...
}

Define a dictionary (based on a map for example):

// java pseudocode
public enum EnvelopeField {field1, field2, field3,... field_n);

interface Envelope //package visible
{
   // typical map-based read fields.
   Object get(EnvelopeField  field);
   boolean isEmpty();

   // new methods similar to existing ones in java.lang.Map, but
   // more semantically aligned with envelopes and fields.
   Iterator fields();
   boolean hasField(EnvelopeField field); 
}

// a "marker" interface
// code that only needs to read envelopes must operate on
// these interfaces.
public interface ReadOnlyEnvelope extends Envelope {} 

// the read-write version of envelope, notice that
// it inherits from Envelope, but not from ReadOnlyEnvelope.
// this is done to make it difficult (but not impossible
// unfortunately) to "cast-up" a read only envelope into a
// mutable one.
public interface MutableEnvelope extends Envelope
{
   Object put(EnvelopeField field); 

   // to "cast-down" or "narrow" into a read only version type that
   // cannot directly be "cast-up" back into a mutable.
   ReadOnlyEnvelope readOnly();
}

// the standard interface for map-based envelopes.
public interface MapBasedEnvelope extends 
   Map
   MutableEnvelope
{
}

// package visible, not public
class EnvelopeImpl extends HashMap 
  implements MapBasedEnvelope, ReadOnlyEnvelope
{
   // get, put, isEmpty are automatically inherited from HashMap
   ... 
   public Iterator fields(){ return this.keySet().iterator(); }
   public boolean hasField(EnvelopeField field){ return this.containsKey(field); }

   // the typecast is redundant, but it makes the intention obvious in code.
   public ReadOnlyEnvelope readOnly(){ return (ReadOnlyEnvelope)this; }
}

public class final EnvelopeFactory
{
    static public MapBasedEnvelope new(){ return new EnvelopeImpl(); }
}

No need to set up read-only internal flags. All you need to do is downcast your envelope instances as Envelope instances (that only provide getters).

Code that expects to read should operate on read-only envelopes and code that expects to change fields should operate on mutable envelopes. Creation of the actual instances would be compartmentalized in factories.

That is, you use the compiler to enforce things to be read-only (or allow things to be mutable) by establishing some code conventions, rules governing what interfaces to use where and how.

You can layer your code into sections that need to write separate from code that only needs to read. Once that's done, simple code reviews (or even grep) can identify code that is using the wrong interface.)

Problems:

Non-public Parent Interface:

Envelope is not declared as a public interface to prevent erroneous/malicious code from casting a read-only envelope down to a base envelope and then back to a mutable envelope. The intended flow is from mutable to read-only only - it is not intended to be bi-directional.

The problem here is that extension of Envelope is restricted to the package that contains it. Whether that is a problem will depend on the particular domain and intended usage.

Factories:

The problem is that factories can (and most likely will) be very complex. Again, the nature of the beast.

Validation:

Another problem introduced with this approach is that now you have to worry about code that expects field X to be present. Having the original monster envelope class partially frees you from that worry because, at least syntactically, all fields are there...

... whether the fields are set or not, that was another matter that still remains with this new model I'm proposing.

So if you have client code that expects to see field X, the client code has to throw some type of exception if the field is not present (or to computer or read a sensible default somehow.) In such cases, you will have to

1. Identify patterns of field presence. Clients that expect field X to be present might be grouped separately (layered apart) from clients that expect some other field to be present.

2. Associate custom validators (proxies to read-only envelope interfaces) that either throw exceptions or compute default values for missing fields according to some rules (rules provided programmatically, with an interpreter, or with a rules engine.)

Lack of Typing:

This might be debatable, but people used to work with static typing might feel uneasy with losing the benefits of static typing by going to a loosely typied map-based approach. The counter-argument of this is that most of the web works on a loose typing approach, even on the Java side (JSTL, EL.)

Problems aside, the larger the maximum number of possible fields and the lower the average number of fields present at any given time, the most effective wrt performance this approach will be. It adds additional code complexity, but that's the nature of the beast.

That complexity doesn't go away, and either will be present in your class model or in your validation code. Serialization and transferring down the wire is much more efficient, though, specially if you expect massive numbers of individual data transfers.

Hope it helps.