Domain Entities and Draft Forms

Recently, I've been reviewing a funds transfer system that uses the S.W.I.F.T. standard message types. After reviewing it, I find that splitting the domain entity from its draft form would make it easier to maintain. Let me explain further.

Background of Funds Transfer System

The system allows users to record funds transfer requests as S.W.I.F.T. messages. Note that the system users are familiar with the said message types, and are not afraid to enter the values into a form that displays all the possible fields. To put things in perspective, a typical message type like MT 103 (Single Customer Credit Transfer) has about 25 fields. Some fields can have several formats. For example, the Ordering Customer (field 50a), can be in one of three formats: option A, option K, and option F. If we include every possible format, MT 103 would easily have over 70 fields.

Modeling Entities

Based on Eric Evan's DDD book, domain entities have a clear identity and a life-cycle with state transitions that we care about.

The funds transfer system modeled each MT as a domain entity. The MT domain entity has a clear identity, and a life-cycle (from draft to approved). It is mapped to its own table with its own corresponding columns to represent its fields. The message fields are implemented as properties. All fields are mutable (via setters methods).

At this point, you might pause and think about anemic domain models. You're probably thinking that there is hardly any behavior on these MT objects. You're probably thinking that these MT objects are just bags of getters and setters. And you're probably correct. But there's a reason for this. Read on.

The system validates the MTs, and displays which fields have errors. But the errors did not stop the system from persisting/storing the message. The system allowed the draft message to be stored for further modification (until it is approved).

This analysis led me to re-think about how the system modeled messages as entities. While it is tempting to think of them as rich domain entities that represent a real-world funds/credit transfer, they're really not. They're actually request forms. Much like the real-world paper forms that one fills-out, and submits for approval, and possibly being re-submitted due to corrections.

Given this, I would model the draft message as another domain entity that I'll call MessageTypeForm (or MT103Form for MT 103). This domain entity will use a map of fields (still in compliance with S.W.I.F.T. messages, where each field is keyed by an alpha-numeric string) (e.g. get("50a")), and not use a separate property for each field (e.g. getOrderingCustomer()). By using a map of fields, the entity would require lesser code, and can still be persisted (or mapped to table rows).

The CreditTransfer (or SingleCustomerCreditTransfer) would be another entity (not the same as the entity with a map of fields — MT103Form). This credit transfer entity shall have a clear identity and a life-cycle (e.g. being amended or cancelled). It can have a reference (by ID, and not by type) to the MT103Form entity from which it was created. It is also easier to establish invariants on this entity, as opposed to having all properties being mutable.

The validation of MT103Form will have to change from using properties (e.g. getOrderingCustomer()) to using a get method with a key parameter (to determine the message field it wants to retrieve) (e.g. get("50a")).

Deeper Insight

Just because a domain entity is largely made up of getters and setters, does not mean it is anemic.

The "draft/form" entity may look like a bag of getters and setters. But it has an identity, and a life-cycle (from draft to approved). So, it is an entity. Many "draft/form" instances will exist in the system simultaneously. The different instances may even have the same field values, but it is important for us to be able to track individual "draft/form" instances.

Discovering the distinction between the "draft/form" and "fund/credit transfer" entities have made things easier to implement. Having two separate domain entities has made the model better (reflect the problem it is trying to solve).

This deeper insight would not have been possible without the help of the domain experts, and my team mates: Tin, Anson, Richie, and Tina. Thanks guys!

One Thing Good Spring Developers Know

In my recent training sessions on the (core) Spring Framework, I was asked, "If there was one thing that a (Java) Spring developer should know, what should that be?" That question caught me off guard. Yes, the (core) Spring Framework does cover a lot of areas (e.g. beans, configuration, aspect-oriented programming, transactions). And it was difficult for me to point out just one thing. I ended up mentioning everything that we covered in our (3 day) training course.

If there was one thing that a (Java) Spring developer should know, what should that be?

As I gave that question more thought, I began to think about the most important one. I ended up thinking of how Spring uses aspects to add behavior to managed objects (usually called beans) as the most important. This is how the Spring Framework supports transactions, security, scope, Java-based configuration, among others. And I'm sharing my thoughts here in this post.

I ended up thinking of how Spring uses aspects to add behavior to managed objects (usually called beans) as the most important.

ORM and Lazy Loading Exceptions

Most developers who use some form of ORM have encountered an exception that signifies that child entities could not be loaded (e.g. LazyInitializationException).

Some developers who have encountered this would use an "open session in view" (OSIV) pattern to keep the session open and prevent this exception from happening. But I find this to be an overkill. Worse, some developers consider the "open session in view" pattern to be the only solution. A possible underlying cause for this misconception could be that the developer is probably not armed with the knowledge of using the Spring Framework effectively to keep the ORM session open longer.

In the case of JPA, the "open entity manager in view" pattern will create an entity manager at the beginning of the request, bind it to the request thread, and close it when the response is completed.

So, if not the OSIV pattern, what would be a better solution?

The short answer is to use the Spring Framework to keep the session open for the duration that you need it (e.g. @Transactional). Keep on reading as I'll provide a longer answer.

Services and Repositories

In a layered architecture, a typical design pattern is to define a domain or application service (usually defined as an interface) to provide business functionality (e.g. start using a shopping cart, adding items to that shopping cart, searching for products). Domain and application service implementations would typically delegate the retrieval/persistence of domain entities to repositories.

Presentation Layer

Business Layer

Data Access (or Infrastructure) Layer

Repositories (or data access objects) are also defined as interfaces to retrieve/persist domain entities (i.e. provide ORM and CRUD access). Naturally, repository implementations use ORM libraries (e.g. JPA/Hibernate, myBATIS) to retrieve and persist domain entities. With this, it uses the ORM framework's classes to connect to the persistent store, retrieve/persist the entity, and close the connection (called session in Hibernate). There's no problem of lazy loading failures at this point.

The problem of lazy loading failures occur when the service retrieves a domain entity using the repository, and wants to load child entities (after the repository method has returned). By the time the repository returns the domain entity, the ORM session gets closed. Because of this, attempts to access/load child entities in the domain service cause an exception.

The code snippets below illustrate how a lazy loading exception can occur when the child items of an order entity is lazily loaded after being returned by the repository.

@Entity
public class Order {
    @OneToMany // defaults to FetchType.LAZY
    private List<OrderItem> items;
    …
    public List<OrderItem> getItems() {…}
}

public class SomeApplicationServiceImpl implements SomeApplicationService {
    private OrderRepository orderRepository;
    …
    @Override
    public void method1(…) {
        …
        order = orderRepository.findById(...);
        order.getItems(); // <-- Lazy loading exception occurs!
        …
    }
    …
}

public class OrderRepositoryImpl implements OrderRepository {
    @PersistenceContext
    private EntityManager em;
    …
    @Override
    public Order findById(...) {...}
    …
}

The repository implementation explicitly uses JPA for its ORM (as illustrated with the use of an EntityManager).

At this point, some developers may opt to use eager fetch to prevent the lazy initialization exception. Telling the ORM to eagerly fetch the child items of an order entity will work. But sometimes, we don't need to load the child items. And eagerly loading this might be unnecessary overhead. It would be great to only load it when we need it.

To prevent the lazy initialization exception (and not be forced to eagerly fetch), we'll need to keep the ORM session open until the calling service method returns. In Spring, it can be as simple as annotating the service method as @Transactional to keep the session open. I find that this approach is better than using "open session in view" pattern (or being forced to use eager fetching), since it keeps the session open only for the duration that we intend it to be.

public class SomeApplicationServiceImpl implements SomeApplicationService {
    private OrderRepository orderRepository;
    …
    @Override
    @Transactional // <-- open the session (if it's not yet open)
    public void method1(…) {
        …
        order = orderRepository.findById(...);
        order.getItems(); // <-- Lazy loading exception should not happen
        …
    }
    …
}

Domain Entities in the Presentation Layer

Even after keeping the ORM session open in the service layer (beyond the repository implementation objects), the lazy initialization exception can still occur when we expose the domain entities to the presentation layer. Again, because of this, some developers prefer the OSIV approach, since it will also prevent lazy initialization exceptions in the presentation layer.

But why would you want to expose domain entities in the presentation layer?

From experience, I've worked with teams who prefer to expose domain entities in the presentation layer. This usually leads to anemic domain model, since presentation layer frameworks need a way to bind input values to the object. This forces domain entities to have getter and setter methods, and a zero-arguments constructor. Having getters and setters will make invariants difficult to enforce. For simple domains, this is workable. But for more complex domains, a richer domain model would be preferred, as it would be easier to enforce invariants.

In a richer domain model, the objects that represent the presentation layer input/output values are actually data transfer objects (DTOs). They represent inputs (or commands) that are carried out in the domain layer. With this in mind, I prefer to use DTOs and maintain a richer domain model. Thus, I don't really run into lazy initialization exceptions in the presentation layer.

Aspects to add behavior to managed objects

Spring intercepts calls to these @Transactional annotated methods to ensure that an ORM session is open.

Transactions (or simply keeping an ORM session open) are not the only behavior provided using aspects. There's security, scope, Java-based configuration, and others. Knowing that the Spring Framework uses aspects to add behavior is one of the key reasons why we let Spring manage the POJOs that we develop.

Conclusion

There you go. That for me is the one most important thing that a Spring Framework developer should know when using the core. Now that I've given my opinion as to what is the one most important thing, how about you? What do you think is the one most important thing to know when tackling Core Spring. Cheers!

Top 3 Improvements New Agile Teams Can Make

At first, I was planning to write about the top mistakes that novice Scrum/agile teams make. But then I wanted to say it in a positive way. So, I ended up writing about the top three improvements new Scrum/agile teams can make. Here it goes.

Focus on Stories, Not Tasks

Focus on stories (or features), and not tasks. Yes, team members still need to pull tasks from the board, and perform them. But don't forget that the team's goal is to complete stories. This means that when a team member has an option to pull a task from the board, the task should be part of the current on-going story. It would be better if team members are discouraged from starting a task that is not part of the current on-going story.

I've seen the following board (see below) several times in my few years of agile. Each story is grouped as a row (left to right) on the board. Notice how the team has completed several tasks, but none of the stories are done.

Scrum Board with Just Getting Random Tasks Done

To Do	In Progress	Done
Task A8	Task A6 Task A3 Task A5	Task A1 Task A2 Task A4 Task A7
Task B8 Task B6	Task B3 Task B7	Task B1 Task B2 Task B4 Task B5

This limiting of tasks is practiced in Kanban as WIP limits (as I've presented in my softtech.ph 2014 talk) (video here).

To put this in the positive, the team is encouraged to achieve a board that looks more like this (below). Here, team members are encouraged to stay on tasks with the on-going story (or feature) until it is done. They're discouraged from starting tasks on another story. But this doesn't mean that team members become idle. They're asked to help other team members to get the story done.

Scrum Board with Focus on Getting Stories Done

To Do	In Progress	Done
	Task A8 Task A6 Task A3	Task A5 Task A1 Task A2 Task A4 Task A7
Task B8 Task B6 Task B3 Task B7 Task B1 Task B2 Task B4 Task B5

Start Only with Ready Stories

I find that only a few agile teams have heard of definition of ready (DoR). Most of them have heard of definition of done (DoD), but not DoR. I've seen teams start sprints with stories that are far from being ready.

Some organizations (or companies) provide a checklist to get stories ready (not just sort of ready). I've seen the checklist include items like: estimated at the right size, prioritized by business value, has "done" criteria. It's difficult (almost impossible) to get a user story (or feature) done, if the team does not have an agreed "done" criteria. This "done" criteria (or acceptance criteria) is usually set before the story can be included in a sprint (as part of backlog refinement).

So, other than DoD, take a look at DoR. It might just help you and your team improve its productivity.

Split Stories to Fit in Sprint

Sometimes, user stories (or features) are too big to fit in a sprint (or iteration). When this happens, it would be better to split the user story, than to extend the sprint (to make it long enough for the team to complete the story).

I've witnessed teams extending their sprint cycles just to accommodate larger stories (or epics). And that's probably because they have limited skills in splitting stories. Many new agile teams attempt to split stories by architectural layers: one story for the UI, another for the database, etc. This results into stories that are not valuable to a user, and become interdependent with each other.

So, do be careful when splitting stories. Following Bill Wake's INVEST model for good user stories is highly recommended. You can refer to Richard Lawrence's Patterns for Splitting User Stories.

Conclusion

The above points may not apply to all teams. But if it does, please let me know. I would love to hear your (or your team's) experiences.

Here's wishing you more success in achieving your team's goals!