Changing The Way We Use JPA

I've been updating some training materials recently, and thinking about better ways of teaching and talking about JPA. One of the things I've been thinking about is how we have typically used JPA, and how that should change given the pains I've experienced (and observed).

JPA is often seen as a set of annotations (or XML files) that provide O/R (object-relational) mapping information. And most developers think that the more mapping annotations they know and use, the more benefits they get. But the past few years of wrestling with small to medium monoliths/systems (with about 200 tables/entities) have taught me something else.

TL;DR

1. Reference entities by ID (only map entity relationships within an aggregate)
2. Don't let JPA steal your Identity (avoid @GeneratedValue when you can)
3. Use ad-hoc joins to join unrelated entities

Reference Entities by Identifier

Only map entity relationships within an aggregate.

Tutorials (and training) would typically go about teaching and covering all possible relationship mappings. After basic mappings, many would start from simple uni-directional @ManyToOne mapping. Then proceed to bi-directional @OneToMany and @ManyToOne. Unfortunately, most often than not, they fail to explicitly point out that it is perfectly fine to not map the relationship. So, beginners would often complete the training thinking that it would be a mistake to not map a related entity. They mistakenly think that a foreign key field must be mapped as a related entity.

In fact, it is not an error, if you change the @ManyToOne mapping below…

@Entity
public class SomeEntity {
    // ...
    @ManyToOne private Country country;
    // ...
}

@Entity
public class Country {
    @Id private String id; // e.g. US, JP, CN, CA, GB, PH
    // ...
}

…into a basic field that contains the primary key value of the related entity.

@Entity
public class SomeEntity {
    // ...
    @Column private String countryId;
    // ...
}

@Entity
public class Country {
    @Id private String id; // e.g. US, JP, CN, CA, GB, PH
    // ...
}

Why is this a problem?

Mapping all entity relationships increases the chances of unwanted traversals that usually lead to unnecessary memory consumption. This also leads to an unwanted cascade of EntityManager operations.

This may not be much if you're dealing with just a handful of entities/tables. But it becomes a maintenance nightmare when working with dozens (if not hundreds) of entities.

When do you map a related entity?

Map related entities only when they are within an Aggregate (in DDD).

Aggregate is a pattern in Domain-Driven Design. A DDD aggregate is a cluster of domain objects that can be treated as a single unit. An example may be an order and its line-items, these will be separate objects, but it's useful to treat the order (together with its line items) as a single aggregate.

https://martinfowler.com/bliki/DDD_Aggregate.html

@Entity
public class Order {
    // ...
    @OneToMany(mappedBy = "order", ...) private List<OrderItem> items;
    // ...
}

@Entity
public class OrderItem {
    // ...
    @ManyToOne(optional = false) private Order order;
    // ...
}

More modern approaches to aggregate design (see Vaughn Vernon's Implementing Domain-Driven Design) advocate a cleaner separation between aggregates. It is a good practice to refer to an aggregate root by storing its ID (unique identifier), not a full reference.

If we expand the simple order example above, the line-item (OrderItem class) should not have a @ManyToOne mapping to the product (since it is another aggregate in this example). Instead, it should just have the ID of the product.

@Entity
public class Order {
    // ...
    @OneToMany(mappedBy = "order", ...) private List<OrderItem> items;
    // ...
}

@Entity
public class OrderItem {
    // ...
    @ManyToOne(optional = false) private Order order;
    // @ManyToOne private Product product; // <-- Avoid this!
    @Column private ... productId;
    // ...
}

But… what if the Product (aggregate root entity) has its @Id field mapped as @GeneratedValue? Are we forced to persist/flush first and then use the generated ID value?

And, what about joins? Can we still join those entities in JPA?

Don't Let JPA Steal Your Identity

Using @GeneratedValue may initially make the mapping simple and easy to use. But when you start referencing other entities by ID (and not by mapping a relationship), it becomes a challenge.

If the Product (aggregate root entity) has its @Id field mapped as @GeneratedValue, then calling getId() may return null. When it returns null, the line-item (OrderItem class) will not be able to reference it!

In an environment where all entities always have a non-null Id field, referencing any entity by ID becomes easier. Furthermore, having non-null Id fields all the time, makes equals(Object) and hashCode() easier to implement.

And because all Id fields become explicitly initialized, all (aggregate root) entities have a public constructor that accepts the Id field value. And, as I've posted long ago, a protected no-args constructor can be added to keep JPA happy.

@Entity
public class Order {
    @Id private Long id;
    // ...
    public Order(Long id) {
        // ...
        this.id = id;
    }
    public Long getId() { return id; }
    // ...
    protected Order() { /* as required by ORM/JPA */ }
}

While writing this post, I found an article by James Brundege (posted back in 2006), Don't Let Hibernate Steal Your Identity (thanks to Wayback Machine), where he says, Don't let Hibernate manage your ids. I wish I heeded his advice earlier.

But beware! When using Spring Data JPA to save() an entity that does not use @GeneratedValue on its @Id field, an unnecessary SQL SELECT is issued before the expected INSERT. This is due to SimpleJpaRepository's save() method (shown below). It relies on the presence of the @Id field (non-null value) to determine whether to call persist(Object) or merge(Object).

public class SimpleJpaRepository // ...
	@Override
	public <S extends T> save(S entity) {
		// ...
		if (entityInformation.isNew(entity)) {
			em.persist(entity);
			return entity;
		} else {
			return em.merge(entity);
		}
	}
}

The astute reader will notice that, if the @Id field is never null, the save() method will always call merge(). This causes the unnecessary SQL SELECT (before the expected INSERT).

Fortunately, the work-around is simple — implement Persistable<ID>.

@MappedSuperclass
public abstract class BaseEntity<ID> implements Persistable<ID> {
	@Transient
	private boolean persisted = false;
	@Override
	public boolean isNew() {
		return !persisted;
	}
	@PostPersist
	@PostLoad
	protected void setPersisted() {
		this.persisted = true;
	}
}

The above also implies that all updates to entities must be done by loading the existing entity into the persistence context first, and applying changes to the managed entity.

Use Ad-hoc Joins to Join Unrelated Entities

And, what about joins? Now that we reference other entities by ID, how can we join unrelated entities in JPA?

In JPA version 2.2, unrelated entities cannot be joined. However, I cannot confirm if this has become a standard in version 3.0, where all javax.persistence references were renamed to jakarta.persistence.

Given the OrderItem entity, the absence of the @ManyToOne mapping causes it to fail to be joined with the Product entity.

@Entity
public class Order {
    // ...
}

@Entity
public class OrderItem {
    // ...
    @ManyToOne(optional = false) private Order order;
    @Column private ... productId;
    // ...
}

Thankfully 😊, Hibernate 5.1.0+ (released back in 2016) and EclipseLink 2.4.0+ (released back in 2012) have been supporting joins of unrelated entities. These joins are also referred to as ad-hoc joins.

SELECT o
  FROM Order o
  JOIN o.items oi
  JOIN Product p ON (p.id = oi.productId) -- supported in Hibernate and EclipseLink

Also, this has been raised as an API issue (Support JOIN/ON for two root entities). I really hope that it will become a standard soon.

In Closing

What do you think about the above changes? Are you already using similar approaches? Do you use native SQL to explicitly retrieve a generated value (e.g. sequence object) to create an entity with a non-null Id field? Do you use entity-specific ID types to differentiate ID values? Let me know in the comments below.

Dealing with Domain Objects in Spring MVC

I was recently surprised by how one code base had public default constructors (i.e. zero-arguments constructors) in all their domain entities, and had getters and setters for all the fields. As I dug deeper, I found out that the reason why the domain entites are the way they are is largely because the team thinks it was required by the web/MVC framework. And I thought it would be a good opportunity to clear up some misconceptions.

Specifically, we'll look at the following cases:

1. No setter for generated ID field (i.e. the generated ID field has a getter but no setter)
2. No default constructor (e.g. no public zero-arguments constructor)
3. Domain entity with child entities (e.g. child entities are not exposed as a modifiable list)

Binding Web Request Parameters

First, some specifics and some background. Let's base this on a specific web/MVC framework - Spring MVC. When using Spring MVC, its data binding binds request parameters by name. Let's use an example.

@Controller
@RequestMapping("/accounts")
... class ... {
    ...
    @PostMapping
    public ... save(@ModelAttribute Account account, ...) {...}
    ...
}

Given the above controller mapped to "/accounts", where can an Account instance come from?

Based on documentation, Spring MVC will get an instance using the following options:

• From the model if already added via Model (like via @ModelAttribute method in the same controller).
• From the HTTP session via @SessionAttributes.
• From a URI path variable passed through a Converter.
• From the invocation of a default constructor.
• (For Kotlin only) From the invocation of a "primary constructor" with arguments matching to Servlet request parameters; argument names are determined via JavaBeans @ConstructorProperties or via runtime-retained parameter names in the bytecode.

Assuming an Account object is not added in the session, and that there is no @ModelAttribute method, Spring MVC will end up instantiating one using its default constructor, and binding web request parameters by name. For example, the request contains "id" and "name" parameters. Spring MVC will try to bind them to "id" and "name" bean properties by invoking "setId" and "setName" methods, respectively. This follows JavaBean conventions.

No Setter Method for Generated ID Field

Let's start with something simple. Let's say that we have an Account domain entity. It has an ID field that is generated by the persistent store, and only provides a getter method (but no setter method).

@Entity
... class Account {
    @Id @GeneratedValue(...) private Long id;
    ...
    public Account() { ... }
    public Long getId() { return id; }
    // but no setId() method
}

So, how can we have Spring MVC bind request parameters to an Account domain entity? Are we forced to have a public setter method for a field that is generated and read-only?

In our HTML form, we will not place the "id" as a request parameter. We will place it as a path variable instead.

We use a @ModelAttribute method. It is called prior to the request handling method. And it supports pretty much the same parameters as a regular request handling method. In our case, we use it to retrieve an Account domain entity with the given unique identifier, and use it for further binding. Our controller would look something like this.

@Controller
@RequestMapping("/accounts")
... class ... {
    ...
    @ModelAttribute
    public Account populateModel(
            HttpMethod httpMethod,
            @PathVariable(required=false) Long id) {
        if (id != null) {
            return accountRepository.findById(id).orElseThrow(...);
        }
        if (httpMethod == HttpMethod.POST) {
            return new Account();
        }
        return null;
    }

    @PutMapping("/{id}")
    public ... update(...,
            @ModelAttribute @Valid Account account, ...) {
        ...
        accountRepository.save(account);
        return ...;
    }

    @PostMapping
    public ... save(@ModelAttribute @Valid Account account, ...) {
        ...
        accountRepository.save(account);
        return ...;
    }
    ...
}

When updating an existing account, the request would be a PUT to "/accounts/{id}" URI. In this case, our controller needs to retrieve the domain entity with the given unique identifier, and provide the same domain object to Spring MVC for further binding, if any. The "id" field will not need a setter method.

When adding or saving a new account, the request would be a POST to "/accounts". In this case, our controller needs to create a new domain entity with some request parameters, and provide the same domain object to Spring MVC for further binding, if any. For new domain entities, the "id" field is left null. The underlying persistence infrastructure will generate a value upon storing. Still, the "id" field will not need a setter method.

In both cases, the @ModelAttribute method populateModel is called prior to the mapped request handling method. Because of this, we needed to use parameters in populateModel to determine which case it is being used in.

No Default Constructor in Domain Object

Let's say that our Account domain entity does not provide a default constructor (i.e. no zero-arguments constructor).

... class Account {
    public Account(String name) {...}
    ...
    // no public default constructor
    // (i.e. no public zero-arguments constructor)
}

So, how can we have Spring MVC bind request parameters to an Account domain entity? It does not provide a default constructor.

We can use a @ModelAttribute method. In this case, we want to create an Account domain entity with request parameters, and use it for further binding. Our controller would look something like this.

@Controller
@RequestMapping("/accounts")
... class ... {
    ...
    @ModelAttribute
    public Account populateModel(
            HttpMethod httpMethod,
            @PathVariable(required=false) Long id,
            @RequestParam(required=false) String name) {
        if (id != null) {
            return accountRepository.findById(id).orElseThrow(...);
        }
        if (httpMethod == HttpMethod.POST) {
            return new Account(name);
        }
        return null;
    }

    @PutMapping("/{id}")
    public ... update(...,
            @ModelAttribute @Valid Account account, ...) {
        ...
        accountRepository.save(account);
        return ...;
    }

    @PostMapping
    public ... save(@ModelAttribute @Valid Account account, ...) {
        ...
        accountRepository.save(account);
        return ...;
    }
    ...
}

Domain Entity with Child Entities

Now, let's look at a domain entity that has child entities. Something like this.

... class Order {
    private Map<..., OrderItem> items;
    public Order() {...}
    public void addItem(int quantity, ...) {...}
    ...
    public Collection<CartItem> getItems() {
        return Collections.unmodifiableCollection(items.values());
    }
}

... class OrderItem {
    private int quantity;
    // no public default constructor
    ...
}

Note that the items in an order are not exposed as a modifiable list. Spring MVC supports indexed properties and binds them to an array, list, or other naturally ordered collection. But, in this case, the getItems method returns an unmodifiable collection. This means that an exception would be thrown when an object attempts to add/remove items to/from it. So, how can we have Spring MVC bind request parameters to an Order domain entity? Are we forced to expose the order items as a mutable list?

Not really. We must refrain from diluting the domain model with presentation-layer concerns (like Spring MVC). Instead, we make the presentation-layer a client of the domain model. To handle this case, we create another type that complies with Spring MVC, and keep our domain entities agnostic of the presentation layer.

... class OrderForm {
    public static OrderForm fromDomainEntity(Order order) {...}
    ...
    // public default constructor
    // (i.e. public zero-arguments constructor)
    private List<OrderFormItem> items;
    public List<OrderFormItem> getItems() { return items; }
    public void setItems(List<OrderFormItem> items) { this.items = items; }
    public Order toDomainEntity() {...}
}

... class OrderFormItem {
    ...
    private int quantity;
    // public default constructor
    // (i.e. public zero-arguments constructor)
    // public getters and setters
}

Note that it is perfectly all right to create a presentation-layer type that knows about the domain entity. But it is not all right to make the domain entity aware of presentation-layer objects. More specifically, presentation-layer OrderForm knows about the Order domain entity. But Order does not know about presentation-layer OrderForm.

Here's how our controller will look like.

@Controller
@RequestMapping("/orders")
... class ... {
    ...
    @ModelAttribute
    public OrderForm populateModel(
            HttpMethod httpMethod,
            @PathVariable(required=false) Long id,
            @RequestParam(required=false) String name) {
        if (id != null) {
            return OrderForm.fromDomainEntity(
                orderRepository.findById(id).orElseThrow(...));
        }
        if (httpMethod == HttpMethod.POST) {
            return new OrderForm(); // new Order()
        }
        return null;
    }

    @PutMapping("/{id}")
    public ... update(...,
            @ModelAttribute @Valid OrderForm orderForm, ...) {
        ...
        orderRepository.save(orderForm.toDomainEntity());
        return ...;
    }

    @PostMapping
    public ... save(@ModelAttribute @Valid OrderForm orderForm, ...) {
        ...
        orderRepository.save(orderForm.toDomainEntity());
        return ...;
    }
    ...
}

Closing Thoughts

As I've mentioned in previous posts, it is all right to have your domain objects look like a JavaBean with public default zero-arguments constructors, getters, and setters. But if the domain logic starts to get complicated, and requires that some domain objects lose its JavaBean-ness (e.g. no more public zero-arguments constructor, no more setters), do not worry. Define new JavaBean types to satisfy presentation-related concerns. Do not dilute the domain logic.

That's all for now. I hope this helps.

Thanks again to Juno for helping me out with the samples. The relevant pieces of code can be found on GitHub.

Isolating the Domain Logic

In one design patterns class, I had an interesting discussion about modelling domain logic. Specifically, it was about isolating the domain logic. An application would typically be divided into three parts:

1. Presentation (e.g. desktop GUI, browser, web service)
2. Domain logic
3. Infrastructure (e.g. persistence storage, e-mail)

The class found it interesting that the dependency arrows were pointing towards the domain logic part. They asked, “Is the diagram intentionally made wrong? Shouldn’t the domain logic part be dependent on the persistence storage?” It was a great question. And I wanted to share and post the discussion and explanation here.

Often Misunderstood

Most developers would usually have this misunderstanding in mind.

Misunderstood

vs.

Proper

And this misunderstanding is largely due to the sequence of operations. It usually starts with a trigger (e.g. a user clicking a button or a link) in the presentation layer, which then calls something within the domain logic layer, which then calls something within the infrastructure layer (e.g. update a database table record).

While this is the correct sequence of operations, there’s something subtle in the way in which the domain logic layer can be implemented. This has something to do with dependency inversion.

Dependency Inversion Principle

The domain logic layer may need something from the infrastructure layer, like some form of access to retrieve from persistence storage. The usual patterns for this are: DAO and repository. I won’t explain these two patterns here. Instead, I would point out that the interface definitions are placed within the domain logic layer, and their implementations are placed in another separate layer.

Placing the (DAO and repository) interface definitions inside the domain logic layer means that it is the domain logic layer that defines it. It is the one that dictates which methods are needed, and what return types are expected. This also marks the boundaries of the domain logic.

This separation between interface and implementation may be subtle, but key. Placing just the interface definitions allows the domain logic part to be free from infrastructure details, and allows it to be unit-tested without actual implementations. The interfaces can have mock implementations during unit testing. This subtle difference makes a big difference in rapid verification of (the development team’s understanding of) business rules.

This separation is the classic dependency inversion principle in action. Domain logic (higher-level modules) should not depend on DAO and repository implementations (low-level modules). Both should depend on abstractions. The domain logic defines the abstractions, and infrastructure implementations depend on these abstractions.

Most novice teams I’ve seen, place the DAO and repository interfaces together with their infrastructure-specific implementations. For example, say we have an StudentRepository and its JPA-specific implementation StudentJpaRepository. I would usually find novice teams placing them in the same package. While this is fine, since the application will still compile successfully. But the separation is gone, and domain logic is no longer isolated.

Now that I’ve explained why and how the domain logic part does not depend on the infrastructure part, I’d like to touch on how the presentation part is accidentally entangled with the domain logic.

Separated Presentation

Another thing I often see with novice teams is how they end up entangling their domain logic with their presentation. And this results into this nasty cyclic dependency. This cyclic dependency is more logical than physical. Which makes it all the more difficult to detect and prevent.

I won’t use a rich GUI presentation example here, since Martin Fowler has already written a great piece on it. Instead, I’ll use a web-browser-based presentation as an example.

Most web-based systems would use a web framework for its presentation. These frameworks usually implement some form of MVC (model-view-controller). The model used is usually the model straight from the domain logic part. Unfortunately, most MVC frameworks require something about the model. In the Java world, most MVC frameworks require that the model follow JavaBean conventions. Specifically, it requires the model to have a public zero-arguments constructor, and getters and setters. The zero-arguments constructor and setters are used to automatically bind parameters (from HTTP POST) to the model. The getters are used in rendering the model in a view.

Because of this implied requirement by MVC frameworks used in the presentation, developers would add a public zero-arguments constructor, getter and setters, to all their domain entities. And they would justify this as being required. Unfortunately, this gets the in the way of implementing domain logic. It gets entangled with the presentation. And worse, I’ve seen domain entities being polluted with code that emits HTML-encoded strings (e.g. HTML code with less-than and greater-than signs encoded) and XML, just because of presentation.

If it is all right to have your domain entity implemented as a JavaBean, then it would be fine to have it used directly in your presentation. But if the domain logic gets a bit more complicated, and requires the domain entity to lose its JavaBean-ness (e.g. no more public zero-arguments constructor, no more setters), then it would be advisable for the domain logic part to implement domain logic, and have the presentation part adapt by creating another JavaBean object to satisfy its MVC needs.

An example I use often is a UserAccount that is used to authenticate a user. In most cases, when a user wishes to change the password, the old password is also needed. This helps prevent unauthorized changing of the password. This is clearly shown in the code below.

public class UserAccount {
  ...
  public void changePassword(
      String oldPassword, String newPassword) {…}
}

But this does not follow JavaBean conventions. And if the MVC presentation framework would not work well with the changePassword method, a naive approach would be to remove the erring method and add a setPassword method (shown below). This weakens the isolation of the domain logic, and causes the rest of the team to implement it all over the place.

public class UserAccount {
  ...
  public void setPassword(String password) {…}
}

It’s important for developers to understand that the presentation depends on the domain logic. And not the other way around. If the presentation has needs (e.g. JavaBean convention), then it should not have the domain logic comply with that. Instead, the presentation should create additional classes (e.g. JavaBeans) that have knowledge of the corresponding domain entities. But unfortunately, I still see a lot of teams forcing their domain entities to look like JavaBeans just because of presentation, or worse, having domain entities create JavaBeans (e.g. DTOs) for presentation purposes.

Arrangement Tips

Here’s a tip in arranging your application. Keep your domain entities and repositories in one package. Keep your repository and other infrastructure implementations in a separate package. Keep your presentation-related classes in its own package. Be mindful of which package depends on which package. The package that contains the domain logic is preferrably at the center of it all. Everything else depends on it.

When using Java, the packages would look something like this:

• com.acme.myapp.context1.domain.model
  • Keep your domain entities, value objects, and repositories (interface definitions only) here
• com.acme.myapp.context1.infrastructure.persistence.jpa
  • Place your JPA-based repository and other JPA persistence-related implementations here
• com.acme.myapp.context1.infrastructure.persistence.jdbc
  • Place your JDBC-based repository and other JDBC persistence-related implementations here
• com.acme.myapp.context1.presentation.web
  • Place your web/MVC presentation components here. If the domain entities needed for presentation do not comply with MVC framework requirements, create additional classes here. These additional classes will adapt the domain entities for presentation-purposes, and still keep the domain entities separated from presentation.

Note that I’ve used context1, since there could be several contexts (or sub-systems) in a given application (or system). I’ll discuss about having multiple contexts and having multiple models in a future post.

That’s all for now. I hope this short explanation can shed some light to those who wonder why their code is arranged and split in a certain way.

Thanks to Juno Aliento for helping me with the class during this interesting discussion.

Happy holidays!

Architectural Layers and Modeling Domain Logic

As I was discussing the PoEAA patterns used to model domain logic (i.e. transaction script, table module, domain model), I noticed that people get the impression (albeit wrong impression) that the domain model pattern is best. So, they set out to apply it on everything.

Not Worthy of Domain Model Pattern

Let's get real. The majority of sub-systems are CRUD-based. Only a certain portion of the system requires the domain model implementation pattern. Or, put it in another way, there are parts of the application that just needs forms over data, and some validation logic (e.g. required/mandatory fields, min/max values on numbers, min/max length on text). For these, the domain model is not worth the effort.

For these, perhaps an anemic domain model would fit nicely.

Anemic Domain Model Isn't As Bad As It Sounds

The anemic domain model isn't as bad as it sounds. There, I said it (at least here in my blog post).

But how does it look like?

package com.acme.bc.domain.model;
...
@Entity
class Person {
 @Id ... private Long id;
 private String firstName;
 private String lastName;
 // ...
 // getters and setters
}
...
interface PersonRepository /* extends CrudRepository<Person, Long> */ {
 // CRUD methods (e.g. find, find/pagination, update, delete)
}

package com.acme.bc.infrastructure.persistence;
...
class PersonRepositoryJpa implements PersonRepository {
 ...
}

In the presentation layer, the controllers can have access to the repository. The repository does its job of abstracting persistence details.

package com.acme.bc.interfaces.web;

@Controller
class PersonsController {
 private PersonRepository personRepository;
 public PersonsController(PersonRepository personRepository) {...}
 // ...
}

In this case, having the Person class exposed to the presentation layer is perfectly all right. The presentation layer can use it directly, since it has a public zero-arguments constructor, getters and setters, which are most likely needed by the view.

And there you have it. A simple CRUD-based application.

Do you still need a service layer? No. Do you still need DTO (data transfer objects)? No. In this simple case of CRUD, you don't need additional services or DTOs.

Yes, the Person looks like a domain entity. But it does not contain logic, and is simply used to transfer data. So, it's really just a DTO. But this is all right since it does the job of holding the data stored-to and retrieved-from persistence.

Now, if the business logic starts to get more complicated, some entities in the initially anemic domain model can become richer with behavior. And if so, those entities can merit a domain model pattern.

Alternative to Anemic Domain Model

As an alternative to the anemic domain model (discussed above), the classes can be moved out of the domain logic layer and in to the presentation layer. Instead of naming it PersonRepository, it is now named PersonDao.

package com.acme.bc.interfaces.web;

@Entity
class Person {...}

@Controller
class PersonsController {
 private PersonDao personDao;
 public PersonsController(PersonDao personDao) {...}
 // ...
}

interface PersonDao /* extends CrudRepository<Person, Long> */ {
 // CRUD methods (e.g. find, find/pagination, update, delete)
}

package com.acme.bc.infrastructure.persistence;

class PersonDaoJpa implements PersonDao {
 ...
}

Too Much Layering

I think that it would be an overkill if you have to go through a mandatory application service that does not add value.

package com.acme.bc.interfaces.web;
...
@Controller
class PersonsController {
 private PersonService personService;
 public PersonsController(PersonService personService) {...}
 // ...
}

package com.acme.bc.application;
...
@Service
class PersonService {
 private PersonRepository personRepository;
 public PersonService(PersonRepository personRepository) {...}
 // expose repository CRUD methods and pass to repository
 // no value add
}

Keep the repository in the domain.model package. Place the repository implementations in another package (e.g. infrastructure.persistence). But why?

The domain.model package is where the repository is defined. The elements in the domain model dictate what methods are needed in the repository interface definition. Thus, the repository definition is placed in the domain.model package. The repository implementations need to follow what new methods are added (or remove unused ones). This packaging follows the dependency inversion principle. The infrastructure.persistence package depends on the domain.model package, and not the other way around.

Application Services for Transactions

So, when would application services be appropriate? The application services are responsible for driving workflow and coordinating transaction management (e.g. by use of the declarative transaction management support in Spring).

If you find the simple CRUD application needing to start transactions in the presentation-layer controller, then it might be a good sign to move them into an application service. This usually happens when the controller needs to update more than one entity that does not have a single root. The usual example here is transferring amounts between bank accounts. A transaction is needed to ensure that debit and credit both succeed, or both fail.

package sample.domain.model;
...
@Entity
class Account {...}
...
interface AccountRepository {...}

package sample.interfaces.web;
...
@Controller
class AccountsController {
 private AccountRepository accountRepository;
 ...
 @Transactional
 public ... transfer(...) {...}
}

If you see this, then it might be a good idea to move this (from the presentation layer) to an application-layer service.

package sample.interfaces.web;
...
@Controller
class AccountsController {
 private AccountRepository accountRepository;
 private TransferService transferService;
 ...
 public ... transfer(...) {...}
}

package sample.application;
...
@Service
@Transactional
class TransferService {
 private AccountRepository accountRepository;
 ...
 public ... transfer(...) {...}
}

package sample.domain.model;
...
@Entity
class Account {...}
...
interface AccountRepository {...}

Domain Model Pattern (only) for Complex Logic

I'll use the double-entry accounting as an example. But I'm sure there are more complex logic that's better suited.

Let's say we model journal entries and accounts as domain entities. The account contains a balance (a monetary amount). But this amount is not something that one would simply set. A journal entry needs to be created. When the journal entry is posted, it will affect the specified accounts. The account will then update its balance.

package ….accounting.domain.model;
...
/** Immutable */
@Entity
class JournalEntry {
 // zero-sum items
 @ElementCollection
 private Collection<JournalEntryItem> items;
 ...
}
...
/** A value object */
@Embeddable
class JournalEntryItem {...}
...
interface JournalEntryRepository {...}
...
@Entity
class Account {...}
...
interface AccountRepository {...}
...
@Entity
class AccountTransaction {...}
...
interface AccountTransactionRepository {...}

Now, in this case, a naive implementation would have a presentation-layer controller create a journal entry object, and use a repository to save it. And at some point in time (or if auto-posting is used), the corresponding account transactions are created, with account balances updated. All this needs to be rolled into a transaction (i.e. all-or-nothing).

Again, this transaction is ideally moved to an application service.

package ….accounting.application;

@Service
@Transactional
class PostingService {...}

If there's a need to allow the user to browse through journal entries and account transactions, the presentation-layer controller can directly use the corresponding repositories. If the domain entities are not suitable for the view technology (e.g. it doesn't follow JavaBean naming conventions), then the presentation-layer can define DTOs that are suitable for the view. Be careful! Don't change the domain entity just to suit the needs of the presentation-layer.

package ….interfaces.web;

@Controller
class AccountsController {
 private AccountRepository accountRepository;
 private AccountTransactionRepository accountTransactionRepository;
 private PostingService postingService;
  ...
}

In Closing...

So, there you have it. Hopefully, this post can shed some light on when (and when not) to use domain model pattern.

Now I think I need a cold one.

One-shot Delete with Hibernate (JPA)

In older versions of Hibernate, I can see the one-shot delete indicated in the manual. But newer versions no longer have this section. I'm not sure why. So, in this post, I take a look if it still works.

The one-shot delete section says:

Deleting collection elements one by one can sometimes be extremely inefficient. Hibernate knows not to do that in the case of an newly-empty collection (if you called list.clear(), for example). In this case, Hibernate will issue a single DELETE.

Suppose you added a single element to a collection of size twenty and then remove two elements. Hibernate will issue one INSERT statement and two DELETE statements, unless the collection is a bag. This is certainly desirable.

However, suppose that we remove eighteen elements, leaving two and then add thee new elements. There are two possible ways to proceed

• delete eighteen rows one by one and then insert three rows
• remove the whole collection in one SQL DELETE and insert all five current elements one by one

Hibernate cannot know that the second option is probably quicker. It would probably be undesirable for Hibernate to be that intuitive as such behavior might confuse database triggers, etc.

Fortunately, you can force this behavior (i.e. the second strategy) at any time by discarding (i.e. dereferencing) the original collection and returning a newly instantiated collection with all the current elements.

One-shot-delete does not apply to collections mapped inverse="true".

The inverse="true" is for (Hibernate Mapping) XML. But in this post, we'll see how "one-shot delete" works in JPA (with Hibernate as the provider).

We will try different approaches and see which one will result to a one-shot delete.

1. Bi-directional one-to-many
2. Uni-directional one-to-many (with join table)
3. Uni-directional one-to-many (with no join table)
4. Uni-directional one-to-many (using ElementCollection)

We'll use a Cart entity with many CartItems.

Bi-directional One-to-Many

For this, we have references from both sides.

@Entity
public class Cart { ...
 @OneToMany(mappedBy="cart", cascade=ALL, orphanRemoval=true)
 Collection<OrderItem> items;
}

@Entity
public class CartItem { ...
 @ManyToOne Cart cart;
}

To test this, we insert one row to the table for Cart, and three or more rows to the table for CartItem. Then, we run the test.

public class CartTests { ...
 @Test
 public void testOneShotDelete() throws Exception {
  Cart cart = entityManager.find(Cart.class, 53L);
  for (CartItem item : cart.items) {
   item.cart = null; // remove reference to cart
  }
  cart.items.clear(); // as indicated in Hibernate manual
  entityManager.flush(); // just so SQL commands can be seen
 }
}

The SQL commands shown had each item deleted individually (and not as a one-shot delete).

delete from CartItem where id=?
delete from CartItem where id=?
delete from CartItem where id=?

Discarding the original collection did not work either. It even caused an exception.

public class CartTests { ...
 @Test
 public void testOneShotDelete() throws Exception {
  Cart cart = entityManager.find(Cart.class, 53L);
  // remove reference to cart
  cart.items = new LinkedList<CartItem>(); // discard, and use new collection
  entityManager.flush(); // just so SQL commands can be seen
 }
}

javax.persistence.PersistenceException:
    org.hibernate.HibernateException:
        A collection with cascade="all-delete-orphan" was no longer referenced by the owning entity instance: ….Cart.items

I tested this with Hibernate 4.3.11 and HSQL 2.3.2. If your results vary, please hit the comments.

Uni-directional One-to-Many (With Join Table)

For this, we make changes to the mapping. This causes a join table to be created.

@Entity
public class Cart { ...
 @OneToMany(cascade=ALL)
 Collection<OrderItem> items;
}

@Entity
public class CartItem { ...
 // no @ManyToOne Cart cart;
}

Again, we insert one row to the table for Cart, and three or more rows to the table for CartItem. We also have to insert appropriate records to the join table (Cart_CartItem). Then, we run the test.

public class CartTests { ...
 @Test
 public void testOneShotDelete() throws Exception {
  Cart cart = entityManager.find(Cart.class, 53L);
  cart.items.clear(); // as indicated in Hibernate manual
  entityManager.flush(); // just so SQL commands can be seen
 }
}

The SQL commands shown had the associated rows in the join table deleted (with one command). But the rows in the table for CartItem still exist (and did not get deleted).

delete from Cart_CartItem where cart_id=?
// no delete commands for CartItem

Hmmm, not exactly what we want, since the rows in the table for CartItem still exist.

Uni-directional One-to-Many (No Join Table)

Starting with JPA 2.0, the join table can be avoided in a uni-directional one-to-many by specifying a @JoinColumn.

@Entity
public class Cart { ...
 @OneToMany(cascade=CascadeType.ALL, orphanRemoval=true)
 @JoinColumn(name="cart_id", updatable=false, nullable=false)
 Collection<OrderItem> items;
}

@Entity
public class CartItem { ...
 // no @ManyToOne Cart cart;
}

Again, we insert one row to the table for Cart, and three or more rows to the table for CartItem. Then, we run the test.

public class CartTests { ...
 @Test
 public void testOneShotDelete() throws Exception {
  Cart cart = entityManager.find(Cart.class, 53L);
  cart.items.clear(); // as indicated in Hibernate manual
  entityManager.flush(); // just so SQL commands can be seen
 }
}

Discarding the original collection also did not work either. It also caused the same exception (as with bi-directional one-to-many).

javax.persistence.PersistenceException:
    org.hibernate.HibernateException:
        A collection with cascade="all-delete-orphan" was no longer referenced by the owning entity instance: ….Cart.items

Uni-directional One-to-Many (with ElementCollection)

JPA 2.0 introduced @ElementCollection. This allows one-to-many relationships to be established with the many-side being either @Basic or @Embeddable (i.e. not an @Entity).

@Entity
public class Cart { ...
 @ElementCollection // @OneToMany for basic and embeddables
 @CollectionTable(name="CartItem") // defaults to "Cart_items" if not overridden
 Collection<OrderItem> items;
}

@Embeddable // not an entity!
public class CartItem {
 // no @Id
 // no @ManyToOne Cart cart;
 private String data; // just so that there are columns we can set
}

Again, we insert one row to the table for Cart, and three or more rows to the table for CartItem. Then, we run the test.

public class CartTests { ...
 @Test
 public void testOneShotDelete() throws Exception {
  Cart cart = entityManager.find(Cart.class, 53L);
  cart.items.clear(); // as indicated in Hibernate manual
  entityManager.flush(); // just so SQL commands can be seen
 }
}

Yey! The associated rows for CartItem were deleted in one shot.

delete from CartItem where Cart_id=?

Closing Thoughts

One-shot delete occurs with uni-directional one-to-many using ElementCollection (where the many-side is an embeddabled, and not an entity).

In the uni-directional one-to-many with join table scenario, deleting entries in a join table doesn't add much value.

I'm not sure why one-shot delete works (or why it works this way) in Hibernate. But I do have a guess. And that is the underlying JPA provider could not do a one-shot delete because it could not ensure that the many-side entity is not referenced by other entities. Unlike the ElementCollection, the many-side is not an entity and cannot be referenced by other entities.

Now, this does not mean that you have to use ElementCollection all the time. Perhaps the one-shot delete only applies to aggregate roots. In those cases, using Embeddable and ElementCollection might be appropriate for a collection of value objects that make up an aggregate. When the aggregate root is removed, then it would be good to see that the "child" objects should be removed as well (and in an efficient manner).

I wish there was a way in JPA to indicate that the child entities are privately owned and can be safely removed when the parent entity is removed (e.g. similar to @PrivateOwned in EclipseLink). Let's see if it will be included in a future version of the API.

Hope this helps.

Reference by Identity in JPA

In a previous post, I mentioned that I opted to reference other aggregates by their primary key, and not by type. I usually use this approach (a.k.a. disconnected domain model) when working with large or complex domain models. In this post, let me try to explain further how it can be done in JPA. Note that the resulting DDL scripts will not create a foreign key constraint (unlike the one shown in the previous post).

Reference by Identity

In most JPA examples, every entity references another entity, or is being referenced by another entity. This results into an object model that allows traversal from one entity to any other entity. This can cause unwanted traversals (and unwanted cascade of persistence operations). As such, it would be good to prevent this, by referencing other entities by ID (and not by type).

The code below shows how OrderItem references a Product entity by its primary key (and not by type).

@Entity
public class Product {
 @Id private Long id;
 // ...
}

@Entity
public class Order {
 // ...
 @OneToMany(mappedBy="order")
 private Collection<OrderItem> items;
}

@Entity
public class OrderItem {
 // ...
 @ManyToOne
 private Order order;
 // @ManyToOne
 // private Product product;
 private Long productId;
 // ...
}

There are several ways to get the associated Product entities. One way is to use a repository to find products given the IDs (ProductRepository with a findByIdIn(List<Long> ids) method). As mentioned in previous comments, please be careful not to end up with the N+1 selects problem.

Custom identity types can also be used. The example below uses ProductId. It is a value object. And because of JPA, we needed to add a zero-arguments constructor.

@Embeddable
public class ProductId {
 private Long id;
 public ProductId(long id) {
  this.id = id;
 }
 public long getValue() { return id; }
 // equals and hashCode
 protected ProductId() { /* as required by JPA */ }
}

@Entity
public class Product {
 @EmbeddedId private ProductId id;
 // ...
}

@Entity
public class Order { // ...
 @OneToMany(mappedBy="order")
 private Collection<OrderItem> items;
}

@Entity
public class OrderItem {
 // ...
 @ManyToOne
 private Order order;
 // @ManyToOne
 // private Product product;
 @Embedded private ProductId productId;
 // ...
}

But this will not work if you're using generated values for IDs. Fortunately, starting with JPA 2.0, there are some tricks around this, which I'll share in the next section.

Generated IDs

In JPA, when using non-@Basic types as @Id, we can no longer use @GeneratedValue. But using a mix of property and field access, we can still use generated value and ProductId.

@Embeddable
@Access(AccessType.FIELD)
public class ProductId {...}

@Entity
@Access(AccessType.FIELD)
public class Product {
 @Transient private ProductId id;
 public ProductId getId() { return id; }
 // ...
 private Long id_;
 @Id
 @GeneratedValue(strategy=...)
 @Access(AccessType.PROPERTY)
 protected Long getId_() { return id_; }
 protected void setId_(Long id_) {
  this.id_ = id_;
  this.id = new ProductId(this.id_);
 }
}

@Entity
public class Order { // ...
 @OneToMany(mappedBy="order")
 private Collection<OrderItem> items;
}

@Entity
public class OrderItem {
 // ...
 @ManyToOne
 private Order order;
 // @ManyToOne
 // private Product product;
 @Embedded private ProductId productId;
 // ...
}

The trick involves using property access for the generated ID value (while keeping the rest with field access). This causes JPA to use the setter method. And in it, we initialize the ProductId field. Note that the ProductId field is not persisted (marked as @Transient).

Hope this helps.

Domain Entities and Draft Forms

Recently, I've been reviewing a funds transfer system that uses the S.W.I.F.T. standard message types. After reviewing it, I find that splitting the domain entity from its draft form would make it easier to maintain. Let me explain further.

Background of Funds Transfer System

The system allows users to record funds transfer requests as S.W.I.F.T. messages. Note that the system users are familiar with the said message types, and are not afraid to enter the values into a form that displays all the possible fields. To put things in perspective, a typical message type like MT 103 (Single Customer Credit Transfer) has about 25 fields. Some fields can have several formats. For example, the Ordering Customer (field 50a), can be in one of three formats: option A, option K, and option F. If we include every possible format, MT 103 would easily have over 70 fields.

Modeling Entities

Based on Eric Evan's DDD book, domain entities have a clear identity and a life-cycle with state transitions that we care about.

The funds transfer system modeled each MT as a domain entity. The MT domain entity has a clear identity, and a life-cycle (from draft to approved). It is mapped to its own table with its own corresponding columns to represent its fields. The message fields are implemented as properties. All fields are mutable (via setters methods).

At this point, you might pause and think about anemic domain models. You're probably thinking that there is hardly any behavior on these MT objects. You're probably thinking that these MT objects are just bags of getters and setters. And you're probably correct. But there's a reason for this. Read on.

The system validates the MTs, and displays which fields have errors. But the errors did not stop the system from persisting/storing the message. The system allowed the draft message to be stored for further modification (until it is approved).

This analysis led me to re-think about how the system modeled messages as entities. While it is tempting to think of them as rich domain entities that represent a real-world funds/credit transfer, they're really not. They're actually request forms. Much like the real-world paper forms that one fills-out, and submits for approval, and possibly being re-submitted due to corrections.

Given this, I would model the draft message as another domain entity that I'll call MessageTypeForm (or MT103Form for MT 103). This domain entity will use a map of fields (still in compliance with S.W.I.F.T. messages, where each field is keyed by an alpha-numeric string) (e.g. get("50a")), and not use a separate property for each field (e.g. getOrderingCustomer()). By using a map of fields, the entity would require lesser code, and can still be persisted (or mapped to table rows).

The CreditTransfer (or SingleCustomerCreditTransfer) would be another entity (not the same as the entity with a map of fields — MT103Form). This credit transfer entity shall have a clear identity and a life-cycle (e.g. being amended or cancelled). It can have a reference (by ID, and not by type) to the MT103Form entity from which it was created. It is also easier to establish invariants on this entity, as opposed to having all properties being mutable.

The validation of MT103Form will have to change from using properties (e.g. getOrderingCustomer()) to using a get method with a key parameter (to determine the message field it wants to retrieve) (e.g. get("50a")).

Deeper Insight

Just because a domain entity is largely made up of getters and setters, does not mean it is anemic.

The "draft/form" entity may look like a bag of getters and setters. But it has an identity, and a life-cycle (from draft to approved). So, it is an entity. Many "draft/form" instances will exist in the system simultaneously. The different instances may even have the same field values, but it is important for us to be able to track individual "draft/form" instances.

Discovering the distinction between the "draft/form" and "fund/credit transfer" entities have made things easier to implement. Having two separate domain entities has made the model better (reflect the problem it is trying to solve).

This deeper insight would not have been possible without the help of the domain experts, and my team mates: Tin, Anson, Richie, and Tina. Thanks guys!

One Thing Good Spring Developers Know

In my recent training sessions on the (core) Spring Framework, I was asked, "If there was one thing that a (Java) Spring developer should know, what should that be?" That question caught me off guard. Yes, the (core) Spring Framework does cover a lot of areas (e.g. beans, configuration, aspect-oriented programming, transactions). And it was difficult for me to point out just one thing. I ended up mentioning everything that we covered in our (3 day) training course.

If there was one thing that a (Java) Spring developer should know, what should that be?

As I gave that question more thought, I began to think about the most important one. I ended up thinking of how Spring uses aspects to add behavior to managed objects (usually called beans) as the most important. This is how the Spring Framework supports transactions, security, scope, Java-based configuration, among others. And I'm sharing my thoughts here in this post.

I ended up thinking of how Spring uses aspects to add behavior to managed objects (usually called beans) as the most important.

ORM and Lazy Loading Exceptions

Most developers who use some form of ORM have encountered an exception that signifies that child entities could not be loaded (e.g. LazyInitializationException).

Some developers who have encountered this would use an "open session in view" (OSIV) pattern to keep the session open and prevent this exception from happening. But I find this to be an overkill. Worse, some developers consider the "open session in view" pattern to be the only solution. A possible underlying cause for this misconception could be that the developer is probably not armed with the knowledge of using the Spring Framework effectively to keep the ORM session open longer.

In the case of JPA, the "open entity manager in view" pattern will create an entity manager at the beginning of the request, bind it to the request thread, and close it when the response is completed.

So, if not the OSIV pattern, what would be a better solution?

The short answer is to use the Spring Framework to keep the session open for the duration that you need it (e.g. @Transactional). Keep on reading as I'll provide a longer answer.

Services and Repositories

In a layered architecture, a typical design pattern is to define a domain or application service (usually defined as an interface) to provide business functionality (e.g. start using a shopping cart, adding items to that shopping cart, searching for products). Domain and application service implementations would typically delegate the retrieval/persistence of domain entities to repositories.

Presentation Layer

Business Layer

Data Access (or Infrastructure) Layer

Repositories (or data access objects) are also defined as interfaces to retrieve/persist domain entities (i.e. provide ORM and CRUD access). Naturally, repository implementations use ORM libraries (e.g. JPA/Hibernate, myBATIS) to retrieve and persist domain entities. With this, it uses the ORM framework's classes to connect to the persistent store, retrieve/persist the entity, and close the connection (called session in Hibernate). There's no problem of lazy loading failures at this point.

The problem of lazy loading failures occur when the service retrieves a domain entity using the repository, and wants to load child entities (after the repository method has returned). By the time the repository returns the domain entity, the ORM session gets closed. Because of this, attempts to access/load child entities in the domain service cause an exception.

The code snippets below illustrate how a lazy loading exception can occur when the child items of an order entity is lazily loaded after being returned by the repository.

@Entity
public class Order {
    @OneToMany // defaults to FetchType.LAZY
    private List<OrderItem> items;
    …
    public List<OrderItem> getItems() {…}
}

public class SomeApplicationServiceImpl implements SomeApplicationService {
    private OrderRepository orderRepository;
    …
    @Override
    public void method1(…) {
        …
        order = orderRepository.findById(...);
        order.getItems(); // <-- Lazy loading exception occurs!
        …
    }
    …
}

public class OrderRepositoryImpl implements OrderRepository {
    @PersistenceContext
    private EntityManager em;
    …
    @Override
    public Order findById(...) {...}
    …
}

The repository implementation explicitly uses JPA for its ORM (as illustrated with the use of an EntityManager).

At this point, some developers may opt to use eager fetch to prevent the lazy initialization exception. Telling the ORM to eagerly fetch the child items of an order entity will work. But sometimes, we don't need to load the child items. And eagerly loading this might be unnecessary overhead. It would be great to only load it when we need it.

To prevent the lazy initialization exception (and not be forced to eagerly fetch), we'll need to keep the ORM session open until the calling service method returns. In Spring, it can be as simple as annotating the service method as @Transactional to keep the session open. I find that this approach is better than using "open session in view" pattern (or being forced to use eager fetching), since it keeps the session open only for the duration that we intend it to be.

public class SomeApplicationServiceImpl implements SomeApplicationService {
    private OrderRepository orderRepository;
    …
    @Override
    @Transactional // <-- open the session (if it's not yet open)
    public void method1(…) {
        …
        order = orderRepository.findById(...);
        order.getItems(); // <-- Lazy loading exception should not happen
        …
    }
    …
}

Domain Entities in the Presentation Layer

Even after keeping the ORM session open in the service layer (beyond the repository implementation objects), the lazy initialization exception can still occur when we expose the domain entities to the presentation layer. Again, because of this, some developers prefer the OSIV approach, since it will also prevent lazy initialization exceptions in the presentation layer.

But why would you want to expose domain entities in the presentation layer?

From experience, I've worked with teams who prefer to expose domain entities in the presentation layer. This usually leads to anemic domain model, since presentation layer frameworks need a way to bind input values to the object. This forces domain entities to have getter and setter methods, and a zero-arguments constructor. Having getters and setters will make invariants difficult to enforce. For simple domains, this is workable. But for more complex domains, a richer domain model would be preferred, as it would be easier to enforce invariants.

In a richer domain model, the objects that represent the presentation layer input/output values are actually data transfer objects (DTOs). They represent inputs (or commands) that are carried out in the domain layer. With this in mind, I prefer to use DTOs and maintain a richer domain model. Thus, I don't really run into lazy initialization exceptions in the presentation layer.

Aspects to add behavior to managed objects

Spring intercepts calls to these @Transactional annotated methods to ensure that an ORM session is open.

Transactions (or simply keeping an ORM session open) are not the only behavior provided using aspects. There's security, scope, Java-based configuration, and others. Knowing that the Spring Framework uses aspects to add behavior is one of the key reasons why we let Spring manage the POJOs that we develop.

Conclusion

There you go. That for me is the one most important thing that a Spring Framework developer should know when using the core. Now that I've given my opinion as to what is the one most important thing, how about you? What do you think is the one most important thing to know when tackling Core Spring. Cheers!

SOA and Referential Data Integrity

One of the issues that tends to pop up is how we maintain referential integrity between services. In this post, I'd like to share my experiences on how referential data integrity between services can ruin your SOA.

Database Constraints Between Services Break Boundaries

Two of the Tenets of Service-Orientation are: "boundaries are explicit", and "services are autonomous". The first one implies that internal (private) implementation details should not be leaked outside of a service boundary. And the second one implies that services are not subservient to other services (or other pieces of code).

For purposes of discussion, let's say we have the following:

• a "customer" service that provides customer-related business capability and persists data in a database
• an "order" service that provides order-related business capability and persists data in the same database
• foreign key constraints between customer-related entities/tables and order-related entities/tables. More specifically, an "order" table contains the unique ID of a customer that placed the order, and that this ID needs to exists in the "customer" table.

Do the above services (customer and order) follow the "boundaries are explicit" tenet? Are the services autonomous? Let's examine further.

The way the "customer" service persists data in a database is (private) implementation detail that is internal to it. Likewise, the way the "order" service persists data in a database is also internal to it. But how would you consider the foreign key constraints between their database tables? Is this internal implementation detail leaking outside of a service boundary? (i.e. leaking outside the "customer" service boundary and onto the "order" service boundary)

If the "order"-related database undergoes some schema changes, will it not affect the "customer"-related database schema? When deploying the schema changes to the "order" service, will it not require the "customer" service to be temporarily unavailable? (e.g. due to database restart) If "services are autonomous", how come the "customer" and "order" services are inter-dependent, such that a change in one requires a restart (or a redeploy) on the other?

Split Service, Split Database

A better approach would be to split the databases of the two services, and do away with foreign key constraints. That would allow for explicit boundaries, and autonomy. But this might be unacceptable to some people at first.

A monolithic application (left) split into services in a service-oriented architecture (right).

How could one ensure that enough customer information is received before orders are placed by that customer? In other words, how can a developer ensure that orders are placed by known customers (an existing customer ID)? Well, in SOA, services don't have to! It is not the responsibility of services to maintain this referential integrity. The responsibility of ensuring that an order is placed by a known customer lies with the process of placing orders (not in the service). It is the orchestration layer's responsibility to maintain this.

What if some customer information was modified (e.g. billing address), shouldn't the related orders be affected? Again, the process, or orchestration layer, can become responsible for this. Is this really a change in the customer? Or is it just a change in the placed order? One possible way is we define the process to say that it would copy the customer's billing information, and attach a copy (i.e. duplicate) with the placed order. This would mean that the order's bill-to (billing) address is defined when the order was placed. Another possible way is to leave the bill-to address undetermined until the order is shipped, at which the billing address as provided by the customer service shall be used.

Correlation Between Services and Context Boundaries

A useful way of thinking about this is the Domain-Driven Design notion of Bounded Context. DDD splits a complex domain up into multiple bounded contexts and maps out the relationships between them. And this results into multiple databases.

Re-usable Services

Services (in SOA) are meant to be reusable. In the example, if we reconsider what the customer service contains, we can probably design it in such a way that it doesn't have to be for the purposes of order taking. It can be designed to be reusable and allow for any party's information (e.g. persons, organizations) and not just customers. It could be re-used to store employees, since employees can become customers in the future. It could be re-used to store suppliers (since the business may need to track the suppliers of products being produced and ordered).

The (business) goals of SOA any IT initiative are:

• increase agility (e.g. support new/changing business processes/models, reduce time to solution)
• reduce cost (e.g. re-use business processes and/or applications, improve utility of existing/legacy application)

These goals are further translated as (technical goals):

• increase usability (i.e. re-usability and accessibility across different applications)
• improve maintainability
• reduce redundancy

When we make the boundaries of services explicit and make them autonomous, we can better achieve the goals of re-usability, and reduce redundancy.

Some other services, that come to mind, can become re-usable (when properly designed and split), are:

• Authentication and authorization - if this were (re-)written for each application, it would be a huge cost.
• Billing (or invoicing)
• Product Catalog

Microservices

In a previous post on SOA, I did mention that I find the term Microservices to be misleading. Although the information found on the web were good, I found them to still be unclear with their implementation. Nonetheless, I did find that they do add an exciting twist if you consider that, Microservices:

• can contain their own GUI,
• and use their own database (i.e. decentralized data management).

Closing Thoughts

Finally, when communicating with business people, don't let reuse become the primary measure. They probably won't understand it. Tell them that it does help save development and maintenance costs. Tell them that it provides better time to market, reduce days of inventory, reduce employee turnover, etc.

Nice root beer from Virgil's Sodas. Is this available here in the Philippines? I'm not related to the product in any way. I just saw their Ad online. And being thirsty, I thought of having one.

After this rather long post, I think it's time for a nice cold drink. Root beer anyone?

Domain-Driven Design: Cargo Shipping Example (Part 2)

One of my colleagues asked an interesting question about my previous post that I thought would be worth posting here. He asked about the persistence of value objects. He pointed out that the Cargo had a Delivery property that was replaced (not modified). So, what happens to its persistence? Does it result into a delete and an insert? Let's see.

Let's see how a Delivery object is used in a Cargo object. In the highlighted lines, we can see that the property is indeed being assigned (not modified or updated) with a new delivery object.

public class Cargo implements Entity<Cargo> {

  private TrackingId trackingId;
  private Location origin;
  private RouteSpecification routeSpecification;
  private Itinerary itinerary;
  private Delivery delivery;

  public Cargo(final TrackingId trackingId, final RouteSpecification routeSpecification) {
    Validate.notNull(trackingId, "Tracking ID is required");
    Validate.notNull(routeSpecification, "Route specification is required");

    this.trackingId = trackingId;
    // Cargo origin never changes, even if the route specification changes.
    // However, at creation, cargo orgin can be derived from the initial route specification.
    this.origin = routeSpecification.origin();
    this.routeSpecification = routeSpecification;

    this.delivery = Delivery.derivedFrom(
      this.routeSpecification, this.itinerary, HandlingHistory.EMPTY
    );
  }

  public TrackingId trackingId() { return trackingId; }

  public Location origin() { return origin; }

  public Delivery delivery() { return delivery; }

  public Itinerary itinerary() {
    return DomainObjectUtils.nullSafe(
        this.itinerary, Itinerary.EMPTY_ITINERARY);
  }

  public RouteSpecification routeSpecification() { return routeSpecification; }

  public void specifyNewRoute(final RouteSpecification routeSpecification) {
    Validate.notNull(routeSpecification, "Route specification is required");

    this.routeSpecification = routeSpecification;
    // Handling consistency within the Cargo aggregate synchronously
    this.delivery = delivery.updateOnRouting(this.routeSpecification, this.itinerary);
  }

  public void assignToRoute(final Itinerary itinerary) {
    Validate.notNull(itinerary, "Itinerary is required for assignment");

    this.itinerary = itinerary;
    // Handling consistency within the Cargo aggregate synchronously
    this.delivery = delivery.updateOnRouting(this.routeSpecification, this.itinerary);
  }

  public void deriveDeliveryProgress(final HandlingHistory handlingHistory) {
    // TODO filter events on cargo (must be same as this cargo)

    // Delivery is a value object, so we can simply discard the old one
    // and replace it with a new
    this.delivery = Delivery.derivedFrom(routeSpecification(), itinerary(), handlingHistory);
  }
  . . .
}

Here is the Delivery value object (summarized). It doesn't have mutator methods. The class is immutable. Note that updateOnRouting() and derivedFrom() methods return new Delivery objects.

public class Delivery implements ValueObject<Delivery> {

  private TransportStatus transportStatus;
  private Location lastKnownLocation;
  private Voyage currentVoyage;
  private boolean misdirected;
  private Date eta;
  private HandlingActivity nextExpectedActivity;
  private boolean isUnloadedAtDestination;
  private RoutingStatus routingStatus;
  private Date calculatedAt;
  private HandlingEvent lastEvent;

  private static final Date ETA_UNKOWN = null;
  private static final HandlingActivity NO_ACTIVITY = null;

  /**
   * Creates a new delivery snapshot to reflect changes in routing, i.e.
   * when the route specification or the itinerary has changed
   * but no additional handling of the cargo has been performed.
   *
   * @param routeSpecification route specification
   * @param itinerary itinerary
   * @return An up to date delivery
   */
  Delivery updateOnRouting(RouteSpecification routeSpecification, Itinerary itinerary) {
    Validate.notNull(routeSpecification, "Route specification is required");

    return new Delivery(this.lastEvent, itinerary, routeSpecification);
  }

  /**
   * Creates a new delivery snapshot based on the complete handling history of a cargo,
   * as well as its route specification and itinerary.
   *
   * @param routeSpecification route specification
   * @param itinerary itinerary
   * @param handlingHistory delivery history
   * @return An up to date delivery.
   */
  static Delivery derivedFrom(
       RouteSpecification routeSpecification,
       Itinerary itinerary,
       HandlingHistory handlingHistory) {
    Validate.notNull(routeSpecification, "Route specification is required");
    Validate.notNull(handlingHistory, "Delivery history is required");

    final HandlingEvent lastEvent = handlingHistory.mostRecentlyCompletedEvent();

    return new Delivery(lastEvent, itinerary, routeSpecification);
  }

  /**
   * Internal constructor.
   */
  private Delivery(
        HandlingEvent lastEvent, Itinerary itinerary,
        RouteSpecification routeSpecification) {
    this.calculatedAt = new Date();
    this.lastEvent = lastEvent;

    this.misdirected = calculateMisdirectionStatus(itinerary);
    this.routingStatus = calculateRoutingStatus(itinerary, routeSpecification);
    this.transportStatus = calculateTransportStatus();
    this.lastKnownLocation = calculateLastKnownLocation();
    this.currentVoyage = calculateCurrentVoyage();
    this.eta = calculateEta(itinerary);
    this.nextExpectedActivity = calculateNextExpectedActivity(routeSpecification, itinerary);
    this.isUnloadedAtDestination = calculateUnloadedAtDestination(routeSpecification);
  }

  public TransportStatus transportStatus() {...}

  public Location lastKnownLocation() {...}

  public Voyage currentVoyage() {...}

  public boolean isMisdirected() {...}

  public Date estimatedTimeOfArrival() {...}

  public HandlingActivity nextExpectedActivity() {...}

  public boolean isUnloadedAtDestination() {...}

  public RoutingStatus routingStatus() {...}

  public Date calculatedAt() {...}
  . . .
}

The above entities don't use JPA annotations. So, I dug into the example's ORM which happens to be Hibernate (Cargo.hbm.xml). I saw the following:

<hibernate-mapping default-access="field">
  <class name="se.citerus.dddsample.domain.model.cargo.Cargo" table="Cargo">
    <id name="id" column="id">...</id>

    <many-to-one name="origin" column="origin_id" not-null="false" cascade="none" update="false" foreign-key="origin_fk"/>

    <component name="trackingId" unique="true" update="false">...</component>

    <component name="delivery" lazy="true">
      <property name="misdirected" column="is_misdirected" not-null="true"/>
      <property name="eta" column="eta" not-null="false"/>
      <property name="calculatedAt" column="calculated_at" not-null="true"/>
      <property name="isUnloadedAtDestination" column="unloaded_at_dest" not-null="true"/>

      <property name="routingStatus" column="routing_status" not-null="true">...</property>

      <component name="nextExpectedActivity" update="true">
        <many-to-one name="location" column="next_expected_location_id" foreign-key="next_expected_location_fk" cascade="none"/>
        <property name="type" column="next_expected_handling_event_type">...</property>
        <many-to-one name="voyage" column="next_expected_voyage_id" foreign-key="next_expected_voyage_fk" cascade="none"/>
      </component>

      <property name="transportStatus" column="transport_status" not-null="true">...</property>
      <many-to-one name="currentVoyage" column="current_voyage_id" not-null="false" cascade="none" foreign-key="current_voyage_fk"/>
      <many-to-one name="lastKnownLocation" column="last_known_location_id" not-null="false" cascade="none" foreign-key="last_known_location_fk"/>
      <many-to-one name="lastEvent" column="last_event_id" not-null="false" cascade="none" foreign-key="last_event_fk"/>
    </component>
    ...
  </class>
</hibernate-mapping>

The JPA annotation counterpart of <component> is @Embedded.

The delivery value object is declared as a <component>. A component is a contained object that is persisted as a value type and not an entity reference. That means, it does not have its own table. Instead, its properties are mapped as columns in the surrounding <class>'s table. So here, the delivery object's properties are actually columns in the Cargo table.

Also note how Hibernate was configured to access by field and not by property. This tells Hibernate to use direct field injection, and not use setter methods.

So, does it result into a delete and an insert? The answer is neither. It results into an update!

Note that not all value objects will be persisted this way. The Leg value objects are deleted whenever a new Itinerary object is assigned to a Cargo. This results into Leg value objects being inserted (after the old ones are deleted). So, please check your ORM and persistence mechanisms to be sure, and see if there are any performance problems.

But why all the hassle just to use value objects? Good question. Here's what the cargo shipping example has to say:

When possible, we tend to favor Value Objects over Entities or Domain Events, because they require less attention during implementation. Value Objects can be created and thrown away at will, and since they are immutable we can pass them around as we wish.

So, I guess in the cargo shipping example, the likelihood of an itinerary being replaced is quite minimal. Thus, they didn't have performance problems even when the child leg objects were being deleted (and new ones inserted).

Domain-Driven Design: Cargo Shipping Example

I've always found the cargo shipping example used in Eric Evan's book to be quite useful in learning DDD. Here are some of my notes.

Bounded Contexts

Eric Evans did mention in his talk "What I've Learned About DDD Since the Book" at QCon London 2009, that ...in Chapter 14 of the book, I finally got around to talking about context boundaries and context maps... but putting context mapping in Chapter 14 was a mistake. It is fundamental. [You] just can't make models work without establishing context boundaries.. Evans said that the reality is that there is always more than one model and mapping these models out is crucial to setting the stage for success of a DDD project.

Ideally, it would be preferable to have a single, unified model. While this is a noble goal, in reality it typically fragments into multiple models. It is useful to recognize this fact of life and work with it. (Wikipedia)

I believe a lot of teams are struggling with DDD because they're looking for a single, cohesive, all-inclusive model of an organization's entire business domain — you know, like an enterprise model. However, when using DDD, that is not the goal. DDD places emphasis on developing models within a context — A description of the conditions under which a particular model applies. Thus, the strategic design principle of Bounded Context.

Unfortunately, the cargo shipping example has only one bounded context — cargo shipping. I was hoping that it had two or more bounded contexts, so that it can illustrate how context mapping works.

Could carrier movement tracking be another context? What about a billing context (where a someone is billed when the cargo is delivered)?

The packages could have been named cargoshipping. But the packages were named as follows:

• se.citerus.dddsample.application
• se.citerus.dddsample.domain.model
• se.citerus.dddsample.infrastructure
• se.citerus.dddsample.interfaces

In Vernon Vaughn's book, a similar naming convention was suggested (for the "optimal purchasing" context):

• com.mycompany.optimalpurchasing.application
• com.mycompany.optimalpurchasing.domain.model
• com.mycompany.optimalpurchasing.infrastructure
• com.mycompany.optimalpurchasing.presentation

Applications

There are three applications (not three contexts) in the example:

• Booking
• Tracking
• Handling (or Incident Logging)

Aggregate Roots

There is one package per aggregate, and to each aggregate belongs entities, value objects, domain events, a repository interface and sometimes factories. The aggregate roots are Cargo, HandlingEvent, Location, and Voyage.

Cargo

The cargo package has the Cargo entity as the aggregate root. It is made up of Itinerary-Leg, Delivery, and RouteSpecification.

Notice how the Cargo entity does not have getter/setter methods that you'd normally see in anemic domain entities. Instead, it has getters that are differently named (not following JavaBean naming conventions)...

public class Cargo implements Entity<Cargo> {

  private TrackingId trackingId;
  private Location origin;
  private RouteSpecification routeSpecification;
  private Itinerary itinerary;
  private Delivery delivery;

  public Cargo(
        final TrackingId trackingId,
        final RouteSpecification routeSpecification) {...}

  public TrackingId trackingId() { return trackingId; }

  public Location origin() { return origin; }

  public Delivery delivery() { return delivery; }

  public Itinerary itinerary() {
    return DomainObjectUtils.nullSafe(
        this.itinerary, Itinerary.EMPTY_ITINERARY);
  }

  public RouteSpecification routeSpecification() {
    return routeSpecification;
  }
  . . .
}

...and mutator methods that aren't named as setters.

public class Cargo implements Entity<Cargo> {

  . . .
  public void specifyNewRoute(
        final RouteSpecification routeSpecification) {...}

  public void assignToRoute(final Itinerary itinerary) {...}

  public void deriveDeliveryProgress(
        final HandlingHistory handlingHistory) {...}
  . . .
}

The life cycle of a cargo begins with the booking procedure, when the tracking id is assigned. It has an origin and a destination (via a RouteSpecification). As the cargo is being handled (transported to its destination), its delivery and transport status changes (from NOT_RECEIVED to CLAIMED).

Handling Event

Although not exactly an aggregate or entity, the HandlingEvent is a domain event used to register the handling event when, for instance, a cargo is unloaded from a carrier at some location at a given time. These events are sent from different applications some time after the event occurred and contain information about the cargo (referenced via tracking id), location, timestamp of the completion of the event, and if applicable, a voyage (referenced via voyage number).

Before I illustrate how a cargo's delivery status is updated through handling events, let me take a closer look at the code behind handling events.

public final class HandlingEvent implements DomainEvent<HandlingEvent> {

  private Type type;
  private Voyage voyage;
  private Location location;
  private Date completionTime;
  private Date registrationTime;
  private Cargo cargo;

  public enum Type implements ValueObject<Type> {
    LOAD(true),
    UNLOAD(true),
    RECEIVE(false),
    CLAIM(false),
    CUSTOMS(false); . . .
  }
  . . .
}

I find it interesting that even though HandlingEvent has a public constructor, it also has a factory — HandlingEventFactory. While the HandlingEvent constructor needs other entities like Cargo, Location, and Voyage, the factory accepts TrackingId, UnLocode, and VoyageNumber to create a new HandlingEvent. Notice how the factory uses the unique IDs of the domain entities to retrieve the entities and create the domain event object.

I also find it interesting that there's HandlingHistory to represent a list (or collection) of HandlingEvents. The list is unmodifiable (i.e. read-only). And has mostRecentlyCompletedEvent() and distinctEventsByCompletionTime() methods.

The HandlingEventRepository is also interesting, since it doesn't have all the CRUD operations. It only has a method to store a HandlingEvent, and to retrieve the handling history of a particular cargo via its unique TrackingId.

public interface HandlingEventRepository {
  void store(HandlingEvent event);
  HandlingHistory lookupHandlingHistoryOfCargo(
      TrackingId trackingId);
}

Updating a Cargo's Delivery Status

Now, let's see how a Cargo's delivery status is updated. Some might be tempted to provide a setStatus(...) method. But if we inspect the cargo shipping domain, the delivery status is based on handling events.

Cargo (aggregate root) entity has a Delivery value object that is the actual transportation of the cargo, as opposed to the customer requirement (RouteSpecification) and the plan (Itinerary). Delivery is updated via a list of handling events (HandlingHistory).

public class Cargo implements Entity<Cargo> {
  private TrackingId trackingId;
  private Location origin;
  private RouteSpecification routeSpecification;
  private Itinerary itinerary;
  private Delivery delivery;
  . . .
  public void deriveDeliveryProgress(
        final HandlingHistory handlingHistory) {
    . . .
  }
}

The delivery of a cargo is defined by its status at the last known location (e.g. on-board carrier in Hong Kong). It's also possible that the cargo is misdirected.

public class Delivery implements ValueObject<Delivery> {
  private TransportStatus transportStatus;
  private Location lastKnownLocation;
  private Voyage currentVoyage;
  private boolean misdirected;
  private Date eta;
  private HandlingActivity nextExpectedActivity;
  private boolean isUnloadedAtDestination;
  private RoutingStatus routingStatus;
  private Date calculatedAt;
  private HandlingEvent lastEvent;
  . . .
}

public enum TransportStatus implements ValueObject<TransportStatus> {
  NOT_RECEIVED, IN_PORT, ONBOARD_CARRIER, CLAIMED, UNKNOWN;
  . . .
}

I find that there's overlap between HandlingEvent, HandlingActivity, and TransportStatus. It could have been merged to simplify and clarify the model. If merged, I'd choose the name of HandlingActivity, since I find the word "event" to be a bit too technology-centric (probably influenced by "domain event"), and the word "activity" sounds more apt for the domain.

Why was HandlingEvent not part of the Cargo aggregate? Good question. At first, I was also thinking that it would be easier if the handling events were part of a cargo aggregate. According to the cargo shipping example:

The main reason for not making HandlingEvent part of the cargo aggregate is performance. HandlingEvents are received from external parties and systems, e.g. warehouse management systems, port handling systems, that call our HandlingReportService webservice implementation. The number of events can be very high and it is important that our webservice can dispatch the remote calls quickly. To be able to support this use case we need to handle the remote webservice calls asynchronously, i.e. we do not want to load the big cargo structure synchronously for each received HandlingEvent. Since all relationships in an aggregate must be handled synchronously we put the HandlingEvent in an aggregate of its own and we are able processes the events quickly and at the same time eliminate dead-locking situations in the system.

Anti-corruption Layer

The cargo shipping example may not have shown context mapping. But it did show one strategic DDD pattern — Anti-corruption Layer. Although not much explanation was put into the code, here's what I can gather from it.

A HandlingReportService interface was defined and exposed as a SOAP-based web service.

public interface HandlingReportService {
  public void submitReport(HandlingReport arg0) throws...;
}

It accepts a HandlingReport. Note that this is not the HandlingEvent domain event. HandlingReport is eventually translated to a HandlingEvent. This effectively shields the cargo shipping context's domain (layer) from being corrupted by all the external applications that would be posting handling events to the system.

<complexType name="handlingReport">
  <complexContent>
    <restriction base="anyType">
      <sequence>
        <element name="completionTime" type="xs:dateTime"/>
        <element name="trackingIds" type="xs:string" maxOccurs="unbounded"/>
        <element name="type" type="xs:string"/>
        <element name="unLocode" type="xs:string"/>
        <element name="voyageNumber" type="xs:string" minOccurs="0"/>
      </sequence>
    </restriction>
  </complexContent>
</complexType>

And since the web service needs to return as soon as possible (due to the huge number of events coming in), it does not write directly to the database. It was implemented to create a HandlingEventRegistrationAttempt that gets sent to a message queue. The message is then asynchronously consumed by HandlingEventRegistrationAttemptConsumer that delegates to HandlingEventService (an application service). HandlingEventServiceImpl (implementation) simply used the HandlingEventRepository to store the handling event. Again, notice how HandlingEventService is using unique IDs as input parameters (not domain entities).

public interface HandlingEventService {
  void registerHandlingEvent(
          Date completionTime,
          TrackingId trackingId,
          VoyageNumber voyageNumber,
          UnLocode unLocode,
          HandlingEvent.Type type)
      throws CannotCreateHandlingEventException;
}

How does cargo.deriveDeliveryProgress(HandlingHistory) get called? From HandlingEventService, the event is stored, and published by calling ApplicationEvents.cargoWasHandled(...). It, in turn, is implemented using a message queue (Yep, another message queue). The event (in the message queue) is consumed by CargoHandledConsumer which then calls CargoInspectionService. And it finally uses repositories to retrieve the cargo in question, and its related handling history.

1. HandlingEventService -> ApplicationEvents
2. ApplicationEvents -> CargoHandledConsumer (via JMS)
3. CargoHandledConsumer -> CargoInspectionService
4. CargoInspectionService -> cargo.deriveDeliveryProgress(...)

Kinda long-winded if you ask me. But looking at the implementation, I think I can understand why asynchronous events were preferred.

Ending Thoughts

Boy, was this a long post or what?!

While the cargo shipping example does leave a lot to be desired, it was still a good example. I learned a lot from it. I hope you do too. If you find other things worth noting in the cargo shipping example, please hit the comments and let me know.