tag:blogger.com,1999:blog-25266288848372136682024-03-18T01:08:02.112-07:00Adapting and LearningThoughts on software development and life in generalLorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.comBlogger60125tag:blogger.com,1999:blog-2526628884837213668.post-21623229980382490552021-06-07T19:30:00.036-07:002021-06-07T22:50:45.573-07:00Changing The Way We Use JPA<p>I've been updating some training materials recently, and thinking about better ways of teaching and talking about JPA. One of the things I've been thinking about is how we have typically used JPA, and how that should change given the pains I've experienced (and observed).</p>
<p>JPA is often seen as a set of annotations (or XML files) that provide O/R (object-relational) mapping information. And most developers think that the more mapping annotations they know and use, the more benefits they get. But the past few years of wrestling with small to medium monoliths/systems (with about 200 tables/entities) have taught me something else.</p>
<p><strong>TL;DR</strong></p>
<ol>
<li>Reference entities by ID (only map entity relationships <em>within</em> an aggregate)</li>
<li>Don't let JPA steal your Identity (avoid <code>@GeneratedValue</code> when you can)</li>
<li>Use ad-hoc joins to join unrelated entities</li>
</ol>
<h3>Reference Entities by Identifier</h3>
<p><em>Only map entity relationships <strong>within</strong> an aggregate.</em></p>
<p>Tutorials (and training) would typically go about teaching and covering all possible relationship mappings. After basic mappings, many would start from simple uni-directional <code>@ManyToOne</code> mapping. Then proceed to bi-directional <code>@OneToMany</code> and <code>@ManyToOne</code>. Unfortunately, most often than not, they fail to explicitly point out that it is perfectly fine to <em>not</em> map the relationship. So, beginners would often complete the training thinking that it would be a mistake to <em>not</em> map a related entity. They mistakenly think that a foreign key field <em>must</em> be mapped as a related entity.</p>
<p>In fact, it is <em>not</em> an error, if you change the <code>@ManyToOne</code> mapping below…</p>
<pre class="brush: java; highlight: [4]">
@Entity
public class SomeEntity {
// ...
@ManyToOne private Country country;
// ...
}
@Entity
public class Country {
@Id private String id; // e.g. US, JP, CN, CA, GB, PH
// ...
}
</pre>
<p>…into a basic field that contains the primary key value of the <em>related</em> entity.</p>
<pre class="brush: java; highlight: [4]">
@Entity
public class SomeEntity {
// ...
@Column private String countryId;
// ...
}
@Entity
public class Country {
@Id private String id; // e.g. US, JP, CN, CA, GB, PH
// ...
}
</pre>
<p><em>Why is this a problem?</em></p>
<p>Mapping all entity relationships increases the chances of unwanted traversals that usually lead to unnecessary memory consumption. This also leads to an unwanted cascade of <code>EntityManager</code> operations.</p>
<p>This may not be much if you're dealing with just a handful of entities/tables. But it becomes a maintenance nightmare when working with dozens (if not hundreds) of entities.</p>
<p><em>When do you map a related entity?</em></p>
<p>Map related entities only when they are <em>within</em> an <a href="https://martinfowler.com/bliki/DDD_Aggregate.html">Aggregate</a> (in DDD).</p>
<blockquote>
<p>Aggregate is a pattern in Domain-Driven Design. A DDD aggregate is a cluster of domain objects that can be treated as a single unit. An example may be an order and its line-items, these will be separate objects, but it's useful to treat the order (together with its line items) as a single aggregate.</p>
<footer>
<cite><a href="https://martinfowler.com/bliki/DDD_Aggregate.html">https://martinfowler.com/bliki/DDD_Aggregate.html</a></cite>
</footer>
</blockquote>
<pre class="brush: java; highlight: [4, 11]">
@Entity
public class Order {
// ...
@OneToMany(mappedBy = "order", ...) private List<OrderItem> items;
// ...
}
@Entity
public class OrderItem {
// ...
@ManyToOne(optional = false) private Order order;
// ...
}
</pre>
<p>More modern approaches to aggregate design (see Vaughn Vernon's <cite>Implementing Domain-Driven Design</cite>) advocate a cleaner separation between aggregates. It is a good practice to refer to an aggregate root by storing its ID (unique identifier), not a full reference.</p>
<p>If we expand the simple order example above, the line-item (<code>OrderItem</code> class) should not have a <code>@ManyToOne</code> mapping to the product (since it is another aggregate in this example). Instead, it should just have the ID of the product.</p>
<pre class="brush: java; highlight: [12, 13]">
@Entity
public class Order {
// ...
@OneToMany(mappedBy = "order", ...) private List<OrderItem> items;
// ...
}
@Entity
public class OrderItem {
// ...
@ManyToOne(optional = false) private Order order;
// @ManyToOne private Product product; // <-- Avoid this!
@Column private ... productId;
// ...
}
</pre>
<p>But… what if the <code>Product</code> (aggregate root entity) has its <code>@Id</code> field mapped as <code>@GeneratedValue</code>? Are we forced to persist/flush first and then use the generated ID value?</p>
<p>And, what about joins? Can we still join those entities in JPA?</p>
<h3>Don't Let JPA Steal Your <code>Id</code>entity</h3>
<p>Using <code>@GeneratedValue</code> may initially make the mapping simple and easy to use. But when you start referencing other entities by ID (and not by mapping a relationship), it becomes a challenge.</p>
<p>If the <code>Product</code> (aggregate root entity) has its <code>@Id</code> field mapped as <code>@GeneratedValue</code>, then calling <code>getId()</code> may return <code>null</code>. When it returns <code>null</code>, the line-item (<code>OrderItem</code> class) will not be able to reference it!</p>
<p>In an environment where all entities always have a non-<code>null</code> <code>Id</code> field, referencing any entity by ID becomes easier. Furthermore, having non-<code>null</code> <code>Id</code> fields all the time, makes <code>equals(Object)</code> and <code>hashCode()</code> easier to implement.</p>
<p>And because all <code>Id</code> fields become explicitly initialized, all (aggregate root) entities have a <code>public</code> constructor that accepts the <code>Id</code> field value. And, as I've <a href="/2016/01/jpa-pitfalls-mistakes.html">posted long ago</a>, a <code>protected</code> no-args constructor can be added to keep JPA happy.</p>
<pre class="brush: java; highlight: [5]">
@Entity
public class Order {
@Id private Long id;
// ...
public Order(Long id) {
// ...
this.id = id;
}
public Long getId() { return id; }
// ...
protected Order() { /* as required by ORM/JPA */ }
}
</pre>
<aside><p>While writing this post, I found an article by James Brundege (posted back in 2006), <cite><a href="https://web.archive.org/web/20171211235806/http://www.onjava.com/pub/a/onjava/2006/09/13/dont-let-hibernate-steal-your-identity.html">Don't Let Hibernate Steal Your Identity</a></cite> (thanks to Wayback Machine), where he says, <q>Don't let Hibernate manage your <code style="color: inherit">id</code>s</q>. I wish I heeded his advice earlier.</p></aside>
<p>But beware! When using Spring Data JPA to <code>save()</code> an entity that does not use <code>@GeneratedValue</code> on its <code>@Id</code> field, an unnecessary SQL <code>SELECT</code> is issued before the expected <code>INSERT</code>. This is due to <code>SimpleJpaRepository</code>'s <code>save()</code> method (shown below). It relies on the presence of the <code>@Id</code> field (non-<code>null</code> value) to determine whether to call <code>persist(Object)</code> or <code>merge(Object)</code>.</p>
<pre class="brush: java">
public class SimpleJpaRepository // ...
@Override
public <S extends T> save(S entity) {
// ...
if (entityInformation.isNew(entity)) {
em.persist(entity);
return entity;
} else {
return em.merge(entity);
}
}
}
</pre>
<p>The astute reader will notice that, if the <code>@Id</code> field is never <code>null</code>, the <code>save()</code> method will always call <code>merge()</code>. This causes the unnecessary SQL <code>SELECT</code> (before the expected <code>INSERT</code>).</p>
<p>Fortunately, the work-around is simple — implement <code>Persistable<ID></code>.</p>
<pre class="brush: java">
@MappedSuperclass
public abstract class BaseEntity<ID> implements Persistable<ID> {
@Transient
private boolean persisted = false;
@Override
public boolean isNew() {
return !persisted;
}
@PostPersist
@PostLoad
protected void setPersisted() {
this.persisted = true;
}
}
</pre>
<p>The above also implies that <em>all</em> updates to entities must be done by loading the existing entity into the persistence context first, and applying changes to the <em>managed</em> entity.</p>
<h3>Use Ad-hoc Joins to Join Unrelated Entities</h3>
<p>And, what about joins? Now that we reference other entities by ID, how can we join unrelated entities in JPA?</p>
<p>In JPA version 2.2, unrelated entities <em>cannot</em> be joined. However, I cannot confirm if this has become a standard in version 3.0, where all <code>javax.persistence</code> references were renamed to <code>jakarta.persistence</code>.</p>
<p>Given the <code>OrderItem</code> entity, the absence of the <code>@ManyToOne</code> mapping causes it to fail to be joined with the <code>Product</code> entity.
<pre class="brush: java; highlight: [10]">
@Entity
public class Order {
// ...
}
@Entity
public class OrderItem {
// ...
@ManyToOne(optional = false) private Order order;
@Column private ... productId;
// ...
}
</pre>
<p>Thankfully 😊, <a href="http://in.relation.to/2016/02/10/hibernate-orm-510-final-release/">Hibernate 5.1.0+</a> (released back in 2016) and <a href="https://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Querying/JPQL#ON">EclipseLink 2.4.0+</a> (released back in 2012) have been supporting joins of unrelated entities. These joins are also referred to as <em>ad-hoc joins</em>.</p>
<pre class="">
SELECT o
FROM Order o
JOIN o.items oi
JOIN Product p ON (p.id = oi.productId) -- supported in Hibernate and EclipseLink
</pre>
<p>Also, this has been raised as an API issue (<a href="https://github.com/eclipse-ee4j/jpa-api/issues/128">Support JOIN/ON for two root entities</a>). I really hope that it will become a standard soon.</p>
<h3>In Closing</h3>
<p>What do you think about the above changes? Are you already using similar approaches? Do you use native SQL to explicitly retrieve a generated value (e.g. sequence object) to create an entity with a non-<code>null</code> <code>Id</code> field? Do you use entity-specific ID types to differentiate ID values? Let me know in the comments below.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-37761542874622737472020-09-17T20:00:00.002-07:002020-09-17T20:00:04.065-07:00Reduce Repetitive Code in Spring MVC Controllers<p>After spending some time doing sustained engineering (a.k.a. maintaining legacy code), I ventured to reduce repetitive code in Spring MVC <code>@Controller</code>s. I started with an abstract base controller class. But I soon found out that it was a dead-end because <code>@RequestMapping</code> is <em>not</em> inherited from (or combined with) parent classes and/or interfaces (see <a href="https://github.com/spring-projects/spring-framework/issues/14722">Spring MVC <code>@RequestMapping</code> Inheritance</a>).</p>
<p>With some free time to further think about this, I took different approach.</p>
<pre class="brush: java">
@Controller
public class RepositoryCrudController {
private final Repositories repositories;
private final RepositoryInvokerFactory repositoryInvokerFactory;
// ...
@RequestMapping("/{repository}")
String index(@PathVariable("repository") String repositoryKey,
Pageable pageable, Model model) {
// ... (only if repository has findAll(Pageable) method)
return repositoryKey + "/index";
}
@GetMapping("/{repository}/{id}")
String show(@PathVariable("repository") String repositoryKey,
@PathVariable("id") Object id, Model model) {
// ... (only if repository has findById method)
return repositoryKey + "/show";
}
@GetMapping(path = "/{repository}", param = "create")
String create(...) {
// ... (only if repository has save method)
return repositoryKey + "/create";
}
@PostMapping("/{repository}")
String save(...) {
// ... (only if repository has save method)
return "redirect:/" + repositoryKey + "/{id}";
}
@GetMapping(path = "/{repository}/{id}", param = "edit")
// ... edit (only if repository has findById and save methods)
@PutMapping("/{repository}/{id}")
// ... update (only if repository has save method)
// @DeleteMapping("/{repository}/{id}")
// ... delete (only if repository has deleteById method)
}
</pre>
<p>This approach is largely inspired by <code>RepositoryEntityController</code> of Spring Data REST.</p>
<p>Instead of defining an abstract base controller class, I created a concrete controller class with the <em>default</em> (or repetitive) behavior. The <em>default</em> behavior relies on invoking methods on Spring Data repositories.</p>
<p>For custom controllers, instead of creating subclasses (of the abstract base class), the controller can simply define handler methods that behave differently. For example:</p>
<pre class="brush: java">
@Entity
public class Article {...}
// Spring Data JPA
public interface ArticleRepository extends CrudRepository<Article, ...> {...}
@Controller
@RequestMapping("/articles")
public class ArticlesController {
// no need for index() handler method
// just set-up "articles/index" view
// no need for show() handler method
// just set-up "articles/show" view
@GetMapping(param = "create")
String create(Model model) {
// Do something that is _not_ *default* behavior
// e.g. provide options for dropdowns, or use form-backing object/JavaBean
// ...
return "articles/create";
}
// no need for save() handler method
// just set-up "articles/show" view
@GetMapping(path = "/{id}", param = "edit")
String edit(@PathVariable("id") ... id, Model model) {
// Do something that is _not_ *default* behavior
// e.g. provide options for dropdowns, or use form-backing object/JavaBean
// ...
return "articles/edit";
}
// no need for update() handler method
// just set-up "articles/show" view
}
</pre>
<p>The above will work because Spring MVC chooses the more specific mapping first. When a <code>GET /articles?create</code> request is received, the <code>ArticlesController</code> will be chosen to handle it (and not <code>RepositoryCrudController</code>). But if <code>ArticlesController</code> handler methods were not defined, the <code>GET /articles?create</code> request would have been handled by the <code>RepositoryCrudController</code>.</p>
<p>With this simple <em>fall-back</em> controller that has the <em>default</em> behavior, developers (team mates) can then focus on the domain, creating views, or creating controllers with custom behavior (e.g. Ajax, Server-generated JavaScript Responses).</p>
<p>That's all for now.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-36636084848207081672019-06-25T15:00:00.000-07:002019-07-02T01:54:42.892-07:00Spring with Rails' jQuery UJS<p>I’ve always wanted to try to see if I could use <a href="https://github.com/rails/jquery-ujs">Rails’ jQuery UJS</a> in a Spring Boot project. The UJS in jquery-ujs stands for <a href="http://en.wikipedia.org/wiki/Unobtrusive_JavaScript">unobtrusive JavaScript</a>. I really like how UJS wires event handlers to eligible DOM elements marked with HTML5 <code>data-*</code> attributes. I find myself wanting to see more of this approach being used in Spring Boot web apps. I wonder why there’s very little mentioned on the web about this. Or, may be I’ve been looking at the wrong places.</p>
<p>Anyway, here are some things jQuery UJS can do, and the related source code is on <a href="https://github.com/lorenzodee/spring-ujs-sjr">GitHub</a> (albeit using a different example).</p>
<h3>Non-GET Links (e.g. DELETE)</h3>
<p>When I need to render a link that deletes an item, I would use a <code><button></code> wrapped in a <code><form></code> with a hidden <code>_method</code> field with a value of <code>delete</code>. The <code><form></code> is not visible to the user. But the visible button is used to submit the <code><form></code>. Some CSS is used to make the button look like a link.</p>
<pre class="brush:html">
<form action="/articles/42" method="post">
<input type="hidden" name="_method" value="delete" />
<button class="btn btn-link">Delete</button>
</form>
</pre>
<p>Thanks to Spring’s <code>HiddenHttpMethodFilter</code> (also automatically configured in Spring Boot), when this <code><form></code> is submitted, it will be received as a request with a method of <code>DELETE</code>. It maps to <code>@DeleteMapping(path="/articles/{id}")</code> in the related <code>@Controller</code>.</p>
<p>While the above works, there is an easier way with jQuery UJS. All that is needed to render a link to delete an item is this:</p>
<pre class="brush:html">
<a href="/articles/42" data-method="delete">Delete</a>
</pre>
<p>jQuery UJS will enhance links that have <code>data-method</code> attribute. When the above example link is clicked, the JavaScript will create a <code><form></code>. The action attribute of this <code><form></code> is set to the value of link’s <code>href</code>. The method is set to <code>post</code>. A hidden <code>_method</code> field is added to the <code><form></code> and set to the value of the link’s <code>data-method</code>. Finally, the <code><form></code> is submitted (and the link is not followed).</p>
<h3>Confirmation Dialogs</h3>
<p>Most often than not, when it comes to deleting anything, it would be good to confirm with the user. This could be implemented as a simple dialog via <code>window.confirm()</code>. If we build from the previous example, the <code><form></code> would look like this:</p>
<pre class="brush:html">
<form action="/articles/42" method="post"
onsubmit="return confirm('Are you sure?')">
<input type="hidden" name="_method" value="delete" />
<button>Delete</button>
</form>
</pre>
<p>While the above works, jQuery UJS shows us a better way. All that is needed to confirm before deleting is this:</p>
<pre class="brush:html">
<a href="/articles/42?delete" data-method="delete"
data-confirm="Are you sure?">Delete</a>
</pre>
<p>jQuery UJS will enhance links (and <code><form></code>s too) that have <code>data-confirm</code> attribute. When the above example link is clicked, the JavaScript will call <code>confirm()</code> to show a dialog containing the text that is the value of the attribute. If the user chooses to cancel, the creation/submission of the <code><form></code> (due to <code>data-method</code>) does not take place.</p>
<h3>Ajax Links</h3>
<p>After deleting an item, the page usually reloads to show that the deleted element has indeed been removed.</p>
<p>Let's say the items are displayed in a table. Each row has a unique <code>id</code>.</p>
<pre class="brush:html">
<table>
<tr id="article:18">
<!-- data cells for /articles/18 -->
<td><a href="/articles/18?delete"
data-method="delete" data-confirm="Are you sure?">Delete</a></td>
</tr>
<tr id="article:42">
<!-- data cells for /articles/42 -->
<td><a href="/articles/42?delete"
data-method="delete" data-confirm="Are you sure?">Delete</a></td>
</tr>
<!-- other rows -->
</table>
</pre>
<p>Assuming that we can simply remove the HTML element that represented the deleted item, then we can probably send the <code>DELETE</code> request asynchronously and get a response that would remove the related HTML element. jQuery UJS makes this as easy as adding <code>data-remote="true"</code> and some minor changes to the server-side controller.</p>
<p>For the HTML, all we need is <code>data-remote="true"</code>.</p>
<pre class="brush:html">
<a href="/articles/42?delete" data-remote="true"
data-method="delete"
data-confirm="Are you sure?">Delete</a>
</pre>
<p>When the link is clicked, jQuery UJS will again send the <code>DELETE</code> request. But this time, it will be sent asynchronously using Ajax. Doing so enables the browser not to reload the entire page. And depending on the server’s response, can update just a portion of the page. Thus, providing a slightly better user experience.</p>
<p>For the server-side controller, we need to send a different response when the request is expecting <code>text/javascript</code>. We add a handler method that will respond with <code>text/javascript</code> by using the <code>produces</code> element of <code>@RequestMapping</code>. The response will remove the related HTML element(s).</p>
<pre class="brush:java">
// inside a @Controller
@DeleteMapping(path="/articles/{id}")
String delete(... id) {
// ... delete article with given identifier
return "redirect:/articles";
}
@DeleteMapping(path="/articles/{id}",
produces="text/javascript")
String delete(... id) {
// ... delete article with given identifier
return "articles/delete";
}
</pre>
<p>The view is a JSP that contains <code>text/javascript</code>. This will be executed by jQuery UJS.</p>
<pre class="brush:html">
<%-- articles/delete.js.jsp --%>
<%@ page contentType="text/javascript" %>
$('#article:${id}').remove();
</pre>
<h3>Partials and Server-generated JavaScript Responses</h3>
<p>Now what happens if we wanted to have an edit link to get some HTML content and show it up in a modal (without a page refresh)?</p>
<p>Here's what we can do. We send a <code>GET</code> request asynchronously. The response is expected to contain JavaScript that would append the HTML in targeted places in the document, and then trigger the modal to appear.</p>
<pre class="brush:html">
<a href="/articles/42?edit" data-remote="true">Edit</a>
</pre>
<p>When the response is expected to be <code>text/javascript</code>, <code>articles/edit.js.jsp</code> is rendered. Otherwise, the usual <code>articles/edit.jsp</code> is rendered.</p>
<pre class="brush:java">
// inside a @Controller
@GetMapping(path="/articles/{id}", params={"edit"})
String edit(... id, ...) {
// ...
return "articles/edit";
}
@GetMapping(path="/articles/{id}", params={"edit"},
produces="text/javascript")
String editViaAjax(... id, ...) {
// ...
return "articles/edit";
}
</pre>
<p>The <code>edit.jsp</code> renders the <code><form></code> (a partial, not a complete HTML document) which has been refactored to its own JSP to avoid repetition.</p>
<pre class="brush:html">
<%-- articles/edit.jsp --%>
<!-- -->
<jsp:include page="_form.jsp" />
<!-- -->
</pre>
<p>The <code>edit.js.jsp</code> renders the same <code><form></code> (a partial, not a complete HTML document) as a string in JS. Then includes it in the modal. It was tricky to render <code>_form.jsp</code> as a string. I had to use <code><c:import></code>.</p>
<pre class="brush:html">
<%-- articles/edit.js.jsp --%>
<%@ page contentType="text/javascript" %>
<c:import var="html" url="…_form.jsp" />
<!-- escape double quotes and remove new lines -->
(function() {
const $modal = $('#...'); // ID of modal element
$modal.find('.modal-body').html('${html}');
if (!$modal.is(':visible')) {
$modal.modal('show');
}
})()
</pre>
<p>For this to work, another <code>InternalResourceViewResolver</code> (IRVR) bean with <code>text/javascript</code> as the <code>contentType</code> is configured. This bean uses the same prefix and a slightly different <span style="white-space: nowrap">suffix: <code>.js.jsp</code>.</span> That way, when the request is expecting <code>text/javascript</code>, the <abbr title="content negotiating view resolver">CNVR</abbr> will favor using the IRVR bean with <code>text/javascript</code> and it ends up rendering <code>articles/edit.js.jsp</code>.</p>
<h3>Ajax Forms</h3>
<p>The <code>data-remote="true"</code> attribute can also be applied to <code><form></code>s. With it, jQuery UJS will handle the form submission as an Ajax request. And when the form is being submitted, the buttons can be disabled by adding <code>data-disabled-with</code>. For example,</p>
<pre class="brush:html">
<form ...>
<!-- ... -->
<button data-disable-with="Saving...">Save</button>
</form ...>
</pre>
<p>When the above form is submitted, jQuery UJS will disable the button and change its content to "Saving...".</p>
<p><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj31mejWDGMdynlJ0Zp_8vaqznoZISiTo40HJKABLYPdNGYw1qY9gHZ6QWobnruWZub_qdLnlgUQHb_Lnh3KtttrZYwv7-WFw3dhtzGzcf6AJaakE8NPjao0HIiGQcJO0NgtcXwr6SRvR8/s240/test.gif" /></p>
<h3>Closing Thoughts</h3>
<p>I’ve barely touched the surface of <a href="https://github.com/rails/jquery-ujs">Rails’ jQuery UJS</a>. There is so much more that it can offer. I look forward to using it (and similar techniques) in my web apps.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-70122898787403038492018-11-12T18:17:00.000-08:002018-11-12T18:17:24.420-08:00Revisions and Immutability<p>Here's a brief post. I'm not sure how to start it. It's one of those "why didn't I think of that" moments while reviewing some existing code. Due to NDAs, I cannot share the actual code. It has something to do with handling revisions. The closest thing I can relate to is how WordPress (WP) handles blog posts and revisions.</p>
<p>In WP, the <a href="https://developer.wordpress.org/reference/functions/wp_insert_post/"><code>wp_insert_post</code></a> function inserts or updates a post. It checks the ID field to determine if it will carry out an <code>INSERT</code> or an <code>UPDATE</code>. If the post is being updated, it checks if changes were made. If so, <a href="https://developer.wordpress.org/reference/functions/wp_save_post_revision/">a revision is saved</a>. A limit for the number of revisions to keep can be set. If so, the oldest ones are deleted.</p>
<p>This sounds like something that can be modeled as a rich domain entity. Here's a first try.</p>
<pre class="brush:java">
@Entity
... class Post {
@Id @GeneratedValue ... id;
... name;
... title;
... content;
... excerpt;
... status; // e.g. 'draft', 'publish', 'inherit'
... type; // e.g. 'post', 'revision'
@OneToMany @JoinColumn(name="parent_post_id") ... List<Post> revisions;
...
// setters and getters
}
</pre>
<pre class="brush:java">
Post post = new Post();
post.setTitle("Lorem Ipsum");
post.setContent("...");
// save post
...
post = // retrieve existing post for updates
post.setContent("..."); // how can we ensure that revision is created?
</pre>
<p>In the first try, the setter methods pose a challenge to ensuring that a revision is created when the post is updated. Let's give it another try. Here's our second try.</p>
<pre class="brush:java; highlight: [7, 19]">
// Immutable class
@Embeddable
... class PostData {
... title;
... content;
... excerpt;
// getters only
... getTitle() { return title; }
... getContent() { return content; }
... getExcerpt() { return excerpt; }
// equals() method to compare with another post data
// to see if there are changes
}
@Entity
... class Post {
@Id @GeneratedValue ... id;
... name; // for a revision, will contain parent ID and revision #
@Embedded ... PostData postData; // read-only
... status; // e.g. 'draft', 'published', 'inherit'
... type; // e.g. 'post', 'revision'
@OneToMany @JoinColumn(name="parent_post_id") ... List<Post> revisions;
...
... getTitle() { return this.postData.getTitle(); }
... getContent() { return this.postData.getContent(); }
... getExcerpt() { return this.postData.getExcerpt(); }
... getName() { return name; }
}
</pre>
<p>This is when I got my "why didn't I think of that" moment!</p>
<p>Note how we encapsulated the post data into its own type — <code>PostData</code>. It is immutable. This makes it possible to ensure that a revision is created when the post is updated.</p>
<pre class="brush:java">
PostData postData = new PostData("Lorem Ipsum", "...", "...");
Post post = new Post(postData);
// save post
...
post = // retrieve existing post for updates
// post.setContent("..."); // not possible
post.updateData(new PostData("...", "...", "...")); // ensure that revision is created
</pre>
<p>And here's how we create revisions.</p>
<pre class="brush:java">
@Entity
... class Post {
...
@Embedded ... PostData postData; // read-only
...
@OneToMany @JoinColumn(name="parent_post_id") ... List<Post> revisions;
...
public Post(PostData postData) {
this(postData, null);
}
/* package private */ Post(PostData postData, Post parent) {
if (postData == null) {
throw new IllegalArgumentException(...);
}
this.postData = postData;
if (parent == null) {
this.type = "post";
this.status = "draft";
this.name = null;
this.revisions = new ArrayList<>();
} else {
this.type = "revision";
this.status = "inherit";
this.name = "" + parent.getId() + "-revision" + (parent.getRevisionsCount() + 1);
this.revisions = null;
}
...
}
...
... void updateData(PostData newPostData) {
if (this.postData.equals(newPostData)) {
// no changes, no revisions added
return;
}
...
// creates a revision
PostData beforePostData = this.postData;
this.revisions.add(0, new Post(beforePostData, this));
// store latest changes
this.postData = newPostData;
// limit to number of revisions to keep
if (this.revisions.size() > ...) {
// delete the excess ones
for (...) {
this.revisions.remove(this.revisions.size() - 1);
}
}
...
}
...
}
</pre>
<p>Like I said, this one is a brief post. Let me know in the <a href="#comments">comments</a> below if it's something you've seen before, or, just like me, it gave you a "why didn't I think of that" moment.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com3tag:blogger.com,1999:blog-2526628884837213668.post-6492251875073100152018-07-18T16:30:00.000-07:002018-07-18T20:58:56.485-07:00Caching in Spring Boot with Spring Security<p>In this post, I’d like to share a lesson learned by one of the teams at O&B. They were using Spring Boot with Spring Security.</p>
<p>By default, anything that is protected by Spring Security is sent to the browser with the following HTTP header:</p>
<pre>
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
</pre>
<p>Essentially, the response will never be cached by the browser. While this may seem inefficient, there is actually a good reason for this default behavior. When one user logs out, we don’t want the next logged in user to be able to see the previous user’s resources (and this is possible if they’re cached).</p>
<p>It makes sense to not cache anything by default, and leave caching to be explicitly enabled. <em>But it’s not good if nothing is cached, as it will lead to high bandwidth usage and slow page loads.</em></p>
<p style="margin-bottom: 0">Good thing it is very easy to enable caching of static content in Spring Boot (even with Spring Security). Simply configure a cache period. And that’s it!</p>
<pre class="" style="line-height: 1">
# Boot 2.x
spring.resources.cache.cachecontrol.max-age=14400
# Boot 1.x
spring.resources.cache-period=14400
</pre>
<p><strong>But there are some gotchas!</strong> With some versions, it ain’t that simple! Let me explain further.</p>
<p style="margin-bottom: 0">There are several ways content can be returned:</p>
<ol style="margin-top: 0">
<li>Static content through Spring Boot auto-configured static resource request handler</li>
<li>Controller method returning view name (e.g. resolves to a JSP)</li>
<li>Controller method returning <code>HttpEntity</code> (or <code>ResponseEntity</code>)</li>
</ol>
<h3>Enable Caching of Static Content</h3>
<p>The first (serving static content) is handled by configuring the said property (usually in <code>application.properties</code> as shown above).</p>
<h3>Set via HttpServletResponse</h3>
<p>In the second case, the controller handler method may choose to set "Cache-Control" headers through a <code>HttpServletResponse</code> method parameter.</p>
<pre class="brush:java">
@Controller
... class ... {
@RequestMapping(...)
public String ...(..., HttpServletResponse response) {
response.setHeader("Cache-Control", "max-age=14400");
return ...; // view name
}
}
</pre>
<p>This works, as long as Spring Security does not overwrite it.</p>
<h3>Set via HttpEntity/ResponseEntity</h3>
<p>In the third case, the controller handler method may choose to set "Cache-Control" headers of the returned HTTP entity.</p>
<pre class="brush:java">
@Controller
... class ... {
@RequestMapping(...)
public ResponseEntity<...> ...(...) {
return ResponseEntity.ok().cacheControl(...).body(...);
}
}
</pre>
<p>This works, as long as Spring Security has not written its own "Cache-Control" headers yet.</p>
<h3>Under the Hood</h3>
<p>To understand when and why it works, here are the relevant sequences.</p>
<p style="margin-bottom: 0">With Spring Security Web 4.0.x, 4.2.0 up to 4.2.4 and above, the following sequence occurs:</p>
<ol style="margin-top: 0">
<li>The <code>HeaderWriterFilter</code> delegates to <a href="https://github.com/spring-projects/spring-security/blob/master/web/src/main/java/org/springframework/security/web/header/writers/CacheControlHeadersWriter.java"><code>CacheControlHeadersWriter</code></a> to write the "Cache-Control" headers (including "Pragma" and "Expires"), if no cache headers exist.</li>
<li>
<div>Controller handler method (if matched) is invoked. The method can:</div>
<ul>
<li>Explicitly set a header in <code>HttpServletResponse</code>.</li>
<li><div>Or, set a header in the returned <code>HttpEntity</code> or <code>ResponseEntity</code> (refer to the <code>handleReturnValue()</code> method of <a href="https://github.com/spring-projects/spring-framework/blob/master/spring-webmvc/src/main/java/org/springframework/web/servlet/mvc/method/annotation/HttpEntityMethodProcessor.java" target="_blank"><code>HttpEntityMethodProcessor</code></a>).</div>
<ul>
<li>Note that <a href="https://github.com/spring-projects/spring-framework/blob/5ea5f81c00d022ef0668aec07f5af4b399650b8e/spring-webmvc/src/main/java/org/springframework/web/servlet/mvc/method/annotation/HttpEntityMethodProcessor.java#L187" target="_blank"><code>HttpEntityMethodProcessor</code></a> only writes the headers (from <code>HttpEntity</code>) to the actual response <em>if they do not exist yet</em>. <span style="color: red">This becomes a problem, since back in #1, the headers have already been set.</span></li>
</ul>
</li>
</ul>
</li>
<li><div>If no controller handles the request, then the Spring Boot auto-configured static resource request handler gets its chance. It tries to serve static content, and if configured to cache, it overwrites the "Cache-Control" headers (and clears the values of "Pragma" and "Expires" headers, if any). The static resource handler is a <a href="https://github.com/spring-projects/spring-framework/blob/master/spring-webmvc/src/main/java/org/springframework/web/servlet/resource/ResourceHttpRequestHandler.java" target="_blank"><code>ResourceHttpRequestHandler</code></a> object (refer to the <a href="https://github.com/spring-projects/spring-framework/blob/master/spring-webmvc/src/main/java/org/springframework/web/servlet/support/WebContentGenerator.java#L415" target="_blank"><code>applyCacheControl()</code></a> method in its <a href="https://github.com/spring-projects/spring-framework/blob/master/spring-webmvc/src/main/java/org/springframework/web/servlet/support/WebContentGenerator.java" target="_blank"><code>WebContentGenerator</code></a> base class).</div>
<ul>
<li>However, in Spring Web MVC 4.2.5, the <a href="https://github.com/spring-projects/spring-framework/blob/v4.2.5.RELEASE/spring-webmvc/src/main/java/org/springframework/web/servlet/support/WebContentGenerator.java#L332" target="_blank"><code>WebContentGenerator</code></a> only writes the "Cache-Control" headers <em>only if it does not exist!</em>. <span style="color: red">This becomes a problem, since back in #1, the headers have already been set.</span></li>
<li>In Spring Web MVC 4.2.6 and above, it adds the "Cache-Control" headers even if it already exists. So, no problem even if the headers have been set in #1.</li>
</ul>
</li>
</ol>
<p style="margin-bottom: 0">With Spring Security Web 4.1.x, 4.2.5, and above (version 4.2.5 is used in Spring Boot 1.5.11), the sequence has changed. It goes something like this:</p>
<ol style="margin-top: 0">
<li>
<div>Controller handler method (if matched) is invoked. The method can:</div>
<ul>
<li>Explicitly set a header in <code>HttpServletResponse</code>.</li>
<li><div>Or, set a header in the returned <code>HttpEntity</code> or <code>ResponseEntity</code> (refer to the <code>handleReturnValue()</code> method of <code>HttpEntityMethodProcessor</code>).</div>
<ul>
<li>Note that <code>HttpEntityMethodProcessor</code> only writes the headers (from <code>HttpEntity</code>) to the actual response <em>if they do not exist yet</em>. <span style="color: #333; text-shadow: 0 0 16px #3f3">No problem, since no headers have been set yet.</span></li>
</ul>
</li>
</ul>
</li>
<li>If no controller handles the request, then the Spring Boot auto-configured static resource request handler gets its chance. It tries to serve static content, and if configured to cache, it overwrites the "Cache-Control" headers (and clears the values of "Pragma" and "Expires" headers, if any).</li>
<li><div>The <code>HeaderWriterFilter</code> delegates to <code>CacheControlHeadersWriter</code> to write the "Cache-Control" headers (including "Pragma" and "Expires"), if no cache headers exist.</div>
<ul>
<li><span style="color: #333; text-shadow: 0 0 16px #3f3">No problem, since it will not overwrite if cache headers have already been set.</span></li>
</ul>
</li>
</ol>
<h3>Working Versions</h3>
<p>The above three cases of controlling caching all work in Spring Boot 1.5.11 and Spring Boot 2.x. But in case upgrading to those versions is not possible, please see the following classes and check if it has your desired behavior (using the above sequences):</p>
<ul>
<li><code>HeaderWriterFilter</code> (see <code>doFilterInternal</code> method)</li>
<li><code>CacheControlHeadersWriter</code> (see <code>writeHeaders()</code> method)</li>
<li><code>WebContentGenerator</code> (see <code>applyCacheControl()</code> method)</li>
<li><code>HttpEntityMethodProcessor</code> (see <code>handleReturnValue()</code> method)</li>
</ul>
<p>Also, be aware that Spring Security Web 4.2.5 and above will write the following HTTP headers (overwrite it, even if they are already set, like in a controller for example):</p>
<ul>
<li><code>X-Content-Type-Options</code> via <code>XContentTypeOptionsHeaderWriter</code></li>
<li><code>Strict-Transport-Security</code> via <code>HstsHeaderWriter</code></li>
<li><code>X-Frame-Options</code> via <code>XFrameOptionsHeaderWriter</code></li>
<li><code>X-XSS-Protection</code> via <code>XXssProtectionHeaderWriter</code></li>
</ul>
<p>This is because, unlike <code>CacheControlHeadersWriter</code>, the header writers for the above do not check if the headers already exist. They simply set their respective HTTP headers. Please refer to their respective header writer classes and issue <a href="https://github.com/spring-projects/spring-security/issues/5193">#5193</a>.</p>
<p>Another option is to have Spring Security ignore static resource requests. That way, the configured cache period will not be overwritten.</p>
<pre class="brush:java">
@Configuration
@EnableWebSecurity
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {
@Override
public void configure(WebSecurity web) throws Exception {
web.ignoring().antMatchers("/css/**", "/js/**");
// If the above paths are served by the
// Spring Boot auto-configured
// static resource request handler,
// and a cache period is specified,
// then it will have a "Cache-Control"
// HTTP header in its response.
// And it would NOT get overwritten by Spring Security.
}
}
</pre>
<p>That's all for now. Hope this clears things up.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com6tag:blogger.com,1999:blog-2526628884837213668.post-70746508461246317022018-06-20T10:00:00.000-07:002018-06-21T18:32:53.308-07:00Dealing with Domain Objects in Spring MVC<p>I was recently surprised by how one code base had public default constructors (i.e. zero-arguments constructors) in <em>all</em> their domain entities, and had getters and setters for <em>all</em> the fields. As I dug deeper, I found out that the reason why the domain entites are the way they are is largely because the team thinks it was required by the web/MVC framework. And I thought it would be a good opportunity to clear up some misconceptions.</p>
<p>Specifically, we'll look at the following cases:</p>
<ol>
<li>No setter for generated ID field (i.e. the generated ID field has a getter but no setter)</li>
<li>No default constructor (e.g. no public zero-arguments constructor)</li>
<li>Domain entity with child entities (e.g. child entities are not exposed as a modifiable list)</li>
</ol>
<h2>Binding Web Request Parameters</h2>
<p>First, some specifics and some background. Let's base this on a specific web/MVC framework - Spring MVC. When using Spring MVC, its data binding binds request parameters by name. Let's use an example.</p>
<pre class="brush:java">
@Controller
@RequestMapping("/accounts")
... class ... {
...
@PostMapping
public ... save(@ModelAttribute Account account, ...) {...}
...
}
</pre>
<p>Given the above controller mapped to "/accounts", where can an <code>Account</code> instance come from?</p>
<p style="margin-bottom: 0.25em">Based on <a href="https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-modelattrib-method-args">documentation</a>, Spring MVC will get an instance using the following options:</p>
<ul style="margin-top: 0">
<li>From the model if already added via <a href="https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-modelattrib-methods"><code>Model</code></a> (like via <i><code>@ModelAttribute</code> method</i> in the same controller).</li>
<li>From the HTTP session via <a href="https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-sessionattributes"><code>@SessionAttributes</code></a>.</li>
<li>From a URI path variable passed through a <code>Converter</code>.</li>
<li>From the invocation of a default constructor.</li>
<li><em>(For Kotlin only)</em> From the invocation of a "primary constructor" with arguments matching to Servlet request parameters; argument names are determined via JavaBeans <code>@ConstructorProperties</code> or via runtime-retained parameter names in the bytecode.</li>
</ul>
<p>Assuming an <code>Account</code> object is <em>not</em> added in the session, and that there is <i>no <code>@ModelAttribute</code> method</i>, Spring MVC will end up instantiating one using its default constructor, and binding web request parameters <em>by name</em>. For example, the request contains "id" and "name" parameters. Spring MVC will try to bind them to "id" and "name" bean properties by invoking "setId" and "setName" methods, respectively. This follows JavaBean conventions.</p>
<h2>No Setter Method for Generated ID Field</h2>
<p>Let's start with something simple. Let's say that we have an <code>Account</code> domain entity. It has an ID field that is generated by the persistent store, and only provides a getter method (but no setter method).</p>
<pre class="brush:java">
@Entity
... class Account {
@Id @GeneratedValue(...) private Long id;
...
public Account() { ... }
public Long getId() { return id; }
// but no setId() method
}
</pre>
<p>So, how can we have Spring MVC bind request parameters to an <code>Account</code> domain entity? Are we forced to have a public setter method for a field that is generated and read-only?</p>
<p>In our HTML form, we will not place the "id" as a request parameter. We will place it as a path variable instead.</p>
<p>We use a <code>@ModelAttribute</code> method. It is called prior to the request handling method. And it supports pretty much the same parameters as a regular request handling method. In our case, we use it to retrieve an <code>Account</code> domain entity with the given unique identifier, and use it for further binding. Our controller would look something like this.</p>
<pre class="brush:java">
@Controller
@RequestMapping("/accounts")
... class ... {
...
@ModelAttribute
public Account populateModel(
HttpMethod httpMethod,
@PathVariable(required=false) Long id) {
if (id != null) {
return accountRepository.findById(id).orElseThrow(...);
}
if (httpMethod == HttpMethod.POST) {
return new Account();
}
return null;
}
@PutMapping("/{id}")
public ... update(...,
@ModelAttribute @Valid Account account, ...) {
...
accountRepository.save(account);
return ...;
}
@PostMapping
public ... save(@ModelAttribute @Valid Account account, ...) {
...
accountRepository.save(account);
return ...;
}
...
}
</pre>
<p>When updating an existing account, the request would be a <code>PUT</code> to "/accounts/{id}" URI. In this case, our controller needs to retrieve the domain entity with the given unique identifier, and provide the same domain object to Spring MVC for further binding, if any. The "id" field will not need a setter method.</p>
<p>When adding or saving a new account, the request would be a <code>POST</code> to "/accounts". In this case, our controller needs to create a <em>new</em> domain entity with some request parameters, and provide the same domain object to Spring MVC for further binding, if any. For new domain entities, the "id" field is left <code>null</code>. The underlying persistence infrastructure will generate a value upon storing. Still, the "id" field will not need a setter method.</p>
<p>In both cases, the <code>@ModelAttribute</code> method <code>populateModel</code> is called <em>prior</em> to the mapped request handling method. Because of this, we needed to use parameters in <code>populateModel</code> to determine which case it is being used in.</p>
<h2>No Default Constructor in Domain Object</h2>
<p>Let's say that our <code>Account</code> domain entity does not provide a default constructor (i.e. no zero-arguments constructor).</p>
<pre class="brush:java">
... class Account {
public Account(String name) {...}
...
// no public default constructor
// (i.e. no public zero-arguments constructor)
}
</pre>
<p>So, how can we have Spring MVC bind request parameters to an <code>Account</code> domain entity? It does not provide a default constructor.</p>
<p>We can use a <code>@ModelAttribute</code> method. In this case, we want to create an <code>Account</code> domain entity with request parameters, and use it for further binding. Our controller would look something like this.</p>
<pre class="brush:java">
@Controller
@RequestMapping("/accounts")
... class ... {
...
@ModelAttribute
public Account populateModel(
HttpMethod httpMethod,
@PathVariable(required=false) Long id,
@RequestParam(required=false) String name) {
if (id != null) {
return accountRepository.findById(id).orElseThrow(...);
}
if (httpMethod == HttpMethod.POST) {
return new Account(name);
}
return null;
}
@PutMapping("/{id}")
public ... update(...,
@ModelAttribute @Valid Account account, ...) {
...
accountRepository.save(account);
return ...;
}
@PostMapping
public ... save(@ModelAttribute @Valid Account account, ...) {
...
accountRepository.save(account);
return ...;
}
...
}
</pre>
<h2>Domain Entity with Child Entities</h2>
<p>Now, let's look at a domain entity that has child entities. Something like this.</p>
<pre class="brush:java">
... class Order {
private Map<..., OrderItem> items;
public Order() {...}
public void addItem(int quantity, ...) {...}
...
public Collection<CartItem> getItems() {
return Collections.unmodifiableCollection(items.values());
}
}
... class OrderItem {
private int quantity;
// no public default constructor
...
}
</pre>
<p>Note that the items in an order are not exposed as a modifiable list. Spring MVC supports indexed properties and binds them to an array, list, or other naturally ordered collection. But, in this case, the <code>getItems</code> method returns an unmodifiable collection. This means that an exception would be thrown when an object attempts to add/remove items to/from it. So, how can we have Spring MVC bind request parameters to an <code>Order</code> domain entity? Are we forced to expose the order items as a mutable list?</p>
<p>Not really. We must refrain from diluting the domain model with presentation-layer concerns (like Spring MVC). Instead, we make the presentation-layer a client of the domain model. To handle this case, we create another type that complies with Spring MVC, and keep our domain entities agnostic of the presentation layer.</p>
<pre class="brush:java">
... class OrderForm {
public static OrderForm fromDomainEntity(Order order) {...}
...
// public default constructor
// (i.e. public zero-arguments constructor)
private List<OrderFormItem> items;
public List<OrderFormItem> getItems() { return items; }
public void setItems(List<OrderFormItem> items) { this.items = items; }
public Order toDomainEntity() {...}
}
... class OrderFormItem {
...
private int quantity;
// public default constructor
// (i.e. public zero-arguments constructor)
// public getters and setters
}
</pre>
<p>Note that it is perfectly all right to create a presentation-layer type that knows about the domain entity. But it is not all right to make the domain entity aware of presentation-layer objects. More specifically, presentation-layer <code>OrderForm</code> knows about the <code>Order</code> domain entity. But <code>Order</code> does not know about presentation-layer <code>OrderForm</code>.</p>
<p>Here's how our controller will look like.</p>
<pre class="brush:java">
@Controller
@RequestMapping("/orders")
... class ... {
...
@ModelAttribute
public OrderForm populateModel(
HttpMethod httpMethod,
@PathVariable(required=false) Long id,
@RequestParam(required=false) String name) {
if (id != null) {
return OrderForm.fromDomainEntity(
orderRepository.findById(id).orElseThrow(...));
}
if (httpMethod == HttpMethod.POST) {
return new OrderForm(); // new Order()
}
return null;
}
@PutMapping("/{id}")
public ... update(...,
@ModelAttribute @Valid OrderForm orderForm, ...) {
...
orderRepository.save(orderForm.toDomainEntity());
return ...;
}
@PostMapping
public ... save(@ModelAttribute @Valid OrderForm orderForm, ...) {
...
orderRepository.save(orderForm.toDomainEntity());
return ...;
}
...
}
</pre>
<h2>Closing Thoughts</h2>
<p>As I've mentioned in previous posts, it is all right to have your domain objects look like a JavaBean with public default zero-arguments constructors, getters, and setters. But if the domain logic starts to get complicated, and requires that some domain objects lose its JavaBean-ness (e.g. no more public zero-arguments constructor, no more setters), do not worry. Define new JavaBean types to satisfy presentation-related concerns. Do not dilute the domain logic.</p>
<p>That's all for now. I hope this helps.</p>
<p>Thanks again to <a href="https://plus.google.com/108603150488137710491">Juno</a> for helping me out with the samples. The relevant pieces of code can be found on <a href="https://github.com/lorenzodee/domain-objects-in-mvc">GitHub</a>.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-81028587325922359202018-01-24T10:00:00.000-08:002018-01-24T10:00:15.296-08:00JasperReports: The Tricky Parts<p>If you have been programming in Java long enough, chances are you needed to generate reports for business users. In my case, I've seen several projects use <a href="https://community.jaspersoft.com/project/jasperreports-library">JasperReports® Library</a> to generate reports in PDF and other file formats. Recently, I've had the privilege of observing Mike and his team use the said reporting library and the challenges they faced.</p>
<h3>JasperReports in a Nutshell</h3>
<p>In a nutshell, generating reports using JasperReports (JR) involves three steps:</p>
<ol>
<li>Load compiled report (i.e. load a <code>JasperReport</code> object)</li>
<li>Run report by filling it with data (results to a <code>JasperPrint</code> object)</li>
<li>Export filled report to a file (e.g. use <code>JRPdfExporter</code> to export to PDF)</li>
</ol>
<p>In Java code, it looks something like this.</p>
<pre class="brush: java">
JasperReport compiledReport = JasperCompileManager.compileReport(
"sample.jrxml");
Map<String, Object> parameters = ...;
java.sql.Connection connection = dataSource.getConnection();
try {
JasperPrint filledReport = JasperFillManager.fillReport(
compiledReport, parameters, connection);
JasperExportManager.exportReportToPdf(
filledReport, "report.pdf");
} finally {
connection.close();
}
</pre>
<p>Thanks to the facade classes, this looks simple enough. <em>But looks can be deceiving!</em></p>
<p>Given the above code snippet (and the outlined three steps), which parts do you think takes the most amount of time and memory? (Sounds like an interview question).</p>
<p style="color: #ccc">If you answered (#2) filling with data, you're correct! If you answered #3, you're also correct, since #3 is proportional to #2.</p>
<p><abbr title="In my humble opinion">IMHO</abbr>, most online tutorials only show the easy parts. In the case of <abbr title="JasperReports">JR</abbr>, there seems to be a lack of discussion on the more difficult and tricky parts. Here, with Mike's team, we encountered two difficulties: out of memory errors, and long running reports. What made these difficulties particularly memorable was that they only showed up during production (not during development). I hope that by sharing them, they can be avoided in the future.</p>
<h3>Out of Memory Errors</h3>
<p>The first challenge was reports running out of memory. During development, the test data we use to run the report would be too small when compared to real operating data. So, <em>design for that</em>.</p>
<p>In our case, all reports were run with a <code>JRVirtualizer</code>. This way, it will flush to disk/file when the maximum number of pages/objects in memory has been reached.</p>
<p>During the process, we also learned that the virtualizer needs to be cleaned-up. Otherwise, there will be several temporary files lying around. And we can only clean-up these temporary files <em>after</em> the report has been exported to file.</p>
<pre class="brush: java">
Map<String, Object> parameters = ...;
JRVirtualizer virtualizer = new JRFileVirtualizer(100);
try {
parameters.put(JRParameter.REPORT_VIRTUALIZER, virtualizer);
...
... filledReport = JasperFillManager.fillReport(
compiledReport, parameters, ...);
// cannot cleanup virtualizer at this point
JasperExportManager.exportReportToPdf(filledReport, ...);
} finally {
virtualizer.cleanup();
}
</pre>
<p>For more information, please see <a href="http://jasperreports.sourceforge.net/sample.reference/virtualizer/index.html">Virtualizer Sample - JasperReports</a>.</p>
<p>Note that JR is <em>not</em> always the culprit when we encountered out-of-memory errors when running reports. Sometimes, we would encounter an out-of-memory error even before JR was used. We saw how JPA can be misused to load the entire dataset for the report (<code>Query.getResultList()</code> and <code>TypedQuery.getResultList()</code>). Again, the error does not show up during development since the dataset is still small. But when the dataset is too large to fit in memory, we get the out-of-memory errors. We opted to avoid using JPA for generating reports. I guess we'll just have to wait until JPA 2.2's <code>Query.getResultStream()</code> becomes available. I wish JPA's <code>Query.getResultList()</code> returned <code>Iterable</code> instead. That way, it is possible to have one entity is mapped at a time, and not the entire result set.</p>
<p>For now, avoid loading the entire dataset. Load one record at a time. In the process, we went back to good ol' JDBC. Good thing JR uses <code>ResultSet</code>s well.</p>
<h3>Long Running Reports</h3>
<div style="float: right; margin: 0 0 1em 1em"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioqvIETp1pEvFafajV0-P6YPTTU5g99gfmFeZOI3LsqLQU2nfDe8zL5_5MlG5nafGcuIiVEJ1U6u0vHMh3guxQ_SEXB69pjqM6hqNuJZJUwRW_JdOgbM21IdAxoYnlGycDOgcUl_fCVB4/s200/imageedit_2_2027264639.jpg" width="129" height="200" data-original-width="288" data-original-height="445" /></div>
<p>The second challenge was long running reports. Again, this probably doesn't happen during development. At best, a report that runs for about 10 seconds is considered long. But with real operating data, it can run for about 5-10 minutes. This is especially painful when the report is being generated upon an HTTP request. If the report can start to write to the response output stream within the timeout period (usually 60 seconds or up to 5 minutes), then it has a good chance of being received by the requesting user (usually via browser). But if it takes more than 5 minutes to fill the report and another 8 minutes to export to file, then the user will just see a timed-out HTTP request, and log it as a bug. Sound familiar?</p>
<p>Keep in mind that reports can run for a few minutes. So, <em>design for that</em>.</p>
<p>In our case, we launch reports on a separate thread. For reports that are triggered with an HTTP request, we respond with a page that contains a link to the generated report. This avoids the time-out problem. When the user clicks on this link and the report is <em>not yet</em> complete, s/he will see that the report is still being generated. But when the report <em>is</em> completed, s/he will be able to see the generated report file.</p>
<pre class="brush: java">
ExecutorService executorService = ...;
... = executorService.submit(() -> {
Map<String, Object> parameters = ...;
try {
...
... filledReport = JasperFillManager.fillReport(
compiledReport, parameters, ...);
JasperExportManager.exportReportToPdf(filledReport, ...);
} finally {
...
}
});
</pre>
<p>We also had to add the ability to stop/cancel a running report. Good thing JR has code that checks for <code>Thread.interrupted()</code>. So, simply interrupting the thread will make it stop. Of course, you'll need to write some tests to verify (expect <code>JRFillInterruptedException</code> and <code>ExportInterruptedException</code>).</p>
<p>And while we were at it, we rediscovered ways to add "listeners" to the report generation (e.g. <a href="http://jasperreports.sourceforge.net/api/net/sf/jasperreports/engine/fill/FillListener.html"><code>FillListener</code></a> and <a href="http://jasperreports.sourceforge.net/api/net/sf/jasperreports/engine/export/JRExportProgressMonitor.html"><code>JRExportProgressMonitor</code></a>) and provide the user some progress information.</p>
<p>We also created utility test classes to generate large amounts of data by repeating a given piece of data over and over. This is useful to help the rest of the team develop JR applications that are designed for handling long runs and out-of-memory errors.</p>
<h3>Further Design Considerations</h3>
<p>Another thing to consider is the opening and closing of the resource needed when filling the report. This could be a JDBC connection, a Hibernate session, a JPA <code>EntityManager</code>, or a file input stream (e.g. CSV, XML). Illustrated below is a rough sketch of my design considerations.</p>
<pre style="line-height: 1">
1. Compiling
- - - - - - - - - - - - - -\
- - - -\ \
2. Filling > open-close \
- - - -/ resource > swap to file
/
3. Exporting /
- - - - - - - - - - - - - -/
</pre>
<p>We want to isolate #2 and define decorators that would open the resource, fill the report, and close the opened resource in a <code>finally</code> block. The resource that is opened may depend on the <code><queryString></code> element (if present) inside the report. In some cases, where there is no <code><queryString></code> element, there is probably no need to open a resource.</p>
<pre class="brush:xml">
<queryString language="hql">
<![CDATA[ ... ]]>
</queryString>
...
<queryString language="csv">
<![CDATA[ ... ]]>
</queryString>
</pre>
<p>Furthermore, we also want to combine #2 and #3 as one abstraction. This single abstraction makes it easier to decorate with enhancements, like flushing the created page objects to files, and load them back during exporting. As mentioned, this is what the <code>JRVirtualizer</code> does. But we'd like a design where this is transparent to the object(s) using the combined-#2-and-#3 abstraction.</p>
<h3>Acknowledgements</h3>
<p>That's all for now. Again, thanks to Mike and his team for sharing their experiences. Yup, he's the same guy who <a href="https://www.entrepreneur.com.ph/startup-tips/this-pinoy-app-developer-donates-his-earnings-to-charity-a1148-20170616-lfrm">donates his app's earnings to charity</a>. Also, thanks to <a href="https://www.blogger.com/profile/09491302535338840746">Claire</a> for the ideas on testing by repeating a given data again and again. The relevant pieces of code can be found on <a href="https://github.com/orangeandbronze/jasperreports-utils">GitHub</a>.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-3132744512122562702018-01-02T10:00:00.000-08:002018-01-02T16:57:01.896-08:00DataSource Routing with Spring @Transactional<p>I was inspired by Carl Papa's use of aspects with the Spring Framework to determine the <code>DataSource</code> to use (either read-write or read-only). So, I'm writing this post.</p>
<div style="float: right; margin: 0 0 1em 1em"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8hNmi5LC84UKANviIEnL6WYiYALS8U2dpy19odAqSP7wDje1jd531BUzCrTFmoZx_e7KeVAwmefqCsWRSs65ej79ITBtTYyzPRxIcmyMWXbXvumJvtnNklqNmzeZmZAF2ANLuMxvOyYo/s1600/imageedit_1_6934082642.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8hNmi5LC84UKANviIEnL6WYiYALS8U2dpy19odAqSP7wDje1jd531BUzCrTFmoZx_e7KeVAwmefqCsWRSs65ej79ITBtTYyzPRxIcmyMWXbXvumJvtnNklqNmzeZmZAF2ANLuMxvOyYo/s1600/imageedit_1_6934082642.png" data-original-width="115" data-original-height="208" /></a></div>
<p>I must admit that I have long been familiar with Spring's <a href="https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/jdbc/datasource/lookup/AbstractRoutingDataSource.html"><code>AbstractRoutingDataSource</code></a>. But I did not have a good idea where it can be used. Thanks to Carl and team, and one of their projects. Now, I know a good use case.</p>
<h3>@Transactional</h3>
<p>With Spring, read-only transactions are typically marked with annotations.</p>
<pre class="brush: java; highlight: [2, 5]">
public class ... {
@Transactional(readOnly=true)
public void ...() {...}
@Transactional // read-write
public void ...() {...}
}
</pre>
<p>To take advantage of this, we use Spring's <a href="https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/transaction/support/TransactionSynchronizationManager.html"><code>TransactionSynchronizationManager</code></a> to determine if the current transaction is read-only or not.</p>
<h3>AbstractRoutingDataSource</h3>
<p>Here, we use Spring's <code>AbstractRoutingDataSource</code> to route to the read-only replica if the current transaction is read-only. Otherwise, it routes to the default which is the master.</p>
<pre class="brush: java; highlight: [4, 5]">
public class ... extends AbstractRoutingDataSource {
@Override
protected Object determineCurrentLookupKey() {
if (TransactionSynchronizationManager
.isCurrentTransactionReadOnly() ...) {
// return key to a replica
}
return null; // use default
}
...
}
</pre>
<p>Upon using the above approach, we found out that the <code>TransactionSynchronizationManager</code> is one step behind because Spring will have already called <code>DataSource.getConnection()</code> <em>before</em> a synchronization is established. Thus, a <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/jdbc/datasource/LazyConnectionDataSourceProxy.html"><code>LazyConnectionDataSourceProxy</code></a> needs to be configured as well.</p>
<p>As we were discussing this, we figured if there was another way to determine if the current transaction is read-only or not (without resorting to <code>LazyConnectionDataSourceProxy</code>). So, we came up with an experimental approach where an aspect captures the <code>TransactionDefinition</code> (from the <code>@Transactional</code> annotation, if any) as a thread-local variable, and an <code>AbstractRoutingDataSource</code> that routes based on the captured information.</p>
<p>The relevant source code can be found on <a href="https://github.com/orangeandbronze/spring-jdbc-routing-datasource">GitHub</a>. Thanks again, <a href="https://www.linkedin.com/in/carl-joseph-echague-papa-59677577/">Carl</a>! BTW, <a href="http://www.imdb.com/name/nm5377013/">Carl</a> is also an award-winning movie director. Wow, talent definitely knows no boundaries.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-30011077105870215392017-04-24T10:00:00.000-07:002017-04-24T10:00:10.402-07:00Apache Spark RDD and Java Streams<p>A few months ago, I was fortunate enough to participate in a few PoCs (proof-of-concepts) that used Apache Spark. There, I got the chance to use resilient distributed datasets (<abbr title="Resilient Distributed Dataset">RDD</abbr>s for short), transformations, and actions.</p>
<p>After a few days, I realized that while Apache Spark and the JDK are very different platforms, there are similarities between RDD transformations and actions, and stream intermediate and terminal operations. I think these similarities can help beginners (like me <i>*grin*</i>) get started with Apache Spark.</p>
<table style="border: 1px solid #ccc; border-collapse: collapse">
<tr><th style="border: 1px solid #ccc; padding: 0.25em 0.5em">Java Stream</th><th style="border: 1px solid #ccc; padding: 0.25em 0.5em">Apache Spark RDD</th></tr>
<tr><td style="border: 1px solid #ccc; padding: 0.25em 0.5em">Intermediate operation</td><td style="border: 1px solid #ccc; padding: 0.25em 0.5em">Transformation</td></tr>
<tr><td style="border: 1px solid #ccc; padding: 0.25em 0.5em">Terminal operation</td><td style="border: 1px solid #ccc; padding: 0.25em 0.5em">Action</td></tr>
</table>
<aside>Note that Apache Spark and the JDK are <em>very</em> different platforms. Apache Spark is an open-source cluster-computing framework that helps with big data processing and analysis. The JDK (Java Development Kit) includes tools for developing, debugging, and monitoring Java applications (not just data processing).</aside>
<h3>Java Streams</h3>
<p>Let's start with streams. Java 8 was released sometime in 2014. Arguably, the most significant feature it brought is the Streams API (or simply streams).</p>
<p style="margin-bottom: 0">Once a <code>Stream</code> is created, it provides many operations that can be grouped in two categories:</p>
<ul style="margin-top: 0">
<li>intermediate,</li>
<li>and terminal.</li>
</ul>
<p><i>Intermediate operations</i> return a stream from the previous one. These intermediate operations can be connected together to form a pipeline. <i>Terminal operations</i>, on the other hand, closes the stream pipeline, and returns a result.</p>
<p>Here's an example.</p>
<pre class="brush: java">
Stream.of(1, 2, 3)
.peek(n -> System.out.println("Peeked at: " + n))
.map(n -> n*n)
.forEach(System.out::println);
</pre>
<p>When the above example is run, it generates the following output:</p>
<pre style="background-color: black; color: #ccc; line-height: 1em; padding: 0.25em">
Peeked at: 1
1
Peeked at: 2
4
Peeked at: 3
9
</pre>
<p>Intermediate operations <em>are</em> lazy. The actual execution does not start until the terminal operation is encountered. The terminal operation in this case is <code>forEach()</code>. That's why, we <em>do not</em> see the following.</p>
<pre style="background-color: black; color: #ccc; line-height: 1em; padding: 0.25em">
Peeked at: 1
Peeked at: 2
Peeked at: 3
1
4
9
</pre>
<p>Instead, what we see is that the operations: <code>peek()</code>, <code>map()</code>, and <code>forEach()</code>, have been joined to form a pipeline. In each pass, the static <code>of()</code> operation returns one element from the specified values. Then the pipeline is invoked: <code>peek()</code> that prints the string "Peeked at: 1", followed by <code>map()</code>, and terminated by <code>forEach()</code> that prints the number "1". Then with another pass starting with <code>of()</code> that returns the next element from the specified values, followed by <code>peek()</code>, and <code>map()</code>, and so on.</p>
<p>Executing an intermediate operation such as <code>peek()</code> does not actually perform any peeking, but instead creates a new stream that, when traversed, contains the same elements of the initial stream, but additionally performing the provided action.</p>
<h3>Apache Spark RDD</h3>
<p>Now, let's turn to Spark's RDD (resilient distributed dataset). Spark's core abstraction for working with data is the resilient distributed dataset (RDD).</p>
<p>An RDD is simply a <em>distributed</em> collection of elements. In Spark all work is expressed as either creating new RDDs, or calling operations on RDDs to compute a result. Under the hood, Spark automatically distributes the data contained in RDDs across your cluster and parallelizes the operations you perform on them.</p>
<p style="margin-bottom: 0">Once created, RDDs offer two types of operations:</p>
<ul style="margin-top: 0">
<li>transformations,</li>
<li>and actions.</li>
</ul>
<p><i>Transformations</i> construct a new RDD from a previous one. <i>Actions</i>, on the other hand, compute a result based on an RDD, and either return it to the driver program or save it to an external storage system (e.g., HDFS).</p>
<p>Here's an example with a rough equivalent using Java Streams.</p>
<pre class="brush: java">
SparkConf conf = new SparkConf().setAppName(...);
JavaSparkContext sc = new JavaSparkContext(conf);
List<Integer> squares = sc.parallelize(Arrays.asList(1, 2, 3))
.map(n -> n*n)
.collect();
System.out.println(squares.toString());
// Rough equivalent using Java Streams
List<Integer> squares2 = Stream.of(1, 2, 3)
.map(n -> n*n)
.collect(Collectors.toList());
System.out.println(squares2.toString());
</pre>
<p>After setting up the Spark context, we call <code>parallelize()</code> which creates an RDD from the given list of elements. <code>map()</code> is a transformation, and <code>collect()</code> is an action. Transformations, like intermediate stream operations in Java, are lazily evaluated. In this example, Spark will not begin to execute the function provided in a call to <code>map()</code> until it sees an action. This approach might seem unusual at first, but it makes a lot of sense when dealing with huge amounts of data (big data, in other words). It allows Spark to split up the work and do them in parallel.</p>
<h3>Word Count Example</h3>
<p>Let's use <a href="https://en.wikipedia.org/wiki/Word_count">word count</a> as an example. Here, we have two implementations: one uses Apache Spark, and the other uses Java Streams.</p>
<p>Here's the Java Stream version.</p>
<pre class="brush: java">
public class WordCountJava {
private static final String REGEX = "\\s+";
public Map<String, Long> count(URI uri) throws IOException {
return Files.lines(Paths.get(uri))
.map(line -> line.split(REGEX))
.flatMap(Arrays::stream)
.map(word -> word.toLowerCase())
.collect(groupingBy(
identity(), TreeMap::new, counting()));
}
}
</pre>
<p>Here, we read the source file line by line and transforming each line in a sequence of words (via the <code>map()</code> intermediate operation). Since we have a sequence of words for each line and we have many lines, we convert them to a single sequence of words using <code>flatMap()</code>. In the end, we group them by their <code>identity()</code> (i.e. the identity of a string is the string itself) and we count them.</p>
<p>When tested against a text file that contains the two lines:</p>
<pre style="background-color: black; color: #ccc; line-height: 1em; padding: 0.25em">
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
</pre>
<p>It outputs the following map:</p>
<pre style="background-color: black; color: #ccc; line-height: 1em; padding: 0.25em">
{brown=2, dog=2, fox=2, jumps=2, lazy=2, over=2, quick=2, the=4}
</pre>
<p>And now, here's the Spark version.</p>
<pre class="brush: java">
public class WordCountSpark {
private static final String REGEX = "\\s+";
public List<Tuple2<String, Long>> count(URI uri, JavaSparkContext sc) throws IOException {
JavaRDD<String> input = sc.textFile(Paths.get(uri).toString());
return input.flatMap(
line -> Arrays.asList(line.split(REGEX)).iterator())
.map(word -> word.toLowerCase())
.mapToPair(word -> new Tuple2<String, Long>(word, 1L))
.reduceByKey((x, y) -> (Long) x + (Long) y)
.sortByKey()
.collect();
}
}
</pre>
<p>When run against the same two-line text file, it outputs the following:</p>
<pre style="background-color: black; color: #ccc; line-height: 1em; padding: 0.25em">
[(brown,2), (dog,2), (fox,2), (jumps,2), (lazy,2), (over,2), (quick,2), (the,4)]
</pre>
<p>The initial configuration of a <code>JavaSparkContext</code> has been excluded for brevity. We create a <code>JavaRDD</code> from a text file. It's worth mentioning that this initial RDD will operate line-by-line from the text file. That's why we split each line into sequence of words and <code>flatMap()</code> them. Then we transform a word into a key-value tuple with a count of one (1) for incremental counting. Once we have done that, we group by words (<code>reduceByKey()</code>) our key-value tuples from the previous RDD and in the end we sort them in natural order. </p>
<h3>In Closing</h3>
<p>As shown, both implementations are similar. The Spark implementation requires more setup and configuration, and is more powerful. Learning about intermediate and terminal stream operations can help get a Java developer started with understanding Apache Spark.</p>
<p>Thanks to Krischelle, <a href="https://github.com/alcaideredb">RB</a>, and <a href="https://plus.google.com/108603150488137710491">Juno</a>, for letting me participate in the PoCs that used Apache Spark.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com8tag:blogger.com,1999:blog-2526628884837213668.post-11945835875725133082016-12-30T10:30:00.000-08:002017-01-02T20:08:50.918-08:00Isolating the Domain Logic<p>In one design patterns class, I had an interesting discussion about modelling domain logic. Specifically, it was about <em>isolating the domain logic</em>. An application would typically be divided into three parts:</p>
<ol>
<li>Presentation (e.g. desktop GUI, browser, web service)</li>
<li>Domain logic</li>
<li>Infrastructure (e.g. persistence storage, e-mail)</li>
</ol>
<figure>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvNq8-7C3t2h6njNWvACy8Lc_GC7C7E6Tnx9l92g8AEdra5VfZ93IX2hcIatau-blQAuu6OQ3oE74KcEfc_3hsx0OQLXLm0hVww2WvgHCxzbUGFMjYCpwyX8iQ0C49ztHg_eRP3XEQ9ew/s1600/Isolating+Domain+Logic+1.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvNq8-7C3t2h6njNWvACy8Lc_GC7C7E6Tnx9l92g8AEdra5VfZ93IX2hcIatau-blQAuu6OQ3oE74KcEfc_3hsx0OQLXLm0hVww2WvgHCxzbUGFMjYCpwyX8iQ0C49ztHg_eRP3XEQ9ew/s320/Isolating+Domain+Logic+1.png" width="152" height="320" /></a></figure>
<p>The class found it interesting that the dependency arrows were pointing towards the domain logic part. They asked, “Is the diagram intentionally made wrong? Shouldn’t the domain logic part be dependent on the persistence storage?” It was a great question. And I wanted to share and post the discussion and explanation here.</p>
<h2>Often Misunderstood</h2>
<p>Most developers would usually have this misunderstanding in mind.</p>
<div>
<figure style="display: inline-block;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxmB0yFFH_KI133jQUcmkhv8owNAuc10CgIx4Vjrnhm18JdUsM5eOeFJ5QverBKr3Ua5bHbrh0ER59IkCNRQ2ewu67l4M-WCrQOucGl7467WkRM2Wlr6cDfeFk9eCvFQCKEn8A8i6YHNc/s1600/Isolating+Domain+Logic+Wrong.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxmB0yFFH_KI133jQUcmkhv8owNAuc10CgIx4Vjrnhm18JdUsM5eOeFJ5QverBKr3Ua5bHbrh0ER59IkCNRQ2ewu67l4M-WCrQOucGl7467WkRM2Wlr6cDfeFk9eCvFQCKEn8A8i6YHNc/s320/Isolating+Domain+Logic+Wrong.png" width="150" height="320" /></a>
<figcaption style="text-align: center">Misunderstood</figcaption>
</figure><span>vs.</span><figure style="display: inline-block">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvNq8-7C3t2h6njNWvACy8Lc_GC7C7E6Tnx9l92g8AEdra5VfZ93IX2hcIatau-blQAuu6OQ3oE74KcEfc_3hsx0OQLXLm0hVww2WvgHCxzbUGFMjYCpwyX8iQ0C49ztHg_eRP3XEQ9ew/s1600/Isolating+Domain+Logic+1.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvNq8-7C3t2h6njNWvACy8Lc_GC7C7E6Tnx9l92g8AEdra5VfZ93IX2hcIatau-blQAuu6OQ3oE74KcEfc_3hsx0OQLXLm0hVww2WvgHCxzbUGFMjYCpwyX8iQ0C49ztHg_eRP3XEQ9ew/s320/Isolating+Domain+Logic+1.png" width="152" height="320" /></a>
<figcaption style="text-align: center">Proper</figcaption>
</figure>
</div>
<p>And this misunderstanding is largely due to the sequence of operations. It usually starts with a trigger (e.g. a user clicking a button or a link) in the presentation layer, which then calls something within the domain logic layer, which then calls something within the infrastructure layer (e.g. update a database table record).</p>
<p>While this <em>is</em> the correct sequence of operations, there’s something subtle in the way in which the domain logic layer can be implemented. This has something to do with dependency inversion.</p>
<h2>Dependency Inversion Principle</h2>
<p>The domain logic layer may need something from the infrastructure layer, like some form of access to retrieve from persistence storage. The usual patterns for this are: DAO and repository. I won’t explain these two patterns here. Instead, I would point out that the interface definitions are placed within the domain logic layer, and their implementations are placed in another separate layer.</p>
<figure>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKMguiIf9YC2I1KnzcQzABl6cOqYmi9r5IeVRdaHKM4wvqnO8nnh1-lQLNEhYgOKHFQMt6GZkdjsCTcSSb9-YfZGNeMnAkHLeqeII23LpsRE-0bIdTGHIQ1LpfGsAUdFlwUKFGezKyARQ/s1600/Isolating+Domain+Logic+2.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKMguiIf9YC2I1KnzcQzABl6cOqYmi9r5IeVRdaHKM4wvqnO8nnh1-lQLNEhYgOKHFQMt6GZkdjsCTcSSb9-YfZGNeMnAkHLeqeII23LpsRE-0bIdTGHIQ1LpfGsAUdFlwUKFGezKyARQ/s320/Isolating+Domain+Logic+2.png" width="320" height="176" /></a></figure>
<p>Placing the (DAO and repository) interface definitions inside the domain logic layer means that it is the domain logic layer that defines it. It is the one that dictates which methods are needed, and what return types are expected. This also marks the boundaries of the domain logic.</p>
<p>This separation between interface and implementation may be subtle, but key. Placing just the interface definitions allows the domain logic part to be free from infrastructure details, and allows it to be unit-tested without actual implementations. The interfaces can have mock implementations during unit testing. This subtle difference makes a big difference in rapid verification of (the development team’s understanding of) business rules.</p>
<p>This separation is the classic <em>dependency inversion principle</em> in action. Domain logic (higher-level modules) should not depend on DAO and repository implementations (low-level modules). Both should depend on abstractions. The domain logic defines the abstractions, and infrastructure implementations depend on these abstractions.</p>
<p>Most novice teams I’ve seen, place the DAO and repository interfaces together with their infrastructure-specific implementations. For example, say we have an <code>StudentRepository</code> and its JPA-specific implementation <code>StudentJpaRepository</code>. I would usually find novice teams placing them in the same package. While this is fine, since the application will still compile successfully. But the separation is gone, and domain logic is no longer isolated.</p>
<p>Now that I’ve explained why and how the domain logic part does not depend on the infrastructure part, I’d like to touch on how the presentation part is accidentally entangled with the domain logic.</p>
<h2>Separated Presentation</h2>
<p>Another thing I often see with novice teams is how they end up entangling their domain logic with their presentation. And this results into this nasty cyclic dependency. This cyclic dependency is more logical than physical. Which makes it all the more difficult to detect and prevent.</p>
<figure>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh392gMIirFuoblLgDmi15FAgF21kOddNYh-6fZylHGuhHhZmzChxTmvnCfXN8oQgSxtDX6iBDX07AQUvkxiGK96E5NQW8H70JpOgkK8q3ztDE1NPI_tuSCxhiK8_orsbbmTyqYXodAwJ8/s1600/Isolating+Domain+Logic+Wrong+2.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh392gMIirFuoblLgDmi15FAgF21kOddNYh-6fZylHGuhHhZmzChxTmvnCfXN8oQgSxtDX6iBDX07AQUvkxiGK96E5NQW8H70JpOgkK8q3ztDE1NPI_tuSCxhiK8_orsbbmTyqYXodAwJ8/s320/Isolating+Domain+Logic+Wrong+2.png" width="320" height="178" /></a></figure>
<p>I won’t use a rich GUI presentation example here, since <a href="http://martinfowler.com/eaaDev/SeparatedPresentation.html">Martin Fowler has already written a great piece on it</a>. Instead, I’ll use a web-browser-based presentation as an example.</p>
<p>Most web-based systems would use a web framework for its presentation. These frameworks usually implement some form of MVC (model-view-controller). The model used is usually the model straight from the domain logic part. Unfortunately, most MVC frameworks require something about the model. In the Java world, most MVC frameworks require that the model follow JavaBean conventions. Specifically, it requires the model to have a public zero-arguments constructor, and getters and setters. The zero-arguments constructor and setters are used to automatically bind parameters (from HTTP POST) to the model. The getters are used in rendering the model in a view.</p>
<p>Because of this implied requirement by MVC frameworks used in the presentation, developers would add a public zero-arguments constructor, getter and setters, to all their domain entities. And they would justify this as being required. Unfortunately, this gets the in the way of implementing domain logic. It gets entangled with the presentation. And worse, I’ve seen domain entities being polluted with code that emits HTML-encoded strings (e.g. HTML code with less-than and greater-than signs encoded) and XML, just because of presentation.</p>
<p>If it is all right to have your domain entity implemented as a JavaBean, then it would be fine to have it used directly in your presentation. But if the domain logic gets a bit more complicated, and requires the domain entity to lose its JavaBean-ness (e.g. no more public zero-arguments constructor, no more setters), then it would be advisable for the domain logic part to implement domain logic, and have the presentation part adapt by creating another JavaBean object to satisfy its MVC needs.</p>
<p>An example I use often is a <code>UserAccount</code> that is used to authenticate a user. In most cases, when a user wishes to change the password, the old password is also needed. This helps prevent unauthorized changing of the password. This is clearly shown in the code below.</p>
<pre class="brush: java">
public class UserAccount {
...
public void changePassword(
String oldPassword, String newPassword) {…}
}
</pre>
<p>But this does not follow JavaBean conventions. And if the MVC presentation framework would not work well with the <code>changePassword</code> method, a naive approach would be to remove the erring method and add a <code>setPassword</code> method (shown below). This weakens the isolation of the domain logic, and causes the rest of the team to implement it all over the place.</p>
<pre class="brush: java">
public class UserAccount {
...
public void setPassword(String password) {…}
}
</pre>
<p>It’s important for developers to understand that the presentation depends on the domain logic. And not the other way around. If the presentation has needs (e.g. JavaBean convention), then it <em>should not</em> have the domain logic comply with that. Instead, the presentation should create additional classes (e.g. JavaBeans) that have knowledge of the corresponding domain entities. But unfortunately, I still see a lot of teams forcing their domain entities to look like JavaBeans just because of presentation, or worse, having domain entities create JavaBeans (e.g. DTOs) for presentation purposes.</p>
<h2>Arrangement Tips</h2>
<p>Here’s a tip in arranging your application. Keep your domain entities and repositories in one package. Keep your repository and other infrastructure implementations in a separate package. Keep your presentation-related classes in its own package. Be mindful of which package depends on which package. The package that contains the domain logic is preferrably at the center of it all. Everything else depends on it.</p>
<p>When using Java, the packages would look something like this:</p>
<ul>
<li><code>com.acme.myapp.context1.domain.model</code>
<ul>
<li>Keep your domain entities, value objects, and repositories (interface definitions only) here</li>
</ul>
</li>
<li><code>com.acme.myapp.context1.infrastructure.persistence.jpa</code>
<ul>
<li>Place your JPA-based repository and other JPA persistence-related implementations here</li>
</ul>
</li>
<li><code>com.acme.myapp.context1.infrastructure.persistence.jdbc</code>
<ul>
<li>Place your JDBC-based repository and other JDBC persistence-related implementations here</li>
</ul>
</li>
<li><code>com.acme.myapp.context1.presentation.web</code>
<ul>
<li>Place your web/MVC presentation components here. If the domain entities needed for presentation do not comply with MVC framework requirements, create additional classes here. These additional classes will <em>adapt</em> the domain entities for presentation-purposes, and still keep the domain entities separated from presentation.</li>
</ul>
</li>
</ul>
<p>Note that I’ve used <code>context1</code>, since there could be several contexts (or sub-systems) in a given application (or system). I’ll discuss about having multiple contexts and having multiple models in a future post.</p>
<p>That’s all for now. I hope this short explanation can shed some light to those who wonder why their code is arranged and split in a certain way.</p>
<p>Thanks to <a href="https://plus.google.com/108603150488137710491">Juno Aliento</a> for helping me with the class during this interesting discussion.</p>
<p>Happy holidays!</p>Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com2tag:blogger.com,1999:blog-2526628884837213668.post-937499457521511142016-10-27T10:00:00.000-07:002016-10-27T10:00:13.253-07:00Architectural Layers and Modeling Domain Logic<p>As I was discussing the <abbr title="Patterns of Enterprise Application Architecture">PoEAA</abbr> patterns used to model domain logic (i.e. transaction script, table module, domain model), I noticed that people get the impression (albeit wrong impression) that the domain model pattern is best. So, they set out to apply it on everything.</p>
<h2>Not Worthy of Domain Model Pattern</h2>
<p>Let's get real. The majority of sub-systems are <abbr title="Create, Read, Update, Delete">CRUD</abbr>-based. Only a certain portion of the system requires the domain model implementation pattern. Or, put it in another way, there are parts of the application that just needs forms over data, and some validation logic (e.g. required/mandatory fields, min/max values on numbers, min/max length on text). For these, the domain model is not worth the effort.</p>
<p>For these, perhaps an anemic domain model would fit nicely.</p>
<h2>Anemic Domain Model Isn't As Bad As It Sounds</h2>
<p>The anemic domain model isn't as bad as it sounds. There, I said it (at least here in my blog post).</p>
<p>But how does it look like?</p>
<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhicwY5YG8lWci-ZtW7ClRa8d9nZ90D8lAnP1nHjavXoMXBZjXDWlaVAxq7rCygEhsLCzaoXn3wVDzv8wv3kmyvgX6SOVbZhK_PgEQpZHlnkbZ1Ptg8wZkDLTGPyDZ8_hwF8RIY1IrL6hs/s1600/Class+Diagram1.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhicwY5YG8lWci-ZtW7ClRa8d9nZ90D8lAnP1nHjavXoMXBZjXDWlaVAxq7rCygEhsLCzaoXn3wVDzv8wv3kmyvgX6SOVbZhK_PgEQpZHlnkbZ1Ptg8wZkDLTGPyDZ8_hwF8RIY1IrL6hs/s320/Class+Diagram1.png" width="320" height="138" /></a></p>
<pre class="brush: java">
package com.acme.bc.domain.model;
...
@Entity
class Person {
@Id ... private Long id;
private String firstName;
private String lastName;
// ...
// getters and setters
}
...
interface PersonRepository /* extends CrudRepository<Person, Long> */ {
// CRUD methods (e.g. find, find/pagination, update, delete)
}
</pre>
<pre class="brush: java">
package com.acme.bc.infrastructure.persistence;
...
class PersonRepositoryJpa implements PersonRepository {
...
}
</pre>
<p>In the presentation layer, the controllers can have access to the repository. The repository does its job of abstracting persistence details.</p>
<pre class="brush: java; highlight: [5]">
package com.acme.bc.interfaces.web;
@Controller
class PersonsController {
private PersonRepository personRepository;
public PersonsController(PersonRepository personRepository) {...}
// ...
}
</pre>
<p>In this case, having the <code>Person</code> class exposed to the presentation layer is perfectly all right. The presentation layer can use it directly, since it has a public zero-arguments constructor, getters and setters, which are most likely needed by the view.</p>
<p>And there you have it. A simple CRUD-based application.</p>
<p>Do you still need a service layer? No. Do you still need <abbr title="Data Transfer Object">DTO</abbr> (data transfer objects)? No. In this simple case of CRUD, you don't need additional services or <abbr title="Data Transfer Object">DTO</abbr>s.</p>
<p>Yes, the <code>Person</code> looks like a domain entity. But it does <em>not</em> contain logic, and is simply used to transfer data. So, it's really just a DTO. But this is all right since it does the job of holding the data stored-to and retrieved-from persistence.</p>
<p>Now, <em>if</em> the business logic starts to get more complicated, <em>some</em> entities in the <em>initially anemic</em> domain model can become richer with behavior. And if so, those entities can merit a domain model pattern.</p>
<h2>Alternative to Anemic Domain Model</h2>
As an alternative to the anemic domain model (discussed above), the classes can be moved out of the domain logic layer and in to the presentation layer. Instead of naming it <code>PersonRepository</code>, it is now named <code>PersonDao</code>.
<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYLWimLZoNaeguRmS3URwXNdpKxKWVwKF9-S4djeq1YGbLyD4Ju_5f5h20Px_fQonoPi6LE8G8kgZ3oCPuxgxxS4ipKqb_x3yO30bTwr7uu0QwfXWID8YwWrigONLxJc1HGIH1vTlb4G8/s1600/Class+Diagram0.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYLWimLZoNaeguRmS3URwXNdpKxKWVwKF9-S4djeq1YGbLyD4Ju_5f5h20Px_fQonoPi6LE8G8kgZ3oCPuxgxxS4ipKqb_x3yO30bTwr7uu0QwfXWID8YwWrigONLxJc1HGIH1vTlb4G8/s200/Class+Diagram0.png" width="200" height="132" /></a></p>
<pre class="brush: java; highlight: [8]">
package com.acme.bc.interfaces.web;
@Entity
class Person {...}
@Controller
class PersonsController {
private PersonDao personDao;
public PersonsController(PersonDao personDao) {...}
// ...
}
interface PersonDao /* extends CrudRepository<Person, Long> */ {
// CRUD methods (e.g. find, find/pagination, update, delete)
}
</pre>
<pre class="brush: java">
package com.acme.bc.infrastructure.persistence;
class PersonDaoJpa implements PersonDao {
...
}
</pre>
<h2>Too Much Layering</h2>
<p>I think that it would be an overkill if you have to go through a mandatory application service that does not add value.</p>
<pre class="brush: java; highlight: [5]">
package com.acme.bc.interfaces.web;
...
@Controller
class PersonsController {
private PersonService personService;
public PersonsController(PersonService personService) {...}
// ...
}
</pre>
<pre class="brush: java; highlight: [7, 8]">
package com.acme.bc.application;
...
@Service
class PersonService {
private PersonRepository personRepository;
public PersonService(PersonRepository personRepository) {...}
// expose repository CRUD methods and pass to repository
// no value add
}
</pre>
<aside style="clear: right; float: right; margin: 1em 0 1em 1em; width: 50%; max-width: 20em; padding: 0.5em; border: 1px solid #ccc; border-radius: 4px">
<p>Keep the repository in the <code>domain.model</code> package. Place the repository implementations in another package (e.g. <code>infrastructure.persistence</code>). But why?</p>
<p>The <code>domain.model</code> package is where the repository is defined. The elements in the domain model dictate what methods are needed in the repository interface definition. Thus, the repository definition is placed in the <code>domain.model</code> package. The repository implementations need to follow what new methods are added (or remove unused ones). This packaging follows the dependency inversion principle. The <code>infrastructure.persistence</code> package depends on the <code>domain.model</code> package, and not the other way around.</p>
</aside>
<h2>Application Services for Transactions</h2>
<p>So, when would application services be appropriate? The application services are responsible for driving workflow and coordinating transaction management (e.g. by use of the declarative transaction management support in Spring).</p>
<p>If you find the simple CRUD application needing to start transactions in the presentation-layer controller, then it might be a good sign to move them into an application service. This usually happens when the controller needs to update more than one entity that does not have a single root. The usual example here is transferring amounts between bank accounts. A transaction is needed to ensure that debit and credit both succeed, or both fail.</p>
<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEIRjONV_l-CgXQo290X8pfpj8EqC8YhSY63rELfTObZ_x55J1yub_ppunCB49inqvXXPmleWrStYFJ9wc2lETUrVpEVFC2vhjHAH0mjawGxxm1AN6VM9tk4umAdhUShD1WeWffnBuMtE/s1600/Class+Diagram2.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEIRjONV_l-CgXQo290X8pfpj8EqC8YhSY63rELfTObZ_x55J1yub_ppunCB49inqvXXPmleWrStYFJ9wc2lETUrVpEVFC2vhjHAH0mjawGxxm1AN6VM9tk4umAdhUShD1WeWffnBuMtE/s320/Class+Diagram2.png" width="320" height="138" /></a></p>
<pre class="brush: java">
package sample.domain.model;
...
@Entity
class Account {...}
...
interface AccountRepository {...}
</pre>
<pre class="brush: java; highlight: [7, 8]">
package sample.interfaces.web;
...
@Controller
class AccountsController {
private AccountRepository accountRepository;
...
@Transactional
public ... transfer(...) {...}
}
</pre>
<p>If you see this, then it might be a good idea to move this (from the presentation layer) to an application-layer service.</p>
<pre class="brush: java; highlight: [6]">
package sample.interfaces.web;
...
@Controller
class AccountsController {
private AccountRepository accountRepository;
private TransferService transferService;
...
public ... transfer(...) {...}
}
</pre>
<pre class="brush: java">
package sample.application;
...
@Service
@Transactional
class TransferService {
private AccountRepository accountRepository;
...
public ... transfer(...) {...}
}
</pre>
<pre class="brush: java">
package sample.domain.model;
...
@Entity
class Account {...}
...
interface AccountRepository {...}
</pre>
<h2>Domain Model Pattern (only) for Complex Logic</h2>
<p>I'll use the double-entry accounting as an example. But I'm sure there are more complex logic that's better suited.</p>
<p>Let's say we model journal entries and accounts as domain entities. The account contains a balance (a monetary amount). But this amount is not something that one would simply set. A journal entry needs to be created. When the journal entry is posted, it will affect the specified accounts. The account will then update its balance.</p>
<pre class="brush: java">
package ….accounting.domain.model;
...
/** Immutable */
@Entity
class JournalEntry {
// zero-sum items
@ElementCollection
private Collection<JournalEntryItem> items;
...
}
...
/** A value object */
@Embeddable
class JournalEntryItem {...}
...
interface JournalEntryRepository {...}
...
@Entity
class Account {...}
...
interface AccountRepository {...}
...
@Entity
class AccountTransaction {...}
...
interface AccountTransactionRepository {...}
</pre>
<p>Now, in this case, a naive implementation would have a presentation-layer controller create a journal entry object, and use a repository to save it. And at some point in time (or if auto-posting is used), the corresponding account transactions are created, with account balances updated. All this needs to be rolled into a transaction (i.e. all-or-nothing).</p>
<p>Again, this transaction is ideally moved to an application service.</p>
<pre class="brush: java">
package ….accounting.application;
@Service
@Transactional
class PostingService {...}
</pre>
<p>If there's a need to allow the user to browse through journal entries and account transactions, the presentation-layer controller can directly use the corresponding repositories. If the domain entities are not suitable for the view technology (e.g. it doesn't follow JavaBean naming conventions), then the presentation-layer can define DTOs that are suitable for the view. <em>Be careful! Don't change the domain entity just to suit the needs of the presentation-layer.</em></p>
<pre class="brush: java">
package ….interfaces.web;
@Controller
class AccountsController {
private AccountRepository accountRepository;
private AccountTransactionRepository accountTransactionRepository;
private PostingService postingService;
...
}
</pre>
<h2>In Closing...</h2>
<p>So, there you have it. Hopefully, this post can shed some light on when (and when not) to use domain model pattern.</p>
<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjctqyB3JBNiEmYY6v0HU6G_hHMG2f5LIZqvE9hTvekCU9axlOylRNPE-301NoLuhVUFPLD5fQpTludiP3iIQrvKJuhaHfTCmI8nzJNFFiYFLC3tfc6iYhh2dtF54eT1_Ky6KV9QgXv2Rs/s1600/Class+Diagram3.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjctqyB3JBNiEmYY6v0HU6G_hHMG2f5LIZqvE9hTvekCU9axlOylRNPE-301NoLuhVUFPLD5fQpTludiP3iIQrvKJuhaHfTCmI8nzJNFFiYFLC3tfc6iYhh2dtF54eT1_Ky6KV9QgXv2Rs/s320/Class+Diagram3.png" width="320" height="107" /></a></p>
<aside>
<p>Now I think I need a cold one. <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8WXjEGDPIzb913nnkZn79_jjU4Aou_i8ImmxuWAEvHvH0QE2VqdIU10-QZFvHsvhUovCzwj6DtXO3_tYKcdWhRriQnEQOvv_D0mjCKsgcVJ59DgL_yJLcDHM7_NBBza7w5WjV3YNNib4/s1600/butterscotch-beer.png" imageanchor="1" ><img border="0" style="vertical-align: top" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8WXjEGDPIzb913nnkZn79_jjU4Aou_i8ImmxuWAEvHvH0QE2VqdIU10-QZFvHsvhUovCzwj6DtXO3_tYKcdWhRriQnEQOvv_D0mjCKsgcVJ59DgL_yJLcDHM7_NBBza7w5WjV3YNNib4/s320/butterscotch-beer.png" width="237" height="320" /></a></p>
</aside>Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com3tag:blogger.com,1999:blog-2526628884837213668.post-76794172502126330282016-08-23T02:43:00.001-07:002021-06-07T19:31:47.113-07:00Spring Security OAuth2 with Google<p>I needed to create a web app using Spring MVC and secure it using OAuth2 with Google as a provider for authentication. <a href="https://jhasaket.blogspot.com/2014/09/securing-spring-mvc-application-using.html">Saket's Blog</a> (posted back in September 2014) provided a good guide. But I needed something slightly different. I needed one that uses Maven (not Gradle) and minus Spring Boot. So, I thought it would just be a simple thing to do. But as I found out, it was not so simple, and I'm writing some details here to help others in using OAuth2 to secure their Spring MVC web apps.</p>
<p>Here's what I configured to make my web application use OAuth2 with Google as the provider.</p>
<ul>
<li>Enable Spring Security with <code>@EnableWebSecurity</code>.</li>
<li>Add an <code>OAuth2ClientAuthenticationProcessingFilter</code> bean to the security filter chain just <em>before</em> the filter security interceptor. This authentication processing filter is configured to know where the authorization code resource can be found. This makes it possible for it to throw an exception that redirects the user to the authorization server for authentication and authorization.</li>
<li>Set an authentication entry point (specifically a <code>LoginUrlAuthenticationEntryPoint</code>) that redirects to the same URL as the one being detected by the <code>OAuth2ClientAuthenticationProcessingFilter</code>. Say, we choose the path “/oauth2/callback”. This path should be the one used by both authentication entry point and authentication processing filter.</li>
<li>Add <code>@EnableOAuth2Client</code> to create an <code>OAuth2ClientContextFilter</code> bean and make an <code>OAuth2ClientContext</code> available in request scope. To make request scope possible in the security filter chain, add a <code>RequestContextListener</code> or <code>RequestContextFilter</code>.</li>
<li>Add the <code>OAuth2ClientContextFilter</code> bean to the security filter chain just <em>after</em> the exception translation filter. This filter handles the exception that redirects the user (thrown by the authentication process filter). It handles this exception by sending a redirect.</li>
</ul>
<h2>Authorization Code Resource</h2>
<p>The authentication processing filter needs to know where to redirect the user for authentication. So, a bean is configured and injected into the authentication process filter.</p>
<pre class="brush: java; highlight: [27]">
@Configuration
@EnableWebSecurity
@EnableOAuth2Client
@PropertySource("classpath:google-oauth2.properties")
public class ... extends WebSecurityConfigurerAdapter {
...
@Value("${oauth2.clientId}")
private String clientId;
@Value("${oauth2.clientSecret}")
private String clientSecret;
@Value("${oauth2.userAuthorizationUri}")
private String userAuthorizationUri;
@Value("${oauth2.accessTokenUri}")
private String accessTokenUri;
@Value("${oauth2.tokenName}")
private String tokenName;
@Value("${oauth2.scope}")
private String scope;
@Value("${oauth2.userInfoUri}")
private String userInfoUri;
@Value("${oauth2.filterCallbackPath}")
private String oauth2FilterCallbackPath;
@Bean
@Description("Authorization code resource")
public OAuth2ProtectedResourceDetails authorizationCodeResource() {
AuthorizationCodeResourceDetails details = new AuthorizationCodeResourceDetails();
...
details.setClientId(clientId);
details.setClientSecret(clientSecret);
details.setUserAuthorizationUri(userAuthorizationUri);
details.setAccessTokenUri(accessTokenUri);
details.setTokenName(tokenName);
String commaSeparatedScopes = scope;
details.setScope(parseScopes(commaSeparatedScopes));
details.setAuthenticationScheme(AuthenticationScheme.query);
details.setClientAuthenticationScheme(AuthenticationScheme.form);
return details;
}
private List<String> parseScopes(String commaSeparatedScopes) {...}
...
@Bean
@Description("Enables ${...} expressions in the @Value annotations"
+ " on fields of this configuration. Not needed if one is"
+ " already available.")
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
</pre>
<p>Note that the authorization code resource details are externalized. These details include the URI for authentication, the URI to exchange an authorization code with an access token, client ID, and client secret.</p>
<h2>Authentication Processing Filter</h2>
<p>With an authorization code resource bean configured, we configure an authentication processing filter bean that will redirect to the authorization code resource when the incoming request is not yet authenticated. Note that the authentication processing filter is injected with an <code>OAuth2RestTemplate</code> that points to the authorization code resource.</p>
<pre class="brush: java; highlight: [11, 24, 26, 27, 28]">
@Configuration
@EnableWebSecurity
@EnableOAuth2Client
@PropertySource("classpath:google-oauth2.properties")
public class ... extends WebSecurityConfigurerAdapter {
@Autowired
private OAuth2ClientContext oauth2ClientContext;
...
@Bean
@Description("Authorization code resource")
public OAuth2ProtectedResourceDetails authorizationCodeResource() {
AuthorizationCodeResourceDetails details = new AuthorizationCodeResourceDetails();
...
return details;
}
@Bean
@Description("Filter that checks for authorization code, "
+ "and if there's none, acquires it from authorization server")
public OAuth2ClientAuthenticationProcessingFilter
oauth2ClientAuthenticationProcessingFilter() {
// Used to obtain access token from authorization server (AS)
OAuth2RestOperations restTemplate = new OAuth2RestTemplate(
authorizationCodeResource(),
oauth2ClientContext);
OAuth2ClientAuthenticationProcessingFilter filter =
new OAuth2ClientAuthenticationProcessingFilter(oauth2FilterCallbackPath);
filter.setRestTemplate(restTemplate);
// Set a service that validates an OAuth2 access token
// We can use either Google API's UserInfo or TokenInfo
// For this, we chose to use UserInfo service
filter.setTokenServices(googleUserInfoTokenServices());
return filter;
}
@Bean
@Description("Google API UserInfo resource server")
public GoogleUserInfoTokenServices googleUserInfoTokenServices() {
GoogleUserInfoTokenServices userInfoTokenServices =
new GoogleUserInfoTokenServices(userInfoUri, clientId);
return userInfoTokenServices;
}
...
}
</pre>
<p>Note that the access token is further checked by using it to access a secured resource (provided by a resource server). In this case, the Google API to retrieve user information like email and photo is used.</p>
<p>Arguably, the authorization code resource does not need to be configured as a bean, since it is only used by the authentication processing filter.</p>
<h2>Authentication Entry Point</h2>
<p>The authentication processing filter and the authentication entry point are configured to detect the same request path.</p>
<pre class="brush: java; highlight: [16, 23]">
@Configuration
@EnableWebSecurity
@EnableOAuth2Client
@PropertySource("classpath:google-oauth2.properties")
public class ... extends WebSecurityConfigurerAdapter {
...
public OAuth2ProtectedResourceDetails authorizationCodeResource() {...}
@Bean
@Description("Filter that checks for authorization code, "
+ "and if there's none, acquires it from authorization server")
public OAuth2ClientAuthenticationProcessingFilter
oauth2ClientAuthenticationProcessingFilter() {
...
OAuth2ClientAuthenticationProcessingFilter filter =
new OAuth2ClientAuthenticationProcessingFilter(oauth2FilterCallbackPath);
...
return filter;
}
...
@Bean
public AuthenticationEntryPoint authenticationEntryPoint() {
return new LoginUrlAuthenticationEntryPoint(oauth2FilterCallbackPath);
}
...
}
</pre>
<h2>So, how does this all work?</h2>
<p>This is how the security filter chain will look like with the added custom filters. Note that for brevity, not all filters were included.</p>
<div style="text-align: center; margin-bottom: 1em">
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Web Browser</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">SecurityContextPersistenceFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">LogoutFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">ExceptionTranslationFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white">OAuth2Client<wbr>ContextFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white">OAuth2Client<wbr>AuthenticationProcessingFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">FilterSecurityInterceptor</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Secured Resource</div>
</div>
<p>So, here's what happens at runtime. The <em>client</em> referred to here, is a web application that uses OAuth2 for authentication.</p>
<ol>
<li>
<div>Request for a secured resource on the <em>client</em> is received. It travels through the security filter chain until <code>FilterSecurityInterceptor</code>. The request has not been authenticated yet (i.e. security context does not contain an authentication object), and the <code>FilterSecurityInterceptor</code> throws an exception (<code>AuthenticationCredentialsNotFoundException</code>). This authentication exception travels up the security filter chain, and is handled by <code>ExceptionTranslationFilter</code>. It detects that an authentication exception occured, and delegates to the authentication entry point. The configured authentication entry point (<code>LoginUrlAuthenticationEntryPoint</code>) redirects the user to a new location (e.g. <code>“/oauth2/callback”</code>). The request for a secured resource is <em>saved</em>, and request processing completes.</div>
<div style="text-align: center; margin-bottom: 1em">
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Web Browser</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">SecurityContextPersistenceFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">LogoutFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; position: relative">
<span>ExceptionTranslationFilter</span>
<span style="position: absolute; left: 80%; top: 50%; display: block; width: 12em; border: 1px solid #888; padding: 0.25em 0.5em; background-color: #eee; font-size: 0.8em; line-height: 1em">Delegate to authentication entry point</span>
</div>
<div>↓ ↑</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white">OAuth2Client<wbr>ContextFilter</div>
<div>↓ ↑</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white">OAuth2Client<wbr>AuthenticationProcessingFilter</div>
<div>↓ ↑</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; position: relative">
<span>FilterSecurityInterceptor</span>
<span style="position: absolute; left: 80%; top: 50%; display: block; width: 8em; border: 1px solid #888; padding: 0.25em 0.5em; background-color: #eee; font-size: 0.8em; line-height: 1em">Throws exception!</span>
</div>
<div> </div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Secured Resource</div>
</div>
</li>
<li><div>Since a redirect is the response of the previous request, a request to the new location is made. This request travels through the security filter chain until <code>OAuth2ClientAuthenticationProcessingFilter</code> determines that it is a request for authentication (e.g. it matches <code>“/oauth2/callback”</code>). Upon checking the request, it determines that there’s no authorization code, and throws an exception (<code>UserRedirectRequiredException</code>) that contains a URL to the authorization code resource (e.g. <code>https://accounts.google.com/o/oauth2/v2/auth?client_id=https://accounts.google.com/o/oauth2/v2/auth?client_id=…&<b>redirect_uri=http://…/…/oauth2/callback</b>&response_type=code&scope=…&state=…</code>). This exception is handled by <code>OAuth2ClientContextFilter</code>. And request processing completes.</div>
<div style="text-align: center; margin-bottom: 1em">
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Web Browser</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">SecurityContextPersistenceFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">LogoutFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">ExceptionTranslationFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white; position: relative">
<span>OAuth2Client<wbr>ContextFilter</span>
<span style="position: absolute; left: 80%; top: 50%; display: block; width: 12em; border: 1px solid #888; padding: 0.25em 0.5em; background-color: #eee; font-size: 0.8em; line-height: 1em; color: #333">Handle exception by sending redirect</span>
</div>
<div>↓ ↑</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white; position: relative">
<span>OAuth2Client<wbr>AuthenticationProcessingFilter</span>
<span style="position: absolute; left: 80%; top: 50%; display: block; width: 8em; border: 1px solid #888; padding: 0.25em 0.5em; background-color: #eee; font-size: 0.8em; line-height: 1em; color: #333">Throws exception!</span>
</div>
<div> </div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">FilterSecurityInterceptor</div>
<div> </div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Secured Resource</div>
</div>
</li>
<li>Just as before, the redirect is followed. This time, it is a redirect to the authorization server (e.g. <code>https://accounts.google.com/o/oauth2/v2/auth</code>). The user is asked to authenticate (if not yet authenticated).</li>
<li>Next, the user is asked to allow/authorize the <em>client</em> to have access to his/her information. After the user decides to allow/authorize the <em>client</em>, the authorization server redirects back to the <em>client</em> (based on the <code>redirect_uri</code> parameter).</li>
<li><div>Request on the <em>client</em> is received. It travels through the security filter chain until <code>OAuth2ClientAuthenticationProcessingFilter</code> determines that it is a request for authentication (e.g. it matches <code>“/oauth2/callback”</code>). It finds that the request contains an authorization code, and proceeds to exchange the authorization code for an access token. Furthermore, it validates the access token by accessing a resource (on a resource server), and creates an <code>Authentication</code> object (with <code>Principal</code> and <code>GrantedAuthority</code> objects). This will be stored in the session and in the security context. And request processing completes with a redirect to the <em>saved</em> request (from #1).</div>
<div style="text-align: center; margin-bottom: 1em">
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Web Browser</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">SecurityContextPersistenceFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">LogoutFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">ExceptionTranslationFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white">OAuth2Client<wbr>ContextFilter</div>
<div>↓</div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white; position: relative">
<span>OAuth2Client<wbr>AuthenticationProcessingFilter</span>
<span style="position: absolute; left: 80%; top: -20%; display: block; width: 14em; border: 1px solid #888; padding: 0.25em 0.5em; background-color: #eee; font-size: 0.8em; line-height: 1em; color: #333">Exchanges authorization code with access token; creates authentication object and stores it in session</span>
</div>
<div> </div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto">FilterSecurityInterceptor</div>
<div> </div>
<div style="padding: 0.25em 0.5em; max-width: 100%; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Secured Resource</div>
</div>
</li>
<li><div>Just as before, the redirect is followed. It travels through the security filter chain. This time, the <code>FilterSecurityInterceptor</code> allows the request to proceed, since there is an authentication object in the security context (retrieved from session). The secured resource is provided to the user (e.g. render a view/page of the secured resource).</div>
<div style="text-align: center; margin-bottom: 1em">
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic">Web Browser</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">SecurityContextPersistenceFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">LogoutFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">ExceptionTranslationFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white">OAuth2Client<wbr>ContextFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; background-color: #373; color: white">OAuth2Client<wbr>AuthenticationProcessingFilter</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto">FilterSecurityInterceptor</div>
<div style="">↓</div>
<div style="padding: 0.25em 0.5em; width: 20em; border: 1px solid #ccc; margin: 0 auto; font-style: italic; position: relative">
<span>Secured Resource</span>
<span style="position: absolute; left: 80%; top: -20%; width: 2em; height: 2em; border: 1px solid #333; background-color: yellow; border-radius: 50%; font-style: normal; font-weight: bold; font-family: monospace; transform: rotate(90deg)">:)</span>
</div>
</div>
</li>
</ol>
<h2>Code and Credits</h2>
<p>The code for the sample web application can be found <a href="https://github.com/lorenzodee/spring-security-oauth2-google">here at my GitHub account</a>.</p>
<p>Again, thanks to <a href="https://jhasaket.blogspot.com/2014/09/securing-spring-mvc-application-using.html">Saket's Blog</a>.</p>
<p>Also, check out <a href="https://examples.javacodegeeks.com/security-architecture-with-spring/">Security Architecture with Spring</a> at <a href="https://www.javacodegeeks.com/">Java Code Geeks</a>.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com5tag:blogger.com,1999:blog-2526628884837213668.post-644484865004164392016-07-21T10:00:00.002-07:002016-07-21T10:00:12.134-07:00One-shot Delete with Hibernate (JPA)<p>
In older versions of Hibernate, I can see the <a href="https://docs.jboss.org/hibernate/orm/5.0/manual/en-US/html/ch20.html#performance-collections-oneshotdelete">one-shot delete</a> indicated in the manual. But newer versions no longer have this section. I'm not sure why. So, in this post, I take a look if it still works.
</p>
<p>The <a href="https://docs.jboss.org/hibernate/orm/5.0/manual/en-US/html/ch20.html#performance-collections-oneshotdelete">one-shot delete</a> section says:</p>
<blockquote>
<p>Deleting collection elements one by one can sometimes be extremely inefficient. Hibernate knows not to do that in the case of an newly-empty collection (if you called <code>list.clear()</code>, for example). In this case, Hibernate will issue a single <code>DELETE</code>.</p>
<p>Suppose you added a single element to a collection of size twenty and then remove two elements. Hibernate will issue one <code>INSERT</code> statement and two <code>DELETE</code> statements, unless the collection is a bag. This is certainly desirable.</p>
<p>However, suppose that we remove eighteen elements, leaving two and then add thee new elements. There are two possible ways to proceed</p>
<ul>
<li>delete eighteen rows one by one and then insert three rows</li>
<li>remove the whole collection in one SQL <code>DELETE</code> and insert all five current elements one by one</li>
</ul>
<p>Hibernate cannot know that the second option is probably quicker. It would probably be undesirable for Hibernate to be that intuitive as such behavior might confuse database triggers, etc.</p>
<p>
Fortunately, you can force this behavior (i.e. the second strategy) at any time by discarding (i.e. dereferencing) the original collection and returning a newly instantiated collection with all the current elements.
</p>
<p>
One-shot-delete does not apply to collections mapped <code>inverse="true"</code>.
</p>
</blockquote>
<p>The <code>inverse="true"</code> is for (Hibernate Mapping) XML. But in this post, we'll see how "one-shot delete" works in <abbr title="Java Persistence API">JPA</abbr> (with Hibernate as the provider).</p>
<p>We will try different approaches and see which one will result to a one-shot delete.</p>
<ol>
<li>Bi-directional one-to-many</li>
<li>Uni-directional one-to-many (with join table)</li>
<li>Uni-directional one-to-many (with no join table)</li>
<li>Uni-directional one-to-many (using <code>ElementCollection</code>)</li>
</ol>
<p>We'll use a <code>Cart</code> entity with many <code>CartItem</code>s.</p>
<h2>Bi-directional One-to-Many</h2>
<p>For this, we have references from <em>both</em> sides.</p>
<pre class="brush: java">
@Entity
public class Cart { ...
@OneToMany(mappedBy="cart", cascade=ALL, orphanRemoval=true)
Collection<OrderItem> items;
}
@Entity
public class CartItem { ...
@ManyToOne Cart cart;
}
</pre>
<p>To test this, we insert one row to the table for <code>Cart</code>, and three or more rows to the table for <code>CartItem</code>. Then, we run the test.</p>
<pre class="brush: java; highlight: [8]">
public class CartTests { ...
@Test
public void testOneShotDelete() throws Exception {
Cart cart = entityManager.find(Cart.class, 53L);
for (CartItem item : cart.items) {
item.cart = null; // remove reference to cart
}
cart.items.clear(); // as indicated in Hibernate manual
entityManager.flush(); // just so SQL commands can be seen
}
}
</pre>
<p>The SQL commands shown had each item deleted individually (and not as a one-shot delete).</p>
<pre>
delete from CartItem where id=?
delete from CartItem where id=?
delete from CartItem where id=?
</pre>
<p>Discarding the original collection did not work either. It even caused an exception.</p>
<pre class="brush: java; highlight: [6]">
public class CartTests { ...
@Test
public void testOneShotDelete() throws Exception {
Cart cart = entityManager.find(Cart.class, 53L);
// remove reference to cart
cart.items = new LinkedList<CartItem>(); // discard, and use new collection
entityManager.flush(); // just so SQL commands can be seen
}
}
</pre>
<pre>
javax.persistence.PersistenceException:
org.hibernate.HibernateException:
A collection with cascade="all-delete-orphan" was no longer referenced by the owning entity instance: ….Cart.items
</pre>
<p>I tested this with Hibernate 4.3.11 and HSQL 2.3.2. If your results vary, please hit the <a href="#comments">comments</a>.</p>
<h2>Uni-directional One-to-Many (With Join Table)</h2>
<p>For this, we make changes to the mapping. This causes a join table to be created.</p>
<pre class="brush: java">
@Entity
public class Cart { ...
@OneToMany(cascade=ALL)
Collection<OrderItem> items;
}
@Entity
public class CartItem { ...
// no @ManyToOne Cart cart;
}
</pre>
<p>Again, we insert one row to the table for <code>Cart</code>, and three or more rows to the table for <code>CartItem</code>. We also have to insert appropriate records to the join table (<code>Cart_CartItem</code>). Then, we run the test.</p>
<pre class="brush: java">
public class CartTests { ...
@Test
public void testOneShotDelete() throws Exception {
Cart cart = entityManager.find(Cart.class, 53L);
cart.items.clear(); // as indicated in Hibernate manual
entityManager.flush(); // just so SQL commands can be seen
}
}
</pre>
<p>The SQL commands shown had the associated rows in the join table deleted (with one command). But the rows in the table for <code>CartItem</code> still exist (and did not get deleted).</p>
<pre>
delete from Cart_CartItem where cart_id=?
// no delete commands for CartItem
</pre>
<p>Hmmm, not exactly what we want, since the rows in the table for <code>CartItem</code> still exist.</p>
<h2>Uni-directional One-to-Many (No Join Table)</h2>
<p>Starting with JPA 2.0, the join table can be avoided in a uni-directional one-to-many by specifying a <code>@JoinColumn</code>.</p>
<pre class="brush: java; highlight: [4]">
@Entity
public class Cart { ...
@OneToMany(cascade=CascadeType.ALL, orphanRemoval=true)
@JoinColumn(name="cart_id", updatable=false, nullable=false)
Collection<OrderItem> items;
}
@Entity
public class CartItem { ...
// no @ManyToOne Cart cart;
}
</pre>
<p>Again, we insert one row to the table for <code>Cart</code>, and three or more rows to the table for <code>CartItem</code>. Then, we run the test.</p>
<pre class="brush: java">
public class CartTests { ...
@Test
public void testOneShotDelete() throws Exception {
Cart cart = entityManager.find(Cart.class, 53L);
cart.items.clear(); // as indicated in Hibernate manual
entityManager.flush(); // just so SQL commands can be seen
}
}
</pre>
<p>Discarding the original collection also did not work either. It also caused the same exception (as with bi-directional one-to-many).</p>
<pre>
javax.persistence.PersistenceException:
org.hibernate.HibernateException:
A collection with cascade="all-delete-orphan" was no longer referenced by the owning entity instance: ….Cart.items
</pre>
<h2>Uni-directional One-to-Many (with <code>ElementCollection</code>)</h2>
<p>JPA 2.0 introduced <code>@ElementCollection</code>. This allows one-to-many relationships to be established with the many-side being either <code>@Basic</code> or <code>@Embeddable</code> (i.e. not an <code>@Entity</code>).</p>
<pre class="brush: java; highlight: [3, 8]">
@Entity
public class Cart { ...
@ElementCollection // @OneToMany for basic and embeddables
@CollectionTable(name="CartItem") // defaults to "Cart_items" if not overridden
Collection<OrderItem> items;
}
@Embeddable // not an entity!
public class CartItem {
// no @Id
// no @ManyToOne Cart cart;
private String data; // just so that there are columns we can set
}
</pre>
<p>Again, we insert one row to the table for <code>Cart</code>, and three or more rows to the table for <code>CartItem</code>. Then, we run the test.</p>
<pre class="brush: java">
public class CartTests { ...
@Test
public void testOneShotDelete() throws Exception {
Cart cart = entityManager.find(Cart.class, 53L);
cart.items.clear(); // as indicated in Hibernate manual
entityManager.flush(); // just so SQL commands can be seen
}
}
</pre>
<p>Yey! The associated rows for <code>CartItem</code> were deleted in one shot.</p>
<pre>
delete from CartItem where Cart_id=?
</pre>
<h2>Closing Thoughts</h2>
<p>One-shot delete occurs with uni-directional one-to-many using <code>ElementCollection</code> (where the many-side is an embeddabled, and <em>not</em> an entity).</p>
<p>In the uni-directional one-to-many with join table scenario, deleting entries in a join table doesn't add much value.</p>
<p>I'm not sure why one-shot delete works (or why it works this way) in Hibernate. But I do have a guess. And that is the underlying JPA provider could not do a one-shot delete because it could not ensure that the many-side entity is not referenced by other entities. Unlike the <code>ElementCollection</code>, the many-side is not an entity and cannot be referenced by other entities.</p>
<p>Now, this <em>does not</em> mean that you have to use <code>ElementCollection</code> all the time. Perhaps the one-shot delete only applies to aggregate roots. In those cases, using <code>Embeddable</code> and <code>ElementCollection</code> might be appropriate for a collection of value objects that make up an aggregate. When the aggregate root is removed, then it would be good to see that the "child" objects should be removed as well (and in an efficient manner).</p>
<p>I wish there was a way in JPA to indicate that the child entities are privately owned and can be safely removed when the parent entity is removed (e.g. similar to <code>@PrivateOwned</code> in EclipseLink). Let's see if it will be included in a future version of the API.</p>
<p>Hope this helps.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-68107015856211375142016-07-20T14:00:00.000-07:002016-07-20T20:45:47.035-07:00Reference by Identity in JPA<p>
In a <a href="/2013/05/domain-driven-design-referencing.html">previous post</a>, I mentioned that I opted to reference other aggregates by their primary key, and not by type. I usually use this approach (a.k.a. disconnected domain model) when working with large or complex domain models. In this post, let me <em>try</em> to explain further how it can be done in <abbr title="Java Persistence API">JPA</abbr>. Note that the resulting <abbr title="Data Definition Language">DDL</abbr> scripts will not create a foreign key constraint (unlike the one shown in the <a href="/2013/05/domain-driven-design-referencing.html">previous post</a>).
</p>
<h2>Reference by Identity</h2>
<p>In most JPA examples, every entity references another entity, or is being referenced by another entity. This results into an object model that allows traversal from one entity to any other entity. This can cause <em>unwanted traversals</em> (and unwanted cascade of persistence operations). As such, it would be good to prevent this, by referencing other entities by ID (and not by type).</p>
<p>The code below shows how <code>OrderItem</code> references a <code>Product</code> entity by its primary key (and not by type).</p>
<pre class="brush: java; highlight: [19, 20, 21]">
@Entity
public class Product {
@Id private Long id;
// ...
}
@Entity
public class Order {
// ...
@OneToMany(mappedBy="order")
private Collection<OrderItem> items;
}
@Entity
public class OrderItem {
// ...
@ManyToOne
private Order order;
// @ManyToOne
// private Product product;
private Long productId;
// ...
}
</pre>
<p>There are several ways to get the associated <code>Product</code> entities. One way is to use a repository to find products given the IDs (<code>ProductRepository</code> with a <code style="white-space: nowrap">findByIdIn(List<Long> ids)</code> method). As mentioned in previous comments, please be careful not to end up with the <em>N+1 selects problem</em>.</p>
<p>Custom identity types can also be used. The example below uses <code>ProductId</code>. It is a value object. And because of JPA, we needed to add a zero-arguments constructor.</p>
<pre class="brush: java; highlight: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 31]">
@Embeddable
public class ProductId {
private Long id;
public ProductId(long id) {
this.id = id;
}
public long getValue() { return id; }
// equals and hashCode
protected ProductId() { /* as required by JPA */ }
}
@Entity
public class Product {
@EmbeddedId private ProductId id;
// ...
}
@Entity
public class Order { // ...
@OneToMany(mappedBy="order")
private Collection<OrderItem> items;
}
@Entity
public class OrderItem {
// ...
@ManyToOne
private Order order;
// @ManyToOne
// private Product product;
@Embedded private ProductId productId;
// ...
}
</pre>
<p>But this will not work if you're using generated values for IDs. Fortunately, starting with JPA 2.0, there are some tricks around this, which I'll share in the next section.</p>
<h2>Generated IDs</h2>
<p>In JPA, when using non-<code>@Basic</code> types as <code>@Id</code>, we can no longer use <code>@GeneratedValue</code>. But using a mix of property and field access, we can still use generated value and <code>ProductId</code>.</p>
<pre class="brush: java; highlight: [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]">
@Embeddable
@Access(AccessType.FIELD)
public class ProductId {...}
@Entity
@Access(AccessType.FIELD)
public class Product {
@Transient private ProductId id;
public ProductId getId() { return id; }
// ...
private Long id_;
@Id
@GeneratedValue(strategy=...)
@Access(AccessType.PROPERTY)
protected Long getId_() { return id_; }
protected void setId_(Long id_) {
this.id_ = id_;
this.id = new ProductId(this.id_);
}
}
@Entity
public class Order { // ...
@OneToMany(mappedBy="order")
private Collection<OrderItem> items;
}
@Entity
public class OrderItem {
// ...
@ManyToOne
private Order order;
// @ManyToOne
// private Product product;
@Embedded private ProductId productId;
// ...
}
</pre>
<p>The trick involves using property access for the generated ID value (while keeping the rest with field access). This causes JPA to use the setter method. And in it, we initialize the <code>ProductId</code> field. Note that the <code>ProductId</code> field is not persisted (marked as <code>@Transient</code>).</p>
<p>Hope this helps.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-88071853655126460382016-01-11T10:00:00.000-08:002016-01-13T18:55:08.885-08:00JPA Pitfalls / Mistakes<p>From my experience, both in helping teams and conducting training, here are some pitfalls/mistakes I have encountered that caused some problems in Java-based systems that use <abbr title="Java Persistence API">JPA</abbr>.</p>
<ul>
<li>Requiring a public no-arg constructor</li>
<li><em>Always</em> using bi-directional associations/relationships</li>
<li>Using <code>@OneToMany</code> for collections that can become huge</li>
</ul>
<h2>Requiring a Public No-arg Constructor</h2>
<p>Yes, a JPA <code>@Entity</code> requires a zero-arguments (or default no-args) constructor. <strong>But this can be made <strong><code>protected</code></strong>. You do not have to make it <code>public</code>.</strong> This allows better object-oriented modeling, since you are <em>not</em> forced to have a publicly accessible zero-arguments constructor.</p>
<blockquote>
<p>The entity class must have a no-arg constructor. The entity class may have other constructors as well. The no-arg constructor must be public <strong>or protected</strong>. [emphasis mine]</p>
<footer>
- from Section 2.1 of the <cite><a href="http://download.oracle.com/otn-pub/jcp/persistence-2_1-fr-eval-spec/JavaPersistence.pdf">Java Persistence API 2.1 Specification (Oracle)</a></cite>
</footer>
</blockquote>
<p>If the entity being modeled has some fields that need to be initialized when it is created, this should be done through its constructor.</p>
<aside style="clear: right; float: right; margin: 1em 0 1em 1em; width: 30%; max-width: 10em; padding: 0.5em; border: 1px solid #ccc; border-radius: 4px">
<p>NOTE: Some JPA providers may overcome a missing no-arg constructor by adding one at build time.</p>
</aside>
<p>Let's say we're modeling a hotel room reservation system. In it, we probably have entities like room, reservation, etc. The reservation entity will likely require start and end dates, since it would not make much sense to create one without the period of stay. Having the start and end dates included as arguments in the reservation's constructor would allow for a better model. Keeping a <em>protected</em> zero-arguments constructor would make JPA happy.</p>
<div class="clear"></div>
<pre class="brush: java; highlight: [9]">
@Entity
public class Reservation { ...
public Reservation(
RoomType roomType, DateRange startAndEndDates) {
if (roomType == null || startAndEndDates == null) {
throw new IllegalArgumentException(...);
} ...
}
...
protected Reservation() { /* as required by ORM/JPA */ }
}
</pre>
<aside style="clear: right; float: right; margin: 1em 0 1em 1em; width: 30%; max-width: 10em; padding: 0.5em; border: 1px solid #ccc; border-radius: 4px">
<p>NOTE: Hibernate (a JPA provider) allows the zero-arguments constructor to be made private. This makes your JPA code non-portable to other JPA providers.</p>
</aside>
<p>It also helps to add a comment in the zero-arguments constructor to indicate that it was added for JPA-purposes (technical infrastructure), and that it is not required by the domain (business rules/logic).</p>
<p>Although I could not find it mentioned in the JPA 2.1 spec, embeddable classes also require a default (no-args) constructor. And just like entities, the <em>required no-args constructor can be made <code>protected</code></em>.</p>
<div class="clear"></div>
<pre class="brush: java; highlight: [12]">
@Embeddable
public class DateRange { ...
public DateRange(Date start, Date end) {
if (start == null || end == null) {
throw new IllegalArgumentException(...);
}
if (start.after(end)) {
throw new IllegalArgumentException(...);
} ...
}
...
protected DateRange() { /* as required by ORM/JPA */ }
}
</pre>
<p>The <a href="https://github.com/citerus/dddsample-core">DDD sample project</a> also hides the no-arg constructor by making it package scope (see <a href="https://github.com/citerus/dddsample-core/blob/master/src/main/java/se/citerus/dddsample/domain/model/cargo/Cargo.java">Cargo</a> entity class where no-arg constructor is near the bottom).</p>
<h2>Always Using Bi-directional Associations/Relationships</h2>
<p>Instructional material on JPA <em>often</em> show a bi-directional association. But this is not required. For example, let's say we have an order entity with one or more items.</p>
<pre class="brush: java; highlight: [4, 11]">
@Entity
public class Order {
@Id private Long id;
@OneToMany private List<OrderItem> items;
...
}
@Entity
public class OrderItem {
@Id private Long id;
@ManyToOne private Order order;
...
}
</pre>
<p>It's good to know that bi-directional associations are supported in JPA. But in practice, it becomes a maintenance nightmare. If order items do not have to know its parent order object, a uni-directional association would suffice (as shown below). The ORM just needs to know how to name the foreign key column in the many-side table. This is provided by adding the <code>@JoinColumn</code> annotation on the one-side of the association.</p>
<pre class="brush: java; highlight: [4, 5, 13]">
@Entity
public class Order {
@Id Long id;
@OneToMany
@JoinColumn(name="order_id", ...)
private List<OrderItem> items;
...
}
@Entity
public class OrderItem {
@Id private Long id;
// @ManyToOne private Order order;
...
}
</pre>
<p>Making it uni-directional makes it easier since the <code>OrderItem</code> no longer needs to keep a reference to the <code>Order</code> entity.</p>
<p>Note that there may be times when a bi-directional association is needed. In practice, this is quite rare.</p>
<p>Here's another example. Let's say you have several entities that refer to a country entity (e.g. person's place of birth, postal address, etc.). Obviously, these entities would reference the country entity. But would country have to reference all those different entities? Most likely, not.</p>
<pre class="brush: java; highlight: [4, 11, 18, 19]">
@Entity
public class Person {
@Id Long id;
@ManyToOne private Country countryOfBirth;
...
}
@Entity
public class PostalAddress {
@Id private Long id;
@ManyToOne private Country country;
...
}
@Entity
public class Country {
@Id ...;
// @OneToMany private List<Person> persons;
// @OneToMany private List<PostalAddress> addresses;
}
</pre>
<p><strong>So, just because JPA supports bi-directional association does <em>not</em> mean you have to!</strong></p>
<h2>Using <code>@OneToMany</code> For Collections That Can Become Huge</h2>
<p>Let's say you're modeling bank accounts and its transactions. Over time, an account can have thousands (if not millions) of transactions.</p>
<pre class="brush: java">
@Entity
public class Account {
@Id Long id;
@OneToMany
@JoinColumn(name="account_id", ...)
private List<AccountTransaction> transactions;
...
}
@Entity
public class AccountTransaction {
@Id Long id;
...
}
</pre>
<img style="display: block; float: right; margin: 1em 0 1em 1em" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5J-1J6Ilxs0VisPkg2hH-ufZ-x0kGS8YNYL9Vs7kj1fI5ymrFz5NblJ64DI5vdUM-1-1AZEFRCh_V0P9U8T05g9yjJ3EPYZsIUb-pd7ezSV3Bd44zXMb7TOlL5xkEdwYGQQgQzQEFy1I/s320/confused-woman1.jpg" />
<p>With accounts that have only a few transactions, there doesn't seem to be any problem. But over time, when an account contains thousands (if not millions) of transactions, you'll most likely experience out-of-memory errors. So, what's a better way to map this?</p>
<p><strong>If you cannot ensure the maximum number of elements in the many-side of the association can all be loaded in memory, better use the <code>@ManyToOne</code> on the opposite side of the association.</strong></p>
<div class="clear"></div>
<pre class="brush: java; highlight: [4, 11, 12]">
@Entity
public class Account {
@Id Long id;
// @OneToMany private List<AccountTransaction> transactions;
...
}
@Entity
public class AccountTransaction {
@Id Long id;
@ManyToOne
private Account account;
...
public AccountTransaction(Account account, ...) {...}
protected AccountTransaction() { /* as required by ORM/JPA */ }
}
</pre>
<p>To retrieve the possibly thousands (if not millions) of transactions of an account, use a repository that supports pagination.</p>
<pre class="brush: java">
@Transactional
public interface AccountTransactionRepository {
Page<AccountTransaction> findByAccount(
Long accountId, int offset, int pageSize);
...
}
</pre>
<p>To support pagination, use the <code>Query</code> object's <code>setFirstResult(int)</code> and <code>setMaxResults(int)</code> methods.</p>
<h2>Summary</h2>
<p>I hope these notes can help developers avoid making these mistakes. To summarize:</p>
<ul>
<li><del style="color: #888">Requiring a public</del> The JPA-required no-arg constructor can be made <code>public</code> or <code><strong>protected</strong></code>. Consider making it <code>protected</code> if needed.</li>
<li><del style="color: #888">Always using</del> Consider uni-directional <em>over</em> bi-directional associations/relationships.</li>
<li><del style="color: #888">Using</del> Avoid <code>@OneToMany</code> for collections that can become huge. Consider mapping the <code>@ManyToOne</code>-side of the association/relationship instead, and support pagination.</li>
</ul>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com3tag:blogger.com,1999:blog-2526628884837213668.post-27406383199103092842015-12-07T11:00:00.000-08:002015-12-08T16:54:02.137-08:00JPA EntityManager Merge vs Persist<p>Here's a quick short post. Putting it here, since it has been tremendously useful for me. I hope this will help others as well.</p>
<p>There have been several questions on <a href="http://stackoverflow.com/search?q=jpa+entitymanager+merge+persist"><abbr title="StackOverflow">SO</abbr> about <code>merge()</code> and <code>persist()</code> operations</a>. But I find that it is <em>not just about</em> one resulting to an <code>UPDATE</code> SQL command, and the other resulting to an <code>INSERT</code> SQL command.</p>
<h2>Results of <code>merge()</code> and <code>persist()</code></h2>
<img border="0" style="float: left; margin: 1em 1em 1em 0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcYlk8v2GlYuSTdoh1fwTwjOWMB1M84WqGw3aHYVs-RYPtf3NrOidqPjS5DhohDICkgDSM4OKWtH-5wTKhSL6qP2TKbphoFHkFSdslY9ZkvgFYtCASioz3JziSBAYL9aklvBL6B1MGyUE/s320/thinking-serious-young-woman-in-glasses-looking-vintage-closeup.jpg" />
<p>Instead of pasting relevant sections from the JPA 2.1 specification in this post, I've created a summary table of the results of the said operations. This is based on sections 3.2.2 (Persisting an Entity Instance) and 3.2.7.1 (Merging Detached Entity State) of the <a href="http://download.oracle.com/otndocs/jcp/persistence-2_1-fr-eval-spec/index.html"><abbr title="Java Persistence API">JPA</abbr> 2.1 specification</a>.</p>
<table style="border-collapse: collapse">
<thead>
<tr style="background-color: #eef">
<th style="padding: 0.5em 1em">Operation</th>
<th style="padding: 0.5em 1em">State</th>
<th style="padding: 0.5em 1em">Result</th>
</tr>
</thead>
<tbody>
<tr style="background-color: #fff; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">persist</td>
<td style="padding: 0.5em 1em">new</td>
<td style="padding: 0.5em 1em">becomes managed</td>
</tr>
<tr style="background-color: #fffae5; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">persist</td>
<td style="padding: 0.5em 1em">managed</td>
<td style="padding: 0.5em 1em">ignored (but cascaded)</td>
</tr>
<tr style="background-color: #fff; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">persist</td>
<td style="padding: 0.5em 1em">removed</td>
<td style="padding: 0.5em 1em">becomes managed</td>
</tr>
<tr style="background-color: #fffae5; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">persist</td>
<td style="padding: 0.5em 1em">detached</td>
<td style="padding: 0.5em 1em">throws exception or commit fails</td>
</tr>
</tbody>
</table>
<table style="border-collapse: collapse">
<thead>
<tr style="background-color: #eef">
<th style="padding: 0.5em 1em">Operation</th>
<th style="padding: 0.5em 1em">State</th>
<th style="padding: 0.5em 1em">Result</th>
</tr>
</thead>
<tbody>
<tr style="background-color: #fff; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">merge</td>
<td style="padding: 0.5em 1em">new</td>
<td style="padding: 0.5em 1em">becomes managed</td>
</tr>
<tr style="background-color: #fffae5; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">merge</td>
<td style="padding: 0.5em 1em">managed</td>
<td style="padding: 0.5em 1em">ignored (but cascaded)</td>
</tr>
<tr style="background-color: #fff; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">merge</td>
<td style="padding: 0.5em 1em">removed</td>
<td style="padding: 0.5em 1em">throws exception or commit fails</td>
</tr>
<tr style="background-color: #fffae5; border-bottom: 1px solid #ccc">
<td style="padding: 0.5em 1em">merge</td>
<td style="padding: 0.5em 1em">detached</td>
<td style="padding: 0.5em 1em">becomes managed</td>
</tr>
</tbody>
</table>
<p>As you can see, both <code>merge()</code> and <code>persist()</code> operations treat new and managed entities the same way. But they only differ in the way they treat removed and detached entities.</p>
<h3>Closing Thoughts</h3>
<p>So, the next time you think that <code>persist()</code> results to an <code>INSERT</code>, and <code>merge()</code> results to an <code>UPDATE</code>, think again!</p>
<p>Here's my take. I personally prefer to use <code>merge()</code> operation to handle <em>new</em>, <em>managed</em>, and <em>detached</em> entities.</p>
<p>But for <em>removed</em> entities (which can only happen when you programmatically remove it), I use <code>persist()</code>. But then again, it's rare (in my experience) to remove an entity, and then reverse its removal in the <em>same</em> persistence context.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-19435296316489074092015-12-04T10:30:00.000-08:002015-12-04T10:30:01.561-08:00Spring-managed Transactions Explained - Part 2 (JPA)<p>In the <a href="//lorenzo-dee.blogspot.com/2015/12/spring-managed-transactions-part1.html">first part of the series</a>, I showed how transactions work in plain-vanilla <abbr title="Java Database Connectivity">JDBC</abbr>. And then I showed how Spring manages JDBC-based transactions. In this second part of the series, I'll show how transactions work in plain-vanilla <abbr title="Java Persistence API">JPA</abbr> first. And then show how Spring manages JPA-based transactions.</p>
<h2>Funds Transfer</h2>
<p>To help illustrate transactions, I'll be using the same case study of transferring funds from one bank account to another. Here, we show code snippets of debit, credit, and transfer methods.</p>
<pre class="brush: java">
... class BankAccountService {
public void transfer(MonetaryAmount amount, ...) {
debit(amount, ...);
credit(amount, ...);
...
}
public void credit(MonetaryAmount amount, AccountId accountId) {
...
}
public void debit(MonetaryAmount amount, AccountId accountId) {
...
}
...
}
</pre>
<h2>JPA Transactions</h2>
<p>In plain-vanilla JPA, transactions are started by calling <code>getTransaction().begin()</code> on the <code>EntityManager</code>. The code snippet below illustrates this.</p>
<pre class="brush: java; highlight: [6, 8]">
import javax.persistence.*;
...
EntityManagerFactory emf = ...;
EntityManager em = emf.createEntityManager();
try {
em.getTransaction().begin();
// make changes through entities
em.getTransaction().commit();
...
} catch(Exception e) {
em.getTransaction().rollback();
throw e;
} finally {
em.close();
}
</pre>
<p>Technically, the <code>EntityManager</code> is in a transaction from the point it is created. So calling <code>begin()</code> is somewhat redundant. Until <code>begin()</code> is called, certain operations such as <code>persist</code>, <code>merge</code>, <code>remove</code> cannot be called. Queries can still be performed (e.g. <code>find()</code>).</p>
<p>Objects that were returned from queries can be changed. Although the JPA specification is <em>somewhat unclear</em> on what will happen to these changes when no transaction has been started.</p>
<p>Now, let's apply JPA to the funds transfer case study.</p>
<p>We defined a <code>BankAccount</code> entity to handle <code>debit()</code> and <code>credit()</code> behavior.</p>
<pre class="brush: java">
import javax.persistence.*;
@Entity
... class BankAccount {
@Id ...;
...
public void debit(MonetaryAmount amount) {...}
public void credit(MonetaryAmount amount) {...}
...
}
</pre>
<p>We add an <code>EntityManagerFactory</code> to <code>BankAccountService</code> to enable the creation of <code>EntityManager</code>s when needed.</p>
<pre class="brush: java; highlight: [4, 7, 9, 14, 24, 26, 29, 39, 41, 44]">
import javax.persistence.*;
... class BankAccountService {
private EntityManagerFactory emf; // injected via constructor
...
public void transfer(MonetaryAmount amount, ...) ... {
EntityManager em = emf.createEntityManager();
try {
em.getTransaction().begin();
BankAccount fromAccount = em.find(BankAccount.class, ...);
BankAccount toAccount = em.find(BankAccount.class, ...);
fromAccount.debit(amount);
toAccount.credit(amount);
em.getTransaction().commit();
...
} catch(Exception e) {
em.getTransaction().rollback();
// handle exception (possibly rethrowing it)
} finally {
em.close();
}
}
public void credit(MonetaryAmount amount, AccountId ...) ... {
EntityManager em = emf.createEntityManager();
try {
em.getTransaction().begin();
BankAccount theAccount = em.find(BankAccount.class, ...);
theAccount.credit(amount);
em.getTransaction().commit();
...
} catch(Exception e) {
em.getTransaction().rollback();
// handle exception (possibly rethrowing it)
} finally {
em.close();</code>
}
}
public void debit(MonetaryAmount amount, AccountId ...) ... {
EntityManager em = emf.createEntityManager();
try {
em.getTransaction().begin();
BankAccount theAccount = em.find(BankAccount.class, ...);
theAccount.debit(amount);
em.getTransaction().commit();
...
} catch(Exception e) {
em.getTransaction().rollback();
// handle exception (possibly rethrowing it)
} finally {
em.close();
}
}
}
</pre>
<h2>Spring-managed JPA Transactions</h2>
<p>The <code>transfer</code>, <code>credit</code>, and <code>debit</code> methods could sure use a template class (something like a <code>JdbcTemplate</code>) to remove all the boilerplate code. Spring previously provided a <code>JpaTemplate</code> class, but was deprecated as of Spring 3.1, in favor of native <code>EntityManager</code> usage (typically obtained through <code>@PersistenceContext</code>).</p>
<p>So, let's do just that — use <code>EntityManager</code> obtained through <code>@PersistenceContext</code>.</p>
<pre class="brush: java; highlight: [4, 5]">
import javax.persistence.*;
... class BankAccountService {
@PersistenceContext
private EntityManager em;
...
public void transfer(MonetaryAmount amount, ...) ... {
try {
em.getTransaction().begin();
BankAccount fromAccount = em.find(BankAccount.class, ...);
BankAccount toAccount = em.find(BankAccount.class, ...);
fromAccount.debit(amount);
toAccount.credit(amount);
em.getTransaction().commit();
...
} catch(Exception e) {
em.getTransaction().rollback();
// handle exception (possibly rethrowing it)
}
}
public void credit(MonetaryAmount amount, AccountId ...) ... {
try {
em.getTransaction().begin();
BankAccount theAccount = em.find(BankAccount.class, ...);
theAccount.credit(amount);
em.getTransaction().commit();
...
} catch(Exception e) {
em.getTransaction().rollback();
// handle exception (possibly rethrowing it)
}
}
public void debit(MonetaryAmount amount, AccountId ...) ... {
try {
em.getTransaction().begin();
BankAccount theAccount = em.find(BankAccount.class, ...);
theAccount.debit(amount);
em.getTransaction().commit();
...
} catch(Exception e) {
em.getTransaction().rollback();
// handle exception (possibly rethrowing it)
}
}
}
</pre>
<p>Our code is a little bit simpler. Since we didn't create an <code>EntityManager</code>, we don't have to close it. But we are still calling <code>getTransaction().begin()</code>. Is there a better way? And how does an <code>EntityManager</code> get injected into the object in the first place?</p>
<p>From my <a href="//lorenzo-dee.blogspot.com/2015/12/spring-managed-transactions-part1.html">previous post in this series</a>, the astute reader is probably already thinking of having Spring do the work for us. And rightfully so!</p>
<h3><code>EntityManager</code> and <code>@PersistenceContext</code></h3>
<p>We tell Spring to inject an <code>EntityManager</code> from the <code>EntityManagerFactory</code> by adding a <a href="//docs.spring.io/spring/docs/current/javadoc-api/org/springframework/orm/jpa/support/PersistenceAnnotationBeanPostProcessor.html"><code>PersistenceAnnotationBeanPostProcessor</code></a> (either through XML <code><bean></code>, or simply using a Java-based configuration via <code>@Configuration</code> classes loaded via <code>AnnotationConfigApplicationContext</code>).</p>
<ul>
<li>When using XML-based configuration, a <code>PersistenceAnnotationBeanPostProcessor</code> is transparently activated by the <code><context:annotation-config /></code> element. And this element also gets transparently activated by <code><context:component-scan /></code>.</li>
<li>When using Java-based <code>@Configuration</code>, the <code>AnnotationConfigApplicationContext</code> is used. And with it, annotation config processors are always registered (one of which is the aforementioned <code>PersistenceAnnotationBeanPostProcessor</code>).</li>
</ul>
<p>By adding a single bean definition, the Spring container will act as a JPA container and inject an <code>EnitityManager</code> from your <code>EntityManagerFactory</code>.</p>
<h3>JPA and <code>@Transactional</code></h3>
<p>Now that we have an <code>EntityManager</code>, how can we tell Spring to begin transactions for us?</p>
<p>We tell Spring to start transactions by marking methods as <code>@Transactional</code> (or mark the class as <code>@Transactional</code> which makes all public methods transactional). This is consistent with the way Spring enables transactions with JDBC.
<pre class="brush: java; highlight: [4]">
import javax.persistence.*;
import org.springframework.transaction.annotation.Transactional;
@Transactional
... class BankAccountService {
@PersistenceContext
private EntityManager em;
...
public void transfer(MonetaryAmount amount, ...) ... {
BankAccount fromAccount = em.find(BankAccount.class, ...);
BankAccount toAccount = em.find(BankAccount.class, ...);
fromAccount.debit(amount);
toAccount.credit(amount);
}
public void credit(MonetaryAmount amount, AccountId ...) ... {
BankAccount theAccount = em.find(BankAccount.class, ...);
theAccount.credit(amount);
}
public void debit(MonetaryAmount amount, AccountId ...) ... {
BankAccount theAccount = em.find(BankAccount.class, ...);
theAccount.debit(amount);
}
}
</pre>
<p>Wow, that was nice! Our code just got a lot shorter.</p>
<p>And just as explained in the <a href="//lorenzo-dee.blogspot.com/2015/12/spring-managed-transactions-part1.html">first part of this series</a>, when Spring encounters this annotation, it proxies the object (usually referred to as a Spring-managed bean). The proxy starts a transaction (if there is no on-going transaction) for methods that are marked as <code>@Transactional</code>, and ends the transaction when the method returns successfully.</p>
<img src="//yuml.me/1f997647" alt="Proxy adds transaction behavior" style="display: block; margin: 0 auto 1em auto;">
<p>A call to <code>debit()</code> will use a transaction. A separate call to <code>credit()</code> will use a transaction. But what happens when a call to <code>transfer()</code> is made?</p>
<p>Since the <code>transfer()</code> method is marked as <code>@Transactional</code>, Spring will start a transaction. This same transaction will be used for calls to <code>debit()</code> and <code>credit()</code>. In other words, <code>debit(amount)</code> and <code>credit(amount)</code> will not start a new transaction. It will use the on-going transaction (since there is one).</p>
<p>But wait! How does Spring know when to inject a proper entity manager? Is it only injected when a transactional method is invoked?</p>
<h3>Shared <code>EntityManager</code></h3>
<p>In one of my <a href="//www.orangeandbronze.com/university">training classes</a>, I tried the following to better understand how Spring injects an <code>EntityManager</code> via <code>@PersistenceContext</code>. And I believe it will help others too. So, here's what I tried:</p>
<pre class="brush: java; highlight: [10, 11, 12, 13]">
import javax.persistence.*;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.beans.factory.InitializingBean;
@Transactional
... class BankAccountService implements InitializingBean {
@PersistenceContext
private EntityManager em;
...
@Override
public void afterPropertiesSet() {
System.out.println(em.toString());
}
...
}
</pre>
<p>An output of something like this was displayed on the console after the application context started.</p>
<pre style="border: 1px solid #ccc; padding: 1em 2em; background-color: #111; color: #1c1">
Shared EntityManager proxy for target factory [...]
</pre>
<p>So what is this <em>shared</em> entity manager?</p>
<p>When the application context starts, Spring injects a <em>shared</em> entity manager. The shared <code>EntityManager</code> will behave just like an <code>EntityManager</code> fetched from an application server's JNDI environment, as defined by the JPA specification. It will delegate all calls to the current transactional <code>EntityManager</code>, if any; otherwise, it will fall back to a newly created <code>EntityManager</code> per operation.</p>
<p>Going back to our question. Spring doesn't inject the right entity manager at the right time. It always injects a shared entity manager. But this shared entity manager is transaction-aware. It delegates to the current transactional <code>EntityManager</code>, if there is an on-going transaction.</p>
<h2>Conclusion</h2>
<img border="0" style="float: right; margin: 1em 0 1em 1em" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEir43c31oDf8LHkAticu7-f37c3luGFqR-inhW-jZ238LAaCQwTyDbK-8gBK1pRaDxfwhug7hDCR3Jz1cn4689GPERXi9armdhKTnWOEyRX9gQIbxETvzLIZ9q3CipN-DDT2cXCa4XmT9c/s1600/butterscotch-beer.png" />
<p>This concludes the two-part series. I hope that by starting off with the plain-vanilla versions of JDBC and JPA (sans DAOs and repositories), I was able to make it clearer as to how Spring is able to manage transactions behind the scenes. And that by having a clearer idea as to what Spring is doing behind the scenes, you can troubleshoot better, understand why you get an <code>TransactionRequiredException</code> saying <span style="font-family: monospace">"No transactional EntityManager available"</span>, and add better fixes to your applications.</p>
<p>Now, it's time for a cold one.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-52291997005541181822015-12-03T10:30:00.000-08:002015-12-06T16:55:39.645-08:00Spring-managed Transactions Explained - Part 1 (JDBC)<p>I've been meaning to write about this for quite some time now. In the <a href="//www.orangeandbronze.com/university">training courses</a> I've been privileged to conduct, I've noticed that course participants have had the most difficulty trying to understand how Spring manages transactions. In the first part of this series, I'll start by showing how transactions work in plain-vanilla <abbr title="Java Database Connectivity">JDBC</abbr>. And then show how Spring manages JDBC-based transactions.</p>
<p>In the second part of this series, I'll show how transactions work in plain-vanilla <abbr title="Java Persistence API">JPA</abbr>. And then show how Spring manages JPA-based transactions.</p>
<aside style="float: right; margin: 1em 0 1em 1em; padding: 1em; border: 1px solid #ccc; border-radius: 4px; width: 30%; min-width: 8em; max-width: 15em">
Update: The <a href="//lorenzo-dee.blogspot.com/2015/12/spring-managed-transactions-part2.html">second part of this series</a> has just been posted.
</aside>
<h2>Funds Transfer</h2>
<p>To help illustrate transactions, I'll be using the often used case study of transferring funds from one bank account to another. Here, we show code snippets of debit, credit, and transfer methods.</p>
<pre class="brush: java">
... class BankAccountService {
public void transfer(MonetaryAmount amount, ...) {
debit(amount, ...);
credit(amount, ...);
...
}
public void credit(MonetaryAmount amount, AccountId accountId) {
...
}
public void debit(MonetaryAmount amount, AccountId accountId) {
...
}
...
}
</pre>
<aside style="border: 1px solid #ccc; border-radius: 4px; padding: 1em; float: right; margin: 1em 0 1em 1em; width: 30%; min-width: 8em; max-width: 15em">
By default a <code>Connection</code> object is in auto-commit mode, which means that it automatically commits changes after executing each statement. If auto-commit mode has been disabled, the method <code>commit()</code> must be called explicitly in order to commit changes; otherwise, database changes will not be saved.
</aside>
<h2>JDBC Transactions</h2>
<p>In plain-vanilla JDBC, transactions are started by setting the <code>Connection</code>'s auto-commit mode to <code>false</code> (or to <em>manual</em> commit mode). Once the <a href="//docs.oracle.com/javase/8/docs/api/java/sql/Connection.html"><code>Connection</code></a>'s auto-commit is set to <code>false</code>, subsequent database changes are not committed until a call to <code>commit()</code> is made. The changes brought about by the SQL operations will not be seen by other connections until the changes are committed. The code snippet below illustrates this.</p>
<div style="clear: both">
<pre class="brush: java; highlight: [4]">
import java.sql.*;
...
try (Connection conn = dataSource.getConnection()) {
conn.setAutoCommit(false);
... // SQL operations here (inserts, updates, deletes, etc)
// database changes will not be saved until commit
conn.commit(); // save changes (all-or-nothing)
} catch (SQLException e) {
// handle exception
conn.rollback(); // rollback changes
}
</pre>
</div>
<p>Now, let's apply this to the funds transfer case study.</p>
<p>The <code>transfer()</code> method is now overloaded with an additional connection parameter.</p>
<pre class="brush: java; highlight: [6, 7, 8]">
import java.sql.*;
...
... class BankAccountService {
public void transfer(MonetaryAmount amount, ...)
throws SQLException {...}
public void transfer(
MonetaryAmount amount, ..., Connection conn)
throws SQLException {...}
...
}
</pre>
<p>The method that does not accept a connection, creates a new connection, and calls the same method that accepts a connection parameter. The method that accepts a connection, uses it to carry out debit and credit operations.</p>
<pre class="brush: java; highlight: [7]">
import java.sql.*;
... class BankAccountService {
public void transfer(MonetaryAmount amount, ...)
throws SQLException {
try (Connection conn = dataSource.getConnection()) {
conn.setAutoCommit(false);
transfer(amount, ..., conn);
conn.commit(); // save changes (all-or-nothing)
} catch (SQLException e) {
// handle exception
conn.rollback(); // rollback changes
}
}
public void transfer(
MonetaryAmount amount, ..., Connection conn)
throws SQLException {
debit(amount, ..., conn);
credit(amount, ..., conn);
...
}
public void credit(MonetaryAmount amount, AccountId accountId)
throws SQLException {...}
public void credit(MonetaryAmount amount, AccountId accountId,
Connection conn) throws SQLException {...}
public void debit(MonetaryAmount amount, AccountId accountId)
throws SQLException {...}
public void debit(MonetaryAmount amount, AccountId accountId,
Connection conn) throws SQLException {...}
...
}
</pre>
<p>The <code>debit()</code> and <code>credit()</code> methods are also overloaded. Just like the <code>transfer()</code> methods, the one that does not accept a connection parameter, creates a new connection, and calls the one that accepts a connection object. The method that accepts a connection, uses it to carry out SQL operations.</p>
<pre class="brush: java; highlight: [19, 35]">
import java.sql.*;
... class BankAccountService {
public void transfer(MonetaryAmount amount, ...)
throws SQLException {
... transfer(amount, ..., conn); ...
}
public void transfer(
MonetaryAmount amount, ..., Connection conn)
throws SQLException {
debit(amount, ..., conn);
credit(amount, ..., conn);
...
}
public void credit(MonetaryAmount amount, AccountId accountId)
throws SQLException {
try (Connection conn = dataSource.getConnection()) {
conn.setAutoCommit(false);
credit(amount, accountId, conn);
conn.commit(); // save changes (all-or-nothing)
} catch (SQLException e) {
// handle exception
conn.rollback(); // rollback changes
}
}
public void credit(MonetaryAmount amount, AccountId accountId,
Connection conn) throws SQLException {
... = conn.prepareStatement(...);
... // SQL operations for credit
}
public void debit(MonetaryAmount amount, AccountId accountId)
throws SQLException {
try (Connection conn = dataSource.getConnection()) {
conn.setAutoCommit(false);
debit(amount, accountId, conn);
conn.commit(); // save changes (all-or-nothing)
} catch (SQLException e) {
// handle exception
conn.rollback(); // rollback changes
}
}
public void debit(MonetaryAmount amount, AccountId accountId,
Connection conn) throws SQLException {
... = conn.prepareStatement(...);
... // SQL operations for debit
}
...
}
</pre>
<p>Notice that we'll need to pass connection objects all over the place. Yes, this looks ugly. But stay with me for a few more minutes and read on.</p>
<p>A pattern emerges from these methods. Whenever a database transaction needs to be made, a connection is either provided or established, then used to carry out SQL operations (or database changes), and the changes are committed (or rolled back) by the method that established the connection.</p>
<p>The method accepting a connection object gives up control of the transaction to its caller (as in the case of <code>credit(amount)</code> calling <code>credit(amount, connection)</code>). The scope of transactions can span one method call, or several methods calls (as in the case of <code>transfer(amount)</code> calling <code>debit(amount, connection)</code> and <code>credit(amount, connection)</code>).</p>
<p>As already mentioned, this looks ugly and error-prone. The connection object is passed all over the place. So, how can we improve on this?</p>
<h2>Spring-managed JDBC Transactions</h2>
<p>First, we'll use Spring's <code>JdbcTemplate</code> to help us deal with JDBC boilerplate code and <code>SQLException</code>s. Second, we'll use Spring's declarative transaction management.</p>
<p>When we start using a <code>JdbcTemplate</code> inside <code>BankAccountService</code>, we no longer need to handle connections explicitly, and we can remove those overloaded methods. The connection object actually gets hidden in some thread local object.</p>
<pre class="brush: java; highlight: [12, 16]">
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.transaction.annotation.Transactional;
@Transactional
... class BankAccountService {
private JdbcTemplate jdbcTemplate; // injected via constructor
...
public void transfer(MonetaryAmount amount, ...) {
...
}
public void credit(MonetaryAmount amount, AccountId accountId) {
jdbcTemplate.update(...); // SQL operations for credit
...
}
public void debit(MonetaryAmount amount, AccountId accountId) {
jdbcTemplate.update(...); // SQL operations for debit
...
}
...
}
</pre>
<p>How does <code>JdbcTemplate</code> get a <code>Connection</code>?</p>
<p>For each <code>JdbcTemplate</code> method, it retrieves a connection from the given <code>DataSource</code> that is provided (either as constructor argument, or as property <code>setDataSource(DataSource)</code>). After the <code>JdbcTemplate</code> method returns, the connection is closed. Without transactions, the connection's auto-commit mode remains <code>true</code>. The good thing is, <code>JdbcTemplate</code> is transaction-aware. That means, it will participate in the on-going transaction, if there is one (i.e. use the on-going transaction's connection).</p>
<p>So, how do we set the connection's auto-commit mode to <code>false</code>, so that a transaction can be started and span more than one method call to <code>JdbcTemplate</code>?</p>
<p>To use transactions, we <em>do not</em> set the connection's auto-commit mode to <code>false</code>. Instead, we tell Spring to start transactions. In turn, Spring will retrieve a connection, set its auto-commit mode to <code>false</code>, and keep using this <em>same</em> connection until the transaction is completed (either with all changes saved/committed, or all changes rolled back).</p>
<p>So, how do we tell Spring to start transactions, and what does <code>@Transactional</code> do?</p>
<p>As you might have guessed, the <code>@Transactional</code> annotation tells Spring when to start a transaction (by setting the connection's auto-commit mode to <code>false</code>).</p>
<p>We tell Spring to start transactions by marking methods as <code>@Transactional</code> (or mark the class as <code>@Transactional</code> which makes all public methods transactional). And for Spring to start noticing <code>@Transactional</code> annotations, we'll need to add <code><tx:annotation-driven /></code> in our XML-based configuration, or add <code>@EnableTransactionManagement</code> in our Java-based <code>@Configuration</code>.</p>
<p>When Spring encounters this annotation, it proxies the object (usually referred to as a Spring-managed bean). The proxy starts a transaction (if there is no on-going transaction) for methods that are marked as <code>@Transactional</code>, and ends the transaction when the method returns successfully.</p>
<img style="display: block; margin: 0 auto 1em auto;" alt="Proxy adds transaction behavior" src="//yuml.me/1f997647" />
<p>Since <code>JdbcTemplate</code> is transaction-aware, it knows how to get the on-going transaction's connection (if one exists), and not create a new one.</p>
<pre class="brush: java; highlight: [4]">
...
import org.springframework.transaction.annotation.Transactional;
@Transactional
... class BankAccountService {
...
public void transfer(MonetaryAmount amount, ...) {
...
debit(amount, ...);
credit(amount, ...);
...
}
public void credit(MonetaryAmount amount, AccountId accountId) {...}
public void debit(MonetaryAmount amount, AccountId accountId) {...}
...
}
</pre>
<p>So, a call to <code>debit()</code> will use a transaction. A separate call to <code>credit()</code> will use a transaction. But what happens when a call to <code>transfer()</code> is made?</p>
<p>Since the <code>transfer()</code> method is marked as <code>@Transactional</code>, Spring will start a transaction. This same transaction will be used for calls to <code>debit()</code> and <code>credit()</code>. In other words, <code>debit(amount)</code> and <code>credit(amount)</code> will not start a new transaction. It will use the on-going transaction (since there is one).</p>
<p>Here's another good blog post to better understand <a href="//spring.io/blog/2012/05/23/transactions-caching-and-aop-understanding-proxy-usage-in-spring">Transactions, Caching and AOP in Spring</a>.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com1tag:blogger.com,1999:blog-2526628884837213668.post-56205891378745358752015-09-21T10:00:00.000-07:002015-09-21T21:10:17.465-07:00Blob Handling in Java and JDBC <p>Whenever I search the web for storing and retrieving <abbr title="Binary Large OBject">BLOB</abbr>s to and from a database, I usually get a complicated sample that uses byte arrays (<code>byte[]</code>) and JDBC code. Since I usually deal with web-based enterprise applications, I need to store BLOBs (e.g. images, spreadsheets, documents, PDFs) to a database and retrieve them later (e.g. as an <code><img></code> tag in an HTML page, or downloaded from a URL).</p>
<h2>JDBC and <abbr title="Binary Large OBject">BLOB</abbr>s</h2>
<p>The typical way of storing BLOBs with JDBC is shown below. Note that the created objects need to be closed properly to prevent resource leaks.</p>
<pre class="brush: java; highlight: [6, 10]">
Connection connection = ...getConnection();
try {
PreparedStatement stmt =
connection.prepareStatement("INSERT...");
try {
// Sometimes, a byte[] is used here :(
InputStream bytes = new FileInputStream("..."); // e.g. image file
try {
... // other parameters set
// Unsure on how to create a Blob and use setBlob(...)
stmt.setBinaryStream(..., bytes);
stmt.execute();
} finally {
bytes.close();
}
} finally {
stmt.close();
}
} finally {
connection.close();
}
</pre>
<p>Note that the code above is possibly an over-simplification. Most often, the <abbr title="Binary Large OBject">BLOB</abbr> is embedded with another persistent object (e.g. a <code>Person</code> with a photo that is stored as a BLOB).</p>
<p>To retrieve the BLOB, the following code is typically seen. Note that the BLOB is only available while the JDBC connection stays open. Thus, some developers resort to using byte arrays (<code>byte[]</code>) to temporarily store and transfer them, <em>before</em> the connection gets closed.</p>
<pre class="brush: java; highlight: [10, 12]">
Connection connection = ...getConnection();
try {
PreparedStatement stmt =
connection.prepareStatement("SELECT...");
try {
... // other parameters set
ResultSet rs = stmt.executeQuery();
try {
InputStream bytes = rs.getBinaryStream(...);
// Sometimes, a byte[] is used with rs.getBytes() :(
try {
// Must use bytes before JDBC connection gets closed
} finally {
bytes.close();
}
} finally {
rs.close();
}
} finally {
stmt.close();
}
} finally {
connection.close();
}
</pre>
<p>The astute reader can see (lines also highlighted) how error-prone BLOB handling is with JDBC. The lesser known <a href="http://docs.oracle.com/javase/6/docs/api/java/sql/Connection.html#createBlob%28%29"><code>Connection.createBlob()</code></a> (introduced in Java 6 / JDBC 4.0) and <a href="http://docs.oracle.com/javase/6/docs/api/java/sql/ResultSet.html#getBlob%28int%29"><code>ResultSet.getBlob(...)</code></a> is not often used.</p>
<h2>File Systems and <abbr title="Binary Large OBject">BLOB</abbr>s</h2>
<p>Another often seen way of handling BLOBs in Java is to store them as files in a file system. But this has some drawbacks, like the tricky to use unique file names, and possible un-tracked tampering of file contents. In some cases, storing them as files may not be permitted when running in a server with a security manager (e.g. Servlet-container like Tomcat with a <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/SecurityManager.html">SecurityManager</a>). The security manager protects the server from trojan servlets, JSPs, JSP beans, tag libraries, or even inadvertent mistakes (e.g. <code style="white-space: nowrap"><% System.exit(1); %></code> in a JSP).</p>
<h2>Introducing <a href="https://github.com/lorenzodee/jBlubble">jBlubble</a></h2>
<p>I find the above handling of BLOBs to be over-complicated. Because of this, I wanted to make BLOB handling easier, and less error-prone. I am also inspired by how Google's AppEngine for Java was handling BLOBs. Thus, I started looking at the BLOB-related code I have worked with in the past years, and came up with a simple API, I called <strong>jBlubble</strong>. The main interface is <code>BlobstoreService</code>.</p>
<pre class="brush: java;">
public interface BlobstoreService {
String createBlob(
InputStream in,
String name, String contentType) ...;
String createBlob(
BlobstoreWriteCallback callback,
String name, String contentType) ...;
BlobInfo getBlobInfo(String blobKey) ...;
void serveBlob(String blobKey, OutputStream out) ...;
int[] delete(String... blobKeys);
}
</pre>
<p>The code is available at <a href="https://github.com/lorenzodee/jBlubble">my GitHub account</a>. As of this writing, a JDBC-implementation of the API is available. Other implementations are also welcome.</p>
<p>The <code>createBlob</code> methods <em>abstract</em> the JDBC-related code of using <a href="http://docs.oracle.com/javase/6/docs/api/java/sql/Blob.html"><code>java.sql.Blob</code>s</a>, and making sure allocated resources are closed properly. To store BLOBs uploaded to a webapp, the code looks something like:</p>
<pre class="brush: java">
// Servlet configured to support multipart/form-data
// HttpServletRequest
request.getPart("...").write(fileName);
// Open an input stream with the file (created from uploaded part)
InputStream in = new FileInputStream(fileName);
try {
... blobKey = blobstoreService.createBlob(in, ...);
} finally {
in.close();
}
</pre>
<p>To serve the previously stored BLOB, the code looks something like:</p>
<pre class="brush: java">
// HttpServletResponse
BlobInfo blobInfo = blobstoreService.getBlobInfo(blobKey);
response.setContentType(blobInfo.getContentType());
OutputStream out = response.getOutputStream();
blobstoreService.serveBlob(blobKey, out);
</pre>
<p>In some cases, like when generating reports, the output stream can be stored as a BLOB too, like so:</p>
<pre class="brush: java">
... JasperPrint print = ...;
... JRExporter exporter = ...; // new JRPdfExporter();
... blobKey = blobstoreService.createBlob(
new BlobstoreWriteCallback() {
@Override
public long writeToOutputStream(OutputStream out) ... {
...
exporter.setParameter(JRExporterParameter.JASPER_PRINT, print);
exporter.setParameter(JRExporterParameter.OUTPUT_STREAM, out);
exporter.exportReport();
...
}
}, ...);
// generated blobKey can then be
// used to identify the generated report
</pre>
<p>Since report generation can take its time (especially when the report has several pages), it is a good idea to store the report in a database, rather than directly writing to a servlet output stream. When the report is complete, it can be downloaded (and re-downloaded) anytime.</p>
<p>The key design consideration was to abstract the persistence-related specifics (like JDBC) and use a simpler API. A callback interface was used to support writing to <code>OutputStream</code>s to ensure that it gets closed properly. A higher level method was used to serve the BLOBs directly to an output stream. The <code>BlobInfo</code> contains a timestamp field that can be used to support <code>Last-Modified</code> HTTP header for caching purposes. As of the moment, BLOBs are immutable and <em>cannot</em> be updated. To change, a new BLOB should be created, delete the old one, and reference the new one.</p>
<p>Instead of embedding the BLOB with their related persistent objects (or entities), I find it better to simply reference the BLOB. So, instead of this...</p>
<pre class="brush: java; highlight: [4]">
@Entity
public class Person {
@Id private Long id;
@Lob byte[] photo;
...
}
</pre>
<p>... I find it better to just reference the BLOB like this.</p>
<pre class="brush: java; highlight: [4]">
@Entity
public class Person {
@Id private Long id;
String photoId; // retrieved via BlobstoreService
...
}
</pre>
<img style="width: 60px; height: auto; float: right; margin: 0 3em 1em 1em; transform: rotate(30deg); border: 0 none" src="http://reedsinc.com/wp-content/uploads/2012/10/Flying-Cauldron-Butterscotch-Beer.png" />
<p>I've started replacing existing BLOB handling code with the API we've developed. So far, it has simplified things. I hope this will help others too. Now, it's time for a cold one.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com1tag:blogger.com,1999:blog-2526628884837213668.post-49887116883322364852015-08-24T10:00:00.000-07:002015-08-31T17:24:14.315-07:00Domain Logic Patterns (Martin Fowler's Revenue Recognition Problem)<p>
In the <a href="http://orangeandbronze.com/enterprise-architect/design-patterns-training">training courses</a> I've given, providing concrete examples (not just UML diagrams) help the course participants understand better. Here in this post, I'm sharing a Java-implementation of the revenue recognition problem (as described in Martin Fowler's <abbr title="Pattern of Enterprise Application Architecture">PoEAA</abbr> book). The problem is used to explain three domain logic patterns: transaction script, table module, and domain model.
</p>
<h2>Revenue Recognition Problem</h2>
<p>Just in case you haven't read the book, here's an excerpt from the book that explains the revenue recognition problem, <em>taken from Martin Fowler's PoEAA book (page 112)</em>.</p>
<blockquote>
<p>Revenue recognition is a common problem in business systems. It's all about when you can actually count the money you receive on your books. If I sell you a cup of coffee, it's a simple matter: I give you the coffee, I take your money, and I count the money to the books that nanosecond. For many things it gets complicated, however. Say you pay me a retainer to be available that year. Even if you pay me some ridiculous fee today, I may not be able to put it on my books right away because the service is to be performed over the course of a year. One approach might be to count only one-twelfth (1/12) of that fee for each month in the year, since you might pull out of the contract after a month when you realize that writing has atrophied my programming skills.</p>
<p>The rules for revenue recognition are many, various, and volatile. Some are set by regulation, some by professional standards, and some by company policy. Revenue tracking ends up being quite a complex problem.</p>
<p>I don't fancy delving into the complexity right now, so instead we'll imagine a company that sells three (3) kinds of products: word processors, databases, and spreadsheets. According to the rules, when you sign a contract for a word processor, you can book all the revenue right away. If it's a spreadsheet, you can book one-third (1/3) today, one-third in sixty (60) days, and one-third in ninety (90) days. If it's a database, you can book one-third today, one-third in thirty (30) days, and one-third in sixty (60) days. There's no basis for these rules other than my own fevered imagination. I'm told that the real rules are equally rational.</p>
<p><em style="font-size: 80%">Emphasis, formatting, and numbers added for clarity</em></p>
</blockquote>
<h2>Domain Logic Patterns Example (Java-implementation)</h2>
<p>The working code is available at <a href="https://github.com/lorenzodee/experiments/tree/master/revenue-recognition" target="_GitHub">my GitHub account</a>.</p>
<p>I used <em>the same</em> integration test (<a href="https://github.com/lorenzodee/experiments/blob/master/revenue-recognition/src/test/java/revenue/recognition/interfaces/facade/AbstractRevenueRecognitionServiceFacadeTests.java">AbstractRevenueRecognitionServiceFacadeTests</a>) and applied it against three different implementations: transaction script, table module, and domain model.</p>
<p>Here are the three (3) patterns:</p>
<ul>
<li><strong>transaction script</strong> — organizes business logic by procedures where each procedure handles a single request from the presentation.</li>
<li><strong>domain model</strong> — an object model of the domain that incorporates both behavior and data.</li>
<li><strong>table module</strong> — a single instance that handles the business logic for all rows in a database table or view.</li>
</ul>
<p style="clear: both; text-align: center;"><img border="0" style="border: 0; max-width: 100%" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCKBp9nCgjllkgq0TnMw-u3Lnii2MJ9PU1FJr3c3kbsMl12rN9Dd326SIZBd04Pg3QG0v4rmpVf9hPxDkwo4hbRaQkdDzTqN0BTKQLkLFGfULrfHS5g-3wedXyIx4XEXwjx_9RHPY0cxs/s1600/Overall.png" /></p>
<p>For transaction script and table module implementations, I used table data gateways.</p>
<p style="clear: both; text-align: center;"><img border="0" style="border: 0; max-width: 100%" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnQ5e4CMchoupI6dYLzLa39zS1hKS3jIlrPLbh3WLkUC0AbGOG9-pPf5735MDUT49uc4v8Pk-Zz5AaAQrLwtdRrG_sEZO4gSv_mesv6XOInbGQ8D8rMKmPT3oHQ0MRbj3XZYlBsm_V2DM/s1600/Transaction+Script.png" alt="UML diagram of transaction script implementation" /></p>
<p style="clear: both; text-align: center;"><img border="0" style="border: 0; max-width: 100%" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyTRjP-IlXpGpEkbZ2vS_ExH-8a5dwL4BFv19dHnKNx9_sjFDhdH2ZjLovbCeMaT52Suugbv-qGuuZXqWqsOHQaQ7UQ4gqM34a2I_b1IcN2axAhpZSruaa0anLh3fQ-fRthCWg09-ZN-Q/s1600/Table+Module.png" alt="UML diagram of table module implementation" /></p>
<p>For domain model, I used data mapper (via JPA/Hibernate). I tried to stick as close as possible to the database schema provided. In fact, all three patterns work with the <em>same</em> database schema.</p>
<p style="clear: both; text-align: center;"><img border="0" style="border: 0; max-width: 100%" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAyDfP5XoO8Wo51EBdxkiArbaw9sbhp7TmOBFtHV1BtoiiNwyUSk0JPS6YZRZEv15oDVpYJ-sG81G48qmqmTCThSazkbZhUhym040hmZXorYcmKHTAWjVtLZBm4zkuM1gMHGaFyGjWwV0/s1600/Domain+Model.png" alt="UML diagram of domain model implementation" /></p>
<p>The transaction script implementation can be found under the <code>revenue.recognition.transaction.script</code> package. The table module implementation is under the <code>revenue.recognition.table.module</code> package. The domain model is found under the following packages: <code>revenue.recognition.interfaces.facade.internal</code>, <code>revenue.recognition.application.impl</code>, and <code>revenue.recognition.domain.model</code>.
<p>Please feel free to review the implementation, and let me know of any items I've missed.</p>
<p>Note that having three (3) different patterns in one application is unnecessary (and quite confusing). I've only done this to better compare the different approaches to organizing domain logic.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com3tag:blogger.com,1999:blog-2526628884837213668.post-42559908953230470152015-07-12T11:00:00.000-07:002015-07-14T00:32:34.864-07:00Effective Design Patterns and Poetry<p>In one of the training courses I've been giving on object-oriented design techniques, I was discussing two principles from the <a href="https://en.wikipedia.org/wiki/Design_Patterns">GoF book, Design Patterns: Elements of Reusable Object-Oriented Software</a> <img style="float: right; margin: 1em 0 1em 1em" width="140" height="175" src="//upload.wikimedia.org/wikipedia/en/7/78/Design_Patterns_cover.jpg" />:</p>
<ul>
<li>"Program to an 'interface', not an 'implementation'." (Gang of Four 1995:18)</li>
<li>Composition over inheritance: "Favor 'object composition' over 'class inheritance'." (Gang of Four 1995:20)</li>
</ul>
<p>And I was thinking how I can make those two phrases stick more. And a thought occurred to me that perhaps if we make those principles rhyme, it'll be easier for developers to remember, and become more effective.</p>
<h3>Rhyme-as-Reason Effect</h3>
<p>In one study <sup><a href="#birds-of-a-feather-flock-conjointly">[1]</a></sup>, researchers compared the perceived truthfulness of rhyming vs. similar non-rhyming statements. For example:</p>
<p style="display: block; margin: 1em auto; max-width: 24em; border-left: 4px solid #ccc; padding: 0.5em; text-align: center; background-color: #ffd">"What sobriety conceals, alcohol reveals" <span style="display: block">vs</span> "What sobriety conceals, alcohol unmasks"</p>
<p>Which one do you find is true (or more truthful)?</p>
<p>Here's another one.</p>
<p style="display: block; margin: 1em auto; max-width: 24em; border-left: 4px solid #ccc; padding: 0.5em; text-align: center; background-color: #ffd">"Caution and measure will win you treasure" <span style="display: block">vs</span> "Caution and measure will win you riches"</p>
<p>The study suggested that "rhyme, like repetition, affords statements an enhancement in processing fluency that can be misattributed to heightened conviction about their truthfulness."</p>
<p>In other words, rhyming makes statements easier to understand. Which, in turn, makes them appear more accurate.</p>
<h3>Object-Oriented Design Principles with Poetry</h3>
<p>Here's my attempt at making those two principles rhyme.</p>
<p style="font-weight: bold; color: #888">Program to an interface, not to an implementation.</p>
<p style="font-family: Georgia, serif; font-weight: bold; color: #369; margin-left: 2em; max-width: 20em; padding: 0.5em; background-color: #ffd">An implementation is good and fine, <span style="display: block">an interface is a wiser design.</span></p>
<p style="font-family: Georgia, serif; font-weight: bold; color: #369; margin-left: 2em; max-width: 25em; padding: 0.5em; background-color: #ffd">Programming to an implementation is logical, <span style="display: block">programming to an interface is exceptional.</span></p>
<p style="font-family: Georgia, serif; font-weight: bold; color: #369; margin-left: 2em; max-width: 30em; padding: 0.5em; background-color: #ffd">Programming to an implementation is what is needed, <span style="display: block">programming to an interface is why we succeeded.</span></p>
<p style="font-weight: bold; color: #888">Favor composition, over inheritance.</p>
<p style="font-family: Georgia, serif; font-weight: bold; color: #369; margin-left: 2em; max-width: 20em; padding: 0.5em; background-color: #ffd">Inheritance if you must, <span style="display: block">in composition we trust.</span></p>
<h3>Next time…</h3>
<p>So the next time I get a chance to run another training course on object-oriented design techniques, I'll give these rhymes, some good times.</p>
<p>If you have rhymes of your own, please feel free to hit the <a href="#comments">comments</a>.</p>
<p id="birds-of-a-feather-flock-conjointly" style="font-size: 0.8em; line-height: 1.1em"><sup>[1]</sup> McGlone, M.S., and Tofighbakhsh, J. (2000) Birds of a feather flock conjointly (?): rhyme as reason in aphorisms. Psychological Science, 11:424-28.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-65149900221515535182015-04-22T19:11:00.001-07:002015-04-22T23:38:15.657-07:00Domain Entities and Draft Forms<p>Recently, I've been reviewing a funds transfer system that uses the <abbr title="Society for Worldwide Interbank Financial Telecommunication">S.W.I.F.T.</abbr> standard message types. After reviewing it, I find that splitting the domain entity from its draft form would make it easier to maintain. Let me explain further.</p>
<h2>Background of Funds Transfer System</h2>
<p>The system allows users to record funds transfer requests as S.W.I.F.T. messages. Note that the system users are familiar with the said message types, and are not afraid to enter the values into a form that displays all the possible fields. To put things in perspective, a typical message type like <abbr title="Message Type">MT</abbr> 103 (Single Customer Credit Transfer) has about 25 fields. Some fields can have several formats. For example, the Ordering Customer (field 50a), can be in one of three formats: option A, option K, and option F. If we include every possible format, MT 103 would easily have over 70 fields.</p>
<h2>Modeling Entities</h2>
<p>Based on Eric Evan's <abbr title="Domain-Driven Design">DDD</abbr> book, domain entities have a clear identity <em>and</em> a life-cycle with state transitions that we care about.</p>
<p>The funds transfer system modeled each <abbr title="Message Type">MT</abbr> as a domain entity. The MT domain entity has a clear identity, and a life-cycle (from draft to approved). It is mapped to its own table with its own corresponding columns to represent its fields. The message fields are implemented as properties. All fields are mutable (via setters methods).</p>
<aside class="" style="max-width: 30em; margin: 1em auto; padding: 0.5em 1em; border: 1px solid #ccc; border-radius: 4px; font-size: 85%; color: #888; overflow: auto">
<img style="float: right; margin-left: 0.5em; border: 0 none" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVLrDT39e_fB3k0I73bEyIWEs9wl9t0Ek8qBr-dMqt0u_h41WC2htqSwMyzFqRsMCH2Rm30pYLvURFoDa2nzQoyUBrqj4mOsfnkC4S9Z7ySPu35nrb_gbO0giNAf1pwdGUxgtq0OR6M8c/s1600/thinking-4.jpg" />
<div>At this point, you might pause and think about <a href="http://www.martinfowler.com/bliki/AnemicDomainModel.html">anemic domain models</a>. You're probably thinking that there is hardly any behavior on these MT objects. You're probably thinking that these MT objects are just bags of getters and setters. And you're probably correct. But there's a reason for this. Read on.</div>
</aside>
<p>The system validates the <abbr title="Message Type">MT</abbr>s, and displays which fields have errors. But the errors did not stop the system from persisting/storing the message. The system allowed the <em>draft</em> message to be stored for further modification (until it is approved).</p>
<p>This analysis led me to re-think about how the system modeled messages as entities. While it is tempting to think of them as rich domain entities that represent a real-world funds/credit transfer, they're really not. They're actually request forms. Much like the real-world paper forms that one fills-out, and submits for approval, and possibly being re-submitted due to corrections.</p>
<p>Given this, I would model the <em>draft</em> message as another domain entity that I'll call <code>MessageTypeForm</code> (or <code>MT103Form</code> for MT 103). This domain entity will use a map of fields (still in compliance with S.W.I.F.T. messages, where each field is keyed by an alpha-numeric string) (e.g. <code>get("50a")</code>), and <em>not</em> use a separate property for each field (e.g. <code>getOrderingCustomer()</code>). By using a map of fields, the entity would require lesser code, and can still be persisted (or mapped to table rows).</p>
<p>The <code>CreditTransfer</code> (or <code>SingleCustomerCreditTransfer</code>) would be another entity (not the same as the entity with a map of fields — <code>MT103Form</code>). This credit transfer entity shall have a clear identity and a life-cycle (e.g. being amended or cancelled). It can have a reference (by ID, and not by type) to the <code>MT103Form</code> entity from which it was created. It is also easier to establish invariants on this entity, as opposed to having all properties being mutable.</p>
<p>The validation of <code>MT103Form</code> will have to change from using properties (e.g. <code>getOrderingCustomer()</code>) to using a get method with a key parameter (to determine the message field it wants to retrieve) (e.g. <code>get("50a")</code>).</p>
<h2>Deeper Insight</h2>
<aside class="" style="max-width: 10em; margin: 1em; float: right; padding: 0.5em; border: 1px solid #ccc; border-radius: 4px; font-size: 85%; color: #888">
<div>Just because a domain entity is largely made up of getters and setters, does not mean it is anemic.</div>
</aside>
<p>The "draft/form" entity may look like a bag of getters and setters. But it has an identity, and a life-cycle (from draft to approved). So, it is an entity. Many "draft/form" instances will exist in the system simultaneously. The different instances may even have the same field values, but it is important for us to be able to track individual "draft/form" instances.</p>
<p>Discovering the distinction between the "draft/form" and "fund/credit transfer" entities have made things easier to implement. Having two separate domain entities has made the model better (reflect the problem it is trying to solve).</p>
<p>This <em>deeper insight</em> would not have been possible without the help of the domain experts, and my team mates: <a href="https://www.linkedin.com/in/christinedesagun">Tin</a>, Anson, <a href="https://www.facebook.com/ichipipot">Richie</a>, and <a href="https://www.linkedin.com/pub/maria-cristina-balsita/98/754/a26">Tina</a>. Thanks guys!</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com1tag:blogger.com,1999:blog-2526628884837213668.post-6810725686357355552015-04-06T17:00:00.000-07:002015-04-06T20:32:19.223-07:00One Thing Good Spring Developers Know<p>
In my recent training sessions on the <a href="http://www.orangeandbronze.com/java/core-spring-training">(core) Spring Framework</a>, I was asked, "If there was one thing that a (Java) Spring developer should know, what should that be?" That question caught me off guard. Yes, the (core) Spring Framework does cover a lot of areas (e.g. beans, configuration, aspect-oriented programming, transactions). And it was difficult for me to point out <em>just</em> one thing. I ended up mentioning everything that we covered in our <a href="http://www.orangeandbronze.com/java/core-spring-training">(3 day) training course</a>.</p>
<aside style="display: block; float: right; width: 30%; max-width: 15em; margin: 1em 0 1em 1em; border: 1px solid #ccc; border-radius: 4px; padding: 0.5em; background-color: #eee">
If there was one thing that a (Java) Spring developer should know, what should that be?
</aside>
<p>As I gave that question more thought, I began to think about the <em>most</em> important one. I ended up thinking of how Spring uses aspects to add behavior to managed objects (usually called beans) as the <em>most</em> important. This is how the Spring Framework supports transactions, security, scope, Java-based configuration, among others. And I'm sharing my thoughts here in this post.</p>
<aside style="display: block; float: left; width: 30%; max-width: 15em; margin: 1em 1em 1em 0; border: 1px solid #ccc; border-radius: 4px; padding: 0.5em; background-color: #eee">
I ended up thinking of how Spring uses aspects to add behavior to managed objects (usually called beans) as the <em>most</em> important.
</aside>
<h2>ORM and Lazy Loading Exceptions</h2>
<p>Most developers who use some form of <abbr title="Object-Relational Mapping">ORM</abbr> have encountered an exception that signifies that child entities could not be loaded (e.g. <a href="http://docs.jboss.org/hibernate/orm/4.3/javadocs/org/hibernate/LazyInitializationException.html"><code>LazyInitializationException</code></a>).</p>
<p>Some developers who have encountered this would use an "open session in view" (<abbr title="Open Session In View">OSIV</abbr>) pattern to keep the session open and prevent this exception from happening. But I find this to be an overkill. Worse, some developers consider the "open session in view" pattern to be the <em>only</em> solution. A possible underlying cause for this misconception could be that the developer is probably not armed with the knowledge of using the Spring Framework effectively to keep the ORM session open longer.</p>
<p>In the case of <abbr title="Java Persistence API">JPA</abbr>, the "open entity manager in view" pattern will create an entity manager at the beginning of the request, bind it to the request thread, and close it when the response is completed.</p>
<p>So, if not the <abbr title="Open Session In View">OSIV</abbr> pattern, what would be a better solution?</p>
<p>The short answer is to use the Spring Framework to keep the session open for the duration that you need it (e.g. <code>@Transactional</code>). Keep on reading as I'll provide a longer answer.</p>
<h2>Services and Repositories</h2>
<p>In a layered architecture, a typical design pattern is to define a <a href="http://dddsample.sourceforge.net/characterization.html#Services">domain or application service</a> (usually defined as an interface) to provide business functionality (e.g. start using a shopping cart, adding items to that shopping cart, searching for products). Domain and application service implementations would typically delegate the retrieval/persistence of domain entities to repositories.</p>
<style>
#layered-layers { text-align: center; border: 1px solid #ccc; border-radius: 8px; padding: 0.5em; }
#layered-layers td { padding: 0.5em 1em }
</style>
<table id="layered-layers">
<tr><td style="background-color: #e0543e; color: white">Presentation Layer</td></tr>
<tr><td style="background-color: #00a86d; color: white">Business Layer</td></tr>
<tr><td style="background-color: #3c8ef3; color: white">Data Access (or Infrastructure) Layer</td></tr>
</table>
<p><a href="http://dddsample.sourceforge.net/characterization.html#Repositories">Repositories</a> (or data access objects) are also defined as interfaces to retrieve/persist domain entities (i.e. provide ORM and <abbr title="Create, Read, Update, Delete">CRUD</abbr> access). Naturally, repository implementations use ORM libraries (e.g. JPA/Hibernate, myBATIS) to retrieve and persist domain entities. With this, it uses the ORM framework's classes to connect to the persistent store, retrieve/persist the entity, and close the connection (called session in Hibernate). There's no problem of lazy loading failures at this point.</p>
<p>The problem of lazy loading failures occur when the service retrieves a domain entity using the repository, and wants to load child entities (<em>after</em> the repository method has returned). By the time the repository returns the domain entity, the ORM session gets closed. Because of this, attempts to access/load child entities in the domain service cause an exception.</p>
<p>The code snippets below illustrate how a lazy loading exception can occur when the child items of an order entity is lazily loaded after being returned by the repository.</p>
<pre class="brush:java highlight:[16]">
@Entity
public class Order {
@OneToMany // defaults to FetchType.LAZY
private List<OrderItem> items;
…
public List<OrderItem> getItems() {…}
}
public class SomeApplicationServiceImpl implements SomeApplicationService {
private OrderRepository orderRepository;
…
@Override
public void method1(…) {
…
order = orderRepository.findById(...);
order.getItems(); // <-- Lazy loading exception occurs!
…
}
…
}
public class OrderRepositoryImpl implements OrderRepository {
@PersistenceContext
private EntityManager em;
…
@Override
public Order findById(...) {...}
…
}
</pre>
<p>The repository implementation explicitly uses JPA for its ORM (as illustrated with the use of an <code>EntityManager</code>).</p>
<p>At this point, some developers may opt to use eager fetch to prevent the lazy initialization exception. Telling the ORM to eagerly fetch the child items of an order entity will work. But sometimes, we don't need to load the child items. And eagerly loading this might be unnecessary overhead. It would be great to only load it when we need it.</p>
<p>To prevent the lazy initialization exception (and not be forced to eagerly fetch), we'll need to keep the ORM session open until the calling service method returns. In Spring, it can be as simple as annotating the service method as <code>@Transactional</code> to keep the session open. I find that this approach is better than using "open session in view" pattern (or being forced to use eager fetching), since it keeps the session open only for the duration that we intend it to be.</p>
<pre class="brush:java highlight:[5, 9]">
public class SomeApplicationServiceImpl implements SomeApplicationService {
private OrderRepository orderRepository;
…
@Override
@Transactional // <-- open the session (if it's not yet open)
public void method1(…) {
…
order = orderRepository.findById(...);
order.getItems(); // <-- Lazy loading exception should not happen
…
}
…
}
</pre>
<h2>Domain Entities in the Presentation Layer</h2>
<p>Even after keeping the ORM session open in the service layer (beyond the repository implementation objects), the lazy initialization exception can still occur when we expose the domain entities to the presentation layer. Again, because of this, some developers prefer the OSIV approach, since it will also prevent lazy initialization exceptions in the presentation layer.</p>
<p>But why would you want to expose domain entities in the presentation layer?</p>
<p>From experience, I've worked with teams who prefer to expose domain entities in the presentation layer. This usually leads to <a href="http://www.martinfowler.com/bliki/AnemicDomainModel.html">anemic domain model</a>, since presentation layer frameworks need a way to bind input values to the object. This forces domain entities to have getter and setter methods, and a zero-arguments constructor. Having getters and setters will make invariants difficult to enforce. For simple domains, this is workable. But for more complex domains, a richer domain model would be preferred, as it would be easier to enforce invariants.</p>
<p>In a richer domain model, the objects that represent the presentation layer input/output values are actually data transfer objects (DTOs). They represent inputs (or commands) that are carried out in the domain layer. With this in mind, I prefer to use DTOs and maintain a richer domain model. Thus, I don't really run into lazy initialization exceptions in the presentation layer.</p>
<h2>Aspects to add behavior to managed objects</h2>
<p>Spring intercepts calls to these <code>@Transactional</code> annotated methods to ensure that an ORM session is open.</p>
<p>Transactions (or simply keeping an ORM session open) are not the only behavior provided using aspects. There's security, scope, Java-based configuration, and others. Knowing that the Spring Framework uses aspects to add behavior is one of the key reasons why we let Spring manage the POJOs that we develop.</p>
<h2>Conclusion</h2>
<p>There you go. That for me is the <em>one most important</em> thing that a Spring Framework developer should know when using the core. Now that I've given my opinion as to what is the <em>one most important</em> thing, how about you? What do you think is the <em>one most important</em> thing to know when tackling Core Spring. Cheers!</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com1tag:blogger.com,1999:blog-2526628884837213668.post-59496416475746150362014-12-15T18:00:00.000-08:002014-12-16T02:33:51.318-08:00Top 3 Improvements New Agile Teams Can Make<p>At first, I was planning to write about the top mistakes that novice Scrum/agile teams make. But then I wanted to say it in a positive way. So, I ended up writing about the top three <em>improvements</em> new Scrum/agile teams can make. Here it goes.</p>
<h2>Focus on Stories, Not Tasks</h2>
<p>Focus on stories (or features), and not tasks. Yes, team members still need to <em>pull</em> tasks from the board, and perform them. But don't forget that the team's goal is to complete stories. This means that when a team member has an option to pull a task from the board, the task should be part of the current on-going story. It would be better if team members are <em>discouraged</em> from starting a task that is not part of the current on-going story.</p>
<p>I've seen the following board (see below) several times in my few years of agile. Each story is grouped as a row (left to right) on the board. Notice how the team has completed several tasks, but none of the stories are done.</p>
<table style="margin: 1em auto; width: 100%; max-width: 800px; border-spacing: 0; border-collapse: separate; table-layout: fixed; border: 1px solid #888; border-radius: 4px">
<caption style="color: #ccc; font-size: 90%">Scrum Board with Just Getting <em>Random</em> Tasks Done</caption>
<tr style="font-family: 'Comic Sans MS', Georgia, serif">
<th style="padding: 0.25em 1em; width: 30%; border-right: 1px solid #ccc; border-bottom: 1px solid #ccc;">To Do</th><th style="padding: 0.25em 1em; width: 40%; border-right: 1px solid #ccc; border-bottom: 1px solid #ccc;">In Progress</th><th style="padding: 0.25em 1em; width: 30%; border-bottom: 1px solid #ccc;">Done</th>
</tr>
<tr style="">
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A8</div>
</td>
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A6</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A3</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A5</div>
</td>
<td style="padding: 0.25em 1em; border-bottom: 1px solid #ccc">
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A1</div>
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A2</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A4</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A7</div>
</td>
</tr>
<tr style="">
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B8</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B6</div>
</td>
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B3</div>
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B7</div>
</td>
<td style="padding: 0.25em 1em; border-bottom: 1px solid #ccc">
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B1</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B2</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B4</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B5</div>
</td>
</tr>
</table>
<aside style="float: right; color: #aaa; width: 30%; max-width: 10em; margin: 0.5em 0 0.5em 1em; padding: 0.5em; border: 1px solid #ccc; border-radius: 4px; font-size: 0.8em; line-height: 1.1em">
<p>This limiting of tasks is practiced in Kanban as <abbr title="Work in Process">WIP</abbr> limits (as I've presented in my softtech.ph 2014 talk) (<a href="https://www.youtube.com/watch?v=qWiL5klTvJg">video here</a>).</p>
</aside>
<p>To put this in the positive, the team is encouraged to achieve a board that looks more like this (below). Here, team members are <em>encouraged</em> to stay on tasks with the on-going story (or feature) until it is done. They're <em>discouraged</em> from starting tasks on another story. But this doesn't mean that team members become idle. They're asked to help other team members to get the story done.</p>
<table style="margin: 1em auto; width: 100%; max-width: 800px; border-spacing: 0; border-collapse: separate; table-layout: fixed; border: 1px solid #888; border-radius: 4px">
<caption style="color: #ccc; font-size: 90%">Scrum Board with Focus on Getting Stories Done</caption>
<tr style="font-family: 'Comic Sans MS', Georgia, serif">
<th style="padding: 0.25em 1em; width: 30%; border-right: 1px solid #ccc; border-bottom: 1px solid #ccc;">To Do</th><th style="padding: 0.25em 1em; width: 40%; border-right: 1px solid #ccc; border-bottom: 1px solid #ccc;">In Progress</th><th style="padding: 0.25em 1em; width: 30%; border-bottom: 1px solid #ccc;">Done</th>
</tr>
<tr style="">
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
</td>
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A8</div>
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A6</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A3</div>
</td>
<td style="padding: 0.25em 1em; border-bottom: 1px solid #ccc">
<div style="transform: rotate(2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A5</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A1</div>
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A2</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A4</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task A7</div>
</td>
</tr>
<tr style="">
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B8</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B6</div>
<div style="transform: rotate(2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B3</div>
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B7</div>
<div style="transform: rotate(-2deg); display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B1</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B2</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B4</div>
<div style="display: inline-block; width: 40px; height: 40px; padding: 10px; margin: 10px; font-size: 12px; background: url('https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVtF7VLHDfbDd1_7vbQMig9DaUPr6bHwPeztNoa1kKce_hq4AHwVrK_XwF4xarEakcYgKTKD1mlbU2hqkZ1YDG4GSCl5PmvgOgIRVWlb9b1NaTnfcpSLiA0UWj_0V6GpxD54fhDAt4-4Y/s1600/yellow-post-it-note.png') no-repeat">Task B5</div>
</td>
<td style="border-right: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 0.25em 1em; ">
</td>
<td style="padding: 0.25em 1em; border-bottom: 1px solid #ccc">
</td>
</tr>
</table>
<h2>Start Only with Ready Stories</h2>
<p>I find that only a <em>few</em> agile teams have heard of <a href="http://www.scruminc.com/definition-of-ready/">definition of ready</a> (DoR). Most of them have heard of <a href="http://www.scruminc.com/definition-done/">definition of done</a> (DoD), but not <abbr title="Definition of Ready">DoR</abbr>. I've seen teams start sprints with stories that are far from being ready.</p>
<p>Some organizations (or companies) provide a checklist to get stories ready (not just <em>sort of</em> ready). I've seen the checklist include items like: estimated at the right size, prioritized by business value, has "done" criteria. It's difficult (almost impossible) to get a user story (or feature) done, if the team does not have an agreed "done" criteria. This "done" criteria (or acceptance criteria) is usually set <em>before</em> the story can be included in a sprint (as part of backlog refinement).</p>
<p>So, other than <abbr title="Definition of Done">DoD</abbr>, take a look at <abbr title="Definition of Ready">DoR</abbr>. It might just help you and your team improve its productivity.</p>
<h2>Split Stories to Fit in Sprint</h2>
<p>Sometimes, user stories (or features) are too big to fit in a sprint (or iteration). When this happens, it would be better to split the user story, than to extend the sprint (to make it long enough for the team to complete the story).</p>
<p>I've witnessed teams extending their sprint cycles just to accommodate larger stories (or epics). And that's probably because they have limited skills in splitting stories. Many new agile teams attempt to split stories by architectural layers: one story for the UI, another for the database, etc. This results into stories that are <em>not valuable</em> to a user, and become <em>interdependent</em> with each other.</p>
<p>So, do be careful when splitting stories. Following Bill Wake's <a href="http://xp123.com/articles/invest-in-good-stories-and-smart-tasks/">INVEST</a> model for good user stories is highly recommended. You can refer to Richard Lawrence's <a href="http://www.agileforall.com/2009/10/patterns-for-splitting-user-stories/">Patterns for Splitting User Stories</a>.</p>
<h2>Conclusion</h2>
<p>The above points may not apply to all teams. But if it does, please let me know. I would love to hear your (or your team's) experiences.</p>
<p style="max-width: 60%"><span style="display: inline-block; vertical-align: middle; max-width: 80%">Here's wishing you more success in achieving your team's goals!</span> <img style="float: right; margin: 0.5em; height: 10em; border: 0 none; transform: rotate(4deg)" alt=""Flying Cauldron" Butterscotch Beer" src="http://reedsinc.com/wp-content/uploads/2012/10/fourth_home.png" /></p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com0tag:blogger.com,1999:blog-2526628884837213668.post-29377218883797996682014-11-16T17:00:00.000-08:002014-11-16T18:36:14.938-08:00Maker-Checker Design Concept<p>We've seen the <a href="http://en.wikipedia.org/wiki/Maker-checker" target="wikipedia">maker-checker</a> concept pop-up several times in our software development experiences with banks. In this post, let me share a possible re-usable design approach. Thanks to <a href="https://plus.google.com/110374340551935178703" target="g+">Tin</a>, <a href="http://www.linkedin.com/pub/richie-panganiban/36/318/41a" target="linkedin">Richie</a>, Tina, Val, and their team, for adding their insights.</p>
<h2>What is Maker-Checker?</h2>
<p>According to <a href="http://en.wikipedia.org/wiki/Maker-checker" target="wikipedia">Wikipedia</a>:</p>
<blockquote>
<p>Maker-checker (or Maker and Checker, or 4-Eyes) is one of the central principles of <em>authorization</em> in the Information Systems of financial organizations. The principle of maker and checker means that for each <em>transaction</em>, there must be at least two individuals necessary for its completion. While one individual may create a transaction, the other individual should be involved in confirmation/authorization of the same. Here the segregation of duties plays an important role. In this way, strict control is kept over system software and data keeping in mind functional division of labor between all classes of employees.</p>
</blockquote>
<aside style="width: 30%; float: right; margin: 1em 0 1em 1em; padding: 1em; border: 1px solid #ccc; border-radius: 8px">By <em>transaction</em>, we're referring to a business transaction (e.g. sale, purchase, remittance request), and <em>not</em> a database transaction.</aside>
<p>Here are some business rules we can derive from the above definition:</p>
<ol>
<li>For any transaction entry, there must be <em>at least</em> two individuals necessary for its completion.</li>
<li>The one who makes the transaction entry (i.e. maker) cannot be the same one who checks (i.e. checker) it.</li>
<li>A transaction entry is only considered completed if it has been checked.</li>
</ol>
<p>Upon further clarification with the domain experts, we've learned the following:</p>
<ol>
<li>The <em>checker</em> cannot make modifications to the transaction entry. Modifications can only be done by <em>maker</em>.</li>
<li>If the <em>checker</em> rejects the transaction entry, it should be returned back to <em>maker</em> (with possible comments or suggested changes). The <em>maker</em> can then resubmit changes later.</li>
<li>There can be cases when the transaction entry needs another level of checking (after the first one). This would result into three individuals necessary for completion.</li>
</ol>
<p>A typical user story for this would be something like: <q>As a <manager>, I want to apply maker-checker policy for each <transaction> being entered, so that I can prevent fraud (or improve quality).</q></p>
<p>Possible usage scenario(s) would be something like this:</p>
<p>For maker:</p>
<ol>
<li>Maker submits a transaction to the system.</li>
<li>System determines submitted transaction to be under the maker-checker policy.</li>
<li>System stores submitted transaction as "for checking".</li>
<li>System displays list of "for checking", "accepted", and "rejected" transactions.</li>
</ol>
<p>For checker:</p>
<ol>
<li>Checker retrieves list of transactions "for checking".</li>
<li>System displays list of transactions "for checking".</li>
<li>Checker selects a transaction.</li>
<li>System shows the transaction.</li>
<li>Checker <em>accepts</em> the transaction.</li>
<li>System records "accepted" transaction.</li>
</ol>
<p>The alternative flow is when the checker rejects the transaction.</p>
<ol start="5">
<li>Checker <em>rejects</em> the transaction.</li>
<li>System records "rejected" transaction.</li>
</ol>
<p>Our analysis shows that the transaction entry can have the following states:</p>
<ol>
<li>for checking,</li>
<li>verified,</li>
<li>and rejected.
</ol>
<figure class="separator" style="clear: both; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiT36PBn8X_wHAnd7w8Ivx_e2jUYHcvBbvH8JoeOQ5LbIaLGSgE02RuWgRxNCXFdseoetFWMFzPCgirrx3kbYQsDd8IWPiHhAoSuT4CPTBGwoGpowGlEV8-0mP9jLUrKvCa-bcyIco8f4Q/s1600/Statemachine+Diagram.png" /><figcaption>The <em>checker</em> can either <em>accept</em> or <em>reject</em> the entry.</figcaption></figure>
<p>In a future post, I'll share one possible design approach for maker-checker.</p>
Lorenzo Deehttp://www.blogger.com/profile/00551506284562303520noreply@blogger.com9