Sunday, November 16, 2014

Maker-Checker Design Concept

We've seen the maker-checker concept pop-up several times in our software development experiences with banks. In this post, let me share a possible re-usable design approach. Thanks to Tin, Richie, Tina, Val, and their team, for adding their insights.

What is Maker-Checker?

According to Wikipedia:

Maker-checker (or Maker and Checker, or 4-Eyes) is one of the central principles of authorization in the Information Systems of financial organizations. The principle of maker and checker means that for each transaction, there must be at least two individuals necessary for its completion. While one individual may create a transaction, the other individual should be involved in confirmation/authorization of the same. Here the segregation of duties plays an important role. In this way, strict control is kept over system software and data keeping in mind functional division of labor between all classes of employees.

Here are some business rules we can derive from the above definition:

  1. For any transaction entry, there must be at least two individuals necessary for its completion.
  2. The one who makes the transaction entry (i.e. maker) cannot be the same one who checks (i.e. checker) it.
  3. A transaction entry is only considered completed if it has been checked.

Upon further clarification with the domain experts, we've learned the following:

  1. The checker cannot make modifications to the transaction entry. Modifications can only be done by maker.
  2. If the checker rejects the transaction entry, it should be returned back to maker (with possible comments or suggested changes). The maker can then resubmit changes later.
  3. There can be cases when the transaction entry needs another level of checking (after the first one). This would result into three individuals necessary for completion.

A typical user story for this would be something like: As a <manager>, I want to apply maker-checker policy for each <transaction> being entered, so that I can prevent fraud (or improve quality).

Possible usage scenario(s) would be something like this:

For maker:

  1. Maker submits a transaction to the system.
  2. System determines submitted transaction to be under the maker-checker policy.
  3. System stores submitted transaction as "for checking".
  4. System displays list of "for checking", "accepted", and "rejected" transactions.

For checker:

  1. Checker retrieves list of transactions "for checking".
  2. System displays list of transactions "for checking".
  3. Checker selects a transaction.
  4. System shows the transaction.
  5. Checker accepts the transaction.
  6. System records "accepted" transaction.

The alternative flow is when the checker rejects the transaction.

  1. Checker rejects the transaction.
  2. System records "rejected" transaction.

Our analysis shows that the transaction entry can have the following states:

  1. for checking,
  2. verified,
  3. and rejected.
The checker can either accept or reject the entry.

In a future post, I'll share one possible design approach for maker-checker.

Tuesday, November 11, 2014

SOA and Referential Data Integrity

One of the issues that tends to pop up is how we maintain referential integrity between services. In this post, I'd like to share my experiences on how referential data integrity between services can ruin your SOA.

Database Constraints Between Services Break Boundaries

Two of the Tenets of Service-Orientation are: "boundaries are explicit", and "services are autonomous". The first one implies that internal (private) implementation details should not be leaked outside of a service boundary. And the second one implies that services are not subservient to other services (or other pieces of code).

For purposes of discussion, let's say we have the following:

  • a "customer" service that provides customer-related business capability and persists data in a database
  • an "order" service that provides order-related business capability and persists data in the same database
  • foreign key constraints between customer-related entities/tables and order-related entities/tables. More specifically, an "order" table contains the unique ID of a customer that placed the order, and that this ID needs to exists in the "customer" table.

Do the above services (customer and order) follow the "boundaries are explicit" tenet? Are the services autonomous? Let's examine further.

The way the "customer" service persists data in a database is (private) implementation detail that is internal to it. Likewise, the way the "order" service persists data in a database is also internal to it. But how would you consider the foreign key constraints between their database tables? Is this internal implementation detail leaking outside of a service boundary? (i.e. leaking outside the "customer" service boundary and onto the "order" service boundary)

If the "order"-related database undergoes some schema changes, will it not affect the "customer"-related database schema? When deploying the schema changes to the "order" service, will it not require the "customer" service to be temporarily unavailable? (e.g. due to database restart) If "services are autonomous", how come the "customer" and "order" services are inter-dependent, such that a change in one requires a restart (or a redeploy) on the other?

Split Service, Split Database

A better approach would be to split the databases of the two services, and do away with foreign key constraints. That would allow for explicit boundaries, and autonomy. But this might be unacceptable to some people at first.

A monolithic application (left) split into services in a service-oriented architecture (right).

How could one ensure that enough customer information is received before orders are placed by that customer? In other words, how can a developer ensure that orders are placed by known customers (an existing customer ID)? Well, in SOA, services don't have to! It is not the responsibility of services to maintain this referential integrity. The responsibility of ensuring that an order is placed by a known customer lies with the process of placing orders (not in the service). It is the orchestration layer's responsibility to maintain this.

What if some customer information was modified (e.g. billing address), shouldn't the related orders be affected? Again, the process, or orchestration layer, can become responsible for this. Is this really a change in the customer? Or is it just a change in the placed order? One possible way is we define the process to say that it would copy the customer's billing information, and attach a copy (i.e. duplicate) with the placed order. This would mean that the order's bill-to (billing) address is defined when the order was placed. Another possible way is to leave the bill-to address undetermined until the order is shipped, at which the billing address as provided by the customer service shall be used.

Correlation Between Services and Context Boundaries

A useful way of thinking about this is the Domain-Driven Design notion of Bounded Context. DDD splits a complex domain up into multiple bounded contexts and maps out the relationships between them. And this results into multiple databases.

Re-usable Services

Services (in SOA) are meant to be reusable. In the example, if we reconsider what the customer service contains, we can probably design it in such a way that it doesn't have to be for the purposes of order taking. It can be designed to be reusable and allow for any party's information (e.g. persons, organizations) and not just customers. It could be re-used to store employees, since employees can become customers in the future. It could be re-used to store suppliers (since the business may need to track the suppliers of products being produced and ordered).

The (business) goals of SOA any IT initiative are:

  • increase agility (e.g. support new/changing business processes/models, reduce time to solution)
  • reduce cost (e.g. re-use business processes and/or applications, improve utility of existing/legacy application)

These goals are further translated as (technical goals):

  • increase usability (i.e. re-usability and accessibility across different applications)
  • improve maintainability
  • reduce redundancy

When we make the boundaries of services explicit and make them autonomous, we can better achieve the goals of re-usability, and reduce redundancy.

Some other services, that come to mind, can become re-usable (when properly designed and split), are:

  • Authentication and authorization - if this were (re-)written for each application, it would be a huge cost.
  • Billing (or invoicing)
  • Product Catalog

Microservices

In a previous post on SOA, I did mention that I find the term Microservices to be misleading. Although the information found on the web were good, I found them to still be unclear with their implementation. Nonetheless, I did find that they do add an exciting twist if you consider that, Microservices:

Closing Thoughts

Finally, when communicating with business people, don't let reuse become the primary measure. They probably won't understand it. Tell them that it does help save development and maintenance costs. Tell them that it provides better time to market, reduce days of inventory, reduce employee turnover, etc.

Nice root beer from Virgil's Sodas. Is this available here in the Philippines? I'm not related to the product in any way. I just saw their Ad online. And being thirsty, I thought of having one.

After this rather long post, I think it's time for a nice cold drink. Root beer anyone?