Sunday, November 10, 2013

Gaining Domain Knowledge of ISO-8583 Messages

Let's talk about how we improved code readability and gained domain knowledge in creating ISO 8583 messages in jPOS.

At first, we were writing code like this. And the team started asking about field 2. The code didn't help in providing much domain knowledge about financial transaction card originated messages.

import org.jpos.iso.*;

ISOMsg m = new ISOMsg();
m.setMTI("0100");
m.set(2, "...");
m.set(3, "000000"); // purchase; no account type specified
m.set(4, "000000001500"); // in acquirer's currency (e.g. USD 15.00)

So, we wanted to improve things and here's what we had in mind.

import static org.junit.Assert.*;
 
import java.util.*;
import org.junit.*;
 
import org.jpos.iso.*;

public class AuthorizationRequestBuilderTest {
  @Test
  public void test() throws Exception {
    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode("000000")
            .withTransactionAmount(...)
            .build();
    assertEquals("0100", msg.getMTI());
    assertEquals("...", msg.getString(2));
    assertEquals("000000", msg.getString(3));
    assertEquals("...", msg.getString(4));
  }
}

We wanted to use the names of the fields, instead of referring to them as field numbers. This helps improve readability and adds to the team's domain knowledge, as they now know that field number 2 is the primary account number (or PAN for short). We also applied the builder pattern and used a fluent interface.

public class AuthorizationRequestBuilder {
  …

  public AuthorizationRequestBuilder() {
  }

  public ISOMsg build() {
    ISOMsg msg = new ISOMsg();
    msg.setMTI("0100");
    msg.set(2, this.pan);
    msg.set(3, this.processingCode);
    msg.set(4, this.transactionAmount);
    return msg;
  }

  public AuthorizationRequestBuilder withPrimaryAccountNumber(String pan) {
    if (!pan.matches("[0-9]{12,19}")) {
      throw new IllegalArgumentException("PAN must be a minimum of 12 digits");
    }
    this.pan = pan;
    return this;
  }

  public AuthorizationRequestBuilder withProcessingCode(String processingCode) {
    this.processingCode = processingCode;
    return this;
  }

  public AuthorizationRequestBuilder withTransactionAmount(String transactionAmount) {
    this.transactionAmount = transactionAmount;
    return this;
  }

  …
}

Primary Account Number

As it turns out, PANs are not just a minimum of 12 (and maximum of 19) digits. It consists of three primary components:

To illustrate, say we have the following PAN, 55417710000xxxx3.

  • 554177 is the IIN
  • 10000xxxx is the individual account identification number
  • 3 is the PAN check digit

With this added knowledge, we can enhance the builder to validate the PAN.

public class AuthorizationRequestBuilder {

  …

  public AuthorizationRequestBuilder withPrimaryAccountNumber(String pan) {
    if (!pan.matches("[0-9]{12,19}")) {
      throw new IllegalArgumentException("PAN must be a minimum of 12 digits and a maximum of 19 digits");
    }
    if (!CheckDigit.isValid(pan)) {
      throw new IllegalArgumentException("PAN contains invalid check digit");
    }
    this.pan = pan;
    return this;
  }

  …
}

ISO specification 7812 and 7813 details the specific requirements for PAN composition. All PANs used in ISO 8583–1987 messages must conform to the ISO PAN encoding requirements.

Processing Code

The processing code (DE 3) contains even more knowledge to be gained. At first, we thought they were just digits. Later, we found out (thanks to the domain experts and supporting documents) that it was a series of six (6) digits used to describe the effect of a transaction on the customer account and the type of accounts affected.

These six (6) digits are composed of three (3) subfields:

  1. Cardholder Transaction Type Code
  2. Cardholder Account Type (From)
  3. Cardholder Account Type (To)

To get a better sense of what transaction types can be used in an authorization request, here are some transaction types (NOTE: Your payment network may differ. Please refer to its documents/manuals):

ValuesDescription
00Purchase
01Withdrawal
28Payment
30Balance Inquiry
40Account Transfer

Cardholder account types can have the following values:

ValuesDescription
00No account specified (NAS)/Default Account
01Savings Account
02Checking Account
03Credit Card Account

So, when a business user says, "balance inquiry on savings account", s/he means processing code 300100.

Did you get a light-bulb moment like I did when I first found out? Smile! If so, hit the comments.

Given the above domain knowledge, we initially set out to create a builder for the processing code to do something like this.


    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode(new ProcessingCode.Builder()
                .purchase()
                .from(AccountType.NOT_SPECIFIED)
                .to(AccountType.NOT_SPECIFIED)
                .build())
            .withTransactionAmount(...)
            .build();

But then, we later found out that the payment network only supports specific processing code combinations. Here are some (NOTE: Table below does not provide a complete list of valid processing codes):

ValuesDescription
000000Purchase; no account specified
001000Purchase from savings account
002000Purchase from checking account
280000Payment; No account specified
280010Payment to savings account
280020Payment to checking account
280030Payment to credit card account
300000Balance inquiry; no account specified.
When no account is specified on a balance inquiry transaction, the issuer may return both checking and savings account balances if applicable.
301000Balance inquiry on savings account
302000Balance inquiry on checking
303000Balance inquiry on credit card (credit line)

Since not all combinations (between transaction type and to-/from- account types) are valid, we thought it would be best to create a builder that helps with the creation of valid processing codes (and not just a simple string of six digits). Here's our rough idea.

public enum AccountType {
  NOT_SPECIFIED, SAVINGS, CHECKING, CREDIT_CARD
}

. . .

public class PurchaseProcessingCodeBuilder {
  public PurchaseProcessingCodeBuilder from(AccountType type) {. . .}
  // does not support a To- account
  public String build() {. . .}
}

. . .

public class PaymentProcessingCodeBuilder {
  // does not support a From- account
  public PaymentProcessingCodeBuilder to(AccountType type) {. . .}
  public String build() {. . .}
}

. . .

public class BalanceInquiryProcessingCodeBuilder {
  public BalanceInquiryProcessingCodeBuilder from(AccountType type) {. . .}
  // does not support a To- account
  public String build() {. . .}
}

. . .

    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode(new PurchaseProcessingCodeBuilder()
                // From- account type is NOT_SPECIFIED
                .build())
            .withTransactionAmount(...)
            .build();

    ISOMsg msg2 = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode(new PurchaseProcessingCodeBuilder()
                // From- account type is NOT_SPECIFIED
                .to(...) // <-- results into a compiler error!
                .build())
            .withTransactionAmount(...)
            .build();

Notice that purchase transactions only support a "from" account type, but no "to" account type. Payment transactions support a "to" account type, but no "from" account type. And, balance inquiry only supports a "from" account type.

The astute reader would probably notice that in the given sample transaction types, only one account type is used (either "from" or "to"), but not both. So, you might ask, "Is there a transaction type that needs both 'from' and 'to' account type values?" Yes, there is — transfers.

Another possible idea is to create separate builders for the transaction types. Something like this,


    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .balanceInquiry()
                // no account type is specified
                // .withTransactionAmount(...) <-- no transaction amount is needed
            .build();

    . . . = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .balanceInquiry()
                .onSavingsAccount() // or .onCheckingAccount() or .onCreditCardAccount()
            .build();

    . . . = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .accountTransfer()
                .fromSavingsAccount()
                .toCheckingAccount()
            .withTransactionAmount(...)
            .build();

Transaction Amount

At first, we simply thought that the transaction amount was a left-zero-padded string with two decimal places, but without the separator (i.e. decimal point). Again, after learning much more from the domain, the amount was actually based on the acquirer's currency. The sample from the document helps explain this.

DE 4 (Amount, Transaction)DE 49 (Currency Code)Currency ExponentCurrency NameActual Monetary Value of DE 4
0000000015009490New Turkish Lira1500 Lira
0000000015001242Canadian Dollar15.00 Dollars
0000000015007883Tunisian Dinar1.500 Dinars

Notice that while the transaction amount (DE 4) value is the same, it means differently based on the value of the currency (DE 49). We've used java.util.Currency#getDefaultFractionDigits() for this.

Message- vs. Domain- Centric

I consider the above ideas to be rather message-centric. After gaining more domain knowledge, I believe a domain-centric design would be of greater help. This domain-centric design would revolve around issuers, acquirers, card holders, merchants, and more. I hope to write more about this when I get some free time in the near future.

Acknowledgements

There is just so much more to learn about ISO-8583 and payment networks. One blog post is definitely not enough. Hopefully, I was able to share some of the things I've learned. Thanks to my team mates, Edge, Claire, and JC, for encouraging me to write this. I've learned so much while working with you guys.

More power to the team, and have fun learning more about the domain.

Wednesday, October 23, 2013

Long Loading Requests in App Engine for Java

In this post, I revisit the loading requests in Google App Engine for Java. The test results are shown below. The following modes are:

Skeleton
Plain-vanilla skeleton webapp with SDK (1.8.2) dependencies.
With Spring Framework Web
Skeleton webapp plus Spring Framework Web (3.2.4.RELEASE) dependencies. This adds almost 3 MB of JARs.
spring-aop-3.2.4.RELEASE.jar335455
spring-beans-3.2.4.RELEASE.jar607755
spring-context-3.2.4.RELEASE.jar863688
spring-core-3.2.4.RELEASE.jar869674
spring-expression-3.2.4.RELEASE.jar196807
spring-web-3.2.4.RELEASE.jar625875
With Spring Web MVC
Webapp with Spring Framework Web MVC. This add one additonal JAR.
spring-webmvc-3.2.4.RELEASE.jar636993
With Spring Web MVC and applicationContext
Webapp with Spring Framework Web MVC and running an empty (zero beans) application context on startup via ContextLoaderListener.
With Google Guice
Skeleton webapp with Google Guice (3.0) dependencies.
With Google Guice (one servlet module)
Webapp with Google Guice servlet context listener, filter, loading a servlet module.
With Google Guice (one servlet module)
Same as before, but built without AOP. The following dependency declaration was used:
        <dependency>
            <groupId>com.google.inject</groupId>
            <artifactId>guice</artifactId>
            <version>${guice.version}</version>
            <classifier>no_aop</classifier>
            <exclusions>
             <exclusion>
              <groupId>aopalliance</groupId>
              <artifactId>aopalliance</artifactId>
             </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.google.inject.extensions</groupId>
            <artifactId>guice-servlet</artifactId>
            <version>${guice.version}</version>
            <exclusions>
             <exclusion>
              <groupId>com.google.inject</groupId>
              <artifactId>guice</artifactId>
             </exclusion>
            </exclusions>
        </dependency>  
  

Test Results (Summarized)

Times are measured by shutting down instance, hitting a URL, looking at request time in logs. All using a frontend instance class of F1 (600MHz, 128MB).

ModeAverage Time (ms)
Skeleton4322.6
With Spring Web4836.0
With Spring Web MVC4819.0
With Spring Web MVC and applicationContext8845.6
With Google Guice (no modules)4437.6
With Google Guice (one servlet module)8260.8
With Google Guice (one servlet module, no AOP)5332.2

Observations

  • Adding some JARs (dependencies) to the web application (even without using any of its classes) increases the time it takes to complete a loading request.
  • Adding an empty (zero beans) application context to the web application increases the time by about 4 seconds.
  • Using Google Guice to load servlets and provide dependency injection can add about 4 seconds (similar to using Spring Framework).
  • Removing AOP from the use of Google Guice can lower the additional loading request time.

Test Results (Raw)

ModeTryTime (ms)
Skeleton14050
Skeleton24384
Skeleton34305
Skeleton44609
Skeleton54265
SkeletonAverage4322.6
With Spring Web15001
With Spring Web25115
With Spring Web34783
With Spring Web44724
With Spring Web54557
With Spring WebAverage4836.0
With Spring Web MVC15205
With Spring Web MVC24900
With Spring Web MVC34486
With Spring Web MVC45164
With Spring Web MVC54340
With Spring Web MVCAverage4819.0
With Spring Web MVC and applicationContext19187
With Spring Web MVC and applicationContext28946
With Spring Web MVC and applicationContext38417
With Spring Web MVC and applicationContext48777
With Spring Web MVC and applicationContext58901
With Spring Web MVC and applicationContextAverage8845.6
With Google Guice (no modules)14578
With Google Guice (no modules)24473
With Google Guice (no modules)34480
With Google Guice (no modules)44223
With Google Guice (no modules)54434
With Google Guice (no modules)Average4437.6
With Google Guice (one servlet module)17845
With Google Guice (one servlet module)28426
With Google Guice (one servlet module)38476
With Google Guice (one servlet module)48394
With Google Guice (one servlet module)58163
With Google Guice (one servlet module)Average8260.8
With Google Guice (one servlet module, no AOP)15702
With Google Guice (one servlet module, no AOP)25020
With Google Guice (one servlet module, no AOP)36036
With Google Guice (one servlet module, no AOP)45223
With Google Guice (one servlet module, no AOP)54680
With Google Guice (one servlet module, no AOP)Average5332.2

Related Links

Sunday, September 29, 2013

Early UI, Early Feedback, Better Results

A few months back, I was reviewing some internal apps we had in our organization, and this thing we humbly call the "library app" got my attention. We have a lot of books (physical copies) lying around, and we encourage everyone in the organization to read and apply/share what they've read. So far, it seems to be working.

A group of young developers was learning Grails, and they built this "library app". I wanted to learn too, so I pushed myself to do something that ran on Google's App Engine, and I ended up learning more.

UI and User Experience

I started looking at the current user interface of the one built using Grails. Here's what I saw — a login page.

After logging in, I get the following.

Thanks to PJ for the screenshots.

I asked myself, "What would a typical user do if s/he was greeted with this landing page?" My first response was, I wouldn't even think of clicking any of the two icons. And here's what I saw after clicking on the "Book Listing" icon/link.

There was nothing there that "entices" me to explore and look at the available titles. So, I set out to create a landing page that would make we want to look at what's available.

Early Feedback

Here's what I came up with (largely inspired by iBooks and other similar apps).

libraryapp.onb-demo.appspot.com

I didn't build the code behind it (at least, not yet). I just started with some static HTML, deployed it to the cloud, and asked some colleagues to try it out. I was pleasantly surprised with their reaction. They asked if the app was a modified version of the one built using Grails. They also asked if I got help from our creatives team, and if it was written using Grails. Anyway, the early UI allowed me to get early feedback. And letting the landing page be seen without the need to login also helped (I guess everyone is too lazy busy to login).

So, I proceeded to turn the static HTML into something more dynamic. I started populating the database with book titles. Then, it occurred to me that I was spending so much time entering the titles, authors, publishers, etc. Even though we have about 50+ titles, it was taking too much time. So, I asked around how the current version was being maintained. I was surprised that they did it pretty much the same way I did (adding titles one by one, and typing everything in while looking at the physical copy). I started thinking of improving this. (Seems like an unarticulated need here)

I saw myself Googling the book's title every time I needed to enter its details. Light bulb moment! Why don't I enhance the entry form and allow the user to Google it up, and populate the entry form?

After some rough JavaScript, here's what I ended up with.

I added a "Search" button that would do the search (using Google Books API). The results are shown, and a button is added to use the book information to populate the form.

After I added this, I was able to add titles in no time! This would not have been possible for a user to articulate or express as a requirement. How would they know that APIs exist to help with finding book information? It's critical to observe and experience what the user goes through when using your app to achieve their goals and/or perform their tasks.

I'm no design guru. This simple enhancement is far from a breakthrough. I hardly observed our office admin team use the existing library app. But I'm glad I had this experience, and I hope sharing it here would help others. I indeed learned more than just building apps!

Future Enhancements

As you can see from the screenshot, I have not implemented the borrowing part of the app (yet). But I'm thinking of ways to add a "comments" section to allow users to post their comments about the book. I'm hoping that these comments shall further "entice" others to grab a copy, read, apply, and share.