Wednesday, February 19, 2014

Show, Don't Just Tell

When designing software, what can we learn about user interface requirements? When using Scrum, what can product owners, and development teams, learn from users? Some organizations still rely on asking people what changes they'd like to see in their software or service. Does asking really work?

It's not what people say, it’s what they do

Gerry McGovern writes in It's not what people say, it's what they do that The worst way to design a website is to get five smart people in a room drinking lattes and posting post-it notes. […] The next worst way is to get 10 customers in a room drinking lattes and giving their opinions on the new design. That model is really, truly broken.

People Cannot Tell You What They Want

UXMyths posted Myth #21: People Can Tell You What They Want. So, if it is a myth, and people cannot tell you what they want, why do we keep asking them?

From reading Malcolm Gladwell's Blink, I can remember the failed experiments:

  • The Iyengar/Fisman study revealed that what the speed-daters say they want and what they were actually attracted to in the moment didn't match when compared.
  • The New Coke is one of the most famous research failures. Despite thousands of sip tests and countless efforts to fine-tune the taste based on the customer feedback, the New Coke was a huge disaster. "Gladwell contends that what people say they like in these tests may not reflect what they will actually buy to sit at home and drink over a week or so." See the full story on Wikipedia.

    the New Coke (Image from Wikipedia)

  • The now acclaimed Aeron office chair received very low ratings in early tests. Despite the ratings, the company decided to go on with manufacturing. The rest is history: Aeron became one of the most iconic and best selling chairs in the history of office furniture. And the irony: once the chair became famous, people started rating it much favorably.

    The Aeron chair (Image from Wikipedia)

Lean Startups

Present an idea, and ask about their pain point? Ask if they would pay X for this product? This is what some teams say they do when they try to "validate" their startup idea.

The problem here is that asking people is not a good approach. They'll say one thing, and do another. It's not that they're lying. It's just that humans are not good at speculating on future use of a product or system. Their answers are neither right nor wrong, they're just unreliable.

Scrum and Product Owners

What can product owners and development teams gain from this insight? What happens during Sprint reviews?

During sprint reviews, it is good that teams show what they've built, and have the product owner use it. That way, it's not just what the product owner says, it's more about what s/he does with it. Better yet, the product owner should find ways to get the product to actual users and see if they actually use it (e.g. download it, or click through it).

Show, Don't Just Tell

People don't know what they want until they see and use it. If you're designing a new system for people to use, have more than one design running, and have people use it. Don't ask them which one they like. Have them use both, and see what they do with it. If they keep using one version over the other, then you have a winner. (web usage analytics? A/B testing anyone?)

If you really really really have to ask, make sure you know what you're asking, and know how to interpret the results.

Sunday, November 10, 2013

Gaining Domain Knowledge of ISO-8583 Messages

Let's talk about how we improved code readability and gained domain knowledge in creating ISO 8583 messages in jPOS.

At first, we were writing code like this. And the team started asking about field 2. The code didn't help in providing much domain knowledge about financial transaction card originated messages.

import org.jpos.iso.*;

ISOMsg m = new ISOMsg();
m.setMTI("0100");
m.set(2, "...");
m.set(3, "000000"); // purchase; no account type specified
m.set(4, "000000001500"); // in acquirer's currency (e.g. USD 15.00)

So, we wanted to improve things and here's what we had in mind.

import static org.junit.Assert.*;
 
import java.util.*;
import org.junit.*;
 
import org.jpos.iso.*;

public class AuthorizationRequestBuilderTest {
  @Test
  public void test() throws Exception {
    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode("000000")
            .withTransactionAmount(...)
            .build();
    assertEquals("0100", msg.getMTI());
    assertEquals("...", msg.getString(2));
    assertEquals("000000", msg.getString(3));
    assertEquals("...", msg.getString(4));
  }
}

We wanted to use the names of the fields, instead of referring to them as field numbers. This helps improve readability and adds to the team's domain knowledge, as they now know that field number 2 is the primary account number (or PAN for short). We also applied the builder pattern and used a fluent interface.

public class AuthorizationRequestBuilder {
  …

  public AuthorizationRequestBuilder() {
  }

  public ISOMsg build() {
    ISOMsg msg = new ISOMsg();
    msg.setMTI("0100");
    msg.set(2, this.pan);
    msg.set(3, this.processingCode);
    msg.set(4, this.transactionAmount);
    return msg;
  }

  public AuthorizationRequestBuilder withPrimaryAccountNumber(String pan) {
    if (!pan.matches("[0-9]{12,19}")) {
      throw new IllegalArgumentException("PAN must be a minimum of 12 digits");
    }
    this.pan = pan;
    return this;
  }

  public AuthorizationRequestBuilder withProcessingCode(String processingCode) {
    this.processingCode = processingCode;
    return this;
  }

  public AuthorizationRequestBuilder withTransactionAmount(String transactionAmount) {
    this.transactionAmount = transactionAmount;
    return this;
  }

  …
}

Primary Account Number

As it turns out, PANs are not just a minimum of 12 (and maximum of 19) digits. It consists of three primary components:

To illustrate, say we have the following PAN, 55417710000xxxx3.

  • 554177 is the IIN
  • 10000xxxx is the individual account identification number
  • 3 is the PAN check digit

With this added knowledge, we can enhance the builder to validate the PAN.

public class AuthorizationRequestBuilder {

  …

  public AuthorizationRequestBuilder withPrimaryAccountNumber(String pan) {
    if (!pan.matches("[0-9]{12,19}")) {
      throw new IllegalArgumentException("PAN must be a minimum of 12 digits and a maximum of 19 digits");
    }
    if (!CheckDigit.isValid(pan)) {
      throw new IllegalArgumentException("PAN contains invalid check digit");
    }
    this.pan = pan;
    return this;
  }

  …
}

ISO specification 7812 and 7813 details the specific requirements for PAN composition. All PANs used in ISO 8583–1987 messages must conform to the ISO PAN encoding requirements.

Processing Code

The processing code (DE 3) contains even more knowledge to be gained. At first, we thought they were just digits. Later, we found out (thanks to the domain experts and supporting documents) that it was a series of six (6) digits used to describe the effect of a transaction on the customer account and the type of accounts affected.

These six (6) digits are composed of three (3) subfields:

  1. Cardholder Transaction Type Code
  2. Cardholder Account Type (From)
  3. Cardholder Account Type (To)

To get a better sense of what transaction types can be used in an authorization request, here are some transaction types (NOTE: Your payment network may differ. Please refer to its documents/manuals):

ValuesDescription
00Purchase
01Withdrawal
28Payment
30Balance Inquiry
40Account Transfer

Cardholder account types can have the following values:

ValuesDescription
00No account specified (NAS)/Default Account
01Savings Account
02Checking Account
03Credit Card Account

So, when a business user says, "balance inquiry on savings account", s/he means processing code 300100.

Did you get a light-bulb moment like I did when I first found out? Smile! If so, hit the comments.

Given the above domain knowledge, we initially set out to create a builder for the processing code to do something like this.


    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode(new ProcessingCode.Builder()
                .purchase()
                .from(AccountType.NOT_SPECIFIED)
                .to(AccountType.NOT_SPECIFIED)
                .build())
            .withTransactionAmount(...)
            .build();

But then, we later found out that the payment network only supports specific processing code combinations. Here are some (NOTE: Table below does not provide a complete list of valid processing codes):

ValuesDescription
000000Purchase; no account specified
001000Purchase from savings account
002000Purchase from checking account
280000Payment; No account specified
280010Payment to savings account
280020Payment to checking account
280030Payment to credit card account
300000Balance inquiry; no account specified.
When no account is specified on a balance inquiry transaction, the issuer may return both checking and savings account balances if applicable.
301000Balance inquiry on savings account
302000Balance inquiry on checking
303000Balance inquiry on credit card (credit line)

Since not all combinations (between transaction type and to-/from- account types) are valid, we thought it would be best to create a builder that helps with the creation of valid processing codes (and not just a simple string of six digits). Here's our rough idea.

public enum AccountType {
  NOT_SPECIFIED, SAVINGS, CHECKING, CREDIT_CARD
}

. . .

public class PurchaseProcessingCodeBuilder {
  public PurchaseProcessingCodeBuilder from(AccountType type) {. . .}
  // does not support a To- account
  public String build() {. . .}
}

. . .

public class PaymentProcessingCodeBuilder {
  // does not support a From- account
  public PaymentProcessingCodeBuilder to(AccountType type) {. . .}
  public String build() {. . .}
}

. . .

public class BalanceInquiryProcessingCodeBuilder {
  public BalanceInquiryProcessingCodeBuilder from(AccountType type) {. . .}
  // does not support a To- account
  public String build() {. . .}
}

. . .

    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode(new PurchaseProcessingCodeBuilder()
                // From- account type is NOT_SPECIFIED
                .build())
            .withTransactionAmount(...)
            .build();

    ISOMsg msg2 = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .withProcessingCode(new PurchaseProcessingCodeBuilder()
                // From- account type is NOT_SPECIFIED
                .to(...) // <-- results into a compiler error!
                .build())
            .withTransactionAmount(...)
            .build();

Notice that purchase transactions only support a "from" account type, but no "to" account type. Payment transactions support a "to" account type, but no "from" account type. And, balance inquiry only supports a "from" account type.

The astute reader would probably notice that in the given sample transaction types, only one account type is used (either "from" or "to"), but not both. So, you might ask, "Is there a transaction type that needs both 'from' and 'to' account type values?" Yes, there is — transfers.

Another possible idea is to create separate builders for the transaction types. Something like this,


    ISOMsg msg = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .balanceInquiry()
                // no account type is specified
                // .withTransactionAmount(...) <-- no transaction amount is needed
            .build();

    . . . = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .balanceInquiry()
                .onSavingsAccount() // or .onCheckingAccount() or .onCreditCardAccount()
            .build();

    . . . = new AuthorizationRequestBuilder()
            .withPrimaryAccountNumber(...)
            .accountTransfer()
                .fromSavingsAccount()
                .toCheckingAccount()
            .withTransactionAmount(...)
            .build();

Transaction Amount

At first, we simply thought that the transaction amount was a left-zero-padded string with two decimal places, but without the separator (i.e. decimal point). Again, after learning much more from the domain, the amount was actually based on the acquirer's currency. The sample from the document helps explain this.

DE 4 (Amount, Transaction)DE 49 (Currency Code)Currency ExponentCurrency NameActual Monetary Value of DE 4
0000000015009490New Turkish Lira1500 Lira
0000000015001242Canadian Dollar15.00 Dollars
0000000015007883Tunisian Dinar1.500 Dinars

Notice that while the transaction amount (DE 4) value is the same, it means differently based on the value of the currency (DE 49). We've used java.util.Currency#getDefaultFractionDigits() for this.

Message- vs. Domain- Centric

I consider the above ideas to be rather message-centric. After gaining more domain knowledge, I believe a domain-centric design would be of greater help. This domain-centric design would revolve around issuers, acquirers, card holders, merchants, and more. I hope to write more about this when I get some free time in the near future.

Acknowledgements

There is just so much more to learn about ISO-8583 and payment networks. One blog post is definitely not enough. Hopefully, I was able to share some of the things I've learned. Thanks to my team mates, Edge, Claire, and JC, for encouraging me to write this. I've learned so much while working with you guys.

More power to the team, and have fun learning more about the domain.

Wednesday, October 23, 2013

Long Loading Requests in App Engine for Java

In this post, I revisit the loading requests in Google App Engine for Java. The test results are shown below. The following modes are:

Skeleton
Plain-vanilla skeleton webapp with SDK (1.8.2) dependencies.
With Spring Framework Web
Skeleton webapp plus Spring Framework Web (3.2.4.RELEASE) dependencies. This adds almost 3 MB of JARs.
spring-aop-3.2.4.RELEASE.jar335455
spring-beans-3.2.4.RELEASE.jar607755
spring-context-3.2.4.RELEASE.jar863688
spring-core-3.2.4.RELEASE.jar869674
spring-expression-3.2.4.RELEASE.jar196807
spring-web-3.2.4.RELEASE.jar625875
With Spring Web MVC
Webapp with Spring Framework Web MVC. This add one additonal JAR.
spring-webmvc-3.2.4.RELEASE.jar636993
With Spring Web MVC and applicationContext
Webapp with Spring Framework Web MVC and running an empty (zero beans) application context on startup via ContextLoaderListener.
With Google Guice
Skeleton webapp with Google Guice (3.0) dependencies.
With Google Guice (one servlet module)
Webapp with Google Guice servlet context listener, filter, loading a servlet module.
With Google Guice (one servlet module)
Same as before, but built without AOP. The following dependency declaration was used:
        <dependency>
            <groupId>com.google.inject</groupId>
            <artifactId>guice</artifactId>
            <version>${guice.version}</version>
            <classifier>no_aop</classifier>
            <exclusions>
             <exclusion>
              <groupId>aopalliance</groupId>
              <artifactId>aopalliance</artifactId>
             </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.google.inject.extensions</groupId>
            <artifactId>guice-servlet</artifactId>
            <version>${guice.version}</version>
            <exclusions>
             <exclusion>
              <groupId>com.google.inject</groupId>
              <artifactId>guice</artifactId>
             </exclusion>
            </exclusions>
        </dependency>  
  

Test Results (Summarized)

Times are measured by shutting down instance, hitting a URL, looking at request time in logs. All using a frontend instance class of F1 (600MHz, 128MB).

ModeAverage Time (ms)
Skeleton4322.6
With Spring Web4836.0
With Spring Web MVC4819.0
With Spring Web MVC and applicationContext8845.6
With Google Guice (no modules)4437.6
With Google Guice (one servlet module)8260.8
With Google Guice (one servlet module, no AOP)5332.2

Observations

  • Adding some JARs (dependencies) to the web application (even without using any of its classes) increases the time it takes to complete a loading request.
  • Adding an empty (zero beans) application context to the web application increases the time by about 4 seconds.
  • Using Google Guice to load servlets and provide dependency injection can add about 4 seconds (similar to using Spring Framework).
  • Removing AOP from the use of Google Guice can lower the additional loading request time.

Test Results (Raw)

ModeTryTime (ms)
Skeleton14050
Skeleton24384
Skeleton34305
Skeleton44609
Skeleton54265
SkeletonAverage4322.6
With Spring Web15001
With Spring Web25115
With Spring Web34783
With Spring Web44724
With Spring Web54557
With Spring WebAverage4836.0
With Spring Web MVC15205
With Spring Web MVC24900
With Spring Web MVC34486
With Spring Web MVC45164
With Spring Web MVC54340
With Spring Web MVCAverage4819.0
With Spring Web MVC and applicationContext19187
With Spring Web MVC and applicationContext28946
With Spring Web MVC and applicationContext38417
With Spring Web MVC and applicationContext48777
With Spring Web MVC and applicationContext58901
With Spring Web MVC and applicationContextAverage8845.6
With Google Guice (no modules)14578
With Google Guice (no modules)24473
With Google Guice (no modules)34480
With Google Guice (no modules)44223
With Google Guice (no modules)54434
With Google Guice (no modules)Average4437.6
With Google Guice (one servlet module)17845
With Google Guice (one servlet module)28426
With Google Guice (one servlet module)38476
With Google Guice (one servlet module)48394
With Google Guice (one servlet module)58163
With Google Guice (one servlet module)Average8260.8
With Google Guice (one servlet module, no AOP)15702
With Google Guice (one servlet module, no AOP)25020
With Google Guice (one servlet module, no AOP)36036
With Google Guice (one servlet module, no AOP)45223
With Google Guice (one servlet module, no AOP)54680
With Google Guice (one servlet module, no AOP)Average5332.2

Related Links