Showing posts with label maven. Show all posts
Showing posts with label maven. Show all posts

July 13, 2017

Java Bean Validation Basics

Abstract

This post summarizes some quick and easy examples for the most common things you would want to do with the Java Beans Validation API (JSR 349, JSR 303). Remember, Beans Validation is independent of Java EE. Although it is built in as part of a Java EE compliant server, the API can also be used just as easily in a Java SE application. All these examples use Java SE.

Table of Contents

  1. Basics
  2. Custom Message Template
  3. Custom Message Template with Variable Replacement
  4. Custom Property Validator
  5. Custom Class Validator
  6. GroupSequence (Short Circuit)

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Java 1.8.0_65_x64
  • NetBeans 8.2
  • Maven 3.0.5 (Bundled with NetBeans)
<dependency>
    <groupId>javax.validation</groupId>
    <artifactId>validation-api</artifactId>
    <version>1.1.0.Final</version>
</dependency>
<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-validator</artifactId>
    <version>5.1.2.Final</version>
</dependency>
<dependency>
    <groupId>javax.el</groupId>
    <artifactId>javax.el-api</artifactId>
    <version>2.2.4</version>
</dependency>
<dependency>
    <groupId>org.glassfish.web</groupId>
    <artifactId>javax.el</artifactId>
    <version>2.2.4</version>
</dependency>
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
</dependency>

Download

Visit my GitHub page https://github.com/mjremijan to see all of my open source projects. The code for this post is located at: https://github.com/mjremijan/thoth-beanvalidation

Basics

This example shows the basics of bean validation using the built-in, standard constraints and the built-in, standard validators.

Listing 1.1 - Bean to validate

package org.thoth.beanvalidation.basics;

import javax.validation.constraints.NotNull;

public class Widget {

    @NotNull
    protected String name;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

Listing 1.2 - How to validate

package org.thoth.beanvalidation.basics;

import java.util.Set;
import javax.validation.ConstraintViolation;
import javax.validation.Validation;
import javax.validation.Validator;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;

public class WidgetTest {

    protected Validator validator;

    @Before
    public void before() {
        validator = Validation.buildDefaultValidatorFactory().getValidator();
    }

    @Test
    public void violations_size() {
        // setup
        Widget w = new Widget();

        // action
        Set<ConstraintViolation<Widget>> violations
            = validator.validate(w);

        // assert
        Assert.assertEquals(1, violations.size());
    }

    @Test
    public void violation_message() {
        // setup
        Widget w = new Widget();

        // action
        Set<ConstraintViolation<Widget>> violations
            = validator.validate(w);

        // assert
        ConstraintViolation<Widget> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("may not be null", v.getMessage());
    }

    @Test
    public void violation_messageTemplate() {
        // setup
        Widget w = new Widget();

        // action
        Set<ConstraintViolation<Widget>> violations
            = validator.validate(w);

        // assert
        ConstraintViolation<Widget> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("{javax.validation.constraints.NotNull.message}", v.getMessageTemplate());
    }

    @Test
    public void violation_propertyPath() {
        // setup
        Widget w = new Widget();

        // action
        Set<ConstraintViolation<Widget>> violations
            = validator.validate(w);

        // assert
        ConstraintViolation<Widget> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("name", v.getPropertyPath().toString());
    }
}

Custom Message Template

This example shows how the built-in, standard constraints can be customized with a custom error message instead of using the built-in, standard error messages.

Listing 2.1 - ValidationMessages.properties

Candy.name.NotNull=A candy name is required.

Listing 2.2 - Bean to validate

package org.thoth.beanvalidation.custommessage;

import javax.validation.constraints.NotNull;

public class Candy {

    @NotNull(message = "{Candy.name.NotNull}")
    protected String name;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

Listing 2.3 - How to validate

package org.thoth.beanvalidation.custommessage;

import java.util.Set;
import javax.validation.ConstraintViolation;
import javax.validation.Validation;
import javax.validation.Validator;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;

public class CandyTest {

    protected static Validator validator;

    @BeforeClass
    public static void before() {
        validator = Validation.buildDefaultValidatorFactory().getValidator();
    }

    @Test
    public void notnull_violation_message() {
        // setup
        Candy candy = new Candy();

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(candy);

        // assert
        ConstraintViolation<Candy> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("A candy name is required.", v.getMessage());
    }

    @Test
    public void notnull_violation_messageTemplate() {
        // setup
        Candy candy = new Candy();

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(candy);

        // assert
        ConstraintViolation<Candy> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("{Candy.name.NotNull}", v.getMessageTemplate());
    }
}

Custom Message Template with Variable Replacement

This example shows how the built-in, standard constraints can be configured with a custom error message which has variable values in the message which are replaced by bean validation at runtime. Examples of variables which can be replaced are the actual value which was validate and the min and max properties of a @Size constraint.

Listing 3.1 - ValidationMessages.properties

Candy.name.Size.message=The candy name "${validatedValue}" is invalid. It must be between {min} and {max} characters long

Listing 3.2 - Bean to validate

package org.thoth.beanvalidation.variablereplacement;

import javax.validation.constraints.Size;

public class Candy {
    private String name;

    @Size(message = "{Candy.name.Size.message}", min=5, max=10)
    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

Listing 3.3 - How to validate

package org.thoth.beanvalidation.variablereplacement;

import java.util.Set;
import javax.validation.ConstraintViolation;
import javax.validation.Validation;
import javax.validation.Validator;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import org.thoth.beanvalidation.variablereplacement.Candy;

public class CandyTest {

    protected static Validator validator;

    @BeforeClass
    public static void before() {
        validator = Validation.buildDefaultValidatorFactory().getValidator();
    }


    @Test
    public void does_the_constraint_have_the_correct_messageTemplate() {
        // setup
        Candy candy = new Candy();
        candy.setName("");

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(candy);

        // assert
        ConstraintViolation<Candy> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("{Candy.name.Size.message}", v.getMessageTemplate());
    }

    @Test
    public void is_the_message_correct_if_size_is_too_small() {
        // setup
        Candy candy = new Candy();
        candy.setName("foo");

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(candy);

        // assert
        ConstraintViolation<Candy> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("The candy name \"foo\" is invalid. It must be between 5 and 10 characters long", v.getMessage());
    }

    @Test
    public void is_the_message_correct_if_size_is_too_big() {
        // setup
        Candy candy = new Candy();
        candy.setName("123456789|1");

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(candy);

        // assert
        ConstraintViolation<Candy> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("The candy name \"123456789|1\" is invalid. It must be between 5 and 10 characters long", v.getMessage());
    }
}

Custom Property Validator

This example shows how to create your own constraint and your own validator for a property of a class.

Listing 4.1 - ValidationMessages.properties

org.thoth.beanvalidation.propertyvalidator.Excludes.message=The value "${validatedValue}" is one of {value} which is forbidden.

Listing 4.2 - Constraint annotation

package org.thoth.beanvalidation.propertyvalidator;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import javax.validation.Constraint;

@Target({
    ElementType.TYPE, ElementType.ANNOTATION_TYPE, ElementType.METHOD, ElementType.FIELD})
@Retention(RetentionPolicy.RUNTIME)
@Constraint(validatedBy = {ExcludesValidator.class})
@Documented
public @interface Excludes {

    String message() default "{org.thoth.beanvalidation.propertyvalidator.Excludes.message}";

    Class[] groups() default {};

    Class[] payload() default {};

    String[] value() default {};
}

Listing 4.3 - Constraint validator

package org.thoth.beanvalidation.propertyvalidator;

import java.util.Arrays;
import java.util.List;
import javax.validation.ConstraintValidator;
import javax.validation.ConstraintValidatorContext;

public class ExcludesValidator
    implements ConstraintValidator< Excludes, String> {

    private List<String> excludeTheseValues;

    @Override
    public void initialize(Excludes arg) {
        String[] strarr = arg.value();
        if (strarr == null) {
            strarr = new String[]{};
        }
        excludeTheseValues = Arrays.asList(strarr);
    }

    @Override
    public boolean isValid(String value, ConstraintValidatorContext cvc) {
        if (excludeTheseValues.contains(value)) {
            return false;
        } else {
            return true;
        }
    }
}

Listing 4.4 - Bean to validate

package org.thoth.beanvalidation.propertyvalidator;

public class Candy {
    private String name;

    public Candy(String name) {
        this.name = name;
    }

    @Excludes({"foo", "bar", "shrubbery"})
    public String getName() {
        return name;
    }
}

Listing 4.5 - How to validate

package org.thoth.beanvalidation.propertyvalidator;

import java.util.Set;
import javax.validation.ConstraintViolation;
import javax.validation.Validation;
import javax.validation.Validator;
import static org.junit.Assert.assertEquals;
import org.junit.BeforeClass;
import org.junit.Test;

public class CandyTest {

    protected static Validator validator;

    @BeforeClass
    public static void before() {
        validator = Validation.buildDefaultValidatorFactory().getValidator();
    }


    @Test
    public void a_non_excludeded_name_should_not_give_you_a_constraint_violation() {
        // setup
        Candy candy = new Candy("hershey");

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(candy);

        // assert
        assertEquals(0, violations.size());
    }


    @Test
    public void do_you_get_a_constraint_violation_if_you_use_excluded_name_foo() {
        // setup
        Candy candy = new Candy("foo");

        // action
        ConstraintViolation<Candy> violation
            = validator.validate(candy).iterator().next();

        // assert
        assertEquals("{org.thoth.beanvalidation.propertyvalidator.Excludes.message}", violation.getMessageTemplate());
        assertEquals("The value \"foo\" is one of [foo, bar, shrubbery] which is forbidden.", violation.getMessage());
    }


    @Test
    public void do_you_get_a_constraint_violation_if_you_use_excluded_name_bar() {
        // setup
        Candy candy = new Candy("bar");

        // action
        ConstraintViolation<Candy> violation
            = validator.validate(candy).iterator().next();

        // assert
        assertEquals("{org.thoth.beanvalidation.propertyvalidator.Excludes.message}", violation.getMessageTemplate());
        assertEquals("The value \"bar\" is one of [foo, bar, shrubbery] which is forbidden.", violation.getMessage());
    }


    @Test
    public void do_you_get_a_constraint_violation_if_you_use_excluded_name_shrubbery() {
        // setup
        Candy candy = new Candy("shrubbery");

        // action
        ConstraintViolation<Candy> violation
            = validator.validate(candy).iterator().next();

        // assert
        assertEquals("{org.thoth.beanvalidation.propertyvalidator.Excludes.message}", violation.getMessageTemplate());
        assertEquals("The value \"shrubbery\" is one of [foo, bar, shrubbery] which is forbidden.", violation.getMessage());
    }
}

Custom Class Validator

This example shows how to create your own constraint and your own validator which applies to an entire class.

Listing 5.1 - ValidationMessages.properties

org.thoth.beanvalidation.classvalidator.IdentificationExists.message=At least one of social security number, drivers license number, or passport number must exist.

Listing 5.2 - Constraint annotation

package org.thoth.beanvalidation.classvalidator;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import javax.validation.Constraint;
import javax.validation.Payload;

@Target({ElementType.TYPE, ElementType.ANNOTATION_TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Constraint(validatedBy = {IdentificationExistsValidator.class})
@Documented
public @interface IdentificationExists {

    String message() default "{org.thoth.beanvalidation.classvalidator.IdentificationExists.message}";

    Class<?>[] groups() default {};

    Class<? extends Payload>[] payload() default {};
}

Listing 5.3 - Constraint validator

package org.thoth.beanvalidation.classvalidator;

import java.util.Objects;
import javax.validation.ConstraintValidator;
import javax.validation.ConstraintValidatorContext;

public class IdentificationExistsValidator implements ConstraintValidator<IdentificationExists, Identification> {

    @Override
    public void initialize(IdentificationExists a) {}

    @Override
    public boolean isValid(Identification t, ConstraintValidatorContext cvc) {
        boolean invalid =
            Objects.equals(t.getDriversLicenseNumber(), null)
            &&
            Objects.equals(t.getPassportNumber(), null)
            &&
            Objects.equals(t.getSocialSecurityNumber(), null)
        ;
        return !invalid;
    }
}

Listing 5.4 - Bean to validate

package org.thoth.beanvalidation.classvalidator;

@IdentificationExists
public class Identification {
    protected String socialSecurityNumber;
    protected String driversLicenseNumber;
    protected String passportNumber;

    public String getSocialSecurityNumber() {
        return socialSecurityNumber;
    }

    public void setSocialSecurityNumber(String socialSecurityNumber) {
        this.socialSecurityNumber = socialSecurityNumber;
    }

    public String getDriversLicenseNumber() {
        return driversLicenseNumber;
    }

    public void setDriversLicenseNumber(String driversLicenseNumber) {
        this.driversLicenseNumber = driversLicenseNumber;
    }

    public String getPassportNumber() {
        return passportNumber;
    }

    public void setPassportNumber(String passportNumber) {
        this.passportNumber = passportNumber;
    }
}

Listing 5.5 - How to validate

package org.thoth.beanvalidation.classvalidator;

import java.util.Set;
import javax.validation.ConstraintViolation;
import javax.validation.Validation;
import javax.validation.Validator;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;

public class IdentificationTest {

protected Validator validator;

    @Before
    public void before() {
        validator = Validation.buildDefaultValidatorFactory().getValidator();
    }

    @Test
    public void violation_if_all_are_missing() {
        // setup
        Identification id = new Identification();

        // action
        Set<ConstraintViolation<Identification>> violations
            = validator.validate(id);

        // assert
        ConstraintViolation<Identification> v
            = violations.stream().findFirst().get();
        Assert.assertEquals("At least one of social security number, drivers license number, or passport number must exist.", v.getMessage());
    }

    @Test
    public void no_violation_if_social_security_number_exists() {
        // setup
        Identification id = new Identification();
        id.setSocialSecurityNumber("a");

        // action
        Set<ConstraintViolation<Identification>> violations
            = validator.validate(id);

        // assert
        Assert.assertEquals(0, violations.size());
    }

    @Test
    public void no_violation_if_drivers_license_number_exists() {
        // setup
        Identification id = new Identification();
        id.setDriversLicenseNumber("a");

        // action
        Set<ConstraintViolation<Identification>> violations
            = validator.validate(id);

        // assert
        Assert.assertEquals(0, violations.size());
    }

    @Test
    public void no_violation_if_passport_number_exists() {
        // setup
        Identification id = new Identification();
        id.setPassportNumber("a");

        // action
        Set<ConstraintViolation<Identification>> violations
            = validator.validate(id);

        // assert
        Assert.assertEquals(0, violations.size());
    }
}

GroupSequence (Short Circuit)

This example shows how to use @GroupSequence as a short circuit when doing validation. This means if the 1st round of validations do not pass, then validation is “short circuited” and the 2nd round of validations is not performed.

By default, all bean validation constraints are put into a “Default” group sequence. However, by putting a @GroupSequence on a class (like shown below) the “Default” group sequence is redefined just for that class. With the @GroupSequence on a class below, what it basically does is that during beans validation the 1st operation is to validate all constraints in the class that aren’t specifically assigned a group. That would be the @NotNull constraint first. If all of those are OK, then the 2nd operation is to validate all constraints that are in the Second.class group. That would be the @Size constraint. If all of those are OK, then 3rd operation is to validate all of the constraints that are in the Third.class group. That would be the @Pattern constraint. If at any time a group fails to validate, validation is “short circuited” and validation goes no farther.

Listing 6.1 - Bean to validate

package org.thoth.beanvalidation.groupsequence;

import javax.validation.GroupSequence;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Pattern;
import javax.validation.constraints.Size;

@GroupSequence({Candy.class, Candy.Second.class, Candy.Third.class})
public class Candy {

    protected interface Second {}
    protected interface Third {}

    private String name;

    @NotNull()
    @Size(min=4, max=10, groups = Second.class )
    @Pattern(regexp = "[a-z]", groups = Third.class)
    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

Listing 6.2 - How to validate

package org.thoth.beanvalidation.groupsequence;

import java.util.Set;
import javax.validation.ConstraintViolation;
import javax.validation.Validation;
import javax.validation.Validator;
import static org.junit.Assert.assertEquals;
import org.junit.Before;
import org.junit.Test;

public class CandyTest {

    private Validator validator;

    @Before
    public void before() {
        validator = Validation.buildDefaultValidatorFactory().getValidator();
    }

    @Test
    public void short_circuits_first_if_null() {
        // setup
        Candy w = new Candy();

        // action
        Set<ConstraintViolation<Candy>> violations
            //= validator.validate(w, CheckGroupSequence.class);
            = validator.validate(w);

        // assert
        assertEquals(1, violations.size());
        assertEquals("may not be null", violations.iterator().next().getMessage());
    }


    @Test
    public void short_circut_if_size_is_in_violation() {
        // setup
        Candy w = new Candy();
        w.setName("foo");

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(w);

        // assert
        assertEquals(1, violations.size());
        assertEquals("size must be between 4 and 10", violations.iterator().next().getMessage());
    }


    @Test
    public void short_circuit_if_pattern_is_in_violation() {
        // setup
        Candy w = new Candy();
        w.setName("SHRUBBERY");

        // action
        Set<ConstraintViolation<Candy>> violations
            = validator.validate(w);

        // assert
        assertEquals(1, violations.size());
        assertEquals("must match \"[a-z]\"", violations.iterator().next().getMessage());
    }
}

Summary

Beans validation is a powerful API, especially since it can be used within a Java EE server or in stand-alone Java SE applications. This is just a very short summary of the basics of the beans validation API, but, typically, it is enough to cover most questions developers have about how to use it.

March 19, 2014

Integration Testing with Maven and an In-Memory Derby Database

Abstract
You have a database-driven application. Your project contains a lot of SQL and you want to test it. Following strict unit testing guidelines you have mocked your data access up to this point (to avoid access to an external resource) and have verified your business rules.  Now you are ready to move to the next step - integration testing and verifying the SQL itself. But how? And more importantly, how to do it quickly and easily - i.e. not relying on an external database server? The purpose of this article is to explore quick and easy integration testing with Maven and an in-memory Derby database.

System Requirements
The code was developed and run using the following system setup. If yours is different, it may work but no guarantees.
  • JDK 1.7.0_17
  • NetBeans 7.4
  • Maven 3.0.5 (bundled with NetBeans)
  • Derby 10.10.1.1
The rest of the software dependencies can be found in the project's pom.xml

Download Project
If you just want to look at the project demonstrating the solution, download the code example from GitHub (https://github.com/mjremijan/MavenIntegrationTestingWithDerby).

Run Project
The project is a standard Maven setup.  Simply execute mvn from the command line after changing to the MavenIntegrationTestingWithDerby directory. Or, within NetBeans, right-click on the project and select "Clean and Build". Using NetBeans is probably easiest.

The project contains Maven integration tests which are NOT run during the unit testing phase of the Maven lifecycle. So executing mvn test won't run the integration test examples!

Configure Integration Testing
The Maven world has two kinds of testing: unit-testing and integration-testing.  Both are driven off the Maven execution life cycle.  Unit-testing you get for free, but integration-testing is a bit more complicated.  Let's learn a little more about each one.

We'll first talk a bit about unit-testing because it is probably something you are already familiar with and we'll use this familiarity to springboard into integration-testing. Unit testing is performed by the maven-surefire-plugin during the test phase of the  life cycle. There is no configuration needed to run simple unit tests. Simply create a src/test/main/**/*Test.java class and Maven will automatically execute it for you. It's nice that unit-testing works so easily, and you'd think it would be the same for integration-testing, but it's not.

Now we'll talk a bit about integration-testing. Integration testing is performed by the maven-failsafe-plugin during the integration-test phase of the life cycle. For integration tests, the default file matching name patterns are as follows:
  1. src/test/main/**/IT*.java
  2. src/test/main/**/*IT.java
  3. src/test/main/**/*ITCase.java  
As you can see, these file name patterns are different than they are for unit tests. The most popular pattern is *IT.java since it most closely matches its unit test counterpart *Test.java. You annotate the *IT.java with the same @Test annotations you would use for a unit test.  In fact the test itself is written almost exactly as if it were a unit test. However the difference is that integration tests use real external resources more and rely on mock objects less.  

After creating *IT.java integration tests you may run your Maven default life cycle and expect the integration tests to run. Your expectations are wrong. To get integration tests to run the maven-failsafe-plugin must be configured in your project's pom.xml.  Configuring the plugin is easy. Listing 1 shows you how to add the plugin to your project's.

Listing 1: Add maven-failsafe-plugin to project's build
<project><build><plugins>...
<!-- 
    The maven-failsafe-plugin is used to run integration tests.
    The tests are part of /src/main/test and should end with 
    *IT.java
-->
<plugin>
    <groupid>org.apache.maven.plugins</groupid>
    <artifactid>maven-failsafe-plugin</artifactid>
    <version>2.12.4</version>
    <executions>
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <skiptests>${skip.integration.tests}</skiptests>
            </configuration>
        </execution>
    </executions>
</plugin>
</plugins></build></project>


In this article's project (download), I have two integration test examples.  The first is SomeTableFindExistingIT.java.  This integration test will attempt to find existing data in the database.  The second is SomeTableInsertAndFindIT.java.  This integration test will attempt to insert data into a database then query to find it. Now that I have my *IT.java files and I have the maven-failsafe-plugin configured in the pom.xml, the integration tests are ready to run right?  Well not quite yet.

Remember, integration tests partially rely on external resources.  For these example integration tests, a database is needed. So the next configuration step I need to make is to get Maven to automatically start and stop a database and fill it with data for my integration tests to use.  Let's take a look at that next.

Configure Derby
Derby is a very interesting project with a history with some twists and turns. Accoring to Wikipedia, Derby started its life with Cloudscape, Inc. The first release of the project was called JBMS.  After this first release the project was renamed and released as Cloudscape.  After a few company acquisitions, the project eventually ended up at IBM but still continued to be called Cloudscape. IBM eventually gave Cloudscape to Apache.  At this point it changed names to Derby. Then, just to make things even more confusing, Sun (now Oracle) incorporated the project into Java 6 and refered to it as Java DB.  So Cloudscape, Derby, Java DB...all the same name for basically the same thing.

We are going to use Derby for our integration tests because the project supports common SQL syntax and it can be run in-memory without the need for a separate database server.  Maven can start the database, run the integration tests, and stop the database.  Nice, clean, and easy.


To do this, we need to add two configurations to the project's pom.xml. Listing 2 shows the first, which are the dependencies to get the Derby JARs from the Maven repo.

Listing 2: Dependencies to get Derby JARs
<project>...
<dependency>
    <groupId>org.apache.derby</groupId>
    <artifactId>derby</artifactId>
    <version>10.10.1.1</version>
</dependency>
<dependency>
    <groupId>org.apache.derby</groupId>
    <artifactId>derbyclient</artifactId>
    <version>10.10.1.1</version>
</dependency>

Next we need to add a plugin to the Maven build.  Listing 3 shows this inmemdb-maven-plugin which is used to control Derby during integration testing.

Listing 3: The inmemdb-maven-plugin
<project><build><plugins>...
<!-- 
    This plugin will take care of starting a Derby database
    and creating the tables/data before integration testing
    begins. It will then cleanup everything afterwards
-->        
<plugin>
    <groupId>com.btmatthews.maven.plugins.inmemdb</groupId>
    <artifactId>inmemdb-maven-plugin</artifactId>
    <version>1.4.2</version>
    <configuration>
        <monitorKey>inmemdb</monitorKey>
        <monitorPort>11527</monitorPort>
    </configuration>
    <executions>
        <execution>
            <id>run</id>
            <goals>
                <goal>run</goal>
            </goals>
            <phase>pre-integration-test</phase>
            <configuration>
                <daemon>true</daemon>
                <type>derby</type>
                <database>${ferris.property.test.data.db}</database>
                <username>${ferris.property.test.data.user}</username>
                <password>${ferris.property.test.data.pass}</password>
                <sources>
                    <script>
                        <sourceFile>src/test/resources/sql/test-data-derby.sql</sourceFile>
                    </script>                               
                </sources>
            </configuration>
        </execution>
        <execution>
            <id>stop</id>
            <goals>
                <goal>stop</goal>
            </goals>
            <phase>post-integration-test</phase>
        </execution>
    </executions>  
</plugin>

Let's take a look at this plugin in more detail.

1st, a big shout out to Brian Matthews for creating this plugin and making it available.

2nd, <phase>pre-integration-test</phase>: This execution is tied to the pre-integration-test phase of the Maven execution life cycle.  This basically means that before Maven runs the integration tests, it will first start an in-memory Derby database instance.

3rd, <database>, <username>, and <password>: These values are all properties defined in the pom.xml. It is important to have this data extracted as properties because we will need to reuse the values later when configuring JPA to connect to the database.  Instead of having duplicate data, put the data in properties.

4th, src/test/resources/sql/test-data-derby.sql: This is the SQL script which creates our tables and fills them with data.  Depending on the complexity of your database, you may want to split this into multiple files.  Tying an SQL script file to a specific integration test is a good idea once your project gets large and has a lot of integration tests. Doing so will hopefully prevent weird dependencies between integration tests and SQL scripts.

5th, <phase>post-integration-test</phase>: This execution is tied to the post-integration-test phase of the Maven execution life cycle. So this basically means once the integration tests are over, Maven will stop the in-memory Derby database instance.

So with some Maven dependencies the inmemdb-maven-plugin plugin, some Maven properties, and an SQL script, we have configured Maven to automatically start, build, and stop a database for the integration tests. Let's next take a closer look at the SQL script and see how the database is built.


Configure Data
In the previous section we looked at the inmemdb-maven-plugin plugin to get the Derby database up and running and loaded with data before our integration tests run. This is a great convenience; We don't have to worry about doing this all in the integration test itself.  If you have only one test it's not that bad, however if you have dozens it can get very tricky having the tests do this database work.  It is better to have Maven take care of it before the integration test start.  Let's see how Maven does this.

The src/test/resources/sql/test-data-derby.sql file is in this article's project (download).  Listing 4 shows this file.

Listing 4:SQL to build the database
-- Create the tables
CREATE TABLE app_sometable
(
   some_varchar varchar(100), 
   some_numeric numeric, 
   some_timestamp timestamp, 
   some_blob blob  
);
 
-- Insert the data
insert into app_sometable (some_varchar, some_numeric, some_timestamp, some_blob)
values ('unit_test_varchar',123,'1977-01-30-10.11.30.766',null);

The SQL script is very simple.  It creates a table named app_sometable with columns of various commonly used types.  Then it inserts a row of test data into the table.  This row can then be used in the integration test to verify that selecting data from the table is working properly.

A final point to make about the SQL is that it is not production ready.  It has no primary key, no indices and no permissions.  When integration testing, you typically will not use the full-fledged production SQL scripts.  Integration tests verify the basic communication between resources is working, so in this case the SQL doesn't need to get too fancy.

Now that we have our database and our data, the next thing to do is figure out how the code is going to connect and get it.  For that, we'll need to configure JPA.

Configure JPA
Java Persistence API (JPA) is the Java standard for object-relational mapping (ORM) to turn relational database data into Java objects. The persistence.xml file is used to configure JPA.  Section 8.2.1 of the JSR 338 specification defines that the persistence.xml file is to be located off the root of the JAR resource as /META-INF/persistence.xml.  Translating this into a Maven integration test, that means we need to put the persistence.xml file in the following location:

src/test/resources/META-INF/persistence.xml

the src/test/resources directory is the standard Maven directory for test resources and META-INF/persistence.xml is where the JPA specification says to put this file.  Now, let's take a look listing 5 and see what's in our integration test persistence.xml.

Listing 5: Integration test persistence.xml
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="1.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd">
  <persistence-unit name="MavenIntegrationTestingWithDerbyPU" transaction-type="RESOURCE_LOCAL">
    <provider>org.hibernate.ejb.HibernatePersistence</provider>
    <class>org.ferris.mavenintegrationtestingwithderby.SomeTable</class>
    <properties>
      <property name="hibernate.connection.username"        value="${ferris.property.test.data.user}"/>
      <property name="hibernate.connection.password"        value="${ferris.property.test.data.pass}"/>
      <property name="hibernate.connection.url"             value="jdbc:derby://localhost/memory:${ferris.property.test.data.db}"/>
      <property name="hibernate.connection.driver_class"    value="${ferris.property.test.data.driver}"/>
      <property name="hibernate.dialect"                    value="${ferris.property.test.data.dialect}"/>
      <property name="hibernate.show_sql"                   value="true"/>
      <property name="hibernate.cache.provider_class"       value="org.hibernate.cache.NoCacheProvider"/>      
    </properties>
  </persistence-unit>
</persistence>

In listing 5, there are a couple things to comment on.

1st MavenIntegrationTestingWithDerbyPU: This is the name of the persistence unit.  We'll need to remember this when we get to coding the integration tests themselves. 

2nd: Remember the Maven properties when setting up the inmemdb-maven-plugin plugin?  Well here they are again, with a few more added to get the integration test persistence.xml configured property.

Although JPA needs persistence.xml, you will need to do one more thing to get the integration tests running properly. What you need to do is make sure Maven does a search and replace of all the properties in persistence.xml. This is easy do do by configuring a resource as showing in listing 6.

Listing 6: Configure resource so Maven performs search & replace of properties
<project><build>...
<!--
  Make sure resources are filtered to get the properties
-->
<testResources>
 <testResource>
  <directory>src/test/resources</directory>
  <filtering>true</filtering>
 </testResource>
</testResources>

This tells Maven to do the search and replace of all files it finds in the src/test/resources directory when it is copying those resources to the /build directory during the unit- and integration-testing phases of the Maven life cycle. 

At this point we have all of the configuration done.  We have configured Maven with the maven-failsafe-plugin plugin to run integration tests during the Maven life cycle. Next we used the inmemdb-maven-plugin plugin to start and build and in-memory Derby database before the integration tests start and to stop the database after the tests are finished.  Then we looked at JPA and made sure the persistence.xml file is configured correctly for the integration tests.  The final thing to do is to create an integration test.

Writing your integration tests
This article's project (download) has two integration tests. Listing 7 shows SomeTableFindExistingIT.java.

Listing 7: The SomeTableFindExistingIT.java integration test
package org.ferris.mavenintegrationtestingwithderby;

import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import org.apache.log4j.Logger;
import static org.junit.Assert.*;
import org.junit.Before;
import org.junit.Test;

public class SomeTableFindExistingIT 
{
    private static Logger log = Logger.getLogger(SomeTableFindExistingIT.class);
    private EntityManager em;

    @Before
    public void setEntityManager() throws Exception 
    {
        try {
            EntityManagerFactory emf
                    = Persistence.createEntityManagerFactory("MavenIntegrationTestingWithDerbyPU");
            em = emf.createEntityManager();
        } catch (Exception e) {
            log.fatal("Cannot create EntityManager", e);
            throw e;
        }
    }
    
    @Test
    public void findSomeTableData() throws Exception 
    {
        String someVarchar = "unit_test_varchar";   
        Long someNumeric = new Long(123);
        Timestamp someTimestamp = new Timestamp(new SimpleDateFormat("yyyy-MM-dd-hh.mm.ss.SSS").parse("1977-01-30-10.11.30.766").getTime());
        
        SomeTable bean = em.find(SomeTable.class, someVarchar);
        assertNotNull(
              String.format("Not found! someVarchar=\"%s\"",someVarchar)
            , bean
        );
        assertEquals(someVarchar, bean.getSomeVarchar());
        assertEquals(someNumeric, bean.getSomeNumeric());
        assertEquals(someTimestamp, bean.getSomeTimestamp());        
    }
}

Let's take a look at this integration test a bit more closely.

1st setEntityManager(): This @Before annotated method will create an EntityManager for the integration test.  Notice the persistence unit name is "MavenIntegrationTestingWithDerbyPU", which is the same as in the persistence.xml file.  It's also important to mention this is a real EntityManager attached to a real Derby database.  No mocks here.

2nd findSomeTableData(): This @Test method sets the following variables: someVarchar, someNumeric, someTimestamp.  It uses them to both find and assert the data in the SomeTable object matches the data in the database. Again, no mocks here. The data in SomeTable comes from the app_sometable table in the Derby database which was pre-populated with data by Maven before the integration test are run.

Now that the integration test is written, we just need to run it and make sure it works. 

Running your integration tests
Integration tests are part of the Maven life cycle and are run during the integration-test phase. Table 1 shows the Maven default life cycle.  Notice the integration-test phase is much later in the life cycle than the test phase. This means if you execute $mvn test you will run your unit tests but not your integration tests!  Eclipse and NetBeans both have GUI shortcuts for testing the project but you must be aware those shortcuts only run $mvn test and will not include the integration tests.  To completely run your integration tests, you must execute $mvn verify or create your own GUI shortcut to do so.

Table 1: The Maven lifecycle

validate
initialize
generate-sources
process-sources
generate-resources
process-resources
compile
process-classes
generate-test-sources
process-test-sources
generate-test-resources
process-test-resources
test-compile
process-test-classes
test
prepare-package
package
pre-integration-test
integration-test
post-integration-test
verify
install
deploy

Conclusion
Running unit tests with Maven is easy.  Running integration tests take a bit more work.  But I guess this makes sense.  After all, an application may integrate with half a dozen system and trying to get them all configured and up and running for an automated integration test can be quite a challenge. But if you just want to connect to a database and run a few queries, I hope this article helped.

References




stop

January 27, 2014

Resolving SQLGrammarException using JPA @ManyToOne and @OneToMany Relationships

Abstract
So let's say your database has a simple parent-child relationship such as the relationship between a user and their addresses.  A single user can have many addresses so it is a one-to-many relationship from the point of view of the user; a many-to-one relationship from the point of view of the address.  You model this with JPA entities as follows:

Listing 1: User entity with collection of Address
@Entity
@Table(name = "app_user")
public class User implements Serializable {
    @EmbeddedId
    UserKey key;

    @OneToMany(mappedBy="user", fetch=FetchType.EAGER, cascade=CascadeType.ALL)
    Collection<Address> addresses;
}

In listing 1, you can see the @OneToMany annotation is used in the User class (#2) because from the point of view of a user, a user can have multiple addresses .  Recall this annotation is used on the owner of the relationship.

Listing 2: Address entity with parent User
@Entity
@Table(name = "app_address")
public class Address implements Serializable {
@EmbeddedId
    AddressKey key;

    @ManyToOne
    private User user;
}

In listing 2, the Address entity uses the @ManyToOne annotation (#2) because from the address point of view, an address can only have one user. That's it, real simple JPA config.

So after coding this up, you start your integration tests.  A test attempts to select data from the database and build a User/Address object tree. Immediately you run into a problem. The SQL generated is not valid and you get a SQLGrammarException complaining about some column which does not exist. listing 3 shows some of what the stack trace may say:

Listing 3: SQLGrammarException stack trace
javax.persistence.PersistenceException: org.hibernate.exception.SQLGrammarException: could not load an entity: 
  .
  .

  Caused by: org.hibernate.exception.SQLGrammarException: could not
  load an entity: [org.ferris.resolving.sqlgrammarexception.User#
  .
  .
  Caused by: java.sql.SQLSyntaxErrorException: Column
  'ADDRESSES1_.USER_BIRTHDAY' is either not in any table in the 
  FROM list or appears within a join specification and is outside
  the scope of the join specification or appears in a HAVING clause
  and is not in the GROUP BY list.

You look at this stack trace, you see Column 'ADDRESSES1_.USER_BIRTHDAY', and the $10,000 question is where is JPA getting this column? It doesn't exist in your database and none of your entities reference a column with this name.  You wired up your JPA entities with annotations in the exact way the book tells you to, so everything should work. Isn't that right?  The purpose of this article is to explore why you get this SQLGrammarException and how to resolve it.

System Requirements
The code was developed and run using the following system setup. If yours is different, it may work but no guarantees.
  • JDK 1.7.0_17
  • NetBeans 7.3.1
  • Maven 3.0.5 (bundled with NetBeans)
The rest of the software dependencies can be found in the project's pom.xml

Download Project
If you just want to look at the project demonstrating the solution, download the code example from GitHub (https://github.com/mjremijan/JPATest).

Run Project
The project is a standard Maven setup.  Simply execute mvn. The project contains Maven integration tests which are NOT run during the unit testing phase of the Maven lifecycle. So executing mvn test won't run the example.

The code is configure so the integration tests will pass and you will not get the SQLGrammerException. You can mess around with the JPA annotations on your own an explore all the wonderful exceptions you can get.

Why the SQLGrammerException ?
To understand why you may be getting an SQLGrammarException, you really need to understand the difference between artificial and natural primary and foreign keys for database tables. It's in the keys that the problem lies.  So let's take a look at these keys.

When studying JPA, you learn the @ManyToOne and @OneToMany annotations can be used to establish a bi-directional relationship between entities. Typically, examples demonstrating how to code this relationship with JPA use very simple artificial primary and foreign keys. As a reminder, an artificial primary key is a column added to a database table which only exists to make rows in the table unique and as such is completely independent of the data stored in the table.  Artificial keys are typically Java long primitive values. Each table in the relationship typically has its own artificial key.  An artificial key relationship is shown in figure 1.

Figure 1: An artificial key database relationship 
Database relationships are sometimes this simple, but not often. This is especially true with legacy systems or any system which favors a natural primary and foreign keys.  As a reminder, a natural primary key specifies the column (or the combination of columns) of the data which make a row unique. A natural key relationship is shown in figure 2.

 Figure 2: A natural key database relationship
Natural keys are almost always a combination of columns because it's the combination of data that's needed to make the row unique. Using natural keys with multiple columns in JPA is not that difficult, but there are some challenges. If you don't do it right, you will get the SQLGrammarException

Now that you got a quick review of artificial and natural keys, let's take a look at how this is contributing to our problem. First look at the error message again.

Column 'ADDRESSES1_.USER_BIRTHDAY' is either not in any table. . .

ADDRESSES1 indicates the problem is with the Address entity. So that makes sense. Now referring back to listing 2, the Address entity has a relationship to its user by embedding the User entity like so:

    @ManyToOne
    private User user;
 
So the USER part of the error now kinda makes sense, but now what about BIRTHDAY? Well refer back to listing 1 and take a look at the properties of the User entity. It has an embedded UserKey as its primary key. Listing 4 shows UserKey.

Listing 4: UserKey is the primary key of  the User entity
@Embeddable
public class UserKey implements Serializable 
{
    private static final long serialVersionUID = 131374817387370831L;
    
    @Column(name = "last_name")
    private String lastName;

    @Column(name = "zip_code")
    private Long zipCode;

    @Column(name = "birthday")
    private Timestamp birthday;

    // getters and setters omitted for brevity

There is is! Inside UserKey is the missing BIRTHDAY. So now that it looks like we've identified all of the parts of the exception, let's walk though what's going on.

Address is an entity managed by JPA. As such, JPA scans through all the properties of Address looking for non-@Transient properties to manage. It finds the private User user property. Since User it is an object and not a simple primative, JPA uses make an intelligent guess about the column name for this property and assumes it starts with "user". Then JPA scans the User entity. In it, JPA finds @EmbeddedId UserKey userKey. Because UserKey is an embedded object, JPA now scans it for fields to manage and finds birthday. Putting this all together in hibernate-generated SQL you get ADDRESSES1_.USER_BIRTHDAY. Not very pretty, so what do we do about it?  We the solution is actually very simple.

Solution
To solve this problem, you need to configure JPA with the relationship between the entities. Recall the relationship between the user and address data is a complicated natural foreign key relationship. Figure 2 shows the primary key of App_User is a combination of LAST_NAME, ZIP, and BIRTHDAY.  For App_Address, these three columns are repeated with an addition of a fourth column to make the combination of LAST_NAME, ZIP, BIRTHDAY, and ADDRESS the primary key for App_Address. App_Address also uses its LAST_NAME, ZIP, and BIRTHDAY columns as a foreign key back to App_User. Though this may sound complicated, JPA annotations make it very easy. Listing 5 shows how to define the relationship in the Address entity.


Listing 5: Address relationship with its user
@ManyToOne(optional=false, fetch=FetchType.EAGER, cascade=CascadeType.ALL)
@JoinColumns({
    @JoinColumn(name="last_name", referencedColumnName="last_name", insertable=false, updatable=false),
    @JoinColumn(name="zip_code",  referencedColumnName="zip_code",  insertable=false, updatable=false),
    @JoinColumn(name="birthday",  referencedColumnName="birthday",  insertable=false, updatable=false)
})    
private User user;

In listing 5 @JoinColumn has been introduced (#3). The name property refers to the name of the column in the App_Address table.  The referencedColumnName propery refers to the name of the column in the App_User table. The insertable and updatable properties are self explanatory but critically important so JPA does not duplicate the columns when generating the INSERT and UPDATE statements. And that's it, very simple.