August 23, 2018

Multiple Derby Network Servers on the same Host

Abstract

Suppose you want to start multiple Derby network servers on the same host. They need to be listening on different ports and ideally store their data in different locations. The listings below show Linux bash and Windows batch scripts to configure starting a Derby network server. In this example, the Derby network server will listen on port 1110. Each Derby network server will also have its own file system location to store its databases.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Apache Derby 10.14.1.0
  • Java zulu11.1+23-ea-jdk11

I am not going to go through the process of downloading and installing these technologies. I’ll leave that as an exercise for you.

Linux bash scripts

Here are the Linux bash scripts to configure starting a Derby network server. We’ll look at the details of each file. First we’ll set the environment.

Listing 1 - setenv.sh

#!/bin/bash
 
export DERBY_HOME=/home/derby/opt/derby
export PATH="$DERBY_HOME/bin:$PATH"
echo "DERBY_HOME=$DERBY_HOME"

export JAVA_HOME=/home/derby/opt/java
echo "JAVA_HOME=$JAVA_HOME"

export NS_HOME=/var/local/derby/1110
mkdir -p $NS_HOME
echo "NS_HOME=$NS_HOME"

export NS_PORT=1110
echo "NS_PORT=$NS_PORT"

export NS_HOST=0.0.0.0
echo "NS_HOST=$NS_HOST"
 
export DERBY_OPTS=""
export DERBY_OPTS="$DERBY_OPTS -Dderby.drda.host=$NS_HOST"
export DERBY_OPTS="$DERBY_OPTS -Dderby.drda.portNumber=$NS_PORT"
export DERBY_OPTS="$DERBY_OPTS -Dderby.system.home=$NS_HOME"

DERBY_HOME is self explanatory: it’s where Derby is unzipped (installed). Add Derby’s bin directory to the PATH.

JAVA_HOME is self explanatory: it’s where Java is unzipped (installed). Add Java’s bin directory to the PATH.

NS_HOME is “Network Server Home”. This is the directory the Derby network server will use to store its configuration and databases. Whenever a new database is created on this Derby network server, a new sub-directory will be created under NS_HOME for the new database. This allows multiple Derby network servers running on the same host to keep their data separate.

NS_PORT is self explanatory: it’s the port the Derby network server uses to listen for connections. This allows multiple Derby network servers to run on the same host.

NS_HOST sets the network interface used by the Derby network server when listening for connections. By default, the Derby network server only listens for connections on the loopback address of 127.0.0.1. This default means clients must run on the same host as the network server - not very useful. By setting the host to 0.0.0.0, the Derby network server will listen for connections on any network interface on the host. If your Derby network server host has multiple network interfaces, NS_HOST should be set to the IP of one of those interfaces. Setting this value allows clients to be remote and run on any host.

DERBY_OPTS is the system property used to get all of the configuration options to Derby. It’s value is created by concatenating together the appropriate Derby system properties with their associated values.

Now that the environment is set, we can start the server.

Listing 2 - start.sh

#!/bin/bash

# Directory of the script
SD=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

# Source in common variables
source $SD/setenv.sh

startNetworkServer -noSecurityManager

SD is Script Directory. The evaluation determines the fully-qualified file system location of the start.sh script and assigns it to SD. This is useful when referencing other scripts.

source self explanatory: it sources in the environment configuration to run the Derby network server.

startNetworkServer -noSecurityManager starts the Derby network server. The DERBY_OPTS variable - set in setenv.sh - is used to configure the network server. The -noSecurityManager command line option needs some explanation. By default, Derby’s Java process runs with a limited security policy. I’ve found that this limited security policy gets in the way of database operations which I normally would just expect to work. By specifying -noSecurityManager, you run the Derby network server without any security policy. This may not be the ideal way for you to run a Derby network server. However, this blog is limited in scope to running the server. Search for my other blog on how to secure a Derby network server.

Now that the server is running, we need to stop it.

Listing 3 - stop.sh

#!/bin/bash

# Directory of the script
SD=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

# Source in common variables
source $SD/setenv.sh

stopNetworkServer

All of this is self explanatory. No further comments are needed for this script.

Windows batch scripts

Here are the Windows batch scripts to configure starting a Derby network server. We will not look at these files in additional details; details are included with the Linux bash scripts.

Listing 1 - setenv.cmd

@echo off
 
set DERBY_HOME=C:\Applications\Derby
set PATH=%DERBY_HOME%\bin;%PATH%
echo DERBY_HOME=%DERBY_HOME%

set JAVA_HOME=C:\Applications\Java
echo JAVA_HOME=%JAVA_HOME%

set NS_HOME=C:\Users\Derby\1110
md %NS_HOME% 2>NUL
echo NS_HOME=%NS_HOME%

set NS_PORT=1110
echo NS_PORT=%NS_PORT%

set NS_HOST=0.0.0.0
echo NS_HOST=%NS_HOST%
 
set DERBY_OPTS=-Dderby.drda.host=%NS_HOST%
set DERBY_OPTS=%DERBY_OPTS% -Dderby.drda.portNumber=%NS_PORT%
set DERBY_OPTS=%DERBY_OPTS% -Dderby.system.home=%NS_HOME%

Listing 2 - start.cmd

@echo off
call setenv.cmd
startNetworkServer -noSecurityManager

Listing 3 - stop.cmd

@echo off
call setenv.cmd
stopNetworkServer

Summary

That’s it. I hope you enjoyed learning how to run multiple Derby network servers on the same host.

May 23, 2018

Apache Derby Database Users and Permissions

Abstract

Apache Derby is awesome! Especially in a Microservices environment where the data for services (may) shrink and not require a heartier RDBMS. Derby is awesome because it’s so easy to use, especially when it come to users and permissions - you don’t need any! But, it may be the case you want to create an application-level user with limited permissions to use in Derby. The purpose of this blog is to document how to create application-level, limited permission users in Derby.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Apache Derby 10.14.1.0
  • Java 1.8.0_152_x64

I am not going to go through the process of downloading and installing these technologies. I’ll leave that as an exercise for you.

Run Derby Network Server

The first thing you must do is run a Derby network server. In my previous blog post titled Multiple Derby Network Servers on the same Host, I give detailed instructions on how to do this. Please refer to that blog post to run your own Derby network server.

Configure Derby Network Server

To configure the Derby network server, you need to create a derby.properties file. But where does the file go? It can go in a couple different places. Let’s take a look.

I’ll first assume that you ignored the Run Derby Network Server section above and instead are running Derby with all its defaults. If that’s the case, you probably started the network server by finding the %DERBY_HOME%\bin\startNetworkServer.bat file and double-clicking it. If you did this - highly not recommended - then Derby thinks the %DERBY_HOME%\bin directory is its system directory. You can confirm this by looking for the %DERBY_HOME%\bin\derby.log file. If confirmed, then you need to create a %DERBY_HOME%\bin\derby.properties file. Wherever the derby.log file is, that’s where you create the derby.properties file.

On the other hand if you didn’t ignore the Run Derby Network Server section above, congratulations! The derby.properties file must go into the directory set by the -Dderby.system.home Java system property.

Now that you know where to put the derby.properties file, here is (an example) of what to put in it:

# Passwords don't expire for 10 years
derby.authentication.native.passwordLifetimeMillis=315360000000

# Use the best hash algorithm you can
derby.authentication.builtin.algorithm=SHA-512

# Use a larger salt length for better security
derby.authentication.builtin.saltLength=128

# Re-hash this number of times for better security
derby.authentication.builtin.iterations=1564

The first property derby.authentication.native.passwordLifetimeMillis is the most important one. It configures how long Derby user passwords can be used before they go stale. By default, a password goes stale in 31 days unless this property is configured.

NOTE Derby uses the word “stale”, whereas we would typically use the word “expired”.

The value of this property is in milliseconds. The example above - 315360000000 - means a user’s password goes stale 10 years after the user is created. This configuration is for all users in every database in that Derby network server. There does not seem to be any per-user password expire setting.

Now you have the derby.properties file in place. Start the network server and let’s use it.

Run ij

ij is to Derby what sqlplus is to Oracle; just a simple command-line interface to execute SQL. Find and run %DERBY_HOME%\bin\ij.bat. For the rest of the blog, the "ij>" prompt will indicate SQL commands that must be executed within ij.

With ij running, we can create a database. Let’s do that next.

Create a Database

First, make sure your Derby network server is configured (as described above), and running (as described above) and is waiting for connections on whatever port you configured.

Second, make sure ij is running. You should be at this prompt:

ij>

Third, issue a connect statement which will create a new database.

ij> connect 'jdbc:derby://localhost:11528/resiste;create=true;' user 'sa_resiste';

Let’s look at this in more detail. localhost assumes the Derby network server is running on the same machine as ij, though it can be any host on your network. 11528 is the port the Derby network server is using to listen for connections. resiste is the name of the database to connect to; remember a Derby network server can have many different databases. create=true tells Derby to create the database if it doesn’t exist already. When created, the resiste database will the file system folder %derby.system.home%\resiste. sa_resiste is name of the admin user for the resiste database. Usually it’s just sa. Next we’ll look at setting the password for this admin user.

Create the Admin User

Derby provides an internal system call to create users and passwords. Make sure you are connected to the resiste database then execute:

ij> CALL SYSCS_UTIL.SYSCS_CREATE_USER('sa_resiste', 'derby123'); 
ij> disconnect;
ij> exit;

Line 1 creates the sa_resiste user with the password derby123. Lines 3 and 4 then disconnect from the database and exits ij.

RESTART THE NETWORK SERVER NOW

After restarting, let’s see if it worked. Connect with sa_resiste and no password. Connection will get authentication failure.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste';
ERROR 08004: Connection authentication failure occurred.  Reason: Userid or password invalid.

Now connect with sa_resiste and password. Connection will succeed.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> 

Good! The admin user is now created. Next we’ll use the admin user to create a table. This table will be used to validate the permissions of the application-level user we’ll create later.

Create Test Table

Now we are going to use the admin user to create a test table. We will do this for a couple reasons.

  1. Verify the admin user has all permissions and is able to execute these SQL commands.
  2. Verify the permissions of the application-level user we’ll create later.
ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> create schema testing;
ij> set schema testing;
ij> create table names (full_name varchar(100));
ij> insert into names values ('rita red');
ij> select * from names;
FULL_NAME
----------------------------------------------------------------------------------------------------
rita red
ij> disconnect; 

Next let’s create the application-level user.

Create the Application User

Now for the fun stuff. Let’s create an application-level user. This will be a user with permission limited to only the operations an application is able to perform. For example, if your Microservice is only going to GET data, then the application-level user should only have SELECT permissions on the database table. We will test the application-level user’s permission, but first let’s create the user.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> CALL SYSCS_UTIL.SYSCS_CREATE_USER('oscar', 'orange');
ij> disconnect;
ij> exit;

NOTE Remember, oscar’s password will go stale in 31 days by default. However, if you have a derby.properties file with the derby.authentication.native.passwordLifetimeMillis property set, then it will go stale after whatever time is set by that property. There is no other way to change the time it takes for the password to go stale.

RESTART THE NETWORK SERVER NOW

After restarting, let’s see if it worked. Connect with oscar. Connection will succeed, but, oscar won’t have the permission to read the test table.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'oscar' password 'orange';
ij> select * from testing.names;
ERROR 42502: User 'OSCAR' does not have SELECT permission on column 'FULL_NAME' of table 'TESTING'.'NAMES'.
ij> disconnect;

Even though the SELECT statement failed, failure means a successful test. oscar has no permissions so should not be able to select from the test table. Let’s configure oscar next.

Configure the Application User

Let’s set some permissions for oscar. Of course the sa_resiste admin user is required to do this.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> set schema testing;
ij> grant select on names to oscar;
ij> disconnect;

This will give oscar only 1 permission: to select from TESTING.NAMES table. Let’s see if it worked.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'oscar' password 'orange';
ij> select * from testing.names;
FULL_NAME
----------------------------------------------------------------------------------------------------
rita red
ij> disconnect;

Congratulations! You now have an application-level user with limited permissions in your Derby database.

Summary

I hope you enjoyed learning how to do simple user administration with Derby.

March 02, 2018

My Thoughts on Jakarta EE

The Announcement

On February 26, 2018, I saw a post on Twitter saying EE4J tallied the results of the naming survey and Jakarta EE is the new brand for the open source enterprise software standard (Milinkovich, 2018). From what I’ve seen on Twitter and read on other blogs, the Java enterprise community has been very supportive of the re-branding. I am supportive as well (I voted for Jakarta EE) and will continue to evangelize Jakarta EE. Though this is a monumental step, the challenges for EE4J and the Jakarta EE brand are far from over. I’d like to share what I think are some of those challenges. You can skip to tl;dr to save time. They are:

  1. Getting the Re-branding to Stick
  2. Release Cadence
  3. Continued Emphasis on the EE Server

Getting the Re-branding to Stick

The announcement hadn’t even been a day old when Twitter posts started from recruiters looking for Jakarta EE experience (Ament, 2018). Java Enterprise software has been around for nearly 20 years, and over that time it has gone through previous cycles of renaming and re-branding. Here they are:

J2EE (1999) -> Java EE (2006) -> Jakarta EE (2018)

Re-branding is hard, and it can be argued that neither Sun nor Oracle did a good job promoting the brands. You can search Twitter for posts joking about how it has been over 10 years since the re-brand to Java EE but recruiters still advertise and look for “J2EE”. I am guilty of this as well. My LinkedIn profile has “J2EE” all over it so I can be found.

So how will EE4J fair promoting the Jakarta EE brand? I think success can be measured by management knowing about it. It’s sometimes dangerous when managers know things. But, if managers start asking, “are we are using Jakarta EE in new projects”, that’s success in branding.

NOTE It’s OK if managers don’t know what Jarkarta EE is, as long as they know it’s the “thing to do” :)

Release Cadence

Java SE is experiencing this challenge right now. The Java community griped about how long it took for Java to evolve and release new versions. Now that Oracle announced Java SE is on a 6 month release cycle (Evens, 2017) with limited long term support options (Azul, 2018), organizations don’t seem to know what to do (be careful what you wish for). Software is moving faster than infrastructure can keep up.

Release cadence can be a problem with Jakarta EE as well. The Java enterprise community may get its wish with rapid releases of Jakarta EE. But if this happens, I don’t think the releases will get the fan fair the community expects. Infrastructure won’t be able to keep up. Organizations will keep to their multi-year upgrade plans meaning long periods of time between server upgrades. Jakarta EE developers will be stuck using old standards like Java EE developers are stuck now.

A rapid release cadence for Jakarta EE may even cause more harm than good. As it is now, if an organization is upgrading servers every few years, the perceived risk of the upgrade is somewhat reduced because the standard may change by only 1 number: Java EE 6 to 7. On paper, that doesn’t look too risky. But if there have been multiple release over the years and the change is from Jakarta EE 9 to 14, the perceived risk is now YIKES!

I think Jakarta EE can support a rapid release cadence. However, to do this it has to face what I consider to be its most daunting challenge: the server.

Continued Emphasis on the EE Server

The final and most glaring challenge EE4J faces is continuing to remain server-technology focused. Will Jakarta EE continue to focus on the standards to create an “EE Server”? I hope not!

20 years ago, the expectation was to install a “heavy” server (WebLogic, WebSphere) and be able to ride that installation with very few changes for 3–5 years and you’re all good. Even security patches over those years were very limited; it was a different era! That’s the way infrastructure worked then, and if we are honest with ourselves, it’s the way infrastructure works now. Infrastructure is a huge impediment to software development. Developers want to use the latest-and-greatest, but infrastructure does not want to “take the risk”. Upgrading server software is always painful: lots of testing, lots of things breaking. So infrastructure avoids upgrades as much as it can.

The development community has gotten around this infrastructure problem by moving functionality out of the server and into the application, hence the Spring framework. Spring is successful precisely because of this infrastructure problem. Infrastructure typically does not care what’s inside the applications running on the server - the applications can change and upgrade as much as they want - just so long as the server itself doesn’t have to change. Is your production WebSphere installation 10 years old? “No problem,” developer’s say, “We aren’t really using anything the server provides anyway.”

Jakarta EE needs to evolve into a standard for an enterprise framework vs. an enterprise server. Spring, Hibernate, and other open source project can continue to push innovation forward. Organizations that want to take the risk and use those bleeding-edge technologies can do so. Once proven, their innovations can be incorporated into the Jakarta EE standard framework for the rest of us. Jakarta EE developers can use the new features added to the framework as quickly as their next product release by a simple POM <version> update…no need to install a new server. As an added bonus, if Jakarta EE sticks to TCK requirements and backward compatibility, Jakarta EE developers will have the confidence things won’t break when they change that often feared <version> number.

If Jakarta EE evolves into a standard for an enterprise framework (I hope so), Jakarta EE applications still need to run in something. Perhaps keeping to the current trend of using the Servlet container (Tomcat, Jetty) would be sufficient. But whatever form it takes, the expectation should be that the “server” would be able to sit for 10 years with minimal updates and not get in the way of the applications running in them. When the EE4J charter was under review, I believe there was some discussion on using the word “runtime” instead of “server”. This is appropriate. Have a Jakarta EE runtime (JEER) that can run for years without requiring major upgrades.

tl;dr

I fully support Jakarta EE as a brand and will continue to evangelize the technology as an open standard for enterprise development.

Successfully branding Jakarta EE will be a challenge. Branding will be successful if managers start asking, “Are we using Jakarta EE”? Jakarta EE should be a buzz word and the “thing to use” even though people may have no idea what it is. Sound familiar?

Release cadence may backfire on Jakarta EE because upgrading server installations will continue to move a lot slower.

Jakarta EE needs to evolve into a enterprise standard framework, deployable as part of an application vs. an enterprise standard server that has to be installed or upgraded on a box. The former results in speed, agility, and innovation. The latter results in stagnation.

Thanks for reading.

References

Milinkovich, M., (2018, February 26). And the Name Is…[Blog Post]. Retrieved from https://mmilinkov.wordpress.com/2018/02/26/and-the-name-is/.

Ament, J. D., (2018, February 26). This just in [Tweet]. Retrieved from https://twitter.com/JohnAment/status/968324168264704000.

Evans, B., (2017, September 6). Java to Move to 6-Monthly Release Cadence [Blog Post]. Retrieved from https://www.infoq.com/news/2017/09/Java6Month.

Azul Systems, (2018, January 17). Azul Systems Announces Enhanced Java SE Support Plans [Blog Post]. Retrieved from https://globenewswire.com/news-release/2018/01/17/1295592/0/en/Azul-Systems-Announces-Enhanced-Java-SE-Support-Plans.html.

January 10, 2018

Bootstrap CDI 2.0 in Java SE

Abstract

This is a quick reference for bootstrapping CDI 2.0 in a Java SE application.

CDI 2.0 (JSR 365)

Maven Dependency

<dependency>
    <groupId>org.jboss.weld.se</groupId>
    <artifactId>weld-se-core</artifactId>
    <version>3.0.2.Final</version>
</dependency>

beans.xml

File location is /META-INF/beans.xml. This file is optional, though I’ve found it’s still good to have.

<beans version="2.0" 
    xmlns="http://xmlns.jcp.org/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
     http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/beans_2_0.xsd" 
    bean-discovery-mode="all"
>
    <scan>
        <exclude name="org.jboss.weld.**" />
    </scan>  
</beans>

Java SE Application

package org.ferris.cdi.example.main;

import javax.enterprise.inject.se.SeContainer;
import javax.enterprise.inject.se.SeContainerInitializer;

/**
 * The main() method for this application
 *
 * @author <a href="mailto:mjremijan@yahoo.com">Mike Remijan</a>
 */
public class Main {
    public static void main(String[] args) {

        SeContainer container
            = SeContainerInitializer.newInstance().initialize();

        Main main
            = container.select(Main.class).get();

        //. . .
    }
}

Summary

That’s it…enjoy!

December 30, 2017

JDBC Connections Cheat Sheet

Abstract

This is a quick reference for JDBC connections for common databases. I seem to have to lookup this information a lot, so I figured it be good to have a reference all in one place.

Derby

<dependency>
    <groupId>org.apache.derby</groupId>
    <artifactId>derbyclient</artifactId>
    <version>10.11.1.1</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.apache.derby</groupId>
    <artifactId>derby</artifactId>
    <version>10.11.1.1</version>
    <scope>test</scope>
</dependency>

Embedded (in-memory)

Class.forName("org.apache.derby.jdbc.EmbeddedDriver");

String connectionUrl
    = "jdbc:derby:C:/My Databases/Derby/Test;user=;password=;create=true";

Connection conn
    = DriverManager.getConnection(connectionUrl);

Remote

Class.forName("org.apache.derby.jdbc.ClientDriver");

String connectionUrl
    = "jdbc:derby://localhost:1527/widget";

String user = "sa";
String pass = "sa";

Connection conn
    = DriverManager.getConnection(connectionUrl, user, pass);

PostgeSQL

<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.1.4.jre7</version>
    <scope>test</scope>
</dependency>
Class.forName("org.postgresql.Driver");

String connectionUrl
    = "jdbc:postgresql://localhost:5432/widget";

String user = "widgetapp";
String pass = "widgetapp";

Connection conn
    = DriverManager.getConnection(connectionUrl, user, pass);

Oracle

Download JDBC drivers from http://www.oracle.com/technetwork/database/features/jdbc/index.html

<dependency>
    <groupId>com.oracle</groupId>
    <artifactId>ojdbc6</artifactId>
    <version>11.2.0.4</version>
    <scope>system</scope>
    <systemPath>${basedir}/lib/ojdbc6.jar</systemPath>
</dependency>
Class.forName("oracle.jdbc.driver.OracleDriver");

String SID
    = "xe";

String connectionUrl
    = "jdbc:oracle:thin:@localhost:1521:" + SID;

String user = "hr";
String pass = "hr";

Connection conn
    = DriverManager.getConnection(connectionUrl, user, pass);

Summary

That’s it…enjoy!

December 22, 2017

Java EE Deployment Descriptor and XML Reference

Abstract

This is a quick reference for the most used Java EE deployment descriptors and XML documents. Most developers do not know the versions of these files change with EE versions and that EE servers use the versions of these files to determine what EE standard to apply to your application. So, as you migrate your application to newer application servers, make sure to update the versions of the Java EE deployment descriptors and XML documents to take advantage of the new EE features.

web.xml

<!-- EE 8 -->
<web-app version="4.0"
xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee  http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd">
<!-- EE 7 -->
<web-app version="3.1" 
         xmlns="http://xmlns.jcp.org/xml/ns/javaee" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd">
<!-- EE 6 -->
<web-app  version="3.0"
xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee  http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"> 

beans.xml

<!-- EE 8 -->
<beans version="2.0" 
xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/beans_2_0.xsd" bean-discovery-mode="all">
<!-- EE 7 -->             
<beans
xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/beans_1_1.xsd"
bean-discovery-mode="all">
<!-- EE 6 -->    
<beans 
xmlns="http://java.sun.com/xml/ns/javaee"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/beans_1_0.xsd" -->

Summary

That’s it…enjoy!

Choosing Java Cryptographic Algorithms Part 3 - Public/Private key asymmetric encryption

Abstract

This is the 3rd of a three-part blog series covering Java cryptographic algorithms. The series covers how to implement the following:

  1. Hashing with SHA–512
  2. Single-key symmetric encryption with AES–256
  3. Public/Private key asymmetric encryption with RSA–4096

This 3rd post details how to implement public/private key, asymmetric, RSA–4096 encryption. Let’s get started.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Java 1.8.0_152_x64
  • NetBeans 8.2 (Build 201609300101)
  • Maven 3.0.5 (Bundled with NetBeans)

Download

Visit my GitHub Page to see all of my open source projects. The code for this post is located in project: thoth-cryptography

Asymmetric Encryption

About

Asymmetric algorithms are based on 2 keys: a public key and a private key. The public key is responsible for encryption and the private key is responsible for decryption. The public key can be freely distributed. With the public key, any client can encrypt a message which only you - with the private key - can decrypt (Asymmetric algorithms, n.d. para. 3).

Asymmetric algorithms are the workhorse of the Internet. Protocols like SSH, OpenPGP, SSL, and TLS rely on asymmetric algorithms (Rouse, 2016, para. 2). Anyone who uses a web browser for something like online banking inherently knows the importance of asymmetric algorithms.

Research done as of today seems to indicate the best and most secure public/private key, asymmetric, encryption algorithm is the following (Sheth, 2017, “Choosing the correct algorithm”, para.2):

  1. Algorithm: RSA
  2. Mode: ECB // It’s really NONE but ECB is needed to get Java to work.
  3. Padding: OAEPWithSHA–512AndMGF1Padding
  4. Key size: 4096 bit

RSA isn’t a block cipher so ECB mode doesn’t make much sense, but, ECB is needed to make Java work even though the mode isn’t used under the covers (Brightwell, 2015). OAEP provides a high level of randomness and padding. Let’s take a look at an example.

Example

Listing 1 is the RsaTest.java unit test. It is a full demonstration on the following:

  1. Generate and store an RSA 4096-bit key
  2. RSA Encryption
  3. RSA Decryption

Listing 2 shows RsaKeyPairProducer.java. This is a helper class which is responsible for producing a new KeyPair. The KeyPair contains both the PublicKey and PrivateKey.

Listing 3 shows RsaPrivateKeyProducer.java. This is a helper class which is responsible for reproducing a PrivateKey from a byte[].

Listing 4 shows RsaPublicKeyProducer.java. This is a helper class which is responsible for reproducing a PublicKey from a byte[].

Listing 5 shows ByteArrayWriter.java and Listing 6 shows ByteArrayReader.java. These are helper classes responsible for reading and writing a byte[] to a file. It’s up to you to determine how to store the byte[] of your keys, but it needs to be stored securely somewhere (file, database, git repository, etc.).

Listing 7 shows RsaEncrypter.java. This is a helper class which is responsible for encryption.

Finally, listing 8 shows RsaDecrypter.java. This is a helper class which is responsible for decryption.

Listing 1 - RsaTest.java class

package org.thoth.crypto.asymmetric;

import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.KeyPair;
import java.security.PrivateKey;
import java.security.PublicKey;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import org.thoth.crypto.io.ByteArrayReader;
import org.thoth.crypto.io.ByteArrayWriter;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class RsaTest {

    static Path privateKeyFile;
    static Path publicKeyFile;

    @BeforeClass
    public static void beforeClass() throws Exception {

        // Store the PrivateKey and PublicKey bytes in the ./target
        // diretory. Do this so it will be ignore by source control.
        // We don't want this file committed.
        privateKeyFile
            = Paths.get("./target/RsaPrivate.key").toAbsolutePath();
        publicKeyFile
            = Paths.get("./target/RsaPublic.key").toAbsolutePath();

        // Create KeyPair for RSA
        KeyPair keyPair
            = new RsaKeyPairProducer().produce();

        // Store the PrivateKey bytes. This is what
        // you want to keep absolutely safe
        {
            ByteArrayWriter writer = new ByteArrayWriter(privateKeyFile);
            writer.write(keyPair.getPrivate().getEncoded());
        }

        // Store the PublicKey bytes.  This you
        // can freely distribute so others can
        // encrypt messages which you can then
        // decrypt with the PrivateKey you keep safe.
        {
            ByteArrayWriter writer = new ByteArrayWriter(publicKeyFile);
            writer.write(keyPair.getPublic().getEncoded());
        }
    }


    @Test
    public void encrypt_and_decrypt() throws Exception {
        // setup
        PrivateKey privateKey
            = new RsaPrivateKeyProducer().produce(
                new ByteArrayReader(privateKeyFile).read()
            );

        PublicKey publicKey
            = new RsaPublicKeyProducer().produce(
                new ByteArrayReader(publicKeyFile).read()
            );

        RsaDecrypter decrypter
            = new RsaDecrypter(privateKey);

        RsaEncrypter encrypter
            = new RsaEncrypter(publicKey);

        String toEncrypt
            = "encrypt me";

        // run
        byte[] encryptedBytes
            = encrypter.encrypt(toEncrypt);
        System.out.printf("Encrypted %s%n", new String(encryptedBytes,"UTF-8"));

        String decrypted
            = decrypter.decrypt(encryptedBytes);

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }

}

Listing 2 - RsaKeyPairProducer.java class

package org.thoth.crypto.asymmetric;

import java.security.KeyPair;
import java.security.KeyPairGenerator;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class RsaKeyPairProducer {

    /**
     * Generates a new RSA-4096 bit {@code KeyPair}.
     *
     * @return {@code KeyPair}, never null
     * @throws RuntimeException All exceptions are caught
     * and re-thrown as {@code RuntimeException}
     */
    public KeyPair produce() {
        KeyPairGenerator keyGen;
        try {
            keyGen = KeyPairGenerator.getInstance("RSA");
            //keyGen.initialize(3072);
            keyGen.initialize(4096);
            return keyGen.generateKeyPair();
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 3 - RsaPrivateKeyProducer.java class

package org.thoth.crypto.asymmetric;

import java.security.KeyFactory;
import java.security.PrivateKey;
import java.security.spec.PKCS8EncodedKeySpec;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class RsaPrivateKeyProducer {

    /**
     * Regenerates a previous RSA {@code PrivateKey}.
     *
     * @param encodedByteArrayForPrivateKey The bytes this method
     * will use to regenerate a previously created {@code PrivateKey}
     *
     * @return {@code PrivateKey}, never null
     * @throws RuntimeException All exceptions are caught
     * and re-thrown as {@code RuntimeException}
     */
    public PrivateKey produce(byte[] encodedByteArrayForPrivateKey) {
        try {
            PrivateKey privateKey = KeyFactory.getInstance("RSA")
                .generatePrivate(new PKCS8EncodedKeySpec(encodedByteArrayForPrivateKey));

            return privateKey;
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 4 - RsaPublicKeyProducer.java class

package org.thoth.crypto.asymmetric;

import java.security.KeyFactory;
import java.security.PublicKey;
import java.security.spec.X509EncodedKeySpec;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class RsaPublicKeyProducer {

    /**
     * Regenerates a previous RSA {@code PublicKey}.
     *
     * @param encodedByteArrayForPublicKey The bytes this method
     * will use to regenerate a previously created {@code PublicKey}
     *
     * @return {@code PublicKey}, never null
     * @throws RuntimeException All exceptions are caught
     * and re-thrown as {@code RuntimeException}
     */
    public PublicKey produce(byte[] encodedByteArrayForPublicKey) {
        try {
            PublicKey publicKey = KeyFactory.getInstance("RSA")
                .generatePublic(new X509EncodedKeySpec(encodedByteArrayForPublicKey));

            return publicKey;
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 5 - ByteArrayWriter.java class

package org.thoth.crypto.io;

import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.nio.file.Files;
import java.nio.file.Path;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class ByteArrayWriter {

    protected Path outputFile;

    private void initOutputFile(Path outputFile) {
        this.outputFile = outputFile;
    }

    private void initOutputDirectory() {
        Path outputDirectory = outputFile.getParent();
        if (!Files.exists(outputDirectory)) {
            try {
                Files.createDirectories(outputDirectory);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }
    }

    public ByteArrayWriter(Path outputFile) {
        initOutputFile(outputFile);
        initOutputDirectory();
    }

    public void write(byte[] bytesArrayToWrite) {
        try (
            OutputStream os
                = Files.newOutputStream(outputFile);

            PrintWriter writer
                =  new PrintWriter(os);
        ){
            for (int i=0; i<bytesArrayToWrite.length; i++) {
                if (i>0) {
                    writer.println();
                }
                writer.print(bytesArrayToWrite[i]);
            }
        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 6 - ByteArrayReader.java class

package org.thoth.crypto.io;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Scanner;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class ByteArrayReader {

    protected Path inputFile;

    public ByteArrayReader(Path inputFile) {
        this.inputFile = inputFile;
    }

    public byte[] read() {
        try (
            Scanner scanner
                =  new Scanner(inputFile);

            ByteArrayOutputStream baos
                = new ByteArrayOutputStream();
        ){
            while (scanner.hasNext()) {
                baos.write(Byte.parseByte(scanner.nextLine()));
            }
            
            baos.flush();
            return baos.toByteArray();

        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 7 - RsaEncrypter.java class

package org.thoth.crypto.asymmetric;

import java.security.PublicKey;
import javax.crypto.Cipher;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class RsaEncrypter {

    protected RsaCipher cipher;

    public RsaEncrypter(PublicKey key) {
        this.cipher = new RsaCipher(Cipher.ENCRYPT_MODE, key);
    }

    public byte[] encrypt(String message) {
        try {
            return cipher
                    .update(message.getBytes("UTF-8"))
                    .doFinal()
            ;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

Listing 8 - RsaDecrypter.java class

package org.thoth.crypto.asymmetric;

import java.security.PrivateKey;
import javax.crypto.Cipher;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class RsaDecrypter {

    protected RsaCipher cipher;

    public RsaDecrypter(PrivateKey key) {
        this.cipher = new RsaCipher(Cipher.DECRYPT_MODE, key);
    }

    public String decrypt(byte[] message) {
        try {
            return new String(
                cipher.update(message).doFinal()
                , "UTF-8"
            );
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

}

Summary

Encryption isn’t easy. And easy examples will result in implementations with security vulnerabilities for your application. If you need a public/private key, asymmetric, encryption algorithm, use RSA/ECB/OAEPWithSHA–512AndMGF1Padding, with a 4096 bit key.

References

Sheth, M. (2017, April 18). Encryption and Decryption in Java Cryptography. Retrieved from https://www.veracode.com/blog/research/encryption-and-decryption-java-cryptography.

  • Best algorithms, modes, and paddings
  • Use 4096-bit key

Brightwell, W., poncho. (2015, May 4). Is ECB mode safe to use with RSA encryption?. Retrieved from https://crypto.stackexchange.com/questions/25420/is-ecb-mode-safe-to-use-with-rsa-encryption.

  • ECB isn’t applicable; Java doesn’t use it under the covers.
  • RSA uses randomness and the padding to avoid ECB issue of the same plaintext generating the same ciphertext

Marilena. (2016, November 29). Java - Asymmetric Cryptography example. Retrieved from https://www.mkyong.com/java/java-asymmetric-cryptography-example/.

  • Writing the public/private keys to file
  • Reading the public/private keys from file
  • Encrypt & Decrypt

Key size. (2017, October 12). Wikipedia. Retrieved from https://en.wikipedia.org/wiki/Key_size.

  • 2048-bit keys are sufficient until 2030
  • An RSA key length of 3072 bits should be used if security is required beyond 2030

user4982. (2013, November 4). How are IVs used in association with RSA Encryption?. Retrieved from https://crypto.stackexchange.com/questions/11403/how-are-ivs-used-in-association-with-rsa-encryption.

  • IV values are not used with RSA

Choosing Java Cryptographic Algorithms Part 2 - Single key symmetric encryption

Abstract

This is the 2nd of a three-part blog series covering Java cryptographic algorithms. The series covers how to implement the following:

  1. Hashing with SHA-512
  2. Single-key symmetric encryption with AES-256
  3. Public/Private key asymmetric encryption with RSA-4096

This 2nd post details how to implement single key, symmetric, AES-256 encryption. Let’s get started.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

NOTE As of Java 1.8.0_161, unlimited cryptography is enabled by default. This means if you are using Java 1.8.0_161 or later, you do not need to install Java Cryptography Extension (JCE) Unlimited Strength separately. See the Java 1.8.0_161 release notes

Acknowledgments

Thanks to Peter Jakobsen. In March 2022, he identified an error in my Aes#encrypt() method. The error caused encryption/decryption process to fail when the string being encrypted was greater than 15 characters long. I did not have unit test for this case.

At the time, I did not have availability to look into this problem. Peter posted the question to https://stackoverflow.com/questions/71422498/javax-crypto-aeadbadtagexception-tag-mismatch-when-password-length-16. Additional thanks to Topaco on Stack Overflow for providing comments.

In November 2022, I was able to update my Aes class to address the problem Peter identified. Using the Template Method Design Pattern, I now have two sub-classes of Aes demonstrating both the multiple-part and single-part encryption operations. I have updated this blog to reflect these new classes.

Download

Visit my GitHub Page to see all of my open source projects. The code for this post is located in project: thoth-cryptography

Symmetric Encryption

About

Symmetric encryption algorithms are based on a single key. This one key is used for both encryption and decryption. As such, symmetric algorithms should only be used where strict controls are in place to protect the key.

Symmetric algorithms are commonly used for encryption and decryption of data in secured environments. A good example of this is securing Microservice communication. If an OAuth-2/JWT architecture is out of scope, the API Gateway can use a symmetric algorithm’s single key to encrypt a token. This token is then passed to other Microservices. The other Microservices use the same key to decrypt token. Another good example are hyperlinks embedded in emails. The hyperlinks in emails contain an encoded token which allow automatic login request processing when the hyperlink is clicked. This token is a strongly encrypted value generated by a symmetric algorithm so it can only be decoded on the application server. And of course, anytime passwords or credentials of any kind need to be protected, a symmetric algorithm is used to encrypt them and the bytes can later be decrypted with the same key.

Research done as of today seems to indicate the best and most secure single key, symmetric, encryption algorithm is the following (Sheth, 2017, “Choosing the correct algorithm”, para.2):

  1. Algorithm: AES
  2. Mode: GCM
  3. Padding: PKCS5Padding
  4. Key size: 256 bit
  5. IV size: 96 bit

AES-256 uses a 256-bit key which requires installation of the Java Cryptography Extension (JCE) Unlimited Strength package. Let’s take a look at an example.

NOTE The Java Cryptography Extension (JCE) Unlimited Strength package is required for 256-bit keys. If it’s not installed, 128-bit keys are the max.

Example

If you don’t already have it, download and install the Java Cryptography Extension (JCE) Unlimited Strength package. It is required to use 256-bit keys. Otherwise, the example below must be updated to use a 128-bit key.

Listing 1 and Listing 2 are the unit tests AesUsingSinglePartEncryptionTest.java and AesUsingMultiplePartEncryptionTest.java. Both unit tests are full demonstrations of the following:

  1. Generate and store an AES 256-bit key
  2. AES Encryption
  3. AES Decryption

One demonstrates multiple-part encryption and the other demonstrates single-part encryption. The difference between the two is how the Cipher#update and Cipher#doFinal methods are used to get the encrypted bytes.

Listing 3 shows AesSecretKeyProducer.java. This is a helper class which is responsible for producing a new key or reproducing an existing key from a byte[].

Listing 4 shows ByteArrayWriter.java and Listing 5 shows ByteArrayReader.java. These are helper classes responsible for reading and writing a byte[] to a file. It’s up to you to determine how to store the byte[] of your key, but it needs to be stored securely somewhere (file, database, git repository, etc.).

Listing 6 shows Aes.java. This is a helper class which is responsible for both encryption and decryption. It is implemented as an abstract class so it can not be instantiated directly. Instead, one of its sub-classes need to be used. There are 2 sub-classes, one to demonstrate the single-part encryption operation and the other to demonstrate the multiple-part encryption operation.

NOTE In reality, you probably won’t need an abstract class. Instead you’ll most likely create 1 class which uses the encryption operation (single-part or multiple-part) you want to use.

Listing 7 shows AesUsingSinglePartEncryption.java. This extends the Aes class and implements the getEncryptedBytes() method. Its implementation demonstrates the single-part encryption operation, which uses a single call to Cipher#doFinal(byte[]) to get the encrypted bytes.

Listing 8 shows AesUsingMultiplePartEncryption.java. This extends the Aes class and implements the getEncryptedBytes() method. Its implementation demonstrates the multiple-part encryption operation which makes multiple calls to Cipher#update(byte[]) and then a final call to Cipher#doFinal(). All the encrypted bytes are store along the way and returned after the call to Cipher#doFinal().

Listing 1 - AesUsingSinglePartEncryptionTest.java class

package org.thoth.crypto.symmetric;

import java.io.ByteArrayOutputStream;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Optional;
import javax.crypto.SecretKey;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import org.thoth.crypto.io.ByteArrayReader;
import org.thoth.crypto.io.ByteArrayWriter;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class AesUsingSinglePartEncryptionTest {

    static Path secretKeyFile;

    @BeforeClass
    public static void beforeClass() throws Exception {
        // Store the SecretKey bytes in the ./target diretory. Do
        // this so it will be ignore by source control.  We don't
        // want this file committed.
        secretKeyFile
            = Paths.get("./target/Aes256.key").toAbsolutePath();

        // Generate a SecretKey for the test
        SecretKey secretKey
            = new AesSecretKeyProducer().produce();

        // Store the byte[] of the SecretKey.  This is the
        // "private key file" you want to keep safe.
        ByteArrayWriter writer = new ByteArrayWriter(secretKeyFile);
        writer.write(secretKey.getEncoded());
    }


    @Test
    public void encrypt_and_decrypt_using_same_Aes256_instance_long() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingSinglePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me1, encrypt me2, encrypt me3, encrypt me4, encrypt me5, encrypt me6, encrypt me7, encrypt me8, encrypt me9, encrypt me10";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.empty());

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    
    @Test
    public void encrypt_and_decrypt_using_same_Aes256_instance_short() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.empty());

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }


    public void encrypt_and_decrypt_with_aad_using_same_Aes256_instance_short() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me aad";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.of("JUnit AAD"));

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.of("JUnit AAD"));

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    public void encrypt_and_decrypt_with_aad_using_same_Aes256_instance_long() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me aad 1, encrypt me aad 2, encrypt me aad 3, encrypt me aad 4, encrypt me aad 5, encrypt me aad 6, encrypt me aad 7, encrypt me aad 8, encrypt me aad 9, encrypt me aad 10";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.of("JUnit AAD"));

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.of("JUnit AAD"));

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }


    @Test
    public void encrypt_and_decrypt_using_different_Aes256_instance_long()
    throws Exception {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aesForEncrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        Aes aesForDecrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me1, encrypt me2, encrypt me3, encrypt me4, encrypt me5, encrypt me6, encrypt me7, encrypt me8, encrypt me9, encrypt me10";

        // run
        byte[] encryptedBytes
            = aesForEncrypt.encrypt(toEncrypt, Optional.empty());

        ByteArrayOutputStream baos
            = new ByteArrayOutputStream();
        baos.write(encryptedBytes);

        String decrypted
            = aesForDecrypt.decrypt(baos.toByteArray(), Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    
    @Test
    public void encrypt_and_decrypt_using_different_Aes256_instance_short()
    throws Exception {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aesForEncrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        Aes aesForDecrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "eNcryPt Me";

        // run
        byte[] encryptedBytes
            = aesForEncrypt.encrypt(toEncrypt, Optional.empty());

        ByteArrayOutputStream baos
            = new ByteArrayOutputStream();
        baos.write(encryptedBytes);

        String decrypted
            = aesForDecrypt.decrypt(baos.toByteArray(), Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    @Test
    public void foo() {
        String s = "ABC";
        int chunkSize = 5;
        String[] chunks = s.split("(?<=\\G.{" + chunkSize + "})");
        System.out.println(Arrays.toString(chunks));
    }
}

Listing 2 - AesUsingMultiplePartEncryptionTest.java class

package org.thoth.crypto.symmetric;

import java.io.ByteArrayOutputStream;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Optional;
import javax.crypto.SecretKey;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import org.thoth.crypto.io.ByteArrayReader;
import org.thoth.crypto.io.ByteArrayWriter;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class AesUsingMultiplePartEncryptionTest {

    static Path secretKeyFile;

    @BeforeClass
    public static void beforeClass() throws Exception {
        // Store the SecretKey bytes in the ./target diretory. Do
        // this so it will be ignore by source control.  We don't
        // want this file committed.
        secretKeyFile
            = Paths.get("./target/Aes256.key").toAbsolutePath();

        // Generate a SecretKey for the test
        SecretKey secretKey
            = new AesSecretKeyProducer().produce();

        // Store the byte[] of the SecretKey.  This is the
        // "private key file" you want to keep safe.
        ByteArrayWriter writer = new ByteArrayWriter(secretKeyFile);
        writer.write(secretKey.getEncoded());
    }


    @Test
    public void encrypt_and_decrypt_using_same_Aes256_instance_long() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me1, encrypt me2, encrypt me3, encrypt me4, encrypt me5, encrypt me6, encrypt me7, encrypt me8, encrypt me9, encrypt me10";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.empty());

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    
    @Test
    public void encrypt_and_decrypt_using_same_Aes256_instance_short() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.empty());

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }


    public void encrypt_and_decrypt_with_aad_using_same_Aes256_instance_short() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me aad";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.of("JUnit AAD"));

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.of("JUnit AAD"));

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    public void encrypt_and_decrypt_with_aad_using_same_Aes256_instance_long() {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aes
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me aad 1, encrypt me aad 2, encrypt me aad 3, encrypt me aad 4, encrypt me aad 5, encrypt me aad 6, encrypt me aad 7, encrypt me aad 8, encrypt me aad 9, encrypt me aad 10";

        // run
        byte[] encryptedBytes
            = aes.encrypt(toEncrypt, Optional.of("JUnit AAD"));

        String decrypted
            = aes.decrypt(encryptedBytes, Optional.of("JUnit AAD"));

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }


    @Test
    public void encrypt_and_decrypt_using_different_Aes256_instance_long()
    throws Exception {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aesForEncrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        Aes aesForDecrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "encrypt me1, encrypt me2, encrypt me3, encrypt me4, encrypt me5, encrypt me6, encrypt me7, encrypt me8, encrypt me9, encrypt me10";

        // run
        byte[] encryptedBytes
            = aesForEncrypt.encrypt(toEncrypt, Optional.empty());

        ByteArrayOutputStream baos
            = new ByteArrayOutputStream();
        baos.write(encryptedBytes);

        String decrypted
            = aesForDecrypt.decrypt(baos.toByteArray(), Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    
    @Test
    public void encrypt_and_decrypt_using_different_Aes256_instance_short()
    throws Exception {
        // setup
        SecretKey secretKey
            = new AesSecretKeyProducer().produce(
                new ByteArrayReader(secretKeyFile).read()
            );

        Aes aesForEncrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        Aes aesForDecrypt
            = new AesUsingMultiplePartEncryption(secretKey);

        String toEncrypt
            = "eNcryPt Me";

        // run
        byte[] encryptedBytes
            = aesForEncrypt.encrypt(toEncrypt, Optional.empty());

        ByteArrayOutputStream baos
            = new ByteArrayOutputStream();
        baos.write(encryptedBytes);

        String decrypted
            = aesForDecrypt.decrypt(baos.toByteArray(), Optional.empty());

        // assert
        Assert.assertEquals(toEncrypt, decrypted);
    }
    
    @Test
    public void foo() {
        String s = "ABC";
        int chunkSize = 5;
        String[] chunks = s.split("(?<=\\G.{" + chunkSize + "})");
        System.out.println(Arrays.toString(chunks));
    }
}

Listing 3 - AesSecretKeyProducer.java class

package org.thoth.crypto.symmetric;

import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class AesSecretKeyProducer {

    /**
     * Generates a new AES-256 bit {@code SecretKey}.
     *
     * @return {@code SecretKey}, never null
     * @throws RuntimeException All exceptions are caught and re-thrown as {@code RuntimeException}
     */
    public SecretKey produce() {
        KeyGenerator keyGen;
        try {
            keyGen = KeyGenerator.getInstance("AES");
            keyGen.init(256);
            SecretKey secretKey = keyGen.generateKey();
            return secretKey;
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    }


    /**
     * Generates an AES-256 bit {@code SecretKey}.
     *
     * @param encodedByteArray The bytes this method will use to regenerate a previously created {@code SecretKey}
     *
     * @return {@code SecretKey}, never null
     * @throws RuntimeException All exceptions are caught and re-thrown as {@code RuntimeException}
     */
    public SecretKey produce(byte [] encodedByteArray) {
        try {
            return new SecretKeySpec(encodedByteArray, "AES");
        } catch (Exception ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 4 - ByteArrayWriter.java class

package org.thoth.crypto.io;

import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.nio.file.Files;
import java.nio.file.Path;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class ByteArrayWriter {

    protected Path outputFile;

    private void initOutputFile(Path outputFile) {
        this.outputFile = outputFile;
    }

    private void initOutputDirectory() {
        Path outputDirectory = outputFile.getParent();
        if (!Files.exists(outputDirectory)) {
            try {
                Files.createDirectories(outputDirectory);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }
    }

    public ByteArrayWriter(Path outputFile) {
        initOutputFile(outputFile);
        initOutputDirectory();
    }

    public void write(byte[] bytesArrayToWrite) {
        try (
            OutputStream os
                = Files.newOutputStream(outputFile);

            PrintWriter writer
                =  new PrintWriter(os);
        ){
            for (int i=0; i<bytesArrayToWrite.length; i++) {
                if (i>0) {
                    writer.println();
                }
                writer.print(bytesArrayToWrite[i]);
            }
        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 5 - ByteArrayReader.java class

package org.thoth.crypto.io;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Scanner;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class ByteArrayReader {

    protected Path inputFile;

    public ByteArrayReader(Path inputFile) {
        this.inputFile = inputFile;
    }

    public byte[] read() {
        try (
            Scanner scanner
                =  new Scanner(inputFile);

            ByteArrayOutputStream baos
                = new ByteArrayOutputStream();
        ){
            while (scanner.hasNext()) {
                baos.write(Byte.parseByte(scanner.nextLine()));
            }
            
            baos.flush();
            return baos.toByteArray();

        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }
}

Listing 6 - Aes.java class

package org.thoth.crypto.symmetric;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.security.SecureRandom;
import java.util.Optional;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public abstract class Aes {

    // If you don't have the Java Cryptography Extension
    // (JCE) Unlimited Strength packaged installed, use
    // a 128 bit KEY_SIZE.
    public static int KEY_SIZE = 256;
    
    public static int IV_SIZE = 12; // 12bytes * 8 = 96bits
    public static int TAG_BIT_SIZE = 128;
    public static String ALGORITHM_NAME = "AES";
    public static String MODE_OF_OPERATION = "GCM";
    public static String PADDING_SCHEME = "PKCS5Padding";

    protected SecretKey secretKey;
    protected SecureRandom secureRandom;

    protected Aes(SecretKey secretKey) {
        this.secretKey = secretKey;
        this.secureRandom = new SecureRandom();
    }


    public byte[] encrypt(String message, Optional<String> aad) {
        try {
            // Transformation specifies algortihm, mode of operation and padding
            Cipher c = Cipher.getInstance(
                String.format("%s/%s/%s",ALGORITHM_NAME,MODE_OF_OPERATION,PADDING_SCHEME)
            );

            // Generate IV
            byte iv[] = new byte[IV_SIZE];
            secureRandom.nextBytes(iv); // SecureRandom initialized using self-seeding

            // Initialize GCM Parameters
            GCMParameterSpec spec = new GCMParameterSpec(TAG_BIT_SIZE, iv);

            // Init for encryption
            c.init(Cipher.ENCRYPT_MODE, secretKey, spec, secureRandom);

            // Add AAD tag data if present
            aad.ifPresent(t -> {
                try {
                    c.updateAAD(t.getBytes("UTF-8"));
                } catch (Exception e) {
                    throw new RuntimeException(e);
                }
            });

            // I demonstrate 2 different ways of getting the
            // encrypted bytes. See the 2 sub-classes which
            // implement the method of this abstract class.
            ByteArrayOutputStream baos = getEncryptedBytes(message, c);
            
            // Concatinate IV and encrypted bytes.  The IV is needed later
            // in order to to decrypt.  The IV value does not need to be
            // kept secret, so it's OK to encode it in the return value
            //
            // Create a new byte[] the combined length of IV and encryptedBytes
            byte[] ivPlusEncryptedBytes = new byte[iv.length + baos.size()];
            // Copy IV bytes into the new array
            System.arraycopy(iv, 0, ivPlusEncryptedBytes, 0, iv.length);
            // Copy encryptedBytes into the new array
            System.arraycopy(baos.toByteArray(), 0, ivPlusEncryptedBytes, iv.length, baos.size());

            // Return
            return ivPlusEncryptedBytes;

        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }


    public String decrypt(byte[] ivPlusEncryptedBytes, Optional<String> aad) {

        try {
            // Get IV
            byte iv[] = new byte[IV_SIZE];
            System.arraycopy(ivPlusEncryptedBytes, 0, iv, 0, IV_SIZE);

            // Initialize GCM Parameters
            GCMParameterSpec spec = new GCMParameterSpec(TAG_BIT_SIZE, iv);

            // Transformation specifies algortihm, mode of operation and padding
            Cipher c = Cipher.getInstance(
                String.format("%s/%s/%s",ALGORITHM_NAME,MODE_OF_OPERATION,PADDING_SCHEME)
            );

            // Get encrypted bytes
            byte [] encryptedBytes = new byte[ivPlusEncryptedBytes.length - IV_SIZE];
            System.arraycopy(ivPlusEncryptedBytes, IV_SIZE, encryptedBytes, 0, encryptedBytes.length);

            // Init for decryption
            c.init(Cipher.DECRYPT_MODE, secretKey, spec, secureRandom);

            // Add AAD tag data if present
            aad.ifPresent(t -> {
                try {
                    c.updateAAD(t.getBytes("UTF-8"));
                } catch (Exception e) {
                    throw new RuntimeException(e);
                }
            });

            // Add message to decrypt
            c.update(encryptedBytes);

            // Decrypt
            byte[] decryptedBytes
                = c.doFinal();

            // Return
            return new String(decryptedBytes, "UTF-8");

        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
    
    
    /**
     * An abstract method to be implemented by a subclass following the 
     * Gang of four <a href="https://www.digitalocean.com/community/tutorials/template-method-design-pattern-in-java">Template Method Design Pattern</a>.
     * This method is to use the {@link Cipher} provided to return the 
     * encrypted bytes of the {@link message} parameter.
     * 
     * @param message The String to be encrypted.
     * @param cipher  The Cipher object used to encrypt the message.
     * @return
     * @throws BadPaddingException
     * @throws IllegalBlockSizeException
     * @throws IOException 
     */
    abstract ByteArrayOutputStream getEncryptedBytes(String message, Cipher cipher) throws BadPaddingException, IllegalBlockSizeException, IOException;
}

Listing 7 - AesUsingSinglePartEncryption.java class

package org.thoth.crypto.symmetric;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.SecretKey;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class AesUsingSinglePartEncryption extends Aes {


    public AesUsingSinglePartEncryption(SecretKey secretKey) {
        super(secretKey);
    }
    

    /**
     * This method demonstrates how to perform a  single-part encryption
     * operation by using the {@link Cipher#doFinal(byte[]) } method to 
     * get all of the encrypted bytes.
     * 
     * @param message The message to encrypt.
     * @param c The {@link Cipher} used to get the encrypted bytes.
     * @return
     * @throws BadPaddingException
     * @throws IllegalBlockSizeException
     * @throws IOException 
     */
    @Override
    ByteArrayOutputStream getEncryptedBytes(String message, Cipher c) throws BadPaddingException, IllegalBlockSizeException, IOException {
        // Create output array to hold all the encrypted bytes
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
         
        // Perform single-part encryption.
        baos.write(
            c.doFinal(message.getBytes("UTF-8"))
        );
        
        return baos;
    }
}

Listing 8 - AesUsingMultiplePartEncryption.java class

package org.thoth.crypto.symmetric;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.SecretKey;

/**
 *
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class AesUsingMultiplePartEncryption extends Aes {


    public AesUsingMultiplePartEncryption(SecretKey secretKey) {
        super(secretKey);
    }
    

    /**
     * This method demonstrates how to perform a  multiple-part encryption
     * operation by using the {@link Cipher#update(byte[]) } and the 
     * {@link Cipher#doFinal() } methods to get all of the encrypted
     * bytes. This method <b>simulates</b> the operation by dividing the
     * {@link message} parameter into many strings and operating on those
     * strings individually. In reality you would probably never do this, but
     * it's an easy way to demonstrate to perform a  multiple-part encryption.
     * 
     * @param message The message to encrypt. This method will <b>simulate</b>
     * multiple-part encryption by <b>unnecessarily</b> dividing this string into many strings.
     * @param c The {@link Cipher} used to get the encrypted bytes.
     * @return
     * @throws BadPaddingException
     * @throws IllegalBlockSizeException
     * @throws IOException 
     */
    @Override
    ByteArrayOutputStream getEncryptedBytes(String message, Cipher c) throws BadPaddingException, IllegalBlockSizeException, IOException {
        // Create output array to hold all the encrypted bytes
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        
        // Chunk the message into many strings to simulate multiple-part encryption
        int chunkSize = 5;
        String[] chunks = message.split("(?<=\\G.{" + chunkSize + "})");
        
        // Add each chunk to a multiple-part encryption.
        // Don't forget to collect the encypted bytes
        // along the way!
        for (String chunk : chunks) {
            baos.write(
                c.update(chunk.getBytes("UTF-8"))
            );
        }
        
        // Finish multi-part encryption. Again,
        // Don't forget to collect the encypted bytes
        // along the way!
        baos.write(
            c.doFinal()
        );
        
        return baos;
    }
}

Summary

Encryption isn’t easy. And easy examples will result in implementations with security vulnerabilities for your application. If you need a single key, symmetric, encryption algorithm, use cipher AES/GCM/PKCS5Padding with a 256 bit key and a 96 bit IV.

References

Java Cryptography Extension (JCE) Unlimited Strength. (n.d.). Retrieved from http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html.

Sheth, M. (2017, April 18). Encryption and Decryption in Java Cryptography. Retrieved from https://www.veracode.com/blog/research/encryption-and-decryption-java-cryptography.

cpast[ Says GCM IV is 96bit which is 96/8 = 12 bytes]. (2015, June 4). Encrypting using AES-256, can I use 256 bits IV [Web log comment]. Retrieved from https://security.stackexchange.com/questions/90848/encrypting-using-aes-256-can-i-use-256-bits-iv.

Bodewes[ Says GCM IV is strongly recommended to be 12 bytes (12*8 = 96) but can be of any size. Other sizes will require additional calculations], M. (2015, July 7). Ciphertext and tag size and IV transmission with AES in GCM mode [Web log comment]. Retrieved from https://crypto.stackexchange.com/questions/26783/ciphertext-and-tag-size-and-iv-transmission-with-aes-in-gcm-mode.

Figlesquidge. (2013, October 18). What’s the difference between a ‘cipher’ and a ‘mode of operation’? [Web log comment]. Retrieved from https://crypto.stackexchange.com/questions/11132/what-is-the-difference-between-a-cipher-and-a-mode-of-operation.

Toust, S. (2013, February 4). Why does the recommended key size between symmetric and asymmetric encryption differ greatly?. Retrieved from https://crypto.stackexchange.com/questions/6236/why-does-the-recommended-key-size-between-symmetric-and-asymmetric-encryption-di.

Karonen, I. (2012, October 5). What is the main difference between a key, an IV and a nonce?. Retrieved from https://crypto.stackexchange.com/questions/3965/what-is-the-main-difference-between-a-key-an-iv-and-a-nonce.

Block cipher mode of operation. (2017, November 6). Wikipedia. Retrieved from https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Initialization_vector_.28IV.29

Jakobsen, Peter. (2022, March 10). javax.crypto.AEADBadTagException: Tag mismatch! when password.length >= 16 - Stack Overflow [Web log post]. Retrieved from https://stackoverflow.com/questions/71422498/javax-crypto-aeadbadtagexception-tag-mismatch-when-password-length-16.

December 20, 2017

Choosing Java Cryptographic Algorithms Part 1 - Hashing

Abstract

This is the 1st of a three-part blog series covering Java cryptographic algorithms. The series covers how to implement the following:

  1. Hashing with SHA–512
  2. Single-key symmetric encryption with AES–256
  3. Public/Private key asymmetric encryption with RSA–4096

This 1st post details how to implement SHA–512 hashing. Let’s get started.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Java 1.8.0_152_x64
  • NetBeans 8.2 (Build 201609300101)
  • Maven 3.0.5 (Bundled with NetBeans)

Download

Visit my GitHub Page to see all of my open source projects. The code for this post is located in project: thoth-cryptography

Hashing

About

Hashing is a one-way cryptographic algorithm which takes in a message of any length and outputs a repeatable, fixed-length, and one-way digest (hash) of the message. Being one-way, it’s supposed to be impossible to regenerate the original message from the hash. Identical messages will always generate the same hash.

A hash can be used to authenticate an original message. A common use of hashing is validating passwords. Instead of storing the password itself, the hash of the password is stored. To verify a password, the stored hash is compared with a new hash of an incoming password during a login process.

Because identical messages generate the same hash, a salt value is used to make the hash more secure (Salt, 2017, para. 1). Consider a case where the same password is used by multiple users. A salt value combined with the original password allows for unique hash values. This is important because if the hashed values are ever compromised, identical hashes let a hacker know those passwords are the same.

SHA–512

Research done as of today seems to indicate the best and most secure algorithm for hashing is SHA–512, which uses 64-bit words (Secure Hash Algorithms, 2017, para.2). Let’s take a look at an example.

NOTE Don’t use MD5 as a secure hash. It has many vulnerabilities (MD5, 2017, para. 1). Limit MD5 use to checksums and data verification.

Example

Listing 1 is the ShaTest.java unit test demonstrating how to hash. Listing 2 is the Sha.java class which does the hash.

Listing 1 - ShaTest.java class

package org.thoth.security.hash;

import java.util.Optional;
import org.junit.Assert;
import org.junit.Test;

/**
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class ShaTest {

    @Test
    public void test_hash_with_optional_to_hex() throws Exception {
        // setup
        String username = "mjremijan";
        String password = "super!secret";
        Sha sha = new Sha();

        // test
        String asHex
            = sha.hashToHex(password, Optional.of(username));

        // assert
        Assert.assertEquals(
              "F38CD5290D11B20159E36740843A8D93CFDFA395CF594F328613EF5C7BA42D9EAC00BF3EE47B7E8CE1587040B36365F05C8E15E9392C288A1D7C4CFB66097848"
            , asHex);
    }

    @Test
    public void test_hash_without_optional_to_hex() throws Exception {
        // setup
        String password = "super!secret";
        Sha sha = new Sha();

        // test
        String asHex
            = sha.hashToHex(password, Optional.empty());

        // assert
        Assert.assertEquals(
              "516A1FE9D87FE5B953D91B48B1A2FFA5AE5F670914C1B6FE0835D8877918DC4E8BC8FB8CCD520DBA940C21B4F294DFD1B4EFF2E06AB110C6A06E35068251C1DD"
            , asHex);
    }


    @Test
    public void test_hash_with_optional_to_base64() throws Exception {
        // setup
        String username = "mjremijan";
        String password = "super!secret";
        Sha sha = new Sha();

        // test
        String asBase64
            = sha.hashToBase64(password, Optional.of(username));

        // assert
        Assert.assertEquals(
              "84ZVKQ0RSGFZ42DAHDQNK8/FO5XPWU8YHHPVXHUKLZ6SAL8+5HT+JOFYCECZY2XWXI4V6TKSKIODFEZ7ZGL4SA=="
            , asBase64);
    }


    @Test
    public void test_hash_without_optional_to_base64() throws Exception {
        // setup
        String password = "super!secret";
        Sha sha = new Sha();

        // test
        String asBase64
            = sha.hashToBase64(password, Optional.empty());

        // assert
        Assert.assertEquals(
              "UWOF6DH/5BLT2RTISAL/PA5FZWKUWBB+CDXYH3KY3E6LYPUMZVINUPQMIBTYLN/RTO/Y4GQXEMAGBJUGGLHB3Q=="
            , asBase64);
    }
}

Listing 2 - Sha.java class

package org.thoth.security.hash;

import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Base64;
import java.util.Optional;

/**
 * @author Michael Remijan mjremijan@yahoo.com @mjremijan
 */
public class Sha {

    public String hashToHex(String hashMe, Optional<String> salt)
    throws NoSuchAlgorithmException, UnsupportedEncodingException {
        byte[] bytes
            = hash(hashMe, salt);

        StringBuilder sp
            = new StringBuilder();

        for (int i = 0; i < bytes.length; i++) {
            sp.append(Integer.toString((bytes[i] & 0xff) + 0x100, 16).substring(1));
        }

        return sp.toString().toUpperCase();
    }

    public String hashToBase64(String hashMe, Optional<String> salt)
    throws NoSuchAlgorithmException, UnsupportedEncodingException {
        return Base64.getEncoder().encodeToString(
            hash(hashMe, salt)
        ).toUpperCase();
    }

    public byte[] hash(String hashMe, Optional<String> salt)
    throws NoSuchAlgorithmException, UnsupportedEncodingException {
        MessageDigest md
            = MessageDigest.getInstance("SHA-512");

        md.update(hashMe.getBytes("UTF-8"));
        salt.ifPresent(s -> {
            try { md.update(s.getBytes("UTF-8")); } catch (Exception e) {throw new RuntimeException(e);}
        });

        return md.digest();
    }
}

Summary

Hashing is pretty easy. Choose a strong hashing algorithm like SHA–512 for securing your application data. Avoid MD5 for securing data. Stay current as to which algorithms are strong and safe. Update your application if you are using an older algorithm which has vulnerabilities or is compromised.

References

Salt (cryptography). (2017, November 3). Wikipedia. Retrieved from https://en.wikipedia.org/wiki/Salt_(cryptography).

Secure Hash Algorithms. (2017, November 25). Wikipedia. Retrieved from https://en.wikipedia.org/wiki/Secure_Hash_Algorithms.

MD5. (2017, November 22). Wikipedia. Retrieved from https://en.wikipedia.org/wiki/MD5.