MuleSoft 4: Using the Amazon DynamoDB Connector

MuleSoft 4: Using the Amazon DynamoDB Connector

Overview

The following provides a quick example of how to use the “Amazon DynamoDB Connector – Mule 4” using Anypoint Studio. While there is a lot of documentation available for both this connector and Amazon’s DynamoDB API, none could be found that specifically shows how to structure the JSON required within the configuration settings for the connector.

Prerequisites

Setting up your own project is of course optional. The following prerequisites assume you want to set up an actual, testable flow within Anypoint Studio. This article is intended to illustrate only a few simple concepts around structuring the JSON that is used to communicate between MuleSoft and DynamoDB. This article is not expected to provide a detailed, end-to-end instruction on how to set-up and configure a MuleSoft flow in its entirety. You’ll need to rely on your own knowledge and experience to create a fully functional application from these specifications.

To create your own project, the following assumptions are made:
• You are using MuleSoft 4 and Anypoint Studio 7 or greater.
• You have installed the Amazon DynamoDB Connector – Mule 4 connector into Anypoint Studio.
• You have an AWS account, have created a DynamoDB database, and have created a table that will hold data as it pertains to this example.
• You have established a global configuration element in your flow that uses the following properties (See configuration settings below):

• accessKey= {your AWS key}
• secretKey= {your AWS secret key}
•endPoint= {your AWS region endPoint – e.g. http://dynamodb.us-east-2.amazonaws.com
•region= {your AWS region – e.g. us-east-2}
• propertiesTableName= {your table name that will hold your property data}
•columnName= {your table key column. In our example we use “appName” for the key column}.

• You have created a flow with an HTTP listener component that can route either a ‘get’ or a ‘put’ to their respective sub-flows (see the Get Operation and Put Operation sections below for more information)

The configuration XML for the global element pertaining to the DynamoDB is as follows:

Requirements

In this example, I’ll be creating an API whereby any number of properties (i.e. key/value pairs) may be stored for a given application by version number. The API will also include a ‘get’ operation to retrieve all properties for an application version.

An example Use Case would involve an application with many configuration properties that need to be stored. The application may then retrieve these properties from the stored data as needed. The properties might include things such as connection-string, login, password, etc.

In a real application we might also include a variable for specifying the ‘environment’; however, this simple example is intended to demonstrate the MuleSoft 4 connector for DynamoDB so we’ll keep it simple for now.

Put Operation

Client Configuration

The following information explains how the API will be called. There are several HTTP client tools that may be used for testing web services. Postman is one example and the one that I generally use for my testing.

MuleSoft Configuration

The MuleSoft flow may include multiple components; however, only those related to the connection with DynamoDB will be described here.

Transform Message

The DynamoDB API passes information using a proprietary JSON format that is unique to the API. Here is the format, where “string” is the name of an attribute. You can see how attribute value types are declared using a lettering system.

AWS provides good documentation around all the API operations which detailed descriptions about how to structure these method calls.

https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Operations_Amazon_DynamoDB.html

The trick is knowing which subset of the JSON structure to pass from MuleSoft to the API. Because the table name is declared through another configuration property in the DynamoDB component, the markup that gets passed only includes the nodes found under the “Item” node.

The following shows the expression used in our example to store multiple key/value pairs under a single application version. Notice that this includes an array of values which is not well described in the AWS documentation. It’s helpful for the MuleSoft developer to understand how this should be formatted.

DynamoDB Put-Item

The Configuration XML for the put item connector is as follows:

The UI editor reflects these settings with the addition of the Item field which is set to #[payload] so it uses output from the previous transform processor. The Table name field is set to a configuration file property; however, this may also be hard coded for simplicity.

Get Operation (All Properties)

Client Configuration

The ‘get’ operation mirrors the ‘put’, with the absence of passing any JSON in the body. In return, however, are the stored properties in the form of JSON.

MuleSoft Configuration

The MuleSoft flow may include multiple components; however, only those related to the connection with DynamoDB will be described here.

Transform Message

The transform component concatenates the URI parameters by combining the application name and version. This combination forms the key that will be used to retrieve the properties stored in the DynamoDB database for the specified application.

DynamoDB Get-Item

The xml for the get operation is configured as follows:

The UI editor reflects these settings with the addition of the Key field which is set to #[payload] so it uses output from the previous transform processor. The Table name field is set to a configuration file property; however, this may also be hard coded for simplicity.

Transform Return Message

The DynamoDB will return the results as a Java object. The following transform may be used to convert this to a relatively easier format to read and consume by the client.

Summary

We’ve learned through this simple example how to use the “Amazon DynamoDB Connector – Mule 4” using Anypoint Studio; and more specifically, how to structure the JSON required within the configuration settings for the connector.
If you would like more information pertaining to MuleSoft, or how we could help you implement a MuleSoft strategy in your organization, contact us at CleanSlate.

Testing the Waters

Testing the Waters

Does testing really pay off in the World of software development?

As software developers, we are constantly hounded about the importance of automated testing. We hear diatribes about test driven development, evangelist who swear by unit testing, and coders who claim testing has changed how they work. By now most people in the industry would agree there is a high level of credence to the importance for this type of testing. What I find surprising, despite the apparent benefits, is that most companies don’t do it; and in many cases it is the same who swear by it who aren’t doing it.

So why aren’t more companies utilizing automated testing in their software development? For starters, it takes more time. Of course, it’s easier to write code without companion unit tests for every method created. Writing tests often means writing as much or perhaps more lines of code than the actual solution. With deadlines looming, it is hard to justify this added effort and our tendency is to just get the application working.

Creating unit tests is hard!

Many times, it is challenge enough to just come up with the logic necessary to build an application. Couple that with factoring your code so that it can be easily tested makes the coding effort doubly as hard. One must consider potentially thorny concepts such as interfaces, dependency injection, and mocking. Questions arise such as what to test, how much coverage is preferred, or whether we care to distinguish between pure unit tests or integration tests.

Testing can also be distracting; it’s easy to lose sight of the true goal when you are worried about constructing the perfect test framework. No amount of testing, however, can overcome a solution that fails to meet its original requirements. While on the topic of requirements, with the popularity of agile methodologies, functional requirements are often more fluid. This flexibility in scope potentially sets us up for not only re-writing code, but also re-writing tests to match. Our once perfect suite of test cases can easily become stale.

Despite all the added complexities, why is automated testing now more important than ever? There are two significant reasons why automated testing has become so crucial in the software development lifecycle. First, I point to the ‘rising cost of defects’ paradigm which illustrates the exponentially increasing cost of finding a bug later in the development cycle. A bug discovered through automated testing is far less costly than the same bug discovered during acceptance testing. Later in the process; the end user must capture, document, and submit the bug. The development team must then prioritize, assign, re-analyze, and refactor. All of this is time and money that could have been saved if detected earlier through automated testing.

The second factor that has brought about increased importance on automated testing is the rise of the agile methodology. Nearly everyone it seems is using some form of agile development practices these days. This means that deployments are happening far more frequently than would happen in the old days of ‘waterfall’. With a repetitive release cycle, it becomes paramount that testing be automated. To execute the same test repeatedly is not only tedious but also invites the possibility of excluding certain tests on future runs. Automated testing not only verifies that new code is working but helps keep us covered in terms of regression.

If you or your company are in the business of developing software, ‘test the waters’ and implement automated testing in your development operations. Despite the initial investment in skills, tooling and time, your leap of faith will become apparent as your product matures. Speaking from experience, done right, a solid framework around automated testing will pay dividends before you reach the end of your first release cycle.

If you would like more information about automated testing, or how we could help you implement testing in your development environment, contact us at CleanSlate.