Processing Elements for Enterprise Integration Needs

In February, AdroitLogic released UltraESB-X, a completely redesigned ESB which is developed by the very same team who developed the UltraESB six years ago. UltraESB-X is developed inline with UltraStudio, a user friendly graphical tool to easily create, manage and test integration flows. In the user’s point of view, UltraESB-X mainly has two new concepts namely connectors and processing elements.

What is a Processing Element?

A processing element is an element which performs a specific processing requirement or business logic on the message. It can be one among a wide range of operations such as filtering using conditional branching, transformation, flow control, scope handling such as a transactional scope and many more. Usually processing elements in the UltraStudio editor are represented by an icon with a  bluish background except for  a few exceptions. One of those exceptions is the scope processing element family which uses a purple background and another set of special flow control processing elements which uses a reddish background.

pe(1)

The structure of a processing element is as shown in the above image. There is an In Port (grey colour) which takes a message in, then depending on the type of processing there can be zero or more Out Ports (light yellow) which gives a processed message out and finally there is the On Exception Port (red) which is used to forward the message in case an exception occurs during the processing logic of the element. UltraESB-X ships with dozens of built-in processing elements which covers a wide range of integration processing requirements and more processing elements are continuously added with each release cycle. Current list of processing elements can be browsed through the developer portal of AdroitLogic here. Apart from the inbuilt processing elements, UltraStudio allows user to write their own custom processing elements as explained in the documentation.

What is a Connector?

A connector is an element which is used to receive or send messages between the ESB and an external system via a given protocol, such as HTTP/S, JMS, Files, Databases, or Web Services. Connectors are categorized into two categories as Ingress and egress connectors based on the message flow. Ingress connectors, represented by an icon with a greenish background, are components dedicated for receiving messages into the integration flow and egress connectors, represented by an icon with an orange background, are the connectors for sending out messages from the integration flow.

Ingress Connectoringress

Structure of an ingress connector is as shown in the above diagram. There is a Processor Port (light yellow in colour) which is basically an out port used to forward  the receiving message into a processing element. Next there is the On Exception Port (red) which will be used to forward the message in case of an exception occurs during the message receiving process. Then an optional In Port (grey in colour) is presented depending on the message exchange pattern of the protocol supported by the connector. For example In Port is there in HTTP connectors which is 2-way (request-response MEP) but it is not there in one-way  connectors such as JMS, File, SFTP etc.

Egress Connectoregress(1)

Moving on to the egress connectors, the structure is as shown above. The In Port (grey) which takes in a message to be sent out via the connector. Again depending on the message exchange pattern of the  protocol used by the connector, there may or may not be a Response Processor Port (light yellow). For example again the HTTP connectors have the Response Processor Port (as it is 2-way) but most of the other one-way connectors does not have that port. Then the On Exception Port which is used to forward the message in case of a failure in the sendout process. Additionally egress connectors have a port called Connector Operation Port which is a special kind of a port which accepts an operation specific to the connector and that special logic will be applied in message preparation before the send-out process.

In the following sections this article will focus on introducing the use of few processing elements leading in to a complete sample scenario at the final stage.

XML Validation –  XSD Validator Processing Element

XML Validation is the process of checking a document written in XML to confirm that it is both well-formed and also “valid” in that it follows a defined structure. Generally this structure is defined using an XSD (XML Schema Definition) file. XSD Validator processing element is focusing on this task of validating an XML payload against one or more given schemas.

xsd-validator

Above diagram shows the port structure of the XSD Validator element. Notice the On Successful Validation Port and On Failed Validation Port which will be used to forward the message when the XML validation results in a success or a failure respectively. Following diagram shows a very basic flow to depict the use of XSD Validator element, captured from AdroitLogic UltraStudio, the integration flow designer. In this flow, there is an HTTP ingress connector which takes in a message and then passes it through an XSD Validator and forwards the message into two different HTTP egress connectors depending on the validation result.xsd-flow

Next diagram below shows the configuration view of validator element in UltraStudio.

xsd-conf

XML to JSON Conversion – XML to JSON Transformer

xml-to-json

XML To JSON Transformer processing element handles conversion of XML payload to JSON. Similar to this, UltraESB-X includes builtin processing elements covering most of the common payload conversion requirements.xml-flow

xml-conf

If-Else Condition Evaluation – Conditions Evaluator

condition-evaluator

When implementing business logic the requirement for conditional operators is inevitable. Conditions Evaluator processing element is the if-then-else implementation of UltraESB-X. Following is a sample XML configuration of the Conditions Evaluator processing element. UltraStudio generates this configuration when we configure the processing element through user interface.

<bean id="0bd5ecf6-6b51-7733-49f9-48b4feaac455" class="org.adroitlogic.x.processor.flowControl.ConditionsEvaluator">
   <property name="ifOutPort" ref="7e0fe076-3d8c-208b-76b9-bfb818b77429"/>
   <property name="elseOutPort" ref="a92a2828-b20b-3fa1-0dd3-fdde9812faec"/>
   <property name="predicateType" value="VARIABLE"/>
   <property name="varKey" value="id_one"/>
   <property name="varType" value="String"/>
   <property name="predicateFunction" value="EQUALS"/>
   <property name="matchingValue" value="3"/>
</bean>

condition-conf

As you can see there are several configuration parameters. One is predicate type which basically defines the type on which the evaluation is done. We can evaluate on a header, a scope variable, a message property or on the payload itself. Upon selecting the predicate type, the name of the variable of the selected type must be given. Then we can select the type which could be one of Boolean, Integer, Long, Float, Double or String. Next the predicate function should be selected and it could be one of EQUALS, MATCH, CONTAINS, NULL, NOT NULL, EXISTS.

EQUALS evaluates java Object.equals() operation. MATCH evaluates String.matches() operation. CONTAINS evaluates String.contains() function. NOT_NULL and EXISTS simply evaluates existence of such value where NULL evaluates the opposite. Predicate functions EQUALS, MATCH and CONTAINS require a matching value to be configured.

Apart from Conditions Evaluator UltraESB-X also consists of processing elements that support Switch-Case, For-Loop and While-Loop as well.

JSON Path Extraction – JSON Path Extractor

json-path-extractor

JSON Path Extractor processing elements allows us to extract a JSON element and store it as a scope variable easily by providing a json path to the configuration. Then we can use that extracted element to make decisions such as using the extracted value in a conditions evaluator. json-flow

json-conf

Failover and Load Balancing – Egress Load Balancer

egress-load-balancer

Load balancing and failover is one of the most sought after features in an ESB. In the context of an ESB load balancing means distribution of workloads across multiple endpoints and failover means basically attempting to send to redundant or alternative end points in case of failure. UltraESB-X supports this requirements through Egress Load Balancer processing element. A sample XML configuration of the Egress Load Balancer processing element is shown below.

elb-flow

<bean id="7b7f6ccc-acba-c2e6-5125-e9aa49e5e944"      class="org.adroitlogic.x.processor.flowControl.lb.EgressLoadBalancer">
   <property name="endpointList">
       					<list>
           <ref bean="dcec8a87-3922-d62d-0da7-dc87e52ccae6"/>
           <ref bean="36faacbf-f4fa-723b-3170-e5c84e7ab262"/>
           <ref bean="14defbf2-cb22-24c6-d717-17ad9b091182"/>
       </list>
   </property>
   <property name="loadBalancingAlgorithm" value="round-robin"/>
   <property name="retryAttempts" value="3"/>
   <property name="safeToRetryErrorCodes" value="E0001"/>
</bean>

When configuring the Egress Load Balancer through the UI, there are three main parameters. One is the load balancing algorithm. Egress Load Balancer supports a list of load balancing algorithms including round robin, round robin with fall over, fall over, weighted, weighted with fail over, random and random with fail over. Re-try attempts parameter must be configured when the LB algorithm is weighted with failover or random with failover. Safe to retry error codes parameter accepts a comma separated list of error codes and it is only required when the LB algorithm is a failover one. Using this parameter we basically say the processing element that it can proceed to retry if the error code is one on the given list.

elb-conf

Getting All The Pieces Together

Here is our complete sample scenario using the above mentioned elements. Note that connections of handling exception paths (On Exception Port connections) are omitted for simplicity.full-flow

Sample Input

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Header><routing xmlns="http://someuri">xadmin;server1;community#1.0##</routing></soapenv:Header>
   <soapenv:Body>
       <m:buyStocks xmlns:m="http://services.samples/xsd">
           <promiseList>
               <id>1</id>
               <orderId>1234-5678-2345-5467-SH1</orderId>
               <comments>grade-1 quality</comments>
           </promiseList>
           <promiseList>
               <id>2</id>
               <orderId>2345-1222-6778-9999-SH2</orderId>
               <comments>grade-2 quality</comments>
           </promiseList>
           <promiseList>
               <id>3</id>
               <orderId>4568-7678-9876-1211-SH2</orderId>
               <comments>grade-3 quality</comments>
           </promiseList>
       </m:buyStocks>
   </soapenv:Body>
</soapenv:Envelope>

Output from XML to JSON Transformer

{
 "Body":{
   "buyStocks":{
     "promiseList":{
       "id":"1",
       "orderId":"1234-5678-2345-5467-SH1",
       "comments":"grade-1 quality"
     },
     "promiseList":{
       "id":"2",
       "orderId":"2345-1222-6778-9999-SH2",
       "comments":"grade-2 quality"
     },
     "promiseList":{
       "id":"3",
       "orderId":"4568-7678-9876-1211-SH2",
       "comments":"grade-3 quality"
     }
   }
 }
}

Message Flow Path

Highlighting the message flow path is an important feature of UltraStudio which can be more than handy in development and debugging. Following diagram shows the message flow path of the sample scenario for a single request.message-path

Inspecting Message Details

Another useful feature of UltraStudio is the ability to inspect message details after a message has been sent simply by clicking the message icon on the message path as shown below.

Screenshot from 2017-06-08 19-50-23

Following is the message details of the result from XML to JSON Transformer.

Screenshot from 2017-06-08 19-49-38

~Rajind Ruparathna

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s