This tutorial is all about JBoss Drools Fusion. Before proceeding with this article, lets assume that readers have a basic knowledge into JBoss Drools and working of a Java n-tier application. To brief, Drools is a Business Rules Management System (BRMS) solution. It provides a core Business Rules Engine (BRE), a web authoring and rules management application (Drools Workbench) and an Eclipse IDE plugin for core development. To have more understanding of JBoss Drools please refer my article here.
Table Of Contents
1. Introduction to Drools Fusion
1. Introduction to Drools Fusion
Drools Fusion is based on event driven architecture which is explained in detail below. Drools Fusion includes complex event processing features. Complex Event Processing (CEP) is used to process a large stream of information and can be used for real-time event monitoring or correlation. Events can be processed in two ways, that is either in the ‘stream’ mode or in the ‘cloud’ mode.
The cloud mode would be useful in the following circumstances: user behavior, market data, and activity monitoring.
The stream mode could be most useful in applications, such as real-time event monitoring, event correlation, and sensor networks.
2. What is Event Driven Architecture
An event represents a significant change of state. As we have said above, Drools Fusion is based on Complex Event Processing (CEP), event-driven architecture is a concept that is very easy to bond to the CEP as it defines a simple architecture to promote the production, detection, consumption, and reaction to the events. The concept of this architecture is to focus on application components.
For example, when a consumer purchases a car, the car’s state changes from “for sale” to “sold”. A car dealer’s system architecture may treat this state change as an event whose occurrence can be made known to other applications within the architecture. From a formal perspective, what is produced, published, propagated, detected or consumed is a (typically asynchronous) message called the event notification, and not the event itself, which is the state change that triggered the message emission.
Events do not travel, they just occur. An event-driven system typically consists of event emitters (or agents), event consumers(or sinks), and event channels.
Event Emitters have the responsibility to detect, gather, and transfer events. An Event Emitter does not know the consumers of the event, it does not even know if a consumer exists, and in case it exists, it does not know how the event is used or further processed.
Event Consumers or Sinks have the responsibility of applying a reaction as soon as an event is presented. The reaction might or might not be completely provided by the sink itself. For instance, the sink might just have the responsibility to filter, transform and forward the event to another component or it might provide a self-contained reaction to such event.
Event channels are conduits in which events are transmitted from event emitters to event consumers. Events can travel either in the form of messages or there can be a point to point communication which basically require a particular framework to work.
2.1 Event processing styles
There are three general styles of event processing: simple, stream, and complex. The three styles are often used together in a mature event-driven architecture.
Simple event processing: Simple event processing concerns events that are directly related to specific, measurable changes of condition. In simple event processing, a notable event happens which initiates downstream action(s). For example, a sensor detecting changes in tire pressures.
Event stream processing: In event stream processing (ESP), both ordinary and notable events happen. ESP is more about real-time processing of huge volume of events. For example, calculating the real-time average transaction volume over time.
Complex event processing: Complex Event Processing (CEP) deals with complex events. A complex event is a set of simple events. For example, a sequence of large withdrawals may raise a suspicious transaction event. The simple events are considered to infer that a complex event has occurred.
3. Complex Event Processing
As we have already mentioned before, that Drools Fusion is based on Complex Event Processing, lets see what CEP is in little detail here. Complex Event Processing (CEP) is a technique in which incoming data about what is happening (event data) is processed more or less as it arrives to generate higher-level, more-useful, summary information (complex events). Event processing platforms have built-in capabilities for filtering incoming data, storing windows of event data, computing aggregates and detecting patterns.
In formal terminology, CEP software is any computer program that can generate, read, discard or perform calculations on complex events. A complex event is an abstraction of one or more base (input) events. Complex events may signify threats or opportunities that require a response from the business. One complex event may be the result of calculations performed on a few or on millions of base events from one or more event sources.
CEP is growing rapidly because CEP, in a technical sense, is the only way to get information from event streams in real-time or near-real time. The system has to process the event data more or less as it arrives so that the appropriate action can be taken quickly.
Event Stream Processing (ESP) was focused on the capabilities of processing streams of events in (near) real time, where the main focus of Complex Event Processing (CEP) was on the correlation and composition of atomic events into complex (compound) events.
In short, CEP is about detecting and selecting the interesting events (and only them) from an event cloud, finding their relationships and inferring new data from them and their relationships.
4. Drools Fusion
Drools Fusion is a Drools module that is a part of the Business Logic Integration Platform. It is the Drools event processing engine covering both CEP and ESP. Each event has a type, a time of occurrence, and possibly, a duration. Both point in time (zero duration) and interval-based events are supported. Events can also contain other data like any other facts—properties with a name and type. All events are facts but not all facts are events. An event’s state should not be changed. However, it is valid to populate the unpopulated values. Events have clear life cycle windows and may be transparently garbage collected after the life cycle window expires (for example, we may be interested only in transactions that happened in the last 24 hours). Rules can deal with time relationships between events.
So far, in our previous Drools article, we’ve dealt with facts that we insert into a KieSession and how they can match a specific rule. Facts are very similar to events, except events have one extra feature: time of occurrence. Events are simply the data about any domain (represented as a Java object), along with the information about the time in which this information was true.
Pretty much anything that we record taking place at a specific time can be an event.
4.1 Declaring CEP-based Rules
As we know that rules should be atomic and when worked together must be able to achieve management of complex scenarios, this approach is also very well aligned with CEP as each different rule can deal with one aspect of the aggregation, composition, or abstraction of other events. They can work together to achieve real-time resolution of very complex event situations.
In the following sections we will see how to define an event. Before that lets have an insight into two main types of events—punctual and interval events.
Punctual events: They are the events that occurred at a specific instance in time. They may represent the exact moment when a change in the reality of our domain model appeared or they may have a life span that is too short to be considered. An example of punctual events is sensor readings, which will associate a specific value from the sensor with a specific time of the reading. Transactions can also be considered punctual events if they are so short lived that we can ignore their duration.
Interval Events: They are events that have two distinctive moments in time: the moment they started and the moment they ended. This makes the interval events a bit more complex to compare than the punctual events. If you have two punctual events, you can only compare whether they happened at the same time, before or after each other. For interval events on the other hand, you might compare cases where one event started and ended during another one, just to name a scenario.
4.2 Fraud Detection
To better understand the concept of CEP and Drools Fusion, lets see an example of fraud detection system. Fraud in banking systems is becoming a major concern. The amount of online transactions is increasing every day. An automatic system for fraud detection is needed. The system should analyze various events happening in a bank and, based on a set of rules, raise an appropriate alarm.
This problem cannot be solved by the standard Drools rule engine. The volume of events is huge and it happens asynchronously. If we simply insert them into the knowledge session, we would soon run out of memory. While the Rete algorithm behind Drools doesn’t have any theoretical limitation on number of objects in the session, we could use the processing power more wisely. Drools Fusion is the right candidate for this kind of task.
4.2.1 Problem description:
Let’s consider the following set of business requirements for the fraud detection system:
- If a notification is received from a customer about a stolen card, block this account and any withdrawals from this account.
- Check each transaction against a blacklist of account numbers. If the transaction is transferring money from/to such an account, then flag this transaction as suspicious with the maximum severity.
- If there are two large debit transactions from the same account within a ninety second period and each transaction is withdrawing more than 300% of the average monthly (30 days) withdrawal amount, flag these transactions as suspicious with minor severity.
- If there is a sequence of three consecutive, increasing, debit transactions originating from a same account within a three minute period and these transactions are together withdrawing more than 90% of the account’s average balance over 30 days, then flag those transactions as suspicious with major severity and suspend the account.
- If the number of withdrawals over a day is 500% higher than the average number of withdrawals over a 30 day period and the account is left with less than 10% of the average balance over a month (30 days), then flag the account as suspicious with minor severity.
- Duplicate transactions check—if two transactions occur in a time window of 15 seconds that have the same source/destination account number, are of the same amount, and just differ in the transaction ID, then flag those transactions as duplicates.
Monitoring:
- Monitor the average withdrawal amount over all of the accounts for 30 days.
- Monitor the average balance across all of the accounts.
4.2.2 Designing a Fraud Detection System:
We already have the requirements here and now we would need a way to flag a transaction as suspicious.
This state can be added to an existing Transaction type, or we can externalize this state to a new event type. We’ll do the latter. The following new events will be defined:
TransactionCreatedEvent— An event that is triggered when a new transaction is created. It contains a transaction identifier, source account number, destination account number, and the actual amount transferred.TransactionCompletedEvent— An event that is triggered when an existing transaction has been processed. It contains the same fields as theTransactionCreatedEventclass.AccountUpdatedEvent— An event that is triggered when an account has been updated. It contains the account number, current balance, and the transaction identifier of a transaction that initiated this update.SuspiciousAccount— An event triggered when there is some sort of a suspicion around the account. It contains the account number and severity of the suspicion. It is an enumeration that can have two valuesMINORandMAJOR. This event’s implementation is shown in the following code.SuspiciousTransaction— Similar toSuspiciousAccount, this is an event that flags a transaction as suspicious. It contains a transaction identifier and severity level.LostCardEvent— An event indicating that a card was lost. It contains an account number.
One of events described—SuspiciousAccount—is shown in the following code. It also defines SuspiciousAccountSeverity enumeration that encapsulates various severity levels that the event can represent. The event will define two properties. One of them is already mentioned, severity and the other one will identify the account— accountNumber.
SuspiciousAccount.java
package com.drools.myexample;
import java.io.Serializable;
import org.apache.commons.lang.builder.ToStringBuilder;
/**
* Marks an account as suspicious
*/
public class SuspiciousAccount implements Serializable {
public enum SuspiciousAccountSeverity {
MINOR, MAJOR
}
private final Long accountNumber;
private final SuspiciousAccountSeverity severity;
public SuspiciousAccount(Long accountNumber, SuspiciousAccountSeverity severity) {
this.accountNumber = accountNumber;
this.severity = severity;
}
private transient String toString;
@Override
public String toString() {
if (toString == null) {
toString = new ToStringBuilder(this).appendSuper(super.toString()).append("accountNumber", accountNumber)
.append("severity", severity).toString();
}
return toString;
}
}
An event represents an active entity, which means that each instance is unique. Therefore, we dont have to override equals() and hashcode() methods. The event classes listed above are lightweight and they carry no reference to any other domain objects. They also implement the Serializable interface which makes it convenient to transfer between JVMs. As best practice, this event is immutable. The two properties above (accountNumber and severity) are marked as final. They can be set only through a constructor.
The events themselves don’t carry a time of occurrence— a time stamp. When the event is inserted into the knowledge session, the rule engine assigns such a time stamp. There is a special implementation of FactHandle called EventFactHandle which is returned by session.insert(). It extends the DefaultFactHandle and adds few additional fields, — startTimestamp and duration. Both contain millisecond values and are of type long.
So, now we have our event classes and we are aware that there is a special FactHandle for events. But we still havn’t seen a way to tell Drools that our class represents an event. There are type declarations in Drools, that help Drools know about our class representing an event. It can define new types and enhance existing types. Lets see an example here:
Event role declaration (cep.drl)
declare TransactionCreatedEvent @role( event ) end
This code can reside inside a normal .drl file. If our event had a time stamp property or a duration property, we could map it into startTimestamp or duration properties of EventFactHandle by using the following mapping:
@duration( durationProperty )
The name in brackets is the actual name of the property of our event that will be mapped to the duration property of EventFactHandle. This can be done similarly for startTimestamp property.
In the following examples, we’ll also see how to define a new type declaration.
4.2.3 Fraud detection rules
Lets assume that a system processes thousands of transactions at any given time. This might be challenging in terms of time and memory consumption because it is not possible to keep all data in memory. A possible solution to this would be just to keep all accounts in memory as accounts would be lesser in number than the number of transactions and keep the transactions only for a certain period.
This is achievable using Drools Fusion by declaring the Transaction as an event. The transaction will be inserted into the knowledge session through an entry-point. Each entry point defines a partition in the input data storage, reducing the match space and allowing patterns to target specific partitions. Matching data from a partition requires explicit reference at the pattern declaration. This makes sense, especially if there are large quantities of data and only some rules are interested in them. We’ll look at entry points in the following example.
4.2.4 Notification
The requirement we’re going to implement here is essentially to block an account whenever a LostCardEvent is received. This rule will match two facts:
a) one of type Account and
b) one of type LostCardEvent.
The rule will then set the the status of this account to blocked. The implementation of the rule is as follows:
Notification rule that blocks an account (cep.drl)
rule notification
when
$account : Account( status != Account.Status.BLOCKED )
LostCardEvent( accountNumber == $account.number )
from entry-point LostCardStream
then
modify($account) {
setStatus(Account.Status.BLOCKED)
};
end
As we know, account is an ordinary fact from the knowledge session, the second fact— LostCardEvent — is an event from an entry point called LostCardStream. Whenever a new event is created and goes through the entry point, LostCardStream, this rule tries to match (checks if its conditions can be satisfied). If there is an account in the knowledge session that didn’t match with this event yet, and all conditions are met, the rule is activated. The consequence sets the status of the account to blocked in a modify block.
As we’re updating the account in the consequence and also matching on it in the condition, we have to add a constraint that matches only the non-blocked accounts to prevent looping (see above: status != Account.Status.BLOCKED).
4.2.5 Test configuration setup
In this section, we will setup a class for unit testing. All of the rules will be written in a file called cep.drl. When creating this file, just make sure it is on the classpath. The creation of KnowledgeBase would be same as shown in my previous article section 4.4. We just need to change the default knowledge base configuration slightly:
Enabling STREAM event processing mode on knowledge base configuration.
KnowledgeBaseConfiguration config = KnowledgeBaseFactory
.newKnowledgeBaseConfiguration();
config.setOption( EventProcessingOption.STREAM );
The above code will enable the STREAM event processing mode.
KnowledgeBaseConfiguration from the preceding code is then used when creating the knowledge base as shown below:
KnowledgeBaseFactory.newKnowledgeBase(config).
So next we will setup click initialization. We already know that every event has a time stamp. This time stamp comes from a clock which is inside the knowledge session. Drools supports several clock types, for example, a real-time clock or a pseudo clock.
The real-time clock is the default and should be used in normal circumstances. The pseudo clock is useful for testing as we have complete control over the time.
If we see the following code, its initialize() method sets up a pseudo clock. This is done by setting the clock type on KnowledgeSessionConfiguration and passing this object to the newStatefulKnowledgeSession method of KnowledgeBase. The initialize method then makes this clock available as a test instance variable called clock when calling session.getSessionClock().
Unit tests setup (CepTest.java)
public class CepTest {
static KnowledgeBase knowledgeBase;
StatefulKnowledgeSession session;
Account account;
FactHandle accountHandle;
SessionPseudoClock clock;
TrackingAgendaEventListener trackingAgendaEventListener;
WorkingMemoryEntryPoint entry;
@Before
public void initialize() throws Exception {
KnowledgeSessionConfiguration conf =
KnowledgeBaseFactory.newKnowledgeSessionConfiguration();
conf.setOption( ClockTypeOption.get( "pseudo" ) );
session = knowledgeBase.newStatefulKnowledgeSession(conf, null);
clock = (SessionPseudoClock) session.getSessionClock();
trackingAgendaEventListener = new TrackingAgendaEventListener();
session.addEventListener(trackingAgendaEventListener);
account = new Account();
account.setNumber(123456l);
account.setBalance(BigDecimal.valueOf(1000.00));
accountHandle = session.insert(account);
As we see, the preceding initialize() method also creates an event listener and passes it into the session. The event listener is called TrackingAgendaEventListener. It simply tracks all of the rule executions. It is useful for unit testing to verify whether a rule is fired or not. Its implementation is as follows:
TrackingAgendaEventListener.java
public class TrackingAgendaEventListener extends
DefaultAgendaEventListener {
List rulesFiredList = new ArrayList();
@Override
public void afterActivationFired(
AfterActivationFiredEvent event) {
rulesFiredList.add(event.getActivation().getRule()
.getName());
}
public boolean isRuleFired(String ruleName) {
for (String firedRuleName : rulesFiredList) {
if (firedRuleName.equals(ruleName)) {
return true;
}
}
return false;
}
public void reset() {
rulesFiredList.clear();
}
}
DefaultAgendaEventListener comes from the org.drools.event.rule package that is a part of drools-api.jar file as opposed to the org.drools.event package that is part of the old API in drools-core.jar.
All of the Drools agenda event listeners must implement the AgendaEventListener interface. Here in our example above, TrackingAgendaEventListener extends DefaultAgendaEventListener so that we don’t have to implement all of the methods defined in the AgendaEventListener interface.
Our listener just overrides the afterActivationFired() method that will be called by Drools every time a rule’s consequence has been executed. Our implementation of this method adds the fired rule name into a list of fired rules — rulesFiredList. Then the convenience method isRuleFired() takes a ruleName as a parameter and checks if this rule has been executed/fired. The reset() method is useful for clearing out the state of this listener, for example, after session.fireAllRules() is called.
Again, back to the test configuration setup(CepTest.java). The last part of the initialize method is account object creation (account = new Account(); …). This is for convenience purposes so that every test does not have to create one. The account balance is set to 1000. The account is inserted into the knowledge session and its FactHandle is stored so that the account object can be easily updated.
4.2.6 Testing the notification rule
After setting up the test infrastructure, we can write a test for the notification rule.
CepTest.java
@Test
public void notification() throws Exception {
session.fireAllRules();
assertNotSame(Account.Status.BLOCKED,account.getStatus());
entry = session
.getWorkingMemoryEntryPoint("LostCardStream");
entry.insert(new LostCardEvent(account.getNumber()));
session.fireAllRules();
assertSame(Account.Status.BLOCKED, account.getStatus());
}
The test verifies that the account is not blocked. Then it gets the LostCardStream entry point from the session by calling: session.getWorkingMemoryEntryPoint(“LostCardStream”). Then the code listing demonstrates how an event can be inserted into the knowledge session through an entry-point— entry.insert(new LostCardEvent(…)).
References
- http://planet.jboss.org/post/getting_started_with_drools_fusion
- https://docs.jboss.org/drools/release/5.5.0.CR1/drools-fusion-docs/pdf/drools-fusion-docs.pdf
- https://docs.jboss.org/drools/release/5.3.0.Final/drools-fusion-docs/html_single/
- https://www.packtpub.com/mapt/book/networking_and_servers/9781783288625/6/ch06lvl1sec41/running-cep-based-scenarios
Under problem description, point no 3, 4 and 5 which you have mentioned. Have you implemented that point. If you can share the sample code or workflow, it will be of great help