Sunday, November 19, 2006

The Case for a Clear Distinction between Events and Content.

I should explain my terminology here. When I refer to Events I mean the information about a business-meaningful event – not the actual real-world experience of the event. Similarly, when I refer to Content, I am also talking about information - in the sense of the ‘content of a book’. So, both Events and Content are categories of information and naturally form part of an Information System, in the broadest sense.

Events:

The real-world proceedings that stimulate business activity – sometimes in a pre-defined sequence but often not. These are the triggers for action.

Content:

The documents, conversations or messages that are produced and consumed by business activities. These are the dialogues we use to share a plan, a concept, a history and/or the details of a person, place or thing.

Events (Event Messages) do carry information. However, the information carried has only one purpose: to provide sufficient context to make the Event meaningful to a person or a software component, working on their behalf. It is important to maintain the logical[1] distinction between Event and Content.


Fuzzy and Precise Events:

Events can be regarded as both highly structured and precise and highly unstructured and imprecise messages within a common Event ‘envelope’ (general structure). For example, a movement tracking system may receive highly structured signals from RFID or GPS devices which are then converted into equally structured human-readable business events, But the same system might also receive much more unstructured Event information, possibly capture a ‘text’ message on a mobile phone that might alert of a delay caused by heavy traffic. The emphasis is placed on the value to the human consumer as opposed to, sometimes unhelpful, or misplaced, information engineering rigour. That’s not to say, however, that over time a loosely defined Event may benefit from being made more structured and precise and that some Events need to be implemented as semantically-agreed/syntax-precise data structures from day one.

To illustrate further:

  • Fuzzy Events -The Event information may not be as complete or as rigorous as, for example, a structured document or data record might require. However, it might be really useful to know that an Event has taken place even if the information conveyed requires a degree of human interpretation. Maintaining separation between the Event and related Content makes it possible to get value from the Event information without confusing it with the, necessarily precise, business Content information. This is because the Event and the Content have fundamentally different business purposes (as illustrated above). Recognising this difference can be the key to avoiding lengthy data modelling and data standards work (around Identity schemes and other codified data) and thus ensures a degree of business value is delivered as early as possible. The Event may not be interpretable by an IT system – but it may be of use to a person, in the same way a scribbled jotting on Post-it-Note might provide valuable information.

  • Precise Events - Paradoxically, the opposite is also true. Content, in the form of a conversation or audio/visual media might be difficult for an IT system to consume and interpret, but is fine for human consumption. In this case, the separation can have the opposite benefit – the Event is always IT ‘friendly’ in the sense it can always be processed in the general sense of routing and subscription, and the Event ‘context information’ may also be processed by rules and derive a new fact or implication.

Aggregated Events become Content over time.

Unfortunately, having said that it’s important to make a logical distinction between Events and Content, the reality is less clear-cut. This is illustrated by the ‘Wave/Particle’ nature of Event information in IT Systems. While they exist as independent, content-light, ‘Business Signals’ during minute-by-minute business operation, they are also ‘Content’ when they are retrieved in sets and aggregations from an ‘Event Log’ or Data Warehouse. Fortunately, from a business point-of-view, this seems to be easily understood. The debate around Event and Content, however, is more prevalent in the IT community and stems from years of building IT applications and databases without a clear distinction between the two.



[1] In some circumstances the physical implementation of an Event-based system may include ‘Payload Data’ (Content) bound to the Event. The same logical separation rules, however, apply. The value of this separation identified in MIT’s/AutoID Inc’s original work on the X-internet and EPC standards.

6 comments:

Brian Connell said...

The CEP community has been working hard for a shared community-wide glossary (with different semantics, but similar concepts to those you have described). I'd encourage you to check out www.complexevents.com for the latest version, and to continue to discuss your ideas in the CEP Yahoo group.

Brian

Nigel Green said...

Brian,
Thanks for your comment. I'm keen to align the 'semantics' of the CEP community with my day-to-day discussions with CxOs - I'm finding a growing interest in the subject - my mission is to draw the business and the CEP community together using the 'language' of business. I'd really like to hear of 'CxO level' disussion approaches to this important concept.

Tom Rose said...

Complexevents.com is a good source of discussion on the topic. As is http://www.opengeospatial.org/projects/groups/sensorweb and their standards for event processing, http://www.epcglobalinc.org/home for RFID, and there are others. The world of sensors in machine monitoring, environment monitoring, etc, has significant experience in event processing.

I see event processing evolving, and if I interpreted Nigel correctly, that evolution seems to be centered on common business events. The world of Raw Events is evolving to include simple and complex business events that are part of the typical enterprise infrastructure. These events, even simple ones, require business context for them to be detected.

CEP.com has done much to generalize event processing concepts, although I see every application of EDA creating their own semantics from the ground up. It would be nice if we could agree on EDA and terminology like CEP.com, then apply semantic mappings to the base definitions, and of course extend the definitions for particular applications.

I’m confident it’s not a technology barrier to bring EDA to the enterprise to significantly improve business operations, but the fractured and non-orchestrated efforts are the real barrier.

The value of the technology from a CXO perspective is an understanding that business is an event driven operation. People are event driven, and currently are the “sensors” and complex event processors of the business. The goal is to automate much of sensor management and event detection to make it more consistent, provide more of it for better operations visibility, and detect business events as they are occurring so people can make better more informed decisions.

From my perspective, the challenge that Nigel talked about is bringing appropriate business context into event stream and complex event processing, and I believe that challenge is made more difficult by not having a context free language, and then a process for applying semantic meanings to the language.

As an example, EPCglobal and OpenGeoSpatial communities in isolation have completely defined context specific sensor standards, and at a context free level are virtually identical.

If I missed your point Nigel, I apologize, and let me know.

Tom

Nigel Green said...

Thanks, Tom, you interpret my post correctly. For adoption of EDA to take-off, we need to find a way to convey compelling stories and possibilities to the business community. I’ve found it necessary to make the distinction between ‘Business Events’ and ‘Content’ (as I define them above) in order to then go on and explain the value of an Event-based approach and how it compliments existing information flows – no one wants to hear ‘throw away and start again’. Using this approach, we’re finding some very interesting potential value of an Event-based approach, in for example, data privacy and protection – using Events as ‘Proxies’ for content and thus managing the channel over which sensitive data might be published. I don’t apologise for avoiding some of the ‘academic’ language of the CEP community, but do wish to create linkage between my CxO discussions and the appropriate standards focussed groups.

I’ll make a separate post later with more snippets from the ‘Business Event’ discussions.

Tom Rose said...

Nigel,

This is a big topic and one I'm very interested in. Moving beyond the mechanics of event processing, and finding a consistent language to lay business semantics over the mechanics is imperative to bring EDA based solutions into business operations.

I started a consortium called FirstOpen to create a focal point for the work being done in this area. Currently, I just have the open source effort Singularity (RFID/Sensor Middleware) under FirstOpen. If you are interested, perhaps we can collaborate to bring the work on Business Events under FirstOpen (firstopen.org).

Tom

Adrian Apthorp said...

Nigel,

I understand your need to explain EDA principles to CxOs, however I'm having a problem drawing the line between event data and content data. I think this may lie somewhere in the difference between a real world and information systems view.

Events reflect state changes in some object or objects. i.e. something has happened in the real world. A record of an event needs to provide contextual information (e.g. when, where, how/who observed) as well as identifying what happened. I think it is the 'what happened' where your distinction between Content and Event lies. 'What happened' refers to the activity and the objects/entities involved. When you refer to 'Content', I believe you are referring to the latter or the information about it.

May be we can use a basic example:

Taking the example of an Order (Content?), events of interest affecting an order are:

- order quoted
- order agreed
- order ammended
- order cancelled

Of course another event relevant to order is when the customer's account is suspended, but lets keep the discussion restricted to simple and non-derived events.

Apart from the fact that these events reflect state changes on the order object they may also involve some change in attributes of the order. If I understand correctly, your definition of Content is the information describing the current state of an object. Is this correct? If so would an event include the delta (i.e. what has changed in an objects attributes) even if the delta includes the whole object when it's created?

Adrian