Remembering the Future

Last week there was some discussion about the value of relational databases and this week it looks like it's messaging's turn for re-examination. It's an interesting proposal and, hey, it's good to revisit sacred cows every once in a while if only to keep them on their toes.

...

If the goal of the data fabric is, after all, is distribution of data and horizontal scaling then it's messaging, not persistence, that was developed with these priorities in mind in the first place. And when one considers the various design patterns for infinite scalability what quickly emerges is a message-oriented view of persistence where individual entities generate and receive messages and most types of locking (e.g. distributed transactions and locks across many entities) are just thrown out the window.

It's clear that the overwhelming need to distribute data across dozens of servers is increasingly blurring the line between persistence and messaging.

SEE ALSO: Jags Ramnarayan

Now, with a data fabric, all applications are sharing a single data model and express interest on the data model through simple, intuitive queries. The underlying fabric is constantly detecting what and how data and relationships are changing and simply sends notifications of the changes to the consuming application.

Note the following advantages with a data fabric:

1. Application processes are connected to each other in a p2p fashion with direct connections between each. This allows the fabric to avoid unnecessary network hops, dramatically reducing latency and CPU costs associated with message transfer.
2. As the data is typically held in multiple locations and often replicated to the process space of the consuming application, the publisher doesn't have to send obese messages - applications merely change the data fabric and underneath the hood the right "delta" event gets propagated to the consuming applications.
3. The receiving application when notified can take immediate business decisions as the contextual information they need is cached right there.

Labels: Architecture, database, messaging