Remembering the Future

In light of some new integration projects I may be involved in and some talks I plan to attend at the Library of Congress [The Inventor's Perspective on Subject-based Access], I though it might be useful to try and distill my current thoughts on Topic Maps. In a traditional integration/business intelligence project data moves from raw data, to locally structured databases, to globally structured data warehouses, to olap dimensional models, to spreadsheet like analysis. Lately, I have been wondering what role Topic Maps might play in this process. The TM sweet spot seem to be the merging of heterogenious metadata (building a merged index from different sources). This also seems to be one of the critical problems with current systems. Merging different datasources into a datawarehouse, is one of the most difficult and costly parts of this process. Semantic merging has turned out to be much more difficult than the syntactic merging of relational join operations. However, semantics is usually defined as the grounding of symbols in the meaning expressed through use and behaviour. For me, one of the ironies of Topic Maps is that while they are focused on semantics, they have no direct support for any kind of behavior. Instead, Topic Maps use a well thought out graph-representation, grounded on a system of atomic identifiers. These grounding identities provide a static basis for the automated merging of metadata. In real systems, however, the merging process is much more subjective and labor intensive. The same kinds of information gets encoded in may different forms in many different systems. This requires a complex set of transformations to merge these perspectives into a common view for analysis. The transformations (behavior) attached to each of these different data sources must respect the semantics of each source, and it is the development of these transformations that is the heavy lifting of integration. What seems to be needed, is a dynamic system for describing and supporting the transformations that might guide manual semantic merging. Maybe a kind of XQuery for knowledge graphs that respect the constraints of topic map structure and identity. I understand that work on some kind of TMQL is well underway and I'll be interested to here what Michel Biezunski and Steven R. Newcomb have to say.