Data modelling layers: do you wanna get logical or physical

An introduction to data models

3 Big data security analytics techniques

Reg Developer: Can't UML be considered a conceptual model? Why do you see a need for a separate conceptual data model?

Donna: Many people do use UML to create abstract, conceptual layer. But while UML does well expressing the interactions between the data and the processes, it does not accommodate some of the core principles around designing data itself and the relationships between data items. Concepts such as identifiers/primary keys, for example, are not expressed in a UML class diagram. A conceptual data model is designed to be easily mapped to a logical data model. UML is more suited to application design [fair enough, but that might just be an omission in MDA, the OMG's Model Driven Architecture - Ed].

We also hear from many of our business users and data modellers that the UML is difficult to understand. Perhaps "difficult" means only that it's not a graphical notation that they are familiar with, i.e. "it doesn't look like a data model to me", or "where's my Visio"? And/or this could be a result of the mistake that many architects make in showing the wrong level of detail to the customer, and that the UML diagrams presented were too low-level to be helpful to business users, and too application-focused to be helpful to the data modellers.

Honestly, another big factor is a societal one. There seems to be separate camps in the UML and ER modelling world and never the two shall speak. This is unfortunate, since I think there are commonalities between the two. I think some of this is changing with efforts like the IMM (Information Management Metamodel) from the OMG, which is attempting to better mesh the UML and ER worlds in a formal metamodel and exchange format.

But, until these two worlds play more nicely together, we'll choose to focus our model on our core audience, the data modeller. But even with that consideration, I do think that the conceptual model as we see it is an easier format for the average business user to understand.

Reg Developer: And XML? How does that fit in? Isn't that data too? I'm sure I remember using data analysis generally and E-R diagramming specifically to design hierarchical IMS databases; and XML is 'just' another data hierarchy.

Donna: XML most certainly fits into these modelling layers. However, similar to the UML discussion, there are currently different camps/factions/silos that don't communicate well. We're seeing our traditional RDBMS data modeller becoming more familiar with XML, but that's a more recent phenomenon. Getting XML developers used to data modelling is a separate challenge - as we mentioned before, the developer crowd can be a harder one to convince of the benefits of a model.

But the benefits are the same - a common conceptual and logical model of the business with different implementation layers - XML being one of those.

We're currently guilty of this disconnect ourselves in our tools. We currently have wizards to export XML from a relational logical design, but we're still treating XML as a translation from the relational world. This actually works well, given the fact that I mentioned earlier that most of our relational database modellers aren't necessarily familiar with XML - allowing them to translate into XML from a format they're familiar with works. But in the longer term, we're working on modelling XML better in its own, native, hierarchical format.

Reg Developer: That's all clear enough, I suppose, and the future directions are interesting, but why do we need a data model at all? Isn't this all just 'stuff' getting between the programmer and her users?

Donna: That's a religious question! There's the basic principal of design, then build. Would you want to live in a house that had been designed without a blueprint?

And then there's reuse - most companies have common data objects such as customer, product, etc. Rather than reinvent the wheel for each new project, better to start from the same core design in the data model.

And what about data quality? In the example above, if every project uses a different definition of 'customer', for example, there are bound to be discrepancies in the data. In fact, there's a whole Master Data Management industry devoted to fixing the problems experienced by companies without a coherent model of their data.

Data governance is yet another issue. The data models help to provide an effective inventory of your data assets. Most companies have a wide variety of database platforms with hundreds, thousands, and even millions of tables. Reverse engineering from these platforms into a physical (and eventually a logical and conceptual model helps companies better understand what they have today – which can be extremely important if these assets form the foundation of regulatory reports, that directors sign off on, on pain of going to jail...

Reg Developer: OK, play the governance card, everybody else does (although you make a good point). But how accepted is data modelling in the community? How often do you need to explain the benefits to customers and potential customers?

I'd say that in the data community, having a data model is a fairly accepted principle, and it is rare that we need to explain the benefits (but it does still happen from time-to-time). Among the developer community, including some DBAs, buy-in is less accepted. I'd say about 50 per cent. I think the attitude and culture is different, more of the "I have a deadline to meet, I don't have time for design", "leave me alone and let me code", etc.

I think we can sum up by saying that understanding data is still important and there is still some point in the old disciplines of data analysis, even though we have moved forward in so many ways (business-centric development, XML and so on). In the end, however, we still have to deal with the old problem of development silos – business analysts, coders, database designers, and so on, all doing their own thing and not talking to each other. ®

SANS - Survey on application security programs

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Oh no, Joe: WinPhone users already griping over 8.1 mega-update
Hang on. Which bit of Developer Preview don't you understand?
Microsoft lobs pre-release Windows Phone 8.1 at devs who dare
App makers can load it before anyone else, but if they do they're stuck with it
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Internet-of-stuff startup dumps NoSQL for ... SQL?
NoSQL taste great at first but lacks proper nutrients, says startup cloud whiz
Next Windows obsolescence panic is 450 days from … NOW!
The clock is ticking louder for Windows Server 2003 R2 users
Ditch the sync, paddle in the Streem: Upstart offers syncless sharing
Upload, delete and carry on sharing afterwards?
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
Batten down the hatches, Ubuntu 14.04 LTS due in TWO DAYS
Admins dab straining server brows in advance of Trusty Tahr's long-term support landing
prev story


SANS - Survey on application security programs
In this whitepaper learn about the state of application security programs and practices of 488 surveyed respondents, and discover how mature and effective these programs are.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.