IBM Blueview

Cognos Analytics and all things IBM

Data Modules

Data Modules as a Single Source of Truth

by 10 Comments

The conversation spurred by my What are Cognos Data Modules blog post continues with an excellent question by Heather, who asks what to do about maintaining a single source of truth in Cognos Analytics while still using data modules. This does pose a challenge. I’ll discuss some options below and rate them using a scale of 1 – 5 Analysis Studio cubes.

Cognos Data Modules are a great single source of truth
Cognos Data Modules can server as a single source of truth, if you let them

Option 1: Don’t Use Data Modules

One common way to solve this problem – sadly – is to not use data modules in any capacity. Turn them off for end users and IT and stick with tried and true Framework Manager. If it aint’ broke, don’t fix it. This approach will preserve single source of the truth but at significant cost:

  • No access to new features… ever (FM is not receiving feature updates)
  • FM development cycles are very long
  • End users will have no ability to model data in Cognos…
  • Which means they won’t be able to do anything complex in Cognos…
  • Which means they’ll use Power BI instead!

Odds are if you’re reading this blog you’d prefer your users work in Cognos vs Power BI or Tableau. Choosing option 1 will definitely preserve single source of the truth but at the cost of relevance.

Final Verdict

Option 2: Treat Data Modules Exactly Like Framework Manager

Contrary to popular belief – and IBM’s marketing – data modules are not just for self-service users. The feature set of data modules has reached near parity to Framework Manager (with some notable exceptions) and has surpassed it in some very significant ways – relative time, for example. It’s also important to understand that Cognos security applies to a data module in just the same manner as a Framework Manager package.

There is in principle no reason you can’t migrate your IT modeling work to data modules wherever possible while restricting their use entirely to the IT team. In this way you can future proof your development and take advantage of a growing list of very cool features while leaving your traditional BI workflow and single source of truth intact.

However you foreclose the opportunity to take Cognos to the next level and enable real self-service.

Final Verdict

Option 3: Use Linked Tables

Linked tables allow to you use tables from one data module as the source for tables in another data module. These tables inherit all changes from the source, including field name changes, field additions/subtractions, calculation changes, SQL changes, etc…

Linked tables in action!

Using these links, it’s possible for IT to build and maintain a high quality, tested and accurate data model while still providing a significant degree of end user flexibility. In the image above, an end user has imported three linked tables over which IT has 100% control and joined them to an excel spreadsheet. To do this follow these steps:

  1. Create one or more data servers for your source databases
  2. Restrict access to create new models using those servers to IT only
  3. Build a data module that contains the tables you wish to govern
  4. Save the data module in a folder where users can access it but not overwrite it
  5. Remove the ‘break link’ capability from users. This means they cannot edit these tables in any way
  6. Teach them how to use link tables and let them go wild!

By following these steps you can ensure that all self-service data modeling for governed sources uses your logic, your calculations and even your joins – the joins you define are imported alongside your linked tables. Your users can still model to their heart’s content with their departmental or personal data but when it comes to your governed enterprise data, your rules come first.

Final Verdict

Option 4: Stop, collaborate and listen

For better or worse, speed and flexibility have decisively crushed accuracy and security in the battle for BI market share. End users routinely show up to meetings with three competing Power BI/Tableau workbooks that disagree on all relevant metrics but were produced in four days and look incredible. And people seem absolutely thrilled a this state of affairs based on recent events!

Therefore Option 4 isn’t so much a technical solution as it is a cultural one. Yes, you adopt the strategy outlined in Option 3 – but you do so cheerfully with the knowledge that your end users absolutely will colossally screw up the data in Cognos, and that it’s your job to identify and fix it when this happens. This is by far the hardest thing for long time Cognos pros to do, and it was extremely hard for me to come around to this way of thinking. The final piece of the puzzle was the following realization:

  • Users are incredible innovators in the art of producing bad data – they truly cannot be stopped
  • Trying to enforce 100% data accuracy in Cognos just drives them to other tools
  • Errors produced in those other tools are completely outside of my ability to monitor, influence or correct – as are those users
  • This cycle absolutely destroys ‘single version of truth’ for an organization as a whole, even if it is maintained for Cognos

Switching to a more open, collaborative approach in Cognos has the benefit increasing user satisfaction, organizational perception and eventually investment in Cognos. I have seen this work at my clients who have embraced it.

Final Verdict

That’s right, the final verdict for Option 4 is five PowerPlay Studio cubes of approval – the highest approval I can give.

PowerPlay was way better than Analysis Studio anyway.

Alias Shortcuts in Cognos Data Module

by 5 Comments

My recent What are Cognos Data Modules post generated some interesting discussion around the alias shortcut functionality of Framework Manager and whether or not it is available in data modules. I initially responded by saying you could make data module alias tables, however SirM challenged me that it isn’t the same at all.

SirM is right.

What is an Alias in Cognos

In my experience the primary reason to build an alias is to tightly control how Cognos executes joins. Imagine you have two tables, Sales_Fact and Time_Dim. Sales_Fact contains the fields order_date and shipped_date, both of which you wish to join to Time_Dim. You could join the tables twice and be done with it, but then you are at the mercy of Cognos as to which join it will include when generating SQL. This will introduce unintended or completely incorrect results when the ‘wrong’ join is used.

Enter Framework Manager alias shortcuts. An alias shortcut is essentially a pointer to another table that has the following properties:

  • Inherits all fields and all changes from the target table automatically
  • Ignores all relationships from the target table
  • Can be repeated as often as necessary

In our example above, we make an alias shortcut of Time_Dim called ‘Shipped_Date’ and join the alias to Sales_Fact. Shipped_Date will inherit all fields from Time_Dim but will have an independent relationship to Sales_Fact.

Alias in Data Modules

Can we do this in data modules? In Cognos 11.0.x the answer was basically ‘No way!’ However recent releases have gone a long way to close this gap, especially once the custom table functionality was added in the 11.1.x release stream. Let’s explore the three options I recommend for building alias shortcuts in Cognos Analytics data modules and see which one fits best.

Copy Tables

Copying a table does exactly what you’d expect – it makes a copy of the table in question. You can create as many copies of a tables as you want, however changes you make to the target table do not flow through to the copied table in any capacity – whether you add or remove fields, rename fields or change calculation logic.

  • Inherits all fields and all changes from target table automatically: NO
  • Ignores all relationships from the target table: YES
  • Can be repeated as often as necessary: YES

View Tables

View tables are like copy tables with a couple very important differences; views can be composites of many underlying tables and they do inherit some changes from the target automatically. Specifically, they will inherit calculation changes made to the fields in the target table but will not inherit name changes or added/removed fields without manually editing the view definition. I will do a deep dive on their usage in a future post.

  • Inherits all fields and all changes from target table automatically: SORTA?
  • Ignores all relationships from the target table: YES
  • Can be repeated as often as necessary: YES

Linked Tables

Linked tables are essentially pointers to tables built and maintained in other data modules. They are extremely useful for BI teams concerned about data quality in Cognos Analytics, as you will see below.

Linked tables inherit all properties from the target table in the model in which they were built. This means that any changes I make in the target module, whether I add or remove fields, create new calculations or edit existing logic will automatically flow through to all linked tables that reference it. This would appear to solve our alias shortcut problem, so what’s the catch?

You can only import a linked table into a data module once. It cannot be copied and any views you build on it have the view limitations outlined above.

  • Inherits all fields and all changes from target table automatically: YES
  • Ignores all relationships from the target table: YES
  • Can be repeated as often as necessary: NO

So What’s the Solution?

Whenever you find yourself reaching for the alias shortcut button in data modules, ask yourself which alias table feature is most important for the task at hand.

  • If the most important thing is automated inheritance of all future changes, build a link table
  • If the most important thing is re-using the same table over and over, build a view table
  • In most instances, do not build a copy table

In reality, a view table should fit your needs in most circumstances. Yes, you will need to manually intervene to inherit certain changes listed above, however this process takes about 30 seconds per table. Not ideal but something most of us can live with.

How to Proceed

Alias tables have historically filled a very important role in building large scale Cognos models in Framework Manager, and their absence in data modules creates challenges to modeling the way we have for over a decade. Certainly many of my most skilled customers feel that without this functionality data modules don’t have much use.

It’s worth considering then how it is that Tableau and Power BI managed to dominate the mid-late 2010s BI market with a total absence of an equivalent to alias shortcuts or many other ‘enterprise’ BI modeling features? The answer lies in the culture and practice that these solutions enabled which delivered faster results for business users. How to apply these practices to Cognos using data modules will be a major theme of this blog going forward.

Level up your Cognos skills with these helpful articles!