I have spent a lot of time showcasing data modules to audiences across the world in-person and in livestreams, helping people understand how and why to use them as part of Cognos Analytics. The most consistent question I receive – by far – is about understanding framework manager vs data modules. There are a lot of outdated opinions and outright misconceptions floating around so let me outline the exact feature differences between framework manager and data modules as of Cognos 11.1.4.
What do data modules and framework manager have in common?
The answer is ‘a lot’ but this wasn’t always the case. In the 11.0 releases data modules were missing many essential framework manager features and didn’t offer compelling reasons to switch. Of course that has changed. As of 11.1.4 framework manager and data modules both:
- Produce data models that can be used with all Cognos 11 features
- Join dozens or hundreds of tables across multiple databases
- Execute cross-grain fact queries (aka the dreaded determinants)
- Build simple or complex calculations and filters
- Build alias, view, union and join virtual tables
- Secure data by groups, roles and users
- Create OLAP-like dimensional hierarchies
- Offer enterprise governance, auditablity and security
Oftentimes people washed their hands of data modules a couple years ago and are surprised to see virtual tables, cross-grain fact queries and security by groups. These features may exist in both but the implementation in data modules is superior from a usability perspective.
What do data modules offer that framework manager does not?
Again, the answer is ‘a lot’. The 11.1. release takes data modules beyond what is possible in FM with a host of powerful capabilities and quality of life enhancements. The following features are either exclusive to data modules or done infinitely better in data modules.
- Natural-language and Ai powered auto-modeling
- Automatic join detection
- Easy integration of excel data
- Ability to easily clean data
- Flexible hierarchies that go up, down and across (navigation paths)
- Easy measure binning and attribute grouping
- Easy extraction of year, month, day, etc… from data data types (split)
- Automatic creation of relative time filters (YTD, MTD, PYMTD, etc…)
- Automatic creation of relative time measures (YTD actuals, PYTD actuals, etc…)
- In-memory materialized views within Cognos Analytics
- In-memory query cache
- Easy multi-model inheritance for single source of truth
- Degenerate dimension aggregation (column dependencies)
Some of these features are absolute game changers for how I craft highly performant, easy to use and self-service friendly data models. Consider the coconut relative time; because this was such a titanic brain buster in framework manager only the most skilled developers could deliver. Now it takes minutes for end users to implement.
What are data modules missing?
There are still some things data modules lack:
- Object level security
- DMR capabilities
- Parameter maps
- Multiple connections for data servers
If I’m being honest, I don’t really recommend you use many of these features for new development in 2019 unless you absolutely have to, particularly DMRs. DMRs are very powerful for those who know MDX but a true maintenance and self-service nightmare in the long run. I cannot count the number of clients who are stranded with critical DMR based reports they cannot understand. In any case, a little bird told me that DMR-like functionality will grace data modules soon.
Going beyond the feature list
Comparing framework manager vs data modules feature for feature, we can see how data modules have few shortcomings and offer huge advantages. While this is a common way for IT folks to think (and I would know, I’m one of them!), I argue that it badly misses the point. By using data modules an IT professional can do weeks of FM work in an afternoon while a self-service user can easily accomplish tasks that will otherwise be done in Power BI. I repeat it often but I’ll say it again – data modules are the key to modernizing your Cognos Analytics environment and delivering content with the speed modern users demand.
What do you recommend?
I’ll parrot Cognos offering manager Jason Tavoularis and say, ‘use data modules unless you can’t.’ And as you can see above, the list of reasons you can’t has become quite short. I start ALL consulting engagements under the assumption that we’ll be building data modules and I’m always happy with the results.
Hi Ryan,
your post is very interesting and rich of information.
Thanks for sharing your knowledge.
I have a question about performance and in the specific on those two points for data modules:
In-memory materialized views within Cognos Analytics
In-memory query cache
there are some benefits on perfi
Data modules use a different query mode?or is the same as with fm with dqm and query service?
thanks
First of all, Magnum BI is the most excellent name I’ve ever had in my comments section, bravo! This is a great question – great enough for a blog post. I’ll write one with an answer.
You can also create your data modules based on data sets. And when the data in the data set changes, the change is reflected in the data module.
Great article.
Thanks.
I love data sets and think everyone should use them.
Hi Ryan
I have a question on data modules, assume we have a student table and created a view of that, later even we are pulling a single column from that view , take it as a example of we pulled Student ID into the report , and when we checked the generated query , we found all the columns from the student table into that query which is really not required… if we pulled single column then its should show single column into the generated query results.
Hi Ryan,
Good to see your idea between Framework Manager and Data Modules.
and I have a question how do we enable the RBAC on data level if use Data Modules only?
There is a RBAC for data that can be set on the data server level. I will try to write a quick explainer post about it. Don’t let me forget!
I was wondering if there already was a way to do Slowly Changing Dimensions and most importantly, use surrogate keys?
In Cognos Analytics 11.1.5, I don’t see a way to use a data source to feed a data module. It looks like in order to use data from a database, you need to use a package — which means you need to use Framework Manager anyway. Am I missing something?
Check out the article here for info about this: Cognos Analytics Data Servers
Great article. I have one question, is it possible to use relative dates with packages from framework manager?
Thanks, Ryan.
Your blog and YouTube were very educative for new me starting up.
I’m very glad to hear this!
Hi Ryan,
There is one major functionality limitation that is worth pointing out. Many of my clients have heavily invested in Transformer and PowerCubes. Unfortunately, Transformer does not support using Data Modules as a data source either directly or as a report. There is an RFE requesting this functionality and it is under consideration. – https://ibm-data-and-ai.ideas.aha.io/ideas/CAOP-I-200
This limitation means many will continue to see Data Modules as the “poorer cousin” of FM thus limiting the uptake of what is an otherwise excellent addition to the product set. I would advise anyone who is interested in this functionality to vote for the RFE 😉
This is extremely true! Two things to note:
1. The 11.1.5 release added direct member access from the data tree in data modules and dashboards, which greatly enhances the dashboard experience for cube sources. That doesn’t solve your problem, but it’s worth noting that you can now get a ‘cube-like’ experience in Cognos without building a cube at all – the feature works on relational sources as well.
2. IBM is definitely aware of this limitation and will be adding cube sources into data modules in the near to medium future, so stay tuned!
I would like to make a few comments after reading this article. To tell the truth, the subject is serious and deserves an in-depth discussion.
I do not quite understand why oppose DM to FM.
In addition, with arguments which in my opinion are not always correct.
DataModules lacks several vital functionalities for the development of models at departmental and enterprise level.
One of them is the possibility of translating the data dictionary.
I have a lot of clients who use multilingual models. My latest project is in: French, English, German, Italian and Chinese.
I can’t (or maybe I don’t know) do it with the DM! The metadata dictionary is not multilingual.
Of course some functionality is missing in FM, but these are functions that did not exist in Cognos before version 11.
Type time data and geographic coordinates. Access to DataSets, …
But to say that in an afternoon with DM you can work the FM work weeks is simply wrong.
Managing Year to Date is not really complicated in FM. It’s been several years that we do the reports without making knots in the brain.
IOl is possible to do it in DMR (by modeling) and in relational (with filters or by modeling). It’s not as good as in Transformer – but it’s hard to do as well;).
You can easily extract the year, month, day from the date and access to the various functions is not more complicated – it is rather simpler – than in DM.
In addition, users like the multidimensional.
Another thing about relative time scales. Most often what we need are the columns Montant_Y, Montant-Y1, Difference-Y-Y1. In reports.
How to do this with DM?
FM is a successful product that fits business and departmental business intelligence needs.
IBM has stopped improving this product to focus on individual productivity tools.
As if the Excel sheet is the main source of data in the company
I work with several clients and I assure you, they still use the databases, with tens of millions of rows and hundreds of tables.
DM and Dashboard brings the functions that were missing in Cognos and that’s great. But why stop evolving proven products?
You can add in FM time and geographic data types. DM can complement the uses of FM.
But opposing these two products is incomprehensible.
This is a very good comment that I largely agree with. I think IBM could have done so much with Framework Manager over the years and didn’t, which is a real shame. Look at tools like Trifacta – clearly there is a market for desktop based data modeling and blending. If only…
Given the current situation I think customers should move everything they can to data modules because they are easier to work with and have a long future of support with IBM. Some of your specific concerns – like multi-language support, which is clearly a major concern for you and many other customers – are on the mid-term road map. In general the next year is going to be dedicated to closing remaining gaps with FM.
Thank you for the thoughtful comment!
A common problem with Cognos (and any other software) is that the same thing can be achieved in many places.
For the professional we all know the quirks about when we should use what and why. For example Cognos can model data in:
Framework Manager, Transformer, Dynamic Cubes, Data Modules, Data Sets, Performance Modeller and so on. And before anyone shouts Relational vs Multidimensional, that’s an end use case question. Should I model data relationally or dimensionally.
Often I think Cognos adds stuff to the truck, without thinking what it should be taking off the truck.
All of the modelling tools have a large amount of overlap, allowing you to achieve a lot of the same functionality in each. It is the edge cases that make the difference.
Unfortunately none of these tools easily allow you to move a model from one tool to another.
So you really need to know of all the pros/cons of each model before you embark on modelling and reporting.
Datasets vs Data Modules – to me is a question of business author vs professional author, with a lot of grey in between.
For example in C10 should I use Query Studio or Report Studio. In CA this line is blurred as there is now only one Report tool, and perhaps this will happen with Datasets and FM at some point.
For some shortcomings in DM we need to ask, are we thinking in the old way? For example, a data warehouse has many dimensions and many facts. Within FM we would use shortcuts and namespaces to remove loop joins, easily create role playing dimensions and so on.
These functions are not available in DM – but are they needed?
Within FM we try and model the whole Data Warehouse in a single package.
Could I achieve the same functionality by having single subject area DMs, that is small DMs that only have a single fact table.
Within the DM I could use link tables to only need to model the dimension tables once.
As DMs produce DQM sources, we can use multiple DMs in a single reports. Multifact queries would then be handled by the reporting engine, rather than the database engine.
This is a very different way of working to what we traditionally do within FM.
. I know the traditional FM way will write a single SQL query that will handle multifacts, pushing all the processing to the DB.
But what would the DM solution look like. Would it be multiple queries, joined within the DQM engine. Which would be more performant?
You raise a lot of great points here. You are right on when it comes to rethinking how you accomplish tasks in data modules vs framework manager – it’s just a different way of doing things. In FM you strive to make everything perfect up front and include all data anyone could possibly request. This makes sense because building FM models is too time intensive to be constantly adjusting them while the traditional BI workflow requires weeks to make even the simplest change. Data Modules are all about speed – build them fast, build them in prod on prod data, get them in front of users, collect feedback and adjust quickly. Design patterns are similar but not as involved. One point you raise – the inability to make shortcuts for example – can basically be accomplished using custom tables in data modules. I have an article here that goes over it. Lots of great thoughts in this comment, thanks for sharing.
In many similar articles about the differences between FM and DM I have yet to see any mention of branching and merging.
We develop and test many branches for many tickets using multiple resources from the same base FM model, then merge in the approved changes.
How can this be handled in DM?
Hi Pete. I sent you an email directly about this, but I figured I’d respond publicly here as well for posterity:
AFAIK there is no data module branch/merge functionality in OOTB Cognos, unfortunately. All you can really do is copy the DMs to a dev folder, then save over the prod version in a manual migration process. This introduces obvious version control issues.
There is a Cognos GitHub integration that does support data modules. It is a paid add-on for Cognos and so I have never used it. I tried to get IBM to let me make a YouTube video about it but got nowhere. Anyway, it would solve your problem if it works as advertised.
https://www.ibm.com/support/pages/cognos-analytics-github-integration