The 2021 Gartner BI Magic Quadrant is Broken for Cognos Analytics

It’s that time of year again – time for the Gartner BI Magic Quadrant. Like a bad ex who just won’t back off, Gartner drops their annual market warping report around Valentine’s Day to ruin the mood. I’ve expressed strong opinions in the past about Gartner’s report and methodology and this year will be no different. Put simply, they did Cognos Analytics dirty again while rewarding some competitors based on evaluation criteria that have nothing to do with the capabilities or quality of the software. In particular I argue that:

• Gartner heavily weights the number of inquires each tool generates for Gartner’s advisory services in determining the rankings. This is a self-reinforcing feedback loop.
• Gartner is rewarding BI vendors that have attachments to ERPs, office suites and other enterprise software. Particularly SAP and Oracle.
• Gartner’s write up of IBM doesn’t align with their ranking on the scatterplot, particularly in completeness of vision.
• Gartner’s Peer Insights customer review site tells a completely different story about Cognos Analytics.

You may be thinking, ‘Ryan of course you feel this way – you’re the most public Cognos fanboy on Earth!’ Guilty as charged. But my issue isn’t with the placement of Microsoft or Tableau in the Leader’s Quadrant. Those tools are crushing it and deserve their placement. What I take issue with is IBM’s location in regard to legacy competitors like SAP or Oracle and the overall parameters that determine the ranking.

How does Gartner evaluate software?

Contrary to popular belief, Gartner is not conducting an in depth evaluation of what they call ‘ABI platforms’ to identify and compare the performance, features and ease of use of the software. Instead, Gartner typically relies on the number of inquiries each tool generates for their own consultancy in addition to surveys and interviews to determine this ranking. The natural outcome is a feedback loop where being a leader last year is the primary criteria for being a leader this year. You simply aren’t going to call Gartner to talk about IBM because Gartner is telling you not to bother.

This year they also seem to be penalizing enterprise tools that don’t come bundled with ERPs or other enterprise software. Unfortunately those tools are typically very poor BI tools – I would know, I’ve worked for them.

Cognos Analytics as a case study

This manifests in a yawning gap between IBM’s scatterplot coordinates and what they actually wrote about Cognos Analytics. Unfortunately the image is what carries all the weight in this report. Comparatively few people will read IBM’s entry, so I’ll summarize it for you below:

IBM Cognos Analytics Strengths

• One of the only tools that offers comprehensive enterprise reporting and self-service. Gartner calls these ‘mode 1 and mode 2’
• Strong product vision combining AI augmented analytics, traditional BI and planning capabilities
• Deployment flexibility on prem and on all cloud platforms

IBM Cognos Analytics Weaknesses

• People don’t call Gartner asking about Cognos
• It doesn’t have ‘adoption drivers’ like associations with popular ERPs or office suites
• It costs about the same as standalone BI tools but more than BI tools that are bundled with ERPs or office suites

Very curious. All of Cognos’ strengths are things BI developers, administrators and users care about. And all of those weaknesses are things that procurement people, industry analysts and – most importantly – Gartner themselves care about. It’s not that I think these are unimportant points, but it seems that software features and quality carry significantly less weight.

Where does Cognos belong?

I’m going to be as objective about this as possible and explain my reasoning, but first take a gander at the world premier of the Ryan Dolley BI Magic Quadrant!

Cognos Analytics completeness of vision

Gartner really got completeness of vision wrong. This is where Cognos Analytics shines. No other BI platform on Earth combines the breadth of present day capabilities with the visionary roadmap of IBM. Gartner sort of acknowledges this in their write up but it’s not at all reflected in the MQ image so let me break it down for you:

Cognos Analytics today

• Integrated AI Assistant chatbot for NLQ capabilities and auto-visualization
• ML-driven one-button forecasting
• Correlation and causation engine suggests relationships in data
• Jupyter notebooks for easy BI/ML integration
• Absolute top enterprise reporting capability in the world
• Huge scale bursting, schedule and event-driven data distribution
• Robust extensions allow incorporation of custom visualization libraries, js code. You can make Cognos do damn near anything. Just ask Paul.
• Easy to build self-service dashboards and visualizations
• Easy self-service data modeling
• Self-service storytelling/narrative BI
• Unlimited scale
• Deployable on prem and on any cloud

It can’t be stressed enough that none of the current ‘leaders’ offer such a wide spectrum of features. In fact they offer only bare bones enterprise reporting if they offer it at all.

Cognos Analytics roadmap

The Cognos Analytics roadmap is very strong. There’s a lot I don’t know and a lot I can’t say, but to give you some idea of where things are going:

• Easy what-if scenario modeling and data science for end users
• AI-driven data prep and data quality evaluation
• Deeper integration with Planning Analytics to provide a single portal for enterprise BI and planning
• Dramatically improved NLQ including ontological customization
• Continued containerization and modernization of the platform

Cognos Analytics ability to execute

Let’s be honest. There are some real concerns with how existing Cognos Analytics customers are executing with the platform. I know this because I hear your struggles. And some of that comes from stability issues and UI/UX quirks with the software that IBM absolutely needs to address. The good news is that IBM knows this.

But the reality is that many long term Cognos customers are simply not able to realize the full capabilities and value of the platform and in many cases I think it’s their own fault. The culture and practice that grew up around Cognos was formed in the late 90s and 2000s and it shows. Features that get extensive and successful use in Power BI and Tableau are left virtually untouched by a majority of Cognos customers. Data modules and data sets have improved massively since their less-than-stellar debuts. Their usage in existing Cognos deployments has yet to catch up with just how darn good they are and it’s a shame. Consider this my plea to you to start delivering with Cognos the same way people deliver with Tableau or Power BI. The platform can support it, the technology is there. What needs to change is our collective ambition as a community of practice.

This is not to say that nobody is executing at a high level. There are some people who are using all of Cognos’ capabilities to the fullest and having huge success. And new customers typically do great. But until us old timers collectively start to take advantage of what this platform has to offer I can’t really fault Gartner’s analysis for ability to execute. It’s off, but only a little.

Cognos Analytics and Gartner Peer Insights

Gartner has a lesser known product called Peer Insights and it tells a very different story. Peer Insights features product reviews by verified users rather than the opinion of analysts. Take a second to check out the Business Intelligence page and sort by average review. Scroll past the minor players with a single glowing review and look at the placement of the platforms included in the Magic Quadrant. As of this writing Cognos Analytics scores worse than Tableau Desktop. It scores better than Microstrategy, ThoughtSpot, Tableau Server, Domo, Power BI, Qlik Sense, Oracle, Sisense, Looker, Amazon, SAP and Board International.

What to make of all this?

I hope I’ve done a good job of walking you through why I strongly disagree with IBM Cognos Analytics’ placement on the 2021 Gartner Business Intelligence Magic quadrant. I’ve been as objective as I can in this analysis and I don’t think Gartner is wrong across the board. I really do like Power BI and Tableau. I’m increasingly impressed with Domo and very intrigued to see what AWS does going forward. But this report badly misses the mark on Cognos and it does so by disregarding what a strong enterprise BI platform it is in 2021 and penalizing IBM for the lack of market awareness that Gartner themselves have caused.

---

Want to continue the conversation? Connect with me on LinkedIn and check out the PMsquare YouTube channel. And don’t miss the great Cognos content below!

• Cognos Union Queries in Reports
• Cognos Relative Dates in 11.2
• The 2021 Gartner BI Magic Quadrant is Broken for Cognos Analytics
• Data Modeling for Success: BACon 2020
• Cognos Analytics 11.1.6 What’s New