The Blog

What being an IBM Champion means to me

January 16, 2020 by Ryan Dolley Leave a Comment

I received some great news on Tuesday from IBM – I have been selected as an IBM Champion again in 2020! And not only that, but my colleagues Cognos Paul Mendelson, Sonya Fournier and Mike DeGeus were selected as well. This brings our total IBM Champion count at PMsquare to 4! I’m extremely proud of our excellent team. We hire only the absolute best Cognos resources and IBM recognizes it. This post isn’t just for back slapping though, lets discuss what being an IBM Champion means to me.

What is an IBM Champion

Directly from the IBM website:

IBM Champions demonstrate both expertise in and extraordinary support and advocacy for IBM technology, communities, and solutions.
The IBM Champion program recognizes these innovative thought leaders in the technical community and rewards these contributions by amplifying their voice and increasing their sphere of influence. IBM Champions are enthusiasts and advocates: IT professionals, business leaders, developers, executives, educators, and influencers who support and mentor others to help them get the most out of IBM software, solutions, and services.

The program runs on a yearly cycle and members must re-apply each year.

What does it mean to me?

Being an IBM champion means more than just a pat on the back and some SWAG. First and foremost it is a recognition of the work I do to feed the Cognos community with this blog, by moderating the Cognos subreddit and contributing to the IBM Analytics community site. Building our community is incredibly important to me and I am so very grateful for the feedback I get from you and from IBM. It keeps me going.

The program is also about being part of a huge community of IBM advocates. The slack is extremely active, extremely helpful and features the top experts on all IBM technologies in the world. The camaraderie in the program is quite amazing and I’ve made friends from all over the world. Meeting fellow champs is the most rewarding part of the program.

Finally, it’s about promoting your professional passion and working with IBM to improve their offerings. Champions receive next level access to the product teams for the respective technologies. If you’ve ever wondered how I know so much about the direction of the product this is a big part of it.

And SWAG, don’t forget the SWAG…

Relative Time in Framework Manager

December 31, 2019 by Ryan Dolley 6 Comments

Kamil asks an excellent question about relative time in framework manager in response to my Framework Manager vs Data Modules article:

Great article. I have one question, is it possible to use relative dates with packages from framework manager?
– Kamil

Like all questions in Cognos, the short answer is ‘no’ and the long answer is ‘yes!’ Let’s take a quick dive into relative time in Framework Manaager.

Relative time in FM? No!

The enrich package screen of Cognos — The Enrich Package interface

There is no ‘easy button’ for using the new relative time functionality with Framework Manager, unfortunately. I was briefly hopeful that this is possible using the enrich package functionality but it’s not there.

Enrich package is an important piece of the Cognos pie so it’s worth talking about briefly. Enriching a package provides needed context that allows Dashboards, Explore and the AI Assistant to do their thing. Most notably, enriching a package will allow Cognos to properly display time and geographic data types and will collect the sample data that allows the AI assistant and Explore to function properly. Enriching a package taxes your system so consider restricting the query subject or running it off hours.

If easy relative time does come to FM this is where I’d expect it to go. It’s worth reiterating that FM itself will receive no changes going forward so it’s time to start making the change to data modules. It’s easier than you think!

Relative time in FM? Yes!

Here’s where things get a little trickier and using relative time with your FM model becomes possible. To make this work we’re going to need to use Data Modules, custom tables and the lookup reference feature.

Step 1: Add a package source to a data module

Adding a package as a source to data modules — A package has been added as a data source to this data module

Click the ‘new’ icon and select data module
Navigate to your package in the folder structure, click on it and click ‘OK’
The data module screen will open with the package visible in the data tray

Step 2: Create a custom time table

Building a custom table in Cognos — Building a custom time table from a package

Click the ‘Custom tables’ tab and click ‘Create custom table’
Click ‘select tables’ and click the package source. All the tables in the package are displayed on the left.
Click ‘create a view of tables’ and click ‘Next’
Don’t forget to give your custom table a new name!
Click ‘invert’ then select only the table with which you want to use relative time
Click finish. Your custom time table will appear in the data tray

Step 3: Add relative time functionality

Adding relative time to the custom table in Cognos — Relative time can be added to your custom time table

Click the ‘Add sources and tables’ button and select ‘Add more sources’
Navigate to the ‘Calendars’ folder in the samples and select the ‘Fiscal calendar’ data module
Click ‘OK’. The FiscalCalendar table will appear in the data tray, hidden by default
Expand your custom time table, click the date you wish to use for relative time and click the ‘properties’ button in the upper right. The properties window will open.
In the properties window, select ‘FiscalCalendar’ in the ‘Lookup reference’ drop down menu.
You now have relative time functionality in your data module!
Rinse and repeat for any additional time or measure fields that require this functionality

Step 4: Join the custom time table to the package

Joining a custom table to a framework manager package — The custom time table can now be joined to the package

Click your custom time table and choose ‘New… Relationship’ in the pop up menu
Select the appropriate table to relate the custom time table to the rest of the package. Oftentimes this is a fact table.
Select the appropriate field(s), cardinality and join type for the join.
Click ‘OK’

There you have it! Relative time in Framework Manager (sort of)

At this point you can save and use your data module, which is made of your pre-existing package plus one or more custom tables. This doesn’t solve the obvious problem that your existing content references the package and not the new data module, but new content can be built off this module. The module will even inherit changes made to the package. And there you have it – relative time in Framework Manager… sort of!

Read on to level up your Cognos skills!

Cognos Analytics Data Servers

December 30, 2019 by Ryan Dolley 2 Comments

Joe Schmoe asks a fairly common question about Cognos Analytics data servers in the comments to my Framework Manager vs Data Modules article:

In Cognos Analytics 11.1.5, I don’t see a way to use a data source to feed a data module. It looks like in order to use data from a database, you need to use a package — which means you need to use Framework Manager anyway. Am I missing something?

Let’s take a look at how data servers work in Cognos Analytics to (eventually) answer Mr. Schmoe’s question.

What is a data server?

A data server is simply a connection to a database that has been defined within Cognos Analytics. This definition contains the database url, connection parameters, username and password necessary for Cognos to authenticate with, issue queries to and receive data from the desired database. A data server can be re-used by an infinite number of data modules to provide self-service users and report authors controlled access to desired data while maintaining high quality row-level security. Longtime Cognoids will no doubt say ‘Ryan, that sounds exactly like what we used to call a data source!’ Yes. Yes it does.

Cognos Analytics data server vs Cognos 10 data source

Cognos Analytics data servers and Cognos 10 data sources are fundamentally the same while having some differences in where and how they are configured. If you’re familiar with Cognos 10 you can breath a sigh of relief as your existing knowledge is almost 100% applicable. You just need to learn a slightly new UI.

Cognos Analytics data server

Introduced in Cognos Analytics
Provides a definition which Cognos uses to connect to a database
Configured via the manage menu
Requires JDBC drivers and uses exclusively JDBC connectivity
Uses Dynamic Query Mode exclusively
One data server can be used by infinite data modules
Source for data modules only – no Framework Manager [Editors note: Commentor Jerzy points out below that you can use data servers as a source for FM for DQM models only. I didn’t know that! Thanks Jerzy!]

Cognos 10 data source

Introduced in Cognos 8? 7? Reportnet? I was in college, it’s all very hazy
Provides a definition which Cognos uses to connect to a database
Configured via legacy administration console
Uses a wide variety of connection types including JDBC, ODBC and others
Uses Dynamic Query Mode or Classic Query Mode
One data source can be used by infinite Framework Manager packages
Source for Framework Manager only – no data modules (we’ll cover a simple workaround later in this article)

As you can see, data servers and data sources serve the same function within Cognos – defining a reusable database connection for Cognos – and have many of the same features. They differ primarily in the types of databases to which they can connect, the connectivity standard they use and how they are configured within Cognos.

Configuring data servers

Configuring a data server is easy and is – for the most part – a one time, ‘set it and forget it’ task most frequently done by Cognos administrators. Like everything in Cognos you have a few options for how to proceed. Before we dig into that though, let’s take a look at the info we need regardless of how we’re configuring data servers.

Data server configuration checklist

Identify which type of database you want to connect to
Make sure you have permission to connect to it!
Check to make sure it’s supported by Cognos
Check to see if there are any special considerations for that database
Acquire the relevant JDBC driver and install it in the ‘drivers’ folder of the Cognos install directory
Ask your DBA for the following information:
- URL for connecting to the database
- Database name
- Schema name
- Username for authentication
- Password for authentication

Configuring a data server from scratch

Now that we have our ducks in a row we can build the data server. This is incredibly easy, though you might not know it from reading these instructions. Let me break it down for you:

Click the manage icon then select ‘Data server connections’
Click the plus button in the upper right of the data server connections window
Select your database type from the list that appears
The data server configuration window will automatically open for you to input your connection url from above.
In authentication method, select ‘use the following signon’ and input your username and password
Click ‘Test’ and ensure Cognos can connect to your database
Click on the ‘Schemas’ tab and set the following:
- In ‘Load options’, choose whether or not you want Cognos to retrieve statistical samples from the database. I generally leave this alone. Turning this off will disable certain advanced features of Cognos – like auto joins in data modules
- In ‘Tables’, select which tables you want to expose via the data server. I highly recommend you select only the tables which you need. Allowing Cognos to profile your entire production database during work hours is an extremely bad idea…
Click ‘Save.’ Your data server is ready for use in data modules!

Converting Cognos 10 data sources to Cognos Analytics data servers

While it’s true that a data source configured in Cognos 10 cannot directly feed a data module, you can easily convert a data source to a data server provided the administrator has configured a jdbc connection for the data source. To do so follow these steps:

Click the manage icon then ‘Administration Console’
Click ‘Configuration’ tab and ‘Data Source Connections’
Click the ‘set Properties’ icon for the data source you wish to convert
Check the ‘Connection’ tab
Check the box next to ‘Allow web-based modeling’
Click ‘OK’
Exit the administration console and open ‘Data Servers’ in the ‘Manage’ menu. You should see your data source list as a data server.

Using data servers in Cognos analytics

By following the process outlined above we can easily use data servers to get our data into data modules and then into reports, dashboards or the new exploration feature without ever needing to reference Framework Manager. I hope this satisfies Mr. Schmoe and as always, contact me if you have any questions!

Read on to level up your Cognos skills!

Cognos Analytics 11.1.5 What’s New

December 21, 2019 by Ryan Dolley 2 Comments

It’s the holiday season and IBM has given us an unexpected gift in the waning days of 2019. No, I am not referencing the fact that Cognos Analytics is currently half-off! IBM rather suddenly dropped a new Cognos release chocked full of stocking stuffer features for all the good little Cognoids across the land. So let’s read on to learn what’s new in Cognos 11.1.5.

Schematics

The schematic feature of Cognos Analytics 11.1.5 enables interactive custom maps for things like stadium or airplane seating charts — Schematics allow you to turn anything into an interactive map

Schematics are undoubtedly the eye catching new feature in Cognos Analytics 11.1.5. Schematics allow Cognos to dynamically visualize data on an image – think a stadium or airplane seating chart, a hospital floor plan or a diagram of a machine. The schematic is interactive and plays nicely with other features.

Schematic Views

One interesting feature of Schematics is the ability to create ‘views.’ Views enable end users to automatically filter the schematic to render only those sections that interest them. In the example above I created two views of the United States – one titled ‘Midwest’ and one titled ‘Rest.’ You can see that unselecting ‘Midwest’ hides those states. For a more complex example, imagine a schematic with a detailed diagram of a car. Users could interactively choose to view only the suspension, transmission, power train, etc… Very powerful!

Schematic Management

Managing Schematics is not exactly straight forward but IBM has provided excellent instructions. Multiple schematics can be bundled together into a package, which allows report authors to select from a library of related schematics. These packages are managed just like the custom visualizations that debuted in Cognos Analytics 11.1.4.

Schematics are currently only available in reporting.

Data Modules

You guys know I love data modules. Well in Cognos Analytics 11.1.5 the changes to Data Modules are so, so, SO GOOD. Just ridiculously good. I got a huge smile of my face using it for the first time. You have no excuse at this point, they’re just too awesome!

Custom Tables

IBM has taken all of the scattered virtual table/view building functionality and very smartly bundled it together into a new, clean and totally kick ass interface called ‘Custom Tables.’ From custom tables you can easily blend tables or spreadsheets from any source together while understanding at a glance how all that blended data is related. I cannot tell you how crucial this feature is as I and my customers build increasingly complex data flows in Cognos.

Members in the data tree

Cognos Analytics 11.1.5 adds the ability for members to appear in the data tree regardless of source type — Members are now visible in the data tree regardless of source type

As of Cognos Analytics 11.1.5, members are now visible in the data tree regardless of source type. Yes, you read that correctly – even for relational sources. You can drag and drop individual members into visualizations, use them to build sets, etc. Even against excel spreadsheets. Holy cow! This change applies across the entire UI, not just data modules.

Show generated SQL and query control

Another huge quality of life upgrade – you can now see generated SQL directly in data modules! And you can now use a property called ‘item list‘ to determine whether the data module fetches the entire table or uses minimized SQL, just like you could in FM.

Dashboards

Dashboards received a host of small quality of life changes plus one big, frequently requested enhancement that I know you’ll love. So let’s start there.

Dashboard to dashboard drill through

Dashboards can now drill through to dashboards — Dashboard to dashboard drill through – it finally happened

Dashboards can now utilize other dashboards as drill-through targets. IBM did a great job of making this extremely straightforward so end users should have no problem with it. It works like this:

Create two dashboards that share at least one data source
Establish a Drill-through definition (pictured above)
Notice there is no mapping of parameters or anything like that!
In the source dashboard, select a visualization element and choose the drill-through icon
The target dashboard will open and a new filter will appear in the ‘All tabs’ filter section
The target dashboard has been filtered by the values passed from the source

Customize tabs

Cognos 11.1.5 brings significant new tab controls

You can now customize the location an appearance of Dashboard tabs. In the example above, I moved the tabs to the left side and formatted the fourth tab to appear in red with some cool spectacles as an icon.

Various and sundry improvements

Here is a summary of additional enhancements to Dashboards:

Show and hide rows and columns in crosstabs
Customize missing values in visualizations
Assistant-suggested questions based on context
Enhancements to forecasting (as in it’s more accurate)

Administration

I don’t write a ton about administration on this blog. Mostly that’s because it’s boring. However there are some important changes that I want to highlight for you.

Save reports to cloud storage

Save to cloud is a new capability that allows you to, well, save reports directly to 3rd party object storage services. This allows you to archive your 10-years-and-running collection of daily PDFs to Amazon S3 instead of in your content store. I think this is hugely helpful, especially for customers on Cognos cloud where archival and bursting capabilities have been severely restricted. It also portends good things as far as integration with 3rd party cloud vendors going forward.

AI Capability

The new AI capability controls who can use the AI Assistant in Dashboards and Explore. I know the ability to turn this on and off is something many Cognos teams have been asking for. My advice to you is to just leave it on! But if you absolutely must lock it away, now you can.

Other new features of Cognos Analytics 11.1.5

There are a host of other small changes that I’m not going to cover here. Many of the changes to Report Authoring, Explorations, etc… are covered above as there are quite a few UI spanning updates in this release. Overall I’m extremely pleased with this one. Schematics are a great new feature while the changes to Data Modules are exceptionally good. I suggest -as always – that you update to 11.1.5 as soon as possible. And as always, hit me up with any questions you may have.

Read on to level up your Cognos skills!

Framework Manager vs Data Modules

November 20, 2019 by Ryan Dolley 15 Comments

I have spent a lot of time showcasing data modules to audiences across the world in-person and in livestreams, helping people understand how and why to use them as part of Cognos Analytics. The most consistent question I receive – by far – is about understanding framework manager vs data modules. There are a lot of outdated opinions and outright misconceptions floating around so let me outline the exact feature differences between framework manager and data modules as of Cognos 11.1.4.

What do data modules and framework manager have in common?

The answer is ‘a lot’ but this wasn’t always the case. In the 11.0 releases data modules were missing many essential framework manager features and didn’t offer compelling reasons to switch. Of course that has changed. As of 11.1.4 framework manager and data modules both:

Produce data models that can be used with all Cognos 11 features
Join dozens or hundreds of tables across multiple databases
Execute cross-grain fact queries (aka the dreaded determinants)
Build simple or complex calculations and filters
Build alias, view, union and join virtual tables
Secure data by groups, roles and users
Create OLAP-like dimensional hierarchies
Offer enterprise governance, auditablity and security

Oftentimes people washed their hands of data modules a couple years ago and are surprised to see virtual tables, cross-grain fact queries and security by groups. These features may exist in both but the implementation in data modules is superior from a usability perspective.

Column dependencies can handle degenerate dimensions, unlike framework manager — Column dependencies go beyond what was possible using determinants in FM

What do data modules offer that framework manager does not?

Again, the answer is ‘a lot’. The 11.1. release takes data modules beyond what is possible in FM with a host of powerful capabilities and quality of life enhancements. The following features are either exclusive to data modules or done infinitely better in data modules.

Natural-language and Ai powered auto-modeling
Automatic join detection
Easy integration of excel data
Ability to easily clean data
Flexible hierarchies that go up, down and across (navigation paths)
Easy measure binning and attribute grouping
Easy extraction of year, month, day, etc… from data data types (split)
Automatic creation of relative time filters (YTD, MTD, PYMTD, etc…)
Automatic creation of relative time measures (YTD actuals, PYTD actuals, etc…)
In-memory materialized views within Cognos Analytics
In-memory query cache
Easy multi-model inheritance for single source of truth
Degenerate dimension aggregation (column dependencies)

Some of these features are absolute game changers for how I craft highly performant, easy to use and self-service friendly data models. Consider ~~the coconut~~ relative time; because this was such a titanic brain buster in framework manager only the most skilled developers could deliver. Now it takes minutes for end users to implement.

Building a dozen relative time filters can be done in as little as five clicks. — It took five clicks to build the relative time filters that take ~1 trillion years in FM

What are data modules missing?

There are still some things data modules lack:

Object level security
DMR capabilities
Parameter maps
Multiple connections for data servers

If I’m being honest, I don’t really recommend you use many of these features for new development in 2019 unless you absolutely have to, particularly DMRs. DMRs are very powerful for those who know MDX but a true maintenance and self-service nightmare in the long run. I cannot count the number of clients who are stranded with critical DMR based reports they cannot understand. In any case, a little bird told me that DMR-like functionality will grace data modules soon.

Going beyond the feature list

Comparing framework manager vs data modules feature for feature, we can see how data modules have few shortcomings and offer huge advantages. While this is a common way for IT folks to think (and I would know, I’m one of them!), I argue that it badly misses the point. By using data modules an IT professional can do weeks of FM work in an afternoon while a self-service user can easily accomplish tasks that will otherwise be done in Power BI. I repeat it often but I’ll say it again – data modules are the key to modernizing your Cognos Analytics environment and delivering content with the speed modern users demand.

What do you recommend?

I’ll parrot Cognos offering manager Jason Tavoularis and say, ‘use data modules unless you can’t.’ And as you can see above, the list of reasons you can’t has become quite short. I start ALL consulting engagements under the assumption that we’ll be building data modules and I’m always happy with the results.

Cognos Analytics Forecasting Overview

November 7, 2019 by Ryan Dolley 3 Comments

Cognos Analytics 11.1.4 brings exciting and long sought forecasting capabilities to the platform. While Cognos is far from the first analytics tool to offer forecasting, IBM’s implementation is more flexible, more powerful and easier to use than competitors. Let’s take a dive into Cognos Analytics forecasting to understand why.

Cognos Analytics forecasting is a powerful and easy to use forecasting tool — Cognos Analytics forecasting is just really cool.

Where is forecasting available?

Forecasting is available in line, column and bar visualizations in Dashboards, Stories and Explorations as of 11.1.4. Report Authoring is on the horizon (in fact there’s a ton of cool stuff coming for Report Authoring.) The ability to forecast column and bar charts sets Cognos apart from Tableau and Power BI. Cognos hides the forecasting button until a visualization meets the following criteria:

Visualization is the proper type (line, column or bar)
Visualization has a measure in the Y axis
Visualization has a time element in the X axis

Once this criteria is satisfied the forecast button appears next to the insights button in the top right of the visualization.

You start from any line, column or bar chart — Forecasting is available for line, column and bar charts

In the above example I use the ‘Great outdoors data module’ available in the samples. Using the Sales table I apply Revenue to the Y axis and nest Year and Month on the X axis – the perfect scenario for forecasting.

Single category forecasts

One click of the forecast button is all it takes to build a predictive forecast — Forecasting a single category is simple

Click the forecast button, turn the feature on and there you have it – your very own Cognos Analytics forecast! You may immediately notice that the slope of the line changed dramatically once a forecast was applied. I first assumed this was a bug until I clicked the yellow ‘!’ next to the forecast button and read these magical words:

Cognos will even automatically re-order your time categories — Forecasting automatically orders time categories chronologically!

That’s right – Cognos automatically reordered the time categories to be in chronological order rather than alphanumeric! I’ve been waiting for this feature for all my life and it’s finally here. Now we just need IBM to automatically apply it to all visualizations, not just forecasts.

Multi-category forecasts

Adding additional categories builds a multi line or bar forecast — Forecasting multiple categories, just as easy

In the above example I add ‘Retailer Type’ from the Retailers table to the color attribute of the visualization. Cognos instantly re-applies the forecast to each individual category. This is one advantage Cognos has on Power BI, which currently forecasts a single category only. This makes data discovery on forecasts extremely fast and easy.

Interacting with forecasts

Hovering over an individual point in the forecast will show you the confidence upper bound, forecast and lower bound for that intersection of the chart:

In this example I hover over the projection for ‘Outdoors Shop’ in September, 2018. You can see Cognos’ predictions for the upper and lower bound.

Focusing on a single category will visualization the confidence interval — You can visually see the upper and lower bound as well

Clicking on the line for ‘Outdoors Shop’ will filter the dashboard by this category, bring the selected category into visual focus and plot the confidence intervals as they evolve in the forecast. Super duper cool.

Configuring forecasts

Forecast configuration options are available in the menu that appears when the forecast button is clicked.

Cognos analytics forecasts have several configuration options to customize how the forecast is generated — Forecast configuration options help tailor results

Let’s take a look at what these options do:

Forecast periods: The number of periods included in the forecast. The default ‘Auto’ will project forward 20%. If you have 10 months of data, auto will generate an additional 2 months of forecast
Ignore last periods: Ignore last periods is useful for cases where you have incomplete data at the end of your chart – for example, the month of November currently has 6 days worth of actuals. Ignoring this incomplete period creates a more accurate forecast.
Confidence level: Controls the confidence level displayed on the tool tip and confidence visualization – options are 90%, 95% and 99%.
Seasonality: Cognos automatically detects seasonal fluctuations in your data and accounts for them in its forecasts – think about retailer revenue during the holidays for example. The default ‘Auto’ setting will build multiple models with different seasonal periods and and select the best one, however you can specify a season period by entering numbers here.

That last option is incredibly powerful and another strong advantage for Cognos vs Power BI and Tableau. The inability to detect and account for seasonal variation when forecasting renders this functionality useless for many industries – retail, hospitality, utilities – and only Cognos has it.

Understanding forecasts

Cognos uses exponential smoothing models to generate these forecasts. I don’t know what the means either but you can read about it here. The important thing to know is that you have access to the forecasting statistical details in the data tray.

Cognos analytics forecast statistical details are available for each category in the visualization — For those that know, here are the statistical details for the forecast

I’m a BI/DW guy and this screen might as well be magical incantations to me, but the fact that Cognos provides this level of detail means that I can always find a friendly wizard to explain it. IBM has a nice explanation that I also don’t understand here.

Bar chart forecasts

Switching to a column/bar chart retains the forecast and forecast parameters that were applied to your line chart.

One thing it doesn’t do, which I just discovered in writing this blog post, is automatically sort the time category in chronological order like it does with line charts. I’m not sure if this is purposeful, an oversight or a bug. You can vote for my feature request to rectify this here.

Cognos Analytics forecasting is the real deal

I have to say, I think this feature is extremely well done and I’m certain end users are going to love it. If you need help prepping your environment for self-service see my guide here. Allowing your users to build their own data modules will make forecasting even more powerful, read more about it here.

IBM was late to the market with it but delivered something much better than their competitors. This has been a recurring story in the last year – Cognos has come so far since the 11.0 releases. It’s up to IBM – and to us – to make sure the broader BI community is aware of the good things going on in Cognos Analytics.