A Fireside Chat with an Auditor

Author: Sam Raco

During a Wisdom 2022 Spotlight Session, TMG’s Sam Raco had a ‘fireside chat’ with a leading license verification auditor from KPMG, Maurice Pagnozzi, his former boss, and promises him there will be “no free passes”. Here is an edited version of the dialog which also includes some additional questions asked after the session.

Sam: So, straight off the bat Maurice, how exactly does a vendor decide who they are going to audit?

Maurice: That’s an interesting question, I started doing this 18 years ago, and what they used to do then is completely different to what they do now.

The traditional model was to put every customer into a program and look at them every 2-3 years and decide if they should review them. The decision might be influenced by the findings of a prior review, and whether agreed actions were taken to remediate non-compliance. Other factors included; did the customer have software with complex licensing metrics where the vendor can help support how to monitor and manage deployment, did the customer decide not to renew their agreement?.

That was the traditional model. I think the impact of COVID caused a shift in attitude that led to vendors treating customers as true customers. Now I would not say this about all vendors, (I have worked with some of them not all) but the focus shifted to trying to get the most out of their relationships.

Their opportunity [to achieve quota] now comes from renewals. So instead of audits, the vendors are initiating license reviews to produce baselines to help them understand what is in the customers environment. –Right, you’ve got a renewal coming in six months so let’s start the process of baselining, here is KPMG to help, so let’s work  together to get to a view that makes sense for us [the vendor] and the customer. 

The actual selection process in the majority, has become more collaborative, unless the customer is keeping them out and blocking them, then it becomes an audit.

Sam: So to be clear it is not true that today it is still used as a tactic for them to meet their sales targets?

Maurice: Look, it is still true because it is linked to sales, linked to renewals, but they are open about it, – you have a renewal coming up and we want to continue to work with you. So yes, it is still linked to sales, but in the traditional model it was about closing the gap in sales targets – now the model has shifted to focus on clients who are renewing. On the other hand, if there is evidence that they have got software but are not paying for S&S or not providing data being requested, the vendors might consider and audit.

Sam: For the next question you can only answer yes or no – is it possible to completely avoid an audit?

Maurice jokingly states that Sam knows him better than that, and that it is not in his nature to give yes/no answers, so the answer he gives is “yes, but…”

Maurice: Can you avoid an audit? I think if you take the mentality of audit out of your head, yes, you can avoid an audit. The reason I say this is that, I have noticed a change in the market where the customer is more connected to their vendor than before and they are reporting regularly on license consumption. They know their environment well in terms of usage and future need and are going proactively to the vendor and saying I need to true up, –so why would I go and audit a client that is actually talking to me and wants to buy more. You can avoid an audit by knowing your environment and proactively communicating with the vendor.

Sam: Ok, if you decide that you do not want to follow that path what other factors might minimise the probability that you would be selected?

Maurice: Well, I think audits will be here for a while, so to minimise it, you should put in place a robust SAM framework, with recognised SAM tools and maintain regular connection with your vendors on current and future needs, true up as they go, then likelihood of a review is very low. This keeps the vendor focused on future rather than historical spend.

The key is ongoing proactive communication.

I think it is also important to switch your focus from minimising the probability of being selected to minimising the issue of a big finding.

Maurice refers to the TMG spotlight session ‘Executive Support for ITAM’ run by Keiren Tilbrook.

During an earlier session, Keiren said that no one wants to be the one to present a big-finding to an executive sponsor. So, the focus should be on trying to minimise the problem of a large exposure. Therefore, it is important to know your environment, you need quality data. Without quality data everything else is going to be a problem. If you’ve got quality data you are more likely to be comfortable to approach your vendor and have open conversations.

Minimising the probability of being selected is also difficult because vendors vary in their approach to audits. Some vendors might choose to audit you because you have decided that you don’t want to stay on their technology, others may be aggressive simply because they want to get paid for what you are using.  The number of vendors in the market has increased and so the probability that you will be audited is higher.. However, vendors are watching the market and initiatives changing the relationship between the customer and the vendor. For example, IBM’s Approved SAM Provider Program, (IASP) which was rolled out three years ago and demonstrating success for both the customer and IBM, with focused optimisation and remediation strategies, with customers proactively managing their environment with their vendor.

Sam: So good quality data is a key factor?

Maurice: Well, it is Sam, because to me, it is your responsibility to know your environment, not the vendors.

Most of the people I have dealt with [during audits] do not manage licenses as a full time job, so when an audit comes along, they become quickly frustrated when they realise they don’t have the quality data needed to demonstrate proof of entitlement and usage data.

They then have to reactively inform the executive sponsor of the need to negotiate down from a large multi-million dollar finding. That’s when you really need C level sponsorship. On the other hand, if you have good quality data rather than give your sponsor an unwelcomed surprise you can plan and prepare, and have the sponsor join you at the negotiating table.

Sam: I will come back to the question of data quality later. Let’s now talk about the actual audit. When you start an audit what are the signs that it is going to be challenging to conduct?

Maurice: Firstly, it is important to remember that it is the vendor that initiates an audit, not KPMG. So, the audit starts with notification and communication from the vendor.

The first sign is that the customer refuses to make contact. Then, when they eventually do make contact, it takes weeks or months to set up the kick-off meeting. Then when we finally get a kick-off meeting, they decide, – you know what, we think we need an NDA and that drags on for months.

The delays may be because they are concerns because they have not looked after their environment and so they want to clean up. They have not been doing anything wrong, most findings are not intentional non-compliance, it is just that the environment has gotten out of hand.

Other signs of a challenging audit are when we need certain data, but they won’t give us access to the systems, and instead print off a whole wad of paper and say you can’t leave the room with this. It is all these little things, but we have been doing this long enough we know how to get through it. We are happy to sit side by side with the customer and coach them on how to find the data they need.  Ideally, I’d like to be in a position where I am also able to give a customer advice during a vendor initiated review or audit, but unfortunately I can’t, all we can do is provide factual information. However, the customer can engage us directly after the review for support and to clean up the environment.

Another sign of a challenging audit is when we give them the first draft of the report and they throw it back at us and say I am not paying -however they should remember we are just the messenger reporting back data provided to us by them.

These are some of the signs, but I think in the last five years, the challenges have become less. A mindset shift seems to be –if I can get the vendor to pay for a review of our environment that gives us information that is accurate and really valuable , and if we are going to get a bill at the end of this hopefully it is a bill I can plan for and get my C- level support.

Sam: So back to the question of data quality, but in the context of how you approach an audit what is the methodology and what you expect to find and what you expect as quality data?

Maurice: Firstly, I would like to note that our role is to review usage against contractual entitlement and report a factual license position at a point in time. While we would like to provide as much guidance as possible to the customer about systemic issues and process improvements our scope is limited to factual reporting and not advisory.

The approach is simple, after the notification we run a kick-off meeting to clarify the scope. We then run a technical workshop where we try to map out the data, we need working collaboratively with the people that own the data rather than attempting to do this ourselves, we then issue a detailed data request.

We hope that they have a tool that can quickly provide what we need. However, we also run our scripts across some of the environment, to get a source of truth.  If the tool is not operating effectively, and the data is not good quality, we need to know.

Once we get the data it is just a matter of analysing it against contract terms, creating a draft report that goes to the customer first to check if it looks right. If report doesn’t look right, we want to know where it is wrong and what evidence exists to support the position. If they don’t have any evidence, we can’t change the finding.  We’ll capture the commentary, but they need to connect with the vendor and negotiate their position.  

A common ‘missing evidence’ scenario is that in the past, the customer had an agreement with the vendor that was ‘off contract’, but never actually put down in writing. This makes it difficult for us to accept as evidence. I can tell you from my experience, having it in writing is gold, this gives you the leverage and we are happy to support it.

We will put everything that makes sense in the report, but we can’t change facts, unless you give us evidence. We don’t do anything but factual comparison of entitlements versus data you provide, so data quality for us is critical.

Sam: How do you decide what is a source of truth?

We assume in the first instance that the customers’ SAM tool is configured correctly and is reporting accurately.  We then run our scripts which have been tailored and refined over, many years and reconcile the reports out of the tool with the output from our scripts.

If they reconcile, great, if they don’t, we go through a process of seeking further evidence to explain the differences, this may involve manually checking a number of servers. We hope this is not needed as it takes time out of their day as well as ours. The strength of the data quality determines the amount of time spent on this activity.

Sam: Let’s switch tact, talk about future of audit, where will it be in 3-5 years’ time?

Maurice: That’s a good question. I thought when I set up vendor reviews over 18 years ago for several mega vendors that this would only last 3-5 years. The style and frequency may have changed but 18 years later, it is still going on, more vendors have entered the arena, some of those that have left continue with reviews but under a renewal program or opportunities to upsell into other offerings such as their cloud platform.

In 3-5 years’ time I am hoping there will be a lot more collaboration between vendors and customers to drive a different approach to reviewing the environment. I think the IASP (IBM Authorized SAM Provider) model put out by IBM will become more popular. If a customer enters into an IASP agreement, IBM will not audit them for the duration of that agreement, instead they will allow approved SAM providers (such as KPMG) to help the customer collate and report on their IBM environment on a quarterly basis with recommendations to remediate and optimise being part of the service offering.

Oracle announced in a conference in Vegas that they are looking to implement a similar program. Microsoft also had a similar program for a while and now have shifted from compliance to trying to move and keep their customers in the cloud. Other vendors will choose to focus primarily on renewals.

However, all vendors unfortunately will have to do some form of auditing to ensure they are being paid for the licenses that are being consumed because not all customers are going to proactively tell their vendor if they are over using or not, even if not doing this intentionally. So, I believe, audits will continue – but I hope it will be more of a collaborative effort, rather than a traditional audit.

Sam: What makes the difference in between an easy or challenging commercial settlement?

Firstly, it is the understanding that a vendor seeking payment for non-compliance is not imposing a penalty, but rather is seeking monies owed for software used in line with the agreed terms.

It helps if the customer has built a strategic relationship with the vendor before the review, audits are often unnecessarily viewed as adversarial, however, it does not need to be that way. It also helps if the customer allows the vendor to be involved throughout the review.

Other factors that make it an easier settlement include; providing unrestricted access to software environment during review, ensuring that no surprises come out at end during the 3-way discussion, taking action on any remediation steps that were agreed. Sometimes the vendor will not chase non-compliance if it was evident the customer has taken action to try resolve the issue.

Sam: ok, so we can follow your advice to make for an easier settlement process, but what can we do to minimise the actual amount of the settlement?

For over 90% of reviews that we have conducted, either on behalf of vendors or directly for the customer we identified non-compliance (and over-licensing).  Once an audit starts, there is not much you can do to minimise the size of the non-compliance, the main work should happen well before that. You should always be proactively working to minimise non-compliance, by ensuring that you deploy based on your agreed contract terms and reviewing your environment regularly. You should know the high-risk products that are likely to cause problems and review these more regularly than others.  I think it is also important to invest in tools that help you to maintain good quality data which as mentioned early is critical.

Finally, having a good relationship and cooperating with the vendor during the audit not only helps to make it a less adversarial discussion as mentioned earlier, but also makes it easier to propose or seek concession and negotiate a lower settlement. However ultimately it is the vendors right to pursue monies owed for software used, and they will make the final call on the amount 

Sam: What are 3 common misconceptions about an audit, or the vendor?

Maurice: The first and most common one is a vendor only invokes an audit when they are falling behind on their sales target. We discussed this at length earlier.

The second is that if the customer can delay the commercial settlement discussion until month end or quarter end, when the vendor is likely to be under pressure to meet their quotas, they will do whatever they can to settle, including giving away concessions.

The third is that if a customer makes it difficult for the auditor to gain access they will go away.

Sam: As your closing remarks, what are three pieces of advice you would like to give to customers?

Maurice: Firstly, stay connected with your vendor.

Secondly, implement a SAM framework that meets your needs, including appropriate tools so you can proactively manage your environment and true up when needed. Ensure you have quality data.

Thirdly, acknowledge the vendor doesn’t want to penalise you for non-compliance. They would prefer working with you to help manage your current and future needs.

Sam: Maurice that’s all the questions I have, thank you for being a great sport and allowing us to have a frank conversation.

If you would like to hear more information about software audits and how you can proactively prepare, please reach out to us via email at info@tmg100.com or visit our website https://tmg100.com.