Max Havey: Hello, and welcome to another edition of Security Visionaries, a podcast all about the world of cyber, data, and tech infrastructure, bringing together experts from around the world and across domains. I'm your host Max Havey, Senior Content Specialist at Netskope, and today we're talking about analyst research. Now you've almost certainly heard the names of many different analyst firms thrown around as the research goes out throughout the year. Things like the Gartner Magic Quadrant, the Forrester Wave, the IDC MarketScape, all of these are illustrating where vendors stand when it comes to key capabilities that consumers are looking for. With that in mind, I've brought in some guests who have experience in this world, and the Gartner Magic Quadrant specifically, to tell us a bit more about these kinds of analyst creations and how they function. First up, we've got Steve Riley, a former Gartner analyst who helped author a number of research notes in his time there, including the Magic Quadrant for CASB, and the Market Guide for ZTNA. Welcome, Steve.
Steve Riley: Thanks, Max. Good to be here.
Max Havey: Glad to have you. And we've also got Mona Faulkner, whose entire job is analyst relations. Welcome, Mona.
Mona Faulkner: Thanks. Glad to be here.
Max Havey: So to start things off here, I think it would be helpful to sort of outline what is an MQ. A Magic Quadrant kind of gets thrown around a lot in industry conversations. So it'd be helpful to just outline why they exist and how does this work? Is it a pay-for-play opportunity? Like, Steve, can you take us through this a little bit?
Steve Riley: Yeah. Well, so let's dispel the myth first. These are not pay-for-play opportunities. In fact, as an analyst, I have no visibility into how much Gartner or how much vendor clients spend with Gartner. The idea there is keeping me ignorant of that information means that how much a vendor spends is much less likely to influence or not at all likely to influence how I would assess them. It's all about vendors creating products that clients are interested in and have expressed a desire to learn more. Now, how these things arise is that when an analyst or a group of analysts realizes that, hey, a new market is emerging, they'll first write a short research note just kind of describing what that is. And then, if it seems that market is gonna persist, the next thing that happens is maybe a couple of years of a market guide. Now, market guides describe what the market is, what problems it attempts to solve, the likelihood that it will succeed at something like that, the risks and benefits of adopting such technology.
Steve Riley: And it lists vendors in that market, but it isn't a comparison just yet. Then if that can survive for a couple of years, yep, that's a good sign that this is a market that has established itself, at least in the beginnings. And then, the analyst will switch to writing a Magic Quadrant. And that's what happened with the CASB. Craig and I watched the market guide for a couple of years, and then we decided it was time to create a Magic Quadrant for that. Now, Magic Quadrants exist to compare vendors in their respective markets. And it compares a couple of different things. One is the vertical axis, the execution axis. Some people think it's all about how much a vendor sells. And while sales is a component, there's like seven or eight other components that go into calculating where on the vertical axis a vendor might land. The horizontal axis is much more about vision, strategy, where is the vendor going to go in the future? How well will they compete against others?
Steve Riley: And again, there's several dimensions that constitute that. And by having these together, an enterprise, any Gartner client really, can use this information along with much other information, like also the critical capabilities note that looks at the tech itself, to determine if this is a vendor who they feel for their use cases is worth partnering and purchasing from, generally over a long term. People don't use MQs for year to year decisions. They use them for more strategic things.
Max Havey: So sort of a thing that helps people better understand the landscape of these capabilities and find the thing that will work best for what they want to be using. What is the best thing they should have on their mind as they're looking for new capabilities?
Steve Riley: Right. And that's why it's so important for clients to read all of the words in an MQ. I know it's tempting to look only at the picture and make a decision. And in fact, I had some clients ask me that one time, just tell me who to buy, to point on the picture who I should buy. I'm like, please, I wrote those words. They're very, very important. Each one of them is lovingly crafted. But this information in the words that aren't necessarily always conveyed in the graphic. So use the whole thing, add it to the SEC, add it to conversations with analysts too. This is how you make a well-rounded purchasing decision.
Max Havey: Absolutely. And that sort of thing takes a lot of information. So that's kind of the other side of the coin on this Mona. Can you tell us a bit about what an analyst relations person does when working with these analyst groups? Can you tell us, what is this role all about as it relates to this research?
Mona Faulkner: Yeah. It's such a great question and it's such a loaded question. And I think I still have to explain to my mom, what do I do for a living every day? [chuckle] The crux of this is if you are doing your job well in the world of analyst relations, you are providing the most quality information to an analyst that is as accurate as possible so that they can do their jobs well. And what is their job? Their job, just to add to what Steve was saying, is to serve their clients with the most accurate up-to-date information about a space, about specific vendors, and what an organization can do with that information to make the right decisions for them. So if you look at the MQ graphic or a Forrester Wave graphic or an IDC MarketScape graphic, you'll see a bunch of dots. But that doesn't mean that a vendor in a specific position is the end-all be-all for an organization. It depends on where an organization is and what their needs are. And so, a graphic can tell quite a big story. A lot of the details, to Steve's point, that's in the write-up of the report tells an even bigger story. It explains the market trends directionally, what the challenges are, what organizations are facing, what are their biggest pain points.
Mona Faulkner: And we all know one organization's pain points are different than another's. So you don't take a look at an MQ, or Wave, or a MarketScape and say, this is cookie cutter for all organizations. That would not be fair or ever accurate. So for an AR professional to do their job well, they need to understand an industry analyst's needs, the markets that they're covering, and help their jobs be so much easier. There is a wealth of information that they could tap into within a vendor. How do you help them navigate that as quickly as possible so they can get their job done?
Max Havey: Yeah, to a degree, kind of helping cut through the buzz and sort of give them the signal for the noise in all the things that are happening in a given year.
Mona Faulkner: Exactly.
Max Havey: Well, in thinking about that then, Steve, as someone who has been in that analyst position, a person who's working on this sort of research, what does that process look like for analysts that are putting together something, say like an MQ? How deep into the technical weeds are you going? What does that process look like in sort of taking in this significant amount of information?
Steve Riley: Well, it begins with the analyst making the case to management that a market is worthy of an MQ because it is a six-month exercise from when you initiate until you publish. And that takes time for analysts who also have been many of the things. So first of all, make the case. Then once approval arrives, you just start the process, which is very well documented. All Gartner Research Notes have a methodology analysts must follow. So that's helpful if you're not trying to invent something out of whole cloth. Typically, the first thing that happens is the analyst will write a welcome pack, inviting a vendor to participate. And it includes things like inclusion criteria. Here are the things that you must have in order to meet the parameters to join the research. Usually, it's around things like certain capabilities that are considered core, a certain amount of revenue, not always, but sometimes a certain amount of deployments across companies or seats, these typical things.
Steve Riley: Now, sometimes people criticize the methodology for saying that MQs often are built such that they ignore startups. And that's absolutely not the case. There can be a sliding scale for these sorts of things, which is what Craig and I often tried to do, especially in the later days of the CASB MQ. Then when vendors respond, the determination is made whether they, the analysts determine whether the vendors meet those inclusion criteria. And then if they do, the next thing that vendors receive is a series of questions. It used to be a spreadsheet, now it's done online that evaluate the vendor in each of those evaluation criteria. Seven of them are, I'll never remember which, seven are horizontal and then eight are vertical, or maybe it's the other way around. It doesn't really matter, but there's 15 different criteria. And through multiple questions, the analysts can have an understanding of how the vendor performs in each of those criteria, which is then used to determine the position in the graph. That's an internal Gartner process that converts the analyst's opinions into the position. And I do have to state, and you can scroll to the bottom of every Gartner research note and find this, that the content in the note represents the opinions of the analysts. And that's true for all firms though, right? It's analysts who write these, it is their impression, their opinions.
Steve Riley: Now, we all strive as much as possible to keep these things fact-based because an opinion without a fact base is kind of pointless, but also the analysts rely a lot on prior experience. So you kind of asked that in your question Max, is how far technically do analysts go, it depends on which part of Gartner they're a member of. So I was in Gartner for IT leaders. My audience was for cloud security research, my audience was CISOs and CIOs. That's who I wrote for, which means no, I didn't like install every product I wrote about. I might have glanced at online documentation if I could find it, or vendors who would share their documentation with me, but I just don't have the capacity. And that's not what my research products are all about, including even the critical capabilities. The way that note is created is by, well, some analysts, not all, want to see demonstrations from vendors. And these demonstrations have a script that is very prescriptive, do this, that, the next thing, do these 37 things in 90 minutes or 120 minutes. And that's how an analyst can determine where a vendor performs technically, which goes into the computation for the use case scores in the critical capabilities note.
Steve Riley: Now, if a customer actually wants to speak with analysts who do install this, at least on the Gartner side, that would be a different group called Gartner for Technical Professionals. They do install the products, they do exercise them on their own, and they write very detailed and lengthy research products as a result of that for buyers who are concerned about the technical minutiae, getting a GTP subscription so you have access to that research is a good choice. But most companies actually feel they don't need to do that.
Max Havey: So Mona, thinking about this as someone who's on the vendor side of this, there's the RFI that comes through, there's demos and things like that. What sort of role does an analyst relations person play in that process, managing that across the line, submitting those sorts of things?
Mona Faulkner: Yeah. So it's an intense process, like Steve was saying, for an MQ it's six months. It varies with MarketScape by IDC and Forrester Wave. But regardless of the analyst firm, it's an intense process. And it's very important that you wear your analyst hat and you want to give a response and answer that helps the analyst, as I said earlier, do their job. So when they ask a question, we answer the question. We don't give a war and peace version of what that answer is, because you're just going to irritate the analyst. And if you're constantly reminding yourself, why are you here? You're here to benefit that analyst firm's end user client. You want organizations that are asking for information and guidance on what different vendors do so that they can make intelligent buying decisions. And those decisions vary from organization to organization. So you put your best foot forward and answer the question to the best of your ability. I would advise be as succinct as possible so that the analyst can do their job. Yes, it does involve opinion, but it also involves a myriad of data points.
Max Havey: Yeah.
Mona Faulkner: And one... Yeah. And one of the data points is what a vendor provides to an analyst. But make no mistake, that is not the end-all be-all to help an analyst form an opinion and understand what the market needs are and where vendors lie. They're getting their data points and they're gathering information from so many different resources. One of them, which is critically important, are from their end user clients. So these are organizations like CIOs and CISOs that Steve was mentioning that are Gartner clients, Forrester, IDC clients. They're calling these analysts saying, "I am dealing with this issue. Here is my technical architecture. Here is where I want to consolidate, for example. How do I even start this process? Right?
Mona Faulkner: And so the analyst will be talking to those end user clients and guiding them. And then there's some discovery. There's conversations. They could be talking to that organization, that individual multiple times, and they are understanding what their pain points are, what purchases they've made, what business struggles they're dealing with, their architectural baggage, and where they're trying to go. And so, they get a lot of data points from their end user clients. They get a lot of data points from vendors. It's kind of limitless. And so, I would say any vendor would be ignorant to think that the information they give to an analyst is the end-all be-all. It is absolutely not.
Max Havey: Mona, you're especially right about that. You're referring to the inquiry process, which for me, I would spend five hours a day on. And the other three were writing stuff or developing presentations. But that was my favorite aspect of being an analyst, was the client inquiry. And learning from each customer or each organization just what they were trying to make this technology do for them. And then, I was able to rely on hundreds of other conversations to give them some conversations too. So it's very much a two-way interaction. But not only with end user clients, it's with vendors too. And I will admit that one of the things that simplifies creating any form of research that evaluates vendors is having a good AR person at the vendor. That is a phenomenal way to make an analyst's life simpler. And by good, I don't mean ass-kisser. I mean someone who cares about how I care about my end user clients and my readers. And it's just great.
Mona Faulkner: Yeah. I think it's really about how do I make this person's job easier and give them their information that they need as quickly and accurately as possible. And that also means you have to understand the different roles that individuals play within your organization, within the vendor side, and also the different analysts and what their coverage is on the analyst firm side. So really understanding all the players and what their needs are and figuring out what to give to them when they're asking for it. And when they may not know what to ask for, but you know that this would be important based on their charter. So it's really understanding the landscape.
Max Havey: And talking about sort of understanding the landscape and the inquiry process and all of this, to what extent does Gartner reach out to vendors, customers to sort of vet this research, to vet these sorts of things? Is that something that happens in this process? And if so, how are they finding these customers? How are they reaching out in that regard?
Steve Riley: It used to be that a defined portion of the MQ calendar was, and it was about two weeks if I'm recalling correctly, was vendors offering the analysts, the authors, sets of customers with whom we could interact. And they didn't have to be Gartner clients. It was just anybody who the vendor wanted us to speak to. That changed when COVID arose. Gartner decided that it was gonna move away from that and instead emphasize more of the peer insights, this mechanism that Mona mentioned earlier. So that's one source of information. The other source, of course, is the people who are Gartner clients and do want to talk, you know, do talk to us during inquiry. We learn a lot then. But it is still the case that every once in a while an analyst might, and this is totally within policy, an analyst might ask a vendor, "Hey, can I just talk to a representative sample?" And even sometimes I would say, can I talk to someone who ultimately shows a competitor of yours? If they're friendly enough, they're going to speak to me, not through like anger or something, but just help me understand where you didn't, or you fell short in some dimension and a competitor won. I want to understand that dynamic. That's useful to me.
Steve Riley: And again, that's where having a good AR person can come in because that person can identify exactly the right kind of customer who almost, the right kind of prospect who almost became a customer, but didn't.
Mona Faulkner: You bring up a really great point. We're not AI bots, machines, robots. We're human beings.
[laughter] Steve Riley: Wait, really?
Mona Faulkner: Not yet.
Steve Riley: Oh, no. [laughter]
Mona Faulkner: Not yet. We're still human beings interacting with other human beings to help make our lives easier so we can get through the end of our day and know that we've accomplished something great. And so, you don't wanna underscore enough the relationship and the trust, the credibility. It is so critical. So when you're working with those analysts over weeks, months, years, you're building that rapport and credibility and trust. And so if you were to ask a hard question, like, give me a customer reference where you didn't necessarily win a deal. Well, that may not be so fun, but it happens. And so, how are you helping that analyst? And why do you think they need that information? Think of it differently. That analyst could be asking your competitor the same question about you, right?
Steve Riley: Yes.
Mona Faulkner: But the goal, if you think about what is the goal, the goal is for me to have a full understanding and spectrum of what is happening in the landscape. Not all vendors win every deals and not all vendors lose every deals. And why? And why? It's because organizations have different needs and it depends on where the market is heading and what bells and whistles an organization or a vendor is bringing to the table for that organization. And so, it's a very complicated job. If I could speak on your behalf for a second, Steve, it's a very complicated job to take all those data points and synthesize it somewhat simply in a report so that the reader can understand and digest within 3-4 pages if you're lucky. And so, how do you help an analyst be successful? And so, you really have to get into the analyst's head for an AR professional to do their jobs well.
Steve Riley: Yeah, I would say one other thing though, is that it isn't entirely incumbent upon asking vendors for things. If a vendor wins a deal that is exceptional in some fashion, it was a breakthrough, maybe they were at the bottom of the list, but they somehow came up to the top of the list or I don't know, you know, whatever it might be a little unusual from a normal way a deal closes. And if it's really big or if it's like their first one to break through in a new market, I love it when vendors reach out to me and say, "I've got a story I wanna tell you about a win we had". And I'm like, "Sure, let's do it". And again, that customer of the vendor does not need to be a Gartner client for that conversation to happen. It's a message for all you AR folks, brag about yourselves to your analysts every once in a while. We do like to hear it.
Mona Faulkner: We officially heard it from Steve. [laughter]
Steve Riley: Brag about your customers, I would say. [chuckle]
Max Havey: Well, I mean, to an extent, all those sorts of conversations, the good and the bad of all that kind of come together for this like sort of 360 degree holistic view of a vendor, of someone who's getting profiled in one of these research notes and keeping all that in mind, all these data points in mind that sort of get distilled down into something like a Gartner Magic Quadrant. What sorts of things should organizations who are looking at like a Magic Quadrant, for instance, what should they be keeping an eye out for when they're reading this? Is there anything specific that you know that they tend to be looking for when they're looking at a research note like that?
Steve Riley: Well, what sort of analyst research is helpful? The whole thing. MQs are decades old by now, at least two, right? And that we know that the format works. It makes Gartner money. It helps move millions of dollars of spend, which kind of blew my mind the first time I learned how to write one of these things. But in reality, they are there for customers to make the best buying decisions that they can. Such decisions are always complicated. They're often fraught with politics. And when an independent third party can offer some suggestion, perhaps it helps the organization feel more confident about whatever decision they make.
Mona Faulkner: Yeah, I would also add in your earlier question or comment, Gartner, Forrester, IDC, none of these firms are pay-for-play. They have stood the test of time and they are here for years and years on end because of the credibility that they have earned. There has to be analyst firms like these that can stand the test of time and that organizations can rely on. It's very important that what an organization does read in MQ, for example, or a Wave or MarketScape, it is very balanced. It talks about strengths of a vendor. It talks about weaknesses. And you as an organization need to decide where are your priorities? Because there is no perfect silver bullet. I'd love to say that we are as close as you can to a silver bullet, but it's not perfect, right?
Mona Faulkner: And so, you think about what your organization's needs are and what their maturity is in their phase of life as an organization and what their needs are and what they need today and what that purchase would help them for the next few years. But that is why they call the analyst and ask for guidance because it's not black and white. They need help and counsel. And these reports are very valuable as a starting point, but it is not the end of the discussion. It gets the conversation going.
Max Havey: For sure.
Steve Riley: Definitely. In the murky seas where there are so many vendors, so many capabilities happening, research notes and reports and things, these can be a lighthouse helping guide folks who are looking for some sort of guidance on their journey. Good analogy.
Max Havey: Well, Steve and Mona, I think that about does it for questions and time that we've got here today. I'm sure we could continue going down this road for a while, but thank you both for coming on here. This is as always a very enlightening conversation about research that I think is constantly going over the LinkedIn airwaves and we're thinking about and seeing. So this is a really nice peek behind the curtain on this. So thank you so much.
Mona Faulkner: Thank you.
Max Havey: Yeah. Thanks so much.
Mona Faulkner: I could talk about this for hours and hours, but I do appreciate the 30 minutes. My favorite subject.
Steve Riley: [laughter] Indeed.
Max Havey: We got to put bounds on it somewhere here. Otherwise, we will go for a cool two hours.
Mona Faulkner: Thank you, Max. Thanks, Steve.
Steve Riley: Yeah. Good to see you again, Mona and Max.
Max Havey: All right. You've been listening to the Security Visionaries podcast and I've been your host, Max Havey. If you enjoyed this episode, please share it with a friend and subscribe to Security Visionaries on your favorite podcasting platform. There you can listen to our back catalog of episodes and keep an eye out for new ones dropping every other week, hosted either by me or my co-host, the great Emily Wearmouth. And with that, we'll catch you on the next episode.
Why Netskope 
){kind=icon}
