As an industry, are we getting better at measuring the value of our output? Article icon

As Helen Dunne reviews a lively morning discussion

NIGEL FAIRBRASS, GROUP DIRECTOR OF MEDIA RELATIONS, G4S

Boards are recognising that reputation can and should be managed as a strategic asset that really can deliver competitive and commercial advantage to a business. They recognise that they can’t outsource that role to the communications department anymore, that reputation changes too fast and that reputation needs to be managed far deeper into the organisation. Measurement is at the heart of that debate.

I do think that the model needs to change. The whole debate needs to be about measuring business objectives and the whole evaluation debate needs to be one that is grounded in talking the language of business. As a comms function, or a reputation management function, we’ve been quite poor at talking the language of the business and grounding the whole evaluation debate as one which the business actually recognises.

Over the years, I’ve been culpable in taking wonderfully academic and extremely abstract presentations on reputation around the world and presenting them to business leaders and seeing them glaze over. They have absolutely no idea how it’s relevant to their business.

For me, the fundamental challenge is actually making an argument as to how reputation management enhances the value of the business commercially. We’ve got a fight on our hands. The corporate risk guys are being showered with a huge amount of resource in many companies and the way I see it, they have an enormous amount of science in the way they go about evaluating risk. The board are saying to them Risk is really, really important. Reputation risk is really important. It’s a primary risk. The risk guys are determining what the reputation risks are for the company: they’re defining what reputation is, how to measure it and identifying key performance indicators.

Corporate affairs and reputation management teams need to step up. If corporate risk teams take on reputation, there’s a risk that you end up with a tick-box approach to reputation management. By defining it as risk, the business potentially loses sight of the fact that reputation management should be an opportunity.

Corporate affairs is in a far better position to make the argument that good reputation management actually facilitates business objectives as much as it mitigates risk.

Looking at meaningful measurement across a range of integrated activities is very, very difficult. Our approach is to try and draw back slightly from the individual campaign measures, and to think more about the longer-term, ultimate goals of the communication. You’re looking at outtake or outcome measures, which might exist above the level of the campaign activity.

In terms of the journey that the discipline has been on, where the old model could reduce your company’s reputation down to a single number, we now acknowledge that there are multiple reputations. The way you try to evaluate outputs and reputation in emerging markets has to be completely different, or certainly the methodology is different. You don’t have the same level of media access or media available to track, you probably have to do a lot more primary research to really understand what’s going on.

Companies, particularly global businesses, have to recognise that they have multiple, different local reputations. Going back to the fact that evaluation has to closely tie in with business objectives, the reality is that every multi-national has a degree of bespoke commercial objectives in its local markets.

To be relevant to that local management team, you have to be able to talk to them in a way that actually reflects their local commercial reality. The pendulum is swinging back to reflect the fact that there needs to be greater respect for those local idiosyncrasies and differences you see in markets anyway.

It feels to me that Barcelona Principles are always slightly behind where current thinking is. They are perhaps useful in co-defining at an industry level the collective wisdom. But when they put the stake through the heart of AVE, it felt to me as if the industry had already definitively moved on at that point.

My gut feel is that actually the external capability and evaluation capability has moved on far quicker than internal. G4S employs more than 600,000 people and every year, the organisation goes through an extremely manual and laborious internal employee survey process, which I think is probably extremely effective but it’s massively expensive.

What it really does do is reflect that boards are now recognising that reputation is massively influenced by the values and culture of an organisation.  It’s an area in which there needs to be significant catch up.

In my experience, people in the front line of businesses are attitudinally extremely good judges of reputational issues. Exposing some of the work we do to some of those frontline people and utilising them in the process is enormously useful because they’ve got a great bullshit detector. Those are the people on the front line of sales and they know precisely very often what’s going to move the needle in one direction or another with a potential customer.

DOMINIC REDFEARN, GLOBAL BRAND AND COMMUNICATIONS DIRECTOR, DIAGEO 

How do you translate what we do as a function into something that makes sense to people on the ground? We’re blessed with the amount of data points we can now get hold of. The amount of tools available, mainly coming from the marketing world, that enable us to understand, both periodically and real-time, what is happening in our markets and on the global stage.

Five years ago or ten years ago, you didn’t have this data available to you. But how you translate that into something that is meaningful is the hardest thing to do but the most important thing to do. Understanding our reputation as a score out of ten is no use to me.

But if I were to understand what actions I could take on the ground that our stakeholders would care about, that would move them from neutral or passive into being supportive in their behaviours or even moving towards advocacy, that then impacts business value. It impacts the plans we have. It impacts the role of communicators, of public affairs, of all our teams that are impacting stakeholders in different ways.

Trying to differentiate between what works and what doesn’t work can only be done through experimentation. I don’t think anybody’s quite cracked it yet. Certainly having a mix of experiments, a mix of data sources, playing with different tools, seeing how that impacts your plan, seeing how you then course-correct your plan on the ground, is the most important thing.

Protecting the reputation of the business, as well as growing the reputation, impacting sales, creating opportunities for a better route to market, dealing with issues that equally affect business value, such as the reputation of alcohol in the marketplace or the perception of how much shared value the business is creating for the different communities it operates in... all of these things have an impact on the business value you’re trying to either grow or protect. Your appreciation of that and your ability to articulate that to the board and your management team is vital. You need to be very clear on your understanding of the business priorities and your role within those.

The world is connected, I don’t think you can tell your story in isolation, on one particular channel. The role of a comms person now is to understand paid, earned and owned media and the relationship between the three, and to be able to plan and integrate campaigns in a way that you understand the impact of each channel at any one time. If you don’t apply that integrated approach, you’re not responding to the connected way in which your stakeholders look at you.

Knowing upfront what the outcomes are that you’re driving for is really key. If you don’t know where you’re going, you don’t understand the relationship between what you’re doing on a day-to-day basis. In times of crisis, you then can’t understand how to get back on track or what a successful outcome could be.

Upfront alignment is key. It has to be aligned with the board and it has to be aligned with the strategy function and the risk function. Everybody has to understand why and what we’re doing in order to build and grow reputation.

A campaign can get picked up by content aggregators and spun, and you can end up with thousands of pieces of coverage. But what difference has that actually made? It’s about the quality of the content or the coverage that you’re getting and whether that’s in line with your plan.

You’ve got the right strategy in place, the right road map that you’re following and everybody understands that, you’ve mapped your stakeholders and the media, and you think Is that piece in that particular title, or on that particular channel helping in the progression of our plan?

When it comes to insight, it’s a very misunderstood word, but fundamentally for us, it’s Have we learned something that we wouldn’t otherwise know? Has that performance of that initiative made us think differently about what we might do next time? It’s as simple as that.

Volume doesn’t matter. Think about the titles or the channels and the stakeholders that matter to you. Understand that if you get one piece in one outlet that’s quite influential in relation to your plan and what you’re trying to achieve, then that’s worth more than 200 pieces across many unknown entities.

We run an annual values survey within the business. We get good insights from that, but you also have to run real-time as well. Quite a lot of big organisations these days have enterprise level internal comms and engagement channels. You get real-time data points from that and you can understand the relative performance of particular campaigns and the engagement levels around them. But there’s no golden thing that sorts it all out. To us, we’re always trying to understand the performance of an individual campaign down to the performance of an individual piece of content. While that is time consuming, equally it enables you to push the service providers that you’re buying these things from and say We need to demonstrate the return on our investment.

Ultimately, every organisation wants their employees to move towards advocacy. The only way to create that shift is to understand where they are now, where you’d ideally like them to be, and what’s working and what moves people along. The importance of traditional, non-digital employee engagement is still huge. How is that production line being impacted by that message in the same way as an office-based team looking at it through a digital channel? We spend a lot of time looking at it and trying to make it better.

 MICHAEL WARREN, DEPUTY DIRECTOR, GCS INSIGHT AND EVALUATION, PRIME MINISTER’S OFFICE AND CABINET 

The challenge that we’re trying to crack is that people send us a whole lot of data as opposed to a whole lot of insight. There’s a big difference between the two. Data itself is just numbers. It’s not really compared against any particular targets or benchmarks, it’s not showing a time series of incremental improvement, so what are we to make of it? It’s just a lot of big numbers.

So in that sense, Twitter impressions are definitely the way forward, because you can talk about big numbers. One example I’d give is about some data I saw about our campaign to support the Union with Scotland, where someone sent in some data saying It’s brilliant, we’ve had a Twitter reach of eight and a half million on this campaign in Scotland, without really realising that there are only four and a half million people in Scotland.

One of the big challenges we have is in creating consistency. We are trying to make sure people collect the same kind of thing so we can compare, improve the quality of the evaluation, make sure we’re collecting the right kind of things, and make sure it’s coherent and aligns to what we’re trying to achieve.

Quality, consistency and coherence are important to us. In government, it’s very difficult to have this conversation, because our objectives are not like in the corporate world. Our objective is to make life better for everyone in this country.

That means that our objectives are quite broad ranging. We came up with this acronym a couple of years ago, CORE, which means ‘Changing our behaviour, Improving operational effectiveness, Managing reputation and risk, and Explaining what the government does’.

That gives you an idea of the breadth of what we’re trying to do as government. It’s everything from trying to get people to slow down on country roads, to fill in their tax returns, to basically uphold the Union and stay as part of the United Kingdom if you live in Scotland, to stopping people going and committing terrorist acts in the Levant. It’s a big range of issues and the list of course is endless. A conversation about return on investment or value to the organisation is a very complicated one in government.

We’ve done a lot of great work in driving evaluation in a lot of areas and we have some world-class examples of how we’ve measured the effect of our communications. But equally we have to accept that not everything we do is up to that gold standard. The programme I’m responsible for is designed to bring everyone up to the level of the best.

You’ve got to be aware that not everything can be delivered at the same standard in terms of evaluation; it depends what you’re trying to achieve as an objective. It’s often difficult to get agreement on what success looks like. The journey is to take people’s perceptions of communications as an art and to move towards a perception of it as a science ultimately.

That’s a narrative that we push heavily in government. What we do is measurable, it can demonstrate a meaningful effect on objectives, and it’s worth investing in. And the best of our examples absolutely make that case.

When does it go wrong? When we have a lack of clarity about our objectives. There can be disagreement. We have a number of different competing pressures in government, so a minister’s perspective on what success looks like might be quite different from your operational director, and might be ultimately quite different from what the man or woman in the street thinks success should look like.

Getting clarity on that is quite tricky and that requires a strategic level of conversation, which goes back to getting the right sort of debate going and having the right influence within your organisation. Just being told what to measure doesn’t really help. You need to be able to influence that decision.

Have a no-fear culture. If you’re the recipient of data, then you should look to welcome bad news as much as good. Often there’s more you can learn from bad news versus good news. You’ve got to provide the right incentives.

We try to keep pressure on people to say What have you learned? What is this data telling us? Don’t just accept the data.

What we’re doing as part of our Evaluation Champions network is saying to certain people You have the responsibility for honesty, transparency and generating insight. Put your hand up and we’ll look to you for it. In classical behavioural science terms, if people make a commitment to do something, they’re more likely to follow through on it. If you can actually get nominated people to make that commitment, you’re going to find life a little bit easier.

The reality is evaluation is always going to be based on a sequence of measures. The key is understanding the logic model that sits behind your communications. You’re going to get rapid, real-time data on your outputs, but very rarely are you going to have real-time data on your outcomes. A lot of the time you’re going to have to wait.

But if your logic model tells you, and you can validate it with some qualitative research, how the system is supposed to work, so you see the connection between your output measures, your outtake measures and your outcome measures. In that sense, there’s less of a problem there.


MARCUS GAULT, MANAGING DIRECTOR (INSIGHT), PRECISE 

It’s not an easy discipline to master. What a lot of people grapple with, because there’s no one answer for everybody, is that everyone has to think dynamically about their own individual situation and what they need to measure because of that.

Over the years, I’ve been working in evaluation for eight or nine years now, I’ve seen progress. We’ve started to move away from the horror of AVEs and that sort of thing and we’re starting to get much better as an industry at measuring the quality of the coverage we’re generating and the output.

Where we’re still falling very short as an industry is then making the leap to how we’re impacting our real objectives, which might be an attitudinal change in your audience or a behavioural change, and bringing in the actual end of objective back to the communications practice and making that link.

That’s hard. In order to get credibility at board level, we have to move as an industry from saying We’ve gained some great coverage and achieved all our output objectives in the media to We are influencing the way this business is performing. We are influencing the way this business is perceived in the minds of the board.

It’s different for everybody which makes it complex. There are different varieties of noise that you have to try and navigate through to isolate the impact of communications. It’s not easy getting to that, but we need to focus on trying to get there, because that’s the only way we will get real credibility at the most senior level of organisations.

Certainly, people are trying to understand, in terms of online, print and social, which channels are working better and which areas could have some improvement. One of the challenges is beyond our world. We don’t as a communications and PR function exist in isolation.

There are other communication channels which belong in different departments. Not many organisations are looking across the piece. What role is external communications having relative to advertising or other activities through which companies try to influence their audience?

You have to collaborate with other departments. You have to get a whole range of additional data together and really look at it. Sometimes I feel that there’s not the will to coordinate. Marketing is protecting its advertising budget. Why would it want to showcase that external communications is doing a fantastic job and is a much more effective communications channel? That would result in money swinging away from bought media to earned media.

So there are some structural challenges within organisations to get to that Holy Grail of I’ve got £5 million, I’ve got 15 channels that I can put that in, how to I divide that up? What role does each channel play and how do I most efficiently spend that money? We’re doing it within PR but there’s a greater challenge across organisations. Reputation in many organisations is across many functions. It’s about customers, it’s about public perceptions, financial analysts and so on.

There’s a whole hotchpotch of different people that matter to your organisation. One of the interesting things we’re seeing is that some corporate comms directors are keen to broaden their own informal remit beyond the media. They’re bringing in data from the marketing team, from the customer service team, to present reputation to the board and take ownership across all departments.

It requires a lot of co-operation internally from a data sharing point of view, but we are seeing some particularly ambitious corporate affairs people using reputation as a platform to take their remit beyond just the media.