CorpComms Magazine

George Bernard Shaw once complained that the only person who treated him sensibly was his tailor. 'He took my measurement anew every time he saw me,' observed the writer in a 1925 essay, 'while all the rest went on with their old measurements and expected them to fit me.'

Some 84 years on, some corporate communicators are making the same complaint about media evaluation firms tasked with the mission of accurately measuring the value of public relations campaigns.

'I've met with numerous evaluation companies and tend to get the same thing in a different package or at a different price,' bemoaned Reuben Aitchison, head of public relations at risk and insurance group Aon, in last month's CorpComms Magazine. 'I'm left feeling that I need something more.'

It's a debate that has long been aired in corporate communications circles, where some practitioners yearn for a standard metric to evaluate media campaigns - something they claim is currently impossible because the methods of measurement vary so widely.

Take the Central Office of Information. The government agency has been trying for some time to encourage core mandatory standards for public relations evaluation and recently tested whether any progress had been made by sending an identical brief consisting of 138 items of coverage to five separate agencies for evaluation.

It asked how many people consumed the coverage, what the cost was per 1,000 people, whether it was favourable and what its tone was.

The answers ranged so enormously that director of news and public relations Neil Martinson wondered how media evaluation firms could generate such varying results on one campaign.

'Within each company's response the logic was impeccable,' he says, 'but each measured in different ways from different sources and used different descriptive terms.'

The COI's need for evaluation is particularly keen because, unlike commercial sales campaigns where there is a monetary result to be totted up, changes in attitudes and behaviours are its main measure of a campaign's success or failure.

Marcus Gault, managing director of Insight, the evaluation arm of media intelligence agency Precise Media, also points out that the COI uses a number of agencies to measure the effectiveness of its various and wide-ranging public information campaigns, while corporate clients normally only use one evaluation firm.

However, it is certainly true that measurement groups do tend to use different methodologies, whether it is tonal analysis of media coverage and media penetration, surveys to assess attitudes and perceptions or maps of the key voices in debates and what influence they had over journalists and publications.

Linguistic challenge

Invariably, they feature metrics to measure costs, coverage, changes in awareness and opinion and outcomes, such as shifts in attitudes and increased sales.

The problem, from an in-house corporate communicator's point of view is that as well as all spouting different measurement philosophies and strong opinions about what is and is not an accurate metric, the evaluation agencies also all have their own language.

Some phrases and methods, such as 'advertising value equivalent' - the amount of money that would have had to have been spent to generate the editorial coverage generated by a media campaign - or 'opportunity to see' - the number of people that a campaign is put before - have become mainstream. Then there are page views and other conventional metrics of the Internet age.

However, other measurement systems remain strictly proprietary to the evaluation firms that invented them and there is little incentive for them to be shared.

Gault says that because the firms have such different approaches, clients should bear in mind their objectives and select those agencies with the most appropriate proprietary methodology for the task in hand. If they do that, he adds, a standardised methodology is not necessary.

'Each firm will have a different set of proprietary measurement tools centred around their own methodology,' he says.

'So it should be quite an important part of the decision that a client makes when choosing a measurement agency. If I was a client, I would want to look at what lay behind the methodology of each firm.

'But the important thing for clients is that they look at the results in the context of how their competitors are doing and what went before.

'As long as you have a consistent methodology over a period of time, that should be able to tell you how you are doing in relation to where you were before and how your competitors are doing.'

Anne Curtis, managing director of media analysis and PR evaluation provider Media Measurement, agrees: 'We think that measurement is an integrated part of the management cycle and has to be linked into business outcomes and return on investment,' she says.

Being able to do that, however, assumes that corporate communication departments do indeed know where they are at the start of a campaign and what they want to achieve.

Benchmarking from the start

Richard Bagnall, managing director of communications measurement agency Metrica, says that this is often not the case and that departments can exhibit a marked reluctance to benchmark at the beginning.

'In a perfect world, people would do prior market research to see where they exactly stand before a campaign and they measure it again afterwards,' he says. 'People say it's too expensive to do that but it doesn't have to be.

'The key is that companies need to set specific objectives and they need to be measurable objectives. Saying that you want to increase general awareness is not enough. You need to be able to say Awareness is at this level. We want to move it to this next level. You have to know at the beginning what it is that you want to measure.

'In 15 years of running Metrica, I continue to be amazed by the number of organisations that don't sit down at the beginning and do this. Everyone wants a silver bullet but there's not a silver bullet for everyone because the objectives will not all be the same.'

Difficult challenge

That will not stop some corporate communicators wanting a common measurement system but Sandra McLeod, chief executive of reputation and communications research group Echo Research, believes that is an impossibility because of the differences between what companies hire agencies to evaluate.

'People are always looking for a magic bullet, so they can do their jobs and you can sort it out,' she agrees. 'But public relations delivers itself in many ways and there are many ways of measuring its projects and campaigns.'

What a PR campaign is trying to do is to change behaviour and attitudes, she argues, but these can express themselves in different ways and therefore need measuring differently.

'You can have very different projects with different objectives and different timelines,' she adds. 'A long-term attitudinal change programme on something like climate change will probably need a different measurement than a campaign that aims to produce an increase in sales.'

The best that could be achieved, she says is probably to develop a series of standard measurement metrics for a small number of types of campaign, so there would be one for long-term behaviour change programmes, another for short-term sales drives and others for other purposes.

'With some campaigns, you're going to get quick answers and others are more long-term,' says McLeod. 'I can see that some firms would love something like this but it is over-simplistic to have one system of evaluation that measures all campaigns. You just can't do it because they're all different.'

Compromise is possible

Jerry Ward, managing director of integrated monitoring and evaluation agency Press Data, feels that a middle way can be found.

He believes that the issue can be broken down into the difference between quantitative and qualitative measurement.

In-house PR often need numeric figures, he says, usually as a financial value that can be understood easily within an organisation, as a measure of performance.

However, very often this measures outputs rather than outcomes. 'To a certain extent in this area of evaluation there is a greater level of convergence over the past few years,' says Ward.

But he adds: 'Whilst there can be standardisation on aspects of methodology, such as the use of audience and readership data, it is important that campaigns in particular are measured against their own clearly defined objectives for success. What may be possible to achieve is a more consistent use of evaluation tools that most suit individual programmes.'

Whether such a view stops the standardisation bandwagon is another matter. It's certainly too late for Bernard Shaw's would-be tailors to learn the lesson.

share me: del.icio.us | digg | reddit | Tweet