Radio InSights

Voltair vs Nielsen Smack-Down at Radio Show: Who Won?

It was to be radio’s Hulk Hogan versus The Rock, a shoot-out at OK corral. Dueling presentations: Voltair versus Nielsen’s PPM.

There were high expectations that sparks would fly maybe punches thrown as 25-Seven’s Geoff Steadman made the case for Voltair while Nielsen’s Matt O’Grady and Arun Ramaswamy defended PPM.

With so much anticipation and the future of radio measurement in the balance you’d think that the NAB would provide the best possible venue to host these two important events.

Think again.

The NAB chose the “Innovation Stage” for both presentations. It is a clever name for what appeared to be unsold booth space in the noisy cramped exhibit area.

It was an environment more appropriate for a Vegomatic demonstration than for two presentations that could impact the way an entire industry is measured.

But whatever.

Did I mention that the NAB decided to lay out a buffet lunch in the area surrounding the Innovation Stage?

Don’t believe it? Look closely at the top picture!

So with a backdrop of sliced roast beef under heat lamps and the sound of popping soft-drinks the face-off began.

But a funny thing happened.

Instead of a smack-down, we witnessed a pair of carefully choreographed ballets with neither dancer directly addressing the other.

Steadman defended Voltair as a useful tool that broadcasters could use to help make sure that their radio stations got full listening credit (from you-know-who).

And Nielsen’s O’Grady, et. al. formally rolled out PPM encoding “enhancements.”

Their even tone and measured words made it seem that these “enhancements” were long planned developments to address some sort of well understood issue that might possibly negatively impact the ratings of a very small handful of outlier stations.

Listening to the Nielsen presentation you’d think no one there had ever heard of Voltair.

So who won? It's too early to tell.

Think of this as only the first round. On points, the first round goes to Nielsen, but there will be many more rounds of what will be an epic battle--not for the two opponents, but for radio.

Nielsen is to be commended for taking the opportunity to share with all broadcasters information that the company had previously shared with only a limited number of clients and friends of the company.

The presentation included a great deal of important information including results of the Baltimore/Washington enhancement test.

It hasn’t gotten the attention it deserves so we’ll be reviewing the test in the next post.

Stay tuned.

Is PPM Killing HD Radio?

Nielsen has now confirmed that PPM under-counts some formats.

Unfortunately, things are even worse than Nielsen admits.

As bad as PPM is in accurately measuring some formats on traditional (analog) AM and FM stations, HD (digital) stations have virtually no chance of getting their full credit from PPM regardless of format.

For reasons we’ll explain, an HD station playing identical content to a traditional analog station is unlikely to encode as well as the analog station.

Consequently, with identical audiences the HD station will have lower ratings.

PPM relies on the presence of masking content to insert inaudible identifying codes in a station’s programming. The greater the level of masking content, the “louder” the codes. The more often programming contains masking content, the more codes broadcast.

Some formats broadcast more masking content than others, so those formats broadcast codes more consistently than other formats. More consistent encoding means better ratings.

On the other hand, if a station broadcasts insufficient masking content, the encoder can’t insert codes and the station is essentially off the air as far as PPM is concerned.

Digital radio such as HD radio adds an additional wrinkle to this process.

To enable HD to deliver multiple digital streams within the bandwidth of an AM or FM channel, the digital channels are highly compressed using a proprietary algorithm that removes a great deal of content.

In this regard, HD radio is like a low quality MP3 copy of a song.

If you want to better understand what this means and you have access to a Voltair, replace the CD version of a song with a low quality MP3 version and watch your green encoding bars turn to red.

Were it not for PPM, the quality degradation that comes with the HD radio compression algorithm wouldn't be a ratings problem for HD Radio.

However, the process of compression eliminates large amounts of masking content, the very content PPM needs to consistently encode the broadcast.

HD radio turns highly encodable analog content into unencodable digital content.

Granted HD Radio has issues that go well beyond algorithms, but the medium shouldn't be further punished because of Nielsen's inability to accurately measure its audience.

Nielsen is currently testing enhancements to the PPM encoder to address encoding flaws that Voltair revealed.

Nielsen hasn’t announced any test details, but we’re skeptical that Nielsen will be testing the encoding reliability of HD-2 and HD-3 channels.

And we really doubt that results will be released publicly any way.

In a follow-up post we’ll get into the messy details of HD radio’s HDC compression algorithm (codec for shot) and explain the problem in greater detail.

 

Radio Continues Unprecedented Growth

Last month we highlighted a significant turning point for radio.

After years (some say a decade) of deteriorating listening, local radio turned a corner in January.

For the first time since PPM was implemented radio put together six consecutive months of year-over-year listening growth.

In August radio managed to continue that trend, notching another month of growth.

Radio listening, as measured by average quarter-hour ratings (AQH), swung from a loss of 4.3% in 2014 to a gain of 3.0% last month, a swing of over 7%.
Equally significant was the proportion of markets that gained listening. Nearly three-quarters (72%) of PPM markets increased AQH compared to 2014.

Considering that only four markets managed to expand listening the year before, that is quite a turnaround in one year.

Oddly, this turnaround has gotten scant attention. Why aren't we hearing more about this?

Perhaps after years of decline radio’s leaders don’t believe the numbers. They assume that this reversal of fortune is temporary, that radio’s fate is to continue to lose audience.

Or perhaps radio’s leaders believe the trend but don’t want to acknowledge the likelihood that Voltair is behind the turnaround.

Admitting that after years of decline Voltair generated unprecedented growth in a matter of months would be an admission that PPM is flawed.

That would be awkward for many reasons.

 

Nielsen: Let's Kill the Quarter-Hour

Little attention has been given to one important comment made by Arun Ramaswamy during the July 21st Nielsen PPM presentation. Here’s what he said:

The final point I’ll make is that the Voltair was more pronounced at the minute level than just the quarter-hour level....(Because of complimentary crediting and the smoothing system) we found that even though the Voltair product was more visible at the minute level, it wasn’t always translated at the quarter-hour level.

So even with Voltair, radio stations fail to get credit for fragmentary listening because of a quarter-hour tradition dating back to the 1940s.

In the 1940s and 1950s, radio programs were 15 minutes or longer in length. It made sense for radio ratings to be measured in 15 minute units, so the quarter-hour metric was created.

Most everything in radio has since changed, but not the quarter-hour.

The quarter-hour metric is an anachronistic throw-back to a era that radio has long passed.

We need to kill the quarter-hour. Here’s why.

Voltair enhances PPM codes so that decoders are more successful at identifying the station a panelist is listening to.

It takes at least five successfully identified minutes of listening for a station to get credit for a quarter-hour. If Voltair helps you get only four minutes credit in a quarter-hour, you don’t get a quarter-hour.

You get zero credit, despite the fact that the panelist was listening to you.

Nielsen argues that the editing rules help radio because you can get 15 minutes of credit even if the meter only successfully identifies five.

But they ignore the flip side.

A station gets no credit if the meter identifies even longer listening spans when it bridges two quarter-hours.

Say a panelist listens for five minutes from 5:00 to 5:05 and the meter successfully credits the station for all five minutes.

The station gets credit for one quarter-hour.

What if the panelist listens for eight minutes from 4:56 to 5:03?

The station gets no credit for any listening–-even if the meter credits the station for all eight minutes.

In diary markets this is hardly an issue. People are not that precise when they write down their listening, generally rounding to the quarter-hour.

PPM is different.

Because PPM measures exposure and the meters take time to identify a radio station (if at all), exposure spans are brief with gaps as the meter struggles to identify the station.

While a diary keeper is unlikely to write down 4:56-5:03, it is quite likely that meters will detect listening over odd periods of time like this.

That’s why Voltair has a bigger impact for Canadian stations than US stations. Canadian radio is measured in minutes rather than quarter-hours.

So Canadian stations get credited with listening that US stations do not.

Nielsen claims that PPM propelled radio audience measurement into the 21st century. Isn’t time to abandon the quarter-hour, an anachronistic vestige of the first half of the 20th?

 

Will Nielsen Ban Voltair?

Last month’s Nielsen presentation announced a number of enhancements to PPM:

So Nielsen continuously updates and enhances it technology and we have been making working on this for more than a year, developing enhancements that I’m pleased to know that we will be rolling out starting Q4 of this year.

Then just 18 days later Nielsen announced that a field-test of the enhancements in Washington, D.C. and Baltimore would begin this month. Word has it that stations hadn’t even agreed to participate when Nielsen made the field-test announcement.

First, no one we’ve spoken to can recall Nielsen announcing any plans to update PPM encoders prior to the company’s July 21st presentation. We couldn’t find any press releases, Nielsen reps hadn’t mentioned it, nothing.

There isn’t any kind of paper trail leading up to the 21st announcement or even a hint that changes to the encoder were in the works.

This all has the feeling of something that was quickly thrown together in response to Voltair, not the latest steps in a well developed long term plan to continually improve the reliability of PPM encoding/decoding.

How can Nielsen announce enhancements prior to field-testing them?

In any case, here are the two key enhancements Nielsen announced:

1. An increase in code density to improve code pick-up in challenging environments.
2. The ability to increase code amplification adding a third level of gain to the code.

It is clear from the descriptions of the two enhancements that Nielsen reverse-engineered Voltair or at least studied it closely enough to determine how it enhances encoding.

The enhancements are simply what Voltair does.

However, Mr. Ramaswamy went one step further.

He assured broadcasters that the algorithm would be tweaked so that the codes would be hidden even with formats with “a lot of more silence” in them. He also declared that the changes will not cause any audio artifacts or compromise user quality.

If you recall from the presentation, Mr. Ramaswamy was critical of Voltair because he claimed that Nielsen’s “golden ears” could hear Voltair at work.

All this raises a number of interesting questions.

If Nielsen is going to integrate Voltair technology into PPM encoders, how can it also claim that PPM is already nearly perfect, and that Voltair enhances encoding only by making inaudible codes audible?

Nielsen claims that Voltair extends the code tones and compromises the watermark. It also amplifies the codes and introduces “audible artifacts like static, popping, and echoes.”

But Nielsen is going to extend the code tones. It is going to “amplify” the codes.

How is it that Nielsen won’t also be making the tones audible and introduce audible artifacts?

And if the tones turn out to be audible in the field test, what will Nielsen do?

Maybe Nielsen will declare the field-test a failure–that PPM’s reliability in crediting listening cannot be improved without introducing unacceptable artifacts.

Then the company will give radio a choice: Accept that PPM misses some listening, or accept audible codes.

And perhaps Nielsen will take the matter one step further. It will finally declare Voltair unacceptable and ban it because it creates the same problems.

Maybe that is the end game. Go through the motions of a field-test, have the field test fail, and get rid of Voltair.

In any case, we expect that this test will be done just like everything else has been done regarding PPM.

The field-study will be secret, detailed results will only be released to a handful of people who sign lengthy non-disclosures agreements, and most of us will never have the information to draw our own conclusions.

We can’t wait.

 

Shocker! Radio is Growing!

Does Voltair Work?

It’s an important question that’s not easy to answer.

Early on with few stations using the box it was easy to tell which stations were using Voltair. They were the stations spiking in the ratings.

Now with penetration increasing, we have Voltair stations competing against other Voltair stations.

If one one station installs a Voltair and spikes, then other stations lose share, even other Voltair stations. That’s because share is a zero sum game.

And if every station in a market has Voltair?

In that case we reach some sort of new equilibrium when finally shares are based on each station’s popularity, not the encodability of their formats.

So if every station has a Voltair, how do we now figure out whether Voltair works?

The answer is to look at over-all radio listening. That’s where Average Quarter-Hours (AQH) come in.

AQH is a measure of the average number of people listening to radio.

If every station finally gets credit for all the listening it generates, not just the part that PPM can decode, then share could remain the same while AQH increases.

So the key to understanding Voltair’s impact on radio is to look at AQH.

The transition to PPM wiped out about 30% of radio listening. We were told at the time that PPM is more accurate than diaries, and that people never really listened to radio as much as they said they did.

Since the roll-out of PPM time spent with radio as measured by PPM has continued to erode each year, something around 5% a year.

We have argued that methodological flaws in PPM and faulty measurement could very well be behind much if not all of the decline.

The problem was that without an alternative to PPM it was difficult to prove.

We speculated very early on that Voltair would increase over-all listening as more and more stations got full credit for their listening.

Now Voltair has provided us the closest thing to a smoking gun confirming that PPM is undercounting listening.

We examined AQH ratings across all non-embedded PPM markets comparing 2015 to 2014, and then comparing that period to the previous year.

The two graphs show what we found.

 

The first graph shows year-over-year changes in AQH ratings with each month indexed to the same month the previous year.

An index of less than 100 means that radio’s audience declined. An index over 100 means that radio’s listening increased–as measured by Nielsen.

The first thing to note is that the index was well below 100 for all of 2014. That means listenership across PPM markets was lower in 2014 than 2013.

And if you go back further, you’ll find the same pattern, a decline every year since PPM rolled out.

Look what happened beginning with February of this year. The index moved steeply upwards.

Since then AQH has increased every single month, even cracking 100 in March.

That means for the first time PPM is showing listenership growth. That is unheard of.

The next graph shows the growth differently. This graph shows the proportion of markets that saw their AQH grow year-over-year.

2014 was typical of previous years. Each year a handful of markets would gain AQH, perhaps three or four, while the majority of markets would lose AQH.

Since the appearance of Voltair we’ve seen the number of markets growing AQH tremendously increase. In July 2015 ten times as many markets grew compared to July 2014.

Maybe there is some other explanation for this unprecedented growth, but the growth coincides with the introduction of Voltair.

Isn’t it likely that Voltair stopped radio’s decline?

Voltair critics will claim that an index of 103 suggesting 3% growth is anemic, and nothing to shout about. However, we have to put the recent rate of growth in the context of over a decade of decline.

PPM declines in 2014 were approaching 5% before Voltair’s appearance. So far this year the swing is 8%, and that is something to shout about.

We predicted that if Voltair worked it would float all boats, that it would help the industry push back on the notion that radio is dying. It has done exactly that.

Still think Voltair is a bad thing for radio?

The Twisted Logic of PPM's Defenders

It was inevitable that the Voltair discussion would descend into the rabbit hole to join the Mad Hatter and the Queen of Hearts.

Some in the industry really want to believe that PPM is as good as Arbitron claimed when it pitched the service to radio.

If Voltair really does increase ratings, it challenges the accuracy of PPM, and some people just can’t accept that.

It has led to dubious claims based on shaky statistics, misguided assertions, and baseless conjecture.

The latest example is a series of anti-Voltair screeds penned by consultant Randy Kabrich.

Randy makes his motives clear at the very beginning:

I am doing this because Voltair has caused tremendous turmoil in the industry that is now spilling over to questions about the currency.

I want this controversy ended as soon as possible and the least amount of damage done.

Damage to whom, you might ask. Apparently he is speaking about damage to Nielsen.

Were it not for Voltair we would not be discussing PPM flaws.

Were it not for Voltair Nielsen might not see any need to update their encoders or tweak their algorithms to fix the flaws.

The flaws of PPM have destroyed formats, driven talented personalities out of the business, distorted what songs we play, and punished radio stations for no reason other than the format they broadcast.

PPM has done irreparable harm to local radio, yet Randy only fears that Voltair may sully Nielsen’s reputation.

And he seemingly is willing to mock, mislead, and distort the facts to protect PPM and Nielsen’s reputation.

Most difficult to understand is how we reconcile this rabid defense of PPM with the admissions of Nielsen.

In their presentation Nielsen admitted that Voltair boosts ratings for “outlier” formats. That means some formats encode better than others.

Nielsen has agreed that encoders need updating and they are rolling out a fix this year.

Why accuse those who claim Voltair works as delusional when Nielsen itself admits that some formats benefit from using Voltair?

At first we were going to ignore his transparent attempt to help Nielsen save face, but his assertions have gained some traction within the industry and clients have asked us to comment on his claims.

Rather than post a lengthy critique here, the rest of this post will be at the link below. If you choose to drop down into the rabbit hole, be our guest.

 

Nielsen's Eric Cartman Moment: Respect My Authority!

An analysis of Nielsen’s Voltair Webinar wouldn’t be complete without noting the tone and rhetorical tricks that Nielsen employed to bury the admission that Nielsen needs to fix PPM.

The presentation was an artful display of rhetorical gymnastics.

In his opening remarks Matt O’Grady managed to invoke the word “currency” four times in the span of two sentences, seven times in the span of five sentences.

Do you think Mr. O’Grady was trying to make a point?

Here’s what he said:

As a currency provider it is our obligation to provide fair, unbiased, and accurate measurement. On the flip side, we also expect the industry to respect the currency, play by the rules, and ensure that all benefits from the use of a common currency.

(That last line probably read a little differently in the script from which he read, but that’s what he said.)

The word “currency” is an interesting word to use in regard to audience estimates subject to error.

Why is PPM a currency, let alone the currency?

The term was first used to sell PPM to radio. Arbitron needed to convince radio that PPM was worth a lot more than diary-based estimates, so the company touted PPM as the new ratings currency.

It worked back then so why not use it again.

Linking PPM to “currency” enabled Mr. O’Grady to imply that all this talk about inadequate encoding, missed listening, and Voltair was debasing the currency.

Voltair is to PPM what Hugo Chavez was to the Venezuelan peso, perhaps.

He was cleverly implying that stations that use Voltair create doubts about the value of Nielsen’s the currency, and we don’t want that.

Mr. O’Grady employed a common rhetorical trick, invoking authority to essentially say, “we know what’s best for PPM and radio.”

It was a moment straight out of South Park: “You will respect my authority!”

That set up the next statement:

Nielsen stands by the quality of service that we provide and it’s also our responsibility as I said earlier to continue to provide ongoing and continuous enhancements to meet the changing needs of the marketplace.

So the message is that Nielsen is the authority, the expert, the only one to determine if, how, and when PPM problems should be fixed.

Any interference in this process is disruptive and potentially damaging.

The problem is that when Jennifer Hutson took her turn at the mic the biggest improvement that she could cite was the meter's longer battery life!

Apple fixed the iPhone battery life problem in a single year and it took Arbitron/Nielsen eight years?

Nielsen isn’t Apple. It’s more like Ma Bell before the 1984 break-up.

Come to think about it, PPM is from the 1980s.

More comments to follow.

 

Nielsen's Non-Statement on PPM and Voltair

We begin our analysis of Nielsen’s highly anticipated Webinar by looking at Nielsen’s conclusions regarding Voltair’s impact on PPM.

Arun Ramaswamy, Nielsen Audio’s chief engineer, reviewed the company’s research on Voltair and then said this:

So here are the conclusions. First Nielsen reiterates its non-support for Voltair. The reasons are as follows:

• Voltair can impact the audio quality for listeners.
• It alters the Nielsen encoding as I listed out.
• Third, it can actually result in crediting in cases where sometimes it’s unintelligible listening.
• It introduces variability through the station-controlled settings, you know there’s a dial that can be changed.
• And then finally, it is not uniformly universally implemented which could result in an unequal market dynamic.

So for these particular reasons, we cannot support Voltair.

It is a curiously worded conclusion.

Nielsen has leveled some pretty serious charges suggesting that Voltair unfairly manipulates PPM codes and the strongest step it is willing to take is to express “non-support”?

What does “non-support” mean? And if Nielsen instead supported Voltair, how would things be different?

If Voltair really does all the bad things that Nielsen says it does, why didn’t Nielsen just ban the box like the Canadians did?

Perhaps Nielsen’s evidence isn’t as strong as it is portrayed in the two data tables presented.

Or perhaps Nielsen is afraid that 25-Seven, the manufacturer of Voltair, might sue if it took stronger steps.

Or perhaps Voltair really has revealed flaws in PPM and pressing the matter by going further than expressing “non-support” for Voltair might invite greater scrutiny.

The announcement that followed Mr. Ramaswamy's conclusions gives us a hint of why “non-support” was the strongest response Nielsen was willing to make.

We’ll explain why in our next installment.

Nielsen to Radio: PPM is perfect. Now We’ll Fix It.

Totalitarian regimes use torturous language when admitting problems and failures. They hope that by couching confessions within reassurances they can hide the truth in plain sight.

We were reminded of this as we listened to the recent Nielsen PPM Webinar.

Officially the presentation was exclusively for Nielsen Audio clients and invited guests, namely the NAB, RAB, and MRC, but we suspect we weren’t the only gate crasher at this long anticipated extravaganza.

We’ve held off offering a detailed critique of what Nielsen had to say because Nielsen's Matt O’Grady made assurances during the presentation that it would be made available to all clients. We’re still waiting.

How long does it take to post a ten page presentation?

Maybe somebody within Nielsen decided there was too much honesty. Maybe the implications of the promises and assurances were a little too transparent.

Whatever the reason for the delay, we’re going to go ahead and examine the presentation, its claims and promises and translate all 4,400 words into plain English.

Here’s the bottom line:

Nielsen claims PPM works like it is supposed to, but it is going to make significant changes to the encoding process to fix the problems that Nielsen claims do not exist.

Nielsen also claims that extensive testing has shown that Voltair doesn’t really increase ratings, but if it does, it might do so by creating spurious codes while degrading the listening experience for panelists.

(If you’re one of the hundreds of radio stations successfully using Voltair all this may be contrary to your own experience, but Nielsen says they’ve proved it.)

In the coming days we will parse and decode the claims of Nielsen. You won’t want to miss it.

Voltair From A Client's Perspective

Today we are turning our blog over to our friend and client Dave Paulus from Max Media, who we believe has summarized this better than anything we could have written.

 Well, as Jimmy Buffett would say, “It’s been quite a summer”. 

When I became so vocal in the “Voltair debate” a few months ago, my only agenda was to use this topic to generate a healthy debate regarding the entire encoding and measurement system process for radio.  

My agenda was never to critique the validity of Voltair as a product.  But, as an industry, we haven’t asked the important questions about the measurement process be it diary or PPM and the Voltair discussion provided a great basis to do just that. 

What I do think I’ve been able to provide throughout this process is a clear perspective from someone who lives with these consequences day in and day out.  That’s why I felt like it was important to write this note. Imagine if you’re sitting in the quagmire of a condensed radio market, with basically the entire marketplace within a few 10^th’s of a ratings point of one another and you see this:

 

You see a station that you feel has not been reported accurately for years and basically all you see is red. Literally and figuratively.

And then you start asking yourself a lot of questions:   Does the guy across the street have this already?  Is it real?  Is it a placebo?   Is this the reason why my numbers suck?  Has Nielsen been scamming us all along?  Have they not been scamming us all along?  Should the industry rethink the entire methodology, TV and radio?   How do these ratings points stack up against TV set top boxes?  Do I really trust a Nielsen radio report that takes 760 pages to explain how a PPM monthly measurement is formulated in Norfolk, VA? 

So, I say, "Let’s ask the source"!  Until last Tuesday the 21^st, we heard nothing but crickets.

I am thrilled and downright giddy that this debate has gotten to this level.  It’s long overdue and very healthy for our industry to openly question and discuss all aspects of the measurement system afforded to us, even when we disagree!  It’s also my feeling that unless the major companies like iHeart, Cumulus, Cox, Entercom, Emmis, Radio One and the like, hold our ratings partner more accountable for their actions and their commitments, we’re really just a bunch of neighbors talking over the hedge.  

To those companies who have the power, I urge you to leverage your weight to ensure that the system works to betterment of our industry.  Otherwise, I promise you that we’ll all be selling 0.7’s forever.

As a reminder:

We are the client here

So, when my kids start asking me at the dinner table “What’s going on with Voltair today?” instead of “How was your day at work?” it’s become obvious I’ve gotten a little “too engaged” on this topic. So, I’m handing the baton over to the aforementioned companies and our radio trades/bloggers that hopefully will take the leadership role.  It’s certainly my desire that this intense discussion continues to hold our measurement system accountable.  While there is no question that there are other pressing issues for our industry, there is nothing that will help the larger markets more quickly than making sure all appropriate encoding is being accurately measured by PPM.    

“Want to survive in business?  Innovate, demand accountability and trust, but always verify”-  Steve Jobs, 2007.

Thank you for your thoughts, opinions and passion on this incredibly important topic for our industry.

Will Nielsen Open Door to its Sausage Making?

As we explained in our last post, editing rules can have a significant impact on ratings.

Computers using algorithms sift through panelist listening logs and make material decisions on what listening will and will not be included in the final numbers.

Nielsen as with Arbitron before it has never allowed broadcasters to see PPM panelist level data.

With diaries there is a physical record of what a listener wrote down, as well as notes on any edits that were done in preparing the diary for data entry.

A Nielsen client can review the diaries to see if the editing rules are being applied consistently and according to official procedures.

There is no equivalent with PPM.

Panelist logs are directly uploaded to Nielsen computers where upon the editing rules are applied and the numbers generated.

Nothing physical or reviewable exists within the PPM ecosystem. The whole process is a black box.

Which raises the question: How do we know that PPM editing is being applied consistently and according to official procedures?

The failures of software and algorithms are well documented. A day does not go by without some story of software errors creating havoc.

This author has reviewed thousands of completed diaries and found them to be full of editing errors, most small but some significant.

Can we believe that the man-made algorithms assigning credit to stations are any more accurate than the people crediting radio stations based on edits to Nielsen paper diaries?

The voltair controversy has proved that PPM is not flawless. Far from it.

PPM has flaws causing it to under-report listening. As a result, PPM distorts ratings and rankers.

Editing rules cannot fix a technology that has been shown to create winners and losers based on format and programming.

Nielsen didn’t create PPM, they inherited it.

But now PPM is a Nielsen product. The company has to own up to the fact that PPM is flawed and in need of a serious fix.

Nielsen also needs to create more transparency so that broadcasters have the ability to watch over the entire PPM process.

How else can Nielsen regain the confidence of broadcasters so they believe that ratings are as accurate as possible.

Enabling Nielsen clients to examine panelist level data would be a good place to start.

We just hope pulling back the curtain isn’t like the proverbial visit to a sausage making factory.

PPM Edits: Adding Insult to Injury.

It would be great if PPM meters captured all radio consumption.

But they don’t.

The meters miss listening. And worse, the amount of listening the meters miss depends on formats, programming, and the environment in which a person listens.

PPM’s 1980s technology simply can’t capture 100% of listening 100% of the time. So PPM meters end up with gaps, logged periods of unidentifiable listening.

Arbitron knew this even before PPM launched, so in an effort to fill in the unidentifiable listening--to patch the problems, the company created editing rules.

Computers review meter logs searching for gaps in listening. Algorithms then use rules to credit the gaps to radio stations.

The problem is that algorithms can’t know for sure what a person was actually listening to. It can only guess based on the information it has, and given PPM’s flaws the information can be very misleading.

Because PPM favors some formats over others the algorithms can end up just making things worse.

The computers give some formats even more quarter-hours than they deserve while adding insult to injury by punishing other formats even more.

Let’s say within a quarter-hour a panelist listens to Station A for four minutes, switches to station B listening seven minutes, and then switches to Station C for four minutes. (See the first illustration below.)

Because it takes at least five minutes of station listening to qualify for a quarter-hour, only Station B should get credit for the quarter-hour.

That’s if the meter accurately logs the listening to all three stations.

But what if Stations A and C are music stations that the meter can easily identify but Station B is difficult for a meter to identify, say maybe a morning show or New/Talk station?

The editing rules that Nielsen applies in this situation could end up awarding all the minutes to Stations A and C, and Station B ends up with nothing!

There are two kinds of gaps.

There are gaps where the meter kind-of knows which station it might be but isn’t sure, and there are gaps where the meter only knows some radio station is playing.

The first is called a one-off incomplete edit while the second is called a medium edit.

If your head isn’t already spinning and you really want to know even more about editing, the Nielsen people have put together a lovely nine page glossy booklet explaining all this entitled From Codes to Credit: Data editing in our radio service.

It is just as interesting as the title suggests. Be sure to ask your Nielsen rep for a copy today (because we can't link to it).

But if you just want to know how Station B got screwed out of a quarter-hour, read on.

The computer looks on either side of the gap to see what the panelist was listening to before and after the gap. In this case it was Station A before, and Station C after the gap.

Since it never identifies Station B in our scenario, the computer splits the gap between Stations A and C.

The additional minutes qualify both stations for a quarter-hour they don’t deserve, and the most listened to station that quarter-hour gets nothing. (See the second illustration.)

The editing rules look reasonable and fair in the overly simplified brochure Nielsen has produced, but in the real world the algorithm can arbitrarily divvy up quarter-hours.

If anything, the rules exacerbate the rating differences between formats that encode well and those that don’t.

Stations that encode well will be given a higher proportion of the gaps because they are more likely to be accurately identified within the editing windows.

The stations that don’t encode could very well represent a higher proportion of the unidentified listening for the very fact that they don’t encode well. However, they will be given a smaller proportion of the credit.

There’s just no way for editing to patch the gaps and make sure every station gets all the credit it deserves. Only 99% decoding accuracy can do that.

iHeart Stations Through the Roof. Is it Voltair?

No group has embraced Voltair as much as iHeartMedia.

Rumor has it that the company has installed hundreds at its stations and continues to buy more as they become available.

If you compete against iHeart in a PPM market, you are probably competing with Voltair.

Because so many of the group’s stations are using Voltair, the company’s recent Nielsen performance serves as a useful metric to measure the impact of the box.

We’ll get into the details in a moment, but here’s the bottom line:

This year iHeart has been able to increase the number of stations gaining share by 50%.

In May 2014 only 42% of iHeart stations were able to better their 2013 May numbers. The percentage of improving stations reached 63% this May. That’s an increase of 50%.

Only a third of iHeart stations declined this year compared to 53% last year.

We’ve been doing these analyses for more than ten years, first with diaries and now PPM.

We’ve written often about how most months the ups and downs hover around 50%.

In any given month you can expect half of stations will go up. The other half will go down.

For a large group to increase share at two-thirds of their stations over the same time period is unheard of.

The over-all performance is pretty amazing, but the details are even more striking.

Look at the graph above. It shows the distribution of gains over the two time periods.

Over the 2013-14 time period most of the station gains were less than 20%. Fewer than one-in-five stations managed to grow their share by more than 20%.

This year the percentage of stations that increased more than 20% doubled to 31% of all the group’s stations.

Maybe iHeart programmers got a lot smarter in 2015, or maybe iHeart stations are finally getting credit for listening that PPM has been missing all along.

The gains were pretty much across most formats too.

We compared the median percent change from the first period to the second in each format where iHeart has a significant presence.

Think of median as the half-way point in a ranker.

Half of iHeart stations in the format did better than the median. Half did worse.

As the chart at the left shows, the median performance for major formats in 2014 was anywhere from flat to a decline of 10%.

Country was the only format where iHeart managed to show a gain in 2014.

Every one of the formats turned around in 2015 with gains from 5% to 15%. Even Country managed to exceed its 2014 performance. Quite a turn around.

Given iHeart’s soft performance in 2014 we would have anticipated a small uptick in 2015 without the stations doing anything differently. In technical terms it is called reversion to the mean.

It is the magnitude of the gains that suggest something more than just a reversion bounce.

Knowing the flaws in PPM and the fact that many formats are not getting credit for listening they have had all along, a far more reasonable explanation is that Voltair has had a major positive impact on iHeart station ratings.

To understand the impact of iHeart’s decision to install Voltair, just add up the shares of all PPM measured iHeart stations.

Those shares increased nearly 9% after the group started installing Voltair. That’s a lot of listeners.

And those shares came from iHeart’s competitors.

Still believe that PPM is accurate and does a good job crediting stations for all the listening they deserve?

You must not work for iHeartMedia.

Did Nielsen Kill the Radio Star?

Did Nielsen Kill the Radio Star?

That’s the title of a well written investigative story at ESPN’s FiveThirtyEightLife. Written by Carl Bialik, it examines the impact of PPM on radio.

It is to date the most thorough story on the growing questions surrounding PPM’s ability to accurately identify radio listening and fully credit radio stations.

It is a piece well worth reading and rereading.

One quote caught our attention become of something that happened a few weeks ago.

Jon Miller, Nielsen's VP of Audience Insights, standing before a room of Talk Radio broadcasters flatly declared that PPM accurately measures Talk Radio listening. (See the video here. Skip to 35:28 to hear my exchange with Jon Miller.)

We're still trying to figure out what Voltair is doing but we've been confident from day one that our system does give the proper amount of credit.

How do we square his confident assurances with an Arbitron engineer quote in the story:

We had concern about various types of formats. It was thought that classical and some talk formats would be challenging to encode and possibly decode reliably.

So inside Arbitron engineers knew that some formats would not encode and decode equally, a point that we have been writing about for seven years. 

Nielsen’s official communique on Voltair was promised before the end of June.

We hope it will include a response addressing the Arbitron engineer's confession that some formats are "challenging" for PPM to accurately credit.

Voltair Case Study #3: Shares Nearly Double.

How about a talk show growing nearly 90% with Voltair?

The host is a well known national host that had been rumored to “encode poorly.”

We wanted to see the impact of Voltair on this one show and see if it resulted in better ratings.

Once again you may be frustrated by the lack of detail here.

We’ll be discussing the results at only one of the host’s affiliates. We’d prefer to look across stations, but only one station was willing to provide data.

We won’t identify the station, the host, or the specific time period we are looking at.

Until Nielsen issues its own communique you can expect stations to remain extremely cautious about releasing rating information or even admitting they’ve installed Voltair.

We’re working with the few cooperative stations that have come forward with data.

To determine the impact of Voltair we compared listenership for two weeks in 2014 and the same two weeks in 2015.

The days were chosen at random. We had not seen the data before choosing the days.

We focused on one quarter-hour, fifteen minutes of data. The quarter-hour was chosen randomly. We didn’t cherry-pick this particular quarter-hour.

We did, however, look at other quarter-hours after the fact to see how similar they were. They varied somewhat, but this quarter-hour appears representative of the entire show.

That said, the graph tell the story.

Listenership significantly increased after the station installed Voltair, not by a little but a lot. Each average minute in the chosen quarter-hour rose.

Much of the growth was fueled by fewer zero listener minutes.

Yes, non-zero minute totals increased too, but the noticeable decline in zero minutes had a bigger impact on the averaged numbers.

The 2014 shows had large gaps with zero listeners that might last for three or more minutes at a time.

The 2015 shows with Voltair still had occasional zeros, but they were fewer in number as well as shorter in duration.

As we’ve written, the purpose of Voltair is to increase the likelihood that codes will be strong enough to be consistently counted by decoders.

This should manifest itself as fewer drop-outs--fewer zeros--that indicate the possibility that decoders are missing listeners.

We would expect then higher more consistent listening numbers, which is what we see here.

So how did it work out for the station?

Show TSL rose 50% and cume rose 60%. This led to a nearly 90% increase in share. The show almost doubled its numbers.

We expected an increase in TSL. However, like the morning show we looked at earlier, Voltair also increased cume.

We have some theories about why increasing code reliability may be impacting cume as well as TSL, but we’ll save that for another post.

We would like to emphasize again that this is one station and one show, one that already has a reputation for poor encoding. We do not represent this to be any more than a single experience with Voltair. Your mileage my vary.

It is Nielsen’s job to prove that gains with Voltair are not real, that the box is creating listeners rather than helping decoders capture listeners that meters would have otherwise missed.

Unless it can prove that PPM today is capturing all listening and Voltair is artificially inflating the numbers, Nielsen has no right to tell radio stations that they cannot use Voltair.

Nielsen should put accuracy ahead of saving face.

Canada Bans Voltair Processor. Is US Next?

Canadian broadcasters pride themselves on the diversity of formats available to Canadians.

While US radio is dominated by a handful of PPM-friendly formats, Canadian broadcasters have made an effort to maintain format diversity despite low ratings fueled by PPM’s flaw of under-reporting listening to some formats.

Now Numeris, the Canadian rating organization, has chosen to jeopardize Canada’s diverse format landscape with a misdirected attempt to maintain the status quo.

Numeris has declared PPM untouchable.

Consequently the organization will perpetuate the un-level playing field that PPM created when it replaced diaries in Canada.

The problem started when Canadian broadcasters, learning of the success of Voltair in the US, decided to try it themselves. A few stations installed Voltair and saw immediate results.

Voltair rating spikes in the US generate stories of miraculous gains and record ratings. In Canada, rating spikes caused consternation and concern.

Ultimately Numeris felt it had to act. It instructed its broadcast members to pull Voltair.

It is a disservice to the industry it ostensibly serves, but more importantly it is a disservice to all Canadians.

Stations that keep Canadian radio diversified will likely suffer economically. Canadian radio will have no choice but to move further towards US radio homogeneity.

By forcing stations to pull Voltair Canadian radio will return to the old way, rewarding PPM friendly formats and punishing formats that PPM has trouble encoding or decoding.

Troubling is the fact that Numeris appears to have made its decision based on erroneous information.

We now have convincing evidence that PPM picks winners and losers based on station programming material, format, and listening environment.

With PPM listeners alone do not pick winners and losers.

PPM penalizes some formats, rewards others, and ultimately distorts ratings. It does not create a level playing field. Quite the contrary, it creates an un-even playing field.

Voltair creates the opportunity for broadcasters to be fairly rated, even with poorly encoding formats.

On Thursday June 11th Numeris issued a directive for broadcasters to turn off their Voltair processors for 60 days so that it could study Voltair’s impact on ratings.

Citing the need for a level playing field, it declared:

Numeris is currently undertaking a full analysis of encoding methodologies that have become available since the introduction of PPM. In the interim, the Numeris board has reaffirmed the importance of a level playing field and has requested that all stations using Voltair suspend use pending the outcome of the encoding review.

OK. We get it. Let’s take a look, they say. We’ll get back to you in 60 days.

What happened next, however, is hard to justify...or even explain.

Less than 24 hours later the board changed its mind. In an email it declared that, “in Numeris’s opinion the result of Voltair’s use is an increase in code density that may differentially impact data.”

It then instructed all station members to immediately remove Voltair claiming in bold letters that Canada’s Level Measurement Playing Field Must be Maintained.

Where do we begin.

It is the density of programming that determines code density. Ideally for accurate measurement PPM needs the densest material.

Formats that have the densest material have the densest encoding. That’s because the encoder can continually add codes to the programming.

(We’re simplifying a little here. Go back and read earlier posts to get a more complete picture of what’s going on.)

So PPM rewards formats with dense programming with lots of codes increasing the likelihood of proper station identification.

What happens if a format doesn’t have dense programming? The encoding becomes sporadic, adding codes only during those moments where there is enough material to mask the codes.

So formats with less dense dynamic programming don’t get codes inserted as often as the dense formats. That’s why some stations end up with big gaps in encoding, minutes with no codes.

Gaps equal lower ratings.

It is quite conceivable that there are well liked formats in Canada with large audiences that look weak in PPM ratings.

So Numeris is right. Voltair increases code density–-not by adding codes but instead by increasing the density of programming that the encoder can then use to add codes.

The encoding process itself does not change with Voltair.

Adding density to low density material helps the encoder do its job which means it levels the playing field for the formats that are being punished.

Formats like Classical, Jazz, Urban Oldies, News, Talk, Sports, and niche ethnic formats get the credit they deserve, and the listeners to those formats are finally heard from.

Numeris should welcome Voltair and encourage every station to add it to the audio chain.

Yes, it will help some stations more than others, but the stations that it will help most are the stations that have been hurt most by PPM.

Voltair only levels the playing field that PPM un-leveled.

Unlike Nielsen, Numeris is a not-for-profit, member-owned organization. Its responsibility is to its members, not to shareholders.

If Numeris is really concerned about the integrity of ratings and making sure ratings reflect the tastes of Canadian listeners, then it should realize that this knee-jerk reaction is wrong.

It needs to speak to independent experts rather than PPM stakeholders that are misrepresenting the issues and downplaying the technological flaws in PPM.

 

 

 

Up 69%! Better Product or Voltair?

What are the chances of a station increasing share by 69% in four months? Pretty small. Right?

And what if the station has been doing the same format since 1987 and hasn’t made any huge programming changes?

Even smaller, right?

So how do we explain KTWV, The Wave, achieving a 4.4 share in May, its highest share since Los Angeles switched to PPM?

The conventional wisdom is that iHeartMedia’s switch to urban contemporary helped KTWV, but let’s be real here.

It would take nearly all of Hot 92.3's listeners coming over to The Wave to pull that off.

The format change can’t explain a 69% increase.

Even if the Program Director, Ralph Stewart, has The Wave sounding better than ever, can we believe that a twenty-eight year old radio station that has averaged no more than a three share in PPM in six years can suddenly pull a 4.4 share and go from 20^th to 5^th in four months?

Only if they’ve installed Voltair.

Smooth Jazz is a format that PPM punishes more than most other formats. The format itself does not encode well, and it is generally played at a low level, complicating decoding.

Both factors conspire to increase the likelihood that Nielsen’s PPM will underestimate the format’s audience.

So Voltair should help Smooth Jazz more than other formats, and the impact should be immediate...like we see with KTWV.

Take a look at the above trend and ask yourself whether explosive growth like this is better product or something more.

There aren’t many Voltairs in service in Los Angeles, but the big players have them. Maybe that’s the reason that market AQH is up 5%.

That’s 5% in a billion dollar market.

With more than 600 stations using the box and many more waiting their turn we continue to hear nothing from Nielsen, The NAB, The Nielsen Advisory Council, or MRC.

How many more miraculous success stories are we going to hear before somebody admits there’s a problem with PPM?

Radio is to Blame for PPM Failures

Are you incensed that PPM isn’t as accurate as Arbitron led you to believe?

Are you frustrated that you might have to shell out money for a processor to fix what Arbitron should have fixed in the first place?

You have only radio itself to blame.

Yes, Steve Morris, Pierre Bouvard, and others at Arbitron did a good job selling PPM insisting that it was more accurate, more precise, and radio’s perfect 21^st century ratings solution.

But let’s face it. Radio really really wanted PPM.

Radio wanted to believe it was more accurate. Radio wanted to believe it was more precise. Anyone who expressed doubt was dismissed as a luddite, someone opposed to progress.

Now we know that PPM is far less accurate and far less precise than Arbitron claimed.

PPM’s fails to accurately capture radio listening, and quite possibly has done more damage than good to the medium.

By underestimating listening, PPM didn’t better prepare radio for the digital world. Instead it gave aid and comfort to radio’s enemies, digital stakeholders who used lower PPM estimates to help perpetuate the false meme of radio’s imminent death.

Asymmetrical information means one side knows a lot more about something than the other. And the side that does has a tremendous advantage over the other.

Arbitron had (and Nielsen still has) a huge asymmetrical advantage over broadcasters because it has refused to provide information on critically important aspects of PPM.

Without fully understanding PPM’s technological limitations broadcasters cannot judge PPM’s accuracy or precision.

The controversy over Voltair is an opportunity to address the disadvantage that PPM information asymmetry has created for radio.

Radio’s leaders need to hold Nielsen accountable for PPM’s failure to live up to its promises, and to make sure the company acknowledges the problem and takes steps to quickly fix it.

PPM technology should finally be put through stringent testing as foreign broadcasters did. Test results should be publically released. No more secrets.

Nielsen should not be allowed to hide behind the propriety information smoke screen that Arbitron used to dodge discussions of PPM’s technological limitations. Perhaps Nielsen could begin by answering the questions we raised in 2009 here, here, here, here, and here.

A more public vetting of PPM’s technology in the early 2000s might have warned broadcasters that all was not as it seemed with PPM.

And broadcasters need to understand that increasing the number of panelists won’t increase the accuracy of PPM as long as the meters fail to properly credit stations for listening they deserve.

Remember those endless discussions of panel sizes and in-tabs?

The distraction over panel sizes is one reason radio missed the bigger problem. That’s the subject of the next Radio Insights.

Illustration: Christopher Sleboda

 

Now We Know: Nielsen PPM Short-Changing Radio

Rumors of double digit rating gains using Voltair, a new processor, are fueling growing confusion over its implications.

Is it really true that a new processor can boost ratings, and by double digits?

And if true, what does it say about the accuracy of Nielsen PPM rating estimates?

Much of the confusion is due to a lack a data.

Stations using the processor are reluctant to even admit they’re using one let alone share ratings information.

But it’s pretty hard to hide.

Every month now we see growing numbers of stations with pretty suspicious rating spikes.  Are we to believe that all these stations just got smarter...and all at once?

And how many "best book ever" stories are we going to read in Nielsen’s monthly rating reports before somebody admits that maybe the box is behind many of these records?

Are we to believe that it's just coincidence that the success stories started appearing soon after Voltair's launch?

Given the lack of hard data, Harker Research decided to do something about it.

We set out to find stations about to install Voltair that would be willing to share ratings data with us so we could see for ourselves what impact Voltair could have.

What we are about to share is the first verified pre/post Voltair data, and what we found was just as amazing as all the rumors.

Two stations agreed to share their Voltair experiences with us.

We were able to track ratings as Voltair was installed. We were able to establish a pre-Voltair baseline of ratings, and then we were able to see what happened with the ratings after the launch.

We were also able to confirm that these markets were pretty quiet over this time, and that the two stations we followed changed nothing other than installing Voltair. These stations were also the first in their markets to get Voltair.

Like all other Voltair clients, the two stations insisted on anonymity, so we won’t be identifying the stations, markets, or formats. Both formats are highly competitive mainstream formats.

Station A was our first look at Voltair, so we looked at full week numbers.

For Station B we did something different. We looked at Voltair’s impact on morning drive. 

Station B plays very little music in morning drive, and it has been reported that PPM is particularly cruel to talk shows.

Since the encoding process does a poor job with spoken word, we wanted to see what impact Voltair would have on morning numbers. We’ll report what happened with morning drive, but we’ll also touch on full week numbers for Station B.

Ironically, Voltair increased both stations’ market share by more than twenty percent, but how that growth came about differed between the two stations.

First, a little background.

Voltair is designed to process station programming to create more consistent encoding. More consistent encoding increases the likelihood that decoders will be able to identify the stations to which panelists are  exposed.

So Voltair should have minimal impact on cume. All it takes is one successful code identification for a station to get credit for a cume listener.

The ratings benefit of Voltair should come primarily by increasing time-spent-listening (TSL).

If Voltair improves the encoding/decoding process, then listening normally missed by the meters will be picked up.

This might manifest itself as longer listening spans, but it could also increase TSL by increasing the number of episodes of listening, what Nielsen calls Occasions.

A 2011 analysis by Arbitron (since taken down) concluded that top performing stations in PPM didn’t have longer listening spans than weaker stations. Instead, leading stations had more Occasions of listening.

So to see the impact of Voltair, we began by looking at its impact on Occasions of listening.

As the graphs illustrate, both stations dramatically increased Occasions. Station A increased full week Occasions by 26%, and grew market share by a similar percentage.

Station B’s morning drive Occasions increased by a whopping 61%!

Full week TSL for Station A rose by 33%, but Station B’s full week TSL increased very little. Almost all the growth was in morning drive.

Station B saw its full week cume increase slightly more than Station A.

Station B’s talky morning show attracts a different audience from the rest of the day. We suspect that PPM was not catching the morning listeners, so the station was losing much more than just a little TSL.

Consequently, when the station started getting more credit in morning drive it gained Occasions, quarter-hours, and cume.

While this is the most definitive published analysis of Voltair to date, it comes with several qualifications.

We’ve only looked at two stations and two formats. We can say with confidence that the two stations’ gains were a direct result of Voltair.

However, as we have repeatedly emphasized, results will vary across formats and content.

The quality of PPM encoding depends on content. Some content encodes well, other content not so well. Voltair’s potential for improving a station's ratings ultimately depends on how poorly PPM is encoding the station's content.

Some stations will benefit more than others. Based on Station B’s morning gains, stations with talk formats should be particularly concerned about PPM’s "blind spots."

Some formats may not benefit at all, so don’t assume that your experience will be similar to either station.

The most important lesson here is that the rumors about Voltair helping improve ratings are true. This is the clearest evidence that PPM as implemented is flawed.

PPM does not capture all listening, so it under-estimates radio listening–potentially by a lot for some formats.

The solution is for Nielsen to admit that the problem exists and agree to a time-table to fix it.

It is likely that PPM will never be perfect. It will never capture all listening, but missing 20%, 50%, or more should be unacceptable.

 We invite additional Voltair users to confidentially share their experiences so that we can have a clearer picture of PPM and its weaknesses. Collectively we can finally get answers to PPM questions never asked as PPM rolled out.