Engagement is not an excuse, it’s a KPI

To continue this debate Avinash and I have been discussing. I asked Avinash to clarify what he meant by customer. In my view Jim Novo and Ron Shevlin are talking about actual customer engagement (customers who bought a product) whereas Avinash is talking about visitors (people who may not have purchased anything and therefore you have no information about them). Avinash said he was talking about visitors and then goes onto say;

We could hire a Consultant who will spend three months and create a complex ¢‚¬Å“Engagement Index¢‚¬ for you that has weighted average/whatever and put all the above metrics in that Index.

They walk out of the door with US $200,000 and you are left holding a ¢‚¬Å“Engagement Index¢‚¬ that no one understands broadly and when the Index goes up or down no one has any idea why and what needs to be done.

The Consultant is happy, you are left holding the bag with a hole (where the $200 grand was).

Hence my recommendation: Figure out your site purpose, figure our what it means to talk to your visitors and have a conversation with them and get them to visit again and again, identify the right metric for you, call a spade a spade, let your entire organization understand what you measure and let everyone big or small drive actions from insights that only a simple metric can give.

Don¢‚¬„¢t Sexify, Simplify. And conquer!

I don’t have any problem with Avinash’s approach, that’s not really the point. The point that I don’t agree with is the fact that we’re using engagement index very effectively as a KPI and the point needs to be made that it can be used effectively.

So here is an example I have also put on Avinash’s blog as a comment.

We defined a simple engagement criteria of a 3 minute session. So we segmented all the visitors that had been on the site for 3 minutes (enough time to make a purchase). By segmenting I mean totally separating the behavior of all the visitors to an index of visitors who had become engaged as per our definition (Engagement Index).

SATAMA deal with a number of very big clients that spend a lot on marketing every quarter and we saved this particular one ¢€š¬2million on advertising spend that would have been entirely wasted.

A direct action came from the analysis and instead of spending ¢€š¬2.5 million, they spent ¢€š¬500,000. They were extremely impressed with what they learned. If you ask them whether measuring engagement was important they would give a resounding yes.

You have global visitors (hundreds of thousands of them a month) coming to a particular product page which could be bought online via a shopping cart. The end goal of these campaigns were sales.

You had maybe 20% of those visitors which were part of the engagement index. As it stands right now, I agree, not very useful.

However because we put the visitors into a segment (remember we were using a tool that could seperate global visitors from our engagement index) we could see how effective the campaigns were and work out not only a cost per conversion, but also the cost per engaged visit.

Now this was extremely useful. All visitors who bought a product were by definition engaged. It took about 3 minutes to purchase anyway, so it stands to reason that you need to have visitors that became engaged at some point with the campaign site if you want them to buy.

Now I could say whether Google was driving more engaged visitors than banner ad placements or rich media ads and which one cost us the most. Search engine marketings cost per engaged visit was much much lower than the banners in this particular case.

Based on previous campaigns, the shopping cart abandonment for this product and engagement numbers I had in front of me I could predict that running banners in the same way as previously was going to be a disaster from a sales perspective. (Einstien once said, ¢‚¬Å“The definition of insanity is doing the same thing over and over and expecting different results¢‚¬)

The abandonment was incredibly high (another engagement metric – though we call that shopping cart abandonment like everyone else) and the engagement from the campaigns incredibly low in comparison to other campaigns. In the size of the company I¢‚¬„¢m discussing ¢€š¬2.5 Million is chicken feed so when i raised my hand and told them not to do the initial plan (banner placements) they tried it anyway. My prediction on cost per engaged visit and cost per acquisition was out by about 10% but regardless the numbers on the first ¢€š¬100K spent was a disaster. (SEE INSANITY above ;o) This was when they started paying attention to what I¢‚¬„¢d been trying to tell them.

So we tried to show them how to improve.

Google engagement costs were much lower for the same amount of traffic coming from banners but they were still quite high (we thought) based on experience.

We looked into trying to improve on this and the banners (took action). The keywords used by visitors to find the product could also be separated based on the engagement index and when we had split which ones were engaged and which ones weren¢‚¬„¢t we could optimize search campaigns based on those keywords.

Sales and engagement rose.

We also looked at the banners down to the ones which engaged and the ones which didn¢‚¬„¢t and directed media spend to the ones which did, again.

Sales and engagement rose.

Overall instead of burning ¢€š¬2.5M on banner ads (which had been done in the past with really dubious success metrics loosely based on generating awareness) they optimized the engagement from all the campaigns and learned a a lot from the journey.

Had I not had the engagement index, I would have not known which keywords those guys used in comparison to global traffic. Nor would I have known which banners worked better than others.

All I would have known was traffic numbers from each source and sales data but my optimization routes would have been more limited to expensive qualitative studies. The data I am talking about all came from a web analytics tool.

We hit the same targets set for a much lower cost than ¢€š¬2.5 million. It cost us in total ¢€š¬500K. So while I didn¢‚¬„¢t charge anywhere near $200K to set-up and run with the engagement index, had I done, it would have still been a huge cost saving for them. (In retrospect maybe I am too cheap! ;o)

Is this a proxy for sales?

Could I have segmented based on visitors who became customers? In this case no, the data we got was too small – worthless – we tried.

Now what Jim Novo and I believe a number of others reading your original post may have thought is that you were talking about customer engagement, in terms of a person you have recorded buying something. I don¢‚¬„¢t disagree with anything Jim said because I believe he is discussing the nurturing phase, more focused around CRM and customer analytics. In the case I¢‚¬„¢ve described we could also have done this, however the time/resources weren¢‚¬„¢t there.

I on the other hand am talking about visitors who have yet to buy and can¢‚¬„¢t be segmented out of the noise as a customer. Basically people that may or may not be interested in your products and services and filtering out what might be useful.

I hope this helps clarify my position.

Steve is a well known analytics specialist, author and speaker. A pioneer since 2002, he established one of the first European web analytics consultancies (Aboavista), later acquired by Satama (now Trainers’ House) in 2006. In 2008 he wrote his first book Cult Of Analytics published on May 14th 2009. He currently serves as CEO at Quru and has presented and keynoted web analytics topics across Europe. These include The Internet Marketing Conference (Stockholm), The Search Engine strategies (Stockholm), IIH (Copenhagen), the IAB Finland (Helsinki), Media Plaza (Amsterdam), Design For Conversion (Amsterdam) The eMetrics Summit (London, Munich, Stockholm), Divia (Helsinki) in addition to sitting on dozens of panels.

Posted in General, Web Analytics
2 comments on “Engagement is not an excuse, it’s a KPI
  1. Ergo Proxy says:

    What did you mean with the question: “Is this a proxy for sales?”

  2. Avinash was arguing that the Engagement Index we had set-up was a proxy for sales, so he was saying that because we couldn’t measure to a sale or some other conversion that good engagement was a bit of an excuse for poor sales.

    My argument was that we had to start somewhere with our improvement processes and as we didn’t have sales coming from the campaigns we had to start somewhere. So I asked the question…. Was it a proxy for sales and answered it myself, no. :)

Leave a Reply

Your email address will not be published. Required fields are marked *



Enter your email address:

Delivered by FeedBurner

Recent Tweets
Site Sponsors

Find out how we're trying to make our clients €1bn, click below!