Mobile Musings: Have you been a respondent yet?

Mobile Musings: Have you been a respondent yet?

Scott Weinberg, Tabla Mobile LLC
Immediate Past President, MN / Upper Midwest MRA Chapter

I’ve been noticing how few market researchers and advertisers have participated in even a single mobile research study. Specifically, I’m referring to an app-based experience, usually using a form of geo-validation and multi-media data capture. I’m not referring to opening a url on your phone and taking a survey, any survey.

Rather, I’m referring to an actual ‘mobile research’ experience, the kind where you’re notified walking into a movie theatre, Best Buy, Target, grocery store, gas station, etc. Alternately, you may be pre-screened and invited to participate, e.g. an out of home ‘assignment.’ The reason I’m curious about this is because of the (profoundly?) unique and different respondent experience these studies entail. Let me give you a few examples.

I took an in-store study, or attempted to, inside a Super Target. I’m not affiliated with this supplier; I have several survey research apps running on my phone (and I never stray far from an electric outlet). Essentially the assignment entailed taking 1 photo of 11 variations of a food product, and responding to a few questions on each. Not difficult; tedious, but not difficult. When I uploaded the first pic, my phone timed out/went into lock mode (set at 1 minute). I tried it three times. I was on an iPhone back then, where pic file sizes range from 1-2 mb, depending on the detail (Androids are similar). This isn’t an issue on a home wifi or similar network, but inside a big box, via your cellular carrier, pic uploads (or any uploads) can be a pickle.

So what did I do? I was calculating that even if I could get the upload to work, I was looking at a 15+ minute boring repetitive survey, while standing in this food aisle. Not much intrigue to this. I was wondering how many others around the country were having this same frustrating experience. I decided to try an experiment of my own: I took 11 random product photos outside the survey (just using my camera) and exited the survey. The survey told me I had an hour to finish up from when I started. I drove home. Resumed the survey on my home wi-fi. At the first upload sequence, I randomly uploaded one of the pics, in about 2 seconds. Answered those questions. Went to the next sequence. Rinse and repeat. Finished in a few minutes. My experiment was to determine if this particular app had any kind of lockout or detection protocols for what I was doing. This supplier is a major player, one of the largest out there. Submitted fine, and my incentive showed up after a few days.

I’ve also noticed recently that Target’s offer free wi-fi. You need to actively accept their terms and login to connect, i.e. it’s not an ‘auto-connect.’ I wonder how many people actually do this? Or how many suppliers tell their potential respondents there is free onsite wi-fi, and to connect to it? I’ve never seen messaging to this effect in a mobile study; have you?

Another example, this time as a project manager rather than a respondent. On a time sensitive, DMA-specific mobile study, a phone recruit to survey app was in effect. Ergo, many of the respondents were ‘first-timers’ to this kind of study. I’m rather keen on these audiences actually; as they bypass the conditioned (i.e. self-select bias) ‘panel people’ who comprise the bulk of all primary online research (and a small but growing portion of mobile respondents). During this study it became apparent that live tech support was needed (and by live I mean immediate, while they were in-aisle). I began emailing my phone number to the potential respondents, and my phone quickly started ringing with confused respondents. They weren’t doing anything wrong, the app was working fine, survey was loading fine, they were just unsure how all this works. Happily however, they were motivated to participate (a healthy incentive didn’t hurt).

So, what are the lessons here? First, suppliers approach signal strength issues differently, with some using offline versions of the app experience (data are uploaded later); others minimize the amount of data uploaded via design. Ask what your options are. Second, when the sampling universe is small, e.g. with specific DMA’s, age groups and such, ergo when each potential response is critical, it’s wise to plan for tech support in advance and have live people ready and on-call to answer questions or take feedback. A confused user may not return to the study if they can’t access the content correctly.

Most importantly, experiencing activities like these make more an impact than reading about it; I always encourage interested parties to experience this methodology as a respondent. It doesn’t matter whether you’re new to the mobile research space or are versed in various fieldwork methods; the technology is rapidly changing, and our assumptions regarding how we should interact are best learned empirically.

Mobile Musings: Disintermediation

My prior blog post ‘What is Mobile Research’ was a simple primer in the form of a Q & A. I thought I would stir the waters a bit more with this submission. I’ve been discussing, and in fact advocating, mobile research (not mobile surveys, which are different) as a methodology with both market research firms and end clients since early 2012. I haven’t kept count but certainly over 50 separate discussions/presentations, roughly split between the ‘channel’ (MR firms) and client siders. I developed two different talk tracks, based on viability with the company type and also awareness of a new business model:

  1. Within the channel, the positioning is more a VAR model (value added reseller), in which the fieldwork would be outsourced to a specialty firm who is equipped with both the technology, i.e., a mobile research smartphone app, & the traffic, i.e. engaged people who have installed said app. The MR firm then adds their ‘secret sauce,’ hence adding the value, to the fieldwork. Same model as with online panel sourcing, for example.
  2. When I’m with end clients (for me, typically CPGs and big box retailers), the people I’m chatting with have comparatively more open-minds about the methodology. The difference can be dramatic. I’ve seen it over and over.  Mobile research can (virtually) take them into their stores, in front of their products, and their competitor’s products, all from the consumer’s perspective. It’s more a case of ‘how do we get started’ as opposed to ‘what about representativeness?’ (I’ll address representativeness in a later post).

Well, is there a point to this? I think there is, and it’s this: end clients are calling upon their MR firms of record to engage in ‘in the moment’ research with today’s mobilized consumer, and the channel has been caught unprepared. And whatever interest is there now will be dwarfed a year from now. And two years from now. In defense of the channel, the rise of mobile research technologies has grown so quickly, it’s not fair to expect MR firms, whom, after all, are hypotheses generators and insight strategists, to have sophisticated smartphone technologies and a mobile panel in-house and ready to go. Noted.

However…the reactions I receive from my channel discussions, and I’m usually in the room with the senior execs, is one of four:

  1. I’m going to ignore this and hope it goes away
  2. We’ll take a wait and see approach; perhaps my clients will magically request a mobile study
  3. Hmmm…maybe this is a paradigm shift, maybe not, but I should get up to speed
  4. This would have been perfect for a qual/ethnography/mystery shop/shelf set sim study we ran last month, let’s add this to our competencies

I think if you’ve spent time observing mobile research as a business model, you’ve become aware that the end clients are leapfrogging, or disintermediating the channel and going straight to the ‘OEMs,’ i.e. those firms who have been quietly developing the apps and promoting them to audiences (recruiting I addressed in my prior post). These OEM suppliers, be they online panel firms or pure mobile plays, have naturally jumped into the void and are assertively getting in front of these end client study sponsors.

“But what about the secret sauce” I’m often asked (I’m paraphrasing) by the channel? Well, two items of note: raw, or semi-raw, data feeds from inside stores restaurants, etc, with pics, vids, and open end audio clips (what I consider the real Voice of the Customer, finally!) from mobilized consumers is as pure and non-biased a data feed as this industry has seen. Moreover, the OEMs are hiring research pros to diversify their service offer as well. Check out their press releases or attend their conference sessions if you don’t take my word for it.

If I was running an MR firm and planning to be around awhile, I would hedge my bets and initiate frank chats with other industry insiders. Specifically, about the implications of smartphone ubiquity, disintermediation, and how to maintain our perceived value as insight strategists and domain experts with these new technologies swirling around us.

Scott Weinberg, Tabla Mobile LLC

Immediate Past President, MN / Upper Midwest MRA Chapter


Mobile Musings: What is Mobile Research?

Since early 2012 I’ve been actively involved in the emerging methodology known as ‘mobile research.’ As a result, I often find myself in interesting conversations with market researchers who are also getting up to speed with this new fieldwork technique. I hope you find of interest these frequent questions I’m asked (and how I reply).


What is mobile research?

This question is not as simple as it appears. Some consider any survey or study activity conducted on a phone or tablet as ‘mobile research.’ Regardless of the information exchange occurring via browser-based survey, email, or SMS exchange; as long as it’s happening via mobile device, this is ‘mobile research.’ In fact, I’ve talked to people who consider a telephone/IDI survey as mobile research, if the respondent was on their mobile phone during it.
 A more stringent definition involves apps designed to enable the native GPS and multi-media capabilities built into smartphones and tablets. The respondent usually has more ‘involvement’ with the study in using these capabilities, compared to responding to directed questions offered in a conventional survey.

Are mobile studies representative?

Another common, and intriguing, question; if I’m being asked ‘are these studies representative of the general population?’ I reply: no, they are not. However, this is what makes mobile research not only intriguing, but a  strength for progressive MR professionals. Specifically, the demographic metrics of smartphone ownership should be of interest for this reason alone. Keep in mind: the percent of the USA population, who have joined online panels, is 1-2%, depending on your source. An enormous amount of MR fieldwork is powered by this tiny slice of the population.
I think it is safe to assume that considerably more than 1-2% of the USA population is in possession of a smartphone. Moreover, the demographics of these owners, from teens on up, working professionals, cell-only households, etc. makes this potential audience of keen interest to consumer brand firms. Certain pockets of these demographics are challenging to reach via traditional advertising mediums of TV, radio and print. However, these same age and demo cohorts are often in possession of a smartphone. And one only nee ds to look around to see how actively involved they are with them.

What about data quality?

Given the reliance of self-report feedback through conventional methods (online surveys, phone/mail surveys), the potential of smartphone-based research should be of keen interest. There is an added layer of validation via mobile-research, with shopping behavior in particular, that is capturing the interest of consumer brand firms and manufactures. Consider the difference between asking ‘are you shopping for a new car?’ in a conventional survey, vs. receiving photographic verification and open end commentary, in audio format, from a car shopper while they are at the dealership (and all the dealerships they are visiting that week or month).
Moreover, mobile research can provide in-store shopper feedback, and purchase verification, of laundry detergent, shampoo, grocery items and other fast moving consumer goods via bar code scanning and receipt photos. The appeal of store promotions, instant coupons, etc. can be measured in-store, to gauge brand loyalty and propensity for on the spot brand switching, etc.

Do I need an app for mobile research?

If one simply requires the respondent to have browser access, then the answer is no. Examples of this may includeclosed audiences, for example at a meeting or conference. Or, the survey may be offered as a convenience, for example as a follow-up to a customer service inquiry, where the survey device doesn’t matter. However, many ‘online surveys’ do not rend er properly on a mobile device, and the importance of this, often overlooked, can quickly lead to respondent frustration and drop offs.
Sophisticated mobile research, involving geo-validation, barcode and multi-media validation, does require an app designed for this kind of activity. Although these apps are designed to conduct similar tasks, there is a surprising amount of variability in the user experience and technical proficiency (i.e. care given to the battery drain problem). As always, it’s good to shop around, and I encourage interested parties to try several apps via the app stores, and experience these as a respondent might.
I hope you find this mini primer in mobile research helpful, and keep an eye out for further newsletter articles
devote to this topic.


Scott Weinberg,

Immediate-Past President of the Upper Midwest / MN MRA Chapter

Originally posted in the MN-MRA Winter 2013 Newsletter