The problem with algorithms and why we created Oomiji

Algorithms, the core formula behind Big Data, are not objective facts. They are opinions embedded in math, In the words of US data scientist Cathy O’Neil, author of Weapons of Math Destruction (2016). Watch this brief entertaining video to better understand how algorithms work as well as the potential mistakes we make in relying too heavily upon them.

As we’ve often noted to our clients at Oomiji, Big Data can be like looking in your car’s rear-view mirror, seeing no traffic behind you and predicting the same going down the road even though you’re headed into rush hour. That’s because it’s built on historical behavior. Sure, we can factor in times when rush hour is expected but only if we have enough experience to record it with reliability. Even then, we don’t know how we’ll do in all that traffic (a/k/a competition).

The root of the problem is that algorithms are built on the marketer’s definition of success. But in this consumer driven world, it’s the customer’s definition that should count. Understanding their success requires asking them how they define it. This falls to the market researchers who almost always are operating in a separate silo from marketing.

What we’ve done at Oomiji is to link market research to respondent files so they can be segmented by their interests and needs (i.e. their success definition), created real-time segmentation tools and integrated them with marketing communications. All of Oomiji’s features are on an integrated, horizontal platform that enables marketing directly to individuals based on what they each specifically want. 

Here’s an example: We do a lot of work in the global wine industry. We love wine, not because we like drinking it (oh, well yes we do that too) but because there is so much competition and choices the consumer must make. In fact, there are nearly 18,000 brands of wine sold in the U.S., each one with multiple SKU’s, perhaps as many as 250,000 SKU’s being sold. This makes wine an ideal place to use data and test how people respond to predictive analysis. Add to that that it’s a consumer goods category that most people don’t understand, other than they know what tastes good to them.

Big Data will take the wines that consumers have bought in the past, where they’ve been bought and apply an algorithm to predict which wines will be purchased in the future and where. The problem is that while consumers are mystified by wine, they are thirsting for education about it and their tastes are changing. Like most consumer goods, it’s an evolutionary market. We can look back to see a slow evolution of taste and changing purchase trends behind it or we can ask consumers “What if” and “Why” questions about their interests and needs for the future.

O’Neil notes that algorithms are usually developed by a homogenous group of people. Consumers are not homogenous and they don’t respond well to general information because there is no such thing as a general consumer. The purpose of market research is often misguided to determine what people have in common. We want to know what drives them apart so we can market to them based on their own specific needs. It’s simply common sense to appeal to people with what is most meaningful to them.

Watch the video, take a look at Oomiji.com and contact us for an online demo to see how it works.