Data philosophizing

As I will tell anyone willing to listen to me wax poetic about how a person with two degrees in literature ends up becoming a data analyst, data (as a concept) is a narrative. The data you gather are all threads that, when woven together, tell a story.

I’ve spent a lot of time recently cleaning data, which is always a tedious task. Analysts are nothing if not tenacious in this regard, but even we get a bit weary of making sure every little detail is as valid as possible. It’s also a lonely task. Even though I often listen to music when doing things like this, my mind still manages to wander.

I started thinking about data purity. There’s this idea that quantitative data is the purest form of narrative; “the numbers don’t lie,” after all.

I had a conversation recently with someone about this concept, and how so many people, some quantitative analysts included, don’t recognize (sometimes willfully) how easy it is to introduce bias into numbers. Anytime you have humans involved in a process, you have bias. Bias can be introduced through study parameters, during analysis, or in the conclusions drawn and recommendations given. Many people bend data to fit a narrative that’s been pre-ordained.

What I like about qualitative analysis, which is my specialty, is that it doesn’t hide or deny bias. It encourages the researcher to think about and state their biases very clearly. Some methodologies actually use the researcher’s bias as part of the study. It’s impossible to completely put aside your own perspective, so why not channel it?

I’m starting to tackle with a lot of philosophical questions about qualitative analysis, and bias, and constructing studies that are useful operationally but true to what qualitative analysis is at its core. It’s an interesting place to be floating for a while, and gives me a lot to think about as clean, and check, and clean, and recheck the quantitative data I’m slowly polishing into something meaningful.

At the heart of it all, that’s really what all data analysis is about, be it quantitative or qualitative – finding meaning.

And meaning, and what meaning is, can launch a hundred different discussions and poetic manifestations…

Advertisements

Statisticians and data analysts are the new prophets

A prophet, in the religious sense, is a person who has a direct line to a god or divine entity, and is acting as a conduit between the divine and humanity. In more recent years, the word prophet has been applied to economists and statisticians. (Nate Silver anyone?) See wikipedia for more fun facts about prophets in major world religions.  See Nate Silver’s website if you occasionally have nightmares of Trump winning and need some reassurance.

What I love about the field of statistics is that we describe trends and predict things, then bring those things to leaders and say, “Here’s what we think may happen.” We tell stories using data. We’re narrative builders. We’re predictors of the future.

That would, of course, make data the new god. Our deity, Big Data, hallowed be thy name. And it makes sense, doesn’t it? The other day, I purchased a new brand of dog food. Suddenly ads for that dog food pop up on all my social media sites, and in my gmail account. Big Data knows all. I sent some emails about the frustrations of being childfree (by choice). I stopped seeing ads for baby formula or birthing centers. Big Data sees all.

Data does not lie. But humans make data imperfect. Humans misinterpret. Humans act unethically. Humans introduce bias into the narratives, or change the narrative to suit their needs. Just like they do to gods.

And analysts are here to correct that. Just like prophets once did.