The last year was a busy one for those of us in the data trenches and 2017 is shaping up to be even more so. Here are the trends that we think will shape the world of data acquisition and management in 2017.
Chatbots
What started as chat-based customer service performed by human resources (“I see you’re looking at our product page. Do you have any questions you’d like to ask?”) has evolved into semi-automated and completely automated real-time response systems designed to answer queries submitted by potential buyers in real time. Microsoft CEO Satya Nadella called this evolution “Conversation as a Service” and it is set to replace all manner of customer service positions. The value proposition is compelling to end users who can text a query to a number or click on a button to trigger AI-driven replies 24/7/365. This robot genie is not going back in the bottle, but it will take years of AI training and several transitional hybrid human-machine systems to perfect.
Anonymized Data
Working with personally-identifiable information can be problematic. It’s one of the few areas covered in US data protection laws including HIPAA regulations and child protection laws, so sharing it is tricky. Once anonymized, however, it can be swapped with partners or overlaid with other datasets to reveal an array of valuable information. Several firms are now aggregating anonymized data into feeds that can give unique insight into prospective clients when handled properly. Other players are forming data cooperatives to co-mingle their internal data with erstwhile competitors’ data. This lets all contributors apply predictive models to improve marketing response rates without exposing proprietary data.
The Non-anonymous “Crowd”
The illusion of anonymity has posed problems for the consumers of crowdsourced or cloud labor. The firms using the workers (the “requesters”) really liked the idea of labor that acted like a machine – no interactions, no negotiations over price, and the ability scale up, scale down, or stop work completely at will. Over time, though, the veneer of anonymity has worn away and the benefit of interacting with crowd workers has become clearer. Valuable feedback on how to improve processes to save time and money is no longer routinely ignored. Workers are organizing to reward better requesters. Requesters are wooing workers rather than offering “take it or leave it” propositions.
The Confluence of Crowdsourcing and the Open Data Movement
As governments move to digitize services such as renewing passports, researching land ownership records, or managing a variety of government records, they soon realize the necessity for significant labor resources to handle these large, often one-off, projects. Meanwhile, the rise of crowdsourcing has highlighted the attractive “portability” of data work steered to pools of qualified labor worldwide. Between the government’s labor requirement and the flexibility of data tasks, countries are building their own internal crowds of local resources to do the kinds of data tasks their governments need to meet government transparency goals and drive digitization efforts.
