Using Small Data to Make Sense of Big Data, October 2015

The telecomunications industry, like any other industry, needs buzzwords and the current talk of the day is, of course, big data.

Today, big Internet players are creating knowledge out of fragmented slices of user behaviour. This includes tapping into the vast amount of data that technology companies collect from users of different services, for example in the case of Google; or of a vast number of topics discussed, as is in the case Facebook. In the Telecommunications industry, on the other hand, widely published success stories of big data utilisation are harder to find, despite everyone talking about the topic at every opportunity. Why might this be?

One possible reason for the lack of commercial success in the CSP domain may lie in the philosophy of big data processing itself. Reading the myriad published articles, blogs and vendor case studies, the discussion seems to largely concentrate on the “how,” rather than the ”what”. In the case of, say, a company like Google, this is understandable. Throwing some sophisticated maths at a very, very large data set will  sooner or later yield some insight. A sort of serendipitous discovery. And since in the domain of internet services there is often no preconceived idea of what we are looking for, this is fine and can be put to use without too much of a problem. 

Despite the ever expanding scope of services CSPs are offering, the CSP domain on the other hand, remains much more confined. Therefore, CSPs are often looking for answers to more specific questions. Answers that often get lost in the “bigness” of all the big data available. The broad stroke approach simply does not suffice. 

This may partly be due to the fact that in the “how” side of things, there is a possible issue in the nature of available data itself. Compared to the data available to the big internet companies, CSPs often have available data sets that are not only extremely dense, but also inconsistent over time. With relatively few users, millions as opposed to billions – using a relatively small set of services – thousands as opposed to tens of thousands – overlap in service usage between customers is significant, rendering many of the ”standard” approaches to big data ineffective.

So, a different approach is called for. One way to improve the results would be to combine big data with small data. That is, distill the broad trends out of the big data using “standard methods” and then hone in on the behaviour of individual users and their immediate surroundings for detailed insight. After all, many CSPs – especially those operating predominantly in the prepaid services arena – have access to consumer behaviour in real time. By reversing  the approach by looking into the user-level detail first and then moderating the results with trend data, from increasingly large circles of users, a CSP might be able to take a step closer to their promise of an individualised service.

Like any technological approach, big data makes for a good servant but a bad master.  The results gained from applications of mathematics to a big data set are only as good as the data set available and unfortunately, in the case of many CSPs, the data leaves much to be desired. For this reason, taking a detail-oriented approach may well prove to be a better, and often less expensive, approach.

Timo Ahomäki, CTO, Tecnotree

This article was orginally published in VanillaPlus

Tecnotree Corporation, Finnoonniitynkuja 7, FIN-02271 Espoo, Finland, Tel +358 9 804 781 | Privacy Statement | Personal Profile

Tecnotree VAT no. FI16515770