Big Data is certainly the buzz term of the year and there is no doubt that the presentation of Big Data technology into the mainstream has presented a vast array of opportunities not just for web companies but for enterprise companies across virtually all verticals.
Platforms such as Hadoop now allow companies to mine and process unstructured data and large amounts of data in a much more efficient manner but they are certainly not the magic bullets or finished articles that end users are demanding.
The explosion of cloud technologies have increased the demand from end users who now want their software as a service and on demand but right now, whilst there are “Big Data�? companies beginning to offer this type of service, the market is a long way from offering recommendations, advanced sentiment or predictive analytics in a SaaS fashion.
The reality is that most organisations outside the big web companies such as Amazon, Google and NetFlix are still finding their feet with the technology and trying to understand how to use it to deliver on their strategic goals. Even the organisations that have defined this are realising that a Hadoop developer is not going to give you the perfect tools you need and in fact data scientists are required to model the algorithms needed to really gain customer or machine to machine insight.
As the Big Data world matures and companies really start to get to grips with how to use it, the demand for data scientists will completely out strip supply as will it also for data visualisation experts who will need to make sense of the mathematical results coming from their real time sentiment analysis algorithms and turn it into a nice end user experience.
Making sense of big data requires a combination of engineering and data science expertise – having the typically low level specialist technical skills as well as the ability to get behind the science of the data is equally important. With so many new big data solutions emerging from distributed scalable systems like Hadoop to NoSQL database solutions and new machine learning techniques, the technology has evolved faster than the workforce skills in the majority of organisations. Also fuelling this evolution is the wide acceptance of open source technology by large enterprise customers. As the de facto technology solution for big data, Hadoop is beginning to gain wider adoption amongst large enterprises as they seek to gain competitive edge in a way traditional platforms did not allow. Whilst there are pockets of talent in the UK & Europe, there is a widespread shortage in trained staff in this geographic region.
The end-user customers and vendor community are all competing for the most talented big data developers and data scientists. Big Data Partnership firmly believe it is critical for end users to adapt and enable themselves to compete in an increasingly data-driven economy by cross-training their workforce to deal with tomorrows challenges today. To gain real competitive edge, you have to have the right talent and the right teams with the relevant skill set. Apart from a minority of early-adopter companies, this does not exist today across may of the enterprises. Recruiting is one option but with very few individuals available, companies will struggle to source suitable candidates without paying a significant premium. Formal training and certification will play an increasing role to identify experts in big data, Apache Hadoop, Apache Cassandra and other key technologies. It is important that companies act now and budget for training existing staff in these new technologies which we believe will help attract talent, retain staff and increase job satisfaction.
In summary, the are still a number of hurdles to overcome for companies wanting to adopt big data including the technology itself but there is no doubt that it will and is changing the way that organisations look at consumers and technologies such as Hadoop and Mahout will become key tools when deciding where and who to advertise to or provide services to.
Big Data Partnership offer a variety of public open enrolment and on-site private big data training including Hadoop and Cassandra training & certification helping the adoption of enterprise big data solutions.
Comments are closed.