Note: This was originally posted by CIO Review on 10/2/2012, available here.
As someone who is dedicated to creating high quality and accessible data, I am encouraged that more and more people are realizing that data in massive volumes does not really mean much unless it is high quality, verifiable, relevant, and accessible. The most important trend I see in the world of data today is that the element to create quality and accessible data is entering the mainstream.
The rise in data marketplaces or data exchanges is a great sign because they enable distribution to new customer segments, more creative pricing, as well as easier data sharing. Companies like Microsoft’s Azure Data Marketplace, docs, Factual and EMC/Greenplum are creating exchanges that demonstrate, to paraphrase Bill Joy, that there may be more valuable data outside your company than in it. These marketplaces/exchanges will foster the sharing and using of external data sources and will drive the creation of business models so people can get valuable data at a fair trade. There will be a natural process of quality control as the organizers and participants of the markets seek to protect their brands and insist that those who provide data show that it is correct. Third parties will review the quality of data, as they do for most other products.
The rising adoption of Application Programming Interfaces (APIs) outside of the world of tech startups will accelerate data accessibility. Most people do not know that most of the traffic for Twitter comes via APIs. The same is true for other popular Internet companies. APIs allow people to incorporate data directly into applications. To make this work, you have to understand how to build and run an API and what kinds of applications it will support. Companies like Apigee are bringing the infrastructure and knowledge of how to publish APIs to market.
I predict that as more great data becomes accessible through APIs and other means, the idea of a data supply chain will become popular. At Factual, we realized that all over the Internet, relevant data was available, but it was incredibly fragmented, and without any clean standards applied. Our first mission is to collect those fragments along with direct contributions from organizations and individuals and to create data about places and products. We have also created many APIs to help mobile app developers, ad targeting efforts, and businesses. We think that as more and more data becomes accessible, more companies will join us in creating data supply chains.
Road to the Future:
The discipline of Data Science will play a much more pivotal role in shaping every aspect of a business, moving from R&D/back office to being indispensable to their core strategy. Its the difference between having a few Hadoop engineers to employing data strategists that will ensure all functions have access to clean, high quality internal and external data, as well as a deep toolkit to do predictive analysis. While it’s exciting to see positions like Chief Data Officer, and Stanford opening a Data Science 101 class, we are still away from really knowing the combination of talent + software + culture required to affect the impact big data can have across the entire organization. I predict Data Scientists will be in the core DNA of many organizations, and a norm in the workforce versus an outlier and a minority.
Challenges faced by Entrepreneurs:
The characteristics of successful entrepreneurs will change in the next decade. Open source and cloud technology have dramatically lowered the costs of the development and production of new products. A startup that required $10 million a decade ago can now get going for $500,000 or in some cases even far less. While that sounds rosy, it also means that there is more competition ever from other startups as well as from nimble groups within larger organizations.
On the consumer side of the business, we have seen a tremendous acceleration around new products and features, and in order to succeed several things have to come together.
Creating great products is of course primary, but communication and marketing skills will become more important than ever as an increasing number of products and services proliferate the market. Attracting talent is the first chore. Once a product is in place, developing market awareness and changing consumer behavior is the next challenge. Successful entrepreneurs will be more balanced between engineering and communication skills.
In addition, it is clear that Lisa Gansky is right on the money in her characterization of how businesses need to embrace the Mesh. The most significant moat will come from data and relationships, and Mesh-oriented products look to every customer to provide information or reach that extends the value of the entire network to the entire consumer base.