Tuesday, September 14, 2010

The explosion of digital information has provided many opportunities for data driven startups

We’re producing data at an amazing rate. In fact, humans generated more data in 2009 alone than in the entire history of our species up to that year, according to Amazon’s former chief scientist Andreas Weigand. And IT firm EMC projects the quantity of digital information we produce will grow by a factor of 44 between 2009 and 2020.”

Ken Elefant of Opus Capital outlined three market trends that startups are using to enter what he calls the "big data" markets:

  • Data Accessibility
  • Data Control
  • Better Tools, Better Business Insights (Analytics)

Startups have recently gained ground by providing a business model that uses at least one of these three methods. Although this is the current trend of monetizing vast amounts of data, what does the future hold for data driven startups?


  1. This is interesting, but I am reminded of past criticism of compute grids and supercomputing (and, for that matter Business Intelligence) as solutions in search of problems.

    For companies or organizations that do identify useful applications that can leverage this data and processing power, some other questions that must be considered are:

    - What will the cost of developing or buying these applications be?
    - What will be the cost of spreading them through the organization -- and will they be welcomed?

    Regardless, companies will need some way to store the growing amounts of data and potentially have ways to easily access it, either for applications or compliance. EMC and the other storage specialists (and startups) stand to profit handsomely from this trend.

  2. Amazon’s former chief scientist Andreas Weigand said that humans generated more data in 2009 alone than in the entire history of our species up to that year. I think that the reason is that we can more easily create video or picture than ever before thanks to digital equipment and Web service. About 3 million user generated contents were produced daily in my company, Daum Communications's online community( http://www.daum.net ) and the number is almost same to it in 10 years ago. But required storage size become bigger and bigger as time goes by. If you want to create a new company, I recommend you building the technology related to a image processing and video analyzing to help multimedia search better.
    In the serveral years later, the data we call information could be a waste without the technology which changes disperse data into structured knowledge.

  3. I believe there are a lot of business chances for data driven startup companies since many IT companies have big data but they have not yet maximized the value of it and no idea how to use it.
    For example, mobile companies have individual data such as originating/terminating call time, handover info, location info, location update signalling data, amount of data used, personal data, etc. By using those, startup company can offer better analytics to mobile operators how to improve overall quality of services and can offer new service like tracking services.
    The only reason mobile operators don't expose their data is that there are legal compliance and privacy issues. Once They are solved, I believe there will be a big chance of business.

  4. Digilata data which enterprizes are accumutating is growing at an amazing speed bacause of the reduction of hardware prices and devepopment of multi-precessing technology. I have worked to make full use of about 10 million customer data, demogaraphic and behavioral, to develop and market new services last 3 years. I always have two major problems.
    1)Data mining
    Compared with the increase of the processing speed, data mining technology remains rather primitive. In most of the case, I have to develop new models from srcatch.
    2)Restriction of individual behavioral data usage
    Getting consent to use from a large number of consumer is not practical. In many countries there exsits regulation over traffic data usage. In US the discussion seems to centre on the protection of privacy. In some country the regulation is much more stricter. For example in Japan, usage of traffic data itself tends to constitute criminal violance for communiction-madiating enterprizes.

    If this second regulation is modenized, the incentive to accumulate more didital data and advantage for enterprizes with large customer base will be enormous.

  5. As digital information increases, economy in digital world will replicate the convetional economy.
    Besides the physical server business, there should be a security/insurance business to protect it from possible pirate or misuse, data bank service to keep the personal data for a really long time (for examples, children's growing video data from the birth), or even digital data making services for non IT-familiar customers.
    Also considering how quickly the barriers among nations in physical world breaks down, breaking the barrier in digital world (let's say seamless and real time traslation among different languages) will make a huge opportunity as well.

  6. SungDuck CHUN (SF'11)September 16, 2010 at 3:14 PM

    But, sometime in the future, we have to think about the quality aspect not the quantity aspect. I don't quite remember the number. But one research said that most of the traffic over TWITTER is just meaningless.
    This is more linked to the A.I. Intuitively, human know what is important data and where we can find it. Advance in A.I will enhance the experience with search engine.

    Then, regarding data itself, currently most of the data is generated by users. Data regarding each person activity or daily life is not continuous. Open platfrom or data sharing between companyies or institution is required to complete overall picture.

  7. It seems like most of the use of "Big Data" right now (besides in finance) is in the consumer advertising/marketing industry. This makes sense because it currently has the three key components mentioned by the author:
    1. Data is readily accessible - I am currently opting out of 106 "ad networks" tracking my moves on the internet.
    2. Data control has been worked out. Ad networks own whatever information they gather about me by implicit agreements when I go to a website.
    3. The rich existing theory around marketing provides fertile ground for companies to develop unique analytics.

    An interesting question is how "Big Data" can be extended to serve businesses seeking to improve their operations? It seems like that market may be underserved, with the main barriers being data accessibility and data control - there is a vast volume of theory in operational research that can be used to take advantage of the data.

    In that sense, big firms may have inherent advantages over small firms due to their history and internal volume of data produced per unit time. Is there a way to level the playing field?