Monday, June 27, 2011

Will HPCC System kill Hadoop?

HPCC System has jolted the whole community who were working on Apache Hadoop. The new developers are in confusion and start ups around Hadoop product are feeling nervous. HPCC System which claims to be better solution than Hadoop has some set of advantages as it claims to be faster than usual Hadoop MapReduce. Also developers who are comfortable writing in C++ application use to use pipes to compute the data now can think of using HPCC System directly. With Oracle taking over the Sun and Apache not being part of JSR committee what would be the future of Hadoop is the question.

I still prefer working on Hadoop as compared to any systems as Hadoop is strongly backed by Yahoo. The Apache community is one of most powerful community around. The community support in Apache project is highest. Hadoop has penetrated into the bloods of the developers and spreading like blood cancer. Yahoo organizes yearly events on Hadoop. We have so many Hadoop Community around the world. Cloudera has commercial distribution of Hadoop. Lot of commercial training on Hadoop is available by many vendors. Not only there are lots of machine images of Hadoop available for Amazon EC2, Eucalyptus, Open Nebula, OpenStack etc. There whole bunch of commercial systems which are built on top of Hadoop .

Given this popularity will HPCC System will kill the Hadoop. I have strong doubt about the same. Might be I am supporter of Yahoo and Apache community but overall if you consider the ecosystem to evolve it takes time.

For now LexisNexis need to release whole source code of HPCC System and then a community will start building.