Apache Spark is hailed as being Hadoop’s successor, claiming its throne as the hottest Big Data platform. What the founding fathers of Spark are saying and doing about its future and its positioning in the market has never been more timely.

You probably did not hear it here first. Spark has been making waves in big data for a while now, and 2017 has not disappointed anyone who has bet on its meteoric rise. That was a pretty safe bet actually, as interpreting market signals, speaking with pundits and monitoring data all pointed to the same direction.

Spark adoption is booming. Its community is growing, and all major big data platforms make a point of interoperating with Spark. If you look at its core contributors and project management committee (PMC) you will see Hadoop heavyweights Cloudera and Hortonworks, and all-round powerhouses such as IBM, Facebook and Microsoft.

You will also see a name you may not recognize, but dominates Spark’s current development and future direction: Databricks. Databricks is a startup founded by Spark’s inventors, Ali Ghodsi and Matei Zaharia. Ghodsi and Zaharia, who started out as fellow researchers and friends in their Berkeley days, are the CEO and CTO of Databricks.

Last week the Spark Summit Europe event attracted more than 1,000 attendees in Dublin. Ghodsi and Zaharia were both there to share news, get in touch with the community and discuss. ZDNet was also there, and the topics we discussed covered a wide spectrum ranging from strategic to hard-core technical

Read the full article on ZDNet