Qubole

Qubole operates as a cloud-native open data lake platform, providing end-to-end services for machine learning, streaming analytics, and ad-hoc data analysis. Founded in 2011 by Ashish Thusoo and Joydeep Sen Sarma, the company was established to simplify large-scale data processing in the cloud. Both founders brought extensive experience from Facebook, where they were instrumental in developing the social media giant's data infrastructure. Thusoo led the data infrastructure team, and Sen Sarma co-created Apache Hive, a data warehouse software project that became an industry standard for SQL on big data. Their journey at Facebook, building one of the world's largest data processing platforms, directly informed Qubole's mission to make big data accessible and manageable for all enterprises.

The company's core offering is the Qubole Data Service (QDS), a self-service platform that automates the management of big data infrastructure on public clouds like AWS, Google Cloud, and Azure. This allows data teams to focus on business outcomes rather than platform administration. The platform serves data engineers, data scientists, and data analysts by providing a unified environment with optimized versions of open-source engines such as Apache Spark, Presto, Hive, and Airflow. Qubole's business model is subscription-based, offering various tiers including an enterprise edition and on-demand packages, which are often facilitated through cloud marketplaces like AWS Marketplace. The company's revenue is generated by providing these managed services to a client roster that has included prominent names like Expedia, Disney, Oracle, and Adobe.

Qubole's platform is designed to reduce the complexity and cost associated with big data workloads. It achieves this through features like workload-aware autoscaling, which automatically adjusts cloud resource usage to meet demand, potentially lowering cloud data lake costs by over 50 percent. The service provides a collaborative environment with built-in notebooks for various data professionals and supports a wide range of data processing needs, from building streaming data pipelines to turning machine learning models into products. A key aspect of the platform is its open nature, which prevents vendor lock-in by allowing clients to use their preferred cloud and open-source tools. In October 2020, Qubole was acquired by Idera, Inc., and was integrated into Idera's Database Tools business unit to expand its portfolio into data lake and data warehouse technologies.

Keywords: data lake platform, big data analytics, machine learning, streaming analytics, cloud-native, Apache Spark, Apache Hive, Presto, data engineering, data infrastructure