The Application Development Framework in CDAP lets Developers build entire Applications for Big Data. But perhaps the most exciting development here at Cask is our new plugin architecture, which has allowed us to build a Drag ‘n Drop UI for Apache Hadoop and Apache Spark. Your Spark or Hadoop code becomes a plugin – … Read more
Incremental data processing is a common challenge that is tackled in data lakes across several organizations. For example, suppose a user has a continual stream of data arriving into their data lake and they want to perform some validity check on their data set to filter out invalid records. With data in motion, it is … Read more
The Cask Data Application Platform is an integrated developer platform for the Hadoop ecosystem. With CDAP, developers can address a broader set of batch and real-time use-cases with easy-to-use abstractions. Developers can write MapReduce programs using CDAP and deploy them as CDAP applications easily, as explained in this guide. Running MapReduce programs inside CDAP has … Read more
In CDAP, a MapReduce program can interact with a CDAP dataset by using it as an input or an output. Before CDAP 3.2.0 users could only have single dataset as the output of a MapReduce job. We wanted to extend this capability and allow writing to multiple datasets from MapReduce jobs to support the following … Read more
The Cask Data Application Platform (CDAP) is an open-source platform to build and deploy data applications on Apache Hadoop™. In a previous blog post we introduced Workflows, a core component of CDAP, in comparison with Apache Oozie. In this post we will discuss the CDAP Workflow engine in greater detail. CDAP Workflows are used to … Read more
We are excited to announce the Cask Data Application Platform (CDAP) 3.2 release. This release brings many enhancements to existing CDAP features as well as lays the foundation for upcoming, advanced features—all designed to further simplify data application development. Cask Hydrator CDAP 3.2 introduces Cask Hydrator—a highly functional framework and UI to support self-service batch … Read more
Java class loading is one of the most fundamental and powerful concepts provided by the Java Platform. Understanding the class loading mechanism helps you when designing and building extensible application frameworks. You can also avoid spending many hours in debugging exceptions such as ClassCastException and ClassNotFoundException, among others. In this post, we will talk about … Read more