Distributed R

Distributed R
Developer(s) HP
Stable release
v1.0.0 / February 17, 2015 (2015-02-17)
Development status Active
Written in C++, R
Operating system Linux
Type machine learning algorithms
License GNU General Public License
Website www.distributedr.org

Distributed R is an open source, high-performance platform for the R language. It splits tasks between multiple processing nodes to reduce execution time and analyze large data sets. Distributed R enhances R by adding distributed data structures, parallelism primitives to run functions on distributed data, a task scheduler, and multiple data loaders.[1] It is mostly used to implement distributed versions of machine learning tasks. Distributed R is written in C++ and R, and retains the familiar look and feel of R. As of February 2015, Hewlett-Packard (HP) provides enterprise support for Distributed R with proprietary additions such as a fast data loader from the Vertica database.[2]

History

Distributed R was begun in 2011 by Indrajit Roy, Shivaram Venkataraman, Alvin AuYoung, and Robert S. Schreiber as a research project at HP Labs.[3] It was open sourced in 2014 under the GPLv2 license and is available at GitHub.

In February 2015, Distributed R reached its first stable version 1.0, along with enterprise support from HP.[4]

Components

Distributed R is a platform to implement and execute distributed applications in R. The goal is to extend R for distributed computing, while retaining the simplicity and look-and-feel of R. Distributed R consists of the following components:

Integration with databases

HP Vertica provides tight integration with their database and the open source Distributed R platform. HP Vertica 7.1 includes features that enable fast, parallel loading from the Vertica database to Distribute R. This parallel Vertica loader can be more than five times (5x) faster than using traditional ODBC based connectors. The Vertica database also supports deployment of machine learning models in the database. Distributed R users can call the distributed algorithms to create machine learning models, deploy them in the Vertica database, and use the model for in-database scoring and predictions. Architectural details of the Vertica database and Distributed R integration are described in the Sigmod 2015 paper.[5]

References

  1. Venkataraman, Shivaram; Bodzsar, Erik; Roy, Indrajit; AuYoung, Alvin; Schreiber, Robert S. (2013). "Presto: Distributed Machine Learning and Graph Processing with Sparse Matrices" (PDF). European Conference on Computer Systems (EuroSys).
  2. Gagliordi, Natalie. "HP adds scale to open-source R in latest big data platform". ZDNet. Retrieved 17 February 2015.
  3. Venkataraman, Shivaram; Roy, Indrajit; AuYoung, Alvin; Schreiber, Robert S. (2012). "Using R for Iterative and Incremental Processing". Workshop on Hot Topics in Cloud Computing (HotCloud).
  4. "HP Delivers Predictive Analytics at Big Data Scale". hp.com. 17 February 2015. Retrieved 17 February 2015.
  5. Prasad, Shreya; Fard, Arash; Gupta, Vishrut; Martinez, Jorge; LeFevre, Jeff; Xu, Vincent; Hsu, Meichun; Roy, Indrajit (2015). "Enabling predictive analytics in Vertica: Fast data transfer, distributed model creation and in-database prediction". ACM SIGMOD International Conference on Management of Data (SIGMOD).

External links

This article is issued from Wikipedia - version of the 12/3/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.