hibd.cse.ohio-state.edu
High-Performance Big Data :: userguide
http://hibd.cse.ohio-state.edu/userguide
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). RDMA for Apache HBase 0.9.1 Userguide. Users can download a PDF version of this userguide HERE. RDMA for Apache Spark 0.9.1 Userguide. Users can download a PDF version of this userguide HERE. RDMA for Apache Hadoop 2.x 1.1.0 Userguide. The following user guide contains the necessary information for users to download, install, test, use, and tune RDMA for the Apache Hadoop package on variou...
hibd.cse.ohio-state.edu
High-Performance Big Data :: Sort
http://hibd.cse.ohio-state.edu/performance/inmemory_plugin
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). In HHH-M mode, we evaluate RDMA-enhanced MapReduce over RDMA-enhanced HDFS with in-memory I/O operations. For detailed configuration and setup, please refer to our userguide. OSU - RI Cluster. TestDFSIO Latency and Throughput. Contact: Dhabaleswar K. (DK) Panda. Dept of Computer Science and Engineering. Columbus, OH 43210.
hibd.cse.ohio-state.edu
High-Performance Big Data :: Sort
http://hibd.cse.ohio-state.edu/performance/withlustre_hdp
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). TestDFSIO Write and TestDFSIO Read. Each node in OSU-RI2 has two fourteen Core Xeon E5-2680v4 processors at 2.4 GHz and 512 GB main memory. The nodes support 16x PCI Express Gen3 interfaces and are equipped with Mellanox ConnectX-4 EDR HCAs with PCI Express Gen3 interfaces. The operating system used is CentOS 7. Each node in OSU-RI2 has two fourteen Core Xeon E5-2680v4 processors at 2....
hibd.cse.ohio-state.edu
High-Performance Big Data :: Sort
http://hibd.cse.ohio-state.edu/performance/mrolustre
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). MapReduce over Lustre (with local disks). MapReduce over Lustre (w/o local disks). Each compute node in this cluster has two twelve-core Intel Xeon E5-2680v3 processors, 128GB DDR4 DRAM, and 320GB of local SSD with CentOS operating system. Each node has 64GB of RAM disk capacity. The network topology in this cluster is 56Gbps FDR InfiniBand with rack-level full bisection bandwidth ...Each ...
hibd.cse.ohio-state.edu
High-Performance Big Data :: Sort
http://hibd.cse.ohio-state.edu/performance/withlustre
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). TestDFSIO Write and TestDFSIO Read. Each compute node in this cluster has two twelve-core Intel Xeon E5-2680v3 processors, 128GB DDR4 DRAM, and 320GB of local SSD with CentOS operating system. Each node has 64GB of RAM disk capacity. The network topology in this cluster is 56Gbps FDR InfiniBand with rack-level full bisection bandwidth and 4:1 oversubscription cross-rack bandwidth. These ex...
hibd.cse.ohio-state.edu
High-Performance Big Data :: dfs
http://hibd.cse.ohio-state.edu/performance/testdfsio
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). OSU - RI Cluster. TestDFSIO Write Latency on HDD. TestDFSIO Write Throughput on HDD. These experiments are performed in 8 DataNodes with a total of 32 maps. Each DataNode has a single 1TB HDD. The NameNode runs in a different node of the Hadoop cluster and the benchmark is run in the NameNode. TestDFSIO Write Latency on SSD. TestDFSIO Write Throughput on SSD. Each node of our testbed has t...
hibd.cse.ohio-state.edu
High-Performance Big Data :: Sort
http://hibd.cse.ohio-state.edu/performance/standalone_plugin
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). OSU - RI Cluster. TestDFSIO Latency and Throughput. Each node in OSU-RI has two 4-core 2.53 GHz Intel Xeon E5630 (Westmere) processors and 24 GB main memory. The nodes support 16x PCI Express Gen2 interfaces and are equipped with Mellanox ConnectX QDR HCAs with PCI Express Gen2 interfaces. The operating system used was RedHat Enterprise Linux Server release 6.4 (Santiago). Each node in OSU...
hibd.cse.ohio-state.edu
High-Performance Big Data :: Sort
http://hibd.cse.ohio-state.edu/performance/mrolustre_hdp
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). MapReduce over Lustre (with local disks). Each node in OSU-RI2 has two fourteen Core Xeon E5-2680v4 processors at 2.4 GHz and 512 GB main memory. The nodes support 16x PCI Express Gen3 interfaces and are equipped with Mellanox ConnectX-4 EDR HCAs with PCI Express Gen3 interfaces. The operating system used is CentOS 7. MapReduce over Lustre (w/o local disks). Columbus, OH 43210.
hibd.cse.ohio-state.edu
High-Performance Big Data :: features
http://hibd.cse.ohio-state.edu/features
Apache Hadoop 2.X Integrated/Plugin. HDP Hadoop 2.X Plugin. CDH Hadoop 2.X Plugin. High-Performance Big Data (HiBD). RDMA-Apache-HBase 0.9.1 Features. Based on Apache HBase 1.1.2. High performance design with native InfiniBand and RoCE support at the verbs-level for Apache HBase. Compliant with Apache HBase 1.1.2 APIs and applications. Easily configurable for native InfiniBand, RoCE, and the traditional sockets based support (Ethernet and InfiniBand with IPoIB). RoCE support with Mellanox adapters. High ...