Big Data Networking

 

Explosive data growth is a reality and the trajectory is continuing to be strong. In order to accommodate and support this level of intensification, more robust and powerful networks are becoming more important than ever before. Data generation and the diversification of data use drive the adoption of more role-based storage solutions within the data center. These factors, coupled with the transition to highly virtualized data center environments, affects how organizations buy and manage server, storage, and network assets and are key drivers in what is propelling Big Data into an everyday reality. The outlook is Big Data in the Cloud.

 

 

Big Data is comprised of datasets that grow so large that they become cumbersome to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analytics, and visualizing. The growth trend continues because of the benefits of working with larger and larger datasets which allows analysts to discover business trends and solve problems. Though a moving target, current limits are on the order of terabytes, exabytes and zetabytes of data.
There are various horizontals where Big Data is applicable, such as meteorology, genomics, biological research, Internet search, finance, and business informatics. Arista is a strong player in this arena and many times the incumbent in all of these horizontals providing great differentiation. One current difficulty of working with Big Data is the use of relational databases and desktop statistics/visualization packages which require massively parallel software running on tens, hundreds, or even thousands of servers. This notion directly ties in to the role of networking and cloud storage, and to the advantages of the Arista’s offering in the form of high density, non-blocking, highly scalable networks in a small footprint.
Data is everywhere, whether it is from users, applications, or machines and it’s growing exponentially with no vertical or industry being spared. Due to this reality, IT organizations everywhere are forced to come to grips with storing, managing and extracting value from every piece of it -– as inexpensively as possible. This begins the real race to cloud computing where the framework needs the ability to process data increasingly in real-time and in greater orders of magnitude -– and at a fraction of what it would typically cost.

 

Application Drivers of Storage Networking

 

There are many key application drivers related to storage networking. The ones that rise to the top of the list are in the Cloud Computing, Hadoop Cluster, Storage Virtualization, High Performance Computing, and Rich Content/Video arenas. Furthermore, the rise of new web-based application architectures in the data center, the increasing use of virtualization tools to consolidate servers and the utilization HPC in core mission-critical applications all drive the need for a high performance, low latency, and highly available networks for Big Data.

The key players driving Big Data are predominantly Internet service providers, application service providers, and storage service providers. These players are among those who need 10 Gigabit Ethernet (GbE) bandwidth at the core of the network to support
applications as well as for remote replication and disk-to-disk backup. Also guaranteed to benefit from 10 GbE are applications such as video editing and rich content applications that consume or generate huge amounts of data in a short time.

 

Hadoop Clusters

 

Data analytics has become a key element of the business decision process over the last decade. Classic reporting on a dataset stored in a database was sufficient until recently, but yesterday’s data gathering and mining techniques are no longer a match for the amount of unstructured data and the time demands required to make it useful. The common limitations for such analysis are compute and storage resources required to obtain the results in a timely manner. A network that is designed for Hadoop applications, rather than standard enterprise applications, can make a big difference in the performance of the Hadoop cluster. Arista Networks satisfies Hadoop cluster network requirements has been successfully deployed in many Hadoop environments. As
costs fall and companies think of new ways to correlate and analyze data, Big Data analytics will become more common. Businesses will especially benefit given their low-cost ability to manage and analyze Big Data.

Hadoop is a very powerful distributed computational framework that can process a wide range of datasets, from a few gigabytes to petabytes of structured and unstructured data. Use of Hadoop has quickly gained momentum, and Hadoop is now the preferred platform for various scientific computational and business analytics. While availability of commodity Linux based servers makes it feasible to build very large clusters, the network is often the bottleneck, resulting in congestion and less efficient use of the cluster.

Hadoop consists of two key services: reliable data storage using the Hadoop Distributed File System (HDFS) and high-performance parallel data processing using MapReduce. MapReduce is essentially the engine that brings speed and agility to the Hadoop platform. With MapReduce, developers can create programs that process massive amounts of unstructured data in parallel across a distributed cluster of processors or stand-alone computers. Hadoop allows enterprises to easily explore this complex data using custom analyses tailored to their information and questions. Big Data problem is not just all about size of the data; it is also about performance and how fast the data can be processed.

Arista offers high performance 1/10 GbE non-blocking, ultra-low latency solutions that can scale from a few racks to some of the largest Hadoop deployments. In addition, Multi-Chassis LAG (MLAG) offers true active/active uplink connectivity from each rack, allowing the full bi-sectional bandwidth of the network to be utilized in a flat layer 2 network. Arista’s Extensible Operating System can easily be integrated with Ganglia and Nagios. Lastly, Arista’s networking solutions offer a true flat-line growth when it comes to price for server-interconnect bandwidth. These factors make Arista’s networking solutions ideal for any Hadoop deployment.

 

High Performance Computing (HPC)

 

High performance computing involves the use of supercomputers or clusters of powerful processors to solve computationally intensive problems. In order for a large number of processors to work together, clusters require interconnects that support highbandwidth
and very low latency communication. Arista Networks has the lowest latency switch in the market today which is used for HPC cluster interconnects to process large amounts of data in such industries as meteorology, genomics,oil and gas, biological research or any other type of business that needs to crunch large amounts of data.

 

Storage Virtualization

 

Virtualization is a requirement for cloud computing as virtual nodes only need a subset of the performance of modern CPUs. As aggressive as this convergence to virtual storage has been, it has also come with some disruptions. For one, it has equated to an over-provisioning of storage and network capacity. Furthermore, there has been an excess of Fibre Channel (FC) storage and Ethernet data network facilities and the propagation of new storage network technologies such as iSCSI. iSCSI over Ethernet is an essentially ideal environment for virtualization due to the fact that it allows administrators to move virtual servers among physical machines without reconfiguring the zoning and logical unit number (LUN) masking in the storage network. Together, these problems can rapidly eliminate any hopes for benefits as the scope of virtual server deployment expands unless the network can play a thriving role in making this transformation a success. Arista’s 10 Gigabit Ethernet solutions alleviate these disruptions and ensure the full benefits of storage virtualization are met and realized.

 

Cloud Computing

 

Cloud computing helps organizations store, manage, share, and analyze their Big Data in a reasonable and simple-to-use way. Today’s cloud Infrastructure-as-a-Service (IaaS) providers, supported by on-demand analytics solution vendors, make Big Data analytics very affordable. As location-independent computing entails shared servers providing resources, software, and data to systems and devices on demand, cloud computing is a very strong use driver for high performance storage networking. Widespread adoption of virtualization and utility computing has caused this natural evolution and has resulted in customers no longer needing expertise or control over the technology infrastructure that supports them. However, in order for those customers to achieve the best performance and service possible, there needs to be robust and high performance networking in place to support it such as what Arista Networks provides with their entire product line of switching solutions.

 

Rich Content / Video

 

Arista Networks is a key player in delivering a fully redundant, cloud based content origination service that hosts all video and high resolution photo content. Arista Networks provides the Ethernet network to interconnect the streaming servers which require highperformance 1/10GbE switching with very large packet buffering. This is perfectly suited to support high-bandwidth video streaming and storage interconnection while maintaining the highest levels of system availability with both device and system level resiliency designed into the rich content / video infrastructure. Additionally, it is often the data that needs to be aggregated, and analyzed for business intelligence as well as for post-production. As reliability and resilience are important, the Arista Extensible Operating System [EOS] with In-Service Software Upgrades and Stateful Fault Repair prove to be invaluable in maintaining service level agreements and overall uptime.