Big Data Cloud
Cloud computing has become a viable, mainstream solution for data processing, storage and distribution, but moving large amounts of data in and out of the cloud presented an insurmountable challenge for organizations with terabytes of digital content. A pioneer in the enablement of high-speed data-intensive workflows throughout the enterprise, Aspera has now unlocked the cloud for big data with its industry-leading high-speed transport solutions.
The Cloud Promise
Cloud computing promises on demand, scalable, pay-as-you-go compute and storage capacity. Compared to an in-house datacenter, the cloud eliminates large upfront IT investments, lets businesses easily scale out infrastructure, while paying only for the capacity they use. It's no wonder cloud adoption is accelerating – the amount of data stored in Amazon Web Services (AWS) S3 cloud storage has jumped from 262 billion objects in 2010 to over 1 trillion objects at the end of the first second of 2012.
The Big Data Challenge
Traditional WAN-based transport methods cannot move terabytes of data at the speed dictated by businesses; they use a fraction of available bandwidth and achieve transfer speeds that are unsuitable for such volumes, introducing unacceptable delays in moving data into, out of, and within the cloud.
The Aspera Solution
Built on our patented fasp™ transfer technology, Aspera On Demand delivers scale-out transfer capacity, enabling efficient, large-scale workflows, with enterprise-grade security, a variety of client options (web, mobile, embedded), and applications for ingest, sharing, collaboration and exchange of big data, available on demand as a subscription service.
Delivering on the Promise
Aspera has solved the big data challenge. Our customers and partners, such as Netflix, Zencoder, SendToNews and others are now leveraging Aspera On Demand with Amazon AWS to meet their IT infrastructure needs, so they can focus their resources on the core functions of their business.
Cloud computing holds a tremendous promise of unlimited, on demand, elastic, computing and data storage resources, without the large upfront investments required when deploying traditional data centers. The cloud eliminates the risk of upfront IT investments and allows businesses to scale out and back by quickly adding or removing resources while paying only for the capacity they use.
From a business perspective, the cloud offers three key advantages:
- Removing computing/storage infrastructure as a limiting factor in meeting un-anticipated demand.
- Eliminating the need to build IT infrastructures that can handle spikes in activity only to sit idle most of the time.
- Reducing the risk of upfront investment and improving cash flow through pay- as-you-go models, charging only for the resources that are actually used.
Cloud adoption by big data businesses has been limited because of the problem of moving their data into and out of the cloud. Often dealing with data sets measuring in tens of terabytes, they have had to rely on traditional means for moving big data:
- Ship hard disk drives to a cloud provider and hope that they don’t get delayed, damaged or lost.
- Attempt to transfer the data via the web, using TCP-based transfer methods such as FTP or HTTP.
Neither option is practical and both impose costs on the business through delayed access to data, risks of loss or damage, and the need to invest resources into maintaining an infrastructure that can support it – either shipping operations or over-engineered IT systems.
To become a viable option for big data management, processing and distribution, cloud services need a high-speed transport mechanism that addresses two main bottlenecks: the degradation in WAN transfer speeds that occurs over distance using traditional transfer protocols, and the “last foot” bottleneck inside the cloud datacenter caused by the HTTP interfaces to the underlying object-based cloud storage.
Built on top of our patented fasp™ transport technology, Aspera’s suite of On Demand Transfer Products solves both technical problems of the WAN and the cloud I/O bottleneck, delivering unrivaled performance for the transfer of large files, or large collections of files, in and out of the cloud.
- Transfers occur at line speed, optimizing the intra-cloud data transfer, delivering consistently high speeds along the entire data path.
- Files of any size and any format can be transferred at any distance, over any network, under any condition.
- Transfer capacity can easily scale-out and back, on demand.
- Offers full support for all Aspera software and use cases.
Users have extraordinary control over individual transfer rates and bandwidth sharing, as well as full visibility into bandwidth utilization. File transfer times can be guaranteed, regardless of the network distance and conditions, including transfers over satellite, wireless, and unreliable long-distance international links. Complete security is built-in, including secure endpoint authentication, on-the-fly data encryption, and integrity verification.
For users of Amazon's S3 cloud storage, Aspera has developed a high-speed software bridge, Direct-to-S3, which transfers data at line speed, from source directly into AWS S3, with no hops or stops in between:
- Enables direct I/O in and out of Amazon S3 storage.
- Ensures intra-cloud I/O keeps up with the fasp™-based transport over the WAN.
- Transparently handles cloud-specific I/O requirements such as S3 multi-part uploads.