And, while public sector data, financial records, customer information and confidential business reports were previously stored in secure facilities where physical access was restricted and closely monitored, in the digital age, vast amounts of data are continuously on the move.

These changes have seen the evolution of a security industry focused on ensuring confidential information is secure and can only be accessed by the people and systems authorised to do so.

Businesses are also realising and tapping into the value of the data they hold, with high-performance data processing driving whole new business models. With business data becoming more and more critical to operations, suddenly losing access to it would make it extremely difficult for many businesses to continue their activities.

The security challenges of high-performance workloads

High-performance computing is growing all the time, with the amount of data being extracted, transferred and stored increasing exponentially.

In addition to the sheer volume of data, high-performance workloads are complex, requiring a large number of nodes, distributed systems, and optimised configurations, which are tuned and tested for the specific business.

They must also deal with the added complexity that comes when data includes health records or financial information over different geographic regions, where local laws restrict the movement of certain types of information.

Add into the mix that these capabilities are increasingly provided in the cloud, and there are a number of extra considerations.

An expanded attack surface

With data being transferred to and from the cloud, as well as being processed and stored beyond the confines of organisation’s own IT infrastructure, there are more channels that malicious actors could use to intercept, disrupt or steal data.

Cloud providers’ data centres, which contain server hardware, network switches and IT tools, further expand this attack surface.

OVHcloud, for example, provides millions of public IP addresses across five continents and faces a constant battle to thwart attacks from hackers and criminal organisations hoping to profit from data theft, fraud and compromised systems.

This larger attack surface means cloud-based data processing is more attractive to malicious individuals or groups conducting distributed denial of service attacks (DDoS), where they attempt to overload an application or network with too much information.

High-performance workloads require high availability and lots of compute power by definition, so can be particularly impacted by these types of attacks. Not only can DDoS attacked overwhelm servers, they can also affect the general speed of traffic on the network — another issue that could particularly impact high-performance workloads.

Protecting high-performance workloads

Malicious actors present a constant threat to customer data, and so cloud providers of high-performance capabilities must go above and beyond when it comes to protecting the information on their networks.

They need to follow security best practice, use the most reliable and tested technology, and abide by regulations set out by governments and regulators.

Decentralising the computing and running high-performance workloads on numerous servers at the same time, means a successful attack on a server is less likely to affect the high-performance environment.

It’s also important for organisations to put disaster recovery and business continuity capabilities in place, so mission-critical workloads can continue to safely and securely operate even if a security issue occurs.

Being aware of the combined security implications of high-performance workloads and the cloud, means organisations can plan the right approach and select the cloud provider that will best manage their risks.

AI, for example, is being used in applications that recognise human emotions or mimic human conversation. Some multinationals, meanwhile, are developing applications to automate back-office processes, power intelligent chatbots for customer service, or that can scan millions of information-heavy documents to find insights.

What unites these technologies is that they involve large amounts of data that must be rapidly and reliably processed. This has created an increased need for businesses to have access to high-performance workload capabilities.

One piece of research suggests the high-performance computing market will grow at a compound annual growth rate of 5.5 per cent between 2020 and 2025 to reach $49.4 billion.

However, many organisations struggle to overcome resourcing issues that can either prevent these capabilities being put in place or stop companies from exploiting them to their full potential.

The resource question

One issue is around cost. The hardware, software and networks aren’t the standard kit being used for other business operations, but specialist products that come with a premium price tag.

In addition, many organisations lack the time to get these capabilities up and running. This is particularly true when you consider the additional burden placed on IT departments due to the Covid-19 pandemic, during which they’ve had to support an increase in remote working and manage the increased risk associated with shadow IT.

High-performance workloads are complex, requiring a large number of nodes, distributed systems and data. While this may all sound quite daunting, it doesn’t have to be — particularly with the right approach and service provider.

To gain high-performance capabilities, businesses don’t have to go all-in — they can start small, both in terms of cost and capabilities, before expanding their usage.

How cloud helps out

One major reason for this is the fact that high-performance capabilities can now be accessed via the cloud. There is no need to invest in expensive and complicated on-premises equipment — this will all be provided by the service provider. No additional space needs to be found and there is no need to have IT staff who are dedicated to the upkeep of the infrastructure.

Service providers will also be able to provide the networking capabilities and tools needed to keep the high-performance capabilities — which are often business critical — available at all times and can support large amounts of data securely.

Even better, businesses only need to pay for the capabilities they use — there is no need to pay for unused capacity on the off chance it could be needed in the future. A business can experiment with small implementations before deciding whether they want to up their capabilities, meaning little money is wasted and there is minimal risk.

A service provider with the answers

In addition to the benefits that cloud offers, service providers can also address other resource challenges around high-performance workloads.

When it comes to pricing, transparency is key. It’s a particularly big advantage for companies that have limited budgets as they are able to have predictable costs, even if they have an unexpected spike in traffic or need to restore a large backup.

Another advantage is having the ability to take on dedicated high-performance servers individually, which they can build on to eventually create a cluster, or clusters. This flexibility to start small really helps businesses dip their toe in the water with high-performance workloads, before embarking on anything more ambitious.

This makes it easy for organisations to gain the capacity they need — so business requirements are fully met without resources being wasted or overcommitted. OVHcloud, for example, offers several storage configuration options so organisations can choose the right disk configuration based on their needs.

Conclusion

With the ability of cloud technology to support data-intensive workloads, and service providers that address the other resource issues, there is no need for businesses to think high-performance capabilities aren’t a realistic option.

They can start playing with small stakes before going big — all with minimal risk. If the new capabilities fulfil the business need, they can be retained or scaled up. And if capabilities fail to meet business needs, businesses can stop using the services without significant financial cost.

In this way, your business can enjoy the benefits of high-performance workloads without putting a strain on resources, opening up a whole new world of possibilities.

According to IDC, more than 55 billion devices, including industrial IoT applications and sensors, will generate more than 73ZB of data by 2025, up from 18ZB in 2019. The analyst firm also expects worldwide storage capacity to increase from 6.8ZB in 2020 to 8.9ZB by 2024.

With the volume of data growing exponentially and technologies improving quickly to handle that growth, the latest and most cutting-edge computing hardware doesn’t come cheap. As a result, organisations need to evolve their procurement practices to better serve customers, improve their bottom line and remain competitive.

Traditionally, when an organisation is planning an update to its IT infrastructure, it sets a budget and finds the solution to fulfil its technical requirements at the right price. Technology upgrades, meanwhile, are often seen as an opportunity to cut costs rather than enhance profitability.

By focusing on the pure dollar amount of a project, organisations are missing the potential competitive advantage that could be gained by employing high-performance capabilities in to rapidly transform raw data into business insights.

Can you afford to?

Cloud technology is well established but high-performance capabilities add a whole new dimension by creating a wave of opportunity for innovation, especially around workloads that contain enormous amounts of data and employ machine learning, pattern recognition and AI.

High-performance systems are built from components that are orders of magnitude faster than standard technologies and can provide a level of performance that until recently was only available to large institutions and government departments with deep pockets.

However, over the past few years, advances in technology have resulted in high-performance capabilities becoming more accessible than ever before, creating an opportunity for organisations that are able to break from the traditional IT procurement mindset.

Can you afford not to?

Instead of looking at IT plans and considering the technologies required to keep doing the same thing, innovative organisations are imagining how much more competitive, efficient and profitable they could be if they were able to run high-performance workloads to create a performance gap between themselves and the competition.

High-performance products sell at a premium compared to standard cloud infrastructures but the additional costs can be recouped with improved efficiencies, scalability and competitiveness.

The next wave of disruptors will use high-performance capabilities to rapidly process, organise and analyse their data to create innovative products, fight off competition and create completely new markets.

According to a recent study by Mordor Intelligence the value of the global high-performance computing industry will increase from $24 billion in 2020 to over $42 billion by 2026.

High-performance systems are quickly becoming the standard for big data workloads, meaning that any organisation wanting to remain competitive in this area will have to determine how high-performance workloads can be integrated into its infrastructure.

Why high-performance capabilities make sense

Choosing to implement high-performance capabilities is a complex task that requires technical expertise and a solid business plan.

Organisations need to identify the most suitable processor, bandwidth and memory for their workloads. Do they have the right security, will their systems be interoperable, and do they really know exactly how much the whole thing will cost?

Some companies may find it counterintuitive to be spending vast sums of money on cloud-based high-performance capabilities when a regular cloud infrastructure could do a similar job but slower. If their workloads are not time critical then maybe spending on premium technology can’t and shouldn’t be justified.  But if workloads are time critical, if you deal in real time data, if AI and machine-learning systems are required to analyse and manipulate data to generate critical information, a change of mindset is all you need to access the high-performance capabilities your business needs.

In parallel with this evolution, more and more uses cases involving publicly sharing data have arisen. The amount of data has exploded, in almost all applications, and IT people don’t want to manage storage anymore (maintenance, failure, scalling…).

As now, all the major languages have their object storage libraries (at least for S3 APIs), and tones of applications integrate by default the connectors to those services. And as many IT people, I switched my reflexes to think Object Storage first, and handle some exceptions where network file systems are a better solution.

When applications need storage performances…

During many years, performances were not the main focus. The goal was to take advantage of unlimited space and public access to store images, documents, code…

Nowadays, part of the applications need more speed because the objects are bigger and plentiful. That’s the case for Big Data use cases and IA scenarios but also for the majority of modern applications.

• Those who work on big data analysis know that you can wait hours before getting your result in many cases. Working on data to analyze users behavior, system deviation, cost optimization requires powered compute resources but also high performance storage. Big Data clusters needs to access the date quickly to provide a result in time.
• Training IA models and fine tuning the predictions requires a lot of data analysis also that takes hours and slow down your progress if your storage solution is the bottleneck.
• Serving media that are bigger and bigger (4K video, HD pictures on smartphones, …) on high speed network (fiber, 5G) challenge the previous solutions.

OVHcloud Standard Object Storage hosts petabytes of data across the world on a strong and resilient infrastructure but our classical offer was not design to address those use cases. The performances of our classical object storage where not enough in those situations.

A new Object Storage service for demanding applications

So what is high performances for an object storage service? What are we speaking about?

There are mainly two things to consider when you need speed on your objects. The first one is the API. This is the first component that answers the requests you’ll send to the cluster, so the simplest API call should react as a flash. That’s the assurance that all the engine and automation tools on the cluster are optimized and not consuming precious milliseconds. The second one is the bandwidth. You’ll expect that now your object is identified, it should be delivered to you as soon as possible using the maximum capacity of disks and network.

So we are excited to provide OVHcloud High Performance Object Storage that fit those requirements in the upcoming weeks. The general availability should be the 21rst February at first in Strasbourg, then mid-March in Gravelines.

Some numbers

We’ll make some tests and compare that new service with the existing service at OVHcloud and the market leader on Object Storage, AWS S3. We’ll do those tests from OVHcloud network on a Public Cloud instance B2-120 which have a 10G public interface. This instance is located in Gravelines (north of France). The three tested services are located in central Europe, Strasbourg (France) for OVHcloud ones and Frankfurt (Germany, closed to Strasbourg) for AWS. The distance between the instance and the cluster are almost the same.

Time-to-First-Byte test

So what is the API efficiency on that new service? Here we want to compare the Time-to-First-Byte which give us an idea how the cluster manage to process the request, identify the object, check the right to access it and return it. So we’ll send tones of 1 Byte objects and measure the time needed to access it.

To be fare in that test, we need to take in consideration that we’ll get out to the OVHcloud network to reach the AWS cluster. Here are the ping we got :

OVHcloud instance to OVHcloud Object Storage

$ ping s3.sbg.perf.cloud.ovh.net
PING s3.sbg.perf.cloud.ovh.net (141.95.161.68) 56(84) bytes of data.
64 bytes from 141.95.161.68 (141.95.161.68): icmp_seq=1 ttl=53 time=10.2 ms

OVHcloud instance to AWS Object Storage

$ ping s3.eu-central-1.amazonaws.com
PING s3.eu-central-1.amazonaws.com (52.219.169.77) 56(84) bytes of data.
64 bytes from s3.eu-central-1.amazonaws.com (52.219.169.77): icmp_seq=1 ttl=45 time=11.0 ms

So we have almost the same latency, let’s go for the Time-to-First-Byte for each services.

OVHcloud High Perf looks a little bit better, not so far away but there is a small advantage for OVHcloud.

Small and medium files test

Now we’ll test not only the API but a normal object usage looking on files from 10KB to 10MB which represent plenty of usages, especially when it comes to serve statics on the web. That could represent code, images, sound…

As you can see, we leaved the field where the latency advantage where on OVHcloud side and we enter in the real performances using the whole chain including writing and reading on disks. Here, OVHcloud High Perf takes clearly the lead on 1MB files with around 55% faster than AWS S3 on download tests.

Large files test

When we move to 100MB files and more, we enter in a different space where the bandwidth is the main part which impact the global result. Here the data could be of course videos but also specific application files or large datasets.

Here, the upload performances are almost the same for AWS and OVHcloud but we still have a real difference on the download test with around 30% advantage in favor of OVHcloud High Perf.

Some take away about this new offer

Regarding the pure performances, OVHcloud High Perf Object Storage is a great challenger for AWS S3. The API is as fast as our competitor. We can notice a real advantage for OVHcloud on medium files (1MB to 100MB) where we go over 50% (1MB) faster than AWS on download and around 40% on upload (1MB).