Moving to PaaS and SaaS on the FPGA world
Cloud computing is the on-demand availability of computer system resources, such as computing power, without direct active management by the user. Cloud computing has enabled companies to deploy their applications in a scalable way minimizing up-front IT infrastructure costs and preventing the underutilization of the available resources. It has enabled enterprises to have access to the exact computing resources as their applications require.
Based on the different requirements, cloud providers provide different options regarding the exposure of the computing resources to the end user; Infrastructure-as-a-Service (IaaS), Platform-as-a-Service or Software-as-a-Service.
- IaaS gives users access to on-premise infrastructure, so users can avoid investing in expensive on-site resources
- PaaS provides users hardware and software tools that allows users to deploy their applications on top of the available platforms
- SaaS make software directly available to users to run their applications.
As the performance of typical general purpose processors does not keep increasing as it used to be due to the end of Moore’s and Dennard’s law, data center have stated deploying alternative computing resources that can provide increased performance, lower latency and better energy efficiency.
FPGA are programmable platforms that can be configured with customized architectures optimized for specific applications. These tailored-made architectures can execute specific applications much faster compared to typical general-purpose processors. In the last years, FPGAs have been evolved from simple programmable logic gates to powerful heterogeneous platforms with specialized modules for machine learning, DSP, networking, and data processing.
Cloud providers like AWS, Huawei, Baidu, and Tencent has started providing FPGAs as a computing resource in their cloud in the last years. However, current cloud providers offer the FPGAs only as IaaS and actually without the virtualization/sharing of the resources as in the typical case of general purpose processors (CPUs).
Cloud providers offer only the FPGAs as infrastructure making hard for the software community to easily deploy their applications. For example, in case that the application needs to be deployed in 8 FPGAs, software developers will have to manually distribute the workload to the available FPGAs. Similarly, if multiple users or applications wants to have access to a cluster of FPGAs, software developers will have to develop their own solution to prevent the conflict of the resources or the serialization of the access to the FPGA resources.
FPGAs offer significant advantages compared with other computing platform like increase performance, lower latency and better energy efficiency. However, the widespread adoption by the software community required the availability of the frameworks that will allow easy deployment. The required frameworks are necessary that will allow software developers to deploy FPGAs in the same way as any computing platform.
Having in mind these issues, InAccel developed the unique FPGA orchestrator that allows easy deployment, instant scaling and seamless resource management of the FPGA clusters. InAccel’s FPGA orchestrator allows the deployment of FPGAs as a Platform ready to be used by the software developers just like any other computing system. At the same time, InAccel’s orchestrator integrated with InAccel’s FPGA repository allows the utilization of FPGAs as a service using off-the-shelf ready to use accelerators.
PaaS using FPGA orchestrator: easy deployment, scaling and resource management
InAccel FPGA orchestrator abstracts away the available FPGA resources serving as an OS for the application that need to be deployed on FPGAs. One of the main advantages of InAccel’s orchestrator is that it allows multiple applications or processes to share the available resources without the user having to do manually the contention/conflict management. InAccel’s orchestrator serves as the scheduler that allows the sharing of the available FPGA resources in an FPGA cluster from multiple applications, multiple processes or multiple users-tenants.
InAccel’s unique approach of an FPGA orchestrator as a middleware, allows the easy deployment of FPGAs by software developers. An FPGA orchestrator is used to schedule the function callings to the available FPGA resources. That methodology allows easy scale-out of an application to multiple FPGAs in the same server without the user having to manually define which function will be executed on which FPGA. As it is done in the software world where the users do not specify in which core the application will be executed, similarly InAccel manager/orchestrator automatically dispatch the functions to the available resources in the FPGA cluster. Also, it automatically configures and initialize the FPGAs with the right bitstream making easier than ever the deployment of FPGA clusters from a software developer. Actually, it serves like a Kubernetes for multiple FPGAs n the same servers. Combining the FPGA manager with InAccel’s Kubernetes plug-in it allows to scale-out also to multiple servers with multiple FPGAs making it ready for enterprise-grade deployment.
SaaS using FPGA library-repository
InAccel main innovation is the decoupling of FPGA binary files (ready-to-use bitstreams) from the software code. As it is common in many software languages like python and Java, the use of ready-to-use libraries for the most widely used function is of paramount importance for the adoption and utilization of the programming language. Therefore, the use of pre-compiled binary files (bitstreams) and the deployment of an FPGA repository that hosts many ready-to-use binaries for several FPGA cards allows the easy utilization of FGPAs from the software community. The FPGA repository expose to the software developers only the required information for the invoking of the accelerators as it is done in any software library (i.e. name of function, input and output arguments).
FPGA vendors, like Xilinx, offer open-source accelerators like the Vitis libraries for widely used applications (compression, encryption, quantitative finance, linear algebra vision, etc.). Using InAccel’s repository for these applications, gives the opportunity to the software developers to instantly utilize FPGAs without being exposed to the time-consuming process of synthesis, place and route that can take several hours. Therefore, it allows software developers to speedup their applications using FPGAs and deploy FPGAs as SaaS.
FPGA offer a unique combination of advantages: high-throughput, low latency and energy efficiency. Using OpenCL and C/C++ is now easier than ever to create your own accelerator. However, in order to allow the widespread adoption we must offer the frameworks that will allow easy deployment in the same way that is done in computing resources like CPUs and GPUs. Inaccel’s unique FPGA orchestrator provides the abstraction layer that servers as an OS for an FPGA cluster and allows easy deployment, instant scaling and seamless resource management for easier integration with the software stack.
You can try it for free on http://inaccel.com