Personal computers
In the early 1980s, IBM, after introduced the personal computer, quickly understood the need for an operating system. Gates had suggested that IBM investigate an OS called CP/M (Control Program for Microcomputers), written by Gary Kildall of Digital Research. Back then, Digital Research was the pioneer on the development of an operating system for a personal computer. After a series of events, IBM adopted MS-DOS to allow easy deployment of the personal computers. Similarly, other companies understood the need for a vendor agnostic Operating system that could be deployed on any processor or PC. This is how Linux, windows and Mac OS prevailed in the domain of operating systems.
Mobile phones
In the early 2000s, each mobile vendor had it’s own operating system (OS). Symbian was used for the Nokia phones, Blackberry had it’s own OS, and Palm had also it’s own OS. As the market was quite fragmented, Android Inc. was founded in Palo Alto, California, in October 2003 by Andy Rubin, Rich Miner, Nick Sears, and Chris White. Rubin described the Android project as 'tremendous potential in developing smarter mobile devices”. In July 2005, Google understood the important of a vendor agnostic OS for mobile phone and acquired Android Inc. for at least $50 million.
Data Centers
In the early 2000s, VMware introduced the world to virtual servers that allowed IT to make more efficient use of idle server capacity. Virtualization is the process of creating a software-based, or virtual, representation of something, such as virtual applications, servers, storage and networks. It is the single most effective way to reduce IT expenses while boosting efficiency and agility for all size businesses. Virtualization allows to run multiple operating systems on one physical machine. It also allows to divide system resources between virtual machines. It also provides Provide fault and security isolation at the hardware level and helps reduce the OpEx in data centers.
Cloud deployment
When multi-core processors arrived and data centers started to increase rapidly, software developers wanted an easy way for scalable deployment of their application. MapReduce and other frameworks like Mesos helped towards an easy and scalable deployment of software applications.
FPGA accelerators
Now it seems that time has come for the acceleration cards and specifically for FPGAs. In the last years, more and more cloud vendors have started providing FPGAs as an alternative instances that can provide major advantages like high-throughput, low-latency, deterministic execution time and energy-efficiency that is translated to OpEx reduction.
FPGAs can offer significant speedups in many applications, but they lack frameworks for easy deployment and scaling. In the CPU-world, software community has moved from a single node deployment era to virtualized deployment era and most recently to multi-tenant, multi-server deployment using Kubernetes.
In the past programming FPGAs was challenging, but now software programmers can develop their own accelerators written in OpenCL or HLS C/C++. However, still the easy deployment remains a challenge as programmers need to be familiar with the FPGA programming paradigm and there is a lack of an abstraction layer that will allow easy and automated deployment, resource management and scaling.
FPGA vendors like Intel and Xilinx already provide several open source accelerators for applications like quantitative finance, databases, genomics, computer vision, machine learning, DNN, compression and encryption. But usually, software developers need an abstraction layer that will allow easy deployment, multi-tenant deployment and easy scaling to multiple node (e.g. FPGA cards).
If we want to make the power of FPGA-based accelerators associable to any software developer, machine learning engineer and data scientist, then a vendor-agnostic OS layer and a Kubernetes-layer is required. This abstraction will allow everyone to enjoy the power of FPGA-based acceleration with minimum effort.
InAccel FPGA orchestrator abstracts away the available FPGA resources serving as an OS layer and “kubernetes-alike” layer for the applications that need to be deployed on FPGAs. The main advantages that InAccel’s orchestrator offers are:
- Easy deployment: Software developers can invoke the functions that want to accelerate in the same way that it is done on any other software framework. Software developers do no need to worry about the buffer management, the configuration file of the FPGAs, etc. InAccel’s unique orchestrator overloads the specific functions that need to be offloaded to the FPGAs. Users can use accelerators in the same way as it would have been done using software libraries. Software developers can deploy their applications using simple C/C++, Python and Java API calls and not OpenCL. They can also utilize accelerators from high level framework like Spark and Jupyter.
- Seamless resource management: Multiple applications or users or processes can share the available resources without the user having to do manually the contention/conflict management. InAccel’s orchestrator serves as the scheduler that allows the sharing of the available FPGA resources in an FPGA cluster from multiple applications, multiple processes or multiple users-tenants.
- Instant Scaling: Users just invoke as many times as they need the function that they want to accelerate without having to manually distributed the functions to the available resources. Whether there are many kernels on a single FPGA or multiple FPGAs on a single server, InAccel’s Coral Orchestrator performance the load-balancing and the distribution of the functions that need to be accelerated to the available resources. InAccel also provides the Kubernetes plugin in which the available resources are advertised to Kubernetes to allow application to scale-out to multiple Server (with multiple FPGAs per server.)
- Heterogeneous deployment: InAccel unique bitstream repository allows the deployment of fully heterogeneous FPGA clusters. InAccel bitstream repository holds different bitstreams for the available FPGAs. The bitstream repository is integrated with the Coral Orchestrator alleviating the user from the complexity of FPGA-specific bitstreams. Users can deploy their application to heterogenous FPGA clusters (e.g. different FPGA cards from the same or different vendors) and InAccel FPGA orchestrator takes care for loading the right bitstream.
InAccel’s unique approach of an FPGA orchestrator as a middleware, allows the easy deployment of FPGAs by software developers. An FPGA orchestrator is used to schedule the function callings to the available FPGA resources. That methodology allows easy scale-out of an application to multiple FPGAs in the same server without the user having to manually define which function will be executed on which FPGA. As it is done in the software world where the users do not specify in which core the application will be executed, similarly InAccel manager/orchestrator automatically dispatch the functions to the available resources in the FPGA cluster.
Also, it automatically configures and initialize the FPGAs with the right bitstream making easier than ever the deployment of FPGA clusters from a software developer. Actually, it serves like a Kubernetes for multiple FPGAs n the same servers. Combining the FPGA manager with InAccel’s Kubernetes plug-in it allows to scale-out also to multiple servers with multiple FPGAs making it ready for enterprise-grade deployment.
Actually, it makes it so easy to utilize the power of FPGAs, that anybody can try it now for free from their browser using Jupyter notebooks. More info at https://inaccel.com/
FPGA offer a unique combination of advantages: high-throughput, low latency and energy efficiency. Using OpenCL and C/C++ is now easier than ever to create your own accelerator. However, in order to allow the widespread adoption of FPGAs, we must offer the frameworks that will allow easy deployment in the same way that is done in computing resources like CPUs and GPUs. InAccel’s unique FPGA orchestrator provides the abstraction layer that servers as an OS for an FPGA cluster and allows easy deployment, instant scaling and seamless resource management for easier integration with the software stack.