Willow: A User-Programmable SSD on OSDI 2014.
Storage device interface is abstracted as a block device for a long time. Various software develop feature rich data driven engines on top of this abstraction, with out the need to manage lower disk rotation scheme. Nand flash based SSDs adopt this interface by developing a Flash Translation Layer (FTL) and Garbage Collection (GC) and many more to hidden its’ distinct “erase-befor-write” feature. However, many of the internal design of SSDs are overlapped with upper level software. And the obscuring the internal mechanism of SSDs makes the data driven storage engines, e.g. file-system and database, unable to optimize their I/Os for the SSDs as they usually do for HDDs. As a result, there is a need to merge some of the complexity data driven engine operations into SSDs.
For instance, an SSD can support complex atomic operations [10, 32, 35], native caching operations [5, 38], a large, sparse storage address space [16], delegating storage allocation decisions to the SSD [47], and offloading file system permission checks to hardware [8]. These new interfaces allow applications to leverage SSDs’ low latency, ample internal bandwidth, and on-board computational resources, and they can lead to huge improvements in performance.
Although these features are useful, the current one-at-a-time approach to implementing them suffers from several limitations. First, adding features is complex and requires access to SSD internals, so only the SSD manufacturer can add them. Second, the code must be trusted, since it can access or destroy any of the data in the SSD. Third, to be cost-effective for manufacturers to develop, market, and maintain, the new features must be useful to many users and/or across many applications. Selecting widely applicable interfaces for complex use cases is very difficult. For example, editable atomic writes [10] were designed to support ARIES-style write-ahead logging, but not all databases take that approach.
The paper then step directly in to its’ design in Section II to provide an framework to enable feature rich SSDs interface. No more motivation needed. The interface Willow provided is very different from the interface of a conventional SSD. It support Remote Procedure Call (RPC) like protocol to enable OS or application to communicate with several internal component storage processor units (SPU), each of which includes a microprocessor, an interface to the inter-SPU interconnect, and access to an array of non-volatile memory.
The custom code is run on both the host os and the willow device SPU. Trust and permission are supported so that only user or RPC call with proper permission are allowed to execute on SPU.
Custom code execution permissions are checked with in the SSDs. Also, the atomic operations and native caching operations can also implemented in SSDs with lower overhead. Evaluation shows that atomic operations implemented inside SSDs have up to 1.5x performance speed up.
