D A D I

Data Accelerator for
Disaggregated Infrastructure

Accelerated container image

Sub-project of containerd, contains a containerd snapshotter and image conversion tools. more

Github
Overlaybd

Sub-project of containerd, contains the backend storage service of overlaybd image format.

Github
Buildkit

Based on moby/buildkit, easily and directly build and export overlaybd images.

Github
P2P data distribution

Use p2p protocol to speed up HTTP file download for registry in large-scale clusters.

Github

About DADI

Introduction

DADI is short for Data Accelerator for Disaggregated Infrastructure. DADI provides a solution for data acceleration which is typically used for container images acceleration, and can be easily expand into other scenarios.

The whole approach of accelerated image service is published at DADI: Block-Level Image Service for Agile and Elastic Application Deployment. USENIX ATC’20

DADI has been widely used in Alibaba and Alibaba Cloud and already been integrated by Alibaba Cloud Registry (ACR), Function Compute and other serverless services.

Key Features

High Performace
Block-device-based image format has much lower complexity than filesystem-based implementations. For example, cross-layer hardlink and non-copy commands like chown are very complex for filesystem-based image without copying up, but is natively supported by overlaybd.

High Reliability
Overlaybd outputs virtual block devices through TCMU, which is a linux kernel module and widely supported in most operation systems. Overlaybd backstore can recover from failures or crashes, which is difficult for FUSE-based image formats.

Core Technology

Overlaybd

Overlaybd is a block level container image format, providing a merged view of block-based layers.

Zfile

Zfile is a compression file format supporting online decompression, which can reduce storage and transmission costs.

P2P

Uses P2P protocol to speed up HTTP file download not only for container images.