Thallus: An RDMA-based Columnar Data Transport Protocol

Appeared in HotInfra 2024.

Abstract

The volume of data generated and stored in contemporary global data centers is experiencing exponential growth. This rapid data growth necessitates e￿cient processing and anal- ysis to extract valuable business insights. In distributed data processing systems, data undergoes exchanges between the compute servers that contribute signi￿cantly to the total data processing duration in adequately large clusters, neces- sitating e￿cient data transport protocols.

Traditionally, data transport frameworks such as JDBC and ODBC have used TCP/IP-over-Ethernet as their under- lying network protocol. Such frameworks require serializing the data into a single contiguous bu￿er before handing it o￿ to the network card, primarily due to the requirement of contiguous data in TCP/IP. In OLAP use cases, this seri- alization process is costly for columnar data batches as it involves numerous memory copies that hurt data transport duration and overall data processing performance. We study the serialization overhead in the context of a widely-used columnar data format, Apache Arrow, and propose lever- aging RDMA to transport Arrow data over In￿niband in a zero-copy manner. We design and implement Thallus, an RDMA-based columnar data transport protocol for Apache Arrow based on the Thallium framework from the Mochi ecosystem, compare it with a purely Thallium RPC-based implementation, and show substantial performance improve- ments can be achieved by using RDMA for columnar data transport.

Publication date:
October 2024

Authors:
Jayjeet Chakraborty

Projects:

Bibtex entry

@inproceedings{thallus-jayjeet,
  author       = {Jayjeet Chakraborty},
  title        = {Thallus: An {RDMA}-based Columnar Data Transport Protocol},
  booktitle    = {HotInfra 2024},
  month        = oct,
  year         = {2024},
}