All MSRG Publications

Nikolai Merkel / Pierre Toussing / Ruben Mayer / Hans-Arno Jacobsen
Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental Study
arXiv Preprint, 2024
ArXiv Preprint
Grier M. Jones / Hans-Arno Jacobsen
Distributed Quantum Computing for Chemical Applications
arXiv Preprint, 2024
ArXiv Preprint
Shashank Motepalli / Hans-Arno Jacobsen
How Does Stake Distribution Influence Consensus? Analyzing Blockchain Decentralization
2024 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), 2024
Conference Paper
Shashank Motepalli / Hans-Arno Jacobsen
Delay Towers to Bootstrap Blockchains
2024 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), 2024
Conference Paper
Herbert Woisetschläger / Alexander Erben / Shiqiang Wang / Ruben Mayer / Hans-Arno Jacobsen
A Survey on Efficient Federated Learning Methods for Foundation Model Training
Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence, 2024
Conference Paper
Jawad Tahir / Chiheb Baili / Matej Svaral / Johannes Friedlein / Christoph Doblander / Hans-Arno Jacobsen
Challenger 2.0: A Step Towards Automated Deployments and Resilient Solutions for the DEBS Grand Challenge
Proceedings of the 18th ACM International Conference on Distributed and Event-based Systems, 2024
Conference Paper
Herbert Woisetschläger / Alexander Erben / Shiqiang Wang / Ruben Mayer / Hans-Arno Jacobsen
Federated Fine-tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly
Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning, 2024
Conference Paper
Nikolai Merkel / Daniel Stoll / Ruben Mayer / Hans-Arno Jacobsen
An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Training
None, 2024
Technical Report
Herbert Woisetschläger / Alexander Erben / Bill Marino / Shiqiang Wang / Nicholas D Lane / Ruben Mayer / Hans-Arno Jacobsen
Federated Learning Priorities Under the European Union Artificial Intelligence Act
None, 2024
Technical Report
Herbert Woisetschläger / Simon Mertel / Christoph Krönke / Ruben Mayer / Hans-Arno Jacobsen
Federated Learning and AI Regulation in the European Union: Who is Responsible? An Interdisciplinary Analysis
None, 2024
Technical Report
Gengrui Zhang / Fei Pan / Yunhao Mao / Sofia Tijanic / Michael Dang’ana / Shashank Motepalli / Shiquan Zhang / Hans-Arno Jacobsen
Reaching consensus in the byzantine empire: A comprehensive review of bft consensus algorithms
ACM Computing Surveys, 2023
Technical Report
Yuqiu Zhang / Tongkun Zhang / Gengrui Zhang / Hans-Arno Jacobsen
Lifting the fog of uncertainties: Dynamic resource orchestration for the containerized cloud
Proceedings of the 2023 ACM Symposium on Cloud Computing, 2023
Conference Paper
Shashank Motepalli / Hans-Arno Jacobsen
Analyzing Geospatial Distribution in Blockchains
2023 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS), 2023
Conference Paper
Nikolai Merkel / Ruben Mayer / Tawkir Ahmed Fakir / Hans-Arno Jacobsen
Partitioner Selection with EASE to Optimize Distributed Graph Processing
2023 IEEE 39th International Conference on Data Engineering (ICDE), 2023
Conference Paper
Jawad Tahir / Raj Mandal / Olha Stefanova / Hans-Arno Jacobsen / Christoph Doblander / Ruben Mayer
pTA: A Programmable Teaching Assistant for Lab Courses
Proceedings of the 2nd International Workshop on Data Systems Education: Bridging education practice with education research, 2023
Conference Paper
Herbert Woisetschläger / Alexander Isenko / Ruben Mayer / Hans-Arno Jacobsen
Fledge: Benchmarking federated machine learning applications in edge computing systems
arXiv Preprint, 2023
ArXiv Preprint
Gengrui Zhang / Yunhao Mao / Shiquan Zhang / Shashank Motepalli / Fei Pan / Hans-Arno Jacobsen
V-Guard: An Efficient Permissioned Blockchain for Achieving Consensus under Dynamic Memberships in V2X Networks
arXiv Preprint, 2023
ArXiv Preprint
Gengrui Zhang / Yunhao Mao / Shiquan Zhang / Shashank Motepalli / Fei Pan / Hans-Arno Jacobsen
V-Guard: An Efficient Permissioned Blockchain for Achieving Consensus under Dynamic Memberships in V2X Networks.
arXiv Preprint, 2023
ArXiv Preprint
Sebastian Frischbier / Jawad Tahir / Christoph Doblander / Arne Hormann / Ruben Mayer / Hans-Arno Jacobsen
The DEBS 2022 Grand Challenge: Detecting Trading Trends in Financial Tick Data
Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems, 2022
Conference Paper
Sebastian Frischbier / Jawad Tahir / Christoph Doblander / Arne Hormann / Ruben Mayer / Hans-Arno Jacobsen
Detecting trading trends in financial tick data: the DEBS 2022 grand challenge
Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems, 2022
Conference Paper

Browse 29 publications

Details:

Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental Study

Abstract

Graph neural networks (GNNs) are a type of neural network capable of learning on graph-structured data. However, training GNNs on large-scale graphs is challenging due to iterative aggregations of high-dimensional features from neighboring vertices within sparse graph structures combined with neural network operations. The sparsity of graphs frequently results in suboptimal memory access patterns and longer training time. Graph reordering is an optimization strategy aiming to improve the graph data layout. It has shown to be effective to speed up graph analytics workloads, but its effect on the performance of GNN training has not been investigated yet. The generalization of reordering to GNN performance is nontrivial, as multiple aspects must be considered: GNN hyper-parameters such as the number of layers, the number of hidden dimensions, and the feature size used in the GNN model, neural network operations, large intermediate vertex states, and GPU acceleration. In our work, we close this gap by performing an empirical evaluation of 12 reordering strategies in two state-of-the-art GNN systems, PyTorch Geometric and Deep Graph Library. Our results show that graph reordering is effective in reducing training time for CPU- and GPU-based training, respectively. Further, we find that GNN hyper-parameters influence the effectiveness of reordering, that reordering metrics play an important role in selecting a reordering strategy, that lightweight reordering performs better for GPU-based than for CPU-based training, and that invested reordering time can in many cases be amortized.

Additional Information

Details:

Distributed Quantum Computing for Chemical Applications

Abstract

In recent years, interest in quantum computing has increased due to technological advances in quantum hardware and algorithms. Despite the promises of quantum advantage, the applicability of quantum devices has been limited to few qubits on hardware that experiences decoherence due to noise. One proposed method to get around this challenge is distributed quantum computing (DQC). Like classical distributed computing, DQC aims at increasing compute power by spreading the compute processes across many devices, with the goal to minimize the noise and circuit depth required by quantum devices. In this paper, we cover the fundamental concepts of DQC and provide insight into where the field of DQC stands with respect to the field of chemistry -- a field which can potentially be used to demonstrate quantum advantage on noisy-intermediate scale quantum devices.

Additional Information

Details:

How Does Stake Distribution Influence Consensus? Analyzing Blockchain Decentralization

Additional Information

Details:

Delay Towers to Bootstrap Blockchains

Additional Information

Details:

A Survey on Efficient Federated Learning Methods for Foundation Model Training

Abstract

Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training across a multitude of clients. However, new approaches to FL often discuss their contributions involving small deep-learning models only and focus on training full models on clients. In the wake of Foundation Models (FM), the reality is different for many deep learning applications. Typically, FMs have already been pre-trained across a wide variety of tasks and can be fine-tuned to specific downstream tasks over significantly smaller datasets than required for full model training. However, access to such datasets is often challenging. By its design, FL can help to open data silos. With this survey, we introduce a novel taxonomy focused on computational and communication efficiency, the vital elements to make use of FMs in FL systems. We discuss the benefits and drawbacks of parameter-efficient fine-tuning (PEFT) for FL applications, elaborate on the readiness of FL frameworks to work with FMs and provide future research opportunities on how to evaluate generative models in FL as well as the interplay of privacy and PEFT.

Details:

Challenger 2.0: A Step Towards Automated Deployments and Resilient Solutions for the DEBS Grand Challenge

Additional Information

Details:

Federated Fine-tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly

Additional Information

Details:

An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Training

Abstract

Recently, graph neural networks (GNNs) have gained much attention as a growing area of deep learning capable of learning on graph-structured data. However, the computational and memory requirements for training GNNs on large-scale graphs can exceed the capabilities of single machines or GPUs, making distributed GNN training a promising direction for large-scale GNN training. A prerequisite for distributed GNN training is to partition the input graph into smaller parts that are distributed among multiple machines of a compute cluster. Although graph partitioning has been extensively studied with regard to graph analytics and graph databases, its effect on GNN training performance is largely unexplored. In this paper, we study the effectiveness of graph partitioning for distributed GNN training. Our study aims to understand how different factors such as GNN parameters, mini-batch size, graph type, features size, and scale-out factor influence the effectiveness of graph partitioning. We conduct experiments with two different GNN systems using vertex and edge partitioning. We found that graph partitioning is a crucial pre-processing step that can heavily reduce the training time and memory footprint. Furthermore, our results show that invested partitioning time can be amortized by reduced GNN training, making it a relevant optimization.

Additional Information

Details:

Federated Learning Priorities Under the European Union Artificial Intelligence Act

Abstract

The age of AI regulation is upon us, with the European Union Artificial Intelligence Act (AI Act) leading the way. Our key inquiry is how this will affect Federated Learning (FL), whose starting point of prioritizing data privacy while performing ML fundamentally differs from that of centralized learning. We believe the AI Act and future regulations could be the missing catalyst that pushes FL toward mainstream adoption. However, this can only occur if the FL community reprioritizes its research focus. In our position paper, we perform a first-of-its-kind interdisciplinary analysis (legal and ML) of the impact the AI Act may have on FL and make a series of observations supporting our primary position through quantitative and qualitative analysis. We explore data governance issues and the concern for privacy. We establish new challenges regarding performance and energy efficiency within lifecycle monitoring. Taken together, our analysis suggests there is a sizable opportunity for FL to become a crucial component of AI Act-compliant ML systems and for the new regulation to drive the adoption of FL techniques in general. Most noteworthy are the opportunities to defend against data bias and enhance private and secure computation

Additional Information

Details:

Federated Learning and AI Regulation in the European Union: Who is Responsible? An Interdisciplinary Analysis

Additional Information

Details:

Reaching consensus in the byzantine empire: A comprehensive review of bft consensus algorithms

Abstract

Byzantine fault-tolerant (BFT) consensus algorithms are at the core of providing safety and liveness guarantees for distributed systems that must operate in the presence of arbitrary failures. Recently, numerous new BFT algorithms have been proposed, not least due to the traction blockchain technologies have garnered in the search for consensus solutions that offer high throughput, low latency, and robust system designs. In this article, we conduct a systematic survey of selected and distinguished BFT algorithms that have received extensive attention in academia and industry alike. We perform a qualitative comparison among all algorithms we review considering message and time complexities. Furthermore, we provide a comprehensive, step-by-step description of each surveyed algorithm by decomposing them into constituent subprotocols with intuitive figures to illustrate the message-passing pattern. We also elaborate on the strengths and weaknesses of each algorithm compared to the other state-of-the-art approaches.

Details:

Lifting the fog of uncertainties: Dynamic resource orchestration for the containerized cloud

Additional Information

Details:

Analyzing Geospatial Distribution in Blockchains

Additional Information

Details:

Partitioner Selection with EASE to Optimize Distributed Graph Processing

Additional Information

Details:

pTA: A Programmable Teaching Assistant for Lab Courses

Additional Information

Details:

Fledge: Benchmarking federated machine learning applications in edge computing systems

Abstract

Federated Machine Learning (FL) has received considerable attention in recent years. FL benchmarks are predominantly explored in either simulated systems or data center environments, neglecting the setups of real-world systems, which are often closely linked to edge computing. We close this research gap by introducing FLEdge, a benchmark targeting FL workloads in edge computing systems. We systematically study hardware heterogeneity, energy efficiency during training, and the effect of various differential privacy levels on training in FL systems. To make this benchmark applicable to real-world scenarios, we evaluate the impact of client dropouts on state-of-the-art FL strategies with failure rates as high as 50%. FLEdge provides new insights, such as that training state-of-the-art FL workloads on older GPU-accelerated embedded devices is up to 3x more energy efficient than on modern server-grade GPUs.

Additional Information

Details:

V-Guard: An Efficient Permissioned Blockchain for Achieving Consensus under Dynamic Memberships in V2X Networks

Abstract

This paper presents V-Guard, a new permissioned blockchain that achieves consensus for vehicular data under changing memberships, targeting the problem in V2X networks where vehicles are often intermittently connected on the roads. To achieve this goal, V-Guard integrates membership management into the consensus process for agreeing on data entries. It binds a data entry with a membership configuration profile that describes responsible vehicles for achieving consensus for the data entry. As such, V-Guard produces chained consensus results of both data entries and their residing membership profiles, which enables consensus to be achieved seamlessly under changing memberships. In addition, V-Guard separates the ordering of transactions from consensus, allowing concurrent ordering instances and periodic consensus instances to order and commit data entries. These features make V-Guard efficient for achieving consensus under dynamic memberships with high throughput and latency performance.

Additional Information

Details: