论文标题
通过弹性资源调度来自适应HTAP
Adaptive HTAP through Elastic Resource Scheduling
论文作者
论文摘要
现代混合交易/分析处理(HTAP)系统使用的集成数据处理引擎对新鲜数据进行分析,这些数据是从交易引擎中摄入的。 HTAP系统通常在设计时考虑新鲜度,并针对固定范围的新鲜要求进行了优化,并以OLTP或OLAP的性能成本解决。但是,数据新鲜度和两种引擎的性能要求可能会随工作量而异。 我们将HTAP作为调度问题,通过弹性资源管理在运行时解决。我们将HTAP系统建模为一组三个单独的引擎:OLTP,OLAP以及资源和数据交换(RDE)引擎。我们设计了一种调度算法,该算法通过弹性资源管理遍历HTAP设计范围,以满足工作量的数据新鲜度要求。我们提出了一种内存系统设计,该设计对当前的最新OLTP和OLAP发动机毫无疑问,我们使用它来评估方法的性能。我们的评估表明,与100个查询序列的静态时间表相比,我们系统对OLAP查询的性能益处随着时间的推移而增加,高达50%,同时保持小的和受控的OLTP吞吐量。
Modern Hybrid Transactional/Analytical Processing (HTAP) systems use an integrated data processing engine that performs analytics on fresh data, which are ingested from a transactional engine. HTAP systems typically consider data freshness at design time, and are optimized for a fixed range of freshness requirements, addressed at a performance cost for either OLTP or OLAP. The data freshness and the performance requirements of both engines, however, may vary with the workload. We approach HTAP as a scheduling problem, addressed at runtime through elastic resource management. We model an HTAP system as a set of three individual engines: an OLTP, an OLAP and a Resource and Data Exchange (RDE) engine. We devise a scheduling algorithm which traverses the HTAP design spectrum through elastic resource management, to meet the data freshness requirements of the workload. We propose an in-memory system design which is non-intrusive to the current state-of-art OLTP and OLAP engines, and we use it to evaluate the performance of our approach. Our evaluation shows that the performance benefit of our system for OLAP queries increases over time, reaching up to 50% compared to static schedules for 100 query sequences, while maintaining a small, and controlled, drop in the OLTP throughput.