论文标题
Tridentse：32全球令牌指导语音增强
TridentSE: Guiding Speech Enhancement with 32 Global Tokens
论文作者
论文摘要
在本文中，我们介绍了Tridentse，这是一种新颖的语音增强体系结构，能够有效捕获全球信息和本地细节。 Tridentse维护T-F BIN级别表示以捕获细节，并使用少量的全球令牌来处理全局信息。通过交叉注意模块在本地和全球表示之间传播信息。为了捕获框架内和框内信息，将全局令牌分为两组以分别在时间和频率轴上进行处理。公制歧视者进一步用于指导我们的模型以达到更高的感知质量。即使计算成本明显降低，Tridents的表现都超过了以前的语音增强方法，在VoiceBank+需求数据集上达到了3.47的PESQ，而DNS无依赖性测试集的PESQ为3.44。可视化表明，全球代币学习了多种且可解释的全球模式。
In this paper, we present TridentSE, a novel architecture for speech enhancement, which is capable of efficiently capturing both global information and local details. TridentSE maintains T-F bin level representation to capture details, and uses a small number of global tokens to process the global information. Information is propagated between the local and the global representations through cross attention modules. To capture both inter- and intra-frame information, the global tokens are divided into two groups to process along the time and the frequency axis respectively. A metric discriminator is further employed to guide our model to achieve higher perceptual quality. Even with significantly lower computational cost, TridentSE outperforms a variety of previous speech enhancement methods, achieving a PESQ of 3.47 on VoiceBank+DEMAND dataset and a PESQ of 3.44 on DNS no-reverb test set. Visualization shows that the global tokens learn diverse and interpretable global patterns.
