Pisets: A Robust Speech Recognition System for Lectures and Interviews

Summary

This paper introduces Pisets, a new speech-to-text system designed to be more accurate and reliable than existing models, especially for long recordings and challenging audio environments.

What's the problem?

Current speech-to-text models, like Whisper, can sometimes make mistakes, adding incorrect words (hallucinations) or misinterpreting what's said, particularly when dealing with lengthy audio or poor sound quality. This is a big issue for professionals like scientists and journalists who need precise transcriptions.

What's the solution?

The researchers built Pisets using a three-step process. First, it uses Wav2Vec2 to get an initial transcription. Then, it employs a system called the Audio Spectrogram Transformer (AST) to filter out likely errors. Finally, it refines the transcription with Whisper. They also used a technique called 'curriculum learning,' where the system learns from easier examples first, and trained it on a lot of Russian speech data. They also improved how the system estimates its own uncertainty, making it more confident in its accurate parts and flagging potentially incorrect sections.

Why it matters?

Pisets is important because it provides a more robust and accurate way to convert speech to text, especially for long audio files and in situations where the audio isn't perfect. This is crucial for fields where accurate records are essential, like scientific research and journalism, and the fact that the code is publicly available allows others to build upon and improve it.