Synthetic spectrograms
Using GANs to make more training data for spoken language identification.
This started with a data problem. We were working on identifying languages in speech where someone switches between languages, and there wasn’t enough multilingual speech data to train on.
The project became a way to explore whether we could create more useful training examples ourselves.
Generate the spectrograms#
We used a generative adversarial network (GAN) to synthesize Mel spectrograms, a representation of how sound changes over time. These synthetic examples went into the training data for code-switched language identification.
We also built a CNN-LSTM baseline to compare against. Generating examples was one part of the work. Evaluating the language-identification system was the other.
- GAN synthesisGenerate Mel spectrograms
- Augment the dataAdd synthetic training examples
- Language identificationEvaluate against a CNN-LSTM baseline
From an experiment to a paper#
The work was published at IEEE IConSIP 2022. The preprint and code are available below.
It was an early piece of my work with generative models, using them to address a training-data problem in another model.