This changed in this version as the current version uses VSQX files (Vocaloid Editor file format) for dataset conversion, training, and testing.

Previous version of the AI Vocaloid Kit used midi files as a main source as music data for training and testing.

Python3 train.py -modelOutput music-model.pt Counts notes and syllables to ensure flow between melody and lyrics.Īllow users to train their own melody-generation model based on their VSQX dataset. Use markov models to generate Japanese lyrics and sync them to generated melodies from song. Using a Seq2Seq transformer deep learning architecture model to either randomly generate melodies from scratch or from an inital seed of midi notes There is also a manual option in which you can take your generated midi fileĪnd song and import it into your vocaloid editor yourself.įor those who don't own a vocaloid editor, you can just listen to the generated midi file and read the text files containing the Japanese lyricsĬonverting a directory of VSQX files into a training and validation dataset This can then be automatically loaded into a Vocaloid software (tested on Vocaloid Editor 4) to have the Vocaloid sing your AI generated song. The script will generate a vsqx file that contains the melody and lyrics built into it automatically. The corpus was web-scaped from the Studio48 site, containing lyrics from groups like AKB48, Nogizaka46, Keyakizaka46, etc. It can also combine verses if one generated verse is too short So if there are 2 notes in the 1st verse and 5 notes in 2nd verse, the lyric verse combine those two verses and will generate a verse of 7 syllables (2+5=7). This means that if a melody verse contains 7 notes, the script will generate a lyric verse of 7 syllables to match it. It also uses markov models to generate its own accompanying Japanese lyrics that is in syllable sync with the melody. Developers and musicians can now have AI support to aid and inspire ideas for their next vocaloid song.Ĭan take a folder of vsqx files and use Seq2seq model (encoders and decoders) in order to study pattern of a song and generate it’s own unique music (in the form of a vsqx file or midi file). A Python kit that uses deep learning to generate vocaloid music.

Vocaloid 2 software