About Bs-RoFormer Training

[dorpxam]

If a nice user can share is experience with is training. Even just say if the (long time to finish) training was doing well that can be cool not only for me but for some others. I continue to dig into the training code, but a little help will be welcome.

[lyndonlauder]

@iver56 @jinhonglu @ZFTurbo

Please may you share your experiences with training this model, if you can share training code too for everyone else to train that will be great.

[dorpxam]

When you say (in the issue comments) that you can't train the model. What is the problem exactly?

[lyndonlauder]

Just because i dont have training code and i am not a programmer to make code myself

[ZFTurbo]

I'm planing to release my training code. I'm preparing it now. I hope in 1-2 weeks I will be able to release it. After release I will post link here.

[dorpxam]

Thank you very much @ZFTurbo. I have just a little question about the "train loss / SDR" listing you provide in one of the first issue (very regular loss value). Is this value the loss value from the validation at epoch (val loss) or the train loss at step? I think epoch because the SDR computed at this level, but I want to have you validation.

[jinhonglu]

I mainly follow Bytedance' template (https://github.com/bytedance/music_source_separation/tree/master/bytesep), which released a few years ago. I think it would be a good start

[jinhonglu]

yes, with some of the modifications to suit the template.

[lyndonlauder]

@jinhonglu Please can you release this modified train code for other people to use?

[dorpxam]

@jinhonglu not so easy to adapt:

• need to adapt all the dependencies (2 years old is an eternity)
• need to adapt the dataset to musdb18hq (only musdb18 support)
• need to convert all the dataset to HDF5 + indexation
• need to add a new BatchDataPreprocessor subclass for avoid concatenation over channels, more of that the official github do not include the BasicBatchDataPreprocessor found in some scripts. (but found in other distribution in Hugging Face)
• need to write Wrapper for BSRoformer and MelBandRoformer.
• even if the code include segmentation (3 secs in the code), no DynamicPool like in the original paper
• no code to test framing methods (TC & OA)

So, like said by @lyndonlauder, if you've already do the job, that can be cool to share it with us?

It's the first time I will train a MSS model, and because i'm running on a single GPU, I want to be sure to not lose 3 months for nothing. The current code i've doing show random step loss values (see the WandB report). Maybe it's correct and this is the epoch loss that is revelant, but i'm just doubtful.

That's why I invoke the masters for a little help to understanding this point.

[jinhonglu]

Hi, due to the company coding regulation, I can not release the code. I am so sorry for that.

However, I can give some cues for the codes.

even if the code include segmentation (3 secs in the code), no DynamicPool like in the original paper

The length of the segmentation should be able to be controlled somewhere in the code with the help of HDF5 and indexation, and the dynamic pool has already been implemented in the template, you may just read the code carefully.

need to adapt the dataset to musdb18hq (only musdb18 support)
I don't think that is hard to replace the audio data.

need to write Wrapper for BSRoformer and MelBandRoformer
Read how they implement and use the model class

no code to test framing methods (TC & OA)
Regarding the framing methods, based on the results in the paper, I don't think the different usage of the framing method is going to affect the result a lot. The current framing method should be sufficient.

Overall, I don't think the above modifications are difficult for people who have been working in the neural network research area for more than two years. Also, since you can spot the differences, I believe you should be capable of figuring out how to modify them, what you need is to take time to understand their template.

Lastly, as if you were using multi-resolution loss in your training, what I can comment on the figure you provided is that a normal training loss should be around 1.5

[dorpxam]

@jinhonglu

Thank you for your answer. I understand for the limitation of the code distribution. No problem.

About training the RoFormer's models, it's more a question of experience in training and the understand of the common look of the step/epoch values you need to get. I do not have any background.

My purpose was about the gain of time spend in a adaptation or the writing on a dedicated training code. Most of the training code for MSS now are build with Lightning (and maybe Hydra for the config), most of MSS datasets work around MusDB18(+HQ), now maybe Moise.ai DB or some In-House datasets.

These days, i've take a look about different open-source code. Even if there's some difference in-between on the way to use Pytorch/and/or/Lightning, the core stay always the same. Configuring the dataloaders, settings the LightningModule with the right optimizer (+sheduler) ... calling the fit(...). It's always redundant even if some strategy are slightly different (audio segmentation / augmentation).

The problem is that each project, even if it has a large generic part, remains almost systematically fixed for the given project. For example, mdx-net was based on a outdated version of Lightning-Hydra-Template, and even mdx-net is generic, it is not enough for more generic MSS training framework.

That's why, I will wait for the @ZFTurbo training code in a first instance, and continue to see if it is possible to create a training framework for MSS projects that is completely generic.

[dorpxam]

The training code was release, thanks to @ZFTurbo : https://github.com/ZFTurbo/Music-Source-Separation-Training