Easy way to convert model? #1627

Hey @oneway234, I am not experienced with this at all. And if you find a solution, please let us know. According to: this, you can just need to load what you are interested into and then export this object.

Hi @TParcollet thanks for help!
I followed the tutorial to load model, then tried to use "torch.onnx.export" to convert the model.
But I got an issue.

File "/home/weiyaodu/ww/speechbrain/sepsep.py", line 10, in
torch.onnx.export(model.modules, dummy_input, '/home/weiyaodu/Desktop', verbose=True)
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/onnx/init.py", line 271, in export
return utils.export(model, args, f, export_params, verbose, training,
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/onnx/utils.py", line 88, in export
_export(model, args, f, export_params, verbose, training, input_names, output_names,
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/onnx/utils.py", line 694, in _export
_model_to_graph(model, args, verbose, input_names,
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/onnx/utils.py", line 457, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args,
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/onnx/utils.py", line 420, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/onnx/utils.py", line 380, in _trace_and_get_graph_from_model
torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/jit/_trace.py", line 1139, in _get_trace_graph
outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/jit/_trace.py", line 125, in forward
graph, out = torch._C._create_graph_by_tracing(
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/jit/_trace.py", line 116, in wrapper
outs.append(self.inner(*trace_inputs))
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/nn/modules/module.py", line 887, in _call_impl
result = self._slow_forward(*input, **kwargs)
File "/home/weiyaodu/anaconda3/envs/sb/lib/python3.8/site-packages/torch/nn/modules/module.py", line 860, in _slow_forward
result = self.forward(*input, **kwargs)
TypeError: forward() takes 1 positional argument but 2 were given

It reported I gave 2 arguments. Is it a bug in speechbrain ?
Here is my convert code.

@pplantinga @Gastron @mravanelli sepformer isn't jittable ?

I've read #639. Then I tried to use onnx , and got this issue.

This issue appears with Core ML conversion too (coremltools). The non-direct workaround would be to use the ONNX converter, which is also how we plan to deploy the model to the browser, and then import the ONNX model to Core ML, but both ways result in the same type error with forward() arguments.

The current goal is to deploy the model on edge devices, and web applications (browsers), where Python is unavailable, and where sending audio data is problematic due to privacy concerns.

I'm using ECAPA_TDNN. Some of the classes in the ECAPA_TDNN.py file have two arguments, the secondary being a length parameter, where PyTorch only expects a single parameter.

This issue appears with Core ML conversion too (coremltools). The non-direct workaround would be to use the ONNX converter, which is also how we plan to deploy the model to the browser, and then import the ONNX model to Core ML, but both ways result in the same type error with forward() arguments.

The current goal is to deploy the model on edge devices, and web applications (browsers), where Python is unavailable, and where sending audio data is problematic due to privacy concerns.

I'm using ECAPA_TDNN. Some of the classes in the ECAPA_TDNN.py file have two arguments, the secondary being a length parameter, where PyTorch only expects a single parameter.

hi Fillip, I want to export the pretrained ECAPA-TDNN model so that it can inference with C++ torch API.
When I tried to do "torch.jit.script(classifier.modules.embedding_model)", errors such as ”ModuleList/Sequential indexing is only supported with integer literals“ occur in ECAPA-TDNN.py. What is the proper way to export the model without modifying model architecture python files?

@sheepHavingPurpleLeaf The issue is it cannot be exported at this stage, as far as I can see. For TorchScript / JIT compatibility some changes need to be made.

Note: I've reverted back to using NVIDIA's NeMo for our project as we need portable models suitable for production cases on edge devices.

Thanks for reply. As @HelloFillip said it can't be exported at this stage.

Closing the issue as it was answered - though let's make a note that there is interesting in a JITable version of SepFormer.

#1142 JITable version of SepFormer is here

Hi @Gastron! Mind reopening this issue while #1142 is being discussed?

can the VAD model be converted to onnx