-
Notifications
You must be signed in to change notification settings - Fork 162
Description
Hello, thank you for your team’s awesome work!
I have some questions about using the bolt framework.
Here's my working environment:
- target platform: Android-aarch64
- build platform: Linux
- Device: Arm v8.2+
- inference Precision: BNN_FP16
- tested bolt version: both 1.2.1 and 1.3.0
- when I run the same model on both versions (1.2.1 and 1.3.0) using their X2bolt and benchmark,
the final latency results were almost the same but the compositions (statistics time report) were different.
Both cases run under loops=1 option but the statistics time of version 1.2.1 seems to be the result of running 10 times. Is it normal?
- I made some custom network, which has a structure like the following:
This network works with version 1.3.0, but not with 1.2.1 because X2bolt of ver 1.2.1 doesn't work properly.
[ X2bolt debug log (version 1.2.1) ]
[ X2bolt debug log (version 1.3.0) ]
as we can see, ver 1.2.1 X2bolt cannot detect an inputs tensor of Mul_21 so I guess benchmark program stops at
bolt/inference/engine/src/cnn.cpp
Line 696 in 4bdc81e
| std::vector<std::string> curOpInputTensorName = this->operatorTensorMap[opName][0]; |
(or nearby, checked with debug options).
In the case of Mul_21, it is executed after the whole left path of the above graph image, so it is expected that it was difficult to reuse the result of ReduceMean op. Of course, there is no problem with the latest version of X2bolt. Is there a way to solve this in the previous version as well?
- The reason I use the previous version of the bolt framework is that I saw a significant difference in latency results depending on the version for a specific network model (e.g. Real-to-Binary network, https://arxiv.org/pdf/2003.11535.pdf?fname=cm&font=TypeI).
I wonder if this faster output of ver 1.2.1 is kind a reporting bug in version 1.2.1, or a possible result by the implementation difference.
Thank you for reading my long issue and I look forward to your answers.