feat: add post 'The Good, the Bad, the Brittle. Fusing Attention in ONNX Graphs'✨#11
feat: add post 'The Good, the Bad, the Brittle. Fusing Attention in ONNX Graphs'✨#11KarelZe wants to merge 38 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds a new blog post about attention fusion in ONNX graphs, covering approaches using onnxruntime and onnxscript. It also includes minor configuration updates and a small stylistic change in another post. The new post provides valuable insights and code examples, although some code snippets are incomplete or have minor issues like missing imports and hardcoded paths. Addressing these will improve the clarity and usability of the examples.
| ir.passes.Sequential( | ||
| onnxscript.rewriter.RewritePass([rule]), | ||
| common_passes.RemoveUnusedNodesPass(), | ||
| common_passes.ShapeInferencePass(), | ||
| )(onnx_model) |
There was a problem hiding this comment.
The common_passes module is used here but not imported in the code snippet. Please add the necessary import.
| ir.passes.Sequential( | |
| onnxscript.rewriter.RewritePass([rule]), | |
| common_passes.RemoveUnusedNodesPass(), | |
| common_passes.ShapeInferencePass(), | |
| )(onnx_model) | |
| ir.passes.Sequential( | |
| onnxscript.rewriter.RewritePass([rule]), | |
| onnxscript.ir.passes.common_passes.RemoveUnusedNodesPass(), | |
| onnxscript.ir.passes.common_passes.ShapeInferencePass(), | |
| )(onnx_model) |
| ), | ||
| name="qkv_bias", | ||
| ) | ||
| num_heads = _ir_utils.get_dim(q_reshaped, 2) |
There was a problem hiding this comment.
The _ir_utils module is used here but not imported in the code snippet. Please add the necessary import.
| num_heads = _ir_utils.get_dim(q_reshaped, 2) | |
| num_heads = onnxscript.ir.utils.get_dim(q_reshaped, 2) |
| gathered_shape = op.Gather(shape, op.Constant(), axis=0) | ||
| q_unsqueezed_shape = op.Unsqueeze(gathered_shape, op.Constant()) |
There was a problem hiding this comment.
Using op.Constant() without arguments in the pattern might match any constant node, which may not be specific enough to correctly identify the shape constants used in the original graph. Consider using pattern.ANY_VALUE if the exact constant value doesn't matter for the match, or matching specific constant values if they are fixed.
No description provided.