Skip to content

Respect the dtype the op was initialized in for non quant mixed op. #11282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
comfyanonymous merged 1 commit into from
Dec 12, 2025
Merged

Respect the dtype the op was initialized in for non quant mixed op. #11282

comfyanonymous merged 1 commit into from
Dec 12, 2025

Conversation

@comfyanonymous
Copy link
Owner

@comfyanonymous comfyanonymous commented Dec 12, 2025

No description provided.

@comfyanonymous comfyanonymous merged commit 5495589 into master Dec 12, 2025
12 checks passed
@comfyanonymous comfyanonymous deleted the temp_pr branch December 12, 2025 04:32
@robsilvia
Copy link

robsilvia commented Dec 12, 2025

This change is causing some regressions with custom nodes. Specifically WanVideoNag no longer works on quantized models.

2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] Traceback (most recent call last):
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/comfy/quant_ops.py", line 481, in fp8_linear
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] output = torch._scaled_mm(
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] RuntimeError: Bias must be BFloat16 or Half, but got Float8_e4m3fn
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR]
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] Traceback (most recent call last):
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/execution.py", line 515, in execute
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/execution.py", line 329, in get_output_data
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/execution.py", line 303, in _async_map_node_over_list
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] await process_inputs(input_dict, i)
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/execution.py", line 291, in process_inputs
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] result = f(**inputs)
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/custom_nodes/comfyui-kjnodes/nodes/model_optimization_nodes.py", line 1797, in patch
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] context = diffusion_model.text_embedding(conditioning[0][0].to(device, dtype))
2025-12-12 04:53:34.200 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] return self._call_impl(*args, **kwargs)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] return forward_call(*args, **kwargs)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/venv/lib/python3.12/site-packages/torch/nn/modules/container.py", line 250, in forward
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] input = module(input)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] return self._call_impl(*args, **kwargs)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] return forward_call(*args, **kwargs)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/comfy/ops.py", line 609, in forward
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] return self._forward(input, self.weight, self.bias)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/comfy/ops.py", line 593, in _forward
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] return torch.nn.functional.linear(input, weight, bias)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/comfy/quant_ops.py", line 212, in torch_dispatch
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] return handler(func, args, kwargs)
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] File "ComfyUI/comfy/quant_ops.py", line 507, in fp8_linear
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] raise RuntimeError(f"FP8 _scaled_mm failed, falling back to dequantization: {e}")
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR] RuntimeError: FP8 _scaled_mm failed, falling back to dequantization: Bias must be BFloat16 or Half, but got Float8_e4m3fn
2025-12-12 04:53:34.201 [Warning] [ComfyUI-0/STDERR]

This was referenced Dec 12, 2025
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Kosinkadink Kosinkadink Awaiting requested review from Kosinkadink Kosinkadink is a code owner

@guill guill Awaiting requested review from guill guill is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@comfyanonymous @robsilvia