Currently on the mainstream AMD driver, the driver will segfault if you try to build an acceleration structure without either ray query or raytracing pipelines enabled. It would be good to test if acceleration structure building commands still work as you expect with only VK_KHR_acceleration_structure enabled.
We found this in wgpu because our own raytracing tests only turned on acceleration structure features, but for some reason those tests we segfaulting. If we enable ray query, suddenly the tests stop failing and pass with flying colors.
Afaict this is valid vulkan.