comment: similar compute rates to turing-lite or pascal in fp or dp4a.
example of the highest config (g710 mp16 @ 1.0ghz):
- 2.0 tflops fp32
- 4.1 tflops fp16 (through vectors)
- 1.0 tops int16 (through vectors)
- 0.5 tops int32
- 0.1 tops int64 (unofficial panvk support)
- 8.2 tops int8 (theoretical/unofficial, panvk expected to be ~ 6 tops)
panvk is expected to currently perform at about 2/3 of theoretical compute in vkpeak. the system driver gets much closer, but if you're like me and having vulkan 1.1, panvk is the only choice...
Update (g710 mp10@955MHz): with clpeak (vulkan) peak fp32 gets to about 1 tflops (80%) and peak fp16 is very close to theoretical, but still lags behind the system driver. peak int8 got to about 3.7 tops from 3.2-ish in vkpeak (the best performing one changed to panvk_faux123, but that doesn't run llama.cpp, but even the driver that runs is still 5x slower than CPU)
sadly panvk_faux123 is proprietary, but the system driver already is anyway...