But for SIMD to be fun, data should be interleaved...

Yes and no.

While interleaved channels would certainly help with "channel-wise" vectorization, it would make things *worse* for multi-channel "block-wise" vectorization. IMO, any object that can do block-wise vectorization would likely do so with multi-channel data as well, for several reasons:

1. same code for single-channel and multi-channel version
2. vector size never has a remainder -> no second loop required -> less code
3. it is more likely to have a multiple-of-8 vector size than having 8+ channels

Of course, you can still do channel-wise vectorization with non-interleaved channels, it's just more expensive to fill each SIMD register with data from multiple locations (instead of loading it from a single memory address). Now, if the vector size is a multiple of 4 (SSE) resp. 8 (AVX), things don't look quite as bad: you can load 4 resp. 8 consecutive floats from each channel into SIMD registers and transpose them once before processing.

Another thing to consider with interleaved channels: if the channel count is not a multiple of 4 (SSE) resp. 8 (AVX), the data would not be properly aligned and you would have to use the slower unaligned instructions.

Finally, interleaved channels are awkward and less cache-friendly for scalar code because channel data is not contiguous.


To sum it up:

+ faster channel-wise vectorization (how much?)
- slower block-wise (multi-channel) vectorization
- no backwards compatibility with existing externals
- worse for scalar code

For me, non-interleaved channels are the clear winner :-)

Christof