Audio mixing: channels stored interleaved vs. not

When downmixing non-interleaved multichannel audio, does it make sense to interleave it first, or just process it as it is?

If as is, sample by sample jumping between channels, or a buffer-full of each channel at a time?