The SSP’s supported sample rates are 48, 96 and 192kHz.
When you load a sample with a 32kHz sample rate, and play it back on the SSP, then yes, the pitch will change, because you are not playing it back at 32kHz and because the SSP does not do automatic resampling of your samples. The software also does not do pitch-detection of your samples, to figure out what the pitch of the sample is. So if you hit A4=440Hz on a piano and sample that at 48kHz, and play it back on the SSP running at 48kHz, it will still be 440Hz. But if you sample it at 32kHz and play it back at 48kHz, then no, it won’t be 440Hz.
Some modules like the granular have tuning / frequency settings you can use to deal with this problem.
The SSP does all sampling and playing back of samples at 32 bit resolution, but that has nothing to do with the sample rate and doesn’t affect the pitch you perceive.