Has anyone seen any experiments about what happens if a pistol has a very high twist rate?

It seems like it may be able to help the marginal calibers penetrate adequately/I suppose it should improve the amount of energy transferred from the powder to the bullet.

I wonder if its because small capacity means pistol rounds have to be loaded with fast powders, and really fast twist could lead to overpressure as the bullet is overcoming initial inertia, or it could skip the rifling

I've never seen results of an experiment on that, and that might work for jacketed bullets. But if I'm carrying a load that I want to achieve maximum penetration then I'm always using hard cast bullets and a fast twist is probably going to make it harder for the lands to get a good grip and could cause accuracy to suffer and result in barrel leading.

I was thinking more for rounds which can't both expand and penetrate - 380/snubbie 38 etc.

I'm not really sure what you are asking or insinuating, but I would think that as with rifle barrels, pistol barrels have twist rates to stabilize bullets designed to be fired in that firearm. Longer for caliber bullets need a faster twist rates than shorter/lighter bullets. Faster twist rates do not increase muzzle velocity, in fact just the opposite. Not sure that twist rate has any significant impact on penetration.

A faster twist rate, may mean increased rotational energy. I could see a faster twist rate decreasing the MV of a rifle, especially if its long enough to capture all the energy of a charge - but the same increased rotation, along with the increased drag, should reduce the acceleration, which would increase the amount of time the powder has to burn in the barrel - I could see this resulting in an overall increase of kinetic energy/possibly MV as well - even if it attains the same MV, significantly increasing the rate of rotation should lead to an significantly increased inertia.

The increased kinetic energy, with the same frontal area, should increase penetration,

I'm not sure how the increased rotational kinetic energy/increased drag/increased dwell time would balance out - hence, the question.

With a pistol, especially the short-barreled ones I'm thinking of, I don't know that the increased rotation will have a significant impact on group size - I know it can be a problem with rifles.

Reminder on rotational kinetic energy
Reminder on Rotational Inertia
Reminder on angular momentum
Muzzle velocity and RPM

Well I am no physicist and can't address your issues point by point, but can't see how twist rate alone that is faster than determined optimum for that firearm and ammunition can increase velocity or energy, nor group size, in a significant manner.