Managed to get the higher sample rates working. The default format and a few others used too much cpu, but dumping to binary worked fine for high rates. The 1 MHz was indeed far too inaccurate.
I tested pin 21's variance, how long it stayed high, at 25 MHz. While not quite 3 us, the variance is still massive.
min 40 ns, avg 75.05 ns, max 360 ns
Wifi connected but not transferring:
min 40 ns, avg 75.09 ns, max 400 ns
Wifi + data:
min 40 ns, avg 76.70 ns, max 880 ns
The cause for big spikes, at least my current guess, is memw waiting for the other core's flash cache miss.