Maybe the whole 64 MB of the swap file are seldom used, but I think the OS doesn't write the whole file all the time, it only uses, what it needs, or am I wrong? And in this case, 64MB may be more than I need in 99% of the time, but it doesn't bother me, cause with a 32MB swap file, I would only save 32MB on my card.
But time will tell, probably you are right and 32MB are enough in 100% of all times.