Changing the resolution of the screen doesn't automatically scale the game.
If the original game runs in 480x420, it means you'd have to scale the screen output to 320x240.
So far you've set the screen surface to 320x240, but the rest of the game still tries to draw off screen into the original 480x420.
Oh, I see you've used the scaler. It is slow because the game first renders to 480x420 and then software scales the whole thing to 320x240. Software scaling will always be slow. Another thing you could do is to resize all the graphics and modify the code and change all the appropriate variables to work with the scaled graphics.