Dingoonity.org

Dingux (Dingoo Linux) => Development => Topic started by: joyrider on October 17, 2009, 04:24:29 pm

Title: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: joyrider on October 17, 2009, 04:24:29 pm
okay this is a very advanced topic and i only just recently found out about it myselve with the help of sms_sdl emulator which ran really crap under dingux and i couldn't figure out why.

people should start with reading this :
http://www.ibm.com/developerworks/library/pa-dalign/
or 
http://lecs.cs.ucla.edu/wiki/index.php/XScale_alignment (even though it's arm related)

now that you know what alignment issues are you might think what does this have to do with dingux ???

well the problem (we didn't know about until today) is that the mips linux kernel seems to handle aligment issues for us but at the cost of speed. in other words it will decrease performance a lot on software that got affected by it.
so now that we know al of this, how can we check if our software has the problem ? It's really easy add the following include file in your main c/cpp file :
#include <sys/sysmips.h>

and in your main function add the following call :
sysmips(MIPS_FIXADE, 0,0,0);
what this function does is disabling the kernel from intercepting the unalligned memory access and handling it by itselve.
recompile your program and run it.
If the program runs fine without crashing your program is not affected by the problem, however if your program crashes with a "Bus Error" message your program is affected by this issue and you should seek out a way to align your data (examples are given on both sites i mentioned above).

to see what kind of an performance impact this has lets take sms_sdl i had compiled it with and without memory alginement and here are the results :
Code: [Select]
/boot # ./sms_sdl ./sonic2.sms  --fullspeed --fskip 0
SMS Plus/SDL v0.9.4aR7 (Build date Oct 17 2009)
(C) Charles Mac Donald in 1998, 1999, 2000
SDL Version by Gregory Montoir ([email protected])

Loaded `./sonic2.sms'.
Initializing SDL... Ok.
Initializing SDL TIMER SUBSYSTEM... Ok.
Initializing SDL CONTROLS SUBSYSTEM... Ok.
Initializing SDL VIDEO SUBSYSTEM... Ok.
Initializing SDL SOUND SUBSYSTEM... Ok.
Initializing virtual console emulation... Ok.
Starting emulation...
Shutting down virtual console emulation...
Shutting down SDL...
[INFO] Frames rendered = 1658.
[INFO] Average FPS = 52.79 (87%).
[INFO] Play time = 31.41 sec.
/boot # ./sms_sdl_aligned  ./sonic2.sms  --fullspeed --fskip 0
SMS Plus/SDL v0.9.4aR7 (Build date Oct 17 2009)
(C) Charles Mac Donald in 1998, 1999, 2000
SDL Version by Gregory Montoir ([email protected])

Loaded `./sonic2.sms'.
Initializing SDL... Ok.
Initializing SDL TIMER SUBSYSTEM... Ok.
Initializing SDL CONTROLS SUBSYSTEM... Ok.
Initializing SDL VIDEO SUBSYSTEM... Ok.
Initializing SDL SOUND SUBSYSTEM... Ok.
Initializing virtual console emulation... Ok.
Starting emulation...
Shutting down virtual console emulation...
Shutting down SDL...
[INFO] Frames rendered = 1665.
[INFO] Average FPS = 122.34 (203%).
[INFO] Play time = 13.61 sec.
/boot #

as you can see the unaligned version runs not even fullspeed it runs at 52 fps (87%) (the normal speed is 60 fps) while the alginment version runs at a whooping 122.34 fps average (203%).
So running a program that is affected with this problem will run very very bad with a lot of slowdowns.

I know of at least 1 other emulator that has this problem snes9x-sdl ! so speed could improve in snes9x-sdl as well given we align the data. i'll look into that some time or contact the current author thats currently doing snes9x ports so he knows about this.

Hope this helps !

Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: vimrc on October 17, 2009, 04:48:41 pm
Thank you. Now I suspect this is the reason why the DOSBOX crashed. 
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: darklegion on October 19, 2009, 12:53:03 am
I know of at least 1 other emulator that has this problem snes9x-sdl ! so speed could improve in snes9x-sdl as well given we align the data. i'll look into that some time or contact the current author thats currently doing snes9x ports so he knows about this.

Hope this helps !


He's Japanese, and his site is here:  http://www9.atpages.jp/~drowsnug95/
Hopefully he understands some amount of English, otherwise someone will have to be a translator.Full (or near full speed) SNES is one of the top things missing from the Dingoo, so if fixing the alignment issues gives the same level of performance increases as sms-sdl, full speed snes should be doable.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: quadomatic on October 19, 2009, 02:27:31 am
I hope he does understand enough english...he probably does, as I would imagine much of the documentation that would teach one how to program on platforms like Dingoo is in english.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: Goon8 on October 19, 2009, 02:57:04 am
Exciting news!

Hopefully it'll be implemented as quick as everything else gets done in this scene! :P

We'll be running SNES in full speed in no time!! WOOHOO!!  8)
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: qbertaddict on October 19, 2009, 07:58:59 am
Are you going to release the aligned version of sms_sdl?  ;D ;D ;D
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: alekmaul on October 19, 2009, 08:57:13 am
Nice to hear that tips.
joyrider, do you think this can also apply for Dingoo OS Native dev also ?
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: joyrider on October 19, 2009, 05:35:37 pm
Well i don't know if the original os (ucos2) does something similar or how it gets handled there so i really don't know alekmaul :/

about sms_sdl yes i'll release the aligned version, but i'm still working on it, i'm basicly implementing the same options as i did with gnuboy (same menu & stuff).
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: drowsnug on October 20, 2009, 03:00:45 pm
Hi, joyrider.

i am drowsnug from Japan.

I've released snes9x source code in my site,
http://www9.atpages.jp/~drowsnug95/?p=252#comments
Please optimise it! (I've tried, but failed :'()

Thank you.

Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: zear on October 20, 2009, 03:09:17 pm
Hello there drowsnug.
Thanks for your work on snes9x and for releasing the source code.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: dukemagus on October 20, 2009, 03:25:45 pm
YEAH! the demigods are joining! soon dingoo will reach it maximum power...
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: quadomatic on October 20, 2009, 05:48:24 pm
awesome, snes9x source code!!!

I hope there are some improvements soon...I've been hoping for fullspeed snes emulation since pocketsnes on gp2x
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: joyrider on October 21, 2009, 12:38:24 am
well i looked at the source code, and unfortunatly the code for allignements was already in place an being used, the bus error came from an unaligned struct in ppu.h (being struct SPPU)

which can be fixed by adding a "dummy padding" var to it like this :
Code: [Select]
    uint8  OAMReadFlip;
    uint16  padding;
    uint8  OAMData [512 + 32];

as i said the code for the alignment stuff was there and is being used, it wouldn't have been used if the FAST_LSB_WORD_ACCESS define would have been defined. so it was alligned al the time. except for that struct (which was made the OAMDATA being unaligned). You can see this in for example following code :
Code: [Select]
#ifdef FAST_LSB_WORD_ACCESS
   //unaligned
    long OpAddress = (*(uint32 *) cpu->PC + reg->X.W) & 0xffffff;
#else
   //aligned
    long OpAddress = (*cpu->PC + (*(cpu->PC + 1) << 8) + (*(cpu->PC + 2) << 16) + reg->X.W) & 0xffffff;
#endif

I did try some other stuff like no flipping of surfaces and a HW_SURFACE which gave me 3-4 fps increase, unfortunatly it has a side effect. Whenever pixel data gets written twice on the same frame on the same pixel it will flicker. (and the snes does seem to do this) so it's not really worth it.

The only thing i can try is to add the scaler from mame4all which i have gotten from alekmaul & used in gnuboy & sms_sdl to be used in snes9x to see ifs any faster. But that's about it, drowsnug did a great job already :)
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: Goon8 on October 21, 2009, 01:30:15 am
Wow!

Good work joyrider & drowsnug. Can't wait to test an updated version!

I hope we can get full speed or near, in the future..  ;D
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: darklegion on October 21, 2009, 02:20:27 am
Is snes9x using profiling? It could be a nice easy speedup, if it isn't already using it.You can do it with GCC4 pretty easily:
http://www.gp32x.com/board/index.php?/topic/28490-advanced-optimization-via-profiling-with-gcc4/page__st__15

EDIT: Just checked the source, and there are .gcno files so it's already using profiling.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: quadomatic on October 21, 2009, 03:28:53 am
any chance psx4all could see improvements from fixing misalignment issues? (:D just hoping for an easy framerate improvement)
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: joyrider on October 21, 2009, 12:53:02 pm
It all depends where the misalignments happen in the code and if there are any to begin with since not all software has the issue. If the source is included of the software in question, you can easily test it with the sysmips call which disables the kernel trapping. If the application crashes with a "bus error" you know it has problems. In order to detect where exactly gdb is needed (which is included in the rootfs) You just run the software through gdb with the kernel trapping of the alignment issues disabled and gdb will show you the exact line where it happens. Unfortunatly with snes9x it was in a place that doesn't have any impact on the emulation speed, the struct i told about in my previous post. However with sms_sdl it was happening in the render code so you can probably guess it was happening quite often, thus the impact & speed increase gained by alligning reads / writes in sms_sdl
So it all depends on the software and where it happens, it's not some magical thing to make a software go twice as fast all of a sudden (unless its in a crucial part of the code).

So full speed snes won't happen, at least not by doing this, someone could try mips asm instructions but i don't know anything about that, so don't expect it to happen soon and neither being done by me :)
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: darklegion on October 22, 2009, 03:14:14 am
any chance psx4all could see improvements from fixing misalignment issues? (:D just hoping for an easy framerate improvement)

I tried joyrider's check for misalignment issues, and psx4all compiled and ran fine without a bus error.I'm not a programmer though, so I'm not 100% I did it correctly.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: quadomatic on October 23, 2009, 01:05:04 am
someone could try mips asm instructions but i don't know anything about that, so don't expect it to happen soon and neither being done by me :)

Any chance any ASM used on the PSP snes emulator would help (PSP does use a MIPS processor, right?)?
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: vimrc on October 25, 2009, 03:13:13 pm
It all depends where the misalignments happen in the code and if there are any to begin with since not all software has the issue. If the source is included of the software in question, you can easily test it with the sysmips call which disables the kernel trapping. If the application crashes with a "bus error" you know it has problems. In order to detect where exactly gdb is needed (which is included in the rootfs) You just run the software through gdb with the kernel trapping of the alignment issues disabled and gdb will show you the exact line where it happens. Unfortunatly with snes9x it was in a place that doesn't have any impact on the emulation speed, the struct i told about in my previous post. However with sms_sdl it was happening in the render code so you can probably guess it was happening quite often, thus the impact & speed increase gained by alligning reads / writes in sms_sdl
So it all depends on the software and where it happens, it's not some magical thing to make a software go twice as fast all of a sudden (unless its in a crucial part of the code).

So full speed snes won't happen, at least not by doing this, someone could try mips asm instructions but i don't know anything about that, so don't expect it to happen soon and neither being done by me :)

Just an additional post:

I also found an interesting code from Notaz's homepage:
http://notaz.gp2x.de/other.php#GP2X

You can get it from:
http://notaz.gp2x.de/releases/gpSP-0.9-2Xb-u6_src.tar.bz2

extract it and you will find in the 'test' directory, there are two files named align_test.c and load_imm_test.c, seems quite helpful.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: Exophase on October 30, 2009, 02:59:09 pm
Just an additional post:

I also found an interesting code from Notaz's homepage:
http://notaz.gp2x.de/other.php#GP2X

You can get it from:
http://notaz.gp2x.de/releases/gpSP-0.9-2Xb-u6_src.tar.bz2

extract it and you will find in the 'test' directory, there are two files named align_test.c and load_imm_test.c, seems quite helpful.

I wrote those to test what ARM9 does with misaligned loads and stores w/o kernel intervention. It was important since it was trying to use the system's behavior for emulating ARM7 misalignment as much as possible. If you want to use it just to cause a misalignment fault then it'll work but really it's pretty easy to do that anyway..

load_imm_test.c isn't related to any of this.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: Kalisiin on October 30, 2009, 06:31:05 pm
awesome, snes9x source code!!!

I hope there are some improvements soon...I've been hoping for fullspeed snes emulation since pocketsnes on gp2x

LOL, lucky.  I can't even make SNES9x work on my Dingoo...in the Dingux application.  All I get is a flash of red and pink screen.
So I just use the native for SNES.  It works fine.

I'd love to see ZS-NES, though, I like that one better than SNES9x.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: dukemagus on October 31, 2009, 12:13:25 am
zsnes are meant to be used on x86 processors...

no way it lands on dingoo
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: Kalisiin on October 31, 2009, 04:20:09 am
zsnes are meant to be used on x86 processors...

no way it lands on dingoo

Oh, well...didn't know that.  Well, at any rate, I can't make the pre-packaged SNES9x work on Dingux, not at all.  It never worked at all.  Part of the package, but it never worked, don't ask me why.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: Rebelphoenix on November 06, 2009, 11:07:51 am
SNES9x doesn't work for me either. Just goes to a blank screen that I can't do anything with.
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: Bouvrie on November 12, 2009, 09:06:19 am
Thank you. Now I suspect this is the reason why the DOSBOX crashed. 
Are you referring to DosBox, the DOS emulator? Is it being ported to the Dingoo liketo  the gp2x Wiz (http://www.gp32x.com/board/index.php?/topic/49585-dosbox-game-test/page__hl__dosbox)? It'd be great!
Title: Re: Watch out for misallignment issues !!!! (they slow down your software a lot)
Post by: abhoriel on November 15, 2009, 09:48:56 pm
I looked at the zsnes code, and it looks pretty unportable. A huge amount of it is written in x86 assembly (the CPU core, etc... even a lot of the UI code!) unfortunately.

It might be worth looking into the PSP version of snes9x though, to see if they optimised it for MIPS