Author Topic: RetroFW - Developer Support Thread  (Read 7348 times)

jbanes (OP)

  • Posts: 91
RetroFW - Developer Support Thread
« on: February 16, 2019, 02:47:34 am »
Interested in developing for the RS-97? Heard about this new RetroFW firmware and want to help get things ported? Then this thread is for you!

Once you've downloaded RetroFW, run the correct .bat script, and flashed your internal SD card, you're ready for the first step.

Tutorial: Configuring a Toolchain for RetroFW Development

This tutorial covers:

  • How to create a compiler toolchain
  • Setting up an IDE
  • Integrate IPK package creation into your build process
  • The difference between Emulators and Systems (and why you should support both!)
  • GPL Considerations
  • Collaborative development
  • ...and more!


With all the programs that need to be ported to RetroFW, a point of confusion has been "who's doing what" and "how can I help?" So with the help of many key developers I've created a document that captures who's working on what, what state it's in, and what is open for adoption.

Project Status: RetroFW ports

This document is open and editable. So if you see something wrong, change it.  If you've started working on an unowned project, add your name so everyone else knows who to collaborate with. If you're porting something new, add it!

The goal is to make this a tool for the community to  successfully manage the RetroFW ecosystem. Please let me know if you have any questions or if there are ways I could improve this tool.

--

Special thanks to pingflood, sauce, TonyJih, and many others for helping me prepare these documents. Additional thanks to 12345rStreet for cataloging all the software available for the RS-97.
« Last Edit: February 16, 2019, 02:59:23 am by jbanes »

benob

  • Posts: 16
Re: RetroFW - Developer Support Thread
« Reply #1 on: February 21, 2019, 08:09:21 am »
That's awesome, thanks.

Is adding IPU support just a matter of adding HW_SURFACE to SDL_SetVideoMode()?
Is it possible to get scaling with IPU (i.e. 640x480 => 320x240)?
What are the constraints on application icons (32x32 .png with transparency?)
Who should I send my ipk files so that they get released through the proper channels?

Are there plans for leveraging the g_ether module and provide network access to the consoles? That could be awesome for downloading and installing ipk packages.

jbanes (OP)

  • Posts: 91
Re: RetroFW - Developer Support Thread
« Reply #2 on: February 21, 2019, 12:11:57 pm »
That's awesome, thanks.

You're welcome! :)

Quote
Is adding IPU support just a matter of adding HW_SURFACE to SDL_SetVideoMode()?

Correct. As long as it's a hardware surface of any size that's 320x240 or smaller, the IPU will scale it to fit according to its aspect ratio.

Quote
Is it possible to get scaling with IPU (i.e. 640x480 => 320x240)?

The kernel and firmware devs are working on adding that support now. At the moment the IPU only does basic upscaling with no deformation support. Downscaling and deformation are planned for future releases. (Hopefully near-future. :) )

Quote
What are the constraints on application icons (32x32 .png with transparency?)

Using the default screen layout, icons are 32x32 PNG files with transparency. Any programs you ship should have a default icon that matches those parameters. However, there are skins that change the grid size, allowing for much larger icons. Those skins have icons to override the defaults.

Quote
Who should I send my ipk files so that they get released through the proper channels?

You can post a link in the releases thread or on the Discord channel and someone with access will be happy to move it to the official IPK folder.

Quote
Are there plans for leveraging the g_ether module and provide network access to the consoles? That could be awesome for downloading and installing ipk packages.

There is a developer mode for networking over the USB cable. Also, you can just mount the device to your PC/Mac using the cable. However, the device isn't really capable or expandable for networking beyond that.

bopbopbop

  • Posts: 87
Re: RetroFW - Developer Support Thread
« Reply #3 on: February 22, 2019, 09:29:27 am »
Hi JBanes!

1. In your guide to IPKs you mention contents and contents.tar.gz - I think these should be control and control.tar.gz

2. I've made a package, and it installs (and even better it works!) but when it installs it gives a bunch of warnings about time and user permissions - "warning, cannot set home to user=1000,group=1000", etc. I presume it's trying to overwrite the files /home, /home/retrofw, etc, with the versions from the tar file. How can I get it not do whatever it is doing to stop the warnings?


jbanes (OP)

  • Posts: 91
Re: RetroFW - Developer Support Thread
« Reply #4 on: February 22, 2019, 12:09:25 pm »
1. In your guide to IPKs you mention contents and contents.tar.gz - I think these should be control and control.tar.gz

Thanks for catching that! The tutorial is now fixed. ;D

2. I've made a package, and it installs (and even better it works!) but when it installs it gives a bunch of warnings about time and user permissions - "warning, cannot set home to user=1000,group=1000", etc. I presume it's trying to overwrite the files /home, /home/retrofw, etc, with the versions from the tar file. How can I get it not do whatever it is doing to stop the warnings

The good news is that it's not trying to overwrite these folders. However, the interesting thing about TAR files is that they capture filesystem attributes like owner, group, read, write, and execute. So what's happening is that the package manager is trying to set these back to whatever was captured when you made the tarball. Which it's not allowed to do because the file system is read-only.

FWIW, these errors are mostly harmless and plague pretty much all of the packages out there. But if it bothers you enough, you can adjust your TAR command to fix it. For example:

Code: [Select]
tar -czvf data.tar.gz --owner=0 --group=0 home
https://unix.stackexchange.com/questions/285237/tar-without-preserving-user

That should wipe out the ownership when making the tar and prevent the errors from occurring.

bopbopbop

  • Posts: 87
Re: RetroFW - Developer Support Thread
« Reply #5 on: February 22, 2019, 12:36:22 pm »
Ace, thanks!

podulator

  • Posts: 8
Re: RetroFW - Developer Support Thread
« Reply #6 on: June 01, 2019, 08:02:57 am »
Hey @jbanes, thanks so much for the guide.
I got everything setup and commander builds fine. Now i'm trying another project and it can't find the linux std includes (errno.h, zlib.h etc)
I updated my path like so, adding the buildroot path to linux includes
Code: [Select]
echo $PATH
/home/mat/bin:/home/mat/.local/bin:/opt/rs97tools/bin:/opt/rs97tools/mipsel-buildroot-linux-uclibc/sysroot/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/jvm/java-12-oracle/bin:/usr/lib/jvm/java-12-oracle/db/bin:/home/mat/Documents/code/aws/aws-elasticbeanstalk-cli/eb/linux/python2.7:/home/mat/installed_apps/processing/latest:/home/mat/installed_apps/android-studio/bin/:/home/mat/installed_apps/riderRS/latest/bin:/opt/buildroot-2018.02.9/output/build/linux-headers-4.15.18/include
but it's still not resolving them.
Any pointers?
Sorry, i'm a bit rusty, it's been a few years ;)
Thanks!

bopbopbop

  • Posts: 87
Re: RetroFW - Developer Support Thread
« Reply #7 on: June 03, 2019, 02:25:35 pm »
@podulator

By default the guide doesn't select zlib, so unless you built it youll need to compile eit as a package. You can compile it without recompiling the whole toolchain - I did this the other day. I think its

 make zlib-package

, and then an additional make for it to add it to your toolchain.

Then if its still not picking it up, try to -I the directory of your libraries on either the Cflags or the lflags, I cant remember, to tell it the path to look.

Good luck!

podulator

  • Posts: 8
Re: RetroFW - Developer Support Thread
« Reply #8 on: June 03, 2019, 07:59:52 pm »
of course! thanks for the reminder @bopbopbop  :)
I haven't looked at buildroot since gp2x days.
Just in case anyone else is in the same boat, add the libzlib library, and make, then link the libzlib-x.y.z folder to zlib in /opt/buildroot-2018.02.9/output/build/ and -I include it

nes6502

  • Posts: 2
Re: RetroFW - Developer Support Thread
« Reply #9 on: June 04, 2019, 09:54:21 pm »
hello, developers, i've used Win10+sub-system(linux) envirment, I tried to compile buildroot-2018.02.9 for RS-97 toolchain, have some errors, could you help me?

error log:
Code: [Select]
printf '        /bin/busybox                     f 4755 0  0 - - - - -\n /dev/console c 622 0 0 5 1 - - -\n\n' >> /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/device_table.txt
echo "/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/bin/makedevs -d /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/device_table.txt /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/target" >> /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
printf '        (cd /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/target; find -print0 | LC_ALL=C sort -z | tar  -cf /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/images/rootfs.tar --null --xattrs-include='\''*'\'' --no-recursion -T - --numeric-owner)\n' >> /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
chmod a+x /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
rm -f /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
PATH="/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/bin:/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/sbin:/opt/rs97tools/bin:/opt/rs97tools/mipsel-buildroot-linux-uclibc/sysroot/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Program Files/WindowsApps/CanonicalGroupLimited.UbuntuonWindows_1804.2019.521.0_x64__79rhkp1fndgsc:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Windows/System32/OpenSSH:/mnt/c/Users/yefeng/.dnx/bin:/mnt/c/Program Files/Microsoft DNX/Dnvm:/mnt/d/Program Files/MATLAB/R2012a/runtime/win64:/mnt/d/Program Files/MATLAB/R2012a/bin:/mnt/c/Program Files (x86)/Windows Kits/8.1/Windows Performance Toolkit:/mnt/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/mnt/c/Users/yefeng/AppData/Local/Microsoft/WindowsApps:/snap/bin" /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/bin/fakeroot -- /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
fakeroot, while creating message channels: Function not implemented
This may be due to a lack of SYSV IPC support.
fakeroot: error while starting the `faked' daemon.
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
fs/tar/tar.mk:14: recipe for target '/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/images/rootfs.tar' failed
make[1]: *** [/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/images/rootfs.tar] Error 1
Makefile:84: recipe for target '_all' failed
make: *** [_all] Error 2

podulator

  • Posts: 8
Re: RetroFW - Developer Support Thread
« Reply #10 on: June 05, 2019, 08:04:44 am »
Hey nes6502, this bug report might help you

https://github.com/Microsoft/WSL/issues/2465

podulator

  • Posts: 8
Re: RetroFW - Developer Support Thread
« Reply #11 on: June 05, 2019, 08:06:13 am »
Anyone got a suggestion of how to set up a test environment on x86_64 linux for testing built mips binaries locally without bouncing to LDK?
Thanks!

bopbopbop

  • Posts: 87
Re: RetroFW - Developer Support Thread
« Reply #12 on: June 05, 2019, 10:42:06 am »
@nes6502 you need to run this command before running make. I think you can run the comand, and then resume make (as in, not do a make clean if you haven't already).

https://stackoverflow.com/questions/45469650/error-while-building-rootfs-system-for-raspberry-pi-3-using-buildroot-lack-of#46480886

Be aware I had issues with g++ even after it built, but only for one really complex project. Everything else seems to work well in the linux windows subsystem.

Also, nice username!  :)

hello, developers, i've used Win10+sub-system(linux) envirment, I tried to compile buildroot-2018.02.9 for RS-97 toolchain, have some errors, could you help me?

error log:
Code: [Select]
printf '        /bin/busybox                     f 4755 0  0 - - - - -\n /dev/console c 622 0 0 5 1 - - -\n\n' >> /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/device_table.txt
echo "/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/bin/makedevs -d /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/device_table.txt /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/target" >> /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
printf '        (cd /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/target; find -print0 | LC_ALL=C sort -z | tar  -cf /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/images/rootfs.tar --null --xattrs-include='\''*'\'' --no-recursion -T - --numeric-owner)\n' >> /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
chmod a+x /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
rm -f /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
PATH="/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/bin:/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/sbin:/opt/rs97tools/bin:/opt/rs97tools/mipsel-buildroot-linux-uclibc/sysroot/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Program Files/WindowsApps/CanonicalGroupLimited.UbuntuonWindows_1804.2019.521.0_x64__79rhkp1fndgsc:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Windows/System32/OpenSSH:/mnt/c/Users/yefeng/.dnx/bin:/mnt/c/Program Files/Microsoft DNX/Dnvm:/mnt/d/Program Files/MATLAB/R2012a/runtime/win64:/mnt/d/Program Files/MATLAB/R2012a/bin:/mnt/c/Program Files (x86)/Windows Kits/8.1/Windows Performance Toolkit:/mnt/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/mnt/c/Users/yefeng/AppData/Local/Microsoft/WindowsApps:/snap/bin" /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/host/bin/fakeroot -- /mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/build/buildroot-fs/fakeroot.fs
fakeroot, while creating message channels: Function not implemented
This may be due to a lack of SYSV IPC support.
fakeroot: error while starting the `faked' daemon.
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
fs/tar/tar.mk:14: recipe for target '/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/images/rootfs.tar' failed
make[1]: *** [/mnt/e/game/rs97/mydev/RS-97/buildroot-2018.02.9/output/images/rootfs.tar] Error 1
Makefile:84: recipe for target '_all' failed
make: *** [_all] Error 2

nes6502

  • Posts: 2
Re: RetroFW - Developer Support Thread
« Reply #13 on: June 05, 2019, 02:23:03 pm »
@bopbopbop  @podulator  First of all,  thks for solving my problem, the toolchain  "mips-linux-gcc / g++" and so on...  is generated and work well  ;D .   i can compile some project, everything is ok, no errors, just can't build out image "rootfs.tar". it doesn't matter.

podulator

  • Posts: 8
Re: RetroFW - Developer Support Thread
« Reply #14 on: June 06, 2019, 04:58:55 pm »
Anyone got a suggestion of how to set up a test environment on x86_64 linux for testing built mips binaries locally without bouncing to LDK?
Thanks!

So, to partly answer my own question, i found this

https://github.com/nongiach/arm_now

but it's not recognising binaries that run on cfw.
i'm using mips32el as the cpu.

Anyone had any joy getting this working?
looks to be a fast way of testing if it worked...

cheers!

bopbopbop

  • Posts: 87
Re: RetroFW - Developer Support Thread
« Reply #15 on: June 06, 2019, 11:59:30 pm »
@podulator  Sounds like you're already making excellent progress, but it seems like it could be quite a bit of hassle to get working properly. It would be amazing if you manage it, however.

You know you can just usb mount your rs97 and the copy binaries across from the command line? You have to dismount to run them, but you can always copy them straight to gmes or emu and have an icon pre set up to run them. It's not terribly slow to do so.

For the platformer I'm building, I just built a sdl 1.2 engine that has two makefile targets - one for windows, and one for cfw. It's only a command switch difference.

What are you trying to build? New software, or are you looking to port something?


podulator

  • Posts: 8
Re: RetroFW - Developer Support Thread
« Reply #16 on: June 08, 2019, 09:53:24 am »
You know you can just usb mount your rs97 and the copy binaries across from the command line? You have to dismount to run them, but you can always copy them straight to gmes or emu and have an icon pre set up to run them. It's not terribly slow to do so.
Yeah, just looking for a quicker way not requiring a device and a copy....build and run is the dream!

What are you trying to build? New software, or are you looking to port something?

A bit of both :)

cpasjuste

  • Posts: 1
Re: RetroFW - Developer Support Thread
« Reply #17 on: June 14, 2019, 09:00:50 am »
Hi @jbanes,

A very (basic) few questions for you :)

- There's no SDL2 yet for the LDK/RS97 ?
- There's no opengles support right (no gpu) ?

Thanks!

Edit: So there's no SDL2 nor opengles on the LDK. I'm probably going to port SDL2, and report progress here.
« Last Edit: June 14, 2019, 11:30:01 am by cpasjuste »

gameblabla

  • Posts: 1363
Re: RetroFW - Developer Support Thread
« Reply #18 on: June 14, 2019, 03:12:08 pm »
Edit: So there's no SDL2 nor opengles on the LDK. I'm probably going to port SDL2, and report progress here.
SDL2 apparently has a software renderer but they don't have a fbdev backend to support it so you'll have to work on that.
Also they have a DirectFB backend but i could never get it to work : it probably assumes 3D acceleration as well but i could be wrong ?
Good luck if you want to seriously work on it.

pingflood

  • Posts: 84
Re: RetroFW - Developer Support Thread
« Reply #19 on: June 14, 2019, 11:52:45 pm »
- There's no SDL2 yet for the LDK/RS97 ?
- There's no opengles support right (no gpu) ?

Hi @cpasjuste,

SDL2 is working with DirectFB in the upcoming RetroFW v2.0, but it's still being evaluated due to lack of OpenGL.

Join our RetroFW discord, so we can talk directly about this:
https://discord.gg/EEzZ7k

Kind regards

 

Post a new topic