Scratchbox

From The Neuros Technology Wiki

Jump to: navigation, search

You want to compile some shiny package for your OSD, but no matter how much you tweak with the package Makefile and environment variables, you can't get past cross compilation hell.

Scratchbox might be a solution for your problems. It tricks the build process in thinking you are building directly on the OSD. Instead it will be running in an emulated ARM processor under QEMU.

Now, before starting, a big warning: the scratchbox folks really love Debian. It seems the only way to get scratchbox up an running is if you have a debian (or debian-derived) system. People even say "scratchbox on non-debian is evil". Evil.

I did this on Ubuntu Dapper and it worked. More power to you if you want to try on something else.

Installing on Gentoo turned out to be a real pain and not worth the effort. I ended up making a minimal Ubuntu chroot for it.

Contents

Installing

You will need to install the following packages for Ubuntu (They may work for other apt-based systems). First you will need to add scratchbox's apt repository to your /etc/apt/sources.list file:

deb http://scratchbox.org/debian stable main

Scratchbox will install in /scratchbox and eats up at least 1.5Gb. You can create /scratcbox as a symlink to somewhere with more space before installing the package if you have disk space issues.

Then install (sudo apt-get install packageName) the following packages:

# qemu                                     (the actual emulator)
# scratchbox-core                          (sbox core environment, common tools and host compiler)
# scratchbox-libs                          (libraries required by core, devkits and toolchains)
# scratchbox-devkit-cputransp              (makes sbox use qemu)
# scratchbox-toolchain-arm-gcc3.4-glibc2.3_1.0.2 (the GNU toolchain) 

This toolchain is used for this guide because neuros-bsp (which we are NOT using here, see the TODO section below) uses libc 2.3.something so it seems the most compatible.

There WAS a version of the toolchain with a closer-match gcc, but it seems to be on longer distributed. Ideally, we'd make our own using the foreign-toolchain support (in scratchbox) from neuros-bsp. --Nick 14:22, 20 November 2006 (CST)

Well, some sbox folks informed me that these old toolchains can still be downloaded from their site, so i put the old toolchain name back --nerochiaro 14 January 2007

To allow compiling against libncooler.so, I had to do this:

rm -rf /scratchbox/compilers/arm-linux-ct401-2.3/arm-unknown-linux-gnu/include
cp -ar neuros-bsp/toolchain/arm-linux/include /scratchbox/compilers/arm-linux-ct401-2.3/arm-unknown-linux-gnu
cp -a neuros-bsp/toolchain/arm-linux/lib/libncooler.so neuros-bsp/toolchain/arm-linux/lib/libcomnmsc.so /scratchbox/compilers/arm-linux-ct401-2.3/arm-unknown-linux-gnu/lib

which is probably rather nasty. --Nick 14:22, 20 November 2006 (CST)

Setting up sbox

Sitwon, who kindly walked me across this setup process, said this differs much from the sbox official documentation, so beware.

Let's suppose for simplicity that you're running as user yesyou and that you want to use that user for scratchbox too. Run this command:

/scratchbox/sbin/sbox_adduser yesyou

And say yes when it asks about adding the user to the sbox group. Then logoff and log back in, and check with groups that you're really into the sbox group.

If so, you can enter the scratchbox environment with

/scratchbox/login  #This should bring you inside a scratchbox shell
sb-menu

Follow these steps in the menu:

# Setup a target
# Setup a new target
# name it something you like. I just picked "OSD"
# Select "arm-linux-ct401-2.3" as compiler
# Select the "cputransp" devkit
# Done selecting
# Select "qemu-arm-0.8.1-sb2" emulation mode (see TODO below for "sbrsh")
# "Do you wish to extract a rootstrap on the target?" > No
# "Do you wish to install files to the target?" > Yes
# Pick "C-library, /etc, Devkits, fakeroot" (the defaults), then OK
# "Do you wish to select the target ?" > Yes

And it's done. You will be dropped into a new sbox shell. Stuff you run in there will believe to be running on the OSD. As best as qemu can fake it, at least. Actually not the OSD but a generic arm-unknown machine.

Using sbox for cross compiling

You have gone all this way, and now it's time to use it. You have to remember that sbox cannot reach outside its chrooted environment, but the external environment can reach in. The following two directories should be noted (it may be helpful to symlink them from somewhere more memorable):

(A) /scratchbox/users/$YOURUSER/home/$YOURUSER/ (your home directory inside sbox) 
(B) /scratchbox/users/$YOURUSER/targets/$YOURTARGET/ (the root of your fake ARM env filesystem)

It generally goes like this:

  1. You copy stuff into (A) from outside sbox.
  2. You login into sbox with your user and configure and make your stuff.
  3. You make install your stuff.
  4. From outside sbox you go into (B), and you notice that you have a pristine tree with just the output files in it, in the right directories.
  5. You copy that tree from (B) over to the real OSD filesystem.
  6. Or if your stuff are dev libs, you copy that tree to the $TOOLCHAIN/arm-linux to be used from the outside of sbox with the Neuros toolchain.

Don't try to run what you built inside sbox. It won't work. Copy it to the OSD and go.

Resetting the environment

Maybe you have messed up a lot with a package and now you want to compile another, but have all the output of your previous work in the sbox installation tree ((B) in the chapters above).

To get back to a pristine tree you need to "reset" the target.

sb-menu
# Reset (Reset a target)
# pick your target
# "Are you sure you want to delete all files of target ?" > Yes
# "Do you wish to reselect the target?" > No
# Install (Install files to a target)
# pick your target
# Pick "C-library, /etc, Devkits, fakeroot" (the defaults), 

You can also do the same thing quicker with a single shell command (two actually). Assuming your target is called "osd" all you need to do is:

sb-conf reset --force osd && sb-conf install --clibrary --etc --devkits --fakeroot osd

You will now have a clean install tree again.

Please notice that resetting a target will not touch anything in your sbox home ((A) in the chapters above). Anything you have in there will remain as it was.

Installing cross-compiled software

When you are cross-compiling your package, you may wonder just how exactly you will install the package. There is a simple way to do this which does not use --prefix=/path/here! Please do not use prefix.

cd foo
./configure
make
make DESTDIR=/home/sbuser/packages/foo-osd install

Adding DESTDIR installs the full path of the package to foo-osd with the complete directory heirarchy. This can be copied into your NFS root, or tarred up and expanded later. Scons has a similar install atrribute.

TODO

  1. Investigate "sbsrh". It allows you to use the real OSD instead of qemu. tradeoff here is in accuracy of emulation v/s speed (emulation+Ghz speed of your PC v/s no emulation+continental drift speed of the OSD)
  2. Investigate the use of Neuros toolchain inside sbox. The sbox one seems pretty compatible so far, but there migt be mismatch that cause subtle problems and much pain down the road. However everyone seems to agree that making custom toolchain for sbox involves large amounts of cursing and tears.

Successes

I managed to compile and use all these items this way:

Python-2.5.tar.bz2 ipython-0.7.2.tar.gz libmad-0.15.1b.tar.gz mpg321-0.2.10.tar.gz strace-4.5.14.tar.bz2 zlib-1.2.3.tar.gz gdb-6.5.tar.bz2 libid3tag-0.15.1b.tar.gz madplay-0.15.2b.tar.gz readline-5.2.tar.gz termcap-1.3.1.tar.gz

(except libao does crash when trying to play music, so it likely still needs work...) --Nick 14:22, 20 November 2006 (CST)

ruby-1.8.5 sqlite-3.3.8 flac-1.1.2 libogg-1.1.3 ffmpeg-SVN-r6716 mpd-0.12.1 --Xorlev 23:36, 29 November 2006 (CST)

samba-3.0.23d wget --Crweb 22:22, 27 January 2007 (CST)

Personal tools