May 3, 2020
Installing solyste on a Turris Omnia
Four years ago, CZ.NIC launched a campaign on Indiegogo to create what they called "The open-source center of your home". The idea sounded kinda interesting to me. I was not really thrilled about the software, I was excited about the hardware. As you can see, the listed features were quite good. Therefore, I decided to pre-order a board. My primary idea was to port my personal Linux distribution called solyste, to it.
Unfortunately, embedded hardware often use custom U-Boot
and also heavily patched Linux kernel. No surprise, it was the
same here. I dislike having to build custom kernels from obscure
and sometimes unmaintained trees, because it is not a long-term
solution. What can we do when manufacturers decide to (silently)
drop the hardware ? We become stuck with old, buggy and flawed
software. For those reasons, I had to wait for patches in
upstream / mainline Linux
repo.
As I write these lines, front LEDs are not really supported by
5.4.x and we can't control them. It seems to work
with OpenWrt and LEDE, though (I didn't test by myself).
Upstreaming support took a long time, but eventually it came later. Even if a few bits are still missing, I knew I could try to boot on the board. Due to the Covid-19 crisis, I had to stay at home like many people across the globe. After all those years, it was the occasion to work on solyste and Omnia, in my spare time.
Kernel
Booting a kernel on the default system was not really hard.
I had to compile it with mvebu_v7_defconfig and
it went surprisingly fine. The device tree file is mandatory too.
The kernel (without loadable modules) will be meticulously tuned
later. Make sure you can access U-Boot prompt with
serial
connection preliminarily. Then, using TFTP I could "send" it to
the board, with the following commands :
tftpboot 0x01000000 zImage
tftpboot 0x02000000 armada-385-turris-omnia.dtb
bootz 0x01000000 - 0x02000000You only need to make sure that your kernel supports Btrfs, to be able to mount the first partition (and to avoid a kernel panic). That's what the vanilla system uses.
Userland
Few years ago, I embraced
musl libc for my own
projects and I never looked back. Rich, by any chance, if you
read these lines, you are truly awesome.
The repository
musl-cross-make
provides a makefile to build the musl cross-compiler. That's
what I use in solyste (before, I was also building the
cross-compiler with my own scripts). Boosted by the previous
success, I wrote the proper code to cross-compile the softwares
for armv7 architecture.
Embedded MMC
It was is the scary part. Managing the eMMC is
sometimes extremely difficult. An innocent tiny mistake can
brick your device for good. What a harsh reward. Turris Omnia
has 8Go NAND flash. It would be very sad to not use all that
space. Anyway, at the moment we simply CAN'T boot from both
USB3, because of U-Boot version. mSATA SSD or mPCIe to USB boot
is
possible,
but it requires you to buy something else, one more time.
Luckily, in addition to the default operating system, a
rescue system is installed in SPI flash. We can exploit it to
install solyste. To switch to rescue, follow this
page.
It is based on a busybox and a few other tools.
There is a script called rescue.sh in
/bin directory to reflash the router, as described
here.
We won't use it because it supports btrfs only.
/ # ls bin/
[ date head mktemp rmdir tr
[[ dd hostid mount sed traceroute
ash df hostname mv seq true
awk dirname id nc setsid umount
basename dmesg ipcalc netmsg sh uname
bunzip2 du kill nice sha256sum uniq
busybox echo killall nslookup sleep unxz
bzcat egrep less pgrep sort uptime
bzip2 expr ln pidof strings vi
cat false lock ping stty wc
chgrp fgrep logger ping6 sync wget
chmod find ls printf tail which
chown free lspci ps tar xargs
clear fsync lsusb pwd tee xz
cmp ftpget md5sum readlink test yes
cp grep mkdir rescue.sh tftp zcat
cttyhack gunzip mkfifo reset time
cut gzip mknod rm touchfdisk binary is present in /sbin
but we still lack e2fsprogs to create
ext4 filesystem. ext4 is perfect for
me (my systems are read-only anyway). Not a problem here as
e2fsprogs is included in base group in
solyste. That means I can easily build a static
mke2fs binary. That's neat, isn't it ?
$ file mke2fs
mke2fs: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, with debug_info, not strippedThat mke2fs binary could be also very useful
in the future. I decided to copy it in /sbin. Maybe
we should ask CZ.NIC to do it, but I feel it would be too much
hassle, as they are focusing on Btrfs. Now, we create the
filesystem :
/ # fdisk /dev/mmcblk0
/ # mke2fs -t ext4 /dev/mmcblk0p1We are (very) good so far.
solyste is finally ready to be deployed in eMMC. Let's
extract the archive stored on an external USB drive, with
tar :
/ # mount -t ext4 /dev/mmcblk0p1 /mnt
/ # tar xvf SOLYSTE-OMNIA.tar.gz -C /mnt
/ # chown -R 0:0 /mnt/*
/ # sync
/ # umount /mntPress the reset button to reboot. Before we edit U-Boot
environment variables, we verify if the ext4
partition is correctly listed :
=> ext4ls mmc 0:1 /
<DIR> 4096 .
<DIR> 4096 ..
<DIR> 16384 lost+found
<DIR> 4096 bin
<DIR> 0 boot
<DIR> 0 dev
<DIR> 0 etc
<DIR> 0 mnt
<DIR> 0 proc
<DIR> 0 root
<DIR> 0 sys
<DIR> 0 tmp
<DIR> 0 varHum, it does not seem to be very good… The directories were simply empty and the files inside them were missing :
=> ext4ls mmc 0:1 /boot
** Can not find directory. **At this point, I thought there was a problem with the eMMC, even if I didn't do something wrong. But it was still correctly detected :
=> mmcinfo
Device: mv_sdh
Manufacturer ID: 90
OEM: 14a
Name: H8G4a
Tran Speed: 52000000
Rd Block Len: 512
MMC version 4.0
High Capacity: Yes
Capacity: 7.3 GiB
Bus Width: 8-bit
Erase Group Size: 512 KiBThanks to TFTP, I could boot on my kernel and the main
partition was totally fine. Something was bad with U-Boot.
Maybe the ext4 implementation is too simple.
In embedded world, filesystems in 64-bit mode can lead to
erratic behavior. I decided to boot once again in rescue mode
to create the ext4 filesystem, in 32-bit this time.
Do you remember when I told you earlier that mke2fs
could be necessary ? So, here we go again :
/ # mke2fs -t ext4 -O ^64bit /dev/mmcblk0p1Then, after extracting the archive and rebooting, the
ext4ls command was more realistic :
=> ext4ls mmc 0:1 /
<DIR> 4096 .
<DIR> 4096 ..
<DIR> 16384 lost+found
<DIR> 4096 bin
<DIR> 4096 boot
<DIR> 4096 dev
<DIR> 4096 etc
<DIR> 4096 mnt
<DIR> 4096 proc
<DIR> 4096 root
<DIR> 4096 sys
<DIR> 4096 tmp
<DIR> 4096 var
=> ext4ls mmc 0:1 /boot
<DIR> 4096 .
<DIR> 4096 ..
17506 armada-385-turris-omnia.dtb
3793992 zImageVICTORY ! As the main ext4 filesystem is
completely detected by U-Boot, we can now setup the boot
itself.
U-Boot variables
The default environment variables can be listed with
printenv. Here the default bootargs
and mmcboot variables. We will modify them :
bootargs=earlyprintk console=ttyS0,115200 rootfstype=btrfs rootdelay=2 root=b301 rootflags=subvol=@,commit=5 rw
mmcboot=setenv bootargs "$bootargs cfg80211.freg=$regdomain"; btrload mmc 0 0x01000000 boot/zImage @; btrload mmc 0 0x02000000 boot/dtb @; bootz 0x01000000 - 0x02000000 First, we will replace bootargs with
setenv. I tried the following, but I had many
kernel panics :
setenv bootargs rootfstype=ext4 root=/dev/mmcblk0p1 rw console=ttyS0,115200The kernel was booting too fast and the eMMC was not
discovered in time. It's also quite common. Let's add
rootdelay. Even default variable has it. I appended
quiet too.
setenv bootargs rootfstype=ext4 root=/dev/mmcblk0p1 rootdelay=2 rw console=ttyS0,115200 quietmmcboot variable is similar to
tftpboot command. Defining
cfg80211.freg makes no sense here, because my board
do not have WiFi. Secondly, btrload is for
btrfs. The ext4 counterpart is
ext4load. With those parameters, we can compose
the variable :
setenv mmcboot "ext4load mmc 0:1 0x01000000 boot/zImage ; ext4load mmc 0:1 0x02000000 boot/armada-385-turris-omnia.dtb ; bootz 0x01000000 - 0x02000000"Do not get fooled by my words, I had to try four or five
times to find the correct value. Once bootargs and
mmcboot variables are setup, we save them with
saveenv command and… we are done ! You can either
push the reset button on the back or directly launch
run mmcboot command.
RAM issues
Usually, when the Omnia was unplugged from electrical source or after a reset, the DDR3 RAM will fail to initialize. Here the output :
Memory config in EEPROM: 0x01
ddr3_tip_pbs_rx failure CS #0
Title: I/F# , Tj, Calibration_n0, Calibration_p0, Calibration_n1, Calibration_p1, Calibration_n2, Calibration_p2,CS0 ,
VWTx, VWRx, WL_tot, WL_ADLL, WL_PH, RL_Tot, RL_ADLL, RL_PH, RL_Smp, Cen_tx, Cen_rx, Vref, DQVref, PBSTx-Pad0,PBSTx-Pad1,PBSTx-Pad2,PBSTx-Pad3,PBSTx-Pad4,PBSTx-Pad5,PBSTx-Pad6,PBSTx-Pa
d7,PBSTx-Pad8,PBSTx-Pad9,PBSTx-Pad10, PBSRx-Pad0,PBSRx-Pad1,PBSRx-Pad2,PBSRx-Pad3,PBSRx-Pad4,PBSRx-Pad5,PBSRx-Pad6,PBSRx-Pad7,PBSRx-Pad8,PBSRx-Pad9,PBSRx-Pad10,
Data: 0,72,19,15,19,15,20,20,CS0 ,
0,0,10,10,0,423,7,1,6,23,10,4,0, 63,63,63,63,31,31,63,63,63,63,63, 0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,
0,0,6,6,0,422,6,1,6,19,10,4,0, 63,63,63,63,31,31,63,63,63,63,63, 0,0,0,0,0,0,0,0,0,0,0, 0,4,2,8,0,0,3,8,8,10,0,
0,0,6,6,0,421,5,1,6,19,10,4,0, 63,63,63,63,31,31,63,63,63,63,63, 0,0,0,0,0,0,0,0,0,0,0, 2,0,0,6,0,0,0,2,6,3,0,
0,0,10,10,0,423,7,1,6,23,10,4,0, 63,63,63,63,31,31,63,63,63,63,63, 0,0,0,0,0,0,0,0,0,0,0, 1,2,1,5,0,0,2,0,4,1,0,
Run_alg: tuning failed 0
DDR3 run algorithm - FAILED 0x1
DDR3 Training Sequence - FAILEDI share it to you in case you noticed it on your board. I do not know if my unit is defective or not. I just hope it will not happen when it's deployed, in a remote location. I would be unable to access it.
solyste on Turris Omnia
After the previous steps, solyste is now perfectly booting. The board is ready to be used. Needless to say, I'm quite happy. Maybe it's time to brag a little bit…
Kernel :
solyste armv7l on 5.4.33-SOLYSTE (/dev/ttyS0)
solyste-omnia login: root
Password:
Welcome to your solyste box!
[solyste-omnia] /root # cat /proc/version
Linux version 5.4.33-SOLYSTE (ypnose@solyste) (gcc version 9.2.0 (GCC)) #1 SMP Sat May 2 13:46:17 CEST 2020
[solyste-omnia] /root # uname -a
Linux solyste-omnia 5.4.33-SOLYSTE #1 SMP Sat May 2 13:46:17 CEST 2020 armv7l GNU/Linux
[solyste-omnia] /root # du -h /boot/zImage
3.6M /boot/zImage
[solyste-omnia] /root # zcat /proc/config.gz | grep CONFIG_MODULES
# CONFIG_MODULES is not setcpuinfo :
[solyste-omnia] /root # cat /proc/cpuinfo
processor : 0
model name : ARMv7 Processor rev 1 (v7l)
BogoMIPS : 1600.00
Features : half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x4
CPU part : 0xc09
CPU revision : 1
processor : 1
model name : ARMv7 Processor rev 1 (v7l)
BogoMIPS : 1600.00
Features : half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x4
CPU part : 0xc09
CPU revision : 1
Hardware : Marvell Armada 380/385 (Device Tree)
Revision : 0000
Serial : 0000000000000000Daemons / services :
[solyste-omnia] /root # perpls -G
[+ +++ +++] crond uptime: 107s/107s pids: 147/146
[+ +++ ---] getty-ttyS0 uptime: 107s/-s pids: 145/-
[+ +++ ---] klogd uptime: 107s/-s pids: 155/-
[- --- ---] monit (service not activated)
[- --- ---] openntpd (service not activated)
[+ +++ +++] sshd uptime: 107s/107s pids: 152/150
[+ +++ ---] syslogd uptime: 107s/-s pids: 144/-
[- --- ---] unbound (service not activated)
[- --- ---] vnstatd (service not activated)
[+ +++ ---] watchdog uptime: 107s/-s pids: 148/-
[solyste-omnia] /root # pstree
init---rc.intro---perpboot-+-perpd-+-crond
| |-klogd
| |-mksh---pstree
| |-sshd
| |-syslogd
| |-2*[tinylog]
| `-watchdog
`-tinylog
[solyste-omnia] /root # print $KSH_VERSION
@(#)MIRBSD KSH R59 2020/04/14The end
Even if I plan to write an article explaining why solyste is different and how it works, you can visit my "Projects" page in the meantime. Have a nice day and take care of you.