by Tykling
19. sep 2017 19:59 UTC
Today I had the pleasure of trying out my new apu2c4. Hit a few snags here and there so I am documenting my experiences for future reference. I have an apu3 as well, but I haven't played with it yet, so that will have to wait for another blog post.
My Qubes installation on my laptop is using the sys-usb vm thing so the first thing I needed to do was to assign the USB serial device to the Qubes VM I was working from. dmesg on the sys-usb vm looks like this after plugging it in:
[12505.096158] usb 3-2: new full-speed USB device number 12 using xhci_hcd [12505.264940] usb 3-2: New USB device found, idVendor=067b, idProduct=2303 [12505.264964] usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [12505.264978] usb 3-2: Product: USB-Serial Controller D [12505.264990] usb 3-2: Manufacturer: Prolific Technology Inc. [12505.265880] pl2303 3-2:1.0: pl2303 converter detected [12505.266966] usb 3-2: pl2303 converter now attached to ttyUSB0
I noted the number of the USB device and pop open a dom0 console and ran:
qvm-usb -a apuvm sys-usb:3-2
In the apuvm dmesg now showed the serial USB device:
[ 7450.975054] usb 2-1: new full-speed USB device number 4 using vhci_hcd [ 7451.080162] usb 2-1: SetAddress Request (4) to port 0 [ 7451.095764] usb 2-1: New USB device found, idVendor=067b, idProduct=2303 [ 7451.095793] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 7451.095810] usb 2-1: Product: USB-Serial Controller D [ 7451.095823] usb 2-1: Manufacturer: Prolific Technology Inc. [ 7451.102423] pl2303 2-1:1.0: pl2303 converter detected [ 7451.108987] usb 2-1: pl2303 converter now attached to ttyUSB0
Neato. screen can speak serial but the default speed is 9600 and the apu boards all run 115200 8n1. So I ran:
$ sudo screen /dev/ttyUSB0 115200
Then I physically connected the serial cable (which must be a null modem cable and not a regular cable!) and powered up the apu2c4.
The very first line during boot contains the bios version (it flashes by really quick!) and mine was 160307 and according to the apu2c4 page that is a few versions older than the newest v4.0.7 (apparently they switched versioning system recently which didn't help the confusion).
The bios update page describes how to create a TinyCore USB stick on different operating systems.
I tried using their FreeBSD instructions but they didn't work, it failed while writing MBR to the USB stick. It has been ages since I've used fdisk(8) and I can't say I miss it. fdisk(8) is so outdated that it itself suggested using gpart(8) when it failed.
The instructions for Linux were a bit thin, so I wound up borrowing a Windows machine and using the Windows TinyCore USB Installer which worked very well. After it was done I mounted the USB stick on my laptop and copied the apu2_v4.0.7.rom over.
When no other boot options are available the apu2c4 boots straight into a memtest86 test which confused me a bit at first. But before memtest86 starts up there is briefly an option to press F10 to get a boot menu when needed.
Booting the TinyCore USB stick was easy since the serial connection was already in place. It even told me which command to run after it finished booting, which took a few seconds:
flashrom -w apu2_v4.0.7.rom -p internal
A few scary sounding errors in the first few lines after starting it did nothing for my confidence, but it kept going and finally wrote and verified the new bios. After rebooting all was well. Things were mostly the same, except a bit less verbose during boot, which was nice actually. The bios now reports version SeaBIOS (version rel-1.10.0.1) during boot, which is weird since I installed version 4.0.7. Whatever, I suppose versioning is not a strength of PC Engines.
Me and serial have never gotten along so I was kind of looking forward to getting this part overwith. I've always found it a bit of a chore to get it working, many pitfalls and stuff one is just assumed to know. Sigh.
I got the latest FreeBSD 11-stable amd64 snapshot which at the time of writing was FreeBSD-11.1-STABLE-amd64-20170822-r322788-memstick.img and dd'ed it onto a USB stick.
My serial console was still attached and screen was still running at 115200 which worked as long as the bios and early stage bootloader had control (until now the bios had been nice enough to redirect the early stage FreeBSD bootloader vidconsole output to the serial port for me). But as soon as FreeBSD started booting proper, the bios looses control and since FreeBSD was still outputting to the default vidconsole, I eventually lost output on the serial port.
Since the apu2c4 has no video output like VGA or HDMI I needed to get this working so I rebooted and in the FreeBSD boot menu I pressed 3 to escape to the boot loader prompt to set the console variable. But something was fucky! Some dimentions were off and the output on the terminal was garbled, it looked like the output is a bit too wide for the terminal, so it was mostly unreadable. I was able to input stuff though, so I wrote:
set console="comconsole"
and pressed enter, and as soon as I did, I lost the serial connection! Not to worry though.
This was because the changes take effect immediately in the bootloader, and FreeBSD had now switched to outputting to the serial console by itself, unassisted by the bios. The default serial speed for FreeBSD is 9600 and my screen was still running at 115200 so that was never going to work. I needed to kill the screen session (ctrl-a k) and start a new screen, this time with speed 9600.
After starting screen again I hit enter a few times, and saw that I still had the familiar OK FreeBSD boot loader prompt. Then I typed boot, hit enter, and watched the boot process of the FreeBSD memstick installer.
I could also have chosen to just set comconsole_speed="115200" at the boot loader prompt. Then I could have kept using the same screen session. But remember I was typing commands more or less blind, so restarting screen was easier.
When booting FreeBSD with console="comconsole" it asks what kind of terminal you are using near the end of the boot process. I don't really understand the choices but I usually go with the default vt100 and that seems to work well enough to get through the installer at least.
During boot I experienced a drop to mountroot which looked like this:
Trying to mount root from ufs:/dev/ufs/FreeBSD_Install [ro,noatime]... mountroot: waiting for device /dev/ufs/FreeBSD_Install ... Mounting from ufs:/dev/ufs/FreeBSD_Install failed with error 19.
I rebooted and tried again and this time there was no problem, it booted fine. It appears there is a timing issue/race condition on the apu2c4 where mountroot is attempted before the USB bus is fully operational. I've seen other users work around this by setting kern.cam.boot_delay="10000" at the loader(8) prompt.
It makes sense to add the variables discussed here to /boot/loader.conf even though the apu2c4 is now reachable over the network. In case anything happens it is nice to be able to plug in the serial connection and have it just work. I added the following to /boot/loader.conf:
console="comconsole" comconsole_speed="115200" kern.cam.boot_delay="10000"
The dmesg for the booted system looks like this:
Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-STABLE #0 r322788: Tue Aug 22 15:32:10 UTC 2017
root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
VT(vga): resolution 640x480
CPU: AMD GX-412TC SOC (998.16-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x730f01 Family=0x16 Model=0x30 Stepping=1
Features=0x178bfbff
Features2=0x3ed8220b
AMD Features=0x2e500800
AMD Features2=0x1d4037ff
Structured Extended Features=0x8
XSAVE Features=0x1
SVM: NP,NRIP,AFlush,DAssist,NAsids=8
TSC: P-state invariant, performance statistics
real memory = 4815060992 (4592 MB)
avail memory = 4084215808 (3895 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table:
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: unblocking device.
ioapic1: Changing APIC ID to 5
ioapic0 irqs 0-23 on motherboard
ioapic1 irqs 24-55 on motherboard
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
Timecounter "TSC" frequency 998161774 Hz quality 1000
random: entropy device external interface
kbd0 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff80f68130, 0) error 19
nexus0
vtvga0: on motherboard
cryptosoft0: on motherboard
acpi0: on motherboard
acpi0: Power Button (fixed)
cpu0: on acpi0
cpu1: on acpi0
cpu2: on acpi0
cpu3: on acpi0
atrtc0: port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x818-0x81b on acpi0
acpi_button0: on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
pcib1: at device 2.2 on pci0
pcib1: failed to allocate initial I/O port window: 0x1000-0x1fff
pci1: on pcib1
igb0: mem 0xfe600000-0xfe61ffff,0xfe620000-0xfe623fff at device 0.0 on pci1
igb0: Using MSIX interrupts with 5 vectors
igb0: Ethernet address: 00:0d:b9:45:c2:94
igb0: Bound queue 0 to cpu 0
igb0: Bound queue 1 to cpu 1
igb0: Bound queue 2 to cpu 2
igb0: Bound queue 3 to cpu 3
igb0: netmap queues/slots: TX 4/1024, RX 4/1024
pcib2: at device 2.3 on pci0
pci2: on pcib2
igb1: port 0x2000-0x201f mem 0xfe700000-0xfe71ffff,0xfe720000-0xfe723fff at device 0.0 on pci2
igb1: Using MSIX interrupts with 5 vectors
igb1: Ethernet address: 00:0d:b9:45:c2:95
igb1: Bound queue 0 to cpu 0
igb1: Bound queue 1 to cpu 1
igb1: Bound queue 2 to cpu 2
igb1: Bound queue 3 to cpu 3
igb1: netmap queues/slots: TX 4/1024, RX 4/1024
pcib3: at device 2.4 on pci0
pci3: on pcib3
igb2: port 0x3000-0x301f mem 0xfe800000-0xfe81ffff,0xfe820000-0xfe823fff at device 0.0 on pci3
igb2: Using MSIX interrupts with 5 vectors
igb2: Ethernet address: 00:0d:b9:45:c2:96
igb2: Bound queue 0 to cpu 0
igb2: Bound queue 1 to cpu 1
igb2: Bound queue 2 to cpu 2
igb2: Bound queue 3 to cpu 3
igb2: netmap queues/slots: TX 4/1024, RX 4/1024
pci0: at device 8.0 (no driver attached)
xhci0: mem 0xfeb22000-0xfeb23fff at device 16.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
xhci0: Unable to map MSI-X table
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ahci0: port 0x4010-0x4017,0x4020-0x4023,0x4018-0x401f,0x4024-0x4027,0x4000-0x400f mem 0xfeb25000-0xfeb253ff at device 17.0 on pci0
ahci0: AHCI v1.30 with 2 6Gbps ports, Port Multiplier supported with FBS
ahcich0: at channel 0 on ahci0
ahcich1: at channel 1 on ahci0
ehci0: mem 0xfeb25400-0xfeb254ff at device 19.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
usbus1: 480Mbps High Speed USB v2.0
isab0: at device 20.3 on pci0
isa0: on isab0
sdhci_pci0: mem 0xfeb25500-0xfeb255ff at device 20.7 on pci0
sdhci_pci0: 1 slot(s) allocated
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (9600,n,8,1)
orm0: at iomem 0xef000-0xeffff on isa0
ppc0: cannot reserve I/O port range
uart1: <16550 or compatible> at port 0x2f8 irq 3 on isa0
hwpstate0: on cpu0
ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
nvme cam probe device init
ugen0.1: <0x1022 XHCI root HUB> at usbus0
ugen1.1: at usbus1
uhub0: <0x1022 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
uhub1: ada0: ACS-2 ATA SATA 3.x device
ada0: Serial Number D797961703
ada0: 600.000MB/s transfers ( on usbus1
SATA 3.x, UDMA6, PIO 1024bytes)
ada0: Command Queueing enabled
ada0: 30533MB (62533296 512 byte sectors)
Trying to mount root from zfs:zroot/ROOT/default []...
GEOM_MIRROR: Device mirror/swap launched (1/1).uhub0:
4 ports with 4 removable, self powered
Root mount waiting for: usbus1
uhub1: 2 ports with 2 removable, self powered
Root mount waiting for: usbus1
ugen1.2: at usbus1
uhub2 on uhub1
uhub2: on usbus1
uhub2: 4 ports with 4 removable, self powered
Root mount waiting for: usbus1
GEOM_ELI: Device mirror/swap.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI: Crypto: software
igb0: link state changed to UP