User Guide

Contents

• User Guide

  • Introduction

  • Host prerequisites

  • Syncing and building the source code

    • Building

      • Firmware only

      • Ubuntu Distribution

      • Minimal BusyBox

      • Check dependencies

      • Fetch Tools

      • Prebuilt board firmware

  • Software Components

    • Firmware

      • Firmware image creation

      • Trusted Firmware-A

      • System Control Processor (SCP)

      • UEFI EDK2

    • File-system

      • Disk image creation

      • GRUB

      • Linux

      • BusyBox

      • Ubuntu

  • Running the software distribution on N1SDP

    • Setting up the N1SDP

  • Update firmware on microSD card

    • L3 Cache Enablement

    • Boot Minimal BusyBox Image on N1SDP

    • Boot Ubuntu on N1SDP

    • Building Kernel Modules Natively

Introduction

The Neoverse N1 System Development Platform (N1SDP) is an enterprise class reference board based on the Neoverse N1 core.

This document is a guide on how to fetch, build from source, and run an Arm Reference Platforms software stack on N1SDP, including a Linux distribution based on either the Ubuntu server distribution or a minimal BusyBox based root filesystem.

The synced workspace includes:

• Scripts to build the board firmware, Linux kernel, and Ubuntu server distribution image.

• Ubuntu server distribution, sources for Linux Kernel, EDK2, firmware and BusyBox.

• Other supporting board files and prebuilt binaries.

Host prerequisites

These instructions assume that:

• Your host PC is running Ubuntu Linux 18.04 LTS and should have at least 50GB free storage space

• You are running the provided scripts in a bash shell environment.

The following utilities must be available on your host PC:

• chrpath

• compression library

• diffstat

• gawk

• makeinfo

• openssl headers

• pip

• repo

To install all the required packages, run:

sudo apt-get update
sudo apt-get install autoconf autopoint bc bison build-essential \
   curl debianutils device-tree-compiler dosfstools flex gettext-base git \
   libssl-dev m4 mtools parted pkg-config python python3 python3-distutils \
   rsync unzip uuid-dev wget

Configure Git, run:

git config --global user.email "[email protected]"
git config --global user.name "Your Name"

Install repo tool

Follow the instructions provided in the repo README file to install the repo tool.

NOTE: The repo tool which gets installed using apt-get command sometimes return errors, in such a case it is recommended to install repo using the curl method:

mkdir -p ~/.bin
PATH="${HOME}/.bin:${PATH}"
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
chmod a+rx ~/.bin/repo

Syncing and building the source code

The N1SDP software stack supports two software distributions:

• A minimal BusyBox root filesystem.

• Ubuntu server.

The instructions below provide the necessary steps to sync and build these distributions.

Create a new folder that will be your workspace and will henceforth be referred to as <n1sdp_workspace> in these instructions:

mkdir <n1sdp_workspace>
cd <n1sdp_workspace>
export N1SDP_RELEASE=refs/tags/N1SDP-2021.10.12

NOTE: Sometimes new features and additional bug fixes will be made available in the git repositories and will not yet have been tagged as part of a release. To pickup these latest changes you can drop the -b <release tag> option from the repo init commands below. However, please be aware that such untagged changes have not yet been formally verified and should be considered unstable until they are tagged in an official release.

Building

Firmware only

Build the firmware files.

repo init \
    -u https://git.gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-manifest.git \
    -b ${N1SDP_RELEASE} \
    -m pinned-n1sdp.xml \
    -g bsp \
    --depth=6
repo sync
./build-scripts/build-all.sh -f none

Ubuntu Distribution

N1SDP provides support for installation and boot of standard Ubuntu 18.04 distribution. The distribution is installed on a disk and since the installed image is persistent it can be used for multiple boots.

Build the firmware and Ubuntu distribtution:

repo init \
    -u https://git.gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-manifest.git \
    -b ${N1SDP_RELEASE} \
    -m pinned-n1sdp.xml \
    -g ubuntu \
    --depth=6
repo sync
./build-scripts/build-all.sh -f ubuntu

The bootable image will be available at the location <n1sdp_workspace>/output/n1sdp/ubuntu.img>

Minimal BusyBox

Build the firmware and BusyBox based root filesystem:

repo init \
    -u https://git.gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-manifest.git \
    -b ${N1SDP_RELEASE} \
    -m pinned-n1sdp.xml \
    -g busybox \
    --depth=6
repo sync
./build-scripts/build-all.sh -f busybox

The bootable image will be available at the location <n1sdp_workspace>/output/n1sdp/busybox.img>

NOTE: To sync the source code with the complete history of components, drop the depth option from the repo init command for any of the distributions/filesystem mentioned above.

Check dependencies

To ensure that all the required dependent packages were installed, run:

./build-scripts/check_dep.sh

"no missing dependencies detected" should get displayed.

Fetch Tools

The tools required to build the software components are fetched using build-scripts/fetch-tools.sh, it is executed as part of build-all.sh.

The script can also be run separately to fetch the tools as shown :

./build-scripts/fetch-tools.sh -f none

Prebuilt board firmware

The board firmware prebuilt binaries are made available as part of the release in the directory bsp/n1sdp-board-firmware/, these binaries can be copied to the onboard SD card to boot the platform to UEFI EDK2 shell console.

There are two methods for updating the microSD card with the firmware binaries:

1. The microSD card from the N1SDP can be removed from N1SDP and can be mounted on a host machine using a card reader.

2. The USB debug cable when connected to host machine will show the microSD partition on host machine which can be mounted.

   $> sudo mount /dev/sdx1 /mnt
   $> rm -rf /mnt/*
   $> sudo cp -a bsp/n1sdp-board-firmware/* /mnt/
   $> sudo umount /mnt
   

NOTE: replace sdx1 with the device and partition of the SD card. Option (2) above is typically preferred, as removing the microSD card requires physical access to the motherboard inside the N1SDP’s tower case.

NOTE:

Please ensure to use the recommended PMIC binary. Refer to page potential-damage for more info. If a PMIC binary mismatch is detected, a warning message is printed in the MCC console recommending the user to switch to appropriate PMIC image. On MCC recommendation ONLY, please update the MB/HBI0316A/io_v123f.txt file on the microSD using the below commands.

Example command to switch to 300k_8c2.bin from the host PC

$> sudo mount /dev/sdx1 /mnt
$> sudo sed -i '/^MBPMIC: pms_0V85.bin/s/^/;/g' /mnt/MB/HBI0316A/io_v123f.txt
$> sudo sed -i '/^;MBPMIC: 300k_8c2.bin/s/^;//g' /mnt/MB/HBI0316A/io_v123f.txt
$> sudo umount /mnt

Software Components

Firmware

Each firmware component has a separate script to build that component. After the firmware components have been built they must be packaged using build-firmware-image.sh.

To make customizations to the firmware

# first make sure to build all the firmware components
<n1sdp_workspace>/build-scripts/build-all.sh -f none

# modify the firmware component

# build the component
<n1sdp_workspace>/build-scripts/build-[COMPONENT].sh -f none

# repackage the firmware files
<n1sdp_workspace>/build-scripts/build-firmware-image.sh -f none

As an alternative the one-liner <n1sdp_workspace>/build-scripts/build-all.sh -f none will trigger the build of all firmware scripts.

NOTE: The valid firmware [COMPONENT] names are :

"arm-tf" : To build Trusted Firmware-A
"scp"    : To build SCP and MCP ROM and RAM firmware binaries
"uefi"   : To build EDK2 for N1SDP

Firmware image creation

Packages the firmware files in a format suitable for the N1SDP.

Build script

<n1sdp_workspace>/build-scripts/build-firmware-image.sh

Output

<n1sdp_workspace>/output/n1sdp/firmware/scp_rom.bin <n1sdp_workspace>/output/n1sdp/firmware/scp_fw.bin <n1sdp_workspace>/output/n1sdp/firmware/mcp_rom.bin <n1sdp_workspace>/output/n1sdp/firmware/mcp_fw.bin <n1sdp_workspace>/output/n1sdp/firmware/uefi.bin <n1sdp_workspace>/output/n1sdp/firmware/n1sdp-board-firmware_primary.tar.gz <n1sdp_workspace>/output/n1sdp/firmware/n1sdp-board-firmware_secondary.tar.gz

Trusted Firmware-A

Based on Trusted Firmware-A.

Build script	<n1sdp_workspace>/build-scripts/build-arm-tf.sh
Checkout path	<n1sdp_workspace>/bsp/arm-tf
Output	<n1sdp_workspace>/output/n1sdp/intermediates/bl31.bin

System Control Processor (SCP)

Based on SCP Firmware.

Build script	<n1sdp_workspace>/build-scripts/build-scp.sh
Checkout path	<n1sdp_workspace>/bsp/scp
Output	<n1sdp_workspace>/output/n1sdp/intermediates/scp-ram.bin <n1sdp_workspace>/output/n1sdp/intermediates/scp_rom.bin <n1sdp_workspace>/output/n1sdp/intermediates/mcp-ram.bin <n1sdp_workspace>/output/n1sdp/intermediates/mcp_rom.bin

UEFI EDK2

Based on UEFI EDK2.

Build script	<n1sdp_workspace>/build-scripts/build-uefi.sh
Checkout path	<n1sdp_workspace>/bsp/uefi/edk2
Output	<n1sdp_workspace>/output/n1sdp/intermediates/uefi.bin

File-system

Each file-system component has a separate script to build that component. Some build scripts depend on the value of -f <FILESYSTEM>. After the file-system components have been built they must be packaged into a disk image using build-disk-image.sh.

# first make sure that all the components have been built at least once
<n1sdp_workspace>/build-scripts/build-all.sh -f <FILESYSTEM>

# modify the component

# build the component
<n1sdp_workspace>/build-scripts/build-[COMPONENT].sh -f <FILESYSTEM>

# repackage the disk image
<n1sdp_workspace>/build-scripts/build-disk-image.sh -f <FILESYSTEM>

NOTE:

The valid filesystem [COMPONENT] names are :

"grub" : To build the GRUB utilities
"linux": To build the Linux kernel image
"perf" : To build the perf tool

The valid [FILESYSTEM] names are :

"busybox" : To build for BusyBox filesystem
"ubuntu"  : To build for Ubuntu filesystem distribution

Disk image creation

Create a disk image for -f <FILESYSTEM>. FILESYSTEM is either busybox or ubuntu.

Build script	<n1sdp_workspace>/build-scripts/build-disk-image.sh
Output	<n1sdp_workspace>/output/n1sdp/<FILESYSTEM>.img <n1sdp_workspace>/output/n1sdp/intermediates/<FILESYSTEM>.esp.img

GRUB

Based on grub.

Build script	<n1sdp_workspace>/build-scripts/build-grub.sh
Checkout path	<n1sdp_workspace>/grub
Output	<n1sdp_workspace>/output/n1sdp/intermediates/grub/output/grubaa64.efi

Linux

Based on Linux 5.10.61 for N1SDP.

Build script	<n1sdp_workspace>/build-scripts/build-linux.sh
Checkout path	<n1sdp_workspace>/linux
Output	<n1sdp_workspace>/output/n1sdp/intermediates/kernel_Image_<FILESYSTEM>

BusyBox

Build a minimal root filesystem based on BusyBox

Build script	<n1sdp_workspace>/build-scripts/build-busybox.sh
Checkout path	<n1sdp_workspace>/busybox
Output	<n1sdp_workspace>/output/n1sdp/intermediates/busybox.initramfs <n1sdp_workspace>/output/n1sdp/intermediates/busybox.root.img

Ubuntu

Build Ubuntu server distribution based on Ubuntu 18.04

Build script	<n1sdp_workspace>/build-scripts/build-ubuntu.sh
Checkout path	<n1sdp_workspace>/tools/ubuntu_bionic
Output	<n1sdp_workspace>/output/n1sdp/intermediates/ubuntu.esp.img <n1sdp_workspace>/output/n1sdp/intermediates/ubuntu.root.img

Running the software distribution on N1SDP

This section provides steps for:

• Setting up the N1SDP with the required board firmware

• Preparing a bootable disk

• Boot the supported software distributions (Minimal BusyBox or Ubuntu Server).

Setting up the N1SDP

After powering up or rebooting the board, any firmware images placed on the board’s microSD will be flashed into either on-board QSPI flash or copied into the DDR3 memory via the IOFPGA.

Configure COM Ports

Connect a USB-B cable between your host PC and N1SDP’s DBG USB port, then power ON the board. The DBG USB connection will enumerate as four virtual COM ports assigned to the following processing entities, in order

COM<n>   - Motherboard Configuration Controller (MCC)
COM<n+1> - Application Processor (AP)
COM<n+2> - System Control Processor (SCP)
COM<n+3> - Manageability Control Processor (MCP)

Please check Device Manager in Windows or ls /dev/ttyUSB* in Linux to identify <n>.

Use a serial port application such as PuTTy or minicom to connect to all virtual COM ports with the following settings:

115200 baud
8-bit word length
No parity
1 stop bit
No flow control

Note: Some serial port applications refer to this as “115200 8N1” or similar.

Before running the deliverables on N1SDP, ensure both BOOT CONF switches are in the OFF position, then issue the following command in the MCC console:

Cmd> USB_ON

This will mount the on-board microSD card as a USB Mass Storage Device on the host PC with the name “N1SDP”.

Enter the following command on the MCC console window to ensure time is correctly set. This is required for the first distribution boot to succeed:

Cmd> debug
Debug> time
Debug> date
Debug> exit

Update firmware on microSD card

The board firmware files are located in <n1sdp_workspace/output/n1sdp/firmware/> after the firmware source build.

Single chip mode:

n1sdp-board-firmware_primary.tar.gz    : firmware to be copied to microSD of N1SDP board in single chip mode.

Multi chip mode:

n1sdp-board-firmware_primary.tar.gz    : firmware to be copied to microSD of primary board.
n1sdp-board-firmware_secondary.tar.gz  : firmware to be copied to microSD of secondary board.

There are two methods for populating the microSD card:

1. The microSD card from the N1SDP can be removed from N1SDP and can be mounted on a host machine using a card reader,

2. The USB debug cable when connected to host machine will show the microSD partition on host machine which can be mounted.

Option (2) above is typically preferred, as removing the microSD card requires physical access to the motherboard inside the N1SDP’s tower case.

The instructions to extract the board firmware package onto the microSD card is as shown below:

$> sudo mount /dev/sdx1 /mnt
$> sudo rm -rf /mnt/*
$> sudo tar --no-same-owner -xzf n1sdp-board-firmware_primary.tar.gz -C /mnt/
$> sudo umount /mnt

NOTE:

• Replace sdx1 with the device and partition of the SD card.

• Follow the similar steps to install the firmware on secondary board with n1sdp-board-firmware_secondary.tar.gz for multi chip mode.

Firmware tarball package contains IOFPGA configuration files, SCP, TF-A, and UEFI binaries.

NOTE: Please ensure to use the recommended PMIC binary. Refer to page potential-damage for more info.

If a PMIC binary mismatch is detected, a warning message is printed in the MCC console recommending the user to switch to appropriate PMIC image. On MCC recommendation ONLY, please update the MB/HBI0316A/io_v123f.txt file on the microSD using the below commands.

Example command to switch to 300k_8c2.bin from the host PC

$> sudo mount /dev/sdx1 /mnt
$> sudo sed -i '/^MBPMIC: pms_0V85.bin/s/^/;/g' /mnt/MB/HBI0316A/io_v123f.txt
$> sudo sed -i '/^;MBPMIC: 300k_8c2.bin/s/^;//g' /mnt/MB/HBI0316A/io_v123f.txt
$> sudo umount /mnt

L3 Cache Enablement

By default, L3 cache is disabled for use. To enable/disable L3 cache support, follow these steps:

1. Run USB_ON command to mount the on-board microSD card on the host PC.

2. Open the file “MB/HBI0316A/io_v123f.txt”.

3. For user to enable/disable L3 cache support, edit the SCC BOOT_GPR1 register in the following manner.

   • To enable L3 cache, update the SOCCON with offset 0x1184 and set the value 0x00000001. The line should now read,

     SOCCON: 0x1184 0x00000001 ; SoC SCC BOOT_GPR1

   • To disable L3 cache, update the SOCCON with offset 0x1184 and set the value 0x00000000. The line should now read,

     SOCCON: 0x1184 0x00000000 ; SoC SCC BOOT_GPR1

4. Save and close the file.

Boot Minimal BusyBox Image on N1SDP

Preparing a bootable disk with BusyBox root filesystem

A bootable disk (USB stick or SATA drive) can be prepared by flashing the image generated from the source build. The image will be available at the location <n1sdp_workspace>/output/n1sdp/busybox.img>

This is a bootable GRUB image comprising Linux kernel and BusyBox binaries. The partitioning and packaging is performed during the build phase.

Use the following commands to prepare the GRUB image on a USB stick or SATA drive:

$ lsblk
$ sudo dd if=busybox.img of=/dev/sdx conv=fsync bs=1M
$ sync

Note: Replace /dev/sdx with the handle corresponding to your USB stick or SATA drive, as identified by the lsblk command.

Booting the board with BusyBox image

Insert the bootable disk created earlier. Shutdown and reboot the board by issuing the following commands to the MCC console:

Cmd> SHUTDOWN
Cmd> REBOOT

Enter the UEFI menu by pressing Esc on the AP console as the EDK2 logs start appearing; from here, enter the UEFI Boot Manager menu and then select the disk.

By default the Linux kernel will boot with ACPI configurations:

* BusyBox N1SDP (ACPI)
  BusyBox N1SDP SINGLE CHIP (Device Tree)
  BusyBox N1SDP MULTI CHIP (Device Tree)

The system will boot into a minimal BusyBox Linux image environment.

Boot Ubuntu on N1SDP

Preparing a bootable Ubuntu disk

A bootable disk (USB stick or SATA drive) can be prepared by formatting it with the distribution image created during source build. The image will be available at the location <n1sdp_workspace/output/n1sdp/ubuntu.img>.

This is a bootable GRUB image comprising Linux kernel and an Ubuntu Server 18.04 file system. The partitioning and packaging is performed during the build.

Use the following commands to burn the GRUB image to a USB stick or SATA drive:

$ lsblk
$ sudo dd if=ubuntu.img of=/dev/sdX bs=1M
$ sync

Note: Replace /dev/sdX with the handle corresponding to your USB stick or SATA drive, as identified by the lsblk command.

Booting the board with Ubuntu image

Insert the bootable disk created earlier and connect the ethernet cable to a working internet connection. This is REQUIRED on first boot in order to successfully download and install necessary Ubuntu packages. Installation will fail if an internet connection is not available.

NOTE: It is also observed that the installation may fail if more than one storage device is present on N1SDP, the error log is as shown below:

Booting `Install Ubuntu on N1SDP Platform'
error: disk `hd1,msdos2' not found.
error: you need to load the kernel first.

Press any key to continue...

Failed to boot both default and fallback entries.

Press any key to continue...

Therefore, it is always recommended to have only one storage device on N1SDP on which you want to install the Ubuntu software.

Shutdown and reboot the board by issuing the following commands to the MCC console:

Cmd> SHUTDOWN
Cmd> REBOOT

Enter the UEFI menu by pressing Esc on the AP console as the EDK2 logs start appearing; from here, enter the UEFI Boot Manager menu and then select the burned disk.

Ubuntu 18.04 will boot in two stages; the first boot is an installation pass, after which a second boot is required to actually enter the Ubuntu Server environment.

To reboot the board after the first boot installation pass has completed, from MCC console:

Cmd> REBOOT

The system will boot into a minimal Ubuntu 18.04 environment.

Login as user root with password root, and install any desired packages from the console:

# apt-get install <package-name>

Building Kernel Modules Natively

Native building of kernel modules typically requires kernel headers to be installed on the platform. However, a bug in deb-pkg currently causes host executables to be packed rather than the target executables.

This can be worked around by building and installing the kernel natively on the platform.

Boot the N1SDP board with Ubuntu filesystem and login as root.

apt-get install -y git build-essential bc bison flex libssl-dev
git clone -b n1sdp https://git.gitlab.arm.com/arm-reference-solutions/linux.git
cd kernel-release/
mkdir out
cp -v /boot/config-5.10.61+  out/.config
make O=out -j4
make O=out modules_install
make O=out install
update-grub
sync

Reboot the board and when Grub menu appears, select the Advanced Boot Options -> 5.10.61 kernel for booting.

---

Copyright (c) 2021, Arm Limited. All rights reserved.