Home

XtratuM User Manual - ESA Microelectronics Section

1. Extended types 2520 typedef long xmLong_t typedef xm_u32_t xmWord_t define XM_LOG2_WORD_SZ 5 typedef xm_s64_t xmTime_t define MAX_XMTIME MAX_S64 2525 typedef xm_u32_t xmAddress_t typedef xmAddress_t xmIoAddress t typedef xm_u32_t xmSize_t typedef xm_s32_t xmSSize_t typedef xm_u32_t xmId_t 2530 Listing 6 2 core include arch arch types h For future compatiblity most data structures contain version information It is a xm u32 t data type with 3 fields version subversion and revision The macros listed next can be used to manipulate those fields 2535 define XM SET VERSION ver _subver rev _ver amp OxFF lt lt 16 _subver amp OxFF lt lt 8 _rev amp OxFF define XM_GET_VERSION _v _v gt gt 16 amp 0xFF define XM_GET_SUBVERSION _v _v gt gt 8 0xFF define XM_GET_REVISION _v v amp OxFF 2540 Listing 6 3 core include xmef h 6 2 Hypercall mechanism A hypercall is implemented by a trap processor instruction that transfers the control to XtratuM code and sets the processor in supervisor mode There are two kinds of hypercalls normal and assembly Each type of hypercall uses a different trap number 2545 define XM HYPERCALL TRAP OxFO define XM ASMHYPERCALL TRAP OxF1 Listing 6 4 core include arch xm_def h The XM ASMHYPERCALL TRAP hypercall entry is needed for the XM sparcv8 flush regwin XM sparc s v8 iret and XM
2. xm 4 usermanual 047d A ie amu P Printed November 26 2012 2385 2390 2395 2400 2405 2410 2415 2420 2425 2430 2435 82 134 Chapter 5 Partition Programming define SET IRQ MASK NR 16 define FORCE IRQS NR 17 define CLEAR IRQS NR 18 define ROUTE IRQ NR 19 define UPDATE PAGE32 NR 20 define SET PAGE TYPE NR 21 define __INVLD_TLB_NR 22 define __RAISE_IPVI_NR 23 define __OVERRIDE_TRAP_HNDL_NR 24 define __FLUSH_CACHE_NR 25 define __SET_CACHE_STATE_NR 26 define __SWITCH_SCHED_PLAN_NR 27 define __GET_GID_BY_NAME_NR 28 define __RESET_VCPU_NR 29 define __HALT_VCPU_NR 30 define __SUSPEND_VCPU_NR 31 define __RESUME_VCPU_NR 32 define __GET_VCPUID_NR 33 define sparc_atomic_add_nr 34 define sparc_atomic_and_nr 35 define sparc_atomic_or_nr 36 define sparc_inport_nr 37 define sparc_outport_nr 38 define sparc_write_tbr_nr 39 define sparc_write_ptdli_nr 40 Listing 5 31 core include arch hypercalls h define sparc_iret_nr 0 define sparc_flush_regwin_nr 1 define sparc_get_psr_nr 2 define sparc_set_psr_nr 3 define sparc_set_pil_nr 4 define sparc_clear_pil_nr 5 define sparc_ctrl_winflow_nr 6 Listing 5 32 core include arch hypercalls h The file core include sparckv8 hypercalls h has additional services for the SPARC v8 architecture 5 19 1 The object interface XtratuM implements internally a kind of virtual file system
3. MK Hh z Examples of valid frequencies 80Mhz Eighty mega hertz 80000000 hertz 20000Khz Twenty mega hertz 20000000 hertz 3125 66 Boolean Valid values are yes true no false Hexadecimal Pattern Ox 0 9a fA F Examples of valid numbers OxFfffFfff 0x0 OxF1 0x80 xm 4 usermanual 047d AX ie amu P Printed November 26 2012 110 134 Chapter 8 Co nfiguration http w ww xtratum org xm 3 x PartitionTable http w ww xtratum org xm 3 x SystemDescription Y anonymous Attrs name idString t version version t anonymous d sequence of partition_e hypMemoryArea e flags memAreaFlagsList_t Attrs size sizeUnit_t all of xttHypervisor Hw Description Channels i hwDescription e channels e i i sequence of sequence of CARS MemoryLayout Devices ProcessorTable choice of console idString_t hypervisor_e anonymous maxMessageLength sizeUnit_t maxNoMessages positivelnteger maxTimeExpiration timeUnit_t DEFAULT 0s DEFAULT 0s tes 1295 PartitionFlagsLis
4. 5 20 Manpages summary Below is a summary of the manpages A detailed information is provided in the document Volume 4 Reference Manual System Management XM get system status Get the current status of the system XM halt system Stop the system XM reset system Reset the system Partition Management XM get partition mmap This function returns a pointer to the memory map table MMT XM get partition status Get the current status of a partition XM halt partition Terminates a partition XM idle self Idles the execution of the calling partition XM params get PCT Return the address of the PCT XM reset partition Reset a partition XM resume partition Resume the execution of a partition XM set partition opmode Informs the internal status of the partition XM shutdown partition Send a shutdown interrupt to a partition XM_suspend partition Suspend the execution of a partition Multicore Management XM get vcpu status Get the current status of a virtual CPU XM get vcpuid Returns the virtual CPU identifier of the calling partition XM halt vcpu Halts a virtual CPU for the partition invoking this service XM reset vcpu Resets a virtual CPU for the partition invoking this service XM resume vcpu Resumes a virtual CPU for the partition invoking this service XM suspend vcp
5. 90 134 Chapter 6 Binary Interfaces 6 4 Partition ELF format A partition image contains all the information needed to execute the partition It does not have loading or booting information It contains one image header structure one or more partition header structures as well as the code and data that will be executed Since multiple partition headers is an experimental feature to support multiprocessor in a partition we will assume in what follows that a partition file contains only one image header structure and one partition header structure All the addresses of partition image are absolute addresses which refer to the target RAM mem ory locations 6 4 1 Partition image header The partition image header is a data structure with the following fields pageTable struct xmImageHdr define XMEF PARTITION MAGIC 0x24584d69 XMi xm u32 t sSignature xm u32 t compilationXmAbiVersion XM s abi version xm u32 t compilationXmApiVersion XM s api version is unused when MPU is set xmAddress_t pageTable Physical address pageTableSize is unused when MPU is set xmSize_t pageTableSize xm_u32_t noCustomFiles struct xefCustomFile customFileTab CONFIG_MAX_NO_CUSTOMFILES xm u32 t eSignature PACKED Listing 6 5 core include xmef h sSignature and eSignature Holds the start and end signatures which identifies the structure as a XtratuM partition image compilationXmAbiVers
6. PACK ARGS h XMCORE xm cf xef xmc p O partitionl xef p 1 partition2 xef container bin PARTITIONS xm cf xef xmc XMPACK check xm cf xef xmc PACK ARGS XMPACK build PACK ARGS 0 exec echo en gt Done container n Listing 5 4 Makefile 5 4 The Hello World example Let s start with a simple code that is not ready to be executed on XtratuM and needs to be adapted void main 1 int counter 0 xprintf Hello World n while 1 4 counter counter 100000 Listing 5 5 Simple example The first step is to initialise the virtual execution environment and call the entry point PartitionMain in the examples of the partition The following files are provided as an example of how to build the partition image and initialise the virtual machine 1315 boot S The assembly code where the headers and the entry point are defined traps c Required data structures PCT and trap handlers std_c c std_c h Minimal C support as memcpy xprintf etc loader lds The linker script that arranges the sections to build the partition image layout The boot S file 1320 xm 4 usermanual 047d AX ic amu P Printed November 26 2012 1325 1330 1335 1340 1345 1350 1355 1360 1365 1370 1375 1380 1385 58 134 Chapter 5 Partition Programming include lt xm h gt include lt xm_inc arch asm_offsets h gt include lt am_inc
7. Volume 2 XtratuM User Manual Chapter 1 Introduction This document describes the XtratuM hypervisor and how to write applications to be executed as XtratuM partitions A hypervisor is a layer of software that provides one or more virtual execution environments for par titions Although virtualisation concepts have been employed since the 60 s IBM 360 the application of these concepts to the server desktop and recently the embedded and real time computer segments is relatively new There have been some attempts in the desktop and server markets to standard ise how a hypervisor should operate but the research and the market is not mature enough In fact there is still not a common agreement on the terms used to refer to some of the new objects introduced Check the glossary A for the exact meaning of the terms used in this document In the case of embedded systems and in particular in avionics the ARINC 653 standard defines a partitioning system Although the ARINC 653 standard was not designed to describe how a hypervisor has to operate some parts of the APEX model of ARINC 653 are quite close to the functionality provided by a hypervisor During the porting of XtratuM to the LEON2 and LEON3 processors we have also adapted the XtratuM API and internal operations to resemble ARINC 653 standard It is not our intention to convert XtratuM in an ARINC 653 compliant system ARINC 653 relies on the idea of a separation ke
8. define XM VT EXT HW TIMER 0 XM VT EXT FIRST define XM VT EXT EXEC TIMER 1 XM VT EXT FIRST define XM VT EXT WATCHDOG TIMER 2 XM VT EXT FIRST define XM VT EXT SHUTDOWN 3 XM VT EXT FIRST define XM VT EXT SAMPLING PORT 4 XM VT EXT FIRST define XM VT EXT QUEUING PORT 5 XM VT EXT FIRST define XM VT EXT CYCLIC SLOT START 8 XM VT EXT FIRST define XM VT EXT MEM PROTECT 16 XM VT EXT FIRST Inter Partition Virtual Interrupts define XM MAX IPVI CONFIG XM MAX IPVI define XM VT EXT IPVIO 24 XM VT EXT FIRST define XM VT EXT IPVI1 25 XM VT EXT FIRST define XM VT EXT IPVI2 26 XM VT EXT FIRST define XM VT EXT IPVIS3 27 XM VT EXT FIRST define XM VT EXT IPVIA 28 XM VT EXT FIRST define XM VT EXT IPVI5 29 XM VT EXT FIRST define XM VT EXT IPVI6 30 XM VT EXT FIRST define XM VT EXT IPVI7 314XM VT EXT FIRST Listing 5 22 core include guest h 1940 1945 1950 1955 1960 1965 1970 Both extended and hardware interrupts can be routed to a different interrupt vector through the i XM route irq O hypercall this hypercall enables a partition to select the most suitable vector to be raised All hardware and extended interrupts can be masked through the following hypercalls XM clear irqmask and XM set irqmaskO Besides all these set of interrupts can be globally disabled enable by using the XM sparc get psr and XM spar
9. 3315 118 134 Chapter 9 Tools 9 1 1 xmcparser Compiles XtratuM XML configuration files SYNOPSIS xmcparser c s xsd file o output file XM CF xml xmcparser d DESCRIPTION xmcparser reads an XtratuM XML configuration file and transforms it into a binary file which can be used directly by XtratuM at run time xmcparser performs internally the folowing steps 1 Parse the XML file Validate the XML data Generate a set of C data structures initialised with the XML data Compiles and links using the target compiler the C data structures An ELF file is produced n A C N The data section which contains the data in binary format is extracted and copied to the output file OPTIONS d Prints the dafault XML schema used to validate the XML configuration file Place output in file s xsd file Use the XML schema xsd file rather than the dafault XtratuM schema C Stop after the stage of C generation do not compile The output is in the form of a C file 9 2 ELF to XEF xmeformat 9 2 1 xmeformat Creates and display information of XEF files SYNOPSIS xmeformat read h s m file xmeformat build m o outfile c p payload file file Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 9 2 ELF to XEF xmeformat 119 134 DESCRIPTION xmeformat converts an ELF or a binary file into an XEF format XtratuM Executable Format An XEF file contai
10. Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 8 3 Hypervisor configuration file XM CF 115 134 Value Description unmapped Allocated to the partition but not mapped by XtratuM in the page table mappedAt It allows to allocate this area to a virtual address shared It is allowed to map this area in other partitions read only The area is write protected to the partition uncacheable Memory cache is disabled rom Not applicable in SPARC v8 boards Only used in ia32 systems Configuration of I O ports There are two ways to allocate a port to a partition using ranges of ports and using the restricted port allocation Both are declared by elements contained in the Partition HwResources IoPorts element Range A range of port addresses is allocated to the partition The attributes of a range element are Range base Required hexadecimal base address Range noPorts Required number of ports in this range Each port is a word 4 bytes Restricted An I O port which is partially controlled by the partition The attributes are Restricted address Required hexadecimal address of the port Restricted mask Optional 4 bytes hexadecimal The bits set in this mask can be read and written by the partition Those bits not allocated to this partition i e the bit not set in the bitmask can be allocated to other partitions Configuration of interrupts The element Partition HwResources Interrupts h
11. User data can be passed to each partition at boot time This information is passed to the partition via the customisation files It is possible to attach up to a maximum of CONFIG MAX NO CUSTOMFILES customisation files per par tition The content of each customisation file is copied into the partition memory space at boot time before the partition boots The buffer where each customisation file is loaded is specified in the partition header by the partition developer See section 6 This is the mechanism used by XtratuM to get the compiled XML system configuration 3 6 Building the final system image The partition binary is not an ELF file It is a custom format file called XEF which contains the machine code and the initialized data See section 6 The container is a single file which contains all the code data and configuration information that is loaded in the target board In the context of the container a component refers to the set of files that are part of an execution unit which can be a partition or the hypervisor itself xmpack is a program that reads all the executable images XEF files and the configuration customisation files and produces the container The container is not a bootable code That is it is like a tar file which contains a set of files In order to be able to start the partitioned system a boot loader shall load the content of the container into the corresponding partition addresses The utility rswbui
12. gt _ernel parameters _uests parameters gt _LibC features gt oad an Alternate Configuration File ave Configuration to an Alternate File XtratuM Source Code autoconfig h E A config Figure 3 3 Menuconfig process to produce a compact and efficient XtratuM executable image Parameters like the processor model or the memory layout of the board are configured here see section 8 1 The configuration interface is the same as the one known as menuconfig used in the Linux kernel see figure 3 3 It is a ncurses based graphic interface to edit the configuration options The selected choices are stored in two files a C include file named core include autoconf h and a Makefile include file named core config Both files contain the same information but with different syntax to be used C programs and in Makefiles respectively Although it is possible to edit these configuration files with a plain text editor it is advisable not to do so since both files shall be synchronized Once configured the next step is to build XtratuM binaries which is done calling the command make Ideally configuring and compiling XtratuM should be done at the initial phases of the design and should not be changed later The build process leaves the objects and executables files in the source directory Although it is possible to use these files direct
13. xm 4 usermanual 047d AX ic amu J Printed November 26 2012 915 920 925 930 935 940 945 950 955 960 965 970 975 36 134 Chapter 2 XtratuM Architecture Monitoring Partitiont Partition 1 Partition 2 XM trace open XM CF xml XM trace event event bitmask XM trace read ss XM trace seek Partition id 1 gt if Y amp trace bitmask 0x01 device MemDisk0 gt lt Partition gt lt Devices gt Y v lt MemoryBlockTable gt E amp T Block name MemDisk0 start 0x40100000 gt gt SS T ye E f size 256KB gt e race race l Trace lt MemoryBlockTable gt stream stream stream lt Devices gt x XtratuM _ XtratuM trace system Figure 2 18 Tracing overview 2 17 Clocks and timers There are two clocks per partition XM HW CLOCK Associated with the native hardware clock The resolution is 1yusec XM EXEC CLOCK Associated with the execution of the partition This clock only advances while the partition is being executed It can be used by the partition to detect overruns This clock relies on the XM HW CLOCK and its resolution is also 1 psec Only one timer can be armed for each clock In the multicore approach XtratuM provides e one global hardware clock XM HW CLOCK global for all virtual CPU e one local execution clock XM EXEC CLOCK for each virtual CPU Each virtual CPU can
14. 1585 1590 1595 1600 1605 1610 1615 1620 60 134 Chapter 5 Partition Programming returns then an endless loop is executed The remaining of this file contains the trap handler routines Note that the assembly routines are only provided as illustrative examples and should not be used on production application systems These trap routines just jump to C code which is located in the file traps c include lt xm h gt include lt xm_inc arch paging h gt include std_c h extern void start void static xm u8 t _pageTable PAGE_SIZE 32 __attribute__ aligned PAGE_SIZE __attribute__ section bss noinit struct xmImageHdr xmImageHdr __XMIHDR sSignature XMEF_PARTITION_MAGIC compilationXmAbiVersion XM_SET_VERSION XM_ABI_VERSION XM_ABI_SUBVERSION XM_ABI_REVISION compilationXmApiVersion XM_SET_VERSION XM_API_VERSION XM_API_SUBVERSION XM_API_REVISION pageTable xmAddress t pageTable pageTableSize 32 PAGE SIZE noCustomFiles 0 dif 0 customFileTab 0 struct xefCustomFile sAddr xmAddress_t 0x40105bc0 size 0 fs endif e8ignature XMEF PARTITION MAGIC void attribute weak ExceptionHandler xm_s32_t trapNr 1 xprintf exception Ox 4x d Wn trapNr trapNr XM halt partition XM PARTITION SELF void attribute weak ExtIrqHandler xm s32 t trapNr 1 xprintf extIrq Ox 4x d Wn trapNr trapNr XM halt partition XM PARTITION
15. 595 600 605 610 24 134 Chapter 2 XtratuM Architecture lt IoMmu gt lt AhbMst id 0 partitionId 1 busRouting processor vendorld 0x1 deviceId 0x20 gt lt AhbMst id 1 partitionId O busRouting memory vendorld 0x1 deviceId 0x20 gt lt AhbMst id 2 partitionId O busRouting memory vendorld 0x1 deviceld 0x20 IoMmu lt XMHypervisor gt lt PartitionTable gt Partition id 0 name Partitioni flags console Uart gt lt PhysicalMemoryAreas gt lt Area start 0x300000 size 512KB gt lt Area start 0x80000000 size 4KB flags iommu gt lt PhysicalMemoryAreas gt lt TemporalRequirements duration 500ms period 500ms gt lt Partition gt Partition id 1 name Partition2 flags system console Uart gt lt PhysicalMemoryAreas gt lt Area start 0x500000 size 512KB flags uncacheable gt lt Area start 0x80001000 mappedAt 0xff000000 size 512KB flags uncacheable iommu gt lt PhysicalMemoryAreas gt lt Partition gt lt PartitionTable gt lt SystemDescription gt Listing 2 1 XML example In the example two master buses AHB1 and AHB2 use an io mmu table having access to the area 0x80000000 0x80000fff area defined by the partition 0 and the bus AHBO which has been allo cated to the partition 1 uses a table which translates any access to the area Oxff000000 0xff070000 to 0x80001000 0x8008ffff In this example the IT
16. 6 4 1 Partition image headet o ss 224 suo ko eR ee Roe Ry we R4 90 6 4 2 Partition control table PCT 91 0 5 XEE format 323 A A a A Nr Vc BGG AO OS 93 6 5 1 Gompr ssionaleofithm urraca RR ROUX UR 95 6 0 Container format ioe y un ee ee OR d AUR ad ovo Re idees 96 7 Booting 99 Zl Boot configuration 40002 eee Gab 2 bs oe SSS OEY cee eae eee x 100 72 Booting process in LEONA one osos ome e o oes a mk a a ow ae aes 101 7 2 1 Monocore partition booting es sse ok e RR a E ee s 102 7 2 2 Multicore partition booting aa 102 8 Configuration 105 8 1 XtratuM source code configuration menuconfig o 105 8 2 Resident software source code configuration menuconfig 107 8 2 1 Memory requirements 422 6 6 eoe ek e o EUR RS RR 108 8 3 Hypervisor configuration file XM_CF o o e 109 8 3 1 Data representation and XPath syntax o o llle 109 8 3 2 The root element SystemDescription llle 111 8 3 3 The SystemDescription XMHypervisor element 111 8 3 4 The SystemDescription HwDescriptionelement 113 8 3 5 The SystemDescription ResidentSw element 114 8 3 6 The SystemDescription PartitionTable Partition element 114 8 3 7 The SystemDescription Channels element ooo o 115 9 Tools 117 9 1 XML configuration parser xmcparser ves raro
17. A partition can halt itself or be halted by a system partition In the halt state the partition is not se lected by the scheduler and the time slot allocated to it is left idle it is not allocated to other partitions 1we will consider that the partition code is composed of an operating system and a set of applications xm 4 usermanual 047d A ic emu PA Printed November 26 2012 215 220 225 230 235 240 245 250 255 260 12 134 Chapter 2 XtratuM Architecture All resources allocated to the partition are released It is not possible to return to normal state In suspended state a partition will not be scheduled and interrupts are not delivered Interrupts raised while in suspended state are left pending If the partition returns to normal state then pending interrupts are delivered to the partition The partition can return to ready state if requested by a system partition by calling XM resume partition hypercall 2 4 2 Partition services Table 2 9 lists the partition services Service Description XM get partition status Get the status of the partition XM halt partition Halt the partition XM reset partition Reset the partition XM_resume_partition Resume the partition XM shutdown partition Shutdown the partition XM suspend partition Suspend the partition Figure 2 9 Partition services 2 5 Virtual CPU states and operation From the CPU point of view
18. GNU Free Documentation License You may not copy modify sublicense or distribute the Document except as expressly provided for under this License Any other attempt to copy modify sublicense or distribute the Document is void and will automatically terminate your rights under this License However parties who have received copies or rights from you under this License will not have their licenses terminated so long as such 330 parties remain in full compliance 10 FUTURE REVISIONS OF THIS LICENSE The Free Software Foundation may publish new revised versions of the GNU Free Documentation License from time to time Such new versions will be similar in spirit to the present version but may differ in detail to address new problems or concerns See http www gnu org copyleft 333 Each version of the License is given a distinguishing version number If the Document specifies that a particular numbered version of this License or any later version applies to it you have the option of following the terms and conditions either of that specified version or of any later version that has been published not as a draft by the Free Software Foundation If the Document does not specify a version number of this License you may choose any version ever published not as a draft by the Free az Software Foundation ADDENDUM How to use this License for your documents To use this License in a document you have written include a copy of the L
19. Perform the installation using the above settings Y n Y Installation completed Listing 4 1 Output of the self executable distribution file xm 4 usermanual 047d A ic emu P Printed November 26 2012 1200 1205 1210 1215 1220 50 134 Chapter 4 XtratuM Configuration and Build 4 5 Compile the Hello World partition 1 Change to the INSTALL PATH xm examples hello world directory 2 Compile the partition make Created by xmuser on xmhost at Wed Feb 3 13 21 02 CET 2011 XM path home xmuser xm2env xm XtratuM Core Version 3 1 2 Arch sparcv8 File home xmuser xm2env xm 1ib xm_core xef Shal 962b930e8278df873599e6b8bc4f cb939eb92a19 Changed 2010 02 03 13 19 51 000000000 0100 XtratuM Library Version 3 1 2 File home xmuser xm2env xm lib libxm a Shai 46f64cf2510646833a320e1e4a8ce20e4cd4e0a9 Changed 2010 02 03 13 19 51 000000000 0100 XtratuM Tools File home xmuser xm2env xm bin xmcparser Shal 8fa5dea03739cbb3436d24d5bc0e33b20906c47a Note that the compilation is quite verbose the compilation commands messages detailed infor mation about the tools libraries used etc are printed The result from the compilation is a file called resident sw 3 To run this file just load it in the tsim or grmon and run go it tsim leon3 mmu serial port A on stdin stdout allocated 4096 K RAM memory in 1 bank s allocated 16 M S
20. Software traps TrapHandlerN 4 TrapHandlerN PartitionMain _asm_ ta 0x81 Software traps Hypercall Li PartitionMain asm ta 0x81 wes Exceptions M Swire Sanct 7 rJ oftware Traps Software Traps Extended Irqs L Sotware Traps y Health Monitoring Hardware Irqs Health Monitoring Cae XtratuM Irq Controller Irq Controller P E Peripherals b Peripherals Figure 2 17 Software traps Extended Irqs Hardware Irqs XtratuM fl Inter Partition Virtual Interrupts define XM_MAX_IPVI CONFIG_XM_MAX_IPVI 885 define XM_VT_EXT_IPVIO 24 XM_VT_EXT_FIRST define XM_VT_EXT_IPVI1 25 XM_VT_EXT_FIRST define XM VT EXT IPVI2 26 XM VT EXT FIRST define XM VT EXT IPVIS3 27 XM VT EXT FIRST define XM VT EXT IPVIA 28 XM VT EXT FIRST 890 define XM VT EXT IPVI5 29 XM VT EXT FIRST define XM VT EXT IPVI6 30 XM VT EXT FIRST define XM VT EXT IPVI7 31 XM VT EXT FIRST Listing 2 3 sources core include guest h IPVIs are raised by partitions without the explicit knowledge of the partition or partitions that will sos receive them The connection or channel that defines the IPVI route is specified in the configuration file XM CF Next example shows the specification of the IPVI route Channels lt Ipvi id 0 sourceld 0 desti
21. are managed by XtratuM thorough the health monitoring system define DATA STORE ERROR Ox2b O define INSTRUCTION ACCESS MMU MISS Ox3c 1 define INSTRUCTION ACCESS ERROR 0x21 2 define R REGISTER ACCESS ERROR 0x20 3 define INSTRUCTION ACCESS EXCEPTION 0x1 4 define PRIVILEGED INSTRUCTION 0x03 5 define ILLEGAL INSTRUCTION Ox2 6 define FP DISABLED Ox4 7 define CP DISABLED Ox24 8 define UNIMPLEMENTED_FLUSH 0x25 9 define WATCHPOINT_DETECTED Oxb 10 define WINDOW_OVERFLOW 0x5 ttdefine WINDUW UNDERFLOW 0x6 define MEM ADDRESS NOT ALIGNED 0x7 11 define FP EXCEPTION Ox8 12 define CP EXCEPTION 0x28 13 define DATA ACCESS ERROR 0x29 14 define DATA ACCESS MMU MISS Ox2c 15 define DATA ACCESS EXCEPTION Ox9 16 define TAG OVERFLOW Oxa 17 define DIVISION BY ZERO Ox2a 18 Listing 5 24 core include arch irqs h Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 5 12 Clock and timer services 73 134 If the health monitoring action associated with the HM event is XM HM AC PROPAGATE then the same trap number is propagated to the partition as a virtual trap The partition code is then in charge of handling the error 5 12 Clock and timer services XtratuM provides the XM get time O hypercall to read the time from a clock and the XM set timer hypercall to arm a timer There are two clocks available define XM HW CLOCK 0x0 defi
22. as the dev directory Most of the libxm hypercalls are implemented using this file system The hypercalls to access the objects are used inter nally by the libxm and shall not be used by the programmer They are listed here just for completeness extern __stdcall xm_s32_t XM_get_gid_by_name xm_u8_t name xm_u32_t entity extern __stdcall xmId_t XM_get_vcpuid void Time management hypercalls extern __stdcall xm_s32_t XM_get_time xm_u32_t clock_id xmTime_t time extern __stdcall xm s32 t XM set timer xm u32 t clock id xmTime t abstime xmTime t interval Partition status hypercalls Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 5 19 Assembly programming 83 134 extern stdcall xm s32 t XM suspend partition xm u32 t partition id extern stdcall xm s32 t XM resume partition xm u32 t partition id extern stdcall xm s32 t XM shutdown partition xm u32 t partition id 2440 extern stdcall xm s32 t XM reset partition xm u32 t partition id xm u32 t resetMode xm u32 t status extern stdcall xm s32 t XM halt partition xm u32 t partition id extern stdcall xm s32 t XM idle self void extern stdcall xm s32 t XM suspend vcpu xm u32 t vcpu id 2445 extern stdcall xm s32 t XM resume vcpu xm u32 t vcpu id extern stdcall xm s32 t XM reset vcpu xm u32 t vcpu id xmAddress t ptdL1 xmAddress t entry xm u32 t status extern stdcall xm s32 t XM halt vcpu xm u32 t vcpu id 2450
23. done by creating the configuration file XM CF file 8 Using the binaries resulted from the compilation of XtratuM and the system configuration file partition developers can implement and test its own partition code by their own 4 The tool xmpack is used to build the complete system hypervisor plus partitions code The result is a single file called container Partition developers shall replace the image of non available partitions by a dummy partition Up to a maximum of CONFIG MAX NO CUSTOMFILES customisation files can be attached to each partition The container shall be loaded in the target system using the corresponding resident software or boot loader For convenience a resident software is provided 3 2 Building XtratuM In the first stage XtratuM shall be tailored to the hardware available on the board and the ex pected workload This configuration parameters will be used in the compilation of the XtratuM code xm 4 usermanual 047d X irem uPA Printed November 26 2012 1050 1055 1060 1065 1070 1075 1080 1085 42 134 Chapter 3 Developing Process Overview Main Menu Arrow keys navigate the menu Enter selects submenus gt Highlighted letters are hotkeys Pressing Y includes lt N gt excludes M modularizes features Press lt Esc gt lt Esc gt to exit lt gt for Help Legend built in excluded M module lt gt module capable gd _ernel features
24. 1 1 Required resources 1 K 1 L 2 Configura amp Build I IH j E i I I i I I UM T i H 3 Install amp Create a Distribution i i T i j 4 Distribution XM_CF L 1 i i 5 Build Execution Environment i I I I 6 Develop Application I 1 i i IH I I 1 7 Build Partition Image i i I 8 Send Partition Image AS l I KC i i 9 Pack the System f IH I 10 Deploy amp Run L k Figure 3 1 Integrator and partition developer interactions 1 Define the resources required by its application and send it to the integrator Prepare the development environment Install the binary distribution created by the integrator Develop the partition application according to the system resources agreed by the integrator gt 4 m Deliver to the integrator the resulting partition image and the required customisation files if any There should be an agreement between the integrator and the partition developers on the resources allocated to each partition The binaries jointly with the XM_CF configuration file defines the partitioned system All partition developers shall use exactly the same XtratuM binaries and configuration files during the development Any change on the configuration shall be agreed with the integrator Since the development of the partitions may be carried out in parallel or
25. 10 Peripheral programming The LEON3 processor implements a memory mapped I O for performing hardware input and output operations to the peripherals There are two hypercalls to access I O registers XM arch inport O and XM_ arch _outport In order to be able to access read from or write to hardware I O port the corresponding ports have to be allocated to the partition in the XM_CF configuration file There are two methods to allocate ports to a partition in the configuration file Range of ports A range of I O ports with no restriction allocated to the partition The Range element is used Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 5 11 Traps interrupts and exceptions 69 134 Restricted port A single I O port with restrictions on the values that the partition is allowed to write in The Restricted element is used in the configuration file A restricted port includes the following information Address Port address Mask Port mask Only those bits that are set can be modified by the partition In the case of a read operation only those bits set in the mask will be returned to the partition the rest of the bits will be resetted The attribute mask is optional A restricted port declaration with no attribute is equivalent to declare a range of ports of size one In the case that both the bitmap and the range of values are specified then the bitmap is applied first and then the range is checked
26. 108 134 Chapter 8 Configuration Read only section addresses RSW memory layout Read only area The resident software will be compiled to use this area Read write section addresses RSW memory layout Read write area used by the resident software CPU frequency KHz The processor frequency is passed to XtratuM via the XM CF file but in the case of the RSW it has to be specified in the source code since it has no run time configuration The processor frequency is used to configure the UART clock Enable UART support Select the serial line to write messages to UART baud rate The baud rate of the UART Note that the baud rate used by XtratuM is configured in the XM CF file and not in the source code configuration Stand alone version If not set then the resident software shall be linked jointly with the container That is the final resident software image shall contain as data the container If set then the container is not linked with the resident software The address where the container will be copied in RAM is specified by the next option Container physical location address Address of the container In case of the stand alone version 8 2 1 Memory requirements The memory footprint of XtratuM is independent of the workload number of partitions channels etc The memory needed depends only on actual workload defined in the XM_CF file The size of the compiled configuration provides an accurate estimation of the me
27. 1540 add fp 48 fp 0x00 reset mov 410 oi t reset b start mov sparc set psr nr 00 nop __XM_AHC nop set sparc iret nr 00 1490 nop 1545 __XM_AHC 0x01 data instruction_access_exception align 8 BUILD_TRAP 0x1 Stack 1495 1550 fill STACK SIZE 4 4 0 0x02 illegal instruction Stack top BUILD TRAP 0x2 previous 0x03 privileged instruction 1500 BUILD_TRAP 0x3 1555 define BUILD IRQ irqnr 0x04 fp_disabled sethi 4hi HwIrqHandlerAsm BUILD_TRAP 0x4 14 jmpl 414 1o 1505 0x05 window_overflow 1560 HwIrgHandlerAsm gO BUILD_TRAP 0x5 mov irqnr 15 nop E ea define BUILD TRAP trapnr 1510 sme io p i Listing 5 6 user examples sparcv8 boot S The xmImageHdr declares the required image header see section 6 and one partition header xmPartitionHdr The entry point of the partition the first instruction executed is labeled start First off the bss section is zeroed the stack pointer sp register is set to a valid address the address of the partition header is passed to the libxm call InitLibxm the virtual trap table register is loaded with the 157 direction of _traptab and finally the user routine PartitionMain is called If the main function Multiple partition headers can be declared to allocate several processors to a single partition experimental feature not documented xm 4 usermanual 047d A ic emu P Printed November 26 2012 1575 1580
28. 2 4 i386 v Second proof of concept 2005 GD DA Linux 2 4 i386 T Cod itten fi tch ode rewritten from scratch 2006 LO peecsecsscenuss ported to Linux 2 6 i386 Temporal amp spatial isolation 2007 ii 120 eee oes Linux independent hypervisor i386 2004 0 01 454 4 Highly Critical features added like ARINC 653 Sparcv8 Full featured highly critical hypervisor Sparcv8 i386 2011 lt 40 gt Highly Critical MMU Multiprocessor Q Linux module _ Temporal amp spatial Ongoing Only temporal isolation isolation developments Figure 1 1 XtratuM evolution The first version of XtratuM 1 0 was initially developed to meet the requirements of a hard real time system The main goal of XtratuM 1 0 was to guarantee the temporal constrains for the real time partitions Other characteristics of this version are Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 1 1 History 3 134 The first partition shall be a modified version of Linux Partition code has to be loaded dynamically There is not a strong memory isolation between partitions Linux is executed in processor supervisor mode Linux is responsible of booting the computer Fixed priority partition scheduling XtratuM 2 0 was a completely new redesign and implementation This new version had nothing in common with the first one but the name It was truly a hypervisor with both spatial an
29. 5 30 Memory areas allocation example The memory layout defined for the board is Memory type Initial End addres Size STRAM 40000000 403FFFFF 4 MB SDRAM 60000000 607FFFFF 8 MB STRAM FFFFFOOO FFFFFFFF 4 KB The memory areas defined and the owners is detailed in the next table NOTE The container address is set in the XtratuM configuration process In order to change it a new XtratuM compilation and installation has to be performed In this example PartitionO defines the following memory areas xm 4 usermanual 047d hr adu Printed November 26 2012 2275 2280 2285 2290 2295 2300 2305 2310 2315 2320 2325 2330 2335 2340 80 134 Chapter 5 Partition Programming Component Initial End addres Size Attributes Mapped at XtratuM 40000000 400FFFFF 1 MB PartitionO 40100000 4013FFFF 256 KB Partition1 40140000 4017FFFF 256 KB PartitionO 60200000 6020FFFF 64 KB 10000 PartitionO 60210000 6021FFFF 64 KB Shared rw 20000 Partition1 60210000 6021FFFF 64 KB Shared ro PartitionO 60220000 6022FFFF 64 KB Uncacheable RSW 60300000 6037FFFF 512KB Container 60400000 6047FFFF 512 KB PartitionO FFFFF000 FFFFFFFF 4 KB e A memory area of 256Kb that is allocated to the physical memory 0x40100000 e A memory area of 64Kb allocated in the physical memory at 0x60200000
30. On a cold reset the PCT table is rebuild resetCounter field is set to zero and resetStatus set to the value given on the hypercall the communication ports are closed the timers are disarmed 5 6 System reset There are two different system reset sequences Warm reset XtratuM jumps to its entry point This is basically a software reset Cold reset A hardware reset if forced See section 8 3 5 1800 The set of actions done on a warm system reset are still under development xm 4 usermanual 047d AX irem P Printed November 26 2012 1805 1810 1815 1820 1825 1830 66 134 Chapter 5 Partition Programming 5 7 Scheduling 5 7 1 Slot identification A partition can get information about which is the current slot being executed quering its PCT This information can be used to synchronise the operation of the partition with the scheduling plan The information provided is Slot duration The duration of the current slot The value of the attribute duration for the current slot Slot number The slot position in the system plan starting in zero Id value Each slot in the configuration file has a required attribute named id which can be used to label each slot with a user defined number The id field is not interpreted by XtratuM and can be used to mark for example the slots at the starts of each period 5 7 2 Managing scheduling plans A system partition can request a plan switch at any time
31. Ox401000008 Figure 8 2 Graphical view of an example XM_CF configuration file see the XML file in section Printed November 26 2012 hie ach xm 4 usermanual 047d 8 3 Hypervisor configuration file XM CF 113 134 Trace Defines where to store the traces messages emitted by XtratuM the value of the attribute device shall be a the name of a device defined in SystemDescription Devices and the hexadecimal bit mask to filter out which traces will not be stored bitmask A health monitoring event element contains the following attributes event name The event s name See 2 13 1 HM Events for the list of available HM events event action The name of the action associated with this event See 2 13 2 HM Actions for the list of available HM actions event log Boolean flag to select whether the event will be logged or not 8 3 4 The SystemDescription HwDescription element It contains three mandatory elements HwDescription ProcessorTable Which holds a sequence of Processor elements Each pro cessor element describes one physical processor the processor clock frequency the fre quency units has to be specified id zero in a mono processor system and an optional features attribute The features attribute contains a list of specific processor features than can be selected Currently only the memory protection workaround XM CPU LEON2 WA1 for the memory mapped processor re
32. The implementation of the bit mask is done as follows oldValue LoadIoReg port StoreloReg port oldValue amp mask value amp mask Listing 5 19 core kernel sparcv8 hypercalls c First the port is read to get the value of the bits not allocated to the partitions then the bits that have to be modified are changed and finally the value is written back The read operation shall not cause side effects on the associated peripheral For example some devices may interpret as interrupt acknowledge to read from a control port Another source of errors may happen when the restricted is implemented as an open collector output In this case if the pin is connected to an external circuit which forces a low voltage then the value read from the io port is not the same that the previous value written The following example declares a range of ports and two restricted ones lt Partition gt lt HwResources gt lt IoPorts gt lt Restricted address 0x3000 mask 0xff gt lt IoPorts gt lt HwResources gt lt Partition gt If the bitmask restriction is used then the bits of the port that are not set in the mask can be allocated to other partitions This way it is possible to perform a fine grain bit level port allocation to partitions That is a single ports can be safely shared among several partitions 5 11 Traps interrupts and exceptions 5 11 1 Traps A partition can not directly manage pro
33. The include header which contains all the definitions and declarations of the 1ibxm a library is xm h 10 This file depends includes also the next list of files user libxm include xm inc config h user libxm include xm_inc autoconf h user libxm include xm_inc arch arch_types h 1755 user libxm include xm_inc xmef h user libxm include xm inc linkage h user libxm include xm inc arch linkage h user libxm include xm inc arch paging h user libxm include xmhypercalls h 1760 user libxm include xm_inc hypercalls h user libxm include xm_inc arch hypercalls h user libxm include xm_inc arch irqs h user libxm include xm_inc arch xm_def h user libxm include xm_inc arch leon h 1765 user libxm include arch xmhypercalls h user libxm include xm_inc objdir h user libxm include xm_inc guest h user libxm include xm_inc arch atomic h user libxm include xm_inc arch guest h 1770 user libxm include xm_inc arch processor h user libxm include xm_inc xmconf h user libxm include xm_inc arch xmconf h user libxm include xm_inc devid h user libxm include xm_inc objects hm h 1775 user libxm include comm h user libxm include xm_inc objects commports h user libxm include hm h user libxm include hmapp h user libxm include xm inc objects hmapp h 1780 user libxm include hypervisor h user libxm include arch hypervisor h user libxm include trace h user libxm include xm inc objects trace h user libxm include status h 1785 user libxm include xm inc objects status h us
34. XtratuM mimics the behaviour of a multicore system When the system is booted CPUO is started and is the software operating systems the responsible of start the execution of the rest of CPUs XtratuM offers the same behaviour to partition for the virtual CPU When booted each partition they have vCPUO running and it is responsability of each partition to start the execution of others virtual CPU s if required virtual CPU s are internal abstractions to each partition virtual CPU s only can be seen and handled by its partition A partition can not access to any service related to virtual CPU s of other partition virtual CPU s as partitions are controlled through a set of services that change its status The virtual CPU s states and transitions are shown in figure 2 10 On start up each virtual CPU is in boot state It has to prepare the virtual CPUs to be able to run the applications it sets up a standard execution environment that is initialises a correct stack and sets up each virtual CPU control registers creates the communication ports requests the hardware devices I O ports and interrupt lines etc that it will use Once the virtual CPU has been initialised it changes to normal mode From the hypervisor point of view there is no difference between the boot state and the normal state In both states the virtual CPUs are scheduled according to the fixed plan and has the same capabilities The normal state is subdivided
35. addition you must do these things in the Modified Version A Use in the Title Page and on the covers if any a title distinct from that of the Document and from those of previous versions which should if there were any be listed in the History section of the Document You may use the same title as a previous version if the original publisher of that version gives permission B List on the Title Page as authors one or more persons or entities responsible for authorship of the modifications in the Modified Version together with at least five of the principal authors of the Document all of its principal authors if it has fewer than five unless they release you from this requirement C State on the Title page the name of the publisher of the Modified Version as the publisher D Preserve all the copyright notices of the Document E Add an appropriate copyright notice for your modifications adjacent to the other copyright no tices F Include immediately after the copyright notices a license notice giving the public permission to use the Modified Version under the terms of this License in the form shown in the Addendum below xm 4 usermanual 047d Heca P Printed November 26 2012 3600 3605 3610 3615 3620 3625 3630 3635 3645 3650 3655 3660 3665 3670 3675 3680 130 134 Appendix A GNU Free Documentation License G Preserve in that license notice the full lists of Inva
36. appendix or a front matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document s overall subject or to related matters and contains nothing that could fall directly within that overall subject Thus if the Document is in part a textbook of mathematics a Secondary Section may not explain 127 134 3520 3525 3530 3535 3540 3545 3555 3560 3565 3570 3575 3580 3585 3590 3595 128 134 Appendix A GNU Free Documentation License any mathematics The relationship could be a matter of historical connection with the subject or with related matters or of legal commercial philosophical ethical or political position regarding them The Invariant Sections are certain Secondary Sections whose titles are designated as being those of Invariant Sections in the notice that says that the Document is released under this License If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant The Document may contain zero Invariant Sections If the Document does not identify any Invariant Sections then there are none The Cover Texts are certain short passages of text that are listed as Front Cover Texts or Back Cover Texts in the notice that says that the Document is released under this License A Front Cover Text may be at most 5 words and a Back Cover Text may be at most 25
37. are defined Cyclic scheduling Pairs lt partition vcpu gt are scheduled in a fixed cyclic basis ARINC 653 scheduling policy This policy ensures that one partition cannot use the processor for longer than scheduled to the detriment of the other partitions The set of time slots allocated to each lt partition vcpu gt is defined in the XM CF configuration file Each lt partition vcpu gt is sched uled for a time slot defined as a start time and a duration Within a time slot XtratuM allocate the system resources to the partition and virtual CPU specified Priority scheduling Under this scheduling policy pairs lt partition vcpu gt are scheduled based on the partition priority The partition priority is specified in the configuration file Priority 0 corresponds to the highest priority All pairs lt partition vcpu gt in normal state ready allocated in the configuration file to a processor attached to this policy are executed taking into account its priority 2 9 3 Cyclic scheduling In general a cyclic plan consists of a major time frame MAF which is periodically repeated The MAF is typically defined as the least common multiple of the periods of the partitions or the periods of the threads of each partition if any This plan has to be specified in the configuration file An XML file describing this schedule is shown below lt Processor id 0 frequency 50Mhz gt lt CyclicPlanTable gt lt Plan
38. but mapped at 0x1000 So internally the partition can access to this area using this virtual address e A memory area of 64Kb allocated in the physical memory at 0x60210000 that is shared with other partitions This partition can read and write on this area e A memory area of 64Kb allocated in the physical memory at 0x60220000 that is not cacheable e A memory area of 4Kb allocated in the physical memory at Oxfffff000 but mapped at 0x2000 On the other hand Partition1 defines e A memory area of 256Kb that is allocated to the physical memory 0x40140000 and mapped at 0x40000000 e A memory area of 64Kb allocated in the physical memory at 0x60210000 that is shared with other partitions This partition can only read on this area 5 17 Releasing the processor In some situations a partition is waiting for a new event to execute a task If no more tasks are pending to be executed then the partition can become idle The idle partition becomes ready again when an interrupt is received The partition can inform to XtratuM about its idle state see XM idle self O In the current imple mentation XtratuM does nothing while a partition is idle that is other partition is not executed but it opens the possibility to use this wasted time in internal bookkeeping or other maintenance activities Also energy saver actions can be done during this idle time Since XtratuM delivers an event on every new slot the idle
39. by calling XM resume vcpu O hypercall The state of the virtual CPU s of a partition depends on the partition state When a partition manage ment service is invoked to a target partition the following effects are produced Partition reset All virtual CPU s of the partition are halted The partition restarts with the vCPUO Partition suspend All virtual CPU s of the partition in normal state are suspended Partition resume All virtual CPU s of the partition as they were when the partition was suspended are restored 275 Partition halt All virtual CPU s of the partition are halted 2 5 1 virtual CPU services Table 2 1 lists the virtual CPU services 2 6 System partitions XtratuM defines two types of partitions normal and system System partitions are allowed to manage and monitor the state of the system and other partitions Some hypercalls only can be invoked from 2x0 xm 4 usermanual 047d A irem uPA Printed November 26 2012 14 134 Chapter 2 XtratuM Architecture Service Description XM get vcpuid Get the vCPU identifier XM halt vcpu Halt the vCPU XM reset vcpu Reset the vCPU XM resume vcpu Resume the vCPU XM suspend vcpu Suspend the vCPU Table 2 1 CPU services a system partition or more properly these hypercall only can succeed when are invoked from system partition Normal partitions requesting these services get a permission error as response It is important
40. equal to 32KB e The size shall be a power or two e The start address shall be a multiple of the size Memory Management Unit MMU If the processor has MMU and XtratuM has been compiled to use it then fine grain page size memory protection provided In this case one or more areas of memory can be allocated to each partition The MMU is used only as a MPU memory protection unit i e the virtual and physical addresses are the same Only the protections bits of the pages are used As a result each partition shall be compiled and linked to the designated addresses where they will be loaded and executed The memory protection mechanism employed is a source code configuration option See section 8 1 The memory areas allocated to a partition are defined in the XM CF file The executable image shall be linked to be executed in those allocated memory areas The XM get physmem map O returns the set of memory areas allocated the partition Available since XtratuM 3 1 5 16 1 Configuration A partition can statically allocates several memory areas to a partition Memory areas have to be properly configured in the XM CF configuration file in a coherent way with respect to the memory available and the allocation of memory areas to XtratuM container booter and other partitions Printed November 26 2012 A ir amu PA xm 4 usermanual 047d 5 16 Memory management 79 134 An example of the memory areas definition in the XM CF con
41. hypercalls h gt define STACK_SIZE 8192 define MIN_STACK_FRAME 0x60 text align 8 global start start _start start Zero out our BSS section set _sbss 00 set _ebss 01 1 st g0 00 Subcc 00 01 g0 bl 1b add 00 0x4 00 set stack top fp mov Agi 00 call init libxm sub fp MIN STACK FRAME sp if 0 set Oxfa000002 Asp sub Asp 4 sp save save save save save save save save restore restore restore restore restore restore restore restore endif Set up TBR set sparc_write_tbr_nr 00 set _traptab 01 __XM_HC call PartitionMain sub fp MIN STACK FRAME sp set 0 o1 mov halt partition nr 00 XM HC 1 b 1b nop ExceptionHandlerAsm 1390 1395 1400 1405 1410 1415 1420 1425 1430 1435 1440 1445 1450 1455 mov sparc get psr nr 00 __XM_AHC mov 00 10 set sparc_flush_regwin_nr o0 __XM_AHC sub fp std 410 std 7 12 std g6 fp 24 std g4 fp 16 std g2 4fp 8 st Agi Afp 4 rd hy g5 st 4g5 tp 48 fp p 40 4fp 32 mov 415 00 call ExceptionHandler sub fp 0x80 sp ld fp gl wr Agi Ay ld 4fp 41 g1 ldd A4fp 8 g2 ldd fp 161 g4 ldd fp 241 g6 ldd 4fp 32 412 ldd fp 40 410 add fp 48 fp mov 10 oi mov sparc set psr nr 00 __XM_AHC set sparc iret nr 00 __XM_AHC ExtIrqHandlerAsm mov sparc_get_psr_nr 00 __XM_AHC mov 00 10 s
42. in the configuration file An XML file describing this schedule is shown below Processor id 0 frequency 50Mhz gt lt CyclicPlanTable gt lt Plan id 0 majorFrame 200ms gt lt Slot id 0 start 0ms duration 20ms partitionld 0 gt lt Slot id 1 start 20ms duration 10ms partitionId 1 For efficiency and simplicity reasons xm 4 usermanual 047d A ic emu P Printed November 26 2012 295 300 305 310 315 320 325 335 340 345 350 16 134 Chapter 2 XtratuM Architecture Name Period WCET Util 96 Partition 1 System Mngmt 100 20 20 Partition 2 Flight Control 100 10 10 Partition 3 Flight Mngmt 100 30 30 Partition 4 IO Processing 100 20 20 Partition 5 IHVM 200 20 10 a Partition set Start Dur Start Dur Start Dur Start Dur Partition O 0 20 100 20 Partition 1 20 10 120 10 Partition 2 40 30 140 30 Partition 3 30 10 70 10 130 10 170 10 Partition 4 180 20 b Detailed execution plan Table 2 3 Partition definition lt Slot id 2 start 30ms duration 10ms partitionld 3 gt lt Slot id 3 start 40ms duration 30ms partitionld 2 gt lt Slot id 4 start 70ms duration 10ms partitionld 3 gt lt Slot id 5 start 100ms duration 20ms partitionld 0 lt Slot id 6 start 120ms duration 10ms partitionld 1 gt lt Slot id
43. inter rupt line To do so the interrupt line shall be allocated to the partition in the configuration file There are two groups of virtual interrupts Hardware interrupts Correspond to the native hardware interrupts Note that SPARC v8 defines only 15 interrupts from 1 to 15 but XtratuM reserves 32 for compatibility with other architectures Interrupts 1 to 15 are assigned to traps 0x11 to Ox1F respectively as in the native hardware Extended interrupts Correspond to the XtratuM extended interrupts These interrupts are assigned from traps OxEO to OxFF define XM VT HW FIRST 0 define XM VT HW LAST 31 define XM VT HW MAX 32 Printed November 26 2012 A ir amu PA xm 4 usermanual 047d 5 11 Traps interrupts and exceptions 71 134 define XM_VT_HW_INTERNAL_BUS_TRAP_NR 1 XM_VT_HW_FIRST define XM_VT_HW_UART2_TRAP_NR 2 XM_VT_HW_FIRST define XM_VT_HW_UART1_TRAP_NR 3 XM_VT_HW_FIRST define XM_VT_HW_IO_IRQO_TRAP_NR 4 XM_VT_HW_FIRST define XM_VT_HW_IO_IRQ1_TRAP_NR 5 XM_VT_HW_FIRST define XM VT HW IO IRQ2 TRAP NR 6 XM_VT_HW_FIRST define XM VT HW IO IRQ3 TRAP NR 7 XM VT HW FIRST define XM VT HW TIMER1 TRAP NR 8 XM VT HW FIRST define XM VT HW TIMER2 TRAP NR 9XM VT HW FIRST define XM VT HW DSU TRAP NR 114XM VT HW FIRST define XM VT HW PCI TRAP NR 14 XM VT HW FIRST define XM VT EXT FIRST 0 define XM VT EXT LAST 31 define XM VT EXT MAX 32
44. is listed in the next table Action Description XM HM AC IGNORE XM HM AC SHUTDOWN M HM AC COLD RESET M HM AC WARM RESET M HM AC PARTITION COLD RESET M HM AC PARTITION WARM RESET M HM AC HYPERVISOR COLD RESET M HM AC HYPERVISOR WARM RESET M HM AC SUSPEND M HM AC HALT M HM AC PROPAGATE P P PS PS P PS PS P PS XM HM AC SWITCH TO MAINTENANCE 2 13 3 HM Configuration No action is performed The shutdown extended interrupt is sent to the failing parti tion The failing partition processor is cold reset The failing partition processor is warm reset The failing partition is cold reset The failing partition is warm reset The hypervisor is cold reset The hypervisor is warm reset The failing partition is suspended The failing partition processor is halted No action is performed by XtratuM The event is redirected to the partition as a virtual trap The current scheduling plan is switched to the maintenance one 755 There are two tables to bind the HM events with the desired handling actions XtratuM HM table which defines the actions for those events that must be managed at system or hypervisor scope Partition HM table which defines the actions for those events that must be managed at hypervisor or partition scope 760 xm 4 usermanual 047d hie adu Printed November 26 2012 765 770 775 780 785 790 795 800 30 134 Ch
45. mark the Start and End of the partition image The ported version of the previous simple code is the following include std c h Helper functions include lt xm h gt xm 4 usermanual 047d A ic emu P Printed November 26 2012 1625 1630 1635 1640 1645 1650 1655 1660 1665 1670 62 134 Chapter 5 Partition Programming void PartitionMain C code entry point int counter 0 xprintf Hello World n while 1 counter counter 100000 Listing 5 9 Ported simple example Listing 5 10 shows the main compilation steps required to generate the final container file which contains a complete XtratuM system of a system of only one partition The partition is only a single 16rs file called simple c This example is provided only to illustrate the build process It is advisable to use some of the Makefiles provided in the xm examples in the installed tree gt Compile the partition souce code simple c gt simple o sparc linux gcc Wall 02 nostdlib nostdinc Dsparcv8 fno strict aliasing V fomit frame pointer include xm inc config h include xm inc arch arch types h I libxm include DCONFIG_VERSION 2 DCONFIG SUBVERSION 1 DCONFIG_REVISION 3 g D_DEBUG_ c o simple o simple c gt Link it with the startup libezamples a sparc linux ld o simple simple o n u start T lib loader lds L lib L xm li
46. notification message is stored in the HM log stream if the HM event is marked to generate a log A system partition can then read those log messages and perform a more advanced error handling As an example of what can be implemented 1 Configure the HM action to stop the faulting partition and log the event 2 The system partition can resume an alternate one a redundant dormant partition which can be implemented by another developer team to achieve diversity 5 Since the differences between fault and error are so subtle and subjective we will use both terms to refer to the original reason of an incorrect state The XtratuM health monitoring subsystem defines four different execution scopes depending on which part of the system has been initially affected 1 Process scope Partition process or thread 2 Partition scope Partition operating system or run time support 3 Hypervisor scope XtratuM code 4 Board scope Resident software BIOS BOOT ROM or firmware The scope where an HM event should be managed has to be greater than the scope where it is believed it can be produced There is not a clear and unique scope for each HM event Therefore the same HM event may be handled at different scopes For example fetching an illegal instruction is considered hypervisor scope if it happens when while XtratuM is executing and partition level if the event is raised while a partition is running XtratuM tries to det
47. only will use one of the virtual CPU available vCPUO Others partitions are multicore and will use several virtual CPUs The system integrator is in charge of allocate the virtual CPUs to real CPUs in the configuration file Figure 2 5 shows an example of this allocation Printed November 26 2012 hie aru xm 4 usermanual 047d 2 2 Architecture 9 134 vCPUO vCPU2 vCPU1 Multicore processor Figure 2 4 Example of mono and multi core partitions Figure 2 5 vCPU CPU allocation scheme xm 4 usermanual 047d hie ahul Printed November 26 2012 195 200 205 210 10 134 Chapter 2 XtratuM Architecture 2 3 System states and services 2 3 1 System states The system s states and its transitions are shown in figure 2 6 Hardware reset XM halt system or XM HM AC HYPERVISOR COLD RESET or XM HM AC HYPERVISOR WARM RESET Figure 2 6 System s states and transitions At boot time the resident software loads the image of XtratuM in main memory and transfers control to the entry point of XtratuM The period of time between starting from the entry point to the execution of the first partition is defined as boot state In this state the scheduler is not enabled and the partitions are not executed see chapter 7 At the end of the boot sequence the hypervisor is ready to start executing partition code The system changes to normal state and the scheduling plan is started Changing f
48. operations These operation can require allocating other resources different from the ones required during the operational mode 360 In order to deal with these issues XtratuM provides multiple scheduling plans that allows to reallocate the timing resources the processor in a controlled way In the scheduling theory this process is known as mode changes Figure 2 12 shows how the modes have been considered in the XtratuM scheduling A Boot a a o e um um uw Initial Normal dd Maintenance plan Figure 2 12 Scheduling modes The scheduler and so the plans is only active while the system is in normal mode Plans are defined in the XM CF file and identified by a identifier Some plans are reserved or have a special meaning 365 Plan 0 Initial plan The system selects this plan after a system reset The system will be in plan 0 until a plan change is requested It is not legal to switch back to this plan That is this plan is only executed as a consequence of a system reset software or hardware Plan 1 Maintenance plan This plan can be activated in two ways 370 e As a result of the health monitoring action XM HM AC SWITCH TO MAINTENANCE The plan switch is done immediately e Requested from a system partition The plan switch occurs at the end the current plan It is advisable to allocate the first slot of this plan to a system partition in order to start the main tenance activity as soon as po
49. p2 cfg size 16 lt Partition gt lt Package gt 9 4 Bootable image creator rswbuild 9 4 1 rswbuild Create a bootable image 360 SYNOPSIS rswbuild contailer bootable DESCRIPTION rswbuild is a shell script that creates a bootable file by combining the resident software code with the container file The container shall be a valid file created with the xmpack tool xs The resident software object file is read from the distribution directory pointer by the XTRATUM PATH variable USAGE EXAMPLES rswbuild container resident sw Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d Volume 2 XtratuM User Manual Chapter 10 Security issues This chapter introduces several security issues related with XtratuM which should be taken into account by partition developers 10 1 Invoking a hypercall from libXM Invoking a hypercall requires a non standard protocol which must be directly implemented in assembly code LibXM is a partition level C library deployed jointly with XtratuM aiming to hide this complexity and ease the development of C partitions From the security point of view XtratuM implements two stacks for each partition one managed by the partition user context and another internal managed directly by XtratuM supervisor context The partition stack is used by the libXM to prepare the call to XtratuM pretty much like the gLibc does Once the hypercall service is invok
50. port Since there is only one single interrupt line to notify for incoming messages on the reception of the interrupt the partition code has to determine which port or ports are ready to perform the operation XtratuM maintains a bitmap in the Partition Control Table to inform about the state of each port A 1 in the corresponding entry indicates that the requested operation can be performed When a new message is available in the channel XtratuM triggers an extended interrupt to the desti nation s 2 12 1 Multicore implications When the plan allocates partitions in several CPU s partitions involved in a communication can be running or not at the same time If they do not overlap in time the reader will receive the interrupt at the begining of its slot If they overlap as soon as the writer partition sends the message XtratuM will deliver the interrupt the reader partition that will be able to receive the message In order to implement the notification of the message a set of interrupts have been added List below shos these interrupts Inter Partition Virtual Interrupts define XM MAX IPVI CONFIG XM MAX IPVI define XM VT EXT IPVIO 24 XM VT EXT FIRST define XM VT EXT IPVI1 25 XM VT EXT FIRST define XM VT EXT IPVI2 26 XM VT EXT FIRST define XM VT EXT IPVI3 27 4XM VT EXT FIRST define XM VT EXT IPVIA 28 XM VT EXT FIRST define XM VT EXT IPVI5 29 XM VT EXT FIRST define XM VT EXT IPVI6 30 XM VT EXT
51. register is initialized with the address 0x00000000 Contrarily to other computers the PROM of the board does not have any 200s kind of resident software like booter that takes the control of the processor after the reset We have developed a small booting code called resident software which is in charge of the initial steps of booting the computer This software is not part of the container produced by the xmpack tool It is prepended to the container by the burning script 210 The board has two kind of RAM memory SRAM 4Mb and SDRAM 128Mb 1Known as BIOS in the personal computer area 99 134 2915 2920 2925 2930 2935 2940 2945 100 134 Chapter 7 Booting 7 1 Boot configuration The resident software is in charge of loading into memory XtratuM its configuration file XM CT and any partition jointly with its customisation file as found in the container The information hold by the XM CT file is used to load any partition image Additionally the resident software informs XtratuM which partitions have to be booted After starting XtratuM assumes that the partitions informed as ready to be booted are in RAM SRAM memory setting them in running state right after it finishes its booting sequence If a partition s code is not located within the container then XtratuM sets the partition in HALT state until a system partition resets it by using XM reset partition O hypercall In this case the RAM imag
52. requires more memory once loaded into memory deflatedFileSize When the XEF COMPRESS flag is set this field holds the size of the segment when uncompressed offset Location of the segment expressed as an offset in the partition binary image 6 5 1 Compression algorithm The compression algorithm implemented is Lempel Ziv Storer Szymanski LZSS It is a derivative of LZ77 that was created in 1982 by James Storer and Thomas Szymanski A detailed description of the algorithm appeared in the article Data compression via textual substitution published in Journal of the ACM The main features of the LZSS are 1 Fairly acceptable trade off between compression rate and decompression speed 2 Implementation simplicity 3 Patent free technology Aside from LZSS other algorithms which were regarded were huffman coding gzip bzip2 LZ77 RLE and several combinations of them Table 6 2 sketches the results of compressing XtratuM s core binary with some of these compression algorithms Algorithm Compressed size Compression rate 96 LZ77 43754 44 2096 LZSS 36880 53 0196 Huffman 59808 23 8096 Rice 32bits 78421 0 1096 RLE 74859 4 6096 Shannon Fano 60358 23 1096 LZ77 Huffman 36296 53 76 Table 6 2 Outcomes of compressing the xm core bin 78480 bytes file xm 4 usermanual 047d A ic emu P Printed November 26 2012 2785 2790 2795 2800 2805 2815 2820 2825 2830 2835 2840 2845 2850 9
53. sections Entitled Acknowledge ments and any sections Entitled Dedications You must delete all sections Entitled Endorsements 6 COLLECTIONS OF DOCUMENTS You may make a collection consisting of the Document and other documents released under this License and replace the individual copies of this License in the various documents with a single copy that is included in the collection provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects You may extract a single document from such a collection and distribute it individually under this License provided you insert a copy of this License into the extracted document and follow this License in all other respects regarding verbatim copying of that document 7 AGGREGATION WITH INDEPENDENT WORKS A compilation of the Document or its derivatives with other separate and independent documents or works in or on a volume of a storage or distribution medium is called an aggregate if the copyright resulting from the compilation is not used to limit the legal rights of the compilation s users beyond what the individual works permit When the Document is included in an aggregate this License does not apply to the other works in the aggregate which are not themselves derivative works of the Docu ment If the Cover Text requirement of section 3 is applicable to these copies of the Document then if the Document is les
54. size Size of the customisation file The address where the custom files are loaded shall belong to the partition 2630 The xmImageHdr structure has to be placed in a section named xmImageHdr An example of how the header of a partition can be created is shown in section 5 4 The remainder of the image is free to the partition developer There is not a predefined format or structure of where the code and data sections shall be placed 6 4 2 Partition control table PCT In order to minimize the overhead of the para virtualised services XtratuM defines a special data 235 structure which is shared between the hypervisor and the partition called Partition control table PCT There is a PCT for each partition XtratuM uses the PCT to send relevant operating information to the partitions The PCT is mapped as read only allowing a partition only to read it Any write access causes a system exception 2640 typedef struct xm_u32_t magic xm_u32_t xmVersion XM version xm_u32_t xmAbiVersion XM s abi version xm_u32_t xmApiVersion XM s api version 2645 xmSize t partCtrlTabSize xm u32 t resetCounter xm u32 t resetStatus xm u32 t cpuKhz define PCT GET PARTITION ID pct pct id amp Oxff 2650 define PCT_GET_VCPU_ID pct pct gt id gt gt 8 xmId_t id xmId_t noVCpus xmId_t schedPolicy xm_u32_t flags 2655 xm u32 t hwIrqs Hw interrupts belonging to the partition xm_s32_t noPhysicalMemAreas xm_s
55. source code jointly with the resident software Customise it for the tar get board processor model etc and a miscellaneous set of code options and limits debugging identifiers length etc See section 8 1 for a detailed description 2 Build XtratuM hypervisor binary user libraries and tools 3 Distribute the resulting binaries to the partition developers All partition developers shall use the same binary version of XtratuM 4 Allocate the available system resources to the partitions according to the resources required to execute each partition e memory areas where each partition will be executed or can use design the scheduling plan e communication ports between partitions e the virtual devices and physical peripherals allocated to each partition configure the health monitoring e etc By creating the XM CF configuration file See section 8 3 for a detailed description 5 Gather the partition images and customisation files from partition developers 6 Pack all the files resident software XtratuM binary partitions and configuration files into the final system image The partition developer activity lAlthough it is not mandatory to name XM CF the configuration file we will use this name in what follows for simplicity 39 134 1010 1015 1020 1025 1030 1035 1040 1045 40 134 Chapter 3 Developing Process Overview A A Integrator Partition Developer1 1 1
56. system partition Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 2 7 Names and identifiers 15 134 2 7 Names and identifiers Each partition is globally identified by a unique identifier id Partition identifiers are assigned by the integrator in the XM CF file XtratuM uses this identifier to refer to partitions System partitions use partition identifiers to refer to the target partition The C macro XM PARTITION SELF can be used by a partition to refer to itself These ids are used internally as indexes to the corresponding data structures The fist id of each object group shall start in zero and the next id s shall be consecutive It is mandatory to follow this order in the XM CF file The attribute name of a partition is a human readable string This string shall contain only the following set of characters upper and lower case letters numbers and the underscore symbol It is advisable not to use the same name on different partitions A system partition can get the name of another partition by consulting the status object of the target partition In order to avoid name collisions all the hypercalls of XtratuM contain the prefix XM Therefore the prefix XM both in upper and lower case is reserved 2 7 1 Virtual CPU identifiers Each virtual CPU in each partition is locally identified by a unique identifier vcpuld Each virtual CPU can identify itself by the symbol XM VCPU SELF These
57. system status hypercalls extern stdcall xm s32 t XM halt system void extern stdcall xm s32 t XM reset system xm u32 t resetMode Object related hypercalls 2455 extern __stdcall xm s32 t XM read object xm bjDesc t objDesc void buffer xm u32 t size xm u32 t flags extern __stdcall xm s32 t XM write object xm bjDesc t objDesc void buffer xm u32 t size xm u32 t flags 2460 extern stdcall xm s32 t XM seek object xmObjDesc t objDesc xm u32 t offset xm u32 t whence extern stdcall xm s32 t XM ctrl object xmObjDesc t objDesc xm u32 t cmd void arg 2465 Listing 5 33 user libxm include xmhypercalls h The following services are implemented through the object interface Communication ports e Console output Health monitoring logs e Memory access 2470 XtratuM and partition status Trace logs Serial ports For example the XM_hm_status hypercall is implemented in the libxm as 2475 xm_s32_t XM hm status xmHmStatus t hmStatusPtr 1 if libXmParams partCtrlTab XM get vcpuid flags amp XM PART SYSTEM return XM PERM ERROR if hmStatusPtr 1 2480 return XM INVALID PARAM return XM_ctrl_object OBJDESC_BUILD OBJ_CLASS_HM XM HYPERVISOR ID 0 XM HM GET STATUS hmStatusPtr xm 4 usermanual 047d A ir emu PA Printed November 26 2012 84 134 Y 2485 Listing 5 34 user libxm common hm c Chapter 5 Partition Programming
58. tar bz2 file in a temporal directory and execute the install script Alternatively if the distributed file is xtratum x x x run then just execute it The install script requires only two parameters 1 The installation path 2 The path to the proper cross compile toolchain Note that it is assumed that the host toolchain binaries can be located in the PATH variable It is necessary to provide again the path to the LEON3 toolchain because it may be located in a different place than in the system where XtratuM was build In any case it shall be the same version than the one used to compile XtratuM xtratum X X X run Verifying archive integrity All good Starting installation Installation log in tmp xtratum installer log 1 Select the directory where XtratuM will be installed The installation directory shall not exist 2 Select the target compiler toolchain binary directory arch sparc 3 Confirm the installation settings Important you need write permision in the path of the installation directory Continue with the installation Y n Y Press Enter for the default value or enter a new one Press TAB to complete directory names 1 Installation directory opt home xmuser xmenv 2 Path to the sparc toolchain opt cross compiler bin opt cross compiler bin Confirm the Installation settings Selected installation path home xmuser xmenv Selected toolchain path opt cross compiler bin 3
59. the level of efficiency 105 XtratuM is a bare metal hypervisor designed to achieve temporal and spatial partitioning for safety critical applications Figure 2 1 shows the complete architecture Supervisor Partitions User Partitions Para virtualised services Partition Control table Partition Control table Partition Control table User Mode PROCESSOR Memory Manager Scheduler IP Communications Clock amp Timers Mgnt Interrupt Mgnt Health Monitor Supervisor Mode Non shared Shared Peripherals Figure 2 1 XtratuM architecture The main components of this architecture are e Hypervisor XtratuM provides virtualisation services to partitions It is executed in supervisor mode of the procesor and virtualises the CPU memory interrupts and some specific peripherals 1o The internal XtratuM architecture includes the following components Memory management XtratuM provides a memory model for the partitions enforcing the spatial isolation It uses the hardware mechanisms to guarantee the isolation Scheduling Partitions are scheduled by using a cyclic scheduling policy Interrupt management Interrupts are handled by XtratuM and depending on the interrupt us nature propagated to the partitions XtratuM provides a interrupt model to the partitions 5 134 120 125 130 135 140 145 150 155 6 134 Chapter 2 XtratuM Architecture that ext
60. words A Transparent copy of the Document means a machine readable copy represented in a format whose specification is available to the general public that is suitable for revising the document straight forwardly with generic text editors or for images composed of pixels generic paint programs or for drawings some widely available drawing editor and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters A copy made in an otherwise Transparent file format whose markup or absence of markup has been arranged to thwart or discourage subsequent modification by readers is not Transparent An image format is not Transparent if used for any substantial amount of text A copy that is not Transparent is called Opaque Examples of suitable formats for Transparent copies include plain ASCII without markup Texinfo in put format LaTeX input format SGML or XML using a publicly available DTD and standard conforming simple HTML PostScript or PDF designed for human modification Examples of transparent image for mats include PNG XCF and JPG Opaque formats include proprietary formats that can be read and edited only by proprietary word processors SGML or XML for which the DTD and or processing tools are not generally available and the machine generated HTML PostScript or PDF produced by some word processors for output purposes only The Title Page mea
61. 2 9 4 Fixed priority scheduling The priority based policy selects between the ready VCPUs allocated to the core the VCPU with a highest priority This scheduling policy is preemptive allowing to switch to a higher priority VCPU at so any time The definition of the CPUs under the priority based scheduling policy and they allocation of pairs partition vcpu gt to it has to be specified in the configuration file A XML file describing this schedule is shown below 445 Processor id 2 frequency 50Mhz gt lt FixedPriority gt lt Partition id 0 vCpuld 3 priority 10 gt lt Partition id 1 vCpuId 1 priority 5 gt lt FixedPriority gt 450 lt Processor gt In this example the CPU2 is defined to be scheduled under the priority based scheduling policy Pairs lt P0 vCPU3 gt and lt P1 vCPUI1 gt are allocated to this processor and as consequence to this policy to this policy independently of the plan 455 xm 4 usermanual 047d A ir emu P Printed November 26 2012 20 134 Chapter 2 XtratuM Architecture Note that if the scheduling policy of a CPU is defined as fixed priority this CPU is not affected by the the cyclic plans of other CPUs In the multicore approach the following aspects have to be taken into account e Only cyclic plans are associated to the multiple plans 460 e If a processor is defined under a priority based policy the policy applies independently of the plan e
62. 32_t noCommPorts xm u8 t name CONFIG ID STRING LENGTH xm u32 t iFlags As defined by the ARCH ET PIL in sparc 2660 xm u32 t hwIrqsPend pending hw irqs xm 4 usermanual 047d A ic em J Printed November 26 2012 2665 2670 2675 2680 2685 2690 2695 2700 92 134 Chapter 6 Binary Interfaces xm u32 t hwIrqsMask masked hw irqs xm u32 t extIrqsPend pending extended irqs xm u32 t extIrqsMask masked extended irqs struct pctArch arch struct schedInfo schedInfo xm u16 t trap2Vector NO_TRAPS xm u16 t hwIrq2Vector CONFIG NO HWIRQS xm u16 t extIrq2Vector XM VT EXT MAX partitionControlTable t Listing 6 8 core include guest h The libxm call XM params get PCT returns a pointer to the PCT The architecture dependent part is defined in struct pctArch 1 xmAddress t tbr ifdef CONFIG_MMU volatile xmAddress_t ptdL1 define ARCH PTDL1 REG ptdL1 volatile xm u32 t faultStatusReg volatile xm u32 t faultAddressReg endif E Listing 6 9 core include arch guest h signature Signature to identity this data structure as a PIT xmAbiVersion The ABI version of the currently running XtratuM This value is filled by the running XtratuM xmApiVersion The API version of the currently running XtratuM This value is filled by the running XtratuM resetCounter A counter of the number of partition resets This counter is incremented when the partition
63. 50088 C8 ees rea 3 9 de EE EERE RR AAA 72 5 12 Clock and timer services e ook X GUN Sex EL NUR LEGUNTUR AORTA RS s 73 5 12 1 Execution time clock su eue e aces dedo mE Ek e EORR RU TR ee 73 5 13 Processor management oe 33 93 4 6009 SAGARA a E RE RR eR 73 5 13 1 Managing stack context 2 0 4 2 3 299 caos eee 9x aa 74 DNE abren A PP a A ee ee ee Bee be a E a 74 Dell Fracednessdees ece eos eonim UR e oe OR eR RU e x x Lu ar ue rue a ee ORE S 74 5 142 Reading traces oa ope td TERRES EE EEE Se FUR ROO Ee 76 5 14 3 Configuration lt lt mde oe deo ok wo oor rob id hehe eed s 76 5 15 System and partition Status soa ox a oom eae XLRUR CH US s 77 5 16 Memory management 223oocoec noo e a RARO 78 5 16 L Configuration o sw Ga es CK Row eai NUR R s e EO A eA S 78 5 17 Releasing the processor 59e RR 45 X EUER UR A E S 80 5 18 Partition customisation files o lt lt lt er reres oo e e Ewa 80 5 19 Assembly programming s ssa e Kor AN ee Pe Ge Ap PO a 81 519 1 The object interfac lu og e Roe a ee dds aaa 82 xm 4 usermanual 047d Ah iram uPA Printed November 26 2012 viii 134 Contents 9 20 Manpages Summary 42 8 a He wow A UY Ue a ee a 84 6 Binary Interfaces 87 6 1 Data representations oe xp bob bob ex X 543 4 34 x4x4 add 87 6 2 Hypetcall mechanism es cem Reo o ook RUE RR RR RUE WO S a A 88 6 3 Executable formats ovetvieW oz o o OR os e e ee ROO 89 6 4 Partition ELE format iuc ke oe x oo a ee a ER Serer 90
64. 6 134 Chapter 6 Binary Interfaces 6 6 Container format A container is a file which contains a set of XEF files The tool xmpack manages container files see chapter 9 A component is an executable binary hypervisor or partition jointly with associated data configura tion or customization file The XtratuM component contains the files xm core bin and XM CT bin A partition component is formed by the partition binary file and zero or more customization files XtratuM is not a boot loader There shall be an external utility the resident software or boot loader which is in charge of coping the code and data of XtratuM and the partition from a permanent memory into the RAM Therefore the the container file is not managed by XtratuM but by the resident software see chapter 7 Note also that the container does not have information regarding where the components shall be loaded into RAM memory This information is contained in the header of the binary image of each component The container file is like a packed filesystem which contains the file metadata name of the files and the content of each file Also the file which contains the executable image and the customisation data of each partition is specified The container holds the following elements 1 The header xmefContainerHdr structure A data structure which holds pointers in the form of Offsets and the sizes to the remaining sections of the file 2 The component tab
65. 6 characters is a fair number GCoverage support Experimental Enable UART support If enabled XtratuM will use the UART to output console messages otherwise the UART can be used by a partition Enable XM partition status accounting Enable this option to collect statistical information of Xtra tuM itself and partitions Note that this feature increases the overhead of most of the XtratuM operations 8 2 Resident software source code configuration menuconfig The resident software RSW configuration parameters are hard coded in the source code in order to generate a self contained stand alone executable code After the configuration of the XtratuM source code the make menuconfig shows the RWS config uration menu The selected configuration is stored in the file user bootloaders rsw config The following parameters can be configured Parameter Type Default value RSW memory layout Container physical location address hex 0x4000 Read only section addresses hex 0x40200000 Read write section addresses hex 0x40090000 CPU frequency KHz int 50000 Enable UART support choice UART1 UART2 Enable UART flow control bool n UART baud rate int 115200 Stack size KB int 8 Stand alone version bool no Load container at a fixed address bool y Stack size KB xm 4 usermanual 047d J hie adu Printed November 26 2012 3055 3060 3065 3070 3075 3080 3085 3090 3095 3100
66. 7 start 130ms duration 10ms partitionId 3 lt Slot id 8 start 140ms duration 30ms partitionId 2 gt Slot id 9 start 170ms duration 10ms partitionId 3 lt Slot id 10 start 180ms duration 20ms partitionId 4 gt Plan lt CyclicPlanTable gt lt Processor gt lt Processor gt Hypervisor P56 Figure 2 11 Scheduling example One important aspect in the design of the XtratuM hypervisor scheduler is the consideration of the overhead caused by the partition s context switch In the document XtratuM Scheduling analysis it is provided a deep analysis of the scheduling aspects related to the partition context switch and its implications 2 8 1 Multiple scheduling plans In some cases a single scheduling plan may be too restrictive For example e Depending on the guest operating system the initialisation can require a certain amount of time Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 2 8 Partition scheduling 17 134 and can vary significantly If there is a single plan the initialisation of each partition can require ss a different number of slots due to the fact that the slot duration has been designed considering the operational mode This implies that a partition can be executing operational work whereas others are still initialising its data e The system can require to execute some maintenance
67. CPU services 4 sse se o ERG A EEE ES 2 6 System partitions uoo Oe bb e eee SH SORRY a ee SS Bae ed dE s 2 7 Namesandidentiflers 00m ss ser Ro eee ee le BH HH HL 2 7 1 Virtual CPU identifiers xa iii dada aria aa RR 8 2 8 Partition scheduling Lucinda bobs ae PRE OEE Roe eR S Som e ROUX 2 8 1 Multiple scheduling plans 405064 44 08 Bega whee memo ME EES 2 8 2 Switching scheduling plans s ss x 9k ela Roo Ry E GUESS S 2 9 XtratuM Multicore Scheduling os o ww XL RA ES x 2 91 Scheduling units 4529 539 bee e oe LY dees AA 2 90 2 Scheduling policies ke AE READER a RR WO E DES ES 2 9 3 Cyclicscheduling aii REX EU VG UE S x 2 9 4 Fixed priority scheduling 22e 2 10 Memory management sos se s x7 eee se Ee ee we ee ae ew 2 11 TO MIMU SUpport r re ee si 03 A ae eed okt E URS es ae Vii v 134 vi 134 Contents 2 12 Inter partition communications IPC ss soe seco xU 2254 eb eee es 25 2 12 Multicore implications 06 4044545 a aS See ees 26 2 13 Health monitor HM a ww diosa RO ee ewe Rar tea ee ow ow a 26 213 1 HM Events aia qe hed od dp bibs aks ALS Ug e de der oe dede Ba eh we 28 2 13 2 AM ACHOMS i r a a ER dex qe ee SUPCR ELA E OR e oe a 29 2 13 3 HM Configuration i ii a BRR a eom om ep ROUES 29 2 13 4 HM notifications i ss sog woe dom to e eoe Ro a ee ea dee DR Greene 30 2 13 5 Multicor implications 5 25244 oko a 31 2 14 Access tO EVICES isse sepe ues R RR Y RO ARD AO ae ara 31 2 15 Traps int
68. DRAM memory in 1 bank allocated 2048 K ROM memory icache 1 4 kbytes 16 bytes line 4 kbytes total dcache 1 4 kbytes 16 bytes line 4 kbytes total tsim gt load resident sw section text addr 0x40200000 size 14340 bytes section rodata addr 0x40203808 size 790 bytes section container addr 0x40203b20 size 48888 bytes section got addr Ox4020fa18 size 8 bytes section eh frame addr 0x4020fa20 size 64 bytes read 38 symbols tsim gt go resuming at 0x40200d24 XM Hypervisor 3 1 r2 Detected 50 0MHz processor gt gt HWClocks LEON clock 1000Khz CPU 0 HwTimer LEON timer 1000Khz Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 4 6 XtratuM directory tree 51 134 CPU 0 sched using cyclic scheduler 2 Partition s created PO Partitioni 0 flags SYSTEM BOOT 0x40080000 0x40080000 0x400fffff P1 Partition2 1 flags SYSTEM BOOT 0x40100000 0x40100000 0x4017ffff Jumping to partition at 0x40082000 Jumping to partition at 0x40102000 I am Partition2 Hello World I am CPU 0 HYPERCALL 0x1 Halted Partitioni Hello World CPU 0 HYPERCALL 0xO Halted 4 6 XtratuM directory tree Next list shows the installed tree directory for XtratuM and XAL home xmuser xmEnvs xm4 xal bin common include arch lib xm bin include arch xm_inc arch objects lib bootloaders rsw Sparcv8 Listing 4 2 Installation
69. EON2 3 processors the program counter register is initialized with the address 0x00000000 In LEON4 processor the program counter register is initialised with a specified address The resident software has to be allocated from this address The resident software is in charge of loading into memory XtratuM its configuration file XM CT and any partition jointly with its customisation file as found in the container The information hold by the XM_CT file is used to load any partition image Additionally the resident software informs XtratuM which partitions have to be booted When the control is transfered from the resident software to XtratuM the setup function start the boot operation Figure 7 2 sketches the booting sequences CPUO CPU1 CPU2 CPU3 Initialisation Local l A Local initialisation Initialisation Local i m Initialisation MN o eee gue piuegeiese em Synchronisation ef ES Plan 0 Plan 0 Plan 0 Plan 0 Execution Execution Execution Execution Processor 0 Processor 1 Processor 2 Processor 3 Figure 7 2 Booting process After the hard reset the CPUO is started and a XtratuM thread is executed This CPUO thread performs a global initialisation and starts the execution of other CPUs by providing the entry point and stack area Each started CPU executes a XtratuM thread performing a local initialisation of the internal local data structures All CPU threads are synchonised in a specific point in order to guarantee
70. FIRST define XM VT EXT IPVI7 31 4XM VT EXT FIRST XtratuM implements a homogeneous policy for access on a per partition basis to the resources It implies that the internal subjects VCPUs or threads in a partition have the same requirements for access to the resources specified in the configuration file Is responsability of the guest OS to define another policy For instance a guest OS could define a Least Provilege Abstraction which assumes that the internal subjects VCPU or threads in a partition have heterogeneous requirements for access to the resources In this case the guest OS could restrict the defined operations to some internal subjects 2 13 Health monitor HM The health monitor is the part of XtratuM that detects and reacts to anomalous events or states The purpose of the HM is to discover the errors at an early stage and try to solve or confine the faulting subsystem in order to avoid a failure or reduce the possible consequences It is important to clearly understand the difference between 1 an incorrect operation instruction function application peripheral etc which is handled by the normal control flow of the software and 2 an incorrect behaviour which affects the normal flow of control in a way not considered by the developer or which can not be handled in the current scope An example of the first kind of errors is the malloc function returns a null pointer because there is not memory en
71. From this point of view a trap generated by a VCPU will e will raise a HM event e XtratuM will handle the event as execute the action defined in the configuration file which is common to all the internal VCPUs e In the case of a propagation action XtratuM identifies the VCPU that generated the trap and delivers to the causing VCPU the associated virtual interrupt e Inthe case of a partition action halt reset etc the action is aplied independently of the causing VCPU 2 14 Access to devices Partition Partition Partition Partition XM sparcv8 HWoutport XM sparcv8 HWinport XM_spardv8_HWoutport XM_sparqv8_HWinport XtratuM XtratuM Device registers Device registers Device Device a Partition using exclusively a device b I O Server partition A partition using exclusively a device peripheral can access the device through the device driver implemented in the partition figure 2 15a The partition is in charge of handling properly the device The configuration file has to specify the I O ports and the interrupt lines that will be used by each partition Two partitions cannot use the same interrupt line XtratuM provides a fine grain access control to I O ports so that several partitions can use read and write different bits of the the same I O port Also it is possible to define a range of valid values that can be written in a
72. L field of the PSR enable interrupts XM sparc flush regwim Save the contents of the register window XM sparc flush tlb The TLB cache is invalidated Assembly hyper call XM sparc get psr Get the ICC and PIL flags from the virtual PSR processor register XM sparc inport Read from a hardware I O port xm 4 usermanual 047d hie acu Printed November 26 2012 86 134 Chapter 5 Partition Programming XM_sparc_outport Write in a hardware I O port XM sparc set pil Set the PIL field of the PSR disallow interrupts XM_sparc_set_psr Set the ICC and PIL flags on the virtual PSR pro cessor register XM_sparcv8_ctrl winflow Check manually the state of register windows Obsolete XM_hm open DEPRECATED XM_mask_irq Obsoleted by XM_set_irqmask XM_request_irq Request to receive an interrupt XM set page type Changes the type of a physical page to a specific page type XM sparc flush cache Flush data cache Assembly hypercall XM update page32 Updates an entry in the page table Printed November 26 2012 hie adu xm 4 usermanual 047d Volume 2 XtratuM User Manual Chapter 6 Binary Interfaces This section covers the data types and the format of the files and data structures used by XtratuM Only the first section describing the data types is needed for a partition developer The remaining 240
73. PU to support mul ticore applications Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 2 1 Multicore architecture 7 134 2 1 Multicore architecture This section introduces the multicore approach for XtratuM The following notation is used CPU It identifies the real CPUs in the hardware virtual CPU It represents the virtualisation of the CPU provided by XtratuM to the partitions Partition It corresponds to the execution environment where the application are executed The hypervisor offers a set of virtual CPUs that corresponds to the real CPUs are available in the platform The goal of the hypervisor is to provide virtual CPUs to partitions in the same way that a multicore OS see the CPUs if not virtualisation layer is present From this point of view the virtualisation layer will e Virtualise all the CPUs to the partitions offering them as virtual CPUs e Initialise the execution of each CPU at the initialisation phase e After the initialiation phase start the execution of the plan 0 e Initialise the virtual CPU identified as vcpuO of each partition From the point of view of the partition e It can be mono or multicore e In monocore partitions the vcpu0 is used It is started by the hypervisor The vcpuO can be allocated by the integrator XM CF configuration file in any of the real CPUs e In multicore partitions only vcpuo is initialised by the hypervisor The vcpuO code has to in
74. Plans involving several processors have to define the same Major Frame MAF An example of the scheduling definition is listed below 465 lt ProcessorTable gt lt Processor id 0 frequency 50Mhz gt lt CyclicPlanTable gt lt Plan id 0 majorFrame 1s gt Slot id 0 start Oms duration 500ms partitionId 470 0 vCpuld 0 gt Slot id 1 start 500ms duration 500ms partitionId 1 vCpuld 0 gt lt Plan gt lt Plan id 1 majorFrame 600ms gt 475 Slot id 0 start Oms duration 410ms partitionId 0 vCpuld 0 gt Slot id 1 start 450ms duration 150ms partitionId 1 vCpuld 0 gt lt Plan gt 480 lt CyclicPlanTable gt lt Processor gt lt Processor id 1 frequency 50Mhz gt lt CyclicPlanTable gt lt Plan id 0 majorFrame 1s gt 485 Slot id 0 start Oms duration 500ms partitionId 2 vCpuld 0 gt lt Slot id 0 start 500ms duration 500ms partitionId 3 vCpuld 0 gt lt Plan gt 490 lt Plan id 1 majorFrame 600ms gt Slot id 0 start Oms duration 400ms partitionId 2 vCpuld 0 gt Slot id 1 start 400ms duration 200ms partitionId 3 vCpuld 0 gt 495 lt Plan gt lt CyclicPlanTable gt lt Processor gt lt Processor id 2 frequency 50Mhz gt lt CyclicPlanTable gt 500 lt Plan id 1 majorFrame 600ms gt Slot id 0 start Oms duration 200ms partitionId 0 vCpuld 1 gt lt Slot id 1 start 200ms durat
75. R 28 define APBUARTO_NR 29 define APBUART1_NR 30 define GRIOMMU_NR 31 oP WNP 2 16 Traces XtratuM provides a mechanism to store and retrieve the traces generated by partitions and XtratuM itself Traces can be used for debugging during the development phase of the application but also to log relevant events during the production phase In order to enforce resource isolation each partition as well as XtratuM has a dedicated trace log stream to store its own trace messages which is specified in the device attribute of the Trace element Trace is an optional element of XMHypervisor and Partition elements The hypercall to write a trace message has a parameter bitmask used to select the traces messages are stored in the log stream The integrator can select which trace messages are actually stored in the log stream with the Trace bitmask attribute If both the corresponding bit set in the value configured in the Partition Trace bitmask and the value of the bitmask parameter of the XM trace event hypercall are set then the event is stored otherwise it is discarded Figure 2 18 sketches the configuration of the traces In the example the traces generated by partition 1 will be stored in the device MemDisk0 which is defined in the Devices section as a memory block device Only those traces whose least significant bit is set in the bitmask parameter will be recorded 2 16 1 Multicore implications No implications
76. R is 7 the page size is 4KB and there will be two tables of 2 MBytes each one Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 2 12 Inter partition communications IPC 25 134 2 12 Inter partition communications IPC Inter partition communications are communications between two partitions XtratuM implements a message passing model which highly resembles the one defined in the ARINC 653 standard A message is a variable block of data A message is sent from a source partition to one or more destination partitions The data of a message is transparent to the message passing system A communication channel is the logical path between one source and one or more destinations Partitions can access to channels through access points named ports The hypervisor is responsible for encapsulating and transporting messages that have to arrive to the destination s unchanged At the partition level messages are atomic entities i e either the whole message is received or nothing is received Partition developers are responsible for agreeing on the format data types endianess padding etc Channels ports maximum message sizes and maximum number of messages queuing ports are entirely defined in the configuration files see section 8 XtratuM provides two basic transfer modes sampling and queuing Sampling port It provides support for broadcast multicast and unicast messages No queuing is supported in this mode A m
77. SELF void attribute weak HwIrgHandler xm s32 t trapNr xprintf hwIrq 0x x 4d n trapNr trapNr XM halt partition XM PARTITION SELF Listing 5 7 user examples common traps c Note that the C trap handler functions are defined as weak Therefore if these symbols are defined elsewhere the new declaration will replace this one The linker script that arranges all the ELF sections is Printed November 26 2012 A ir amu PA xm 4 usermanual 047d 5 4 The Hello World example 61 134 OUTPUT_FORMAT binary OUTPUT FORMAT elf32 sparc elf32 sparc elf32 sparc OUTPUT ARCH sparc ENTRY start SECTIONS 1 text ALIGN 8 ALIGN 4K _Sguest text init text rodata ALIGN 8 4 rodata rodata rodata data ALIGN 8 _sdata data edata bss 1 bss noinit _sbss COMMON bss _ebss _eguest DISCARD note comment Listing 5 8 user examples sparcv8 loader lds The section text ini which contains the headers is located at the beginning of the file as defined by the ABI The section xm ct1 which contains the PCT table is located at the start of the bss section to avoid being zeroed at the startup of the partition The contents of these tables has been initialized by XtratuM before starting the partition The symbols sguest and eguest
78. T does not change In this case bytes are sent with a timeout XtratuM load address Physical RAM address where XtratuM shall be copied This value shall be the same than the one specified in the XM CF file Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 3050 8 2 Resident software source code configuration menuconfig 107 134 Console initial buffer length Size of the internal console buffer This buffer it used to store the mes sages written by XtratuM or by partitions using the XM console write hypercall The larger the buffer the lower the chances of loosing data Enable voluntary preemption support Enable experimental features Enable this option to be able to select experimental ones This option does not do anything on itself just shows the options marked as experimental Kernel stack size KB Size of the stack allocated to each partition It is the stack used by XtratuM when attending the partition hypercalls Do not change reduce this value unless you know what you are doing Debug and profiling support XtratuM is compiled with debugging information gcc flag ggdb and assert code is included This option should be used only during the development of the XtratuM hypervisor Maximum identifier length B The maximum string length including the terminating 0x0 char acter of the names partition name port name plan etc Since the names are only used for debugging 1
79. TSO Total Store Ordering mode This is the standard SPARC v8 working mode If changed to PSO Partial Store Ordering mode then random errors will happen Memory allocation e Care shall be taken to avoid overlapping the memory allocated to each partition e If MMU in not used then the partition code shall be linked to work on the allocated memory areas If the memory allocated in the XM CF file is changed then the linker script of the affected partition shall be updated accordingly Reserved names The prefix xm both in upper and lower case is reserved for XtratuM identifiers Stack management XtratuM manages automatically the register window of the partitions The par tition code is responsible of initialising the stack pointer to a valid memory area and reserve enough space to accommodate all the data that will be stored in the stack Otherwise a stack overflow may occur Data Alignment By default all data structures passed to or shared with XtratuM shall by aligned to 8 bytes Units definition and abbreviations KB KByte or Kbyte is equal to 1024 2 bytes Kb Kbit is equal to 1024 219 bits MB MByte or Mbyte is equal to 1048576 1024 1024 2 bytes Mb Mbit is equal to 1048576 1024 1024 220 bits KHz Kilo Hertz is equal to 1000 Hertzs MHz Mega Hertz is equal to 1000 000 Hertzs XtratuM memory footprint XtratuM does not use dynamic memory allocation T
80. W define TRACE OPCODE CODE BIT 1 256 vcpus 256 partitions xmTime t timestamp define XMTRACE PAYLOAD LENGTH __PACKED Listing 5 26 core include objects trace h 6 4 xmWord t payload XMTRACE_PAYLOAD_LENGTH typedef struct xmTraceEvent xmTraceEvent t define TRACE OPCODE CRIT MASK Ox7 TRACE OPCODE CRIT BIT define TRACE_OPCODE_SYS_MASK Ox1 TRACE OPCODE SYS BIT define TRACE OPCODE CODE MASK Oxffff TRACE OPCODE CODE BIT define TRACE OPCODE VCPUID MASK Oxff TRACE OPCODE VCPUID BIT define TRACE OPCODE VCPUID BIT 8 define TRACE OPCODE PARTID MASK Oxff TRACE OPCODE PARTID BIT define TRACE OPCODE PARTID BIT O partitionId Identify the partition that issued the trace event This field is automatically filled by XtratuM The value XM HYPERVISOR ID is used to identify XtratuM traces moduleId For the traces issued by the partitions this field is user defined For the traces issued by XtratuM this field identifies an internal subsystem TRACE MODULE HYPERVISUR Traces related to XtratuM core events TRACE MODULE PARTITION Traces related to partition operation TRACE MODULE SCHED Traces concerning scheduling code This field is user defined for the traces issued by the partitions For the traces issued by XtratuM the values of the code field depends on the value of the moduleId If moduleId TRACE MODULE HYPERVISOR TRACE EV HYP HALT The hypervisor is
81. XAL a basic partition execution environment 4 2 1 XtratuM configuration With respect to the processor the following options have to be selected in the menuconfig e CPU LEON2 LEON3 LEON4 e Board type e Memory protection Scheme Enable SMP to enable disable the use of multicore The number of CPUs have to be defined Enable VMM update hypercalls Enable cache Enable cache snoop e Enable instruction burst fetch Flush cache after context switch 4 2 2 Resident software configuration It permits to specify the configuration parameters of the resident software as e Stack size KB e Stand alone version e Load container at a fixed address e RSW memory layout Container physical location address Read only section addresses and Read write section addresses Enable RSW output e Autodetect CPU frequency Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 4 3 Generating binary a distribution 47 134 4 2 3 Xtratum Abstraction Layer XAL configuration XAL is a minimal layer to give support to bare C applications The XAL configuration only specifies the stack size to be used by the bare C partition 4 2 4 Compiling XtratuM 1190 For running XtratuM in the simulator select the appropriate processor model from the menuconfig menus 5 Compile XtratuM sources make gt Configuring and building the XtratuM hypervisor gt Building XM Core kernel sparcv8 kernel mmu kernel kl
82. XtratuM Hypervisor for LEON4 Volume 2 XtratuM User Manual Miguel Masmano Alfons Crespo Javier Coronel November 2012 Reference xm 4 usermanual 047d ai2 This page is intentionally left blank iii 134 DOCUMENT CONTROL PAGE TITLE XtratuM Hypervisor for LEON4 Volume 2 XtratuM User Manual AUTHOR S Miguel Masmano Alfons Crespo Javier Coronel LAST PAGE NUMBER 134 VERSION OF SOURCE CODE XtratuM 4 for LEON3 REFERENCE ID xm 4 usermanual 047d SUMMARY This guide describes the fundamental concepts and the features provided by the API of the XtratuM hypervisor DISCLAIMER This documentation is currently under active development Therefore there are no explicit or implied warranties regarding any properties including but not limited to correctness and fitness for purpose Contributions to this documentation new material suggestions or corrections are welcome REFERENCING THIS DOCUMENT techreport xm 4 usermanual 047d title XtratuM Hypervisor for LEON4 Volume 2 XtratuM User Manual author Miguel Masmanoand Alfons Crespoand Javier Coronel institution Universidad Polit cnica de Valencia number xm 4 usermanual 0474d year November 2012 Copyright November 2012 Miguel Masmano Alfons Crespo Javier Coronel Permission is granted to copy distribute and or modify this document under the terms of the GNU Free Documentation License Version 1 2 or any later version p
83. a coherent initialisation before the scheduling plan execution xm 4 usermanual 047d A irem uPA Printed November 26 2012 2950 2955 2960 2965 2970 2975 2980 2985 2990 2995 3000 3005 102 134 Chapter 7 Booting The global initialisation consists on the following Initializes the internal console Initializes the interrupt controller Detects the processor frequency information extracted from the XML configuration file Initializes memory manager enabling XtratuM to keep track of the use of the physical memory Initializes hardware and virtual timers Initializes the scheduler Initializes the communication channels Wakes up other processors VD 0 Oo Ul KR 0 Nme Booting partitions are set in NORMAL state and non booting ones are set in HALT state n eo Opens the sync barrier 11 Finally the setup function calls the scheduler and becomes into the idle task Other processors processors perform the local initialisation 1 Sets up a valid virtual memory map 2 Initializes the timer 3 Waits in a sync barrier 4 Finally calls the scheduler and becomes into the idle task These scheme implies an important design aspect with respect to the monocore version of XtratuM The internal code of XtratuM is not a non preemptive code block like it is in the monocore version The multicore design is fully preemptive A set of low grain atomic sections have been defined in
84. about to halt TRACE EV HYP RESET The hypervisor is about to perform a software reset TRACE EV HYP AUDIT INIT The first message after the audit startup If moduleId TRACE MODULE PARTITION The field auditEvent partitionId has the identifier of the affected partition The recorded events are TRACE EV PART SUSPEND The affected partition has been suspended TRACE EV PART RESUME The affected partition has been resumed TRACE EV PART HALT The affected partition has been halted TRACE EV PART SHUTDOWN A shutdown extended interrupt has been delivered to the parti tion xm 4 usermanual 047d hie adu Printed November 26 2012 2105 2110 2115 2120 2125 2130 2135 2140 2145 2150 2155 2160 2165 2170 2175 2180 2185 76 134 Chapter 5 Partition Programming TRACE EV PART IDLE The affected partition has been set in idle state TRACE EV PART RESET The affected partition has been reset If moduleId TRACE MODULE SCHED The field auditEvent partitionId has the identifier of the partition that requested the plan switch or XM HYPERVISOR ID if the plan switch is the consequence of the XM HM AC SWITCH TO MAINTENANCE health monitoring action The field auditEvent newPlanId has the identifier of the new plan TRACE EV SCHED CHANGE REQ A plan switch has been requested TRACE EV SCHED CHANGE COMP A plan switch has been carried out criticality Determines the im
85. agement Currently only the tbr processor control register has been virtualised This register should be loaded with the hypercall XM write_register32 with the address of the partition trap table This operation xm 4 usermanual 047d A ic emu P Printed November 26 2012 2045 2050 2055 2060 2065 2070 2075 2080 2085 2090 2095 2100 74 134 Chapter 5 Partition Programming is usually done only once when the partition boots see listing 5 4 5 13 1 Managing stack context The SPARC v8 architecture following the RISC ideas tries to simplify the complexity of the processor by moving complex management tasks to the compiler or the operating system One of the most particular features of SPARC v8 is the register window concept and how it should be managed Both register window overflow and underflow cause a trap This trap has to be managed in supervisor mode and with traps disabled bit ET in the psr register is unset to avoid overwriting valid registers It is not possible to emulate efficiently this behaviour XtratuM provides a transparent management of the stack Stack overflow and underflow is directly managed by XtratuM without the intervention of the partition The partition code shall load the stack register with valid values In order to implement a content switch inside a partition if the partition is a multi thread environ ment the function XM sparcv8 flush regwinO can be used to fl
86. ags system console Uart gt lt PhysicalMemoryAreas gt Area start 0x40100000 size 512KB flags gt lt PhysicalMemoryAreas gt lt TemporalRequirements duration 500ms period 500ms gt lt HwResources gt lt Interrupts lines 4 gt lt HwResources gt lt Partition gt lt PartitionTable gt lt lt track id hello xml sample gt gt lt SystemDescription gt Listing 5 11 XML configuration file example xm 4 usermanual 047d A ir emu PA Printed November 26 2012 1690 1695 1700 1705 1710 1715 1720 1725 1730 1735 1740 64 134 Chapter 5 Partition Programming In order to avoid inconsistences between the memory Area attribute of the configuration and the parameter passed to the linker the examples common xpath tool can be used from a Makefile to is extract the information from the configuration file cd user examples hello world common xpath c f xm cf sparcv8 xml SystemDescription PartitionTable Partition 1 PhysicalMemoryAreas Area 1 start 0x40050000 Listing 5 12 Using xpath to recover to memory area of the first partition The attribute SystemDescription PartitionTable Partition 1 PhysicalMemoryAreas Area 1 start is the xpath reference to the attribute which defines the first region of memory allocated to the first partition which in the example is the place where the partition will be loaded 5 4 1 Included headers
87. all start in Oxffffffff restriction imposed by LEON4 e The largest page size which better fits with the partition s area sizes e A new possible flag iommu has been added to PhysicalMemoryAreas Area flags in order to indicate that only the areas with this new flags shall be mapped in the io mmu Following is an example of use of the io mmu with two partitions lt xml version 1 0 gt lt SystemDescription xmlns http www xtratum org xm 3 x version 1 0 0 name Example_iommu gt lt HwDescription gt lt ProcessorTable gt lt Processor id 0 frequency 50Mhz gt lt Sched gt lt CyclicPlan gt Plan id 0 majorFrame 500ms gt Slot id 0 start Oms duration 250ms partitionId 0 gt Slot id 1 start 250ms duration 250ms partitionId 1 gt lt Plan gt lt CyclicPlan gt lt Sched gt lt Processor gt lt ProcessorTable gt lt Devices gt lt Uart id 0 baudRate 115200 name Uart gt lt Devices gt lt MemoryLayout gt lt Region type stram start 0x0 size 32MB gt lt Region type rom start 0x80000000 size 32MB lt MemoryLayout gt lt HwDescription gt lt XMHypervisor console Uart gt lt PhysicalMemoryAreas gt lt Area start 0x40000000 size 512KB flags uncacheable gt lt PhysicalMemoryAreas gt xm 4 usermanual 047d A ic emu PA Printed November 26 2012 530 535 540 545 550 555 560 565 570 575 580 585 590
88. alues in the port1 whereas partition 2 reads them include lt xm h gt include std_c h define PORT NAME porti define PORT SIZE 48 void PartitionMain partition entry point int counter 0 int portDEsc portDesc XM create sampling port PORT NAME PORT SIZE XM SOURCE PORT if portDesc lt 0 xprintf s cannot be created PORT NAME return xm 4 usermanual 047d AX ic amu P Printed November 26 2012 1850 1855 68 134 Chapter 5 Partition Programming while 1 counter if counter 1000 XM write sampling message portDesc counter sizeof counter Listing 5 17 Partition 1 include lt xm h gt include std_c h define PORT_NAME port2 define PORT_SIZE 48 void PartitionMain partition entry point int value int previous 0 int portDesc xm_u32_t flags portDesc XM_create_sampling_port PORT_NAME PORT_SIZE XM DESTINATION PORT if portDesc lt 0 xprintf s cannot be created PORT NAME return while 1 XM read sampling message portDesc amp value sizeof value amp flags if value previous 1 xprintf d n value previous value Listing 5 18 Partition 2 5 9 1 Message notification When a message is sent into a queuing port or written into a sampling port XtratuM triggers the extended interrupt XM VT EXT OBJDESC By default this interrupt is masked when the partition boots 5
89. andatory to use this tool to deploy the application hypervisor and the partitions in the target machine The following checks are done e The binary image of the partitions fits into the allocated memory as defined in the XM CF e The size of the customisation files fits into the area reserved by each partition e The memory allocated to XtratuM is big enough to hold the XtratuM image plus the configuration file build A new container is created Two kind of components can be defined h to create an H ypervisor component The hypervisor entry is composed of the name of the XtratuM xef file and the binary config uration file the result of processing the XM CF file p to create a P artition The partition entries are composed of The id of the partition as specified in the XM CF file Note that this is the mechanism to bind the configuration description with the actual image of the partition The part file which shall contains the executable image And zero or more custom files There shall be the same number of customisation files than that specified in the field noCustomFiles of the xmImageHdr structure The elements that are part of each component are separated by By default xmpack stores the files sequentially in the container If the offset parameter is specified then the file is placed at the given offset The offset is defined with respect to the start of the container The specified offset shall not overl
90. ap with existing data The remaining files of the container will be placed after the end of this file list Shows the contents components and the files of each component of a container If the option c is given the blocks allocated to each file are also shown xm 4 usermanual 047d Heca P Printed November 26 2012 3395 3400 3405 3410 3415 3420 3425 3430 122 134 Chapter 9 Tools USAGE EXAMPLES A new container with one hypervisor and one booting partition The hypervisor container has two files the hypervisor binary and the configuration table xmpack build build h core xm core bin xm ct bin p partitioni bin o container x The same example but the second container has now two files the partition image and a customisa tion file xmpack xmpack build h core xm core bin xm cf bin p partitioni bin pi cfg p partition2 bin p2 cfg container bin 3445 List the contents of the container xmpack list container bin Package file container bin version 1 0 0 gt lt XMHypervisor file core xm core bin fileSize 97188 offset 0x0 size 97192 gt lt Module file xm_cf bin size 8976 gt 3450 lt XMHypervisor gt Partition file partitioni bin fileSize 29996 offset 0x19eb8 size 30000 gt lt Module file p1 cfg size 16 gt lt Partition gt lt Partition file partition2 bin fileSize 30292 offset 0x213f8 size 30296 gt 3455 Module file
91. apter 2 XtratuM Architecture Note that the same HM event can be binded with different recovery actions in each partition HM table and in the XtratuM HM table The HM system can be configured to send a HM message after the execution of the HM action It is possible to select whether a HM event is logged or not See the chapter 8 2 13 4 HM notification The log events generated by the HM system those events that are configured to generate a log are stored in the device configured in the XM_CF configuration file In the case that the logs are stored in a log stream then they can be retrieved by system partitions by using the XM_hm_X services Health monitoring log messages are fixed length messages defined as follows struct xmHmLog define XM_HMLOG_SIGNATURE Oxfecf xm_ui6_t signature xm u16 t checksum xm u32 t opCode define HMLOG OPCODE EVENT MASK Ox1fff HMLOG OPCODE EVENT BIT define HMLOG OPCODE EVENT BIT 19 Bits 18 and 17 free define HMLOG_OPCODE_SYS_MASK Ox1 HMLOG OPCODE SYS BIT define HMLOG_OPCODE_SYS_BIT 16 define HMLOG_OPCODE_VALID_CPUCTXT_MASK Oxi HMLOG_OPCODE_VALID_CPUCTXT_BIT define HMLOG_OPCODE_VALID_CPUCTXT_BIT 15 define HMLOG_OPCODE_MODID_MASK 0x7f lt lt HMLOG_OPCODE_MODID_BIT define HMLOG_OPCODE_MODID_BIT 8 define HMLOG_OPCODE_PARTID_MASK Oxff HMLOG OPCODE PARTID BIT define HMLOG OPCODE PARTID BIT O xm u32 t seq xmTime t timestamp unio
92. artitions These functions are coded in examples common std c c Some of these is functions are strlen print str xprintf O which are similar to the functions provided by a stdio The use of xprintf is illustrated in the next example include lt xm h gt include std_c h header of the std_c h void PartitionMain partition entry point int counter 0 while 1 counter if counter 1000 xprintf d n counter Listing 5 16 Ported dummy code 1 xprintf O performs some format management in the function parameters and invokes the hypercall which stores it in a kernel buffer This buffer can be sent to the serial output or other device 1840 5 9 Inter partition communication Partitions can send receive messages to from other partitions The basic mechanisms provided are sampling and queuing ports The use of sampling ports is detailed in this section Ports need to be defined in the system configuration file XM_CF Source and destination ports are connected through channels Assuming that ports and channel linking the ports are defined in the configuration file the next partition code shows how to use it 1845 XM_create_sampling port and XM create queuing port O hypercalls return object descriptors An object descriptor is an integer where the 16 least significant bits are a unique id of the port and the upper bits are reserved for internal use In this example partition 1 writes v
93. as the attribute lines which is a list of the interrupt number in the range 0 to 16 allocated to the partition 8 3 7 The SystemDescription Channels element This is an optional element with no attributes and which contains a list of channel elements There are two types of channels SamplingChannel Shall contain one Source element and one or more Destination elements It has the following attributes QmaxMessageLength Required The maximum message size that can be stored on this chan nel refreshPeriod Optional The duration of validity of a written message When a message is read after this period the validity flag will be false QueuingChannel Shall contain one Source element and one Destination element It has the following attributes CmaxMessageLength Required The maximum message size that can be stored on this chan nel xm 4 usermanual 047d A ic emu PA Printed November 26 2012 3235 3240 3245 3250 3255 116 134 Chapter 8 Configuration maxNoMessages Required The maximum number of messages that will be stored in the 3260 channel Note The QueuingChannel 0validPeriod attribute has been removed with respect to XtratuM 2 2 x versions The arguments maxNoMsgs and maxMsgSize of the hypercalls XM create queuing port O and XM crea te sampling port O shall match the values of the attributes maxNoMessages and maxNoMessages The XML schema which defines the config
94. at shall be loaded into RAM The tool xmeformat converts from ELF or plain data files to XEF format see chapter 9 struct xefHdr define XEF SIGNATURE 0x24584546 xm u32 t signature xm u32 t version define XEF DIGEST 0x1 define XEF COMPRESSED 0x4 define XEF RELOCATABLE 0x10 define XEF TYPE MASK OxcO define XEF TYPE HYPERVISOR 0x00 define XEF TYPE PARTITION 0x40 define XEF TYPE CUSTOMFILE 0x80 define XEF ARCH SPARCv8 0x400 define XEF ARCH MASK Oxff00 xm u32 t flags xm u8 t digest XM DIGEST BYTES xm u8 t payload XM PAYLOAD BYTES xmSize t fileSize xmAddress_t segmentTabOffset xm s32 t noSegments xmAddress t customFileTabOffset xm s32 t noCustomFiles xmAddress_t imageDffset xmSize_t imageLength xmSize t deflatedImageLength xmAddress t pageTable xmSize t pageTableSize xmAddress t xmImageHdr xmAddress t entryPoint __PACKED Listing 6 10 core include xmef h hie adu xm 4 usermanual 047d Printed November 26 2012 2705 2710 2715 2720 2725 2730 2735 2740 2745 2750 2755 2760 2765 2770 2775 2780 94 134 Chapter 6 Binary Interfaces signature A 4 bytes word to identify the file as a XEF format version Version of the XEF format flags Bitmap of features present in the XEF image It is a 4 bytes word The existing flags are XEF DIGEST If set then the digest field is valid and shall be used to check the integrity of t
95. ation process which involves several steps to finally obtain a binary code which has to be loaded in the embedded system The goal of chapter 5 is to provide a view of the API provided by XtratuM to develop applications to be executed as partitions The chapter puts more emphasis in the development of bare applications than applications running on a real time operating system Chapter 6 deals with the concrete structure and internal formats of the different components involved in the system development system image partition format partition tables The chapter ends with the description of the hypercall mechanism Chapter 7 and 8 detail the booting process and the configuration elements of the system respectively Finally chapter 8 provides information of the preliminar tools developed to analyse system configura tion schemas XML format and generate the appropriate internal structures to configure XtratuM for a specific payload 1 1 History The term XtratuM derives from the word stratum In geology and related fields it means Layer of rock or soil with internally consistent characteristics that distinguishes it from con tiguous layers In order to stress the tight relation with Linux and the open source the S was replaced by X XtratuM would be the first layer of software the one closer to the hardware which provides a rock solid basis for the rest of the system First XtratuM version proof of concept Linux
96. b start group sparc linux gcc print libgcc file name 1xm lxef lexamples end group Ttext 0x40080000 gt Convert the partition ELF to the XEF format xmeformat build c simple o simple xef Compile the configuration file xmcparser o xm cf bin xmc xm cf sparcv8 xml 4 Convert the configuration file to the XEF format xmeformat build c m xm cf bin xmc o xm cf xef xmc gt Pack all the XEF files of the system into a single container xmpack build h xm lib xm core xef xm cf xef xmc p O simple xef container bin Build the final bootable file with the resident sw and the container rswbuild container bin resident sw Listing 5 10 Example of a compilation sequence The partition code shall be compiled with with the flags nostdlib and nostdinc to avoid using host specific facilities which are not provided by XtratuM The bindings between assembly and C are done considering that not frame pointer is used fomit frame pointer 1 9 All the object files traps o boot o and simple o are linked together and the text section is positioned in the direction 0x40050000 This address shall be the same than the one declared in the XM CF file lt SystemDescription xmlns http www xtratum org xm 3 x version 1 0 0 1685 name example gt lt HwDescription gt lt MemoryLayout gt Region type rom start 0x0 size 4MB gt lt Region type stram start 0x40000000
97. bss 03 text rodata 04 Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 8 3 Hypervisor configuration file XM CF 109 134 poo c A A a 1 Listing 8 1 Resident software memory footprint example The next table summarises the size of the resident software for different configurations Board type Debug messages size LEON3 disabled 0x01f44 LEON3 enabled Ox01ff4 LEON2 disabled 0x01f44 LEON2 enabled OxO1ff4 8 3 Hypervisor configuration file XM CF The XM CF file defines the system resources and how they are allocated to each partition 3105 For an exact specification of the syntax mandatory optional elements and attributed and how many times an element can appear the reader is referred to the XML schema definition in the Appendix A 8 3 1 Data representation and XPath syntax When representing physical units the following syntax shall be used in the XML file Time Pattern 0 9 0 9 mu sS Examples of valid times 3110 9s nine seconds 10ms ten milliseconds 0 5ms zero point five milliseconds 500us five hundred microseconds 0 5ms Size Pattern 0 9 0 9 4 C MK 7B 3115 Examples of valid sizes 90B ninety bytes 50KB fifty Kilo bytes 50 1024 bytes 2MB two mega bytes 2 1024 1024 bytes 2 5KB two point five kilo bytes 2560B 3120 It is advised not to use the decimal point on sizes Frequency Pattern 0 9 0 9 4
98. c set psrO hypercalls mimicking the way the underneath architec ie ture wi orks An example of interrupt management at partition level is include std c h include lt xm h gt void IrqHandler int irqnr i xprintf Hardware irq d nHalting Mn irqnr XM halt partition XM PARTITION SELF void ExtIrqHandler int irqnr xprintf Extended irq d nHalting Mn irqnr 1985 1990 xm 4 usermanual 047d hie adu Printed November 26 2012 1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 72 134 Chapter 5 Partition Programming XM halt partition XM PARTITION SELF void PartitionMain void xm_s32_t err xmTime_t oneshoot XM_get_time XM_HW_CLOCK amp oneshoot oneshoot xmTime_t 100000 1 second XM_set_timer XM_HW_CLOCK oneshoot 0 XM enable irqsO err XM unmask irq XM VT EXT HW TIMER xprintf Unmask extirq XM VT EXT HW TIMER d n err err XM unmask irq XM VT HW IO IRQO TRAP NR xprintf Unmask hwirg 4 d n err while 1 4 XM idle self Listing 5 23 Virtual trap management example 5 11 3 Exceptions Exceptions are the traps triggered by the processor in response to an internal condition Some ex ceptions are caused by normal operation of the processor e g register window over underflow but others are caused by abnormal situations e g invalid instruction Error related exception traps
99. cessor traps XtratuM provides a para virtualized trap system called virtual traps XtratuM defines 256 32 traps The first 256 traps correspond directly with to the hardware traps The last 32 ones are defined by XtratuM Specifically for LEON4 16 extended hardware interrupts are defined These extended hardware in terrupts are allocated from the 0x30 to Ox3F in the trap partition list Next is shown the list of the mapping of extended hardware interrupts to traps xm 4 usermanual 047d A ir emu PA Printed November 26 2012 1860 1865 1870 1875 1880 1885 1890 A 1895 1900 1905 1910 1915 1920 1925 1930 1935 70 134 Chapter 5 Partition Programming define EXT HW INTERRUPT LEVEL 16 BASE TRAP EXT HW INTERRUPTS 15 GRIOMMU 31 define EXT HW INTERRUPT LEVEL 15 BASE TRAP EXT HW INTERRUPTS 14 APBUART1 30 define EXT HW INTERRUPT LEVEL 14 BASE TRAP EXT HW INTERRUPTS 13 APBUARTO 29 define EXT HW INTERRUPT LEVEL 13 BASE TRAP EXT HW INTERRUPTS 12 MEMSCRUBL2CACHE 28 define EXT HW INTERRUPT LEVEL 12 BASE TRAP EXT HW INTERRUPTS 11 AHBSTAT 27 define EXT HW INTERRUPT LEVEL 11 BASE TRAP EXT HW INTERRUPTS 10 GR1553B 26 define EXT HW INTERRUPT LEVEL 10 BASE TRAP EXT HW INTERRUPTS 9 GRETH_GBIT1 25 define EXT HW INTERRUPT LEVEL 9 BASE TRAP EXT HW INTERRUPTS 8 GRETH GBITO 24 define EXT HW INTERRUPT LEVEL 8 BASE_TRAP_EXT_HW_INTERRUPTS 7 SPWROUTER3 23 de
100. chine or domain It refers to the environment created by the 3765 hypervisor to execute user code Abbreviated terms Term Description ABI Application Binary Interface API Application Programming Interface IMA Integrated Modular Avionics MMU Memory Management Unit PCT Partition Control Table PIT Partition Information Table XML eXtended Markup Languaje 133 134 This page is intentionally left blank
101. control its execution time using the local execution clock Associated to each each clock the partition can arm one timer In the multicore approach the partition any of the virtual CPU can arm one time based on the global hardware clock Each virtual CPU can arm a one timer based on the local execution clock It is responsability of the guestOS or partition runtime of handling more than one timers based on the same clock using the appropriated data structures 2 18 Status Relevant internal information regarding the current state of the XtratuM and the partitions as well as accounting information is maintained in an internal data structure that can be read by system partitions This optional feature shall be enabled in the XtratuM source configuration and then recompile the XtratuM code By default it is disabled The hypercall is always present but if not enabled then XtratuM does not gather statistical information and then some status information fields are undefined It is enabled in the XtratuM menuconfig Objects XM partition status accounting Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 2 19 Summary 37 134 Monitoring Partition 1 Partition 2 Partitiont A A XM status open XM status read Partition Partition2 XtratuM status status status structure structure structure XtratuM Figure 2 19 Status overview 2 19 Summary Next is a brief summary of
102. cts as the boot loader The XEF XtratuM Executable Format has been designed as a robust format to copy the partition code and data from the partition developer to the final target system The XtratuM image shall also be in XEF format From the resident software point of view XtratuM is just another image that has to be copied into the appropriate memory area The main features of the XEF format are e Simpler than the ELF The ELF format is a rich and powerful specification but most of its features are not required Content checksum It allows to detect transmission errors Compress the content This feature greatly reduce the space of the image consequently the deploy time Encrypt the content Not implemented Partitions can be placed in several non contiguous memory areas The container is a file which contains a set of XEF files It is like a tar file with important internal dif ferences The resident software shall be able to manage the container format to extract the partitions XEF files and also the XEF format to copy them to the target memory addresses The signature fields are constants used to identify and locate the data structures The value that shall contain these fields on each data structure is defined right above the corresponding declaration xm 4 usermanual 047d X iram ula Printed November 26 2012 2555 2560 2565 2570 2575 2580 2585 2590 2595 2600 2605 2610 2615
103. d temporal isolation This version was developed for the x86 architecture but never released XtratuM 2 1 was the first porting to the LEON2 processor and several safety critical features were added Just to mention the most relevant features Bare metal hypervisor Employs para virtualisation techniques A hypervisor designed for embedded systems some devices can be directly managed by a desig nated partition Strong temporal isolation fixed cyclic scheduler Strong spatial isolation all partitions are executed in the user mode of the processor and do not share memory Resource allocation via a configuration table Robust communication mechanisms ARINC sampling and queuing ports Version 2 1 was a prototype to evaluate the capabilities of the LEON2 processor to support a hypervisor system XtratuM 2 2 was a more mature hypervisor on the LEON2 processor This version has most of the final functionality XtratuM 3 1 introduces several changes The main changes are Audit events have been added as the XtratuM tracing subsistem Virtual interrupt subsystem has been rewritten from scratch Cache instruction burst fetch capability can be either enabled or disabled during the XtratuM building process Cache snoop feature can be either enabled or disabled during the XtratuM building process Several partitions can be built with the same virtual addresses Hypercalls behaviour is more robust XtratuM 3 2 introduces severa
104. due to intellectual property restrictions the binary image of some partitions may not be available to a partition developer team In this case it is advisable to use dummy partitions to replace those non available rather than changing the configuration file 3 1 Development at a glance The first step is to buid the hypervisor binaries The integrator shall configure and compile the XtratuM sources to produce xm core xef The hypervisor image which implements the support for partition execution libxm a A helper library which provides a C interface to the para virtualised services via the hypercall mechanism Printed November 26 2012 Ah ir amu PA xm 4 usermanual 047d 3 2 Building XtratuM 41 134 Distribution j Partition 1 Source code 41 4l XtratuM Binary Distribution Partition 2 Source code Partition2 xef B Gd XtratuM Binary ES Distribution xm_core xef XM CFxml xmc xsd Figure 3 2 The big picture of building a XtratuM system xmc xsd The XML schema specification to be used in the XM CF configuration file tools A set of tools to manage the partition images and the XM CF file The result of the build process can be prepared to be delivered to the partition developers as a binary distribution The next step is to define the hypervisor system and resources allocated to each partition This is
105. e of the partition shall be loaded by a system partition through the XM memory copy O hypercall Note that there may be several booting partitions All those partitions will be started automatically at boot time The boot sequence is sketched in figure 7 1 0 The deployment tool to burn the PROM of the board writes first the resident software and right after it the container which should contain the XtratuM core and the booting partitions components Note that the container can also contain the non booting partitions D When the processor is started reset the resident software is executed It is a small code that performs the following actions 1 Initializes a stack required to execute C code 2 Installs a trap handler table only for the case that its code would generate a fault XtratuM installs a new trap handler during its initialisation 3 Checks that the data following the resident software in the PROM is a container a valid signature and seeks the XtratuM hypervisor through container a valid signature 4 Copies the XtratuM hypervisor and booting partitions into RAM memory 2 5 The address of the container which contains the ctCompTab is copied in the 4g2 processor register Jumps to the entry point of the hypervisor in RAM memory 8 XtratuM assumes no initial processor state So the first code has to be written in assembly code kernel sparcv8 head s and performs the next actions the g2 registe
106. ead Read a health monitoring log entry XM hm seek Sets the read position in the health monitoring stream XM hm status Get the status of the health monitoring log stream Trace Management XM trace event Records a trace entry XM trace open Open a trace stream XM trace read Read a trace event XM trace seek Sets the read position in a trace stream XM trace status Get the status of a trace stream Interrupt Management XM clear irqmask Unmask interrupts XM clear irqpend Clear pending interrupts XM raise ipvi Generate a inter partition virtual interrupt ipvi to a partition as specified in the configuration file XM route irq Link an interrupt with the vector generated when the XM_set_irqmask Mask interrupts XM set irqpend Set some interrupts as pending Miscelaneous XM get gid by name Returns the identifier of an entity defined in the configuration file XM multicall Execute a sequence of hypercalls XM read console Print a string in the hypervisor console XM write console Print a string in the hypervisor console XM write register32 Modify a processor internal register Sparcv8 specific XM_sparc_atomic_add Atomic add XM_sparc_atomic_and Atomic bitwise AND XM_sparc_atomic_or Atomic bitwise OR XM sparc clear pil Clear the PI
107. ection Entitled Endorsements Such a section may not be included in the Modified Version N Do not retitle any existing section to be Entitled Endorsements or to conflict in title with any Invariant Section O Preserve any Warranty Disclaimers If the Modified Version includes new front matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document you may at your option designate some or all of these sections as invariant To do this add their titles to the list of Invariant Sections in the Modified Version s license notice These titles must be distinct from any other section titles You may add a section Entitled Endorsements provided it contains nothing but endorsements of your Modified Version by various parties for example statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard You may add a passage of up to five words as a Front Cover Text and a passage of up to 25 words as a Back Cover Text to the end of the list of Cover Texts in the Modified Version Only one passage of Front Cover Text and one of Back Cover Text may be added by or through arrangements made by any one entity If the Document already includes a cover text for the same cover previously added by you or by arrangement made by the same entity you are acting on behalf of you may not add another but you may replace the old one on ex
108. ed XtratuM changes the stack to its own stack This second stack may contain sensitive information but it is located inside the memory space of XtratuM not exposed It is normal to observe that the partition stack is modified when a hypercall is called however this behaviour is far from being considered an actual security issue 10 2 Preventing covert side channels due to scheduling slot over run This version of XtratuM is non preemptible once the kernel starts an activity e g a service it cannot be interrupted until its completion This behaviour includes any hypercall invocation if a partition calls an hypercall just before a partition context switch must be performed XtratuM will not carry out the action until the hypercall is finished This overrun can be exploited to gain information The information is obtained by measuring the temporal cost of the last hypercall There are two types of information that can be retrieved 1 Whether the target partition was executing an hypercall at the end of the slot or not If the spy partition start at the nominal slot start time or not 2 In the case of being executing a hypercall how much time XtratuM needed to attend it the cost of the last hypercall 123 134 3470 3475 3480 3485 3490 3495 3500 3505 3510 3515 124 134 Chapter 10 Security issues In the case of a covert channel the maximum bandwidth is determined by the duration of the longest hype
109. ends the concept of processor interrupts by adding 32 additional sources of interrupt events Clock and timer management XtratuM provides one global clock and one local clock to the partitions Partitions can set one timer for each clock IP communication Inter partition communication is related to the communications between two partitions or between a partition and the hypervisor XtratuM implements a message passing model which highly resembles the one defined in the ARINC 653 A communica tion channel is the logical path between one source and one or more destinations Two basic transfer modes are provided sampling and queuing Partitions can access to chan nels through access points named ports The hypervisor is responsible for encapsulating and transporting messages Health monitor The health monitor is the part of XtratuM that detects and reacts to anoma lous events or states The purpose of the HM is to discover the errors at an early stage and try to solve or confine the faulting subsystem in order to avoid or reduce the possible consequences Tracing facilities XtratuM provides a mechanism to store and retrieve the traces generated by partitions and XtratuM itself Traces can be used for debugging during the development phase of the application but also to log relevant events or states during the production phase e API Defines the para virtualised services provided by XtratuM The access to these services is
110. er libxm include obt h user libxm include xm inc objects obt h 3xpath is a small shell script frontend to the xm11int utility Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 5 5 Partition reset 65 134 5 4 2 The Hello World example in multicore In a multicore platform when the partition is reset the virtual CPUO is booted and it is in charge of the initialisation of the other virtual CPUs In the Hello World example vcpuO resets the other vcpus 1790 include std_c h include lt xm h gt void PartitionMain void extern struct xmImageHdr xmImageHdr extern xmAddress_t start char str Hello World xm s32 t i xprintf Hello World n xprintf This is Partition d vCPU d n XM PARTITION SELF XM_get_vcpuid if XM get vcpuid 0 for i21 i lt 4 i XM reset vcpu i xmImageHdr pageTable xmAddress t start 0 while 1 1 counter counter 100000 XM_halt_partition XM_PARTITION_SELF Listing 5 13 Multicore simple example 5 5 Partition reset A partition reset is an unconditional jump to the partition entry point There are two modes to reset a partition XM WARM RESET and XM COLD RESET see section 2 3 On a warm reset the state of the partition is mostly preserved Only the field resetCounter of the PCT is incremented and the field resetStatus is set to the value given on the hypercall see XM partition reset 1795
111. ermine the most likely scope target and to deliver the HM to the corresponding upper scope Fault What is believed to be the original reason that caused an error 5Error The manifestation of a fault 6The term level is used in the ARINC 653 standard to refer to this idea xm 4 usermanual 047d AX ic amu P Printed November 26 2012 705 710 715 720 725 730 735 28 134 Chapter 2 XtratuM Architecture Monitoring Partition XM hm open XM hm read XM hm seek BM st XtratuM log message a HMlog Stream XML HM configuration subsystem i i HM HM actions events fault gt error gt failure Y fault error failure Y fault error failure N Figure 2 14 Health monitoring overview 2 13 1 HM Events zo There are three sources of HM events e Events caused by abnormal hardware behaviour Most of the processor exceptions are managed as health monitoring events e Events detected and triggered by partition code These events are usually related to checks or A assertions on the code of the partitions Health monitoring events raised by partitions are 745 a special type of tracing message see sections 2 16 Highly critical tracing messages are considered as HM events e Events triggered by XtratuM Caused by a violation of a sanity check performed by XtratuM on its internal state or the state of a partition When a HM event is detected the relevant
112. errupts and exceptions 2 leen 32 PASS c MM A A 32 2 15 2 Interrupts ss s seret 6 nes Pathe heb dad ed hoes m eeu 32 2 15 3 Interpartition virtual interrupt management lere 33 2 15 4 Multicore implications ssn ee e gU ea RR RO OX 34 2 16 Tates oe pog AAA e de ec LUN RR UR RR a 35 2 16 1 Multicor implications 4525 238 x oko moy Uy ws e UR B eeees 35 2 17 Clocks and timers cuca oou se RU m a RAR 36 2l 8 SAUE 52 2 ods d uso a OTORGA E E beg do dep De Ow edicere Ba a ye AD USE OE s 36 2 19 Summary s s i Reeves RR xe e E doo ox RO dk RO XO qe A CUR RR Re 37 3 Developing Process Overview 39 3 1 Developmentata glance 2 ee RADA Ee ee AAA EEE 40 3 2 Building XtratuM 4 1 loe cmo Rom y e o eR RO ee ee ew Wn 41 3 3 System configuration 22444 sU E AGAT ee eee 42 34 Compiling partition code 4 4 44 4086 a OR my UR TS ea 43 3 5 Passing parameters to the partitions customisation files 44 3 6 Building the final system image gt s s lt s cu ccc o o e ee 44 4 XtratuM Configuration and Build 45 4 1 Developing environment 1 se eaae E e a RR e a 45 4 2 Compile XtratuM Hypervisor llle ss 45 4 2 1 XtratuM configuration ask RR aaa ee ee ee ee ee we 46 4 2 2 Resident software configuration o o e mm e 46 4 2 3 Xtratum Abstraction Layer XAL configuration o o 47 4 2 4 Compiling XtratuM 2 cocci n on ERA does REE EES 47 4 3 Generating bina
113. essage remains in the source port until it is transmitted through the channel or it is overwritten by a new occurrence of the message whatever occurs first Each new instance of a message overwrites the current message when it reaches a destination port and remains there until it is overwritten This allows the destination partitions to access the latest message A partitions write operation on a specified port is supported by XM write sampling message hypercall This hypercall copies the message into an internal XtratuM buffer Partitions can read the message by using XM read sampling message which returns the last message written in the buffer XtratuM copies the message to the partition space Any operation on a sampling port is non blocking a source partition can always write into the buffer and the destination partition s can read the last written message The channel has an optional configuration attribute named refreshPeriod This attribute de fines the maximum time that the data written in the channel is considered valid Messages older than the valid period are marked as invalid When a message is read a bit is set accordingly to the valid state of the message Queuing port It provides support for buffered unicast communication between partitions Each port has associated a queue where messages are buffered until they are delivered to the destination partition Messages are delivered in FIFO order Sending and receiving message
114. et sparc_flush_regwin_nr o0 __XM_AHC sub fp std 10 std 412 std 86 std g4 fp 16 std g2 4fp 8 st Agi Afp 4 rd hy g5 st g5 Afpl mov 415 00 call ExtIrqHandler sub fp 0x80 sp la fp g1 wr Agi Ay ld 4fp 4 g1 ldd fp 8 g2 ldd fp 16 g4 ldd fp 241 g6 ldd 4fp 32 412 ldd 4fp 40 10 add fp 48 fp mov 410 o1 mov sparc set psr nr 00 __XM_AHC set sparc iret nr 00 __XM_AHC 48 fp p 40 4fp 32 4fp 24 Printed November 26 2012 hie adu xm 4 usermanual 047d 5 4 The Hello World example 59 134 HwIrqHandlerAsm jmpl 414 1o mov sparc_get_psr_nr 00 ExceptionHandlerAsm g0 __XM_AHC 1460 3 1515 mov 00 10 mov trapnr 15 A set sparc_flush_regwin_nr nop o0 __XM_AHC define BAD_TRAP trapnr sub fp 48 fp 1465 1 b 1b X 1520 std 410 fp 40 nop PN std 412 fp 32 nop PM std g6 4fp 24 nop std g4 fp 16 std 4g2 fp 8 1470 define SOFT_TRAP trapnr 1525 st Agi Afp 4 Ast b 1b IN rd Ay 185 nop SN st g5 tp nop PN mov A15 00 nop call HwIrqHandler 1475 1530 sub fp 0x80 sp define BUILD_EXTIRQ trapnr sethi hi ExtIrqHandlerAsm ld 4fpl Agi 414 N wr Agi hy jmpl 414 1o ld 4fp 4 gl 1480 ExtIrqHandlerAsm 4gO 1535 ldd Zfp 8 g2 mov trapnr 32 15 ldd fpti6 g4 nop ldd fp 241 g6 ldd fp 32 412 align 4096 ldd fp 40 410 1485 _traptab
115. feature can also be used to synchronise the operation of the partition with the scheduling plan 5 18 Partition customisation files A partition is composed of a binary image code and data and zero or more additional files customi sation files To ease the management of these additional files the header of the partition image see section 6 4 1 holds the fields noModules and moduleTab where the first is the number of additional files which have to be loaded and the second is an array of data structure which defines the loading Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 5 19 Assembly programming 81 134 address and the sizes of these additional files During its creation the partition is responsible for filling these fields with the address of a pre allocated memory area inside its memory space 2345 These information shall be used by the loader software for instance the resident software or a man ager system partition in order to know the place where to copy into RAM these additional files If the size of any of these files is larger than the one specified on the header of the partition or the memory address is invalid then the loading process shall fail These additional files shall be accessible by part of the loader software For example they must be 2350 packed jointly with the partition binary image by using the xmpack tool 5 19 Assembly programming This section describes the assembly programming c
116. figuration file can be shown in 5 16 1 lt MemoryLayout gt lt MemoryLayout gt lt XMHypervisor console Uart gt lt XMHypervisor gt lt ResidentSw gt lt PhysicalMemoryAreas gt lt PhysicalMemoryAreas gt lt ResidentSw gt lt PartitionTable gt lt PhysicalMemoryAreas gt lt Area start 0x40100000 lt Area start 0x60200000 lt Area start 0x60210000 shared gt lt Area start 0x60220000 lt Area start 0xfffff000 lt PhysicalMemoryAreas gt lt Partition gt lt PhysicalMemoryAreas gt lt PhysicalMemoryAreas gt lt Partition gt lt PartitionTable gt lt Partition id 0 name Partition0 lt PhysicalMemoryArea size 512KB gt lt Area start 0x60300000 size 512KB gt size 256KB gt size 64KB mappedAt 0x10000 gt size 64KB mappedAt 0x20000 flags size 64KB flags uncacheable gt size 4KB lt Region type stram start 0x40000000 size 4MB gt lt Region type sdram start 0x60000000 size 8KB gt lt Region type sdram start 0xfffff000 size 4KB gt console Uart gt lt TemporalRequirements duration 500ms period 500ms gt Partition id 1 name Partitioni flags system console Uart gt lt Area start 0x40140000 size 256KB mappedAt 0x40000000 lt Area start 0x60210000 size 64KB flags shared read only gt lt TemporalRequirements duration 500ms period 500ms gt Listing
117. fine EXT HW INTERRUPT LEVEL 7 BASE TRAP EXT HW INTERRUPTS46 SPWROUTER2 22 define EXT HW INTERRUPT LEVEL 6 BASE TRAP EXT HW INTERRUPTS45 SPWROUTER1 21 define EXT HW INTERRUPT LEVEL 5 BASE TRAP EXT HW INTERRUPTS44 SPWROUTERO 20 define EXT HW INTERRUPT LEVEL 4 BASE TRAP EXT HW INTERRUPTS43 GRGPIO3 19 define EXT HW INTERRUPT LEVEL 3 BASE TRAP EXT HW INTERRUPTS42 GRGPIO2 18 define EXT HW INTERRUPT LEVEL 2 BASE TRAP EXT HW INTERRUPTS41 GRGPIO1 17 define EXT HW INTERRUPT LEVEL 1 BASE TRAP EXT HW INTERRUPTS40 GRGPIOO 16 Listing 5 20 core include arch irqs h The structure of the virtual trap table mimics the native trap table structure Each entry is a 16 bytes large and contains the trap handler routine which in the practice is a branch or jump instruction to the real handler At boot time or after a reset a partition shall setup its virtual trap table and load the virtual tbr register with the start address of the table The tbr register can be managed with the hypercalls XM read register32 and XM write register32 with the appropriate register name define TBR REG32 0 Listing 5 21 Register name If a trap is delivered to the partition and there is not a valid virtual trap table then the health monitoring event XM EM EV PARTITION UNRECOVERABLE is generated 5 11 2 Interrupts In order to properly manage a peripheral a partition can request to manage directly a hardware
118. from the entry point entry address in the calling parameters and using a stack address defined in the call Halt a virtual CPU the hypercall XM halt vcpu O permits to halt the execution of a virtual CPU 3010 Suspend a virtual CPU the hypercall XM suspend vcpu OO permits to suspend the execution of a vir tual CPU Resume a virtual CPU the hypercall XM resume vcpu O permits to resume the execution of a virtual CPU Get the virtual CPU identifier the hypercall XM get vcpuIdO permits to know the virtual CPU where sus the calling partition thread is under execution xm 4 usermanual 047d A ic amu P Printed November 26 2012 This page is intentionally left blank Volume 2 XtratuM User Manual Chapter 8 Configuration This section describes how XtratuM is configured There are two levels of configuration A first level which affects the source code to customise the resulting XtratuM executable image Since XtratuM does not use dynamic memory to setup internal data structures most of these configuration parameters are related to the size or ranges of the statically created data structures maximum number of partitions channels etc The second level of configuration is done via an XML file This file configures the resources allocated to each partition 8 1 XtratuM source code configuration menuconfig The first step in the XtratuM configuration is to configure the source code This task is done using
119. gisters bug Also the ProcessorTable Processor element defines the scheduling plan of this processor It is specified in the element Processor Sched CyclicPlan Plan The Plan element has the required attributes name and majorFrame and contains a sequence of Slot elements Each S1ot element has the following attributes Slot id Slot Id s shall meet the id s rules defined in section 2 7 This value can be retrieved by the partition at run time see section 5 7 1 Slot duration Time duration of the slot Slot partitionId Id of the partition that will be executed during this slot Slot start Offset with respect to the MAF start Slots intervals shall not overlap HwDescription MemoryLayout Defines the memory layout of the board All the memory allocated to partitions resident software and XtratuM itself shall be in the range of one of these areas HwDescription Devices The device element contains the sequence the XtratuM devices Currently XtratuM implements two types of devices UART and memory blocks Uart Has the required attributes Uart name Uart baudRate and Uart id This element associates the hardware device eid with the name and programs the transmission speed MemoryBlockTable This element contains a sequence of one or more Block elements A memory block device defines an area of RAM ROM or FLASH memory This block of 3200 memory can then be used to store traces health monito
120. hat derivative works of the document must them selves be free in the same sense It complements the GNU General Public License which is a copyleft license designed for free software We have designed this License in order to use it for manuals for free software because free software needs free documentation a free program should come with manuals providing the same freedoms that the software does But this License is not limited to software manuals it can be used for any textual work regardless of subject matter or whether it is published as a printed book We recommend this License principally for works whose purpose is instruction or reference 1 APPLICABILITY AND DEFINITIONS This License applies to any manual or other work in any medium that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License Such a notice grants a world wide royalty free license unlimited in duration to use that work under the conditions stated herein The Document below refers to any such manual or work Any member of the public is a licensee and is addressed as you You accept the license if you copy modify or distribute the work in a way requiring permission under copyright law A Modified Version of the Document means any work containing the Document or a portion of it either copied verbatim or with modifications and or translated into another language A Secondary Section is a named
121. he XEF file XEF_COMPRESSED If set then the partition binary image is compressed XEF_CIPHERED future extension to inform whether the partition binary is encrypted or not XEF_CONTENT Specifies what kind of file is digest when the XEF_DIGEST flag is set this field holds the result of processing whole the XEF file supposing the digest field set to 0 The MD5 algorithm is used to calculate this field Despite the well known security flaws we selected the MD5 digest algorithm because it has a reasonable trade off between calculation time and the security level Note that the digest field is used to detect accidental modifications rather than intentional attacks In this scenario the MDS is a good choice payLoad This field holds 16 bytes which can freely be used by the partition supplier It could be used to hold information such as partition s version etc The content of this field is used neither by XtratuM nor the resident software fileSize XEF file size in bytes segmentTabUffset Offset to the section table noSegments Number of segments held in the XEF file In the case of a customisation file there will be only one segment customFileTabOffset Offset to the custom files table noCustomFiles Number of custom files imageOffset Offset to the partition binary image imageLength Size of the partition binary image deflatedImageLength When the XEF COMPRESS flag is set this field holds the size of the uncom pres
122. herefore all internal data structures are declared statically The size of these data structures are defined during the source code configuration process The following configuration parameters are the ones that have an impact on the memory needed by XtratuM Maximum identifier length Defines the space reserved to store the names of the partitions ports scheduling slots and channels Kernel stack size For each partition XtratuM reserves a kernel stack Do not reduce the value of this parameter unless you know the implications Partition memory areas if the WPR is used and MMU disabled Due to the hardware device WPR used to force memory protection the area of memory allocated to the partitions shall fulfil the next conditions Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 5 3 XAL development environment 55 134 e The size shall be greater than or equal to 32KB e The size shall be a power of two e The start address shall be a multiple of the size Configuration of the resident software RSW The information contained in the XM CF regarding the RSW is not used to configure the RSW itself That information is used e by XtratuM to perform a system cold reset e and by the xmcparser to check for memory overlaps Partition declaration order The partition elements in the XM CF file shall be ordered by id and the id s shall be consecutive starting at zero 5 3 XAL development environment XAL is a m
123. ibc klibc sparcv8 objects drivers gt Linking XM Core text data bss dec hex filename 114545 11548 103800 229893 38205 xm core Oe05cacb4e969a5a34130c22def008fb xm core xef gt Done gt Configuring and building the User utilities gt Building XM user libxm tools tools xmpack tools xmcparser tools xmbuildinfo tools rswbuild tools xef xal bootloaders rsw examples gt Done 4 3 Generating binary a distribution The generated files from the compilation process are in source code directories In order to distribute the compiled binary version of XtratuM to the partition developers a distribution package shall be generated There are two distribution formats 1195 Tar file It is a compressed tar file with all the XtratuM files and an installation script make distro tar xm 4 usermanual 047d A ie amu P Printed November 26 2012 48 134 Chapter 4 XtratuM Configuration and Build Self extracting installer It is a single executable file which contains the distribution and the installa tion script make distro run The final installation is exactly the same regarding the distribution format used make distro run gt Installing XM in tmp xtratum X X X xtratum X X X xm Generating XM shaisums Installing XAL Generating XAL shaisums Installing XM examples Generating XM examples shaisums Setting read only og w permission Deleting empty fi
124. ic non preemptable but at some designated safe places in the code the pre emption is allowed Scenario 1 covert channel XM send message IN 3 Partition activation Scenario 2 covert channel prevented XM send message Partition activation Figure 10 1 Covert channel caused by an incorrect scheduling plan and a solution Printed November 26 2012 X tie erm uPA xm 4 usermanual 047d Volume 2 XtratuM User Manual Appendix A XML Schema Definition See attached document xm 4 usermanual XML spec pdf 125 134 This page is intentionally left blank 3550 Volume 2 XtratuM User Manual GNU Free Documentation License Version 1 2 November 2002 Copyright C 2000 2001 2002 Free Software Foundation Inc 51 Franklin St Fifth Floor Boston MA 02110 1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document but changing it is not allowed Preamble The purpose of this License is to make a manual textbook or other functional and useful document free in the sense of freedom to assure everyone the effective freedom to copy and redistribute it with or without modifying it either commercially or noncommercially Secondarily this License preserves for the author and publisher a way to get credit for their work while not being considered responsible for modifications made by others This License is a kind of copyleft which means t
125. icense in the document and put the following copyright and license notices just after the title page Copyright c YEAR YOUR NAME 3745 Permission is granted to copy distribute and or modify this document under the terms of the GNU Free Documentation License Version 1 2 or any later version published by the Free Software Foundation with no Invariant Sections no Front Cover Texts and no Back Cover Texts A copy of the license is included in the section entitled GNU 3750 Free Documentation License If you have Invariant Sections Front Cover Texts and Back Cover Texts replace the with Texts line with this with the Invariant Sections being LIST THEIR TITLES with the Front Cover Texts being LIST and with the Back Cover Texts being LIST ass If you have Invariant Sections without Cover Texts or some other combination of the three merge those two alternatives to suit the situation If your document contains nontrivial examples of program code we recommend releasing these ex amples in parallel under your choice of free software license such as the GNU General Public License to permit their use in free software Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d Volume 2 XtratuM User Manual Glossary of Terms and Acronyms E Glossary hypervisor The layer of software that using the native hardware resources provides one or more virtual machines partitions partition Also known as virtual ma
126. id 0 majorFrame 800ms gt lt Slot id 0 start Oms duration 200ms partitionId 0 vCpuld 0 gt Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 2 9 XtratuM Multicore Scheduling 19 134 Slot id 1 start 200ms duration 200ms partitionId 2 vCpuld 0 gt lt Slot id 2 start 400ms duration 200ms partitionId O vCpuId 1 lt Slot id 3 start 600ms duration 200ms partitionId 1 vCpuld 0 gt Plan lt CyclicPlanTable gt 420 lt Processor gt lt Processor id 1 frequency 50Mhz gt lt CyclicPlanTable gt lt Plan id 0 majorFrame 800ms gt lt Slot id 0 start Oms duration 300ms partitionId 3 vCpuId 0 gt 425 lt Slot id 1 start 300ms duration 200ms partitionId 0 vCpuld 2 Slot id 1 start 500ms duration 200ms partitionId 2 vCpuld 1 Plan lt CyclicPlanTable gt lt Processor gt 430 This plan produces a scheduling as detailed in the next figure 2 13 MAF _ _ oy CPU 0 lt P0 0 gt lt P2 0 gt lt P0 1 gt lt P1 0 gt CPU 1 P3 009 lt P0 3 gt lt P2 1 gt Plan 0 Figure 2 13 Cyclic plan example In this scheduling plan the configuration file defines e partition PO allocates its vCPUO and vCPU1 in CPUO and VCPU2 in CPU1 e partition P1 allocates its vCPUO in CPUO 435 e partition P2 allocates its vCPUO in CPUO and vCPU1 in CPU1 e partition P3 allocates its vCPUO in CPU1
127. in three sub states Ready The virtual CPU is ready to execute code but it is not scheduled because it is not in its time slot Running The virtual CPU of the partition is being executed by the processor where the virtual CPU is allocated Idle Ifthe virtual CPU does not want to use the processor during its allocated time slot it can relinquish the processor and wait for an interrupt or for the next time slot see XM idle self Printed November 26 2012 A ir au PA xm 4 usermanual 047d 2 6 System partitions 13 134 XM reset vcpu or XM reset partition or XM HM AC PARTITION COLD WARM RESET XM suspend vcpu or XM suspend partition or XM HM AC SUSPEND Normal Ready Running Idle XM halt vcpu XM resume vcpu or or XM halt partition XM resume partition or XM_HM_AC_HALT Figure 2 10 Virtual CPU states and transitions A virtual CPU can halt itself or be halted by another vCPU of the same partition When halted the virtual CPU is not selected by the scheduler and the time slot allocated to it is left idle it is not allocated to other partitions All resources allocated to the virtual CPU are released It is not possible to return 2 to normal state In suspended state the virtual CPU is not be scheduled and interrupts are not delivered Interrupts raised while in suspended state are left pending The virtual CPU can return to ready state if requested by another vCPU of the the partition
128. information error scope offending partition identifier zo memory address faulting device etc is gathered and used to select the apropiate HM action The next table shows the event detected by XtratuM Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 2 13 Health monitor HM 29 134 Action Description XM HM EV INTERNAL ERROR M HM EV UNEXPECTED TRAP X X XM HM EV PARTITION ERROR XM HM EV PARTITION INTEGRITY XM HM EV MEM PROTECTION M HM EV OVERRUN M HM EV SCHED ERROR M HM EV WATCHDOG TIMER M HM EV ARM UNDEF INSTR X X X X X XM HM EV ARM PREFETCH ABORT XM HM EV ARM DATA ABORT 2 13 2 HM Actions M HM EV PARTITION UNRECOVERABLE M HM EV INCOMPATIBLE INTERFACE Internal XtratuM error XtratuM detected a mismatch between the partition s di gest stored in the container and the one calculated from the partition s section inside the container Either XtratuM or a partition triggered a memory protec tion error The processor received a watchdog timer signal The processor detected an unknown instruction An instruction prefetching error was encountered accord ing to the conditions described in the ARM architecture manual A data access abort has been detected according to the con ditions described in the ARM architecture manual Once an HM event is raised XtratuM has to react quickly to the event The set of configurable HM actions
129. inimal developing environment to create bare C applications It is provided jointly with the XtratuM core Currently it is only the minimal libraries and scripts to compile and link a C application More features will added in the future mathematic lib etc In the previous versions of XtratuM XAL was included as part of the examples of XtratuM It has been moved outside the tree of XtratuM to create an independent developer environment When XtratuM is installed the XAL environment is also installed It is included in the target directory of the installation path target directory xal XAL components xal examples examples of XAL use xm xm examples Listing 5 1 Installation tree The XAL subtree contains the following elements xal bin utilities xpath xpathstart common compilation rules config mk config mk dist rules mk include headers arch irqs h assert h autoconf h config h ctype h irqs h limits h stdarg h stddef h stdio h stdlib h string h xm 4 usermanual 047d AX ic amu P Printed November 26 2012 1290 1295 1300 56 134 Chapter 5 Partition Programming xal h lib libraries libxal a loader lds shalsum txt Listing 5 2 XAL subtree A XAL partition can e Be specified as system or user 1305 e Use all the XtratuM hyperca
130. ion 360ms partitionId 2 vCpuld 1 gt 505 lt Plan gt lt CyclicPlanTable gt lt Processor gt lt Processor id 3 frequency 50Mhz gt 510 lt FixedPriority gt Printed November 26 2012 A ir amu PA xm 4 usermanual 047d 2 9 XtratuM Multicore Scheduling 21 134 Partition id 0 vCpuId 2 priority 10 gt Partition id 1 vCpuId 1 priority 5 gt lt FixedPriority gt lt Processor gt lt ProcessorTable gt xm 4 usermanual 047d ie ce uPA Printed November 26 2012 515 22 134 Chapter 2 XtratuM Architecture 2 10 Memory management Partitions and XtratuM core can be allocated at defined memory areas specified in the XM CF A partition define several memory areas Each memory area can define some attributes or rights to permit to other partitions to access to their defined areas or allow the cache management The following attributes area allowed unmapped Allocated to the partition but not mapped by XtratuM in the page table mappedAt It allows to allocate this area to a virtual address shared It is allowed to map this area in other partitions read only The area is write protected to the partition uncacheable Memory cache is disabled The implementation of the whole set of features will depend on the available hardware For systems with a MMU Memory Management Unit most or all of them will be available though in systems with MPU Memory Protection Unit or WPR Write P
131. ion XtratuM ABI version used to compile the partition That is the ABI version of the libxm and other accompanying utilities used to build the XEF file compilationXmApiVersion XtratuM API version used to compile the partition That is the API version of the libxm and other accompanying utilities used to build the XEF file The current values of these fields are define define define define define define XM_ABI_VERSION 3 XM_ABI_SUBVERSION 1 XM_ABI_REVISION O XM_API_VERSION 3 XM_API_SUBVERSION 1 XM_API_REVISION 2 Listing 6 6 core include hypercalls h Note that these values may be different to the API and ABI versions of the running XtratuM This information is used by XtratuM to check that the partition image is compatible Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 6 4 Partition ELF format 91 134 noCustomFiles The number of extra files accompanying the image If the image were Linux then one of the modules would be the initrd image Up to CONFIG MAX NO FILES can be attached The moduleTab table contains the locations in the RAM s address space of the partition where the modules shall be copied if any See section 5 18 2620 customFileTab Table information about the customisation files struct xefCustomFile 1 xmAddress t sAddr xmSize t size 2625 __PACKED Listing 6 7 core include xmef h sAddr Address where the customisation file shall be loaded
132. is WARM reset On a COLD reset it is set to zero resetStatus If the partition had been reset by a XM reset partition hypercall then the value of the parameter status is copied in this field Zero otherwise id The identifier of the partition It is the unique number specified in the XM CF file to unequivocally identify a partition hwIrqs A bitmap of the hardware interrupts allocated to the partition Hardware interrupts are allo cated to the partition in the XM CF file noPhysicalMemoryAreas The number of memory areas allocated to the partition This value defines the size of the physicalMemoryAreas array name Name of the partition hwIrqsPend Bitmap of the hardware interrupts allocated to the partition delivered to the partition extIrqsPend Bitmap of the extended interrupts allocated to the partition delivered to the partition Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 6 5 XEF format 93 134 hwIrqsMask Bitmap of the extended interrupts allocated to the partition delivered to the partition extIrqsMask In the current version there is no specific architecture data 6 5 XEF format The XEF is a wrapper for the files that may be deployed in the target system There are three kind of files e Partition images e The XtratuM image e Customisation files An XEF file has a header see listing 6 5 and a set of segments Each segment like in ELF represents a block of memory th
133. itialise the rest of virtual CPUs required The virtual CPUs can be allocated by the integrator XM CF configuration file in any of the real CPUs 2 2 Architecture Next figure 2 2 presents the approach in a monocore processor In the monocore architecture XtratuM is in charge of the virtualisation of the CPU to the partitions A partition uses the virtual CPU to execute the internal code of the partition The CPU is initialised by the hypervisor and as soon as the system is initialised the scheduling plan is started The plan specifies the sequence of slots to be executed Each slot identifies a temporal window and the partition to be executed Figure 2 3 presents the approach in a multicore processor In the multicore architecture XtratuM virtualises the available CPUs offering to the partitions virtual CPUs A partition can use one or more virtual CPUs to execute the internal code Figure 2 4 shows an example of how partitions can use one or more virtual CPUs to implement mono or multicore partitions xm 4 usermanual 047d X ici A Printed November 26 2012 160 165 170 175 180 185 8 134 Chapter 2 XtratuM Architecture Monocore processor Figure 2 2 Monocore approach vCPUO vCPU2 vCPUO vCPU2 vCPUO vCPU2 vCPU1 vCPU3 vCPU1 vCPU3 vCPU1 vCPU3 XtratuM Multicore CPUO cPU2 processon CPU1 CPU3 Figure 2 3 Multicore approach io In this case one of the partition is monocore and
134. l changes The main changes are Any exception caused by a partition when the processor is in supervisor mode causes a partition unrecoverable error xm 4 usermanual 047d A ic emu PA Printed November 26 2012 55 60 65 70 75 80 85 4 134 Chapter 1 Introduction e A cache flush is executed always in order to guarantee that a partition is unable to retrieve XM 90 information This operation is forced XtratuM 3 5 is the developing version for XtratuM to LEON4 processor It introduces important changes with respect to monocore versions Changes impact on most of the features A summary is e XtratuM exports multiple virtual CPUS to the partitions 95 e Processor initialisation e Interrupt management e Scheduling plan e XML configuration file When released it will be XtratuM 4 0 Printed November 26 2012 XK tir LI PA xm 4 usermanual 047d Volume 2 XtratuM User Manual Chapter 2 XtratuM Architecture This chapter introduces the architecture of XtratuM 100 The concept of partitioned software architectures was developed to address security and safety issues The central design criteria involves isolating modules of the system into partitions Temporal and spatial isolation are the key aspects in a partitioned system Based on this approach the Integrated Modular Avionics IMA is a solution allowed by the Aeronautic Industry to manage the increment of the functionalities of the software maintaining
135. ld creates an bootable ELF file with the resident software and the container to be executed on the target hardware Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d Volume 2 XtratuM User Manual Chapter 4 XtratuM Configuration and Build 4 1 Developing environment XtratuM has been compiled and tested with the following package versions Package Version Linux package name Purpose host gcc 4 2 3 4 4 3 gcc 4 2 req Build host utilities make 3 81 3build1 make req Core libncurses 5 7 20100313 5 libncurses5 dev req Configure source code binutils 2 18 1build1 2 20 1 binutils req Core sparc toolchain linux 3 4 4 req Core libxml2 2 6 27 2 7 6 libxml2 dev req Configuration parser tsim 2 0 10 2 0 15 opt Simulated run grmon 1 1 31 opt Deploy and run perl 5 8 8 12 5 10 1 opt Testing makeself 2 1 5 makeself opt Build self extracting distribution Packages marked as req are required to compile XtratuM Those packages marked as opt are needed to compile or use it in some cases 4 2 Compile XtratuM Hypervisor It is not required to be supervisor root to compile and run XtratuM 1150 The first step is to prepare the system to compile XtratuM hypervisor 1 Check that the GNU LIBC Linux GCC 3 4 4 toolchain for SPARC LEON is installed in the system It can be downloaded from http www gaisler com sparc linux 3 4 4 x x x tar bz2 2 Make a deep clean to be sure that there is not previous configu
136. le section which contains an array of xmefComponent structures Each element contains information of one component 3 The file table section which contains an array of files xmefFile structure in the container 4 The string table section Contains the names of the files of the original executable objects This is currently used for debugging 5 The file data table section with the actual data of the executable XtratuM and partition images and the configuration files The container header has the following fields struct xmefContainerHdr xm_u32_t signature define XM_PACKAGE_SIGNATURE 0x24584354 XCT xm_u32_t version define XMPACK_VERSION 3 define XMPACK_SUBVERSION O define XMPACK REVISION O xm u32 t flags define XMEF CONTAINER DIGEST 0x1 xm u8 t digest XM DIGEST BYTES xm u32 t fileSize xmAddress t partitionTabOffset xm s32 t noPartitions xmAddress t fileTabOffset xm s32 t noFiles xmAddress t strTabOffset xm s32 t strLen xmAddress t fileDataOffset Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 2855 6 6 Container format 97 134 xmSize t fileDataLen __PACKED Listing 6 12 core include xmef h signature Signature field version Version of the package format 2860 flags digest Not used Currently the value is zero fileSize The size of the container partitionTabOffset The offset relative to the start of the file to the partition array
137. les directories gt Done gt Generating XM distribution xtratum X X X tar bz2 gt Done gt Generating self extracting binary distribution xtratum X X X run gt Done The files xtratum x x x tar bz2 or xtratum x x x run contain all the files requires to work de velop and run with the partitioned system This tar file contains two root directories xal and xm and an installation script INSTALL PATH xal examples xm examples common include bootloaders Figure 4 1 Content of the XtratuM distribution Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 4 4 Installing a binary distribution 49 134 The directory xm contains the XtratuM kernel and the associated developer utilities Xal stands for XtratuM Abstraction Layer and contains the partition code to setup a basic C execution environment Xal is provided for convenience and it is not mandatory to use it Xal is only useful for those partitions with no operating system Although XtratuM core and related libraries are compiled for the LEON3 processor some of the host configuration and deploying tools xmcparser xmpack and xmeformat are host executables If the computer where XtratuM was compiled on and the computer where it is being installed are different processor architectures 32bit and 64bit the tools may not run properly 4 4 Installing a binary distribution Decompress the xtratum x x x
138. lication each partition developer requires the following files libxm a Para virtualised services The include files are distributed jointly with the library and they should be provided by the integrator xm 4 usermanual 047d A ic emu P Printed November 26 2012 1090 1095 1100 1105 1110 A 1115 1120 1125 1130 1135 1140 1145 44 134 Chapter 3 Developing Process Overview XM CF xml The system configuration file This file describes the whole system The same file should be used by all the partners xm core bin The hypervisor executable This file is also produced by the integrator and delivered to the other partners xmpack The tool that packs together into a single system image container all the components xmeformat For converting an ELF file into an XEF one xmcparser The tool to translate the configuration file XM CF xm1 into a C file which should be compiled to produce the configuration table XM CT Partition developer should use an execution environment as close as possible to the final system the same processor board and the same hypervisor framework To achieve this goal they should use the same configuration file as the one used by the integrator But the code of other partitions may be replaced by dummy partitions This dummy partition code could execute for instance just a busy loop to waste time 3 5 Passing parameters to the partitions customisation files
139. lls according to the type of partition e Use the standard input output C functions printf sprintf etc The available functions are defined in the include stdio h e Define interrupt handlers and all services provided by XtratuM An example of a Xal partition is include lt xm h gt include lt stdio h gt define LIMIT 100 void SpentTime int n int i j int x y 1 for i 0 i lt n i for j 0 j lt n j K xX xX y F void PartitionMain void long counter 0 printf P d XAL Partition n XM_PARTITION_SELF counter 1 while 1 counter SpentTime 2000 printf P d Counter d Nn XM PARTITION SELF counter XM_halt_partition XM_PARTITION_SELF Listing 5 3 XAL partition example 1310 In the xal examples subtree the reader can find several examples of XAL partitions and how these examples can be compiled Next is shown the Makefile file XAL_PATH path to the XTRATUM directory XAL_PATH xal XMLCF path to the XML configuration file XMLCF xm cf sparcv8 xml Printed November 26 2012 A ir amu PA xm 4 usermanual 047d 5 4 The Hello World example 57 134 PARTITIONS partition files xef format composing the example PARTITIONS partitioni xef partition2 xef all container bin resident sw include XAL PATH common rules mk partitioni dummy xal o LD o 0 LDFLAGS Ttext call xpathstart 1 XMLCF
140. lso the attribute entryPoint is used by XtratuM in the case of a cold system reset In that case XtratuM will give back the control of the system to the resident software by jumping to this address 8 3 6 The SystemDescription PartitionTable Partition element Attribute description id Required See the section 2 7 for a description on how to identify XtratuM objects Cname Optional console Optional The console device where the output of the hypercall XM_write_console O is copied to flags Optional List of features Possible values are fp If set the partition is allowed to use floating point operations By default not set system If set the partition has system privileges By default not set Partition elements PhysicalMemoryAreas Sequence of memory areas allocated to the partition HwResources Contains the list of interrupts and IO ports allocated to the partition PortTable Contains the sequence of communication ports queuing and sampling ports of the partition Trace Configuration of the trace facility of the partition Same attributes are that of the SystemDescription XMHypervisor Trace element TemporalRequirements An element which has two mandatory attributes period and duration This data is not checked by XtratuM Reserved for future use Configuration of memory areas The attributes are Ostart size and flags The Oflags attribute is a list of the following 3230 values
141. ly to develop partitions it is advisable to install the binaries in a separate read only directory to avoid accidental modifications of the code It is also possible to build a TGZ package with all the files to develop with XtratuM which can be delivered to the partition developers See chapter 4 3 3 System configuration The integrator jointly with the partition developers has to define the resources allocated to each partition by creating the XM CF file It is an XML file which shall be a valid XML against the XMLSchema defined in section 8 3 Figure 3 4 shows a graphical view of the configuration schema The main information contained in the XM CF file is 2TGZ Tar Gzipped archive Printed November 26 2012 A ir amu PA xm 4 usermanual 047d 3 4 Compiling partition code 43 134 Memory The amount of physical memory available in the board and the memory allocated to each partition Processor How the virtual processors are allocated to each partition the scheduling plan Peripherals Those peripherals not managed by XtratuM can be used by one partition The I O port ranges and the interrupt line if any Health monitoring How the detected errors are managed by the partition and XtratuM direct action delivered to the offending partition create a log entry reset etc Inter partition communication The ports that each partition can use and the channels that link the source and destination ports Tracing Where t
142. m file xef data in b07715208bbfe72897a259619e7d7a6d custom file xef List the header of the XEF custom file xm 4 usermanual 047d AX ic amu P Printed November 26 2012 3320 3325 3330 3335 3340 3345 3350 3355 3360 3365 3370 3375 3380 3385 3390 120 134 Chapter 9 Tools xmeformat read h custom file xef XEF header signature 0x24584546 version 1 0 0 flags XEF DIGEST XEF CONTENT CUSTOMFILE digest b07715208bbfe72897a259619e7d7a6d payload 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file size 232 segment table offset 80 no segments 1 customFile table offset 104 no customFiles 0 image offset 104 image length 127 XM image s header 0x0 Build the hypervisor XEF file xmeformat build o xm_core xef c core xm_core List the segments and headers of the XtratuM XEF file xmeformat read s xm_core xef Segment table 1 segments segment O physical address 0x40000000 virtual address 0x40000000 file size 68520 compressed file size 32923 48 0596 xmeformat read h xm core xef XEF header signature 0x24584546 version 1 0 0 flags XEF DIGEST XEF COMPRESSED XEF CONTENT HYPERVISOR digest 6698cfcf9311325e46e79ed50dfc9683 payload 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file size 33040 segment table offset 80 no segments 1 customFile table offset 104 no customFiles 1 image offset 112 image length 68520 XM image s header 0x40010b78 comp
143. mory to store all traces The amount of memory reserved to store traces is a configuration parameter of the sources see section 8 1 In order to be able to store the traces of a partition as well as the traces generated by XtratuM it has to be properly configured in the XM CF configuration file The bitmask attribute is used to filter which traces are stored XMHypervisor console Uart healthMonitorDevice MemDiski gt Trace device MemDisk0 bitmask 0x00000005 gt lt XMHypervisor gt lt PartitionTable gt Partition console Uart flags system id 0 name Partition0 gt lt PhysicalMemoryAreas gt lt Area size 64KB start 0x40080000 gt lt PhysicalMemoryAreas gt Printed November 26 2012 AX ir eu PA xm 4 usermanual 047d 5 15 System and partition status 77 134 Trace device MemDisk0 bitmask 0x3 gt lt TemporalRequirements duration 500ms period 500ms gt 2190 lt Partition gt lt PartitionTable gt Listing 5 27 Trace definition example The traces recoded by XtratuM can be selected masked at module granularity In the example of listing 5 14 3 the TRACE BM HYPERVISOR and TRACE BM SCHED events will be recorded 2195 but not TRACE BM PARTITION 5 15 System and partition status The hypercalls XM get partition status and XM get system status return information about a given partition and the system respectively The data structure returned are 2200 ty
144. mory what will use XtratuM to run it Note that it is not the size of the file but the memory required by all the sections including those not allocatable ones bss The resident software can be executed in ROM or RAM memory if the ROM technology allows to run eXecute in Place XiP code The resident software has no initialised data segment only the text and rodata segments are required the got and eh frame segments are not used The memory footprint of the RSW depends on whether the debug messages are enabled or not and it can be obtained from the final resident software code the file created with the build rsw helper utility using the readelf utility The next example shows where the size of the RSW code is printed column labeled MemSiz sparc linux readelf 1 resident sw Elf file type is EXEC Executable file Entry point 0x4020102c There are 5 program headers starting at offset 52 Program Headers Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align LOAD 0x0000d4 0x00004000 0x00004000 0x196f0 0x196f0 RW 0x1 00 segment LOAD 0x0197c4 0x40090000 0x0001d6f0 0x00048 0x00048 RW 0x4 01 segment LOAD 0x019808 0x40090048 0x40090048 0x00000 0x02000 RW Ox8 02 segment LOAD 0x019810 0x40200000 0x40200000 0x01f44 0x01f44 R E 0x8 03 segment GNU STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RWE Ox4 04 segment Section to Segment mapping Segment Sections 00 container 01 got eh frame 02
145. n TrapHandlerN i Li aE XM mask irq XM CFxml ir a XM mask irq 5 y g TD n XM unmask irq Rma y E XM unmask irq 32 63 Software Traps Extended Irqs _ Software Traps _ Software Traps E Extended Irqs Health Monitoring Hardware Irqs a Health Monitoring Hardware Irqs XtratuM XtratuM 1 F irq Controller A Irq Controller P a Peripherals d Peripherals Figure 2 16 Hardware and extended interrupts delivery Partitions shall manage this new set of events in the same way standard traps are These new inter rupts are vectored in the trap table These trap handlers are invoked by XtratuM on the occurrence of an event alike a standard LEONS3 trap 2 15 3 Interpartition virtual interrupt management ss Additionally to the interrupts propagated by XtratuM to the partitions XtratuM provides the mech anisms to send an interrupt from a partition to other or others This are the inter partition virtual interrupts IPVI A IPVI is an interrupt or more properly an event generated by a partition to inform to other or others partitions about some relevant state Next table shows the list of IPVIs available to be used by the partitions 880 xm 4 usermanual 047d X irem uPA Printed November 26 2012 34 134 Chapter 2 XtratuM Architecture Partition Partition
146. n define XM HMLOG PAYLOAD LENGTH 4 struct hmCpuCtxt cpuCtxt xmWord t payload XM_HMLOG_PAYLOAD_LENGTH __PACKED Listing 2 2 HM log definition where signature A magic number to identify the content of the structure as HM log checksum A MD5 digestion of the data structure allowing to verify the integrity of its content opCode The bits of this field codifies Bits 31 19 eventId Identifies the event that caused this log Bits 16 sys Set if the HM event was generated by the hypervisor otherwise clear Bits 15 8 validCpuCtxt Set if the field cpuCtxt see below holds a valid processor context Printed November 26 2012 AX ir emu PA xm 4 usermanual 047d 2 14 Access to devices 31 134 Bits 7 0 partitionId The Id attribute of the partition that caused the event seq Number of sequence enabling to sort the event respect other events It is codified as unsigned integer incremented each time a new HM event is logged timeStamp A time stamp of when the event was detected Either cpuCtxt or payload When validCpuCtxt bit is set opCode then this field holds the proces sor context when the HM event was generated otherwise this field may hold information of the generated event For instance an application can manually generate a HM event specifying this information 2 13 5 Multicore implications Health monitor management has not visibility on the VCPUs internal to the partitions
147. n and the hypervisor health monitoring table SystemDescription ResidentSw This is an optional element which for providing information to XtratuM about the resident software SystemDescription PartitionTable This is a container element which holds all the partition elements SystemDescription Channels A sequence of channels which define port connections SystemDescription HwDescription Contain the configuration of physical and virtual resources 8 3 3 The SystemDescription XMHypervisor element There are two optional attributes console and healthMonitoringDevice The values of these attributes shall be the name of a device defined in the SystemDescription HwDescription Devices section Mandatory elements PhysicalMemoryAreas Sequence of memory areas allocated to XtratuM Optional elements HealthMonitoring Contains a sequence of health monitoring event elements Not all HM actions can be associated with all HM events Consult the allowed actions in the Volume 4 Reference Manual xm 4 usermanual 047d A ic emu MM Printed November 26 2012 3130 3135 3140 3145 3150 3155 3160 A 112 134 Chapter 8 Configuration itianlId Y duration ims lid Mistart ias A ck name Tm eme E DERE x40150000 oryllock iname MenDiszkA irire 234KD start 0x402000001 nase Partitions flags syste console Uart lid Y PhyzicalMmecryAreas Arsa lflags uncachoable sire 512K start
148. n I O port see section 5 10 When a device is used by several partitions an user implemented I O server partition figure 2 15b may be in charge of the device management An I O server partition is a specific partition which xm 4 usermanual 047d Ah ic emu P Printed November 26 2012 805 810 815 820 825 830 865 32 134 Chapter 2 XtratuM Architecture accesses and controls the devices attached to it and exports a set of services via the inter partition communication mechanisms provided by XtratuM sampling or queuing ports enabling the rest of partitions to make use of the managed peripherals The policy access priority FIFO etc is imple mented by the I O server partition Note that the I O server partition is not part of XtratuM It should if any be implemented by the user of XtratuM XtratuM does not force any specific policy to implement I O server partitions but a set of services to implement it 2 15 Traps interrupts and exceptions 2 15 1 Traps A trap is the mechanism provided by the LEON3 processor to implement the asynchronous transfer of control When a trap occurs the processor switches to supervisor mode and unconditionally jumps into a predefined handler SPARC v8 defines 256 different trap handlers The table which contains these handlers is called trap table The address of the trap table is stored in a special processor register called tbr Both the tbr and the contents of the
149. nationId 1 2 gt 900 XIpvi id 1 sourceId 0 destinationId 2 gt lt Channels gt Listing 2 4 IPVI route specification It defines that XM VT EXT IPVIO Ipvi identifier 0 can be generated by partition with identifier O and it will be received by partitions with identifier 1 and 2 Additionally the same partition 0 can raise os XM_VT_EXT_IPVI1 Ipvi with identifier 1 that can be received by partition 2 2 15 4 Multicore implications Interrupts are allocated to the partitions in the configuration file In multicore operating system it is responsible of attach the interrupts to the CPUs In the same way the guest OS is responsible of the assignement of the interrupts allocated to it to the appropriated VCPU so The following interrupt symbols for LEON4 has been added Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 2 16 Traces 35 134 define GPTIMER UO T1 NR define GPTIMER UO T2 NR define GPTIMER UO T3 NR define GPTIMER UO T4 NR define GPTIMER UO T5 NR define GPTIMER U1 NR 6 define GPTIMER_U2_NR 7 define GPTIMER_U3_NR 8 define GPTIMER_U4_NR 9 define IRQ_AMP_NR 10 define GRPCIDMA_NR 11 define GRGPIOO_NR 16 define GRGPIO1_NR 17 define GRGPIO2_NR 18 define GRGPIO3_NR 19 define SPWROUTERO_NR 20 define SPWROUTER1 NR 21 define SPWROUTER2_NR 22 define SPWROUTER3_NR 23 define GRETH_GBITO_NR 24 define GRETH_GBIT1_NR 25 define AHBSTAT_NR 27 define MEMSCRUBL2CACHE_N
150. ne XM EXEC CLOCK 0x1 Listing 5 25 core include hypercalls h XtratuM provides one timer for each clock The timers can be programmed in one shot or in periodic mode Upon expiration the extended interrupts XM VT EXT HW TIMER and XM VT EXT EXEC TIMER are triggered These extended interrupts correspond with traps 256 XM VT EXT HW TIMER and 256 XM VT EXT EXEC TIMER respectively 5 12 1 Execution time clock The clock XM EXEC CLOCK only advances while the partition is being executed or while XtratuM is exe cuting a hypercall requested by the partition The execution time clock computes the total time used by the target partition This clock relies on the XM HW CLOCK and so its resolution is also 1ysec Its precision is not as accurate as that of the XM HW CLOCK due to the errors introduced by the partition switch The execution time clock does not advance when the partition gets idle or suspended Therefore the XM EXEC CLOCK clock should not be used to arm a timer to wake up a partition from an idle state The code below computes the temporal cost of a block of code include lt xm h gt include std_c h void PartitionMain xmTime_t ti t2 XM_get_time XM_EXEC_CLOCK amp t1 code to be measured XM_get_time XM_EXEC_CLOCK amp t2 xprintf Initial time lld final time 411d t1 t2 xprintf Difference 4lldNn t2 t1 XM halt partition XM PARTITION SELF 5 13 Processor man
151. ne grain error detection and a coarse grain fault management e It is possible to implement advanced fault analysis techniques in system partitions e An I O Server partition can handle a set of devices used by several partitions e XtratuM implements a highly configurable health monitoring and handling system 1000 e The logs reported by the health monitoring system can be retrieved and analysed by a system partition online xm 4 usermanual 047d A ic emu MM Printed November 26 2012 38 134 Chapter 2 XtratuM Architecture e XtratuM provides a tracing service that can be used to both debug partitions and online monitor ing 1005 e The same tracing mechanism is used to handle partition and XtratuM traces Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d Volume 2 XtratuM User Manual Chapter 3 Developing Process Overview XtratuM is a layer of software that extends the capabilities of the native hardware There are important differences between a classical system and a hypervisor based one This chapter provides an overview of the XtratuM developing environment The simplest scenario is composed of two actors the integrator and the partition developer or partition supplier There shall be only one integrator team and one or more partition developer teams in what follows we will use integrator and partition developer for short The tasks to be done by the integrator are 1 Configure the XtratuM
152. ngth B int 16 Hypervisor Enable voluntary preemption support boo n Kernel stack size KB int 8 MMU Drivers Reserve UARTI bool n Reserve UART2 bool n Enable early output bool n CPU frequency MHz int Early UART baudrate int Select early UART port choice UART1 UART2 Enable UART flow control bool n DSU samples UART port bool n Objects Verbose HM events boo y Enable emulation of application HM events bool y Enable XM partition status accounting bool n Enable partition id symbol on console write bool n Sparc cpu Processor model Board Enables the specific board features write protection units timers UART and interrupt con troller SPARC memory protection schema Select the processor mechanism that will be used to implement memory protection If MMU is available then it is the best choice With WPR only write protection can be enforced Enable cache If selected the processor cache is enabled All partitions will be executed with cache enabled unless explicitly disabled on each partition through the XM CF file If this option is not selected the cache memory is disabled Neither XtratuM nor the partitions will be able to use the cache Flush cache after context switch Forces a cache flush after a partition context switch DSU samples UART port If the UART port used to print console messages is also used by the DSU Debugging Support Unit then this option shall be set If the DSU is used then the control bits of the UAR
153. ns for a printed book the title page itself plus such following pages as are needed to hold legibly the material this License requires to appear in the title page For works in formats which do not have any title page as such Title Page means the text near the most prominent appearance of the work s title preceding the beginning of the body of the text A section Entitled XYZ means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language Here XYZ stands for a specific section name mentioned below such as Acknowledgements Dedications Endorsements or History To Preserve the Title of such a section when you modify the Docu ment means that it remains a section Entitled XYZ according to this definition The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document These Warranty Disclaimers are considered to be included by reference in this License but only as regards disclaiming warranties any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License 2 VERBATIM COPYING You may copy and distribute the Document in any medium either commercially or noncommercially provided that this License the copyright notices and the license notice saying this License applies to the Document are rep
154. ns one or more segments A segment is a block of data that shall be copied in a contiguous area of memory when loaded in main memory The content of the XEF can optionally be compressed An XEF file has a header and a set of segments The segments corresponds to the allocatable sec tions of the source ELF file In the header there is a reserved area 16 bytes to store user defined information This information is called user payload build A new XEF file is created using file as input m The source file is not an ELF file but a user defined customisation In this case no consistency checks are performed Customisation files are used to attach data to the partitions See the xmpack command This data will be accessible to the partition at boot time It is commonly used as partition defined run time configuration parameters o file Places output in file file The XEF segments are compressed using the LSZZ algorithm p file The first 16 bytes of the file are copied into the payload area of the XEF header The size of the file shall be at least 16 bytes otherwise an error is returned The MD5 sum value is printed if no errors read Shows the contents of the XEF file h Print the content of the header Lists the segments and its attributes Lists the table of custom files This options only works for partition and hypervisor XEF files USAGE EXAMPLES Create a customisation file xmeformat build m o custo
155. o o e ee 117 9 1 1 xmcparsef canoso a e as 118 9 2 ELF to XEF xmeformat seces titi canada a a S 3 3 3s 118 922 1 xime format ssa eeue le sers el a Sos URL uela 118 9 3 Container builder xm pack gt saa arrear ereet se c e i 120 OL xHpack suu ke be eat GE d die A REC ded ee wey 120 Printed November 26 2012 Ah ird uPA xm 4 usermanual 047d Contents ix 134 9 4 Bootable image creator rswbuild 2222s 122 9 41 rswbuild s cesa tod rra m ee TA DOSE RSS 122 10 Security issues 123 10 1 Invoking a hypercall from libXM 2 ee 123 10 2 Preventing covert side channels due to scheduling slot overrun 123 A XML Schema Definition 125 GNU Free Documentation License 127 T APPEICABILITY AND DEFINITIONS 4 2 4 xke d 424 84420484044 3544444 127 2 VERBATIM COPYING 5 31 a A eae OE a a we ee ee ee ea A 128 32 COPYING IN QUANTITY sos ica gaenc 6b ERR are PHS OE EW eee Bed ee a ee gs 129 4 MODIFICATIONS oia e eae wee eR A GL Geared 129 5 COMBINING DOCUMENTS xoc 064 444 ROSE EO VEU EORR ey x Roy Ruso US Ue 130 6 COLLECTIONS OF DOCUMENTS lee eer 131 7 AGGREGATION WITH INDEPENDENT WORKS llle 131 S TRANSLATION EM itis RU RUN ib ius Ae Dee c RA ee erts a 131 O TERMINATION 44er eer ck ORO o e v C ERU GG ED 131 10 FUTURE REVISIONS OF THIS LICENSE 0 csse e ori nEn 132 ADDENDUM How to use this License for your documents enn 132 Glossary of Te
156. o store trace messages and what messages shall be traced Since XM CF defines the resources allocated to each partition this file represents a contract between the integrator and the partition developers A partner the integrator or any of the partition developers should not change the contents of the configuration file on its own All the partners should be aware of the changes and should agree in the new configuration in order to avoid problems later during the integration phase XM CF xml Figure 3 4 Graphical representation of an XML configuration file In order to reduce the complexity of the XtratuM hypervisor the XM CF is parsed and translated into a binary format which can be directly used by XtratuM The XML data is translated into a set of initialised data structures ready to be used by XtratuM Otherwise XtratuM would need to contain a XML parser to read the XM CF information See section 9 1 1 The resulting configuration binary is passed to XtratuM as a customisation file 3 4 Compiling partition code Partition developers should use the XtratuM user library named 1ibxm a which has been generated during the compilation of the XtratuM source code to access the para virtualised services The resulting binary image of the partition shall be self contained that is it shall not contain linking information The ABI of the partition binary is described in section 6 In order to be able to run the partition app
157. onvention in order to invoke the XtratuM hypercalls The register assignment convention for calling a hypercall is 00 Holds the hypercall number 01 05 Holds the parameters to the hypercall 2355 Once the processor registers have been loaded a ta instruction to the appropriate software trap number shall be called see section 6 2 The return value is stored in register 00 For example following assembly code calls the XM get time xm u32 t clockId xmTime t time mov Oxa 400 GET TIME NR mov i0 01 mov i1 02 ta OxfO cmp 00 O XM OK bne error In SPARC v8 the get time nr constant has the value Oxa i0 holds the clock id and i1 is a pointer which points to a xmTime t variable The return value of the hypercall is stored in 00 and 2360 then checked if XM OK Below is the list of normal hypercall number constants listing 5 19 and assembly hypercalls list ing 5 19 define __MULTICALL_NR O 2365 define __HALT_PARTITION_NR 1 define __SUSPEND_PARTITION_NR 2 define __RESUME_PARTITION_NR 3 define __RESET_PARTITION_NR 4 define __SHUTDOWN_PARTITION_NR 5 2370 define __HALT_SYSTEM_NR 6 define __RESET_SYSTEM_NR 7 define __IDLE_SELF_NR 8 define __GET_TIME_NR 9 2375 define __SET_TIMER_NR 10 define __READ_OBJECT_NR 11 define __WRITE_OBJECT_NR 12 define __SEEK_OBJECT_NR 13 define __CTRL_OBJECT_NR 14 2380 define __CLEAR_IRQ_MASK_NR 15
158. order to avoid race conditions between the internal threads of XtratuM 7 2 1 Monocore partition booting XtratuM provides virtual CPUs to the partitions A monocore partition will use the virtual CPU identifed as vCPUO Its operation is exactly the same as the monocore version of XtratuM After a partition reset the vCPUO is initialised to the default values specified in the configuration file Although the monocore partition uses the vCPUO it can be allocated to any of the available 7 2 2 Multicore partition booting Multicore partition can use several virtual CPUs vCPUO vCPU1 vCPU2 to implement the partition XtratuM follows the approach for virtual CPUs than the hardware provides At partition boot XtratuM only starts vCPUO for the partition It is responsability of the partition code in the initialised vCPUO thread to start the execution of the additional cores An important aspect to be considered is that the virtual CPUs are local to each partition It means each partition handles its virtual CPUs which are completely hidden to other partitions In order to handle virtual CPUs XtratuM provides some services hypercalls to partitions to handle its virtual CPUs These hypercall are Printed November 26 2012 D tir eru PA xm 4 usermanual 047d 7 2 Booting process in LEON4 103 134 Start a virtual CPU the hypercall XM reset vcpuO permits to start the execution of a virtual CPU identified by vcpuld in the call
159. ough to attend the request This error is typically handled by the program by checking the return value As for the second kind an attempt to execute an undefined instruction processor instruction may not be properly handled by a program that attempted to execute it Printed November 26 2012 A ir amu PA xm 4 usermanual 047d 2 13 Health monitor HM 27 134 The XtratuM health monitoring system will manage those faults that cannot or should not be man aged at the scope where the fault occurs The XtratuM HM system is composed of four logical blocks HM event detection to detect abnormal states using logical probes in the XtratuM code HM actions a set of predefined actions to recover a fault or confine an error HM configuration to bind the occurence of each HM event with the appropriate HM action HM notification to report the occurrence of the HM events Since HM events are by definition the result of a unexpected behaviour of the system it may be difficult to clearly determine which is the original cause of the fault and so what is the best way to handle the problem XtratuM provides a set of coarse grain actions see section 2 13 2 that can be used at the first stage right when the fault is detected Although XtratuM implements a default action for each HM event the integrator can map an HM action to each HM event using the XML configuration file Once the defined HM action is carried out by XtratuM a HM
160. partition needs to 1 have access to the peripheral control and data registers 2 be informed about triggered interrupts 3 be able to block mask and unmask the associated interrupt line Printed November 26 2012 tir aru PA xm 4 usermanual 047d 2 15 Traps interrupts and exceptions 33 134 Partition Exceptions TrapHandlerN handled throw the health monitor Hu VirapTable ras Exceptions Switaps T vires ___ XM_CF xml oa Tm o Y Y XM hm Parition hm Software Traps _ Table _ Table Extended Irqs Health Monitoring Scope Hardware Irqs XtratuM Irq Controller Dr Peripherals Figure 2 15 Exceptions handled by the health monitoring subsystem A hardware interrupt can only be allocated to one partition in the XM CF configuration file The partition can then mask and unmask the hardware line in the native interrupt controller by using the XM mask irqO and XM unmask irq functions XtratuM extends the concept of processor traps by adding 32 additional interrupts This new range is used to inform the partition about events detected or generated by XtratuM 870 Figure 2 16 shows the sequence from the occurrence of an interrupt to the partition s trap handler Partition Hardware interrupt Extended cre puend tern et delivered to interrupts partition Partitio
161. pedef struct Current state of the partition ready suspended xm u32 t state define XM STATUS IDLE OxO define XM STATUS READY 0x1 2205 define XM STATUS SUSPENDED 0x2 define XM STATUS HALTED 0x3 Number of virtual interrupts received xm u64 t noVIrqs OPTIONAL 2210 Reset information xm u32 t resetCounter xm u32 t resetStatus xmTime t execClock Total number of partition messages 2215 xm u64 t noSamplingPortMsgsRead OPTIONAL xm u64 t noSamplingPortMsgsWritten OPTIONAL xm u64 t noQueuingPortMsgsSent OPTIONAL xm u64 t noQueuingPortMsgsReceived OPTIONAL xmPartitionStatus t 2220 Listing 5 28 core include objects status h typedef struct xm_u32_t resetCounter Number of HM events emitted 2225 xm u64 t noHmEvents OPTIONAL Number of HW interrupis received xm u64 t nolrqs OPTIONAL Current major cycle interation xm u64 t currentMaf OPTIONAL 2230 Total number of system messages xm u64 t noSamplingPortMsgsRead OPTIONAL xm u64 t noSamplingPortMsgsWritten OPTIONAL xm u64 t noQueuingPortMsgsSent OPTIONAL xm 4 usermanual 047d A ic emu PA Printed November 26 2012 2235 2240 2245 2250 2255 2260 2265 2270 78 134 Chapter 5 Partition Programming xm u64 t noQueuingPortMsgsReceived OPTIONAL xmSystemStatu
162. plicit permission from the previous publisher that added the old one The author s and publisher s of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version 5 COMBINING DOCUMENTS You may combine the Document with other documents released under this License under the terms defined in section 4 above for modified versions provided that you include in the combination all of the Invariant Sections of all of the original documents unmodified and list them all as Invariant Sections of your combined work in its license notice and that you preserve all their Warranty Disclaimers Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 3725 131 134 The combined work need only contain one copy of this License and multiple identical Invariant Sections may be replaced with a single copy If there are multiple Invariant Sections with the same name but different contents make the title of each such section unique by adding at the end of it in parentheses the name of the original author or publisher of that section if known or else a unique number Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work In the combination you must combine any sections Entitled History in the various original docu ments forming one section Entitled History likewise combine any
163. portance criticality of the event that motivated the trace message Next the intended use of it can be of the following levels XM TRACE NOTIFY A notification messages of the progress of the application at coarse grained level XM TRACE DEBUG A detailed information message intended to be used for debugging during the development phase XM TRACE WARNING Traces which inform about potentially harmful situations XM TRACE UNRECOVERABLE Traces of this level are managed also by the health monitoring sub system a user HM event is generated and handled according to the HM configuration Note that both the normal partition trace message is stored and the HM event is generated Jointly with the opCode and the user data the XM trace event function has a bitmask parameter that is used to filter out trace events If the logical AND between the bitmask parameter and the bitmask of the XM CF configuration file is not zero then the trace event is logged otherwise it is discarded Traces of XM TRACE UNRECOVERABLE critically always raises a health monitoring event regarding the bitmask 5 14 2 Reading traces Only one system partition can read from a trace stream A standard partition can not read its own trace messages it is only allowed to store traces on it If the trace stream is stored in a buffer RAM or FLASH When the buffer is full the oldest events are overwritten 5 14 3 Configuration XtratuM statically allocates a block of me
164. provided through hypercalls e Partitions A partition is an execution environment managed by the hypervisor which uses the virtualised services Each partition consists of one or more concurrent processes concurrency must be implemented by the operating system of each partition because it is not directly sup ported by XtratuM that share access to processor resources based upon the requirements of the application The partition code can be an application compiled to be executed on a bare machine a real time operating system or runtime support and its applications or a general purpose operating system and its applications Partitions need to be virtualised to be executed on top of a hypervisor Depending on the type of execution environment the virtualisation implications in each case can be summarised as Bare application The application has to be virtualised using the services provided by XtratuM The application is designed to run directly on the hardware so the the provided and non provided services of XtratuM must be taken into account Mono core Operating system application When the application runs on top of a real time operat ing system it uses the services provided by the operating system and does not need to be directly virtualised However the operating system has to deal with the virtualisation and be virtualised ported on top of XtratuM Multi core Operating system application A guest OS can use multiple virtual C
165. r is saved in a global C variable general purpose registers are cleared memory access rights are cleared PSR WIM TBR and Y processor control registers are initialized sets up a working stack and jumps to C code code kernel setup c The setup O function carries out the following actions Initializes the internal console Initializes the interrupt controller Detects the processor frequency information extracted from the XML configuration file De e der p Initializes memory manager enabling XtratuM to keep track of the use of the physical mem ory 2PSR Processor Status Register 3WIM Window Invalid Mask 4TBF Trap Base Register 5Y Extended data register for some arithmetic operations Printed November 26 2012 A ir au PA xm 4 usermanual 047d 7 2 Booting process in LEON4 101 134 Initializes hardware and virtual timers Initializes the scheduler Initializes the communication channels Booting partitions are set in NORMAL state and non booting ones are set in HALT state 9 go s gx 1 Finally the setup function calls the scheduler and becomes into the idle task Partition code is executed according to the configured plan A system partition can load from PROM or other source serial line etc the image of other partitions 6 The new ready partition is activated via a XM reset partition service 7 2 Booting process in LEON4 In the standard boot procedure of the L
166. rations make distclean 3 In the root directory of XtratuM copy the file xmconfig sparc into xmconfig and edit it to meet your system paths The variable XTRATUM PATH shall contain the root directory of XtratuM Also s if the sparc linux toolchain directory is not in the PATH then the variable TARGET CCPREFIX shall 45 134 1160 1165 1170 1175 1180 1185 46 134 Chapter 4 XtratuM Configuration and Build contain the path to the actual location of the corresponding tools The prefix of the SPARC v8 tool chain binaries shall be sparc 1inux otherwise edit also the appropiate variables In the seldom case that the host toolchain is not in the PATH then it shall be specified in the HOST CCPREFIX variable cp xmconfig sparc xmconfig vi xmconfig Configure the XtratuM sources The ncurses5 library is required to compile the configuration tool In a Debian system with internet connection the required library can be installed with the following command sudo apt get install libncurses5 dev The configuration utility is executed compiled and executed with the next command make menuconfig Note The menuconfig target configures the XtratuM source code the resident software and the XAL environment Therefore two different configuration menus are presented a The XtratuM itself b The Resident Software which is charge of loading the system from ROM RAM c The
167. rcall divided by the clock resolution A rough estimation supposing that the maximum message length is 4096 Bytes is 4 bits at each partition context switch In the case of a side channel the bandwidth is drastically reduced due to the uncertainty randomness introduced by the execution of the target partition There are several strategies to address this issue 1 At integrator level design a scheduling plan that lest some idle time before the end of a slot and the beginning of the next one The Xoncrete scheduling tool is able to implement that solution automatically Figure 10 1 shows two scenarios scenario 1 where the integrator has not left spare time between one partition slot and the next one enabling the partition in light grey overrunning the start of the dark gray one Scenario 2 sketches the same case but leaving spare time between one slot and the next one So in this case execution overruns can not occur 2 At partition level stop invoking hypercalls some time before the end of the slot This way there will be no hypercalls being executed when the slot end occurs so the next partition will start always with no delay 3 At hypervisor level a Change the design of XtratuM to convert it preemptable Hypercalls would be interrupted when the end of the slot is reached and later resumed when the partition is active again b Implement the partial preemptability in XtratuM voluntary preemption XtratuM is by default atom
168. ressed image length 32928 48 06 9 3 Container builder xmpack 9 3 1 xmpack Create an XtratuM system image container Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 3435 9 3 Container builder xmpack 121 134 SYNOPSIS xmpack build h xm file Ooffset conf file Ooffset p id part_file offset custom file 20ffset container xmpack list c container DESCRIPTION xmpack manipulates the XtratuM system container The container is a simple filesystem designed to contain the XtratuM hypervisor core and zero or more XEF files The container is an envelope to deploy all the system hypervisor and partitions from the host to the target At boot time the resident software is in charge of reading the contents of the container and coping the components to the RAM areas where the hypervisor and he partitions will be executed Note that XtratuM has no knowledge about the container structure The container is organised as a list of components Each component is a list of XEF files A component is used to store an executable unit which can be the XtratuM hypervisor or a partition Each compo nent is a list of one or more files The first file shall be a valid XtratuM image see the XtratuM binary file header with the configuration file once parsed and compiled into XEF format The rest of the components are optional xmpack is a helper utility that can be used to deploy an XtratuM system It is not m
169. riant Sections and required Cover Texts given in the Document s license notice H Include an unaltered copy of this License I Preserve the section Entitled History Preserve its Title and add to it an item stating at least the title year new authors and publisher of the Modified Version as given on the Title Page If there is no section Entitled History in the Document create one stating the title year authors and publisher of the Document as given on its Title Page then add an item describing the Modified Version as stated in the previous sentence J Preserve the network location if any given in the Document for public access to a Transparent copy of the Document and likewise the network locations given in the Document for previous versions it was based on These may be placed in the History section You may omit a network location for a work that was published at least four years before the Document itself or if the original publisher of the version it refers to gives permission K For any section Entitled Acknowledgements or Dedications Preserve the Title of the section and preserve in the section all the substance and tone of each of the contributor acknowledge ments and or dedications given therein L Preserve all the Invariant Sections of the Document unaltered in their text and in their titles Section numbers or the equivalent are not considered part of the section titles M Delete any s
170. ring logs or the console output of a partition Below is the list of attributes of the Block element 1Some key processor registers needed to guarantee the spatial isolation are mapped memory addresses which are not moni tored protected by the write protection mechanism The workaround consists in protecting this register area using the watchpoint mechanism The workaround is only applicable if the watchpoint facility is present 2The large number of nested elements is for future compatibility with multiple plans and scheduling policies xm 4 usermanual 047d A ic emu PA Printed November 26 2012 3165 3170 3175 3180 3185 3190 3195 3205 3210 3215 3220 3225 114 134 Chapter 8 Configuration MemoryBlockTable Block name Required Name which identifies the device This name is only used to refer this device in the configuration file Once compiled the configuration file this name is removed MemoryBlockTable Block start Required Starting address of the memory block MemoryBlockTable Block size Required Size of the memory block 8 3 5 The SystemDescription ResidentSw element The element PhysicalMemoryAreas is used to declare the memory areas where the resident soft ware will by located This information is included in the configuration file for completeness all the memory areas of the board shall be described in the configuration file and used only to check memory overlaps errors A
171. rmal ready running idle XM halt partition or XM HM AC HALT XM suspend partition or XM HM AC SUSPEND Figure 2 8 Partition states and transitions On start up each partition is in boot state It has to prepare the virtual machine to be able to run the applications it sets up a standard execution environment that is initialises a correct stack and sets up the virtual processor control registers creates the communication ports requests the hardware devices I O ports and interrupt lines etc that it will use Once the partition has been initialised it changes to normal mode The partition receives information from XtratuM about the previous executions if any From the hypervisor point of view there is no difference between the boot state and the normal state In both states the partition is scheduled according to the fixed plan and has the same capabilities Although not mandatory it is recommended that the partition emits a partition s state change event when changing from boot to normal state The normal state is subdivided in three sub states Ready The partition is ready to execute code but it is not scheduled because it is not in its time slot Running The partition is being executed by the processor Idle If the partition does not want to use the processor during its allocated time slot it can relinquish the processor and wait for an interrupt or for the next time slot see XM idle self
172. rms and Acronyms 133 xm 4 usermanual 047d Ah iram uPA Printed November 26 2012 This page is intentionally left blank Volume 2 XtratuM User Manual Preface The target readers for this document are software developers who need to use the services of XtratuM directly The reader is expected to have an in depth knowledge of the LEON3 SPARC v8 architecture and experience in programming device drivers It is also advisable to have some knowledge of the ARINC 653 and other related standards Typographical conventions The following typographical conventions are used in this document e typewriter used in assembler and C code examples and to show the output of commands e italic used to introduce new terms e bold face used to emphasize or highlight a word or paragraph Code Code examples are printed inside a box as shown in the following example static inline void XM sparc set psr xm u32 t flags 1 asm volatile mov TO STR sparc set psr nr 00 n t mov 40 AoiNnaNt N __DO_XMAHC r flags oO o1 Listing 1 Sample code Caution sign The caution sign stresses information that is critical to the integrity or continuity of the system Acknowledgements The porting to LEON4 multicore has been achieved in the framework of the ESA Project ESA ITT 6185 System Impact of Distributed Multicore systems leaded by Astrium EADS xi 134 This page is intentionally left blank
173. rnel which basically consists in extending and enforcing the isolation between processes or a group of processes ARINC 653 defines both the API and operation of the partitions but also how the threads or processes are managed inside each partition It provides a complete APEX In a bare metal hypervisor and in particular in XtratuM a partition is a virtual computer rather than a group of strongly isolated processes When multi threading or tasking support is needed in a partition then an operating system or a run time support library has to provide support to the application threads In fact it is possible to run a different operating system on each XtratuM partition It is important to point out that XtratuM is a bare metal hypervisor with extended capabilities for highly critical systems XtratuM provides a raw close to the native hardware virtual execution envi ronment rather than a full featured one Therefore although XtratuM by itself can not be compat ible with the ARINC 653 standard the philosophy of the ARINC 653 has been employed when applicable This document is organised as follows Chapter 2 describes the XtratuM architecture describing how the partitions are organised and sched uled also an overview of the XtratuM services is presented Chapter 3 outlines the development process on XtratuM roles elements etc 1 134 35 40 45 50 2 134 Chapter 1 Introduction Chapter 4 describes the compil
174. roduced in all copies and that you add no other conditions whatsoever to those of this License You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute However you may accept compensation in exchange for copies If you distribute a large enough number of copies you must also follow the conditions in section 3 You may also lend copies under the same conditions stated above and you may publicly display copies Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 3640 129 134 3 COPYING IN QUANTITY If you publish printed copies or copies in media that commonly have printed covers of the Document numbering more than 100 and the Document s license notice requires Cover Texts you must enclose the copies in covers that carry clearly and legibly all these Cover Texts Front Cover Texts on the front cover and Back Cover Texts on the back cover Both covers must also clearly and legibly identify you as the publisher of these copies The front cover must present the full title with all words of the title equally prominent and visible You may add other material on the covers in addition Copying with changes limited to the covers as long as they preserve the title of the Document and satisfy these conditions can be treated as verbatim copying in other respects If the required texts for either cover are too voluminous to fit legibly you should put the fir
175. rom boot to normal state is performed automatically the last action of the set up procedure The system can switch to halt state by the health monitoring system in response to a detected error or by a system partition invoking the service XM halt system O In the halt state the scheduler is disabled the hardware interrupts are disabled and the processor enters in an endless loop The only way to exit from this state is via an external hardware reset It is possible to perform a warm or cold hardware reset system reset by using the hypercall see XM reset system On a warm reset the system increments the reset counter and a reset value is passed to the new rebooted system On a cold reset no information about the state of the system is passed to the new rebooted system 2 3 2 System states Table 2 7 lists the system services Service Description XM get system status Get the system status XM halt system Halt the system XM reset system Reset the system Figure 2 7 System services Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 2 4 Partition states and services 11 134 2 4 Partition states and services 2 4 1 Partition states Once XtratuM is in normal state partitions are started The partition s states and transitions are shown in figure 2 8 XM HM AC PARTITION COLD RESET or XM HM AC PARTITION WARM RESET or XM reset partition XM resume partition No
176. rotection Registers most of the may not be See section 5 16 for more details Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 2 11 IO MMU support 23 134 2 11 IO MMU support LEON4 processor implements an io mmu in order to enable logic partitioning of io resources any partition access to IO resources is translated through the mmu allowing an effective spatial isolation Nevertheless since DMA accesses were not translated its use permitted to break spatial isolation a partition could program a DMA transference overwriting or retrieving the content of area belonging to another partition or the hypervisor The io mmu is a mmu interposed between master AHB devices and slaves ones enabling an effective real spatial isolation XtratuM allows to the system integrator to make use of the io mmu through the xml configuration file More concretely the element SystemDescription Hypervisor IoMmu permits to define a table in order to link one or a set of AHB master buses to a partition A maximum of 6 of such links are allowed restriction imposed by the LEON4 implementation Additionally the system integrator must define the bus to use for accesses by the AHB master ObusRouting which can be either processor or memory and the vendor and device PNP Id of the master Eventually in order to avoid huge io memory tables XtratuM implements e The lowest possible ITR area protected size Taken into account that the upper part sh
177. ryadistribution se oo e o e 47 4 4 Installing a binary distribution ooo e 49 4 5 Compile the Hello World partition o oo e eee 50 Printed November 26 2012 Ah ira uPA xm 4 usermanual 047d Contents vii 134 46 XtratuM directory tree Lv xo a A ee oe OPE X PUR URN RUSO SR e 51 5 Partition Programming 53 5 1 Partition definition o ou 3 3 G6 asada aa AR 53 5 2 Implementation requirements 2 eer 53 5 3 XAL development environment ellen 55 5 44 The Hello World example oses tooted d 9o RR a RO Ww Ex XXE 57 5 41 Included headers 3 555 ew EX Lo GUESS 64 5 4 2 The Hello World example in multicore o ooo 65 5 5 Parttioh eset a a A A A A S Ts 65 5 6 System TeseLs aee ee Res Gem VOR oce oue God Ue A ae RR d 65 5 7 scheduling 4 4 pd 2b 8449 EG dub SUE GE RC bo ade x 66 5 7 1 Slotidentificatlon lt e Rex e SSS Oe Oe eoe e RR o e SS Oa XR 66 5 7 2 Managing scheduling plans s i corate aa e e Row ox oy ss 66 5 8 Console Output x sce ood eR eR RR x A e RODA cs EORR A E OR S UR ee 66 5 9 Inter partition coinriuhicatiOn ss ss e sue Fk n o a m RESTI n 67 5 9 1 Message notification zese lere 68 5 10 Peripheral programming omic ono ey Ro eoe Roe ew ROUX 68 5 11 Traps interrupts and exceptions 4e s s RE sda S RR gom ee NON E n 69 NIC A 69 5 11 2 Interr pts 4 3 a a eu so eu e Ue d d md x 70 S13 EXCepulons s o 4
178. s are performed by two hypercalls XM send queuing message O and XM receive queuing message O respectively XtratuM implements a classical producer consumer circular buffer without blocking The sending operation writes the message from parti tion space into the circular buffer and the receive one performs a copy from the XtratuM circular buffer into the destination memory If the requested operation cannot be completed because the buffer is full when trying to send a message or empty when attempting to receive a message then the operation returns im mediately with the corresponding error The partition s code is responsible for retrying the operation later Queuing ports can specify a policy FIFO or PRIORITY for the source or destination XtratuM does not handle this attribute but provides the attribute to the partitions in order to be handled by the guest operating system to wake blocked threads in write or read operations on the port under any of these disciplines 3XtratuM defines the maximum length of a message xm 4 usermanual 047d A ic emu P Printed November 26 2012 615 620 625 630 635 640 645 650 655 660 665 670 675 680 685 690 695 700 26 134 Chapter 2 XtratuM Architecture In order to optimise partition s resources and reduce the performance loss caused by polling the state of the port XtratuM triggers an extended interrupt when a new message is written sent to a
179. s t Listing 5 29 core include objects status h The field execClock of a partition is the execution time clock of the target partition The rest of the fields are self explained Those fields commented as OPTIONAL contain valid data only if XtratuM has been compiled with the flag Enable system partition status accounting enabled 5 16 Memory management XtratuM implements a flat memory space on the SPARC v8 architecture LEON2 and LEON3 proces sors The addresses generated by the control unit are directly emitted to the memory controller without any translation Therefore each partition shall be compiled and linked to work on the designated memory range The starting address and the size of each partition is specified in the system configu ration file Two different hardware features can be used to implement memory protection Write Protection Registers WPR In the case that there is no MMU support then it is possible to use the WPR device of the LEON2 and LEON3 processors The WPR device can be programmed to raise a trap when the processor tries to write on a configured address range Since read memory operations are not controlled by the WPR it is not possible to enforce com plete read write memory isolation in this case Also due to the internal operation of the WPR device all the memory allocated to each partition has to be contiguous and has to meet the following conditions e The size shall be greater than or
180. s than one half of the entire aggregate the Document s Cover Texts may be placed on covers that bracket the Document within the aggregate or the electronic equivalent of covers if the Document is in electronic form Otherwise they must appear on printed covers that bracket the whole aggregate 8 TRANSLATION Translation is considered a kind of modification so you may distribute translations of the Document under the terms of section 4 Replacing Invariant Sections with translations requires special permission from their copyright holders but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections You may include a translation of this License and all the license notices in the Document and any Warranty Disclaimers provided that you also include the original English version of this License and the original versions of those notices and disclaimers In case of a disagreement between the translation and the original version of this License or a notice or disclaimer the original version will prevail If a section in the Document is Entitled Acknowledgements Dedications or History the require ment section 4 to Preserve its Title section 1 will typically require changing the actual title 9 TERMINATION xm 4 usermanual 047d AX ic emu PA Printed November 26 2012 3685 3690 3695 3700 3705 3710 3715 3720 132 134 Appendix A
181. section noPartitions Number of partitions plus one XtratuM is also a component in the container 2865 componentOffset The offset relative to the start of the file to the component s array section fileTabOffset The offset relative to the start of the container file to the files s array section noFiles Number of files XtratuM core the XM CT file partition binaries and partition customization files in the container strTabOffset The offset relative to the start of the container file to the strings table 2870 strLen The length of the strings table This section contains all names of the files fileData 0ffset The offset relative to the start of the container file to the file data section fileDataLen The length of the file data section This section contains all the contents of all the components Each entry of the partition table section describes all the XEF files that are part of each partition s Which contains the following fields struct xmefPartition xm s32 t id xm s32 t file 2880 xm u32 t noCustomFiles xm s32 t customFileTab CONFIG MAX NO CUSTOMFILES PACKED Listing 6 13 core include xmef h id The identifier of the partition 2885 file The index into the file table section of the XEF partition image noCustomFiles Number of customisation files of this component customFileTab List of custom file indexes The metadata of each file is store in the file table section xm 4
182. sections contain material for more advanced users The libxm a library provides a friendly interface that hides most of the low level details explained in this chapter 6 1 Data representation The data types used in the XtratuM interfaces are compiler and machine cross development indepen A dent This is specially important when manipulating the configuration files XtratuM conforms the following conventions 2495 Unsigned Signed Size bytes Alignment bytes xm_u8_t xm_s8_t 1 1 xmui6_t xms i6t 2 4 xm u32t xm s32t 4 4 xm u64t xm s64t 8 8 Table 6 1 Data types These data types have to be stored in big endian format that is the most significant byte is the rightmost byte 0x 00 and the least significant byte is the leftmost byte 0x 03 C declaration which meet these definitions are presented in the list below Basic types 2500 typedef unsigned char xm u8 t define MAX U8 OxFF typedef char xm s8 t define MAX 88 OxT7F typedef unsigned short xm ui16 t 2505 define MAX U16 OxFFFF typedef short xm s 16 t define MAX 816 Ox7FFF typedef unsigned int xm u32 t define MAX U32 OxFFFFFFFF 2510 typedef int xm s32 t 87 134 88 134 Chapter 6 Binary Interfaces define MAX 832 Ox7FFFFFFF typedef unsigned long long xm u64 t define MAX U64 OxFFFFFFFFFFFFFFFFULL 2515 typedef long long xm s64 t define MAX S64 Ox7FFFFFFFFFFFFFFFLL Listing 6 1 core include arch arch_types h
183. sed partition binary image xmImageHdr Pointer to the partition image header structure xmImageHdr The xmeformat tool copies the address of the corresponding section in this filed entryPoint Address ofthe starting function Additionally analogically to the ELF format XEF contemplates the concept of segment which is a portion of code data with a size and a specific load address A XEF file includes a segment table see listing 6 5 which describes each one of the sections of the image custom data XEF files have only one section lAccording to our tests the time spent by more sophisticated digest algorithms such as SHA 2 Tiger or Whirlpool in the LEON3 processor was not acceptable As illustration 100 Kbytes took several seconds to be digested by a SHA 2 algorithm in this processor Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 2810 6 5 XEF format 95 134 struct xefSegment 1 xmAddress t physAddr xmAddress t virtAddr xmSize t fileSize xmSize t deflatedFileSize xmAddress t offset PACKED Listing 6 11 core include xmef h startAddr Address where the segment shall be located while it is being executed This address is the one used by the linker to locate the image If there is not MMU then physAddress virtAddr fileSize The size of the segment within the file This size could be different from the memory required to be executed for example a data segment bss segment usually
184. size 4MB gt Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 5 4 The Hello World example 63 134 Region type sdram start 0x60000000 size 1MB gt lt MemoryLayout gt lt ProcessorTable gt lt Processor id 0 frequency 50Mhz gt lt CyclicPlanTable gt lt Plan id 0 majorFrame 2ms gt Slot id 0 start 0ms duration 1ms partitionId 0 lt Slot id 1 start 1ms duration 1ms partitionId 1 lt Plan gt lt CyclicPlanTable gt lt Processor gt lt ProcessorTable gt lt Devices gt lt Uart id 0 baudRate 115200 name Uart gt lt MemoryBlock name MemDisk0 start 0x4000 size 256KB lt lt MemoryBlock name MemDisk0 start 0x40100000 size 256KB gt lt MemoryBlock name MemDisk1 start 0x40150000 size 256KB gt lt MemoryBlock name MemDisk2 start 0x40200000 size 256KB gt gt lt Devices gt lt HwDescription gt lt XMHypervisor console Uart gt lt PhysicalMemoryArea size 512KB gt Container device MemDisk0 offset 0x0 lt XMHypervisor gt lt track id hello xml sample gt gt lt PartitionTable gt Partition id 0 name Partitioni flags system console Uart gt lt PhysicalMemoryAreas gt lt Area start 0x40080000 size 512KB gt lt PhysicalMemoryAreas gt lt TemporalRequirements duration 500ms period 500ms gt lt Partition gt lt Partition id 1 name Partition2 fl
185. sparcv8 get flags O calls In this case the XtratuM entry code does not prepare the processor to execute C code Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 6 3 Executable formats overview 89 134 On the host developer On the host integrator On the target board a dl T gt AA 5 i STRAM libxm ELF file XEF file eee xefHdr I E t 00 i deplo lid Container XEF 1 segment 01 Appl c gt partitionControlTable ABI Data Segment 01 Structures xmlmageHdr partitioninformationTable partitionControlTable xefHdr XtratuM DRAM i i pi l l P pariitioninformationTable 1 i l i l i i i i l i i i T Y Other XEF files T NS TAN sparc linux gcc Xe sparc linux ld xmeformat xmpack rsw Figure 6 1 Executable formats 6 3 Executable formats overview XtratuM core does not have the capability to load partitions It is assumed that when XtratuM starts its execution all the partition code and data required to execute each partition is already in main memory Therefore XtratuM does not contain code to manage executable images The only information required by XtratuM to execute a partition is the address of the partition image header xmImageHdr The partition images as well as the XtratuM image shall be loaded by a resident software which a
186. ssible after the plan switch Once the maintenance activities have s been completed it is responsibility of a system partition to switch to another plan if needed A system partition can also request a switch to this Plan x x gt 1 Any plan greater than 1 is user defined A system partition can switch to any of these defined plan at any time xm 4 usermanual 047d AX ic amu P Printed November 26 2012 410 18 134 Chapter 2 XtratuM Architecture 2 8 2 Switching scheduling plans When a plan switch is requested by a system partition through a hypercall the plan switch is not immediate all the slots of the current plan will be completed and the new plan will be started at the end of the current one The plan switch that occurs as a consequence of the XM HM AC SWITCH TO MAINTENANCE action is syn chronous The current slot is terminated and the Plan 1 is started immediately 2 9 XtratuM Multicore Scheduling In this section the scheduling policies implemented and how they are specified in the configuration file are described 2 9 1 Scheduling units The scheduling unit is a virtual CPU of a partition By default if the virtual CPU is not specified it is assumed that it refers to the vCPUO vcpuld 0 In general we refer to this unit as a pair represented as partition vcpu gt 2 9 2 Scheduling policies XtratuM provides different policies that can be attached to any of the CPU Two basic policies
187. st ones listed as many as fit reasonably on the actual cover and continue the rest onto adjacent pages If you publish or distribute Opaque copies of the Document numbering more than 100 you must either include a machine readable Transparent copy along with each Opaque copy or state in or with each Opaque copy a computer network location from which the general network using public has access to download using public standard network protocols a complete Transparent copy of the Document free of added material If you use the latter option you must take reasonably prudent steps when you begin distribution of Opaque copies in quantity to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy directly or through your agents or retailers of that edition to the public It is requested but not required that you contact the authors of the Document well before redistribut ing any large number of copies to give them a chance to provide you with an updated version of the Document 4 MODIFICATIONS You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above provided that you release the Modified Version under precisely this License with the Modified Version filling the role of the Document thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it In
188. system it uses the services provided by the operating system and does not need to be virtualised But the operating system has to deal with the virtualisation The operating system has to be virtualised ported on top of XtratuM 1240 5 2 Implementation requirements Below is a checklist of what the partition developer and the integrator should take into accout when using XtratuM It is advisable to revisit this list to avoid incorrect assumptions lpara virtualised operations provided by the hypervisor 53 134 1245 1250 1255 1260 1265 1270 1275 1280 1285 54 134 Chapter 5 Partition Programming Development host If the computer where XtratuM was compiled on and the computer where it is being installed are different processor architectures 32bit and 64bit the tools may not run properly Check that the executable files in xm bin are compatible with the host architecture Para virtualised services Partition s code shall use the para virtualised services The use of native services is considered an error and the corresponding exception will be raised PCT The Partition Control Table is located within the partition mapped in read only mode Its content shall be modified by using the appropriate hypercalls In a multicore platform there are as PCT as virtual CPUs Each virtual CPU see only its own PCT Store ordering XtratuM has been implemented considering that the LEON2 processor is operating in
189. t E DEFAULT none DEFAULT mone console idString t memoryLayout e devices e anonymous SamplingChannel QueuingChannel rs rs id id t healthMonitorDevice idString t name idString t Y 56 anonymous all of all of sequence of sequence of sequence of maxMessageLength sizeUnit_t Attrs Attrs refreshPeriod timeUnit t DEFAULT 0s DEFAULT 0s HwResources PortTable TemporalRequirements Tace HealthMonitor Trace HealthMonitor PhysicalMemoryAreas PhysicalMemoryAreas Region choice of Processor t hwResources_e partitionPorts_e trace e pio anonymous duration timeUnit t bitmask hex t Attrs Attrs healthMonitor e device idString_t period timeUnit t all of sequence of sequence of 2 loPorts Interrupts Port anonymous ioPorts e Attrs lines hwIrgldList t Attrs y sequence of sequence of all of aa Destination Destination Source anonymous direction direction t anonymous action hmAction t discipline discipline t Attrs name idS log yntf t init seq
190. tain attributes The XPath syntax is used to refer to the objects elements and attributes Examples SystemDescription PartitionTable The element PartitionTable contained inside the element SystemDescription which is the root element the starting slash symbol SystemDescription name Refers to the attribute name of the element SystemDescription Trace bitmask Refers to the attribute bitmask of a Trace element The location of the element Trace in the xml element hierarchy is relative to the context where the reference appears 8 3 2 The root element SystemDescription Figure 8 1 is a graphical representation of the schema of the XML configuration file The types of the attributes are not represented see the appendix A for the complete schema specification An arrow ended with circle are optional elements Figure 8 2 on page 112 is a compact graphical representation of the nested structure of a sample XM CF configuration file the listing is the actual xml file for this representation Solid lined boxes represent elements Dotted boxes contain attributes The nested boxes represent the hierarchy of elements The root element is SystemDescription which contain the mandatory version name and xmlns attributes The xmlns name space shall be http www xtratum org xm 2 3 There are five second level elements SystemDescription XMHypervisor Specifies the board resources memory and processor pla
191. the same tool than the one used in Linux which is commonly called make menuconfig There are two different blocks that shall be configured 1 XtratuM source code and 2 the resident sofware The configuration menu of each block is presented one after the other when executed the make menuconfig from the root source directory The selected configuration are stored in the files core config and user bootloaders rsw config for XtratuM and the resident software respec tively The next table lists all the XtratuM configuration options and its default values Note that since there are logical dependencies between some options the menuconfig tool may not show all the options Only the options that can be selected are presented to the user 3020 3025 3030 Parameter Type Default value Processor SPARC cpu choice Leon2 Leon3 Board choice TSim GR CPCI XCAV GR PCI XC2V SIMLEON SPARC memory protection schema choice MMU Support AMBA bus PnP bool y Enable cache bool y Enable cache snoop boo n Enable instruction burst fetch bool n Flush cache after context switch bool n Continues 105 134 3035 3040 3045 106 134 Chapter 8 Configuration Parameter Type Default value Physical memory layout XM load address hex XM virtual address hex Enable experimental features bool n Debug and profiling support bool y Dump CPU state when a trap is raised bool y Max identifier le
192. the ideas and concepts that shall be kept in mind to understand the internal operation of XtratuM and how to use the hypercalls e A partition behaves basically as the native computer Only those services that have been explicitly para virtualised should be managed in a different way e Partition s code may not be self modifying The partition is responsible to appropriately flush sso cache in order to guarantee the coherency e Partition s code is always executed with native interrupts enabled This behaviour is enforced by XtratuM e Partition s code is not allowed to disable native interrupts only their own virtual interrupts e XtratuM code is non preemptive It should be considered as a single critical section 985 A e Partitions are scheduled by using a predefined scheduling cyclic plan e Inter partition communication is done through messages e There are two kinds of virtual communication devices sampling ports and queuing ports e All hypercall services are non blocking e Regarding the capabilities of the partitions XtratuM defines two kinds of partitions system and o standard e Only system partitions are allowed to control the state of the system and other partitions and to query about them e XtratuM is configured off line and no dynamic objects can be added at run time e The XtratuM configuration file XM CF describes the resources that are allowed to be used by ou each partition e XtratuM provides a fi
193. to point out that system partition rights are related to the capability to manage the system and not to the capability to access directly to the native hardware or to break the isolation a system partition is scheduled as a normal partition and it can only use the resources allocated to it in the configuration file Table 2 2 shows the list of hypercalls reserved for system partitions A hypercall labeled as partial indicates that a normal partition can invoke it if a system reserved service is not requested For instance a normal partition can invoke the service XM halt partition if the target partition is itself To use this service with another partition the invoking partition has to be defined as system partition Hypercall System XM get gid by name Partial XM get partition status Partial XM get system status Yes XM halt partition Partial XM halt system Yes XM hm open Yes XM hm read Yes XM hm seek Yes XM hm status Yes XM memory copy Partial XM reset partition Partial XM reset system Yes XM resume partition Yes XM shutdown partition Partial XM suspend partition Partial XM switch sched plan Yes XM trace open Yes XM trace read Yes XM trace seek Yes XM trace status Yes Table 2 2 List of system reserved hypercalls A partition has system capabilities if the System Description Partition Table Partition Oflags attribute contains the flag system in the XML configuration file Several partitions can be defined as
194. trap table are exclusively managed by XtratuM All native traps jump into XtratuM routines The trap mechanism is used for several purposes Hardware interrupts Used by peripherals to request the attention of the processor Software traps Raised by a processor instruction commonly used to implement the system call mech anism in the operating systems Processor exceptions Raised by the processor to inform about a condition that prevents the execu tion of an instruction There are basically two kinds of exceptions those caused by the normal operation of the processor such as register window under overflow and those caused by an abnormal situation such as a memory error XtratuM defines 32 new interrupts called extended interrupts These new interrupts are used to inform the partition about XtratuM specific events In the case of SPARC v8 these interrupts are vectored starting at trap handler 224 The trap handler raised by a trap can be changed by invoking the XM route irq hypercall Partitions are not allowed to directly access read or write the tbr register XtratuM implements a virtual trap table instead 2 15 2 Interrupts Although in a fully virtualised environment a partition should not need to manage hardware interrupts XtratuM only virtualises those hardware peripherals that may endanger the isolation but leaves to the partitions to directly manage non critical devices In order to properly manage peripherals a
195. tree xm 4 usermanual 047d AX ic amu P Printed November 26 2012 This page is intentionally left blank Volume 2 XtratuM User Manual Chapter 5 Partition Programming This chapter explains how to build a XtratuM partition partition developer tutorial 5 1 Partition definition A partition is an execution environment managed by the hypervisor which uses the virtualised services Each partition consists of one or more concurrent processes implemented by the operating system of 125 each partition sharing access to processor resources based upon the requirements of the application The partition code can be e An application compiled to be executed on a bare machine bare application e A real time operating system and its applications e A general purpose operating system and its applications 1230 Partitions need to be virtualised to be executed on top of XtratuM For instance the partitions cannot manage directly the hardware interrupts enable disable interrupts which have to be replaced by hypercalls to ask for the hypervisor to enable disable the interrupts Depending on the type of execution environment the virtualisation implies Bare application The application has to be virtualised using the services provided by XtratuM The 125 application is designed to run directly on the hardware and it has to be aware about it Operating system application When the application runs on top of a real time operating
196. u Suspends a virtual CPU for the partition invoking this service Time Management XM get time Retrieve the time of the clock specified in the pa rameter XM set timer Arm a timer Plan Management XM get plan status Return information about the status of the scheduling plan XM switch sched plan Request a plan switch at the end of the current MAF Inter Partition Communication XM create queuing port Create a queuing port XM create sampling port Create a sampling port XM get queuing port info Get the info of a queuing port from the port name Printed November 26 2012 hr adu xm 4 usermanual 047d 5 20 Manpages summary 85 134 XM get queuing port status Get the status of a queuing port XM get sampling port info Get the info of a sampling port from the port name XM get sampling port status Get the status of a sampling port XM read sampling message Reads a message from the specified sampling port XM receive queuing message Receive a message from the specified queuing port XM send queuing message Send a message in the specified queuing port XM write sampling message Writes a message in the specified sampling port Memory Management XM memory copy Copy a memory area of a specified size from a source to a destination Health Monitor Management XM hm r
197. ublished by the Free Software Foundation with no Invariant Sections no Front Cover Texts and no Back Cover Texts A copy of the license is included in the section entitled GNU Free Documentation License Changes Version Date Comments 0 1 November 2011 xm 4 usermanual 047 Initial document 0 2 March 2012 xm 4 usermanual 047b IOMMU included Fixed minor bugs 0 3 April 2012 xm 4 usermanual 047c IOMMU updated to support paging transla tion 0 4 November 2012 xm 4 usermanual 047d Final release Code version 4 0 Typos corrected New subsection about IPVI management XML schema specification described in a separate annex xm 4 usermanual 047d AX ic amu P Printed November 26 2012 This page is intentionally left blank Volume 2 XtratuM User Manual Contents Preface 1 Introduction Vlad aU eee antem Bur pcd cad 2 XtratuM Architecture 2 1 Multicore architecture s s a book p n RU Ee rU UE ER RR d ps 2 2 ArchiteCtHEe a aa Beek ro Ue AC Ra Ub RRA d 2 3 System states and services 4 2e oor ROSE eee ABA SESS 2 31 Systemi States oo ede e e Shee ee wees ob bode ea oe 23 2 DOY STE STATES o o is oos Ee owe e db dx e Qa ge ee a 2 4 Partition states and services erens enere k us 2 4 Partition states 5 2 o 3 6p eid ba quote EEE pe d mes 2 42 Partition Services o kb ABL E EUR X a EON We Sere Ue FIER es 2 5 Virtual CPU states nd Operation sss s suo o A a a e eens 2 5 1 virtual
198. uence of choice of Restricted name hmString t type portType t Range Attrs Y anonymous anonymous address hex t base hex t Attrs mask hex t DEFAULT 0x0 DEFAULT 0x0 noPorts positivelnteger anonymous processor e Size sizeUnit t features processorFeaturesList_t DEFAULT none DEFAULT none memoryArea e Attrs start hex t frequency freqUnit t type memRegion t id id t anonymous baudRate positivelnteger anonymous sequence of all of Source Attrs id idString t Attrs name idString t name idString t lanTable Attrs anonymous flags memAreaFlagsList t DEFAULT none DEFAULT none mappedAt hex t y cyclicPlan_e Attrs name idString t size sizeUnit t start hex t X sequence of plan e idiid t majorFrame timeUnit t Attrs name idString t sequence of anonymous duration timeUnit t idiid t partitionid id t Attrs start timeUnit t md ipcPort e partitionld id t partitionName idString t portName idString_t Figure 8 1 XML Schema diagram Printed November 26 2012 xm 4 userm anual 047d 8 3 Hypervisor configuration file XM CF 111 134 An XML file is organised as a set of nested elements each element may con
199. uration file is in the appendix A 3265 Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d Volume 2 XtratuM User Manual Chapter 9 Tools This section describes the tools to assist the integrator and the partition developers in the process of building the final system file They are xmcparser System XML configuration parser xmeformat Converts ELF files into XEF ones xmpack Creates the container file 3270 rswbuild Creates a bootable file image 9 1 XML configuration parser xmcparser The utility xmcparser translates the XML configuration file containing the system description into bi nary form that can be directly used by XtratuM In the first place the configuration file is checked both syntactically and semantically i e the data is correct This tool uses the 1ibxm12 library to read parse and validate the configuration file against s the XML schema specification Once validated by the library the xmcparser performs a set of non syntactical checks e Memory area overlapping e Memory region overlapping e Memory area inside any region 3280 Duplicated Partition s name and id Allocated CPUs Replicated port s names and id Cyclic scheduling plan Cyclic scheduling plan slot partition ids 3285 Hardware IRQs allocated to partitions IO port alignment IO ports allocated to partitions Allowed health monitoring actions 117 134 3290 3295 3300 3305 3310
200. usermanual 047d A ic emu P Printed November 26 2012 98 134 Chapter 6 Binary Interfaces 2890 struct xmefFile 4 xmAddress t offset xmSize t size xmAddress t nameOffset __PACKED 2895 Listing 6 14 core include xmef h offset The offset relative to the start of the file data table section to the data of this file in the container size The size reserved to store this file It is possible to define the size reserved in the container to 2900 store a file independently of the actual size of the file See the section 9 3 1 tool nameOffset Offset relative to the start of the strings table of the name of the file The strings table contains the list of all the file names The file data section contains the data with padding if fileSize lt size of the files Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d Volume 2 XtratuM User Manual Chapter 7 Booting PROM 256Mb S SRAM 4Mb 0x0000 0000 a 0x4000 0000 Er METWCUS i m o 1 o LI LI Partition1 xef 03 O tc o a Partition1 xef O E E E N XM reset parition O 5 Custom_CT 1 m Sie la gt 0 mn GUTES Partition 2 xef A Partition2 xef Partition2 xef beorinos Custom CT2 See TEE EET Custom_CT 2 LAE EEE muB ELL gt 0x403F FFFF Ox1FFF FFFF lt Figure 7 1 Booting sequence In the standard boot procedure of the LEON2 processor the program counter
201. ush spill all register windows in the current CPU stack After calling this function all the register windows but the current one are stored in RAM and then marked as free The partition context switch code should basically carry out the next actions 1 call XM sparcv8 flush regwinO CL 66599 2 store the current g i and I registers in the stack 3 switch to the new thread s stack and 4 restore the same set of registers Note that there is no complementary function to reload fill the registers because it is done auto matically The XM sparcv8 flush regwinO service can also be used set the processor in a know state before executing a block of code All the register windows will be clean and no window overflow will happen during the next 7 nested function calls 5 14 Tracing 5 14 1 Trace messages The hypercall XM trace event stores a trace message in the partition s associated buffer A trace message is a xmTraceStatus t structure which contains an opCode and an associated user defined data struct xmTraceEvent define XMTRACE SIGNATURE Oxc33c xm u16 t signature xm u16 t checksum xm u32 t opCodeH opCodeL HIGH define TRACE OPCODE SEQ MASK OxfffffffO TRACE OPCODE SEQ BIT define TRACE OPCODE SEQ BIT 4 Printed November 26 2012 AX ir amu PA xm 4 usermanual 047d 5 14 Tracing 75 134 define TRACE OPCODE CRIT BIT 1 define TRACE_OPCODE_SYS_BIT 0 LO
202. using the hypercall XM set planO The system will change to the new plan at the end of the current MAF If XM set plan is called several times before the end of the current plan then the plan specified in the last call will take effect The hypercall XM get plan status O returns information about the plans The xmPlanStatus_t con tains the following fields typedef struct xmTime_t switchTime xm_s32_t next xm_s32_t current xm_s32_t prev xmPlanStatus_t Listing 5 14 core include objects status h switchTime The absolute time of the last plan switch request After a reset both warm and cold the value is set to zero current Identifier of the current plan next The plan identifier that will be active on the next major frame If no plan switch is going to occur then the value of next is equal to the value of current prev The identifier of the plan executed before the current one After a reset both warm and cold the value is set to 1 5 8 Console output XtratuM offers a basic service to print a string on the console This service is provided through a hypercall Printed November 26 2012 A ir emu PA xm 4 usermanual 047d 5 9 Inter partition communication 67 134 XM write console Partition 1 Start execution n 29 Listing 5 15 Simple hypercall invocation Additionally to this low level hypercall some functions have been created to facilitate the use of the console by the p
203. vcpulds are used internally as indexes to the corresponding data structures 2 8 Partition scheduling XtratuM schedules partitions in a fixed cyclic basis ARINC 653 scheduling policy This policy ensures that one partition cannot use the processor for longer than scheduled to the detriment of the other partitions The set of time slots allocated to each partition is defined in the XM CF configuration file during the design phase Each partition is scheduled for a time slot defined as a start time and a duration Within a time slot XtratuM allocates the processor to the partition If there are several concurrent activities in the partition the partition shall implement its own schedul ing algorithm This two level scheduling scheme is known as hierarchical scheduling XtratuM is not aware of the scheduling policy used internally on each partition In general a cyclic plan consists of a major time frame MAF which is periodically repeated The MAF is typically defined as the least common multiple of the periods of the partitions or the periods of the threads of each partition if any For instance consider the partition set of figure 2 3a its hyper period is 200 time units milliseconds and has a CPU utilisation of the 90 The execution chronogram is depicted in figure 2 11 One of the possible cyclic scheduling plans can be described in terms of start time and duration as it is shown in the table 2 3b This plan has to be specified

XtratuM User Manual - ESA Microelectronics Section

Contents

Download Pdf Manuals

Related Search

Related Contents