Home

NAT-MCH Users Manual

Contents

1. 99 Table 6 Pin Assignment of the Front panel Connector 52 GbE Uplink 99 Table 7 Pin Assignment of the Front panel Connector 100 100 Table 8 Pin Assignment of the Clock Connector S3 sene 100 Table 9 MCH Connector Tongue 1 sie ecce ceed 103 Table 10 MCH Connector Tongue 2 e S eo E DEOR NOTI eite Dore dues es 106 Table TT MCH Connector Tongue 109 Table 12 Connector Tongue 4 112 Version 1 25 N A T GmbH 8 User s Manual Glossary AMC Advanced Mezzanine Card ATCA Advanced Telecom Computing Architecture BMC Baseboard Management Controller BT Block Transfer CM Carrier Manager CPU Central Processing Unit CU Cooling Unit Fan EMMC Enhanced Module Management Controller MMC on CU or PU via IPMB 0 FRU Field Replaceable Unit hotswap capable resource HPI Hardware Platform Interface SW management interface defined by SAF Dc Inter Integrated Circuit 2 wire serial bus IPM Intelligent Platform Management IPMB IPM Bus I2C type 0 dual redundant local IPMB IPMB L non redundant local IPMB IPMC IPM Controller e g MCH CPU IPMI IPM Interface KCS Keyboard Controller Style LAN Local Area Network LED ID Light Emitting Diode Identifier LUN Logical Unit Number LVDS Low Voltage Differentia
2. entere 50 7 2 1 Remote management control protocol eese eese enne etn ennt 50 72 2 Supported IPMI messages idit cie a re teet EG I E Mr e e He EA e eni 50 7 3 HOST SOFTWARE OVERVIEW erede RU 51 7 3 1 JAVA GUI application NATView 51 43 2 dpmitool iuo a e RC nn REN Ce EPOD 52 79 3 usate o p Ee ERE EE 53 8 COMMAND LINE 2 tosta setae ee ao seen e een sesso seen s eoe seen P eee eaae 57 9 WEBSERVER e 60 91 PCIE VIRTUAL SWITCH CONFIGURATION 2 222 40 0000000000000000000000000005000005500 63 10 CLOCK MODULE CONFIGURATION Q 00 ccccsssscssccsssccsscssssscssssssssscssscssescsssssssssssssssssascssssesescoess 64 10 1 CEOCK MODULE VERSION 2 c debiti ei 65 10 1 1 PLL Reference Input Configuration eese esee eene trennen rennen nennen trennen 65 10 1 2 PLE Mode Configuration a etes d eee 66 10 1 5 Clock 67 10 1 4 Clock Output Configuration as rre Pr qa 68 10 2 CLOCK MODULE VERSION en petet a eitis aon dbase 69 10 2 1 PLL Reference Input Configuration eese
3. Please note that some events e g temperature events are handled by the local shelf manager refer to chapter 4 3 2 4 3 2 Temperature Management The local shelf manager receives temperature events from the following sources e Local temperature sensors the Temperature sensors on the AMC modules Temperature sensors on Cooling Units and Power Modules In case the MCH receives a temperature event temperature going high event from a FRU i e the temperature of a certain module has reached a critical level it increases the fan speed of the cooling units to the maximum level and starts monitoring the temperature sensors of the respective FRU As soon as the temperature returns to normal level the fan speed will be decreased to a level which is 10 higher than the level it was when the temperature event occurs The initial normal fan level can be set in the MCH configuration menu by the value of the configuration parameter Default Fan Level Within the MCH global parameter section Version 1 25 N A T GmbH 32 User s Manual 4 4 NAT MCHs operating with unmanaged Power Modules The NAT MCH is capable to manage systems which use so called power through modules In this case the detection of modules is not done by the power modules i e presence of PS1 signal but the NAT MCH itself starts scanning all AMC slots by IPMI messages Of course due to the limited functionality of s
4. User s Manual NAT MCH User s Manual Version 1 25 User s Manual NAT MCH has been designed by N A T GmbH Konrad Zuse Platz 9 53227 Bonn Oberkassel Phone 49 228 96 58 64 0 Fax 49 228 96 58 64 10 Internet http www nateurope com Version 1 25 N A T GmbH User s Manual Disclaimer The following documentation compiled by N A T GmbH henceforth called N A T represents the current status of the product s development The documentation is updated on a regular basis Any changes which might ensue including those necessitated by updated specifications are considered in the latest version of this documentation N A T is under no obligation to notify any person organization or institution of such changes or to make these changes public in any other way We must caution you that this publication could include technical inaccuracies or typographical errors N A T offers no warranty either expressed or implied for the contents of this documentation or for the product described therein including but not limited to the warranties of merchantability or the fitness of the product for any specific purpose In no event will N A T be liable for any loss of data or for errors in data utilization or processing resulting from the use of this product or the documentation In particular N A T will not be responsible for any direct or indirect damages including
5. 39 344 SEL Configuration Flags 0 teo ett t geo i De eb sven as 40 9 493 Switch Configuration reet cre cy Tee 41 5 4 6 Module Configuration 8 optional eese eene 42 5 4 7 PCle Switch Configuration 9 optional eee eerte ener 42 5 4 8 5 Switch Configuration 9 optional eese ener 43 241 9 NEP Configuration LO es Spe RR banal 43 5 4 10 DH GP Configuration 44 6 UPDATING THE FIRMWARE sisi sssccseonssesscsssasosccssesseosesssnessossesenassosessnssessenassnsveonscssessesessonseseen 45 6 1 FIRMWARE UPDATE FROM WITHIN THE RUNNING FIRMWARE 2 004010 0006 0000000000000 45 6 2 FIRMWARE UPDATE FROM BOOTLOADER 45 6 3 45 6 3 1 a 45 6 3 2 Firmmware2 7 or later 46 64 FIRMWARE UPDATE VIA WEB INTERFACE FW 2 7 OR 46 7 MANAGEMENT INTERPFACE rires 40 tn eet aaa n ooa k o a oe nsss 48 7 1 SOFTWARE STRUCTURES RB BRRERSETE 48 7 22 COMMUNICATION BETWEEN HOST SYSTEM AND MCH
6. DyFLz DyFL 1 DyFL DFL DyFL DyFL 1 Time slope is critical to avoid The slope is not critical entering the Critical Status when the DFL is very low Glossary Fan Speed FS Actual Fan Speed Maximum Fan Speed FSmax Fans at 100 RPMs Default_Fan_Level DFL The Fan Speed after a MCH reboot configurable Dynamic_Fan_Level DyFL The new Fan Speed after a Temp Sensor alarm Fan_Step_Up FSU Fan_Time_Up FTU Fan_Step_Down FSD Fan_Time_Down FTD Fan_Time_Down_Dy FTDy Possible MCH Status NORMAL No Temp Alarm Non_Critical At least one Temp Sensor has reported that the upper noncritical threshold has been crossed Critical At least one Temp Sensor has reported that the upper critical threshold has been crossed Non_Recoverable At least one Temp Sensor has reported that the non recoverable threshold has been crossed Fan control description After an MCH reboot MCH Status Normal Fan Speed FS DFL Version 1 25 GmbH 129 User s Manual DyFL DFL Non Critical Event DyFL DyFL 1 FS FS FSU each FTU seconds until FS FSmax Normal FS FS FSD each FTD seconds until FS DyFL When FS DyFL then Critical Event FS Fsmax until normal status DyFL DyFL 1 Version 1 25 N A T GmbH 130 User s Manual Appendix F Updating the Backplane EEPROM via MCH CLI A combined backplane FRU image file can be downloa
7. 89 11 3 OR VALUE TO SRIO MODULE BYTE 90 11 4 AND VALUE TO SRIO MODULE REGISTER 91 11 5 WRITE SRIO MODULE SWITCH REGISTER INDIRECT LONG 92 11 6 OR VALUE TO SRIO MODULE SWITCH REGISTER INDIRECT LONG essen 93 11 7 AND VALUE TO SRIO MODULE SWITCH REGISTER INDIRECT LONG seen 94 12 UNMANAGED AMC MODULE CONFIGURATION eee e eee ee ee 95 13 N 1 REDUNDANCY AND REDUNDANT LOAD SHARING e eene eee ene 1010 96 14 HARDWARE ino aola oe eter duae eto EP ek erba oe sea secadsedaccosssdecoosss 97 14 1 KNOWN HARDWARE ISSUES 97 15 GIVE USA HINT 98 APPENDIX 99 APPENDIX 1 CONSOLE CONNECTOR 5232 2 0 212 0 0 000 99 APPENDIX A2 ETHERNET CONNECTOR GBE UPLINK eese eee 99 APPENDIX A3 ETHERNET CONNECTOR 100 BASET MANAGEMENT PORT 100 APPENDIX 4 EXTERNAL CLOCK REFERENCE CONNECTOR 4 11200 100 APPENDIX 5 BASE6 12 CONNECTOR TONGUE
8. 101 APPENDIX NAT MCH CLOCK CONNECTOR 2 104 APPENDIXA7 HUB MODULE CONNECTOR TONGUE 107 APPENDIXA8 HUB MODULE CONNECTOR TONGUE 4 ccce 110 APPENDIX B ACCESS FROM A DIFFERENT 4 ee ee eee ee eo 0 113 APPENDIX B 1 NETWORK SETUP teet io WS WR RESET S 113 APPENDIXB2 MCH IP ADDRESS CONFIGURATION 1 en enhn nne n rennen nnne nnn 113 APPENDIX B 3 LINUX GATEWAY 22 022 114 APPENDIX B4 TESTI PING BETWEEN AND GATEWAY 21 1 0 1 115 APPENDIX B 5 TEST2 PING BETWEEN CONTROL HOST AND GATEWAY 0 444004404 000 enn 115 APPENDIX B6 TEST3 PING BETWEEN CONTROL HOST AND 116 APPENDIX C N A T DEFINED OEM IPMI MESSAGES 118 APPENDIX C 1 IPMI MESSAGES TO READ AND WRITE REGISTER cene eene emen eene eene sereni 118 APPENDIX C2 REGISTER ACCESS USING 0 eene n e 121 APPENDIX C unu 122 APPENDIX D ADAPTING HOST APP
9. Command Parameter Description lo loads a new firmware image and stores in onboard Flash device see chapter 6 for details NOTE This command is only available in the initial bootstrap loader bi Board Information Prints the vital product information record i e Serial number Hardware revision and release codes br Baud Rate Configuration Configures the MCH s serial interface baud rate by entering the desired baud rate e g 19200 or 9600 The configured baud rate is stored in an on board EEPROM and therefore valid after a reboot Please note that if the baud rate was changed the terminal s baud rate has to be re configured too bs Boot String Configuration This command allows to select between loading the MCH firmware from FLASH or to download a firmware image using TFTP Default is to start the MCH firmware from FLASH ip IP configuration Configures IP addresses net mask broadcast address and gateway telnetd pw Telnet Password Configuration optional Allows to change the password which is verified before starting a telnet session A password length of not less than 8 to maximum 16 characters is required Entering just CR without any other characters will delete the password On the next telnet session start no password verification will be executed Version 1 25 N A T GmbH 57 User s Manual reboot Reboot the MCH update firmware Up
10. Hot plug support Disabled Enables PCIe Hot Plug Support PCIe early Ekey Disabled Executes the E Keying before Payload power is applied PCIe clustering Disabled Enables PCIe clustering for PCle x48 HUB modules If PCIe clustering is enabled the mode transparent upstream non transparent upstream parameters can be set for each switch separately Communication between the two clusters is not possible in this case Version 1 25 N A T GmbH 42 User s Manual Remarks The power up sequence and delays defined by the backplane FRU info device might not be sufficient in all situations for systems with PCIe boards as usually all I O boards need to be ready before the CPU board may start its PCI scan Therefore an extra delay in the MCH configuration record can be defined for the PCIe upstream host The settings for the second PCIe cluster are only valid if PCIe clustering is enabled via the corresponding flag Remarks PCIe HUB PCIe PCB V1 x only for PCIe Gen 3 refer to chapter 9 1 PCIe Virtual Switch Configuration optional 5 4 8 5 Switch Configuration 9 optional By the SRIO configuration parameters the operation of the optional SRIO Hub module is controlled Configuration Option default Description operating mode No uplink Operation mode of the SRIO Switches uplink dual x4 Interconnect default Dual Uplink Mode Si
11. READ Ox0b define IPMI_CMD_WRITE_BLCK I2C addresses of the Hub clock IPMI devices Device Site 1 Site 2 Clock Module Ox14 0 18 Hub Module 0x16 Oxla Appendix C2 Register access using ipmitool This chapter gives a list of example calls for ipmitool to access registers on the clock or hub modules Please note that register access IPMI messages have to be double bridged to be delivered to the related modules which requires at least ipmitool version 1 8 11 Read the Clock Module board identifier gt ipmitool H 132 147 160 251 P t 0x82 T 0x14 raw 0x30 0x01 0x00 0x00 0x00 Read the Clock Module PCB version 1 1 001 H 132 147 160 251 P t 0x82 T 0x14 raw 0x30 0x01 0x00 0x00 0x01 Version 1 25 N A T GmbH 121 User s Manual Read the Clock Module reference O selection ipmitool H 132 147 160 251 P t 0x82 T 0x14 raw 0x30 0x01 0x00 0x00 0x04 Write the Clock Module reference O selection to Oxaa ipmitool 132 147 160 251 P t 0x82 t 0x14 raw 0x30 0x02 0x00 0x00 0x04 Appendix C 3 It is also possible to access any AMC power module cooling unit etc by ipmitool via the MCH In this example the firmware of power module 1 shall be updated with ipmitool Please refer to the ipmitool documentation for further parameters ipmitool I lan H 192 168 1 146 A none T 0x82
12. 1 12 RxFDI 716 Version 1 25 N A T GmbH 111 User s Manual MCH Signal MCH Signal 80 TxFFI2 RxFFI2 4 8 TxFFI2 190 82 GND GND 9 85 GND 6 Table 12 Connector Tongue 4 Version 1 25 N A T GmbH 112 User s Manual 2 Appendix access from a different subnetwork This section describes how to setup the MCH in a subnetwork and to access the MCH from a different subnetwork Please note that this appendix describes the network configuration using the defaults stored in the so called board configuration EEPROM Therefore the following explanations only apply if the IP address source in the configuration is set to board configuration Appendix B 1 Network setup The following scheme shows the example network configuration that is used in this description MCH IP Address 192 168 0 134 network 192 168 0 x first subnetwork 4 gt Linux Gateway IP Adress 192 168 0 70 first port ethl Linux Gateway IP Adress 132 147 160 70 second port eth0 network 132 147 160 x second subnetwork V Control host IP address 132 147 160 47 default gateway 132 147 160 70 The term gateway is used in that manner that a gateway changes IP addresses using Network Address Translation see below whereas a router does not change IP addresses Please note that the
13. 29 3 1 3 Mezzanine 29 4 31 4 1 SYSTEM STARTUP AND REDUNDANCY OPERATION 31 42 AMC MODULE STARTUP SEQUENCING 22 0 2 0000000000000000000000000000000000000000 ente te anne 31 Version 1 25 N A T GmbH 4 User s Manual 4 3 LOCAL SHELF MANAGER teri eee RE eee 32 434 SensorEventEog iu sce SER NAUTIQUE BNR nai aR ete 32 4 3 0 Temperature Management 32 44 NAT MCHS OPERATING WITH UNMANAGED POWER MODULES 33 5 MCH CONFIGURATION 5 ois otto e rnno 06 eb oet 34 5 1 CONFIGURATION VIA CONSOLE PORT satis Ant nete d nA m ec e o dt ae e p eee 34 52 CONFIGURATION VIA THE WEB enne enne enne nein nenne 35 5 3 CUSTOMIZING THE NETWORK CONFIGURATION 2 22 eterne ener 35 5 4 DISPLAYING AND CHANGING OPERATIONAL PARAMETERS 36 5 4 1 Global Parameters 3 ne 37 3 4 2 Shelf Manager Configuration 4 ertet tte eee e ett 39 543 Carrier Manager Configuration 5 ie ettet tite nte tere
14. cp examples openhpi nat conf usr local etc openhpi openhpi conf 7 3 3 5 Running OpenHPI Before the OpenHPI daemon is started please start up the uTCA system If the OpenHPI daemon is not able to establish a RMCP connection to the MCH it will silently terminate To start the OpenHPI daemon login as root on the Linux PC and call gt usr local sbin openhpid c usr local etc openhpi openhpi conf Using the provided configuration file openhpi_nat conf you will find daemon logging information on your system in tmp openhpi00 log As mentioned above the OpenHPI daemon will silently terminate on error conditions If a RMCP connection cannot be established between the OpenHPI daemon and the MCH please note following startup time can last up to 1 minute depending on the number of found resources sensors and FRU data check that the MCH can be reached over the network ping MCH IP address if there is no ping reply please check the MCH s network configuration or the IP address and port settings in openhpi conf check that the openhpi daemon is running ps aux grep openhpi Once the daemon was started successfully one may start an application to communicate with the MCH The OpenHPI package provides some applications e g hpitree displays found resources hpitop Version 1 25 N A T GmbH 55 User s Manual displays resource information about found resources and hpisensor
15. Configure the PLL to automatically select the reference input between IC4 Version 1 25 N A T GmbH 70 User s Manual 10 2 3 Clock Output Configuration Description Determines which clock source is driven on a specific clock output of the clock module Syntax Clk gen3 out dst src Parameter Description dst destination clock identifier 1 AMC 1 2 CLK1 2 3 CLK1 3 4 CLK1 4 5 CLK1 5 6 CLK1 AMC 6 7 CLK1 7 8 CLK1 AMC 8 9 CLK1 AMC 9 10 CLK1 AMC 10 11 CLK1 11 12 CLK1 AMC 12 13 CLK2 AMC 1 14 CLK2 AMC 2 15 CLK2 3 16 CLK2 AMC 4 17 CLK2 AMC 5 18 CLK2 AMC 6 19 CLK2 AMC 7 20 CLK2 8 21 CLK2 AMC 9 22 CLK2 AMC 10 23 CLK2 AMC 1 24 CLK2 AMC 12 25 CLK3 AMC 1 26 CLK3 AMC 2 27 CLK3 AMC 3 28 CLK3 AMC 4 29 CLK3 AMC 5 30 CLK3 AMC 6 31 CLK3 AMC 7 32 CLK3 AMC 8 33 CLK3 AMC 9 34 CLK3 10 35 CLK3 11 36 CLK3 AMC 12 37 CLK1 Updat 38 CLK3 Update 39 EXT single ended 1 Version 1 25 N A T GmbH 71 User s Manual 40 EXT single ended 2 41 EXT single ended 3 42 EXT single en
16. 0 t 0xc2 b 7 hpm upgrade fw hpm Parameters used linterface lan H 192 168 1 146 ip address 192 168 1 146 none authentication none T 0x82 target address of the carrier 80h Carrier Number 2 B0 bus id of the carrier IPMB 0 t Oxc2 target address power module 1 has IPMB L address 2 b7 bus is IPMB L hpm upgrade firmware hpm Other possible commands are targetcap shows target capabilities hpm compprop id select shows component properties lt select gt 1 show firmware revision lt select gt 2 show component type hpm upgstatus status of the last long duration command hpm activate activate upgraded firmware Version 1 25 N A T GmbH 122 User s Manual The I2C addresses of several FRU devices are Device target address AMC I 0x72 2 0 74 12 0 88 CU Oxa8 CU 2 Oxaa PM 1 Oxc2 PM2 Oxc4 PM3 Oxc6 PM 4 Oxc8 Version 1 25 N A T GmbH 123 User s Manual Appendix D Adapting a Host App from Compatible to Native mode Appendix D 1 Preface The NAT MCH firmware has changed over time Starting with firmware release 2 5 a new concept called ShM CM Separation was introduced To keep all older software operational with this new firmware the configuration option Compatible Mode was added Enabling this compatible mode forced the M
17. In both cases network address translation is performed by the gateway Ping the the control host from the MCH nat ping 132 147 160 47 192 168 0 134 gt 132 147 160 47 Echo Request 56 bytes seq 0 132 147 160 47 gt 192 168 0 134 Echo Reply 56 bytes seq 0 ttl 63 192 168 0 134 gt 132 147 160 47 Echo Request 56 bytes seq 1 132 147 160 47 gt 192 168 0 134 Echo Reply 56 bytes seq 1 ttl 63 192 168 0 134 gt 132 147 160 47 Echo Request 56 bytes seq 2 132 147 160 47 gt 192 168 0 134 Echo Reply 56 bytes seq 2 ttl 63 Version 1 25 N A T GmbH 116 User s Manual 4 132 147 160 47 ping statistics packets tx 3 rx 3 and the network sniffer dump gives No Time Source Destination Prot Info 46 12 307430 192 168 0 134 132 147 160 47 Echo ping request 12 307485 132 147 160 70 132 147 160 47 Echo ping request 12 307643 132 147 160 47 132 147 160 70 Echo ping reply 49 12 307658 132 147 160 47 192 168 0 134 ICMP Echo ping reply If the ping test between the control host and the or vice versa fails but and Test 2 succeeded there is probably something wrong with the gateway configuration Version 1 25 N A T GmbH 117 User s Manual Appendix C N A T defined OEM IPMI Messages Appendix C 1 messages to read and write register The following chapter lists the IPMI extensions defined by N A T for re
18. Management Interface Port sese pete eee qe 21 27229 Console Port 5 USB Telnet von i ee tete te ee ee 21 24 NAT MCH LC CONNECTORS AND INDICATORS AT THE FACE PLATE 23 2 4 1 LED Tndic lors i ue a ete e n 23 242 GbE Uplink POrt zie seh e p e petet pa e a News ede 23 24 3 Management Interface Pore 23 2 4 4 Gonsole Port 5 USB Telnet c e 24 2 5 NAT MCH M4 CONNECTORS AND INDICATORS AT THE FACE PLATE 25 2 5 1 LED Indicator Srs ae eerte eee 25 2 5 2 SMA Connectors External Clock Reference eese eene ener nee 25 252 Dual GbEUplinkPOftz use aere ea ic ete datei epe i redet 26 2 3 4 Management InterfaGe POrt RR ye eee ee 26 2 5 5 Console Port USB Telnet and RS232 via RIAS 26 3 MEZZANINE MODULE OPTIONS ea oe erra netto eoo Se bano e eR Too aee PE Pe SE ee poer ea 28 3 1 1 28 3 1 2
19. N A T GmbH 50 User s Manual 7 3 Host soflware overview 7 3 1 N A T JAVA GUI application NATView NATView is a graphical application that can be used to monitor and control an uTCA system using IPMI messages The application is written entirely in Java it should therefore run on every operating system that is able to execute the Sun Java Runtime Environment JRE NATView 1 22a 2008 11 21 Registered for Friedolin Lemkueth Resources FRU 003 N A T GmbH Germany NAT MCH 1 Temp CPU 2 Temp 1 0 Temperature deg C 2 HotSwap 43 00 MIN 4 Version Change 2 E FRU 005 N A T GmbH Germany NAMC8560 8E1 100 00 Sensor 2 LUN 0 Temp 1 0 43 0 degree Celsius non critical 0 0 i FRU 006 GmbH Germany NAMC8560 8E1 AMC FRU 007 GmbH Germany NAMC8560 8E1 FRU 009 N A T GmbH Germany NAMC 16ADSP 1 Temp FPGA 2 Temp EB 3 HotSwap 89 48 FRU 010 N A T GmbH Germany NAMC 16ADSP NOS AMC FRU 011 GmbH Germany NAMC 16ADSP 9 AMC FRU 012 GmbH Germany NAMC 16ADSP FRU 013 GmbH Germany NAMC STM4 FRU 014 GmbH Germany NAMC STM4 FRU 040 Rittal AG picoTCA Cooling Unit FRU 050 Rittal AG picoTCA Power Module is Wl FRU 060 GmbH Germany NAT MCH CLK E FRU 061 GmbH Germany PCIe Hub Module FRU 252 NMCH Internal critical 1
20. Rapid Spanning Tree 121 521 321 455 IE 6 4 jab Link Status BCMS396 counters Configure PCIe Virtual Switches Virtual Switch 0 none poene Virtual Switch 1 Virtual Switch 2 Backup Settings Virtual Switch 3 Board Information 5 System Information Moo Reboot NAT MCH Virtual Switch 5 Update MCH Webpage Apply Home Note You need to click apply before you can save your changes to EEPROM Save current configuration to PCIe EEPROM NETT Restore current configuration from PCIe EEPROM peas Reset switch configuration to defaults Figure 13 NAT MCH PCIe Virtual Switch Configuration via web interface Once the PCIe Virtual Switch Configuration is set up for its benefits the configuration can be stored permanently in the EEPROM by pressing the Save button In case the configuration has been mixed up but the Safe button hasn t been pressed yet the previous configuration can easily be restored by pressing the Restore button Note that the configuration might differ depending on the used backplane Therefore it might be necessary to setup the configuration again when the NAT MCH is used in a system with a different backplane Further please note that operation of PCIe on AMC backplane Port 8 11 is not foreseen by the MTCA specification and therefore is not supported by the current MCH firmware But as there are requests for having this connectivity even beyond specif
21. to pulse high Response Data 1 Completion Code 2 PICMG Identifier 0x00 Read Register Long Word Command Byte Data Field Request Data 1 Identifier 0x00 2 0 00 3 Device Number 4 Register Number 5 Register Number 3 Register Number 2 1 0 Register Number 0 lt Response Data Completion Code PICMG Identifier 2 0x00 Read Register Value 0 Read Register Value 1 Din 1 Read Register Value 2 Read Register Value 3 Write Register Long Word Command yte Data Field Request Data PICMG Identifier 0x00 0x00 Device Number Register Number Register Number 3 2 1 B 1 2 3 4 6 7 3 2 Register Number 1 Register Number 0 Version 1 25 N A T GmbH User s Manual 8 Data 0 Data 1 10 Data 2 11 Data 3 Response Data 1 Completion Code 2 Identifier 0x00 Read Register Long Word Indirect Command Byte Data Field Request Data 1 PICMG Identifier 0x00 2 0x00 3 Device Number 4 Register Number 0 5 Register Number 1 6 Register Number 2 7 Register Number 3 Response Data 1 Completion Code 2 PICMG Identifier 2 0x00 3 Read Register Value 0 4 Read Register Value 1 2 Read Register Value 2 6 Read Register Value
22. value to write Example Clk wreg 0x10 Oxff Write value Oxff to register at offset 0x10 Version 1 25 GmbH 84 User s Manual 2 10 5 2 Value to Clock Module Register Description ORs a value to a clock module register This can be used to set specific bits of a register NOTE This configuration item only should be used if the required functionality has not yet been implemented by another configuration item Please ask NAT for adding support of the needed functionality as the register layout might differ depending on the clock module version used Syntax orreg offs value Parameter Description offs register offset value value to OR with current register value Example orreg 0x10 0 01 Set bit 0 0x01 in clock module register at offset 0x10 Version 1 25 N A T GmbH 85 User s Manual 10 5 3 AND Value to Clock Module Register Description a value to a clock module register This can be used to clear specific bits of a register NOTE This configuration item only should be used if the required functionality has not yet been implemented by another configuration item Please ask NAT for adding support of the needed functionality as the register layout might differ depending on the clock module version used Syntax Clk andreg offs value Parameter Description offs register offset value value to
23. 2 9 Use CLK2 from AMC slot 9 as input for PLL reference REFO Version 1 25 N A T GmbH 65 User s Manual 10 1 2 PLL Mode Configuration Description This configuration item is used to configure the operation mode of the PLL on the clock module Syntax pll mode mode ref Parameter Description mode PLL operation mode normal mode use PLL reference input lt ref gt 2 holdover mode 3 free running mode 4 automatic selection between REFO and ref PLL reference input 1 PLL input REFO 2 PLL input 3 PLL input REF2 Example pll mode 4 1 Configure the PLL to automatically select the reference input between REFO and REFI Version 1 25 N A T GmbH 66 User s Manual 10 1 3 Clock Type Configuration Description Determines the signal type for a specific clock type Syntax type clk type Parameter Description clk clock identifier 1 CLK1 2 CLK2 3 CLK3 4 CLK1 UPDATE 5 CLK3 UPDATE type clock signal type 0 NONE 1 8 kHz 2 1 544 MHz 3 2 048 MHz 4 3 088 MHz 5 4 096 MHz 6 6 312 MHz 7 8 192 MHz 8 8 448 MHz 9 16 384 MHz 10 19 44 MHz 11 20 MHz Stratum 3 12 32 768 MHz 13 34 368 MHz 14 44 736 MHz 15 65 536 MHz 16 PCIe clock 17 PLL REFO input
24. 4 PLL input IC4 freq input clock frequency in Hz The following input frequencies are support by the PLL on the clock module 2 kHz 4 kHz 8 kHz 1 544 MHz 2 048 MHz 5 MHz 6 312 MHz 6 480 MHz 10 MHz 19 44 MHz 25 MHz 25 92 MHz 31 25 MHz 38 88 MHz 51 84 MHz 62 5 MHz 77 76 MHz 125 MHz 155 52 MHz 156 25 MHz Furthermore frequencies which are a multiple of 2 kHz are supported up to 125MHz or frequencies which are a multiple of 8 KHz As the input clock has to be divided down to 2kHz or 8kHz in this case and there is only one divider available in the PLL chip only one clock which is different from the standard clocks above can be configured except the divider is the same for 2 or more input clocks Example gen3 pll ref 1 10000000 A 10MHz clock is provided to PLL reference input Version 1 25 N A T GmbH 69 User s Manual 10 2 2 PLL Mode Configuration Description This configuration item is used to configure the operation mode of the PLL on the clock module Syntax Clk gen3 pll mode mode ref Parameter Description mode PLL operation mode 1 automatic selection between 1 4 2 forced mode reference according to parameter ref 3 free running mode ref PLL reference input only for forced mode 1 PLL input 2 PLL input IC2 3 PLL input IC3 4 PLL input IC4 Example gen3 pll mode 1 0
25. AND with current register value Example clk_andreg 0x10 Ox7f Clear bit 7 0 80 in clock module register at offset 0x10 Version 1 25 N A T GmbH 86 User s Manual 11 SRIO Module Configuration The SRIO module mounted on the NAT MCH can be configured via a text based script file similar to the configuration file used for the Ethernet switch on the NAT MCH The different functions of the SRIO module can be set by using so called configuration items Each configuration item consists of an identifier and one or more parameters Comments within the configuration begin with a hash character like Item lt lt srio_port_init gt gt initialize SRIO port The current SRIO module configuration can be downloaded from the NAT MCH e g by using the Backup Settings function in the web based configuration interface refer to chapter 9 The generated text file can be changed with a standard text editor and can be used as a starting point for a user defined configuration The changed configuration can be uploaded again e g via the web based configuration interface The following chapters describe the configuration items which can be used to configure the SRIO module For information about the functionality provided by the SRIO module please refer to the latest SRIO Module Technical Reference Manual Version 1 25 N A T GmbH 87 User s Manual 11 1 SRIO Port Ini
26. Face plate reference clock 1 input 2 input output 2 input output Clock Input Amplifier fixed modular modular Fabric Clock Support PCIe HCSL HCSL HCSL Spread Spectrum support F_CLK e Fat Pipe Mezzanines e e e PCIe XAUI PCIe XAUI Variants SRIO SRIO PCIe 2 Uplink ports at face plate XAULSRIO 2 GbE PCIe Support Gen 1 M GbE Support 1 5 6 1 5 6 1 5 6 Management Software External User Interface RMCP RMCP RMCP RMCP NATview Graphical User Interface 1 Table 2 Variants and Features Version 1 25 N A T GmbH User s Manual 2 2 NAT MCH Gen2 Connectors and Indicators at the Face Plate AMC PM Cu Indicator Leds Ejector Handle Blue Led External Clock Console Interface Input for Telecom 5232 19200 8 n 1 Clocking Option 100 BaseT 1000BaseT Mangement Link Farbric A uplink Figure 5 NAT MCH Gen 2 Front Panel 2 2 1 LED Indicators The NAT MCH is equipped with two sets of indicator LEDs e Aindicator LEDs according to 0 specification 16 indicator LEDs displaying the status of AMC modules Cooling Units and Power Modules The four AMC 0 conformant LEDs are assigned to the following functions Blue LED Blue LED function according to AMC O Red LED severe fault Green LED MCH has taken over role of primary MCH Yellow LED MCH is redundant Th
27. Shelf manager to adapt the fan speed based on temperature events from the AMCs Emergency shutdown disabled FRU on critical non recoverable event Shelf manager shuts down an AMC if a threshold based sensor reaches the critical or non recoverable limit SYSTEM on critical non recoverable event Shelf manager shuts down all AMCs if a threshold based sensor reaches the critical or non recoverable limit Send SEND MSG confirmation no If set to yes the Shelf Manager immediately returns a to SMS SEND MSG response to the BMC as a confirmation that the SEND MSG command was received This SEND_MSG confirmation does not contain response data to the embedded request As soon as the Shelf Manager receives the response from the targeted destination the response is embedded into another SEND MSG response that is delivered to the BMC The BMC can distinguish between the first and the second received SEND by checking the data contents of the received SEND MSG response Please refer to IPMI Specification V2 0 sections about bridged messages for more details Use external Shelf Manager no Allows an external Shelf manager to operate via the RMCP interface If enabled the local shelf manager is suspended Please note that usage of an external Shelf Manager is not yet supported 5 4 3 Carrier Manager Configuration 5 The carrier manager controls the communication and operation of the AMCs power modules and cooling units Config
28. akie ineeie sae isna 69 10 2 2 PLE Mode Configuration getto te areis he Ups 70 10 2 3 Clock Output Configuration iat t ard repetere ra 71 Version 1 25 N A T GmbH 5 User s Manual 10 3 CLOCK MODULE VERSION e n ERE Pee e e ELE NS 74 10 3 1 PLL Reference Input Configuration eese esee eene trennen eene tenerent 74 10 5 2 Mode ep tc e Ea de ud 75 10 3 3 PLL OC Clock Output Configuration esses eene teen 76 10 3 4 Clock Output Config urotion ooi ert 76 10 4 CLOCK MODULE VERSION PHYSICS esee ense nennen een 80 10 4 1 Clock Output Configuration a er rte perdere ere ere i aeni 80 10 4 2 Clock Termination Configuration eese eee eene enne 82 10 5 COMMON COMMANDS FOR ALL CLOCK 84 10 5 1 Write Clock Module Register ee de ee tr d eere tpe 64 10 5 2 OR Value to Clock Module Register eese enne nennen rennen 65 10 5 5 AND Value to Clock Module Register eese 56 11 SRIOMODULE CONFIGURATION ssccccsssssssssssccessssccccssscccssssccccessccccsscascecssscccessecescsssscccnssscecees 87 111 SRIO PORT INITIALIZATION 88 11 2 WRITE SRIO MODULE BYTE 5
29. by any external Shelf or System Manager to control the operation of the NAT MCH and the system The onboard CPU supports TCP IP and RMCP accesses 2 3 5 Console Port USB Telnet The console port provides an interface to the Command Line Interface CLI of the onboard CPU The console interface can be used to set the operational and configuration parameters of the NAT MCH Once the IP configuration has been set the console interface can be switched to a Telnet session by connecting via Telnet In case a password had been configured for a Telnet session a check of this password is done when starting the Telnet session For details regarding configuration of a Telnet password please refer to chapter 8 Version 1 25 N A T GmbH 21 User s Manual 2 3 5 1 Console Port of V3 x Gen3 USB The NAT MCH of generation 3 V3 x uses a USB interface as console port The console port provides a USB CDC type of interface Interoperability has been tested with the standard drivers included in Windows 2000 XP Vista Windows 7 and Linux When connecting first time to a windows machine a new device will be installed NAT MCH console To successfully complete the installation a device information file must be provided to the system N AT provides the required mch inf text file for download from its web site or ftp server Under Windows the new device can be accessed by standard terminal programs like terrater
30. lost profits lost savings delays or interruptions in the flow of business activities including but not limited to special incidental consequential or other similar damages arising out of the use of or inability to use this product or the associated documentation even if N A T or any authorized N A T representative has been advised of the possibility of such damages The use of registered names trademarks etc in this publication does not imply even in the absence of a specific statement that such names are exempt from the relevant protective laws and regulations patent laws trade mark laws etc and therefore free for general use In no case does N A T guarantee that the information given in this documentation is free of such third party rights Neither this documentation nor any part thereof may be copied translated or reduced to any electronic medium or machine form without the prior written consent from N A T GmbH This product and the associated documentation is governed by the N A T General Conditions and Terms of Delivery and Payment Version 1 25 N A T GmbH 3 User s Manual Table of Contents DISCLAIMER d 3 H Ic 4 erro DL 8 LIST OF TABLES QP 8 1 BOARD SPECIFICATION 11 NATSMGELEEN
31. modules For any slot which is populated by an AMC module a green LED on the face plate is lit For all AMC modules found in the system the reads in the FRU information and sensor data records If power negotiation is successful it directs the power module to power up the AMC modules either in the activation sequence defined in the Carrier Activation and Current descriptor record of the backplane FRU device or if the record is not found according to the site number Version 1 25 N A T GmbH 31 User s Manual 4 3 Local Shelf Manager The local Shelf manager of the NAT MCH provides management of the following resources within an uTCA system Sensor Event Log SEL Temperature management and Cooling Unit control 4 3 1 Sensor Event Log The MCH provides a System Event Log SEL that stores all events that occur in an uTCA system The stored events are kept in the MCH s DRAM and are not stored into persistent memory Events can be read by an application using the IPMI message SEL ENTRY In the MCH s default configuration events are removed from the SEL on read Keeping events in the SEL after reading them can be configured using the MCH configuration menu refer to chapter 5 To remove single events from the SEL in this configuration IPMI message DELETE SEL ENTRY has to be sent to the to clear the whole SEL message CLEAR SEL has to be sent to
32. prints all configuration settings mchcfg menu based utility to set and modify configuration parameters ip Basic Network configuration IP addresses Parameters which should keep their values can be acknowledged by simply hitting return For details about the MCH configuration please refer to chapter 5 4 5 2 Configuration via the web Interface configuration parameters are accessible via the web interface as well Please refer to Chapter 9 for the usage of the web interface It can be necessary to setup a basic configuration like the IP address of the MCH before the web interface can be used 5 3 Customizing the Network Configuration The MCH requires IP address parameters to be adapted to make it working in a company s network environment Entering ip at the command line will show the actual configuration and allow to change these parameters by line editing the displayed values IP address parameters must be entered in form factor If an IP address is configured to 0 0 0 0 it will be ignored by the MCH In case a non zero gateway IP address is configured the related routing configuration will be performed automatically when the MCH starts up After all changes are done the system asks for confirmation of the new configuration If the new values shall be written into the I2C EEPROM the question should be answered with y and the new values will be become effective after the next power c
33. three AMC 0 conformant LEDs are assigned to the following functions Blue LED Blue LED function according to Red LED severe fault Bicolor Led Green Yellow o Green is primary management controller o Yellow LED MCH is redundant standby The 16 bi color LEDs give an immediate visual feedback of the status of the corresponding FRU device Their functions are green AMC CU or PM module fully inserted and operational green blinking activation de activation under progress red module faulty or did not progress into operational state communication fault red blinking PM error e g power supply faulty 2 5 2 SMA Connectors External Clock Reference This input output can be used to feed an external reference clock into the NAT MCH or provide a reference clock for other systems The input clock can be distributed by the Telecom Clocking Module to any AMC slot in the system The output can provide a clock sourced from any of the AMCS or a local clock generated by the onboard PLL For technical data of this input output please refer to the NAT MCH Features section Version 1 25 N A T GmbH 25 User s Manual 2 5 3 Dual GbE Uplink Port The NAT MCH has two 10 100 1000 BaseT autonegotiation uplink ports which are connected to the internal GbE switch circuit The ports can be joined together in link aggregation mode to double the uplink throughput performance In alternative they ca
34. 0 0 non recoverable 200 FRU 253 RittalRES PicoTCA FRU 254 MCH LogicalShM Resource discovery done 7 3 1 1 Supported Java Releases NATView requires Sun Java 1 6 or higher This Sun Java version was chosen to guarantee the largest possible JRE installation basis on Window Linux and Mac OS X systems If in doubt enter java version to a command prompt window An appropriate copy of the Sun JRE can be obtained at java sun com 7 3 1 2 Getting NATView NATView be downloaded from the N A T FTP server ftp nateurope com Username is natmch password is natmch as well For NAT MCH firmware 1 24 or higher download latest released archive for an older firmware version use NATView 1 25 Both archives contain all necessary data and documentation Version 1 25 N A T GmbH 51 User s Manual 7 3 1 3 Installing and running NATView NATView is usually distributed as a zip archive The following steps will install NatView on the system 1 Extract the content of the zip archives to an empty directory This empty directory will be the root directory of the application 2 Start the application from within the root directory Otherwise the application cannot find the board images in the subdirectory images From the command line type cd root directory java jar natview jar 7 3 1 4 Further information More detailed information can be found in the documentat
35. 0 134 192 168 0 134 56 84 bytes of data 64 bytes from 192 168 0 134 icmp seq 1 ttl 255 time 0 276 ms 64 bytes from 192 168 0 134 icmp seq 2 ttl 255 time 0 247 ms 64 bytes from 192 168 0 134 icmp seq 3 ttl 255 time 0 249 ms 64 bytes from 192 168 0 134 icmp seq 4 ttl 255 time 0 235 ms 192 168 0 134 ping statistics 3 packets transmitted 4 received 0 packet loss time 3001ms rtt min avg max mdev 0 235 0 244 0 258 0 008 ms Appendix B 5 Test2 ping between control host and gateway This test shows whether the control host can communicate with the gateway and vice versa using the ping tool Ping the gateway from the control host control ping 132 147 160 70 PING 132 147 160 70 132 147 160 70 56 84 bytes of data 64 bytes from 132 147 160 70 icmp seq 1 tt1l 64 time 0 237 ms 64 bytes from 132 147 160 70 icmp seq 2 ttl 264 time 0 193 ms 64 bytes from 132 147 160 70 icmp seq 3 tt1l 64 time 0 192 ms 132 147 160 70 ping statistics packets transmitted 3 received 0 packet loss time 1999ms rtt min avg max mdev 0 192 0 207 0 237 0 024 ms ping the control host from the gateway Version 1 25 N A T GmbH 115 User s Manual root natcg2 ping 132 147 160 47 PING 132 147 160 47 132 147 160 47 56 84 bytes of data 64 bytes from 132 147 160 47 icmp seq 1 tt1l 64 time 0 232 ms 64 bytes from 132 147 160 47 icmp seq 2 tt1l 64 time 0 193 ms 64 bytes fr
36. 06 Version 1 25 N A T GmbH 105 User s Manual MCH Signal MCH Signal 80 0 216 14 i feo 85 GND 6 Table 10 Connector Tongue 2 Version 1 25 N A T GmbH 106 User s Manual Appendix A7 Hub Module Connector Tongue 3 MCH Signal MCH Signal 6 TRSVD RD 15 GND Version 1 25 N A T GmbH 107 User s Manual 61 66 85 15 68 RxFF5 0 69 5 2 12 6 9 GND GND 8 754 TxFDb 6 Version 1 25 N A T GmbH 108 User s Manual MCH Signal MCH Signal 807 RxFF6 4 8 10 82 GND 0 9 GND 86 Table 11 MCH Connector Tongue 3 Version 1 25 N A T GmbH 109 User s Manual Appendix A8 Hub Module Connector Tongue 4 MCH Signal MCH Signal 6 TRSVD RD 165 GND Version 1 25 N A T GmbH 110 User s Manual 42 19 GND GND GND 110 ne 685 108 66 15 8 UTXERIIX 103 69 TxFFli 02 72 TxFGli RxFGl 9 GND GND _ 75
37. 11 4 AND Value to SRIO Module Register Description a value to a SRIO module byte register This can be used to clear specific bits of a register Syntax srio andregb offs value Parameter Description offs register offset value value to AND with current register value Example srio andregb 0x10 Ox7f Clear bit 7 0x80 in SRIO module register at offset Ox10 Version 1 25 N A T GmbH 91 User s Manual 11 5Write SRIO Module Switch Register Indirect Long Description Write a value to a SRIO module switch register Please refer to the Switch to Fabric Port Mapping tables in the SRIO Technical Reference Manuals to select correct switch dev parameter Please refer to the SRIO switch User Manual for a description of the switch registers Syntax srio wregil switch dev offs value Parameter Description switch dev switch device number 0 1 offs register offset value value to write Example srio wregil 0x10 Oxff Write value Oxff to SRIO module switch register at offset Ox10 Version 1 25 N A T GmbH 92 User s Manual 2 11 6 Value to SRIO Module Switch Register Indirect Long Description ORs a value to a SRIO module switch register This can be used to set specific bits of a register Please refer to the Switch to Fabric Port Mapping tables in the SRIO Technical Reference Manuals to select correct switch dev parameter Pl
38. 18 PLL input 19 EXT REF input 20 100 us tick 21 SW CLK 22 SYNC CLK 23 HOLDOVER NOTE Refer to the Clock Module Technical Reference Manual for a description of the different clock types Example type 1 3 Output a 2 048 MHz clock signal on Version 1 25 N A T GmbH 67 User s Manual 10 1 4 Clock Output Configuration Description Enables or disables the output of a specific clock type Syntax Clk out clk slot ena Parameter Description clk clock identifier he 2 CLK2 3 CLK3 4 CLK1 UPDATE 5 CLK3 UPDATE slot AMC slot number 1 12 optional only used if reference source is CLK1 or CLK2 else ignored ena 0 disable clock output 1 enable clock output Example clk_out 1 lg dh Enable output of CLK1 on AMC slot 1 Version 1 25 O N A T GmbH 68 User s Manual 10 2Clock Module Version 3 x 10 2 1 PLL Reference Input Configuration Description This configuration item is used to configure the reference inputs of the PLL on the clock module The clock source that is provided to one of these PLL reference inputs has to be configured via the configuration item gen3 out described below Syntax Clk gen3 pll ref ref freq Parameter Description ref selects the reference input of the PLL 1 PLL input 2 PLL input IC2 3 PLL input IC3
39. 2 7 or later If the MCH works with firmware 2 7 or later it is possible to update several components of the MCH like firmware bootloader microprocessor etc via web interface For this purpose a TAR compressed collection of firmware updates is supplied by N A T Update procedure e Download TAR file to local space Access the web interface select Update from the web interface and open the saved file e Clicking Upload displays a table with the current version of each updateable component of the MCH as well as the new version provided by N A T Select which components should be updated if the version in the TAR file is newer than the current version the update is selected automatically If the version provided in the TAR file is older than the current version and Update this device is checked manually the component will be downgraded Version 1 25 N A T GmbH 46 User s Manual e After selecting the components to be updated click Update and WAIT UNTIL UPDATE HAS COMPLETED The update may take several minutes depending on which and how many components are upgraded A notification is given when the update is completed and successful Note The update function works with TAR compressed files provided by N A T only Uploading a binary firmware file or a zip compressed firmware file in the web interface will not work If no updateable components are visible in the second step
40. 3 Write Register Long Word Indirect Command 2 e Data Field PICMG Identifier 2 0x00 0x00 Device Number Register Number 0 Register Number 1 Register Number 2 Register Number 3 Data 0 Data 1 Data 2 Data 3 Completion Code PICMG Identifier 2 0x00 Request Data 0 1 Response Data 1 1 Byte Data Field 1 Identifier 2 0x00 2 0x00 3 Device Number 4 Length 1 2 Request Data Response Data Completion Code PICMG Identifier 2 0x00 Version 1 25 N A T GmbH 120 User s Manual 3 N Data Write Block Command Byte Data Field Request Data 1 Identifier 0x00 2 0 00 3 Device Number 4 Length 5 N Data Response Data 1 Completion Code 2 PICMG Identifier 0x00 Controller Vendor specific extensions NetFn is IPMI_NETFN_CONTRO define IPMI_CMD_READ_REG 0x01 define IPMI CMD WRITE 0x02 define IPMI CMD SET BIT 0x03 define CMD CLR 0x04 define IPMI CMD PULSE BITL 0x05 define CMD PULSE BITH 0x06 define IPMI READ 0x07 define IPMI CMD WRITE REG 0x08 define IPMI READ REG 0x09 define IPMI CMD WRITE REG LWI 0x0a define
41. ATU RES terere He De ERE CREER E 12 LAT oCPUdr d Memory e BERE tees 12 7 2 PME Gnd Management p te t pe T eae Eee ee 12 1 1 3 Supported Fabrics and Compliance eese eene trennen 12 1 1 4 Clock o teet 12 ERS External Glock Reference Input t i 12 JO XEEDS euet MM LEM LT RI LM DPA DUI M DEL CCR ST 13 2 OVERVIEW NAT MCH PRODUCT 4 4 4 01 no eee tn oeste tee seen ae seen ee esee eee enne eta 14 2 NAT MCH VARIANTS rent ER RE EP CEPR n Os 17 22 GEN2 CONNECTORS AND INDICATORS AT THE 4 18 2 2 1 LED Indicators aea esteem mete HE 16 2 2 2 SMA Connector External Clock Reference esee 19 2 2 5 100 BaseT interface primary Management Interface Port eese 19 2 2 4 GigaBit Ethernet GbE Uplink Port eese eene tenen nennen eene 19 2 222 Console Dbe Portas s aod e Up e ER uc Hi e 19 23 GEN3 CONNECTORS AND INDICATORS AT THE 20 2 3 1 ee eS 20 2 3 2 SMA Connectors External Clock Reference eese eene ener 21 2 3 3 Dual GbE Uplink Port e tiir t d tre hayes a Ree e e Rd ege n 21 2 3 4
42. CH firmware to behave in exactly the same way as it did with firmware releases before 2 5 there was only one single ShM CM unit for the host software This document shall describe the necessary changes that need to be made to make a MicroTCA host application compatible to the NAT MCH native mode Appendix D2 Changes of the Firmware 15 1 Overview The main differences between the compatible and the native mode are as follows Compatible Mode Native Mode Single Shelf Carrier Manager Separate Shelf and Carrier Manager AMC access via single bridged messages AMC access via double bridged messages Resource Browsing using Shelf Carrier Resource browsing using a combination of Manager repository the Shelf and the Carrier Manager repository SEL uses AMC sensor numbering SEL uses Shelf Manager sensor numbering Hot swap detection via SEL Hot swap detection via SEL and Carrier Manager repository these new features of the native mode will be covered more detailed in the following sections of this document The overall structure of the NAT MCH firmware from release 2 5 on is illustrated in the following diagram Version 1 25 N A T GmbH 124 User s Manual Host Application Carrier Manager To access the different levels of a system a host application needs to know where to send its IPMI messages to talk to the Shelf Manager directly using standard IPMI messages e To talk to th
43. Commands for all Clock Modules 02 05 2013 Added chapter 9 1 PCIe Virtual Switch Configuration hn 16 05 2013 Added some missing descriptions in MCH configuration WW 1 24 05 06 2013 chapter 5 4 2 added description of Send SEND MSG Ww confirmation to SMS flag 25 06 2013 Added SMA connector description Se 20 08 2013 Typo correction reworked wording Se 28 08 2013 Added hint the PCIe is not supported on AMC Ports 8 11 Te 16 12 2013 Corrected Tonguel 4 connector pin description te 1 25 4 02 2014 Added Faceplate description for Gen3 with PCIe se Gen3 option Added chapter 2 5 MCH M4 Update of Table 2 MCH Variants and Features Added Block Diagram MCH M4 18 03 2014 Added Appendix F Updating the Backplane EEPROM hn via MCH CLI 6 05 2014 Added chapter 12 Unmanaged AMC Module hn Configuration 26 05 2014 Added chapter 11 SRIO Module Configuration ww Version 1 25 N A T GmbH 134
44. Description change the 802 1P protocol settings For more details please refer to the Ethernet Switch Configuration Manual Port Mirroring Provides a graphical configuration menu to change the port mirroring settings For more details please refer to the Ethernet Switch Configuration Manual Jumbo frame Provides a graphical configuration menu to change the Jumbo frame settings For more details please refer to the Ethernet Switch Configuration Manual Link Provides a graphical configuration menu to Aggregation change the Link Aggregation settings For more details please refer to the Ethernet Switch Configuration Manual Configure Provides a graphical configuration menu to PCIe Virtual change the PCIe Virtual Switch settings For Switches more details please refer to the chapter PCIe optional Virtual Switch Configuration optional Maintenance Functions Backup Backup current configuration settings to the Settings onboard FLASH or an external file or load settings from the onboard FLASH or an external file Save current configuration to onboard FLASH memory Saves the current configuration e g of the Ethernet switch or the clock module to the onboard FLASH memory saved configuration can be loaded during power up if enabled via the MCH configuration refer to chapter 5 Restore current Restores the configuration from the onboard configuration FLASH memory fro
45. FROM COMPATIBLE TO NATIVE MODE 124 Version 1 25 N A T GmbH 6 User s Manual APPENDIX D 1 PREPAC Eis 124 APPENDIXD2 CHANGES OF THE 0 nennen 124 15 1 LOAA EATA T EA 124 15 2 SEPARATE SHELF AND CARRIER 0 0000 nnne nnn rene 125 15 3 AMC ACCESS VIA DOUBLE BRIDGED MESSAGES sese 126 15 4 RESOURCE BROWSING USING A COMBINATION OF SHELF MANAGER AND CARRIER MANAGER IREPOSTEORY 52 59 ttim os tte e ets Eee Seti de Bass 127 15 5 SEL USES SHELF MANAGER SENSOR NUMBERING 0 0 127 15 6 HOTSWAP DETECTION VIA SEL AND CARRIER MANAGER REPOSITORY 128 APPENDIX ALTERNATIVE COOLING UNIT SCHEME eee tona sese ette 129 APPENDIX UPDATING THE BACKPLANE EEPROM VIA 131 DOCUMENTS HISTORY 132 Version 1 25 N A T GmbH 7 User s Manual List of Figures Figure 1 NAT MCH Gen 2 Block Diagram with PCIe Mezzanine Option
46. I For more information about OpenHPI please refer to the README files that come with the OpenHPI package 7 3 3 2 Configuring OpenHPI By default simply calling configure will build all plugins that can be built all other plugins will be silently disabled If a plugin should be disabled any flags that are passed to configure will be passed to the configure program For communication between OpenHPI and the NAT MCH at least the plugin ipmidirect must be enabled this is the default in OpenHPI 2 7 3 Please try configure help for more information on the options Further configuration information can be obtained from the README file provided in the openhpi 2 7 3 package 7 3 3 3 Compiling OpenHPI OpenHPI can be compiled by calling make The official release of OpenHPI 2 7 3 contains a bug that is fixed since 2 8 x versions src event c 156 warning error might be used uninitialized this has to be fixed by replacing int error by int error SA_OK in the file src event c Version 1 25 N A T GmbH 54 User s Manual 7 3 3 4 Installing OpenHPI OpenHPI is installed by calling make install Note that there is no daemon configuration file installed by default so one has to copy it manually gt cp openhpi conf example usr local etc openhpi openhpi conf Alternatively the provided configuration file openhpi nat conf can be used for configuration
47. K2 AMC 23 CLK2 AMC 24 CLK2 AMC 25 CLK3 AMC 26 CLK3 AMC 27 CLK3 AMC 28 CLK3 AMC 29 CLK3 AMC 30 CLK3 31 CLK3 AMC 32 CLK3 AMC 33 CLK3 AMC 34 CLK3 AMC 35 CLK3 AMC 36 CLK3 AMC 37 CLK1 Updat 38 CLK3 Updat 39 EXT sing 40 EXT sing 41 EXT singl 42 EXT sing 43 EXT 44 EXT 45 PLL 46 PLL IC2 47 PLL 48 PLL ICA 49 PLL IGS 50 PLL IC6 SIN 52 PLL ICS8 53 PLL SYNC1 54 PLL SYNC2 55 PLL SYNC3 source cl SYNC CLK input OCk identifier 0 disabled 1 5 CLK 2 CLK 3 CLK AMC AMC AMC N Version 1 25 N A T GmbH 77 User s Manual 4 CLK1 AMC 4 5 CLK1 AMC 5 6 CLK1 6 7 CLK1 AMC 7 8 CLK1 8 9 CLK1 9 10 CLK1 10 11 CLK1 AMC 1 12 CLK1 AMC 12 13 CLK2 AMC 1 14 CLK2 AMC 2 15 CLK2 AMC 3 16 CLK2 AMC 4 17 CLK2 AMC 5 18 CLK2 AMC 6 19 CLK2 AMC 7 20 CLK2 AMC 8 21 CLK2 AMC 9 22 CLK2 AMC 10 23 CLK2 AMC 11 24 CLK2 AMC 12 25 CLK3 AMC 1 26 CLK3 AMC 2 2 CLK3 AMC 3 28 CLK3 AMC 4 29 CLK3 AMC 5 30 CLK3 AMC 6 31 CLK3 AMC 7 32 CLK3 AMC 8 33 CLK3 AMC 9 34 CLK3 AMC 10 35 CLK3 AMC 11 36 CLK3 AMC 12 37 CLK1 Updat 38 CLK3 Updat 39 EXT single ended 1 40 EXT single ended 2 41 EXT singl
48. K2 AMC 11 28 CLK2 AMC 12 29 o 30 X 9T X 32 X 33 CLK3 AMC 1 34 CLK3 AMC 2 35 CLK3 AMC 3 36 CLK3 AMC 4 37 CLK3 AMC 5 Version 1 25 N A T GmbH 80 User s Manual 38 CLK3 AMC 6 39 CLK3 AMC 7 40 CLK3 8 41 CLK3 AMC 9 42 CLK3 AMC 10 43 CLK3 AMC 11 44 CLK3 AMC 12 Src source clock identifier 0 disabled 1 CLK1 1 2 AMC 2 3 3 4 CLK1 4 5 CLK1 AMC 5 6 CLK1 AMC 6 7 CLK1 AMC 7 8 CLK1 8 9 CLK1 AMC 9 10 CLK1 AMC 10 11 CLK1 11 12 CLK1 AMC 12 13 X 14 X T5 X 16 X 17 CLK2 AMC 1 18 2 2 19 CLK2 3 20 CLK2 AMC 4 21 CLK2 AMC 5 22 CLK2 AMC 6 23 CLK2 AMC 7 24 CLK2 AMC 8 25 CLK2 9 26 2 AMC 10 27 CLK2 AMC 11 28 CLK2 AMC 12 41 100MHz OSC only with HCSL option NOTE Refer to the Clock Module Technical Reference Manual for a description of the different clock types Example clk_phys_out 1 41 Output the 100MHz OSC clock output on CLK 1 of AMC slot 1 Version 1 25 N A T GmbH 81 User s Manual 10 4 2 Clock Termination Configuration Description Determines if 100 R termination in the multiplexer is enabled or disabled Syntax phys termination dst ter
49. LAN security protocol configuration qos cfg Quality of service menu priority configuration 4081 802 1p Quality of service configuration mirr cfg Ethernet port mirroring configuration mac amc Print MAC addresses for MAC slots As MAC addresses are read from the MCH s gigabit controller only AMCs are detected which already have had IP traffic routed through the MCH s gigabit controller to that point in time The mapping to the AMC slot number is done using the different ports of the gigabit controller show xlinkinfo Print information about XAUI ports show xmact Print XAUI MAC table diag Menu driven diagnostic tool This menu shall be used on explicit advice only fan ctl FAN control Command to get fan properties and speed level and to set the fan speed level shutdown fru id all Graceful shutdown of FRU or all FRUs fru start fru id Graceful start for FRU Table 3 List of CLI commands Version 1 25 N A T GmbH 59 User s Manual 9 Webserver The NAT MCH has an integrated embedded webserver which allows users to view and change configuration parameters of the NAT MCH Before the webserver can be used it has to be enabled via the MCH configuration refer to Chapter 0 Web access The onboard webserver can be accessed with any standard web browser by entering the IP address of the NAT MCH s management port into the browser s address line Afte
50. MCH Module single width full size M4 double width full size Processor MCH Gen2 ColdFire MCF5470 200 MHz MCH Gen 3 MCH LC Coldfire MCF54452 266 MHz Front I O 2 RJ45 connectors 1 2 SMA and 1 Mini USB connector Main Memory 32 64 MByte SDRAM Flash PROM 16 32 64 MByte Flash PROM on board programmable Operating System OK 1 Base Module 8 5W typ Mezzanine Modules for tongue 2 3 4 adding Power consumption Clock SW PCIe 6W IW per active connection XAUI 29W SRIO 15W Environmental Temperature operating to 65 C with forced cooling conditions Temperature storage 40 C to 85 C Humidity 10 to 90 rh non condensing PICMG AMC 0 Rev 2 0 PICMG AMC 2 Rev 1 0 PICMG SFP 0 Rev 1 0 System Fabric Plane Format IPMI Specification V1 5 Rev 1 0 PICMG pq TCA 0 Rev 1 0 Table 1 NAT MCH Technical Data Standards compliance Version 1 25 N A T GmbH 11 User s Manual 1 1 NAT MCH Features 1 1 1 CPU and Memory Gen 2 Freescale ColdFire 5470 200MHz Gen 3 LC Freescale ColdFire 54452 266MHz DRAM 64MB FLASH 32MB 1 1 2 IPMI and Management The MCH supports Management and IPMI Interfaces for 12 AMCs e 2 cooling units e 1 4 power units 1 1 3 Supported Fabrics and Compliance Fabric A Gigabit Ethernet Option non blocking low latency Layer 2 Gigabit Ethernet switch Support for 12 AMCs and 1 GbE Uplink Por
51. Manual 2 3 NAT MCH Gen3 Connectors and Indicators at the Face Plate GbE Uplink1 GbE Reference Clock USB Management Port Uplink 2 In Out Console Blue Led Primary Secondary Indicator Led Extraction Handle 7 CUPM 6 1 Fault Led LNK FRU Status Leds Fat Pipe Uplinks Fat Pipe Link Status Uplinks Figure 6 NAT MCH Gen 3 Front Panel with SRIO XAUI uplink optional 12 n CU 6 AMC 1 PCle Link Figure 7 NAT MCH Gen 3 Front Panel with PCIe option 2 3 1 LED Indicators The NAT MCH is equipped with three sets of indicator LEDs e 3 indicator LEDs according to 0 specification e 16 indicator LEDs displaying the status of AMC modules Cooling Units and Power Modules e 2 link indication LEDs for Fatpipe Link Status SRIO XAUI option only OR 12 link indication LEDs for PCIe link status PCIe Gen3 only The three AMC 0 conformant LEDs are assigned to the following functions Blue LED Blue LED function according to 0 e Red LED severe fault Bicolor Led Green Yellow o Green is primary management controller o Yellow LED MCH is redundant standby Version 1 25 N A T GmbH 20 User s Manual The 16 bi color LEDs give an immediate visual feedback of the status of the corresponding FRU device Their functions are green AMC CU or PM module fully inserted and operational green blinking activation de activation under pro
52. able if MCH is not in Managed Ethernet Mode Figure 2 NAT MCH Gen 3 Block Diagram with PCIe Mezzanine Option shown Backplane IPMI Controller Local Bus Tongue 1 Fabric A RJ45 D 1000BaseT 8 Uplink EN Gigabit Ethernet Uplink Optional Face Plate NAT MCH Block Diagram Figure 3 Gen LC Block Diagram with PCIe Mezzanine Option shown Version 1 25 N A T GmbH 15 User s Manual 12 AMCs 2CUs 2PMs 1000Base T SGMII Rj 45 gt lt Gigabit Ethernet 1000Base T Ph y 1000Base X 81 45 lt lt Fabric Mil 5 to 12 AMCs Gigabit Ethernet micro USB Atmega UART SPI 1000Base X 1000Base X USB 1602 lt gt Freescale SWITCH Update Fabric A 5 to second 2 MCF54452 NAT MCH 64MB SDRAM bus 8 DDR2 g Coldfire CPU local bus 64MB s FLASH 1000 5 Base X p to Hub LEDs 83 Module a IPMB L Sri s8 8 2 ER and to secon Altera NAT MCH gz 0 to CUs and 35 FPGA 3 PC CLK Clock ext CLK connector transceiver FRU Mo
53. ad Disabled If enabled SEL entries are not deleted upon read Allocate SEL non volatile yes Allocate system event Log in non volatile MRAM on RAM Ignore version change sensor Gen3 MCH Ignore version change sensor if enabled Version 1 25 N A T GmbH 40 User s Manual 5 4 5 GbE Switch Configuration 7 The GbE Switch Configuration is used to configure the Ethernet switch located on the base board of the NAT MCH The different configuration options of the GbE switch are described in a separate manual refer to Ethernet Switch Configuration Manual Configuration Option default Description configuration source Specifies the source for the GbE switch configuration no configuration e load from FLASH Ignore Backplane FRU Info no Ignore backplane FRU information in switch management platform and assume a standard mapping scheme use show_fruinfo 253 to display assumed backplane mapping Version 1 25 N A T GmbH 41 User s Manual 5 4 6 CLK Module Configuration 8 optional The following options are used to configure the CLK module optionally mounted on the NAT MCH The clock module can be configured via a text based configuration file For more information please refer to chapter 10 Configuration Option default Description configuration source none Specifies the source of the CLK module configuration no configuration l
54. ading and writing registers on the hub clock mezzanine modules Read Register Command Byte Data Field PICMG Identifier 0x00 0x00 Register Number Completion Code PICMG Identifier 0x00 Register Read value Request Data Response Data Write Register Command Byte Data Field Request Data 1 Identifier 0x00 2 0 00 3 Register Number 4 Register Write value Response Data 1 Completion Code 2 PICMG Identifier 0x00 Set Bit Command Byte Data Field Request Data 1 Identifier 0x00 2 0 00 3 Register Number 4 Bit Number 0 7 to set Response Data 1 Completion Code 2 Identifier 0x00 Clear Bit Command Byte Data Field Request Data 1 Identifier 0x00 2 0 00 3 Register Number 4 Bit Number 0 7 to clear Version 1 25 N A T GmbH 118 User s Manual Response Data 1 Completion Code PICMG Identifier 0x00 Pulse Bit low Command Byte Data Field Request Data 1 Identifier 0x00 2 0 00 3 Register Number 4 Bit Number 0 7 to pulse low Response Data 1 Completion Code 2 PICMG Identifier 0x00 Pulse Bit high Command Byte Data Field Request Data 1 Identifier 0x00 2 0 00 3 Register Number 4 Bit Number 0 7
55. an be set by the MCH configuration menu see Chapter Configuration allow the flexible adoption of the PCIe hub to a certain environment The PCIe module V2 3 Gen3 features 12 LEDs on the face plate driven by the FPGA to visualize the link status of the PCIe connections Version 1 25 N A T GmbH 29 User s Manual 3 1 3 2 SRIO Switching Module x24 and x48 The SRIO module x24 x48 supports switching of 1 or 4 lanes SRIO for up to 6 12 AMC slots The baud rate of each port can be selected independently between 1 25 Gbit s 2 5 Gbit s and 3 125Gbit s With the help of E Keying the baud rate of the respective MCH SRIO port is configured according the supported baud rate of the connected AMC 3 1 3 3 XAUI 10 Gigabit Ethernet Switching Module X24 and X48 The XAUI module X24 X48 supports switching of IOGDE for up to 6 12 AMC slots and a 2 MCH Additional the XAUI Module can be equipped with an optional face plate module available in Q4 2008 With this module there are also two 10 gigabit Ethernet interfaces accessible at the face plate of the MCH There are two versions of the face plate module available a CX4 and a SFP version The version offers two CX4 interfaces 10 GbE via copper cable The SFP version offers two SFP cages These cages can be assembled with any SFP transceiver There are different SFP transceivers available from different vendors for different optical interfaces If a c
56. ase Update webserver description in ch 9 WW 1 18 19 09 2009 Added remark regarding GbE Uplink porton NAT MCH ww Gen3 in chapter 0 12 10 2009 Added information for updating by Web Interface cs Added update_mch instructions for Firmware 2 7 or later 14 10 2009 Added ipmitool usage example to update firmware of a cs power module 07 12 2009 Added missing NAT IPMI extensions cs 17 02 2010 Added mchcfg gt clock e keying cs 09 03 2010 Updated Webserver chapter 9 ww 09 03 2010 Update clock configuration chapter 10 ww 25 03 2010 Removed config flag ekey for PCIe hl 26 03 2010 Updated NATView related sections Also added SS Appendix D 1 19 16 07 2010 Added note to only use tar archives when using web cs interface update function Version 1 25 N A T GmbH 133 User s Manual 1 20 09 08 2010 In chapter 7 3 1 2 corrected FTP server link Ww 1 21 10 08 2011 Added Block Diagram Gen 3 LC se Added new company contact data Added restriction concerning Managed Ethernet Mode Minor changes e g labeling Changed Boot String to new value 0xd0100008 12 02 2013 Added telnet password hn 1 22 11 03 2013 Updated chapter 5 4 9 NTP Configuration 10 hn 1 23 30 04 2013 Added chapter 10 3 Clock Module Version 4 and hn 10 4 Clock Module Version Physics renamed former chapter 10 3 Common Commands for V2 x and V3 x into 10 5 Common
57. be stored in the onboard FLASH memory e g by using the Save current configuration to onboard FLASH memory function in the web based configuration interface This can be used to load the configuration for the clock module during startup of the NAT MCH Note that loading the clock module configuration from FLASH has to be enabled via the MCH configuration refer to chapter 5 4 6 The following chapters describe the configuration items which can be used to configure the clock module Note that the configuration items might differ depending on the clock module version Therefore the description of the configuration items is divided in different subchapters below For more information about the functionality provided by the clock module please refer to the latest Clock Module Technical Reference Manual Version 1 25 N A T GmbH 64 User s Manual 10 1 Clock Module Version 2 x 10 1 1 PLL Reference Input Configuration Description This configuration item is used to configure the clock source for the reference inputs of the PLL on the clock module Syntax Clk pll ref ref src slot Parameter Description ref selects the reference input of the PLL 1 PLL input REFO 2 PLL input REF1 SEC reference source CLK1 2 CLK2 3 CLK3 4 CLK1 UPDATE 5 CLK3 UPDATE slot AMC slot number 1 12 only used if reference source is or CLK2 else ignored Example pll ref 1
58. browser window and loads the Webpage N A T homepage Home Displays the Welcome page of the NAT MCH web based configuration interface Attention The most changes need a reboot of the NAT MCH to apply The webserver will show a notice if a reboot is necessary Version 1 25 GmbH 62 User s Manual 9 1 Virtual Switch Configuration optional The setup function Configure PCIe Virtual Switches provides a graphical configuration menu to change the PCIe Virtual Switch settings It is possible to configure up to 6 Virtual Switches for clustering features For each Virtual Switch an Upstream AMC port and the corresponding downstream ports can be selected separately Furthermore a non transparent AMC port can be configured on Virtual Switch 0 NAT MCH by N A T Setup Show MCH Configuration PCIe Virtual Switch configuration Change MCH Cenfiguration Switch BOMS396 166 AR Select Host AMCs Upstream for each virtual switch that shall be enabled first 10 VLAN Select Host AMCs Non Transparent Upstream for each virtual switch that shall be enabled afterwards 802 Select which AMCs shall be connected to each virtual switch as downstrean in the 8021 Port Mirroring Ys let Gal we ell We ede d ee le Vitel waa M M M M Link Aggregation AMC AMC 8 1 ena
59. ce selected by fru 14 For valid FRU numbers please refer to MTCA R1 0 table 3 3 show cu Show cooling unit show pm Power Module Status Shows the actual power allocation status for all AMC modules and Cooling Units Version 1 25 N A T GmbH 58 User s Manual show sensorinfo fru id Shows the sensor values of the selected FRU version Print firmware version information ni Print network configuration arp Manipulate the system ARP Use arp to obtain a list of options dhcp dbg Configure DHCP client debug dhcp info Print DHCP client information ifconfig Print IP configuration ping IP address Issue ICMP echo request This command supports several command line options For more information on the command s usage type ping rdate Get date from remote host Gets the date from a time server default TCP port 37 and sets the date 1f desired Use rdate route 2 Modify routing tables This command can be used to manually modify the IP routing tables If called without any parameters it prints the current routing table entries For more information on the command s usage type route CAUTION Misconfiguration of the routing table may lead to loss of Ethernet connectivity sw_mp Ethernet switch management platform vlanp Port based VLAN configuration vlanq cfe 802 1Q VLAN protocol configuration vlanx cfg 802 1x V
60. connected to an USB host e g a PC the RS232 receive path will be disabled That means if the serial console is connected to a terminal program the debug outputs of the MCH firmware will still show up but it will not be possible to interact Version 1 25 N A T GmbH 27 User s Manual 3 Mezzanine Module Options The NAT MCH base board is a modular design and can be completed with additional functionalities by adding Mezzanine PCBs for the tongues 2 4 Hub Module PCle SRIO 10GbE LED Module Base Module Figure 10 NAT MCH Mezzanine Options 3 1 1 Base Module The NAT MCH Base Module carries the main CPU the IPMI controllers for up to 12 AMC modules Power and Cooling Units and backplane devices and the front panel connectors The Base module is available in two assembly options Base 6 for systems with up to 6 modules Base 12 for systems with up to 12 AMC modules The Gen2 base module can optionally be equipped with a GbE Level 2 switch for Fabric A for up to 12 AMC modules backplane backup link and front panel uplink e Option GbE The Gen3 base module is always equipped with the GbE Level 2 switch Note The GbE option is mandatory for redundant operation of the Gen2 MCH Version 1 25 N A T GmbH 28 User s Manual 3 1 2 Clock Modules Available Options Telecom Clocking SSC Spread Spectrum clocking for FCLK A only in combina
61. d AMC module can be activated by using a so called configuration item The configuration item consists of an identifier and three parameters The following example shows the configuration item pwr on which can be used to activate the payload power of an unmanaged AMC module amc pwr on 7 20 O0 As shown above the configuration item identifier is always separated from the parameter list by an equals sign The parameters are always separated by a comma sign Comments within the configuration begin with a hash character like Item amc pwr gt gt initialize AMC pwr on Syntax amc pwr on fru id current max t delay Params fru id amc fru id 5 30 current max maximum current consumption 100mA steps t delay pwr on delay time steps max 65535 Example Enable 1 fru id 5 with a maximum power consumption of 2 0A without a delay time amc 5 20 0 Enable 4 fru id 8 with maximum power consumption of 3 5A and a delay time of 35 seconds amc pwr on 8 35 35 The current AMC module configuration can be downloaded from the NAT MCH e g by using the Backup Settings function in the web based configuration interface refer to Version 1 25 N A T GmbH 95 User s Manual chapter 9 The generated text file can be changed with a standard text editor a
62. d communicates via IPMI messages via I2C with hardware modules e g Cooling Units Power Modules AMC cards that run hardware module specific software The next figure gives an example for a software implementation that uses a graphical JAVA application running on top of the OpenHPI see 7 3 3 middleware on the host system JAVA GUI Application SMS PC RMCP over ETH BMC NAT MCH IPMB I2C HW specific SW Figure 12 Software structure details The system management software is described more detailed in the following chapters The software on the MCH can be divided up into two parts first a layer structured stack that implements the interfaces to external systems host and hardware and the message handling and second helper modules to save data IPMI data base FRU inventory SDR repository SEL and communication information session management Shelf Manager capability is optional and can be configured see chapter 5 4 2 to allow usage of an external shelf Version 1 25 N A T GmbH 49 User s Manual manager e g on a host system or an AMC module Note that the SEL on the MCH is kept in memory but not stored into persistent memory 7 2 Communication between host system and MCH Communication between a host system and the MCH is done via IPMI messages To allow a flexible and common available transport of these IPMI messages the MCH offers an Ethernet based management interface 7 2 1 Re
63. date the MCH firmware in FLASH history Recall console history buffer supported on MCH Gen3 only update fpga Update the FPGA firmware mch MCH configuration parameters Shows the MCH configuration parameters as they are stored in the onboard I2C PROM Parameters can be modified by mmch mchcfg Modify MCH configuration parameters see chapter 5 4 cmu_dbg Configure CM upper part debug csif dbg Configure CM ShM interface debug imsg dbg Configure IPMI message debug Ishm Configure local ShM debug red dbg Configure redundancy module debug rmcp dbg Configure RMCP interface debug sdrrep dbg Configure SDR repository debug sel dbg Configure System Event Log debug idb info Print IPMI data base information imsg info IPMI message information Prints the implementation status of the supported IPMI messages on the Host RMCP interface and can be used to print a list of IPMI messages that are supported by the MCH Ishm info Print local ShM information sdrrep info SDR repository information Prints an overview of the Sensor Data Repository of the MCH sel info System Event Log information Prints the Sensor Event Log status and stored events in raw and decoded format session info Session information Print status of currently active sessions show ekey Show all activated connections show fru Show all FRUs show fruinfo fru id FRU contents Shows the contents of a FRU devi
64. ded 4 43 EXT differential 1 44 EXT differential 2 45 PLL IC 46 PLL IC2 47 PLL IC3 48 PLL ICA 49 PLL SYNC1 50 PLL SYNC2 51 PLL SYNC3 52 SYNC CLK input source clock identifier 0 disabled 1 1 2 CLK1 2 CLK 3 4 CLK1 AMC 4 5 CLK1 AMC 5 6 CLK1 AMC 6 7 AMC 7 8 CLK1 8 9 CLK1 AMC 9 10 CLK1 AMC 10 11 CLK1 AMC 11 12 CLK1 AMC 12 13 CLK2 AMC 1 14 CLK2 2 15 CLK2 AMC 3 16 CLK2 AMC 4 17 CLK2 AMC 5 18 CLK2 6 19 CLK2 AMC 7 20 CLK2 8 21 CLK2 AMC 9 22 CLK2 AMC 10 23 CLK2 AMC 1 24 CLK2 AMC 12 25 CLK3 AMC 1 26 CLK3 AMC 2 27 CLK3 3 28 CLK3 AMC 4 29 CLK3 AMC 5 30 CLK3 AMC 6 31 CLK3 AMC 7 32 CLK3 8 33 CLK3 AMC 9 34 CLK3 10 Version 1 25 N A T GmbH 72 User s Manual 35 CLK3 11 36 CLK3 AMC 12 37 CLK1 Update 38 CLK3 Update 39 EXT single ended 1 40 EXT single ended 2 41 EXT single ended 3 42 EXT single ended 4 43 EXT differential 1 44 EXT differential 2 45 PLL default 30 72MHz 46 PLL OC2 default 10MHz 47 PLL OC3 default 2 048MHz 48 OCA default 19 44MHz 49 PLL OC5
65. ded via the CLI diagnose menu of the chassis individual information e g like the serial number have usually to be picked up before downloading the new combined backplane image file Typically this information has to be entered manually afterwards again e g with NATView The EEPROM of the backplane can be updated with a combined backplane FRU image file as follow 1 Set up a CLI connection to the e g via the RS 232 interface baud rate 19200 data 8bit parity none stop Ibit USB or telnet Type diag when the connection to the MCH is set up Choose submenu 2 submenu UPDATE menu in diag menu Select 3 update backplane EEPROM Enter the complete path and file name of the EEPROM file located a server e g 192 168 137 70 home download released combined_backplane_fru bin 6 Confirm Enter EEPROM I2C address RET 164 0xa4 just with ENTER I2C address is 4 po ees The following response is shown in the CLI when the file is written successfully to the EEPROM writing EEPROM contents please wait EEPROM contents successfully updated UPDATE RET 0 0x0 In case of a redundant system the backplane has a separate EEPROM for each MCH Therefore the update procedure has to be done on both MCHs separately Version 1 25 N A T GmbH 131 User s Manual Documents History Vers
66. default 77 76MHz 50 PLL OC6 default 155 52MHz 51 PLL OC7 default 156 25MHz 52 PLL FSYNC 8kHz 53 PLL MFSYNC 2kHz 54 12 8MHz OSC 55 25MHz OSC only with HCSL option 56 LEVEL 0 57 LEVEL 1 58 SYNC NOTE Refer to the Clock Module Technical Reference Manual for a description of the different clock types Example gen3 out 1 50 Output the PLL clock output OC6 on CLK 1 of AMC slot 1 Version 1 25 O N A T GmbH 73 User s Manual 10 3Clock Module Version 4 x 10 3 1 PLL Reference Input Configuration Description This configuration item is used to configure the reference inputs of the PLL on the clock module The clock source that is provided to one of these PLL reference inputs has to be configured via the configuration item gen4 out described below Syntax Clk 4 pll ref ref freq Parameter Description ref selects the reference input of the PLL 1 PLL input 2 PLL input IC2 3 PLL input IC3 4 PLL input IC4 5 PLL input IC5 6 PLL input 1 6 7 PLL input 8 PLL input 8 freq input clock frequency in Hz The following input frequencies are support by the PLL on the clock module 2 kHz 4 kHz 8 kHz 1 544 MHz 2 048 MHz 5 MHz 6 312 MHz 6 480 MHz 10 MHz 19 44 MHz 25 MHz 25 92 MHz 31 25 MHz 38 88 MHz 51 84 MHz 62 5 MHz 77 76 MHz 125 MHz 155 52 MHz 156 25 MHz Furthermore
67. designated management interface too 2 4 3 Management Interface Port The GbE uplink port is used as the management interface port for external hosts to communicate with the onboard Shelf Carrier manager by RMCP The management port can be used by any external Shelf or System Manager to control the operation of the NAT MCH and the system The onboard CPU supports TCP IP and RMCP accesses Version 1 25 GmbH 23 User s Manual 2 4 4 Console Port USB Telnet The console port provides an interface to the Command Line Interface CLI of the onboard CPU The console interface can be used to set the operational and configuration parameters of the NAT MCH Once the IP configuration has been set the console interface can be switched to a Telnet session by connecting via Telnet For a detailed description please refer to chapter 2 3 above Version 1 25 N A T GmbH 24 User s Manual 7 2 5 M4 Connectors Indicators at the Face Plate 4 12 CU 6 AMC i PCle Link p Figure 9 NAT MCH Gen 4 Front Panel 2 5 1 LED Indicators The NAT MCH is equipped with four sets of indicator LEDs e 3 indicator LEDs according to 0 specification indicator LED reflecting the RTM module s status e 16 indicator LEDs displaying the status of AMC modules Cooling Units and Power Modules e 12 indicator LEDs for PCIe Link Status The
68. dule for IPMI Connector to on backplane Hub Module 1000Base X for IPMI MEG EDDA 5 2 5 SATA SSD HDD 2 PCle 4 Ref 5 243 Half Half aie 8 cost po P 1 8 1 8 SATA 22 8 SSD 8 5 1 Figure 4 4 Block Diagram with SSD Option shown Version 1 25 N A T GmbH 16 User s Manual 2 1 NAT MCH Variants The following table lists the available variants and its features for the NAT MCH NAT MCH d Gen2 Gen3 M4 low cost Base Functionality e e e e Management of up to 12 AMCs 2 4 4 4 CUs 4 PMs Onboard Shelf Manager Console Port USB RS232 USB USB RMCP GbE based management interface ie 5 5 GbE Uplink Ports 1 1 Port Trunking with second GbE 4 8 Uplink Port Power fail safe SDR Repository e e Bicolour LEDs for status indication 8 5 4 of AMCs PMs CUs Clock and Fat Pipe mezzanines Real Time Clock JTAG JSM support Faceplate pub Full Full Mid Full Compact Clock Module Mezzanine AMC clocks supported 1 223 F 1 2233 F 1 233 F Stratum 3 PLL Accuracy Stratum 3 Stratum 3E Option
69. e 16 bi color LEDs at the upper border of the MCH give an immediate visual feedback of the status of the corresponding module Their functions are green AMC CU or PM module fully inserted and operational green blinking activation de activation under progress red module faulty or did not progress into operational state communication fault red blinking PM error e g power supply faulty Version 1 25 N A T GmbH 18 User s Manual 2 2 2 SMA Connector External Clock Reference This input can be used to provide the NAT MCH with a system wide clock which can be distributed by the Telecom Clocking Module to any AMC slot in the system For technical data of this input please refer to the NAT MCH Features section 2 2 3 100 BaseT interface primary Management Interface Port The 100BaseT interface provides an autonegotiation 10 100 MBit link to the onboard CPU This is the default management port and can be used by any external Shelf or System Manager to control the operation of the NAT MCH and the system The onboard CPU supports TCP IP and RMCP accesses Note The management port can be switched to the GbE uplink port by a configuration option see chapter Configuration below 2 2 4 GigaBit Ethernet GbE Uplink Port The GbE Uplink Port provides a direct access to the onboard GbE Switch for Fabric A Thus the Fabric connections can be extended to other shelfs or systems if required Optionally the GbE Up
70. e Carrier Manager the host application needs to send singly encapsulated IPMI messages to the Shelf Manager To investigate the Carrier Manager Repository it needs to set up a proper SDR message from the Shelf to the Carrier Manager e g Get SDR This request is then encapsulated into a Send Message Request message that is being addressed from the host application to the Shelf Manager If the host application needs to investigate AMC module directly it needs to perform another layer of encapsulation Also have a look at section 12 3 about double bridged messages 15 2 Separate Shelf and Carrier Manager From MCH firmware release 2 5 on a separate external Shelf Manager ShM is supported Therefore a logical separation of the Shelf Manager and the Carrier Manager CM Version 1 25 N A T GmbH 125 User s Manual functionality was necessary Both entities have their own device repositories which are maintained independently from each other The Carrier Manager creates and updates its repository by scanning its hardware devices The Shelf Manager periodically scans the Carrier Manager s repositories to update all changes As the CM repository adds a devices as soon as it is visible to the management interface hotswap state M1 and above the ShM repository will add new devices only if they reside in hotswap state M2 or above A consequence of this is that a newly inserted AMC module with a pulled out hotswap handle
71. e GbE switch it can either be connected to an external shelf or system manager by the GbE uplink port or to any shelf or system controller running on one of the AMC modules connected to the MCH by a GbE backplane link Remark Although it is possible to configure both the management and the Gigabit Ethernet ports to obtain its IP configuration from DHCP this approach is not useful both Ethernet interfaces will be located in the same subnet and it is inexplicit to which physical interface packets are routed Such a configuration will lead to discarding of packets Also note that DHCP is only allowed for the Ethernet port that is used for the management interface refer to the option Management interface at GbE port This means if the option Management interface at GbE port is set to disabled DHCP can only be used for the IP address source Mgmt If the option is enabled DHCP can only be used for the IP address source GbE Remark Not available on NAT MCH Gen3 Version 1 25 N A T GmbH 38 User s Manual 5 4 2 Shelf Manager Configuration 4 The following configuration parameters control the operation of the MCH s onboard local shelf manager Configuration Option default Description Allow Shelf FRU invalid Yes Allows startup and operation of the MCH even though no valid Shelf FRU record is found on the backplane device Temperature management Enabled Allows the local
72. e I2C bus NAT has added a third option which can overrule the previous two by setting the carrier number in the MCH configuration record Remark If the handle of an AMC module is opened the NAT MCH transitions the module into state M6 and waits for the Quiesced event message As modern CPUs and operating systems may take up to minutes to shut down completely a configurable timeout has been added which defines how long the MCH will wait for the Quiesced event message before it turns Remark If a management problem in an uTCA system occurs it might be extremely helpful to switch on certain debug options to identify the cause of the problem Therefore the NAT MCH allows setting debug levels for specific areas like E Keying Cooling Unit management etc Note In a complex system it is easy to overload the system by debugs especially if several debug options are turned on at the same time As the display is flooded with message in this case it might become difficult to turn off the debugs again as access to the CLI is limited In such a case the system should be power cycled and the bootstrap process should be aborted by typing 2 immediately after the system has come up This allows modifying the configuration parameters without disturbance from any debug output 5 4 4 SEL Configuration Flags 6 The SEL flags configure the behavior of the Sensor Event Log SEL Configuration Option Description Keep on Re
73. e ended 3 42 EXT single ended 4 43 EXT differential 1 44 EXT differential 2 45 PLL OC 46 PLL OC2 47 PLL OC3 48 PLL OCA 49 PLL OC5 50 PLL 6 51 PLL OC7 52 PLL FSYNC 8kHz 53 PLL MFSYNC 2kHz Version 1 25 N A T GmbH 78 User s Manual 54 55 56 12 8MHz OSC 25M 57 58 NOTE Refer to different clock types Example Hz OSC only with HCSL option EVEL 0 LEVEL 1 SYNC CLK the Clock Module Technical Reference Manual for a description of the 4 out 1 50 Output the PLL clock output OC6 on CLK 1 of AMC slot 1 Version 1 25 N A T GmbH 79 User s Manual 10 4 Clock Module Version Physics 10 4 1 Clock Output Configuration Description Determines which clock source is driven on a specific clock output of the clock module Syntax phys out dst src Parameter Description dst destination clock identifier 1 CLK1 AMC 1 2 CLK1 AMC 2 3 CLK1 AMC 3 4 CLK1 AMC 4 5 CLK1 AMC 5 6 CLK1 AMC 6 7 CLK1 AMC 7 8 1 1 8 9 CLK1 AMC 9 10 CLK1 AMC 10 11 CLK1 AMC 11 12 CLK1 AMC 12 13 X 14 X 15 X 16 X 17 CLK2 AMC 1 18 CLK2 AMC 2 19 CLK2 3 20 CLK2 AMC 4 21 CLK2 AMC 5 22 CLK2 AMC 6 23 CLK2 AMC 7 24 CLK2 AMC 8 25 CLK2 AMC 9 26 CLK2 AMC 10 27 CL
74. e no hardware issues known at the time this manual was released V2 0 PCB release there are no hardware issues known at the time this manual was released Boards with release codes earlier than 070717 need to be upgraded to support MCH hot swap capability and MCH redundancy support V1 1 PCB release Boards with release codes earlier than 070717 need to be upgraded to support hot swap capability and redundancy support V1 0 PCB release 2 Hot swap support not possible Version 1 25 O N A T GmbH 97 User s Manual 15 Give us a Hint This MCH is a quite new product based on a new technology and it has to work together with products from other vendors which might be new as well If you discover a problem you can help us improving the product and fixing the problem quickly The MCH configuration menu allows you to set debug options for certain functionalities see chapter 5 and chapter 8 In case you discover a problem please switch on the respective debug option and provide us with a log file taken from the output of the console port together with a brief description of the problem we will care Thank you in advance N A T GmbH Version 1 25 N A T GmbH 98 User s Manual Appendix A Connector Pinouts Appendix A 1 Console Connector RS232 The following table shows the pin assignment of the signals of the RS232 interface which is wired to the Mini USB connector S1 T
75. ease refer to the SRIO switch User Manual for a description of the switch registers Syntax srio orregil switch dev offs value Parameter Description switch dev switch device number 0 1 offs register offset value value to OR with current register value Example srio orregb 0x10 0x01 Set bit 0 0x01 in SRIO module switch register at offset 0x10 Version 1 25 N A T GmbH 93 User s Manual 2 11 7 AND Value to SHIO Module Switch Register Indirect Long Description ANDs a value to a SRIO module switch register This can be used to clear specific bits of a register Please refer to the Switch to Fabric Port Mapping tables in the SRIO Technical Reference Manuals to select correct switch dev parameter Please refer to the SRIO switch User Manual for a description of the switch registers Syntax srio andregil switch dev offs value Parameter Description switch dev switch device number 0 1 offs register offset value value to AND with current register value Example srio andregil 0x10 0 7 Clear bit 7 0x80 in SRIO module switch register at offset Ox 10 Version 1 25 N A T GmbH 94 User s Manual 12 Unmanaged AMC Module Configuration Unmanaged AMC modules can be configured via a text based script file similar to the configuration file used for the Ethernet switch or clock module on the NAT MCH The payload power of an unmanage
76. frequencies which are a multiple of 2 are supported up to 125MHz or frequencies which are a multiple of 8 kHz As the input clock has to be divided down to 2kHz or 8kHz in this case and there is only one divider available in the PLL chip only one clock which is different from the standard clocks above can be configured except the divider is the same for 2 or more input clocks Example Clk gen4 pll ref 1 10000000 A 10MHz clock is provided to PLL reference input Version 1 25 N A T GmbH 74 User s Manual 10 3 2 PLL Mode Configuration Description This configuration item is used to configure the operation mode of the PLL on the clock module Syntax 4 11 mode 11 mode ref Parameter Description number of PLL to configure 1 PLL 1 2 PLL 2 mode PLL operation mode 1 automatic selection between IC8 2 forced mode reference according to parameter ref 3 free running mode ref PLL reference input only for forced mode 1 PLL input 2 PLL input IC2 3 PLL input IC3 4 PLL input IC4 5 PLL input 5 6 PLL input IC6 7 PLL input 8 PLL input 8 Example Clk 4 pll mode 1 1 0 Configure the PLL 1 to automatically select the reference input between 8 Version 1 25 N A T GmbH 75 User s Manual 10 3 3 PLL OC C
77. gateway must be reachable from both the MCH and the control host on different subnetworks that are differentiated by the network masks Appendix B2 MCH IP address configuration To use the MCH in a subnetwork only one IP address needs to be configured However the second IP address might be used for a different purpose The MCH IP address can be configured using the CLI command ip and the configuration can be printed using the command bi nat bi Board Configuration Information Version 1 25 N A T GmbH 113 User s Manual Board Identifier 0b0321 0285 Serial Number 1285 Manufacturer ID 01 Board Code 0503 Layout Version 2 1 Revison Code gt 071025 CPU Coldfire 162 MHz DRAM 32 MB SRAM 32 KB EEPROM 32 MB IEEE Address 00 40 42 05 01 1 TCP IP Configuration IP Address 192 168 0 134 IP Net Mask 255 255 255 0 IP Broadcast 192 168 0 255 IP Gateway 192 168 0 70 2 IP Address 0 05 00 gt Net Mask 0 0 0 0 IP Broadcast 0 0 0 0 2 IP Gateway 0 0 0 0 0100008 Boot String Oxd Issuing the route CLI command the gives nat route IP routing tables Destination Gateway Flags Ref Use 0 0 0 0 192 168 0 70 UG 1 3 192 168 0 0 U 0 0 2 routings Appendix Linux Gateway configuration Interface nato nato The linux gateway was configured to forward IP packets and to use NAT Network Address Translation also kn
78. ge three of the four possible LUN numbers are used for sensor addressing LUN 2 is reserved for message bridging e The Shelf Manager performs another layer of sensor ID mapping the same way as the Carrier Manager does As a host application will usually read out the SEL from the ShM all its events will use data from the ShM repository Although it is still possible to read out sensor data directly it is not recommended as it is impossible for a host application to retrieve the board local sensor ID from the ShM sensor ID According to the specifications there is no way for a host application to map a ShM repository sensor to the corresponding sensor of the CM or AMC board repository All sensors related requests need to be directed to the ShM that will retrieve the appropriate data Version 1 25 N A T GmbH 127 User s Manual 15 6 Hotswap Detection via SEL and Carrier Manager Repository As mentioned above the ShM and CM repository handle detect resources in a different manner The CM adds all detected resources to its repository once it is detected which means that all devices show up in the CM repository if they are in hotswap state or higher The ShM repository only contains devices that are in hotswap state M2 or higher This implies tiny differences on when a device will show up in which repository During a typical hotswap cycle the following information is contained in the ShM and the CM repositories Vi
79. gress red module faulty or did not progress into operational state communication fault red blinking PM error e g power supply faulty 2 3 2 SMA Connectors External Clock Reference This input output can be used to feed an external reference clock into the NAT MCH or provide a reference clock for other systems The input clock can be distributed by the Telecom Clocking Module to any AMC slot in the system The output can provide a clock sourced from any of the AMCs or a local clock generated by the onboard PLL For technical data of this input output please refer to the NAT MCH Features section 2 3 3 Dual GbE Uplink Port The NAT MCH has two 10 100 1000 BaseT autonegotiation uplink ports which are connected to the internal GbE switch circuit The ports can be joined together in link aggregation mode to double the uplink throughput performance In alternative they can be used as separate ports of the Ethernet Switch e g with using VLAN functionality to divide up traffic paths If the MCH operates in Managed Ethernet Mode Spanning Tree only one front Ethernet interface is available as the switch connection to the other front interface is used internally 2 3 4 Management Interface Port One of the GbE uplink ports is used as the management interface for external hosts to communicate with the onboard Shelf Carrier manager by RMCP By default the management port is assigned to GbE uplink port 1 The management port can be used
80. he parameters of the console port are 79200 8 N 1 PSCO RTS PSCO RXD 3 orn socs a 5 edb Table 5 Pin Assignment of the Front panel Connector 51 RS232 Appendix A2 Ethernet Connector GbE Uplink The following table shows the pin assignment of the RJ45 connector of the GbE Uplink port This connector carries the 1000BaseT signals of the Ethernet interface of the Gigabit Ethernet Switch MDI0 MDIO MDD MDII MDIB jo xj Table 6 Pin Assignment of the Front panel Connector S2 GbE Uplink Version 1 25 N A T GmbH 99 User s Manual Appendix A3 Ethernet Connector 100 BaseT Management The following table shows the pin assignment of the RJ45 connector that carries the 100 signals of the Ethernet interface of the ColdFire CPU Termination is the 100BaseT termination used for pins 4 5 7 and 8 5 66 7 Tem Tem 8 Table 7 Pin Assignment of the Front panel Connector 100 BaseT Appendix A4 External Clock Reference Connector The following table shows the pin assignment of the signals of the external reference clock interface Gen3 only SMAI is located directly besides the USB console jack SMA2 is placed next to GbE2 interface Pin Signal Signal Pin No EXTREF P EXTREF N Shield Table 8 Pin Assignment of the Clock Connector S3 Version 1 25 N A T GmbH 100 User s Ma
81. ication it will be supported in future MCH firmware versions Version 1 25 N A T GmbH 63 User s Manual 10 Clock Module Configuration The clock module mounted on the NAT MCH can be configured via a text based script file similar to the configuration file used for the Ethernet switch on the NAT MCH The different functions of the clock module can be set by using so called configuration items Each configuration item consists of an identifier and one or more parameters The following example shows the configuration item c1k type which can be used to assign a specific clock type e g 2 048 MHz to one of the output clocks type 1 3 As shown above the configuration item identifier is always separated from the parameter list by an equals sign The parameters are always separated by a comma sign Comments within the configuration begin with a hash character like Item lt lt clk_type gt gt clock type configuration The current clock module configuration can be downloaded from the e g by using the Backup Settings function in the web based configuration interface refer to chapter 9 The generated text file can be changed with a standard text editor and can be used as a starting point for a user defined configuration The changed configuration can be uploaded again e g via the web based configuration interface The clock module configuration can also
82. ied by N A T for Firmware 2 7 and later running a TAR file containing firmware images for several components is supplied must be available on a TFTP server within the customer s network The MCH must be connected to this network via the management Ethernet port and a terminal must be connected to the console port 6 1 Firmware Update from within the running MCH firmware The firmware can be updated while the normal MCH firmware is running Real time behavior of the MCH may be slowed down during Flash update To perform the update continue as described below firmware update 6 2 Firmware Update from Bootloader In case that the MCH firmware image is corrupted e g due to aborted update process the firmware can be updated via bootloader The MCH bootstrap process is executed in two phases after power up an initial bootstrap loader is started this loader in turn starts the final MCH application firmware image To initiate a firmware update the normal bootstrapping process needs to be stopped before the initial bootloader enters the second phase by typing x on the console terminal during the initial bootstrap phase When the bootloader is stopped the command line mode of the bootloader is displayed Note on Gen3 This option does no longer exist for the Gen3 The firmware upgrade can be executed from within the running MCH only 6 3 Firmware Update 6 3 1 Firmware 2 6 or lower The firmware update is started by ty
83. iguration 20 08 2008 Added description for XAUI and SRIO Hub Module ks Changed Mini BNC to SMA connector 1 11 10 09 2008 Added IP address source configuration for GbE interface cg in ch 0 Updated CLI reference in ch 8 Added note about IP address source configuration in Appendix 1 12 09 02 2009 In ch 0 added description of backward compatibility Cg configuration and in ch 5 4 2 removed description of OpenHPI compatibility because it is unused now Version 1 25 N A T GmbH 132 User s Manual 1 13 10 02 2009 chapter 5 added CLK and SRIO module configuration WW in chapter 9 updated web configuration interface description added chapter 10 for clock module configuration 1 14 15 03 2009 Added variants hl Added appendix private IPMI message definitions 1 15 17 04 2009 editorial changes Cg updated OpenHPI and Command line interface chapters Added ipmitool call syntax chapter and appendix about clock and hub module register access via ipmitool 20 4 2009 Added USB console connection description for MCH hl Gen3 1 16 02 07 2009 Added note about GbE option of Gen2 MCH in chapter ww 3 1 1 03 07 2009 Added CLI command sw_mp in ch 8 ww Added note about DHCP restrictions in ch 0 1 17 20 08 2009 Corrected some typos Added clarifications related to hl MCH Gen3 Checked chapter about NATView also added a new SS section about how to download the latest rele
84. ion Date Description Author 1 0 10 6 2006 Initial Version based on V1 6 of the Quick Install Guide hl Extracted Firmware Release Info into separate document 1 1 26 07 2007 Updated chapters about management interface and cg command line interface for MCH firmware V2 0 added glossary list of figures list of tables 1 2 17 08 2007 Added SEL configuration bit description Cg 20 08 2007 Reworked hl 1 3 06 09 2007 Added Gigabit Ethernet Switch configuration bit ww description 1 4 26 09 2007 In GeSwitchFlags added bit description for enabling ww 802 1Q VLAN support 1 5 08 10 2007 Added ping and route CLI command s and IP Cg configuration description 1 6 10 10 2007 Removed Flag FLAG_POWER_UP_DELAY hl Added Flag SUPPORT 16 11 2007 Updated chapter Command line interface cg 1 7 20 11 2007 Added RemoteAccessFlag in MCH configuration hl register overview table 07 12 2007 Added Chapter Local Shelf Manager hl Added chapter Sensor Event Log cg 1 8 11 02 2008 Added pin out for the backplane connector of the 4 Ks MCH tongue 25 02 2008 Added chapter access from a different cg subnetwork Updated chapter Command line interface 26 02 2008 New chapter about JAVA application NatView SS 1 9 8 5 2008 Changed chapter Configuration and adapted to hl new configuration structure 1 10 20 06 2008 Added description of PCle clustering parameters in ww MCH conf
85. ion that is accompanying the software 7 3 2 lpmitool Ipmitool available at http ipmitool sourceforge net is an open source utility that runs on a Linux PC for managing and configuring devices that support the Intelligent Platform Management Interface IPMI is an open standard for monitoring logging recovery inventory and control of hardware that is implemented independent of the main CPU BIOS and OS The service processor or Baseboard Management Controller BMC is the brain behind platform management and its primary purpose is to handle the autonomous sensor monitoring and event logging features The ipmitool program provides a simple command line interface to this BMC It features the ability to read the sensor data repository SDR and print sensor values display the contents of the System Event Log SEL print Field Replaceable Unit FRU inventory information read and set LAN configuration parameters and perform remote chassis power control Ipmitool was not written to provide large scale management application functionality The functionality is easily accomplished by sending simple IPMI request messages and parsing the returned response It is intended to be used by system administrators who like the simplicity and scriptability of command line utilities as well as those debugging or developing their own BMC implementations Please note that ipmitool is still under development N A T recommends to use at least versi
86. ity has been tested with the standard drivers included in Windows 2000 XP Vista Windows 7 and Linux When connecting first time to a windows machine a new device will be installed NAT MCH console To successfully complete the installation a device information file must be provided to the system N AT provides the required mch inf text file for download from its web site or ftp server Under Windows the new device can be accessed by standard terminal programs like terraterm COMxx port The new device and the assigned COM port can be looked up in the Windows device manager Note Do not start the terminal program before the USB link has been established because the normal terminal programs needs an active link to connect to To avoid the loss of log messages the MCH keeps a history buffer which can be recalled by the CLI command history Version 1 25 N A T GmbH 26 User s Manual Under Linux the new device can be accessed via device descriptor ttyX YZ by standard terminal programs like minicom The name of the newly generated device descriptor is system dependent and needs to be looked up in the devices directory 2 5 5 2 Console Port of MCH M4 RS 232 via RJ45 Connector The RJ45 Connector connects via a RS 232 level converter to the Coldfire UART to provide a serial console debug interface It can be used as an alternative to the USB interface Please note If the USB interface is
87. l Signal MCH uTCA Carrier Hub MCMC uTCA Carrier Manager Controller MMC on MCH MMC AMC Module Management Controller interfaced to carrier via IPMB L MTCA Micro Telecommunications Computing Architecture uTCA MTCM uTCA Carrier Manager NetFn Network Function functional class of message NMCH N A T MCH OEM Original Equipment Manufacturer OpenHPI specific HPI implementation http www openhpi org PEF Platform Event Filtering PET Platform Event Trap PM Power Module PU POH Power On Hours PU Power Unit PM RCS Remote Console Software SW running on a remote system RDR Resource Data Record logical representation of physical entity RMCP Remote Management Control Protocol UDP based IPMI over LAN RPT Resource Presence Table data base of logical resources SAF Service Availability Forum http www saforum org SDR Sensor Data Record sensor description SEL System Event Log SM Shelf Manager Version 1 25 GmbH 9 User s Manual SMI SMIC SMS SSID SSOID SW TCA UDP uTCA Version 1 25 System Management Interrupt Server Management Interface Chip type of interface to an IPMI BMC System Management Software SW running on BMC System Software Identifier System Sensor Owner Identifier Software Telecom Computing Architecture User Datagram Protocol micro TCA N A T GmbH 10 User s Manual 1 Board Specification MCH Module MicroTCA
88. link Port can take over the management port functionality see above 2 2 5 Console Dbg Port The console port provides an interface to the Command Line Interface CLI of the onboard CPU The console interface can be used to set the operational and configuration parameters of the NAT MCH Once the IP configuration has been done the console interface can be switched to a Telnet session In case a password had been configured for a Telnet session a check of this password is done when starting the Telnet session For details regarding configuration of a Telnet password please refer to chapter 8 Important Note The console port has changed from Gen2 MCH to Gen3 MCH from a RS232 type interface to an USB type interface To prevent any mismatch the connector type has been changed too so that a false connection can be excluded as far as possible 2 2 5 1 Console Port of MCH V1 x and V2 x Gen1 and Gen2 HS232 The MCHs of generation 1 and 2 have a RS232 based console port interface The default parameters of the console port are 19200 8 1 The default baud rate can be changed within configuration settings see Chap 5 Important Note Despite the connector type may indicate that the console port of the NAT MCH is an USB port it is realized as a RS232 interface Never connect this port to the USB port of a computer or to a hub Unpredictable damage might be the result Version 1 25 N A T GmbH 19 User s
89. lock Output Configuration Description Determines which frequency shall be output on a specific PLL clock output OC Syntax 4 oc cfg oc freq Parameter Description output clock of pll 1 PLL 2 PLL OC2 3 PLL OC3 4 PLL 004 y PLL OCI 6 PLL OC6 yo gt PLL 967 freq output clock frequency in Hz Example clk_gen4_oc_cfg 1 8000 Outputs 8 kHz on PLL clock output OC1 10 3 4 Clock Output Configuration Description Determines which clock source is driven on a specific clock output of the clock module Syntax clk_gen4_out dst src Parameter Description dst destination clock identifier 1 CLK1 AMC 1 2 CLK1 AMC 2 3 CLK1 AMC 3 4 AMC 4 5 CLK1 5 6 CLK1 AMC 6 7 CLK1 AMC 7 8 CLK1 AMC 8 9 AMC 9 0 CLK1 AMC 10 1 AMC 11 2 CLK1 AMC 12 Version 1 25 N A T GmbH 76 User s Manual Src y a TS 0 N 159 01 1 2 3 4 differential 1 differential 2 13 CLK2 14 CLK2 AMC 15 CLK2 AMC 16 CLK2 AMC 17 CLK2 AMC 18 CLK2 AMC 19 CLK2 AMC 20 CLK2 AMC 21 CLK2 AMC 22 CL
90. m port The new device and the assigned COM port can be looked up in the Windows device manager Note Do not start the terminal program before the USB link has been established because the normal terminal programs needs an active link to connect to To avoid the loss of log messages the MCH keeps a history buffer which can be recalled by the CLI command history Under Linux the new device can be accessed via device descriptor ttyX YZ by standard terminal programs like minicom The name of the newly generated device descriptor is system dependent and needs to be looked up in the devices directory Version 1 25 N A T GmbH 22 User s Manual 2 4 NAT MCH LC Connectors and Indicators at the Face Plate GbE Uplink1 USB Management Port Console Blue Led Primary Secondary Indicator Led Extraction Handle Fault Led Figure 8 NAT MCH Gen LC Front Panel 2 4 1 LED Indicators The NAT MCH is equipped with 3 indicator LEDs according to AMC 0 specification The three AMC 0 conformant LEDs are assigned to the following functions Blue LED Blue LED function according to 0 e Red LED severe fault Bicolor Led Green Yellow o Green is primary management controller o Yellow LED MCH is redundant standby 2 4 2 GbE Uplink Port The NAT MCH LC has one 10 100 1000 BaseT autonegotiation uplink port The port is connected to the onboard GbE switch This is the
91. m onboard FLASH memory Generate Can be used to generate the current configuration configuration file e g of the Ethernet switch or the clock module as a text file The configuration file will be generated from the current settings of the devices Download configuration file Can be used to download the current configuration file stored in the onboard FLASH memory Upload configuration file Can be used to upload a configuration file If the checkbox Save configuration to FLASH Version 1 25 N A T GmbH 61 User s Manual Function Sub Functions Description memory is checked the new configuration will also be stored in the onboard FLASH memory Verify Can be used to verify a configuration file with configuration file the configuration stored in onboard FLASH memory Board Shows the hardware information and network Informations configuration of the NAT MCH If there is a clock or hub mezzanine installed on the NAT MCH additional information about the hardware version s of these mezzanines is displayed as well Reboot NAT Reboots the NAT MCH MCH Please note that after reboot the connection has to be re established manually via web browser Update MCH Opens a collection of firmware updates shows current and new versions of each component and let user decide which components to update refer to chapter 6 4 N A T Opens a new
92. mination Parameter Description dSt clock identifier 1 1 2 2 3 3 4 4 5 5 6 CLK1 AMC 6 7 1 1 7 8 8 9 CLK1 9 10 CLK1 AMC 10 11 CLK1 AMC 11 12 CLK1 AMC 12 T3 14 X 1 16 X 17 CLK2 1 18 CLK2 AMC 2 19 CLK2 AMC 3 20 CLK2 AMC 4 21 CLK2 AMC 5 22 CLK2 AMC 6 23 CLK2 AMC 7 24 CLK2 AMC 8 25 CLK2 AMC 9 26 CLK2 AMC 10 27 CLK2 AMC 11 28 CLK2 12 2 95 m XC S m X 32x X termination Version 1 25 enable disable 100 R termination in multiplexer N A T GmbH 82 User s Manual Example phys termination 28 1 Enables 100 termination on CLK 2 of AMC slot 12 Version 1 25 N A T GmbH 83 User s Manual 10 5Common Commands for all Clock Modules 10 5 1 Write Clock Module Register Description Write a value to a clock module register NOTE This configuration item only should be used if the required functionality has not yet been implemented by another configuration item Please ask NAT for adding support of the needed functionality as the register layout might differ depending on the clock module version used Syntax wreg offs value Parameter Description offs register offset value
93. mote management control protocol Transport over Ethernet usually requires some high level protocols to be used The MCH implements a Remote Management Control Protocol RMCP defined by the Distributed Management Task Force http www dmtf org interface whereby the IPMI messages are embedded into the RMCP messages that are sent using UDP via an Ethernet channel The RMCP fulfills all requirements that are necessary to transfer IPMI messages over the Ethernet Presence echo mechanism RMPC ping pong messages Message flow control via sequence numbers Support for multi session Transfer of session header Support for authenticated access Per message authentication disable User level authentication disable The MCH supports access to the RMPC interface via its front Ethernet management interface Since MCH firmware version 2 0 the MCH supports access to the RMCP interface via its backplane Ethernet interface Selection between these two interfaces can be configured refer to chapter 5 4 7 2 2 Supported IPMI messages Handling of IPMI messages is done within a separate module of the software The message module s configuration and a list of supported messages can be obtained using the CLI command imsg info refer to chapter 8 This command allows printing the compile time configuration list of supported message modules the run time configuration and a list of request messages that are supported Version 1 25
94. n be used as separate ports of the Ethernet Switch e g with using VLAN functionality to divide up traffic paths If the MCH operates in Managed Ethernet Mode Spanning Tree only one front Ethernet interface is available as the switch connection to the other front interface is used internally 2 5 4 Management Interface Port One of the GbE uplink ports is used as the management interface for external hosts to communicate with the onboard Shelf Carrier manager by RMCP By default the management port is assigned to GbE uplink port 1 The management port can be used by any external Shelf or System Manager to control the operation of the NAT MCH and the system The onboard CPU supports TCP IP and RMCP accesses 2 5 5 Console Port USB Telnet and RS232 via RJ45 The console port provides an interface to the Command Line Interface CLI of the onboard CPU The console interface can be used to set the operational and configuration parameters of the NAT MCH Once the IP configuration has been set the console interface can be switched to a Telnet session by connecting via Telnet In case a password had been configured for a Telnet session a check of this password is done when starting the Telnet session For details regarding configuration of a Telnet password please refer to chapter 8 2 5 5 1 Console Port of MCH M4 USB The NAT MCH M4 uses a USB interface as console port The console port provides a USB CDC type of interface Interoperabil
95. n detail Version 1 25 O N A T GmbH 36 User s Manual 5 4 1 MCH Global Parameters 3 The MCH global parameter section contains basic operational and interface settings Configuration Option default Management interface at GbE port disabled board configuration RMCP access Telnet access WEB access IP address source Mgmt IP address source GbE board configuration RMCP session activity timeout minutes RMCP session activity timeout seconds Default Fan Level Enable backward compatibility Enable alternative cooling scheme No Control rear transition module fans Description Allows re routing of the management interface for RMCP and telnet access to the GbE Uplink port on the face plate Default is using the 100BaseT port Allows to setup RMCP connections via the management port Required for tools like openHPI ipmiTool NatView Allows remote login to the MCH via telenet Allows access to the MCH from any browser Source of the MCHs own IP address on the management port Valid options are board configuration value from local EEPROM see chap 5 3 DHCP ShM Link record IP address is taken from Carrier FRU device Shelf FRU info record link record IP address is taken from Carrier FRU device Carrier FRU info record Source of the MCHs own IP address on the Gigabit Ethernet port Valid options are b
96. nctionality for the various system fabrics as defined in the AMC x standard series i e Gigabit Ethernet GbE PCI Express PCIe Serial Rapid I O SRIO or XAUI 10Gigabit Ethernet The NAT MCH can also provide a centralized clock distribution to all AMCS in the system The following block diagram gives an overview about the main MCH building blocks Backplane FPGA RJ45 100 BaseT Management Tongue 1 RJ45 GigaBit Gigabit Uplink BNC on faceplate Tongue 2 Indicator Leds Face Plate Tongue 3 7 12 4 Block Diagram Alternatives 10GbE SRIO HUB PCB Optional Figure 1 Gen 2 Block Diagram with PCIe Mezzanine Option shown Version 1 25 N A T GmbH 14 User s Manual Indicator Leds Backplane RJ45 1000BaseT Mng Uplink Tongue 1 RJ45 1000BaseT Uplink Gigabit Ethernet Uplink BNC on Tongue 2 faceplate BNC on faceplate 195 1 1 2907 Face Plate Controller Tongue 3 Interconnect AMC 7 12 Extended Fat Pipe Fabric D G NAT MCH SRIO Gen Block Diagram nea _ wueece Optoa Tongue 4 Ethernet port only avail
97. nd can be used as a starting point for a user defined configuration The changed configuration can be uploaded again e g via the web based configuration interface The AMC module configuration can also be stored in the onboard FLASH memory e g by using the Save current configuration to onboard FLASH memory function in the web based configuration interface This can be used to load the configuration for the AMC module during startup of the NAT MCH Once the AMC module configuration is stored in the onboard FLASH memory an AMC module PS1 signal necessary is powered when it is inserted according to the configuration parameters A power up reset in a system with an inserted module will also power up the AMC module according to the configuration parameters 13 N 1 Redundancy and Redundant Load Sharing Since the MCH firmware version 2 15 the MCH supports up to four power modules With three or four power modules it is possible to realize N 1 redundancy or redundant load sharing The MCH get the information how to configure the power modules form the backplane FRU To configure the backplane FRU you can use NATview Version 1 25 N A T GmbH 96 User s Manual 14 Hardware Releases Version Status V1 0 Early access boards for evaluation purpose only V1 1 Pre Series boards V2 x Series boards with Harting Plug Table 4 PCB Releases 14 1 Known Hardware Issues V2 x PCB release there ar
98. ngle Uplink BP Fabric Update 5 4 9 NTP Configuration 10 The MCH offers two client applications to obtain the date and time over the internet 1 Time Protocol TP The Time Protocol is a network protocol defined in RFC 868 to provide a site independent and machine readable date and time A client uses TCP or UPD port 37 to request the time as 32 bit integer representing the number of seconds since 1 January 1900 GMT from the server Since the TP functionality was superseded by NTP SNTP several time server do not longer support this protocol and request packets may be lost without reply The Time Protocol client application on is rdate 2 Network Time Protocol NTP Network Time Protocol NTP is a networking protocol defined in RFC 5905 for clock synchronization between computer systems over packet switched variable latency data networks NTP provides UTC including scheduled leap second adjustments No information about time zones or daylight saving time is transmitted The Simple Network Time Protocol SNTP is a less complex implementation of NTP using the same protocol but without requiring the storage of time states and adjustments over extended periods of time Version 1 25 O N A T GmbH 43 User s Manual The NTP SNTP client application on is ntpdate Since the does not store time states and adjustments the differentiation between NTP and SNTP is obsolete
99. nual Appendix A5 Base6 12 Connector Tongue 1 6 Reserved for TCK 165 164 8 RESVD 168 9 PWR TxFA 1 162 Version 1 25 N A T GmbH 101 User s Manual MCH Signal MCH Signal GND 49 GND GND 122 GND GND 60 TxFA IIi IPMBLSDA4 66 TxFA I2 IPMBLSSDA 6 105 608 RxPA I2 IPMBL SCL 7 15 69 12 8 7 12 72 PWR 5 8 9 3 GND 5 8 75 IPMBLSDA 9 6 IPMBL SCL 10 Version 1 25 N A T GmbH 102 User s Manual MCH Signal MCH Signal 80 XOVER2 IPMBLSCLII 9 8 IPMBLSDA dI 90 82 GND 189 85 GND 86 Table 9 Connector Tongue 1 Version 1 25 N A T GmbH 103 User s Manual Appendix Connector Tongue 2 MCH Signal MCH Signal 6 RSVD 165 GND GND GND GND GND Version 1 25 N A T GmbH 104 User s Manual MCH Signal MCH Signal GND 49 GND GND 122 GND GND 60 6 6 CLK26 05 08 0 69 7 GND 2 CLK28 9 13 GND GND 8 75 CLK29
100. oad from FLASH 5 4 7 PCle Switch Configuration 9 optional By the PCIe configuration parameters the operation of the optional PCIe Hub module is controlled Configuration Option default Description operating mode cluster 1 transparent Operation mode of the PCle Switches PCle cluster 1 default transparent NT Intelligent Adapter mode NT Dual Host mode Upstream transparent slot number cluster 1 Upstream non transparent slot number cluster 1 AMC slot number where the host CPU resides in i e root complex PCIe cluster 1 AMC slot number where the non transparent host CPU resides in for dual host mode PCIe cluster 1 operating mode cluster 2 transparent Operation mode of the PCle Switches PCle cluster 2 default transparent NT Intelligent Adapter mode NT Dual Host mode AMC slot number where the host CPU resides in i e root complex PCIe cluster 2 Upstream non transparent slot AMC slot number where the non transparent host CPU number cluster 2 resides in for dual host mode PCIe cluster 2 upstream slot power up delay 5sec Delay applied to the slot where the upstream CPU resides in The delay is applied before payload power is turned on 100 MHz spread spectrum clock Disabled If enabled the FCLK A clock will be of spread spectrum type with 100 MHz means If disabled the FCLK A is a 100 fixed clock Upstream transparent slot number cluster 2
101. oard configuration saved value from local EEPROM see chap 5 3 DHCP ShM Link record IP address is taken from Carrier FRU device Shelf FRU info record link record IP address is taken from Carrier FRU device Carrier FRU info record Timeout for remote RMCP sessions being inactive minutes Timeout for remote RMCP sessions being inactive seconds Default fan level in percent after power up A value of 0 remains the default value of the cooling unit active Enables backward compatibility to firmware version less or equal V2 4 If this mode is enabled CM and ShM behave as to use common SDR repository and SEL and single bridged IPMI messages are directly delivered to the target devices PMs CUs AMCs Clock HUB modules If this mode is disabled single bridged IPMI messages are delivered to the CM and target devices have to be accessed using double bridged IPMI messages which will be extracted first by the ShM and then by the CM Refer to Appendix E Enable control of RTM fans Version 1 25 N A T GmbH 37 User s Manual Remark An external Shelf or System Manager can be connected to the NAT MCH by an Ethernet connection running the RMCP protocol By default the management port is routed to the front panel 100 BaseT port Alternatively the management port can be switched to one link of the onboard GbE Switch If the management port resides on th
102. om 132 147 160 47 icmp seq 3 tt1l 64 time 0 195 ms 132 147 160 47 ping statistics packets transmitted 3 received 0 packet loss time 2002ms rtt min avg max mdev 0 193 0 206 0 232 0 024 ms Appendix B6 Test3 ping between control host This test shows whether the control host can communicate with the MCH and vice versa using the ping tool Ping the MCH from the control host control S ping 192 168 0 134 PING 192 168 0 134 192 168 0 134 56 84 bytes of data 64 bytes from 192 168 0 134 icmp seq 1 ttl 254 time 4 73 ms 64 bytes from 192 168 0 134 icmp seq 2 ttl 254 time 0 382 ms 64 bytes from 192 168 0 134 icmp seq 3 ttl 254 time 0 342 ms 64 bytes from 192 168 0 134 icmp seq 4 ttl 254 time 0 346 ms 192 168 0 134 ping statistics 4 packets transmitted 4 received 0 packet loss time 3000ms rtt min avg max mdev 0 342 1 450 4 732 1 895 ms and a network sniffer dump gives No Time Source Destination Prot Info 25 5 616704 132 147 160 47 192 168 0 134 ICMP Echo ping request 26 5 616755 192 168 0 70 192 168 0 134 ICMP Echo ping request 27 5 616939 192 168 0 134 192 168 0 70 ICMP Echo ping reply 28 5 616953 192 168 0 134 132 147 160 47 Echo ping reply Note that a ping request issued from the control host is received by the gateway and then delivered to the ping response is first sent to the gateway and then delivered to the control host
103. ompliant AMC is recognized via E Keying the dedicated port of the switch is automatically enabled by the firmware thus the switch is in a forwarding state Version 1 25 N A T GmbH 30 User s Manual 4 Operation The NAT MCH should be operated in MTCA R1 0 compliant uTCA shelf with forced air cooling only This manual describes the operation of the following NAT MCH PCB versions and firmware releases NAT MCH V2 1 Firmware V2 1 and later NAT MCH V3 2 Firmware V2 6 and later NAT MCH V3 3 Firmware V2 6 and later NAT MCH V3 4 Firmware V2 6 and later NAT MCH Base M4 Firmware V2 14 and later 4 1 System Startup and Redundancy Operation After power up the NAT MCH tries to read the backplane FRU information and locates the available power modules The NAT MCH determines the primary and the secondary power module according to the role of the power module In parallel the MCH tries to locate a second MCH in the system and if one exists it starts the negotiation process to assign primary and backup roles to the MCHs If a second MCH is found in a redundant system the primary MCH immediately starts backing up its internal database to the redundant MCH on a frequent basis via the GbE backplane link To run the MCH in redundant configuration the Gigabit Ethernet option GbE is mandatory 4 2 AMC Module Startup Sequencing After the NAT MCH has completed its initial startup sequence it scans the system for available AMC
104. on 1 8 11 since this version fixes some already known bugs e g wrong decoding of SEND MESSAGE responses Version 1 25 N A T GmbH 92 User s Manual 7 3 2 1 ipmitool call syntax The following list gives examples for ipmitool call syntax Common syntax to access the NAT MCH over RMCP ipmitool H ip address P command Common syntax to access a target device behind the NAT MCH ipmitool H ip address P t target 2 gt command Common syntax to access a target device behind the NAT MCH using double bridged OEM messages ipmitool H ip address P t target 2 gt T lt target2_I2c gt raw nfn lt cmd gt data where nfn is the network function code cmd the IPMI message command and data the IPMI message data part given as hexadecimal values 7 3 3 OpenHPI This chapter gives a brief overview on how to install and start the OpenHPI for communication to the NAT MCH Please note that due to the availability of the JAVA base Natview tool N A T support for making OpenHPI interworking with the NAT MCH is obsolete now However N A T continues performing tests using different OpenHPI versions OpenHPI is an open source project created with the intent of providing an implementation of the Service Availability Forum s Hardware Platform Interface HPI http www saforum org HPI provides a universal interface for creating resource system models
105. on the MCH The NTP client of the MCH can be enabled and configured by the following parameters Configuration Option default Description NTP server IP 0 0 0 0 IP address of the NTP server in dot notation NTP check for time delay Time interval to request a time update from the time minutes server Minutes and hours are added to define the poll NTP check for time delay hours 0 NTP local time offset Local time offset to GMT in hours This is an integer number between 12 and 12 which is added to the retrieved time to define the current time on the MCH NOTE Negative numbers of hours are displayed as 256 lt hours gt in the MCH configuration configuration flags NTP client protocol Selects the client for TP or NTP SNTP protocol NTP client disabled Enables disables time request client on 5 4 10 DHCP Configuration 11 The built in DHCP client can be configured by the following parameter Configuration Option default Description Hostname jii Designated Hostname for this MCH Note The DHCP client is activated by setting the configuration option JP address source to in the global parameters Version 1 25 N A T GmbH 44 User s Manual 6 Updating the MCH firmware Firmware updates are supplied to customers as a result of bug fixes or enhancements implemented by N A T To execute a firmware update the binary image suppl
106. on the ShM repository is scanned The application shall scan all records for FruDeviceLocator records Forevery FruDeviceLocator record a new FRU device must be generated find the corresponding sensor entries of this FRU device save the EntityID and EntityInstance of the FruDeviceLocator record Scan the ShM repository for records with this matching pair Periodically read the SEL and process its events These events correspond to ShM repository entries e Periodically scan the CM repository to find all devices in hotswap state Synchronize both repositories entries using the FRU ID of an entry Note This complicated procedure is necessary to make the behavior of the Shelf Manager exactly how it is required by the specifications 15 5 SEL uses Shelf Manager Sensor Numbering There is no sensor numbering that is valid in every area of a MicroTCA system Every administrative layer has its own numbering AMC modules perform a sensor numbering on board level Although it is not requested all board sensor numbers usually start with 1 e Carrier Manager needs to merge all sensors of all carrier resources into one single repository It is therefore necessary to perform a mapping between the board level sensor numbering to a carrier wide sensor numbering The 8 bit limitation of the sensor ID value in the SDRs is not sufficient allowing only 256 sensor for a carrier at max to extend this sensor ID ran
107. own as masquerading The linux system kernel 2 4 and 2 6 can be configured as gateway by the commands root cho 1 gt proc sys net ipv4 ip_forward root iptables A POSTROUTING t nat o ethl j MASQUERADE To disable the gateway configuration enter root iptables D POSTROUTING t nat o ethl j MASQUERADI root cho 1 gt proc sys net ipv4 ip_forward Version 1 25 N A T GmbH 114 User s Manual Appendix B4 Test1 ping between and gateway This test shows whether the MCH can communicate with the gateway and vice versa An echo request is issued to the remote system using the CLI command ping which should send back an echo reply message Only if packets are transmitted and received correctly from both systems the communication between the systems work Ping the gateway from the MCH nat ping 192 168 0 70 192 168 0 134 gt 192 168 0 70 Echo Request 56 bytes seq 0 192 168 0 70 gt 192 168 0 134 Echo Reply 56 bytes seq 0 ttl 64 192 168 0 134 gt 192 168 0 70 Echo Request 56 bytes seq 1 192 168 0 70 gt 192 168 0 134 Echo Reply 56 bytes seq 1 ttl 64 192 168 0 134 gt 192 168 0 70 Echo Request 56 bytes seq 2 192 168 0 70 gt 192 168 0 134 Echo Reply 56 bytes seq 2 ttl 64 192 168 0 70 ping statistics packets tx 3 rX 3 ping the MCH from the gateway root ping 192 168 0 134 PING 192 168
108. ping update firmware at the command line prompt The path information of the new binary image is needed ip address has to be written in the form factor lt ip address path to the new firmware image gt Version 1 25 N A T GmbH 45 User s Manual If the given information is correct the new firmware image will be loaded and stored into the onboard Flash device Warning Do not power cycle the system during this process When programming was successful power cycle the system to boot the new firmware 6 3 2 Firmware 2 7 or later The firmware update is started by typing update mch at the command line prompt The path information of the TAR file is needed lt ip address gt has to be written in the form factor XXX lt ip address path to the tar file gt If the given information is correct each component can be selected for updating separately If the version provided in the TAR file is older than the current version and the update option is chosen the component will be downgraded Warning Do not power cycle the system during this process When programming was successful power cycle the system to boot the new firmware Problem tracking In most cases the download does not work file protections on the TFTP server are not set correctly or firewalls prevent the MCH from downloading the image 6 4 Firmware Update via web interface FW
109. please re check the file type Version 1 25 N A T GmbH 47 User s Manual 7 Management Interface For interfacing to an external shelf or system controller the NAT MCH is equipped with an Ethernet based management port The management port can either reside on the front panel 100 BaseT port or on the GbE Uplink port On the MCH Gen3 one of the GbE Uplink ports is the designated management port by default this is port GbE 1 The assignment can be changed in the MCH configuration record An external shelf or system controller can communicate with the MCH by the exchange of IPMI messages The IPMI messages are encapsulated in RMCP packets for transport via Ethernet By this standard interface the user has access to a broad range of open source and commercial tools to access and manage the MCH The N A T graphical system management JAVA application N A T NatView utilizes this interface as well 7 1 Soflware structure The following figure gives a basic overview about software structure on the MCH i eM RMCP over ETH BMC NAT MCH IPMB 2 HW specific SW Figure 11 Software structure overview Version 1 25 N A T GmbH 48 User s Manual The System management software SMS is running on a host system that communicates via the Remote management control protocol RMCP via Ethernet with the MCH The MCH itself implements base management controller BMC functionality an
110. r the web browser has connected to the onboard webserver the user is asked to enter a username and the associated password The default values for these are Username root Password nat The functions supported by the webserver may vary with new firmware releases Currently the following functions are supported Function Sub Functions Description Setup Functions Show MCH Shows the NAT MCH configuration Configuration corresponding to the command line interface command lt gt Change MCH Provides a graphical configuration menu to Configuration change the basic configuration of the NAT MCH corresponding to the command line interface command lt mchcfg gt Switch The dropdown menu can be used to select the switch device to be configured Port VLAN Provides a graphical configuration menu to change the port based VLAN settings For more details please refer to the Ethernet Switch Configuration Manual 802 10 VLAN Provides a graphical configuration menu to change the 802 1Q VLAN settings For more details please refer to the Ethernet Switch Configuration Manual 802 1X Provides a graphical configuration menu to change the 802 1X protocol settings For more details please refer to the Ethernet Switch Configuration Manual 802 1P Provides a graphical configuration menu to Version 1 25 N A T GmbH 60 User s Manual Function Sub Functions
111. s displays information about found sensors NATView may also been used to communicate to the MCH and to display the found resources To stop the OpenHPI daemon please enter killall openhpid Please note that when stopping the OpenHPI daemon in that way open sessions will not be closed by the daemon However the MCH implements a 60 seconds inactivity timeout condition in its firmware that closes open sessions after this timeout 7 3 3 6 Known issues with OpenHPI N A T tested most of OpenHPI versions since version 2 6 0 up to 2 14 0 but there are still open issues that have to be fixed AMC hotswap does not work Event Only sensors are not supported Resource detection is unreliable and changes when using different OpenHPI versions e g physical devices are not detected Version 1 25 O N A T GmbH 56 User s Manual 8 Command line interface The NAT MCH has a command line interface which allows users to initially set up configuration parameters as well as supplying low level diagnostic information The command line interface is available at the console port The commands supported by the command line interface may vary with the firmware version For an actual list of available commands enter 7 at the command line prompt Please note that some commands are intentionally not listed in the table because they are used for testing purpose only The command line interface supports the following commands
112. shown 14 Figure 2 NAT MCH Gen 3 Block Diagram with PCIe Mezzanine Option shown 15 Figure 3 NAT MCH Gen LC Block Diagram with PCIe Mezzanine Option shown 15 Figure 4 NAT MCH Block Diagram with SSD Option shown 16 Figure 5 NAT MCH Gen 2 Front Panel abs eget es och tet 18 Figure 6 NAT MCH Gen 3 Front Panel with SRIO XAUI uplink optional 20 Figure 7 NAT MCH Gen 3 Front Panel with PCIe option eee 20 Figure 8 Gen LC Front Panel ird 23 Figure 9 NAT MCH Gen 4 Front 25 Figure 10 NAT MCH Mezzanine Options 00 28 Figure 11 Software structure 1 a 48 Figure 12 Software structure details rt ipae 49 Figure 13 NAT MCH PCIe Virtual Switch Configuration via web interface 63 List of Tables Table 1 NAT MCH Technical Data 11 Table 2 MCH Variants Features oe er eaae e danses ea 17 DISEOF CLA Commands S e eiie Cui 59 Table 4 PCB ReledseSu veste t estis ho ie arte tL IUE 97 Table 5 Pin Assignment of the Front panel Connector 51 5232
113. sible in CM Visible ShM Action Transition repository repository Board not yet inserted Board insertion hotswap handle is 0 gt 1 Yes No open pulled out Close hotswap handle push in 1 gt 2 gt 3 gt 4 Yes Yes Open hotswap handle out 4 gt 5 gt 6 gt 1 Yes No Board is removed from chassis 1 gt 0 The host application can use the ShM SEL to monitor all state transitions from M2 and above To monitor the state transitions from MO to MI inclusive it is necessary for the host application to periodically scan the Carrier Manager repository Version 1 25 N A T GmbH 128 FS DFLI DyFL DFL Reboot NAT MCH User s Manual Appendix E Alternative cooling unit scheme The standard cooling unit scheme increases the fan speed of the cooling units to 100 as soon as temperature event is sent by one of the resources in the system Sometime this could be annoying especially when testing a system in the Lab as some systems have quite noisy fans The alternative cooling unit scheme controls the fans as shown in the picture below to find the best fan speed for the current system configuration Fen Speed Status Normal Checking if possible to reduce fan speed In case that the board has always the same power consumption this is not an stable state It will continue oscilating between Normal and Non Critical DyFL DyFL 1 SE
114. t PICMG AMC 2 R1 0 PICMG SFP 1 R1 0 Fabric D G PCI Express Option 12 AMCs x1 x4 each PICMG R1 0 Serial Rapid I O Option 12 AMCs dual face plate uplink PICMG AMC 4 XAUI 10GbE Option 12 AMCs dual face plate uplink PICMG AMC 2 1 1 4 Clock Distribution e Telecom Stratum 3 PLL with reference from either 1 of the 12 AMCs or external clock via front panel Spread Spectrum Clock 100MHz mean or oscillator 100MHz fixed 1 1 5 External Clock Reference Input External Clock input for Telecom clocking module via front panel SMA1 SMA2 Supported input frequencies 2KHz 8kHz 1 544MHz 2 048MHz 8 192MHz 16 384 19 44MHZ Version 1 25 N A T GmbH 12 User s Manual Input Range 300mV 5V 1 1 6 LEDs Standard LEDs according to AMC O specification Bi color indicator LEDs for status indication of each of the AMC modules 2 cooling units and 2 power modules Version 1 25 N A T GmbH 13 User s Manual 2 Overview NAT MCH Product Family The NAT MCH is a MicroTCA Carrier Hub in the form factor of a single width full size Advanced Mezzanine Card AMC It provides the central management and data switching functionality for all MicroTCA systems The NAT MCH comprises of a base module and numerous optional daughter cards which can be mounted on the base module The NAT is MicroTCA 0 R1 0 compliant and delivers switching and hub fu
115. ter The NAT MCH Gen2 provides the console port via a RS232 interface 100 type terminal or a PC running a terminal program e g TeraTerm minicom etc has to be connected to the console port 19200 8 N 1 by the supplied cable Important Note Despite the connector type may indicate that the console port of the NAT MCH Gen2 is an USB port it is realized as a RS232 interface Never connect this port to the USB port of a computer or to a hub Unpredictable damage might be the result The NAT MCH Gen3 or LC provides the console port via a standard USB interface The identifies itself as a CDC type device and is supported by the standard usbser sys driver within Windows2000 XP Vista An appropriate configuration file can be downloaded from N A T s web site or FTP server file mch inf After the connection has been recognized by the USB driver a terminal program e g TeraTerm minicom etc can be used via the assigned virtual COM port Version 1 25 N A T GmbH 34 User s Manual Pressing return displays the prompt at the console The MCH is providing a low level command line interface CLI which allows to set certain operational parameters and to display run time information from the MCH and the system Entering 2 will display a list of available commands For displaying and modifying the MCH configuration the following commands have been implemented mch
116. tialization Description Initializes a SRIO switch port and configures it to a specific port speed Please refer to the Switch to Fabric Port Mapping tables in the SRIO Technical Reference Manuals Syntax srio port init switch dev port speed Parameter Description switch dev switch device number 0 1 port physical switch port number speed port speed 0 1 25 Gbhaud Gbaud 2 3 125 Gbaud Sie 15 uu Gbaud 4 6 25 Gbaud Example srio port init 0 9 2 Initializes port 9 of the first switch device to 3 125 Gbaud Version 1 25 N A T GmbH 88 User s Manual 11 2Write SRIO Module Byte Register Description Write a value to a SRIO module byte register Syntax srio wregb offs value Parameter Description Offs register offset value value to write Example srio_wregb 0x10 Oxff Write value Oxff to register at offset 0x10 Version 1 25 N A T GmbH 89 User s Manual 11 3 OR Value to SRIO Module Byte Register Description ORs a value to a SRIO module byte register This can be used to set specific bits of a register Syntax srio orregb offs value Parameter Description offs register offset value value to OR with current register value Example srio orregb 0x10 0x01 Set bit 0 0x01 in SRIO module byte register at offset 0x10 Version 1 25 N A T GmbH 90 User s Manual
117. tion with option X24 or X48 3 1 2 1 Telecom Clocking Module The clock module provides the telecom clocks CLK1 and CLK2 to the 12 AMC modules as well as the fabric clock FCLK A The telecom clocks can be sourced from either of the AMC modules by means of CLK2 as an input to the MCH or from the front panel external clock reference input 3 1 2 2 Spread Spectrum Clocking Module SSC The Spread Spectrum Clocking module SSC distributes a 100 MHz clock to any of the AMC slots The clock can be configured to be a standard 100 MHz clock or a 100MHz Spread Spectrum Clock see Table 2 The clock is switched onto the individual AMC slots according to the E Keying definitions acquired from the AMC module in the respective slot 3 1 3 HUB Mezzanine Modules Optionally the NAT MCH can be equipped with mezzanine modules which provide the data switching functionalities for the fabrics D G of the uTCA backplane The available and planned options are X24 PCIe Hub module for up to 6 AMC modules X48 PCIe Hub module for up to 12 AMC modules SRIO Serial Rapid I O module for up to 12 AMC modules XAUI 10 GbE module for up to 12 AMC modules For a detailed description of these modules and their technical data please refer to the corresponding hardware manual 3 1 3 1 PCI Express Switching Module X24 and X48 The PCIe module X24 X48 supports switching of 1 4 PCIe lanes for up to 6 12 AMC slots Several configuration options which c
118. typically for chassis and rack based servers but extendable for other problem domains such as clustering virtualization and simulation The current version of OpenHPI can be downloaded from http www openhpi org documentation is available on http openhpi sourceforge net manual book1 html Please note that it is highly recommended to use the newest OpenHPI version that is available 7 3 3 1 Overview OpenHPI OpenHPI is used by N A T as middleware to allow communication between the N A T JAVA GUI HPI application and the NAT MCH Of course it is also possible to run other applications on top of OpenHPI OpenHPI runs on a standard Linux PC N A T tested the current OpenHPI version on Debian 3 1 http www debian org kernel 2 4 27 and on Debian 4 0 kernel 2 6 24 but it is known to run on other Linux distributions and other kernels as well Once a Linux PC has been setup the preferred download directory should be changed and all distributed files should be copied into this directory cd your download directory Version 1 25 N A T GmbH 53 User s Manual cp distributed files Extract the provided OpenHPI tarball tar xzvf openhpi 2 7 3 tar gz and use the standard Linux mechanisms cd openhpi 2 7 3 configure amp amp make amp amp make install to install the software The following sections give some more detailed information on how to configure compile and install OpenHP
119. uch a power module functionalities like power sequencing or hot swap are not available Version 1 25 N A T GmbH 33 User s Manual 5 Configuration The operation of the NAT MCH be adapted to certain environments by configuration options The basic idea behind the configuration options is that a customer usually has to touch as less parameters as possible Therefore in most cases the default configuration will work The MCH configuration parameters are divided into several sections according to functionality e Global parameter contains configuration values for basic operation Shelf Manager parameter configuration options for the local Shelf manager Carrier Manager parameter configuration options for the Carrier manager SEL parameter Sensor Event Log configuration GbE switch parameter configuration options for the onboard Fabric A GbE switch CLK module parameter configuration options for clock module optional PCIe SRIO parameter configuration options for the PCIe SRIO Hub module optional e NTP parameter configuration options for optional Network Time Protocol support DHCP parameter configuration of the built in DHCP client 5 1 Configuration via console port configuration parameters are stored in the NAT MCH configuration record in an onboard I2C EEPROM In order to change the configuration the MCHs console port needs to be connected to a host compu
120. uration Option default Description Carrier number default Allows the setting of a Carrier Number If the value is 0 the Carrier number is loaded from the Carrier FRU device or from backplane I O Expander Quiesced event timeout 10 Timeout in seconds the MCH waits for a quiesce event from a FRU device in response of IPMI FRU CONTROL message Quiesced The value of 255 defines an infinite timeout Version 1 25 N A T GmbH 39 User s Manual Allow carrier FRU invalid Yes Allows the MCH to operate with internal default values if no valid backplane FRU device is found If the flag is not set and the backplane FRU content is found invalid the carrier manager will not enter normal operation mode Overrules the backplane FRU device and uses internal defaults Shutdown all resources of the system if the carrier manager of the MCH goes into M6 state Enables Clock Source if present that matches a Clock Receiver of AMC if present in Clock Configuration Record on AMC Disables Clocks if AMC is removed Allows debugging or analyzing of certain areas of a uTCA system Overrule carrier FRU Shutdown system if MCH goes down Enable Clock E keying Debugging Flags Remark According to the uTCA specification the Carrier Number is defined either in the backplane FRU device or by DIP switches which can be read from an I O expander at the I2C address 0x3e of the backplan
121. will be visible in the CM repository but not in the one of the ShM This has also consequences when the module is being pulled out of the chassis the ShM repository removes the device after the M6 to MI transition although the hardware 15 still in the system The only way to get the whole picture from a host application is to periodically scan the CM repository 15 3 AMC access via double bridged messages According to the IPMI specification every media change of an IPMI message needs to be encapsulated into a SendMessage container When operating in compatible mode this means that a host application had to use single bridged messages to communicate with all devices on the IPMB L AMCs MCHs and IPMB 0 Cus PMs The SendMessage request with its encapsulated message was directed to address 0x20 unpacked by the ShM CM and then sent to the inquired device As the device response arrived the ShM CM encapsulated it into a SendMessage response which is then sent back to the host application With introduction of the native mode and its new logical level the IPMI message handling changed as well The concept is explained below by an example sending GetDeviceld request to AMCI FRU 5 1 The host application constructs an IPMI request message for IPMB L address 0x72 2 This request is encapsulated in the first SendMessage request directed to the first Carrier Manager Its address on the ShM CM interface is calculated with the formula 0
122. x80 Carrier Number 2 See 1 clause 3 4 3 Shelf Carrier Manager Interface for details 3 This request is encapsulated once more in another SendMessage request directed to the Shelf Manager Its address is 0x20 4 This whole message is then send to the Shelf Manager It removes the outer SendMessage request message and processes the result by sending it to the first Carrier Manager 5 The Carrier Manager removes the leftover SendMessage and processes the result by sending the GetDeviceld request to the appropriate AMC module 6 After the Carrier manager has received the GetDeviceld response from the AMC module it encapsulates it into a SendMessage response for the Shelf Manager 7 The Shelf Manager encapsulates the response into another SendMessage response message for the host application 8 The Shelf Manager sends the complete response to the host application Version 1 25 N A T GmbH 126 User s Manual 15 4 Hesource browsing using a combination of Shelf Manager and Carrier Manager Repository The ShM repository shows all devices that have a hotswap state equal or above M2 This means that an AMC module with pulled hotswap handle will not be visible in the ShM repository The only way to find all devices that are currently connected to the system is to scan both the ShM and the CM repositories A common algorithm for a host application would look like this e Onstartup of the host applicati
123. ycle Please refer to Appendix B also for more information about the IP address configuration Version 1 25 N A T GmbH 35 User s Manual 5 4 Displaying and changing MCH operational parameters Entering lt mch gt at the command line will show a list of operational parameters which allow to adapt the to certain environments and configurations With the command lt mchcfg gt it is possible to change these parameters The command mchcfg will lead to a menu driven utility which offers to modify the parameters within the individual sections according to their functionality Currently the configuration menu contains the following entries no action print complete configuration reset to defaults nodify MCH global configuration odify ShM configuration odify CM configuration odify SEL configuration odify GbE switch configuration odify CLK module configuration optional odify PCIe SRIO configuration optional odify NTP configuration nodify DHCP configuration print menu print menu quit and save configuration 18 1 Mlle B 3 3 5 5 5 3 3 4 Entering the respective number will guide to the associated configuration menu 2 reset to defaults will reset all configuration parameters to a well known healthy setup The following sections will describe the individual configuration options i

Download Pdf Manuals

Related Search

Related Contents

Global Upholstery Co. 5440LM-1 User's Manual    Tecumseh AWG5524EXG Performance Data Sheet  Radit Manual  MANUAL DEL USUARIO  1 - Brutal Deluxe Software  Optimized design delivers the best choice for workstation  Intel GA-8I955X PRO User's Manual  Documents du stage    

Copyright © All rights reserved. Failed to retrieve file