600-2701 User`s Manual
Contents
1. Microsystems SEEREERER EPPES BEEBEEEE EBEEBEEE 1 2 3 4 11 Nf CYCLONE MICROSYSTEMS PCI E 401 REV C P N 270 RO401 03 REV C ENG 0401C COPYRIGHT 2006 Figure 5 2 PCle 401 Physical Configuration 600 2701 PCI Express Expansion System User s Manual Revision 1 0 March 2007 14 CYCLONE Microsystems PHYSICAL CONFIGURATION o CH nok lt Rear View 20 00 Fan Drive Bays Front View Door Open i Top Vi Side Vievv Op View 6 969 Front Vievv Door Closed Figure 5 3 600 2701 Chassis 600 2701 PCI Express Expansion System User s Manual Revision 1 0 March 2007 5 3 14 CYCLONE Microsystems CHAPTER 6 REFERENCE 6 1 REFERENCE MANUALS PEX 8532 Versatile PCI Express Switch Data Book Version 1 1 PLX Technology Inc Sunnyvale CA 800 759 3735 www plxtech com PCI Express Base Specification Revision 1 0a PCI Express Card Electromechanical Specification Revision 1 0a PCI Special Interest Group PCISIG 5440 SW Westgate Dr 217 Portland OR 97221 503 291 2569 503 297 1090 Fax WWW pcisig org 600 2701 PCI Express Expansion System User s Manual 6 1 Revision 1 0 March 20072. Each LED corresponds to one lane of that particular link One bank is labeled UP LN 0 upstream and the other bank is labeled DWN LN 0 downstream The eight upstream LEDs give the state of the link from the PC host that the PCIe 401 is installed into to the PCle 401 If the PCle 401 is installed into a x8 of x16 slot then all eight of the upstream LEDs should be on Since the PCIe 401 upstream port is an x8 link only eight lanes of a x16 slot will be used If less than the eight LEDs are on there is a problem with the link between the PC host and the PCIe401 If the PCIe 401 is installed into a x4 PCIe slot down shifting then only the first four LEDs will be on and the link is trained at x4 If none of the upstream LEDs are on turn off the host PC and make sure that the PCle 401 is properly seated into the PCIe connector The eight downstream LEDs indicate the status of the link connecting the PCIe 401 to the PCle 411 via the PCI Express cable If the PCIe 401 is installed into an x8 or a x16 slot and the eight upstream LEDs are on the PCIe 401 is connected via the PCIe cable to the PCIe 411 all eight of the downstream LEDs should be on If the PCIe 401 is installed into a x4 slot there still should be all eight downstream LEDs on If less than eight LEDs are on check to make sure that the cable is properly connected to each board 600 2701 PCI Express Expansion System User s Manual 4 1 Revision 1 0 March 2007 SYSTEM POWER
3. Express Expansion System User s Manual Revision 1 0 March 2007 CONTENTS L I CYCLONE Microsystems LIST OF FIGURES Figure 1 1 600 2701 Block Diagram nn nn eee ete eee 1 2 Figure 5 1 PCle 411 Physical Contouratton nn nn 5 1 Figure 5 2 PCle 401 Physical Configur ati on nu narrar cnn 5 2 Figure 5 3 600 2701 Chassis cnn nn 5 3 LIST OF TABLES Table E Eet e aaa dianas 1 3 Table 3 1 PCle 411 Power Reouirements nn nano ee terre vene eee 3 1 Table 3 2 PCle 401 Power RequirementS au aaa aaa aaa seen ne sene e eve cnc crac 3 1 Table 3 3 Power Supplied Per PCle Got 3 1 Table 4 1 Klezatt PS SS a e Aaa a aaas 4 2 ii 600 2701 PCI Express Expansion System User s Manual Revision 1 0 March 2007 14 CYCLONE Microsystems 1 1 CHAPTER 1 INTRODUCTION INTRODUCTION The Cyclone Microsystems 2701 PCI Express Expansion System is a PCI Express PCle expansion system that allows the user to add up to eight PCI Express add in cards to a PC host system Most PCs contain few PCI Express slots making them poorly suited for embedded systems requiring a wealth of different I O boards and co processor resources The 2701 PCI Express Expansion System s eight PCI Express slots are organized as two x8 slots and six x4 slots All expansion slots accommodate full length and full height cards and are cooled by two 108 CFM fans A 460 watt supply powers the rack mounted expansion chassis The Expansion System supports 20 Gb s bi directional t
4. UP 14 CYCLONE Microsystems 4 4 LINK INDICATION LED DEFINITION PCIE 411 The PCIe 411 has several sets of surface mount located on the top side of the PCB see Figure 5 2 The LEDs are arranged in groups of either eight of four LEDs As with the above board each LED corresponds to an individual lane within that particular link The are eleven groups of LEDs and their definition is shown in Table 4 1 Additionally there are sixteen individual LEDs These LEDs are labeled PWRA through PWRH and ATTA through ATTH These PWR LEDs are illuminated and simply indicate when an add in card is installed and powered in the corresponding slot The ATT LEDs are indicators for the attention buttons Attention buttons are utilized in when Hot Plugging boards into the PCIe 411 Hot Plug is described in section 4 5 The LEDs labeled UP and S2 the LEDs between the PCIe 401 and the PCle 411 and between the two PCI Express switches should be on at all times The slot LEDs indicate the active lanes to the corresponding add in card If no add in card is installed into a particular slot the LEDs for that slot will be off If a x1 or x4 add in card is installed into a x8 slot only 1 position 0 or 4 LEDs LEDs 0 through 3 respectively will be on Similarly x1 add in cards installed into x4 slots should only have the first LED on Add in cards that are down shifted x8 cards in x4 slots should have all four of the lane indicator LEDs on Table 4 1 PCle 411 L
5. both the links that are trained and the individual lanes A detailed explanation of the LEDs follows later in this manual One essential requirement for system initialization is the ability of the Host system s BIOS to be able to enumerate the many bridges inherent in a complex PCI Express design The links from the PCle 401 to the PCle 411 are created with PCI Express Switches Each link looks like a PCI to PCI bridge to the Host s BIOS The number of bridges can add up quickly Older BIOS may not have the ability to handle the number of bridges Make sure that the BIOS on the host computer has the latest updated BIOS If required contact the host system s manufacturer to make sure that the BIOS used can handle the large number of bridges that it will see in the system 600 2701 PCI Express Expansion System User s Manual 2 1 Revision 1 0 March 2007 14 CYCLONE Microsystems CHAPTER 3 EXPANSION SYSTEM OPERATION 3 1 600 2701 CHASSIS The PCIe 411 is installed into an eight slot rack mountable chassis The chassis provides access to two x8 PCI Express slots and six x4 PCI Express slots All slots can accommodate standard height and full length PCI Express add in cards The chassis also contains three drive bays that may be used by the user to install peripherals for their particular application There are several ATX four pin peripheral power connectors available from t
6. mn mn mn pn SS wll Cable 12x Infiniband PCle 411 Expansion System PCI Express x8 Cable Interface PCI Express Switch PCI Express Switch Figure 1 1 600 2701 Block Diagram 600 2701 PCI Express Expansion System User s Manual Revision 1 0 March 2007 14 INTRODUCTION CYCLONE Microsystems 1 2 SPECIFICATIONS The specifications in table 1 1 detail the 600 2701 expansion system chassis including the PCle 411 expansion backplane and the 460W power supply The chassis also includes fan which provide 180 CFM cooling Table 1 1 Specifications Physical Height 7 inches Width 17 inches Depth 22 inches Other 19 inch rack mountable Electrical Voltage 90 264 VAC full range Frequency 47 63 Hz Input Current 8 0 A for 115 VAC 4 0 A for 230 VAC Inrush Current 65 A max for 115 VAC 125 A max for 230 VAC Environmental Operating Temperature 0 to 40 Degrees Celsius Relative Humidity 0 to 90 non condensing Storage Temperature 20 to 70 Degrees Celsius 1 3 STANDARDS PCI Express Base Specification Revision 1 1 PCI Express Card Electro Mechanical Specification 1 1 600 2701 PCI Express Expansion System User s Manual Revision 1 0 March 2007 14 CYCLONE Microsystems 2 1 CHAPTER 2 THEORY OF OPERATION THEORY OF OPERATION The basic PCI Express Link consists of dual unidirectional differential links implemented as a transmit pair
7. 600 2701 EIGHT SLOT PCI EXPRESS EXPANSION SYSTEM USER S MANUAL The information in this document has been carefully checked and is believed to be entirely reliable However no responsibility is assumed for inaccuracies Furthermore Cyclone Microsystems Inc reserves the right to make changes to any products herein to improve reliability function or design Cyclone Microsystems Inc neither assumes any liability arising out of the application or use of any product or circuit described herein nor does it convey any license under its right or the rights of others Revision 1 0 March 2007 Cyclone P N 800 2701 Copyright 2006 by Cyclone Microsystems Inc CONTENTS L I CYCLONE Microsystems CHAPTER 1 Lt ANTRODUCTIO NN da ta de dedo do nd 1 1 12 SPECIFIGATIONS EE 1 3 ES STANDARDS A Ee E 1 3 CHAPTER 2 21 THEORY OF ORERATION 00 o ida 2 1 CHAPTER 3 324 60022701 GHASSIS SAA A A E A ee eee ao eee 3 1 3 2 POVER CONSIDERATIONS ah ea 3 1 CHAPTER 4 431 SYSTEM le WEE 4 1 42 SEATING OF GARDS dedo 4 1 4 3 LINK INDICATION LED DEFINITION PCle A01 eee eeeees eee sesa vene eseve eee ecet 4 1 4 4 LINK INDICATION LED DEFINITION PCleAn eee e eee des eee see ee eee eneve eee ecet 4 2 45 HOT PLUG PGJEAA das int assesi loa 4 2 CHAPTER 5 541 PHYSIGALGONFIGURATION i ani vidat kua ash dek d r i shue DA EES ENEE dat de hh 5 1 CHAPTER 6 6 11 REFERENCE MANUA S es e A A a rA E A e wast a ea dae eene eens 6 1 i 600 2701 PCI
8. ED Banks LED LABEL LED FUNCTION SLOT A 0 gt 3 Slot A is a four lane port SLOT B 0 53 Slot B is a four lane port SLOT C 0 gt 3 Slot C is a four lane port SLOT D 0 33 Slot D is a four lane port SLOT E 0 gt 3 Slot E is an four lane port SLOT F 0 33 Slot F is an four lane port SLOT G 0 gt 7 Slot Gis an eight lane port SLOT H 0 gt 7 Slot H is an eight lane port UP 0 0 gt 7 x8 link from the PCle 401 to the PCle 411 U14 UP 1 0 gt 7 x8 link between PCle switches U22 to U14 S2 0 gt 7 x8 link between PCle switches U14 to U22 4 5 HOT PLUG PCIE 411 The Hot Plug capabilities of the PCIe 411 allow for insertion and extraction of an add in card from a running system without adversely affecting the system Board insertion or extraction is accomplished for faulty board repair or system reconfiguration without system down time The PLX PEX 8532 switches used on the PCle 411 contain one Hot Plug Controller per port each of which is compliant with the PCI Standard Hot Plug R1 0 Hot Plug support requires significant software development Cyclone does not provide O S support for Hot Plug and users wishing to develop software to support HotPlug should consult chapter 9 of the PEX 8532 documentation A description of the Hot Plug sequence from an operator s point of view including a description of the PWR and ATT LEDs and ATT button can be found in the PCle 411 User s Manual 4 2 600 2701 PC
9. EH SEH ti 7 oO O DES t t tal t 3 Ett e tj it tj t t tal t 3 SO EE tt ita Hita ita et tH De lt ta t ta GES Rita GES ES K tl H tal K taj GES 0 7 kt ta D i tt D i t tH D i Gei tt ht tg Kt 4 Hitaj H tal Het 4 Ht 4 Kita Lt 4 p t EH lt ta t EH lt ta lt ta ht tal lt ta t EH et tal kt tal at tal kt tal kit at tal t tal lt tal Hes Re ty EEEH GE G FEH KG EEEH IA taj gel et tal lt tal tal 4 et ta GES SES GES GES GES GES EE GES PWRA Et oy ites tt GES GES GES GI EI Ett SW1 tt tt ii tt tt tt tt it esi esi Gi Gei etja j 34 KI GES ATTA t t Es t 4 t 4 Gr 1 frat ES Ale a E GES GE GES ees ES G EEEH GES PWRB MES P sti MES MES tst MES MES Pa en Er o e s o e e A B ES ATTB o Hi x m o a w oO T a a CORWRE SL ce latte wo E I t H De E PWRD s S J J 3 J 3 3 P RE ATTD D D D E Oj 7 mi e E C4 e m FRE ATIF elle elle elle elle elle elle elle elle gja EENG Ss Ok SPRS 1 f SW8 ATTH s a a a aa aa aa D SS SB E Hi 023 7 R K K o_o eGo Aena EE aa aa De o Er U22 Ces gt a gt a o sE E RO E E 1 OS 3c UP1 ss fe aj BIE Te 0 Sa 1 a a aa J2 a Es E s 1 1450 SH 03 Des Se PWR LEDE ls sl fe e J10 E F Sk 57 355 O pe OJ goje it 1 22 Da o_o o o_o o J20 o_o 1 Bo o Ho eGo so so o y 600 2701 PCI Express Expansion System User s Manual 4900 5 1 Revision 1 0 March 2007 PHYSICAL CONFIGURATION ld CYCLONE
10. I Express Expansion System User s Manual Revision 1 0 March 2007 14 CYCLONE Microsystems 5 1 CHAPTER 5 PHYSICAL CONFIGURATION PHYSICAL CONFIGURATION Figure 5 1 is a physical diagram not to scale of the PCIe 411 backplane showing the location designators of jumpers connectors and major ICs Refer to this figure when component locations are referenced in the manual text Figure 5 2 is a physical diagram of the PCle 401 adapter Figure 5 3 is a drawing of the 2701 chassis showing the location of the PCle 411 the power supply fans etc Figure 5 1 PCle 411 Physical Configuration it 11200 gt j 650 4 3100 1800 gt 5400 PO 400 v s ce x legt lea Bo DEED DE x x o o so Ja N x 1300 O pi o_o CR3 Be J1 J2 Js J4 J5 J6 J7 J8 O sh W oo 7 0 et tal et tal bet tal et tal et taj et Fal let tal et tal EE EH GC esi EFH Gite ees Gite Gite EFH Gite EZE E E SES eet SES SES GES SES ae 52 EE G U14 E 2 eos GE sees Rete sees ieee ieee BeOS a LpOIE H ROE GG ZE Gef ROE ROE Gef tee JOE EE GE REH GE Gi Ered eel eel GE f 3 Et 3 GE f 3 tj B
11. able is used to connect the x8 link between the PCle 401 and the PClIe 411 The cable should be connected between both boards and the 600 2701 power cord should be connected to the chassis and plugged in to a power socket There is a power button on the front of the 600 2701 Expansion Chassis but it is not connected and serves no practical purpose in this application When the host PC is powered on a signal is sent over the cable to turn on the 600 2701 If the chassis does not power up make sure that all cards are seated properly the chassis is plugged in and the PCI Express cable is connected properly There is only one LED on the front of the expansion chassis that has meaning The green LED indicates that the chassis is powered SEATING OF CARDS Unlike standard PC applications the 2701 Expansion Systems has a narrow lower gate that precisely engages the lower end of the PCI Express Add In board s face panel The purpose is to insure correct electrical connectoring mating of up plugged boards Failure to accurately mate the lower end of the face panel with the chassis lower gate will lead to the board not being recognized by the host LINK INDICATION LED DEFINITION PCIE 401 There are many LED indicators on both of the printed circuit boards PCBs that tell the user the status of all links and individual lanes Also see Figure 5 1 The PCle 401 has two banks of eight LEDs These LEDs are located on the top side or component side of the PCB
12. and a receive pair The signaling rate for PCI Express is 2 5 Gigabits second Lane direction A link supports at least one Lane The PCI Express link from the PCle 401 over the cable to the PCle 411 is an eight lane x8 link The PCIe 411 provides two eight lane x8 slots and six four lane x4 slots Each slot can accommodate either single lane x1 x4 or x8 add in cards In the case where a x8 add in card is installed into a x4 slot only the first four lanes on the add in card will be utilized This situation is termed down shifting Per the PCI Express Specification down shifting is only allowed in this case All slot connectors on the PCle 411 are mechanically x8 with the x4 slots leaving the upper four lanes not connected Up plugging i e plugging a smaller link card into a larger link connector is fully allowed Once the PCle 401 is installed into the host PC the cable connected to the PCle 411 the chassis is plugged into an AC power outlet and any desired add in cards are installed the system is ready to be turned on A number of things happen at this point First the PCI Express links are initialized This is a purely hardware initialization where each PCI Express link is set up following a negotiation of lane widths by the two ends of each link No firmware or operating system software is involved Once the links are initialized or trained there are LED indicators on both of the Cyclone Microsystems cards that indicate
13. he power supply The chassis contains a 460W power supply Tables 3 1 through 3 3 show the power consumption for the Cyclone Microsystems boards and the power supplied to the PCI Express slots Note that the PCIe 401 is installed in and powered by the host supply In a fully populated system with each PCI Express add in card drawing 25W there is an additional 180W 5V and 12V available for use via the peripheral power connectors The 12V available from the peripheral power connector is limited to 10A 3 2 POWER CONSIDERATIONS Table 3 1 PCle 411 Power Requirements Voltage Current Typical Current Maximum 3 3V 0 86 Amps 1 1 Amps 5V 2 71 Amps 3 36 Amps 12V 0 01 Amps 0 02 Amps Table 3 2 PCle 401 Power Requirements Voltage Current Typical Current Maximum 3 3V 0 37 Amps 0 46 Amps 12V 0 30 Amps 0 40 Amps Table 3 3 Power Supplied Per PCle Slot Voltage Current Maximum Voltage Tolerance 3 3V 3 0 Amps 9 12V 2 1 Amps 8 Required by the PCI Express Card Electromechanical Specification Revision 1 1 Note that the total power for each PCIe shall not exceed 25W Airflow in the 600 2701 chassis is provided by two 108 CFM located at the front of the chassis 600 2701 PCI Express Expansion System User s Manual 3 1 Revision 1 0 March 2007 14 CYCLONE Microsystems 4 1 4 2 4 3 CHAPTER 4 SYSTEM POWER UP SYSTEM POWER UP The host PC with the PCle 401 installed controls power up for the entire system The 12x Infiniband c
14. raffic to and from the host system and utilizes non blocking PCI Express switches for excellent peer to peer I O bandwidth For PCs with modern BIOSs the 2701 Expansion System is recognized by the host system upon boot up requires no hardware specific drivers and is entirely host operating system agnostic The 600 2701 system is composed from three elements a PCI Express Host Bus Cable Adapter an Expansion System Cable and an Expansion Chassis Our PCle 401 Host Cable Bus Cable Adapter card is inserted into a host computer PCIe slot PCIe expansion cable links the PCI host with the expansion chassis The expansion chassis is populated with the PCle 411 Switched Backplane with Hot Plug support PCI Express is a high performance general purpose I O inter connect defined for a wide variety of computing and communication platforms Key PCI attributes such as its usage model load store archi tecture and software interfaces are maintained whereas its parallel bus implementation is replaced by a serial interface PCI Express take advantage of recent advances in point to point inter connects Switch based technology and packetized protocol to deliver new levels of performance 600 2701 PCI Express Expansion System User s Manual 1 1 Revision 1 0 March 2007 INTRODUCTION 1 2 14 CYCLONE Microsystems e e E e A I I PCle 401 x8 l I Bridge Host PCI Express l System I I PCI Express x8 I mn
Download Pdf Manuals
Related Search