Xilinx hijinx, Part 1: The ML403 out-of-box experience

Diving in the deep end off the ML403 board

Lewin Edwards (sysadm@zws.com), Design Engineer, Freelance

Lewin A.R.W. Edwards works for a Fortune 50 company as a wireless security/fire safety device design engineer. Prior to that, he spent five years developing x86, ARM and PA-RISC-based networked multimedia appliances at Digi-Frame Inc. He has extensive experience in encryption and security software and is the author of two books on embedded systems development. He can be reached at sysadm@zws.com.

Summary: Discover reasons you might choose an FPGA-based system over a traditional hard-IP microcontroller, and identify the learning curve traditional programmers face when meeting RAM-based programmable logic for the first time. In this new series, Lewin Edwards unpacks the Xilinx ML403 Embedded Development Kit and sorts out some of its idiosyncrasies.

View more content in this series

Date: 30 Jan 2007
Level: Intermediate
Activity: 2245 views
Comments:

This series of articles is an up-and-running approach to using the Xilinx ML403 board with the Embedded Development Kit (EDK) to develop a high-performance PowerPC®-based application. The target market for this series is developers who are familiar with off-the-shelf PowerPC parts and now need the flexibility of a Field Programmable Gate Array (FPGA) design; the overall series goal is to illustrate the advantages and disadvantages of this approach to system design and to demonstrate the learning curve that the beginning developer needs to climb.

The task at hand is sufficiently difficult that it needs to be broken up into fairly small chunks. To best follow this series, you should be an experienced C programmer and reasonably conversant with the PowerPC architecture. Prior experience with programmable logic design is not required; an explicit goal of this series is to help pull you up the aforementioned learning curve.

FPGAs -- The benefit of soft hardware

Before diving into the details, what benefits are offered by using an FPGA, and where would you choose to use one? In a nutshell, the principal benefit of the FPGA is unparalleled design flexibility as to where you place the border between hardware and software. You might have an existing application that's implemented entirely in software (perhaps as a technology demonstration running on a regular PowerPC single-board computer). You can put a high-performance core such as a PowerPC inside your FPGA and run your original code with little modification. Later, you can migrate various parts of the application into virtual "hardware" inside the FPGA, thereby freeing up CPU resources for other tasks. If you haven't yet been able to tie down exactly what parts of your project need to be done in software and which in hardware, an FPGA is probably the ideal solution. Using complex programmable logic of this sort can eliminate the need for co-processors and exotic special-purpose hardware.

FPGAs are also frequently used as stepping stones toward a fully custom Application Specific Integrated Circuit (ASIC). The 30,000-foot view of this process is that you first get your application working inside the FPGA. You might even sell this design, and wait to see if the market justifies large-scale production. When everything looks good, you throw it over the wall to an ASIC design house and they turn it into custom silicon for you. The details are a bit more complex than that, of course; your chip design will probably need to be massaged to fit the ASIC, and the design of your overall circuit will certainly need to be adapted.

Please note that there are some rather specious PowerPoint presentations floating around that purport to demonstrate that using an FPGA is variously cheaper, lower-power-consumption, or otherwise more desirable than using "real" off-the-shelf parts (one vendor even claims through various sophisms that it's now cheaper to implement an 8-bit microcontroller core inside an FPGA than it is to buy one as a real part; this is more or less on par with claiming that technology has now reached such a state that it is now possible to abolish Tuesdays and skip straight to Wednesday). The reality of the situation is that the programmability of an FPGA makes it inherently larger (in terms of transistor count) than an equivalent hardwired chip. To squeeze functionality into small physical areas, FPGAs tend to use leading-edge tiny geometries, which implies relatively high leakage currents.

This means, for example, that you will usually see a decrease in power consumption when and if you migrate to an ASIC. Your ASIC will be based on older processes (the polite word is "mature"), with a larger feature size. It also won't have all the extra switching logic the FPGA needs to contain, since it's not called upon to implement any design other than the one you've hardcoded into it.

In this series, I work specifically with the Xilinx ML403 EDK, which is a high-end development platform. This kit presently costs US$895. That might sound rather expensive, especially to a free tools maven like myself. However, on closer inspection it's really pretty good value -- the FPGA chip by itself is in the US$200 ballpark in single-piece quantity, and the package also includes "full" versions of Xilinx's design software (more on this later) as well as a USB-based downloader cable. More importantly, the board itself has a very rich set of interfaces and features, which makes it really quite a cost-effective single-board computer in its own right:

two USB peripheral ports and one host port
one serial port
Gigabit Ethernet
AC97 audio codec with analog front-ends and standard 3.5mm stereo jacks for line-in, line-out, headphone out, and microphone in
PS/2 keyboard and mouse interfaces
analog VGA interface
64MB DDR SDRAM and 8MB of linear flash, plus 1MB of ZBT SRAM
4Kbit I2C EEPROM
32Mbit platform flash, SystemACE configuration interface with CompactFlash slot, and a CPLD that can program the FPGA from main linear flash

All this mention of memory brings me to an important point that won't be obvious to someone from a traditional software background, namely the Virtex-4 is a RAM-based FPGA.

The Virtex-4 is a RAM-based FPGA

This means that its internal state must be loaded from some non-volatile storage medium on power-up. RAM-based logic can be quite confusing if you're accustomed to working with flash-based chips, where you burn them once and they remain configured until explicitly erased. There are several strategies for bootstrapping FPGA configuration in your design, and the ML403 lets you choose which one you want to use:

For development purposes, you can load the FPGA directly from a host PC over a JTAG interface.
The Virtex-4 can bootstrap itself directly off a special-purpose "Platform flash" chip. The ML403 has a 32Mb platform flash, which can be programmed either through SystemACE, or directly over JTAG. Although this method has the lowest component count, it's not necessarily the cheapest, because the special flash memories are fairly costly; other storage options might be considerably cheaper even when you consider the external components required to interface them to the FPGA. Some designs might use an external microcontroller to load the bitstream from an external storage device. This approach would typically be used if you're using the FPGA only to implement peripherals, and the implication is usually that you already have another microprocessor in the system. For example, an add-in PCI card might use this approach to initialize an FPGA. You would probably not choose this method for a system where the FPGA itself is going to be the main processor.
Xilinx provides a canned solution called SystemACE, available in two flavors: one of them uses linear flash, and the other uses a CompactFlash card. (The CompactFlash flavor is fitted on the ML403.) This is really just a special case of the previous example. The ML403 also has a CPLD (Complex Programmable Logic Device) onboard, which can configure the FPGA from a bitstream stored in the main NOR flash.

Observe two important points from this: First, your design needs to accommodate the need to configure the FPGA on power-up; second, you need to pay very close attention to the circuits connected to the FPGA's I/O lines; there will be a finite -- possibly quite long -- time during system initialization when all those lines will be floating.

So, just to make sure you fully understand the total system architecture: Your design needs to have some method to configure the FPGA at power-up, and it then needs some nonvolatile memory to store the PowerPC code it will actually run. If you use the CPLD method of configuring the FPGA, you can store both the FPGA bitstream and the required code in a single NOR flash chip; it's probably the simplest system design.

First impressions and initial setup

When I first opened the shipping container, I thought Xilinx had sent me two kits, since the package is comprised of two superficially identical glossy boxes. The first box contains the ML403 board itself, with a 512MB CompactFlash card preinstalled on it, the Embedded Software Development Kit (three CDs), ISE 7.1 (another three CDs), and the Development Kit Reference CD. The second box contains a bundle of cables -- I believe the appropriate collective noun is an "astonishment" of cables -- plus another copy of ISE, a Xilinx edition of ModelSim, and an evaluation version of ChipScope Pro, and some trial software, for a grand total of 17 disks full of goodies.

Seeing all these disks can leave you a bit at sea. I suggest you follow the installation process in the order described below.

Note that all the software supplied in this kit requires registration before you can install it. Also, parts of the Xilinx Web site and the HTML files on the CDs apparently do not work properly using Firefox; beware. Likewise, be aware that the registration URLs for ISE, ChipScope, and the EDK are different; if your installation key is rejected, check that you went to the correct URL for the product you've installed.

For reasons that will become apparent a little later, I recommend you set up a dedicated PC for developing with this software. The more RAM the better -- 2GB is a sweet spot. For performance reasons, antivirus software is best avoided -- which means that for security reasons, you can't allow a network connection to the machine.

Note that I haven't compared performance between the Linux® and Windows® XP versions of the software. It might be less disruptive to use the Linux flavor.

After you've installed everything off the CDs, the getting started guide directs you to install service packs off the first CD. I suggest you simply go to Xilinx's Web site and download all the latest service packs for absolutely everything (this will be about 800MB of material), since the files on the CD are obsolete anyway. So, in summary, here are the steps you'll need to carry out to get everything installed and functional:

Copy "Development Kit Reference CD" to hard drive.
Install ISE using the BaseX license serial number in Box 2. Note that this software is only licensed for one year from your first activation.
Install the EDK.
Optionally install ChipScope (the license is only for a 60-day evaluation, however) and Modelsim XE. Select the free limited edition.
Download all the updates and install them. This can be quite challenging because of odd incompatibilities in Xilinx's Web site. I suggest you use Microsoft® Internet Explorer, with all possible security settings, pop-up blockers, and so forth completely disabled. (This still didn't solve all the problems for me, by the way -- though apparently it does work for most people. I eventually had to use wget to solve it).

You should allow one to one and a half entire working days to unpack, connect, install, and download everything and get all the patches and so forth installed. Since this software contains roughly a zillion tiny little files, its performance is severely impacted by filesystem hook software like antivirus programs. Therefore I set up a separate PC, with no Internet connection, to work with the board. This machine is an Athlon XP-M 2800+ with 2Gbytes of RAM, running Windows XP Professional. For reasons that will become clear over the course of this series, it's probably a good idea to follow my example and set up a dedicated machine for nothing else other than FPGA work; have no other software installed on it at all, if possible.

The ISE user interface allows you to add entities either in a schematic view or as HDL objects (either Verilog or VHDL). I was extremely puzzled that there seemed to be no method in the schematic interface to add anything other than a 16-bit accumulator. This is an illusion; if you look at the screenshot below, there are user interface elements hidden in the top panel. To reveal them, you need to drag down the divider just above the words No Options; this will show you the list of other symbols available for use. The PowerPC 405 core is under the heading GigaBit IO/Processor, and your schematic needs to be E size or bigger in order to place it.

Figure 1. User interface elements

Since we're working with the whole EDK, you needn't worry about the vagaries of ISE just for the moment; simply launch Xilinx Platform Studio. When you start Platform Studio, you are presented with a generic new-project dialog. Select Create a new project using Base System Builder Wizard and click OK. In the next dialog, select or create a directory to store the project files (I used C:\XILPRJ\IBM-1). Click OK and the wizard proper will launch.

Select I would like to create a new design and click Next. You're now asked to select from a list of standard target platforms; this version of the EDK supports eight different target boards from Xilinx. Note that both the ML403 (which I'm using) and the Spartan-3 Starter Board (see Resources) are listed here, but you won't be able to instantiate a PowerPC core on the Spartan chip.

In the next dialog, you're asked to select between Microblaze or PowerPC 405 processor cores. Note that you can actually instantiate more than one processor inside the FPGA (the Virtex-4 supports two PPC405 cores), but the wizard doesn't support such designs. When you select PowerPC and click Next, you're given some options related to the processor; leave everything at defaults, except that you should change the "Processor Clock Frequency" drop-down to 300MHz and click the Cache Enabled checkbox.

In the next dialog are options relating to the onboard serial port and debugging LEDs.

In the next dialog, you see some more peripherals: push-buttons (the north/south/east/west/center constellation near the keyboard connector), I2C EEPROM, SysACE CompactFlash slot, and DDR SDRAM controller.

Next, you have the Ethernet MAC, 1MB SRAM (256K x 32), and 8MB flash (2M x 32) controllers. The linear flash controller is disabled by default.

In the last I/O configuration dialog, you set the BRAM size. The default is 16KB. BRAM is on-chip memory implemented using FPGA cells. This is a very expensive way to add memory to your design. The advantage is that you might not need any external RAM at all if you can get away with a very small amount on chip.

In the next dialog, you can configure which memory spaces will be cacheable for instructions and data.

Next, you can select where the standard debug in/out channels should be routed (your only options will be "RS232_Uart" or "None") and whether you want the Base System Builder wizard to create two sample applications. Leave these two samples checked; they're a useful starting point.

In the next dialog, you can select what memory segments will be used by the memory test program for instructions, data, and heap -- leave these at the default of plb_bram_if_cntlr_1.

Next, you perform the same selections for the peripheral test program. Locate everything in SRAM_256Kx32.

At last, click Generate and then Finish.

Right about this time, if you had Cygwin on your machine when you installed the Xilinx software, you'll encounter a problem that had me baffled for a week. Namely, when you try to make your designs, you'll see an error message "*** target pattern contains no `%'. Stop." This issue is documented on Xilinx's Web site (see Resources), but the answer given there isn't really satisfactory. The underlying problem is an incompatibility between the Xilinx tools and GNU make 3.81 and later. Xilinx's recommended workaround is to downgrade to 3.79.1, which is installed with the Xilinx toolchain. This is definitely not the full story, however: if you download make 3.79.1 from gnu.org and build it, the resulting executable will fail with the exact same error. The version of make.exe found inside the Xilinx directory is built with MinGW, not with Cygwin, and you should be very wary of this if you follow Xilinx's workaround suggestion of overwriting your existing make.exe with the older one from the Xilinx bundle. It's not clear if this will adversely impact other build processes you're trying to run inside Cygwin; my system appears to be fine, but your mileage may vary, particularly if your projects use unusually complex makefiles.

It is partly for this reason that I reiterate my earlier assertion that you're probably well advised to dedicate a PC just to this one task of FPGA development.

At this point, you've assembled your first EDK project, though you haven't actually put any useful code into it yet. The next article gets down to starting up the processor core, writing some actual PowerPC code, and wiggling some pins!

Resources

Learn

The Spartan-3 Starter Board is a sub-US$100 entry path to developing fairly complex FPGA applications. You can't instantiate a PowerPC in this chip, however, and the price doesn't include software. You can, however, use the free version of ISE to build and debug Microblaze-cored applications on this board. Note that there is no free version of EDK.
Learn more about the Xilinx ML403 Evaluation Platform.
ML40x Known Issues lists known errata and issues for the ML403; read this before you start experimenting with the board.
The rather odd make.exe incompatibility in ISE/EDK is documented here.
See the Xilinx ML403 Evaluation Platform Demos and Reference Designs for direct download links for several reference designs that run on the ML403.
wget is a useful GNU package for retrieving files using HTTP, HTTPS, and FTP.
cygwin and minGW both help to make a Windows environment more UNIX®-like. cygwin is governed mainly by the GPL, while minGW is mainly in the public domain -- note that the word 'mainly' is used here to warn you that some components in each of the collections are exceptions to this overgeneralization.
The IBM Semiconductor Solutions Technical Library Semiconductor Solutions Technical Library hosts a wealth of information -- from specifications and user manuals to product briefs and errata and much more.
Keep abreast of all the Power Architecture-related news that's fit to print: subscribe to IBM microNews.