Level: Introductory Thomas Burger (twburger@bigfoot.com), Owner, Thomas Wolfgang Burger Consulting
01 Aug 2000 This first of two articles describes the birthing pains of Linux 2.4, along with the anticipation and frustration of waiting for the new functionality. TW Burger discusses what Linux users will get with the new kernel release 2.4 and offers an educated guess as to the upcoming release date. Part 1 describes how we might estimate the release date, what the general functionality of the new release will be, and what to expect in terms of new extensions to the main hardware capability. Part 2 details the nitty-gritty expectations surrounding specialized hardware. Linux 2.4: "Better late than never"
First it was promised in the fall of 1999. Then it was spring 2000. On May 25, 2000, Linux Kernel 2.4.0.0-test1 was released as a prototype and on June 23, 2000, 2.4.0.0-test2 was posted to the Linux Kernel Archives.
The frustration of waiting for the much-anticipated Linux kernel 2.4 release has spawned the latest communal wave of anxiety in the open source world. Linux enthusiasts want to replace the Windows operating systems so badly that Linux version releasing is starting to emulate other operating systems' vendor patterns. Typically these include long waits, missed deadlines, and major changes that may not be backward compatible. Happily, there are efforts afoot to correct this trend in Linux development. With the release of Linux 2.4, more control will be placed on changes, and upgrades will be available in smaller time intervals.  | |
The elastic schedule
Linus Torvalds posted this message in the 2.4 Kernel directory on May 24, 2000:
No. It doesn't really exist yet.
But I'll be gone for three weeks, and in the meantime there's a "2.4.0-test1" kernel here. It's not a real 2.4.0 release, but we should be getting closer. There's going to be other test-kernels after this one, and we'll find bugs. And bad behaviour. And wonderful features which we'll document some day.
Alan Cox is most likely to maintain a "2.4.0-ac" series while I'm gone. Not the same thing as 2.4.0 either, but by testing these test-kernels out and giving feedback, we'll get to that some day.
Have fun. And let's see how many people find this without it even being announced ;)
Linus
|
|
To get right to the point, Linus Torvalds announced in a keynote speech (delivered Tuesday, June 13, 2000, at a San Francisco banquet benefitting the
Linux Debian Project) that the next Linux release (2.4) will be out in the fall of 2000. Linux version 2.2 came about thirty months after the last major release, Linux 2.0. The Linux 2.0 release was posted eighteen months after version 1.2 was released. Torvalds said that long delays between releases would be eliminated through a policy of limiting the changes made in each release. To meet this release target date, planned changes and additions which cannot be completed by the required date will be placed in the following release, 2.5. The idea is to eliminate the large and often painful upgrade problems that have happened in the past. Torvalds started adopting the small modification release philosophy late last year.
Why are we waiting?
There is no actual delay of the 2.4 release. There was never a scheduled date for the release. It was always going to be ready when it was ready. The two main problems have been: one, a matter of perception caused by the Linux community, and two, stories published by a press fixated on release dates. Some stories, for example, had the audacity to claim that without a doubt Linux 2.4 would be released February 17, 2000, in order to deliberately challenge or usurp the release of Windows 2000.
Another source of false deadlines, and also false features, are the
developer's mailing lists. Linux is an open source project. Anyone can subscribe to one of the many developer's mailing lists and receive any and all correspondence among developers. The problem is that many subscribers think that the correspondence among top developers (Linus Torvalds, Alan Cox, or David S. Miller) reflect the official release feature list instead of the exchange of ideas that it is.
There will be no release of Linux 2.4 until it is ready. The feature freeze is in place and everyone that knows better is waiting for the release date to set itself based on the completion of coding and testing. Limiting the changes and holding the release until thorough testing has been done is the only sane option for Linux 2.4.
The Linux open source development cycle
The method used to develop Linux 2.4 involves a Linux 2.3.x version. This is the developer's version. Linux 2.2.x is the current release. The features announced for 2.4 exist, in various stages of completion, in 2.3. Independent developers, as they discover a particular need or a bug in the existing 2.3 code submit additions and changes. Linus Torvalds and other key developers then test code submissions for quality and side effects. When 2.3.x is declared stable, it will be released as 2.4.
The Linux 2.4 kernel changes
There are a large number of changes and additions to the 2.4 release. The exact list of changes is still a matter of speculation and will not be truly known until near the release date. However,
Linux 2.4 will have many changes to the kernel that will result in more speed and reliability and a much greater base of supported hardware.
General design changes
Linux ELF and POSIX foundation strengthened
Linux kernel 2.4 is more dependent on the ELF (Executable and Linking Format) than Linux 2.2. ELF is an advanced binary format that includes support for multiple code and data sections and increases the support for shared libraries. ELF, most notably used in the Solaris OS, is like the Win32 format, but better designed. Fully exploiting the ELF binary format will allow Linux kernel developers to make some pieces of code more modular and easy to maintain. ELF will allow drivers to be initialized based on how they are linked rather than by having an explicit initialization line in the core code.
With the adoption of support for POSIX clocks and timers, Linux 2.4 becomes more POSIX compliant, allowing for non-rtc devices to be used as clocks internally.
Enterprise support: threading, scheduling, and capacity
Three changes in Linux 2.4 can be described as enterprise level OS designs. These features do not degrade Linux in standard environments and add much greater capacities to standard Linux.
First: Linux 2.4 can handle many more simultaneous processes or threads because it is more scalable on multiprocessor systems and provides a configurable process limit. The threading model used in Linux 2.4 has been changed to a scalable or "soft" version. Previously the thread limit was 1024, which led to poor performance with large numbers of users. The limit is now set at run time, so the only real limit is the amount of memory available.
Second: The scheduler has been revised to be more efficient on systems with more concurrent processes.
Third: Capacity has been greatly increased in several areas.
- The new kernel can, amazingly, handle about 4.2 billion users and groups.
- 4 gigabytes of RAM is supported on Intel hardware.
- 16 Ethernet cards can be installed on one system.
- 10 IDE controllers can be installed on one system.
- The file size restriction of 2 gigabytes has been removed.
| Definition of a daemon
The ancient Greeks defined a daemon as a protective spirit that presided over a person's destiny. In UNIX daemon stands for a Disk and Execution Monitor. Daemons are server processes that offer services. They run in the background and generally lie dormant until a local or remote process calls them.
|
|
Resource management
Resource management code has been completely rewritten to make it much more scalable and general purpose. This will, in the long term, allow for greater control over the resource management subsystem and make it easier to use.
Integrated Web server
Newly integrated into Linux 2.4 is a kernel Web server, khttpd. This kernel-space http
daemon
won't have to exist in user-space. It sends data to
kernel-space to be taken to the network connection. This results in faster
response times. The khttpd can, however, only run static pages. Apache or another httpd will have to be used to run CGI programs, because khttpd is not
designed as a replacement for Web servers. If it receives a
request for CGI, khttpd will pass the request to user space where a Web
server can process it.
| Note on "races"
A race is a software condition that occurs when the execution of a program is affected by the order and timing of a thread's execution. Race conditions can be corrected using synchronization variables to control the threads' timing and access of shared resources. If a program depends on order of execution, then threading that program may not provide a
good solution because the order in which threads execute is non-deterministic. Programming languages that support threading have mechanisms to force thread synchronization.
|
|
Networking
Linux 2.4 has a completely rewritten networking layer. It has been made as un-serialized as possible to allow for better scaling. The network subsystem is redesigned for stability on multiprocessor systems and many
races
have been eliminated. There are also many updates to the existing network driver set as well as many new devices, including support for ATM network adapters for high-speed networking.
The Linux 2.4 rewrite includes placing firewall and Internet protocol
functions into the kernel. The network subsystem has been split into two
pieces: a packet filtering layer and a network address translation (NAT)
layer. Each is more generic than its previous version and allows sophisticated routing through any Linux box. The rewrite includes a new user-space tool to manage the available functionality. To make the upgrade easier by providing backward compatibility, modules that will allow use of the Linux 2.0 ipfwadm or Linux 2.2 ipchains are included.
Much of the PPP (point-to-point protocol) code has undergone major rewriting and modularization. The kernel now combines the PPP layers from the ISDN layer and the serial device PPP layer (like those used for dial-up connections with modems). ISDN has also been updated to support many new cards. The PLIP (PPP over parallel ports) layer has been improved as well, and now uses the new parallel port abstraction layer and PPP over Ethernet (PPPoE) support, a protocol used by some DSL providers.
Enterprise level changes to networking introduce features that will better enable Linux to be integrated into existing network infrastructures. Linux 2.4 adds DECnet support for interoperating with specialized Digital/Compaq systems and ARCNet protocols, as well as hardware that allows for better interoperation with specialized systems (including older Digital/Compaq ones).
Linux 2.4 will be able to auto-detect Windows shares based on Server Message Block (SMB) protocol. This makes Linux a better candidate for use in mixed (heterogeneous) networks. The Linux 2.4 kernel removes the compile-time requirement of selecting support for mounting drives from Windows 9x or NT. 2.4 will be able to auto-detect the remote system type and enable bug fixes (some versions of Windows have SMB bugs) as needed. (This is for an SMB client only. The Samba package can be used if there are server needs.)
The UNIX network-sockets model has been changed. With Linux 2.2, processes waiting on an event from a network socket were all woken up at event detection. Only one succeeded in getting the event, and the rest put themselves back to sleep. Linux is an efficient system and does this quickly, but eliminating this "stampede effect" would increase network speed significantly. The new model is called "wake one". This new code wakes up only one process in the case of activity. Apache and other servers will begin using this new functionality when 2.4 ships.
The IPv4 implementation is more scalable, and the use of colon-mode for IP "aliasing" has been removed. Linux is still the only operating system completely compatible with the letter of the
IPv4 specification.
Miscellaneous
Linux 2.2 added support for a flexible binary loader module designed to allow association of binary types (based on extension or file header information) with applications. This allows Windows applications on a Linux machine to be run with the WINE Windows Emulator. 2.4 continues to support this.
Linux 2.2 and Linux 2.0 included built-in support for Java. Direct support to run Java binaries is removed from Linux 2.4. The misc binary loader, provided with Linux 2.2, must be used to start Java applications. No functionality is lost, but the loader will have to be configured into the startup scripts.
Exceptions and potential limitations
The Linux 2.4 kernel is not compatible with Linux 2.2 in some areas. Many of the internal structures have changed, as have some file formats in /proc filesystem. But in most cases applications will run without problems. The exceptions will be those applications that work at a kernel level (daemons). The names of most files in the /dev directory have changed, but there are compatibility names available.
Although it will not be present in Linux 2.4, there is ongoing work supporting the NetBEUI protocol used by Microsoft operating systems. Microsoft is moving away from this protocol towards TCP/IP, but this protocol is still important for a number of Windows-based network environments. Kernel developers have said that the protocol is too messy and buggy to be supported in the kernel. NetBEUI may not be stable enough to be included in the 2.4 release.
Linux 2.4 is not likely to include support for cryptography in the main distribution. Import and export regulations for cryptography are different throughout the world and, to keep development simple, the standard kernel will be not include encryption. Patches are available to add encryption.
Linux kernel 2.4 does not directly address the desktop OS issue. Nonetheless it does provide the distribution vendors with a set of tools to greatly enhance the installation process.
Hardware changes
CPU support
Linux 2.4 adds support for three new architectures: Intel ia64 (Itanium/Merced),
IBM S/390, and Hitachi SuperH. (Intel Itanium/Merced CPU is not yet available.) Changes made to the 2.4 kernel are based on the current specifications and existing support for current 64-bit CPU architectures like the Alpha and Sparc64.
Linux 2.4 hardware support, in general, is very similar to Linux 2.2. All Intel chips 386 to Pentium III are supported, as are the compatible AMD and Cyrix chips. The Crusoe CPU, designed by Torvalds' employer Transmeta, uses a Code-Morphing firmware that emulates the i386 so there is no need for a Crusoe-specific port of Linux.
Support for high-end hardware to increase speeds comes from support of non-Intel varieties of the Memory Type Range Registers (MTRR) for the AMD K7 processors and the Cyrix processors' variation called MCR, which will improve performance on some high bandwidth devices.
Where Linux 2.2 included support for the Memory Type Range Registers (MTRR) used on the newest Intel chips to increase performance of some kinds of high-bandwidth devices, Linux 2.4 will take this even further by supporting MTRR (and MCR) variants from alternative vendors AMD and Cyrix.
There will be no support for any CPU previous to the Intel 386. Anyone wanting to install Linux on an old 286 box will have to look for specialized versions like the Embeddable Linux Kernel Subset (ELKS).
The 2.2 kernel scales well up to four processors. Linux 2.4 should support sixteen or more processors.
Advanced Programmable Interrupt Controller
Linux 2.2 included support for the IO-APIC in order to allow interrupts to be spread across multiple processors. Linux 2.4 will support the use of multiple IO-APIC controllers for extremely high-end servers.
Memory usage
Linux 2.4 will use about the same amount of memory as Linux 2.2. Exact numbers will change with each configuration. Some key subsystems, such as the file-caching layer, will use less memory to do the same work, so some memory decreases may be seen.
The way Linux handles shared memory has been changed. Linux 2.4 will be more compliant with industry standards. The changes require a special "shared memory" filesystem to be mounted in order for shared memory segments to work. Distribution vendors will handle this when they are ready for Linux 2.4.
| Other hardware supported
- IDE/ATAPI multi-slot CD changer reporting configuration option
- Various IDE chipset support added
- Amiga Gayle IDE interface support added
- Atari Falcon IDE interface support added
- Amiga Buddha/Catweasel IDE interface support added
- Amiga IDE Doubler support added
- PowerMac IDE driver
- USB CPiA video camera driver
- Integraphics Cyber2000 frame buffer console driver
- EFS filesystem support (read-only)
- USB Abstract Control Model support
- 16-color VGA frame buffer console driver
- DECnet networking support
- SGI disk label support
- USB printer support
- CM8338/CM8738 sound driver support
|
|
Other hardware
Linux 2.4 can now be run on machines on which the keyboard is not pre-initialized by the BIOS, or on machines without keyboards.
Linux 2.4 kernel has expanded support for digitizer pads, including mouse emulation.
It has greater support of PC, Atari, Amiga, and Macintosh hardware.
Hardware support issues
Linux kernel 2.4 adds a large number of additions to the list of supported hardware. The exact list of what the supported hardware will be when 2.4 is released is a matter of some speculation.
Hardware detection and driver software design/coding are complex problems. First, a detailed knowledge of the hardware is needed. This requires cooperation from the vendor and the hardware designers themselves. The hardware may be designed to meet a standard, but problems such as noncompliance arise with some aspects of standard or firmware certain devices. People often wonder, "Why can't Linux use this card if Windows can?". Sometimes Windows also cannot. Often a driver supplied with the hardware fails or has side effects and an update is needed. Due to the lack of popular support for Linux among recreational users, vendors are more apt to ignore Linux for devices that are used for personal systems (as opposed to those devices used on servers). No one puts a VooDoo 3000 AGP video card in a dedicated Apache server.
OK, so what and when?
The changes that are being made to the Linux 2.4 kernel code appear to have three major goals in addition to the obvious bug fixes.
One: Greater speed, especially on large-scale servers. This should once and for all stop Microsoft from commissioning tests that make NT look better under very specific circumstances. There should no longer be any configuration and load combination in which Linux is slower. Paul Ferris, of Linux Today, offers this perspective:
There were a series of benchmarks in 1999 that involved a
company called "Mindcraft". While the benchmarks showed
that Linux was significantly slower than NT for some rigged
tests -- and some of the data was eventually proven to be
pointing in the right direction, the methods used were so suspect
as to cause a lot of people to dis the company. They were supposedly
paid for by an independent entity, but that entity was soon revealed
to be none other than Microsoft. The outcry was deafening.
The positive side of things are about to hit the market -- many
of the programmers involved in the open source philosophy simply
looked at the data as a nice way to point out where work was needed,
and they improved not just the Linux kernel, but also Samba and other
aspects of Linux.
That's why the 2.4 release of Linux was referred to as such.
Two: Greater market penetration. Greater and easier hardware support through the addition of many drivers and changes to the code should ensure that Linux will be more broadly adopted.
Three: Fewer kernel variants. It seems strange to have laptop and enterprise server elements in the same standard release. The purpose is to eliminate the plethora of custom versions. The same Linux kernel will work, as is, on an embedded device, a laptop, a desktop, and a network server. Variant elimination is achieved with the new drivers and scalability that have been added to the kernel.
I have a somewhat optimistic estimate of early September for the 2.4 release. But realistically it may even be as late as spring of 2001. This may be later than we all would like, but it would be a nice way to start the millennium (the real start, not that year 2000 nonsense). One thing is a certain: Linux kernel 2.4 will be released when it is ready and not a day sooner.
Resources -
The Linux 2.3 kernel can be found at Linux Headquarters, which supplies documentation and news about Linux kernels.
- Read both articles in this "Meet the 2.4 Linux kernel" series.
- Learn how to acquire kernel source, configure and boot your new kernel, add a feature, fix a flaw, or just have fun tinkering with operating system source code in our "Hacking the Linux kernel" tutorial series. Hack and be free.
- Build your skills in Linux systems administration with our certification exam study guides. Whether you choose to take the exams or not, our certification-prep tutorial series will immerse you in Linux fundamentals as well as advanced topics.
-
Find more resources for Linux developers in the developerWorks Linux zone, including our newest how-to tutorials.
-
Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
-
Build your next development project on Linux with IBM trial software, available for download directly from developerWorks.
-
Get involved in the developerWorks community by participating in developerWorks blogs.
About the author | | | Thomas Wolfgang Burger is the owner of Thomas Wolfgang Burger Consulting. He has been a consultant, instructor, writer, analyst, and applications developer since 1978. He can be reached at twburger@bigfoot.com. |
Rate this page
| 
