Porting Unix to the 386: the Final Step

Porting Unix to the 386: the Final Step

Running light with 386BSD

William Frederick Jolitz and Lynne Greer Jolitz

Dr. Dobb's Journal

July 01, 1992

Over the past couple of months, we've discussed the minimal code and methodology required to fill in the missing pieces of the incomplete Net/2 tape, leading to an operational 386BSD kernel. Still, the step from a few changes to a real system is great, as evident by the items that had to be provided with this kernel: bootstraps, file systems, an installation mechanism, binaries of utilities, and documentation.

However, we decided it was time to step away from the kernel and make all of 386BSD available and accessible, so that it could become the generic research and educational platform we envisioned when we wrote "386BSD: A Modest Proposal," the software specification for 386BSD, in mid-1989. So, on March 17, 1992 we launched 386BSD Release 0.0.

386BSD Release 0.0--Liftoff

386BSD Release 0.0 consisted of:

One distribution installation floppy.
One 8-floppy multivolume compressed TAR-format source distribution (31 Mbytes uncompressed).
One 6-floppy multivolume compressed TAR-format binary distribution (28 Mbytes uncompressed).
One 360K MS-DOS difference floppy (for those who want to do all the work themselves).
Release notes regarding installation procedures, manifests, and registration/bug report forms. All of 386BSD Release 0.0 was released under freely redistributable and modifiable terms (with attribution to the authors maintained--a small and reasonable request for so much).

With the assistance of several dedicated network volunteers (among them John Sokel, Dan Kionka, and members of the Silicon Valley Computer Society), 386BSD Release 0.0 was made widely available via the Internet. Within one month, an estimated 100,000 sites had obtained 386BSD Release 0.0. (Several networks, particularly in Australia, melted down over the transmission traffic and had to be regulated.) The enthusiastic response from Internet, BBSs, and various user groups (through which copies were widely distributed) has far exceeded our expectations.

Another pleasant surprise was the number of software contributions, bug fixes, and suggestions from early users. People were eager not only to supply their code and knowledge, but to help others get their systems running too.

Finally, it was gratifying to learn that our little system, bugs and all, was still capable of being complete enough to be used for the rest of its own development. There's little question that, in less than a month, 386BSD Release 0.0 was an unqualified success.

386BSD Release 0.1--The Second Stage

386BSD Release 0.1 is the most recent release as of this writing. It consists of:

A single distribution installation floppy, referred to as "Tiny 386BSD."
One 15-floppy multivolume compressed TAR-format source distribution.
One 10-floppy multivolume compressed TAR-format binary distribution.
Installation notes, manifests, and registration/bug report forms.

The major Internet sites from which you can download 386BSD 0.1 are agate.berkeley.edu and reyes.stanford.edu. Additionally, we're available to answer questions and provide some support on CompuServe (CIS#76703,4266) and in the UNIX/BSD conference on Bix.

For limited program development, a 386/486 system should contain at least two Mbytes of RAM and a 40-Mbyte hard disk. To make full use of the source tree and generate new software distributions, a 200-Mbyte disk is recommended. (You can also use 386BSD's version of NFS to obtain space via the Ethernet off a central shared server.) Performance obviously improves with faster processors and more memory.

Just in case anyone is wondering, the entire 386BSD system was created on a 386SX laptop with three Mbytes of RAM and one 100-Mbyte disk.

What's in 386BSD Release 0.1

Thanks to many knowledgeable users, 386BSD 0.1 is a more robust version of 386BSD, supporting broader combinations of PC hardware and simpler installation procedures. 386BSD also provides for more features and functionality.

386BSD 0.1 also contains many utilities which can be used in development work (see Table 1), including a C compiler, C++ compiler, loader, network protocol family (TCP/IP), and so forth. 386BSD also contains a complete set of Internet networking facilities (including NFS), a program development environment for gigabyte-sized programs, document-preparation and text-editing tools, and database mechanisms. And finally, it can rebuild itself from its own source tree.

Table 1: 386BSD Release 0.1 utilities.

  apropos    env       lpr        pwd        timed
  ar         eqn       lprm       query      timedc
  arp        expand    lptest     ranlib     tip
  as         expr      ls         rcp        tn3270
  bad144     false     m4         rdist      touch
  badsect    find      machine    rdump      tput
  basename   finger    mail       reboot     tr
  biff       fmt       mailstats  renice     trace
  cal        fold      make       restore    traceroute
  calendar   from      man        rlogin     troff
  cat        fsck      mesg       rm         true
  cc         fstat     mkdep      rmail      tsort
  checknr    ftp       mkdir      rmdir      tty
  chgrp      g++       mkfifo     rmt        tunefs
  chmod      gcc       mknod      route      ul
  chown      gdb       mkstr      routed     umount
  chpass     genclass  more       rrestore   uncompress
  chroot     grep      mount      rsh        unexpand
  cksum      groff     mountd     savecore   unifdef
  clear      grops     mset       script     uniq
  clri       grotty    mtree      sed        unvis
  cmp        groups    mv         sh         update
  col        halt      named      shar       uptime
  colcrt     head      newfs      showmount  users
  colrm      hexdump   nfsd       shutdown   uudecode
  column     hostname  nfsiod     size       uuencode
  comm       id        nfsstat    slattach   vacation
  compress   ifconfig  nice       sleep      vipw
  config     inetd     nld        sliplogin  vis
  cp         init      nm         soelim     w

  cpio       install   nohup      split      wall
  cpp        kdump     nroff      strings    wc
  csh        kill      nslookup   strip      what
  ctags      ktrace    nsquery    stty       whatis
  cu         last      nstest     su         whereis
  cut        ld        old        swapon     which
  date       leave     pac        symorder   who
  dd         lex       pagesize   sync       whoami
  df         ln        passwd     syslogd    whois
  dirname    locate    paste      talk       write
  disklabel  lock      pic        tar        xargs
  diskpart   logger    ping       tbl        xstr
  du         login     portmap    tee        yacc
  dump       logname   printenv   telnet     yes
  dumpfs     lpc       printf     test       yyfix
  echo       lpd       ps         tftp        zcat
  elvis      lpq       psroff     time

Qualifying a PC to Run 386BSD

Release 0.1 can be run in as little as a single floppy disk, using the Tiny 386BSD diskette available through the DDJ Careware Project. Send us a formatted, error-free, high-density 3.5 or 5.25 floppy diskette and an addressed, stamped diskette mailer, in care of: Tiny 386BSD, Dr. Dobb's Journal, 411 Borel Ave., San Mateo, CA 94402, and we'll send you the latest copy. There's no charge, but if you want to slip in a dollar or so to help out the kids at the Children's Support League of the East Bay, we know they'd appreciate it. (You can also obtain this software directly from the sites mentioned above.) In addition to experimenting with a very minimal 386BSD system prior to loading any software on the hard disk, Tiny 386BSD allows you to validate 386BSD operation on a PC. Simply insert the floppy into the drive and boot up the PC. If it boots and prompts you for a shell command (#), you're ready for installation of the rest of the system. If it fails, it's time to skull out the PC configuration, jumpers, BIOS setup menu, and all the other little "bright spots" that make for interesting compatibility problems. The general rule here is to isolate the problem by comparing cases that work with those that don't until an explanation can be formed. In general, nifty hardware features are the source of most compatibility problems, along with bizarre hardware combinations that are "on the edge." Both should be avoided or defeated when they are behaving suspiciously. Mainstream hardware from patient and understanding firms helps a great deal. Another problem arises with non-standard "old" equipment that you just happen to have lying around. Do yourself a favor and leave these particular PCs to MS-DOS, since other hidden surprises possibly await. In short, avoid the "pathological" cases wherever possible.

Installing 386BSD Release 0.1

To install the rest of 386BSD, we must allocate a large portion (greater than or equal to 40 Mbytes) of formatted disk space on a hard disk--either the entire contents of a disk drive or the remaining contents of a disk drive after other systems have been partitioned. The process discussed here is an exact duplicate of the mechanism we devised five years ago for system-installation procedures on Symmetric Computer Systems' 375 computers. (This was contributed to Berkeley, and parts appeared in the 4.3BSD Tahoe release.) Since 386BSD 0.1 is experimental software, we recommend that it be run with a dedicated disk on a dedicated system, until it matures. Disk space must be formatted by a utility. IDE and SCSI drives are already preformatted. ESDI controllers have formatting programs in ROM that can be run from a MS-DOS Debug utility, and the geometry of the drive can be obtained from these programs (in terms of cylinders, tracks or heads, and sectors). ESDI drives use the last cylinder to hold bad block tables; currently these are not used, and the last cylinder must be ignored. In addition, since 386BSD uses its own bad block revectoring (a la DEC standard 144), sector sparing should not be used. If possible, the formatted drive should have the same low-level geometry of the hard drive itself. However, drives greater than 1024 cylinders are run in a logical translation mode by the disk controller to make up for a limitation in MS-DOS. While this logical formatting will work, the 386BSD file system will not be as efficient, since its clever rotational placement algorithms won't mesh with what the physical drive is actually doing. With the drive geometry information in hand, a disktab entry describing how 386BSD is to use the disk space must be written; Example 1 is a sample disktab entry. In general, we prefer a 5- to 10-Mbyte root partition, a swap partition about twice as big as the amount of RAM memory, and a /usr partition that contains the remainder. Each partition has a size, offset, and type associated with it. Both size and type are in units of sectors, and each partition is arranged to start at the beginning of a cylinder, so that the rotational placement algorithm won't be thrown off by a logical partition offset. We then use the disklabel command from the floppy-based system (analogous to fdisk) to install a machine-readable version of the disktab entry onto the hard disk, along with a bootstrap program.

Example 1: A sample disktab entry.

  cp3100|Connor Peripherals 100MB IDE:\
          :dt=ST506:ty=winchester:se#512:nt#8:ns#33:nc:#766:sf:\
          :pa#12144:oa#0:ta=4.2BSD:ba#4096:fa#512:\
          :pb#12144:ob#12144:tb=swap:\
          :pc#202224:oc#0: \
          :ph#177936:oh#24288:th=4.2BSD:bh#4096:fh#512:

Once the hard-disk configuration is completed, we must do a "high-level" formatting of the 386BSD partitions that will hold files. Analogous to the MS-DOS format program that creates a blank file structure on a hard disk, the newfs program is executed off the floppy and initializes the root and user partitions of the hard disk.

Next, the root partition is made accessible as a file system by means of the mount command, and the contents are transferred from the floppy-based system to the hard-disk root file system and dismounted, making the hard disk bootable. The floppy-based system is shut down gracefully, and the hard-disk version booted in its stead. At this point, the floppy disk can now be used to load on the remainder of the system. We mount the empty user file system and proceed to reload the file system with the tar utility to extract the system from a multivolume floppy dump.

For those with access, the boot floppy allows the restore to occur over the network, thus eliminating the need for extracting from floppy dumps.

System Configuration

All the configuration files are located in the /etc directory of the root file system, including brief notes on setting up the system. The elvis text editor can be used to modify them as needed. Further configuration and expansion can be accomplished by loading the sources from another multivolume floppy dump, and recompiling the system and its utilities.

386BSD documentation, including the installation procedures for the rest of 386BSD (binary and/or source), should be available as part of the online 386-BSD release. If it is not online, contact the sysop or moderator to have it installed, and send e-mail via CompuServe at 76703,4266.

Perspective: The Importance of Thinking Small

In the spirit of "running light without overbyte," 386BSD is a minimalist system. This approach has allowed us to easily discuss the important paradigms effectively leveraged in UNIX and other modern operating systems. Over the course of this series, our minimalist approach has forced us to cleave to a basic understanding of the functional core of the system, and not get bogged down in the minutiae of building obscure utilities. It has also provided us with a bit of "editorial" oversight on the contents of our system. Occasionally, it is as necessary to discard an item as it is to create one--otherwise, we would be "hip deep" in relics held past their prime. In addition, a minimalist system is an excellent educational and training platform in the teaching of operating systems, net-working, file systems, C++, and software management.

Another virtue of our minimalist approach is that the sizes of the source and the operating binaries was greatly reduced in bulk. Paring down redundant utilities and source code increases ease of use without loss of functionality.

Where is 386BSD Heading

In subsequent releases, we expect 386BSD to grow in both size of distribution and stature of function quality, without losing its minimalist design. Some topics now ripe for exploration are as follows:

Many UNIX systems have adjusted only "fitfully" to life on the PC; it's almost as if they were immiscible from the very start. Yet many good features developed in MS-DOS and Windows are missing from the UNIX paradigm. It's a shame that parochial attitudes keep UNIX systems architects from leveraging the largest programming environment present in the world today.
File I/O and networking transmission rates seem to be the limiting factor in UNIX systems performance, especially as the audio/video data demands of multimedia are starting to become significant. The problem is not with the hardware, but with the software and overall architectures. As such, PCs and workstations provide actual data transfer rates at only a fraction of the ten-Mbyte per second possible with state-of-the-art hardware technologies. It's hard to believe that, while we're on the verge of 100-MIP processors, most PCs will be transferring data at less than an order of magnitude faster than the original, wheezing 8-bit PC.
Light-weight processes formed in a tiny fraction of a millisecond are a necessary component to experimentation with new programming architectures. Multiprocessor versions of these processes should allow extensions into the time and space domains of multi-threaded models.
The ability to explore the file-system metaphor without the need for kernel programming is an interesting challenge. File systems are a popular area of study these days, because they are an ideal vehicle for exploring system performance (bandwidth), integrity (file stability and recovery), distributed systems (locality and replication), and the central universal abstraction of the applications program interface (like that in Plan 9; see DDJ, January 1991).

Farewell to the Porting Series--Onward to New Topics

With the completion of 386BSD and its widespread availability, it's time to bid farewell to our "Porting UNIX to the 386" series. After 17 installments (and, believe it or not, quite a few shortcuts), we actually finished our port, and we are happy that people can finally use the system about which we have spent so long writing.

With each new version of 386BSD, we hope to see it become more affordable, available, accessible, modifiable, and understandable. 386BSD still has quite a way to go towards becoming a mature operating system, but already it has traveled the "useful" portion of the distance.

As such, we intend to explore topics such as networking, which impact not only 386BSD, but modern operating systems in general. While we may revisit 386BSD in our discussions, it is important to view it in the context of other modern operating systems approaches on the UNIX side (Plan 9, Mach, Minix, and the like) and in the broader commercial domain (MS-DOS, Windows-NT, OS/2 2.0, and so forth). 386BSD is really a microcosm of what these "big" operating systems are all about.

Given support and encouragement, we also intend to continue the educational and research direction upon which 386BSD was based, and we will continue to assist other groups who wish to head in this direction with us. However, the growth of 386BSD and discussion of new approaches depends on the continued goodwill and enthusiasm of its user base. Everyone is welcome to participate in this process.

Bill was the principal developer of 2.8 and 2.9BSD and was the chief architect of National Semiconductor's GENIX project, the first virtual-memory, microprocessor-based UNIX system. Prior to establishing TeleMuse, a market research firm, Lynne was vice president of marketing at Symmetric Computer Systems. They conduct seminars on BSD, ISDN, and TCP/IP. Send e-mail questions or comments to ljolitz@cardio.ucsf.edu. (c) 1992 TeleMuse.

Previous installments:

June 1992 - Porting Unix to the 386: MISSING PIECES, PART II - A "bare-bones" execve( ) system call is implemented this month.

May 1992 - Porting Unix to the 386: MISSING PIECES, PART 1 - Over the next couple of months, we present the glue that lets you build an operational kernel.