From: a...@lxorguk.ukuu.org.uk (Alan Cox)
Subject: Streams and Linux
Date: 1998/06/29
Message-ID: <m0yqSLz-000aOnC@the-village.bc.nu>
X-Deja-AN: 366956150
Approved: g...@greenie.muc.de
Sender: muc.de!l-linux-kernel-owner
Newsgroups: muc.lists.linux-kernel
[Red Hat firmly off, Linux hacker hat firmly on]
> pennies worth. All of the following is my personal opinion, worth
> every penny you have paid for it. :-)
I can't be bothered to even discuss the more stupid poltical aspects
of all this. I'm just going to explain technical issues - nothing more
or less.
> "convert" Streams code to "Linux-specific" alternatives necessarily
> preclude this kind of "conversion" as an option. Given this
> premise, we have two alternatives:
The are several things people muddle up
1. Streams
This is an API design for layered networking. Even the inventor
of the streams concept said of the way it was used
"The idea loses something when it is shouted"
Streams are technical very flawed and not a good thing to put in
a kernel. Indeed our Solaris, SunOS, SCO etc emulation layers
turn streams into sockets as fast as possible. (note btw ibcs2
does this -without- kernel patches), as could Caldera. Although
for the syscall registering issue there may be a sane reason for a
registering those. Ask Linus.
2. TLI/XTLI
These are the interfaces _most_ people actually use. TLI and XTLI
isolate the user from the innards of streams. TLI is actually
very flexible, it tends to be unpopular with non commercial people
because TLI programming is painful.
Alexey Kuznetsov wrote a complete TLI emulation over sockets layer
for Linux. Entirely in user space. Its free, it exists it addresses
most of the issues people often think are "streams". This TLI
implementation is free software.
So most TLI/XTLI applications are sorted. Finally almost all
applications in the commercial world are written to the socket API
(maybe emulated onto TLI). Thats another reason vendors are running
from streams as fast as they can.
> Besides, despite the strongly-held opinions of many persons, the
> jury is still out on whether or not "Streams performance" is all
> that bad. And even if the performance of Streams _is_ that bad,
The jury returned the final verdict about four years ago. There is no
argument about the technical flaws in streams: SGI do not use streams
internally for networking, Sun moved away from streams for the performance
parts of their networking (their papers imply what actually occurs is
"Hi Im a streams module but I do Sun funky network too" "Hi Im the other
module on this stack right now, I also do sun funky networking - lets
stuff this streams s*** and talk sanely"). Sun moved socket() and friends
back into the kernel. Sequent moved sockets and socket API stuff back into
the kernel. Unixware Im reliably informed is currently doing the same.
So from a technical point of view streams is dead. There are people who
worked on streams for years at companies like Spider, who specialised
in making streams go as fast as is possible who will tell you it doesnt
go fast enough and equally importantly you cannot make it scale if you
want to be top of the pile.
Thats the technical status.
> isn't the most important issue. If I had a choice between _no
> application_ and a _slow application_ which met my critical needs,
> I'd choose the latter 100% of the time. Would you?
The Linux kernel is a technical project. Streams are "not interesting"
technically. Productization is a vendor issue.
> a mistake to tie "NetWare on Streams" with "NetWare performance".
> The two are entirely unrelated.
Indeed - netware is fast "despite streams" and "despite the netware
protocol weaknesses" - its a victory of sheer human stubbonness and hacking
skill over technically poor tools. You may care to benchmark netware 3
versus netware for unix on pure packet turn around time for a null NCP
operation. Now compare it with 2.1.x knfsd (socket based kernel code)
turning around an NFS no-op.
You will find it interesting.
> A: Because certain extensions to the kernel must be made in order to
> support the Streams loadable module. For example, Streams
> introduces some new system calls which are NOT present in the base
> kernel (e.g. putpmsg, getpmsg). The kernel system-call table must
> be modified to support these Streams entry points.
Syscall numbers you didnt register with Linus as far as I can tell, which
means since 2.2 uses more numbers you are likely to see breakages. Irrespective
of any streams issues those two calls as NULL hooks you could get dropped
into 2.1.x. Ask Linus - he did it for AFS. I cannot put anything like that
into 2.0 until it has official 2.1.x syscall numbers from the man himself
or I cause a back compatibility monster with a syscall numbering collision.
> that the kernel changes required to support a loadable Streams
> module are strikingly tiny. But there is such a "religious"
> opposition to Streams among _kernel developers_ that they refuse to
> allow even these tiny patches into the "base kernel". They seem to
It is entirely a technical opposition.
> fear the introduction of support for a Streams module as such a
> terrible "pollution" of their kernel that they have not and
> apparently will not allow it to happen. So Caldera is stuck--the
Putting support for something in the kernel implies a maintenance commitment
that in this case is not there. Ask Caldera or any vendor about the effect
of shipping a package you cant maintain. They can write in big letters
"this package is an add on its not supported", it makes no odds. So there
are real issues adding anything to a kernel - especially as its damned
hard to remove something from the kernel, even when its way more dumb than
say streams
That said I see no reason why Caldera shouldnt ask and get a pair of syscall
numbers from Linus they can hook.
> cases (e.g. NetWare for Linux), the software under consideration is
> proprietary and hence is not a candiate for a "freeware"
> reimplementation.
Actually there is MARS_NWE which is effectively exactly that. Caldera are
really selling a "branded" genuine netware. That I suspect makes the case
even more awkward for a rewrite. Again your cite is a political and
productization issue.
> A: I've been truly amazed at the "group opinion" on this subject.
If you were technically aware of whats going on in networking you wouldnt
be. Streams is being dropped globally. Streams was originally a victory
of standards people over sanity, its now being blasted into oblivion because
1. Networks have sped up by a factor of 100 in 3 years - Memory hasnt
2. Network performance is the hottest checklist item and its going to grow
faster and faster.
> Most people say that opposition to Streams is on technical grounds,
> but I believe the real opposition is a political one. Many people
I've never seen Linus with a political agenda. And even the people who get
political about streams (eg Larry McVoy) do so from a technical base point.
> be going out of their way to make Streams painful for everyone, as
> if to fulfill their own prophecies on the "badness" of Streams.
They don't need fulfilling.
> The fascinating thing going on here is the fear of the "success" of
> Linux: If someone wrote a "major application which depended on
It wouldnt be a success - it would be an app that screws the future of
the Linux networking stack. Right now we are the most shit hot piece of
PC networking software on the planet. The only people who can touch us
are FreeBSD (who also dont do streams).
Having an application that is streams dependant that makes it hard to break
the streams stuff isnt acceptable from a purely technical standpoint. Nor is
it good for the long term future of Linux.
What it means 3 years down the line is Linux would be bottom of the webstone
benchmark and all the nice "linux is fast" graphs that sell Linux webservers
will be gone, dead and history.
Linux is _technically superior_ - that is its fundamental superiority. The
core Linux product must continue to be so and the Linux core team agenda
is purely technical superiority
> the masses. (Not to mention David's unsupported assertion that the
> mere presence of Streams in the kernel cause the kernel itself to
> slow down--this just ain't so.)
It does. We cannot do zero copy page flipping in a streams environment. When
we do that the GCOM code _will_ probably break. I suspect it'll break in a
fixable way where it'll be even slower again but with 1Gigabit networking
once the cards support it , non zero copy networking is _not_ an option if
we want to stay top of the pile. So we will end up breaking the GCOM stuff
possibly pretty badly although I hope not, there is no malice in breaking
of things.
Another very worrying issue is field of interest in the support. Caldera
ship only x86 Linux. Has anyone ever even verified the GCOM streams code
is 64bit clean big and little endian, alignment clean, copes with ARM
packing rules and all the other little portability wonders the mainstream
code has to meet.
> I disagree with David's assertion that Streams will "creap" [sic]
> like a "fungus" into the rest of the kernel. Streams has a
> well-defined kernel interface that is not likely to change much
No streams does NOT have a well defined kernel interface. It has a poorly
defined set of interactions with core Linux data structures. You may not
be a good enough programmer to see that - I don't know your background. I've
worked on Linux since it appeared, I've worked for people like 3Com who
had this problem badly. Anything that has data structure interfaces with
other code leaks through the structures over the boundary, or ties down
those structures.
Streams depends on the sk_buffs in the kernel. In a zero copy world those
buffers may well just not exist any more
> it's maintained by some nice guys at Gcom, NOT the _kernel
> developers_. Linux Streams (LiS) has a lifetime of its own
For how long. Who maintains it if Caldera pull out of Linux to concentrate
on DOS only. Who maintains it if Caldera decide GCOM are charging too much.
Are Caldera committed to 2.2 support of it, have they discussed getting
the syscall numbers (which is all the actually need) from Linus ?
> A: This remains to be seen, but a couple come to mind. Firstly,
> Caldera's NetWare for Linux (NW4L) would definitely NOT be on Linux
> were it not for Streams. You can decide for yourself whether or
These are entirely non-technical issues, and you seem to be a bit short
of actual examples except the netware client, which has a free version
shipped in the standard kernels.
The decisions people like Linus and Davem make on what goes into the
kernel are purely technical ones. If Caldera think there is a market in
shipping a modified "streams aware" kernel good for them. Streams is a
marketing and productization issue not a technical issue. Streams ran out
of technical arguments a very long time ago. Right now every vendor is
trying to figure out how to get streams OUT of their box while keeping
legacy API compatibility code from bloating the kernel another 200 K.
We don't want to put that 200K in. Maybe to Caldera its a worthwhile
commercial product. As I've said - reserve your syscall numbers I cannot
see an opposition to that. Thats the advice I gave in 1996 or so when
this first came up.
Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
|