From: David Anderson <dander...@swri.edu>
Subject: Stream Processing (provided by SVR4) for Linux?
Date: 1999/04/05
Message-ID: <7eba4m$cvd$1@usenet44.supernews.com>#1/1
X-Deja-AN: 463007796
Content-Type: text/plain
X-Complaints-To: newsabuse@remarQ.com
X-Trace: 923347926 IJVAWDTGTE 6FCEAA usenet44.supernews.com
Organization: Posted via RemarQ, http://www.remarQ.com - Discussions start here!
Mime-Version: 1.0
Newsgroups: comp.os.linux.development.system
Content-Disposition: inline
"SVR4 provides mounted streams and a streams processing module named
'connld' that we can use to provide a named stream pipe with unique
connections for the server." - Advanced Programming in the Unix
Environment by W. Richard Stephens
Is there a similar module under Linux?
------------------ Posted via SearchLinux ------------------
http://www.searchlinux.com
From: "G. Sumner Hayes" <sjhal...@yahoo.com>
Subject: Re: Stream Processing (provided by SVR4) for Linux?
Date: 1999/04/07
Message-ID: <370B77E5.7CA01FDF@yahoo.com>#1/1
X-Deja-AN: 463665341
Content-Transfer-Encoding: 7bit
References: <7eba4m$cvd$1@usenet44.supernews.com>
X-Accept-Language: en,es
Content-Type: text/plain; charset=x-user-defined
Organization: Netcom
X-NETCOM-Date: Wed Apr 07 8:21:40 AM PDT 1999
Mime-Version: 1.0
Newsgroups: comp.os.linux.development.system
David Anderson wrote:
>
> "SVR4 provides mounted streams and a streams processing module named
> 'connld' that we can use to provide a named stream pipe with unique
> connections for the server." - Advanced Programming in the Unix
>
> Environment by W. Richard Stephens
>
> Is there a similar module under Linux?
Others have pointed you at ftp.gcom.com for a STREAMS implementation,
but I'd like to note that most of the commercial Unices are moving away
from STREAMS back to a socket-based kernel system -- it's just not
possible to get decent high-end performance out of STREAMS. They were
an interesting experiment, but have been deemed a failure in the eyes
of most.
That's not to say that the concepts behind TLI are without merit, just
that STREAMS are a poor way of getting there.
--Sumner
From: David Grothe <d...@gcom.com>
Subject: Re: Stream Processing (provided by SVR4) for Linux?
Date: 1999/04/07
Message-ID: <370BD323.57248CD0@gcom.com>#1/1
X-Deja-AN: 463792480
Content-Transfer-Encoding: 7bit
References: <7eba4m$cvd$1@usenet44.supernews.com> <370B77E5.7CA01FDF@yahoo.com>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: Gcom, Inc
MIME-Version: 1.0
Reply-To: d...@gcom.com
NNTP-Posting-Date: Wed, 07 Apr 1999 17:50:27 EDT
Newsgroups: comp.os.linux.development.system
"G. Sumner Hayes" wrote:
> Others have pointed you at ftp.gcom.com for a STREAMS implementation,
> but I'd like to note that most of the commercial Unices are moving away
> from STREAMS back to a socket-based kernel system -- it's just not
> possible to get decent high-end performance out of STREAMS. They were
> an interesting experiment, but have been deemed a failure in the eyes
> of most.
One of the most common STREAMS strawmen is that it is useful only as an
environment for TCP/IP. Most people who are interested in STREAMS are
interested in protocol stacks in which the stacks get deep and whose
specific configuration is fairly unpredictable until the final end user
says "Put it together like this."
Chances are good that the original poster is not asking about TCP/IP in
STREAMS but wants to know if the STREAMS environment exists so that he can
port his existing driver code from SCO or Solaris. That's what most of my
e-mail concerning STREAMS is about.
-- Dave -- The author of LiS
From: David Grothe <d...@gcom.com>
Subject: Re: Stream Processing (provided by SVR4) for Linux?
Date: 1999/04/08
Message-ID: <370CBB1B.66BD327E@gcom.com>#1/1
X-Deja-AN: 464040018
Content-Transfer-Encoding: 7bit
References: <7eba4m$cvd$1@usenet44.supernews.com> <370B77E5.7CA01FDF@yahoo.com>
To: "G. Sumner Hayes" <sjhal...@yahoo.com>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: Gcom, Inc
Mime-Version: 1.0
Reply-To: d...@gcom.com
Newsgroups: comp.os.linux.development.system
"G. Sumner Hayes" wrote:
> Others have pointed you at ftp.gcom.com for a STREAMS implementation,
> but I'd like to note that most of the commercial Unices are moving away
> from STREAMS back to a socket-based kernel system -- it's just not
> possible to get decent high-end performance out of STREAMS. They were
> an interesting experiment, but have been deemed a failure in the eyes
> of most.
SCO OpenServer uses STREAMS TCP/IP; SCO UnixWare 2.x uses STREAMS TCP/IP;
SCO UnixWare 7 uses STREAMS TCP/IP; Solaris 2.6 for SPARC (and presumably
Intel) uses STREAMS TCP/IP. I don't have a Solaris 2.7 here, but I doubt
if it is any different.
Just what do you mean by "most of the commercial Unices"? If you go by
counts of units in the field, the ones that I just enumerated are the vast
majority of commercial Unix systems.
Are you saying that TCP/IP on Solaris SPARC is slow? I don't think so.
-- Dave
From: "G. Sumner Hayes" <sjhal...@yahoo.com>
Subject: Re: Stream Processing (provided by SVR4) for Linux?
Date: 1999/04/08
Message-ID: <370CD3F3.B06141FC@yahoo.com>#1/1
X-Deja-AN: 464072317
Content-Transfer-Encoding: 7bit
References: <7eba4m$cvd$1@usenet44.supernews.com> <370B77E5.7CA01FDF@yahoo.com>
<370CBB1B.66BD327E@gcom.com>
X-Accept-Language: en,es
Content-Type: text/plain; charset=x-user-defined
Organization: Netcom
X-NETCOM-Date: Thu Apr 08 9:06:48 AM PDT 1999
Mime-Version: 1.0
Newsgroups: comp.os.linux.development.system
David Grothe wrote:
>
> "G. Sumner Hayes" wrote:
>
> > Others have pointed you at ftp.gcom.com for a STREAMS
> > implementation, but I'd like to note that most of the commercial
> > Unices are moving away from STREAMS back to a socket-based kernel
> > system -- it's just not possible to get decent high-end performance
> > out of STREAMS. They were an interesting experiment, but have been
> > deemed a failure in the eyes
> > of most.
>
> SCO OpenServer uses STREAMS TCP/IP; SCO UnixWare 2.x uses STREAMS
> TCP/IP; SCO UnixWare 7 uses STREAMS TCP/IP; Solaris 2.6 for SPARC (and
> presumably Intel) uses STREAMS TCP/IP. I don't have a Solaris 2.7
> here, but I doubt if it is any different.
Indeed, Solaris is the best illustration of my point. Let me emphasize
that I think the gcom work is quite valuable for supporting existing
STREAMS-based code and obviously not all new development requires
screaming performance. Moving on...
Solaris (in recent versions, certainly 2.6 and later) uses
kernel sockets-based TCP/IP in the fast path and has STREAMS relegated
to secondary status. There is also a TCP/IP-over-streams implementation,
but the kernel sockets are preferred unless you need STREAMS (and the
corresponding performance hit). This is fairly transparent to the
application programmer.
Earlier Solaris development was aimed at having STREAMS be
the primary kernel interface with sockets over STREAMS, but the idea
was trashed precisely because it couldn't perform anywhere near as fast
as the socket-centric solution. (It's of historical interest to note
that kernel sockets were reintroduced at the same time that zero-copy
TCP/IP with hardware checksuming was introduced, but I doubt there's
any real correlation other than that fast networking cards were starting
to deploy and kernels had to do what they could to keep up).
So STREAMS are still there in the kernel, but the motion is away from
them.
Sequent offers a fastlan alternative to STREAMS even though they have
a pretty decent parallel STREAMS implementation; they developed the
fastlan stuff after the fact when it became apparent that STREAMS was
too slow for fast networking. Their TCP/IP is over the fastlan api.
SGI also relegates STREAMS to second-class status, at least in recent
versions (not sure about older versions).
If Sun and SGI can't make it fast and Linus, David, and Alan are against
it, then there's probably something wrong with the idea in the first
place.
If you've got existing STREAMS-based code to port, the gcom stuff is
great; it'd be foolish to rewrite unless the performance issues are
important to your code. If you're developing new code, you're almost
certainly better off forgetting that STREAMS ever existed, IMO.
TLI over sockets seems feasible and cool, and TLI is possibly the
most-used part of STREAMS in practice.
Sumner
From: David Grothe <d...@gcom.com>
Subject: Re: Stream Processing (provided by SVR4) for Linux?
Date: 1999/04/08
Message-ID: <370CE80B.CB783249@gcom.com>#1/1
X-Deja-AN: 464107125
Content-Transfer-Encoding: 7bit
References: <7eba4m$cvd$1@usenet44.supernews.com> <370B77E5.7CA01FDF@yahoo.com>
<370CBB1B.66BD327E@gcom.com> <370CD3F3.B06141FC@yahoo.com>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
Organization: Gcom, Inc
MIME-Version: 1.0
Reply-To: d...@gcom.com
NNTP-Posting-Date: Thu, 08 Apr 1999 13:31:54 EDT
Newsgroups: comp.os.linux.development.system
"G. Sumner Hayes" wrote:
> Solaris (in recent versions, certainly 2.6 and later) uses
> kernel sockets-based TCP/IP in the fast path and has STREAMS relegated
> to secondary status. There is also a TCP/IP-over-streams implementation,
> but the kernel sockets are preferred unless you need STREAMS (and the
> corresponding performance hit). This is fairly transparent to the
> application programmer.
If this is so, then perhaps you can account for the following phenomenon:
On a Solaris 2.6 SPARC, I go to /kernel/drv and do an "nm" on "tcp" and
"ip". I examine the resulting list of external symbols. I find plenty of
references to the STREAMS routines (putnext, allocb, etc). But I find _no_
references to any symbol containing the characters "sock".
I am supposed to believe that there are two parallel implementations of
TCP/IP in Solaris? If so, what is the driver name of the "socket based
TCP/IP"?
My understanding of what actually happened with Solaris is this: There used
to be a user-level library that implemented socket calls in terms of TLI
calls. This was highly ineffecient. So they moved the socket interface
into the kernel but still copy user data into STREAMS messages which get
passed to the STREAMS based TCP/IP stack. What gets bypassed is the extra
API library in user space.
Also, it is my understanding that HP-UX has gone from a BSD style of TCP/IP
to a STREAMS based version. Just the opposite of the "trend" that you and
others have cited.
I claim there is no such "trend". Implementations that started out as
STREAMS based have tended, largely, to stay that way and implementations
that started out BSD style have tended, largely, to stay that way as well.
SGI might be an example of a switch in one direction, but HP is an example
of a switch the other way.
-- Dave
From: m...@toaster.roan.co.uk (Mike Jagdis)
Subject: Re: Stream Processing (provided by SVR4) for Linux?
Date: 1999/04/09
Message-ID: <slrn7grlt6.ga6.mike@toaster.roan.co.uk>#1/1
X-Deja-AN: 464384893
Cache-Post-Path: toaster.roan.co.uk!m...@toaster.roan.co.uk
References: <7eba4m$cvd$1@usenet44.supernews.com> <370B77E5.7CA01FDF@yahoo.com>
<370CBB1B.66BD327E@gcom.com>
X-Complaints-To: usenet@starburst.uk.insnet.net
X-Trace: starburst.uk.insnet.net 923653905 5353 212.19.64.2
(9 Apr 1999 10:31:45 GMT)
Organization: Roan Technology Ltd
X-Cache: nntpcache 2.3.3 (see http://www.nntpcache.org/)
User-Agent: slrn/0.9.5.4 (UNIX)
NNTP-Posting-Date: 9 Apr 1999 10:31:45 GMT
Newsgroups: comp.os.linux.development.system
In article <370CBB1B.66BD3...@gcom.com>, David Grothe wrote:
>
>SCO OpenServer uses STREAMS TCP/IP; SCO UnixWare 2.x uses STREAMS TCP/IP;
>SCO UnixWare 7 uses STREAMS TCP/IP; Solaris 2.6 for SPARC (and presumably
>Intel) uses STREAMS TCP/IP. I don't have a Solaris 2.7 here, but I doubt
>if it is any different.
It depends what you mean by "STREAMS TCP/IP" :-). Most of the SVR3
derived systems used the Lachman stack which used a BSD-ish IP
module with STREAMS wrappers front and back. The socket interface
was dumped behind a STREAMS message interface and accessed by
simply setting up the socket call number and arguments in a
buffer instead of on the stack and passing it down through the
STREAMS head using an ioctl.
Back when Wyse were in the large(ish) systems market (around a
decade ago) they mapped all the internal socket stuff back in to
the system call table and their socket library bypassed the STREAMS
layer. SCO introduced the system call bypass in, I think, the net100
patch to 3.2v5.0.0. Even with today's network speeds the overheads
in dereferencing through STREAMS module stacks and the cache pollution
implications become significant - especially once you start talking
about fast pathing IP.
If connection set up is important a TLI/STREAMS interface tends
to be a serious millstone (some of this is due to poor TLI library
implementations - but not all).
STREAMS is *really* useful to vendors who need to ship closed
modules that drop in to a system and interoperate with other
closed modules. It's no great surprise that the Open Source
community has little use for it :-).
Mike
--
A train stops at a train station, a bus stops at a bus station.
On my desk I have a work station...
.----------------------------------------------------------------------.
| Mike Jagdis | Internet: mailto:m...@roan.co.uk |
| Roan Technology Ltd. | |
| 54A Peach Street, Wokingham | Telephone: +44 118 989 0403 |
| RG40 1XG, ENGLAND | Fax: +44 118 989 1195 |
`----------------------------------------------------------------------'
|