Path: gmdzi!unido!mcsun!uunet!stanford.edu!ames!sparkyfs.erg.sri.com!zwicky
From: zwi...@erg.sri.com (Elizabeth Zwicky)
Newsgroups: comp.org.usenix,comp.unix.admin,comp.unix.large
Subject: LISA V transcript 1: What is System Administration Panel
Message-ID: <1991Oct18.193543.4134@erg.sri.com>
Date: 18 Oct 91 19:35:43 GMT
Sender: n...@erg.sri.com
Organization: SRI International, Menlo Park, CA
Lines: 334
Originator: zwi...@sparkyfs.erg.sri.com
The transcript from Rob Kolstad's panel on "What is System
Administration" follows. It was transcribed by Elizabeth D. Zwicky,
zwi...@erg.sri.com, and the file is named "sysadmin-panel"
This is a transcript of a session at the LISA V conference in San
Diego. It is not a verbatim transcript of what happened; these
transcripts are amateur efforts, done by various people with a wide
range of attempts at accuracy. No speaker is representing the company
or companies for which he or she works, unless otherwise noted.
Corrections are welcome, and should be mailed to zwi...@erg.sri.com.
In particular, please let me know if you recognize yourself as one of
the many unidentified speakers. These transcripts, and other items
relevant to the conference (copies of overheads, papers, and programs)
are available for anonymous ftp on ftp.erg.sri.com in ~ftp/pub/lisaV.
That archive is growing as I finish off transcripts and transfer data.
It will be shadowed at other sites, to be announced later.
Panel members:
John Sellens
Jody Kravitz
Rob Kolstad
Tina Darmohray
Steve Simmons
Rob: Why do computers need so much administration?
(Missing stuff)
Peg Schafer: You need to tailor the computers to the users, it's a
craft sort of thing.
Steve Simmons: Configurability is expensive - cars don't come user
configurable.
Peg: It should be like a Harley; you can put it together a lot of ways but all
the parts fit together.
John Sellens: Getting back to the question, a single machine
doesn't need very much administration. It's when you start having lots
of machines that you have problems.
Tina Darmohray: Problems of scale.
Jeff ? from the audience: The vendors don't have to adminster the
machines, so they don't worry about it. We do have to administer the
machines, and we have multiple vendors to worry about.
?? (Rob somebody?) from audience: The tools exist, but they're from
individual vendors, they don't work on heterogeneous systems.
Rob K.: How about Tivoli?
Rob somebody: Too expensive.
Rob K.: So what you want is something right now that does everything,
works on all machines, and is free?
Jody Kravitz: What scares me is that there are theoretically 2
porting bases, but there are 2,000 different versions of UNIX. I heard
this morning that there are 4 different versions of dump for System V
Release 4.
Steve Simmons: Look, I make my living off the shortcomings of the
vendors.
Rob K.: Couldn't you say "The vendors provide me with an industry"
instead?
Steve Simmons: Individual vendor's tools are OK for homogeneous
systems, and I encourage my clients to use them. But they make policy
assumptions, and different vendors make conflicting assumptions.
Rob: I have a second question: How does management view
administrators? Peg, your management respects you, right?
Peg: Well, at one time. At Bellcore. For a system administrator to get
respect from users, they just have to go on vacation, but managers
think you take the computer out of a box and it works.
John Sellens: Different levels get confused. Someone who's called a
System Administrator might be anything from a graduate student who
runs a single machine because his advisor told him he had to, to the
guy who shapes the computing policy for the entire university.
Management doesn't necessarily see the difference.
Jody Kravitz: When you do system administration well, it's
very nearly invisible.
Steve Simmons: You have to educate both the managers and the users.
Use a newsletter to make invisible tasks visible.
Tina: The problem we have is that the managers expect system
administrators to grow up to be real programmmers.
Brian Keves: It only recently got to the point where you could ask a
headhunter for a system administrator and get back something more than
a blank look and a kernel hacker.
Rob: I wish Michael Sunday were here. It used to be that a junior C
programmer, straight out of college got $45,000, and a system
administrator was expected to know 4 languages, 16 systems, and
everything about every network, and got $30,000 "because we like you".
[Peg: I don't work that cheap. Rob: She's not cheap, but she's easy to use.]
Now I've seen ads which say that the system administrator is expected
to be the most technically competent person on site, and are offering
$65,000, the same money that lead engineers get.
Bob Arnold: It's a PR problem. Education is the key. We have weekly
meetings, for communication. When they started we got beat up really
bad every week for about 6 months, but now they're very dull. We just
keep them up so it won't get bad. We also do a newsletter, and we
fixed internal problems and speeded up our response time.
Rob: If you do a good enough job, you have to go on vacation for a
very long time. Things degrade very slowly, and then there's a
catastrophe. People go away, and everything seems fine, and management
figures that they obviously didn't need an administrator after all,
and then something goes wrong and boom! all the entropy comes back at
once.
Steve Simmons: The increased respect isn't just at companies with
large sites. I have more and more clients with 4 computers that are
hiring me to work 5 hours a month, but they know they need me for
those 5 hours.
Carl Shipley: Part of the problem is that administrators do all the
high-level stuff and all the low-level stuff. Some summer hire can
come in and order you around, even though you may be the most
technically knowledgeable person in the company. It's a real morale
problem; people keep getting beaten up, and all I can tell them is
"Great job; you got beaten up really *well* today."
John Sellens: Specialization helps. Get low level people and
let them get beaten up.
Carl: Even if you specialize, someone has to do the junk jobs.
Rob: Why are there junk jobs? Can't you automate them away? Redo the
backup system so that you don't have to deal with it all the time, for
instance.
Carl: You can automate the backups, but there are still restores, and
there are still other junk jobs that don't automate. And management
hates specialization. They want everybody to do everything, so that
things don't fall apart when people go on vacation.
Steve Simmons: A lot of people think of operators as well-trained
monkeys. We've found that it works much better to give them real tasks
that they can do to help automate the stuff that they do. Train them
up. It takes a lot of education, a lot of hands-on work, but it pays
off.
Carl: I've got to get some operators, I see.
Rob: You have to let the operators move up, too.
Steve: Training operators gets you the very best system
administrators.
Ken ?? U of Washington: I talked to a system administrator for a major
vendor, and he had no connection to the developers. Why aren't vendors
their own best customers? The developers had some neat ideas, but
they kept putting out products that had nasty problems that could have
been found in 5 minutes if the system administrators had ever tested
the software.
Jody Kravitz: One place I worked I went from being a
developer to working in customer service, and it was real obvious that
the developers had never been to a customer site, never talked to the
customers, never seen what a salesman promised.
Rob: Anybody ever have trouble with NIH? The vendor wasn't willing to
deal with something because they didn't write it?
[Audience laughs bitterly]
Jody Kravitz: The vendors scramble to keep up with
each other, but they neglect the public domain. An awful lot of good
ideas come out in the public domain.
Steve: The scale at vendor sites is not the same as the scale at most
customer sites. Vendors have these huge networks, and most customers
have 5 computers. Even though the administrators on the vendor
networks may have good tools, they're not the same tools that the
customer needs. Furthermore, within the vendors, many of the
programmers administer their own machines. They've got a base of
people who don't need much administration.
Peg: The vendors all have their own systems on site, anyway; all the
tools would have to be rewritten for heterogeneous sites.
Rob: Jeff Polk and I have this paper on why it takes software so long
to get out of a vendor. You don't hear about what the vendors are
doing until they get done, and that takes 2 to 5 years. What you can
see is that vendors are sending more people to conferences, they're
running BOFs, they're paying attention to what people are saying.
They've got the picture, but it will be 2 to 5 years before you see
the results of that.
Ken?: What I was trying to say was that they ship good tools, but they
have stupid bugs in them because they haven't been tested by an
administrator on a real network.
Steve Lammert: Developers have degrees, masters or doctorates in
computer science. Most administrators don't have advanced degrees in
computer science; a lot of them have no degrees at all. [A chorus of
boos and hisses from the audience, evincing general disgreement with
this proposition] The developers think that having the degree means
that they know how to do it.
Rob: I can tell you about people with degrees; what about the
professor at University of Colorado who is mounting filesystems
read-write from Illinois. He's a tenured faculty member, can't be
fixed.
Bjorn Satdeva: There's no system administration expertise being used
in writing tools. I had an experience where I was writing an article
about system administration tools, and I called up this manager who
was saying how marvelous their product was, on and on, and then I
asked him how much system administration expertise the developers had.
There was a long pause, and then he said if I printed that he'd sue.
Jeff ??: What we need is tools to move duties down the line. Automate
it and move it to lower-level people.
Rob: Like fly-by-wire in a 747. Your secretary can do it if the
computer knows how.
Jeff: If we automate this far, do we put ourselves out of a job? If
our goal is to push things down, what are we going to do afterwards?
John Furlani: Can we get to the point where you can put 4 systems
together without a system administrator, or add the 101st computer to
your network without a system administrator? We need to do something
like the Japanese, who got reliability by designing for manufacturing;
we need to design UNIX for administration.
Rob: I think Steve Simmons is right; it's configurability that's the
problem. I was at a life insurance company that was converting from an
IBM mainframes to workstations. The system administrator said that he
figured out how to make the users happy: he gave them a little program
that would let them change the background colour of any window on
their screen, and "Now that they have complete control over their
environment, they're happy."
John: There are fundamental flaws in the way the operating system
works; it's not just a matter of complexity and configurability.
Steve Simmons: Yes, we need design for administrability, but its very
low on the vendors' priority lists, and it gets bumped.
Peg: The vendors aim at companies with 4 or 5 systems, and spend a lot
of time on them. They don't care about the people with big sites.
Rob: Come on; any company that can sell a thousand computers instead
of 5 will choose to sell a thousand.
Tina: I'm not sure I like the idea of vendors doing auto-configuring.
Remember the 386i?
??: The problem with vendors who "help" you by providing high-level
tools is that doing anything non-standard is very difficult. Sure, the
low-level stuff is available underneath, but it's not documented
anymore, and you have to go read their programs to see what's going
on. There are almost always new hidden files in the process, too.
Steve Simmons: And as soon as you make one change by hand, all the
tools break.
Greg Rose: In 1977 I managed a system with 1,000 users. This is
comparable in size to a major university site. In fact, it was a major
university site. It should be as easy to do now as it was then, but it
seems to be getting worse instead. Making machines faster and smaller
shouldn't inherently make them harder to administer. But the vendors
keep adding bells and whistles without regard to the administration
problem.
Rob: Look, the vendors are not controlling the world. The users demand
bells and whistles, and the vendors have to provide them as soon as
possible.
Greg: This is another PR problem. We are the users, and we've got to
beat on the vendors and let them know what we want.
Jody Kravitz: There's a scale problem, too. 10 years ago, each
user was happy with 500K, but now they want 10 megabytes. Users
expectations have scaled up.
Rob: 10 years ago, networks weren't a problem.
Steve Simmons: I have one account with 35 Suns, about half of them on
secretaries' desks. The secretaries are the happiest users they have.
The machines are all set up to start up the window system
automatically when they log in, and log them out when the window
system exits. Furthermore, they are the easiest users to deal with.
They know exactly what they want, and they'll take "You can't do that"
as an answer.
Steve ??: I think salaries for system administrators are going up
because they scale to match sales. As more machines are sold, and as
they get cheaper, the users get less expert.
Tina: Our network has doubled, and the expertise has dropped
considerably. Originally, all of users were designing chips and
writing operating systems, and now we have all sorts of people with
computers.
Rob: Expertise per user. I always wanted to invent something to
measure. The unit could be the Ritchie; Tina here is 900
milliRitchies, and the secretary is 1 microRitchie.
Boyd Roberts: All the tools are either so complex that they're
difficult to use, or they're so specific that they're too fragile.
Steve Simmons: For a single machine in your house, SparcStations and
the new DEC stations autoconfigure perfectly well. You take it out of
the box and answer a few questions and it works. This is encouraging.
Steve from the audience: Managers like time management, and system
administration just doesn`t work like that [Rob: Today, we're going to
crash at 4:30 precisely.] How do you come up with the sort of
predictions that the managers need, when what you do varies so often?
Rob: You keep logs, you make graphs, and you are really, really nice
to your manager - take him out to dinner.
Audience: Color graphs. They've got to look good.
Pat Wilson: I've now got undergraduates in dormitory rooms who want to
run UNIX, and want to be on my network, and at Dartmouth we have a
policy of a network port by every pillow. Right now the port is
Appletalk, and we have 8,000 Macintoshes, but I've got to let the
undergraduates onto the network with UNIX machines, and there isn't
anything that configures *that* well. Scares me to death.
|