Path: gmdzi!unido!mcsun!uunet!usenix!std-unix
From: j...@usenix.org
Newsgroups: comp.std.unix
Subject: Standards Update, IEEE 1003.1: System services interface
Message-ID: <385@usenix.ORG>
Date: 28 Jun 90 18:34:23 GMT
Sender: std-u...@usenix.ORG
Reply-To: std-u...@uunet.uu.net
Lines: 610
Approved: jsq@usenix (Moderator, John Quarterman)
Posted: Thu Jun 28 19:34:23 1990
From: < j...@usenix.org>
An Update on UNIX*-Related Standards Activities
June, 1990
USENIX Standards Watchdog Committee
Jeffrey S. Haemer, Report Editor
IEEE 1003.1: System services interface
Paul Rabin < r...@osf.org> reports on the April 23-27 meeting in Salt
Lake City, UT:
1. Introduction
The POSIX 1003.1 working group is the oldest POSIX group, responsible
for specifying general-purpose operating system interfaces for
portable applications. This group developed the original 1988 POSIX
standard, and is now responsible for writing supplements and revisions
to that standard. This work includes
+ corrections and clarifications to the 1988 POSIX standard
+ material that was too controversial to handle before
+ new interfaces requested by other POSIX working groups
Like other working groups developing ``base standards,'' the 1003.1
working group is responsible for writing both C language and
language-independent versions of the specifications that it develops.
So far, the group has concentrated on the C language versions, but
there is increasing pressure to make progress on the language-
independent specifications.
The working group recently completed a revision of the 1988 POSIX
standard, and is currently working on a supplement to that revision.
There has been a lot of turnover in the group since the 1988 POSIX
standard was completed, but there are still a few old-timers to
provide continuity. About 15 people attended the last two meetings.
This seems to be a good size for getting work done. This is
definitely a technical crowd; check your politics at the door.
For more information about the group and how to participate, contact
the chair, Donn Terry, at d...@hpfcla.fc.hp.com or hplabs!hpfcla!donn.
__________
* UNIXTM is a Registered Trademark of UNIX System Laboratories in
the United States and other countries.
June, 1990 Standards Update IEEE 1003.1: System services interface
- 2 -
Send comments and proposals to the secretary, Keith Stuck, at
k...@sp7040.uucp. I've made this report a bit more detailed than
usual in order to give the technical details wider exposure. New
proposals, and comments on any of the current active proposals or
issues are welcome.
2. 1003.1a Status
1003.1a is the recently completed revision to the 1988 POSIX standard.
No new interfaces or features were introduced, but the text was
revised in several ways. The main reason for the revision was to
prepare the text for balloting as an ISO standard, so the document had
to be made to look like an ISO standard. This meant adding ISO
boiler-plate, changing external document references to pretend that
only standards exist, changing internal cross-references so that
chapters are renamed sections, and sections are renamed clauses and
subclauses, changing ``will'' to ``shall,'' etc., ad nauseam. While
the working group was having fun with all that, they took the
opportunity to do some cleaning up. They corrected errors, clarified
unclear language, and changed the function synopses to use ANSI C
prototypes. The group did make one normative change, which was to
specify reserved namespaces. This will allow implementations and
revisions to the standard to define extensions without breaking
existing, conforming applications. It's messier than you might think.
After four recirculation ballots, IEEE balloting was completed in
April. Now it has to get through the ISO balloting process. See the
recent snitch report on 1003.5 for a description of how IEEE and ISO
balloting is synchronized, and what all of the acronyms mean.
ISO has been balloting 1003.1a as ISO/IEC DIS 9945-1. After the first
ISO ballot, JTC1 approved 9945-1 for promotion to full IS status.
This approval was overruled by the ISO Central Secretariat on the
grounds that the document format was still not satisfactory (still
haven't caught all of those ``will''s). Rather than publish the
current document and then immediately revise, ballot, and publish it
again, it was decided to create a new DIS and to start a second round
of ISO balloting. This will cause a delay in the publication of the
international POSIX standard (and hence also the IEEE POSIX.1
standard). The U.S. Technical Advisory Group (TAG) is responsible for
generating the U.S. ballot. Assuming that no normative changes are
introduced by the ISO balloting process, the resulting document will
be published by IEEE as IEEE Std 1003.1-1990.
June, 1990 Standards Update IEEE 1003.1: System services interface
- 3 -
3. 1003.1b Status
Since 1003.1a is now out of IEEE's hands, the working group spent the
Utah meeting working on 1003.1b, the first supplement to 1003.1a.
This will include some corrections and clarifications that didn't make
it into 1003.1a, but will mainly consist of new interfaces and
features.
1003.1b has been in the works for several meetings, so the draft
already contains a lot of material. The first day was devoted to
revision of the draft, the rest of the week to considering new
proposals. The previously announced schedule for 1003.1b specified
the Utah meeting as the cutoff date for new proposals. Unfortunately,
some expected proposals were not received, and some that were received
were not ready for incorporation, so the cutoff was deferred until the
next meeting, in Danvers, Massachusetts.
Draft 2 of 1003.1b was distributed just before the meeting in Utah.
Draft 3 should be available before the Danvers meeting. 1003.1b is
expected to be approved sometime in early 1991, and to be published by
IEEE as a separate supplement to the IEEE Std 1003.1-1990.
3.1 New Features in the Current Draft of 1003.1b
Draft 2 of P1003.1b includes a new data interchange format, and new
interface specifications for symbolic links, environment list access,
and file-tree walking. These had been proposed and generally accepted
at previous meetings. Many new issues were raised and discussed.
3.1.1 Symbolic_Links P1003.1b adds BSD symbolic links to the 1988
POSIX standard as a new required feature. New interfaces for
symlink(), readlink(), and lstat() are specified, and the definition
of pathname resolution is amended to include the handling of symbolic
links. Implementations may optionally enforce a limit on the number
of symbolic links that can be tolerated during the resolution of a
single pathname, for instance to detect loops. The new symbol
{_POSIX_SYMLOOP} is defined to be the minimum value of such a limit.
A new error, [ELOOP], is returned if such a limit is exceeded.
Symbolic links that are encountered in pathname prefixes are always
resolved. Symbolic links named by the final component of a pathname
will be resolved or not, depending on the particular interface. By
default, such symbolic links will be resolved, unless specified
otherwise. The interfaces that will not resolve symbolic links named
by pathname arguments are:
+ readlink()
If the pathname argument names a symbolic link, the contents of
the link will be returned.
+ lstat()
If the pathname argument names a symbolic link, a stat structure
will be returned for the link itself.
June, 1990 Standards Update IEEE 1003.1: System services interface
- 4 -
+ unlink()
If the pathname argument names a symbolic link, the link itself
will be removed.
+ rmdir()
If the pathname argument names a symbolic link, the link will not
be followed and the call will fail.
+ open()
Symbolic links are followed, unless both O_CREAT and O_EXCL are
set. If both O_CREAT and O_EXCL are set, and the pathname
argument names an existing symbolic link, the link will not be
followed and the call will fail.
+ link()
If the new pathname names a symbolic link, the link will not be
followed and the call will fail. If the old pathname names a
symbolic link, the link will be followed. This is the BSD
behavior. SVR4.0 does not follow the link in this case, thus
supporting hard links to symbolic links. The working group felt
that the SVR4 behavior unnecessarily restricts implementations
(for instance, those that do not implement symbolic links with
inodes), and has much more complex semantics.
+ rename()
If the old pathname names a symbolic link, the link will not be
followed. Instead, the symbolic link itself will be renamed. If
the new pathname names a symbolic link, it will not be followed.
Instead, the symbolic link will be removed, and a new hard link
will be created naming the file that was previously named by the
old pathname.
The 1988 POSIX standard specifies that if the new pathname names
an existing file, rename() will fail if the new and old pathnames
do not either both name directories or both name non-directory
files. This rule needs to be expanded to include the case of the
new pathname naming a symbolic link. Should the rename() call
fail depending on whether or not the symbolic link named by the
new pathname itself names a directory or a non-directory file?
This will be resolved at the next meeting.
Symbolic links are not required to have any attributes other than
their file type and their contents. This is intended to provide the
simplest semantics and to allow the greatest latitude for
implementations.
3.1.2 Other_BSD_Interfaces P1003.1b also includes specifications for
the following interfaces:
June, 1990 Standards Update IEEE 1003.1: System services interface
- 5 -
+ fchmod()
+ fchown()
+ fsync()
+ ftruncate()
3.1.3 Environment_List The ANSI-C standard defines the getenv()
function to retrieve the value corresponding to a given name in a
program's environment list, but does not specify the implementation or
initialization of that list. The 1988 POSIX standard specified the
traditional list implementation using the external variable environ,
and specified the initialization of the list by the exec functions.
In an attempt to extend the set of high-level interfaces to the
environment list, and to pave the way for the possible eventual
removal of environ, the working group has included putenv() and
clearenv() interfaces in 1003.1b. Three problems have been identified
with these high-level interfaces:
+ They cause static data to be shared between the application and
the implementation. Neither the application nor the
implementation can easily manage the storage for environment
"name=value" strings.
+ They are not robust. The interactions between the high-level
interfaces and access via environ is not specified.
+ They can't be easily extended to handle multiple lists. There is
no way to copy a list, or to build a new list for execle() or
execve().
The putenv() and clearenv() interfaces may be removed from 1003.1b at
the next meeting if a revised proposal does not appear.
3.1.4 File_Tree_Walk The 1003.1 working group promised the 1003.2
group (Shell and Utilities) that a mechanism would be provided for
walking an directory tree of unbounded depth using any given (non-
zero) number of free file descriptors. The Berkeley folks have
implemented a set of high-level interfaces defined by David Korn of
Bell Labs, and proposed them for inclusion in 1003.1b. These
interfaces support every option and search order required by the
1003.2 commands. The 1003.1 group wants a simpler interface suitable
for typical application programs, so Keith Bostic will put the
proposal on a ``weight-reducing diet'' and resubmit it for the next
draft.
The high-level file-tree walk interfaces can be implemented using only
the existing 1003.1 interfaces. Since 1003.1 does not define a
portable way to save and restore file position for a directory and
cannot hold a file descriptor open for each directory level, the
June, 1990 Standards Update IEEE 1003.1: System services interface
- 6 -
implementation must read and save all directory entries each time a
new directory is visited. This requires only a single file descriptor
(or whatever resource is used by by opendir()). If the underlying
system does provide a way to save and restore file position for
directories, the file-tree walk implementation can use it to reduce
memory consumption.
There was a discussion about whether it is possible (and preferable)
to improve the low-level directory interfaces instead of adding new,
high-level interfaces. Do the high-level interfaces really add new
functionality for portable applications? Do they belong with the
low-level operating system interfaces specified in 1003.1?
Walking an unbounded file tree requires an unbounded number of
directory file positions to be supported using a bounded number of
file descriptors. Either seekdir() and telldir() are needed, or an
unbounded number of opendir()'s must use a bounded number of file
descriptors. The working group has already rejected seekdir() and
telldir() because they cannot easily be supported on implementations
that use a non-linear directory format. A prohibition of simple
implementations of opendir() using file descriptors is also likely to
be rejected.
The underlying problem is that the orderedness of directory entries
was implicit in the traditional implementations, but was not made
fully explicit in 1003.1, partly out of a desire to permit alternate
implementations (for instance, b-trees). As a result, orderedness
must now be imposed by the application. On a non-linear directory
implementation, if positioning is not supported, even opendir() must
read in the whole directory.
3.1.5 Data-Interchange_Format The 1988 POSIX standard specified two
data-interchange formats based on existing utilities. These define
the data-stream encoding of a sequence of files, together with their
pathnames and other attributes. The first format is based on tar and
encodes files as a stream of 512-byte blocks. The second format is
based on cpio and encodes files as an unblocked byte stream.
The ISO POSIX group (JTC1/SC22/WG15) pointed out that both of these
formats are incompatible with accepted international and U.S.
standards. After some arm twisting, the 1003.1 working group agreed
to devise a new data interchange format based on IS 1001: 1986, which
is more or less equivalent to ANS X3.27-1987, the familiar ANSI
labeled tape format.
The current draft of 1003.1b includes the framework for the new
specification, but a lot more work is needed. Previous meetings
discussed alternate proposals. The topic has been strangely quiet
lately, considering the confusion that may be expected when it goes to
ballot. It wasn't discussed at the Utah meeting at all.
June, 1990 Standards Update IEEE 1003.1: System services interface
- 7 -
3.2 {_POSIX_PATH_MAX}: A Clarification
The 1988 POSIX standard included two conflicting statements regarding
{PATH_MAX} and {_POSIX_PATH_MAX}: one said that the null was included
in the count; the other said that the null was excluded. Traditional
implementations have included the trailing null; some new
implementations have excluded the null.
One alternative or the other had to be endorsed. The working group
decided that {_POSIX_PATH_MAX} should not include the trailing null,
since specifying this will not break currently conforming
applications.
3.3 Headers and Name-Space Control
Since 1003.1b is adding many new identifiers to the standard, there
was discussion about whether new identifiers should be declared in new
headers, or whether existing headers could be used, with new feature-
test-macros to control visibility of the additional identifiers. It
was agreed that although both headers and feature-test macros control
identifier visibility, their functions are complementary. Headers are
appropriately used to divide name-spaces horizontally, by
functionality. Feature-test macros are appropriately used to divide
name-spaces vertically, by specification level.
With this understanding, the group decided that new identifiers will
be declared in the ``right place.'' A new header will be created only
if no existing header is functionally appropriate.
A new feature-test macro will be specified by 1003.1b and subsequent
revisions: _POSIX_1_SOURCE. This macro takes ordinal values, starting
with 2 for 1003.1b, and will be incremented by 1 for every subsequent
revision. If the value is 1, the effect will be the same as if
_POSIX_SOURCE were defined.
There are two changes here. The new name was used to indicate that
the macro only controls the visibility of identifiers defined in
POSIX.1. The usage was changed to allow the value to indicate the
particular revision or supplement to the standard, rather than having
to create a new macro each time. This should simplify the
construction and maintenance of header files.
3.4 Requests
Two requests were made by vendors trying to support POSIX behavior on
non-UNIX file systems:
+ that {_POSIX_LINK_MAX} be reduced from 6 to 2
+ that {_POSIX_PATH_MAX} be reduced from 255 to 252
June, 1990 Standards Update IEEE 1003.1: System services interface
- 8 -
Both requests were rejected. Either of these changes could have made
existing conforming applications non-conforming. Even where the risk
of breaking applications seemed small, the working group was reluctant
to set a precedent without a pretty good rationale to protect them
against similar requests in the future.
3.5 New Proposals
Five proposals for new interfaces were submitted for inclusion in
1003.1b, many of which provoked lively discussion. Some were
accepted, some were rejected, and others were deferred to allow a
revised proposal to be submitted or to allow more time for
consideration.
3.5.1 seteuid(),_setegid() Bob Lenk and Mike Karels proposed a set
of changes to the way the effective user and group id's are handled,
in order to provide better support for setuid/setgid programs.
+ Require that all implementations support the functionality of the
saved user ID and saved group ID. These process attributes are
set by the exec functions and by privileged calls to setuid() and
setgid().
+ Add seteuid() and setegid() as new functions that change only the
effective user ID and effective group ID, respectively. A change
is allowed if the proposed new user/group ID is the same as
either the real user/group ID or the saved user/group ID.
+ Redefine the {_POSIX_SAVED_IDS} option to apply only to non-
privileged calls to setuid() and setgid().
This proposal has general support in the working group, and will be
included in the next draft of 1003.1b.
The discussion of this proposal led to a general lament about how
unclear the group model is in the 1988 POSIX standard, perhaps the
result of a hasty marriage between the System V and BSD models. At
the next meeting, the working group intends to add new text to
P1003.1b to clarify the relation between the effective group ID and
the supplementary group list.
3.5.2 Magnetic_Tape_Support The 1003.10 working group
(Supercomputing Profiles) proposed new interfaces to support basic
controller functions for magnetic tape drives, based on the ioctl()
commands supported in 4.3BSD. Although support for these interfaces
would be optional in 1003.1b, the working group decided that the
functions should be further specified according to whether they are:
+ required for all types of tape drives;
June, 1990 Standards Update IEEE 1003.1: System services interface
- 9 -
+ required only for 9-track tape drives;
+ required only for cartridge tape drives; or
+ optional on all types of tape drives.
The proposal needed further revision, but was generally supported by
the working group.
The submitted proposal also included interfaces for mounting labeled
tape volumes. These were considered to be inappropriate for inclusion
at this time and will be deferred until a later revision of the
standard.
3.5.3 Checkpoint/Restart The 1003.10 working group also proposed new
(optional) interfaces for checkpointing and restarting processes.
This proposal is based on two existing implementations. The
interfaces are intended to protect very-long-running applications from
both scheduled shutdowns and unexpected failures of the system.
The 1003.1 working group was not happy to have to deal with this and
had lots of questions. Were programming interfaces for portable
applications really needed, or was a command interface sufficient?
How much state needed to be saved in the checkpoint? What if the
processes depended on additional state information that was not in the
checkpoint, such as file data or the states of other communicating
processes or devices? In this case, the restart would only be
successful if this additional state had not changed since the
checkpoint. How could such changes be detected or prevented? What is
the set of interfaces that an application can use and be sure that it
can be checkpointed and restarted successfully, without dependencies
on additional state? Should applications have a mechanism for
checkpointing themselves, or for blocking an external request that
they be checkpointed?
Because a checkpoint/restart mechanism will have a major impact on
implementations, and the requirements are not yet clear, the working
group was unwilling to endorse the current proposal. A task force
made up of representatives of the 1003.1 and 1003.10 working groups
will be created to clarify the requirements and revise the proposal.
This proposal is going to need a lot more discussion, so checkpoint
restart interfaces will almost certainly not be included in P1003.1b,
but they may be adopted in a subsequent revision.
3.5.4 Messaging The UniForum proposal for new messaging interfaces
has been before the 1003.1 working group for a couple of meetings now.
The proposed interfaces are intended as replacements for the message
catalog interfaces specified in XPG3, and differ from those in several
ways (since the discussion was fairly contentious, I'll try to be
objective):
June, 1990 Standards Update IEEE 1003.1: System services interface
- 10 -
+ The XPG3 interfaces identify a message by the triple: , where catalog name is a file name, and set
ID and msg ID are integers. The UniForum interfaces identify a
message by the triple: ,
where locale name, domain name, and message name are all strings.
The locale for messages is specified by the new LC_MESSAGES
category of the locale. Advocates of the UniForum proposal claim
that string identifiers are easier to use and are more robust
against errors during application development and maintenance.
+ In the XPG3 scheme, each message catalog is an ordinary file.
Message catalogs must be specified by filename and explicitly
opened before messages can be retrieved. The NLSPATH environment
variable provides a search path for message catalogs that can be
parameterized by (among other things) the language, territory,
and codeset fields of the LANG environment variable. In the
UniForum scheme, groups of messages are specified by an abstract
``domain.'' A default domain can be set to control message
accesses, or the domain can be explicitly specified for an
individual message access. Advocates of the UniForum proposal
claim that the binding of message catalogs to files unnecessarily
restricts implementations and imposes a more complex interface on
application developers.
+ The XPG3 interface includes an additional string argument that is
returned in case no message specified by can be
retrieved from the message catalog. In the UniForum proposal,
the message name itself is returned if the message cannot be
found. Advocates of the UniForum proposal point out that the
message name string makes a separate, ``default'' message string
unnecessary.
In addition, the UniForum proposal includes a specification of the
message source file format that differs from the format specified in
XPG3.
+ In the XPG3 format, message strings are implicitly delimited: at
the beginning by the preceding message ID followed by a single
space or tab character, and at the end by an unescaped newline.
In the UniForum format, all strings, including domain names,
message ID's, and message strings, are explicitly delimited by
double quotes ("). Adjacent strings separated by white-space
characters are concatenated. Advocates of the UniForum proposal
claim that the new format provides better support for multi-line
strings and for leading and trailing white-space characters in
strings.
+ In the XPG3 format, the message ID and its corresponding message
string are implicitly determined by their position on a source
line. In the UniForum format, explicit directives are provided
for message ID's and message strings. Advocates of the UniForum
June, 1990 Standards Update IEEE 1003.1: System services interface
- 11 -
proposal claim that the new format is extensible. New attributes
may be added to message entries, such as screen coordinates or
font size.
+ The XPG3 format includes directives for deleting individual
messages and sets of messages, and the associated gencat utility
takes no switches. In the UniForum proposal, all deletion
semantics are provided by switches on the associated gentext
utility.
There was much discussion of the interfaces; less about the source
file format. The most divisive issue was whether string message ID's
are preferable to numeric message ID's. Among those who felt that the
new interfaces are better, there was disagreement about whether the
advantages outweighed the cost of conversion for applications and
implementations based on the XPG3 interfaces. The rationale
accompanying the UniForum proposal described several ways to convert
applications from the XPG3 interfaces to the proposed new interfaces.
The working group asked X/Open to submit the XPG3 messaging interfaces
as an alternate proposal, since they represent existing practice, and
X/Open has agreed to do so. X/Open has said that they will follow
POSIX if POSIX endorses a different interface. The decision regarding
which, if any, messaging proposal to include in 1003.1b will be made
at the POSIX meeting in Danvers.
It's hard to predict the fate of this proposal. The UniForum proposal
represents the consensus of one of the leading internationalization
working groups and is reported to have some support within X/Open. On
the other hand, the POSIX working groups are obliged to respect
existing practice. Watch this space.
3.5.5 /dev/stdin,_/dev/fd/n,_etc. There was an unofficial proposal
from members of the 1003.2 working group that open() be extended to
recognize the special strings "/dev/stdin", "/dev/stdout",
"/dev/stderr", and "/dev/fd/n", and return a new file descriptor
dup()'ed from STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO, or file
descriptor n, respectively. This proposal was intended to allow
simplified command line parsing, by eliminating special casing for
``-'' and ``--'' arguments. The proposal was rejected after a short
exploration of the possible semantics of these pathnames when used
with link(), rename(), etc..
4. Conclusion
As you can see, there's a lot going on. Even though most of the
attention has shifted to other working groups, the 1003.1 group is
busy revising and extending the 1988 standard. The group is small
now, by POSIX standards, but their work is as important as ever.
June, 1990 Standards Update IEEE 1003.1: System services interface
Volume-Number: Volume 20, Number 56
|