KornShell has perhaps been the most powerful shell scripting language to be used in Unix/AIX environments over the years. Its 1988 version (ksh88) already had more features than Bourne and C shells, the other two most commonly used shells, put together. In 1993, major enhancements were made to the 1988 version of the KornShell that made it even more powerful. In this article the author discusses some of those enhancements. KornShell has been used in all releases of AIX and, therefore, the enhancements discussed here would be of interest to both AIX programmers and developers. This article is not intended to cover a detailed discussion of KornShell in general and, therefore, no attempt will be made to include features that were available in prior releases of the shell. This new version, ksh93, is a superset of the 1988 version.

Introduction

ksh93 provides an alternative to two other popular scripting languages, namely, Tcl and Perl. As a programming language, it compares favorably in speed and functionality with both of these languages. Moreover, like Tcl, it is extensible and embeddable with a C language application programming interface.

In AIX 5L as in other releases of AIX, the default shell is KornShell. However, the Kornshell used in AIX 5L is an enhanced version of ksh88 that is POSIX-compliant. AIX 5L also includes a copy of the 1993 version. It is supplied as /usr/bin/ksh93. This version is POSIX compliant as well. With the exception of POSIX-specific items, ksh93 is backward compatible with ksh88.

It is important to note that in AIX 5L the path for the shell is /usr/bin/ksh. This is especially important for the root user. In previous versions of AIX, the path was /bin/ksh and it relied on the existence of the link between /bin and /usr/bin. If this link were accidentally destroyed, the system would become unbootable because there would be no shell available for root and many of the system commands.

Back to top

Features of ksh93

Enhancements discussed in this section include arrays, below, as well as the following topics:

• Compound variables
• Compound assignments
• Name reference variables
• Internationalization support

Arrays

ksh93 supports both indexed arrays (available in ksh88) and associative arrays. Indexed arrays are arrays whose subscripts are integer constants or integer expressions. In ksh93, the range for the indexed array subscripts has been increased. It is 0 to 4095 compared with 0 to 1023 in ksh88.

Associative arrays are arrays whose subscripts are strings. The -A attribute of typeset specifies that a variable is an associative array. The syntax of associative arrays includes the syntax for access and for returning a list of the indexes.

Access syntax

	${variable[index]}
		- All of the characters between the [ ]s are included in the index, 
		  including space. 

Example

	$typeset -A sales
	$sales[magazine]=50
	$print ${sales[magazine]}
	50
	$print ${sales[" magazine "]}

	$sales[ book ]=100
	$print ${sales[book]}

	$print ${sales[ book ]}
	100

Returning a list of the indexes

${!varname[@]}	Expands to the list of subscripts of the array varname that
			are set.
${!varname[*]}	For a variable that is not an array, the value is 0 if the variable
			is set. Otherwise it is null.

			- * and @ differ only when accompanied by double quotes. When used 
			in double quotes, ksh93 expands the ${!varname[*]} format 
			to one argument, and the ${!varname[@]} format to separate
			arguments for each array subscript.

Example

	typeset -A color
	color=([apple]=yellow  [banana]=green  [grape]=purple)

	for i in ${!color[*]}  
	do 
		print $i ${color[$i]}
	done

	for i in ${!color[@]} 
	do  
		print $i ${color[$i]}
	done
	
	for i in "${!color[*]}"
	do 
		print $i ${color[$i]}
	done

	for i in "${!color[@]}"
	do 
		print $i ${color[$i]}
	done

	------------------------ Output ---------------------------
	grape  purple
	apple  yellow
	banana  green

	grape  purple
	apple  yellow
	banana  green

	grape apple banana

	grape  purple
	apple  yellow
	banana  green

Compound variables

In ksh93 a variable is defined by a name=value pair where the name space for the variable is a hierarchy of identifiers with . (dot) as the delimiter. A variable with a . (dot) in the name is called a compound variable. To create a variable with a period in its name, a variable whose name consists of everything up to the period must already exist. For example, new.var is legal as long as new already exists. Variables whose names contain periods cannot be exported.

The expanded name space allows an aggregate definition for a variable. Variable names that begin with .sh are reserved for use by ksh93. The following is a definition of a compound variable named grid.

	$grid=(
		# maximum size of grid
		integer maximum=10
		# maximum width 
		integer width=20
		# current index within the grid
		integer index=30
		typeset entries
		)

Example

	$print ${grid.maximum}
	10
	$print ${grid.width}
	20
	$print ${grid.index}
	30

This example defines the compound variable grid, with the aggregate members maximum, width, index, and entries. A reference of ${grid.index} provides the value associated with the index aggregate. Using the eval command we can create additional variables with the same aggregates. We can, for example, define variables row and col to have the same definition as grid:

	$eval row="$grid"
	$eval col="$grid"

Example

	$print ${row.maximum}
	10
	$print ${row.width}
	20
	$print ${row.index}
	30

	$print ${col.maximum}
	10
	$print ${col.width}
	20
	$print ${col.index}
	30

Compound assignments

Compound assignments can be used to assign values to an indexed or associative array, and to assign values to related compound variables. It is a mechanism for assigning values to one or more variables. It can take the following forms:

varname=(value ...)  		To assign values to indexed array named 
					varname.
varname=([expression]=value ...)  To assign values to associative array named
					varname.
varname=(assignment ...) 	To assign values to a set of variables whose 
					names are of the form varname.name.

Spaces and tabs are not allowed before the =
Spaces and tabs are allowed after ( and before )
The first value is associated with index 0, 
   the second value with index 1, etc.

Example 1

	$files=(*)

	for i in ${!files[@]}
	do
		print ${files[$i]}
	done

	------------ Output ------------

	List of files in the current directory

Example 2

	$age=( [John]=18 [Pat]=21 [Phillip]=23 )
	$print ${!age[*]}
	John Pat  Phillip

Example 3

	$point=( a=3.5 b=4.2 )
	$print ${point.a}
	3.5
	$print ${point.b}
	4.2

Creating indexed arrays using compound assignment

	variable=(value... )

Example

	$ab= ( one two )
	$print ${ab[0]} ${ab[1]}
	one two
	$IFS=:
	PATH=/bin:/usr/bin:/usr/local:/etc
	paths= ( ${PATH} )
	$print ${paths[0]} ${paths[2]}
	/bin /usr/local

Creating associative arrays using compound assignment

Example

	typeset -A color
	color=([sky]=blue  [fire]=red [marigold]=yellow)

	for i in ${!color[@]}  
	do 
		print $i ${color[$i]}
	done

	-------------------- Output --------------------
	marigold  yellow
	fire  red
	sky  blue

Name reference variables

It is often useful to pass the name of a variable rather than the value of the variable, as an argument to a function. A name reference variable, called nameref, provides a mechanism for providing a local name for this variable. The typeset -n attribute specifies that a variable is a reference to another variable. The name for a name reference variable must be an identifier. Once a name reference is created, each reference to this variable causes the variable named by the name reference to be used.

A nameref variable cannot have periods in its name.

Example 1

	$old=123
	$typeset -n new=old
	$echo $new
	123

A nameref variable can also be used to change value of the original variable.

	$new=345
	$echo $old
	345

This nameref feature is particularly useful for passing variable names to functions.

Example 2

	function getFile {

	typeset -n filename=${1}
	typeset message="${2}"

	print -n "${message}"
		read filename
	}

	function canRead  {

	typeset -n file=${1}

		if [[ -r ${file} ]]
		then
			return 0
		else
			return 1
		fi
	}

	function endPrint {

	print ${fromFile} copied to ${toFile}
	}

	getFile toFile "File to copy to: "
	getFile fromFile "File to copy from:  "

	if canRead fromFile
	then
		cp ${fromFile} ${toFile}
		endPrint
		exit 0
	else
		print -u2 cannot read from ${fromFile}
		exit 1
	fi

Internationalization support

ksh93 provides support for internationalization. Double-quoted strings preceded by a $ are checked for message substitution. If the message appears in the message catalog and the shell variable LANG is defined to some locale other than C or POSIX, then ksh93 will substitute the string with the corresponding string from the message catalog. Otherwise, the string is unchanged.

Executing ksh93 -D on a shell script will output all messages identified for internationalization.

Back to top

KornShell Development Kit

ksh93 is extensible through the KornShell Development Kit (KDK). You can write your own built-in functions in C and load them into the current shell environment through the builtin command. This feature is available on operating systems that would let you load and link code into the current process at run time.

A built-in command is executed without creating a separate process. The command is invoked as a C function by ksh93. If this function has no side effects in the shell process, then the behavior of this built-in is identical to that of the equivalent stand-alone command. The primary difference in this case is performance: the overhead of process creation is eliminated.

Discipline functions

Each variable can have one or more functions associated with it by defining functions whose names are of the form varname.action, where varname is the name of the variable and action is the name of the discipline function. The discipline functions named get, set, and unset can be set for any variable. These functions are called when operations are performed on the corresponding variable varname. Through the KornShell Development Kit, you can also add disciplines unique to your environment.

When a variable is referenced, as in $myVariable, ksh93 will invoke the get discipline associated with myVariable. The default discipline is to simply return the current value associated with myVariable. From the shell level, you can define a myVariable.get discipline function.

The set discipline is called when a value is assigned to a variable. Within the set discipline, the special variable .sh.name is the name of the variable whose value is being set. The value of the special variable .sh.value is the value returned from the discipline.

Back to top

Changes and additions to ksh93

Along with arithmetic features below, other changes and additions include the following:

• New parameter expansions
• New quoting mechanisms
• Input/Output additions
• New behavior for functions
• New pattern matching capabilities
• String pattern matching
• Pre-defined variables
• New compound commands
• Control command
• New PATH search rules
• Changes to return values
• Additions and changes to built-in commands
• New ksh93 invocation options
• Preset aliases
• Using ksh93 to make a suid script
• Shell I/O
• Key binding
• FPATH

Arithmetic features

ksh93 supports double precision floating point arithmetic. The typeset attributes exponential (-E) and float (-F) are used to define real variables. Each can be specified with a number that specifies:

	-E	The number of significant figures to use when the number is expanded.
	-F	The number of places after the decimal when the number is expanded.

Example

	$typeset -E4 x
	$typeset -F5 y
	$x=1234.567	
	$y=1234.567
	$print $x
	1235
	$print $y
	1234.56700

Note of Caution: Mixed mode arithmetic may lead to inconsistent results.

The following operators have been added:

• The conditional operator ?:
• The comma operator ,
• The postfix and prefix operators ++ and --
• The unary +

You can use functions from the ANSI C math library within arithmetic expressions. For integer constants, you can specify arithmetic bases up to base 64.

The following functions are available:

abs, acos, asin, atan, cos, cosh, exp, int, log, sin, sinh, sqrt, tan, tanh

New parameter expansions

The following parameter expansions are available in ksh93:

${!varname}	Expands to the name of the variable defined by varname. In most 
		cases, varname will simply be expanded to varname. 
		However, if varname is a reference variable, this format will 
		expand to the variable name that varname is referring to.

Example

	$x=10
	$print ${!x}
	x
	$nameref foo=bar
	$print ${!foo}
	bar

${variable:offset}		Substring starting at offset.
${variable:offset:[length]}	Up to length characters of
				variable starting at offset.							

The basic action of this expansion is to return (extract) a substring of the variable’s value. The value of the variable is not modified.

Example

	$alphabet=abcdefghijklmnopqrstuvwxyz
	$print ${alphabet:12}
	mnopqrstuvwxyz
	$print ${alphabet:12:3}
	mno
	$print ${alphabet:12:30}
	mnopqrstuvwxyz
	$seven=7; two=2
	$print ${alphabet:${seven}:${two}}
	hi

${@:offset} Positional parameters starting at offset.	
${*:offset}

${@:offset:length} Up to length positional parameters starting at offset.
${*:offset:length}

${varname[@]:offset} Array elements of varname starting at offset.
${varname[*]:offset}

${varname[@]:offset:length} Up to length array elements of varname starting at offset.
${varname[*]:offset:length}

Example

	$set foo/fun/bar  hello.world  /dev/null
	$print ${@:2}
	hello.world  /dev/null
	$print ${*:2:4}
	hello.world  /dev/null
	$x=( foo/fun/bar  hello.world  /dev/null)
	$print ${x[@]:1:2}
	hello.world  /dev/null

${variableOPpattern/string}

The basic action of this expansion is to return a modified version of the variable’s value. The value of the variable is not modified.

OP operator	Action
/		Value of variable with the first occurrence of pattern 
		replaced by string.
/#		If variable begins with pattern, pattern 
		is replaced by string.
/%		If variable ends with pattern, pattern 
		is replaced by 	string.
//		Value of variable with each occurrence of pattern 
		replaced by string.

Example

	$path=/chapters/chapter01/subchapter
	$print ${path/chapter/JUNK}
	/JUNKs/chapter01/subchapter

	$print ${path/#chapter/JUNK}
	/chapters/chapter01/subchapter

	$path=chapters/chapter01/subchapter
	$print ${path/#chapter/JUNK}
	JUNKs/chapter01/subchapter

	$print ${path//chapter/JUNK}
	JUNKs/JUNK01/subJUNK

New quoting mechanisms

ksh93 processes a single quoted string preceded by a $, $’...’, using ANSI C string conventions.

An ANSI C string is defined by preceding the single-quoted string with a $. For example, $'*' is the literal asterisk, *. With ANSI C strings, all characters between the single quotes retain their literal meaning, except for escape sequences.

	$print -r $’${PWD} \nreturns the cwd.’
	${PWD}
	returns the cwd.

	$print $’${PWD} \nreturns the cwd.’
	${PWD}
	returns the cwd.

ksh93 processes a double quoted string preceded by a $, $"...", as a string that needs to be translated when the locale is not C or POSIX. The $ is ignored in the C or POSIX locale.

ANSI C string support provides an essential feature for shell programmers. Consider, for example, having to process variables with embedded tabs in their values. Without ANSI C string support, we would not be able to effectively test the value of the variable for embedded tabs. As an example, consider the following script:

Example 1

	$print $’hello\n\tworld’

	hello
		world

Example 2

	$print "foo\tbar" > /tmp/foobar
	$read aline <  /tmp/foobar
	$if [[ "${aline}" == "foo\tbar" ]]
	then 
		print TRUE
	fi

The comparison will fail. When the conditional is replaced with ANSI C strings the comparison will succeed as shown in the following:

	$print "foo\tbar" > /tmp/foobar
	$read aline < /tmp/foobar
	$if [[ "${aline}" == $'foo\tbar' ]]
	then      
		print TRUE
	fi

Input/Output additions

The redirection operators >& digit and <& digit can be followed by a - to cause a given file descriptor to be moved rather than duplicated.

Example

	exec 3<&4-	# Moves file descriptor 4 to 3.

The printf built-in command and the -f format option to print can be used to produce formatted output using ANSI C formatting conventions. Additionally, print and printf can contain the following format arguments:

%b	Expand escape sequences in each argument
%q	Quote strings that contain special characters
%P	Treat argument as regular expression and convert it to a shell pattern.

The read built-in command has options to specify a timeout (-t) and to specify the line delimiter (-d) character. In addition, you can specify that read split fields into an indexed array.

	read -t 15 line	# Waits up to 15 seconds to read a line into variable named 
				# line.
	read -d : field	# Reads up to : into variable field
	read -A var 	# Stores the fields in var as an indexed array
			# starting at index 0, using the
			# characters in the IFS variable as delimiters.

New behavior for functions

If you declare a function with the function varname format, then ksh93 executes the function in a separate function environment.

If you declare a function in varname() format, then ksh93 executes the function in the current environment, like a dot script.

The second format is provided for compatibility with the POSIX standards. The primary distinction is in the scope of the varable name. In the following POSIX function bar, variable foo has global scope and is redefined to a value of 6.

	typeset foo=5
	bar()
	{
		typeset foo=6
		echo $foo
	}
	bar
	6
	echo $foo
	6

On the other hand, in the following definition, a local variable foo is defined and has precedence over the global variable foo.

	typeset foo=5
	function bar
	{
		typeset foo=6
		echo $foo
	}
	bar
	6
	echo $foo
	5

New pattern matching capabilities

The pattern matching construct [:character_class:] inside [ ], matches the specified set of characters. The following character classes are available:

Character Class		Same AS

[:alnum:]		a-z, A-Z, 0-9
[:alpha:]		as a-z, A-Z
[:blank:]		space or tab
[:cntrl:]		the set of control characters
[:digit:] 		0-9
[:graph:] 		digits, characters, and punctuation
[:lower:] 		a-z
[:print:]		digits, characters, punctuation, and space character 
[:punct:]  		any punctuation
[:space:] 		space, tab, newline, vertical tab, form feed, carriage return
[:upper:]		A-Z
[:xdigit:]		a hexadecimal digit a-f, A-F, 0-9

Example

	para[[:digit:]t] matches para0, para1, ..., para9, and parat

String pattern matching

	string = = pattern

• True if string matches pattern
• Spaces required on either side of = =

Pre-defined variables

The following are a few of the 16 new variables that have been added to ksh93:

.sh.name			Is set inside a discipline function to the name of the
				variable associated with the discipline function.
.sh.subscript			Is set inside a discipline function to the name of
				the subscript of the variable associated with the
				discipline function.
.sh.value			Is set inside a set discipline function to the 
				value that is intended to be assigned to the variable 
				associated with the set discipline function. 
				The value of .sh.value when the 
				discipline function completes will be the value of 
				the assignment.

Example

	unset foo
	function foo.set
	{
		print "old: ${.sh.name}[ ${.sh.subscript}]=${.sh.value}"
		.sh.value=good
	}
	foo[3]=bad
	old:foo[3]=bad
	print "new: foo[3]=${foo[3]}"
	new:foo[3]=good

.sh.version	Version of shell.

				$print ${.sh.version}
				Version M-12/28/93e

FIGNORE ksh93 ignores each name that matches the pattern defined by the value of FIGNORE.

Example

	$print /usr/*/include/* 		# Displays a listing of files in all include
						# directories.

	/usr/local/include/sys /usr/gcc/include/sys
	$FIGNORE=gcc      			# Pattern to match C source files
	$print usr/*/include/*
	/usr/local/include/sys

HISTCMD		Is the number of the current command in the history file.
ERRNO 		Variable has been deleted from ksh93. Error messages
		produced by ksh93 give the reason for system failures
		in most cases.
ENV 		Variable is now effective only for interactive shells.
SECONDS		Variable now has a granularity of milliseconds rather than
		seconds.
TMOUT 		Variable also sets the timeout for the select command.

New compound commands

The negation compound command (reserved word ! ) negates the return value of an expression.

Example

	if   ! who | grep -s ‘sdutta’
	then 
		print sdutta is not logged on
	fi

The return value of ! is:

• True (zero) if the return value of the expression is False (non-zero)
• False (non-zero) if the return value of the expression is True (zero)

Control command

The new arithmetic for compound command is similar to the ANSI C language for command:

	for ((init; test; increment))
	do
		list-of-commands
	done

New PATH search rules

The following changes have been made to the way ksh93 searches for commands:

• Only special built-ins are searched for first. (All built-ins were searched for first in earlier versions of the KornShell.) Special built-ins are built-in commands that are executed in the current environment e.g. alias, break, continue etc.
• Functions are searched next.
• Non-special built-ins are searched next.
• External commands are searched next which are found by searching the directories listed in $PATH.
• When searching PATH, ksh93 treats directories that are also in FPATH as function directories. In earlier versions of the shell, functions found in FPATH were found only after searching PATH.
• Built-ins can be associated with a pathname. In this case the built-in is only used when the PATH search would yield this PATH.
• ksh93 turns the trackall option on by default.

Changes to return values

The return values have been changed as follows:

• Commands that are not found return 127.
• Commands that are found but cannot execute return 126.
• Commands that terminate because of a signal return 256 plus the signal number. You can use kill -l to get the name of the signal given the return value.

Additions and changes to built-in commands

The -? option has been added to all regular built-in commands and all other commands. It displays a list of options available for the command.

The following built-ins have been added:

• builtin can be used to list built-in commands or to add dynamically linked libraries and new built-in commands.
• command eliminates functions from the search order. In front of a special built-in command, command causes the special built-in to behave like a regular built-in command.
• disown causes ksh93 not to send a HUP signal to the specified jobs.
• false has a False return value. It was an alias in ksh88.
• true has a True return value. It was an alias in ksh88.
• getconf displays the value of system configuration parameters.
• hist is the same as fc in ksh88. fc is now an alias.
• printf is nearly the same as the ANSI C Programming Language printf function.

The alias command and the typeset command with the -f option ignore the -x option since aliases and functions are no longer exported to scripts.

The following options have been added to exec:

• -a name to specify the command name argument.
• -c to clear the environment.

The following options have been added to kill or have changed:

• -n to specify the signal by signal number.
• -s to specify the signal by signal name.
• -l without arguments lists only the signal names, not their numbers. -l can be used to convert a signal name to and from a signal number.

The following options have been added to unset:

• The -n option unsets a name reference variable.
• The -v option specifies that only variables will be unset.

The following options have been added to whence:

• The -a option displays all matches for each name.
• The -f option skips the search for functions.

The built-in dot (.) command now executes functions as well as files. ksh93 executes the specified function in the current environment. ksh93 uses the search path specified by the FPATH variable to find the function. In addition, arguments given to . (dot) are restored when the . (dot) script or function completes.

New ksh93 invocation options

The following invocation options to ksh93 have been added:

• -n displays many warning messages.
• -D displays the list of double quoted strings that are preceded by a $ in the given script but does not execute the script.

Preset aliases

The following are defined by the shell:

	command=’command’
	fc=hist
	float=’typeset -E’
	nameref=’typeset -n’
	redirect=’command exec’
	times=’{ { times; } 2>&1; }’

Using ksh93 to make a suid script

Requirements for a suid script include:

• #! directing the KornShell be used
• Executable by user, group, and other
• No read permission
• Add suid permission by chmod u+s on the file

Add a -p option to #! to increase security to force a separate process if one is not normally done.

Example

	#! /usr/bin/ksh -p

Shell I/O

ksh93 versions allow direct socket access to servers as long as the operating system supports socket connections of the form:

	/dev/tcp/hostid/portid   
	/dev/udp/hostid/portid

where:

• hostid is the dotted decimal number of the host
• portid is the port the server is listening on

Key binding

In ksh93 it is possible to intercept keys as they are entered and bind them to new meanings. Each time the shell processes characters entered from the keyboard (except for those used in search strings or as an argument to an edit directive such as r in vi), a trap named KEYBD is evaluated and the action associated with this trap can be used to change the value of the entered key so it would perform a different operation.

FPATH

ksh93 functions are not inherited across different invocations of ksh93. A child process, for example, does not have access to the functions defined within the parent ksh93 invocation. This has historically limited the re-usability of KornShell functions. As a solution, ksh93 will search the colon-separated list of directories given by the FPATH for an executable file with the same name as the function.

Back to top

Summary

ksh93, the latest major revision of the KornShell language, has a number of significant enhancements over its previous major release, the 1988 version. The new version provides an alternative to Tcl and Perl and compares favorably in speed and functionality. Like Tcl, it is extensible and embeddable with a C language application programming interface.

Resources

• For information on kornshell, see www.kornshell.com.
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• Introduction
• Features of ksh93
• KornShell Development Kit
• Changes and additions to ksh93
• Summary
• Resources
• About the author
• Comments

Dig deeper into AIX and UNIX on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers