July Release Confetti
150+ Courses, Challenges, and Learning Activities
Launching in July!
Learn More

What makes up a UNIX system?

Introduction 
UNIX operating systems are extensively used in contemporary servers of diverse processor architectures, workstations, network and mobile devices. This guide introduces you to the UNIX operating system and offers you an overview of the most relevant concepts of a UNIX System. 
History

The term UNIX refers to a family of time sharing, multitasking, multiuser operating systems. The first UNIX version was developed in 1969 at the Bell Laboratories by Ken Thompson using an idle PDP-7. He was joined by Dennis Ritchie soon after [1].

Originally UNIX was intended to be used inside the Bell System, for programmer to programmers. Thompson and Ritchie worked on developing different UNIX versions for many years. Ritchie designed the C language in order to support UNIX, then a third version resulted from rewriting the operating system in C. 

As UNIX progressed, it became widely used at AT&T and as the original code was made available, it was progressively adopted by other computer science organizations. Thompson and Ritchie worked on developing different UNIX versions for many years. Ritchie designed the C language in order to support UNIX, then a third version resulted from rewriting the operating system in C. 

Programmers at the University of California at Berkeley made the most significant modifications to the original source, resulting in the Berkeley Software Distribution (BSD) UNIX. One of the most important advances made to UNIX by the programmers at Berkeley was to provide support for the Internet networking protocols (TCP/IP).

In 1988, the 4.2BSD, AT&T UNIX, and other operating systems were blended into UNIX System V release 4. This new generation of the OS combined the best features to become a standard for the UNIX OS [1]. This new SVR4 became the basis for AT&T UNIX, Sun Microsystems’s SUNOS, IBM’s AIX and HP’s HP-UX.


Main components of a UNIX Operating System

This section introduces the three components of a UNIX operating system and describes how these components interact with each other. The three main components of a UNX system are:

  • The kernel
  • The shell
  • The file system

user_45094_582bab7fe8110.jpg


The kernel

The kernel, also called the core of a UNIX system, manages the physical reso urces of a hardware computer, devices, memory, processes and daemons.

The kernel controls the functions between the system programs and the hardware. It also schedules and executes all commands.

The kernel manages functions such as:

  • File system and structures
  • Device management, such as storing data to the disk
  • Process management, such as daemons
  • Memory management, such as swap space

The Shell

A shell is an interface between the user and the kernel. It acts as a command interpreter. The shell accepts commands issued by the user, interprets these commands and then sends the executable to the kernel. The three most common shells in UNIX are: Bourne shell, C shell and Korn shell. There are also popular open source UNIX shells such as Z shell and bash, both are extensions of Bourne shell.

In order to identify the shell you are using in a UNIX system, execute the following command, in the example below. Here we see that the shell in use is zsh:


echo $SHELL
/bin/zsh

The File system

The UNIX file system is a hierarchy of directories, subdirectories and files that are organized or grouped for special purposes.

The Figure 2 depicts a UNIX file system example:


user_45094_582bac1f93f4d.jpg






For further information, check out the Unix file System tutorial.

The Figure 3 illustrates the layers of a UNIX system. Everything below the system call interface and above the hardware is the Kernel. The kernel provides the file system, the process control subsystem, the memory management and the operating system functions over system calls. Programmers use the kernel system calls to deploy useful functions, for example compilation and file manipulation.


user_45094_582bac8a75d5c.png












In the following sub-sections, we will discuss the main functions of the UNIX kernel.

Developer Environment

The programmer interface in UNIX is defined by the system calls. Most systems programs in UNIX are written in C, and all the system calls are C functions. As the details of the system calls are known only by the compiler, this makes portability a main feature of UNIX programs.

System calls for file manipulation are for example: creat, open, read, write, close, link, unlink and trunc.

A new process is created by the fork system call. The new process is a copy of the process space of its process parent (the process that invoked the fork system call).

Signals are used to handle exception conditions. In UNIX there are 20 different signals. For example, the interrupt signal is used to interrupt a command before it completes execution, usually done by the user typing ^C.

User Interface

The developers and the users of a UNIX system can use system programs that are already compiled and available for execution. These programs can be divided by function, for example directory and file manipulation: mkdir, ls, touch, file, rm, link, unlink, cat among others. The program for sorting files: sort, for comparing files: diff, to look for patterns: grep and awk. Additionally, there are available text editors, such as vi,vim , sed among others. These commands are executed by the user interpreter or shell.

The standard I/O refers to the three types of files that a process can open. They are known as standard input (file descriptor 0), standard output (1) and standard error (2). Then the program can read from the standard input, and it can send output to the user’s screen or to a file with the standard output.

 A user can direct the standard output to a file called test1, or send the files test1 as an input to a program:

[mbeuses@myserver linuxacademy]$  ls > test1
[mbeuses@myserver linuxacademy]$ lpr < test1

Process Management

In UNIX when a command is executed, a process is created, and the operating system creates an environment for the process to run. This process structure contains all the information that the system needs to know about a process such as process identifier, scheduling priority, and pointers to other process control blocks. To manage functions, the OS uses processes known as daemons.

CPU Scheduling is a mechanism designed to allow a process to use the CPU while another process is waiting for I/O, in this way the system makes fair and efficient use of the CPU resources. The CPU scheduling can be adjusted according to scheduling priority policies.

The following example run on a UNIX Solaris server shows the output of the command ps -ef. The output shows that the first process that is executed when the system boots is sched (the swapper) followed by the init process, pageout, and so on. [2]


[mbeuses@myserver linuxacademy]$  ps -ef
UID PID PPID C STIME TTY TIME CMD
root 0 0 0 18:04:04 ? 0:15 sched
root 5 0 0 18:04:03 ? 0:05 zpool-rpool
root 1 0 0 18:04:05 ? 0:00 /sbin/init
root 2 0 0 18:04:05 ? 0:00 pageout
root 3 0 0 18:04:05 ? 2:52 fsflush
root 6 0 0 18:04:05 ? 0:02 vmtasks
daemon 739 1 0 19:03:58 ? 0:00 /usr/lib/nfs/nfs4cbd
root 9 1 0 18:04:06 ? 0:14 /lib/svc/bin/svc.startd
root 11 1 0 18:04:06 ? 0:45 /lib/svc/bin/svc.configd
daemon 559 1 0 18:04:49 ? 0:00 /usr/sbin/rpcbind
netcfg 47 1 0 18:04:19 ? 0:01 /lib/inet/netcfgd
dladm 44 1 0 18:04:17 ? 0:00 /sbin/dlmgmtd
netadm 51 1 0 18:04:22 ? 0:01 /lib/inet/ipmgmtd
root 372 338 0 18:04:43 ? 0:00 /usr/lib/hal/hald-addon-cpufreq
root 67 1 0 18:04:30 ? 0:02 /lib/inet/in.mpathd
output truncated...
Memory Management

Memory management is the process of managing the physical memory of the computer which consists of primary memory and secondary memory. The operating system keeps track of the memory allocation (in use) and which memory is free, in order to allocate memory to processes when needed and to de-allocate memory when the process finishes. Memory management includes swapping and demand paging.

  • Swapping is used to handle memory contention between processes. If there is memory contention, processes are swapped out of memory until there is enough memory available. The scheduler process decides which processes to swap in or out.

  • Demand Paging permits execution with only parts of each process in memory. Demand paging is done when a process needs a page and the page is not actually loaded into main memory, then a page fault occurs to the kernel, and the operating system allocates memory and the page is read into this memory space.
File System

The UNIX file system is composed of files and directories. Directories are actually a type of file.

An inode is a list of information related to a particular object (either a file, a directory or a symbolic link). The inode stores the type of object, permissions, ownership and the location where the data is stored.

When working with directories, there are different commands such as pwd that enables you to identify in which directory you are currently working, and move up and down the directory hierarchy such as cd.

The command echo lets you echo the string to the standard output, then using the redirection you can write the output in a file. The command file lets you determine the file type, and ls is a useful command to list information about the files of a directory. It can receive many arguments to display different types of file information.


[mbeuses@myserver linuxacademy]$ pwd 
/export/home
[mbeuses@myserver ~]$ cd linuxacademy
[mbeuses@myserver linuxacademy]$ echo "this is a test" >> test1.output
[mbeuses@myserver linuxacademy]$ cat test1.output
this is a test
[mbeuses@myserver linuxacademy]$ file test1.output
test1.output: ASCII text
[mbeuses@myserver linuxacademy]$ ls -lart test1.output
-rw-r--r-- 1 mbeuses mygroup 15 Nov 7 20:08 test1.output
[mbeuses@myserver linuxacademy]$ cd ..
[mbeuses@myserver linuxacademy]$ pwd
/export/home


The command mkdir is used to create a directory. Then the command pwd is used to display the current directory, while ls lists the directory contents and rm can delete a directory: 


[mbeuses@myserver ~]$ mkdir linuxacademy
[mbeuses@myserver ~]$ cd linuxacademy
[mbeuses@myserver ~]$ pwd
/export/home/linuxacademy
[mbeuses@myserver ~]$ ls
test1.output
[mbeuses@myserver ~]$ rm test1.output
[mbeuses@myserver ~]$ ls


When working with files, these commands enable you to view the file contents, file type, and also display any other information about a file. The following are examples of using the commands: echo, cat, tail, and head.


mbeuses@myserver:~$  echo $SHELL
/usr/bin/bash

mbeuses@myserver:/etc$ cat release
Oracle Solaris 11.1 SPARC
Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
Assembled 06 November 2013

mbeuses@myserver:/etc$ head path_to_inst
#
# Caution! This file contains critical kernel state
#
"/fcoe" 0 "fcoe"
"/iscsi" 0 "iscsi"
"/pseudo" 0 "pseudo"
"/scsi_vhci" 0 "scsi_vhci"
"/scsi_vhci/disk@g5000cca0162a94f0" 3 "sd"
"/scsi_vhci/disk@g5000cca0162aa07c" 4 "sd"
"/options" 0 "options"
mbeuses@myserver:/etc$

mbeuses@myserver:/etc$ tail path_to_inst
"/pci@380/pci@1/pci@0/pci@6/SUNW,emlxs@0/fp@1,0" 9 "fp"
"/pci@380/pci@1/pci@0/pci@6/SUNW,emlxs@0,1" 3 "emlxs"
"/pci@380/pci@1/pci@0/pci@6/SUNW,emlxs@0,1/fp@0,0" 10 "fp"
"/pci@380/pci@1/pci@0/pci@6/SUNW,emlxs@0,1/fp@1,0" 11 "fp"
"/pci@380/pci@1/pci@0/pci@7" 21 "pcieb"
"/pci-performance-counters@340" 0 "iospc"
"/pci-performance-counters@300" 1 "iospc"
"/pci-performance-counters@3c0" 2 "iospc"
"/pci-performance-counters@380" 3 "iospc"
"/ramdisk-root" 0 "ramdisk"

I/O System

There are three main categories of I/O in UNIX: block devices, character devices and sockets for network communication.

  • Block devices: this category includes disks and tape. A block device is essential to isolate disk details from the rest of the kernel. Block devices are accessible thru the system file (example: /dev/sda the first scsi drive on the first scsi bus)
  • Character devices: this category includes printers, terminals (example: /dev/null a bit bucket or bottomless sink for data)

In order to display the block and character devices in a UNIX system, execute the following command in the shell. The following truncated output shows some /dev devices of a UNIX server:


$ ls -la /dev/
mbeuses@myserver:~$ ls -la /dev | more
total 356
drwxr-xr-x 253 root sys 253 Jul 20 11:34 .
drwxr-xr-x 25 root root 27 Jul 20 10:32 ..
lrwxrwxrwx 1 root root 30 May 28 2013 allkmem -> ../devices/pseudo/mm@0:allkmem
lrwxrwxrwx 1 root root 27 May 28 2013 arp -> ../devices/pseudo/arp@0:arp
lrwxrwxrwx 1 root root 29 May 28 2013 bge -> ../devices/pseudo/clone@0:bge
lrwxrwxrwx 1 root root 56 May 28 2013 bge0 -> ../devices/pci@0,600000/pci@0/pci@1/pci@0/network@4:bge0

lrwxrwxrwx 1 root root 30 May 28 2013 console -> ../devices/pseudo/cn@0:console
drwxr-xr-x 3 root root 3 May 28 2013 cpu
lrwxrwxrwx 1 root root 33 May 28 2013 crypto -> ../devices/pseudo/crypto@0:crypto

lrwxrwxrwx 1 root root 31 May 28 2013 ixgbe -> ../devices/pseudo/clone@0:ixgbe
lrwxrwxrwx 1 root root 53 May 28 2013 ixgbe0 -> ../devices/pci@300/pci@1/pci@0/pci@1/network@0:ixgbe0

lrwxrwxrwx 1 root root 6 May 28 2013 stderr -> ./fd/2
lrwxrwxrwx 1 root root 6 May 28 2013 stdin -> ./fd/0
lrwxrwxrwx 1 root root 6 May 28 2013 stdout -> ./fd/1


For further detail refer to Understanding dev and its subdirs and files 

Inter-process communication

The inter-process communication is the mechanism the operating system provides to let processes share data. The pipe is the inter-process mechanism most distinctive of UNIX. Other inter-process mechanisms are: signal, socket, semaphore, shared memory and file.


In the following example, we first issue the command cat to display the content of the file path_to_inst, and pass its output as an input to the command grep. We use the pattern “fp” to find the fiber channel adaptor instances on the mentioned file, and then we pass the output of grep to the command wc to count (using the flag -l) the lines in the file path_to_inst that contain the pattern “fp”.


mbeuses@myserver:/etc$ cat path_to_inst | grep fp | wc -l
132


With these subsections we complete describing the main functions of the Unix Kernel. For further details about differences between UNIX and Linux kernels check out Differentiating UNIX and Linux.

Operating System Virtualization

Operating system virtualization refers to the method in which the kernel allows system hardware to execute multiple instances of user-space instances, also called containers.

Operating system virtualization can be used for consolidation of multiple isolated hosts and services on a single physical server.

Examples of UNIX operating system level virtualization are: Solaris containers or zones; AIX workload partitions (wpars); and Docker for Linux licensed by Apache, among others.


user_45094_582bafdabe461.png

Unix-like Variants

In the nighties, Unix-like systems spread in popularity as Linux and BSD distributions.

The Linux kernel was developed in 1991 by Linus Torvalds. Linux is a Unix-like operating system based on the free and open-source software model. For further information check out: https://www.gnu.org/distros/free-distros.html

https://www.debian.org/releases/lenny/ia64/ch01s02.html.en

https://www.ubuntu.com/

More recently in 2000, Apple released the Open-source Darwin OS which is the core of macOS, iOS, watchOS and tvOS.

Further Resources

The following Linux Academy resources can be followed in order to deepen your knowledge:

Linux Essentials Certification

Linux Foundation Certified System Engineer

Linux Foundation Certified System Administrator

Linux+ LPICLevel 1 Exam 1

References

[1]A. Silberschatz, P. B. Galvin, G. Gagne, and A. Silberschatz, Operating system concepts: Addison-Wesley Reading, 1998.

[2] https://docs.oracle.com/cd/E23824_01/html/821-1451/spprocess-1.html

[3] http://www.vim.org/about.php


  • post-author-pic
    Rilindo F
    11-16-2016

    Noice.

  • post-author-pic
    Milagros B
    11-16-2016

    Thanks!

  • post-author-pic
    Carlos U
    11-16-2016

    Hi Milagros, pretty strong guide overall. Nice one! Bookmarking this. On a side note, you might want to fix a few typos and presentation errors, for example, you have a new line in some places breaking the sentence (probably after copy/pasting). There's also a bit of duplicate information in the Introduction. The example in the inter-process communication section is the same as I/O,  etc.Thanks for this guide.


  • post-author-pic
    Thomas H
    11-16-2016

    Nice work, Milagros.

  • post-author-pic
    Fraser M
    11-16-2016

    Good stuff, Thanks for the guide

  • post-author-pic
    Jaf S
    11-17-2016

    helpful!

  • post-author-pic
    Hunter F
    11-18-2016

    Thank you for your contribution!!! 

  • post-author-pic
    Alejandro N
    11-20-2016

    Nice intro guide,  @mbeuses  Good job!

  • post-author-pic
    Max D
    11-24-2016

     @mbeuses Great Job!

  • post-author-pic
    Tomislav M
    11-25-2016

    Tnx man, a great tutorial!

  • post-author-pic
    David G
    11-29-2016

    Excellent! Great Work!

  • post-author-pic
    Grégory C
    11-29-2016

    Thanks for this tutorial !!! Very good !!!

  • post-author-pic
    Larry S
    12-30-2016

    Love it! 

  • post-author-pic
    Manisha S
    12-30-2016

    Excellent job  @mbeuses !! It is very very very well written! Thanks for your contribution.

  • post-author-pic
    Nizam U
    02-02-2017

     I can't see that much differences between UNIX commands and Linux commands here. 

  • post-author-pic
    Prakyath R
    02-02-2017

    Very good content!!

  • post-author-pic
    Ivan K
    02-02-2017

     @durjoy  thats because Linux system are based off Unix system

  • post-author-pic
    Nizam U
    02-03-2017

    So does it mean that, if you are well experienced with linux commands,  you can administrate UNIX to?

  • post-author-pic
    Ivan K
    02-03-2017

     @durjoy  to a certain extent you should be able to recognize some common commands but there are some significant technical differences between the two systems

    These differences are very well explained here ---> http://www.ibm.com/developerworks/aix/library/au-unix-difflinux.html


  • post-author-pic
    Nizam U
    02-14-2017

    Thank you ivan

  • post-author-pic
    John R
    02-24-2017

    nice


  • post-author-pic
    Sami N
    05-04-2017

    Thanks lots

  • post-author-pic
    Andrew K
    05-19-2017

    Rockin article. Been a Linux/System engineer for 5 years now, IT longer, and I learned a thing or two.

  • post-author-pic
    Sagar G
    05-24-2017

    Neatly explained....Thank You

Looking For Team Training?

Learn More