overview edit doc new doc remove

Jan 26, 2018

updated at: May 28, 2019

LPIC 2 - Linux Engineer

This document is not done yet!

Capacity Planning

Measure and troubleshoot resource usage

iostat

To measure the input/output of a device load you can use:

$ iostat

iotop

Similar to the top command.

# iotop

vmstat

This utility produces information about virtual memory statistics

$ vmstat <delay> <count>

Output explained |

Output Explanation
r Number of processes currently allocated to the processorrun queue
b Number of processes currently allocated to the block queue
swpd Amount of virtual memory being used (swap space)
free Amount of RAM no in use, not cached and not buffered
buff Amount of RAM that is buffered
cache Amount of RAM that is cached
si Amount of memory being swapped in from disk per second
so Amount of memory being swapped out from disk per second
bi Amount of blocks per second being received from a block device
bo Amount of blocks per second being sent to a block device
in Interrupts per second
cs Number of context switches
us Percentage of time spent running non-kernel code (user code)
sy Percetage of time spent running kernel code (system code)
id Percentage of idle time
wa Percentage of time waiting for I/O
st Percentage of stolen CPU time by a virtual machine

The first row will always show average measurements since the machine has booted.

free

free shows you a current overview of the total amount of physical and virtual memory of a system, aswell as the amount of free memory and memory in buffers.

$ free

netstat

netstat shows network connections, routing tables, interface statistics, and so on.

$ netstat

Some options:

Option Function
-r The kernel routing tables
-g Multicast group membership information for IPv4 and IPv6
-i Lists all network interfaces and certain properties
-s Lists summary of statistics for each protocol

ss

This utility is used to show socket statistics. It can display statistics for the sockets: PACKET, TCP, UDP, DCCP, RAW, Unix and more. It is similar to netstat but ss is newer and can display more TCP and state information.

A good set of options:

$ ss -tupna

-t: tcp, -u: udp, -p: processes using sockets, -n: do not try to resolve service names, -a: all

iptraf

iptraf is a network monitoring utility for IP networks and can be used to monitor the load on a network. It intercepts packets on the network and gives out various pieces of information about the current IP traffic.

# iptraf-ng # depends on distribtution, sometimes just iptraf

ps

The ps command shows a list of currently running processes on your system.

$ ps 

If you would run ps aux you will see a list of currently running processes but with some extra tables: %CPU is the CPU usage. %MEM is the ratio of process physical memory on the machine. VSZ is the virtual memory size including swapped out memory from shared libraries. RSS resident set size. How much RAM is allocated to that process. TTY controlling terminal associated with the process and STAT is the process status code.

pstree

To have a nice tree-view of currently running processes use:

$ pstree

top

Dynamic real-time overview of running processes.

$ top

htop

Is like top but extended, more color and more features.

$ htop

w

The w command displays information about the users currently logged in.

$ w

ssof

The lsof command is used to list information about open files

$ lsof <file>

With the -F option you will get an unformatted list that can easily read by programming languages. Adding +D option will list all open files within a specified directory. Using -u option will show open files by a user.

uptime

This command shows how long a system has been running, how many users are logged on, the system load averages and the current time.

$ uptime

sar

The sar command collects, reports or saves system activity information. the system needs to be enabled to collect data if you want to use sar.

$ sar

Common used options:

Option Function
-d Outputs disk statistics
-b Outputs I/O and transfer rate statistics
-c System calls
-p Paging
-w Swapping activity
-q Run queue
-r Free memory and swap over time

Predict future resource needs

Monitor IT infrastructure

A very handy tool to collect system performance statistics periodically is collectd. It is a daemon that provides mechanisms to store the values in a variety of ways. these statistics can be used to find the current performance bottlenecks and predict future system load.

collectd only collects data, not displaying.

Other monitor tools are: Icanga2, Nagios, MRTG and Cacti. They can be used to measure, collect and display resourse performace statistics.

Predict future growth

Steps to predict future needs:

Resource Exhaustion

Most of the time a single resource which gets exhausted will be extractable from collectd measurement data. This is what we call a bottleneck: a single point within the system that narrows throughput and slows down everything below. It is important to note that only fixing the bottleneck is not enough, you will need to get the bigger picture and change the system if needed or prevent further bottlenecks if it can be done.

Linux Kernel

Kernel components

Kernel type

The linux kernel was originally designed to be a monolithic kernel but that means that all drivers for all the various types of supported hardware, regardless if you would need it or not, would be in that kernel. Instead of this a system was introduced to solve this issue: kernel modules.

So it is not monolithic but it is also not a microkernel. A microkernel is designed so the least possible amount of code is run in supervisor mode.

The best way to describe the Linux is kernel is by saying it is a hybrid kernel, it is capable of loading and unloading code as microkernels do and runs alsmost exclusively in supervisor mode as monolithic kernels do.

Linux kernels are stored in a compression form, there are 2 types of compressed kerneltypes, both compressed with gzip:

The maximum allowed kernelsize for a zImage is 512kb, there is no limit on bzImage.

3.0 and up

The Linux kernel has a convention about version numbering. A version looks like: linux-A.B.C.tar.xz.

xz

The recent Linux kernels are compressed with in xz format when downloading from source. To decompress run:

$ tar xvf linux-<>.tar.xz

If you would need to compress:

$ tar -Jcvf <archive>.tar.xz <folder>

Kernel modules

Linux kernel modules are object files stored as .ko extension. There are compiled by a C compiler but not linked into a complete executable. Kernel modules can be loaded and unloaded into the kernel to extend its functionality when needed. Every kernel has its own set of modules.

Modules are stored in /lib/modules/.

Type of modules:

Type Function
block Modules for a few block-specific devices such as RAID or IDE
cdrom Device drivers for nonstandard CD-ROM drivers
fs Drivers for filesystems
ipv4 IP processing
misc Anything that does not fit in the others
net Network interface driver modules
scsi Contains driver modules for the SCSI controller
video Drivers for video adapters

Compiling a Linux kernel

A common location to store and unpack kernel sources is: usr/src.

Cleaning the kernel

To clean the kernel, which deletes most generated files, run:

$ make clean

To delete the current configuration and all generated files (a good idea before building/configuring):

$ make mrproper

This will delete the main configuration file.

To remove editor backup files, patch leftover files and the like:

$ make distclean

Create a .config file

In order to configure the kernel you will need to create a .config file. There are over 500 possible optionts you can write in that file. An option is for example telling the kernel to install an option directly into the kernel rather then building it as a module. Some options must be compiled as a module, some only directly in the kernel.

An example of this file:

CONFIG_X86=y
CONFIG_MMU=y
CONFIG_64BIT=y

It is strongly discouraged to edit file file manually instead use the make functionality. There are 4 targets that will help you to configure a .config file.

make defconfig

Makes the default config file. Most of the time this is not what you want but makes a good start point.

make config

This is a straigtforward approach, its biggest advantage is that you do not need a full-screen display, it can be used on systems with very limited display capabilities. The system will present a sequential list of questions regarding kernel options. There are a lot of questions and you can not go back or forth. This method is very error prone.

make menuconfig

This will create a text-mode window based on the ncurses library (interactive). You are able to switch between options. It is a nice TUI. If you would prefer a dark-themed one run:

$ make nconfig
make xconfig

Will show a graphical user interface based on QT if you prefer GTK run:

$ make gconfig
make oldconfig

Will move your .config file to .config.old and a new .config will be created.

Any other make configuration options will use a .config if available and will not start with a clean one.

Compiling

Use the following make seqquence:

$ make clean
$ make zImage/bzImage # or just run make 
# make modules
# make modules_install

Add -jX to make where X is amount of cores to compile with multithreading.

After compilation you will find the kernel in /usr/src/linux/arch/x86_64/boot (if you compiled a 64 bit kernel). Modules will be installed in /lib/modules/kernel-version.

Install the new kernel

You will need to copy the kernel image to your boot folder and call it vmlinuz-X where X is the version number. After that configure your bootloader so it will load the custom kernel.

Initrd

A RAM disk is a chunck of memory that the kernel sees as if it were a disk. It can be mounted like any other disk. These RAM disks hold scripts and modules to aid the boot process. Instead of that the kernel contains all the modules, the modules will be loaded from this RAM disk so it is much cleaner and you don't need to build another kernel everytime. The Initial Ram Disk is mounted as the root filesystem, so programs can be load from it and modules can easily be loaded. After this step the real root file system will be mounted.

There are a few possible scripts to create your own Ram disk file:

Example of making your own initramfs:

# mkinitcpio -k <path/to/kernel> -g <output-file>
Manually make initrd

So initrd are compressed archives that contain the files of a minimal root file system. This file system contains modules, scripts and some additional binaries required to allow the kernel to properly continue its boot.

What type of archive to use depends on the distribtution and kernel version. Older use tar newer use cpio.

To manually create the RAM disks you need a block device, where you will put the files the file system need. Suitable block devices are:

The best option is a RAM disk because it is very fast. To check the available RAM disk devices:

$ ls -la /dev/ram0

The number of RAM disks available is set by an option in the kernel configuration: CONFIG_BLK_DEV_RAM_COUNT.

Next create an empty file system:

# mke2fs -m0 /dev/ram0 300 # 300 is the size, you can make it smaller

Next mount the device:

# mount -t ext2 /dev/ram0 /mnt

Now the stub for the console device needs to be created. This will be the device node that will be used when the initrd is active.

# mkdir /mnt/dev
# mknod /mnt/dev/tty1 c 4 1

The block device has been created, now you need to copy all the files: modules, scripts, binaries to here. This is most likely the content of /boot/ and lib/modules. The most important file to copy over is /linuxrc this is the connection between the initrd and the kernel. The kernel will search for this file first. A good move is to link the file to /bin/sh:

# ln -s /bin/sh /mnt/linuxrc
# chmod +x /mnt/linurc # important

Now unmount. and copy the disk to a file:

# dd if=/dev/ram0 bs=1k count=300 of=/boot/initrd

Deallocate the RAM disk:

# freeramdisk /dev/ram0

This created initrd is very minimal, real disks will contain a more complex /linuxrc boot file, thatt load modules, mounts the real filesystem etc.

Patch the kernel

To patch the kernel you need a patch file this can be created with the diff command. the patch command can be used to update the old version to a new version with the help of the patch file.

Some options:

# patch -s # silent
# patch -E # remove output files that are empty
# patch -R # rollback, reverse

DKMS

Dynamic Kernel Module Support is a framework capable of automatically compile and/or install kernel modules for every kernel available on the system. Both kernel and kernel modules may be upgraded independent of each other.

After the system upgrades a kernel, a hook will take care of deciding whether any kernel modules need to be compiled and/or installed for the new kernel.

The command:

$ dkms

Can be used to add/remove modules to or from the source tree.

Dracut

Just like DKMS dracut is an event-driven tool that will generate new initramfs images whenever it needs to, for example after downloading a new kernel via the package manager.

Kernel runtime management and troubleshooting

Manipulating modules

To list all modules:

$ lsmod

To load a module it modprobe is generaly used, because it keeps up if a module is need to load another module. However in the past insmod was used to load, this tool however doesn't look at if a module needs other modules.

To unload you can use modprobe also but also rmmode.

Modules are load from /lib/modules.

# modprobe -rv <module.ko>

To get information about a module run:

$ modinfo <module.ko>

Configuring modules

A commonly used configuration file is /etc/modules.conf.

Here you can specify some options like:

Option Function
keep Cause the default paths to be retained and added to any paths specified
depfile=<path> This directive overrides the default location for the modules dependency files
path= <path> Additional directory to search for modules
option <module> <option> Options for the modules
alias <alias2> Aliases can be used to associate generic name with a specific module
pre-installed <module> <command> This causes the execute a specifc shell command prior insertion of a module
install <module> <command> Override the default module-insertion command
post-install <module> <command> Execute a shell command after insertion of a module
pre-remove <module> <command> Execute command prior removing
post-remove <module> <command> Execute command post removing

Module dependency file

In order for modprobe to determine module dependencies and install modules automatically it will first read the file /lib/modules/<kernel-version>/modules.dep.

This file contains all the modules that are available on the system listed with the full path and .ko extension.

When the systeem boots, most distribution will keep modprobe and this file in sync by the command:

# depmod -a

This command generates the file /lib/modules/<kernel>/modules.dep by reading all the modules so modprobe knows in which order modules needs to be loaded/unloaded.

kmod vs kerneld

Both kmod and kerneld provide dynamic loading of kernel-modules. kerneld is a daemon and kmod a thread in the kernel itself. The communication with kerneld is done through System V IPC. kmod directly operates from the kernel, it is considered a module.

kmod has replaced kerneld.

They both use modprobe to dynamically load/unload modules.

To enable the use of modules CONFIG_MODULES needs to be set in the kernel config.

Summary

Instead of using all the make commands, you can use the general:

# make all # creates image, modules
/proc

The /proc file system is a pseudo file system, its content is directly presented by the kernel. You can extract hardware information, network, host parameters, memory and performance information from it. It is possible to change the values in /proc at runtime, this however is only applicable to the current boot.

/

The most important directories are /bin, /var, /srv, /tmp, /lib.., /mnt, /opt, /boot, /sbin and /etc.

The /usr directory mimics the root file system and is used by users to override and enhance the system. For example /usr/bin instead of /bin.

/usr/local this contains another mimic of the root file system and is used for local system specifc files.

Tools to troubleshoot

To trace system calls and signals use:

# strace <command>

By default strace reports the name of the system call, arguments and the return value.

To connect to a running process use the -p flag, in combination with the -f flag you can connect to a running daemon.

To print strings from non-text files use:

$ strings

Another tool similar to strace is:

$ ltrace

System Startup

$  

The Linux boot process

Here I will first show you the code how it really starts

First the header.S file kicks in:

arch/x86/boot/header.S

jump to start_of_setup to kick off.

start_of_setup:
# Force %es = %ds
    movw    %ds, %ax
    movw    %ax, %es
    cld

# Apparently some ancient versions of LILO invoked the kernel with %ss != %ds,
# which happened to work by accident for the old code.  Recalculate the stack
# pointer if %ss is invalid.  Otherwise leave it alone, LOADLIN sets up the
# stack behind its own code, so we can't blindly put it directly past the heap.

    movw    %ss, %dx
    cmpw    %ax, %dx    # %ds == %ss?
    movw    %sp, %dx
    je  2f      # -> assume %sp is reasonably set

    # Invalid %ss, make up a new stack
    movw    $_end, %dx
    testb   $CAN_USE_HEAP, loadflags
    jz  1f
    movw    heap_end_ptr, %dx
1:  addw    $STACK_SIZE, %dx
    jnc 2f
    xorw    %dx, %dx    # Prevent wraparound

2:  # Now %dx should point to the end of our stack space
    andw    $~3, %dx    # dword align (might as well...)
    jnz 3f
    movw    $0xfffc, %dx    # Make sure we're not zero
3:  movw    %ax, %ss
    movzwl  %dx, %esp   # Clear upper half of %esp
    sti         # Now we should have a working stack

# We will have entered with %cs = %ds+0x20, normalize %cs so
# it is on par with the other segments.
    pushw   %ds
    pushw   $6f
    lretw
6:

# Check signature at end of setup
    cmpl    $0x5a5aaa55, setup_sig
    jne setup_bad

# Zero the bss
    movw    $__bss_start, %di
    movw    $_end+3, %cx
    xorl    %eax, %eax
    subw    %di, %cx
    shrw    $2, %cx
    rep; stosl

# Jump to C code (should not return)
    calll   main

# Setup corrupt somehow...
setup_bad:
    movl    $setup_corrupt, %eax
    calll   puts
    # Fall through...

    .globl  die
    .type   die, @function
die:
    hlt
    jmp die

    .size   die, .-die

    .section ".initdata", "a"
setup_corrupt:
    .byte   7
    .string "No setup signature found...\n"

arch/x86/boot/main.c

void main(void)
{
    /* First, copy the boot header into the "zeropage" */
    copy_boot_params();

    /* Initialize the early-boot console */
    console_init();
    if (cmdline_find_option_bool("debug"))
        puts("early console in setup code\n");

    /* End of heap check */
    init_heap();

    /* Make sure we have all the proper CPU support */
    if (validate_cpu()) {
        puts("Unable to boot - please use a kernel appropriate "
             "for your CPU.\n");
        die();
    }

    /* Tell the BIOS what CPU mode we intend to run in. */
    set_bios_mode();

    /* Detect memory layout */
    detect_memory();

    /* Set keyboard repeat rate (why?) and query the lock flags */
    keyboard_init();

    /* Query Intel SpeedStep (IST) information */
    query_ist();

    /* Query APM information */
#if defined(CONFIG_APM) || defined(CONFIG_APM_MODULE)
    query_apm_bios();
#endif

    /* Query EDD information */
#if defined(CONFIG_EDD) || defined(CONFIG_EDD_MODULE)
    query_edd();
#endif

    /* Set the video mode */
    set_video();

    /* Do the last things and invoke protected mode */
    go_to_protected_mode();

{

/arch/x86/boot/pm.c

void go_to_protected_mode(void)
{
    /* Hook before leaving real mode, also disables interrupts */
    realmode_switch_hook();

    /* Enable the A20 gate */
    if (enable_a20()) {
        puts("A20 gate not responding, unable to boot...\n");
        die();
    }

    /* Reset coprocessor (IGNNE#) */
    reset_coprocessor();

    /* Mask all interrupts in the PIC */
    mask_all_interrupts();

    /* Actual transition to protected mode... */
    setup_idt();
    setup_gdt();
    protected_mode_jump(boot_params.hdr.code32_start,
                (u32)&boot_params + (ds() << 4));
}

arch/x86/boot/pmjump.S

/*
 * void protected_mode_jump(u32 entrypoint, u32 bootparams);
 */
GLOBAL(protected_mode_jump)
    movl    %edx, %esi      # Pointer to boot_params table

    xorl    %ebx, %ebx
    movw    %cs, %bx
    shll    $4, %ebx
    addl    %ebx, 2f
    jmp 1f          # Short jump to serialize on 386/486
1:

    movw    $__BOOT_DS, %cx
    movw    $__BOOT_TSS, %di

    movl    %cr0, %edx
    orb $X86_CR0_PE, %dl    # Protected mode
    movl    %edx, %cr0

    # Transition to 32-bit mode
    .byte   0x66, 0xea      # ljmpl opcode
2:  .long   in_pm32         # offset
    .word   __BOOT_CS       # segment
ENDPROC(protected_mode_jump)

    .code32
    .section ".text32","ax"
GLOBAL(in_pm32)
    # Set up data segments for flat 32-bit mode
    movl    %ecx, %ds
    movl    %ecx, %es
    movl    %ecx, %fs
    movl    %ecx, %gs
    movl    %ecx, %ss
    # The 32-bit code sets up its own stack, but this way we do have
    # a valid stack if some debugging hack wants to use it.
    addl    %ebx, %esp

    # Set up TR to make Intel VT happy
    ltr %di

    # Clear registers to allow for future extensions to the
    # 32-bit boot protocol
    xorl    %ecx, %ecx
    xorl    %edx, %edx
    xorl    %ebx, %ebx
    xorl    %ebp, %ebp
    xorl    %edi, %edi

    # Set up LDTR to make Intel VT happy
    lldt    %cx

    jmpl    *%eax           # Jump to the 32-bit entrypoint
ENDPROC(in_pm32)

arch/x86/boot/compressed/head_32.S

    __HEAD
ENTRY(startup_32)
    cld
    /*
     * Test KEEP_SEGMENTS flag to see if the bootloader is asking
     * us to not reload segments
     */
    testb   $KEEP_SEGMENTS, BP_loadflags(%esi)
    jnz 1f

    cli
    movl    $__BOOT_DS, %eax
    movl    %eax, %ds
    movl    %eax, %es
    movl    %eax, %fs
    movl    %eax, %gs
    movl    %eax, %ss

...

/*
 * Do the extraction, and jump to the new kernel..
 */
                /* push arguments for extract_kernel: */
    pushl   $z_output_len   /* decompressed length, end of relocs */

    movl    BP_init_size(%esi), %eax
    subl    $_end, %eax
    movl    %ebx, %ebp
    subl    %eax, %ebp
    pushl   %ebp        /* output address */

    pushl   $z_input_len    /* input_len */
    leal    input_data(%ebx), %eax
    pushl   %eax        /* input_data */
    leal    boot_heap(%ebx), %eax
    pushl   %eax        /* heap area */
    pushl   %esi        /* real mode pointer */
    call    extract_kernel  /* returns kernel location in %eax */
    addl    $24, %esp

/*
 * Jump to the extracted kernel.
 */
    xorl    %ebx, %ebx
    jmp *%eax

arch/x86/boot/compressed/head_32.S

asmlinkage __visible void *extract_kernel(void *rmode, memptr heap,
                  unsigned char *input_data,
                  unsigned long input_len,
                  unsigned char *output,
                  unsigned long output_len)
{
    const unsigned long kernel_total_size = VO__end - VO__text;
    unsigned long virt_addr = LOAD_PHYSICAL_ADDR;

    /* Retain x86 boot parameters pointer passed from startup_32/64. */
    boot_params = rmode;

    /* Clear flags intended for solely in-kernel use. */
    boot_params->hdr.loadflags &= ~KASLR_FLAG;

    sanitize_boot_params(boot_params);

    if (boot_params->screen_info.orig_video_mode == 7) {
        vidmem = (char *) 0xb0000;
        vidport = 0x3b4;
    } else {
        vidmem = (char *) 0xb8000;
        vidport = 0x3d4;
    }

    lines = boot_params->screen_info.orig_video_lines;
    cols = boot_params->screen_info.orig_video_cols;

    console_init();
    debug_putstr("early console in extract_kernel\n");

    free_mem_ptr     = heap;    /* Heap */
    free_mem_end_ptr = heap + BOOT_HEAP_SIZE;

    /* Report initial kernel position details. */
    debug_putaddr(input_data);
    debug_putaddr(input_len);
    debug_putaddr(output);
    debug_putaddr(output_len);
    debug_putaddr(kernel_total_size);

    /*
     * The memory hole needed for the kernel is the larger of either
     * the entire decompressed kernel plus relocation table, or the
     * entire decompressed kernel plus .bss and .brk sections.
     */
    choose_random_location((unsigned long)input_data, input_len,
                (unsigned long *)&output,
                max(output_len, kernel_total_size),
                &virt_addr);

    /* Validate memory location choices. */
    if ((unsigned long)output & (MIN_KERNEL_ALIGN - 1))
        error("Destination physical address inappropriately aligned");
    if (virt_addr & (MIN_KERNEL_ALIGN - 1))
        error("Destination virtual address inappropriately aligned");
#ifdef CONFIG_X86_64
    if (heap > 0x3fffffffffffUL)
        error("Destination address too large");
    if (virt_addr + max(output_len, kernel_total_size) > KERNEL_IMAGE_SIZE)
        error("Destination virtual address is beyond the kernel mapping area");
#else
    if (heap > ((-__PAGE_OFFSET-(128<<20)-1) & 0x7fffffff))
        error("Destination address too large");
#endif
#ifndef CONFIG_RELOCATABLE
    if ((unsigned long)output != LOAD_PHYSICAL_ADDR)
        error("Destination address does not match LOAD_PHYSICAL_ADDR");
    if (virt_addr != LOAD_PHYSICAL_ADDR)
        error("Destination virtual address changed when not relocatable");
#endif

    debug_putstr("\nDecompressing Linux... ");
    __decompress(input_data, input_len, NULL, NULL, output, output_len,
            NULL, error);
    parse_elf(output);
    handle_relocations(output, output_len, virt_addr);
    debug_putstr("done.\nBooting the kernel.\n");
    return output;
}

arch/x86/kernel/head_32.S

ENTRY(startup_32)
    movl pa(initial_stack),%ecx

    /* test KEEP_SEGMENTS flag to see if the bootloader is asking
        us to not reload segments */
    testb $KEEP_SEGMENTS, BP_loadflags(%esi)
    jnz 2f

/*
 * Set segments to known values.
 */
    lgdt pa(boot_gdt_descr)
    movl $(__BOOT_DS),%eax
    movl %eax,%ds
    movl %eax,%es
    movl %eax,%fs
    movl %eax,%gs
    movl %eax,%ss
2:
    leal -__PAGE_OFFSET(%ecx),%esp

....

There are 7 phases in the boot process

  1. Kernel loader loading, setup and execution
  2. Register setup
  3. Kernel decompression
  4. Kernel and memory initialization
  5. Kernel setup
  6. Enable of remaining CPU
  7. Init process creation

The kernel final step is to launch one of the following in the order that it is described:

If none of the above succeed, the kernel panics

Init

Init is the parent of all processes, at the moment init is of course systemd. The processes init starts is defined by a runlevel. A runlevel is a configuration that defines which processes are started.

NOTE: You will notice that there is some difference with each distribtution, runlevels are convention, they do not have a real linux standard.

Runlevels:

Level Function
0 Halt
1 Single user mode
2 Mulituser without NFS
3 Full multiuser mode
4 Custom
5 X11, graphical
6 Reboot
7 Custom
8 Custom
9 Custom
s/S Brings system in singel user mode
A,B,C On demand runlevels

init will read the file /etc/inittab to determine what it should do, you can define runlevel and which process / action should be executed by each runlevel.

Systemd

To see all overrides

systemd-delta
systemctl get-default
systemctl list-units --type=target
systemctl list-unit-files --type=target

Change system state at runtime:

systemctl isolate <target>

Chkconifg

Another tool to manage startup process / scripts is chkconfig.

System recovery

GRUB

Press shift key right to enter the GRUB shell after BIOS/UEFI initialization, enter help command for available commands.

commands to be used: update-grub and grub-mkconfig.

config file: /boot/grub/grub.cfg, do not edit this file directly use the directory: /etc/grub.d/

Booting is as follows: BIOS access the initial sector on the hard drive, called MBR and load data found there in memory. The execution is transfered to it. If GRUB is used, GRUB will be on the MBR and tries to load stage 2. Stage 2 presents a menu to the end-user.

With grub-install you can install stage 1 to the MBR.

press 'e' in the menu on a selected target to edit the boot parameters of that specific target.

fsck

fsck will check the filesystem based on /etc/fstab.

Errors:

Error Meaning
0 No errors
1 Filesystems errors
2 System should be rebooted
4 Filesystem errors left uncorrected
8 Operational error
16 Usage or syntax error
128 Shared library error

Alternative bootloaders

Filesystems and Devices

Operating the Linux file system

create a filesystem example:

$ mkfs -t ext2 /dev/sda1

Using systemd to mount a device on boot:

[Unit]
Description=Data for User
[Mount]
What=/dev/sda2
Where=/home/user/data
Type=ext4
Options=defaults
[Install]
WantedBy=multi-user.target

swap:

mkswap and swapon

$  

Maintaining a Linux file system

change the frequency of checks with tune2fs, dumpe2fs will provide information regarding parameters and options in the superblock, debuggin can be done with debugfs or badblocks

Creating snapshots with btrfs

btrfs subvolume create

btrfs subvolume snapshot file file

zfs -> zpool

Creating and configure filesystem options

$  

mkisofs dd mke2fs

writing to cdrom, mkisofs makes image and cdrecord writes to cdrom

Advanced Sorage Device Administration

Configure RAID

hardware raid vs software raid

To configure RAID one can mdadm , another option is via /etc/raidtab.

$ mdadm -C /dev/md0 -l raid5 -n 3 /dev/partition1 /dev/partition2 /dev/partition3

Adjust storage device access

$  

Logical volume manager

$  

Networking Configuration

$  

Basic network configuration

$  

Advanced network configuration and troubleshooting

$  

Troubleshooting network issues

$  

System Maintenance

$  

Make and install from source

$  

Backup operations

$  

Notify users on syste-related issues

$  

Domain Name Server

$  

Basic DNS server configuration

/etc/named.conf /etc/bind/named.conf

rndc

Create and maintain DNS zones

$  

Secure DNS server

$  

HTTP services

$  

Basic Apace configuration

$  

Apache configuration for HTTPS

$  

Implementing Squid

$  

Implementing Nginx

$  

File Sharing

$  

SAMBA

$  

Configure NFS server

$  

Network Client Management

$  

DHCP configuration

$  

PAM authentication

$  

LDAP client usage

$  

Configure OpenLDAP server

$  

E-mail Services

$  

Using e-mail servers

$  

Managing e-mail delivery

$  

Managing mailbox access

$  

System security

$  

Configure a router

$  

Managing FTP servers

$  

Secure shell

$  

Security tasks

$  

OpenVPN

$  
1