通过《 The Linux Command Line》入门linux命令行

原文发于我的独立博客：通过《The Linux Command Line》入门linux命令行

此书网站：The Linux Command Line ，它是免费的。

它有中文版，而且还有一个好听的名字：快乐的 Linux 命令行

学习有两种方法，一种是系统的学，一种是根据实际需求来学。两种各有优缺点，前一种，知识不会有缺漏，对日后的融会贯通和触类旁通有好处，但是这样慢，而且有可能不如针对性学习的精神集中。后一种，只找需求的学，能够任务完成就行，这样快，它的缺点就是前一种的优点。显然通看整本书属于第一种。

作者写的很好，娓娓道来，讲的很清楚，就像一本故事书一样。我很喜欢这样的书。作者比较幽默，一些用词看了会让人发笑。

这本书的内容很浅，真的只是入门。

正如作者所说，这是一场linux的journey(旅行)，就把看书当旅行吧。

以下是随手记的，只适合自己看。

命令

• date
• cal-calendar
• df-current amount of free space on your disk drives
• free-display the amount of free memory
• exit-closing the terminal emulator window

2 - Navigation

• pwd - Print name of current working directory
• cd - Change directory

cd             home directory
cd -           previous working directory
cd ~user_name  home directory of user
.              点号代表当前文件夹

• ls - List directory contents

ls -a   all files
ls -l   long format
ls -lt  sort the result by the file's modification time
ls -lt --reverse

• file – Determine file type

• less – View file contents

  和cat的区别？参考

4 - Manipulating Files And Directories

• cp

  如果文件已经存在，直接覆盖，没有提示，要提示用-i

• mv

  move (rename) files

• mkdir

mkdir dir1 dir2

• rm – Remove Files And Directories

rm -r playground    # 递归删除

• ln – Create Links

ln myfile hard       # hard link
ln -s myfile soft    # symbolic link

软链接”和“硬链接”的区别：硬链接指向磁盘区块，软链接指向绝对路径。

hard links的缺点：

1.cannot span physical devices.

2.cannot reference directories, only files.

5 - Working With Commands

• type – Indicate how a command name is interpreted

tsc@tsc:~$ type ls
ls is aliased to `ls --color=auto'  # alias

tsc@tsc:~$ type cd
cd is a shell builtin               # builtin

tsc@tsc:~$ type cp
cp is /bin/cp                       # executable program

• which – Display An Executable’s Location

tsc@tsc:~$ which cp
/bin/cp

只用于exe

• help – Get help for shell builtins

  注意是shell builtin

tsc@tsc:/bin$ help cd
cd: cd [-L|[-P [-e]] [-@]] [dir]
    Change the shell working directory.

理解注释：[]是可选，|是互斥。

• --help – Display Usage Information

tsc@tsc:~$ mkdir --help

• man – Display a command’s manual page

没有example

tsc@tsc:~$ man ls

• apropos – Display a list of appropriate commands
• info – Display a command’s info entry
• whatis – Display a very brief description of a command
• alias – Create an alias for a command

起别名前先检查是否存在，用type：

tsc@tsc:~$ type test
test is a shell builtin

使用：

alias name='string' # name后不能有空格
tsc@tsc:~$ alias foo='cd /usr; ls; cd -'
tsc@tsc:~$ foo

终端关闭后作用就消失了。

去除：

tsc@tsc:~$ unalias foo

• trick:多条命令于一行

command1; command2; command3...
tsc@tsc:~$ cd /usr; ls

• man .info .–help的用法和区别

  help最简单，info最详细，man在两者之间。

6 – Redirection

• cat - Concatenate files

cat file1 # 输出文件到屏幕，没有分页
cat > file.txt # cat没有参数时接受stdin
this is test.
               # ctrl+d 结束
# < 接受文件输入
tsc@tsc:~$ cat < lazy_dog.txt

• sort - Sort lines of text
• uniq - Report or omit repeated lines

# -d 显示重复的
tsc@tsc:~$ ls /bin /usr/bin | sort | uniq -d | less

• grep - Print lines matching a pattern

“global regular expression print”,find功能。

tsc@tsc:~$ ls /bin /usr/bin | sort | uniq | grep zip

• wc - Print newline, word, and byte counts for each file

联合使用看条目个数：

tsc@tsc:~$ ls /bin /usr/bin | sort | uniq | wc -l

• head - Output the first part of a file
• tail - Output the last part of a file
• tee - Read from standard input and write to standard output and files

这个名字很有意思，tee有三通管的意思，配合pipe使用，它的作用是从stdin读入，复制到stdout和文件。

# 一方面输出到ls.txt，一方面传给grep
tsc@tsc:~$ ls /usr/bin/ | tee ls.txt | grep zip

• I/O redirection 的作用

  I/O redirection allows us to change where output goes and where input comes from.

• >重定向

tsc@tsc:~$ ls -l /usr/bin/ > ls-output.txt
tsc@tsc:~$ > ls-output2.txt  # 创建一个空文件

要注意重复>时，原来的文件会被清空。

• >> 追加

tsc@tsc:~$ ls -l /usr/bin >> ls-output2.txt

文件不存在会新建。

• 输出error

tsc@tsc:~$ ls -l /bin/usr 2> ls-error.txt

我的问题：在不知道有没有error的情况下，>和2>要如何同时执行？

• 都输出到一个文件

有两种方法：

# 1.老版本
# 顺序很重要
tsc@tsc:~$ ls -l /bin/usr > ls-output.txt 2>&1

# 2.新，用&>
tsc@tsc:~$ ls -l /bin/usr &> ls-output.txt
tsc@tsc:~$ ls -l /bin/usr &>> ls-output.txt # 追加

• Pipelines

tsc@tsc:~$ ls -l /usr/bin | less

千万不要把>用成|，不然有些后果很严重。

• filters

tsc@tsc:~$ ls /bin /usr/bin | sort | less

7 – Seeing The World As The Shell Sees It

这节讲expansion， 就是*、~之类的，本质就是变量。

• echo – Display a line of text

It prints out its text arguments on standard output

tsc@tsc:~$ echo this is a test
this is a test

• Tilde Expansion

tsc@tsc:~/playground$ echo ~
/home/tsc

• Arithmetic Expansion

形式：

$((expression))

tsc@tsc:~/playground$ echo $((2+2))
4

• brace expansion

  括号中间不能有空格。

  用途：批量创建有顺序的文件。

• Parameter Expansion

• Command Substitution

  command的结果也可以作为expansion

echo $(ls)

• Double Quotes

  “$”, “” (backslash), and “`” (backquote) 失效，但是parameter expansion, arithmetic expansion, and command substitution 仍有效。

注意区别：

echo $(cal)
echo "$(cal)"

空格、tab、换行符都被用来分割，双引号则抑制。所以前一个命令不会换行，后一个会。

• Single Quotes

  抑制所有expansions

• escape character

和双引号一起使用，抑制部分符号

8 – Advanced Keyboard Tricks

这节讲键盘技巧，提高效率。主要是command line和history的技巧。command line的：移动，删除。history的上下一个命令。

• clear – Clear the screen

• history – Display the contents of the history list

• Cursor Movement

• completion

  补全

• history

  history expansion-使用history中1209行的命令：

tsc@tsc:~$ !1209

• History Commands快捷键

9 – Permissions

思考：多用户为什么存在？想想计算中心。

多用户存在要解决的问题：

1.一个用户的错误操作不能使计算机crash

2.一个用户不能影响其他用户

• 几个相关文件夹

/etc/passwd
User accounts .

/etc/group
groups

/etc/shadow
user's password

• rwx

Owner Group World

• id – Display user identity

$ id
uid=1000(tsc) gid=1000(tsc) groups=1000(tsc),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),113(lpadmin),128(sambashare)

# root
# id
uid=0(root) gid=0(root) groups=0(root)

• chmod – Change File Mode
• File Modes中八进制的妙用

$ chmod 600 foo.txt

• umask – Set Default Permissions

• su – Run A Shell With Substitute User And Group IDs

  默认是superuser

$ su -
exit

• sudo – Execute A Command As Another User

  纠正自己的认识，su,sudo是改变用户的，不一定是super，默认才是。

• chown – Change File Owner And Group

• chgrp – Change Group Ownership

• passwd - change user password

10 – Processes

进程

Processes are how Linux organizes the different programs waiting for their turn at the CPU.

• ps - report a snapshot of the current processes.

• top - display Linux processes

• & - Putting A Process In The Background

• jobs - list the jobs that have been launched from our terminal.

• fg - Returning A Process To The Foreground

  为什么要回到前台来？后台的程序不受terminal的控制。

• Ctrl-z - Stopping (Pausing) A Process

  注意和ctrl-c的区别，c是结束。

• bg - resume the program’s execution in the background

• kill - “kill” processes

• killall - kill processes by name

• halt

• poweroff

• reboot

• shutdown

Part 2 – Configuration And The Environment

11 – The Environment

什么是environment？

configuration的作用：store program settings

• printenv - print all or part of environment

# 所有environment
tsc@tsc:~$ printenv | less

# 某一个environment
tsc@tsc:~$ printenv USER

tsc@tsc:~$ echo $HOME

printenv显示的一些常见变量：

SHELL
PATH   # 所有的，最新的

• set

  will show both the shell and environment variables

• printenv和set的区别

  printenv只显示environment variables，set显示shell和environment variables.

• 追加

PATH=$PATH:$HOME/bin

• export

  tells the shell to make the contents of PATH available to child processes of this shell.

• 启动顺序

Login Shell Sessions ：

/etc/profile     # global
~/.bash_profile
~/.bash_login    # 如果上面那个不存在,读这个
~/.profile       # 如果上面那个不存在,读这个

.profile文件中有PATH，并且将$HOME/bin添加上了，所以启动系统后$HOME/bin中的命令是直接可以用的。$HOME是用户目录。

Non-Login Shell Sessions ：

/etc/bash.bashrc # global
~/.bashrc        # user's

• Login和Non-Login的区别？

  参考，

• PATH的作用

  命令从中找。

• 改环境变量时改哪个文件？

.profile：

add directories

define additional environment variables

.bashrc：

everything else

• Text Editors

有两种：

1.graphical ：gedit

2.text based ：nano, vi, emacs

• nano

ctrl-x : exit
ctrl-o : save, write out
#      : comments

• 使.bashrc生效

首先要知道原理，启动流程，在session一开始才会read .bashrc，所以需要重启terminal才会生效。当然，可以强行重读.bashrc，用source命令：

source .bashrc

12 – A Gentle Introduction To vi

• why vi?

  always available

  lightweight and fast

• vi 操作

:q
:q!   # 强制退出
i     # insert
esc   # to command mode
:w    # save ,有:的命令叫ex command

# move
h, j, k, l
ctrl-f/b
numberG
gg # 第一个字符
G last line of the file
0 (zero) 行首
^ To the first non-whitespace character on the current line.
$ end of current line
w beginning of next word or punctuation
W ignoring punctuation
b beginning of previous word or punctuation
B ignoring punctuation


# edit
u    # undo last change
i    # insert
# append
a    # append, i是前插
A    # 直接到最后append
# opening a line
o    # 新建一行在下面
O    # 新建一行在上面

# delete
x    # current character
3x
dd   # current line,d有cut的作用
dw   # 
dW   # ignoring punctuation
d$   # current cursor to end
d0   # current cursor to beginning
dG   # current cursor to end of file
d20G # current cursor to 20 line，不是要删除20行
5dd  # 5行
# Cutting, Copying, And Pasting
p    # 小写粘贴在后一行（dd），普通在光标前
P    # 大写粘贴在前一行（dd），普通在光标后

p    # paste after cursor
P    # upper case, paste before cursor
  # copy
yy   # copy current line
5yy  # copy 5 line
yW   # ignoring punctuation
yw   # 
y$   # current cursor to end of line
y0   # current cursor to beginning of line
y^   # current cursor to first non-whitespace char
yG  # current cursor to last line of file
y20G # current cursor to 20th line

# redo,undo
u  # undo
ctrl+r # redo

J    # join line

# search, replace
fa   # 行内搜索a，按;继续搜索
/    # 文件内搜索，n继续
:%s/line/Line/g    # 替换
:%s/line/Line/gc   # 有提醒的替换
%代表对所有内容进行操作
:1,2s/line/Line/gc # 对1-2行进行操作

Ctrl-e, Ctrl-y # Scroll down and scroll up

The leading tilde characters (”~”) indicate that no text exists on that line.

• modal editor

  command mode

• Switching case of characters 大小写转换

You can change the case of text:

Toggle case “HellO” to “hELLo” with g~ then a movement.

Uppercase “HellO” to “HELLO” with gU then a movement.

Lowercase “HellO” to “hello” with gu then a movement.

Alternatively, you can visually select text then press ~to toggle case, or U to convert to uppercase, or u to convert to lowercase.

• 常规操作

# 到行尾
$  反：0
A  反：I
Word movement: w, e, b

Linux vi and vim editor: Tutorial and advanced features

13 – Customizing The Prompt

• PS1-prompt string one

就是这个：

tsc@tsc:~$

更改它：

tsc@tsc:~$ PS1="<u@h w>$ "
<tsc@tsc ~>$ 
u   # username of the current user.
h   # Hostname of the local machine minus the trailing domain name.
w   # Name of the current working directory.
$   # This displays a “$” character unless we have superuser privileges. In that case, it displays a “#” instead.

改颜色：

通过escape code 实现，escape code 的开头是八进制的033 ，例子：

33[0;30m
33是开头
0   是optional character attribute，0代表text color是normal，1是bold
30m 是instruction
tsc@tsc:~$ PS1="[33[0;31m]<u@h w>$[33[0m] "
33[0m   # 代表结束修改

Part 3 – Common Tasks And Essential Tools

要知道package是什么，如何install, remove, update

14 – Package Management

linux的哪个发行版才是最好的？考虑的因素不是界面，而是：

1.packaging system

2.vitality of the distribution’s support community

• packaging 的两大阵营

Debian “.deb” camp

Red Hat “.rpm” camp

Packaging System     Distributions (Partial Listing)
Debian Style (.deb)  Debian, Ubuntu, Linux Mint, Raspbian
Red Hat Style (.rpm) Fedora, CentOS, Red Hat Enterprise Linux, OpenSUSE

• package files

• shared libraries

  软件都是有依赖的，被依赖的就放在这里

  dependency

• package tool

  分为high, low-level两种，

  high:metadata search, dependency

  low-level: installing, removing

  Red Hat系列使用同一low-lvel program(rpm), high-level则不同，yum使用者：Fedora, Red Hat Enter, CentOS.

Distributions	Low-Level Tools	High-Level Tools
Debian-Style	dpkg	apt-get, aptitude
Fedora, Red Hat, Enterprise Linux, CentOS	rpm	yum

- 查找和安装package的方法
- What is the difference between apt-get update and upgrade?

15 – Storage Media

• 什么是mounting?

attaching the device to the file system tree.

• mount – Mount a file system

没有参数时，显示已经mounted的文件系统。

如何看？device on mount_point type file_system_type (options).

• free - Display amount of free and used memory in the system

• partition number 是什么？

• Unable to enumerate USB device under virtualbox

  usb3.0插的3.0的口上在vm中读不出来，改插在2.0的口上就可以，真是。。。

• 磁盘、分区及Linux文件系统 [Disk, Partition, Linux File System]

  inode的作用：实现文件存储的关键。

  ​

• tail的妙用

# ubuntu
tail -f /var/log/syslog

实时观察系统在做什么。

• partition layout 是什么？
• IDE、SATA、SCSI、SAS、FC、SSD硬盘类型介绍
• fsck - check and repair a Linux filesystem

16 – Networking

• IP Addresses, Host Names, and Domain Names

  明白host name与domain name的区别。

• ping, ping6 - send ICMP ECHO_REQUEST to network hosts

• traceroute - print the route packets trace to network host

• ip - show / manipulate routing, devices, policy routing and tunnels

  DHCP (Dynamic Host Configuration Protocol)

• netstat - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

• ftp — Internet file transfer program

  ftp是明文传输。

• Wget - The non-interactive network downloader.

• ssh

  rlogin and telnet 和ftp有一样的缺陷。

  ssh解决了两大问题：First, it authenticates that the remote host is who it says it is (thus preventing so-called “man in the
  middle” attacks), and second, it encrypts all of the communications between the local and
  remote hosts.

ssh root@ip -p 端口号

• windows下的ssh软件

  PuTTY

17 – Searching For Files

找文件。

• locate - find files by name

  locate查找的数据库是如何更新的？由updatedb 程序更新，可以手动更新。

• locate vs find: usage, pros and cons of each other

• find - search for files in a directory hierarchy

$ find ~ -type f | wc
# 文件名有jpg，大小大于1M
$ find ~ -type f -name "*.jpg" -size +1M | wc -l

• test

$ find ~ ( -type f -not -perm 0600 ) -or ( -type d -not -perm 0700 )

顺序很重要。

• action

把查找的结果作为命令的输入参数

$ find ~ -type f -name 'foo*' -exec ls -l '{}' ';'
$ find ~ -type f -name 'foo*' -exec ls -l '{}' +

$ find ~ -type f -name 'foo*' -print | xargs ls -l

{} is a symbolic representation of the current pathname

semicolon is a required delimiter indicating the end of the command.

+：the system only has to execute the ls command once

• 文件名中空格的处理

$ find ~ -type f -name '*.jpg' -print0 | xargs --null ls -l

• touch - change file timestamps

  文件不存在时会创建。

• stat - display file or file system status

  which is a kind of souped-up version of ls.

18 – Archiving And Backup

管理大量的文件-压缩。

• compression archiving区别？

本节讲3个内容：

compression

archiving

synchronization

• gzip, gunzip, zcat - compress or expand files

gzip压缩，gunzip解压。

gzip会替换原来的文件。

• zcat - equivalent to gunzip with the -c option
• zless
• bzip2, bunzip2 - a block-sorting file compressor
• linux压缩和打包是两个要领，分别对应zip和tar，window下面是不是没有区分？
• tar -

tape archive

.tar or .tgz, which indicate a “plain” tar archive and a gzipped archive, respectively.

# c create
$ tar cf playground.tar playground
# t list
$ tar tf playground.tar
# x extract
$ tar xf ../playground.tar
# r append

# extract a single file
$ tar xf archive.tar pathname

注意区分mode与option，c是mode，f是option。

mode要先写。

• incremental backups - 增量式备份
• linux shell 重定向 减号-的作用

studout或者stdin就可以用减号（-）来替代

• 不解压缩直接查看.gz和.bz2命令

使用zcat可以查看*.gz文件内容，使用bzcat可以直接查看*.bz2文件.

• 用ssh从远程服务器复制文件到本地

$ ssh root@xxx.xxx.xx.xx -p 端口号 'tar cf - filename' | tar xf -
root@xxx.xxx.xx.xx's password: 

先tar，传回来再导出。

• zip - 既压缩也归档
• zip,unzip的作用是交换indows文件，linux用tar,gzip
• rsync - 备份与同步

rsync options source destination
$ rsync -av playground foo

19 – Regular Expressions

后面几章都是关于文字处理的。先讲regular expression.

• metacharacters & literal characters

metacharacters是^ $ . [ ] { } - ? * + ( ) | ，其余就是literal characters。

• The Any Character

就是点号，the dot or period character

• anchors

The caret (^) (只匹配行首出来的)and dollar sign ($) （只匹配行尾出现的）

$ grep -h '^zip' dirlist*.txt
$ grep -h 'zip$' dirlist*.txt
$ grep -h '^zip$' dirlist*.txt

• 点号不是任意长度吗？加上^$就代表一个？

$ grep -h '.zip' dirlist*.txt
$ grep -i '^..j.r$' /usr/share/dict/words

• Bracket Expressions And Character Classes

[] 用来只匹配一个字符。

$ grep -h '[bg]zip' dirlist*.txt
^ : negation,只在第一个位置时有效
$ grep -h '[^bg]zip' dirlist*.txt # 注意出现的位置,表示取反
- : range
$ grep -h '^[A-Za-z0-9]' dirlist*.txt

# 要匹配-（短横线），就把它放到第一位
$ grep -h '[A-Z]' dirlist*.txt
$ grep -h '[-AZ]' dirlist*.txt

• POSIX Character Classes

出现的原因：posix和ASCII不一样。

• alternation

[]是单个，它(|)是string

$ echo "AAA" | grep -E 'AAA|BBB|CCC'
$ grep -Eh '^(bz|gz|zip)' dirlist*.txt

• Quantifiers

匹配的个数

? - Match An Element Zero Or One Time ,一个字符出现的次数，和word里面的?不同，word里面代表任意一个字符，这里的?不能单独使用，要跟在一个字符后面。

$ echo "(555) 123-4567" | grep -E '^(?[0-9][0-9][0-9])? [0-9][0-9][0-9]-[0-9][0-9][0-9][0-9]$'

* - Match An Element Zero Or More Times

$ echo "This works." | grep -E '[[:upper:]][[:upper:][:lower:] ]*.'

和?一样，前面要有字符。

+ - Match An Element One Or More Times

至少出现一次。

$ echo "This that" | grep -E '^([[:alpha:]]+ ?)+$'

{ } - Match An Element A Specific Number Of Times

$ echo "(555) 123-4567" | grep -E '^(?[0-9]{3})? [0-9]{3}-[0-9]{4}$'

• find使用reg

$ find . -regex '.*[^-_./0-9a-zA-Z].*'
#                   反             0-无穷个字符

• locate使用reg

$ locate --regex 'bin/(bz|gz|zip)'

• less和vi中也可以使用

注意它们的区别，在vi中reg是basic

# less
/^([0-9]{3}) [0-9]{3}-[0-9]{4}

# vi
/([0-9]{3}) [0-9]{3}-[0-9]{4}

20 – Text Processing

真是一切皆文件。

• cat的一些option

$ cat -A foo.txt # 显示所有符号，包括控制符
$ cat -ns foo.txt # n显示行号，s抑制多余的空行

• sort

$ sort > foo.txt
c
b
a

# 多个文件
$ sort file1.txt file2.txt file3.txt > final_sorted_list.txt

du - estimate file space usage

$ du -s /usr/share/* | head # 显示前10个
$ du -s /usr/share/* | sort -nr | head # 大到小排序后显示前10个

按指定列排序(k)：

$ ls -l /usr/bin | sort -nr -k 5 | head

多个key排序：

$ sort --key=1,1 --key=2n distros.txt
# 一个field内多个排序：
$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt

指定分割符：

$ sort -t ':' -k 7 /etc/passwd | head

• uniq

$ sort foo.txt | uniq

• cut

# 取第3列
$ cut -f 3 distros.txt

每个field的长度可能不同，如果要取所有行中时间中的年，就不可能用字符位置去表示。而先把field取出来，就规整了。

$ cut -f 3 distros.txt | cut -c 7-10

• expand - 把tab换为等量空格

$ expand distros.txt | cut -c 23-

• 改cut的默认分割符，用d option

$ cut -d ':' -f 1 /etc/passwd | head

• paste

adds one or more columns of text to a file

$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt > distros-by-date.txt
$ cut -f 1,2 distros-by-date.txt > distros-versions.txt
$ cut -f 3 distros-by-date.txt > distros-dates.txt
$ paste distros-dates.txt distros-versions.txt

• join

It joins data from multiple files based on a shared key field.

$ join distros-key-names.txt distros-key-vernums.txt | head

• comm

$ comm file1.txt file2.txt
# -n 抑制列
$ comm -12 file1.txt file2.txt

• diff

$ diff file1.txt file2.txt
# context format 看两个各自都有什么，没有什么
$ diff -c file1.txt file2.txt
# unified format 看从旧到新的变化
$ diff -u file1.txt file2.txt

• patch

apply changes to text files

用来更新文件，比如代码。

$ diff -Naur file1.txt file2.txt > patchfile.txt
$ patch < patchfile.txt

• tr

转换

$ echo "lowercase letters" | tr a-z A-Z
LOWERCASE LETTERS
#　多对一的转换
$ echo "lowercase letters" | tr [:lower:] A
# squeeze
$ echo "aaabbbccc" | tr -s ab
abccc

• sed

stream editor

# search-and-replace
$ echo "front" | sed 's/front/back/'
# 谁跟在command后面就是delimiter
$ echo "front" | sed 's_front_back_'
# 指定要处理的行数
$ echo "front" | sed '1s/front/back/'

# p：打印出来
# -n：  not to print every line by default
$ sed -n '1,5p' distros.txt
$ sed -n '/SUSE/p' distros.txt
$ sed -n '/SUSE/!p' distros.txt

# 一般出输出到屏幕，用i原地
$ sed -i 's/lazy/laxy/; s/jumped/jimped/' foo.txt

• reg的back references

$ sed 's/([0-9]{2})/([0-9]{2})/([0-9]{4})$/3-1-2/' distros.txt

• 全部替换

$ echo "aaabbbccc" | sed 's/b/B/'
$ echo "aaabbbccc" | sed 's/b/B/g'

• sed用script
• aspell

拼写检查

$ aspell check foo.txt

# -H (HTML) checking-mode option
$ aspell -H check foo.txt

改完了会产生一个bak文件，存在原来的文本。

21 – Formatting Output

讲文本输出格式。

• nl

显示行号

$ nl distros.txt | head
# 和cat -n 一样

$ sort -k 1,1 -k 2n distros.txt | sed -f distros-nl.sed | nl

• fold – Wrap Each Line To A Specified Length

$ echo "The quick brown fox jumped over the lazy dog." | fold -w 12
# 不打断单词
$ echo "The quick brown fox jumped over the lazy dog." | fold -w 12 -s

• fmt – A Simple Text Formatter

it fills and joins lines in text while preserving blank lines and indentation.

$ fmt -w 50 fmt-info.txt | head
$ fmt -cw 50 fmt-info.txt
# 一行如果不足50个字符，会把第二行的补过来
# 只对'# '开头的行操作
$ fmt -w 50 -p '# ' fmt-code.txt

• pr – Format Text For Printing

used to paginate text.

# l:page length
# w:width
$ pr -l 15 -w 65 distros.txt

首尾会有空行。

• printf – Format And Print Data

“print formatted”

使用：printf “format” arguments

$ printf "I formatted the string: %s
" foo
$ printf "%d, %f, %o, %s, %x, %X
" 380 380 380 380 380 380

Multiple optional components ：

%[flags][width][.precision]conversion_specification

• Document Formatting Systems

tex后来取代了troff.

• groff

$ zcat /usr/share/man/man1/ls.1.gz | head
$ man ls | head
$ zcat /usr/share/man/man1/ls.1.gz | groff -mandoc -T ascii | head

# 存为postscript到桌面
$ zcat /usr/share/man/man1/ls.1.gz | groff -mandoc > ~/Desktop/foo.ps

# ps转pdf
$ ps2pdf ~/Desktop/foo.ps ~/Desktop/ls.pdf

• tbl

22 – Printing

介绍了ps的由来。

CUPS (Common Unix Printing System) ：provides print drivers and print-job management

Ghostscript： a PostScript interpreter, acts as a RIP.

• pr – Convert Text Files For Printing

pr is used to adjust text to fit on a specific page size, with optional page headers and margins.

# 3列
$ ls /usr/bin | pr -3 -w 65 | head

CUPS 有两种printing，lpr, lp.

• lpr – Print Files (Berkeley Style)

$ ls /usr/bin | pr -3 | lpr
lpr: Error - no default destination available.
# 查看printer
$ lpstat -a
lpstat: No destinations added.

• lp – Print Files (System V Style)
• a2ps

“Anything to PostScript.”

$ ls /usr/bin | pr -3 -t | a2ps -o ~/Desktop/ls.ps -L 66 -R

• lpstat – Display Print System Status

# 查询printer状态
$ lpstat -a
$ lpstat -a

• lpq – Display Printer Queue Status

$ lpq
PDF is ready
no entries

• lprm / cancel – Cancel Print Jobs

23 – Compiling Programs

compile

linker，解决共享问题，用共享库。

$ which gcc
/usr/bin/gcc

• 用ftp下载source code

$ mkdir src
$ cd src
$ ftp ftp.gnu.org
Name (ftp.gnu.org:me): anonymous
ftp> cd gnu/diction
ftp> ls
ftp> get diction-1.11.tar.gz
ftp> bye

$ tar xzf diction-1.11.tar.gz
# 先查看是不是在一个文件夹里，以免解压散开
$ tar tzvf diction-1.11.tar.gz | less

共享的头文件在/usr/include ，The header files in this directory were installed when we installed the compiler.

• Building The Program

一般需要两步。

# 由于script不在默认的目录，需要./告诉shell它在当前目录
$ ./configure
config.status: creating Makefile
config.status: creating diction.1
config.status: creating diction.texi
config.status: creating diction.spec
config.status: creating style.1
config.status: creating test/rundiction
config.status: creating config.h

Makefile is a configuration file that instructs the make program exactly how to build the program. Without it, make will refuse to run.

$ make

• make只complie需要complie的文件，省时间。
• Installing The Program

$ sudo make install

Part 4 – Writing Shell Scripts

24 – Writing Your First Script

• script开头的#!

shebang，指明interpreter，每个script开头都应该加这行。

• 让script可以执行

$ chmod 755 hello_world

owner的权限是x就可以了。755所有人可以执行，700只有owner可以执行。必须是可读的。

• 执行

$ ./hello_world

必须要加上./ ,因为文件位置的问题。自动识别的命令的目录都写在了PATH中，可以查看它：

$ echo $PATH

• 把某个路径添加到PATH

在.bashrc （它在~中）中添加：

# 把~/bin添加在PATH的前面
export PATH=~/bin:"$PATH"

# 在当前terminal中生效
$ . .bashrc
# 或者
$ source .bashrc

• script放哪？

 ~/bin ： 自己用
 /usr/local/bin ： 同一个系统多人用
 /usr/local/sbin： 系统管理员用
 /usr/local：   locally supplied software，

• line-continuations

就是

• vi中对script的配置

:syntax on
:set hlsearch
:set tabstop=4
:set autoindent

永久：
写在~/.vimrc，没有就创建

25 – Starting A Project

shell是动态语言

uppercase letters to designate constants and lower case
letters for true variables.

$ vim ~/bin/sys_info_page
$ chmod 755 ~/bin/sys_info_page
$ sys_info_page
$ sys_info_page > sys_info_page.html
$ firefox sys_info_page.html

• Variables And Constants

uppercase letters to designate constants and lower case letters for true variables.

Note that in an assignment, there must be no spaces between the variable name, the equals sign, and the value.

a=z                  # Assign the string "z" to variable a.
b="a string"         # Embedded spaces must be within quotes.
c="a string and $b"  # Other expansions such as variables can be 
                     # expanded into the assignment.
d="$(ls -l foo.txt)" # Results of a command.
e=$((5 * 7))         # Arithmetic expansion.
f="		a string
"   # Escape sequences such as tabs and newlines.

大括号的作用

$ filename="myfile"
$ touch "$filename"
$ mv "$filename" "$filename1"
mv: cannot move 'myfile' to '': No such file or directory

$ mv "$filename" "${filename}1"

• here document

command << token
text
token

# shell example
cat << _EOF_
<HTML>
    <HEAD>
        <TITLE>$TITLE</TITLE>
    </HEAD>
</HTML>
_EOF_

# terminal example
$ cat << _EOF_
> $foo
> "$foo"
> '$foo'
> $foo
> _EOF_

26 – Top-Down Design

讲程序设计的思想。

• shell functions

两种形式：

function name {
commands
return
}

and the simpler (and generally preferred) form:

name () {
commands
return
}

in order for function calls to be recognized as shell functions and not interpreted as the names of external programs, shell function definitions must appear in the script before they are called.

• local variables

要加local

funct_1 () {
    local foo # variable foo local to funct_1
    foo=1
    echo "funct_1: foo = $foo"
}

27 – Flow Control: Branching With if

shell的if语句挺复杂的。

if commands; then
    commands
[elif commands; then
    commands...]
[else
    commands]
fi

x=5
if [ "$x" -eq 5 ]; then
    echo "x equals 5."
else
    echo "x does not equal 5."
fi  

• exit status

程序结果后返回给系统的。查询：

$ ls -d /usr/bin
$ echo $?

true - do nothing, successfully

false - do nothing, unsuccessfully

• test

语法：

test expression
and the more popular:
[ expression ]

expression是true时返回0，否则返回1

test和[ 本质上是一样的。

使用：

#!/bin/bash

# test-file: Evaluate the status of a file

FILE=~/.bashrc

if [ -e "$FILE" ]; then
    if [ -f "$FILE" ]; then
        echo "$FILE is a regular file."
    fi
    if [ -d "$FILE" ]; then
        echo "$FILE is a directory."
    fi
    if [ -r "$FILE" ]; then
        echo "$FILE is readable."
    fi
    if [ -w "$FILE" ]; then
        echo "$FILE is writable."
    fi
    if [ -x "$FILE" ]; then
        echo "$FILE is executable/searchable."
    fi
else
    echo "$FILE does not exist"
    exit 1
fi

exit  # ?

When a script “runs off the end” (reaches end of file), it terminates with an exit status of the last command executed.

• String Expressions

#!/bin/bash
# test-string: evaluate the value of a string
ANSWER=maybe
if [ -z "$ANSWER" ]; then
    echo "There is no answer." >&2
    exit 1
fi
if [ "$ANSWER" = "yes" ]; then
    echo "The answer is YES."
elif [ "$ANSWER" = "no" ]; then
    echo "The answer is NO."
elif [ "$ANSWER" = "maybe" ]; then
    echo "The answer is MAYBE."
else
    echo "The answer is UNKNOWN."
fi

• Integer Expressions
• test的选项真多！！！
• test用regex

[[]]

# 检验是不是数
if [[ "$INT" =~ ^-?[0-9]+$ ]]; then

• (( )) - Designed For Integers

$ if ((1)); then echo "It is true."; fi
It is true.

if ((INT == 0)); 
if ((INT < 0)); 
if (( ((INT % 2)) == 0)); 

• Combining Expressions

Operation   test   [[ ]] and (( ))
AND         -a           &&
OR          -o           ||
NOT         !            !

例子：

# [[]]
if [[ "$INT" -ge "$MIN_VAL" && "$INT" -le "$MAX_VAL" ]];
# test
if [ "$INT" -ge "$MIN_VAL" -a "$INT" -le "$MAX_VAL" ];

• Control Operators: Another Way To Branch

The && (AND) and || (OR)

$ mkdir temp && cd temp
$ [[ -d temp ]] || mkdir temp

• []和[[]]的区别

[[]]和[]一样，不过有两个新特性，：

string1 =~ regex

== operator

• [[]]和表达式之间要有空格

[[ "$count" -gt 5 ]]
[[ "$count" -gt 5]] # 会报错，执行起来会很可怕

28 – Reading Keyboard Input

• read – Read Values From Standard Input

read a single line of standard input.

read [-options] [variable...]

# -n option,suppresses the trailing newline on output
echo -n "Please enter an integer -> "
read int

read var1 var2 var3 var4 var5 # 多个

# -p prompt
read -p "Enter one or more values > "
# -t seconds
# -s Silent mode. 
if read -t 10 -sp "Enter secret passphrase > " secret_pass; 

• IFS

Internal Field Separator

file_info=$(grep "^$user_name:" $FILE)

# 写在一行，IFS只改变跟在后面的 command
IFS=":" read user pw uid gid name home shell <<< "$file_info"

• here string

The <<< operator indicates a here string.

• You Can’t Pipe read

就像函数与子函数，凡是这一类都不能pip。

• Validating Input
• Menus

就是提示和if。

• 一个程序中出现多个exit是个不好的设计

那应该如何做？

29 – Flow Control: Looping With while / until

有3各loop

• while

语法：
while commands; do commands; done

#!/bin/bash
# while-count: display a series of numbers
count=1
while [[ "$count" -le 5 ]]; do
    echo "$count"
    count=$((count + 1))
done
echo "Finished."

• Breaking Out Of A Loop

break

continue

• until

和while相反

count=1
until [[ "$count" -gt 5 ]]; do
    echo "$count
    count=$((count + 1))
done
echo "Finished.

• read file with loop

#!/bin/bash

# while-read

while read distro version release; do
    printf "distro: %s	version: %s	released: %s
" 
        "$distro" 
        "$version" 
        "$release"
done < distros.txt

30 – Troubleshooting

讲错误的处理。

• 引号不对称

• 特定符号缺失

• 意外的expansion

  是否报错取决于expansion的值。

• 双引号的作用

# number为空时会报错
number=
if [ $number = 1 ]; then
# 这样就不会了  “”=1
[ "$number" = 1 ]

Multi-word string也应该用双引号括起来。

所以，总是用双引号把变量括起来。

• logical error
• 给出错误信息是比较好的做法
• unix文件名称什么字符都可以用，有两个例外

/ , null character(a zero byte)

• the POSIX Portable Filename Character Set

uppercase letters A-Z, lowercase letters a-z, numerals 0-9, period (.), hyphen (-), and underscore(_).

• 检查input
• 要会写测试

利用echo输出

在更改的地方加注释，方便恢复。

• test case

case要反映edge and corner

• debug技术

commenting out :部分注释

tracing :在一些关键点增加输出，查看是否执行到这里了

可以在script的第一行加入x option:

#!/bin/bash -x

也可以用set设置部分tracing

set -x # turn on
...
set +x # turn off

• 输出中间过程中变量的值
• 推荐《The Art of Unix Programming》

31 – Flow Control: Branching With case

• case的用法

  1 #!/bin/bash
  2 
  3 # case-menu
  4 
  5 clear
  6 echo "
  7 please select:
  8 
  9 1. display system information
 10 2. display disk space
 11 3. display home space utilization
 12 0. quit
 13 "
 14 read -p "enter selection [0-3] > "
 15 
 16 case "$REPLY" in
 17     0)  echo "program terminated"
 18         exit
 19         ;;
 20     1)  echo "hostname: $HOSTNAME"
 21         uptime
 22         ;;
 23     2)  df -h
 24         ;;
 25     3) if [[ "$(id -u)" -eq 0 ]]; then
 26             echo "home space utilization (all users)"
 27             du -sh /home/*
 28         else
 29             echo "home space utilization ($USER)"
 30             du -sh "$HOME"
 31         fi
 32         ;;
 33     *)  echo "invalid entry" >&2
 34         exit 1
 35         ;;
 36 esac

32 – Positional Parameters

讲处理command的option

• positional parameter

$0是当前命令的路径。

注意它的含义：

Note that $0 always contains the full pathname of the first item on the command line (i.e., the name of the program) and does not contain the name of the shell function as we might expect.

是整个路径，不是命令的名字。

大于9时用大括号括起来。

• 参数个数：$#
• shift

让$1下次变成$2，所有参数都移动，$#减1

• basename

去除文件名前面的字符，只要文件名。

• positional parameter也可以用在function中

• Difference between echo -e “” and echo $“”

  e的意思就是扩展，支持转义，e和$支持的符号部分不相同。

• $*和$@的区别

不加双引号时是一样的，遇到空格就拆分，加双引号时不一样，$*会将所有参数放到一个字符串中，$@会将每个输入的参数分别当作参数，也就是不区分输入时的空格。

$@经常用。

33 – Flow Control: Looping With for

讲for

# 两种风格
for variable [in words]; do
    commands
done

# c语言的
for (( expression1; expression2; expression3 )); do
    commands
done

例子：

[me@linuxbox ~]$ for i in A B C D; do echo $i; done

Create list:

itscs-MacBook-Pro:~ itsc$ for i in {A..D}; do echo $i; done
# 文件列表 pathname expansion
itscs-MacBook-Pro:learnCommandLine itsc$ for i in dis*.txt; do echo "$i"; done

34 – Strings And Numbers

• 一般情况下将变量括起来，防止被拆分
• parameter expansion是什么？

变量是其中的一种

• 大括号的作用

$a
${a}
a="foo"
echo "${a}_file" # 和其它字符相连时防止形成不存在的变量

• 处理空的和不存在的字符

parameter为空或者不存在就用word，存在就用它自己。
${parameter:-word}

和上面基本一样，区别：要赋值。
${parameter:=word}

unset和empty就发发送word到error
${parameter:?word}

${parameter:+word}

• String operation

${#parameter} # 换成长度

$ foo="This string is long."
$ echo "'$foo' is ${#foo} characters long.
'This string is long.' is 20 characters long

${parameter:offset}
${parameter:offset:length}
[me@linuxbox ~]$ foo="This string is long."
[me@linuxbox ~]$ echo ${foo:5}
string is long.
[me@linuxbox ~]$ echo ${foo:5:6}
string

删除一部分
${parameter#pattern}
${parameter##pattern}
[me@linuxbox ~]$ foo=file.txt.zip
[me@linuxbox ~]$ echo ${foo#*.}
txt.zip
[me@linuxbox ~]$ echo ${foo##*.}
zip

反着删除
${parameter%pattern}
${parameter%%pattern}
[me@linuxbox ~]$ foo=file.txt.zip
[me@linuxbox ~]$ echo ${foo%.*}
file.txt
[me@linuxbox ~]$ echo ${foo%%.*}
file

替换
${parameter/pattern/string}
${parameter//pattern/string}
${parameter/#pattern/string}
${parameter/%pattern/string}

• 可以用expansion来提高script的效率
• 大小写转换

可以用来做什么？比如数据库的查找，匹配的时候把输入和数据库中的都统一大小写。

• declare
• shell不能进行浮点运算

itscs-MacBook-Pro:learnCommandLine itsc$ echo $((3.3+4.2))
-bash: 3.3+4.2: syntax error: invalid arithmetic operator (error token is ".3+4.2")

解决：perl, awk.书里用最简单的bc

为什么连浮点运算都不支持？真是麻烦。

35 – Arrays

• scalar variables

Single value

• shell的array只能一维

[me@linuxbox ~]$ a[1]=foo
[me@linuxbox ~]$ echo ${a[1]}
foo
[me@linuxbox ~]$ declare -a a

# assign
name[subscript]=value
name=(value1 value2 ...)
[me@linuxbox ~]$ days=(Sun Mon Tue Wed Thu Fri Sat)
[me@linuxbox ~]$ days=([0]=Sun [1]=Mon [2]=Tue [3]=Wed [4]=Thu [5]=Fri [6]=Sat)

• 遍历array

[me@linuxbox ~]$ animals=("a dog" "a cat" "a fish")
[me@linuxbox ~]$ for i in ${animals[*]}; do echo $i; done
[me@linuxbox ~]$ for i in ${animals[@]}; do echo $i; done
[me@linuxbox ~]$ for i in "${animals[*]}"; do echo $i; done
[me@linuxbox ~]$ for i in "${animals[@]}"; do echo $i; done

• bash的array不一定是要连续的

所以需要有方法知道哪些位置上有值

[me@linuxbox ~]$ foo=([2]=a [4]=b [6]=c)
[me@linuxbox ~]$ for i in "${foo[@]}"; do echo $i; done
a
b
c
[me@linuxbox ~]$ for i in "${!foo[@]}"; do echo $i; done
2
4
6

• array的追加

$ foo=(a b c)
$ foo[100]=e
$ echo ${foo[@]}
a b c e
$ foo+=(k l)
$ echo ${foo[@]}
a b c e k l
$ for i in "${foo[@]}"; do echo $i; done
a
b
c
e
k
l
$ for i in "${!foo[@]}"; do echo $i; done
0
1
2
100
101
102

subscript不是连续的

• associative arrays

下标可以是字符

36 – Exotica

讲一些不常用的，在特定场景下使用的

• Group Commands And Subshells

Group command:
{ command1; command2; [command3; ...] }
Subshell:
(command1; command2; [command3;...])

它们是做什么的？

manage redirection

{ ls -l; echo "Listing of foo.txt"; cat foo.txt; } > output.txt
(ls -l; echo "Listing of foo.txt"; cat foo.txt) > output.txt

结合pip

{ ls -l; echo "Listing of foo.txt"; cat foo.txt; } | lpr

• 注意大括号的使用

due to the way bash implements
group commands, the braces must be separated from the commands by a space and the
last command must be terminated with either a semicolon or a newline prior to the closing brace.

• group和subshell的区别

subshell和它的名字一样，返回时enviroment会丢失，所以一般情况下用group.

echo "foo" | read
echo $REPLY # 这就是subshell的例子，reply是空的

commands in pipelines are always executed in subshells

• process substitution

用来解决subshell的问题

read < <(echo "foo")
echo $REPLY

• asynchronous execution
• 什么是异步？

什么是阻塞，非阻塞，同步，异步？

举个打电话的例子：

阻塞/同步：打一个电话一直到有人接为止
非阻塞：打一个电话没人接，每隔10分钟再打一次，知道有人接为止
异步：打一个电话没人接，转到语音邮箱留言（注册），然后等待对方回电（call back)

看起来异步是最高效，充分利用资源，可以想像整个系统能支持大规模并发。但问题是调试很麻烦，不知道什么时候call back。

• Named pipes

create a connection between two processes and can be
used just like other types of files.

named pipe是一个文件起着pipe的作用

[me@linuxbox ~]$ mkfifo pipe1
[me@linuxbox ~]$ ls -l pipe1
prw-r--r-- 1 me me 0 2009-07-17 06:41 pipe1

# terminal 1
[me@linuxbox ~]$ ls -l > pipe1

# terminal2
[me@linuxbox ~]$ cat < pipe1

问题

• [[:upper:]]哪一节的？

• 每个文件夹下的.和..的作用？为什么必须要有它们？

  它们代表当前和前一级目录，这个大家都知道，可以为什么要有它们？设计原理是什么？

  参考：SO，只说了有，没有说什么为什么。

• shell中各种括号[]、[[]]的作用与区别？

  参考。

需要了解的知识

• linux文件系统

磁盘、分区及Linux文件系统 [Disk, Partition, Linux File System]

• 硬盘接口

IDE、SATA、SCSI、SAS、FC、SSD硬盘类型介绍

• Understand synopsis in manpage

  可以直接在命令man man中看。

• vi color

ENABLING VI SYNTAX COLORS

cd ~
vi .vimrc
syntax on
colorscheme desert

• vi indent

Redefine tab as 4 spaces

添加到vimrc中即永久：

set tabstop=4

• 标准错误输出>&2输出到哪了？

  屏幕。

• vim color

/usr/share/vim/vim74/colors/

我觉得还不错的：

koehler

industry

murphy

ron

• ls如何按文件类型排序？

How to make ls sort by file extension and then name?

ls -lX

• 平铺式窗口管理器-Awesome
• vi comment

Reveal the line numbers:

:set number

then

:5,$s/^/#/g     comment

or this:

:5,$s/^/#/      uncomment

• vim如何复制到clipboard?

Vim: copy selection to OS X clipboard still fail to work

To check if your Vim has clipboard support: vim --version | grep -i clipboard. You should see +clipboard. If you see -clipboard your Vim was compiled without clipboard support…

itscs-MacBook-Pro:learnCommandLine itsc$ vim --version | grep -i clipboard
-clipboard       +job             +path_extra      +user_commands
+eval            -mouse_dec       +statusline      -xterm_clipboard

我的是减号，说明不支持。

Getting Vim with +clipboard support

要先装Homebrew，装完后的vim在 /usr/local/bin/vim,原来的在/usr/bin，也存在，只不过PATH中，local的在前面，命令会先用它。

• 单引号、双引号区别

echo $1
echo "$1"
echo '$1'

$ ./longest words
words
words
$1

双引号与无引号在这里是一样的，双引号的作用体现在字符串为空的时候，为空的时候是空字符串，可以判断，而无引号就麻烦了，比如等号，一边为空会报错，说等号是个二元操作符两边都要有值。单引号的作用就是禁止转义，全部为字符串。

• linux中shell变量#,@,0,1,$2的含义解释

$#
添加到Shell的参数个数

$! shell parameter, which will always contain the process ID of the
last job put into the background.

网站

The Bash Hackers Wiki