shell脚本读取文件+读取命令行参数+读取标准输入+变量赋值+输出到文件

读取url_list文件批量下载网页

url_list

http://www.tianyancha.com/company/2412078287
http://www.4399.com/special/1.htm
http://www.we7.cc/
http://kongzhong.tmall.com/
http://dianying.2345.com/
http://www.takefoto.cn/viewnews-1521788.html
http://www.x4jdm.com/bf/429-1-1.html
http://www.douyu.com/546715
http://www.zjedu.gov.cn/default.html
http://dl.xunlei.com/

 download.sh

#!/bin/bash
for line in $(cat $1)
do
    id=$(echo $line| getid | awk '{$id=10000+$1;print $id;}')
    echo $line | gethtmlfile $id > "./result/"${id}".html"
done

运行：

[spider@zhangsuosheng]$ chmod +x ./download.sh 
[spider@zhangsuosheng]$ ./download.sh url_list

 1、sh文件格式

http://www.runoob.com/linux/linux-shell.html

2、bash按行读取文件+bash读取命令行参数

测试文件：url_list_zss

[spider@zhangsuosheng]$ cat url_list_zss 
cccccc
ddddddddd
aaaaaa

正确写法：

#!/bin/bash
for line in $(cat $1)
do
    echo $line
done

[spider@zhangsuosheng]$ chmod +x ./download.sh 
[spider@zhangsuosheng]$ ./download.sh url_list
cccccc
ddddddddd
aaaaaa

不合适的写法：

#!/bin/bash
for line in 'cat $1'
do
        echo $line
done

[spider@zhangsuosheng]$ chmod +x ./download_testhtml.sh 
[spider@zhangsuosheng]$ ./download_testhtml.sh url_list_zss 
cat $1

https://www.jb51.net/article/122918.htm

3、读取命令行参数

https://blog.csdn.net/qq_30145093/article/details/78191941

https://blog.csdn.net/ruidongliu/article/details/9717905

 4、加法运算

用的awk

5、读取管道中的标准输入 直接读/xargs

https://www.cnblogs.com/wangqiguo/p/6464234.html

6、变量赋值

https://blog.csdn.net/lemontree1945/article/details/79126819

7、字符串拼接

https://www.jb51.net/article/44207.htm