Sysdig Falco和OpenShift的集成

安装步骤参考

https://falco.org/docs/installation/

• 安装部署

我选择了一个3.11的环境进行安装部署，步骤如下

git clone https://github.com/falcosecurity/falco/
cd falco/integrations/k8s-using-daemonset

oc apply -f k8s-with-rbac/falco-account.yaml
oc apply -f k8s-with-rbac/falco-service.yaml

mkdir -p k8s-with-rbac/falco-config
k8s-using-daemonset$ cp ../../falco.yaml k8s-with-rbac/falco-config/
k8s-using-daemonset$ cp ../../rules/falco_rules.* k8s-with-rbac/falco-config/
k8s-using-daemonset$ cp ../../rules/k8s_audit_rules.yaml k8s-with-rbac/falco-config/


oc create configmap falco-config --from-file=k8s-with-rbac/falco-config

因为在OpenShift上有一些安全控制，所以需要打开使用root运行的权限，并给与集群管理员的权限。

oc adm policy add-scc-to-user anyuid -z falco-account -n sample1
oc adm policy add-scc-to-user   privileged -z falco-account -n sample1

oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:sample1:falco-account

部署

oc apply -f k8s-with-rbac/falco-daemonset-configmap.yaml

查看日志,发现报了一些错误。

[root@node2 k8s-using-daemonset]# oc logs falco-daemonset-9vwnb 
* Setting up /usr/src links from host
* Unloading falco-probe, if present
* Running dkms install for falco

Creating symlink /var/lib/dkms/falco/0.19.0/source ->
                 /usr/src/falco-0.19.0

DKMS: add completed.
Error! echo
Your kernel headers for kernel 3.10.0-957.el7.x86_64 cannot be found at
/lib/modules/3.10.0-957.el7.x86_64/build or /lib/modules/3.10.0-957.el7.x86_64/source.
* Running dkms build failed, couldn't find /var/lib/dkms/falco/0.19.0/build/make.log
* Trying to load a system falco-probe, if present
* Trying to find precompiled falco-probe for 3.10.0-957.el7.x86_64
Cannot find kernel config
Wed Jan 29 04:02:05 2020: Falco initialized with configuration file /etc/falco/falco.yaml
Wed Jan 29 04:02:05 2020: Loading rules from file /etc/falco/falco_rules.yaml:
Wed Jan 29 04:02:05 2020: Loading rules from file /etc/falco/falco_rules.local.yaml:
Wed Jan 29 04:02:05 2020: Loading rules from file /etc/falco/k8s_audit_rules.yaml:
Wed Jan 29 04:02:05 2020: Unable to load the driver. Exiting.
Wed Jan 29 04:02:05 2020: Runtime error: error opening device /host/dev/falco0. Make sure you have root credentials and that the falco-probe module is loaded.. Exiting.

需要在运行的宿主机上安装kernel-devel模块

yum -y install kernel-devel-$(uname -r)

然后再运行

[root@node2 k8s-using-daemonset]# oc logs falco-daemonset-2hrdr -f
* Setting up /usr/src links from host
* Unloading falco-probe, if present
* Running dkms install for falco

Creating symlink /var/lib/dkms/falco/0.19.0/source ->
                 /usr/src/falco-0.19.0

DKMS: add completed.

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
make -j1 KERNELRELEASE=3.10.0-957.el7.x86_64 -C /lib/modules/3.10.0-957.el7.x86_64/build M=/var/lib/dkms/falco/0.19.0/build....
cleaning build area...

DKMS: build completed.

falco-probe.ko.xz:
Running module version sanity check.
modinfo: ERROR: missing module or filename.
 - Original module
mkdir: cannot create directory '/lib/modules/3.10.0-957.el7.x86_64/kernel/extra': Read-only file system
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/3.10.0-957.el7.x86_64/kernel/extra/
cp: cannot create regular file '/lib/modules/3.10.0-957.el7.x86_64/kernel/extra/falco-probe.ko.xz': No such file or directory

depmod....

DKMS: install completed.
* Trying to load a dkms falco-probe, if present
falco-probe found and loaded in dkms (xz)
Wed Jan 29 06:07:26 2020: Falco initialized with configuration file /etc/falco/falco.yaml
Wed Jan 29 06:07:26 2020: Loading rules from file /etc/falco/falco_rules.yaml:
Wed Jan 29 06:07:26 2020: Loading rules from file /etc/falco/falco_rules.local.yaml:
Wed Jan 29 06:07:26 2020: Loading rules from file /etc/falco/k8s_audit_rules.yaml:
Wed Jan 29 06:07:27 2020: Starting internal webserver, listening on port 8765

•  验证

• bash进入容器会被监控和输出INFO信息

[root@master .kube]# oc exec -it myconntrack-2-c98j6 bash
I have no name!@myconntrack-2-c98j6:/usr/local/tomcat$ exit
exit

falco日志输出

06:08:28.753580928: Notice A shell was spawned in a container with an attached terminal (user=<NA> k8s.ns=sample1 k8s.pod=myconntrack-2-c98j6 container=476ccfe4b5fe shell=bash parent=docker-runc-cur cmdline=bash terminal=34819 container_id=476ccfe4b5fe image=registry.example.com/conntrack) k8s.ns=sample1 k8s.pod=myconntrack-2-c98j6 container=476ccfe4b5fe k8s.ns=sample1 k8s.pod=myconntrack-2-c98j6 container=476ccfe4b5fe
06:08:54.628722525: Critical Falco internal: syscall event drop. 2 system calls dropped in last second. (ebpf_enabled=0 n_drops=2 n_drops_buffer=2 n_drops_bug=0 n_drops_pf=0 n_evts=17717)

critical Falco internal的日志忽略不计，上面的notice会输出container with an attached terminal的信息。

• 删除以及重新命名文件会输出信息

将容器提升权限后以root运行，然后bash进去创建和删除文件。falco提示

07:15:50.195115379: Warning Shell history had been deleted or renamed (user=root type=open command=bash fd.name=/root/.bash_history name=/root/.bash_history path=<NA> oldpath=<NA> k8s.ns=sample1 k8s.pod=myconntrack-1-wdt99 container=37e121d938e6) k8s.ns=sample1 k8s.pod=myconntrack-1-wdt99 container=37e121d938e6 k8s.ns=sample1 k8s.pod=myconntrack-1-wdt99 container=37e121d938e6

• 在宿主上下载文件, 也会在falco pod上打出日志

06:11:46.703934222: Error File below / or /root opened for writing (user=root command=wget -O jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 parent=bash file=/root/jq program=wget container_id=host image=<NA>) k8s.ns=<NA> k8s.pod=<NA> container=host k8s.ns=<NA> k8s.pod=<NA> container=host k8s.ns=<NA> k8s.pod=<NA> container=host

• Falco规则

详细参考 https://falco.org/docs/rules/

详细的监控规则存放在 falco/integrations/k8s-using-daemonset/k8s-with-rbac/falco-config目录下falco_rules.yaml。

[root@node2 falco-config]# cat falco_rules.yaml | grep -A 10  'terminal'
  desc: A shell was used as the entrypoint/exec point into a container with an attached terminal.
  condition: >
    spawned_process and container
    and shell_procs and proc.tty != 0
    and container_entrypoint
  output: >
    A shell was spawned in a container with an attached terminal (user=%user.name %container.info
    shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty container_id=%container.id image=%container.image.repository)
  priority: NOTICE
  tags: [container, shell, mitre_execution]

# For some container types (mesos), there isn't a container image to
# work with, and the container name is autogenerated, so there isn't
# any stable aspect of the software to work with. In this case, we
# fall back to allowing certain command lines.

- list: known_shell_spawn_cmdlines
  items: [