bcc -execsnoop 性能---未完

　　目前使用到的bcc程序主要包括两个部分，一部分是python语言，一部分是c语言。python部分主要做的工作是BPF程序的加载和操作BPF程序的map，并进行数据处理。c部分会被llvm编译器编译为BPF字节码，经过BPF验证器验证安全后，加载到内核中执行。python和c中出现的陌生函数可以查下面这两个手册

python 等函数：Python链接

c等函数：链接

bcc 安装：bcc_install

bcc program book： book url

https://www.ebpf.top/post/bpf_learn_path/

什么是 bcc

• Bcc 的开源项目：https://github.com/iovisor/bcc
• eBPF 虚拟机使用的是类似于汇编语言的指令，对于程序编写来说直接使用难度非常大。bcc 提供了一个名为 bcc 的 python 库，简化了 eBPF 应用的开发过程
• Bcc 收集了大量现成的 eBPF 程序可以直接拿来使用，可以通过以下工具分布图感受一下

 

https://github.com/brendangregg/perf-tools/blob/master/execsnoop

 其execsnoop 代码实现如下：

#!/bin/bash
#
# execsnoop - trace process exec() with arguments.
#             Written using Linux ftrace.
#
# This shows the execution of new processes, especially short-lived ones that
# can be missed by sampling tools such as top(1).
#
# USAGE: ./execsnoop [-hrt] [-n name]
#
# REQUIREMENTS: FTRACE and KPROBE CONFIG, sched:sched_process_fork tracepoint,
# and either the sys_execve, stub_execve or do_execve kernel function. You may
# already have these on recent kernels. And awk.
#
# This traces exec() from the fork()->exec() sequence, which means it won't
# catch new processes that only fork(). With the -r option, it will also catch
# processes that re-exec. It makes a best-effort attempt to retrieve the program
# arguments and PPID; if these are unavailable, 0 and "[?]" are printed
# respectively. There is also a limit to the number of arguments printed (by
# default, 8), which can be increased using -a.
#
# This implementation is designed to work on older kernel versions, and without
# kernel debuginfo. It works by dynamic tracing an execve kernel function to
# read the arguments from the %si register. The sys_execve function is tried
# first, then stub_execve and do_execve. The sched:sched_process_fork
# tracepoint is used to get the PPID. This program is a workaround that should be
# improved in the future when other kernel capabilities are made available. If
# you need a more reliable tool now, then consider other tracing alternatives
# (eg, SystemTap). This tool is really a proof of concept to see what ftrace can
# currently do.
#
# From perf-tools: https://github.com/brendangregg/perf-tools
#
# See the execsnoop(8) man page (in perf-tools) for more info.
#
# COPYRIGHT: Copyright (c) 2014 Brendan Gregg.
#
#  This program is free software; you can redistribute it and/or
#  modify it under the terms of the GNU General Public License
#  as published by the Free Software Foundation; either version 2
#  of the License, or (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software Foundation,
#  Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
#
#  (http://www.gnu.org/copyleft/gpl.html)
#
# 07-Jul-2014    Brendan Gregg    Created this.

### default variables
tracing=/sys/kernel/debug/tracing
flock=/var/tmp/.ftrace-lock; wroteflock=0
opt_duration=0; duration=; opt_name=0; name=; opt_time=0; opt_reexec=0
opt_argc=0; argc=8; max_argc=16; ftext=
trap ':' INT QUIT TERM PIPE HUP    # sends execution to end tracing section

function usage {
    cat <<-END >&2
    USAGE: execsnoop [-hrt] [-a argc] [-d secs] [name]
                     -d seconds      # trace duration, and use buffers
                     -a argc         # max args to show (default 8)
                     -r              # include re-execs
                     -t              # include time (seconds)
                     -h              # this usage message
                     name            # process name to match (REs allowed)
      eg,
           execsnoop                 # watch exec()s live (unbuffered)
           execsnoop -d 1            # trace 1 sec (buffered)
           execsnoop grep            # trace process names containing grep
           execsnoop 'udevd$'        # process names ending in "udevd"

    See the man page and example file for more info.
END
    exit
}

function warn {
    if ! eval "$@"; then
        echo >&2 "WARNING: command failed "$@""
    fi
}

function end {
    # disable tracing
    echo 2>/dev/null
    echo "Ending tracing..." 2>/dev/null
    cd $tracing
    warn "echo 0 > events/kprobes/$kname/enable"
    warn "echo 0 > events/sched/sched_process_fork/enable"
    warn "echo -:$kname >> kprobe_events"
    warn "echo > trace"
    (( wroteflock )) && warn "rm $flock"
}

function die {
    echo >&2 "$@"
    exit 1
}

function edie {
    # die with a quiet end()
    echo >&2 "$@"
    exec >/dev/null 2>&1
    end
    exit 1
}

### process options
while getopts a:d:hrt opt
do
    case $opt in
    a)    opt_argc=1; argc=$OPTARG ;;
    d)    opt_duration=1; duration=$OPTARG ;;
    r)    opt_reexec=1 ;;
    t)    opt_time=1 ;;
    h|?)    usage ;;
    esac
done
shift $(( $OPTIND - 1 ))
if (( $# )); then
    opt_name=1
    name=$1
    shift
fi
(( $# )) && usage

### option logic
(( opt_pid && opt_name )) && die "ERROR: use either -p or -n."
(( opt_pid )) && ftext=" issued by PID $pid"
(( opt_name )) && ftext=" issued by process name "$name""
(( opt_file )) && ftext="$ftext for filenames containing "$file""
(( opt_argc && argc > max_argc )) && die "ERROR: max -a argc is $max_argc."
if (( opt_duration )); then
    echo "Tracing exec()s$ftext for $duration seconds (buffered)..."
else
    echo "Tracing exec()s$ftext. Ctrl-C to end."
fi

### select awk
if (( opt_duration )); then
    [[ -x /usr/bin/mawk ]] && awk=mawk || awk=awk
else
    # workarounds for mawk/gawk fflush behavior
    if [[ -x /usr/bin/gawk ]]; then
        awk=gawk
    elif [[ -x /usr/bin/mawk ]]; then
        awk="mawk -W interactive"
    else
        awk=awk
    fi
fi

### check permissions
cd $tracing || die "ERROR: accessing tracing. Root user? Kernel has FTRACE?
    debugfs mounted? (mount -t debugfs debugfs /sys/kernel/debug)"

### ftrace lock
[[ -e $flock ]] && die "ERROR: ftrace may be in use by PID $(cat $flock) $flock"
echo $$ > $flock || die "ERROR: unable to write $flock."
wroteflock=1

### build probe
if [[ -x /usr/bin/getconf ]]; then
    bits=$(getconf LONG_BIT)
else
    bits=64
    [[ $(uname -m) == i* ]] && bits=32
fi
(( offset = bits / 8 ))
function makeprobe {
    func=$1
    kname=execsnoop_$func
    kprobe="p:$kname $func"
    i=0
    while (( i < argc + 1 )); do
        # p:kname do_execve +0(+0(%si)):string +0(+8(%si)):string ...
        kprobe="$kprobe +0(+$(( i * offset ))(%si)):string"
        (( i++ ))
    done
}
# try in this order: sys_execve, stub_execve, do_execve
makeprobe sys_execve

### setup and begin tracing
echo nop > current_tracer
if ! echo $kprobe >> kprobe_events 2>/dev/null; then
    makeprobe stub_execve
    if ! echo $kprobe >> kprobe_events 2>/dev/null; then
        makeprobe do_execve
        if ! echo $kprobe >> kprobe_events 2>/dev/null; then
            edie "ERROR: adding a kprobe for execve. Exiting."
        fi
    fi
fi
if ! echo 1 > events/kprobes/$kname/enable; then
    edie "ERROR: enabling kprobe for execve. Exiting."
fi
if ! echo 1 > events/sched/sched_process_fork/enable; then
    edie "ERROR: enabling sched:sched_process_fork tracepoint. Exiting."
fi
echo "Instrumenting $func"
(( opt_time )) && printf "%-16s " "TIMEs"
printf "%6s %6s %s
" "PID" "PPID" "ARGS"

#
# Determine output format. It may be one of the following (newest first):
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#           TASK-PID    CPU#    TIMESTAMP  FUNCTION
# To differentiate between them, the number of header fields is counted,
# and an offset set, to skip the extra column when needed.
#
offset=$($awk 'BEGIN { o = 0; }
    $1 == "#" && $2 ~ /TASK/ && NF == 6 { o = 1; }
    $2 ~ /TASK/ { print o; exit }' trace)

### print trace buffer
warn "echo > trace"
( if (( opt_duration )); then
    # wait then dump buffer
    sleep $duration
    cat -v trace
else
    # print buffer live
    cat -v trace_pipe
fi ) | $awk -v o=$offset -v opt_name=$opt_name -v name=$name 
    -v opt_duration=$opt_duration -v opt_time=$opt_time -v kname=$kname 
    -v opt_reexec=$opt_reexec '
    # common fields
    $1 != "#" {
        # task name can contain dashes
        comm = pid = $1
        sub(/-[0-9][0-9]*/, "", comm)
        sub(/.*-/, "", pid)
    }

    $1 != "#" && $(4+o) ~ /sched_process_fork/ {
        cpid=$0
        sub(/.* child_pid=/, "", cpid)
        sub(/ .*/, "", cpid)
        getppid[cpid] = pid
        delete seen[pid]
    }

    $1 != "#" && $(4+o) ~ kname {
        if (seen[pid])
            next
        if (opt_name && comm !~ name)
            next

        #
        # examples:
        # ... arg1="/bin/echo" arg2="1" arg3="2" arg4="3" ...
        # ... arg1="sleep" arg2="2" arg3=(fault) arg4="" ...
        # ... arg1="" arg2=(fault) arg3="" arg4="" ...
        # the last example is uncommon, and may be a race.
        #
        if ($0 ~ /arg1=""/) {
            args = comm " [?]"
        } else {
            args=$0
            sub(/ arg[0-9]*=(fault).*/, "", args)
            sub(/.*arg1="/, "", args)
            gsub(/" arg[0-9]*="/, " ", args)
            sub(/"$/, "", args)
            if ($0 !~ /(fault)/)
                args = args " [...]"
        }

        if (opt_time) {
            time = $(3+o); sub(":", "", time)
            printf "%-16s ", time
        }
        printf "%6s %6d %s
", pid, getppid[pid], args
        if (!opt_duration)
            fflush()
        if (!opt_reexec) {
            seen[pid] = 1
            delete getppid[pid]
        }
    }

    $0 ~ /LOST.*EVENT[S]/ { print "WARNING: " $0 > "/dev/stderr" }
'

### end tracing
end

python 版本依赖于bcc bpf 如下：

#!/usr/bin/python
# @lint-avoid-python-3-compatibility-imports
#
# execsnoop Trace new processes via exec() syscalls.
#           For Linux, uses BCC, eBPF. Embedded C.
#
# USAGE: execsnoop [-h] [-T] [-t] [-x] [-q] [-n NAME] [-l LINE]
#                  [--max-args MAX_ARGS]
#
# This currently will print up to a maximum of 19 arguments, plus the process
# name, so 20 fields in total (MAXARG).
#
# This won't catch all new processes: an application may fork() but not exec().
#
# Copyright 2016 Netflix, Inc.
# Licensed under the Apache License, Version 2.0 (the "License")
#
# 07-Feb-2016   Brendan Gregg   Created this.

from __future__ import print_function
from bcc import BPF
from bcc.utils import ArgString, printb
import bcc.utils as utils
import argparse
import re
import time
import pwd
from collections import defaultdict
from time import strftime


def parse_uid(user):
    try:
        result = int(user)
    except ValueError:
        try:
            user_info = pwd.getpwnam(user)
        except KeyError:
            raise argparse.ArgumentTypeError(
                "{0!r} is not valid UID or user entry".format(user))
        else:
            return user_info.pw_uid
    else:
        # Maybe validate if UID < 0 ?
        return result


# arguments
examples = """examples:
    ./execsnoop           # trace all exec() syscalls
    ./execsnoop -x        # include failed exec()s
    ./execsnoop -T        # include time (HH:MM:SS)
    ./execsnoop -U        # include UID
    ./execsnoop -u 1000   # only trace UID 1000
    ./execsnoop -u user   # get user UID and trace only them
    ./execsnoop -t        # include timestamps
    ./execsnoop -q        # add "quotemarks" around arguments
    ./execsnoop -n main   # only print command lines containing "main"
    ./execsnoop -l tpkg   # only print command where arguments contains "tpkg"
    ./execsnoop --cgroupmap ./mappath  # only trace cgroups in this BPF map
"""
parser = argparse.ArgumentParser(
    description="Trace exec() syscalls",
    formatter_class=argparse.RawDescriptionHelpFormatter,
    epilog=examples)
parser.add_argument("-T", "--time", action="store_true",
    help="include time column on output (HH:MM:SS)")
parser.add_argument("-t", "--timestamp", action="store_true",
    help="include timestamp on output")
parser.add_argument("-x", "--fails", action="store_true",
    help="include failed exec()s")
parser.add_argument("--cgroupmap",
    help="trace cgroups in this BPF map only")
parser.add_argument("-u", "--uid", type=parse_uid, metavar='USER',
    help="trace this UID only")
parser.add_argument("-q", "--quote", action="store_true",
    help="Add quotemarks (") around arguments."
    )
parser.add_argument("-n", "--name",
    type=ArgString,
    help="only print commands matching this name (regex), any arg")
parser.add_argument("-l", "--line",
    type=ArgString,
    help="only print commands where arg contains this line (regex)")
parser.add_argument("-U", "--print-uid", action="store_true",
    help="print UID column")
parser.add_argument("--max-args", default="20",
    help="maximum number of arguments parsed and displayed, defaults to 20")
parser.add_argument("--ebpf", action="store_true",
    help=argparse.SUPPRESS)
args = parser.parse_args()

# define BPF program
bpf_text = """
#include <uapi/linux/ptrace.h>
#include <linux/sched.h>
#include <linux/fs.h>

#define ARGSIZE  128

enum event_type {
    EVENT_ARG,
    EVENT_RET,
};

struct data_t {
    u32 pid;  // PID as in the userspace term (i.e. task->tgid in kernel)
    u32 ppid; // Parent PID as in the userspace term (i.e task->real_parent->tgid in kernel)
    u32 uid;
    char comm[TASK_COMM_LEN];
    enum event_type type;
    char argv[ARGSIZE];
    int retval;
};

#if CGROUPSET
BPF_TABLE_PINNED("hash", u64, u64, cgroupset, 1024, "CGROUPPATH");
#endif
BPF_PERF_OUTPUT(events);

static int __submit_arg(struct pt_regs *ctx, void *ptr, struct data_t *data)
{
    bpf_probe_read(data->argv, sizeof(data->argv), ptr);
    events.perf_submit(ctx, data, sizeof(struct data_t));
    return 1;
}

static int submit_arg(struct pt_regs *ctx, void *ptr, struct data_t *data)
{
    const char *argp = NULL;
    bpf_probe_read(&argp, sizeof(argp), ptr);
    if (argp) {
        return __submit_arg(ctx, (void *)(argp), data);
    }
    return 0;
}

int syscall__execve(struct pt_regs *ctx,
    const char __user *filename,
    const char __user *const __user *__argv,
    const char __user *const __user *__envp)
{

    u32 uid = bpf_get_current_uid_gid() & 0xffffffff;

    UID_FILTER

#if CGROUPSET
    u64 cgroupid = bpf_get_current_cgroup_id();
    if (cgroupset.lookup(&cgroupid) == NULL) {
      return 0;
    }
#endif

    // create data here and pass to submit_arg to save stack space (#555)
    struct data_t data = {};
    struct task_struct *task;

    data.pid = bpf_get_current_pid_tgid() >> 32;

    task = (struct task_struct *)bpf_get_current_task();
    // Some kernels, like Ubuntu 4.13.0-generic, return 0
    // as the real_parent->tgid.
    // We use the get_ppid function as a fallback in those cases. (#1883)
    data.ppid = task->real_parent->tgid;

    bpf_get_current_comm(&data.comm, sizeof(data.comm));
    data.type = EVENT_ARG;

    __submit_arg(ctx, (void *)filename, &data);

    // skip first arg, as we submitted filename
    #pragma unroll
    for (int i = 1; i < MAXARG; i++) {
        if (submit_arg(ctx, (void *)&__argv[i], &data) == 0)
             goto out;
    }

    // handle truncated argument list
    char ellipsis[] = "...";
    __submit_arg(ctx, (void *)ellipsis, &data);
out:
    return 0;
}

int do_ret_sys_execve(struct pt_regs *ctx)
{
#if CGROUPSET
    u64 cgroupid = bpf_get_current_cgroup_id();
    if (cgroupset.lookup(&cgroupid) == NULL) {
      return 0;
    }
#endif

    struct data_t data = {};
    struct task_struct *task;

    u32 uid = bpf_get_current_uid_gid() & 0xffffffff;
    UID_FILTER

    data.pid = bpf_get_current_pid_tgid() >> 32;
    data.uid = uid;

    task = (struct task_struct *)bpf_get_current_task();
    // Some kernels, like Ubuntu 4.13.0-generic, return 0
    // as the real_parent->tgid.
    // We use the get_ppid function as a fallback in those cases. (#1883)
    data.ppid = task->real_parent->tgid;

    bpf_get_current_comm(&data.comm, sizeof(data.comm));
    data.type = EVENT_RET;
    data.retval = PT_REGS_RC(ctx);
    events.perf_submit(ctx, &data, sizeof(data));

    return 0;
}
"""

bpf_text = bpf_text.replace("MAXARG", args.max_args)

if args.uid:
    bpf_text = bpf_text.replace('UID_FILTER',
        'if (uid != %s) { return 0; }' % args.uid)
else:
    bpf_text = bpf_text.replace('UID_FILTER', '')
if args.cgroupmap:
    bpf_text = bpf_text.replace('CGROUPSET', '1')
    bpf_text = bpf_text.replace('CGROUPPATH', args.cgroupmap)
else:
    bpf_text = bpf_text.replace('CGROUPSET', '0')
if args.ebpf:
    print(bpf_text)
    exit()

# initialize BPF
b = BPF(text=bpf_text)
execve_fnname = b.get_syscall_fnname("execve")
b.attach_kprobe(event=execve_fnname, fn_name="syscall__execve")
b.attach_kretprobe(event=execve_fnname, fn_name="do_ret_sys_execve")

# header
if args.time:
    print("%-9s" % ("TIME"), end="")
if args.timestamp:
    print("%-8s" % ("TIME(s)"), end="")
if args.print_uid:
    print("%-6s" % ("UID"), end="")
print("%-16s %-6s %-6s %3s %s" % ("PCOMM", "PID", "PPID", "RET", "ARGS"))

class EventType(object):
    EVENT_ARG = 0
    EVENT_RET = 1

start_ts = time.time()
argv = defaultdict(list)

# This is best-effort PPID matching. Short-lived processes may exit
# before we get a chance to read the PPID.
# This is a fallback for when fetching the PPID from task->real_parent->tgip
# returns 0, which happens in some kernel versions.
def get_ppid(pid):
    try:
        with open("/proc/%d/status" % pid) as status:
            for line in status:
                if line.startswith("PPid:"):
                    return int(line.split()[1])
    except IOError:
        pass
    return 0

# process event
def print_event(cpu, data, size):
    event = b["events"].event(data)
    skip = False

    if event.type == EventType.EVENT_ARG:
        argv[event.pid].append(event.argv)
    elif event.type == EventType.EVENT_RET:
        if event.retval != 0 and not args.fails:
            skip = True
        if args.name and not re.search(bytes(args.name), event.comm):
            skip = True
        if args.line and not re.search(bytes(args.line),
                                       b' '.join(argv[event.pid])):
            skip = True
        if args.quote:
            argv[event.pid] = [
                b""" + arg.replace(b""", b"\"") + b"""
                for arg in argv[event.pid]
            ]

        if not skip:
            if args.time:
                printb(b"%-9s" % strftime("%H:%M:%S").encode('ascii'), nl="")
            if args.timestamp:
                printb(b"%-8.3f" % (time.time() - start_ts), nl="")
            if args.print_uid:
                printb(b"%-6d" % event.uid, nl="")
            ppid = event.ppid if event.ppid > 0 else get_ppid(event.pid)
            ppid = b"%d" % ppid if ppid > 0 else b"?"
            argv_text = b' '.join(argv[event.pid]).replace(b'
', b'\n')
            printb(b"%-16s %-6d %-6s %3d %s" % (event.comm, event.pid,
                   ppid, event.retval, argv_text))
        try:
            del(argv[event.pid])
        except Exception:
            pass


# loop with callback to print_event
b["events"].open_perf_buffer(print_event)
while 1:
    try:
        b.perf_buffer_poll()
    except KeyboardInterrupt:
        exit()

目前都是依赖于 kprobe实现