cilium/ebpf Support for __ksym and __kfunc
release ·
Breaking changes
This release requires at least Go 1.19.
github.com/cilium/ebpf/btf:
Handle.Spec()now takes abase *Specargument.nilis accepted if the Handle is for vmlinux. If Handle is for a (split BTF) kernel module, pass the result ofLoadKernelSpec().NewHandle()now takes a*Builderinstead of a*Spec. Loading BTF into the kernel now goes via a newBuildertype. See the ‘Additions’ section below.
github.com/cilium/ebpf/link:
- KprobeMultiOptions.Addresses: changed from []uint64 to []uintptr The old API didn’t take 32-bit architectures into account.
Fixes
Work around the kernel rejecting some Datasec
The kernel erroneously rejects Datasec where a Typedef, Volatile, Const, Restrict or typeTag follows a Pointer, Struct, Union or Array. There is now a workaround in place, see https://github.com/cilium/ebpf/pull/954.
Marshaling Map keys and values is more efficient
The marshaling code in the library now uses sync.Pool to re-use bytes.Buffer, which makes common map operations cheaper, see https://github.com/cilium/ebpf/pull/1053.
CO-RE relocations are a lot more efficient
The CO-RE code now does much less copying of BTF types, which makes CO-RE relocation a lot faster, especially against large types such as sk_buff. See https://github.com/cilium/ebpf/pull/1084.
Additions
__ksym (kfunc) support
It’s now possible to use new-style BPF helpers aka kfunc with the library. Going forward, all new BPF ‘helper’ functionality in the kernel will be exposed as kfuncs, and new helpers won’t be added. See https://github.com/cilium/ebpf/pull/966 and https://github.com/cilium/ebpf/pull/996.
__kconfig support
Tracing programs often need to vary their behaviour based on kernel configuration, such as CONFIG_HZ. Such references to __kconfig variables are now automatically populated from a variety of sources such as /proc/config.gz. Note that if you run your application implementing ebpf-go in a container, it will need access to the host’s /boot on some distributions that don’t ship /proc/config(.gz). (notably, Debian-based distros)
As a special mention, the LINUX_HAS_SYSCALL_WRAPPER kconfig is also supported. This allows writing portable kprobes using the BPF_KSYSCALL macro from bpf_tracing.h.
See https://github.com/cilium/ebpf/pull/951, https://github.com/cilium/ebpf/pull/960 and https://github.com/cilium/ebpf/pull/995.
Overwritable perf buffer support
The perf reader now allows creating “overwritable” perf buffers, which always contain the most recent events in case the buffer ever gets full. This is in contrast to regular perf buffers which drop recent events if there is no more space. This is useful to implement “flight recorder” type functionality for events sourced from BPF.
See https://github.com/cilium/ebpf/pull/953.
btf.Builder
Constructing custom BTF type blobs is now possible through the new btf.Builder type. Call btf.NewBuilder() to obtain one, Builder.Add(t btf.Type) to add any types you need, followed by btf.NewHandle(b btf.Builder) to load the types into the kernel. Builder also has a Marshal() method for serializing the type collection into the canonical BTF format so it can be stored for later use.
What’s Changed
- cs.RewriteConstants: define error MissingConstantsError by @alban in https://github.com/cilium/ebpf/pull/904
- internal: add Memoize by @lmb in https://github.com/cilium/ebpf/pull/909
- Fix ProgramInfo.MapIDs for programs without maps on old kernels by @lmb in https://github.com/cilium/ebpf/pull/905
- link: make KprobeMultiOptions.Addresses a []uintptr by @lmb in https://github.com/cilium/ebpf/pull/913
- btf: export API to create BTF from scratch by @lmb in https://github.com/cilium/ebpf/pull/859
- Update CI dependencies by @lmb in https://github.com/cilium/ebpf/pull/916
- map: replace hacky unsafe.Pointer conversion with PROT_NONE page by @lmb in https://github.com/cilium/ebpf/pull/915
- link: use strings.Map in sanitizeSymbol by @lmb in https://github.com/cilium/ebpf/pull/919
- btf: skip .data..percpu in TestRoundtripVMlinux by @lmb in https://github.com/cilium/ebpf/pull/922
- link: close perfEventLink.fd before perf event in Close() by @ti-mo in https://github.com/cilium/ebpf/pull/918
- map: ignore BPF_F_RDONLY_PROG for DevMap compatibility check by @aibor in https://github.com/cilium/ebpf/pull/930
- Remove deprecated
+buildGo build tags by @ti-mo in https://github.com/cilium/ebpf/pull/888 - link: use /sys/kernel/tracing if available by @spikat in https://github.com/cilium/ebpf/pull/906
- collection: work around excessive copying of kernel BTF by @alban in https://github.com/cilium/ebpf/pull/920
- sys: add fd leak tracing instrumentation by @ti-mo in https://github.com/cilium/ebpf/pull/732
- program: set name from object info for pinned file by @aibor in https://github.com/cilium/ebpf/pull/932
- Allow specifying the tracefs prefix in options by @brycekahle in https://github.com/cilium/ebpf/pull/842
- Fixed broken links pointing to Cilium doc by @PhilipSchmid in https://github.com/cilium/ebpf/pull/942
- link: use statfs for tracefs mount detection by @brycekahle in https://github.com/cilium/ebpf/pull/944
- btf: ensure that TypesIterator is not affected by Spec.Add by @lmb in https://github.com/cilium/ebpf/pull/937
- run-tests: allow using local kernel by @lmb in https://github.com/cilium/ebpf/pull/952
- require Go 1.19 by @lmb in https://github.com/cilium/ebpf/pull/950
- Support __kconfig macro for LINUX_KERNEL_VERSION by @eiffel-fl in https://github.com/cilium/ebpf/pull/951
- btf: work around kernel Datasec bug by @lmb in https://github.com/cilium/ebpf/pull/954
- link: always include requested symbol in kprobe creation errors by @ti-mo in https://github.com/cilium/ebpf/pull/959
- Fix expected fs type for /sys/kernel/debug/tracing by @brycekahle in https://github.com/cilium/ebpf/pull/958
- collection: ensure LINUX_KERNEL_VERSION variable is a btf.Int. by @eiffel-fl in https://github.com/cilium/ebpf/pull/961
- add tests for FSType and getTracefsPath by @lmb in https://github.com/cilium/ebpf/pull/962
- Link to maintainers file by @xmulligan in https://github.com/cilium/ebpf/pull/957
- docs: update various bits of documentation by @lmb in https://github.com/cilium/ebpf/pull/936
- fix some comments by @cuishuang in https://github.com/cilium/ebpf/pull/972
- link: new link from fd by @Asphaltt in https://github.com/cilium/ebpf/pull/971
- Add support for loong64 by @zhaixiaojuan in https://github.com/cilium/ebpf/pull/975
- Allow debugfs fstype for /sys/kernel/debug/tracing by @brycekahle in https://github.com/cilium/ebpf/pull/976
- Add error case for creating unspecified map type. by @tommyp1ckles in https://github.com/cilium/ebpf/pull/974
- perf: Add support for overwritable buffer. by @eiffel-fl in https://github.com/cilium/ebpf/pull/953
- update
platformPrefixlist with libbpf values by @paulcacheux in https://github.com/cilium/ebpf/pull/982 - link: document AttachCgroup and flags by @ti-mo in https://github.com/cilium/ebpf/pull/980
- perf: clean up tests by @lmb in https://github.com/cilium/ebpf/pull/979
- btf: remove Spec.firstTypeID() by @lmb in https://github.com/cilium/ebpf/pull/990
- elf_reader, linker: add kfunc support by @rgo3 in https://github.com/cilium/ebpf/pull/966
- CI: enable gofmt linter by @lmb in https://github.com/cilium/ebpf/pull/992
- link: use BPF links to attach Tracing and LSM prog types by @mmat11 in https://github.com/cilium/ebpf/pull/837
- bpf2go: spring cleaning by @lmb in https://github.com/cilium/ebpf/pull/910
- btf: fix off-by-one in Spec.TypeByID by @lmb in https://github.com/cilium/ebpf/pull/993
- link/executable: lazy load symbol table by @mmat11 in https://github.com/cilium/ebpf/pull/991
- btf: remove type alias for []Type by @lmb in https://github.com/cilium/ebpf/pull/989
- sys: introduce TypeID by @lmb in https://github.com/cilium/ebpf/pull/912
- elf: freeze .kconfig map by @lmb in https://github.com/cilium/ebpf/pull/1000
- btf: clean up handling of split BTF in loadRawSpec and inflateRawTypes by @lmb in https://github.com/cilium/ebpf/pull/999
- elf: use per-instruction metadata for .kconfig references by @lmb in https://github.com/cilium/ebpf/pull/994
- btf: avoid repeated kernel BTF copies in Handle.Spec by @lmb in https://github.com/cilium/ebpf/pull/1001
- docs: document API stability by @lmb in https://github.com/cilium/ebpf/pull/1003
- bpf2go: Fall back to default module name when debug.ReadBuildInfo is not available by @folbricht in https://github.com/cilium/ebpf/pull/1004
- update github.com/frankban/quicktest by @lmb in https://github.com/cilium/ebpf/pull/1010
- asm: update eBPF built-in functions by @florianl in https://github.com/cilium/ebpf/pull/1011
- ProgramInfo provides CreatedByUid by @zachcheu in https://github.com/cilium/ebpf/pull/1006
- btf: optimize Spec.Copy by @lmb in https://github.com/cilium/ebpf/pull/1002
- Add support for
LINUX_HAS_SYSCALL_WRAPPERkconfig by @paulcacheux in https://github.com/cilium/ebpf/pull/995 - program: improve error handling in haveSyscallWrapper by @lmb in https://github.com/cilium/ebpf/pull/1022
- fix
container-allmake invocation by @paulcacheux in https://github.com/cilium/ebpf/pull/1021 - make: allow interrupting container-all with ctrl-c by @lmb in https://github.com/cilium/ebpf/pull/1024
- refactor tracefs event handling into an object by @lmb in https://github.com/cilium/ebpf/pull/1023
- Print out the relocation name in the error message by @kuroa-me in https://github.com/cilium/ebpf/pull/1026
- allow skipping kernel version check for feature tests if test skipped via env variable by @abhipranay in https://github.com/cilium/ebpf/pull/1025
- replace deprecated rand.Seed calls by @kwakubiney in https://github.com/cilium/ebpf/pull/978
- elf: support kernel module kfunc calls by @shun159 in https://github.com/cilium/ebpf/pull/996
- tracefs: minor cleanups by @lmb in https://github.com/cilium/ebpf/pull/1027
- elf_reader: Add support for XDP frags by @ptzianos in https://github.com/cilium/ebpf/pull/1037
- program: move examples to VerifierError in main package by @lmb in https://github.com/cilium/ebpf/pull/1040
- sys: prevent clobbering existing entries in fd tracing registry by @ti-mo in https://github.com/cilium/ebpf/pull/1043
- btf: avoid triggering feature probe on import by @lmb in https://github.com/cilium/ebpf/pull/1045
- program: update Benchmark doc by @lmb in https://github.com/cilium/ebpf/pull/1046
- collection: Add code to parse kernel config files by @eiffel-fl in https://github.com/cilium/ebpf/pull/960
- btf: move LoadKernelSpec into separate package by @lmb in https://github.com/cilium/ebpf/pull/1015
- link: add test for UDP iter by @lmb in https://github.com/cilium/ebpf/pull/1044
- marshalers: Pool bytes.Buffers in marshalbytes by @ti-mo in https://github.com/cilium/ebpf/pull/1053
- CI: try to fix job timeouts by @lmb in https://github.com/cilium/ebpf/pull/1056
- Pre-allocate map sys.Errors by @hmahmood in https://github.com/cilium/ebpf/pull/1052
- internal/sys: don’t panic in init() by @lmb in https://github.com/cilium/ebpf/pull/1055
- internal/kconfig: Add possibility to filter kconfig file by @eiffel-fl in https://github.com/cilium/ebpf/pull/1051
- bpf2go: argument parsing unit test by @mejedi in https://github.com/cilium/ebpf/pull/1054
- bpf2go: some options take defaults from the environment by @mejedi in https://github.com/cilium/ebpf/pull/1047
- map: disentangle per-CPU handling by @lmb in https://github.com/cilium/ebpf/pull/1067
- btf: move Spec.Add to dedicated type by @lmb in https://github.com/cilium/ebpf/pull/1069
- Keep code multi OS aware by @florianl in https://github.com/cilium/ebpf/pull/1075
- Benchmark batch lookups vs. MapIterator, remove MapIterator.prevKey by @ti-mo in https://github.com/cilium/ebpf/pull/1077
- testutils: print newline after random seed by @lmb in https://github.com/cilium/ebpf/pull/1085
- btf: don’t copy types during CO-RE relocation by @lmb in https://github.com/cilium/ebpf/pull/1084
- Makefile: require user to provide extract-vmlinux by @lmb in https://github.com/cilium/ebpf/pull/1088
- btf: fix CO-RE bitfield relocations by @lmb in https://github.com/cilium/ebpf/pull/1086
New Contributors
- @alban made their first contribution in https://github.com/cilium/ebpf/pull/904
- @spikat made their first contribution in https://github.com/cilium/ebpf/pull/906
- @PhilipSchmid made their first contribution in https://github.com/cilium/ebpf/pull/942
- @eiffel-fl made their first contribution in https://github.com/cilium/ebpf/pull/951
- @cuishuang made their first contribution in https://github.com/cilium/ebpf/pull/972
- @Asphaltt made their first contribution in https://github.com/cilium/ebpf/pull/971
- @zhaixiaojuan made their first contribution in https://github.com/cilium/ebpf/pull/975
- @tommyp1ckles made their first contribution in https://github.com/cilium/ebpf/pull/974
- @zachcheu made their first contribution in https://github.com/cilium/ebpf/pull/1006
- @kuroa-me made their first contribution in https://github.com/cilium/ebpf/pull/1026
- @abhipranay made their first contribution in https://github.com/cilium/ebpf/pull/1025
- @kwakubiney made their first contribution in https://github.com/cilium/ebpf/pull/978
- @shun159 made their first contribution in https://github.com/cilium/ebpf/pull/996
- @ptzianos made their first contribution in https://github.com/cilium/ebpf/pull/1037
- @hmahmood made their first contribution in https://github.com/cilium/ebpf/pull/1052
- @mejedi made their first contribution in https://github.com/cilium/ebpf/pull/1054
Full Changelog: https://github.com/cilium/ebpf/compare/v0.10.0...v0.11.0