cilium/ebpf Releasing the BTF package!

release · May 25, 2022

This release makes package btf public, allowing type information embedded in eBPF ELF objects and vmlinux to be read programmatically from Go. This enables use cases like searching and inspecting the kernel’s types and function signatures at runtime, or ensuring alignment between structures defined in Go and C. Package btf stands on its own and does not require adopting other subpackages, potentially making it useful beyond the domain of eBPF. Additionally, parsing BTF was made significantly faster and now consumes less memory.

Note: Modifications to btf.Types are not (yet) reflected in the BTF info loaded into the kernel. This will be implemented in the near future, and will provide the flexibility of, for example, creating maps with arbitrarily crafted key/value type information for pretty-printing map dumps, among many other exciting use cases.

Users of the ringbuf and perf packages can now avoid allocations when reading samples with the addition of ReadInto().

Feature probes for program helpers (features.HaveProgramHelper()) were added, as well as a few miscellaneous probes for large instruction limit, bounded loops and the supported eBPF ISA. This now brings the library mostly on par with bpftool’s probing capabilities, except for a few program types that require BTF.

There was also one important bugfix where CO-RE relocations in bpf2bpf subprograms were not applied. Users are strongly encouraged to upgrade.

Enjoy!

Breaking changes

• Deprecations:
  • CollectionSpec.RewriteMaps is deprecated in favour of CollectionOptions.MapReplacements
  • features.HaveProgType is deprecated in favour of features.HaveProgramType
• CollectionSpec.BTF is now called CollectionSpec.Types
• link.K(ret)probe and link.Tracepoint signatures have changed, they now take an additional options parameter. Pass nil to retain the old behaviour.

Features

• features: add misc probes by @florianl in https://github.com/cilium/ebpf/pull/541
• bpf2go: generate Go types from BTF by @lmb in https://github.com/cilium/ebpf/pull/552
• features: add LinuxVersionCode function to obtain LINUX_VERSION_CODE by @ti-mo in https://github.com/cilium/ebpf/pull/585
• link: support bpf perf event link, add bpf_cookies by @mmat11 in https://github.com/cilium/ebpf/pull/565
• link/kprobe: Allow . in symbol name by @chenhengqi in https://github.com/cilium/ebpf/pull/600
• btf: Implement support for bitfield relocations by @joamaki in https://github.com/cilium/ebpf/pull/573
• kprobe: fall back to tracefs if kernel doesn’t support dots in PMU symbol by @chenhengqi in https://github.com/cilium/ebpf/pull/605
• link/kprobe: specify symbol offset by @xpu22 in https://github.com/cilium/ebpf/pull/618
• collection: Add ReplaceMaps to CollectionOptions by @mauriciovasquezbernal in https://github.com/cilium/ebpf/pull/646
• fix: support bpf_core_type_exists() by @holyspectral in https://github.com/cilium/ebpf/pull/655
• perf, ringbuf: add ReadBuffer mehods by @lmb in https://github.com/cilium/ebpf/pull/663
• Make BTF for data sections optional by @ti-mo in https://github.com/cilium/ebpf/pull/675
• btf: Add spec types iterator by @brb in https://github.com/cilium/ebpf/pull/678
• features: add HaveProgramHelper API by @rgo3 in https://github.com/cilium/ebpf/pull/375
• Probe kernel bpf2bpf support by @ti-mo in https://github.com/cilium/ebpf/pull/657

Bug fixes

• elf_reader: Fix cgroup/sock_{create,release} program type by @joamaki in https://github.com/cilium/ebpf/pull/580
• Execute and test CO-RE relocations of bpf2bpf subprogs by @ti-mo in https://github.com/cilium/ebpf/pull/590
• syscalls: omit AttachType field in haveProgAttach by @ti-mo in https://github.com/cilium/ebpf/pull/652
• prog: recognize ENOTSUPP in testRun() by @ti-mo in https://github.com/cilium/ebpf/pull/653
• Assorted fixes for kernel 4.14 by @ti-mo in https://github.com/cilium/ebpf/pull/658

Miscellaneous

• collection: Rename CollectionSpec.BTF to .Types, copy in Copy() by @ti-mo in https://github.com/cilium/ebpf/pull/581
• collection: document garbage collection behaviour of *Map and *Program by @ti-mo in https://github.com/cilium/ebpf/pull/579
• prog_test: compare BTF program tags instead of raw insns by @ti-mo in https://github.com/cilium/ebpf/pull/583
• collection: don’t copy btf.Spec in CollectionSpec.Copy() by @ti-mo in https://github.com/cilium/ebpf/pull/584
• Moved Instruction.Symbol and Instruction.Reference into per-instruction metadata by @dylandreimerink in https://github.com/cilium/ebpf/pull/582
• examples: generate types via bpf2go by @lmb in https://github.com/cilium/ebpf/pull/575
• Add Maps to Instruction metadata by @dylandreimerink in https://github.com/cilium/ebpf/pull/567
• CI: use Go 1.18 by @lmb in https://github.com/cilium/ebpf/pull/599
• examples: include bpf_helpers.h from common.h by @ti-mo in https://github.com/cilium/ebpf/pull/614
• examples: add example for CO-RE kernel struct (tcp_sock) by @bigmikes in https://github.com/cilium/ebpf/pull/602
• examples: fentry - add comments to illustrate difference with tcprtt by @ti-mo in https://github.com/cilium/ebpf/pull/615
• btf: reduce unnecessary copies when parsing BTF by @lmb in https://github.com/cilium/ebpf/pull/623
• asm: add Instruction Source metadata by @lmb in https://github.com/cilium/ebpf/pull/609
• Memory improvements to loadRawSpec by @paulcacheux in https://github.com/cilium/ebpf/pull/634
• btf: split string table ahead of time by @lmb in https://github.com/cilium/ebpf/pull/637
• sys: update types by @florianl in https://github.com/cilium/ebpf/pull/630
• btf: fix up inflated types immediately if possible by @lmb in https://github.com/cilium/ebpf/pull/638
• Go 1.17 by @lmb in https://github.com/cilium/ebpf/pull/640
• btf: stop relying on TypeID when printing Types by @lmb in https://github.com/cilium/ebpf/pull/633
• link: do not use regexp pkg by @kolyshkin in https://github.com/cilium/ebpf/pull/642
• btf: make copying types infallible by @lmb in https://github.com/cilium/ebpf/pull/631
• Deprecate CollectionSpec.RewriteMaps() by @ti-mo in https://github.com/cilium/ebpf/pull/651
• CI: retry two times on error 42 by @lmb in https://github.com/cilium/ebpf/pull/654
• Use instruction metadata to simplify linking and CO-RE relocation by @lmb in https://github.com/cilium/ebpf/pull/606
• btf: allow users to obtain COREFixups by @lmb in https://github.com/cilium/ebpf/pull/661
• CI: run test suite against LTS kernel 4.14 by @ti-mo in https://github.com/cilium/ebpf/pull/670
• btf: introduce Bits type and remove Int.Bits and Int.Offset by @lmb in https://github.com/cilium/ebpf/pull/660
• ringbuf: reduce allocations and syscalls by @lmb in https://github.com/cilium/ebpf/pull/667
• Export package btf by @lmb in https://github.com/cilium/ebpf/pull/665
• btf: don’t cache kernel BTF in LoadKernelSpec by @lmb in https://github.com/cilium/ebpf/pull/676

New Contributors

• @ningmingxiao made their first contribution in https://github.com/cilium/ebpf/pull/568
• @dylandreimerink made their first contribution in https://github.com/cilium/ebpf/pull/582
• @xmulligan made their first contribution in https://github.com/cilium/ebpf/pull/589
• @bigmikes made their first contribution in https://github.com/cilium/ebpf/pull/602
• @gujun4990 made their first contribution in https://github.com/cilium/ebpf/pull/620
• @xpu22 made their first contribution in https://github.com/cilium/ebpf/pull/618
• @kolyshkin made their first contribution in https://github.com/cilium/ebpf/pull/642
• @mauriciovasquezbernal made their first contribution in https://github.com/cilium/ebpf/pull/646
• @holyspectral made their first contribution in https://github.com/cilium/ebpf/pull/655
• @brb made their first contribution in https://github.com/cilium/ebpf/pull/678

Full Changelog: https://github.com/cilium/ebpf/compare/v0.8.1...v0.9.0

View source