The 🤗 Kernels project aims to revolutionize how custom kernels are created, shared, and utilized within the AI landscape. Our core mission is to make this process frictionless, secure, and deeply integrated with the Hugging Face Hub. After months of intensive development and a significant redesign, we’re excited to unveil major updates that transform the project’s capabilities.
A pivotal enhancement is the introduction of a dedicated “kernel” repository type on the Hugging Face Hub. This specialized category empowers users to quickly understand a kernel’s specific compute requirements, such as supported accelerators, operating systems, and backend versions. You can explore all available kernels and their details at huggingface.co/kernels.
This strategic move not only provides tailored information but also elevates kernels to first-class citizens of the Hub. It significantly boosts discoverability, allowing the community to observe trends across kernels, models, and their applications. This fosters a more integrated and dynamic AI development ecosystem for custom compute.
Fortifying Security: Trusted Publishers and Code Signing
Security remains paramount for the Kernels project, as custom kernels execute native code with the same privileges as your Python process. A malicious kernel could inflict real harm, making robust safeguards essential. We’ve always emphasized reproducibility, enabling you to recompile and verify kernels against their public source using Nix for pure, isolated builds and embedding the Git SHA1 for provenance.
Building on these foundations, we’ve rolled out two critical new layers of defense: trusted kernel publishers and an advanced code signing mechanism. By default, the `kernels` package now only loads kernels from organizations explicitly trusted by the community. This significantly mitigates the risk of executing inadvertently compromised code.
For flexibility, you can explicitly opt-in to load kernels from untrusted sources by setting trust_remote_code=True: from kernels import get_kernel; kernel_module = get_kernel("Atlas-Inference/gdn", version=1, trust_remote_code=True). Users must now request access to publish kernel repositories on the Hub, allowing us to carefully review each application and ensure community safety.
Kernel signing offers an additional protective layer, addressing scenarios where a trusted publisher’s Hub credentials might be compromised. Kernels are now cryptographically signed using a private key unique to the developer, verifiable with a public key. This prevents attackers from signing and distributing malicious kernels, even if they gain repository access.
We leverage Sigstore’s cosign for signing with ephemeral private keys, which are only valid for a limited duration, further minimizing risk. We also verify that the kernel’s signature originates from a trusted GitHub workflow and repository. While `kernel-builder` fully supports this, `kernels` will enable signature verification upon loading after further testing. Setup details are in the kernels 0.16.0 release notes.
Streamlined Development: CLIs, Agentic AI, and Compatibility
Our Command Line Interfaces (CLIs) have been completely revamped for improved clarity and maintainability. `kernels` now focuses on loading and preparing kernels, while `kernel-builder` is dedicated solely to the building process. This specialization makes both tools leaner, more intuitive, and highly efficient for developers.
We’ve also significantly expanded support for various frameworks and backends, ensuring the Kernels project remains versatile across a wide array of AI development needs. This broad compatibility makes working with custom kernels more accessible. Comprehensive details on these updates are available in our latest documentation.
A particularly exciting development is the robust foundation we’ve laid for agentic kernel development. The combined power of `kernel-builder` and `kernels` now enables AI agents to generate and optimize kernels from scratch. Agents can scaffold, build, benchmark, and iteratively refine kernels, accelerating the pace of innovation in custom compute.
`kernel-builder` is designed with agents in mind, enforcing a predictable structure for source code and offering an agent-optimized CLI for programmatic interpretation. Moreover, tight integration with Hugging Face Jobs facilitates easy benchmarking across diverse hardware, allowing agents to collect performance data and inform continuous optimization.
Enhanced User Experience and Technical Refinements
Setting up your development environment for `kernel-builder` is now significantly simpler. We offer a convenient one-click installation script, and for ephemeral instances, our Terraform setup guide ensures streamlined deployment. You can now concentrate on kernel development, not configuration complexities.
Upon building, each kernel automatically receives a detailed System Card, outlining its usage and interfaces. When pushed to the Hugging Face Hub, this card becomes the kernel’s front matter, offering immediate transparency and simplifying discovery for the community. Users can quickly understand and utilize new kernels effectively.
Determining a kernel’s compatibility with your specific system is straightforward with our updated API. The has_kernel() method provides a quick boolean answer. For detailed insights, get_kernel_variants() offers granular explanations, like “CPU (x86_64) does not match system CPU (aarch64)”, proving invaluable for informed development decisions.
We’ve critically improved our manylinux_2_28 support. Previously, static linking of `libstdc++` sometimes led to conflicts and segfaults with dynamically linked versions from dependencies like PyTorch, due to global initialization issues. This impacted the stability of advanced C++ features within kernels.
Our solution ensures kernels now dynamically link `libstdc++`, effectively eliminating these conflicts and enhancing stability. This is achieved by compiling kernels with the official `manylinux_2_28` toolchain, providing robust compatibility. Developers can now leverage modern C++ features with confidence, avoiding unexpected runtime errors.
The 🤗 Kernels project is continuously evolving to offer the best possible experience for both kernel developers and users. We deeply value community feedback and encourage contributions to help shape its future. Your insights are essential as we continue to build and refine this powerful and secure ecosystem!
Source: Hugging Face Blog
