During the National Day holiday, NVIDIA published an article on its developer blog:
This means that starting from CUDA 11.8, developers can install new versions of the CUDA development package on Jetson just like on PCs!
This is indeed a good thing. Why do I say that?
Originally, there were two types of systems on Jetson: one is the NV packaged Jetpack, which ordinary developers are already familiar with. The other one is based on NV’s BSP package, a system made by third-party manufacturers (built on NV’s various components, including kernel source code).
However, in both cases, there was a fixed version of the CUDA development package/runtime/driver, such as 11.x or 10.x, built-in. It could not be changed. Once you choose to flash a certain version of JetPack, the version of CUDA is fixed to the version of JetPack.
However, many users requested the ability to download a new driver/development package directly, just like on PCs, to use the new CUDA version. Therefore, NVIDIA announced in this article that they provide this feature for Jetson as well. Whether you are using Jetpack or a third-party system, you can now install the CUDA development package like installing software (starting from CUDA 11.8).
CUDA Upgrade Package on Jetson
The reason it was not provided earlier is that CUDA on Jetson is different from that on ordinary systems; it is basically fixed with the CUDA driver in the system (developed from Jetpack or BSP), and it cannot be easily changed. The reason it cannot be easily changed is that the Jetson system often needs to be a stable system that has been time-tested (e.g., shipped to end users). If users can update freely, it would cause problems with the stability of the entire Jetpack system or the BSP-derived systems, which would require manufacturers to retest and revalidate stability, which is an issue. However, there is indeed a demand for being able to use the latest CUDA development packages released every three months, which creates a contradiction.
What to do?
NVIDIA decided to make a simple modification to the basic system (including Jetpack and systems derived from BSP) starting from Jetpack 5.0 and CUDA 11.8:
Starting from CUDA 11.8, CUDA introduces an upgrade path, providing Jetson developers with the option to update the CUDA driver and CUDA toolkit to the latest version.
The blue box shown in the figure below describes the components that are present by default in the NVIDIA JetPack 5.0 SDK. The dashed line separates the Jetson Linux BSP from other components of the NVIDIA JetPack SDK. The green box indicates the CUDA components you can upgrade to through this feature.
By introducing the CUDA driver upgrade (also known as the CUDA compatibility package), these upgrades become possible, as shown in Figure 2.
This upgrade package mainly includes the CUDA driver ( <span>libcuda.so.*</span>) and its dependencies, allowing you to access the latest and most powerful CUDA features that come with each quarterly CUDA version.
Without this package, you could only use the features provided by the default CUDA driver packaged in the Jetson Linux BSP. You had no mechanism to upgrade to the latest CUDA driver and toolkit.
With this package, existing Jetson users who have invested a long and thorough validation cycle into the Jetson Linux BSP can upgrade to the latest CUDA version. This upgrade is done on the existing Jetson Linux BSP, which remains unchanged.
In simple terms, it introduces something called an intermediate compatibility layer (cuCompat). Why does this solve the issue? Because the previously stable built-in driver directly interacted with lower-level components such as NVMAP/NVHOST/EGL/NVGPU, so it could not be easily changed. Now, starting from JP 5.0, the driver interacts with cuCompat, which then interacts with the lower-level NVMAP/NVHOST/EGL/NVGPU, thus adding an intermediate layer. You can simply understand it as adding an abstraction layer that separates from the original driver; this abstraction layer itself can be tested and stabilized. The new version of the driver becomes a higher-level software above it.
With this separation of layers, NVIDIA believes that drivers can now be updated freely (*), thus adding the following new features:
We can directly install new CUDA development packages (including drivers and development tools/runtimes, etc.) on the original version. After installation, users can directly use the new version of CUDA on Jetson, which is the latest CUDA version released every three months as mentioned in the original text. NVIDIA believes that the introduction of this cuCompat layer eliminates the previous drawbacks of the Jetson drivers directly interacting with lower-level components, allowing us to enjoy both stability and the latest features.
How to Upgrade CUDA on Jetson
In CUDA 11.8, the CUDA download page now shows a new architecture aarch64-Jetson, as shown in Figure 3, along with the related aarch64-Jetson CUDA installer, and provides step-by-step instructions on how to download and use the local installer or the CUDA network repository to install the latest CUDA version.
The new <span>aarch64-Jetson</span> CUDA installer packages the CUDA toolkit and upgrade package together. The provided step-by-step installation instructions ensure that the CUDA upgrade package and the corresponding CUDA toolkit for Jetson devices are downloaded and installed.
Note: NVIDIA emphasizes that currently only two versions of CUDA can coexist, unlike PCs that can support any number. These two versions are: the version that comes with Jetpack (or other Jetson systems derived from BSP) and the version installed by the user. If a user attempts to install 11.8 and 11.9 (which is not yet available), the latter will overwrite the former, meaning it will become the built-in version + 11.9, rather than built-in version + 11.8 + 11.9, which differs from PCs.
In summary, this is a great feature that introduces a new compatibility layer, allowing the Jetson system to maintain stability without frequent updates while still enjoying the benefits of the latest CUDA development packages.
More:
New Features Revealed in CUDA Toolkit 11.8
Are NVIDIA’s numerous Jetson modules compatible?