Edge AI Acceleration - Qualcomm Cloud AI 100 Ubuntu Configuration and SDK Setup

I am configuring a server with a Qualcomm Cloud AI 100 (AIC100) accelerator, and I need the correct drivers and platform SDK to run machine learning models. You can find these devices online. For example, this eBay listing is provided purely as a reference so you know what hardware to look for, though you should always verify the seller’s legitimacy before making a purchase. Similar to standard Ubuntu configuration guides, this walkthrough covers the exact steps to install the Qualcomm Package Manager (QPM3), resolve Linux kernel compatibility issues, build the AIC100 driver, and run a test model.

Update System and Install Prerequisites

Before installing the Qualcomm packages, update your package lists and install the necessary build tools and kernel headers:

1
sudo apt update sudo apt install xdg-utils dkms build-essential linux-headers-$(uname -r)

Next, install the Qualcomm Package Manager (QPM3) using the downloaded Debian package:

1
sudo dpkg -i ./QualcommPackageManager3.3.0.131.3.Linux-x86.deb

Install the AIC100 Driver

Use QPM3 to install the base AIC100 ultra driver package via the command line interface:

1
sudo qpm3 install ./aic100-ultra-driver.qpm --cli-only --no-sandbox

Fix Kernel Compatibility

The AIC100 module relies on specific kernel versions to build and load successfully. If checking the loaded modules (lsmod | grep aic100) returns nothing, you must install a supported kernel.

Install the Required Kernel

Install the 5.15.0-173-generic kernel and its corresponding headers:

1
sudo apt install --install-recommends linux-image-5.15.0-173-generic linux-headers-5.15.0-173-generic

Update GRUB and Reboot

Set the newly installed kernel as the default boot option in GRUB:

1
sudo grub-set-default "Advanced options for Ubuntu>Ubuntu, with Linux 5.15.0-173-generic" sudo update-grub

Remove any conflicting newer kernels, update GRUB again, and reboot the system for the changes to take effect:

1
sudo apt autoremove --purge sudo update-grub sudo reboot

Install the QAIC Platform SDK

After logging back in, confirm you are running the correct kernel by typing uname -r. Once verified, navigate to your extracted SDK directory and run the installation script to compile the kernel modules:

1
2
cd ~/qaic-platform-sdk-1.20.1.2/x86_64/deb
sudo ./install.sh

Verify Hardware and Run Inference

Check that the system correctly recognizes the PCIe device and has loaded the qaic and aic100 drivers:

1
lsmod | grep aic100 lsmod | grep qaic lspci | grep -i qualcomm aic100u-info

Finally, validate the hardware setup by activating the Qualcomm efficient transformers virtual environment and running a GPT-2 text generation script:

1
2
3
source /opt/qeff-env/bin/activate && \
cd /workspace/efficient-transformers/examples && \
python text_generation/basic_inference.py --model-name gpt2 --prompt "Hello, how are you?"

References & Further Reading