コンテンツにスキップ

Getting Started with pylon AI#

This topic explains how to set up your development environment for using pylon AI features.

In order to use pylon AI, you have to install the following components:

  • pylon Software Suite
  • pylon Supplementary Package for pylon AI
    The installation of the supplementary package is described in this topic.

The pylon Supplementary Package for pylon AI delivers the backend service that allows the AI vTools to execute deep learning models. This backend service is the NVIDIA Triton Inference Server with Basler add-ons that support model encryption and licensing.

This is an overview of where the Triton Inference Server is placed in the development environment:

Overview of AI Development Environment

You have two options for installing the Triton Server:

  • On a local computer
  • On a remote computer in the network

The first option is recommended for working alone on your own projects while the second option is better suited when working in a team.

Basler doesn't provide live Triton server instances and doesn't recommend publishing the server as a public internet endpoint. You should always consider it as part of your development environment and protect it accordingly.

The pylon Supplementary Package for pylon AI includes a modified Docker image to prepare the Triton Inference Server for use with the pylon AI vTools and the pylon AI Agent. It also contains setup and launch scripts, as well as release notes and third-party licenses information.

システム要件#

Ubuntu and Ubuntu for WSL#

  • x86_64 CPU：最新のIntel Core i5（またはそれ以降）または同等のもの。
  • Memory: Minimum 8 GB of available RAM. 16 GB or more of available RAM are recommended.
  • ディスクタイプ：SSDをお勧めします。
  • Disk space: Minimum 20 GB of free disk space for installation. The space required for the model repository depends on the size and number of models you plan to deploy. Models can range from a few megabytes to several gigabytes each.
  • GPU models: All NVIDIA models based on the architectures mentioned here
  • CUDA version: 12+
  • GPU cards: Examples of suitable cards include NVIDIA RTX A5000, NVIDIA RTX A2000 Laptop GPU, NVIDIA Tesla T4, etc.

Jetson Family with JetPack 6#

  • Modules: All NVIDIA Jetson
  • Memory: Minimum 8 GB of available RAM. 16 GB or more of available RAM are recommended.
  • ディスクタイプ：SSDをお勧めします。
  • Disk space: Minimum 20 GB of free disk space for installation. The space required for the model repository depends on the size and number of models you plan to deploy. Models can range from a few megabytes to several gigabytes each.

前提条件#

情報

  • Internet connection is required during the installation process.

  • Make sure that the ports 8000, 8001, 8002, and 22350 are not blocked on your local computer.
    Due to the variety and complexity of possible scenarios at the user's end, Basler is unable to provide a general solution. Therefore, potential access issues have to be handled by customers themselves with support from their IT departament.

Ubuntu#

For more information, see install docker on Ubuntu.

If your computer has an NVIDIA GPU, see Installing NVIDIA drivers and the NVIDIA Container Toolkit.

Ubuntu for WSL#

情報

Don't install any Linux display drivers inside WSL. Drivers should only be installed directly on Windows.

Follow these instructions to install Ubuntu OS on WSL.

Follow these instructions to install Docker within Ubuntu on WSL.

Jetson Family (JetPack 6.x)#

Your device must be flashed using SDK Manager. JetPack 6.x is required. This already includes Docker.

All OS Variants#

After you have prepared the operating system, make sure that the following requirements are fulfilled:

  • CodeMeter Runtime is installed.
    The installation of CodeMeter Runtime is part of the pylon Software Suite installation.
  • Licenses for AI vTools are available.
    For more information, visit the Basler website.

  • Docker Engine is installed.
    To check whether you have Docker Engine installed:

    docker --version

  • For GPU usage: NVIDIA Container Toolkit is installed.
    To check whether you have this toolkit installed:

    nvidia-container-toolkit -version

    If it hasn't been installed yet, read the following section.

Installing NVIDIA Drivers and the NVIDIA Container Toolkit (Required for GPU Usage)#

The NVIDIA Container Toolkit provides the libraries and tools required for GPU-accelerated computing. The Triton Inference Server uses CUDA to run GPU-accelerated operations, like model inference. The NVIDIA Container Toolkit version should match the version expected by the Triton Server and the models you are deploying.

To check the version of your NVIDIA drivers, open your terminal and type this command:

nvidia-smi

If no drivers or toolkit have been installed yet, follow the instructions on this page: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

Check this page to ensure that your NVIDIA driver is compatible with the NVIDIA Container Toolkit and the Triton Inference Server: https://docs.nvidia.com/deeplearning/triton-inference-server/release-notes/rel-24-11.html#rel-24-11

Installing the Triton Inference Server#

Online Installation#

To install the Triton Inference Server from the pylon Supplementary Package for pylon AI, follow these steps:

  1. Change to the directory that contains the pylon-supplementary-package-for-pylon-ai-*.tar.gz archive that you downloaded from the Basler website.

  2. Extract the pylon Supplementary Package for pylon AI tar.gz archive into a directory of your choice (replace ./pylon_ai_setup if you want to extract into a different directory).

    mkdir pylon_ai_setup && tar -xzvf ./pylon-supplementary-package-for-pylon-ai-*.tar.gz -C pylon_ai_setup

  3. Switch to the folder into which you extracted the pylon Supplementary Package for pylon AI.

    cd pylon_ai_setup

  4. Run the setup script with root permissions.

    sudo chmod +x setup-pylon-ai-triton-inference-server.sh && sudo ./setup-pylon-ai-triton-inference-server.sh

Offline Installation#

In cases where internet connection is not available, follow these steps.

  1. On a computer with internet connection, visit dockerhub and download the Basler Triton Server Docker image.
    The image is available under this link: https://hub.docker.com/repository/docker/basler/tritonserver/general

  2. Transfer the image on the computer on which you want to install the Triton Server.
    You can use the following commands to perform step 1 and 2.

    docker pull basler/tritonserver:[tagname]

    docker save basler/tritonserver:[tagname] | gzip > tritonserver.[tagname].tgz

    例： 

    docker image save basler/tritonserver:24.11-py3_1.3.0 | gzip > tritonserver.24.11-py3_1.3.0_rc4.tgz

  3. Change to the directory that contains the pylon-supplementary-package-for-pylon-ai-*.tar.gz archive that you downloaded from the Basler website.

  4. Extract the pylon Supplementary Package for pylon AI tar.gz archive into a directory of your choice (replace ./pylon_ai_setup if you want to extract into a different directory).

    mkdir pylon_ai_setup && tar -xzvf ./pylon-supplementary-package-for-pylon-ai-*.tar.gz -C pylon_ai_setup

  5. Copy saved .tar file to the pylon_ai_setup folder.

  6. Run the setup script with root permissions.

    sudo chmod +x setup-pylon-ai-triton-inference-server.sh && sudo ./setup-pylon-ai-triton-inference-server.sh

Launching the Triton Inference Server#

This section details the three different ways to start the Inference Server. Choose the one that best suits your needs bearing in mind the following information regarding running the server and vTools remotely.

Running Server and vTools Remotely

If you're running the Triton Inference Server on a remote computer and are also running the AI vTools or AI Agent from another location, be aware of the following important restrictions and considerations:

  • The time it takes for data to travel between the vTools making the inference request and the Triton Inference Server can impact the overall response time.
  • High latency can slow down the inference process, especially for real-time applications.
  • Limited network bandwidth can lead to slow or unreliable communication, particularly when transferring large amounts of data.
  • Remote functionalities require ports 8000, 8001, and 8002 to be accessible. (This may require support from your IT department, e.g., running firewall and proxy checks.)

Launching the Server When Required#

If you don't want to launch the Triton Inference Server as a Docker container with the startup of your computer, you can use the following shell script to start the Triton Inference Server when required.

  1. Switch to the folder into which you extracted the pylon Supplementary Package for pylon AI.

  2. Launch the Triton Inference Server.

    sudo chmod +x run-pylon-ai-triton-inference-server.sh && sudo ./run-pylon-ai-triton-inference-server.sh

Launching the Server on Boot#

To start the Triton Inference Server Docker container automatically when you boot your computer or restart it automatically in case of a crash, you can create a systemd service that meets your needs by running the "run-pylon-ai-triton-inference-server.sh" shell script with additional parameters.

  1. Switch to the folder into which you extracted the pylon Supplementary Package for pylon AI.

  2. Launch installer script.

    sudo chmod +x run-pylon-ai-triton-inference-server.sh && sudo ./run-pylon-ai-triton-inference-server.sh --service

After you have installed the Triton Inference Server as a service, you can stop it temporarily when required. Bear in mind that it will be started automatically after a reboot of your device.

To temporarily stop the systemd service responsible for running the Triton Inference Server, open the terminal (in any directory) and run:

sudo systemctl stop tritonserver.service

If you want to bring back the service, you have to start it by running:

sudo systemctl start tritonserver.service

If you're having issues with the currently running server managed by the service, you can restart it by running:

sudo systemctl restart tritonserver.service

If you want to permanently disable the service, you can do this by running:

sudo systemctl disable tritonserver.service

Launching the Server Manually#

Customizing the Configuration#

Start the Docker container and replace the following parameters as required:

  • [MODEL_REPOSITORY]: Home directory for models. Make sure the same location is used by the pylon AI Agent.
  • [HTTP_PORT]: HTTP port
  • [GRPC_PORT]: GRPC port
  • [METRICS_PORT]: Metrics port
  • [IMAGE_TAG]: Image tag
docker run --rm --gpus=1 -p[HTTP_PORT]:8000 -p[GRPC_PORT]:8001 -p[METRICS_PORT]:8002 --shm-size 2g -v[MODEL_REPOSITORY]:/models basler/tritonserver:[IMAGE_TAG] tritonserver --model-repository=/models --model-control-mode=explicit --load-model="*"
Examples for Starting the Container#
  • Ubuntu
docker run --rm --gpus=1 -p8000:8000 -p8001:8001 -p8002:8002 --shm-size 2g -v/opt/pylonai/model_repository:/models basler/tritonserver:24.11-py3_1.3.0 tritonserver --model-repository=/models --model-control-mode=explicit --load-model="*"
  • Jetson Family
docker run --rm -p8000:8000 -p8001:8001 -p8002:8002 --shm-size 2g -v/opt/pylonai/model_repository:/models basler/tritonserver:24.11-py3_1.3.0 tritonserver --model-repository=/models --model-control-mode=explicit --load-model="*"

ライセンス#

You can find licensing information here or in the license file contained in the pylon Supplementary Package for pylon AI.

トラブルシューティング#

CodeMeter is configured during the pylon installation so that the Triton Inference Server can access the required licenses.

In rare cases, this may fail. If that happens, configure CodeMeter as a network server:

  1. Open CodeMeter Control Center and click the WebAdmin button at the bottom right.
  2. In CodeMeter WebAdmin, navigate to Configuration > Server > Server Access.
  3. In the Network Server area, select Enable and confirm with Apply.
  4. Return to the CodeMeter Control Center.
  5. In the Process menu, click Restart CodeMeter Service to restart CodeMeter as a network server.

Known Issues#

None