Title

NVIDIA Triton LLM Inference Server for Llama3.2 models

About Or Title

About This Workshop

Desc Long

Leverage the power of NVIDIA GPUs and advanced deep learning frameworks, such as Triton and TensorRT, to gain hands-on experience deploying scalable Llama 3.2 models for large language model inference. This workshop offers a comprehensive introduction to deploying and optimizing AI models using NVIDIA Triton, focusing on key tools and techniques to enhance inference performance and reduce latency. Attendees will explore real-world scenarios using A10 shape standalone or A10 shape within OKE, learning to streamline model management and fully utilize GPU resources for efficient AI deployments.

Workshop Length

1 hour

Outline

Lab 1 - Provision the resources for A10 instance
Lab 2 - Provision the resources for oke

Prerequisites

Administrative access to OCI Tenancy
Ability to spin-up A10 instances in OCI
Ability to create resources with Public IP addresses (load Balancer, Instances, OCI API Endpoint)

Other Workshops you might like

NVIDIA Morpheus on OCI: AI-Driven Cybersecurity Powered by NVIDIA GPUs

NVIDIA Morpheus on OCI: AI-Driven Cybersecurity Powered by NVIDIA GPUs

Automated NVIDIA Morpheus with Oracle Cloud Infrastructure (OCI) by leveraging NVIDIA GPUs such as (..)

2 hrs
Manage Quota and Schedule Priority in OKE

Manage Quota and Schedule Priority in OKE

The workshop will be about deploying Kueue and Kuberay in OKE and using them with jobs with (..)

3 hrs
Deploy WebLogic Application to WebLogic for OKE Stack [WLS for OKE] through WebLogic Kubernetes Toolkit UI

Deploy WebLogic Application to WebLogic for OKE Stack [WLS for OKE] through WebLogic Kubernetes Toolkit UI

This workshop shows an end to end workshop of the migration of an on-premise WebLogic Server Domain (..)

2 hrs 704 Views
Migrate WebLogic to WebLogic on OKE

Migrate WebLogic to WebLogic on OKE

Explore migrating a WebLogic Server domain from on-premises to WebLogic Server on OKE provisioned (..)

2 hrs 8034 Views