Lab 2: Create and train custom models

Introduction

In this session, you will learn how to create projects, train a custom model and analyze text through endpoints.

Estimated Lab Time: 45 minutes

Objectives

In this lab, you will:

  • Learn how to create project.
  • Learn how to train a custom NER and custom Text Classification model.
  • Learn how to create endpoint and analyze text using OCI console and Python SDK

Prerequisites

  • A Free tier or paid tenancy account in OCI (Oracle Cloud Infrastructure)
  • Familiar with OCI object storage to upload data.

Policy Setup

Follow these steps to configure required policies.

  • 1. Navigate to Dynamic Groups

    Log into OCI Cloud Console. Using the Burger Menu on the top left corner, navigate to Identity & Security and click it, and then select Dynamic Groups item under Identity.

    OCI Hamburger menu

  • 2. Create Dynamic Group

    Click Create Dynamic Group

    OCI Create policy

    all {resource.type='ailanguagemodel'}

  • 3. Navigate to Policies

    Log into OCI Cloud Console. Using the Burger Menu on the top left corner, navigate to Identity & Security and click it, and then select Policies item under Identity. OCI Hamburger menu

  • 4. Create Policy

    Click Create Policy OCI Create policy

  • 5. Create a new policy with the following statements:

    To allow dynamic group created above to access object storage in your tenancy, create a new policy with the below statement:

    Allow dynamic-group language-service-dyn-grp-for-custom-models to manage objects in tenancy

    OCI Create policy screen

    For more details on policies required for Language Service, please refer Language Service documentation

Task 1: Create a Project

A Project is a way to organize multiple models in the same workspace. Follow below steps to create a project.

  1. Log into OCI Cloud Console. Using the Burger Menu on the top left corner, navigate to Analytics and AI menu and click it, and then select Language Service item under AI services.

    OCI Language Screen

  2. Select Projects under "Custom models" header on the left hand side of the console.

    Project List

  3. The Create Project button navigates User to a form where they can specify the compartment in which to create a Language Project.

    Create Project Panel

  4. Once the details are entered click the Create Button. Wait until project is in ACTIVE state. If the project is successfully created it will show up in projects pane.
    Active Project

Task 2: Create Custom NER Model

  1. Create and Train Custom NER Model

    1. Upload the training data to Object Storage:

      • Download Custom NER offerletter dataset from this link.

      • Extract the zip file contents into a directory

      • Upload the training dataset files to object storage:

        • Log into OCI Cloud Console. Using the Burger Menu on the top left corner, navigate to Storage and click it, and then select Buckets item under Object Storage and Archive Storage. OCI Hamburger menu
        • Create bucket and upload Custom NER offerletter data extracted above. Upload Objects

        For more details on uploading data to Object Storage, refer Putting Data into Object Storage

    2. Navigate to Project: Navigate to the project you created in Task 1.

    3. Navigate to Models: Under models, click on create and train model.

    4. Choose model type: Choose Named entity recognition for model type.

      Model type

    5. Specify training dataset details: Choose existing labeled dataset and Object Storage under training data section. Choose the bucket created in previous step and choose CNER_offer_trainset_01_labels.jsonl as training data set. Skip Validation data and Test data to default values, so that the system chooses random samples from train set.

      Training data

    6. Specifiy model properties: Specify model name, compartment details and click Next

    7. Create Model: Click "Create and train" and this will kick of the process. Wait until training is complete and Model is in ACTIVE state. Time taken to complete model training is dependent on the data set size. For this dataset, it takes approximately 15 minutes.

      Active model

    8. Check Work request Logs: Optionally, you can check the model training progress. Navigate to Work requests under Resources on left navigation menu and click on Create model Work request and and navigate to Log messages. Work request list Create model work request

    9. Review Model Metrics: Navigate to Metrics under Resources in left navigation menu to review model metrics. Model metrics

  2. Create an Endpoint

    1. Ensure the model status is ACTIVE Active model

    2. Under Model endpoints, click on create model endpoint

    3. Specify the name of the endpoint and click on Create Create model endpoint panel

    4. Wait till the endpoint status changes to ACTIVE

    5. Under Resources, click on Analyze link.

    6. Enter text, and click on Analyze to see the result

      Analyze screen

Task 3: Create Custom Classification Model

  1. Create and Train Custom Classification Model

    1. Upload the training data to Object Storage:

      1. Download and extract the ticket dataset from this link.

      2. Upload training data to object storage:

        • Log into OCI Console. Using the Burger Menu on the top left corner, navigate to Storage and click it, and then select Buckets item under Object Storage and Archive Storage. OCI Hamburger menu
        • Create bucket and upload the extracted data. Upload Objects

        For more details on uploading data to Object Storage, refer Putting Data into Object Storage

    2. Navigate to Project: Navigate to the project you created in Task 1.

    3. Navigate to Models: Under models, click on Create and train model.

    4. Choose model type: Choose Text classification for model type. Select Single label as classification model type

      Model type

    5. Specify training dataset details: Select bucket created in preview step and choose TicketData_train.csv in date file drop down. Skip Validation data and Test data to default values, so that the system chooses random samples from train set. Training data

    6. Specify Model properties: specify the model name and the compartment for creation and click on Next

    7. Create Model: Click Create and train and this will kick of the model training process. Wait until training is complete and Model is in ACTIVE state. Time taken to complete model training is dependent on the data set size. For this dataset, it takes approximately 30 minutes.

      Active model

    8. Check Work request Logs: Optionally, you can check the model training progress. Navigate to Work requests under Resources on left navigation menu and click on Create model Work request and navigate to Log messages. Work request list Create model work request

    9. Review Model Metrics: Navigate to Metrics under Resources in left navigation menu to review model metrics. Model metrics

  2. Create Endpoint and Analyze text

    1. Wait till the model training completes and status is changed to ACTIVE Active model

    2. Under Model endpoints, click on Create model endpoint.

    3. Select the model to associate and click Create model endpoint. Create model endpoint panel

    4. Under Resources, click on Analyze link.

    5. Enter text, and click on Analyze to see the result

      Analyze screen

Task 4: Using Python SDK to Create Custom Models

All the above tasks performed to create custom models and analyzing text can be done through Python code. Pre-requisites:
For using Python SDK, please follow setup steps described in Lab 1: Task 2

Download code file and save it your directory.

Download code file and save it your directory.

To know more about the Python SDK visit Python OCI-Language

Summary

Congratulations!
In this lab you have learnt how to create custom NER and Classification models and analyze text using OCI Console and Python SDK.

You may now proceed to the next lab.

Acknowledgements

Authors

  • Raja Pratap Kondamari - Product Manager, OCI Language Service
  • Sahil Kalra - Oracle AI OCI Language Services

Last Updated By/Date

  • Sahil Kalra - Oracle AI OCI Language Services, March 2023

How to Translate This Page

You must be on the livelabs.oracle.com domain to use translations.
They are not available on apexapps.oracle.com.

For the best translation experience, we recommend Google Chrome.

  1. Right-click anywhere on the page and choose “Translate to
    [Your Language]”    .
  2. If that option doesn’t appear, click the ⋮ three-dot menu in the
    top-right corner of Chrome.
  3. Select “Translate” from the dropdown.
  4. Then, click the translate icon Translate icon in the address bar.
  5. If needed, click the ⋮ three-dot menu within the Google
    Translate popup and choose your preferred language.