From 06732ed655e70610fff4e66c80edb04e8b716e4d Mon Sep 17 00:00:00 2001 From: Chi Lo Date: Wed, 2 Oct 2024 17:37:32 +0000 Subject: [PATCH 01/11] add README.md --- quantization/image_classification/trt/resnet50/README.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 quantization/image_classification/trt/resnet50/README.md diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md new file mode 100644 index 000000000..0e114fe4c --- /dev/null +++ b/quantization/image_classification/trt/resnet50/README.md @@ -0,0 +1 @@ +# ONNX PTQ for yolov3 using TensorRT EP From ab81c740b9589429253287f6ba8d674016acdc53 Mon Sep 17 00:00:00 2001 From: Chi Lo Date: Wed, 2 Oct 2024 18:00:08 +0000 Subject: [PATCH 02/11] update README --- quantization/image_classification/trt/resnet50/README.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md index 0e114fe4c..874059639 100644 --- a/quantization/image_classification/trt/resnet50/README.md +++ b/quantization/image_classification/trt/resnet50/README.md @@ -1 +1,6 @@ -# ONNX PTQ for yolov3 using TensorRT EP +# ONNX PTQ for using TensorRT EP +Following is the end-to-end example using ORT quantization tool to quantize ONNX model and run/evaluate the quantized model with TRT EP. + +## Note +We suggest to use ImageNet 2012 classification dataset to do the model calibration and evaluation. In addition to the sample code we provide below, TensorRT model optimizer which leverages torchvision.datasets already provides +the ability to work with ImageNet dataset. From 081679bf507f0fa4cad06a8a73745dff85cba846 Mon Sep 17 00:00:00 2001 From: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Date: Wed, 2 Oct 2024 13:15:56 -0700 Subject: [PATCH 03/11] Update README.md --- .../trt/resnet50/README.md | 48 ++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md index 874059639..12eec1635 100644 --- a/quantization/image_classification/trt/resnet50/README.md +++ b/quantization/image_classification/trt/resnet50/README.md @@ -1,6 +1,52 @@ # ONNX PTQ for using TensorRT EP Following is the end-to-end example using ORT quantization tool to quantize ONNX model and run/evaluate the quantized model with TRT EP. -## Note +## Environment setup +### dataset We suggest to use ImageNet 2012 classification dataset to do the model calibration and evaluation. In addition to the sample code we provide below, TensorRT model optimizer which leverages torchvision.datasets already provides the ability to work with ImageNet dataset. + +#### Prepare ImageNet dataset +You can either download from [Kaggle](https://www.kaggle.com/c/imagenet-object-localization-challenge/data) or origianl image-net website: val [tarball](https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar) and devkit [tarball](https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz) +```shell +mkdir ILSVRC2012 +cd ILSVRC2012 +wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar --no-check-certificate +wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz --no-check-certificate +``` +Untar the tarballs to `val` and `ILSVRC2012_devkit_t12` folder separately. + +The dataset layout should look like this: + +``` +|-- ILSVRC2012_devkit_t12 +| |-- COPYING +| |-- data +| | |-- ILSVRC2012_validation_ground_truth.txt +| | `-- meta.mat +| |-- evaluation +| | |-- VOCreadrecxml.m +| | |-- VOCreadxml.m +| | |-- VOCxml2struct.m +| | |-- compute_overlap.m +| | |-- demo.val.pred.det.txt +| | |-- demo.val.pred.txt +| | |-- demo_eval.m +| | |-- eval_flat.m +| | |-- eval_localization_flat.m +| | |-- get_class2node.m +| | `-- make_hash.m +| `-- readme.txt +|-- meta.bin +`-- val + |-- n01440764 + | |-- ILSVRC2012_val_00000293.JPEG + | |-- ILSVRC2012_val_00002138.JPEG + | |-- ILSVRC2012_val_00003014.JPEG +... +``` +Note: If the data in `val` folder is not grouped by class, please run following command to reconstruct the layout +```shell +cd val/ +wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash +``` From e9ffdefc24747f11bacf9e31c334f2ff0244bdba Mon Sep 17 00:00:00 2001 From: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Date: Fri, 4 Oct 2024 16:22:03 -0700 Subject: [PATCH 04/11] Update README.md --- .../trt/resnet50/README.md | 41 ++++++++++++++++--- 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md index 12eec1635..bb0877b05 100644 --- a/quantization/image_classification/trt/resnet50/README.md +++ b/quantization/image_classification/trt/resnet50/README.md @@ -16,7 +16,7 @@ wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz --no-ch ``` Untar the tarballs to `val` and `ILSVRC2012_devkit_t12` folder separately. -The dataset layout should look like this: +The dataset layout should look like this: Following sample code expects this dataset layout. ``` |-- ILSVRC2012_devkit_t12 @@ -38,15 +38,44 @@ The dataset layout should look like this: | | `-- make_hash.m | `-- readme.txt |-- meta.bin +|-- synset_words.txt `-- val - |-- n01440764 - | |-- ILSVRC2012_val_00000293.JPEG - | |-- ILSVRC2012_val_00002138.JPEG - | |-- ILSVRC2012_val_00003014.JPEG + |-- ILSVRC2012_val_00000001.JPEG + |-- ILSVRC2012_val_00000002.JPEG + |-- ILSVRC2012_val_00000003.JPEG ... ``` -Note: If the data in `val` folder is not grouped by class, please run following command to reconstruct the layout + +However, if you are using ImageNet, then please run following command to reconstruct the layout to be grouped by class. ```shell cd val/ wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash ``` +``` +|-- ILSVRC2012_devkit_t12 +| |-- COPYING +| |-- data +| | |-- ILSVRC2012_validation_ground_truth.txt +| | `-- meta.mat +| |-- evaluation +| | |-- VOCreadrecxml.m +| | |-- VOCreadxml.m +| | |-- VOCxml2struct.m +| | |-- compute_overlap.m +| | |-- demo.val.pred.det.txt +| | |-- demo.val.pred.txt +| | |-- demo_eval.m +| | |-- eval_flat.m +| | |-- eval_localization_flat.m +| | |-- get_class2node.m +| | `-- make_hash.m +| `-- readme.txt +|-- meta.bin +`-- val + |-- n01440764 + | |-- ILSVRC2012_val_00000293.JPEG + | |-- ILSVRC2012_val_00002138.JPEG + | |-- ILSVRC2012_val_00003014.JPEG +... +``` + From 674a5dfc633735a02b73a3d608306e958e5f15f7 Mon Sep 17 00:00:00 2001 From: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Date: Mon, 14 Oct 2024 10:43:01 -0700 Subject: [PATCH 05/11] Update README.md --- quantization/image_classification/trt/resnet50/README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md index bb0877b05..5a6077a63 100644 --- a/quantization/image_classification/trt/resnet50/README.md +++ b/quantization/image_classification/trt/resnet50/README.md @@ -1,8 +1,9 @@ -# ONNX PTQ for using TensorRT EP -Following is the end-to-end example using ORT quantization tool to quantize ONNX model and run/evaluate the quantized model with TRT EP. +# ONNX PTQ overview +Following is the end-to-end example using ORT quantization tool to quantize ONNX model, specifially image classification model, and run/evaluate the quantized model with TRT EP. ## Environment setup ### dataset +First, prepare dataset for calibration as well as evaluation. We suggest to use ImageNet 2012 classification dataset to do the model calibration and evaluation. In addition to the sample code we provide below, TensorRT model optimizer which leverages torchvision.datasets already provides the ability to work with ImageNet dataset. @@ -16,7 +17,7 @@ wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz --no-ch ``` Untar the tarballs to `val` and `ILSVRC2012_devkit_t12` folder separately. -The dataset layout should look like this: Following sample code expects this dataset layout. +The dataset layout should look like below and the sample code expects this dataset layout ``` |-- ILSVRC2012_devkit_t12 @@ -79,3 +80,5 @@ wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/ ... ``` +## Quantize an ONNX model + From c8f7ce56f7bc6009f38eb0037bbce341a144432c Mon Sep 17 00:00:00 2001 From: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Date: Mon, 14 Oct 2024 16:56:35 -0700 Subject: [PATCH 06/11] Update README.md --- quantization/image_classification/trt/resnet50/README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md index 5a6077a63..5be3b2a75 100644 --- a/quantization/image_classification/trt/resnet50/README.md +++ b/quantization/image_classification/trt/resnet50/README.md @@ -3,9 +3,10 @@ Following is the end-to-end example using ORT quantization tool to quantize ONNX ## Environment setup ### dataset -First, prepare dataset for calibration as well as evaluation. -We suggest to use ImageNet 2012 classification dataset to do the model calibration and evaluation. In addition to the sample code we provide below, TensorRT model optimizer which leverages torchvision.datasets already provides -the ability to work with ImageNet dataset. +First, prepare the dataset for calibration. TensorRT recommends calibration data size to be at least 500 for CNN and ViT models. +Generally, the dataset used for calibration should differ from the one used for evaluation. However, to simplify the sample code, we will use the same dataset for both calibration and evaluation. We recommend utilizing the ImageNet 2012 classification dataset for this purpose. + +In addition to the sample code we provide below, TensorRT model optimizer which leverages torchvision.datasets already provides the ability to work with ImageNet dataset. #### Prepare ImageNet dataset You can either download from [Kaggle](https://www.kaggle.com/c/imagenet-object-localization-challenge/data) or origianl image-net website: val [tarball](https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar) and devkit [tarball](https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz) @@ -79,6 +80,7 @@ wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/ | |-- ILSVRC2012_val_00003014.JPEG ... ``` +Lastly, download `synset_words.txt` from https://github.com/HoldenCaulfieldRye/caffe/blob/master/data/ilsvrc12/synset_words.txt into `ILSVRC2012` (top-level folder) ## Quantize an ONNX model From 6fec3fd22c059ea5a001229994ae0baf64b235ab Mon Sep 17 00:00:00 2001 From: Chi Lo Date: Tue, 15 Oct 2024 00:54:27 +0000 Subject: [PATCH 07/11] update script --- .../resnet50/e2e_tensorrt_resnet_example.py | 26 ++++++++++++------- 1 file changed, 16 insertions(+), 10 deletions(-) mode change 100644 => 100755 quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py diff --git a/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py b/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py old mode 100644 new mode 100755 index 9e87a6b9e..2c9cf895f --- a/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py +++ b/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py @@ -9,6 +9,7 @@ import onnxruntime from onnxruntime.quantization import CalibrationDataReader, create_calibrator, write_calibration_table +# onnxruntime.set_default_logger_severity(0) class ImageNetDataReader(CalibrationDataReader): def __init__(self, @@ -126,12 +127,13 @@ def preprocess_imagenet(self, images_folder, height, width, start_index=0, size_ return: list of matrices characterizing multiple images ''' def preprocess_images(input, channels=3, height=224, width=224): - image = input.resize((width, height), Image.ANTIALIAS) + image = input.resize((width, height), Image.Resampling.LANCZOS) # Image.ANTIALIAS was removed in Pillow 10.0.0 input_data = np.asarray(image).astype(np.float32) if len(input_data.shape) != 2: input_data = input_data.transpose([2, 0, 1]) else: input_data = np.stack([input_data] * 3) + # image normalization mean = np.array([0.079, 0.05, 0]) + 0.406 std = np.array([0.005, 0, 0.001]) + 0.224 for channel in range(input_data.shape[0]): @@ -153,7 +155,8 @@ def preprocess_images(input, channels=3, height=224, width=224): for image_name in batch_filenames: image_filepath = images_folder + '/' + image_name - img = Image.open(image_filepath) + # Note: There is one image ILSVRC2012_val_00019877.JPEG which has 4 channels, so here we convert it to RGB with 3 channels for all images + img = Image.open(image_filepath).convert("RGB") image_data = preprocess_images(img) image_data = np.expand_dims(image_data, 0) unconcatenated_batch_data.append(image_data) @@ -163,7 +166,7 @@ def preprocess_images(input, channels=3, height=224, width=224): return batch_data, batch_filenames, image_size_list def get_synset_id(self, image_folder, offset, dataset_size): - ilsvrc2012_meta = scipy.io.loadmat(image_folder + "/devkit/data/meta.mat") + ilsvrc2012_meta = scipy.io.loadmat(image_folder + "/ILSVRC2012_devkit_t12/data/meta.mat") id_to_synset = {} for i in range(1000): id = int(ilsvrc2012_meta["synsets"][i, 0][0][0][0]) @@ -178,7 +181,7 @@ def get_synset_id(self, image_folder, offset, dataset_size): index = index + 1 file.close() - file = open(image_folder + "/devkit/data/ILSVRC2012_validation_ground_truth.txt", "r") + file = open(image_folder + "/ILSVRC2012_devkit_t12/data/ILSVRC2012_validation_ground_truth.txt", "r") id = file.read().strip().split("\n") id = list(map(int, id)) file.close() @@ -318,11 +321,14 @@ def get_dataset_size(dataset_path, calibration_dataset_size): calibration_table_generation_enable = True # Enable/Disable INT8 calibration # TensorRT EP INT8 settings - os.environ["ORT_TENSORRT_FP16_ENABLE"] = "1" # Enable FP16 precision - os.environ["ORT_TENSORRT_INT8_ENABLE"] = "1" # Enable INT8 precision - os.environ["ORT_TENSORRT_INT8_CALIBRATION_TABLE_NAME"] = "calibration.flatbuffers" # Calibration table name - os.environ["ORT_TENSORRT_ENGINE_CACHE_ENABLE"] = "1" # Enable engine caching - execution_provider = ["TensorrtExecutionProvider"] + execution_provider = [ + ('TensorrtExecutionProvider', { + 'trt_int8_enable': True, + 'trt_fp16_enable': True, + 'trt_engine_cache_enable': True, + 'trt_int8_calibration_table_name': 'calibration.flatbuffers', # The implicit quantization is deprecated in TRT 10 + }) + ] # Convert static batch to dynamic batch [new_model_path, input_name] = convert_model_batch_to_dynamic(model_path) @@ -343,7 +349,7 @@ def get_dataset_size(dataset_path, calibration_dataset_size): model_path=augmented_model_path, input_name=input_name) calibrator.collect_data(data_reader) - write_calibration_table(calibrator.compute_range()) + write_calibration_table(calibrator.compute_data()) # Run prediction in Tensorrt EP data_reader = ImageNetDataReader(ilsvrc2012_dataset_path, From f56a16ce8cf0d5a44274fb56e9e971d337608e14 Mon Sep 17 00:00:00 2001 From: Chi Lo Date: Tue, 15 Oct 2024 00:59:41 +0000 Subject: [PATCH 08/11] update script --- .../resnet50/e2e_tensorrt_resnet_example.py | 26 ++++++++++++------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py b/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py index 9e87a6b9e..2c9cf895f 100644 --- a/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py +++ b/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py @@ -9,6 +9,7 @@ import onnxruntime from onnxruntime.quantization import CalibrationDataReader, create_calibrator, write_calibration_table +# onnxruntime.set_default_logger_severity(0) class ImageNetDataReader(CalibrationDataReader): def __init__(self, @@ -126,12 +127,13 @@ def preprocess_imagenet(self, images_folder, height, width, start_index=0, size_ return: list of matrices characterizing multiple images ''' def preprocess_images(input, channels=3, height=224, width=224): - image = input.resize((width, height), Image.ANTIALIAS) + image = input.resize((width, height), Image.Resampling.LANCZOS) # Image.ANTIALIAS was removed in Pillow 10.0.0 input_data = np.asarray(image).astype(np.float32) if len(input_data.shape) != 2: input_data = input_data.transpose([2, 0, 1]) else: input_data = np.stack([input_data] * 3) + # image normalization mean = np.array([0.079, 0.05, 0]) + 0.406 std = np.array([0.005, 0, 0.001]) + 0.224 for channel in range(input_data.shape[0]): @@ -153,7 +155,8 @@ def preprocess_images(input, channels=3, height=224, width=224): for image_name in batch_filenames: image_filepath = images_folder + '/' + image_name - img = Image.open(image_filepath) + # Note: There is one image ILSVRC2012_val_00019877.JPEG which has 4 channels, so here we convert it to RGB with 3 channels for all images + img = Image.open(image_filepath).convert("RGB") image_data = preprocess_images(img) image_data = np.expand_dims(image_data, 0) unconcatenated_batch_data.append(image_data) @@ -163,7 +166,7 @@ def preprocess_images(input, channels=3, height=224, width=224): return batch_data, batch_filenames, image_size_list def get_synset_id(self, image_folder, offset, dataset_size): - ilsvrc2012_meta = scipy.io.loadmat(image_folder + "/devkit/data/meta.mat") + ilsvrc2012_meta = scipy.io.loadmat(image_folder + "/ILSVRC2012_devkit_t12/data/meta.mat") id_to_synset = {} for i in range(1000): id = int(ilsvrc2012_meta["synsets"][i, 0][0][0][0]) @@ -178,7 +181,7 @@ def get_synset_id(self, image_folder, offset, dataset_size): index = index + 1 file.close() - file = open(image_folder + "/devkit/data/ILSVRC2012_validation_ground_truth.txt", "r") + file = open(image_folder + "/ILSVRC2012_devkit_t12/data/ILSVRC2012_validation_ground_truth.txt", "r") id = file.read().strip().split("\n") id = list(map(int, id)) file.close() @@ -318,11 +321,14 @@ def get_dataset_size(dataset_path, calibration_dataset_size): calibration_table_generation_enable = True # Enable/Disable INT8 calibration # TensorRT EP INT8 settings - os.environ["ORT_TENSORRT_FP16_ENABLE"] = "1" # Enable FP16 precision - os.environ["ORT_TENSORRT_INT8_ENABLE"] = "1" # Enable INT8 precision - os.environ["ORT_TENSORRT_INT8_CALIBRATION_TABLE_NAME"] = "calibration.flatbuffers" # Calibration table name - os.environ["ORT_TENSORRT_ENGINE_CACHE_ENABLE"] = "1" # Enable engine caching - execution_provider = ["TensorrtExecutionProvider"] + execution_provider = [ + ('TensorrtExecutionProvider', { + 'trt_int8_enable': True, + 'trt_fp16_enable': True, + 'trt_engine_cache_enable': True, + 'trt_int8_calibration_table_name': 'calibration.flatbuffers', # The implicit quantization is deprecated in TRT 10 + }) + ] # Convert static batch to dynamic batch [new_model_path, input_name] = convert_model_batch_to_dynamic(model_path) @@ -343,7 +349,7 @@ def get_dataset_size(dataset_path, calibration_dataset_size): model_path=augmented_model_path, input_name=input_name) calibrator.collect_data(data_reader) - write_calibration_table(calibrator.compute_range()) + write_calibration_table(calibrator.compute_data()) # Run prediction in Tensorrt EP data_reader = ImageNetDataReader(ilsvrc2012_dataset_path, From 7d2ece020c4ac4ae829d03101d9afeb050dc9851 Mon Sep 17 00:00:00 2001 From: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Date: Wed, 23 Oct 2024 11:23:23 -0700 Subject: [PATCH 09/11] Update README.md --- quantization/image_classification/trt/resnet50/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md index 5be3b2a75..e116dc227 100644 --- a/quantization/image_classification/trt/resnet50/README.md +++ b/quantization/image_classification/trt/resnet50/README.md @@ -1,6 +1,8 @@ # ONNX PTQ overview Following is the end-to-end example using ORT quantization tool to quantize ONNX model, specifially image classification model, and run/evaluate the quantized model with TRT EP. +## Note +Implicit quantization is deprecated in TRT 10.1 (using calibration table in TRT EP to set `setDynamicRange`), we suggest to use explicit quantization aka QDQ format. ## Environment setup ### dataset First, prepare the dataset for calibration. TensorRT recommends calibration data size to be at least 500 for CNN and ViT models. From c628f0c17f277d1e4bd22eeb345105a530468973 Mon Sep 17 00:00:00 2001 From: Chi Lo Date: Wed, 23 Oct 2024 18:28:06 +0000 Subject: [PATCH 10/11] update scritp --- .../trt/resnet50/e2e_tensorrt_resnet_example.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py b/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py index 2c9cf895f..8e2aa8678 100755 --- a/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py +++ b/quantization/image_classification/trt/resnet50/e2e_tensorrt_resnet_example.py @@ -310,6 +310,11 @@ def get_dataset_size(dataset_path, calibration_dataset_size): Untar the model into the workspace ''' + ################################################################################################################### + # Note: Implicit quantization is deprecated in TRT 10.1 (using calibration table in TRT EP to set setDynamicRange), + # we suggest to use explicit quantization aka QDQ format. + ################################################################################################################### + # Dataset settings model_path = "./resnet50-v2-7.onnx" ilsvrc2012_dataset_path = "./ILSVRC2012" From d3eab454c179dec5de23fb474a4684b236efa115 Mon Sep 17 00:00:00 2001 From: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Date: Wed, 23 Oct 2024 11:35:35 -0700 Subject: [PATCH 11/11] Update README.md --- quantization/image_classification/trt/resnet50/README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/quantization/image_classification/trt/resnet50/README.md b/quantization/image_classification/trt/resnet50/README.md index e116dc227..1813cb864 100644 --- a/quantization/image_classification/trt/resnet50/README.md +++ b/quantization/image_classification/trt/resnet50/README.md @@ -85,4 +85,10 @@ wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/ Lastly, download `synset_words.txt` from https://github.com/HoldenCaulfieldRye/caffe/blob/master/data/ilsvrc12/synset_words.txt into `ILSVRC2012` (top-level folder) ## Quantize an ONNX model - +```shell +python e2e_tensorrt_resnet_example.py +```` +The e2e script will do +* Preprocess the image in the dataset +* Run calibration and generate calibration table (deprecated) +* Launch ORT session with TRT EP and run the evaluation (top 1 and top 5)