TensorRT Onnx Parser 使用案例分享.pdf

1、NVIDIABest Practices of TensorRT ONNX ParserWANG Meng，2020/12#page#OUTLINE口ONNX Introduction口 TF2ONNX Introduction口 TensorRT ONNX Parser口Optimization口 Refit口Summary#page#ONNX INTRODUCTIONONNX： Open Neural Network ExchangeTraining frameworkDeployment targetOPTcCaffehttps:/ INTRODUCTIONONNX: an open a

2、nd interoperable format for ML modelsTraining frameworkDeployment targetOPyiONNXFocus hardware innovation onpue uado ueFreedom to use toolls） ofNN optimizations for a singleinteroperablechoice compatible with ONNXformat forformat instead of manyMLmodelshttps:/ INTRODUCTIONONNX Overview口https:/ ONNX

3、is an open specification that consists of the following componentsA definition of an extensible computation graph modelDefinitions of standard data types.Definitions of built-in operators.口 Operator sets define the available built-in operators and their version (currently 6-12）口 The newest operator

4、set supports around 160 operatorshttps:/ INTRODUCTIONIntermediate Representation口 Model is top-level ONNX construct and represented in protocol buffers as the typeonnx.ModelProto.Model is consisted of graph and associated metadata.Graph defines the computational logic of a model and contains a list

5、of nodes that form adirected acyclic graph based on their inputs and outputs. The nodes in the graph are sortedtopologically.Edges in the computation graph are established by outputs of one node being referenced byname in the inputs of a subsequent node.Nodes are comprised of a name， the name of an

6、operator that it invokes，a list of named口inputs，a list of named outputs，and a list of attributesAU node output names MUST be unique within a graphhttps:/ INTRODUCTIONStructure of onnx.proto3onnx.proto3 is a general network definition protobuf.message NodeProtofmessage GraphProto frepeated string inp