Warning, /HeterogeneousCore/SonicTriton/test/README.md is written in an unsupported language. File is not indexed.
0001 # SONIC TritonClient tests
0002
0003 Test modules `TritonImageProducer`, `TritonIdentityProducer`, and `TritonGraphProducer` (`TritonGraphFilter`, `TritonGraphAnalyzer`) are available.
0004 They generate arbitrary inputs for inference (with Inception/DenseNet, a simple identity model that allows ragged batching, or Graph Attention Network, respectively) and print the resulting output.
0005
0006 First, the relevant data for the image classification networks should be downloaded:
0007 ```
0008 ./fetch_model.sh
0009 ```
0010
0011 A local Triton server will be launched automatically when the tests run.
0012 The local server will use Apptainer with CPU by default; if a local Nvidia GPU is available, it will be used instead.
0013 (This behavior can also be controlled manually using the "device" argument to [tritonTest_cfg.py](./tritonTest_cfg.py).)
0014
0015 ## Test commands
0016
0017 Run the image test:
0018 ```
0019 cmsRun tritonTest_cfg.py --maxEvents 1 --modules TritonImageProducer TritonImageProducer --models inception_graphdef densenet_onnx
0020 ```
0021
0022 Run the identity test with ragged batching:
0023 ```
0024 cmsRun tritonTest_cfg.py --maxEvents 1 --modules TritonIdentityProducer --models ragged_io
0025 ```
0026
0027 Run the graph test:
0028 ```
0029 cmsRun tritonTest_cfg.py --maxEvents 1 --modules TritonGraphProducer
0030 ```
0031
0032 ## Caveats
0033
0034 * Local CPU server requires support for AVX instructions.
