Fails with openvino backend

[srinivasbakki]

This example didn't work for openvino on amd64.

Problems:

1. I had to modify the iot edge network - The inferencemodule is on the host network, see deployment-amd64.template.json:

"createOptions": {
"HostConfig":{
"PortBindings": {},
"Binds":["/tmp/.X11-unix:/tmp/.X11-unix","/dev:/dev"],
"NetworkMode":"host",
"IpcMode":"host",
"previleged":true
},
"NetworkingConfig":{
"EndpointsConfig":{
"host":{}
}
},
The capture module couldn't address inference module by name.

2. After removing NetworkConfig i could proceed further but Openvino failed:

2020-06-29 21:01:48.414589245 [W:onnxruntime:, graph.cc:814 Graph] Initializer convolution8_W appears in graph inputs and will not be treated as constant value/weight. This may fail some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2020-06-29 21:01:48.414607717 [W:onnxruntime:, graph.cc:814 Graph] Initializer convolution8_B appears in graph inputs and will not be treated as constant value/weight. This may fail some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
loaded after 1.7521090507507324 s
CLOUD STORAGE STATUS: True
trying to make IOT Hub manager
INITIALIZED AFTER 1.0028696060180664 s

• Serving Flask app "main" (lazy loading)
• Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
• Debug mode: off
• Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
  [WARN] 2020-06-29T21:01:51z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'ai.onnx.preview.training' not recognized by nGraph
  [WARN] 2020-06-29T21:01:51z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'com.microsoft.nchwc' not recognized by nGraph
  [WARN] 2020-06-29T21:01:51z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'com.microsoft' not recognized by nGraph
  [WARN] 2020-06-29T21:01:51z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'ai.onnx.ml' not recognized by nGraph
  [WARN] 2020-06-29T21:01:51z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'com.microsoft.mlfeaturizers' not recognized by nGraph
  [WARN] 2020-06-29T21:01:51z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'ai.onnx.training' not recognized by nGraph
  E: [ncAPI] [ 528983] [python] ncDeviceOpen:1003 Failed to find booted device after boot
  2020-06-29 21:02:08.991818030 [E:onnxruntime:, sequential_executor.cc:281 Execute] Non-zero status code returned while running OpenVINO-EP-subgraph_1 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0' Status Message: /code/onnxruntime/onnxruntime/core/providers/openvino/backends/basic_backend.cc:41 onnxruntime::openvino_ep::BasicBackend::BasicBackend(const onnx::ModelProto&, onnxruntime::openvino_ep::GlobalContext&, const onnxruntime::openvino_ep::SubGraphContext&) [OpenVINO-EP] Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0Can not init Myriad device: NC_ERROR

3. Openvino backend loads the model and makes a reboot but couldn't find the device back again within the container ? I mapped the dev on host and container :
   "Devices": [
   {
   "PathOnHost": "/dev",
   "PathInContainer": "/dev",
   "CgroupPermissions": "rwm"
   }
   ]
   This did remove the error: E: [ncAPI] [ 528983] [python] ncDeviceOpen:1003 Failed to find booted device after boot

but it still cannot init the device:

2020-06-30 18:52:10.059441680 [W:onnxruntime:, graph.cc:814 Graph] Initializer convolution8_W appears in graph inputs and will not be treated as constant value/weight. This may fail some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2020-06-30 18:52:10.059456585 [W:onnxruntime:, graph.cc:814 Graph] Initializer convolution8_B appears in graph inputs and will not be treated as constant value/weight. This may fail some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
loaded after 1.5334908962249756 s
CLOUD STORAGE STATUS: True
trying to make IOT Hub manager
INITIALIZED AFTER 1.0015590190887451 s

• Serving Flask app "main" (lazy loading)
• Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
• Debug mode: off
• Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
  [WARN] 2020-06-30T18:52:13z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'ai.onnx.preview.training' not recognized by nGraph
  [WARN] 2020-06-30T18:52:13z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'com.microsoft.nchwc' not recognized by nGraph
  [WARN] 2020-06-30T18:52:13z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'com.microsoft' not recognized by nGraph
  [WARN] 2020-06-30T18:52:13z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'ai.onnx.ml' not recognized by nGraph
  [WARN] 2020-06-30T18:52:13z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'com.microsoft.mlfeaturizers' not recognized by nGraph
  [WARN] 2020-06-30T18:52:13z src/ngraph/frontend/onnx_import/ops_bridge.cpp 190 Domain 'ai.onnx.training' not recognized by nGraph
  2020-06-30 18:52:18.611212446 [E:onnxruntime:, sequential_executor.cc:281 Execute] Non-zero status code returned while running OpenVINO-EP-subgraph_1 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0' Status Message: /code/onnxruntime/onnxruntime/core/providers/openvino/backends/basic_backend.cc:41 onnxruntime::openvino_ep::BasicBackend::BasicBackend(const onnx::ModelProto&, onnxruntime::openvino_ep::GlobalContext&, const onnxruntime::openvino_ep::SubGraphContext&) [OpenVINO-EP] Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0Can not init Myriad device: NC_ERROR

EXCEPTION: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running OpenVINO-EP-subgraph_1 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0' Status Message: /code/onnxruntime/onnxruntime/core/providers/openvino/backends/basic_backend.cc:41 onnxruntime::openvino_ep::BasicBackend::BasicBackend(const onnx::ModelProto&, onnxruntime::openvino_ep::GlobalContext&, const onnxruntime::openvino_ep::SubGraphContext&) [OpenVINO-EP] Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0Can not init Myriad device: NC_ERROR

Any help please ?

[srinivasbakki]

This problem is with openvino i believe cause the demo inference engine doesn't work with my neural stick

[srinivasbakki]

Ok, i fixed that with standalone openvino installation. Now, am back to same problem

[oliviajain]

Can not init Myriad device: NC_ERROR means that the neural stick isn't being recognized properly. Have you tried running a myriad sample app that comes with the inference engine?