Skip to content

(TG/TGG) Choose your pipeline #178

(TG/TGG) Choose your pipeline

(TG/TGG) Choose your pipeline #178

Manually triggered February 25, 2025 23:31
Status Failure
Total duration 1h 34m 10s
Artifacts 4

pipeline-select-galaxy.yaml

on: workflow_dispatch
build-artifact  /  ...  /  check-docker-images
7s
build-artifact / build-docker-image / check-docker-images
build-artifact  /  ...  /  🐳️ Build image
0s
build-artifact / build-docker-image / 🐳️ Build image
build-artifact  /  🛠️ Build Release ubuntu 20.04
6m 27s
build-artifact / 🛠️ Build Release ubuntu 20.04
Matrix: tg-frequent-tests / tg-frequent-tests
Matrix: tg-model-perf-tests / tg-model-perf-tests
Waiting for pending jobs
Matrix: tg-unit-tests / TG-tests
Matrix: tg-unit-tests / TG-UMD-tests
Matrix: tgg-frequent-tests / tgg-frequent-tests
Matrix: tgg-model-perf-tests / tgg-model-perf-tests
Matrix: tgg-unit-tests / TGG-tests
Fit to window
Zoom out
Zoom in

Annotations

5 errors, 22 warnings, and 26 notices
tgg-model-perf-tests / TGG CNN model perf tests
Process completed with exit code 1.
tgg-model-perf-tests / TGG CNN model perf tests
Process completed with exit code 2.
tg-unit-tests / TG Llama3-70b unit tests: models/demos/llama3/tests/test_llama_attention.py#L284
test_llama_attention_inference[wormhole_b0-True-256-1-page_params0-paged_attention-mesh_device0] AssertionError: PCC value is lower than 0.99 for some of the outputs. Check Warnings! assert False
tg-unit-tests / TG Llama3-70b unit tests
Process completed with exit code 1.
tg-unit-tests / TG Llama3-small unit tests
Process completed with exit code 1.
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
tgg-model-perf-tests / TGG CNN model perf tests: python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tgg-model-perf-tests / TGG CNN model perf tests: python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tgg-model-perf-tests / TGG CNN model perf tests: python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tgg-model-perf-tests / TGG CNN model perf tests: python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tgg-model-perf-tests / TGG CNN model perf tests: python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tgg-model-perf-tests / TGG CNN model perf tests: python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tg-frequent-tests / tg-frequent-tests (TG unit/distributed frequent tests, wormhole_b0, unit, 90, XXXXX): python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tg-frequent-tests / tg-frequent-tests (TG unit/distributed frequent tests, wormhole_b0, unit, 90, XXXXX): python_env/lib/python3.8/site-packages/huggingface_hub/file_download.py#L1142
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
tg-unit-tests / TG distributed ops tests: tests/ttnn/distributed/test_distributed_layernorm_TG.py#L45
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG distributed ops tests: tests/ttnn/distributed/test_distributed_layernorm_TG.py#L45
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tg-unit-tests / TG DRAM Prefetcher unit tests: tests/ttnn/unit_tests/operations/test_prefetcher_TG.py#L14
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
tgg-unit-tests / TGG unit tests: tests/ttnn/distributed/test_mesh_device_TGG.py#L10
record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
disk-usage-after-startup
Disk usage is 53 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
weka-mount-hugepages-service-found
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
weka-mount-hugepages-service-found
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.

Artifacts

Produced during runtime
Name Size
TTMetal_build_any
186 MB
eager-dist-ubuntu-20.04-any
333 MB
packages-ubuntu-20.04-amd64-Release-x86_64-linux-clang-17-libcpp
89.4 MB
perf-report-csv-LLM-wormhole_b0-
344 Bytes