Skip to content

Generate and update models ops tests and reimplement the models ops test failure updation script #2690

Generate and update models ops tests and reimplement the models ops test failure updation script

Generate and update models ops tests and reimplement the models ops test failure updation script #2690

Triggered via pull request February 24, 2025 14:49
Status Failure
Total duration 2h 7m 5s
Artifacts 9

on-pr.yml

on: pull_request
docker-build  /  build-image
18s
docker-build / build-image
spdx  /  check-spdx-headers
4s
spdx / check-spdx-headers
pre-commit  /  pre-commit
1m 27s
pre-commit / pre-commit
Matrix: build / forge-build
Matrix: test / run-tests
check-all-green
2s
check-all-green
Fit to window
Zoom out
Zoom in

Annotations

206 errors and 1 warning
pre-commit / pre-commit
Process completed with exit code 1.
test_add.test_module[Add0-[((2, 1, 2048), torch.float32), ((1, 2048), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[1.3599805832, 1.2153040171, 0.3786585331, ..., 0.6706525087, 1.7745194435, 0.3440857530]], [[1.1024224758, 0.8026893139, 0.4665691257, ..., 0.8005325794, 1.5674996376, 0.1167586446]]]), compiled_model=tensor([[[1.3599805832, 1.2153040171, 0.3786585331, ..., 0.6706525087, 1.7745194435, 0.3440857530]], [[0.9646650553, 0.4311635494, 0.8605672121, ..., 1.2258419991, 0.9294204712, 0.9939993024]]])
test_add.test_module[Add0-[((2, 1, 1536), torch.float32), ((1, 1536), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[1.4630522728, 1.6477220058, 0.1509722471, ..., 1.4470398426, 0.5820842981, 1.1419926882]], [[1.3005423546, 1.5406150818, 0.7836635709, ..., 1.5978899002, 0.8315522671, 1.7088755369]]]), compiled_model=tensor([[[1.4630522728, 1.6477220058, 0.1509722471, ..., 1.4470398426, 0.5820842981, 1.1419926882]], [[0.5512740016, 1.4046423435, 1.5053542852, ..., 0.6760584116, 1.2907935381, 1.5603219271]]])
test_add.test_module[Add0-[((2, 1, 1024), torch.float32), ((1, 1024), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[0.7349550724, 1.1238288879, 0.2648654580, ..., 0.8409469128, 0.9050856233, 1.4962310791]], [[0.7036654949, 0.3833577633, 0.3880954981, ..., 1.2408131361, 1.5215494633, 1.0661480427]]]), compiled_model=tensor([[[0.7349550724, 1.1238288879, 0.2648654580, ..., 0.8409469128, 0.9050856233, 1.4962310791]], [[0.5946006775, 0.1729068160, 0.9467984438, ..., 0.5874912739, 0.8263171315, 0.2909160852]]])
test_add.test_module[Add46-[((1, 256), torch.int64)]]: forge/test/models_ops/test_add.py#L18561
TypeError: Dtype mismatch: framework_model.dtype=torch.int64, compiled_model.dtype=torch.int32
test_add.test_module[Add1-[((1, 120, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1924707890]], [[1.4644359350]], [[0.7846915722]], [[0.8282446265]], [[1.0036369562]], [[1.3302928209]], [[1.1863075495]], [[1.5926588774]], [[1.1518421173]], [[1.3285204172]], [[1.0451076031]], [[1.0979313850]], [[0.7185398936]], [[0.8650730848]], [[0.9901025891]], [[1.2147359848]], [[1.3938817978]], [[1.4962255955]], [[0.8572435975]], [[0.9784827232]], [[1.3778226376]], [[1.6114081144]], [[1.0933140516]], [[1.5703699589]], [[1.1156225204]], [[1.2491211891]], [[1.6489522457]], [[0.7323789597]], [[0.8814451694]], [[1.0696315765]], [[1.0013141632]], [[1.6282145977]], [[0.8721243143]], [[0.9660477042]], [[0.8468939066]], [[0.7279336452]], [[0.9043439031]], [[1.6260131598]], [[1.4193233252]], [[1.4385504723]], [[1.2225098610]], [[0.9398723841]], [[1.2808065414]], [[0.7293667793]], [[0.8349310160]], [[0.9384491444]], [[1.5116831064]], [[1.4893747568]], [[0.9744666219]], [[1.1781729460]], [[1.5159945488]], [[1.6932806969]], [[1.3946552277]], [[1.2637605667]], [[1.5314573050]], [[0.9018129706]], [[1.2893861532]], [[0.8085613847]], [[0.8496710658]], [[0.9379223585]], [[1.4224506617]], [[1.3972942829]], [[0.9000378847]], [[1.3472676277]], [[1.4707001448]], [[1.1331055164]], [[1.2153048515]], [[1.3120665550]], [[1.5064024925]], [[1.6763112545]], [[0.8109023571]], [[1.0129792690]], [[1.3927190304]], [[1.6104888916]], [[1.6313178539]], [[1.6373925209]], [[1.2957214117]], [[0.7614228129]], [[1.2422103882]], [[0.8834114671]], [[0.7302370667]], [[1.6404603720]], [[1.5763940811]], [[0.6974501610]], [[1.2898001671]], [[1.1119841337]], [[1.1139335632]], [[0.9673357010]], [[1.3884922266]], [[0.9000623822]], [[1.3795098066]], [[1.4490681887]], [[1.5541498661]], [[1.3831696510]], [[0.7013465166]], [[0.8718656898]], [[1.4458715916]], [[1.3008648157]], [[0.8061721325]], [[0.9083043933]], [[1.6665887833]], [[1.5331230164]], [[0.9782015681]], [[1.0703717470]], [[0.7199150920]], [[1.1872270107]], [[0.8196846843]], [[0.8105357885]], [[1.1686643362]], [[1.2712867260]], [[0.9914489985]], [[1.4929029942]], [[0.8919445872]], [[1.6498992443]], [[1.5388641357]], [[0.7745726705]], [[1.0717719793]], [[1.2187755108]], [[1.2691646814]], [[1.3148012161]]]]), compiled_model=tensor([[[[1.1924707890]], [[0.8148299456]], [[0.6633052826]], [[0.2286634445]], [[1.3055853844]], [[0.9863031507]], [[1.1027418375]], [[1.2536740303]], [[0.7628709674]], [[1.1382660866]], [[1.3192000389]], [[0.9223065376]], [[0.1159237027]], [[0.7854200006]], [[0.8128086329]], [[0.6726787686]], [[0.8068621159]], [[1.3291288614]], [[0.8704766035]], [[0.8339579105]], [[0.6927861571]], [[1.1125488281]], [[1.0079274178]], [[1.1160495281]], [[0.6277523637]], [[1.3501378298]], [[1.2956428528]], [[0.9237640500]], [[0.8132946491]], [[0.8395851851]], [[0.5404931903]], [[1.1268116236]], [[0.6457512975]], [[1.0180232525]], [[0.4824424386]], [[0.0413870811]], [[0.7002156377]], [[1.5226066113]], [[1.6835284233]], [[0.8462309241]], [[1.1808503866]], [[1.1559228897]], [[0.9755937457]], [[0.5071379542]], [[0.9486122727]], [[0.4561477900]], [[1.2769508362]], [[0.8887930512]], [[1.7509087324]], [[1.6343026161]], [[2.6938037872]], [[1.9780235291]], [[2.1134800911]], [[1.4381518364]], [[2.2766494751]], [[1.3393878937]], [[1.5243244171]], [[0.4717222452]], [[1.1534569263]], [[0.5437101722]], [[2.3805334568]], [[2.1952209473]], [[1.8717925549]], [[1.7291785479]], [[2.4024157524]], [[1.7025163174]], [[1.2690907717]], [[1.2935867310]], [[1.7511062622]], [[1.7696478367]], [[0.8046296239]], [[1.3133471012]], [[1.4113487005]], [[1.5270676613]], [[2.2407677174]], [[2.2585611343]], [[1.4315385818]], [[1.0232164860]], [[0.8506837487]], [[0.5651270151]], [[1.6746479273]], [[2.3368244171]], [[2.5706095695]], [[1.0940093994]], [[1.7166328430]], [[1.2639145851]], [[1.2058053017]], [[1.0001254082]], [[2.1825125217]], [[1.5895904303]], [[1.9450144768]], [[2.1591041088]], [[2.6421155930]], [[1.7767993212]], [[1.0637261868]], [[1.9813156128]], [[2.1842279434]], [[1.6359007359]], [[0.6158173680]], [[1.3820121288]], [[1.9801402092]], [[1.4038034678]], [[0.7981007099]], [[1.5333373547]], [[0.7468454838]], [[1.2454074621]], [[1.1195642948]], [[1.0777006149]], [[2.2947158813]], [[1.4905998707]], [[0.5606157184]], [[1.1611907482]], [[1.3861601353]], [[1.9556381702]], [[1.8524155617]], [[1.3283585310]], [[0.8848351836]], [[1.4883816242]], [[1.7438490391]], [[1.1415363550]]]])
test_add.test_module[Add1-[((1, 16, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1939241886]], [[1.4658894539]], [[0.7861450315]], [[0.8296980858]], [[1.0050904751]], [[1.3317463398]], [[1.1877610683]], [[1.5941123962]], [[1.1532955170]], [[1.3299739361]], [[1.0465610027]], [[1.0993849039]], [[0.7199933529]], [[0.8665265441]], [[0.9915560484]], [[1.2161893845]]]]), compiled_model=tensor([[[[1.1939241886]], [[1.3211288452]], [[1.0412155390]], [[0.1681953073]], [[0.4926538467]], [[1.0074961185]], [[0.7951934338]], [[1.8284451962]], [[0.6315381527]], [[0.9021398425]], [[0.4995732307]], [[0.4334368110]], [[0.2304555178]], [[1.0986579657]], [[1.0169975758]], [[1.2608580589]]]])
test_add.test_module[Add1-[((1, 240, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1613526344]], [[1.4333178997]], [[0.7535734773]], [[0.7971265316]], [[0.9725188613]], [[1.2991747856]], [[1.1551895142]], [[1.5615408421]], [[1.1207239628]], [[1.2974023819]], [[1.0139894485]], [[1.0668133497]], [[0.6874217987]], [[0.8339549899]], [[0.9589844942]], [[1.1836178303]], [[1.3627636433]], [[1.4651074409]], [[0.8261255026]], [[0.9473646283]], [[1.3467046022]], [[1.5802900791]], [[1.0621960163]], [[1.5392519236]], [[1.0845043659]], [[1.2180030346]], [[1.6178340912]], [[0.7012608647]], [[0.8503270745]], [[1.0385134220]], [[0.9701960683]], [[1.5970964432]], [[0.8410062194]], [[0.9349296093]], [[0.8157758117]], [[0.6968155503]], [[0.8732258081]], [[1.5948951244]], [[1.3882052898]], [[1.4074323177]], [[1.1913918257]], [[0.9087542892]], [[1.2496883869]], [[0.6982486844]], [[0.8038129210]], [[0.9073310494]], [[1.4805650711]], [[1.4582567215]], [[0.9433485270]], [[1.1470549107]], [[1.4848763943]], [[1.6621625423]], [[1.3635370731]], [[1.2326424122]], [[1.5003392696]], [[0.8706948757]], [[1.2582681179]], [[0.7774432898]], [[0.8185529709]], [[0.9068042636]], [[1.3913326263]], [[1.3661762476]], [[0.8689197898]], [[1.3161495924]], [[1.4395821095]], [[1.1019873619]], [[1.1841868162]], [[1.2809484005]], [[1.4752843380]], [[1.6451931000]], [[0.7797842622]], [[0.9818611741]], [[1.3616009951]], [[1.5793707371]], [[1.6001996994]], [[1.6062743664]], [[1.2646033764]], [[0.7303047180]], [[1.2110922337]], [[0.8522933722]], [[0.6991189718]], [[1.6093423367]], [[1.5452759266]], [[0.6663320661]], [[1.2586820126]], [[1.0808660984]], [[1.0828154087]], [[0.9362176061]], [[1.3573741913]], [[0.8689442873]], [[1.3483917713]], [[1.4179501534]], [[1.5230318308]], [[1.3520516157]], [[0.6702284217]], [[0.8407475948]], [[1.4147535563]], [[1.2697467804]], [[0.7750540376]], [[0.8771862984]], [[1.6354706287]], [[1.5020049810]], [[0.9470834732]], [[1.0392537117]], [[0.6887969971]], [[1.1561089754]], [[0.7885665894]], [[0.7794176936]], [[1.1375463009]], [[1.2401685715]], [[0.9603309035]], [[1.4617848396]], [[0.8608264923]], [[1.6187810898]], [[1.5077459812]], [[0.7434545755]], [[1.0406539440]], [[1.1876573563]], [[1.2380466461]], [[1.2836831808]], [[1.3613102436]], [[1.1950461864]], [[0.9211316705]], [[1.4016904831]], [[0.6854715943]], [[0.8687427044]], [[1.0399310589]], [[0.9215393662]], [[0.9901793599]], [[0.7552852035]], [[1.0587384701]], [[1.2719743252]], [[0.8393631577]], [[1.1394363642]], [[1.5230214596]], [[1.1136958599]], [[1.1789921522]], [[1.1219615936]], [[1.2662867308]], [[1.4830157757]], [[1.6387190819]], [[1.4826240540]], [[1.6398028135]], [[1.1289352179]], [[0.7159352899]], [[0.9280574322]], [[1.5055487156]], [[1.1618547440]], [[0.9165728688]], [[0.7819401622]], [[0.6971700191]], [[0.7430919409]], [[1.0636776686]], [[1.4392991066]], [[1.4354165792]], [[0.6828801036]], [[1.4769871235]], [[0.7738413215]], [[1.0593909025]], [[0.9623597264]], [[1.0687884092]], [[1.0669246912]], [[0.7164210677]], [[0.7333770990]], [[1.0868563652]], [[1.1715621948]], [[0.9379585981]], [[1.3534456491]], [[0.7150668502]], [[1.1313524246]], [[1.6048057079]], [[0.9611500502]], [[1.6165976524]], [[1.3461729288]], [[0.7138655782]], [[1.4814447165]], [[1.1073987484]], [[0.9418926239]], [[1.5649226904]], [[0.7610484362]], [[1.2187485695]], [[1.0604116917]], [[1.5221524239]], [[1.3046683073]], [[1.4053487778]], [[1.3416755199]], [[1.0448586941]], [[1.0599433184]], [[0.7530553341]], [[1.4360181093]], [[1.5620851517]], [[1.5072084665]], [[0.8124068975]], [[1.1873960495]], [[0.8126289845]], [[0.8898539543]], [[0.8737432957]], [[1.3359684944]], [[0.8671394587]], [[1.1541874409]], [[1.1861300468]], [[1.4874075651]], [[0.7871360183]], [[0.8218399286]], [[0.8747652769]], [[1.5150628090]], [[0.9853635430]], [[1.5868403912]], [[1.3458998203]], [[1.2284090519]], [[1.1613740921]], [[1.0662553310]], [[1.2278292179]], [[1.0509237051]], [[1.1615828276]], [[1.2288925648]], [[0.7739934921]], [[0.9030303359]], [[1.5688424110]], [[0.7593227029]], [[1.1291929483]], [[1.6597154140]], [[1.3457145691]], [[1.1792526245]], [[0.7317910790]], [[1.4127850533]], [[0.8089556694]], [[1.0231639147]], [[0.9973378778]], [[1.0910522938]], [[1.1705651283]], [[1.5774998665]], [[1.2275154591]], [[1.6129424572]], [[1.4709522724]], [[0.8489890695]], [[1.3893480301]], [[0.8116480112]], [[0.9531834722]], [[1.3121573925]]]]), compiled_model=tensor([[[[1.1613526344]], [[1.5249770880]], [[1.7245111465]], [[0.9210493565]], [[1.3505897522]], [[1.6944515705]], [[0.8003258109]], [[2.0040967464]], [[1.3811956644]], [[1.5067905188]], [[0.6095312238]], [[0.6455270648]], [[1.2008302212]], [[1.9355669022]], [[1.2989851236]], [[1.6350157261]], [[1.2476643324]], [[1.5346825123]], [[0.8690923452]], [[0.4297428727]], [[1.2927756310]], [[1.7325015068]], [[1.5078036785]], [[2.4640054703]], [[0.8933912516]], [[1.9885509014]], [[2.6151683331]], [[1.1115899086]], [[1.2592298985]], [[1.4635250568]], [[1.7132937908]], [[1.7561863661]], [[1.4652962685]], [[0.9121308923]], [[0.5601723194]], [[1.0100221634]], [[0.9547418356]], [[1.8345258236]], [[1.3017680645]], [[1.6498332024]], [[1.5868426561]], [[1.5327208042]], [[1.3922095299]], [[0.7406721711]], [[0.9189903140]], [[0.9814928174]], [[1.7915432453]], [[1.4640591145]], [[0.9672173262]], [[1.7065682411]], [[2.0951709747]], [[1.9135704041]], [[1.5807652473]], [[1.3043627739]], [[2.0041885376]], [[0.9785480499]], [[1.1542072296]], [[0.9824644327]], [[0.8565819263]], [[0.8017668128]], [[1.3843195438]], [[1.1907658577]], [[0.5913726687]], [[0.8428386450]], [[1.6202583313]], [[0.5646792650]], [[1.2239234447]], [[0.9477255940]], [[1.0689542294]], [[1.5699195862]], [[0.3549597263]], [[0.9319675565]], [[1.2946953773]], [[1.0430300236]], [[1.5183529854]], [[1.6541429758]], [[1.2974227667]], [[0.5022693276]], [[0.6361024380]], [[0.6101202965]], [[0.7076767683]], [[1.2618076801]], [[1.5699703693]], [[0.8342173100]], [[0.8325102329]], [[0.9207009077]], [[1.1244689226]], [[0.8103104830]], [[1.2340438366]], [[0.7662809491]], [[0.7902218103]], [[1.2921481133]], [[1.7041707039]], [[1.6375124454]], [[0.7990078330]], [[0.7426530719]], [[1.4831645489]], [[0.8614108562]], [[0.1956138015]], [[0.2821297050]], [[1.9691795111]], [[1.6543035507]], [[0.4363705516]], [[1.0697789192]], [[0.9012847543]], [[1.4908204079]], [[1.0606679916]], [[1.0016984940]], [[0.8578205705]], [[0.8996016979]], [[1.2057592869]], [[1.5768785477]], [[0.3948409557]], [[1.9031937122]], [[1.5842292309]], [[0.8509262800]], [[0.5621696711]], [[1.1660112143]], [[0.8976693153]], [[1.5092654228]], [[1.1062438488]], [[1.2246062756]], [[0.8448465466]], [[1.4493286610]], [[0.3504617810]], [[0.9474323392]], [[0.5255995989]], [[0.8693708181]], [[0.4867873788]], [[0.0969203711]], [[0.4921145439]], [[1.5016132593]], [[0.9447849393]], [[1.4434481859]], [[1.7584860325]], [[0.5020771623]], [[0.6726805568]], [[0.8760742545]], [[0.7764745355]], [[1.6651179790]], [[1.0956192017]], [[1.0735771656]], [[0.9916610718]], [[0.6799663901]], [[0.9620779753]], [[1.1723413467]], [[1.6983714104]], [[1.3828223944]], [[1.1960700750]], [[0.4888126254]], [[0.7520750165]], [[1.0234611034]], [[1.0639910698]], [[1.7740411758]], [[1.5296519995]], [[0.8286082149]], [[1.1368949413]], [[0.8486590385]], [[0.9517481327]], [[0.6778550148]], [[0.6218374372]], [[0.6212703586]], [[0.1665846109]], [[0.9039462209]], [[1.2772258520]], [[0.9495597482]], [[0.4835195541]], [[1.5748023987]], [[0.8697152734]], [[1.0034234524]], [[1.2036421299]], [[1.2555779219]], [[1.6559718847]], [[0.8015041947]], [[1.0273109674]], [[1.6960356236]], [[0.7600573301]], [[1.0578738451]], [[1.1157321930]], [[0.5175952315]], [[1.4782030582]], [[0.9159752131]], [[1.0034472942]], [[0.9724557996]], [[1.1045300961]], [[1.0801150799]], [[0.9276129603]], [[1.3572621346]], [[0.6147294641]], [[0.9622020721]], [[1.4226162434]], [[1.5818560123]], [[0.8953309655]], [[0.5653306842]], [[0.5580617189]], [[0.3531856537]], [[0.4953044653]], [[1.3510191441]], [[0.3469783068]], [[1.1749548912]], [[1.4454231262]], [[1.3551056385]], [[0.2887961268]], [[0.4776014090]], [[0.8188512921]], [[0.9688076973]], [[1.0686726570]], [[0.9678095579]], [[0.7001571059]], [[0.5774827003]], [[0.8948463202]], [[1.2373757362]], [[0.5894936919]], [[1.3014211655]], [[0.7964753509]], [[1.2102385759]], [[0.6316989064]], [[0.2870747447]], [[1.8184108734]], [[0.8634483814]], [[1.4610947371]], [[1.7472255230]], [[0.8505840302]], [[1.4314484596]], [[0.5935672522]], [[1.4847972393]], [[0.2429450750]], [[0.7142546177]], [[0.3413030505]], [[0.7312100530]], [[1.1133346558]], [[1.0198228359]], [[1.2218015194]], [[1.7162498236]], [[1.3755109310]], [[0.3493513465]], [[0.8365921974]], [[0.4922936559]], [[1.0075664520]], [[1.6402595043]]]])
test_add.test_module[Add1-[((1, 144, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.5470958352]], [[0.8190610409]], [[0.1393166780]], [[0.1828697324]], [[0.3582620621]], [[0.6849179268]], [[0.5409326553]], [[0.9472839832]], [[0.5064672232]], [[0.6831455231]], [[0.3997327089]], [[0.4525565505]], [[0.0731649995]], [[0.2196981907]], [[0.3447276950]], [[0.5693610311]], [[0.7485068440]], [[0.8508506417]], [[0.2118687034]], [[0.3331078291]], [[0.7324478030]], [[0.9660332203]], [[0.4479391575]], [[0.9249951243]], [[0.4702475667]], [[0.6037462950]], [[1.0035773516]], [[0.0870040655]], [[0.2360702753]], [[0.4242566228]], [[0.3559392691]], [[0.9828396440]], [[0.2267494202]], [[0.3206728101]], [[0.2015190125]], [[0.0825587511]], [[0.2589690089]], [[0.9806382656]], [[0.7739484310]], [[0.7931755185]], [[0.5771350265]], [[0.2944974899]], [[0.6354315877]], [[0.0839918852]], [[0.1895561218]], [[0.2930742502]], [[0.8663082123]], [[0.8439998627]], [[0.3290917277]], [[0.5327980518]], [[0.8706195951]], [[1.0479058027]], [[0.7492803335]], [[0.6183856726]], [[0.8860824108]], [[0.2564380765]], [[0.6440112591]], [[0.1631864905]], [[0.2042961717]], [[0.2925474644]], [[0.7770757675]], [[0.7519194484]], [[0.2546629906]], [[0.7018927932]], [[0.8253252506]], [[0.4877305627]], [[0.5699300170]], [[0.6666916013]], [[0.8610275388]], [[1.0309362411]], [[0.1655274630]], [[0.3676043749]], [[0.7473441958]], [[0.9651139379]], [[0.9859429002]], [[0.9920176268]], [[0.6503465176]], [[0.1160479188]], [[0.5968354940]], [[0.2380365729]], [[0.0848621726]], [[0.9950854778]], [[0.9310191274]], [[0.0520752668]], [[0.6444252729]], [[0.4666092396]], [[0.4685586691]], [[0.3219608068]], [[0.7431173325]], [[0.2546874881]], [[0.7341349125]], [[0.8036932945]], [[0.9087750316]], [[0.7377948165]], [[0.0559716225]], [[0.2264907956]], [[0.8004967570]], [[0.6554899216]], [[0.1607972383]], [[0.2629294991]], [[1.0212138891]], [[0.8877481818]], [[0.3328266740]], [[0.4249968529]], [[0.0745401978]], [[0.5418521762]], [[0.1743097901]], [[0.1651608944]], [[0.5232894421]], [[0.6259117723]], [[0.3460741043]], [[0.8475280404]], [[0.2465696931]], [[1.0045242310]], [[0.8934891820]], [[0.1291977763]], [[0.4263970852]], [[0.5734005570]], [[0.6237897873]], [[0.6694263816]], [[0.7470533848]], [[0.5807893276]], [[0.3068748713]], [[0.7874337435]], [[0.0712147951]], [[0.2544859052]], [[0.4256743193]], [[0.3072825670]], [[0.3759225607]], [[0.1410284042]], [[0.4444816709]], [[0.6577174664]], [[0.2251063585]], [[0.5251795650]], [[0.9087646604]], [[0.4994391203]], [[0.5647353530]], [[0.5077047944]], [[0.6520299315]], [[0.8687589765]], [[1.0244623423]], [[0.8683671951]], [[1.0255460739]], [[0.5146784186]]]]), compiled_model=tensor([[[[0.5470958352]], [[1.0311832428]], [[0.9289300442]], [[0.6287892461]], [[0.5588996410]], [[0.7509227991]], [[0.5221673846]], [[0.9744406343]], [[0.8542096019]], [[1.4065092802]], [[1.1192140579]], [[0.4195013642]], [[0.8342167735]], [[0.2776042223]], [[0.6881833076]], [[0.8157854676]], [[1.1013599634]], [[1.2018400431]], [[0.2123544812]], [[0.3505496383]], [[1.1033687592]], [[1.4216600657]], [[0.6699624658]], [[1.5625054836]], [[0.4693791270]], [[1.0191633701]], [[1.8924478292]], [[0.3322188258]], [[1.1367325783]], [[1.0544942617]], [[0.3538695574]], [[1.7483490705]], [[0.6182129383]], [[0.5466301441]], [[1.0505063534]], [[0.1276718974]], [[0.7617822289]], [[1.3251147270]], [[1.5801655054]], [[1.3819085360]], [[1.2665483952]], [[0.9202377200]], [[0.9643549919]], [[0.4279999137]], [[0.2266761661]], [[1.0131571293]], [[1.7124580145]], [[1.6352729797]], [[0.4255633354]], [[1.0042587519]], [[0.9673132896]], [[1.2218244076]], [[0.9070883393]], [[1.2384189367]], [[1.0372865200]], [[0.6946902275]], [[1.1142060757]], [[0.9346587658]], [[0.2754969001]], [[0.3984521031]], [[0.9359057546]], [[1.5510468483]], [[0.5240912437]], [[1.5727978945]], [[1.4552898407]], [[1.0002043247]], [[1.0153687000]], [[1.0170116425]], [[1.3729214668]], [[1.3659247160]], [[0.6111750007]], [[0.8805616498]], [[0.8054023981]], [[1.1522090435]], [[1.8388500214]], [[1.0354050398]], [[1.0636041164]], [[1.0598280430]], [[1.2266147137]], [[0.7013538480]], [[0.1007179618]], [[1.6919351816]], [[1.0240395069]], [[0.3593038917]], [[0.9258278608]], [[0.8417263031]], [[0.9231885672]], [[1.1835253239]], [[1.2546975613]], [[1.1516946554]], [[1.4891519547]], [[0.9367470741]], [[1.5821878910]], [[0.8335075378]], [[0.2932198048]], [[0.8227128983]], [[1.4147535563]], [[1.3614059687]], [[1.7459917068]], [[1.0011091232]], [[2.0135414600]], [[1.8972818851]], [[0.5922198296]], [[1.4818096161]], [[0.9492686391]], [[1.3654971123]], [[0.3841083050]], [[0.3581314087]], [[1.6509547234]], [[2.3417804241]], [[1.3003315926]], [[1.9131827354]], [[0.7457271814]], [[1.6883561611]], [[1.5507128239]], [[0.2258328199]], [[0.9867249131]], [[1.3398687840]], [[1.6836543083]], [[2.2084364891]], [[1.1701970100]], [[1.9655940533]], [[1.9184658527]], [[1.8120195866]], [[1.0943744183]], [[1.2937543392]], [[1.7830288410]], [[1.0806293488]], [[1.6144695282]], [[0.7324864864]], [[0.8031349778]], [[1.5851809978]], [[0.9208791852]], [[1.3790671825]], [[1.4365842342]], [[1.3560967445]], [[1.5744429827]], [[1.7459280491]], [[1.4088078737]], [[1.5254392624]], [[1.7538964748]], [[1.5567858219]], [[1.9507809877]], [[1.1347376108]]]])
test_add.test_module[Add1-[((1, 288, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.8819184899]], [[1.1538836956]], [[0.4741393328]], [[0.5176923871]], [[0.6930847168]], [[1.0197405815]], [[0.8757553101]], [[1.2821066380]], [[0.8412898779]], [[1.0179681778]], [[0.7345553637]], [[0.7873792052]], [[0.4079876542]], [[0.5545208454]], [[0.6795503497]], [[0.9041836858]], [[1.0833294392]], [[1.1856732368]], [[0.5466913581]], [[0.6679304838]], [[1.0672705173]], [[1.3008558750]], [[0.7827618122]], [[1.2598178387]], [[0.8050702214]], [[0.9385689497]], [[1.3384000063]], [[0.4218267202]], [[0.5708929300]], [[0.7590792775]], [[0.6907619238]], [[1.3176622391]], [[0.5615720749]], [[0.6554954648]], [[0.5363416672]], [[0.4173814058]], [[0.5937916636]], [[1.3154609203]], [[1.1087710857]], [[1.1279981136]], [[0.9119576812]], [[0.6293201447]], [[0.9702542424]], [[0.4188145399]], [[0.5243787766]], [[0.6278969049]], [[1.2011308670]], [[1.1788225174]], [[0.6639143825]], [[0.8676207066]], [[1.2054421902]], [[1.3827284575]], [[1.0841029882]], [[0.9532083273]], [[1.2209050655]], [[0.5912607312]], [[0.9788339138]], [[0.4980091453]], [[0.5391188264]], [[0.6273701191]], [[1.1118984222]], [[1.0867421627]], [[0.5894856453]], [[1.0367155075]], [[1.1601479053]], [[0.8225532174]], [[0.9047526717]], [[1.0015141964]], [[1.1958501339]], [[1.3657588959]], [[0.5003501177]], [[0.7024270296]], [[1.0821669102]], [[1.2999365330]], [[1.3207654953]], [[1.3268402815]], [[0.9851691723]], [[0.4508705735]], [[0.9316581488]], [[0.5728592277]], [[0.4196848273]], [[1.3299081326]], [[1.2658417225]], [[0.3868979216]], [[0.9792479277]], [[0.8014318943]], [[0.8033813238]], [[0.6567834616]], [[1.0779399872]], [[0.5895101428]], [[1.0689575672]], [[1.1385159492]], [[1.2435977459]], [[1.0726175308]], [[0.3907942772]], [[0.5613134503]], [[1.1353194714]], [[0.9903125763]], [[0.4956198931]], [[0.5977521539]], [[1.3560365438]], [[1.2225708961]], [[0.6676493287]], [[0.7598195076]], [[0.4093628526]], [[0.8766748309]], [[0.5091324449]], [[0.4999835491]], [[0.8581120968]], [[0.9607344270]], [[0.6808967590]], [[1.1823506355]], [[0.5813923478]], [[1.3393468857]], [[1.2283117771]], [[0.4640204310]], [[0.7612197399]], [[0.9082232118]], [[0.9586124420]], [[1.0042490959]], [[1.0818760395]], [[0.9156119823]], [[0.6416975260]], [[1.1222563982]], [[0.4060374498]], [[0.5893085599]], [[0.7604969740]], [[0.6421052217]], [[0.7107452154]], [[0.4758510590]], [[0.7793043256]], [[0.9925401211]], [[0.5599290133]], [[0.8600022197]], [[1.2435872555]], [[0.8342617750]], [[0.8995580077]], [[0.8425274491]], [[0.9868525863]], [[1.2035815716]], [[1.3592849970]], [[1.2031898499]], [[1.3603687286]], [[0.8495010734]], [[0.4365011454]], [[0.6486232877]], [[1.2261145115]], [[0.8824206591]], [[0.6371387243]], [[0.5025060177]], [[0.4177358747]], [[0.4636577964]], [[0.7842435241]], [[1.1598649025]], [[1.1559824944]], [[0.4034459591]], [[1.1975529194]], [[0.4944071770]], [[0.7799567580]], [[0.6829255819]], [[0.7893542647]], [[0.7874905467]], [[0.4369869232]], [[0.4539429545]], [[0.8074221611]], [[0.8921279907]], [[0.6585244536]], [[1.0740115643]], [[0.4356327057]], [[0.8519182801]], [[1.3253716230]], [[0.6817159057]], [[1.3371634483]], [[1.0667388439]], [[0.4344314337]], [[1.2020106316]], [[0.8279646635]], [[0.6624584794]], [[1.2854886055]], [[0.4816142917]], [[0.9393143654]], [[0.7809775472]], [[1.2427182198]], [[1.0252342224]], [[1.1259145737]], [[1.0622413158]], [[0.7654245496]], [[0.7805091739]], [[0.4736211896]], [[1.1565840244]], [[1.2826509476]], [[1.2277743816]], [[0.5329727530]], [[0.9079618454]], [[0.5331948400]], [[0.6104198098]], [[0.5943091512]], [[1.0565344095]], [[0.5877053142]], [[0.8747532964]], [[0.9066959620]], [[1.2079734802]], [[0.5077018738]], [[0.5424057841]], [[0.5953311324]], [[1.2356286049]], [[0.7059293985]], [[1.3074061871]], [[1.0664656162]], [[0.9489749074]], [[0.8819398880]], [[0.7868211269]], [[0.9483950734]], [[0.7714895606]], [[0.8821486831]], [[0.9494584203]], [[0.4945593476]], [[0.6235961914]], [[1.2894082069]], [[0.4798885584]], [[0.8497588038]], [[1.3802812099]], [[1.0662803650]], [[0.8998184204]], [[0.4523569345]], [[1.1333508492]], [[0.5295215249]], [[0.7437297702]], [[0.7179037333]], [[0.8116182089]], [[0.8911310434]], [[1.2980656624]], [[0.9480813146]], [[1.3335082531]], [[1.1915180683]], [[0.5695549250]], [[1.1099139452]], [[0.5322138667]], [[0.6737493277]], [[1.0327231884]], [[1.0507578850]], [[1.2607759237]], [[0.7247039676]], [[0.8864623308]], [[1.1430736780]], [[0.4021158218]], [[1.2471522093]], [[0.4722006917]], [[0.8925531507]], [[0.8006535172]], [[0.6223254204]], [[0.9517474174]], [[1.2991212606]], [[0.7395021319]], [[0.5888148546]], [[0.7007424831]], [[0.3900877237]], [[1.1113588810]], [[0.6455300450]], [[0.5519917607]], [[0.5976111889]], [[1.1731399298]], [[1.1504487991]], [[1.2694227695]], [[1.0670235157]], [[0.7186820507]], [[0.7459378242]], [[1.0333769321]], [[1.2966792583]], [[1.0216079950]], [[0.6490892172]], [[0.6506232023]], [[0.4129273891]], [[0.9936859012]], [[0.6050706506]], [[0.4398739934]], [[1.3241001368]], [[0.5609562993]], [[0.8287811279]], [[1.0289087296]], [[0.9015910625]], [[0.5492210388]], [[0.4815010428]], [[1.2842030525]], [[0.9670812488]], [[1.3004741669]], [[0.7180415988]], [[1.0329396725]]]]), compiled_model=tensor([[[[0.8819184899]], [[1.2905809879]], [[0.6117566228]], [[0.2748561502]], [[0.6144023538]], [[0.9872891903]], [[0.7578272820]], [[1.2063671350]], [[1.0140738487]], [[1.6125154495]], [[0.6422309875]], [[0.4687388539]], [[0.6594837308]], [[1.1170353889]], [[0.3021837473]], [[1.4577327967]], [[1.5587748289]], [[1.4526448250]], [[0.9559433460]], [[0.4654363394]], [[0.7262198329]], [[1.8695368767]], [[1.1443428993]], [[1.0461037159]], [[0.5686058402]], [[1.1867637634]], [[1.7802474499]], [[0.9739457369]], [[0.7481628656]], [[1.1660766602]], [[1.0003373623]], [[1.6088039875]], [[0.4255697131]], [[0.4076765776]], [[1.1362328529]], [[0.6355115771]], [[0.2586230636]], [[1.2449125051]], [[1.2727414370]], [[1.3262169361]], [[0.6524124146]], [[1.1804633141]], [[0.6234925985]], [[0.7781375647]], [[0.4177418351]], [[0.3882884979]], [[1.5366132259]], [[1.1733272076]], [[0.5794606209]], [[0.9295134544]], [[1.1411900520]], [[1.9625688791]], [[0.7275142074]], [[0.8961406946]], [[1.2745496035]], [[0.5665950775]], [[0.7864114046]], [[0.6786054373]], [[0.6635442972]], [[1.0593845844]], [[1.3904476166]], [[1.5434199572]], [[0.3506729007]], [[1.5681738853]], [[1.0652906895]], [[0.9305160046]], [[1.2031782866]], [[1.0125243664]], [[1.1775515079]], [[1.7073320150]], [[0.9749337435]], [[0.4915892482]], [[1.4845354557]], [[1.7454097271]], [[1.6429271698]], [[1.6019043922]], [[1.3679342270]], [[1.0388302803]], [[1.3856425285]], [[0.7031388283]], [[0.1926941872]], [[1.4320185184]], [[1.0928537846]], [[0.9421565533]], [[0.9451145530]], [[0.9518136382]], [[1.1292307377]], [[0.9450285435]], [[0.7822364569]], [[0.3810953498]], [[1.2687476873]], [[0.8856007457]], [[1.5525890589]], [[1.3194885254]], [[0.3104778528]], [[0.2801598907]], [[1.3262844086]], [[0.8254917860]], [[1.0898035765]], [[1.0349967480]], [[1.7903680801]], [[1.5708017349]], [[0.9769369960]], [[0.4988374710]], [[0.9997662902]], [[0.8888364434]], [[0.9131596684]], [[1.0734157562]], [[1.0686025620]], [[0.7127540112]], [[0.7715033293]], [[1.0055313110]], [[0.2886002660]], [[1.8109288216]], [[1.1538258791]], [[0.9800461531]], [[0.5848827958]], [[1.2297117710]], [[1.0392913818]], [[1.0629239082]], [[1.4850080013]], [[0.5772201419]], [[1.1739001274]], [[1.1340763569]], [[0.7461866140]], [[0.4069098234]], [[0.6140337586]], [[0.7448495030]], [[0.7054163814]], [[0.3869484663]], [[1.1642048359]], [[1.0522904396]], [[0.4690603018]], [[0.9172146320]], [[1.8058063984]], [[0.5386640429]], [[0.9306843281]], [[0.8786532879]], [[1.3446345329]], [[1.7300952673]], [[1.1596635580]], [[1.2248980999]], [[1.0093631744]], [[0.5940698981]], [[0.7383006215]], [[0.7085132003]], [[1.7106468678]], [[0.9491928220]], [[0.5231848359]], [[0.4519074559]], [[0.5850267410]], [[0.8982954025]], [[0.9395377040]], [[1.3882429600]], [[1.0947352648]], [[0.7077455521]], [[0.9744313955]], [[0.9193599820]], [[0.8727592826]], [[1.2172904015]], [[0.6491720080]], [[1.3919873238]], [[0.8689889908]], [[0.1644607186]], [[0.7733674645]], [[1.4431345463]], [[0.9426790476]], [[1.6042072773]], [[0.4451043010]], [[0.7527263165]], [[1.7713172436]], [[0.6460267901]], [[1.9049454927]], [[1.4730522633]], [[1.0465383530]], [[1.6934378147]], [[0.7938317657]], [[1.1985013485]], [[1.0119162798]], [[1.0852077007]], [[1.0523424149]], [[0.6674425602]], [[1.5030601025]], [[1.4079259634]], [[1.2637164593]], [[1.5319538116]], [[0.8442266583]], [[0.4034960866]], [[0.9104084969]], [[1.2754373550]], [[1.6775839329]], [[1.0951545238]], [[0.9739524126]], [[1.2100857496]], [[0.3570188284]], [[0.2599551082]], [[0.6919922233]], [[1.2640715837]], [[0.4004518986]], [[0.8563141823]], [[0.9132691622]], [[0.8861645460]], [[0.4125419259]], [[0.3938953280]], [[0.7332614660]], [[1.8460478783]], [[1.1237337589]], [[1.6750257015]], [[1.6144652367]], [[1.1031165123]], [[1.0992052555]], [[0.6234764457]], [[1.3412384987]], [[1.1457130909]], [[1.1194753647]], [[0.7569130063]], [[0.7360689640]], [[0.7995284200]], [[1.7021248341]], [[0.5470088720]], [[1.4125430584]], [[1.4078139067]], [[1.4750069380]], [[1.1136422157]], [[0.1784448624]], [[1.4374086857]], [[0.1957352161]], [[0.4323809147]], [[1.2142665386]], [[0.6173107624]], [[0.6784688234]], [[1.2818841934]], [[1.4743580818]], [[1.9413571358]], [[1.1064112186]], [[0.4062357545]], [[0.9223269820]], [[0.4107528925]], [[0.3109846115]], [[0.9190632105]], [[1.2791144848]], [[1.2253818512]], [[0.3572148085]], [[0.9373351336]], [[1.7143471241]], [[0.6262775660]], [[1.5294530392]], [[0.2023904920]], [[0.6212516427]], [[0.5845983624]], [[0.5906627774]], [[1.0082302094]], [[1.1991169453]], [[1.0248451233]], [[1.0223071575]], [[0.6893806458]], [[0.0441462398]], [[1.0224981308]], [[1.1920220852]], [[0.2591621280]], [[0.5325894356]], [[1.5007548332]], [[1.1491987705]], [[1.8549207449]], [[1.0292843580]], [[1.1905257702]], [[1.1300239563]], [[0.6929511428]], [[1.8924626112]], [[0.8223097920]], [[1.0685437918]], [[0.4495807886]], [[0.3425320387]], [[1.3117964268]], [[0.6296908855]], [[0.2178926468]], [[1.7841215134]], [[0.8222935200]], [[0.9522280693]], [[1.5562748909]], [[1.2103344202]], [[1.1194603443]], [[0.5959815979]], [[0.9722241163]], [[1.3725852966]], [[1.0164916515]], [[1.0131268501]], [[1.1399058104]]]])
test_add.test_module[Add1-[((1, 1280, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6227548122]], [[0.8947200179]], [[0.2149756551]], ..., [[0.8623308539]], [[0.9484047294]], [[0.9666029215]]]]), compiled_model=tensor([[[[0.6227548122]], [[1.7608976364]], [[0.6885750890]], ..., [[1.4462819099]], [[1.8873362541]], [[1.5632492304]]]])
test_add.test_module[Add1-[((1, 256, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.5006824136]], [[0.7726476192]], [[0.0929032564]], [[0.1364563107]], [[0.3118486404]], [[0.6385045052]], [[0.4945192337]], [[0.9008705616]], [[0.4600538015]], [[0.6367321014]], [[0.3533192873]], [[0.4061431289]], [[0.0267515779]], [[0.1732847691]], [[0.2983142734]], [[0.5229476094]], [[0.7020934224]], [[0.8044372201]], [[0.1654552817]], [[0.2866944075]], [[0.6860343814]], [[0.9196197987]], [[0.4015257359]], [[0.8785817027]], [[0.4238341451]], [[0.5573328733]], [[0.9571639299]], [[0.0405906439]], [[0.1896568537]], [[0.3778432012]], [[0.3095258474]], [[0.9364262223]], [[0.1803359985]], [[0.2742593884]], [[0.1551055908]], [[0.0361453295]], [[0.2125555873]], [[0.9342248440]], [[0.7275350094]], [[0.7467620969]], [[0.5307216048]], [[0.2480840683]], [[0.5890181661]], [[0.0375784636]], [[0.1431427002]], [[0.2466608286]], [[0.8198947906]], [[0.7975864410]], [[0.2826783061]], [[0.4863846302]], [[0.8242061734]], [[1.0014923811]], [[0.7028669119]], [[0.5719722509]], [[0.8396689892]], [[0.2100246549]], [[0.5975978374]], [[0.1167730689]], [[0.1578827500]], [[0.2461340427]], [[0.7306623459]], [[0.7055060267]], [[0.2082495689]], [[0.6554793715]], [[0.7789118290]], [[0.4413171411]], [[0.5235165954]], [[0.6202781796]], [[0.8146141171]], [[0.9845228791]], [[0.1191140413]], [[0.3211909533]], [[0.7009307742]], [[0.9187005162]], [[0.9395294785]], [[0.9456042051]], [[0.6039330959]], [[0.0696344972]], [[0.5504220724]], [[0.1916231513]], [[0.0384487510]], [[0.9486720562]], [[0.8846057057]], [[0.0056618452]], [[0.5980118513]], [[0.4201958179]], [[0.4221452475]], [[0.2755473852]], [[0.6967039108]], [[0.2082740664]], [[0.6877214909]], [[0.7572798729]], [[0.8623616099]], [[0.6913813949]], [[0.0095582008]], [[0.1800773740]], [[0.7540833354]], [[0.6090764999]], [[0.1143838167]], [[0.2165160775]], [[0.9748004675]], [[0.8413347602]], [[0.2864132524]], [[0.3785834312]], [[0.0281267762]], [[0.4954387546]], [[0.1278963685]], [[0.1187474728]], [[0.4768760204]], [[0.5794983506]], [[0.2996606827]], [[0.8011146188]], [[0.2001562715]], [[0.9581108689]], [[0.8470757604]], [[0.0827843547]], [[0.3799836636]], [[0.5269871354]], [[0.5773763657]], [[0.6230129600]], [[0.7006399632]], [[0.5343759060]], [[0.2604614496]], [[0.7410203218]], [[0.0248013735]], [[0.2080724835]], [[0.3792608976]], [[0.2608691454]], [[0.3295091391]], [[0.0946149826]], [[0.3980682492]], [[0.6113040447]], [[0.1786929369]], [[0.4787661433]], [[0.8623512387]], [[0.4530256987]], [[0.5183219314]], [[0.4612913728]], [[0.6056165099]], [[0.8223455548]], [[0.9780489206]], [[0.8219537735]], [[0.9791325927]], [[0.4682649970]], [[0.0552650690]], [[0.2673872113]], [[0.8448784351]], [[0.5011845827]], [[0.2559026480]], [[0.1212699413]], [[0.0364997983]], [[0.0824217200]], [[0.4030074477]], [[0.7786288261]], [[0.7747463584]], [[0.0222098827]], [[0.8163168430]], [[0.1131711006]], [[0.3987206817]], [[0.3016895056]], [[0.4081181884]], [[0.4062544703]], [[0.0557508469]], [[0.0727068782]], [[0.4261860847]], [[0.5108919144]], [[0.2772883773]], [[0.6927754283]], [[0.0543966293]], [[0.4706822038]], [[0.9441355467]], [[0.3004798293]], [[0.9559273720]], [[0.6855027080]], [[0.0531953573]], [[0.8207744956]], [[0.4467285872]], [[0.2812224030]], [[0.9042524695]], [[0.1003782153]], [[0.5580782890]], [[0.3997414708]], [[0.8614821434]], [[0.6439980865]], [[0.7446784973]], [[0.6810052991]], [[0.3841884732]], [[0.3992730975]], [[0.0923851132]], [[0.7753478885]], [[0.9014148712]], [[0.8465382457]], [[0.1517366767]], [[0.5267257690]], [[0.1519587636]], [[0.2291837335]], [[0.2130730748]], [[0.6752983332]], [[0.2064692378]], [[0.4935172200]], [[0.5254598856]], [[0.8267373443]], [[0.1264657974]], [[0.1611697078]], [[0.2140950561]], [[0.8543925285]], [[0.3246933222]], [[0.9261701703]], [[0.6852295995]], [[0.5677388310]], [[0.5007038116]], [[0.4055850506]], [[0.5671589971]], [[0.3902534842]], [[0.5009126067]], [[0.5682223439]], [[0.1133232713]], [[0.2423601151]], [[0.9081721902]], [[0.0986524820]], [[0.4685227275]], [[0.9990451932]], [[0.6850443482]], [[0.5185823441]], [[0.0711208582]], [[0.7521147728]], [[0.1482854486]], [[0.3624936938]], [[0.3366676569]], [[0.4303821325]], [[0.5098949671]], [[0.9168295860]], [[0.5668452382]], [[0.9522722363]], [[0.8102820516]], [[0.1883188486]], [[0.7286778688]], [[0.1509777904]], [[0.2925132513]], [[0.6514871716]], [[0.6695218682]], [[0.8795398474]], [[0.3434678912]], [[0.5052262545]], [[0.7618376017]], [[0.0208797455]], [[0.8659161329]], [[0.0909646153]], [[0.5113170743]], [[0.4194174409]], [[0.2410893440]], [[0.5705113411]], [[0.9178851843]], [[0.3582660556]], [[0.2075787783]], [[0.3195064068]]]]), compiled_model=tensor([[[[0.5006824136]], [[1.0833477974]], [[0.1110562086]], [[0.7675865293]], [[0.5814625621]], [[1.3749606609]], [[0.9302139878]], [[1.7932457924]], [[1.3674836159]], [[1.3608345985]], [[0.7057628632]], [[0.7968124151]], [[0.0291234851]], [[0.6526416540]], [[1.0567274094]], [[0.7176864147]], [[1.5162909031]], [[0.8479704857]], [[1.0810496807]], [[0.2880194783]], [[0.8246139288]], [[1.6699842215]], [[0.4614775181]], [[1.3097944260]], [[0.8109716773]], [[1.1752138138]], [[1.9228601456]], [[1.0305089951]], [[0.9995988011]], [[1.1786675453]], [[1.1050875187]], [[1.5775334835]], [[0.4781830907]], [[1.2506749630]], [[1.0430227518]], [[0.3705975413]], [[0.2833987474]], [[1.5364664793]], [[1.4344650507]], [[1.0363306999]], [[0.5581429601]], [[1.1543004513]], [[0.6859285831]], [[0.6291643977]], [[1.1190173626]], [[0.7156679630]], [[0.9704498649]], [[1.6686431170]], [[0.3301787972]], [[1.4490891695]], [[1.2032957077]], [[1.7087013721]], [[1.6378870010]], [[0.8311963677]], [[0.9579418898]], [[1.0897927284]], [[0.7372111678]], [[0.5308936238]], [[0.8487515450]], [[1.1007540226]], [[1.1074311733]], [[1.3518707752]], [[0.6930467486]], [[0.7697823048]], [[1.5999943018]], [[0.6748240590]], [[1.4036210775]], [[0.7495485544]], [[0.8659349084]], [[1.5589847565]], [[0.7656476498]], [[0.5788133144]], [[0.7978674769]], [[1.6396186352]], [[1.9307172298]], [[1.5490343571]], [[0.6193802953]], [[0.2392237782]], [[1.4746016264]], [[0.3025166392]], [[0.5460868478]], [[1.5064201355]], [[1.0947132111]], [[0.2485121489]], [[0.8000474572]], [[0.8700510859]], [[1.1739122868]], [[0.3325448632]], [[1.2653946877]], [[1.1192004681]], [[1.0589790344]], [[1.4133529663]], [[1.4217550755]], [[1.6082479954]], [[0.4277974963]], [[0.7034077644]], [[1.5208597183]], [[1.0721201897]], [[0.1941823959]], [[0.8429183364]], [[1.7675752640]], [[1.3874099255]], [[1.1294555664]], [[0.8937734962]], [[0.3886032104]], [[1.0207626820]], [[0.1447409987]], [[0.1624627113]], [[1.4292457104]], [[1.1861082315]], [[1.1628693342]], [[1.4072325230]], [[1.1781387329]], [[1.2112262249]], [[1.2314789295]], [[0.7690786719]], [[1.1927492619]], [[1.2691438198]], [[1.3597199917]], [[0.6425309777]], [[1.3034188747]], [[0.6378029585]], [[0.9813074470]], [[1.1829757690]], [[0.9426329136]], [[0.3475562334]], [[1.2770906687]], [[0.5963469148]], [[0.7398305535]], [[0.1748156548]], [[0.7787978649]], [[0.7381939292]], [[1.0013251305]], [[0.7252888680]], [[1.1447434425]], [[1.3771451712]], [[1.4588483572]], [[0.5633358955]], [[1.3754683733]], [[1.0138785839]], [[1.3044414520]], [[1.7729154825]], [[1.5133036375]], [[0.5622811317]], [[0.4008420706]], [[1.2544045448]], [[1.1566540003]], [[1.2808033228]], [[0.7197513580]], [[0.3123416305]], [[0.1334270239]], [[0.1668441892]], [[1.3709633350]], [[1.5535278320]], [[1.1276317835]], [[0.6145148873]], [[0.8178412318]], [[0.4642982483]], [[0.8554183245]], [[1.0430237055]], [[0.4856245518]], [[1.2424443960]], [[0.9182355404]], [[0.2088889480]], [[0.6493129730]], [[0.7188361287]], [[0.4653552771]], [[1.0898897648]], [[0.5885711908]], [[1.0847048759]], [[1.4840241671]], [[1.1220130920]], [[1.5738663673]], [[1.4431281090]], [[0.9975739717]], [[1.1009795666]], [[0.6033211946]], [[0.3333778977]], [[0.9069392681]], [[0.2684272528]], [[1.2295963764]], [[0.7039475441]], [[1.5562336445]], [[0.8000905514]], [[1.2128707170]], [[0.6899572611]], [[0.7915813923]], [[0.9015500546]], [[0.9757099748]], [[1.7020306587]], [[1.1031522751]], [[1.4291446209]], [[0.8792157173]], [[0.9529598951]], [[0.6342623234]], [[1.0525233746]], [[0.3331921101]], [[0.7084751725]], [[0.9348928332]], [[1.1062912941]], [[0.6541148424]], [[1.4603705406]], [[1.0427803993]], [[1.0933626890]], [[0.6376793385]], [[1.1437503099]], [[1.1743478775]], [[1.0291879177]], [[1.1689777374]], [[1.3791936636]], [[1.1073300838]], [[0.8540123105]], [[0.9149041176]], [[0.6280267239]], [[1.3783056736]], [[0.5695406795]], [[0.7668276429]], [[0.7210475802]], [[1.0442273617]], [[1.0575194359]], [[1.0337744951]], [[1.4431829453]], [[1.0379821062]], [[1.3695058823]], [[0.5026146173]], [[1.0429258347]], [[0.5018640161]], [[1.0990166664]], [[0.6048610806]], [[0.5165776014]], [[0.9275364280]], [[1.1373569965]], [[1.1138606071]], [[1.7617065907]], [[1.3167059422]], [[0.7532671690]], [[1.0603981018]], [[0.7477260828]], [[0.9575616717]], [[1.1988310814]], [[1.6289732456]], [[1.7912716866]], [[1.3225145340]], [[1.0912194252]], [[1.2269043922]], [[0.5664322376]], [[1.7082040310]], [[0.3873854280]], [[1.3478140831]], [[0.5135604143]], [[0.9858227372]], [[0.9751204848]], [[1.5082988739]], [[0.7858108878]], [[0.2723414302]], [[0.9851081371]]]])
test_add.test_module[Add1-[((1, 512, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.8575292826]], [[1.1294944286]], [[0.4497501254]], [[0.4933031797]], [[0.6686955094]], [[0.9953513741]], [[0.8513661027]], [[1.2577173710]], [[0.8169006705]], [[0.9935789704]], [[0.7101661563]], [[0.7629899979]], [[0.3835984468]], [[0.5301316381]], [[0.6551611423]], [[0.8797944784]], [[1.0589402914]], [[1.1612840891]], [[0.5223021507]], [[0.6435412765]], [[1.0428812504]], [[1.2764666080]], [[0.7583726048]], [[1.2354285717]], [[0.7806810141]], [[0.9141797423]], [[1.3140108585]], [[0.3974375129]], [[0.5465037227]], [[0.7346900702]], [[0.6663727164]], [[1.2932730913]], [[0.5371828675]], [[0.6311062574]], [[0.5119524598]], [[0.3929921985]], [[0.5694024563]], [[1.2910716534]], [[1.0843818188]], [[1.1036089659]], [[0.8875684738]], [[0.6049309373]], [[0.9458650351]], [[0.3944253325]], [[0.4999895692]], [[0.6035076976]], [[1.1767416000]], [[1.1544332504]], [[0.6395251751]], [[0.8432314992]], [[1.1810530424]], [[1.3583393097]], [[1.0597138405]], [[0.9288191199]], [[1.1965157986]], [[0.5668715239]], [[0.9544447064]], [[0.4736199379]], [[0.5147296190]], [[0.6029809117]], [[1.0875091553]], [[1.0623528957]], [[0.5650964379]], [[1.0123262405]], [[1.1357586384]], [[0.7981640100]], [[0.8803634644]], [[0.9771250486]], [[1.1714609861]], [[1.3413697481]], [[0.4759609103]], [[0.6780378222]], [[1.0577776432]], [[1.2755473852]], [[1.2963763475]], [[1.3024511337]], [[0.9607799649]], [[0.4264813662]], [[0.9072689414]], [[0.5484700203]], [[0.3952956200]], [[1.3055188656]], [[1.2414525747]], [[0.3625087142]], [[0.9548587203]], [[0.7770426869]], [[0.7789921165]], [[0.6323942542]], [[1.0535507202]], [[0.5651209354]], [[1.0445683002]], [[1.1141266823]], [[1.2192084789]], [[1.0482282639]], [[0.3664050698]], [[0.5369242430]], [[1.1109302044]], [[0.9659233689]], [[0.4712306857]], [[0.5733629465]], [[1.3316473961]], [[1.1981816292]], [[0.6432601213]], [[0.7354303002]], [[0.3849736452]], [[0.8522856236]], [[0.4847432375]], [[0.4755943418]], [[0.8337228894]], [[0.9363452196]], [[0.6565075517]], [[1.1579614878]], [[0.5570031404]], [[1.3149577379]], [[1.2039226294]], [[0.4396312237]], [[0.7368305326]], [[0.8838340044]], [[0.9342232347]], [[0.9798598289]], [[1.0574867725]], [[0.8912227750]], [[0.6173083186]], [[1.0978672504]], [[0.3816482425]], [[0.5649193525]], [[0.7361077666]], [[0.6177160144]], [[0.6863560081]], [[0.4514618516]], [[0.7549151182]], [[0.9681509137]], [[0.5355398059]], [[0.8356130123]], [[1.2191981077]], [[0.8098725677]], [[0.8751688004]], [[0.8181382418]], [[0.9624633789]], [[1.1791924238]], [[1.3348958492]], [[1.1788005829]], [[1.3359794617]], [[0.8251118660]], [[0.4121119380]], [[0.6242340803]], [[1.2017252445]], [[0.8580314517]], [[0.6127495170]], [[0.4781168103]], [[0.3933466673]], [[0.4392685890]], [[0.7598543167]], [[1.1354756355]], [[1.1315932274]], [[0.3790567517]], [[1.1731636524]], [[0.4700179696]], [[0.7555675507]], [[0.6585363746]], [[0.7649650574]], [[0.7631013393]], [[0.4125977159]], [[0.4295537472]], [[0.7830329537]], [[0.8677387834]], [[0.6341352463]], [[1.0496222973]], [[0.4112434983]], [[0.8275290728]], [[1.3009824753]], [[0.6573266983]], [[1.3127741814]], [[1.0423495770]], [[0.4100422263]], [[1.1776213646]], [[0.8035754561]], [[0.6380692720]], [[1.2610993385]], [[0.4572250843]], [[0.9149251580]], [[0.7565883398]], [[1.2183289528]], [[1.0008449554]], [[1.1015253067]], [[1.0378521681]], [[0.7410353422]], [[0.7561199665]], [[0.4492319822]], [[1.1321947575]], [[1.2582616806]], [[1.2033851147]], [[0.5085835457]], [[0.8835726380]], [[0.5088056326]], [[0.5860306025]], [[0.5699199438]], [[1.0321452618]], [[0.5633161068]], [[0.8503640890]], [[0.8823067546]], [[1.1835842133]], [[0.4833126664]], [[0.5180165768]], [[0.5709419250]], [[1.2112393379]], [[0.6815401912]], [[1.2830170393]], [[1.0420764685]], [[0.9245857000]], [[0.8575506806]], [[0.7624319196]], [[0.9240058661]], [[0.7471003532]], [[0.8577594757]], [[0.9250692129]], [[0.4701701403]], [[0.5992069840]], [[1.2650190592]], [[0.4554993510]], [[0.8253695965]], [[1.3558920622]], [[1.0418912172]], [[0.8754292130]], [[0.4279677272]], [[1.1089615822]], [[0.5051323175]], [[0.7193405628]], [[0.6935145259]], [[0.7872290015]], [[0.8667418361]], [[1.2736763954]], [[0.9236921072]], [[1.3091191053]], [[1.1671289206]], [[0.5451657176]], [[1.0855247974]], [[0.5078246593]], [[0.6493601203]], [[1.0083340406]], [[1.0263687372]], [[1.2363867760]], [[0.7003147602]], [[0.8620731235]], [[1.1186845303]], [[0.3777266145]], [[1.2227630615]], [[0.4478114843]], [[0.8681639433]], [[0.7762643099]], [[0.5979362130]], [[0.9273582101]], [[1.2747321129]], [[0.7151129246]], [[0.5644256473]], [[0.6763532758]], [[0.3656985164]], [[1.0869696140]], [[0.6211408377]], [[0.5276025534]], [[0.5732219815]], [[1.1487507820]], [[1.1260595322]], [[1.2450336218]], [[1.0426342487]], [[0.6942928433]], [[0.7215486169]], [[1.0089876652]], [[1.2722899914]], [[0.9972187877]], [[0.6247000098]], [[0.6262339950]], [[0.3885381818]], [[0.9692966938]], [[0.5806814432]], [[0.4154847860]], [[1.2997109890]], [[0.5365670919]], [[0.8043919206]], [[1.0045194626]], [[0.8772018552]], [[0.5248318315]], [[0.4571118355]], [[1.2598139048]], [[0.9426920414]], [[1.2760848999]], [[0.6936523914]], [[1.0085504055]], [[0.7469345927]], [[0.8390371203]], [[0.5567505956]], [[1.0303730965]], [[1.0193556547]], [[0.8509584069]], [[0.7488216162]], [[0.5530577898]], [[1.2070450783]], [[0.4890606403]], [[1.0661053658]], [[0.6931459308]], [[0.6200386882]], [[0.9510952234]], [[0.6015442014]], [[0.9764751196]], [[0.9594631195]], [[0.4900279641]], [[0.9445219636]], [[1.0742373466]], [[1.0591881275]], [[0.7983333468]], [[0.4513788819]], [[0.7841956615]], [[1.0349265337]], [[0.6788341403]], [[1.0510632992]], [[1.1942539215]], [[0.6001968980]], [[0.8662036061]], [[1.0680222511]], [[0.9004616141]], [[0.9030384421]], [[0.9237053990]], [[0.4681988358]], [[0.9005668163]], [[1.2075076103]], [[1.3118295670]], [[1.1551481485]], [[0.9282742143]], [[1.0947797298]], [[0.6180328727]], [[0.4469285011]], [[0.4313121438]], [[1.3600776196]], [[1.1786673069]], [[0.5156558156]], [[1.0568940639]], [[1.2388565540]], [[1.3610801697]], [[1.2984701395]], [[1.2486495972]], [[0.7466430664]], [[0.6858018637]], [[1.2717971802]], [[1.1414625645]], [[0.5603832006]], [[1.3107812405]], [[1.1028519869]], [[1.1338404417]], [[0.5478845239]], [[1.0047225952]], [[0.6859914660]], [[1.2519509792]], [[0.7713024020]], [[1.0559289455]], [[0.9500836134]], [[1.0740067959]], [[0.6913589239]], [[1.1050584316]], [[0.5120372176]], [[0.9742001891]], [[0.5229767561]], [[0.3680039048]], [[0.4597448111]], [[1.2560076714]], [[1.1317905188]], [[1.3303805590]], [[1.2618333101]], [[0.4147499800]], [[0.5200571418]], [[0.7804813981]], [[0.5365565419]], [[1.2084709406]], [[0.4832687974]], [[0.6173219681]], [[0.3782269955]], [[0.5773999095]], [[1.2725114822]], [[1.2706527710]], [[1.2191915512]], [[1.2473363876]], [[1.3058660030]], [[0.7332412004]], [[1.0812737942]], [[1.3067378998]], [[1.0266821384]], [[1.3611109257]], [[1.1206041574]], [[1.1720968485]], [[0.6862766147]], [[1.1011865139]], [[0.9187259674]], [[0.7418640256]], [[0.5794177651]], [[0.5807144046]], [[0.4765322804]], [[1.1969377995]], [[1.2167382240]], [[0.8043663502]], [[0.5719296932]], [[1.2477254868]], [[1.1810171604]], [[0.8984397054]], [[0.6252050400]], [[1.3207966089]], [[1.0657429695]], [[0.4817000031]], [[1.3398141861]], [[1.2409596443]], [[0.6790272593]], [[1.1423499584]], [[0.5771781802]], [[0.7829155326]], [[1.2858233452]], [[0.8819322586]], [[0.5076636076]], [[0.6941562295]], [[0.7255501151]], [[0.7648082972]], [[0.9091230035]], [[1.3236875534]], [[0.8880428672]], [[0.5525527000]], [[0.8868998289]], [[1.1010162830]], [[1.1092927456]], [[0.4043034315]], [[0.7718014717]], [[0.4897004366]], [[0.6479299068]], [[1.0414193869]], [[0.5062075853]], [[1.0471360683]], [[1.2856616974]], [[0.8940668702]], [[0.5280288458]], [[0.6821302176]], [[0.9704547524]], [[0.4801136851]], [[1.1096777916]], [[0.4073379040]], [[0.3806260228]], [[0.3754423857]], [[0.7598410249]], [[1.1974891424]], [[0.3880332112]], [[1.2768661976]], [[0.6612612605]], [[1.0077147484]], [[0.8840741515]], [[0.4104131460]], [[1.2759371996]], [[1.1304943562]], [[1.3582705259]], [[1.1138787270]], [[0.5312381983]], [[1.2785645723]], [[0.8881449103]], [[1.0983809233]], [[0.4603581429]], [[0.7174594402]], [[0.3703339100]], [[0.6665264368]], [[0.9691382647]], [[0.4686918259]], [[1.0206547976]], [[1.1296761036]], [[0.9309273362]], [[0.5267310143]], [[0.4736128449]], [[0.7070143819]], [[1.0807517767]], [[1.3544709682]], [[1.1487872601]], [[0.8049680591]], [[1.0365809202]], [[0.3707412481]], [[0.4342220426]], [[1.0945765972]], [[0.5780650973]], [[1.1018204689]], [[0.5083066225]], [[0.6136172414]], [[0.4494282007]], [[1.1221930981]], [[0.8103251457]], [[1.2460732460]], [[1.1707087755]], [[1.1379439831]], [[0.8773507476]], [[0.7066836953]], [[0.7525568604]], [[0.9277229905]], [[1.1091275215]], [[0.5109777451]], [[1.2809116840]], [[0.8069076538]], [[0.4422999024]], [[0.5907437801]], [[1.3036816120]], [[1.3185362816]], [[0.3981328607]], [[1.2139217854]], [[1.1118522882]], [[1.1572319269]], [[1.2845373154]], [[0.5917975307]], [[1.0191605091]], [[1.0658893585]], [[0.7135260105]], [[1.0285992622]], [[0.7174159884]], [[1.1704033613]]]]), compiled_model=tensor([[[[0.8575292826]], [[1.2001924515]], [[0.1576659083]], [[0.8020580411]], [[0.4637765884]], [[0.9230792522]], [[0.9659639597]], [[1.1811567545]], [[1.0998113155]], [[1.6225578785]], [[1.0790004730]], [[0.7035893798]], [[0.1202192903]], [[0.4158164263]], [[0.6009347439]], [[1.1998798847]], [[1.4442123175]], [[1.4048823118]], [[0.6396138668]], [[0.7370626926]], [[0.9748380184]], [[1.3475704193]], [[0.6025542617]], [[1.6757814884]], [[0.9200907350]], [[1.6362547874]], [[1.0637943745]], [[0.8037513494]], [[0.7666935921]], [[1.7483780384]], [[1.2353140116]], [[2.7252461910]], [[1.5433938503]], [[1.6306681633]], [[0.8564426303]], [[0.8285319209]], [[0.2372532487]], [[1.5824406147]], [[1.7798366547]], [[1.4600226879]], [[2.0425868034]], [[1.0916287899]], [[1.6656420231]], [[0.3211721182]], [[0.9633308053]], [[1.9122192860]], [[1.2769465446]], [[2.1029551029]], [[1.0892241001]], [[1.6571726799]], [[2.7426404953]], [[2.0275754929]], [[1.6980398893]], [[1.7462139130]], [[1.9403307438]], [[1.7831323147]], [[1.0713551044]], [[1.3630222082]], [[1.1964796782]], [[0.6123057604]], [[1.0096352100]], [[2.2375466824]], [[1.6382887363]], [[1.6873842478]], [[1.3326289654]], [[1.5911917686]], [[1.2050193548]], [[1.2450168133]], [[1.9292056561]], [[1.6957650185]], [[1.0851380825]], [[1.9854083061]], [[1.0266838074]], [[2.3633637428]], [[2.1383993626]], [[2.6498796940]], [[2.2373943329]], [[0.8964050412]], [[1.5039381981]], [[1.2769900560]], [[0.7712340951]], [[1.4751398563]], [[1.7289314270]], [[1.1019899845]], [[1.7010171413]], [[1.7676408291]], [[1.1107661724]], [[1.0409038067]], [[2.2922723293]], [[0.8786723018]], [[2.0869166851]], [[1.5024025440]], [[1.7238707542]], [[2.2459402084]], [[0.7707800269]], [[0.7544648647]], [[1.5475249290]], [[2.2442693710]], [[2.0406751633]], [[1.7611246109]], [[1.5897549391]], [[1.0761327744]], [[1.7565890551]], [[0.6766742468]], [[0.5697878003]], [[1.9974330664]], [[1.2181837559]], [[0.3628337979]], [[1.2724976540]], [[1.4451236725]], [[1.4691472054]], [[1.1292335987]], [[1.4611251354]], [[2.0728855133]], [[1.9016289711]], [[1.4917114973]], [[1.7973129749]], [[2.1308093071]], [[1.0007480383]], [[1.3219949007]], [[2.2170739174]], [[1.6020703316]], [[0.4502180219]], [[1.5795128345]], [[1.7879507542]], [[1.5910565853]], [[1.5042905807]], [[1.1502168179]], [[0.7136865258]], [[1.1109519005]], [[0.5383834243]], [[0.7693409324]], [[1.6035127640]], [[1.6604485512]], [[2.0207948685]], [[1.8558323383]], [[1.6920348406]], [[1.6680917740]], [[1.8326696157]], [[1.5869984627]], [[2.1663722992]], [[2.0866718292]], [[2.3344268799]], [[1.1063702106]], [[1.3542580605]], [[0.9007643461]], [[1.8217600584]], [[1.6797344685]], [[1.1941096783]], [[0.4644003510]], [[1.3091646433]], [[0.6743428111]], [[1.1384122372]], [[0.9490186572]], [[1.5491183996]], [[0.7559779882]], [[1.8132162094]], [[0.8340341449]], [[1.5390383005]], [[1.6744089127]], [[1.8625407219]], [[0.9651645422]], [[1.4267933369]], [[1.0821596384]], [[1.7262017727]], [[2.2793815136]], [[1.7861661911]], [[1.2506307364]], [[0.4508128762]], [[1.7206609249]], [[2.0963637829]], [[1.5768573284]], [[1.6712529659]], [[0.9934185147]], [[0.1821965575]], [[0.9831928611]], [[1.8132660389]], [[1.8303244114]], [[2.0274584293]], [[0.7104672790]], [[1.3714936972]], [[0.8596138954]], [[1.7124745846]], [[1.6825959682]], [[1.2258772850]], [[1.9190238714]], [[1.2979981899]], [[0.6037362218]], [[0.7372722626]], [[1.4897582531]], [[1.3623442650]], [[1.9320021868]], [[0.7358820438]], [[1.6070048809]], [[1.6315571070]], [[1.3467710018]], [[1.7825136185]], [[2.1140005589]], [[1.1996173859]], [[1.5900709629]], [[1.1243553162]], [[1.1556894779]], [[1.0289793015]], [[0.4251711369]], [[1.4392656088]], [[1.5539143085]], [[1.8765010834]], [[1.7218348980]], [[1.8936744928]], [[1.2532702684]], [[1.2878594398]], [[1.3027093410]], [[1.5384430885]], [[2.0878582001]], [[1.5996390581]], [[1.9929411411]], [[0.9881131649]], [[1.1908941269]], [[1.5380086899]], [[1.1467499733]], [[0.7972890139]], [[1.7030944824]], [[1.6155114174]], [[1.6204478741]], [[0.7208098769]], [[2.2080595493]], [[1.1866400242]], [[1.4514305592]], [[0.9699211717]], [[1.5697066784]], [[1.6798169613]], [[1.9415917397]], [[1.7313971519]], [[2.3270401955]], [[1.9131863117]], [[1.0379053354]], [[1.6391561031]], [[0.7745786905]], [[1.6663930416]], [[1.2166020870]], [[1.4319236279]], [[1.5961616039]], [[1.3832694292]], [[1.5583198071]], [[1.7911863327]], [[1.4596369267]], [[1.8994724751]], [[1.4560446739]], [[1.0095058680]], [[1.4579174519]], [[0.7385275364]], [[1.6651022434]], [[1.5183204412]], [[0.8704178333]], [[1.1306893826]], [[1.4524376392]], [[1.1182863712]], [[2.4874036312]], [[1.5765740871]], [[0.9195970297]], [[1.2723474503]], [[1.5352041721]], [[1.7223484516]], [[2.0825920105]], [[2.3103346825]], [[2.1242918968]], [[1.6827905178]], [[1.7389343977]], [[2.1379218102]], [[1.2023782730]], [[1.9716312885]], [[0.6523467302]], [[1.3750796318]], [[1.1215844154]], [[1.2052314281]], [[1.0293326378]], [[2.4467370510]], [[0.9611052871]], [[0.7154606581]], [[1.6283549070]], [[1.2166228294]], [[0.7671896815]], [[0.7799763083]], [[1.7803325653]], [[0.6429398656]], [[1.2454398870]], [[1.1296789646]], [[1.3791359663]], [[0.7692775130]], [[0.6952179670]], [[0.2589014173]], [[1.1585400105]], [[1.3116261959]], [[0.6901603341]], [[1.0277174711]], [[0.2491824627]], [[1.7297222614]], [[1.0723693371]], [[1.2788963318]], [[0.5271777511]], [[1.0689792633]], [[1.3448563814]], [[0.7402169704]], [[1.3474802971]], [[0.6096336246]], [[0.6339100003]], [[1.2281837463]], [[0.7491844893]], [[1.3977982998]], [[0.4481288791]], [[0.1411890984]], [[1.3362350464]], [[1.0955452919]], [[0.4765564203]], [[0.8053911328]], [[1.2084470987]], [[1.1421155930]], [[1.4806761742]], [[1.3951719999]], [[1.5139091015]], [[1.5270276070]], [[1.1961309910]], [[0.8209366798]], [[1.0057734251]], [[1.7108345032]], [[1.1902736425]], [[1.3432796001]], [[0.5701828599]], [[1.5378180742]], [[0.4359030724]], [[0.6303409934]], [[0.6148982048]], [[1.8985724449]], [[1.3364098072]], [[0.7588420510]], [[1.6761090755]], [[1.1060397625]], [[1.1356997490]], [[0.9787989855]], [[1.7734458447]], [[0.8209749460]], [[0.9709702134]], [[1.3724482059]], [[1.2376850843]], [[0.7510981560]], [[1.5959185362]], [[1.3023754358]], [[1.5547666550]], [[0.6445395947]], [[0.8528273702]], [[0.6511801481]], [[1.1111459732]], [[0.7650070190]], [[1.3583387136]], [[1.4053772688]], [[1.6183717251]], [[0.4462670088]], [[1.3500680923]], [[0.5422837138]], [[0.9391174912]], [[0.3913900852]], [[0.2065189481]], [[0.1858202815]], [[1.4509575367]], [[1.1555013657]], [[1.3981623650]], [[1.0965645313]], [[1.0151765347]], [[0.9288395047]], [[0.8125447631]], [[0.7315537333]], [[1.3855805397]], [[0.2485027313]], [[0.6885975003]], [[0.2137898207]], [[1.0915374756]], [[1.2278151512]], [[1.5854835510]], [[1.3820955753]], [[1.5365397930]], [[1.7579220533]], [[0.6860464215]], [[1.4526795149]], [[1.9742578268]], [[0.6925668120]], [[1.6017990112]], [[1.6256244183]], [[1.6716916561]], [[0.6166850328]], [[0.9098251462]], [[0.8156998754]], [[0.7174608111]], [[0.6117798090]], [[0.6547924280]], [[0.5114901662]], [[1.2242264748]], [[1.1405919790]], [[1.1411576271]], [[0.9965093136]], [[1.5876427889]], [[1.6618618965]], [[1.2432291508]], [[0.3718621135]], [[1.0525003672]], [[1.6241483688]], [[0.2873747349]], [[1.3563001156]], [[0.9926975369]], [[1.1459527016]], [[1.1103409529]], [[0.6716848612]], [[0.7467820644]], [[1.8900060654]], [[1.1136982441]], [[1.0430732965]], [[0.7741590142]], [[0.5460400581]], [[1.2806985378]], [[0.8603605628]], [[1.4355990887]], [[0.9726317525]], [[1.1066961288]], [[0.8686225414]], [[0.9767237306]], [[1.5341241360]], [[0.0504578948]], [[0.9922752976]], [[0.4055736065]], [[0.7894771099]], [[1.3152654171]], [[0.6909883618]], [[0.7475339770]], [[1.5039846897]], [[0.5601560473]], [[0.9777237773]], [[0.9258729219]], [[1.2704555988]], [[0.8207780719]], [[1.2260283232]], [[0.3255383372]], [[0.9669243097]], [[0.6988929510]], [[1.0978255272]], [[1.2834125757]], [[0.5981218219]], [[1.2173681259]], [[0.8165432215]], [[0.6857398152]], [[0.8697797060]], [[0.6422622204]], [[1.7603673935]], [[1.6309589148]], [[1.6785702705]], [[1.1189146042]], [[0.7442741990]], [[1.7579883337]], [[0.8312751651]], [[0.9020650387]], [[0.8011379242]], [[0.5256346464]], [[0.7265797257]], [[0.5863713622]], [[0.9959354997]], [[0.8732218742]], [[0.7662373185]], [[1.7563755512]], [[1.1258082390]], [[0.8768648505]], [[0.3532485366]], [[1.1574939489]], [[1.0183255672]], [[1.2149668932]], [[1.4583182335]], [[0.8874040246]], [[1.6400185823]], [[0.8228977919]], [[0.2676283121]], [[1.5155203342]], [[0.3331905603]], [[1.0227701664]], [[0.2706133127]], [[0.7630570531]], [[0.9096294641]], [[1.7061514854]], [[0.5169041157]], [[1.3513360023]], [[1.6300863028]], [[1.3423221111]], [[0.5803250074]], [[1.2377030849]], [[1.0897190571]], [[1.2744610310]], [[1.6742122173]], [[0.4750971794]], [[1.4729528427]], [[1.2368690968]], [[1.0011106730]], [[1.0953425169]], [[1.5687527657]], [[1.0690221786]], [[0.9843560457]], [[1.2124843597]], [[0.7603266835]], [[1.5375943184]], [[1.4566502571]], [[0.9513447285]], [[1.0859515667]], [[1.1685860157]], [[1.2674055099]], [[1.1347281933]], [[1.3529055119]], [[1.2791579962]]]])
test_add.test_module[Add1-[((1, 768, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6126392484]], [[0.8846044540]], [[0.2048600912]], [[0.2484131455]], [[0.4238054752]], [[0.7504613400]], [[0.6064760685]], [[1.0128273964]], [[0.5720106363]], [[0.7486889362]], [[0.4652761221]], [[0.5180999637]], [[0.1387084126]], [[0.2852416039]], [[0.4102711082]], [[0.6349044442]], [[0.8140502572]], [[0.9163940549]], [[0.2774121165]], [[0.3986512423]], [[0.7979912162]], [[1.0315766335]], [[0.5134825706]], [[0.9905385375]], [[0.5357909799]], [[0.6692897081]], [[1.0691207647]], [[0.1525474787]], [[0.3016136885]], [[0.4898000360]], [[0.4214826822]], [[1.0483829975]], [[0.2922928333]], [[0.3862162232]], [[0.2670624256]], [[0.1481021643]], [[0.3245124221]], [[1.0461816788]], [[0.8394918442]], [[0.8587189317]], [[0.6426784396]], [[0.3600409031]], [[0.7009750009]], [[0.1495352983]], [[0.2550995350]], [[0.3586176634]], [[0.9318516254]], [[0.9095432758]], [[0.3946351409]], [[0.5983414650]], [[0.9361630082]], [[1.1134492159]], [[0.8148237467]], [[0.6839290857]], [[0.9516258240]], [[0.3219814897]], [[0.7095546722]], [[0.2287299037]], [[0.2698395848]], [[0.3580908775]], [[0.8426191807]], [[0.8174628615]], [[0.3202064037]], [[0.7674362063]], [[0.8908686638]], [[0.5532739758]], [[0.6354734302]], [[0.7322350144]], [[0.9265709519]], [[1.0964796543]], [[0.2310708761]], [[0.4331477880]], [[0.8128876090]], [[1.0306572914]], [[1.0514862537]], [[1.0575610399]], [[0.7158899307]], [[0.1815913320]], [[0.6623789072]], [[0.3035799861]], [[0.1504055858]], [[1.0606288910]], [[0.9965625405]], [[0.1176186800]], [[0.7099686861]], [[0.5321526527]], [[0.5341020823]], [[0.3875042200]], [[0.8086607456]], [[0.3202309012]], [[0.7996783257]], [[0.8692367077]], [[0.9743184447]], [[0.8033382297]], [[0.1215150356]], [[0.2920342088]], [[0.8660401702]], [[0.7210333347]], [[0.2263406515]], [[0.3284729123]], [[1.0867573023]], [[0.9532915950]], [[0.3983700871]], [[0.4905402660]], [[0.1400836110]], [[0.6073955894]], [[0.2398532033]], [[0.2307043076]], [[0.5888328552]], [[0.6914551854]], [[0.4116175175]], [[0.9130714536]], [[0.3121131063]], [[1.0700676441]], [[0.9590325952]], [[0.1947411895]], [[0.4919404984]], [[0.6389439702]], [[0.6893332005]], [[0.7349697948]], [[0.8125967979]], [[0.6463327408]], [[0.3724182844]], [[0.8529771566]], [[0.1367582083]], [[0.3200293183]], [[0.4912177324]], [[0.3728259802]], [[0.4414659739]], [[0.2065718174]], [[0.5100250840]], [[0.7232608795]], [[0.2906497717]], [[0.5907229781]], [[0.9743080735]], [[0.5649825335]], [[0.6302787662]], [[0.5732482076]], [[0.7175733447]], [[0.9343023896]], [[1.0900057554]], [[0.9339106083]], [[1.0910894871]], [[0.5802218318]], [[0.1672219038]], [[0.3793440461]], [[0.9568352699]], [[0.6131414175]], [[0.3678594828]], [[0.2332267761]], [[0.1484566331]], [[0.1943785548]], [[0.5149642825]], [[0.8905856609]], [[0.8867031932]], [[0.1341667175]], [[0.9282736778]], [[0.2251279354]], [[0.5106775165]], [[0.4136463404]], [[0.5200750232]], [[0.5182113051]], [[0.1677076817]], [[0.1846637130]], [[0.5381429195]], [[0.6228487492]], [[0.3892452121]], [[0.8047322631]], [[0.1663534641]], [[0.5826390386]], [[1.0560923815]], [[0.4124366641]], [[1.0678842068]], [[0.7974595428]], [[0.1651521921]], [[0.9327313304]], [[0.5586854219]], [[0.3931792378]], [[1.0162093639]], [[0.2123350501]], [[0.6700351238]], [[0.5116983056]], [[0.9734389782]], [[0.7559549212]], [[0.8566353321]], [[0.7929621339]], [[0.4961453080]], [[0.5112299323]], [[0.2043419480]], [[0.8873047233]], [[1.0133717060]], [[0.9584950805]], [[0.2636935115]], [[0.6386826038]], [[0.2639155984]], [[0.3411405683]], [[0.3250299096]], [[0.7872551680]], [[0.3184260726]], [[0.6054740548]], [[0.6374167204]], [[0.9386941791]], [[0.2384226322]], [[0.2731265426]], [[0.3260518909]], [[0.9663493633]], [[0.4366501570]], [[1.0381269455]], [[0.7971864343]], [[0.6796956658]], [[0.6126606464]], [[0.5175418854]], [[0.6791158319]], [[0.5022103190]], [[0.6128694415]], [[0.6801791787]], [[0.2252801061]], [[0.3543169498]], [[1.0201289654]], [[0.2106093168]], [[0.5804795623]], [[1.1110019684]], [[0.7970011830]], [[0.6305391788]], [[0.1830776930]], [[0.8640716076]], [[0.2602422833]], [[0.4744505286]], [[0.4486244917]], [[0.5423389673]], [[0.6218518019]], [[1.0287864208]], [[0.6788020730]], [[1.0642290115]], [[0.9222388864]], [[0.3002756834]], [[0.8406347036]], [[0.2629346251]], [[0.4044700861]], [[0.7634440064]], [[0.7814787030]], [[0.9914966822]], [[0.4554247260]], [[0.6171830893]], [[0.8737944365]], [[0.1328365803]], [[0.9778729677]], [[0.2029214501]], [[0.6232739091]], [[0.5313742757]], [[0.3530461788]], [[0.6824681759]], [[1.0298420191]], [[0.4702228904]], [[0.3195356131]], [[0.4314632416]], [[0.1208084822]], [[0.8420796394]], [[0.3762508035]], [[0.2827125192]], [[0.3283319473]], [[0.9038606882]], [[0.8811694980]], [[1.0001435280]], [[0.7977442145]], [[0.4494028091]], [[0.4766585827]], [[0.7640976310]], [[1.0274000168]], [[0.7523287535]], [[0.3798099756]], [[0.3813439608]], [[0.1436481476]], [[0.7244066596]], [[0.3357914090]], [[0.1705947518]], [[1.0548208952]], [[0.2916770577]], [[0.5595018864]], [[0.7596294284]], [[0.6323118210]], [[0.2799417973]], [[0.2122218013]], [[1.0149238110]], [[0.6978020072]], [[1.0311949253]], [[0.4487623572]], [[0.7636603713]], [[0.5020445585]], [[0.5941470861]], [[0.3118605614]], [[0.7854831219]], [[0.7744656205]], [[0.6060683727]], [[0.5039315820]], [[0.3081677556]], [[0.9621550441]], [[0.2441706061]], [[0.8212153912]], [[0.4482558966]], [[0.3751486540]], [[0.7062051892]], [[0.3566541672]], [[0.7315850854]], [[0.7145730853]], [[0.2451379299]], [[0.6996319294]], [[0.8293472528]], [[0.8142981529]], [[0.5534433126]], [[0.2064888477]], [[0.5393056273]], [[0.7900364995]], [[0.4339441061]], [[0.8061732054]], [[0.9493639469]], [[0.3553068638]], [[0.6213135719]], [[0.8231321573]], [[0.6555715799]], [[0.6581484079]], [[0.6788153648]], [[0.2233088017]], [[0.6556767821]], [[0.9626175761]], [[1.0669395924]], [[0.9102581143]], [[0.6833841801]], [[0.8498897552]], [[0.3731428385]], [[0.2020384669]], [[0.1864221096]], [[1.1151875257]], [[0.9337773323]], [[0.2707657814]], [[0.8120039701]], [[0.9939664602]], [[1.1161900759]], [[1.0535800457]], [[1.0037596226]], [[0.5017530322]], [[0.4409118295]], [[1.0269070864]], [[0.8965724707]], [[0.3154931664]], [[1.0658912659]], [[0.8579619527]], [[0.8889504075]], [[0.3029944897]], [[0.7598325610]], [[0.4411014318]], [[1.0070610046]], [[0.5264123678]], [[0.8110389113]], [[0.7051935792]], [[0.8291167617]], [[0.4464688897]], [[0.8601683378]], [[0.2671471834]], [[0.7293101549]], [[0.2780867219]], [[0.1231138706]], [[0.2148547769]], [[1.0111176968]], [[0.8869004846]], [[1.0854904652]], [[1.0169432163]], [[0.1698599458]], [[0.2751671076]], [[0.5355913639]], [[0.2916665077]], [[0.9635809064]], [[0.2383787632]], [[0.3724319339]], [[0.1333369613]], [[0.3325098753]], [[1.0276213884]], [[1.0257626772]], [[0.9743015170]], [[1.0024462938]], [[1.0609759092]], [[0.4883511662]], [[0.8363837004]], [[1.0618479252]], [[0.7817920446]], [[1.1162208319]], [[0.8757141232]], [[0.9272068143]], [[0.4413865805]], [[0.8562964201]], [[0.6738359332]], [[0.4969739914]], [[0.3345277309]], [[0.3358243704]], [[0.2316422462]], [[0.9520478249]], [[0.9718482494]], [[0.5594763160]], [[0.3270396590]], [[1.0028353930]], [[0.9361271262]], [[0.6535496712]], [[0.3803150058]], [[1.0759065151]], [[0.8208529949]], [[0.2368099689]], [[1.0949240923]], [[0.9960695505]], [[0.4341372252]], [[0.8974599242]], [[0.3322881460]], [[0.5380254984]], [[1.0409332514]], [[0.6370422244]], [[0.2627735734]], [[0.4492661953]], [[0.4806600809]], [[0.5199182630]], [[0.6642329693]], [[1.0787974596]], [[0.6431528330]], [[0.3076626658]], [[0.6420097947]], [[0.8561262488]], [[0.8644027710]], [[0.1594133973]], [[0.5269114375]], [[0.2448104024]], [[0.4030398726]], [[0.7965293527]], [[0.2613175511]], [[0.8022460938]], [[1.0407717228]], [[0.6491768360]], [[0.2831388116]], [[0.4372401834]], [[0.7255647182]], [[0.2352236509]], [[0.8647878170]], [[0.1624478698]], [[0.1357359886]], [[0.1305523515]], [[0.5149509907]], [[0.9525991082]], [[0.1431431770]], [[1.0319762230]], [[0.4163712263]], [[0.7628247142]], [[0.6391841173]], [[0.1655231118]], [[1.0310471058]], [[0.8856043816]], [[1.1133804321]], [[0.8689887524]], [[0.2863481641]], [[1.0336744785]], [[0.6432548761]], [[0.8534908891]], [[0.2154681087]], [[0.4725694060]], [[0.1254438758]], [[0.4216364026]], [[0.7242482305]], [[0.2238017917]], [[0.7757647634]], [[0.8847860694]], [[0.6860373020]], [[0.2818409801]], [[0.2287228107]], [[0.4621243477]], [[0.8358617425]], [[1.1095808744]], [[0.9038971663]], [[0.5600780249]], [[0.7916908860]], [[0.1258512139]], [[0.1893320084]], [[0.8496866226]], [[0.3331750631]], [[0.8569304943]], [[0.2634165883]], [[0.3687272072]], [[0.2045381665]], [[0.8773030043]], [[0.5654351115]], [[1.0011832714]], [[0.9258187413]], [[0.8930539489]], [[0.6324607134]], [[0.4617936611]], [[0.5076668262]], [[0.6828329563]], [[0.8642374277]], [[0.2660877109]], [[1.0360215902]], [[0.5620176196]], [[0.1974098682]], [[0.3458537459]], [[1.0587915182]], [[1.0736463070]], [[0.1532428265]], [[0.9690318108]], [[0.8669622540]], [[0.9123418927]], [[1.0396473408]], [[0.3469074965]], [[0.7742705345]], [[0.8209992647]], [[0.4686359763]], [[0.7837092280]], [[0.4725259542]], [[0.9255133271]], [[0.4776553512]], [[0.4299848080]], [[0.7422571778]], [[0.7937294841]], [[0.3720968366]], [[0.6605818272]], [[0.9061505198]], [[0.5666342974]], [[0.7685496211]], [[0.4957891703]], [[0.7916324735]], [[0.2541855574]], [[0.3223682642]], [[0.3625873923]], [[1.0758881569]], [[0.4818406105]], [[0.6150174737]], [[0.3741348982]], [[1.1155276299]], [[1.1047177315]], [[0.2392893434]], [[0.2110477090]], [[0.2373861670]], [[0.6139712930]], [[0.4889307022]], [[0.2891098261]], [[0.4370465875]], [[0.7108480930]], [[0.3551372886]], [[0.7271739244]], [[0.5017247200]], [[0.3741009831]], [[0.6850754023]], [[1.0275118351]], [[0.2783430219]], [[0.6395998597]], [[0.4319987893]], [[1.1070358753]], [[0.1420010328]], [[0.1370520592]], [[1.1090750694]], [[0.3000376821]], [[0.7122443318]], [[0.5732296109]], [[0.5110541582]], [[0.5047087669]], [[0.9341029525]], [[0.6402811408]], [[0.1295753717]], [[0.3212025762]], [[0.4459325075]], [[0.8679908514]], [[0.2928158045]], [[1.0878424644]], [[0.5050179958]], [[0.5266016722]], [[1.0081616640]], [[0.8676865101]], [[1.0404428244]], [[0.9056048393]], [[0.4647057056]], [[0.2846464515]], [[0.5791811347]], [[1.0302014351]], [[0.4485727549]], [[0.1526976824]], [[0.8213468790]], [[1.1031184196]], [[0.4740410447]], [[0.2023708820]], [[0.1628528237]], [[0.7416823506]], [[0.5785228014]], [[0.3638890386]], [[0.7174485922]], [[0.8062620759]], [[1.0140421391]], [[1.0045738220]], [[0.5415408015]], [[0.1754990816]], [[0.1645711064]], [[1.0832273960]], [[0.8374102712]], [[0.8343363404]], [[0.1837721467]], [[1.0793905258]], [[1.0900542736]], [[1.0678128004]], [[0.1945901513]], [[0.4277209640]], [[0.2725007534]], [[1.0898654461]], [[0.4015426636]], [[0.3881053925]], [[0.8783391118]], [[0.3850864768]], [[0.3701215982]], [[0.5726469159]], [[0.5683228970]], [[0.2268934250]], [[1.0332167149]], [[0.3958218098]], [[0.7937400937]], [[1.0512745380]], [[0.8685408831]], [[0.6871725917]], [[1.0418112278]], [[0.6835957766]], [[0.3850543499]], [[1.0893824100]], [[0.7347229123]], [[0.1285422444]], [[0.4740368724]], [[0.2757989168]], [[1.0547591448]], [[0.5338097215]], [[0.1606202126]], [[0.5849509835]], [[0.9303891063]], [[0.7462828755]], [[0.7744848728]], [[0.6628192663]], [[0.8027920127]], [[0.4945566654]], [[0.4174825549]], [[0.1490284204]], [[0.2397142053]], [[0.8330839276]], [[0.3203259706]], [[0.6881158948]], [[0.7759783864]], [[0.6517845988]], [[0.2922077775]], [[1.0945467949]], [[0.3256329298]], [[1.0276329517]], [[0.2186285257]], [[0.4961084127]], [[0.8883670568]], [[0.4120923281]], [[1.0364041328]], [[0.2723044157]], [[0.1964969039]], [[0.3909230828]], [[0.6972078681]], [[1.0767605305]], [[0.3776822090]], [[0.7951967716]], [[0.4910291433]], [[0.5079514980]], [[0.9840343595]], [[0.2288897038]], [[0.6694893241]], [[1.0865871906]], [[0.5476765633]], [[1.0046017170]], [[0.4624042511]], [[1.0188636780]], [[0.1327174902]], [[0.5443208218]], [[0.5285594463]], [[0.7784251571]], [[0.8125463724]], [[1.0002911091]], [[0.5418896675]], [[0.5965551138]], [[0.9587882757]], [[0.4810960889]], [[1.0546994209]], [[0.2834765911]], [[0.5622777343]], [[0.5895271897]], [[0.8394876122]], [[0.9582207203]], [[0.5371307731]], [[0.2021423578]], [[0.8641287684]], [[0.7659485340]], [[0.8172323108]], [[0.3079488277]], [[0.9381804466]], [[1.0899600983]], [[0.6597683430]], [[0.1493584514]], [[0.9673776031]], [[0.2456570268]], [[0.7313187122]], [[0.6890415549]], [[0.3823643923]], [[0.7904804349]], [[0.1691663265]], [[0.7302623987]], [[0.2994071841]], [[0.5623158216]], [[0.6806970835]], [[1.0423548222]], [[0.3778225183]], [[0.9367007017]], [[0.5528657436]], [[0.3789187670]], [[0.1809828281]], [[0.1576234698]], [[1.1046879292]], [[0.4916828871]], [[0.6413570642]], [[0.7519388199]], [[0.9562487602]], [[1.0431196690]], [[1.0218912363]], [[0.2459411025]], [[0.5362742543]], [[0.3205138445]], [[0.3307067752]], [[0.7349861860]], [[1.0857067108]], [[0.2158306837]], [[0.9189848900]], [[0.3571493030]], [[0.5189955235]], [[1.0132681131]], [[0.5032975078]], [[0.6619105339]], [[0.2668663859]], [[1.0420031548]], [[0.5517477393]], [[0.2506874204]], [[0.7627878189]], [[0.2608739734]], [[0.2196289897]], [[0.6468422413]], [[1.0128045082]], [[0.4748906493]], [[0.8517161012]], [[1.0460432768]], [[0.9480131269]], [[0.3541021347]], [[0.5615727305]], [[0.4590402246]], [[0.2143211961]], [[0.6165892482]], [[0.9926002026]], [[1.0376404524]], [[0.6630330086]], [[0.7299441695]], [[0.3999218345]], [[0.9938038588]], [[0.4083284140]], [[0.2690562606]], [[0.6933794022]], [[0.9160753489]], [[0.1655868888]], [[1.0683664083]], [[0.7962481976]], [[0.2660762668]], [[0.5086789131]], [[1.0501725674]]]]), compiled_model=tensor([[[[0.6126392484]], [[1.2847993374]], [[1.0160138607]], [[1.2693874836]], [[1.4212834835]], [[2.3957853317]], [[1.8067994118]], [[1.6497118473]], [[1.5160260201]], [[1.3800323009]], [[1.3064551353]], [[1.6005483866]], [[1.6512989998]], [[1.9601306915]], [[1.6164029837]], [[1.6097412109]], [[1.9245719910]], [[1.3664436340]], [[1.8692334890]], [[0.6696540117]], [[2.0294227600]], [[1.4287543297]], [[1.3829226494]], [[1.8492763042]], [[1.9277071953]], [[1.3387179375]], [[1.2250795364]], [[1.0212728977]], [[0.8859246373]], [[0.9770479202]], [[0.9892371893]], [[1.8137917519]], [[0.2374306917]], [[0.6004611850]], [[0.9479789734]], [[0.7635777593]], [[0.5917453766]], [[1.1472525597]], [[0.7865327001]], [[1.2317757607]], [[1.1798388958]], [[0.4441328645]], [[1.2247608900]], [[0.0905500054]], [[1.0226666927]], [[1.1868164539]], [[1.3895325661]], [[0.9884651303]], [[1.0884656906]], [[1.2369925976]], [[1.3197257519]], [[1.7293443680]], [[0.7098842859]], [[1.0727012157]], [[1.4801776409]], [[0.2418187261]], [[1.2930548191]], [[0.1234154701]], [[0.2045398355]], [[1.1550202370]], [[1.1481279135]], [[0.8600751758]], [[0.3194243312]], [[1.0265192986]], [[1.6776773930]], [[1.4126365185]], [[1.2075133324]], [[1.5905725956]], [[1.7954502106]], [[1.6137953997]], [[0.8286987543]], [[0.7832444310]], [[1.5611045361]], [[1.1539914608]], [[1.4845077991]], [[0.9443597198]], [[1.4038182497]], [[0.2443515658]], [[1.0906814337]], [[0.7320560813]], [[0.9337904453]], [[1.4632613659]], [[1.4846388102]], [[0.9817238450]], [[0.8220419288]], [[0.5516623259]], [[0.4593209624]], [[1.1571905613]], [[1.1278827190]], [[0.8502892852]], [[1.1452194452]], [[1.2103493214]], [[1.4099234343]], [[1.3333654404]], [[0.5659285188]], [[0.9578503966]], [[1.2075853348]], [[0.8140281439]], [[0.4364193678]], [[0.4325580001]], [[1.3253519535]], [[1.5005913973]], [[1.0985537767]], [[1.2797951698]], [[0.1398817301]], [[1.0972954035]], [[0.5149897337]], [[0.4405116439]], [[0.7021362185]], [[0.7748602629]], [[0.3825830221]], [[1.3529112339]], [[0.5807139874]], [[1.3827395439]], [[1.0386538506]], [[1.0400578976]], [[1.1456129551]], [[0.9158973694]], [[1.1292204857]], [[1.1569693089]], [[0.8227207661]], [[0.9624983072]], [[0.4528711438]], [[1.6120047569]], [[0.3369519711]], [[0.8797501922]], [[0.8990117311]], [[0.9069194794]], [[1.1384121180]], [[0.4042670727]], [[1.1263208389]], [[1.6356709003]], [[0.2014245391]], [[1.0763012171]], [[1.7242183685]], [[1.3094674349]], [[0.8055772185]], [[0.6267769337]], [[0.8594372869]], [[1.1547892094]], [[1.3672578335]], [[1.2528786659]], [[1.3709373474]], [[0.8524004221]], [[0.3359656334]], [[0.9610252976]], [[1.6263048649]], [[1.1979488134]], [[1.0935943127]], [[0.8229061961]], [[0.1400037408]], [[0.1709723473]], [[1.3182597160]], [[0.9411504269]], [[1.1480791569]], [[0.1307947040]], [[1.6400890350]], [[0.4380089641]], [[0.8500742316]], [[0.6224029064]], [[1.3691478968]], [[0.9948672652]], [[0.9480074048]], [[0.5095565319]], [[0.6035228968]], [[1.3836289644]], [[0.5853728056]], [[1.1615338326]], [[0.4958323836]], [[1.3816725016]], [[1.2827050686]], [[0.5330341458]], [[1.7376055717]], [[0.6885040402]], [[0.6305160522]], [[1.0934945345]], [[0.9451226592]], [[0.9119152427]], [[1.4458801746]], [[0.1576229334]], [[1.1332480907]], [[0.4226775169]], [[1.6680239439]], [[1.2445876598]], [[1.4015262127]], [[1.3785165548]], [[0.8573858142]], [[0.6743203998]], [[1.0355303288]], [[1.4556453228]], [[1.5962462425]], [[1.2893085480]], [[0.7186721563]], [[0.8240745664]], [[0.6640875936]], [[0.2640556693]], [[0.5556254983]], [[1.2639942169]], [[1.0477461815]], [[1.3508286476]], [[1.2026064396]], [[1.1886200905]], [[0.6963486671]], [[0.9974403381]], [[0.5140721798]], [[1.0149235725]], [[1.0223200321]], [[1.0911922455]], [[1.3983223438]], [[0.8444306254]], [[0.8843479156]], [[1.1669619083]], [[0.6695883870]], [[1.3737998009]], [[1.0526404381]], [[1.2752029896]], [[0.3498058319]], [[1.0496865511]], [[1.2025928497]], [[0.3159953356]], [[1.1349006891]], [[1.4383280277]], [[1.6453288794]], [[1.3275856972]], [[0.2613739967]], [[1.5299053192]], [[0.2602577806]], [[0.6402902603]], [[0.4558212161]], [[0.9366688132]], [[1.3269431591]], [[1.8576347828]], [[0.6302710772]], [[1.4143818617]], [[1.6265064478]], [[0.7495439053]], [[0.7884989977]], [[1.0388441086]], [[0.9865222573]], [[1.3550720215]], [[1.5914535522]], [[1.2005062103]], [[0.8923559785]], [[1.2920346260]], [[1.6774952412]], [[0.8823253512]], [[1.4878342152]], [[0.1982974410]], [[1.4543871880]], [[0.7748268247]], [[0.2464106083]], [[1.3077206612]], [[1.4468449354]], [[1.0746600628]], [[0.6312167048]], [[0.7790499926]], [[0.9195779562]], [[1.1930985451]], [[1.2566304207]], [[0.6363571882]], [[0.8210837245]], [[1.3929480314]], [[1.4141800404]], [[1.6696994305]], [[1.5049270391]], [[1.1231745481]], [[1.0634955168]], [[1.4801609516]], [[1.2590243816]], [[0.7796180844]], [[0.4874807596]], [[0.8763029575]], [[0.4816260338]], [[1.3006384373]], [[0.7318294048]], [[0.8392173648]], [[1.7959675789]], [[1.3754868507]], [[0.6007851362]], [[1.4453048706]], [[0.9797057509]], [[1.0866384506]], [[1.0618031025]], [[2.0796980858]], [[1.6812306643]], [[2.5373702049]], [[1.4113801718]], [[1.3508670330]], [[0.5058811903]], [[0.8935808539]], [[0.7964126468]], [[1.8689804077]], [[2.1022953987]], [[1.8945679665]], [[1.0271627903]], [[0.9288477898]], [[1.8206104040]], [[1.4753583670]], [[1.3073869944]], [[2.0076546669]], [[1.1788567305]], [[2.2260773182]], [[1.3040659428]], [[1.4189537764]], [[1.3648840189]], [[1.8771333694]], [[1.8185632229]], [[3.4382107258]], [[2.2413094044]], [[2.0677287579]], [[0.9465488195]], [[1.2514548302]], [[0.9109070897]], [[1.9000020027]], [[2.4696271420]], [[2.2930040359]], [[2.2815110683]], [[1.5965597630]], [[2.3723914623]], [[0.8603610396]], [[1.5050965548]], [[2.4746520519]], [[1.3838727474]], [[2.6422491074]], [[1.9354590178]], [[2.6077294350]], [[3.5365159512]], [[2.5945770741]], [[2.4315469265]], [[2.0029740334]], [[2.0259866714]], [[1.8531718254]], [[2.0701599121]], [[2.1804168224]], [[1.3508627415]], [[1.3079271317]], [[1.8872189522]], [[3.2373540401]], [[2.5754861832]], [[2.5747611523]], [[1.7179993391]], [[1.9157209396]], [[2.1155438423]], [[2.0252065659]], [[2.1283161640]], [[2.6452736855]], [[1.8267173767]], [[2.7579760551]], [[1.2132956982]], [[3.0068135262]], [[2.4631180763]], [[3.5405578613]], [[2.6474239826]], [[1.5910613537]], [[2.0927491188]], [[1.9897241592]], [[1.1013202667]], [[2.2189254761]], [[1.8796958923]], [[1.7149174213]], [[1.8627212048]], [[1.7743721008]], [[1.2092382908]], [[1.9356389046]], [[3.0627901554]], [[1.8477802277]], [[2.9874773026]], [[1.5558798313]], [[1.8826551437]], [[2.6651489735]], [[0.9460638762]], [[1.6016631126]], [[1.6695210934]], [[2.5003185272]], [[2.0576295853]], [[1.9772517681]], [[2.5009937286]], [[1.9855127335]], [[2.6145079136]], [[1.5627379417]], [[1.5143810511]], [[2.3694014549]], [[1.9381847382]], [[1.3082990646]], [[1.9379069805]], [[2.4449617863]], [[2.2284786701]], [[1.9400577545]], [[1.7861289978]], [[2.8127992153]], [[2.4590821266]], [[1.8723027706]], [[2.0154581070]], [[2.3502509594]], [[1.1160075665]], [[2.1576600075]], [[3.0725395679]], [[2.0451641083]], [[0.6608750224]], [[2.4659655094]], [[2.6076951027]], [[2.1282236576]], [[1.7682229280]], [[2.1097407341]], [[1.4181568623]], [[1.2313792706]], [[1.5169248581]], [[1.6490278244]], [[1.9212672710]], [[2.4415259361]], [[2.2367002964]], [[2.2774751186]], [[2.6165854931]], [[2.1887512207]], [[1.9790605307]], [[1.9198820591]], [[2.5306496620]], [[2.4902074337]], [[2.8822772503]], [[2.0687849522]], [[1.8810281754]], [[1.0920443535]], [[2.3473873138]], [[2.4194779396]], [[1.9421298504]], [[0.5074310899]], [[1.7196934223]], [[0.8027705550]], [[1.4250694513]], [[1.6291654110]], [[1.6940532923]], [[1.4418413639]], [[2.7376053333]], [[1.3668283224]], [[1.7057944536]], [[1.9952664375]], [[2.4717228413]], [[1.0840055943]], [[2.1751985550]], [[1.1282248497]], [[1.7455551624]], [[2.2935512066]], [[2.1847345829]], [[2.0868473053]], [[0.4775733948]], [[2.6362545490]], [[2.3963522911]], [[2.2232995033]], [[2.1940543652]], [[1.0425589085]], [[1.0968611240]], [[1.7524145842]], [[2.8102638721]], [[2.5829305649]], [[2.1974239349]], [[1.6277592182]], [[1.8983659744]], [[1.5967221260]], [[1.8115600348]], [[2.0387825966]], [[1.2349385023]], [[2.2242774963]], [[1.9058637619]], [[0.7111553550]], [[1.3966543674]], [[2.2581615448]], [[1.9319989681]], [[2.0974605083]], [[0.8482221961]], [[1.9527466297]], [[2.3510360718]], [[2.3399691582]], [[2.5700280666]], [[2.5576958656]], [[1.8749256134]], [[1.5995395184]], [[1.1973047256]], [[1.8889935017]], [[1.2457716465]], [[1.1657190323]], [[1.5862995386]], [[1.8062589169]], [[1.9646565914]], [[2.4827551842]], [[2.3427269459]], [[2.1380708218]], [[2.0972955227]], [[2.0793805122]], [[2.0545210838]], [[2.4332692623]], [[1.9909231663]], [[2.5593914986]], [[1.7359678745]], [[1.3405991793]], [[2.4576475620]], [[1.5923849344]], [[0.8783162236]], [[1.9325655699]], [[2.5579202175]], [[2.5777115822]], [[0.7576700449]], [[3.0607087612]], [[1.9372196198]], [[2.2473897934]], [[1.8931858540]], [[1.8002314568]], [[2.3377048969]], [[2.6462082863]], [[2.0836505890]], [[2.9943666458]], [[2.2693295479]], [[1.8470360041]], [[2.0004286766]], [[1.0881807804]], [[2.2922675610]], [[1.8939489126]], [[1.6876378059]], [[2.1403608322]], [[2.1730372906]], [[2.0085713863]], [[2.4433531761]], [[1.8390433788]], [[2.5747222900]], [[1.5938475132]], [[1.2154915333]], [[1.7041221857]], [[1.6980329752]], [[2.0305602551]], [[2.0169553757]], [[1.1281700134]], [[2.1298344135]], [[2.4407727718]], [[1.2411930561]], [[2.5820686817]], [[1.6975775957]], [[1.4171856642]], [[1.6448955536]], [[1.7079312801]], [[2.0430123806]], [[2.6770575047]], [[2.5490894318]], [[2.7350831032]], [[2.0681326389]], [[1.9966527224]], [[2.7066144943]], [[2.1135072708]], [[2.1335916519]], [[1.1755639315]], [[1.6906957626]], [[2.1122374535]], [[1.2308497429]], [[1.0500020981]], [[3.4394295216]], [[1.1447603703]], [[1.3113223314]], [[2.0852017403]], [[1.6112942696]], [[1.1555157900]], [[1.5976965427]], [[2.3042311668]], [[0.6561325788]], [[1.4502598047]], [[1.4592287540]], [[2.1307442188]], [[0.9457106590]], [[1.6666777134]], [[0.6475367546]], [[1.5687589645]], [[2.2034051418]], [[1.4414641857]], [[1.9517776966]], [[1.0384047031]], [[2.0780453682]], [[1.2406331301]], [[1.7416948080]], [[1.4409965277]], [[1.4011693001]], [[1.3811714649]], [[1.4451811314]], [[2.3342161179]], [[0.9672920108]], [[0.7198982239]], [[1.2746539116]], [[1.3744841814]], [[1.8599383831]], [[0.6956352592]], [[0.7422550321]], [[2.0261144638]], [[1.9932048321]], [[1.3647475243]], [[1.2305493355]], [[1.2675635815]], [[1.1903040409]], [[2.4475209713]], [[2.1161994934]], [[2.2318627834]], [[1.5944170952]], [[2.1591389179]], [[1.7946082354]], [[1.9572036266]], [[1.7890419960]], [[1.5016119480]], [[1.4993977547]], [[1.5436656475]], [[1.8229780197]], [[0.7076258063]], [[1.3922975063]], [[0.8836020231]], [[2.1523113251]], [[1.7926740646]], [[1.2107822895]], [[1.7866199017]], [[2.0228738785]], [[1.4151389599]], [[1.6561564207]], [[2.7083377838]], [[1.5731332302]], [[1.5417602062]], [[2.2978768349]], [[1.8048982620]], [[1.0197699070]], [[2.5689182281]], [[1.9207156897]], [[1.5669262409]], [[1.0021938086]], [[1.0122436285]], [[1.5895566940]], [[1.5285730362]], [[0.8092445731]], [[1.8269070387]], [[2.2193837166]], [[2.2482719421]], [[1.1043691635]], [[1.8965046406]], [[1.2286930084]], [[1.3172914982]], [[0.6924899817]], [[0.2391647100]], [[0.3091518283]], [[2.1676588058]], [[1.3594446182]], [[1.9698956013]], [[1.7561602592]], [[1.5505784750]], [[1.1046645641]], [[1.7907090187]], [[0.9408040047]], [[2.2968308926]], [[0.3507485986]], [[1.0683232546]], [[0.9857742190]], [[1.3872470856]], [[2.1478366852]], [[1.7414052486]], [[1.4622098207]], [[1.8110802174]], [[2.3387472630]], [[1.6464242935]], [[1.7139790058]], [[2.6530718803]], [[1.0672132969]], [[1.9933679104]], [[2.4932761192]], [[1.7841987610]], [[1.1697916985]], [[1.8800296783]], [[1.2469937801]], [[1.6056798697]], [[0.9578014016]], [[1.5572735071]], [[0.5278249979]], [[1.6521646976]], [[1.5527687073]], [[1.8032001257]], [[1.6926729679]], [[2.4715511799]], [[2.0873689651]], [[1.7234015465]], [[1.2142677307]], [[1.4172137976]], [[2.5624651909]], [[0.4544686675]], [[1.8021951914]], [[1.4658420086]], [[1.8690576553]], [[1.9521789551]], [[1.0924329758]], [[0.8325417638]], [[2.6377520561]], [[1.7632641792]], [[1.7439229488]], [[0.9657251835]], [[1.3678379059]], [[2.2542760372]], [[1.4037462473]], [[1.4685748816]], [[1.8236267567]], [[1.2359704971]], [[1.4835586548]], [[1.5493826866]], [[1.8001058102]], [[0.7245556712]], [[1.0450589657]], [[1.0194532871]], [[0.9725016356]], [[1.7611985207]], [[1.2553027868]], [[1.6735062599]], [[1.7654244900]], [[1.3804740906]], [[1.4142068624]], [[1.1884090900]], [[1.3350558281]], [[0.8620188832]], [[2.2143335342]], [[0.7008385658]], [[1.4918987751]], [[1.3344490528]], [[1.9376916885]], [[2.2101497650]], [[1.5036303997]], [[1.3469265699]], [[1.2364348173]], [[0.8898710012]], [[1.0841038227]], [[1.2608656883]], [[2.7296915054]], [[1.7304069996]], [[2.4811725616]], [[1.3596812487]], [[1.1468870640]], [[2.6548738480]], [[1.2181899548]], [[1.4475929737]], [[0.9516216516]], [[1.4512550831]], [[1.1619448662]], [[0.7206761241]], [[1.6423406601]], [[1.0177131891]], [[0.8694836497]], [[2.2868351936]], [[2.0222301483]], [[1.2353727818]], [[1.0885820389]], [[2.0871546268]], [[1.8499560356]], [[1.4526863098]], [[1.9035083055]], [[1.2300615311]], [[1.7379571199]], [[1.3231043816]], [[1.1438457966]], [[2.4367780685]], [[0.8798409104]], [[1.6363316774]], [[0.5541524887]], [[1.6404782534]], [[1.2015752792]], [[1.8588250875]], [[1.0939009190]], [[2.1510286331]], [[1.6792905331]], [[2.2943058014]], [[1.2601904869]], [[1.3873966932]], [[1.4820153713]], [[2.2082509995]]]])
test_advindex.test_module[Advindex1-[((1, 2), torch.float32)]]: forge/test/models_ops/test_advindex.py#L34
IndexError: index 933 is out of bounds for dimension 0 with size 1
test_advindex.test_module[Advindex0-[((32, 2), torch.float32), ((1,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L24
IndexError: index 775 is out of bounds for dimension 0 with size 32
test_advindex.test_module[Advindex3-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L60
IndexError: index 556 is out of bounds for dimension 0 with size 169
test_advindex.test_module[Advindex5-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L86
IndexError: index 725 is out of bounds for dimension 0 with size 169
test_avgpool2d.test_module[Avgpool2D9-[((1, 1536, 8, 8), torch.float32)]]: forge/test/models_ops/test_avgpool2d.py#L1472
RuntimeError: Tensor 1 - stride mismatch: expected [64, 1], got [0, 0]
test_broadcast.test_module[Broadcast1-[((1, 1, 1, 128), torch.bool)]]: forge/test/models_ops/test_broadcast.py#L172
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((2, 13, 1), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[ 44.], [239.], [933.], [760.], [963.], [379.], [427.], [503.], [497.], [683.], [101.], [866.], [756.]], [[399.], [878.], [376.], [ 56.], [868.], [794.], [ 33.], [126.], [119.], [391.], [254.], [824.], [841.]]]), compiled_model=tensor([[[ 44.], [ 0.], [239.], [ 0.], [933.], [ 0.], [760.], [ 0.], [963.], [ 0.], [379.], [ 0.], [427.]], [[ 0.], [503.], [ 0.], [497.], [ 0.], [683.], [ 0.], [101.], [ 0.], [866.], [ 0.], [756.], [ 0.]]])
test_cast.test_module[Cast0-[((2, 1, 1, 13), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((2, 1, 7, 7), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 1, 1, 128), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427., 503., 497., 683., 101., 866., 756., 399., 878., 376., 56., 868., 794., 33., 126., 119., 391., 254., 824., 841., 269., 969., 549., 480., 481., 12., 363., 360., 995., 85., 822., 999., 711., 288., 778., 43., 896., 489., 771., 757., 283., 395., 782., 671., 840., 969., 673., 441., 111., 280., 503., 706., 476., 327., 699., 326., 363., 474., 775., 700., 218., 632., 568., 12., 277., 645., 110., 80., 348., 521., 119., 916., 881., 490., 682., 519., 944., 633., 169., 863., 409., 233., 219., 78., 735., 83., 222., 858., 235., 416., 446., 835., 877., 212., 505., 946., 156., 315., 284., 750., 108., 71., 847., 108., 75., 84., 884., 148., 144., 534., 519., 260., 907., 514., 175., 663., 613., 157., 133., 120., 249., 389.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760., 0., 963., 0., 379., 0., 427., 0., 503., 0., 497., 0., 683., 0., 101., 0., 866., 0., 756., 0., 399., 0., 878., 0., 376., 0., 56., 0., 868., 0., 794., 0., 33., 0., 126., 0., 119., 0., 391., 0., 254., 0., 824., 0., 841., 0., 269., 0., 969., 0., 549., 0., 480., 0., 481., 0., 12., 0., 363., 0., 360., 0., 995., 0., 85., 0., 822., 0., 999., 0., 711., 0., 288., 0., 778., 0., 43., 0., 896., 0., 489., 0., 771., 0., 757., 0., 283., 0., 395., 0., 782., 0., 671., 0., 840., 0., 969., 0., 673., 0., 441., 0., 111., 0., 280., 0., 503., 0., 706., 0., 476., 0., 327., 0., 699., 0., 326., 0., 363., 0., 474., 0.]]]])
test_cast.test_module[Cast0-[((1, 1, 256, 256), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast2-[((1, 1, 256, 256), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 128), torch.int32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast0-[((1, 12, 128, 128), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 384), torch.int32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast1-[((1, 1, 32, 32), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 1, 32, 32), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 1, 256, 256), torch.float32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast2-[((1, 32), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 16, 192, 640), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_concatenate.test_module[Concatenate52-[((256, 1536, 1, 1), torch.float32), ((384, 1536, 1, 1), torch.float32), ((384, 1536, 1, 1), torch.float32)]]: forge/test/models_ops/test_concatenate.py#L12958
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 6442450944 B DRAM buffer across 12 banks, where each bank needs to store 536870912 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7f0ba4a797e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7f0ba4a44e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x1ebfb43) [0x7f0b95906b43] --- tt::tt_metal::allocate_tensor_on_devices(tt::tt_metal::TensorSpec const&, std::vector<tt::tt_metal::v0::IDevice*, std::allocator<tt::tt_metal::v0::IDevice*> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xdbf0e) [0x7f0ba5654f0e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xdba65) [0x7f0ba5654a65] --- tt::runtime::ttnn::operations::creation::run(tt::target::ttnn::EmptyOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f0ba5aa09f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f0ba5aa093e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f0ba586bb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x561b5ffe4b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x561b5ffd4a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x561b5ffda574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x561b5fff3172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x561b5ffcf97f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d]
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((1024, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7f0ba5636288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7f0b93e367a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7f0b93e29fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7f0b93e90829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7f0b96007fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7f0b960083e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7f0b9600772b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f0b960073e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f0b960072d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f0b9600649f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f0b93e8fb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f0b9600dfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f0b9600e7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f0b93e276bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f0b93e16f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f0ba5631365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f0ba5aa09f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f0ba5aa093e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f0ba586bb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x561b5ffe4b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x561b5ffd4a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x561b5ffda574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x561b5fff3172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x561b5ffcf97f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30]
test_conv2d.test_module[Conv2D1-[((1, 1280, 3000, 1), torch.float32), ((1280, 1280, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[633.6943969727], [945.1707763672], [946.9923706055], ..., [953.5204467773], [948.5270996094], [940.5952148438]], [[635.0733642578], [950.9418945312], [948.4122924805], ..., [941.4169311523], [953.1663208008], [947.6665649414]], [[652.5844116211], [962.0007324219], [961.6907958984], ..., [961.0711059570], [963.7894287109], [960.6210937500]], ..., [[647.2447509766], [947.0679321289], [955.3330078125], ..., [965.5653686523], [965.4138183594], [966.5710449219]], [[639.2039184570], [952.5633544922], [947.1241455078], ..., [951.1453247070], [948.8290405273], [956.2284545898]], [[642.4246215820], [964.2901000977], [956.3983764648], ..., [979.0764160156], [967.5105590820], [959.4161376953]]]]), compiled_model=tensor([[[[636.], [952.], [956.], ..., [944.], [952.], [936.]], [[632.], [976.], [944.], ..., [956.], [956.], [964.]], [[644.], [976.], [992.], ..., [972.], [984.], [984.]], ..., [[644.], [952.], [964.], ..., [972.], [956.], [984.]], [[636.], [976.], [956.], ..., [960.], [948.], [964.]], [[648.], [984.], [964.], ..., [980.], [972.], [964.]]]])
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((384, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7f0ba5636288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7f0b93e367a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7f0b93e29fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7f0b93e90829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7f0b96007fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7f0b960083e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7f0b9600772b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f0b960073e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f0b960072d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f0b9600649f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f0b93e8fb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f0b9600dfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f0b9600e7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f0b93e276bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f0b93e16f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f0ba5631365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f0ba5aa09f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f0ba5aa093e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f0ba586bb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x561b5ffe4b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x561b5ffd4a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x561b5ffda574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x561b5fff3172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x561b5ffcf97f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30]
test_conv2d.test_module[Conv2D1-[((1, 512, 3000, 1), torch.float32), ((512, 512, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7f0ba5636288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7f0b93e367a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7f0b93e29fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7f0b93e90829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7f0b96007fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7f0b960083e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7f0b9600772b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f0b960073e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f0b960072d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f0b9600649f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f0b93e8fb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f0b9600dfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f0b9600e7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f0b93e276bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f0b93e16f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f0ba5631365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f0ba5aa09f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f0ba5aa093e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f0ba586bb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x561b5ffe4b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x561b5ffd4a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x561b5ffda574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x561b5fff3172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x561b5ffcf97f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30]
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((768, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7f0ba5636288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7f0b93e367a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7f0b93e29fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7f0b93e90829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7f0b96007fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7f0b960083e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7f0b9600772b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f0b960073e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f0b960072d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f0b9600649f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f0b93e8fb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f0b9600dfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f0b9600e7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f0b93e276bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f0b93e16f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f0ba5631365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f0ba5aa09f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f0ba5aa093e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f0ba586bb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x561b5ffe4b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x561b5ffd4a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x561b5ffda574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x561b5fff3172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x561b5ffcf97f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30]
test_conv2d.test_module[Conv2D17-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7f0ba4a797e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7f0ba4a44e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7f0b94a46833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7f0b94a46699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7f0b9600739a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f0b960072d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f0b9600649f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7f0b94a4569a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f0b9600dfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f0b9600e7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7f0b93e7df8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7f0b93e16d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f0ba5631365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f0ba5aa09f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f0ba5aa093e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f0ba586bb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x561b5ffe4b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x561b5ffd4a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x561b5ffda574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x561b5fff3172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x561b5ffdb39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x561b5ffd599e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x561b5ffcf97f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x561b5fff310b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x561b5ffcf790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x561b5ffd1c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x561b5ffd09ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x561b5fff24d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x561b5ffd4702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x561b5ffe538c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x561b5ffda61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x561b5ffef62c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x561b600f7464]
test_conv2d.test_module[Conv2D516-[((1, 64, 147, 147), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[144.5903320312, 146.8498992920, 148.8105468750, ..., 149.3406066895, 144.7736358643, 150.6027526855], [154.3025665283, 151.8691406250, 145.3336944580, ..., 149.5803680420, 149.9938964844, 151.3875579834], [148.8137664795, 149.7992095947, 149.2318572998, ..., 153.0614166260, 147.5972442627, 157.3220672607], ..., [148.7757873535, 154.6059722900, 147.5536193848, ..., 143.4194946289, 144.4598236084, 142.1168823242], [151.3589782715, 147.3620910645, 148.7239379883, ..., 147.4233245850, 147.5432281494, 149.1203155518], [147.1671905518, 147.2913665771, 146.9279327393, ..., 145.6754302979, 149.8158264160, 147.3131408691]], [[140.7183990479, 144.2240142822, 138.4780426025, ..., 143.4065856934, 139.7475128174, 148.9764556885], [148.4001770020, 145.3048095703, 140.8634948730, ..., 144.0821838379, 144.9893646240, 145.4882049561], [144.0110778809, 147.9010772705, 145.4528198242, ..., 146.0332183838, 149.6196746826, 147.2288208008], ..., [141.2115783691, 145.7017669678, 145.4076232910, ..., 139.7186431885, 137.5715332031, 140.2689666748], [149.5879821777, 147.2990875244, 147.5193023682, ..., 142.9397125244, 141.5359954834, 139.4499053955], [141.3296051025, 139.4872283936, 142.7448730469, ..., 143.9296875000, 138.2050781250, 142.2441558838]], [[140.0239105225, 144.8146820068, 146.0733337402, ..., 146.1657104492, 143.6526947021, 151.0458221436], [150.7275238037, 146.1761474609, 138.6365814209, ..., 148.8564910889, 147.3908081055, 151.4685974121], [143.5762023926, 147.9291687012, 146.8162078857, ..., 149.2857055664, 146.2794036865, 144.9579467773], ..., [144.2277832031, 147.8683166504, 151.5809936523, ..., 140.3446655273, 140.5933532715, 141.3680419922], [151.0900573730, 148.8614501953, 141.8109741211, ..., 147.4037628174, 144.2274475098, 143.9186248779], [145.4204254150, 142.1257171631, 145.7842712402, ..., 143.2630615234, 145.9327850342, 146.4102020264]], ..., [[134.7500457764, 143.8243408203, 139.1240997314, ..., 139.3364562988, 139.1329498291, 149.3361816406], [148.0199737549, 143.2199707031, 134.1446075439, ..., 144.3830718994, 138.9013824463, 142.9081268311], [140.7188110352, 141.4164886475, 142.2360839844, ..., 144.0925140381, 143.3411560059, 144.8574523926], ..., [140.1158294678, 142.4669189453, 142.7184906006, ..., 132.8171234131, 134.7518768311, 137.1859741211], [144.0683593750, 140.9325408936, 142.5516357422, ..., 142.4145355225, 140.0834960938, 143.4492034912], [138.6913299561, 137.5113525391, 136.3057708740, ..., 140.7984924316, 138.6292724609, 147.8159637451]], [[140.1265411377, 146.6797485352, 147.7295074463, ..., 145.9743957520, 146.7216491699, 151.2752227783], [152.9240570068, 149.2510375977, 143.1198272705, ..., 143.7355499268, 145.5089721680, 151.1294708252], [146.5888214111, 147.2342224121, 150.8453369141, ..., 147.5011138916, 149.2423400879, 147.6086120605], ..., [144.1636505127, 146.8798522949, 144.5800323486, ..., 136.1079711914, 137.8281555176, 142.2316894531], [149.4996032715, 147.0644531250, 147.9989776611, ..., 146.5351257324, 142.9446868896, 140.9440765381], [140.8222198486, 141.2861022949, 140.4582977295, ..., 144.4272460938, 143.2029571533, 145.1761779785]], [[144.1779174805, 145.3422698975, 143.0585327148, ..., 147.6744079590, 146.9033508301, 154.4301300049], [157.9369659424, 150.3078613281, 148.8754882812, ..., 149.0836639404, 148.6465454102, 154.1902313232], [146.4096221924, 152.5247650146, 153.4474792480, ..., 153.1373748779, 152.7331237793, 148.0358734131], ..., [147.6069030762, 154.3356475830, 151.9167022705, ..., 141.9215240479, 142.0694274902, 143.5206756592], [152.3010711670, 153.6093597412, 152.7453918457, ..., 153.0643463135, 146.0541992188, 148.6673736572], [150.5096740723, 140.0808410645, 147.7682189941, ..., 150.7264862061, 145.5346069336, 152.1295776367]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[142., 146., 147., ..., 149., 142., 147.], [153., 150., 143., ..., 148., 148., 150.], [147., 146., 146., ..., 149., 146., 156.], ..., [145., 151., 145., ..., 142., 142., 141.], [150., 146., 147., ..., 145., 146., 147.], [145., 144., 144., ..., 142., 146., 145.]], [[139., 143., 137., ..., 140., 136., 147.], [147., 144., 139., ..., 143., 143., 141.], [141., 146., 142., ..., 143., 148., 144.], ..., [140., 144., 142., ..., 138., 136., 139.], [148., 144., 145., ..., 142., 139., 137.], [140., 137., 141., ..., 142., 137., 140.]], [[139., 143., 144., ..., 145., 141., 148.], [149., 145., 137., ..., 146., 144., 149.], [141., 145., 144., ..., 148., 143., 142.], ..., [141., 146., 148., ..., 139., 139., 140.], [149., 145., 140., ..., 145., 142., 142.], [143., 140., 143., ..., 140., 144., 144.]], ..., [[133., 141., 138., ..., 138., 139., 149.], [146., 141., 132., ..., 142., 137., 142.], [140., 140., 139., ..., 143., 141., 142.], ..., [139., 140., 140., ..., 131., 132., 134.], [143., 140., 140., ..., 139., 139., 142.], [135., 136., 135., ..., 138., 138., 147.]], [[138., 144., 146., ..., 146., 144., 150.], [149., 147., 141., ..., 142., 143., 148.], [146., 144., 149., ..., 144., 147., 146.], ..., [143., 145., 143., ..., 134., 137., 141.], [148., 146., 147., ..., 145., 141., 139.], [139., 140., 140., ..., 143., 141., 143.]], [[141., 143., 141., ..., 146., 145., 153.], [155., 149., 147., ..., 146., 145., 151.], [145., 150., 152., ..., 150., 149., 145.], ..., [146., 152., 150., ..., 140., 138., 143.], [149., 151., 150., ..., 153., 144., 147.], [149., 138., 146., ..., 150., 144., 150.]]]])
test_conv2d.test_module[Conv2D518-[((1, 64, 73, 73), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[152.2259826660, 145.7771911621, 136.8717651367, ..., 140.4228363037, 139.6538238525, 139.1413421631], [151.6834259033, 143.0866241455, 140.6794281006, ..., 143.0262756348, 144.8131103516, 139.7744598389], [145.7750549316, 146.3048248291, 143.8915710449, ..., 141.5709228516, 148.6718750000, 145.0983734131], ..., [140.6621551514, 145.6005554199, 144.2947540283, ..., 143.6639862061, 140.7307891846, 142.5148315430], [145.8384552002, 149.2470703125, 145.4356994629, ..., 141.1892089844, 141.5986022949, 142.9530639648], [145.2829742432, 151.6161651611, 146.1195831299, ..., 139.2726593018, 140.4278106689, 139.2372436523]], [[146.7124176025, 151.6222991943, 136.7942504883, ..., 137.8146514893, 138.8585662842, 139.5961761475], [149.2644500732, 144.1325531006, 143.4168243408, ..., 142.7654876709, 142.1508483887, 141.0497894287], [145.1377716064, 145.3028564453, 143.3576507568, ..., 139.8086853027, 145.8719024658, 142.0796661377], ..., [145.3661041260, 143.0798339844, 137.8923645020, ..., 139.2701416016, 138.6484222412, 139.6412200928], [145.1908874512, 145.6476440430, 142.2646942139, ..., 138.0929107666, 138.2590484619, 144.4056243896], [143.4348297119, 143.0313262939, 142.1493225098, ..., 140.3582000732, 146.5320739746, 137.0543212891]], [[150.0823059082, 143.9370422363, 138.4825897217, ..., 136.4377136230, 139.2084960938, 137.6576232910], [150.1298522949, 145.9816741943, 142.4092559814, ..., 142.6005096436, 146.3442687988, 137.2540588379], [140.4223327637, 143.3134002686, 141.5794067383, ..., 137.3434295654, 145.7196960449, 142.0828247070], ..., [144.6627502441, 146.5066375732, 140.1581573486, ..., 141.2984161377, 141.3062286377, 143.0517883301], [141.2523498535, 147.6969604492, 142.5403747559, ..., 140.4929199219, 140.3352355957, 138.5057525635], [138.2369537354, 145.4068908691, 146.2414703369, ..., 141.8226776123, 137.8505096436, 140.3447113037]], ..., [[147.7367401123, 145.0466156006, 140.2163543701, ..., 140.2339477539, 139.2320404053, 136.4789581299], [145.3736572266, 145.5703887939, 143.2475433350, ..., 139.9946746826, 142.2647399902, 138.7484130859], [144.3772583008, 145.1488342285, 145.6207122803, ..., 137.1410064697, 145.8037567139, 145.4990539551], ..., [139.1538238525, 142.3866729736, 141.2739105225, ..., 141.3141174316, 137.8016510010, 139.2997283936], [144.3430786133, 141.6739807129, 144.3353729248, ..., 138.9774780273, 137.9251251221, 137.9963073730], [142.9682922363, 141.9563751221, 144.3373565674, ..., 140.6261901855, 138.6582794189, 135.2939910889]], [[154.3753967285, 149.1354675293, 141.2576141357, ..., 142.9516601562, 141.2592315674, 136.3096313477], [152.1967315674, 153.2928619385, 146.0510711670, ..., 145.8645935059, 150.6549987793, 143.8348083496], [152.5205841064, 144.7463378906, 146.8518981934, ..., 143.7776794434, 151.7975616455, 145.5718536377], ..., [145.7089538574, 148.1030120850, 141.7923736572, ..., 146.3828125000, 145.0093536377, 144.0106811523], [142.0967102051, 152.5759429932, 148.8596649170, ..., 143.5588073730, 140.8418731689, 144.6685485840], [143.5753784180, 148.1425781250, 145.0681762695, ..., 146.1747589111, 140.6717529297, 141.0068817139]], [[157.2637481689, 155.8987274170, 145.4244384766, ..., 143.9485473633, 150.4982452393, 149.9707641602], [156.8666076660, 156.2886657715, 147.1239776611, ..., 148.1587829590, 149.9689331055, 149.1417694092], [154.3150329590, 151.1999969482, 151.9582977295, ..., 150.6881866455, 150.9894256592, 152.1382141113], ..., [151.2051849365, 148.8263854980, 149.0452117920, ..., 153.9624176025, 146.6357727051, 146.3478088379], [152.0362854004, 157.2517089844, 149.7593688965, ..., 148.9168548584, 146.1963806152, 145.5578308105], [150.8255004883, 153.9378051758, 151.1777191162, ..., 143.6191253662, 146.9277343750, 143.7196350098]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[152., 145., 135., ..., 138., 138., 138.], [149., 141., 138., ..., 141., 143., 138.], [144., 144., 142., ..., 138., 147., 142.], ..., [138., 144., 143., ..., 143., 140., 141.], [143., 148., 144., ..., 140., 138., 139.], [144., 148., 143., ..., 138., 138., 137.]], [[146., 149., 135., ..., 137., 137., 138.], [148., 143., 142., ..., 140., 141., 139.], [143., 143., 141., ..., 139., 142., 139.], ..., [144., 141., 136., ..., 137., 136., 140.], [144., 143., 141., ..., 135., 135., 144.], [141., 141., 141., ..., 138., 144., 136.]], [[148., 142., 136., ..., 134., 136., 136.], [147., 145., 141., ..., 139., 145., 135.], [139., 142., 142., ..., 136., 143., 139.], ..., [142., 144., 136., ..., 139., 140., 141.], [139., 146., 140., ..., 139., 137., 136.], [137., 143., 145., ..., 141., 135., 139.]], ..., [[145., 141., 138., ..., 137., 136., 133.], [141., 143., 141., ..., 138., 140., 138.], [142., 143., 142., ..., 135., 143., 144.], ..., [137., 140., 138., ..., 139., 137., 136.], [141., 138., 142., ..., 137., 137., 136.], [141., 139., 141., ..., 137., 136., 132.]], [[154., 146., 140., ..., 142., 140., 134.], [152., 152., 145., ..., 144., 149., 142.], [151., 144., 145., ..., 141., 148., 144.], ..., [144., 146., 140., ..., 143., 143., 142.], [139., 150., 147., ..., 141., 138., 143.], [142., 146., 142., ..., 144., 139., 139.]], [[156., 154., 145., ..., 140., 150., 149.], [154., 155., 146., ..., 146., 148., 146.], [152., 150., 150., ..., 149., 149., 150.], ..., [150., 145., 148., ..., 152., 145., 143.], [149., 154., 147., ..., 147., 142., 143.], [148., 151., 147., ..., 141., 145., 141.]]]])
test_conv2d.test_module[Conv2D521-[((1, 192, 71, 71), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[446.1499328613, 439.1386413574, 434.7010192871, ..., 444.4459838867, 445.5372009277, 421.7436218262], [440.6473083496, 445.6108398438, 437.2178649902, ..., 439.5466308594, 429.5158081055, 431.2489318848], [432.5221252441, 441.4844970703, 427.5329589844, ..., 426.8597106934, 426.2628173828, 426.5025024414], ..., [436.9982299805, 428.2520446777, 432.4375305176, ..., 431.2190246582, 422.9773559570, 431.0123291016], [431.6502380371, 432.1030273438, 422.4632568359, ..., 422.2445983887, 426.3515930176, 434.2157897949], [440.1680908203, 429.5302734375, 428.5905151367, ..., 422.3388977051, 422.3145751953, 431.9900512695]], [[450.5713500977, 431.4642028809, 435.4295043945, ..., 444.5227050781, 449.1511840820, 424.6429748535], [444.6395874023, 437.6932067871, 431.0646362305, ..., 450.2820129395, 435.8485412598, 424.6563720703], [434.7045593262, 438.0631713867, 428.8527221680, ..., 431.3270568848, 425.4383544922, 428.4918518066], ..., [430.4143676758, 425.7986450195, 430.9101867676, ..., 440.3790283203, 426.6940307617, 431.3434753418], [429.0862731934, 434.0317993164, 417.7901306152, ..., 422.2330627441, 418.9494628906, 438.0585937500], [436.3234863281, 428.4629211426, 426.1383361816, ..., 424.9827575684, 419.9928283691, 433.1638183594]], [[453.0535278320, 448.7880859375, 439.6737670898, ..., 447.6930236816, 453.1118469238, 426.3739929199], [451.5821228027, 443.2859802246, 439.6748962402, ..., 448.9718322754, 438.2503356934, 431.6145324707], [434.7266540527, 440.1850280762, 424.9765625000, ..., 438.2863769531, 429.3562316895, 427.5081481934], ..., [441.4554443359, 430.2238159180, 434.4473266602, ..., 439.5085144043, 424.2333374023, 430.3134460449], [437.1465454102, 438.3939208984, 423.3197937012, ..., 425.7508544922, 423.9235534668, 439.2757873535], [443.4285888672, 433.0761108398, 432.6029663086, ..., 426.0624389648, 426.8219299316, 429.5516357422]], ..., [[453.8680114746, 446.4806518555, 434.6709594727, ..., 455.5362548828, 444.9443969727, 433.0402221680], [452.1379699707, 444.3343505859, 445.6448669434, ..., 449.3179626465, 438.0439147949, 430.9378967285], [435.1301574707, 450.6653442383, 439.9476013184, ..., 435.2501525879, 433.1171569824, 436.8756713867], ..., [443.2767639160, 434.5807800293, 436.6314697266, ..., 439.7209167480, 427.7367248535, 435.8312377930], [439.5126342773, 443.5432739258, 422.8628540039, ..., 427.2470397949, 430.1888732910, 432.3753356934], [442.5853271484, 437.5104675293, 439.4239196777, ..., 428.2676696777, 429.5149841309, 439.9341735840]], [[451.0899658203, 439.7808227539, 436.0216369629, ..., 445.0820312500, 446.1300354004, 428.2850646973], [437.6214294434, 436.9980773926, 433.5463256836, ..., 445.3561706543, 434.3788146973, 423.7322692871], [429.3497619629, 437.1765747070, 427.9953308105, ..., 432.7474060059, 430.1040039062, 425.7378845215], ..., [430.3336486816, 418.1121520996, 430.6287841797, ..., 433.4544372559, 423.3660888672, 425.9108886719], [433.1035156250, 438.1240539551, 417.4965515137, ..., 421.3310546875, 416.8927612305, 435.6394042969], [441.5576782227, 428.5754699707, 430.2178344727, ..., 425.6445007324, 417.5131225586, 433.5842895508]], [[449.6306152344, 430.5096740723, 429.0747985840, ..., 443.6839599609, 440.6118774414, 419.7160339355], [439.8436889648, 432.2811279297, 431.9723510742, ..., 438.8259887695, 429.8714599609, 428.9606018066], [432.6675415039, 438.9447326660, 422.2742614746, ..., 427.5867004395, 420.1712036133, 429.8712158203], ..., [430.6726074219, 426.7988891602, 431.3893127441, ..., 432.4360656738, 415.1751708984, 429.2268066406], [428.8143005371, 432.2532958984, 409.3340759277, ..., 419.8291625977, 418.7747192383, 433.3817138672], [437.2994384766, 429.8687438965, 426.2240295410, ..., 427.5977172852, 418.4593505859, 431.1557922363]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[454., 438., 430., ..., 440., 448., 416.], [444., 442., 442., ..., 434., 426., 426.], [426., 436., 430., ..., 428., 426., 424.], ..., [428., 422., 436., ..., 438., 424., 428.], [434., 428., 420., ..., 418., 424., 438.], [440., 430., 436., ..., 410., 418., 430.]], [[448., 430., 434., ..., 442., 454., 422.], [442., 434., 428., ..., 448., 430., 416.], [432., 440., 424., ..., 426., 424., 424.], ..., [426., 426., 430., ..., 436., 422., 434.], [422., 428., 416., ..., 418., 412., 432.], [436., 422., 426., ..., 420., 422., 434.]], [[450., 438., 430., ..., 446., 456., 420.], [456., 444., 430., ..., 440., 428., 432.], [430., 436., 430., ..., 436., 424., 422.], ..., [444., 432., 432., ..., 438., 418., 432.], [438., 440., 424., ..., 426., 428., 438.], [448., 430., 430., ..., 428., 434., 422.]], ..., [[454., 448., 434., ..., 456., 446., 436.], [452., 444., 442., ..., 444., 438., 430.], [432., 454., 438., ..., 428., 428., 438.], ..., [446., 436., 436., ..., 440., 430., 430.], [438., 446., 426., ..., 428., 432., 432.], [438., 440., 428., ..., 428., 432., 444.]], [[448., 436., 438., ..., 452., 442., 432.], [444., 438., 436., ..., 448., 426., 416.], [434., 432., 424., ..., 432., 430., 424.], ..., [430., 414., 436., ..., 436., 418., 428.], [424., 442., 426., ..., 412., 414., 438.], [442., 438., 424., ..., 418., 420., 430.]], [[450., 428., 432., ..., 438., 440., 410.], [440., 426., 430., ..., 434., 428., 432.], [434., 440., 414., ..., 426., 414., 428.], ..., [432., 432., 426., ..., 432., 412., 424.], [432., 430., 404., ..., 416., 420., 436.], [438., 436., 424., ..., 420., 414., 430.]]]])
test_conv2d.test_module[Conv2D524-[((1, 384, 35, 35), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[866.4609985352, 863.1013793945, 861.1953735352, ..., 855.0989990234, 840.2864379883, 847.4705810547], [875.8906250000, 877.8523559570, 859.6842041016, ..., 848.9233398438, 850.2925415039, 858.7671508789], [868.1475830078, 869.0169677734, 869.6140747070, ..., 854.4511108398, 861.1550292969, 877.7504882812], ..., [864.5758056641, 878.4547729492, 872.5308227539, ..., 842.3846435547, 860.4345703125, 849.8447265625], [868.0847167969, 862.4199829102, 862.9453735352, ..., 864.3540649414, 877.8132324219, 868.6017456055], [871.2556762695, 879.8225097656, 878.2050170898, ..., 872.2765502930, 869.0935058594, 879.1612548828]], [[868.2838745117, 870.1877441406, 851.5869140625, ..., 854.5398559570, 840.5256347656, 852.4250488281], [869.6328125000, 867.2081298828, 859.9768066406, ..., 847.4663085938, 836.6455078125, 863.3948364258], [871.5944213867, 870.6660766602, 870.5272827148, ..., 853.5545043945, 862.3136596680, 875.3734130859], ..., [878.3984985352, 876.8424682617, 869.4633178711, ..., 844.9426879883, 851.4474487305, 842.9067993164], [860.2482910156, 873.5963745117, 859.1198730469, ..., 858.1121826172, 868.3348999023, 867.8046875000], [865.4127197266, 872.1913452148, 874.1326904297, ..., 859.1614990234, 874.2630615234, 881.2441406250]], [[888.0032348633, 881.4804077148, 883.1143798828, ..., 870.9645385742, 858.5325317383, 874.5654296875], [892.3177490234, 886.7066040039, 869.8302612305, ..., 855.6344604492, 854.0808715820, 871.8780517578], [878.8849487305, 890.1928100586, 873.0169067383, ..., 853.2687377930, 878.5581054688, 881.8147583008], ..., [887.5659179688, 881.7904663086, 885.1101684570, ..., 852.0249023438, 868.1282348633, 864.4175415039], [882.2733764648, 898.7310180664, 882.9476928711, ..., 876.7208862305, 889.6736450195, 877.4073486328], [882.8348388672, 900.7940673828, 893.5076904297, ..., 871.3868408203, 871.6420898438, 907.1170043945]], ..., [[866.0539550781, 873.6541137695, 859.7090454102, ..., 854.6393432617, 847.8032836914, 853.6879272461], [877.1873779297, 877.3082275391, 873.7100830078, ..., 843.3228149414, 838.6892089844, 858.4749755859], [866.7675781250, 881.3967285156, 876.8773803711, ..., 847.8247070312, 862.9194946289, 883.8536376953], ..., [864.8302612305, 882.9354858398, 869.4403686523, ..., 848.6573486328, 855.0936279297, 840.2236328125], [865.5090332031, 870.6389160156, 856.5808715820, ..., 873.1376953125, 876.6796875000, 865.3845825195], [875.5214843750, 882.4806518555, 875.6942138672, ..., 871.4704589844, 874.3905639648, 883.8245239258]], [[890.4667358398, 871.8570556641, 873.5860595703, ..., 872.9151611328, 864.4789428711, 872.6695556641], [895.8981323242, 878.6236572266, 876.0807495117, ..., 870.8341674805, 862.5111694336, 864.1347656250], [879.8024291992, 895.7070312500, 887.4442749023, ..., 866.4889526367, 884.2224731445, 888.7036132812], ..., [880.4244384766, 882.7206420898, 883.0745239258, ..., 860.8098144531, 871.7084350586, 868.9788818359], [879.2639160156, 887.0864868164, 882.5966186523, ..., 876.7026367188, 891.1694946289, 875.5422363281], [878.4291992188, 889.8889770508, 895.8454589844, ..., 883.0544433594, 889.5227050781, 898.1986694336]], [[870.4875488281, 862.5688476562, 857.4489746094, ..., 855.5870971680, 853.5465698242, 862.2807006836], [876.2697143555, 883.4018554688, 872.8113403320, ..., 854.5828857422, 849.0681762695, 859.8845214844], [868.9295654297, 883.5656127930, 872.2227172852, ..., 858.1094970703, 869.1029052734, 886.0479125977], ..., [875.6414184570, 888.0523681641, 872.6617431641, ..., 839.7261962891, 869.1243896484, 855.5270385742], [869.0084228516, 876.9665527344, 868.3314819336, ..., 863.1638793945, 874.4510498047, 865.7653808594], [866.9951171875, 881.1233520508, 882.6040039062, ..., 870.8356323242, 871.3972778320, 887.7750244141]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[868., 856., 872., ..., 860., 864., 872.], [884., 872., 868., ..., 864., 852., 864.], [884., 872., 880., ..., 868., 856., 872.], ..., [876., 880., 872., ..., 860., 868., 860.], [880., 860., 872., ..., 864., 892., 872.], [884., 884., 864., ..., 884., 868., 892.]], [[872., 864., 848., ..., 864., 868., 860.], [872., 860., 872., ..., 860., 856., 868.], [872., 860., 880., ..., 868., 868., 876.], ..., [880., 876., 876., ..., 864., 864., 856.], [860., 872., 880., ..., 856., 868., 876.], [852., 876., 876., ..., 872., 868., 892.]], [[880., 876., 868., ..., 864., 852., 876.], [904., 888., 884., ..., 868., 868., 860.], [872., 892., 860., ..., 868., 872., 876.], ..., [892., 884., 876., ..., 860., 864., 864.], [880., 876., 880., ..., 872., 884., 864.], [880., 888., 888., ..., 872., 872., 900.]], ..., [[856., 860., 864., ..., 864., 860., 848.], [872., 860., 864., ..., 864., 844., 852.], [872., 880., 876., ..., 844., 860., 880.], ..., [884., 880., 880., ..., 872., 864., 856.], [876., 864., 856., ..., 868., 888., 872.], [872., 872., 864., ..., 876., 868., 876.]], [[876., 872., 868., ..., 876., 860., 872.], [896., 872., 872., ..., 872., 868., 864.], [884., 892., 876., ..., 872., 884., 884.], ..., [884., 892., 880., ..., 872., 884., 876.], [888., 884., 888., ..., 872., 888., 876.], [872., 900., 892., ..., 880., 888., 896.]], [[864., 852., 860., ..., 848., 856., 864.], [884., 876., 872., ..., 856., 852., 856.], [860., 868., 856., ..., 860., 880., 880.], ..., [880., 876., 876., ..., 848., 876., 856.], [868., 876., 876., ..., 860., 868., 864.], [872., 884., 880., ..., 864., 868., 884.]]]])
test_conv2d.test_module[Conv2D526-[((1, 224, 35, 35), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[500.3655395508, 489.4805603027, 488.6658325195, ..., 498.1962585449, 485.7242431641, 486.2211608887], [503.1024780273, 495.0940551758, 496.3491210938, ..., 490.6144409180, 489.8194274902, 489.8637390137], [498.7118530273, 494.7251892090, 487.2424621582, ..., 487.9169311523, 495.7423400879, 503.2265319824], ..., [498.4992370605, 518.2477416992, 501.5748901367, ..., 495.7089843750, 495.1419067383, 481.3142089844], [495.4396972656, 508.5327758789, 506.0981445312, ..., 495.9432983398, 509.0792236328, 495.8169555664], [490.7167053223, 501.0510864258, 505.4312744141, ..., 497.9179992676, 504.0027465820, 511.4380798340]], [[523.4204711914, 508.6375427246, 507.7442932129, ..., 516.4190673828, 496.1727905273, 508.5605773926], [518.5764160156, 515.1816406250, 513.1814575195, ..., 503.7810974121, 504.7005310059, 501.2022399902], [516.2860717773, 514.2275390625, 512.8796997070, ..., 509.4268798828, 516.1939697266, 511.4122619629], ..., [519.9572753906, 523.6141967773, 522.6586914062, ..., 512.8377075195, 522.3417968750, 503.4328613281], [507.0760498047, 521.7188720703, 518.8154907227, ..., 512.7859497070, 525.6207885742, 506.8907165527], [513.5719604492, 518.0314331055, 518.3280029297, ..., 520.2257080078, 526.4733276367, 524.2277221680]], [[517.4378051758, 492.0399169922, 502.1506652832, ..., 497.5054626465, 492.6845092773, 497.6663208008], [510.4721374512, 498.3109741211, 509.2692565918, ..., 498.5930175781, 493.6991882324, 495.6730346680], [510.4767150879, 503.6236572266, 497.2259826660, ..., 500.2600097656, 504.0603942871, 509.7193603516], ..., [507.8258056641, 517.9182739258, 515.7431030273, ..., 502.1661071777, 510.3352966309, 494.9486083984], [507.4066772461, 513.5437011719, 500.5325012207, ..., 501.5936584473, 517.8285522461, 502.3205566406], [507.5396728516, 518.0348510742, 508.5066223145, ..., 515.4725341797, 511.8544921875, 514.9838867188]], ..., [[513.7340087891, 496.5385742188, 499.4902954102, ..., 498.3547363281, 491.6601867676, 497.3186645508], [514.0239257812, 502.1475219727, 507.0024414062, ..., 496.7256774902, 498.2815246582, 507.7489013672], [507.5863952637, 512.1275634766, 508.1878051758, ..., 497.1116638184, 509.3461303711, 511.1389160156], ..., [513.8406372070, 523.5374755859, 516.0022583008, ..., 502.9671936035, 510.3093566895, 494.7483520508], [498.4073181152, 507.9766235352, 509.7402648926, ..., 500.9189758301, 523.5086059570, 503.1337280273], [510.0997619629, 515.4460449219, 505.5416870117, ..., 514.4185791016, 521.7772827148, 521.4566650391]], [[515.0075073242, 509.7844238281, 509.4789123535, ..., 513.8734741211, 494.9673156738, 509.1307067871], [531.3638916016, 509.8542175293, 510.9261169434, ..., 507.6830749512, 516.3020019531, 502.5183105469], [521.5582275391, 516.9340209961, 512.7407226562, ..., 517.3104858398, 520.7433471680, 511.6295166016], ..., [520.1965332031, 532.0809936523, 530.5952148438, ..., 514.3910522461, 517.5521240234, 500.3165893555], [509.4289855957, 520.7044677734, 517.2095947266, ..., 513.5690917969, 534.2973022461, 505.8474731445], [508.5122375488, 521.9270019531, 518.3892822266, ..., 518.3449096680, 525.2764892578, 526.2169189453]], [[508.9140930176, 502.6706542969, 506.0242919922, ..., 500.0994873047, 491.6975708008, 498.5713806152], [509.7074584961, 504.4242248535, 506.1207580566, ..., 498.9416198730, 500.4948730469, 498.3580932617], [510.6957397461, 505.6954956055, 501.4795532227, ..., 496.7246704102, 507.0613403320, 508.7590332031], ..., [509.0237731934, 523.0478515625, 517.4692382812, ..., 505.1237487793, 516.2896118164, 499.3422851562], [501.1950378418, 515.9891357422, 505.2563781738, ..., 502.7289733887, 519.1705932617, 506.7718811035], [503.6128234863, 515.0332031250, 503.2246704102, ..., 514.8377075195, 521.2158203125, 521.6481933594]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[498., 494., 486., ..., 504., 480., 486.], [502., 498., 492., ..., 490., 484., 490.], [496., 496., 488., ..., 480., 498., 502.], ..., [502., 516., 498., ..., 490., 498., 480.], [492., 502., 502., ..., 496., 510., 504.], [492., 506., 502., ..., 498., 500., 510.]], [[524., 508., 504., ..., 512., 494., 496.], [520., 512., 512., ..., 492., 508., 496.], [516., 508., 508., ..., 516., 516., 512.], ..., [512., 524., 512., ..., 498., 524., 498.], [504., 524., 524., ..., 508., 520., 506.], [512., 512., 520., ..., 516., 528., 524.]], [[512., 486., 506., ..., 492., 492., 498.], [506., 498., 508., ..., 498., 492., 490.], [508., 494., 488., ..., 504., 512., 508.], ..., [500., 508., 516., ..., 502., 500., 494.], [512., 512., 498., ..., 500., 512., 508.], [510., 524., 498., ..., 502., 508., 520.]], ..., [[516., 492., 498., ..., 496., 496., 506.], [508., 502., 502., ..., 494., 494., 502.], [506., 504., 506., ..., 498., 502., 512.], ..., [512., 528., 512., ..., 494., 506., 486.], [492., 510., 508., ..., 498., 524., 502.], [504., 510., 516., ..., 512., 524., 524.]], [[516., 508., 520., ..., 508., 492., 512.], [520., 508., 508., ..., 504., 516., 508.], [520., 512., 510., ..., 520., 532., 512.], ..., [520., 536., 532., ..., 512., 512., 496.], [502., 520., 532., ..., 508., 528., 504.], [504., 516., 520., ..., 524., 520., 528.]], [[516., 500., 502., ..., 510., 488., 500.], [508., 510., 504., ..., 494., 500., 502.], [516., 504., 502., ..., 502., 506., 512.], ..., [506., 528., 516., ..., 504., 512., 498.], [508., 520., 502., ..., 500., 510., 508.], [498., 512., 506., ..., 512., 512., 524.]]]])
test / run-tests (n300, 2)
Process completed with exit code 1.
test_add.test_module[Add46-[((1, 32), torch.int64)]]: forge/test/models_ops/test_add.py#L18561
TypeError: Dtype mismatch: framework_model.dtype=torch.int64, compiled_model.dtype=torch.int32
test_add.test_module[Add46-[((1, 128), torch.int64)]]: forge/test/models_ops/test_add.py#L18561
TypeError: Dtype mismatch: framework_model.dtype=torch.int64, compiled_model.dtype=torch.int32
test_add.test_module[Add1-[((1, 72, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1927615404]], [[1.4647266865]], [[0.7849823833]], [[0.8285354376]], [[1.0039277077]], [[1.3305835724]], [[1.1865983009]], [[1.5929496288]], [[1.1521329880]], [[1.3288111687]], [[1.0453984737]], [[1.0982222557]], [[0.7188307047]], [[0.8653638959]], [[0.9903934002]], [[1.2150267363]], [[1.3941725492]], [[1.4965163469]], [[0.8575344086]], [[0.9787735343]], [[1.3781135082]], [[1.6116988659]], [[1.0936048031]], [[1.5706608295]], [[1.1159132719]], [[1.2494120598]], [[1.6492431164]], [[0.7326697707]], [[0.8817359805]], [[1.0699223280]], [[1.0016050339]], [[1.6285053492]], [[0.8724151254]], [[0.9663385153]], [[0.8471847177]], [[0.7282244563]], [[0.9046347141]], [[1.6263039112]], [[1.4196140766]], [[1.4388412237]], [[1.2228007317]], [[0.9401631951]], [[1.2810972929]], [[0.7296575904]], [[0.8352218270]], [[0.9387399554]], [[1.5119738579]], [[1.4896655083]], [[0.9747574329]], [[1.1784636974]], [[1.5162853003]], [[1.6935715675]], [[1.3949460983]], [[1.2640514374]], [[1.5317480564]], [[0.9021037817]], [[1.2896769047]], [[0.8088521957]], [[0.8499618769]], [[0.9382131696]], [[1.4227414131]], [[1.3975851536]], [[0.9003286958]], [[1.3475584984]], [[1.4709908962]], [[1.1333962679]], [[1.2155957222]], [[1.3123573065]], [[1.5066932440]], [[1.6766020060]], [[0.8111931682]], [[1.0132701397]]]]), compiled_model=tensor([[[[1.1927615404]], [[1.6824965477]], [[1.0235810280]], [[1.0732088089]], [[0.9069300890]], [[0.6992873549]], [[1.0360896587]], [[1.0836420059]], [[0.4896509051]], [[1.5765525103]], [[1.2290732861]], [[0.4029533267]], [[0.6159117818]], [[0.5846289396]], [[0.7116078734]], [[0.7896433473]], [[1.3899457455]], [[1.0038596392]], [[0.8443251252]], [[1.0351226330]], [[1.5395443439]], [[1.6021494865]], [[0.4022322893]], [[1.0498074293]], [[1.1690658331]], [[1.1575577259]], [[1.0626960993]], [[0.2482550740]], [[1.1556056738]], [[1.2103263140]], [[0.5870874524]], [[1.3061580658]], [[0.1996111274]], [[0.7608464956]], [[0.2741503119]], [[0.1460411549]], [[0.6805799603]], [[1.5048716068]], [[1.0183440447]], [[1.5390250683]], [[0.7220262289]], [[1.1973433495]], [[1.4272422791]], [[0.1115111709]], [[0.5142747164]], [[0.7647963166]], [[1.3884195089]], [[1.4117476940]], [[0.9744666219]], [[1.0119088888]], [[1.0758159161]], [[1.7336610556]], [[0.7188166380]], [[0.7711930871]], [[1.2100782394]], [[0.4620421529]], [[0.9182553291]], [[0.2025364041]], [[0.5470993519]], [[0.8485864401]], [[0.9005036354]], [[1.1754205227]], [[1.0617492199]], [[1.0996534824]], [[1.2883820534]], [[0.8937568665]], [[1.1202814579]], [[1.4337720871]], [[1.7838113308]], [[1.7976250648]], [[1.0893950462]], [[0.7806043029]]]])
test_add.test_module[Add1-[((1, 480, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6432905197]], [[0.9152557254]], [[0.2355113626]], [[0.2790644169]], [[0.4544567466]], [[0.7811126113]], [[0.6371273398]], [[1.0434787273]], [[0.6026619077]], [[0.7793402076]], [[0.4959273934]], [[0.5487512350]], [[0.1693596840]], [[0.3158928752]], [[0.4409223795]], [[0.6655557156]], [[0.8447015285]], [[0.9470453262]], [[0.3080633879]], [[0.4293025136]], [[0.8286424875]], [[1.0622279644]], [[0.5441338420]], [[1.0211898088]], [[0.5664422512]], [[0.6999409795]], [[1.0997719765]], [[0.1831987500]], [[0.3322649598]], [[0.5204513073]], [[0.4521339536]], [[1.0790343285]], [[0.3229441047]], [[0.4168674946]], [[0.2977136970]], [[0.1787534356]], [[0.3551636934]], [[1.0768330097]], [[0.8701431155]], [[0.8893702030]], [[0.6733297110]], [[0.3906921744]], [[0.7316262722]], [[0.1801865697]], [[0.2857508063]], [[0.3892689347]], [[0.9625028968]], [[0.9401945472]], [[0.4252864122]], [[0.6289927363]], [[0.9668142796]], [[1.1441004276]], [[0.8454750180]], [[0.7145803571]], [[0.9822770953]], [[0.3526327610]], [[0.7402059436]], [[0.2593811750]], [[0.3004908562]], [[0.3887421489]], [[0.8732704520]], [[0.8481141329]], [[0.3508576751]], [[0.7980874777]], [[0.9215199351]], [[0.5839252472]], [[0.6661247015]], [[0.7628862858]], [[0.9572222233]], [[1.1271309853]], [[0.2617221475]], [[0.4637990594]], [[0.8435388803]], [[1.0613086224]], [[1.0821375847]], [[1.0882122517]], [[0.7465412021]], [[0.2122426033]], [[0.6930301785]], [[0.3342312574]], [[0.1810568571]], [[1.0912802219]], [[1.0272138119]], [[0.1482699513]], [[0.7406199574]], [[0.5628039241]], [[0.5647533536]], [[0.4181554914]], [[0.8393120170]], [[0.3508821726]], [[0.8303295970]], [[0.8998879790]], [[1.0049697161]], [[0.8339895010]], [[0.1521663070]], [[0.3226854801]], [[0.8966914415]], [[0.7516846061]], [[0.2569919229]], [[0.3591241837]], [[1.1174085140]], [[0.9839428663]], [[0.4290213585]], [[0.5211915374]], [[0.1707348824]], [[0.6380468607]], [[0.2705044746]], [[0.2613555789]], [[0.6194841266]], [[0.7221064568]], [[0.4422687888]], [[0.9437227249]], [[0.3427643776]], [[1.1007189751]], [[0.9896838665]], [[0.2253924608]], [[0.5225917697]], [[0.6695952415]], [[0.7199844718]], [[0.7656210661]], [[0.8432480693]], [[0.6769840121]], [[0.4030695558]], [[0.8836284280]], [[0.1674094796]], [[0.3506805897]], [[0.5218690038]], [[0.4034772515]], [[0.4721172452]], [[0.2372230887]], [[0.5406763554]], [[0.7539121509]], [[0.3213010430]], [[0.6213742495]], [[1.0049593449]], [[0.5956338048]], [[0.6609300375]], [[0.6038994789]], [[0.7482246161]], [[0.9649536610]], [[1.1206569672]], [[0.9645618796]], [[1.1217406988]], [[0.6108731031]], [[0.1978731751]], [[0.4099953175]], [[0.9874865413]], [[0.6437926888]], [[0.3985107541]], [[0.2638780475]], [[0.1791079044]], [[0.2250298262]], [[0.5456155539]], [[0.9212369323]], [[0.9173544645]], [[0.1648179889]], [[0.9589249492]], [[0.2557792068]], [[0.5413287878]], [[0.4442976117]], [[0.5507262945]], [[0.5488625765]], [[0.1983589530]], [[0.2153149843]], [[0.5687941909]], [[0.6535000205]], [[0.4198964834]], [[0.8353835344]], [[0.1970047355]], [[0.6132903099]], [[1.0867435932]], [[0.4430879354]], [[1.0985355377]], [[0.8281108141]], [[0.1958034635]], [[0.9633826017]], [[0.5893366933]], [[0.4238305092]], [[1.0468605757]], [[0.2429863214]], [[0.7006863952]], [[0.5423495770]], [[1.0040903091]], [[0.7866061926]], [[0.8872866035]], [[0.8236134052]], [[0.5267965794]], [[0.5418812037]], [[0.2349932194]], [[0.9179559946]], [[1.0440230370]], [[0.9891463518]], [[0.2943447828]], [[0.6693338752]], [[0.2945668697]], [[0.3717918396]], [[0.3556811810]], [[0.8179064393]], [[0.3490773439]], [[0.6361253262]], [[0.6680679917]], [[0.9693454504]], [[0.2690739036]], [[0.3037778139]], [[0.3567031622]], [[0.9970006347]], [[0.4673014283]], [[1.0687782764]], [[0.8278377056]], [[0.7103469372]], [[0.6433119178]], [[0.5481931567]], [[0.7097671032]], [[0.5328615904]], [[0.6435207129]], [[0.7108304501]], [[0.2559313774]], [[0.3849682212]], [[1.0507802963]], [[0.2412605882]], [[0.6111308336]], [[1.1416532993]], [[0.8276524544]], [[0.6611904502]], [[0.2137289643]], [[0.8947228789]], [[0.2908935547]], [[0.5051018000]], [[0.4792757630]], [[0.5729902387]], [[0.6525030732]], [[1.0594377518]], [[0.7094533443]], [[1.0948803425]], [[0.9528901577]], [[0.3309269547]], [[0.8712859750]], [[0.2935858965]], [[0.4351213574]], [[0.7940952778]], [[0.8121299744]], [[1.0221478939]], [[0.4860759974]], [[0.6478343606]], [[0.9044457078]], [[0.1634878516]], [[1.0085241795]], [[0.2335727215]], [[0.6539251804]], [[0.5620255470]], [[0.3836974502]], [[0.7131194472]], [[1.0604932308]], [[0.5008741617]], [[0.3501868844]], [[0.4621145129]], [[0.1514597535]], [[0.8727309108]], [[0.4069020748]], [[0.3133637905]], [[0.3589832187]], [[0.9345119596]], [[0.9118207693]], [[1.0307948589]], [[0.8283954859]], [[0.4800540805]], [[0.5073098540]], [[0.7947489023]], [[1.0580513477]], [[0.7829800248]], [[0.4104612470]], [[0.4119952321]], [[0.1742994189]], [[0.7550579309]], [[0.3664426804]], [[0.2012460232]], [[1.0854721069]], [[0.3223283291]], [[0.5901531577]], [[0.7902806997]], [[0.6629630923]], [[0.3105930686]], [[0.2428730726]], [[1.0455751419]], [[0.7284532785]], [[1.0618462563]], [[0.4794136286]], [[0.7943116426]], [[0.5326958299]], [[0.6247983575]], [[0.3425118327]], [[0.8161343932]], [[0.8051168919]], [[0.6367196441]], [[0.5345828533]], [[0.3388190269]], [[0.9928063154]], [[0.2748218775]], [[0.8518666625]], [[0.4789071679]], [[0.4057999253]], [[0.7368564606]], [[0.3873054385]], [[0.7622363567]], [[0.7452243567]], [[0.2757892013]], [[0.7302832007]], [[0.8599985242]], [[0.8449494243]], [[0.5840945840]], [[0.2371401191]], [[0.5699568987]], [[0.8206877708]], [[0.4645953774]], [[0.8368244767]], [[0.9800152183]], [[0.3859581351]], [[0.6519648433]], [[0.8537834287]], [[0.6862228513]], [[0.6887996793]], [[0.7094666362]], [[0.2539600730]], [[0.6863280535]], [[0.9932688475]], [[1.0975908041]], [[0.9409093857]], [[0.7140354514]], [[0.8805410266]], [[0.4037941098]], [[0.2326897383]], [[0.2170733809]], [[1.1458387375]], [[0.9644286036]], [[0.3014170527]], [[0.8426552415]], [[1.0246176720]], [[1.1468412876]], [[1.0842313766]], [[1.0344108343]], [[0.5324043036]], [[0.4715631008]], [[1.0575582981]], [[0.9272237420]], [[0.3461444378]], [[1.0965425968]], [[0.8886132240]], [[0.9196016788]], [[0.3336457610]], [[0.7904838324]], [[0.4717527032]], [[1.0377122164]], [[0.5570636392]], [[0.8416901827]], [[0.7358448505]], [[0.8597680330]], [[0.4771201611]], [[0.8908196092]], [[0.2977984548]], [[0.7599614263]], [[0.3087379932]], [[0.1537651420]], [[0.2455060482]], [[1.0417690277]], [[0.9175517559]], [[1.1161417961]], [[1.0475945473]], [[0.2005112171]], [[0.3058183789]], [[0.5662426353]], [[0.3223177791]], [[0.9942321777]], [[0.2690300345]], [[0.4030832052]], [[0.1639882326]], [[0.3631611466]], [[1.0582726002]], [[1.0564138889]], [[1.0049527884]], [[1.0330975056]], [[1.0916271210]], [[0.5190024376]], [[0.8670349717]], [[1.0924991369]], [[0.8124433160]], [[1.1468720436]], [[0.9063653946]], [[0.9578580856]], [[0.4720378518]], [[0.8869476914]], [[0.7044872046]], [[0.5276252627]], [[0.3651790023]], [[0.3664756417]], [[0.2622935176]], [[0.9826990962]], [[1.0024995804]], [[0.5901275873]], [[0.3576909304]], [[1.0334866047]], [[0.9667783976]], [[0.6842009425]], [[0.4109662771]], [[1.1065578461]], [[0.8515042663]], [[0.2674612403]], [[1.1255753040]], [[1.0267207623]], [[0.4647884965]], [[0.9281111956]], [[0.3629394174]], [[0.5686767697]], [[1.0715844631]], [[0.6676934958]], [[0.2934248447]], [[0.4799174666]], [[0.5113113523]], [[0.5505695343]], [[0.6948842406]], [[1.1094486713]], [[0.6738041043]], [[0.3383139372]], [[0.6726610661]], [[0.8867775202]], [[0.8950540423]], [[0.1900646687]], [[0.5575627089]], [[0.2754616737]], [[0.4336911440]], [[0.8271806240]], [[0.2919688225]], [[0.8328973651]], [[1.0714230537]], [[0.6798281074]], [[0.3137900829]], [[0.4678914547]], [[0.7562159896]], [[0.2658749223]], [[0.8954390883]], [[0.1930991411]], [[0.1663872600]], [[0.1612036228]], [[0.5456022620]], [[0.9832503796]], [[0.1737944484]], [[1.0626275539]], [[0.4470224977]], [[0.7934759855]], [[0.6698353887]], [[0.1961743832]], [[1.0616984367]], [[0.9162556529]], [[1.1440317631]], [[0.8996400237]], [[0.3169994354]], [[1.0643258095]], [[0.6739061475]], [[0.8841421604]], [[0.2461193800]], [[0.5032206774]], [[0.1560951471]], [[0.4522876740]], [[0.7548995018]], [[0.2544530630]], [[0.8064160347]], [[0.9154373407]], [[0.7166885734]], [[0.3124922514]], [[0.2593740821]], [[0.4927756190]], [[0.8665130138]], [[1.1402320862]], [[0.9345484376]], [[0.5907292962]], [[0.8223421574]], [[0.1565024853]], [[0.2199832797]], [[0.8803378940]], [[0.3638263345]], [[0.8875817657]]]]), compiled_model=tensor([[[[0.6432905197]], [[1.5860929489]], [[1.4839851856]], [[1.1095695496]], [[1.6394541264]], [[2.0266568661]], [[1.5623590946]], [[2.5243744850]], [[1.3247685432]], [[1.3422672749]], [[1.8293621540]], [[1.9349204302]], [[0.5706655979]], [[1.1835074425]], [[1.4325603247]], [[1.0546545982]], [[2.4515738487]], [[2.0158317089]], [[1.7694278955]], [[1.8955497742]], [[1.6903976202]], [[1.5831627846]], [[1.8931937218]], [[1.8487652540]], [[1.5473380089]], [[1.5743913651]], [[1.1608679295]], [[1.1797194481]], [[0.9860122800]], [[1.5345501900]], [[1.6283421516]], [[2.5970394611]], [[1.1148750782]], [[1.0623140335]], [[1.7483360767]], [[1.0405085087]], [[1.8282469511]], [[2.5342912674]], [[1.5722303391]], [[1.6632347107]], [[1.8768817186]], [[1.5229551792]], [[2.0142798424]], [[1.0946760178]], [[1.0420372486]], [[1.2041490078]], [[2.2432034016]], [[1.9425747395]], [[1.0248345137]], [[1.1430916786]], [[1.6923389435]], [[1.5908164978]], [[1.4416050911]], [[1.8068041801]], [[2.0217666626]], [[1.5093097687]], [[1.7660236359]], [[1.3428159952]], [[1.0201561451]], [[1.1997160912]], [[1.4562170506]], [[2.2313537598]], [[1.3786284924]], [[1.4650183916]], [[1.5005602837]], [[0.9205338955]], [[1.6909657717]], [[1.5894851685]], [[1.4298171997]], [[1.6881048679]], [[1.4642975330]], [[1.1849291325]], [[1.4845908880]], [[1.3957200050]], [[1.4590294361]], [[2.0612955093]], [[1.2894487381]], [[1.0915758610]], [[0.9363771081]], [[1.0460840464]], [[1.0789448023]], [[2.7420978546]], [[2.7112345695]], [[1.1945953369]], [[1.5281562805]], [[1.7653794289]], [[1.5544381142]], [[1.4244418144]], [[1.6190359592]], [[1.3161529303]], [[0.8469919562]], [[1.1085669994]], [[1.3286389112]], [[1.5624438524]], [[0.0902764201]], [[0.2314071655]], [[1.8668450117]], [[2.3097288609]], [[1.1695282459]], [[1.9689261913]], [[1.7125620842]], [[1.3063913584]], [[1.3269093037]], [[1.3009154797]], [[1.8186228275]], [[1.8591769934]], [[1.1381189823]], [[1.0493803024]], [[1.4690322876]], [[1.8055412769]], [[2.0784380436]], [[2.1179778576]], [[0.9960234165]], [[2.0122787952]], [[1.5023179054]], [[0.6677139997]], [[1.1236047745]], [[2.2159206867]], [[1.7663099766]], [[1.5214191675]], [[1.3563704491]], [[1.3365907669]], [[1.8058402538]], [[1.4934303761]], [[1.1522114277]], [[1.3686857224]], [[1.0935850143]], [[0.6575663686]], [[1.2035012245]], [[1.2425329685]], [[0.9912986755]], [[1.3100032806]], [[1.1249506474]], [[1.5807856321]], [[1.5166168213]], [[1.0965490341]], [[2.1506147385]], [[1.6580374241]], [[1.7662297487]], [[1.9897947311]], [[2.1240136623]], [[2.0421373844]], [[1.6841794252]], [[1.1410852671]], [[0.7584071755]], [[0.8722981811]], [[1.7760847807]], [[1.1541067362]], [[0.9490612149]], [[0.8512895107]], [[0.6369504333]], [[0.9708097577]], [[0.8400620222]], [[0.8447772861]], [[1.2869737148]], [[0.7349209785]], [[1.4250295162]], [[0.1661503315]], [[0.5071737766]], [[0.5157698989]], [[0.7135948539]], [[0.7856910229]], [[1.0506021976]], [[0.7729890943]], [[0.7200428843]], [[1.0869300365]], [[0.6944676638]], [[1.4320430756]], [[0.4332394600]], [[0.8272414207]], [[1.7381489277]], [[0.3761420250]], [[1.4599676132]], [[1.0507605076]], [[1.0372548103]], [[1.7043678761]], [[0.6955987215]], [[0.6430185437]], [[1.2860949039]], [[0.1072397828]], [[1.4495565891]], [[1.0913509130]], [[1.5293928385]], [[1.3190903664]], [[0.8330075741]], [[1.0656106472]], [[1.3584403992]], [[0.5599542856]], [[0.2324775457]], [[1.2183134556]], [[1.7585444450]], [[1.1703993082]], [[0.9701071978]], [[0.9641240835]], [[0.6735355258]], [[0.8158783913]], [[0.2118984461]], [[1.0762689114]], [[0.2441940308]], [[0.8807785511]], [[0.6234783530]], [[1.5102820396]], [[0.6983748674]], [[1.1073065996]], [[0.9458004832]], [[1.2852017879]], [[1.0783727169]], [[1.7398881912]], [[1.2536394596]], [[1.2976171970]], [[1.3143935204]], [[1.2756410837]], [[1.3734918833]], [[1.0153359175]], [[1.1239473820]], [[0.6585488915]], [[0.6740365624]], [[0.3822367191]], [[1.3755034208]], [[0.6563866138]], [[1.1509908438]], [[1.7988681793]], [[1.5780091286]], [[1.3942213058]], [[0.9887727499]], [[1.0772136450]], [[0.9736913443]], [[1.0532585382]], [[0.5128701925]], [[0.6240890026]], [[0.7025446296]], [[1.4692983627]], [[1.0231510401]], [[1.3390438557]], [[1.1348956823]], [[0.3971082568]], [[1.6884371042]], [[0.1620073318]], [[0.8208938241]], [[1.5072610378]], [[0.7119740844]], [[0.8861027360]], [[0.9012952447]], [[1.0488648415]], [[1.6656954288]], [[0.7355698943]], [[1.0088002682]], [[0.3571452498]], [[0.5910126567]], [[0.4874385595]], [[0.4701657295]], [[1.5092194080]], [[1.3240481615]], [[0.4311599731]], [[0.8999097943]], [[1.1467518806]], [[0.1392654777]], [[1.5595889091]], [[0.5665434599]], [[0.7062265873]], [[0.4960951805]], [[0.8419075012]], [[0.8744081855]], [[1.4949700832]], [[1.0758250952]], [[0.6459352374]], [[0.8322960734]], [[0.7796265483]], [[1.4818115234]], [[1.3749754429]], [[0.8315638900]], [[1.2457473278]], [[0.5702579021]], [[1.0161709785]], [[0.3893322349]], [[0.1322010756]], [[1.3040784597]], [[0.3744184971]], [[1.4107502699]], [[0.7911782861]], [[1.1093621254]], [[1.1368401051]], [[0.1245256662]], [[1.5452265739]], [[1.4389731884]], [[1.3646998405]], [[0.7592597008]], [[1.0266125202]], [[1.3701941967]], [[0.6172183752]], [[1.0986692905]], [[1.6448457241]], [[0.8362820745]], [[1.1224486828]], [[0.8640732169]], [[0.3410556316]], [[1.1856603622]], [[0.7435573936]], [[0.9468892813]], [[1.0356345177]], [[0.5026663542]], [[0.6182712913]], [[1.2342550755]], [[1.1052479744]], [[0.6321507096]], [[0.1892203689]], [[1.3278805017]], [[1.3074564934]], [[0.7250729203]], [[1.0390214920]], [[0.4370844364]], [[1.0160447359]], [[1.1123795509]], [[0.4167149067]], [[0.8946704865]], [[1.5376076698]], [[0.3898609877]], [[0.9615180492]], [[0.8347385526]], [[0.5713572502]], [[1.0407650471]], [[0.6021842360]], [[0.5894266963]], [[1.1019103527]], [[1.3491635323]], [[1.7385807037]], [[1.3502504826]], [[1.2279213667]], [[1.7156231403]], [[0.5907594562]], [[0.9260461926]], [[0.4085202813]], [[1.8786287308]], [[1.4627962112]], [[1.1028025150]], [[1.1330316067]], [[1.0316700935]], [[1.6049160957]], [[1.0271592140]], [[1.7390085459]], [[1.2954764366]], [[0.3823590279]], [[1.6627001762]], [[1.3369359970]], [[1.1441019773]], [[1.6484005451]], [[0.8784171939]], [[1.5073474646]], [[0.3282203674]], [[1.1474816799]], [[0.6562858224]], [[1.1535165310]], [[0.8223814964]], [[0.9577039480]], [[0.8755698204]], [[1.4556984901]], [[0.6794393659]], [[1.5139436722]], [[0.8331122994]], [[1.4824255705]], [[0.3680773973]], [[0.1129165292]], [[0.4119288921]], [[1.8584219217]], [[1.4071812630]], [[1.2876267433]], [[1.7812303305]], [[0.3061612844]], [[1.0781075954]], [[0.9523036480]], [[1.1408522129]], [[1.4197332859]], [[0.9201971292]], [[0.5385010242]], [[0.1713696122]], [[0.3693567514]], [[1.2868671417]], [[1.4342957735]], [[1.2082762718]], [[1.7020332813]], [[1.5586743355]], [[1.2357254028]], [[1.6423799992]], [[1.5353609324]], [[1.3682303429]], [[1.1724286079]], [[1.6445872784]], [[1.6160199642]], [[0.3670911789]], [[1.2709438801]], [[1.3650162220]], [[0.5141481757]], [[0.6773358583]], [[0.9235632420]], [[0.7739562988]], [[0.9538368583]], [[1.0993044376]], [[1.0128610134]], [[1.1830574274]], [[1.1936874390]], [[1.7439882755]], [[1.4445157051]], [[0.2704749703]], [[1.6646010876]], [[1.5720013380]], [[1.0796248913]], [[1.5574651957]], [[1.6437653303]], [[0.3694860339]], [[1.3546001911]], [[0.6652881503]], [[0.8953785896]], [[1.4195116758]], [[1.1981751919]], [[0.1678333282]], [[1.2554483414]], [[0.6564683318]], [[0.4239755273]], [[0.6409259439]], [[1.5412329435]], [[0.8628174067]], [[0.7406799793]], [[0.6612382531]], [[0.9435313344]], [[0.7591368556]], [[0.2681972980]], [[1.1831872463]], [[0.1627495885]], [[0.7053192258]], [[1.5131113529]], [[0.9360464215]], [[0.9490385056]], [[1.4581763744]], [[1.1453273296]], [[0.8286214471]], [[0.5965632796]], [[1.3307435513]], [[0.7596735954]], [[1.4636592865]], [[0.2137073874]], [[0.2024887800]], [[0.1788253784]], [[1.2611703873]], [[1.3293912411]], [[0.2315820456]], [[1.5807569027]], [[0.4851566553]], [[1.3578138351]], [[1.1398043633]], [[0.3204129934]], [[1.6336282492]], [[1.3205339909]], [[1.7525398731]], [[0.8867415190]], [[0.7172477841]], [[1.7041357756]], [[1.0881819725]], [[1.3323907852]], [[0.9500837922]], [[1.2986679077]], [[0.6984318495]], [[1.2528644800]], [[0.8309126496]], [[0.1922032833]], [[0.8135689497]], [[1.3641623259]], [[1.5342931747]], [[0.7031964660]], [[1.0833247900]], [[0.7237421870]], [[1.4870373011]], [[1.1216611862]], [[0.9390876293]], [[0.9042983651]], [[1.5470068455]], [[0.0222479701]], [[0.3076067567]], [[1.2334625721]], [[0.8989537954]], [[0.7425754666]]]])
test_add.test_module[Add1-[((1, 672, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.9084333777]], [[1.1803985834]], [[0.5006542206]], [[0.5442072749]], [[0.7195996046]], [[1.0462554693]], [[0.9022701979]], [[1.3086215258]], [[0.8678047657]], [[1.0444830656]], [[0.7610702515]], [[0.8138940930]], [[0.4345025420]], [[0.5810357332]], [[0.7060652375]], [[0.9306985736]], [[1.1098444462]], [[1.2121882439]], [[0.5732062459]], [[0.6944453716]], [[1.0937852859]], [[1.3273707628]], [[0.8092767000]], [[1.2863326073]], [[0.8315851092]], [[0.9650838375]], [[1.3649148941]], [[0.4483416080]], [[0.5974078178]], [[0.7855941653]], [[0.7172768116]], [[1.3441772461]], [[0.5880869627]], [[0.6820103526]], [[0.5628565550]], [[0.4438962936]], [[0.6203065515]], [[1.3419758081]], [[1.1352859735]], [[1.1545131207]], [[0.9384725690]], [[0.6558350325]], [[0.9967691302]], [[0.4453294277]], [[0.5508936644]], [[0.6544117928]], [[1.2276457548]], [[1.2053374052]], [[0.6904292703]], [[0.8941355944]], [[1.2319571972]], [[1.4092433453]], [[1.1106178761]], [[0.9797232151]], [[1.2474199533]], [[0.6177756190]], [[1.0053488016]], [[0.5245240331]], [[0.5656337142]], [[0.6538850069]], [[1.1384133101]], [[1.1132569313]], [[0.6160005331]], [[1.0632302761]], [[1.1866627932]], [[0.8490681052]], [[0.9312675595]], [[1.0280292034]], [[1.2223651409]], [[1.3922739029]], [[0.5268650055]], [[0.7289419174]], [[1.1086816788]], [[1.3264515400]], [[1.3472805023]], [[1.3533551693]], [[1.0116840601]], [[0.4773854613]], [[0.9581730366]], [[0.5993741155]], [[0.4461997151]], [[1.3564230204]], [[1.2923567295]], [[0.4134128094]], [[1.0057628155]], [[0.8279467821]], [[0.8298962116]], [[0.6832983494]], [[1.1044548750]], [[0.6160250306]], [[1.0954724550]], [[1.1650308371]], [[1.2701125145]], [[1.0991322994]], [[0.4173091650]], [[0.5878283381]], [[1.1618342400]], [[1.0168274641]], [[0.5221347809]], [[0.6242670417]], [[1.3825514317]], [[1.2490856647]], [[0.6941642165]], [[0.7863343954]], [[0.4358777404]], [[0.9031897187]], [[0.5356473327]], [[0.5264984369]], [[0.8846269846]], [[0.9872493148]], [[0.7074116468]], [[1.2088656425]], [[0.6079072356]], [[1.3658618927]], [[1.2548267841]], [[0.4905353189]], [[0.7877346277]], [[0.9347380996]], [[0.9851273298]], [[1.0307638645]], [[1.1083909273]], [[0.9421268702]], [[0.6682124138]], [[1.1487712860]], [[0.4325523376]], [[0.6158234477]], [[0.7870118618]], [[0.6686201096]], [[0.7372601032]], [[0.5023659468]], [[0.8058192134]], [[1.0190550089]], [[0.5864439011]], [[0.8865171075]], [[1.2701022625]], [[0.8607766628]], [[0.9260728955]], [[0.8690423369]], [[1.0133674145]], [[1.2300965786]], [[1.3857998848]], [[1.2297047377]], [[1.3868834972]], [[0.8760159612]], [[0.4630160332]], [[0.6751381755]], [[1.2526293993]], [[0.9089355469]], [[0.6636536121]], [[0.5290209055]], [[0.4442507625]], [[0.4901726842]], [[0.8107584119]], [[1.1863797903]], [[1.1824972630]], [[0.4299608469]], [[1.2240678072]], [[0.5209220648]], [[0.8064716458]], [[0.7094404697]], [[0.8158691525]], [[0.8140054345]], [[0.4635018110]], [[0.4804578424]], [[0.8339370489]], [[0.9186428785]], [[0.6850393414]], [[1.1005263329]], [[0.4621475935]], [[0.8784331679]], [[1.3518865108]], [[0.7082307935]], [[1.3636783361]], [[1.0932536125]], [[0.4609463215]], [[1.2285254002]], [[0.8544795513]], [[0.6889733672]], [[1.3120033741]], [[0.5081291795]], [[0.9658292532]], [[0.8074924350]], [[1.2692331076]], [[1.0517489910]], [[1.1524294615]], [[1.0887563229]], [[0.7919394374]], [[0.8070240617]], [[0.5001360774]], [[1.1830987930]], [[1.3091658354]], [[1.2542891502]], [[0.5594876409]], [[0.9344767332]], [[0.5597097278]], [[0.6369346976]], [[0.6208240390]], [[1.0830492973]], [[0.6142202020]], [[0.9012681842]], [[0.9332108498]], [[1.2344882488]], [[0.5342167616]], [[0.5689206719]], [[0.6218460202]], [[1.2621434927]], [[0.7324442863]], [[1.3339211941]], [[1.0929806232]], [[0.9754897952]], [[0.9084547758]], [[0.8133360147]], [[0.9749099612]], [[0.7980044484]], [[0.9086635709]], [[0.9759733081]], [[0.5210742354]], [[0.6501110792]], [[1.3159232140]], [[0.5064034462]], [[0.8762736917]], [[1.4067962170]], [[1.0927953720]], [[0.9263333082]], [[0.4788718224]], [[1.1598657370]], [[0.5560364127]], [[0.7702446580]], [[0.7444186211]], [[0.8381330967]], [[0.9176459312]], [[1.3245805502]], [[0.9745962024]], [[1.3600232601]], [[1.2180330753]], [[0.5960698128]], [[1.1364288330]], [[0.5587287545]], [[0.7002642155]], [[1.0592381954]], [[1.0772728920]], [[1.2872908115]], [[0.7512188554]], [[0.9129772186]], [[1.1695885658]], [[0.4286307096]], [[1.2736670971]], [[0.4987155795]], [[0.9190680385]], [[0.8271684051]], [[0.6488403082]], [[0.9782623053]], [[1.3256361485]], [[0.7660170197]], [[0.6153297424]], [[0.7272573709]], [[0.4166026115]], [[1.1378737688]], [[0.6720449328]], [[0.5785066485]], [[0.6241260767]], [[1.1996548176]], [[1.1769635677]], [[1.2959377766]], [[1.0935382843]], [[0.7451969385]], [[0.7724527121]], [[1.0598917007]], [[1.3231941462]], [[1.0481228828]], [[0.6756041050]], [[0.6771380901]], [[0.4394422770]], [[1.0202007294]], [[0.6315855384]], [[0.4663888812]], [[1.3506150246]], [[0.5874711871]], [[0.8552960157]], [[1.0554234982]], [[0.9281059504]], [[0.5757359266]], [[0.5080159307]], [[1.3107180595]], [[0.9935961366]], [[1.3269890547]], [[0.7445564866]], [[1.0594544411]], [[0.7978386879]], [[0.8899412155]], [[0.6076546907]], [[1.0812772512]], [[1.0702598095]], [[0.9018625021]], [[0.7997257113]], [[0.6039618850]], [[1.2579491138]], [[0.5399647355]], [[1.1170095205]], [[0.7440500259]], [[0.6709427834]], [[1.0019993782]], [[0.6524482965]], [[1.0273792744]], [[1.0103671551]], [[0.5409320593]], [[0.9954260588]], [[1.1251413822]], [[1.1100922823]], [[0.8492374420]], [[0.5022829771]], [[0.8350997567]], [[1.0858306885]], [[0.7297382355]], [[1.1019673347]], [[1.2451580763]], [[0.6511009932]], [[0.9171077013]], [[1.1189262867]], [[0.9513657093]], [[0.9539425373]], [[0.9746094942]], [[0.5191029310]], [[0.9514709115]], [[1.2584116459]], [[1.3627336025]], [[1.2060523033]], [[0.9791783094]], [[1.1456838846]], [[0.6689369678]], [[0.4978325963]], [[0.4822162390]], [[1.4109816551]], [[1.2295714617]], [[0.5665599108]], [[1.1077980995]], [[1.2897605896]], [[1.4119842052]], [[1.3493742943]], [[1.2995536327]], [[0.7975471616]], [[0.7367059588]], [[1.3227012157]], [[1.1923666000]], [[0.6112872958]], [[1.3616853952]], [[1.1537561417]], [[1.1847445965]], [[0.5987886190]], [[1.0556266308]], [[0.7368955612]], [[1.3028550148]], [[0.8222064972]], [[1.1068329811]], [[1.0009877682]], [[1.1249108315]], [[0.7422630191]], [[1.1559624672]], [[0.5629413128]], [[1.0251042843]], [[0.5738808513]], [[0.4189080000]], [[0.5106489062]], [[1.3069118261]], [[1.1826946735]], [[1.3812847137]], [[1.3127374649]], [[0.4656540751]], [[0.5709612370]], [[0.8313854933]], [[0.5874606371]], [[1.2593750954]], [[0.5341728926]], [[0.6682260633]], [[0.4291310906]], [[0.6283040047]], [[1.3234155178]], [[1.3215568066]], [[1.2700955868]], [[1.2982404232]], [[1.3567700386]], [[0.7841452956]], [[1.1321778297]], [[1.3576419353]], [[1.0775861740]], [[1.4120149612]], [[1.1715083122]], [[1.2230010033]], [[0.7371807098]], [[1.1520905495]], [[0.9696300626]], [[0.7927681208]], [[0.6303218603]], [[0.6316184998]], [[0.5274363756]], [[1.2478419542]], [[1.2676423788]], [[0.8552704453]], [[0.6228337884]], [[1.2986295223]], [[1.2319211960]], [[0.9493438005]], [[0.6761091352]], [[1.3717007637]], [[1.1166471243]], [[0.5326040983]], [[1.3907182217]], [[1.2918636799]], [[0.7299313545]], [[1.1932539940]], [[0.6280822754]], [[0.8338196278]], [[1.3367273808]], [[0.9328363538]], [[0.5585677028]], [[0.7450603247]], [[0.7764542103]], [[0.8157123923]], [[0.9600270987]], [[1.3745915890]], [[0.9389469624]], [[0.6034567952]], [[0.9378039241]], [[1.1519203186]], [[1.1601969004]], [[0.4552075267]], [[0.8227055669]], [[0.5406045318]], [[0.6988340020]], [[1.0923235416]], [[0.5571116805]], [[1.0980402231]], [[1.3365658522]], [[0.9449709654]], [[0.5789329410]], [[0.7330343127]], [[1.0213588476]], [[0.5310177803]], [[1.1605819464]], [[0.4582419991]], [[0.4315301180]], [[0.4263464808]], [[0.8107451200]], [[1.2483932972]], [[0.4389373064]], [[1.3277703524]], [[0.7121653557]], [[1.0586187840]], [[0.9349782467]], [[0.4613172412]], [[1.3268413544]], [[1.1813985109]], [[1.4091746807]], [[1.1647828817]], [[0.5821422935]], [[1.3294687271]], [[0.9390490055]], [[1.1492850780]], [[0.5112622380]], [[0.7683635354]], [[0.4212380052]], [[0.7174305320]], [[1.0200424194]], [[0.5195959210]], [[1.0715589523]], [[1.1805801392]], [[0.9818314314]], [[0.5776351094]], [[0.5245169401]], [[0.7579184771]], [[1.1316559315]], [[1.4053750038]], [[1.1996912956]], [[0.8558721542]], [[1.0874850750]], [[0.4216453433]], [[0.4851261377]], [[1.1454807520]], [[0.6289691925]], [[1.1527246237]], [[0.5592107177]], [[0.6645213366]], [[0.5003322959]], [[1.1730971336]], [[0.8612292409]], [[1.2969772816]], [[1.2216129303]], [[1.1888480186]], [[0.9282548428]], [[0.7575877905]], [[0.8034609556]], [[0.9786270857]], [[1.1600315571]], [[0.5618818402]], [[1.3318157196]], [[0.8578117490]], [[0.4932039976]], [[0.6416478753]], [[1.3545856476]], [[1.3694403172]], [[0.4490369558]], [[1.2648259401]], [[1.1627564430]], [[1.2081360817]], [[1.3354413509]], [[0.6427016258]], [[1.0700646639]], [[1.1167933941]], [[0.7644301057]], [[1.0795032978]], [[0.7683200836]], [[1.2213075161]], [[0.7734494805]], [[0.7257789373]], [[1.0380513668]], [[1.0895235538]], [[0.6678909659]], [[0.9563759565]], [[1.2019445896]], [[0.8624284267]], [[1.0643436909]], [[0.7915832996]], [[1.0874266624]], [[0.5499796867]], [[0.6181623936]], [[0.6583815217]], [[1.3716821671]], [[0.7776347399]], [[0.9108116031]], [[0.6699290276]], [[1.4113218784]], [[1.4005118608]], [[0.5350834727]], [[0.5068418384]], [[0.5331802964]], [[0.9097654223]], [[0.7847248316]], [[0.5849039555]], [[0.7328407168]], [[1.0066422224]], [[0.6509314179]], [[1.0229680538]], [[0.7975188494]], [[0.6698951125]], [[0.9808695316]], [[1.3233058453]], [[0.5741371512]], [[0.9353939891]], [[0.7277929187]], [[1.4028298855]], [[0.4377951622]], [[0.4328461885]], [[1.4048693180]], [[0.5958318114]], [[1.0080385208]], [[0.8690237403]], [[0.8068482876]], [[0.8005028963]], [[1.2298970222]], [[0.9360752702]], [[0.4253695011]], [[0.6169967055]], [[0.7417266369]], [[1.1637849808]], [[0.5886099339]], [[1.3836365938]], [[0.8008121252]], [[0.8223958015]], [[1.3039557934]], [[1.1634806395]], [[1.3362369537]], [[1.2013989687]], [[0.7604998350]], [[0.5804405808]], [[0.8749752641]], [[1.3259955645]], [[0.7443668842]], [[0.4484918118]], [[1.1171410084]], [[1.3989124298]], [[0.7698351741]], [[0.4981650114]], [[0.4586469531]], [[1.0374765396]], [[0.8743169308]], [[0.6596831679]], [[1.0132427216]], [[1.1020562649]], [[1.3098362684]], [[1.3003678322]], [[0.8373349309]], [[0.4712932110]], [[0.4603652358]], [[1.3790216446]], [[1.1332044601]], [[1.1301305294]], [[0.4795662761]], [[1.3751846552]], [[1.3858482838]], [[1.3636069298]], [[0.4903842807]], [[0.7235150933]], [[0.5682948828]], [[1.3856595755]], [[0.6973367929]], [[0.6838995218]], [[1.1741333008]], [[0.6808806062]], [[0.6659157276]], [[0.8684410453]], [[0.8641170263]], [[0.5226875544]], [[1.3290108442]], [[0.6916159391]], [[1.0895342827]], [[1.3470685482]], [[1.1643350124]], [[0.9829667211]], [[1.3376052380]], [[0.9793899059]], [[0.6808484793]], [[1.3851765394]], [[1.0305171013]], [[0.4243363738]], [[0.7698310018]], [[0.5715930462]], [[1.3505532742]], [[0.8296038508]], [[0.4564143419]], [[0.8807451129]], [[1.2261831760]], [[1.0420770645]], [[1.0702790022]], [[0.9586133957]], [[1.0985860825]], [[0.7903507948]], [[0.7132766843]], [[0.4448225498]], [[0.5355083346]], [[1.1288781166]], [[0.6161201000]], [[0.9839100242]], [[1.0717725754]], [[0.9475787282]], [[0.5880019069]], [[1.3903410435]], [[0.6214270592]], [[1.3234270811]], [[0.5144226551]], [[0.7919025421]], [[1.1841611862]], [[0.7078864574]], [[1.3321982622]], [[0.5680985451]], [[0.4922910333]], [[0.6867172122]], [[0.9930019975]], [[1.3725547791]], [[0.6734763384]], [[1.0909909010]], [[0.7868232727]], [[0.8037456274]], [[1.2798285484]], [[0.5246838331]], [[0.9652834535]], [[1.3823814392]], [[0.8434706926]], [[1.3003958464]], [[0.7581983805]], [[1.3146579266]], [[0.4285116196]], [[0.8401149511]]]]), compiled_model=tensor([[[[0.9084333777]], [[1.2100459337]], [[0.6144800186]], [[0.7231509686]], [[0.3106740117]], [[1.0394752026]], [[0.5322440267]], [[1.2881319523]], [[0.5580722690]], [[1.3202767372]], [[0.9252283573]], [[1.3522800207]], [[0.7584570050]], [[0.6040940285]], [[1.0519936085]], [[1.3366656303]], [[1.2705032825]], [[1.5343155861]], [[0.9791449904]], [[1.1567504406]], [[1.4923672676]], [[1.5447022915]], [[1.0245604515]], [[0.9689082503]], [[0.9845474362]], [[0.6972094774]], [[1.4244952202]], [[0.5983247757]], [[0.8721249700]], [[1.1776661873]], [[1.2024906874]], [[1.8120651245]], [[1.0979878902]], [[0.5993582606]], [[0.9805114865]], [[0.7269101739]], [[0.3887581229]], [[1.1279317141]], [[0.9201846719]], [[1.2992308140]], [[0.9870274067]], [[0.6348556876]], [[0.9136317968]], [[0.2463678718]], [[1.1029019356]], [[0.2576903701]], [[1.3482754230]], [[1.6533603668]], [[0.3251305223]], [[0.4929475188]], [[1.3820335865]], [[1.5451309681]], [[1.6067247391]], [[1.2866623402]], [[0.9825531244]], [[0.4762052894]], [[0.6772934198]], [[0.1847941875]], [[0.3869591355]], [[1.1848421097]], [[1.1368253231]], [[0.7783999443]], [[0.9005805850]], [[1.4827249050]], [[0.9093256593]], [[1.2707833052]], [[0.8257660866]], [[1.1557490826]], [[1.0943341255]], [[1.0345265865]], [[0.2243095636]], [[0.9279742837]], [[1.0909684896]], [[1.2271897793]], [[1.4071238041]], [[1.0730899572]], [[1.1703014374]], [[0.8042379618]], [[1.1141328812]], [[1.1679832935]], [[0.5770153403]], [[1.3523931503]], [[1.0501034260]], [[0.0792250037]], [[0.9592262506]], [[0.6148940921]], [[1.3853504658]], [[0.4190530777]], [[1.2857110500]], [[1.1771291494]], [[0.7119821906]], [[1.3995393515]], [[1.7154896259]], [[1.1368432045]], [[0.4320123792]], [[0.5549864173]], [[1.7341898680]], [[0.7441046238]], [[1.0131494999]], [[1.1878354549]], [[1.1485737562]], [[1.4696718454]], [[0.7585117221]], [[0.5234281421]], [[0.3635888696]], [[1.1067824364]], [[0.3655270934]], [[0.8180829883]], [[0.7163505554]], [[0.6035212874]], [[1.2892184258]], [[1.2867343426]], [[0.2296907306]], [[1.0141501427]], [[1.5872812271]], [[0.6728504300]], [[0.4027152658]], [[1.1245222092]], [[0.9199287891]], [[1.2117089033]], [[1.1349399090]], [[0.6291035414]], [[0.4609155655]], [[1.4412208796]], [[0.1713123322]], [[0.6602337956]], [[0.5028241277]], [[0.2886116505]], [[0.8240826130]], [[0.1299406886]], [[0.8761429787]], [[1.1694943905]], [[0.6771956682]], [[1.2623641491]], [[1.4143004417]], [[1.1095197201]], [[1.4960122108]], [[0.7908648252]], [[1.4415810108]], [[1.1564005613]], [[1.8534469604]], [[1.4629294872]], [[1.9231262207]], [[0.9012494683]], [[0.2049254775]], [[0.8680700660]], [[0.9304143786]], [[1.3483903408]], [[1.1615829468]], [[0.1746739745]], [[0.7842497826]], [[0.6347420216]], [[1.3435730934]], [[1.4730949402]], [[0.9071584344]], [[0.7525637746]], [[0.9534995556]], [[0.6127769947]], [[0.7258619070]], [[0.5601019859]], [[0.8160441518]], [[0.6648763418]], [[0.3380839229]], [[0.8112453818]], [[0.7711133957]], [[1.2766239643]], [[0.9552103281]], [[1.5578477383]], [[0.2563441396]], [[0.5724416971]], [[1.2531664371]], [[1.2597408295]], [[1.5881649256]], [[0.9995957613]], [[0.9294392467]], [[1.0690326691]], [[1.3616259098]], [[0.8098915219]], [[1.8653950691]], [[0.6684874296]], [[1.3518534899]], [[0.6777673960]], [[1.0114716291]], [[0.7928017974]], [[1.1158812046]], [[1.2014951706]], [[0.7301201224]], [[1.2108169794]], [[0.7020403147]], [[1.6346790791]], [[1.8193680048]], [[1.4320081472]], [[0.8501318693]], [[0.6948903203]], [[1.0327887535]], [[1.0299537182]], [[0.2507345080]], [[1.2019026279]], [[1.0096063614]], [[0.6226482391]], [[0.9802248478]], [[1.5264329910]], [[0.7807366848]], [[0.2749155760]], [[0.4535081387]], [[1.4197341204]], [[1.2926678658]], [[1.2289791107]], [[1.6050475836]], [[1.4706616402]], [[0.5028206110]], [[1.1062363386]], [[1.4302642345]], [[1.3450253010]], [[1.0754106045]], [[1.3278748989]], [[0.1606289148]], [[0.8114572763]], [[1.3531290293]], [[0.5679624081]], [[0.9590579867]], [[1.6721351147]], [[0.7020609379]], [[1.4367213249]], [[0.3588859439]], [[0.7681288719]], [[0.2369352579]], [[0.9368860126]], [[0.6682890654]], [[0.9753562808]], [[0.6410802603]], [[1.1161915064]], [[0.5735361576]], [[1.1730129719]], [[1.5785148144]], [[0.2182148695]], [[1.1429140568]], [[0.9795166254]], [[1.0791989565]], [[0.9102364182]], [[1.1988832951]], [[1.4876472950]], [[1.0009074211]], [[0.7765061855]], [[1.4789731503]], [[0.6572865248]], [[1.5767443180]], [[0.2541809678]], [[0.6900267005]], [[0.5796473026]], [[1.0992655754]], [[1.0592603683]], [[1.1182808876]], [[1.0190036297]], [[0.3883210421]], [[1.0264523029]], [[0.6214287281]], [[0.9969695210]], [[0.9788318872]], [[0.7176420689]], [[0.9674912691]], [[0.9216134548]], [[1.3120691776]], [[1.6706048250]], [[1.2426713705]], [[0.9283027053]], [[1.2112742662]], [[1.5901961327]], [[1.6003880501]], [[1.5835568905]], [[0.4864743948]], [[0.3497454524]], [[0.1814523339]], [[1.2037829161]], [[1.1840472221]], [[0.5919502378]], [[1.9094228745]], [[0.5532948971]], [[1.2106773853]], [[0.7717097402]], [[0.6675022840]], [[0.6241621375]], [[0.9675378203]], [[0.9113206267]], [[0.8160767555]], [[1.4149708748]], [[1.0145411491]], [[0.6493053436]], [[0.5020034909]], [[1.0083611012]], [[0.6005051732]], [[1.3698458672]], [[1.0936900377]], [[1.2760031223]], [[0.4453179836]], [[0.5152152181]], [[1.4365240335]], [[0.4378890395]], [[0.7885467410]], [[0.9955444932]], [[0.9020565152]], [[2.1759154797]], [[1.7242567539]], [[1.7247719765]], [[2.2376446724]], [[2.1554121971]], [[2.1456084251]], [[3.2373390198]], [[2.0226840973]], [[1.7793278694]], [[1.9194684029]], [[2.3578433990]], [[1.2443194389]], [[1.5010688305]], [[2.1223509312]], [[1.8876359463]], [[2.6904981136]], [[2.5207626820]], [[2.4761774540]], [[2.4347386360]], [[2.2321634293]], [[2.1455955505]], [[2.0001199245]], [[2.3880593777]], [[2.3935728073]], [[2.5249481201]], [[1.9547433853]], [[1.7467210293]], [[1.7195193768]], [[1.7913103104]], [[1.7139979601]], [[2.6670789719]], [[2.1136798859]], [[1.8797087669]], [[1.9027192593]], [[1.7361297607]], [[2.7058308125]], [[3.5340986252]], [[2.5094277859]], [[2.5506114960]], [[2.2622520924]], [[1.8474843502]], [[2.9248042107]], [[1.8748657703]], [[1.2411477566]], [[2.1536576748]], [[2.9847826958]], [[2.7151424885]], [[1.2114462852]], [[1.7865415812]], [[2.0170576572]], [[2.4814949036]], [[1.8516347408]], [[2.5014605522]], [[2.6105775833]], [[2.2220439911]], [[2.0961098671]], [[2.0866017342]], [[1.1709206104]], [[1.8126435280]], [[1.6179211140]], [[2.2380850315]], [[1.4771006107]], [[2.3597533703]], [[2.2710781097]], [[1.8896417618]], [[2.5915265083]], [[1.6429624557]], [[1.5886015892]], [[2.1073136330]], [[1.6395814419]], [[2.0321273804]], [[1.6065869331]], [[1.6517692804]], [[1.4759837389]], [[2.2774226665]], [[2.2006874084]], [[2.0009558201]], [[1.7942960262]], [[1.9321477413]], [[2.0235381126]], [[3.1140663624]], [[3.4312355518]], [[2.1400604248]], [[2.1935656071]], [[2.7652175426]], [[2.3137695789]], [[2.2352659702]], [[1.9440398216]], [[2.0560667515]], [[1.4044451714]], [[1.4891583920]], [[1.5467839241]], [[1.7818856239]], [[0.2055360079]], [[1.0670723915]], [[2.7223105431]], [[2.7528223991]], [[1.3801852465]], [[2.8553788662]], [[2.5323066711]], [[1.8435583115]], [[1.5908416510]], [[2.2604393959]], [[2.5230932236]], [[1.9796042442]], [[2.1166603565]], [[1.9290671349]], [[1.7867867947]], [[2.5866184235]], [[2.2943434715]], [[2.5396206379]], [[1.9205739498]], [[2.5329384804]], [[1.6487088203]], [[1.0005974770]], [[1.4878821373]], [[2.6194562912]], [[2.3141603470]], [[2.4838340282]], [[1.8831405640]], [[1.5278707743]], [[2.3314673901]], [[2.2331738472]], [[1.9002315998]], [[1.4117164612]], [[1.5041137934]], [[0.7859941125]], [[1.4901584387]], [[1.9226796627]], [[1.1362335682]], [[1.9958667755]], [[2.0493397713]], [[2.1135797501]], [[1.6833729744]], [[1.4174065590]], [[2.7597968578]], [[1.7768783569]], [[2.5146348476]], [[2.0358600616]], [[2.1433670521]], [[2.0563070774]], [[2.0827476978]], [[1.9773017168]], [[0.7851676941]], [[1.7878917456]], [[2.0760734081]], [[1.8005487919]], [[1.4718626738]], [[0.9004299641]], [[1.5516149998]], [[1.7400314808]], [[1.8370598555]], [[1.5973833799]], [[1.4569392204]], [[1.6522128582]], [[1.9519016743]], [[0.9032585621]], [[0.6062592268]], [[0.8719566464]], [[0.7226560712]], [[1.0909447670]], [[1.6584677696]], [[0.8804082274]], [[1.3794250488]], [[1.8553334475]], [[1.2641222477]], [[1.5975013971]], [[0.5455796123]], [[1.1729831696]], [[2.4576280117]], [[1.3693401814]], [[2.2474820614]], [[1.4944558144]], [[1.7125630379]], [[1.7138364315]], [[0.7685480714]], [[1.3763225079]], [[1.5028872490]], [[0.8477876186]], [[1.5965905190]], [[1.3436954021]], [[1.6175483465]], [[2.0800106525]], [[1.2820600271]], [[1.9504112005]], [[2.1678764820]], [[1.3366255760]], [[0.7485556006]], [[1.5637245178]], [[2.1498286724]], [[1.7368495464]], [[1.7179620266]], [[1.1138291359]], [[1.5931744576]], [[1.2615133524]], [[0.2929256558]], [[1.3057399988]], [[1.1866028309]], [[1.8380421400]], [[0.6603385210]], [[2.3629312515]], [[1.4489544630]], [[1.9032658339]], [[1.8690650463]], [[1.5157265663]], [[1.7362606525]], [[2.4445047379]], [[1.6058927774]], [[1.9649437666]], [[1.6705367565]], [[2.0847716331]], [[1.7347645760]], [[1.3289380074]], [[1.7498219013]], [[1.3358957767]], [[0.9297507405]], [[0.9264358878]], [[2.1652712822]], [[1.1066381931]], [[1.8031578064]], [[2.1782746315]], [[2.2532589436]], [[1.5320241451]], [[1.1947584152]], [[1.3234183788]], [[1.9331967831]], [[1.4187164307]], [[1.0115050077]], [[0.8818412423]], [[1.7016897202]], [[2.4576334953]], [[1.1460577250]], [[1.4337089062]], [[1.2558991909]], [[0.8946968913]], [[2.0609850883]], [[0.3347344995]], [[1.1415576935]], [[2.1017265320]], [[0.9507287145]], [[1.4968940020]], [[1.2866373062]], [[1.3065831661]], [[2.2343881130]], [[1.6466989517]], [[1.1707606316]], [[0.8803624511]], [[0.9066287875]], [[1.4780917168]], [[0.4957841039]], [[1.5298888683]], [[2.3167405128]], [[0.6148149967]], [[1.4957714081]], [[1.6035988331]], [[0.5339369774]], [[1.9479150772]], [[1.3842637539]], [[1.2301250696]], [[0.5092878938]], [[1.0467274189]], [[1.2039580345]], [[2.2465782166]], [[1.2522583008]], [[1.6173950434]], [[1.2209314108]], [[1.1898455620]], [[2.3735904694]], [[2.1262793541]], [[1.7556240559]], [[2.0349695683]], [[0.9185809493]], [[1.1844347715]], [[0.8521307111]], [[1.0460197926]], [[1.6362686157]], [[0.4107335210]], [[2.1157145500]], [[1.7779140472]], [[1.4670205116]], [[1.2228283882]], [[0.1709958315]], [[2.1705262661]], [[1.9011132717]], [[1.6122062206]], [[1.3603255749]], [[1.7164919376]], [[2.2678537369]], [[1.5054094791]], [[1.5238274336]], [[1.7039620876]], [[0.8844705224]], [[2.0892934799]], [[1.5851008892]], [[1.0590093136]], [[1.2530498505]], [[1.7065652609]], [[1.9205608368]], [[1.9870646000]], [[0.5808738470]], [[0.9296095967]], [[1.3903732300]], [[2.0787308216]], [[0.9173107147]], [[0.4609431028]], [[2.0898370743]], [[1.5761603117]], [[0.9788118601]], [[1.4952857494]], [[0.8890246749]], [[1.1265554428]], [[2.0292136669]], [[0.6961540580]], [[1.5720279217]], [[2.4724993706]], [[1.1420192719]], [[1.5323079824]], [[1.7601671219]], [[1.1385703087]], [[1.3094367981]], [[1.5751839876]], [[1.2077670097]], [[1.1140699387]], [[1.7068177462]], [[1.8979969025]], [[2.2886269093]], [[1.6453484297]], [[1.7598607540]], [[1.0593278408]], [[1.7400527000]], [[1.0384204388]], [[2.5367310047]], [[2.0092327595]], [[1.7892118692]], [[1.5112056732]], [[1.3327699900]], [[1.6375617981]], [[1.1504907608]], [[2.4557099342]], [[1.4994196892]], [[0.9540922642]], [[2.3222959042]], [[1.8723379374]], [[1.3199270964]], [[2.6265647411]], [[1.0876674652]], [[2.4185976982]], [[0.4304662347]], [[1.5272073746]], [[1.4282702208]], [[1.4492261410]], [[1.7424030304]], [[1.1136257648]], [[0.9556840658]], [[1.7302389145]], [[1.2602646351]], [[2.4743216038]], [[1.0944118500]], [[2.1612396240]], [[0.7427238822]], [[0.5044853687]], [[1.2795805931]], [[1.9709289074]], [[1.9602879286]], [[2.2578313351]], [[2.2125241756]], [[1.1943802834]], [[1.4241292477]], [[1.8547847271]], [[1.1571869850]], [[1.8476715088]]]])
test_add.test_module[Add1-[((1, 960, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6389201880]], [[0.9108853936]], [[0.2311410308]], [[0.2746940851]], [[0.4500864148]], [[0.7767422795]], [[0.6327570081]], [[1.0391082764]], [[0.5982915759]], [[0.7749698758]], [[0.4915570617]], [[0.5443809032]], [[0.1649893522]], [[0.3115225434]], [[0.4365520477]], [[0.6611853838]], [[0.8403311968]], [[0.9426749945]], [[0.3036930561]], [[0.4249321818]], [[0.8242721558]], [[1.0578575134]], [[0.5397635102]], [[1.0168194771]], [[0.5620719194]], [[0.6955706477]], [[1.0954017639]], [[0.1788284183]], [[0.3278946280]], [[0.5160809755]], [[0.4477636218]], [[1.0746639967]], [[0.3185737729]], [[0.4124971628]], [[0.2933433652]], [[0.1743831038]], [[0.3507933617]], [[1.0724625587]], [[0.8657727838]], [[0.8849998713]], [[0.6689593792]], [[0.3863218427]], [[0.7272559404]], [[0.1758162379]], [[0.2813804746]], [[0.3848986030]], [[0.9581325650]], [[0.9358242154]], [[0.4209160805]], [[0.6246224046]], [[0.9624439478]], [[1.1397302151]], [[0.8411046863]], [[0.7102100253]], [[0.9779067636]], [[0.3482624292]], [[0.7358356118]], [[0.2550108433]], [[0.2961205244]], [[0.3843718171]], [[0.8689001203]], [[0.8437438011]], [[0.3464873433]], [[0.7937171459]], [[0.9171496034]], [[0.5795549154]], [[0.6617543697]], [[0.7585159540]], [[0.9528518915]], [[1.1227606535]], [[0.2573518157]], [[0.4594287276]], [[0.8391685486]], [[1.0569382906]], [[1.0777672529]], [[1.0838420391]], [[0.7421708703]], [[0.2078722715]], [[0.6886598468]], [[0.3298609257]], [[0.1766865253]], [[1.0869097710]], [[1.0228434801]], [[0.1438996196]], [[0.7362496257]], [[0.5584335923]], [[0.5603830218]], [[0.4137851596]], [[0.8349416852]], [[0.3465118408]], [[0.8259592652]], [[0.8955176473]], [[1.0005993843]], [[0.8296191692]], [[0.1477959752]], [[0.3183151484]], [[0.8923211098]], [[0.7473142743]], [[0.2526215911]], [[0.3547538519]], [[1.1130383015]], [[0.9795725346]], [[0.4246510267]], [[0.5168212056]], [[0.1663645506]], [[0.6336765289]], [[0.2661341429]], [[0.2569852471]], [[0.6151137948]], [[0.7177361250]], [[0.4378984571]], [[0.9393523932]], [[0.3383940458]], [[1.0963486433]], [[0.9853135347]], [[0.2210221291]], [[0.5182214379]], [[0.6652249098]], [[0.7156141400]], [[0.7612507343]], [[0.8388777375]], [[0.6726136804]], [[0.3986992240]], [[0.8792580962]], [[0.1630391479]], [[0.3463102579]], [[0.5174986720]], [[0.3991069198]], [[0.4677469134]], [[0.2328527570]], [[0.5363060236]], [[0.7495418191]], [[0.3169307113]], [[0.6170039177]], [[1.0005890131]], [[0.5912634730]], [[0.6565597057]], [[0.5995291471]], [[0.7438542843]], [[0.9605833292]], [[1.1162867546]], [[0.9601915479]], [[1.1173703671]], [[0.6065027714]], [[0.1935028434]], [[0.4056249857]], [[0.9831162095]], [[0.6394223571]], [[0.3941404223]], [[0.2595077157]], [[0.1747375727]], [[0.2206594944]], [[0.5412452221]], [[0.9168666005]], [[0.9129841328]], [[0.1604476571]], [[0.9545546174]], [[0.2514088750]], [[0.5369584560]], [[0.4399272799]], [[0.5463559628]], [[0.5444922447]], [[0.1939886212]], [[0.2109446526]], [[0.5644238591]], [[0.6491296887]], [[0.4155261517]], [[0.8310132027]], [[0.1926344037]], [[0.6089199781]], [[1.0823733807]], [[0.4387176037]], [[1.0941650867]], [[0.8237404823]], [[0.1914331317]], [[0.9590122700]], [[0.5849663615]], [[0.4194601774]], [[1.0424902439]], [[0.2386159897]], [[0.6963160634]], [[0.5379792452]], [[0.9997199178]], [[0.7822358608]], [[0.8829162717]], [[0.8192430735]], [[0.5224262476]], [[0.5375108719]], [[0.2306228876]], [[0.9135856628]], [[1.0396525860]], [[0.9847760201]], [[0.2899744511]], [[0.6649635434]], [[0.2901965380]], [[0.3674215078]], [[0.3513108492]], [[0.8135361075]], [[0.3447070122]], [[0.6317549944]], [[0.6636976600]], [[0.9649751186]], [[0.2647035718]], [[0.2994074821]], [[0.3523328304]], [[0.9926303029]], [[0.4629310966]], [[1.0644079447]], [[0.8234673738]], [[0.7059766054]], [[0.6389415860]], [[0.5438228250]], [[0.7053967714]], [[0.5284912586]], [[0.6391503811]], [[0.7064601183]], [[0.2515610456]], [[0.3805978894]], [[1.0464099646]], [[0.2368902564]], [[0.6067605019]], [[1.1372829676]], [[0.8232821226]], [[0.6568201184]], [[0.2093586326]], [[0.8903525472]], [[0.2865232229]], [[0.5007314682]], [[0.4749054313]], [[0.5686199069]], [[0.6481327415]], [[1.0550673008]], [[0.7050830126]], [[1.0905100107]], [[0.9485198259]], [[0.3265566230]], [[0.8669156432]], [[0.2892155647]], [[0.4307510257]], [[0.7897249460]], [[0.8077596426]], [[1.0177776814]], [[0.4817056656]], [[0.6434640288]], [[0.9000753760]], [[0.1591175199]], [[1.0041539669]], [[0.2292023897]], [[0.6495548487]], [[0.5576552153]], [[0.3793271184]], [[0.7087491155]], [[1.0561230183]], [[0.4965038300]], [[0.3458165526]], [[0.4577441812]], [[0.1470894217]], [[0.8683605790]], [[0.4025317430]], [[0.3089934587]], [[0.3546128869]], [[0.9301416278]], [[0.9074504375]], [[1.0264245272]], [[0.8240251541]], [[0.4756837487]], [[0.5029395223]], [[0.7903785706]], [[1.0536808968]], [[0.7786096931]], [[0.4060909152]], [[0.4076249003]], [[0.1699290872]], [[0.7506875992]], [[0.3620723486]], [[0.1968756914]], [[1.0811018944]], [[0.3179579973]], [[0.5857828259]], [[0.7859103680]], [[0.6585927606]], [[0.3062227368]], [[0.2385027409]], [[1.0412048101]], [[0.7240829468]], [[1.0574758053]], [[0.4750432968]], [[0.7899413109]], [[0.5283254981]], [[0.6204280257]], [[0.3381415009]], [[0.8117640615]], [[0.8007465601]], [[0.6323493123]], [[0.5302125216]], [[0.3344486952]], [[0.9884359837]], [[0.2704515457]], [[0.8474963307]], [[0.4745368361]], [[0.4014295936]], [[0.7324861288]], [[0.3829351068]], [[0.7578660250]], [[0.7408540249]], [[0.2714188695]], [[0.7259128690]], [[0.8556281924]], [[0.8405790925]], [[0.5797242522]], [[0.2327697873]], [[0.5655865669]], [[0.8163174391]], [[0.4602250457]], [[0.8324541450]], [[0.9756448865]], [[0.3815878034]], [[0.6475945115]], [[0.8494130969]], [[0.6818525195]], [[0.6844293475]], [[0.7050963044]], [[0.2495897412]], [[0.6819577217]], [[0.9888985157]], [[1.0932204723]], [[0.9365390539]], [[0.7096651196]], [[0.8761706948]], [[0.3994237781]], [[0.2283194065]], [[0.2127030492]], [[1.1414685249]], [[0.9600582719]], [[0.2970467210]], [[0.8382849097]], [[1.0202474594]], [[1.1424710751]], [[1.0798610449]], [[1.0300405025]], [[0.5280339718]], [[0.4671927691]], [[1.0531880856]], [[0.9228534102]], [[0.3417741060]], [[1.0921721458]], [[0.8842428923]], [[0.9152313471]], [[0.3292754292]], [[0.7861135006]], [[0.4673823714]], [[1.0333418846]], [[0.5526933074]], [[0.8373198509]], [[0.7314745188]], [[0.8553977013]], [[0.4727498293]], [[0.8864492774]], [[0.2934281230]], [[0.7555910945]], [[0.3043676615]], [[0.1493948102]], [[0.2411357164]], [[1.0373985767]], [[0.9131814241]], [[1.1117714643]], [[1.0432242155]], [[0.1961408854]], [[0.3014480472]], [[0.5618723035]], [[0.3179474473]], [[0.9898618460]], [[0.2646597028]], [[0.3987128735]], [[0.1596179008]], [[0.3587908149]], [[1.0539023876]], [[1.0520436764]], [[1.0005824566]], [[1.0287272930]], [[1.0872569084]], [[0.5146321058]], [[0.8626646399]], [[1.0881288052]], [[0.8080729842]], [[1.1425018311]], [[0.9019950628]], [[0.9534877539]], [[0.4676675200]], [[0.8825773597]], [[0.7001168728]], [[0.5232549310]], [[0.3608086705]], [[0.3621053100]], [[0.2579231858]], [[0.9783287644]], [[0.9981291890]], [[0.5857572556]], [[0.3533205986]], [[1.0291163921]], [[0.9624080658]], [[0.6798306108]], [[0.4065959454]], [[1.1021875143]], [[0.8471339345]], [[0.2630909085]], [[1.1212050915]], [[1.0223505497]], [[0.4604181647]], [[0.9237408638]], [[0.3585690856]], [[0.5643064380]], [[1.0672142506]], [[0.6633231640]], [[0.2890545130]], [[0.4755471349]], [[0.5069410205]], [[0.5461992025]], [[0.6905139089]], [[1.1050784588]], [[0.6694337726]], [[0.3339436054]], [[0.6682907343]], [[0.8824071884]], [[0.8906837106]], [[0.1856943369]], [[0.5531923771]], [[0.2710913420]], [[0.4293208122]], [[0.8228102922]], [[0.2875984907]], [[0.8285270333]], [[1.0670526028]], [[0.6754577756]], [[0.3094197512]], [[0.4635211229]], [[0.7518456578]], [[0.2615045905]], [[0.8910687566]], [[0.1887288094]], [[0.1620169282]], [[0.1568332911]], [[0.5412319303]], [[0.9788800478]], [[0.1694241166]], [[1.0582571030]], [[0.4426521659]], [[0.7891056538]], [[0.6654650569]], [[0.1918040514]], [[1.0573281050]], [[0.9118853211]], [[1.1396614313]], [[0.8952696919]], [[0.3126291037]], [[1.0599554777]], [[0.6695358157]], [[0.8797718287]], [[0.2417490482]], [[0.4988503456]], [[0.1517248154]], [[0.4479173422]], [[0.7505291700]], [[0.2500827312]], [[0.8020457029]], [[0.9110670090]], [[0.7123182416]], [[0.3081219196]], [[0.2550037503]], [[0.4884052873]], [[0.8621426821]], [[1.1358618736]], [[0.9301781058]], [[0.5863589644]], [[0.8179718256]], [[0.1521321535]], [[0.2156129479]], [[0.8759675622]], [[0.3594560027]], [[0.8832114339]], [[0.2896975279]], [[0.3950081468]], [[0.2308191061]], [[0.9035839438]], [[0.5917160511]], [[1.0274641514]], [[0.9520996809]], [[0.9193348885]], [[0.6587416530]], [[0.4880746007]], [[0.5339477658]], [[0.7091138959]], [[0.8905183673]], [[0.2923686504]], [[1.0623025894]], [[0.5882985592]], [[0.2236908078]], [[0.3721346855]], [[1.0850725174]], [[1.0999271870]], [[0.1795237660]], [[0.9953127503]], [[0.8932431936]], [[0.9386228323]], [[1.0659282207]], [[0.3731884360]], [[0.8005514741]], [[0.8472802043]], [[0.4949169159]], [[0.8099901676]], [[0.4988068938]], [[0.9517942667]], [[0.5039362907]], [[0.4562657475]], [[0.7685381174]], [[0.8200104237]], [[0.3983777761]], [[0.6868627667]], [[0.9324314594]], [[0.5929152369]], [[0.7948305607]], [[0.5220701098]], [[0.8179134130]], [[0.2804664969]], [[0.3486492038]], [[0.3888683319]], [[1.1021690369]], [[0.5081215501]], [[0.6412984133]], [[0.4004158378]], [[1.1418086290]], [[1.1309986115]], [[0.2655702829]], [[0.2373286486]], [[0.2636671066]], [[0.6402522326]], [[0.5152116418]], [[0.3153907657]], [[0.4633275270]], [[0.7371290326]], [[0.3814182281]], [[0.7534548640]], [[0.5280056596]], [[0.4003819227]], [[0.7113563418]], [[1.0537927151]], [[0.3046239614]], [[0.6658807993]], [[0.4582797289]], [[1.1333167553]], [[0.1682819724]], [[0.1633329988]], [[1.1353560686]], [[0.3263186216]], [[0.7385252714]], [[0.5995105505]], [[0.5373350978]], [[0.5309897065]], [[0.9603838921]], [[0.6665620804]], [[0.1558563113]], [[0.3474835157]], [[0.4722134471]], [[0.8942717910]], [[0.3190967441]], [[1.1141233444]], [[0.5312989354]], [[0.5528826118]], [[1.0344426632]], [[0.8939674497]], [[1.0667238235]], [[0.9318857789]], [[0.4909866452]], [[0.3109273911]], [[0.6054620743]], [[1.0564823151]], [[0.4748536944]], [[0.1789786220]], [[0.8476278186]], [[1.1293992996]], [[0.5003219843]], [[0.2286518216]], [[0.1891337633]], [[0.7679632902]], [[0.6048037410]], [[0.3901699781]], [[0.7437295318]], [[0.8325430155]], [[1.0403230190]], [[1.0308547020]], [[0.5678217411]], [[0.2017800212]], [[0.1908520460]], [[1.1095083952]], [[0.8636912107]], [[0.8606172800]], [[0.2100530863]], [[1.1056714058]], [[1.1163351536]], [[1.0940937996]], [[0.2208710909]], [[0.4540019035]], [[0.2987816930]], [[1.1161463261]], [[0.4278236032]], [[0.4143863320]], [[0.9046200514]], [[0.4113674164]], [[0.3964025378]], [[0.5989278555]], [[0.5946038365]], [[0.2531743646]], [[1.0594975948]], [[0.4221027493]], [[0.8200210333]], [[1.0775554180]], [[0.8948218226]], [[0.7134535313]], [[1.0680921078]], [[0.7098767161]], [[0.4113352895]], [[1.1156632900]], [[0.7610038519]], [[0.1548231840]], [[0.5003178120]], [[0.3020798564]], [[1.0810401440]], [[0.5600906610]], [[0.1869011521]], [[0.6112319231]], [[0.9566700459]], [[0.7725638151]], [[0.8007658124]], [[0.6891002059]], [[0.8290729523]], [[0.5208376050]], [[0.4437634945]], [[0.1753093600]], [[0.2659951448]], [[0.8593648672]], [[0.3466069102]], [[0.7143968344]], [[0.8022593260]], [[0.6780655384]], [[0.3184887171]], [[1.1208277941]], [[0.3519138694]], [[1.0539138317]], [[0.2449094653]], [[0.5223893523]], [[0.9146479964]], [[0.4383732677]], [[1.0626850128]], [[0.2985853553]], [[0.2227778435]], [[0.4172040224]], [[0.7234888077]], [[1.1030415297]], [[0.4039631486]], [[0.8214777112]], [[0.5173100829]], [[0.5342324376]], [[1.0103152990]], [[0.2551706433]], [[0.6957702637]], [[1.1128681898]], [[0.5739575028]], [[1.0308825970]], [[0.4886851907]], [[1.0451446772]], [[0.1589984298]], [[0.5706017613]], [[0.5548403859]], [[0.8047060966]], [[0.8388273120]], [[1.0265719891]], [[0.5681706071]], [[0.6228360534]], [[0.9850692153]], [[0.5073770285]], [[1.0809804201]], [[0.3097575307]], [[0.5885586739]], [[0.6158081293]], [[0.8657685518]], [[0.9845016599]], [[0.5634117126]], [[0.2284232974]], [[0.8904097080]], [[0.7922294736]], [[0.8435132504]], [[0.3342297673]], [[0.9644613862]], [[1.1162410975]], [[0.6860492826]], [[0.1756393909]], [[0.9936585426]], [[0.2719379663]], [[0.7575996518]], [[0.7153224945]], [[0.4086453319]], [[0.8167613745]], [[0.1954472661]], [[0.7565433383]], [[0.3256881237]], [[0.5885967612]], [[0.7069780231]], [[1.0686358213]], [[0.4041034579]], [[0.9629816413]], [[0.5791466832]], [[0.4051997066]], [[0.2072637677]], [[0.1839044094]], [[1.1309689283]], [[0.5179638267]], [[0.6676380038]], [[0.7782197595]], [[0.9825296998]], [[1.0694006681]], [[1.0481722355]], [[0.2722220421]], [[0.5625551939]], [[0.3467947841]], [[0.3569877148]], [[0.7612671256]], [[1.1119875908]], [[0.2421116233]], [[0.9452658296]], [[0.3834302425]], [[0.5452764630]], [[1.0395491123]], [[0.5295784473]], [[0.6881914735]], [[0.2931473255]], [[1.0682840347]], [[0.5780286789]], [[0.2769683599]], [[0.7890687585]], [[0.2871549129]], [[0.2459099293]], [[0.6731231809]], [[1.0390853882]], [[0.5011715889]], [[0.8779970407]], [[1.0723242760]], [[0.9742940664]], [[0.3803830743]], [[0.5878536701]], [[0.4853211641]], [[0.2406021357]], [[0.6428701878]], [[1.0188810825]], [[1.0639214516]], [[0.6893139482]], [[0.7562251091]], [[0.4262027740]], [[1.0200848579]], [[0.4346093535]], [[0.2953372002]], [[0.7196603417]], [[0.9423562884]], [[0.1918678284]], [[1.0946474075]], [[0.8225291371]], [[0.2923572063]], [[0.5349598527]], [[1.0764535666]], [[0.2590462565]], [[0.4965249896]], [[0.8066537380]], [[0.2046210766]], [[0.9167627096]], [[0.9029374123]], [[0.9527663589]], [[0.3238900304]], [[1.1406641006]], [[0.3462828398]], [[1.1418337822]], [[0.1628183722]], [[0.1971786022]], [[0.9497617483]], [[0.6949200630]], [[0.6715082526]], [[0.3657844067]], [[0.4329258800]], [[0.4964698553]], [[0.1555858850]], [[0.6686479449]], [[0.7310930490]], [[0.6422402859]], [[0.8041287661]], [[1.1170890331]], [[0.7756056190]], [[0.4596273899]], [[0.4368902445]], [[0.3227608204]], [[0.2960541844]], [[0.5621386766]], [[0.5542308092]], [[0.8650948405]], [[0.4289451838]], [[1.0412654877]], [[0.2918235064]], [[0.6440846920]], [[1.0921235085]], [[1.1398561001]], [[0.3530324697]], [[0.7317280769]], [[0.7017249465]], [[0.4082377553]], [[0.4699161649]], [[0.7780905962]], [[0.2949810028]], [[0.7251614332]], [[0.8590259552]], [[0.4456285238]], [[1.0579841137]], [[0.6097583771]], [[0.8695199490]], [[1.1378167868]], [[0.4898294210]], [[0.9143676758]], [[0.4996489882]], [[0.5696271062]], [[0.5579280257]], [[0.6395573020]], [[0.4537793994]], [[0.7598570585]], [[0.6615037918]], [[0.9596084952]], [[0.5414609313]], [[0.6928036809]], [[0.4566684365]], [[0.2239367366]], [[0.8449972272]], [[0.7066425681]], [[0.4424217343]], [[0.4736111164]], [[0.7734242678]], [[0.5522551537]], [[0.9694015384]], [[0.6709851623]], [[0.8315319419]], [[0.8605117202]], [[0.5199804306]], [[0.8589960337]], [[1.0116302967]], [[0.6656727791]], [[0.7405100465]], [[0.6608313918]], [[0.9888473153]], [[0.4315659404]], [[0.3768786788]], [[0.8605403900]], [[0.2075065374]], [[0.6506798267]], [[0.4127180576]], [[0.9726803899]], [[0.1771440506]], [[0.9458298683]], [[1.1373419762]], [[0.7438336611]], [[0.6093502641]], [[1.1348385811]], [[0.4289874434]], [[0.6025968194]], [[0.4237111211]], [[0.5736429691]], [[0.7513330579]], [[0.7094442844]], [[0.5482261181]], [[0.1427432895]], [[0.6682656407]], [[0.6278526187]], [[0.7178819776]], [[1.0257887840]], [[1.1286683083]], [[0.3460223675]], [[0.6114863753]], [[0.4423267841]], [[0.1818407774]], [[0.2788340449]], [[1.1119441986]], [[0.9341941476]], [[0.9112391472]], [[0.9205352068]], [[0.2454240918]], [[0.6951732039]], [[1.1080209017]], [[0.3636206388]], [[1.0874607563]], [[0.7740828395]], [[0.9944353700]], [[0.4284697771]], [[0.8731724620]], [[0.1994197369]], [[0.6109158397]], [[0.8094391823]], [[0.7926108241]], [[1.0610675812]], [[1.1339921951]], [[1.0975420475]], [[0.9783596992]], [[0.9941946268]], [[0.5781487226]], [[0.2546401620]], [[0.4570500255]], [[0.5865313411]], [[0.3724703193]], [[0.8982490301]], [[0.8181772232]], [[0.7233395576]], [[0.7634439468]], [[1.0818897486]], [[0.8247792721]], [[0.2041938901]], [[0.2797232270]], [[0.8630384207]], [[0.7108032107]], [[0.8865094185]], [[0.1433036327]], [[0.1812289357]], [[0.9492768049]], [[0.9624864459]], [[0.1903080940]], [[0.8324562311]], [[0.2522414923]], [[1.0213174820]], [[0.7995588183]], [[1.1370570660]], [[0.1496676207]], [[0.1613255143]], [[0.9807146192]], [[1.0865751505]], [[0.9490035772]], [[0.9750785232]], [[0.2215279341]], [[1.0045704842]], [[0.1705900431]], [[0.7543644309]], [[0.3151462078]], [[0.4440658689]], [[0.7311980128]], [[0.4460735917]], [[0.5635521412]], [[0.1687514186]], [[0.1712269783]], [[0.8363451362]], [[0.4310243726]], [[0.3384687901]], [[0.4453724623]], [[0.2338793278]], [[0.2429634929]], [[0.9418497682]], [[0.8967846632]], [[0.7092653513]], [[0.3695337772]], [[0.8090140820]], [[0.6651262045]]]]), compiled_model=tensor([[[[0.6389201880]], [[1.2988183498]], [[0.4935047030]], [[0.8327759504]], [[0.7430298924]], [[1.4203960896]], [[0.5478624701]], [[1.2198748589]], [[1.0463795662]], [[0.9424073696]], [[0.4326074719]], [[1.0653885603]], [[0.6656162739]], [[1.7549519539]], [[1.7778736353]], [[1.6280913353]], [[2.3371217251]], [[2.8266682625]], [[1.7233885527]], [[2.8066430092]], [[2.0063772202]], [[2.2574613094]], [[2.2264621258]], [[2.8090763092]], [[0.9900739193]], [[1.7364144325]], [[2.3852984905]], [[1.0908193588]], [[2.6368048191]], [[2.3892490864]], [[2.0745279789]], [[2.8275501728]], [[1.8663077354]], [[1.8529963493]], [[2.0438735485]], [[1.8804848194]], [[1.7554677725]], [[2.5041904449]], [[1.8839771748]], [[1.9220557213]], [[1.5123081207]], [[1.7782084942]], [[2.2129344940]], [[2.6301920414]], [[1.2535920143]], [[1.3045489788]], [[2.5638051033]], [[1.8336691856]], [[2.1064994335]], [[3.0162501335]], [[2.3920106888]], [[2.6603012085]], [[2.5753228664]], [[2.0905015469]], [[2.8495230675]], [[1.3002748489]], [[1.6352093220]], [[1.3164962530]], [[2.3966603279]], [[2.1842830181]], [[1.7510709763]], [[1.8441718817]], [[1.8961627483]], [[2.2418699265]], [[2.2160911560]], [[2.2436954975]], [[2.5408573151]], [[2.1251621246]], [[2.5762119293]], [[2.3229131699]], [[1.1348443031]], [[1.5164811611]], [[2.1527218819]], [[3.1456284523]], [[2.3137321472]], [[2.4061968327]], [[2.1000676155]], [[0.9857425690]], [[2.2369620800]], [[1.7766824961]], [[1.4638401270]], [[2.6323511600]], [[2.3444774151]], [[1.1861650944]], [[2.0781769753]], [[1.8114900589]], [[1.8767488003]], [[2.3324170113]], [[1.9817268848]], [[1.2954241037]], [[1.6196727753]], [[1.7989380360]], [[1.9368805885]], [[3.4290533066]], [[2.7163670063]], [[1.3702468872]], [[2.2778139114]], [[2.3700301647]], [[1.6643960476]], [[1.6365320683]], [[2.5894105434]], [[2.1530618668]], [[1.1289794445]], [[1.4827246666]], [[1.3523398638]], [[2.0534567833]], [[0.2137469649]], [[0.3457288146]], [[2.3392951488]], [[2.8848013878]], [[1.4647631645]], [[2.7656149864]], [[1.9082925320]], [[2.2600765228]], [[2.1695592403]], [[1.3792740107]], [[2.1941807270]], [[2.3817381859]], [[1.7110695839]], [[1.6679674387]], [[2.1652464867]], [[2.3354914188]], [[2.3344736099]], [[2.8545722961]], [[1.0163989067]], [[2.2159254551]], [[1.8771529198]], [[0.9241573215]], [[1.4486880302]], [[2.3061099052]], [[2.1599524021]], [[2.1282973289]], [[1.5306375027]], [[1.8109310865]], [[2.6637656689]], [[1.9420301914]], [[1.6661075354]], [[1.8255512714]], [[1.6947757006]], [[1.4754860401]], [[2.1771242619]], [[2.0600609779]], [[1.9660054445]], [[1.7738424540]], [[1.1757898331]], [[1.8437470198]], [[2.3570694923]], [[1.5933077335]], [[2.4020915031]], [[1.7748816013]], [[1.7983037233]], [[2.0677905083]], [[2.5225954056]], [[2.8163404465]], [[2.4544999599]], [[1.1588692665]], [[1.5702981949]], [[0.9810434580]], [[2.1703796387]], [[1.4513704777]], [[1.3527536392]], [[1.2531181574]], [[0.6882754564]], [[1.0390908718]], [[1.2618222237]], [[1.3512433767]], [[1.5598362684]], [[1.4232705832]], [[1.4750003815]], [[0.6324067116]], [[1.4468834400]], [[0.8118239045]], [[1.6650964022]], [[1.4667679071]], [[1.0993716717]], [[1.5893378258]], [[1.1623456478]], [[1.3637266159]], [[1.5942943096]], [[1.5279954672]], [[0.9868919253]], [[1.2225570679]], [[2.5952053070]], [[1.0157142878]], [[2.2002203465]], [[1.7273399830]], [[1.4170174599]], [[2.0992150307]], [[0.7835580111]], [[1.4139406681]], [[2.1830840111]], [[0.9493522048]], [[1.5968674421]], [[1.6136507988]], [[1.6769257784]], [[1.5438482761]], [[1.0416548252]], [[1.7364830971]], [[1.5604838133]], [[1.0490456820]], [[0.7535116076]], [[2.0406250954]], [[1.8805844784]], [[1.3271431923]], [[1.1797764301]], [[1.8140907288]], [[0.9938030243]], [[1.7376227379]], [[0.8927022219]], [[1.6395819187]], [[0.7404720187]], [[1.2819378376]], [[1.1862115860]], [[1.8961097002]], [[1.1948616505]], [[1.6711031199]], [[1.0546979904]], [[1.5231361389]], [[1.9821190834]], [[1.8341147900]], [[1.7177363634]], [[2.2922365665]], [[1.9950120449]], [[1.7897975445]], [[1.4401869774]], [[1.7630248070]], [[1.2678070068]], [[1.0166168213]], [[1.0062783957]], [[0.8081930280]], [[1.8809726238]], [[1.5687904358]], [[1.7134102583]], [[2.7467145920]], [[2.3838653564]], [[1.5781142712]], [[1.7130248547]], [[1.2237656116]], [[1.2617788315]], [[1.7003198862]], [[1.1779662371]], [[1.4992029667]], [[1.0415866375]], [[1.9700987339]], [[1.7805628777]], [[1.3554978371]], [[1.9963860512]], [[0.4836470485]], [[2.1953282356]], [[0.5769989491]], [[1.0575573444]], [[2.0733466148]], [[1.6254334450]], [[1.2399430275]], [[1.1044481993]], [[1.3639454842]], [[1.6701211929]], [[1.4612668753]], [[1.2686684132]], [[0.5234751105]], [[0.8029619455]], [[1.2749166489]], [[1.2349525690]], [[2.3929803371]], [[2.0054097176]], [[0.7641801238]], [[1.2601857185]], [[1.7944668531]], [[1.0502828360]], [[2.1955349445]], [[0.8299707770]], [[0.9711878896]], [[0.5233606696]], [[1.4499315023]], [[1.0938169956]], [[1.5491821766]], [[2.0142633915]], [[0.8212296367]], [[1.2754153013]], [[1.4228732586]], [[1.9977407455]], [[1.5385346413]], [[0.9274030328]], [[2.1442885399]], [[1.1516772509]], [[1.9309833050]], [[0.7217119336]], [[0.7794787884]], [[1.6897404194]], [[0.8521829247]], [[1.6062281132]], [[1.4602787495]], [[1.7674450874]], [[1.6265258789]], [[0.5120745897]], [[1.7370116711]], [[2.2847456932]], [[1.4924877882]], [[1.4640924931]], [[1.3584856987]], [[1.6289601326]], [[1.2070409060]], [[1.3389408588]], [[2.2600481510]], [[1.4344725609]], [[1.2512040138]], [[1.4473224878]], [[1.0540201664]], [[1.8835759163]], [[1.1806180477]], [[1.0369954109]], [[1.4585574865]], [[1.1763201952]], [[0.9358327389]], [[1.9240455627]], [[1.9382293224]], [[0.8710749149]], [[0.6941512823]], [[2.0346300602]], [[1.8466453552]], [[1.2668386698]], [[1.6014542580]], [[0.5440105796]], [[1.5553388596]], [[1.9586144686]], [[1.3672717810]], [[1.6885459423]], [[2.1046092510]], [[1.1233680248]], [[1.2182781696]], [[0.9203943610]], [[0.6413967013]], [[2.0395698547]], [[1.4195789099]], [[0.7438098192]], [[1.7975316048]], [[2.2267472744]], [[2.7383880615]], [[2.2874479294]], [[2.1152982712]], [[2.1009936333]], [[0.9152886271]], [[1.8365706205]], [[1.1887100935]], [[2.0777392387]], [[2.4123048782]], [[1.8443818092]], [[1.9055993557]], [[1.2182819843]], [[2.2483658791]], [[1.3518779278]], [[2.6296868324]], [[1.7055060863]], [[1.0770152807]], [[2.2515110970]], [[2.0496702194]], [[1.4741882086]], [[2.3921861649]], [[1.0291817188]], [[2.1202750206]], [[0.4899244308]], [[1.1542129517]], [[0.7547579408]], [[2.0482516289]], [[1.5928993225]], [[1.9268118143]], [[1.7761304379]], [[1.5091757774]], [[0.8382238150]], [[1.9331524372]], [[1.0083961487]], [[2.3296236992]], [[0.4900735021]], [[0.3689658046]], [[0.4288831949]], [[2.0745491982]], [[2.3184199333]], [[2.1970067024]], [[2.6391491890]], [[1.1922249794]], [[2.0227007866]], [[1.3242721558]], [[1.8608531952]], [[2.3651986122]], [[1.5856065750]], [[1.5383391380]], [[0.9307010770]], [[1.1801809072]], [[1.6118710041]], [[2.1742095947]], [[1.7657295465]], [[2.0826246738]], [[1.7768194675]], [[1.4551670551]], [[1.7576396465]], [[2.3710260391]], [[1.7638989687]], [[1.6531395912]], [[0.8251370192]], [[1.6096036434]], [[1.1304185390]], [[1.5766422749]], [[0.7961763740]], [[2.2476558685]], [[1.2625426054]], [[1.4407041073]], [[1.9037697315]], [[2.2319669724]], [[1.0762115717]], [[1.3851712942]], [[1.2678990364]], [[1.7583084106]], [[2.1950538158]], [[2.0549750328]], [[1.1255359650]], [[1.4896340370]], [[1.8566446304]], [[1.9482378960]], [[1.5724108219]], [[1.9313230515]], [[1.5113176107]], [[0.8884894848]], [[1.9501223564]], [[1.3380683661]], [[1.6201450825]], [[1.2206969261]], [[1.6130194664]], [[1.9404928684]], [[1.3846451044]], [[1.2795050144]], [[1.1254463196]], [[1.4127736092]], [[1.3131471872]], [[1.6607258320]], [[1.0869407654]], [[1.6200883389]], [[1.5962095261]], [[0.7536966801]], [[1.6620368958]], [[0.2924330831]], [[1.1222553253]], [[0.2718600631]], [[1.7468438148]], [[2.4895768166]], [[0.3518910408]], [[1.4085410833]], [[1.6820220947]], [[2.1915731430]], [[2.1295261383]], [[1.3358027935]], [[1.8972176313]], [[1.2454270124]], [[1.6742912531]], [[0.9374002814]], [[0.5569246411]], [[2.1021339893]], [[1.6636974812]], [[1.5155081749]], [[0.9996660352]], [[1.8389116526]], [[0.9183868766]], [[1.5760370493]], [[1.4336316586]], [[1.2631682158]], [[1.7537162304]], [[1.8029299974]], [[0.7939642072]], [[1.0934326649]], [[1.2033085823]], [[1.5729315281]], [[2.1266028881]], [[2.0662882328]], [[1.9578158855]], [[1.2479333878]], [[1.7894411087]], [[1.1774518490]], [[0.6499646902]], [[2.0856971741]], [[1.2668957710]], [[0.8197728395]], [[1.1062601805]], [[0.8672386408]], [[1.4735059738]], [[1.1799733639]], [[1.7347635031]], [[2.0619297028]], [[1.5214183331]], [[2.1762106419]], [[2.2315676212]], [[1.4822542667]], [[0.8232965469]], [[1.1214367151]], [[2.4820446968]], [[0.8938096762]], [[1.9327883720]], [[1.6334704161]], [[1.2296009064]], [[1.6991429329]], [[1.7009205818]], [[1.4806916714]], [[0.4004490376]], [[1.9594316483]], [[1.1161067486]], [[1.6140422821]], [[1.6396151781]], [[0.8340461254]], [[1.9471063614]], [[1.9913508892]], [[0.5819440484]], [[1.6814767122]], [[1.9434244633]], [[1.4819810390]], [[0.7639879584]], [[1.4381244183]], [[1.5458033085]], [[1.8890557289]], [[1.3906540871]], [[1.1733026505]], [[1.2506834269]], [[1.8914725780]], [[0.8234792948]], [[1.0396403074]], [[1.1780738831]], [[0.4264145494]], [[1.0300681591]], [[0.3761454225]], [[1.8356484175]], [[1.5349524021]], [[1.1758304834]], [[1.5201163292]], [[2.4134454727]], [[2.0978548527]], [[1.6189188957]], [[0.8855298758]], [[1.5625845194]], [[1.6539891958]], [[2.2259950638]], [[1.6356565952]], [[2.2437901497]], [[1.4957149029]], [[0.4436801076]], [[1.4788613319]], [[1.3157564402]], [[1.6061086655]], [[1.7302756310]], [[1.0858030319]], [[0.9462101460]], [[1.1579592228]], [[1.6591892242]], [[2.4637479782]], [[0.9327768087]], [[0.7732331753]], [[1.9461920261]], [[0.7964320183]], [[1.3217235804]], [[1.0169489384]], [[1.2107156515]], [[1.0532023907]], [[1.1558041573]], [[1.3351438046]], [[0.7843061090]], [[1.4814438820]], [[1.2847602367]], [[2.3094558716]], [[0.4327772856]], [[1.5439014435]], [[1.6418018341]], [[1.6699597836]], [[2.4799439907]], [[1.7508995533]], [[1.8534994125]], [[1.8582549095]], [[1.7099490166]], [[0.9781553149]], [[2.3281936646]], [[1.5823061466]], [[1.6840436459]], [[0.7140824199]], [[1.7164359093]], [[1.7795374393]], [[1.4735395908]], [[1.2874834538]], [[0.7765902877]], [[1.8361166716]], [[1.1641805172]], [[1.8821854591]], [[2.4204339981]], [[2.1218876839]], [[1.7477912903]], [[1.5830814838]], [[1.4579468966]], [[1.0890700817]], [[0.2989229560]], [[2.1687474251]], [[1.7306339741]], [[1.3406019211]], [[1.0476143360]], [[2.4894409180]], [[1.7544082403]], [[1.2263457775]], [[0.5317156315]], [[1.7310724258]], [[1.4487860203]], [[2.2024619579]], [[1.8902075291]], [[1.7423844337]], [[1.2647770643]], [[1.3749401569]], [[1.6840031147]], [[1.8012895584]], [[1.5273509026]], [[1.4383857250]], [[1.0774629116]], [[1.0908963680]], [[2.0304865837]], [[1.5028542280]], [[1.7112162113]], [[2.2429251671]], [[1.6274894476]], [[2.0039343834]], [[0.6275576353]], [[1.7411286831]], [[0.8552755117]], [[0.9490455985]], [[1.0259432793]], [[1.1347725391]], [[1.5794568062]], [[1.5336185694]], [[0.6177737117]], [[1.6415812969]], [[2.3925213814]], [[0.8481150866]], [[1.8010163307]], [[1.5259532928]], [[1.7656083107]], [[1.2884104252]], [[1.4999831915]], [[1.5202929974]], [[1.1242389679]], [[1.4932074547]], [[1.6829164028]], [[1.2290197611]], [[2.2363400459]], [[0.7895829082]], [[0.8658518195]], [[1.5578114986]], [[1.3085157871]], [[1.9705107212]], [[1.2205266953]], [[1.3987293243]], [[1.1603055000]], [[1.3221619129]], [[1.5414502621]], [[1.1528912783]], [[1.0589461327]], [[0.9921824932]], [[1.5483164787]], [[1.8819913864]], [[1.5733687878]], [[2.3494188786]], [[1.6173179150]], [[1.3198715448]], [[2.0789260864]], [[1.7027032375]], [[2.1534948349]], [[2.5537614822]], [[0.9177682996]], [[1.2379645109]], [[0.5274739265]], [[2.1062641144]], [[1.2003819942]], [[1.0198884010]], [[2.3215997219]], [[1.2153373957]], [[1.9068410397]], [[1.6556181908]], [[1.0930092335]], [[1.1043345928]], [[1.8099434376]], [[1.2760341167]], [[1.7543935776]], [[1.5820648670]], [[1.4604362249]], [[1.1224498749]], [[1.2251083851]], [[1.8501992226]], [[1.0212533474]], [[1.4556055069]], [[1.8414361477]], [[1.9255690575]], [[1.1461676359]], [[0.7067813873]], [[2.2583217621]], [[1.4114665985]], [[1.3319324255]], [[1.0285203457]], [[1.7530515194]], [[2.3051898479]], [[2.3391928673]], [[2.2974309921]], [[2.5036263466]], [[2.8295099735]], [[2.1983921528]], [[3.8512187004]], [[2.2057085037]], [[2.2252609730]], [[2.4837827682]], [[3.2838156223]], [[1.5057592392]], [[2.3213868141]], [[2.5588340759]], [[2.1501719952]], [[2.7550983429]], [[2.5620036125]], [[3.4644827843]], [[2.8100388050]], [[2.7571377754]], [[2.7811517715]], [[2.8399860859]], [[3.3147964478]], [[3.2990813255]], [[2.6545066833]], [[2.3746349812]], [[1.9508521557]], [[1.9338434935]], [[2.4099137783]], [[2.6833219528]], [[2.7665269375]], [[2.9162821770]], [[2.1204752922]], [[2.3053321838]], [[2.6330151558]], [[3.0927457809]], [[4.0796265602]], [[2.6599116325]], [[3.4762320518]], [[2.6976170540]], [[1.9817891121]], [[3.5712094307]], [[2.0193572044]], [[1.3443940878]], [[2.6841173172]], [[3.8812046051]], [[3.0736503601]], [[1.9467797279]], [[2.7162022591]], [[2.8486881256]], [[2.7192144394]], [[2.2968249321]], [[2.8441181183]], [[2.7085161209]], [[2.7222504616]], [[2.9723274708]], [[3.0078594685]], [[1.7175710201]], [[2.4262051582]], [[1.9014602900]], [[3.1155061722]], [[1.7690463066]], [[2.5124268532]], [[2.8480749130]], [[2.6893343925]], [[2.6407308578]], [[2.5949461460]], [[2.2684671879]], [[2.2570071220]], [[2.0318777561]], [[2.9659173489]], [[1.7229695320]], [[2.0056307316]], [[2.1399738789]], [[2.3393802643]], [[2.9747865200]], [[2.7612295151]], [[2.6043987274]], [[2.1133742332]], [[3.0215387344]], [[3.3176856041]], [[4.4304056168]], [[2.1602151394]], [[2.2480807304]], [[3.5723156929]], [[2.8660259247]], [[2.7641105652]], [[2.1671605110]], [[2.3463289738]], [[1.7582514286]], [[1.5020806789]], [[2.0727682114]], [[2.3703150749]], [[0.7051126957]], [[1.7285375595]], [[3.6967360973]], [[3.3857643604]], [[1.6971490383]], [[3.1496055126]], [[2.7124037743]], [[1.9969489574]], [[2.0103168488]], [[2.6720066071]], [[3.2455244064]], [[2.2658858299]], [[3.0152621269]], [[2.0782270432]], [[2.2882080078]], [[3.5360784531]], [[3.2915360928]], [[2.7499895096]], [[2.5096383095]], [[3.0919997692]], [[1.9142830372]], [[1.3278501034]], [[2.1233091354]], [[2.7717738152]], [[2.8966581821]], [[3.2001962662]], [[2.1861054897]], [[2.4431912899]], [[2.7985620499]], [[2.9600300789]], [[2.8953847885]], [[1.7588822842]], [[2.2758178711]], [[1.1429795027]], [[1.9171218872]], [[2.3379440308]], [[1.6331272125]], [[2.3069825172]], [[2.6665332317]], [[2.6324200630]], [[2.5003178120]], [[1.8162038326]], [[3.3099370003]], [[2.0908832550]], [[2.5959079266]], [[2.7381937504]], [[2.7073459625]], [[2.3560652733]], [[2.4136953354]], [[2.6080622673]], [[1.1947592497]], [[2.6146297455]], [[2.6043949127]], [[2.4894170761]], [[2.1897108555]], [[1.2777467966]], [[2.2679474354]], [[2.6089980602]], [[2.3600690365]], [[2.1952297688]], [[1.9751069546]], [[2.4983966351]], [[2.2408039570]], [[1.1374735832]], [[1.3241360188]], [[0.9367995858]], [[1.2306723595]], [[1.3609992266]], [[2.4884846210]], [[0.9148886800]], [[2.1825914383]], [[2.8500118256]], [[1.8652923107]], [[2.0641880035]], [[1.5377545357]], [[1.4593069553]], [[2.9175612926]], [[1.6503877640]], [[2.6784615517]], [[2.1031253338]], [[2.2793436050]], [[2.1193990707]], [[0.7686277628]], [[1.9019246101]], [[1.9880762100]], [[1.4230060577]], [[2.4797155857]], [[2.3296999931]], [[1.8209071159]], [[2.5488333702]], [[1.5817232132]], [[1.9895883799]], [[2.3040468693]], [[2.3059060574]], [[1.5400861502]], [[2.3323001862]], [[2.9277002811]], [[1.8396100998]], [[2.2704715729]], [[2.0791864395]], [[1.8141314983]], [[2.2063105106]], [[0.9243448973]], [[2.1575117111]], [[1.4724090099]], [[2.5685510635]], [[0.7170946598]], [[2.8311834335]], [[2.1157300472]], [[2.5532131195]], [[2.7874691486]], [[2.5070550442]], [[2.6911392212]], [[3.2802009583]], [[2.4574236870]], [[2.4004287720]], [[1.7825133801]], [[2.3991580009]], [[2.1786322594]], [[1.5587446690]], [[2.5054073334]], [[2.0114092827]], [[1.5104267597]], [[1.5472161770]], [[3.1044974327]], [[1.7887538671]], [[1.8646881580]], [[2.3153343201]], [[2.9736337662]], [[2.1001636982]], [[1.9386042356]], [[1.3240584135]], [[1.9717621803]], [[2.2253296375]], [[1.8313279152]], [[0.9294857383]], [[2.3914823532]], [[2.5672113895]], [[2.0247116089]], [[2.0906040668]], [[2.2502927780]], [[0.9017009139]], [[2.0796470642]], [[1.1727855206]], [[2.0854692459]], [[2.9080665112]], [[1.7831436396]], [[1.5757583380]], [[2.1485440731]], [[1.3345096111]], [[2.8460888863]], [[1.8191815615]], [[1.4721629620]], [[1.4688968658]], [[1.2100387812]], [[1.8989802599]], [[0.5218719244]], [[1.5584522486]], [[3.0104219913]], [[0.9031757712]], [[1.6915766001]], [[1.9063076973]], [[0.6251527071]], [[2.0482149124]], [[2.1834499836]], [[1.9842461348]], [[1.0758895874]], [[1.2735975981]], [[1.8703085184]], [[2.7690408230]]]])
test_add.test_module[Add1-[((1, 96, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.2459141016]], [[1.5178792477]], [[0.8381349444]], [[0.8816879988]], [[1.0570802689]], [[1.3837361336]], [[1.2397508621]], [[1.6461021900]], [[1.2052855492]], [[1.3819637299]], [[1.0985510349]], [[1.1513748169]], [[0.7719832659]], [[0.9185164571]], [[1.0435459614]], [[1.2681792974]], [[1.4473251104]], [[1.5496689081]], [[0.9106869698]], [[1.0319261551]], [[1.4312660694]], [[1.6648514271]], [[1.1467573643]], [[1.6238133907]], [[1.1690658331]], [[1.3025646210]], [[1.7023956776]], [[0.7858223319]], [[0.9348885417]], [[1.1230748892]], [[1.0547575951]], [[1.6816579103]], [[0.9255676866]], [[1.0194910765]], [[0.9003372788]], [[0.7813770175]], [[0.9577872753]], [[1.6794564724]], [[1.4727666378]], [[1.4919937849]], [[1.2759532928]], [[0.9933157563]], [[1.3342498541]], [[0.7828101516]], [[0.8883743882]], [[0.9918925166]], [[1.5651264191]], [[1.5428180695]], [[1.0279099941]], [[1.2316162586]], [[1.5694378614]], [[1.7467241287]], [[1.4480986595]], [[1.3172039986]], [[1.5849006176]], [[0.9552563429]], [[1.3428294659]], [[0.8620047569]], [[0.9031144381]], [[0.9913657308]], [[1.4758939743]], [[1.4507377148]], [[0.9534812570]], [[1.4007110596]], [[1.5241434574]], [[1.1865488291]], [[1.2687482834]], [[1.3655098677]], [[1.5598458052]], [[1.7297545671]], [[0.8643457294]], [[1.0664227009]], [[1.4461624622]], [[1.6639322042]], [[1.6847611666]], [[1.6908359528]], [[1.3491647243]], [[0.8148661852]], [[1.2956538200]], [[0.9368548393]], [[0.7836804390]], [[1.6939036846]], [[1.6298373938]], [[0.7508935332]], [[1.3432435989]], [[1.1654274464]], [[1.1673769951]], [[1.0207791328]], [[1.4419355392]], [[0.9535057545]], [[1.4329531193]], [[1.5025115013]], [[1.6075932980]], [[1.4366130829]], [[0.7547898889]], [[0.9253090620]]]]), compiled_model=tensor([[[[1.2459141016]], [[1.0340647697]], [[0.5653267503]], [[0.3302959204]], [[0.3118000627]], [[1.1341459751]], [[0.9378650188]], [[1.4618458748]], [[1.2964828014]], [[1.5308226347]], [[0.8844418526]], [[1.0286352634]], [[0.2210917473]], [[1.1680202484]], [[0.9377326965]], [[0.8014040589]], [[1.2389695644]], [[1.0983805656]], [[1.1475552320]], [[1.0712819099]], [[0.7867957950]], [[1.1084175110]], [[1.2318291664]], [[1.2277557850]], [[1.0086843967]], [[0.8349524736]], [[1.2771945000]], [[0.9569998980]], [[0.5309362411]], [[1.0555912256]], [[0.3512288332]], [[1.4466662407]], [[0.2271723151]], [[1.0743741989]], [[0.9881686568]], [[0.2309532166]], [[0.3315879703]], [[1.4679858685]], [[1.0489867926]], [[1.0845279694]], [[1.3283915520]], [[1.2306580544]], [[0.6233332753]], [[0.2639709115]], [[0.5764793754]], [[1.0849223137]], [[0.8675936460]], [[0.8159333467]], [[0.4865965247]], [[1.2791895866]], [[0.9710702300]], [[1.8456575871]], [[1.0519739389]], [[0.6486096978]], [[1.4917651415]], [[0.7440613508]], [[1.5495531559]], [[0.6414812803]], [[0.3291386366]], [[0.3380817771]], [[0.7774683237]], [[0.9861490130]], [[1.0417611599]], [[1.0269501209]], [[1.4290406704]], [[1.3491560221]], [[1.0330395699]], [[1.0594881773]], [[0.8385707736]], [[1.9541437626]], [[0.5883243084]], [[0.4998229146]], [[0.7064986825]], [[1.0706689358]], [[1.8342959881]], [[1.0476934910]], [[1.5373592377]], [[1.0360078812]], [[1.0707864761]], [[0.3940286636]], [[0.6628341675]], [[1.8462545872]], [[1.2278873920]], [[0.4544646740]], [[1.1460801363]], [[1.1235973835]], [[0.6782103777]], [[0.5204688907]], [[1.4165614843]], [[0.4917340279]], [[0.9354569316]], [[1.0117805004]], [[1.4885549545]], [[1.3919384480]], [[0.1944850683]], [[0.3329601288]]]])
test_add.test_module[Add1-[((1, 576, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.8284466863]], [[1.1004118919]], [[0.4206675291]], [[0.4642205834]], [[0.6396129131]], [[0.9662687778]], [[0.8222835064]], [[1.2286348343]], [[0.7878180742]], [[0.9644963741]], [[0.6810835600]], [[0.7339074016]], [[0.3545158505]], [[0.5010490417]], [[0.6260785460]], [[0.8507118821]], [[1.0298576355]], [[1.1322014332]], [[0.4932195544]], [[0.6144586802]], [[1.0137987137]], [[1.2473840714]], [[0.7292900085]], [[1.2063460350]], [[0.7515984178]], [[0.8850971460]], [[1.2849282026]], [[0.3683549166]], [[0.5174211264]], [[0.7056074739]], [[0.6372901201]], [[1.2641904354]], [[0.5081002712]], [[0.6020236611]], [[0.4828698635]], [[0.3639096022]], [[0.5403198600]], [[1.2619891167]], [[1.0552992821]], [[1.0745263100]], [[0.8584858775]], [[0.5758483410]], [[0.9167824388]], [[0.3653427362]], [[0.4709069729]], [[0.5744251013]], [[1.1476590633]], [[1.1253507137]], [[0.6104425788]], [[0.8141489029]], [[1.1519703865]], [[1.3292566538]], [[1.0306311846]], [[0.8997365236]], [[1.1674332619]], [[0.5377889276]], [[0.9253621101]], [[0.4445373416]], [[0.4856470227]], [[0.5738983154]], [[1.0584266186]], [[1.0332703590]], [[0.5360138416]], [[0.9832436442]], [[1.1066761017]], [[0.7690814137]], [[0.8512808681]], [[0.9480424523]], [[1.1423783302]], [[1.3122870922]], [[0.4468783140]], [[0.6489552259]], [[1.0286951065]], [[1.2464647293]], [[1.2672936916]], [[1.2733684778]], [[0.9316973686]], [[0.3973987699]], [[0.8781863451]], [[0.5193874240]], [[0.3662130237]], [[1.2764363289]], [[1.2123699188]], [[0.3334261179]], [[0.9257761240]], [[0.7479600906]], [[0.7499095201]], [[0.6033116579]], [[1.0244681835]], [[0.5360383391]], [[1.0154857635]], [[1.0850441456]], [[1.1901259422]], [[1.0191457272]], [[0.3373224735]], [[0.5078416467]], [[1.0818476677]], [[0.9368407726]], [[0.4421480894]], [[0.5442803502]], [[1.3025647402]], [[1.1690990925]], [[0.6141775250]], [[0.7063477039]], [[0.3558910489]], [[0.8232030272]], [[0.4556606412]], [[0.4465117455]], [[0.8046402931]], [[0.9072626233]], [[0.6274249554]], [[1.1288788319]], [[0.5279205441]], [[1.2858750820]], [[1.1748399734]], [[0.4105486274]], [[0.7077479362]], [[0.8547514081]], [[0.9051406384]], [[0.9507772326]], [[1.0284042358]], [[0.8621401787]], [[0.5882257223]], [[1.0687845945]], [[0.3525656462]], [[0.5358367562]], [[0.7070251703]], [[0.5886334181]], [[0.6572734118]], [[0.4223792553]], [[0.7258325219]], [[0.9390683174]], [[0.5064572096]], [[0.8065304160]], [[1.1901154518]], [[0.7807899714]], [[0.8460862041]], [[0.7890556455]], [[0.9333807826]], [[1.1501097679]], [[1.3058131933]], [[1.1497180462]], [[1.3068969250]], [[0.7960292697]], [[0.3830293417]], [[0.5951514840]], [[1.1726427078]], [[0.8289488554]], [[0.5836669207]], [[0.4490342140]], [[0.3642640710]], [[0.4101859927]], [[0.7307717204]], [[1.1063930988]], [[1.1025106907]], [[0.3499741554]], [[1.1440811157]], [[0.4409353733]], [[0.7264849544]], [[0.6294537783]], [[0.7358824611]], [[0.7340187430]], [[0.3835151196]], [[0.4004711509]], [[0.7539503574]], [[0.8386561871]], [[0.6050526500]], [[1.0205397606]], [[0.3821609020]], [[0.7984464765]], [[1.2718998194]], [[0.6282441020]], [[1.2836916447]], [[1.0132670403]], [[0.3809596300]], [[1.1485388279]], [[0.7744928598]], [[0.6089866757]], [[1.2320168018]], [[0.4281424880]], [[0.8858425617]], [[0.7275057435]], [[1.1892464161]], [[0.9717623591]], [[1.0724427700]], [[1.0087695122]], [[0.7119527459]], [[0.7270373702]], [[0.4201493859]], [[1.1031122208]], [[1.2291791439]], [[1.1743025780]], [[0.4795009494]], [[0.8544900417]], [[0.4797230363]], [[0.5569480062]], [[0.5408373475]], [[1.0030626059]], [[0.5342335105]], [[0.8212814927]], [[0.8532241583]], [[1.1545016766]], [[0.4542300701]], [[0.4889339805]], [[0.5418593287]], [[1.1821568012]], [[0.6524575949]], [[1.2539343834]], [[1.0129938126]], [[0.8955031037]], [[0.8284680843]], [[0.7333493233]], [[0.8949232697]], [[0.7180177569]], [[0.8286768794]], [[0.8959866166]], [[0.4410875440]], [[0.5701243877]], [[1.2359364033]], [[0.4264167547]], [[0.7962870002]], [[1.3268094063]], [[1.0128085613]], [[0.8463466167]], [[0.3988851309]], [[1.0798790455]], [[0.4760497212]], [[0.6902579665]], [[0.6644319296]], [[0.7581464052]], [[0.8376592398]], [[1.2445938587]], [[0.8946095109]], [[1.2800364494]], [[1.1380462646]], [[0.5160831213]], [[1.0564421415]], [[0.4787420630]], [[0.6202775240]], [[0.9792514443]], [[0.9972861409]], [[1.2073041201]], [[0.6712321639]], [[0.8329905272]], [[1.0896018744]], [[0.3486440182]], [[1.1936804056]], [[0.4187288880]], [[0.8390813470]], [[0.7471817136]], [[0.5688536167]], [[0.8982756138]], [[1.2456494570]], [[0.6860303283]], [[0.5353430510]], [[0.6472706795]], [[0.3366159201]], [[1.0578870773]], [[0.5920582414]], [[0.4985199571]], [[0.5441393852]], [[1.1196681261]], [[1.0969769955]], [[1.2159509659]], [[1.0135517120]], [[0.6652102470]], [[0.6924660206]], [[0.9799050689]], [[1.2432074547]], [[0.9681361914]], [[0.5956174135]], [[0.5971513987]], [[0.3594555855]], [[0.9402140975]], [[0.5515988469]], [[0.3864021897]], [[1.2706283331]], [[0.5074844956]], [[0.7753093243]], [[0.9754368663]], [[0.8481192589]], [[0.4957492352]], [[0.4280292392]], [[1.2307312489]], [[0.9136094451]], [[1.2470023632]], [[0.6645697951]], [[0.9794678092]], [[0.7178519964]], [[0.8099545240]], [[0.5276679993]], [[1.0012905598]], [[0.9902730584]], [[0.8218758106]], [[0.7197390199]], [[0.5239751935]], [[1.1779625416]], [[0.4599780440]], [[1.0370228291]], [[0.6640633345]], [[0.5909560919]], [[0.9220126271]], [[0.5724616051]], [[0.9473925233]], [[0.9303805232]], [[0.4609453678]], [[0.9154393673]], [[1.0451546907]], [[1.0301055908]], [[0.7692507505]], [[0.4222962856]], [[0.7551130652]], [[1.0058438778]], [[0.6497515440]], [[1.0219806433]], [[1.1651713848]], [[0.5711143017]], [[0.8371210098]], [[1.0389395952]], [[0.8713790178]], [[0.8739558458]], [[0.8946228027]], [[0.4391162395]], [[0.8714842200]], [[1.1784250736]], [[1.2827470303]], [[1.1260654926]], [[0.8991916180]], [[1.0656971931]], [[0.5889502764]], [[0.4178459048]], [[0.4022295475]], [[1.3309949636]], [[1.1495847702]], [[0.4865732193]], [[1.0278114080]], [[1.2097738981]], [[1.3319975138]], [[1.2693874836]], [[1.2195670605]], [[0.7175604701]], [[0.6567192674]], [[1.2427145243]], [[1.1123799086]], [[0.5313006043]], [[1.2816987038]], [[1.0737693310]], [[1.1047577858]], [[0.5188019276]], [[0.9756399989]], [[0.6569088697]], [[1.2228684425]], [[0.7422198057]], [[1.0268464088]], [[0.9210010171]], [[1.0449242592]], [[0.6622763276]], [[1.0759757757]], [[0.4829546213]], [[0.9451175928]], [[0.4938941598]], [[0.3389213085]], [[0.4306622148]], [[1.2269251347]], [[1.1027078629]], [[1.3012979031]], [[1.2327506542]], [[0.3856673837]], [[0.4909745455]], [[0.7513988018]], [[0.5074739456]], [[1.1793882847]], [[0.4541862011]], [[0.5882393718]], [[0.3491443992]], [[0.5483173132]], [[1.2434288263]], [[1.2415701151]], [[1.1901090145]], [[1.2182537317]], [[1.2767833471]], [[0.7041586041]], [[1.0521911383]], [[1.2776553631]], [[0.9975994825]], [[1.3320282698]], [[1.0915215015]], [[1.1430141926]], [[0.6571940184]], [[1.0721038580]], [[0.8896433711]], [[0.7127814293]], [[0.5503351688]], [[0.5516318083]], [[0.4474496841]], [[1.1678552628]], [[1.1876556873]], [[0.7752837539]], [[0.5428470969]], [[1.2186428308]], [[1.1519346237]], [[0.8693571091]], [[0.5961224437]], [[1.2917139530]], [[1.0366604328]], [[0.4526174068]], [[1.3107315302]], [[1.2118769884]], [[0.6499446630]], [[1.1132674217]], [[0.5480955839]], [[0.7538329363]], [[1.2567406893]], [[0.8528496623]], [[0.4785810113]], [[0.6650736332]], [[0.6964675188]], [[0.7357257009]], [[0.8800404072]], [[1.2946048975]], [[0.8589602709]], [[0.5234701037]], [[0.8578172326]], [[1.0719337463]], [[1.0802102089]], [[0.3752208352]], [[0.7427188754]], [[0.4606178403]], [[0.6188473105]], [[1.0123367310]], [[0.4771249890]], [[1.0180535316]], [[1.2565791607]], [[0.8649842739]], [[0.4989462495]], [[0.6530476213]], [[0.9413721561]], [[0.4510310888]], [[1.0805952549]], [[0.3782553077]], [[0.3515434265]], [[0.3463597894]], [[0.7307584286]], [[1.1684064865]], [[0.3589506149]], [[1.2477836609]], [[0.6321786642]], [[0.9786321521]], [[0.8549915552]], [[0.3813305497]], [[1.2468545437]], [[1.1014118195]], [[1.3291878700]], [[1.0847961903]], [[0.5021556020]], [[1.2494819164]], [[0.8590623140]], [[1.0692982674]], [[0.4312755466]], [[0.6883768439]], [[0.3412513137]], [[0.6374438405]], [[0.9400556684]], [[0.4396092296]], [[0.9915722013]], [[1.1005935669]], [[0.9018447399]], [[0.4976484179]], [[0.4445302486]], [[0.6779317856]], [[1.0516691208]], [[1.3253883123]], [[1.1197046041]], [[0.7758854628]], [[1.0074982643]], [[0.3416586518]], [[0.4051394463]], [[1.0654940605]], [[0.5489825010]], [[1.0727379322]], [[0.4792240262]], [[0.5845346451]], [[0.4203456044]], [[1.0931104422]], [[0.7812425494]], [[1.2169907093]], [[1.1416261196]], [[1.1088614464]], [[0.8482681513]], [[0.6776010990]], [[0.7234742641]], [[0.8986403942]], [[1.0800448656]], [[0.4818951488]], [[1.2518290281]], [[0.7778250575]], [[0.4132173061]], [[0.5616611838]], [[1.2745989561]], [[1.2894537449]], [[0.3690502644]], [[1.1848392487]], [[1.0827696323]], [[1.1281492710]], [[1.2554547787]], [[0.5627149343]], [[0.9900779724]], [[1.0368067026]], [[0.6844434142]], [[0.9995166659]], [[0.6883333921]], [[1.1413207054]], [[0.6934627891]], [[0.6457922459]], [[0.9580646157]], [[1.0095369816]], [[0.5879042745]], [[0.8763892651]], [[1.1219580173]], [[0.7824417353]], [[0.9843570590]], [[0.7115966082]], [[1.0074398518]], [[0.4699929953]], [[0.5381757021]], [[0.5783948302]], [[1.2916955948]], [[0.6976480484]], [[0.8308249116]], [[0.5899423361]], [[1.3313350677]], [[1.3205251694]], [[0.4550967813]], [[0.4268551469]], [[0.4531936049]], [[0.8297787309]], [[0.7047381401]], [[0.5049172640]], [[0.6528540254]], [[0.9266555309]], [[0.5709447265]], [[0.9429813623]], [[0.7175321579]], [[0.5899084210]], [[0.9008828402]], [[1.2433192730]], [[0.4941504598]], [[0.8554072976]], [[0.6478062272]], [[1.3228433132]], [[0.3578084707]], [[0.3528594971]], [[1.3248825073]], [[0.5158451200]], [[0.9280517697]], [[0.7890370488]], [[0.7268615961]], [[0.7205162048]], [[1.1499104500]], [[0.8560885787]], [[0.3453828096]], [[0.5370100141]], [[0.6617399454]], [[1.0837982893]], [[0.5086232424]], [[1.3036499023]], [[0.7208254337]], [[0.7424091101]], [[1.2239691019]], [[1.0834939480]], [[1.2562502623]], [[1.1214122772]], [[0.6805131435]], [[0.5004538894]], [[0.7949885726]], [[1.2460088730]]]]), compiled_model=tensor([[[[0.8284466863]], [[0.8045368195]], [[0.7934416533]], [[1.1187661886]], [[0.6650812030]], [[0.7200669050]], [[0.5365635753]], [[1.5217444897]], [[0.9177681208]], [[0.8798126578]], [[0.9499593973]], [[1.0915967226]], [[0.9199852347]], [[1.0570499897]], [[0.7190465927]], [[0.5776382089]], [[0.7458560467]], [[1.7668561935]], [[0.8820570707]], [[1.0002222061]], [[0.7489980459]], [[1.8782018423]], [[1.3707714081]], [[1.8255860806]], [[0.4976158142]], [[0.8642453551]], [[1.1088562012]], [[1.0096476078]], [[0.4703910351]], [[0.6451401114]], [[1.0670564175]], [[1.2007042170]], [[0.4296491146]], [[0.7260978222]], [[0.6026200056]], [[0.1422302723]], [[1.1249637604]], [[1.2092381716]], [[1.4004666805]], [[1.6772280931]], [[1.2784540653]], [[0.8144481778]], [[1.5100208521]], [[0.6003657579]], [[0.4073885679]], [[1.2152347565]], [[1.4338092804]], [[0.8053202033]], [[0.6359066963]], [[0.6413750648]], [[1.7581567764]], [[1.4144935608]], [[0.7426786423]], [[1.0361146927]], [[1.6492495537]], [[0.8354990482]], [[1.2512742281]], [[0.6587838531]], [[0.8398662806]], [[0.6198822260]], [[1.0273363590]], [[0.7337259650]], [[0.3271552920]], [[1.3677548170]], [[0.9784293175]], [[1.0086245537]], [[1.1786864996]], [[1.1512542963]], [[0.9860134125]], [[1.9582612514]], [[0.3239384890]], [[1.2280154228]], [[0.7987508178]], [[1.2940003872]], [[1.7070879936]], [[1.2368880510]], [[1.5195287466]], [[0.2211304307]], [[0.6261104941]], [[0.4617377520]], [[0.6148481369]], [[1.9046242237]], [[1.1414794922]], [[0.6800501347]], [[0.9682325125]], [[0.8073388338]], [[1.2853710651]], [[0.3836286068]], [[1.2453846931]], [[1.1740528345]], [[1.1145895720]], [[1.6410731077]], [[1.2039573193]], [[1.5894366503]], [[0.0214672089]], [[0.6035897136]], [[1.1618342400]], [[1.2666931152]], [[0.8061217070]], [[1.0959987640]], [[1.3958816528]], [[1.3170814514]], [[1.1243929863]], [[0.7388710380]], [[0.9620177746]], [[0.6581068635]], [[0.5693656206]], [[0.5874661803]], [[1.1955552101]], [[1.4169106483]], [[0.7159829736]], [[0.8824484944]], [[0.9434765577]], [[1.6032509804]], [[1.5434995890]], [[0.2699247003]], [[1.1973556280]], [[1.4961388111]], [[1.1163362265]], [[0.6515629292]], [[1.5472090244]], [[0.6592244506]], [[0.8709716797]], [[1.3092534542]], [[0.2863572836]], [[0.8777444363]], [[0.4276187420]], [[0.8703230619]], [[0.5081078410]], [[0.5361223221]], [[0.9579568505]], [[1.5328505039]], [[0.4357069731]], [[1.2946584225]], [[1.2944085598]], [[0.7111359835]], [[0.5784962773]], [[0.4981063604]], [[1.5894960165]], [[1.1932199001]], [[1.4985975027]], [[1.4530841112]], [[1.8145728111]], [[1.3905762434]], [[0.9563478231]], [[0.3925198317]], [[1.2603442669]], [[0.7008899450]], [[0.4658009410]], [[0.7354476452]], [[1.0013979673]], [[0.1774439216]], [[1.2011837959]], [[1.0149695873]], [[1.1729333401]], [[0.9146695137]], [[1.1988058090]], [[0.6542731524]], [[0.5447785854]], [[1.2228841782]], [[0.8390574455]], [[0.5361334085]], [[0.6977301836]], [[0.2127723694]], [[0.5250065923]], [[1.0369256735]], [[1.1692843437]], [[1.0468575954]], [[0.7853042483]], [[1.3959169388]], [[1.7713401318]], [[0.5337734818]], [[1.3966915607]], [[1.0237344503]], [[0.1467080712]], [[1.3165552616]], [[1.3185203075]], [[1.1980543137]], [[1.4464769363]], [[0.7095139027]], [[0.8371916413]], [[1.2727367878]], [[1.1490020752]], [[0.7922458649]], [[1.3172494173]], [[1.4762721062]], [[0.4289668798]], [[1.3468310833]], [[0.7678248286]], [[0.9206156731]], [[1.2892853022]], [[1.7759023905]], [[0.2636935115]], [[0.8761613369]], [[0.8115230799]], [[0.2867153883]], [[0.9827463627]], [[1.4311463833]], [[1.0121462345]], [[0.6703178287]], [[1.5190346241]], [[1.0259307623]], [[1.1212102175]], [[0.1768986583]], [[0.2641842365]], [[1.6570649147]], [[0.8725239635]], [[1.4505889416]], [[0.9039245844]], [[0.8535752892]], [[0.8500842452]], [[0.4140815139]], [[1.0887174606]], [[0.9742571115]], [[0.9960634708]], [[1.2252616882]], [[1.0833230019]], [[0.8708763123]], [[1.2207101583]], [[0.3884533048]], [[0.6441941261]], [[1.1480100155]], [[1.1000936031]], [[0.9257237315]], [[0.7891262770]], [[1.0339705944]], [[1.0424613953]], [[0.5072277784]], [[0.8336629272]], [[1.3754162788]], [[1.5026617050]], [[1.1227726936]], [[1.1514838934]], [[1.5069077015]], [[1.0714304447]], [[0.5111455917]], [[1.3596789837]], [[0.2988693714]], [[0.8705852628]], [[1.3634237051]], [[0.9680609703]], [[1.7904345989]], [[0.8061368465]], [[1.2276568413]], [[1.7525649071]], [[0.3636197448]], [[1.6331944466]], [[0.4435241818]], [[0.9338547587]], [[0.8302560449]], [[0.7335572243]], [[0.8772013187]], [[1.5306527615]], [[0.8726804256]], [[1.0200978518]], [[0.7138779163]], [[0.5545659065]], [[1.0397018194]], [[0.3411412835]], [[0.8686634898]], [[0.7759282589]], [[1.0872361660]], [[1.0957343578]], [[1.5145215988]], [[1.0909531116]], [[1.1597580910]], [[0.8885974884]], [[1.3365833759]], [[1.6288654804]], [[1.0132629871]], [[0.9797597528]], [[1.1339279413]], [[0.5502746701]], [[1.2058703899]], [[0.7375765443]], [[0.9003958106]], [[1.2273405790]], [[0.4095094800]], [[1.1609959602]], [[0.7080897093]], [[1.0239453316]], [[0.4336135983]], [[0.9258559346]], [[0.9330216646]], [[1.3845856190]], [[1.9094905853]], [[0.9335497618]], [[1.1139643192]], [[1.3778368235]], [[0.7640882730]], [[0.6554111242]], [[0.9501479864]], [[1.0890623331]], [[1.0983551741]], [[0.9543296099]], [[0.5973476171]], [[0.8458520770]], [[0.6533899903]], [[1.1900217533]], [[0.9070916176]], [[1.1418911219]], [[1.5758272409]], [[0.4436302781]], [[1.0840251446]], [[0.8978536129]], [[0.1679324508]], [[0.7194197178]], [[1.6822451353]], [[1.4894460440]], [[1.2056362629]], [[0.8679777980]], [[0.5256834626]], [[1.2261633873]], [[1.2829186916]], [[0.9107475877]], [[1.7777783871]], [[0.8703434467]], [[1.3567026854]], [[0.9925556779]], [[1.2696977854]], [[0.5985218883]], [[1.0306849480]], [[0.7737017274]], [[1.1892414093]], [[1.7646389008]], [[1.9418854713]], [[1.7487540245]], [[1.4026975632]], [[1.5850381851]], [[0.6922453046]], [[0.1976323724]], [[0.3844258785]], [[1.4426726103]], [[1.0472013950]], [[0.9099685550]], [[1.3711349964]], [[1.4582598209]], [[1.6205878258]], [[1.8764235973]], [[1.5694925785]], [[0.4469006658]], [[0.4615888000]], [[1.6308991909]], [[1.3483294249]], [[0.9429563284]], [[0.9501486421]], [[0.7801446319]], [[1.5791809559]], [[1.0064346790]], [[0.6910943985]], [[1.0145113468]], [[1.0002561808]], [[1.2886836529]], [[1.3515515327]], [[1.5832043886]], [[0.7197381258]], [[0.3487481475]], [[1.5818367004]], [[1.0946760178]], [[1.4192674160]], [[0.9941189885]], [[0.0855955482]], [[0.9603789449]], [[0.9226614833]], [[1.3822185993]], [[1.1415904760]], [[1.2019629478]], [[0.6420117021]], [[0.4621944427]], [[0.8400972486]], [[0.2013716698]], [[0.8757616282]], [[0.8156776428]], [[0.5444100499]], [[0.2127594948]], [[0.5188360810]], [[1.0024545193]], [[1.0096799135]], [[1.6571049690]], [[1.6401846409]], [[1.5111949444]], [[0.5988386869]], [[1.3863515854]], [[1.4679278135]], [[0.8080729842]], [[1.6074330807]], [[1.7146034241]], [[0.8190209866]], [[0.3278078437]], [[1.3039762974]], [[1.4339644909]], [[0.5763464570]], [[0.8282136321]], [[1.0709342957]], [[0.7694177628]], [[1.4438705444]], [[1.8543286324]], [[1.2688300610]], [[0.4294434786]], [[1.4296710491]], [[1.5751806498]], [[1.4781217575]], [[0.2664659023]], [[1.7302765846]], [[1.0593044758]], [[0.1308166981]], [[1.5895284414]], [[1.1074800491]], [[0.8886106610]], [[0.9564538002]], [[0.3396409750]], [[0.8792173266]], [[1.2565877438]], [[1.3130919933]], [[0.6894999743]], [[1.2231628895]], [[0.9579789042]], [[0.7427549362]], [[1.3865170479]], [[1.4897372723]], [[1.1206159592]], [[0.5304687023]], [[1.0274214745]], [[1.1436070204]], [[1.2055114508]], [[0.1530910730]], [[0.9074538350]], [[0.2134056091]], [[0.6790918112]], [[1.4699198008]], [[0.2972597480]], [[0.8488375545]], [[1.3023008108]], [[0.8957271576]], [[0.7513998747]], [[0.9038653374]], [[1.3627302647]], [[0.2744007111]], [[1.2131195068]], [[0.8257108331]], [[0.9518390894]], [[0.4776969552]], [[0.6092582345]], [[1.4799478054]], [[0.8114991784]], [[1.1032185555]], [[0.5098418593]], [[1.3674411774]], [[0.9877684712]], [[0.0768911242]], [[1.1263719797]], [[1.4719882011]], [[1.3274099827]], [[1.5977206230]], [[1.0652353764]], [[1.4900268316]], [[0.9873348475]], [[1.0783946514]], [[0.5738025308]], [[0.9471971989]], [[0.1271356940]], [[0.6857559085]], [[0.6919651031]], [[0.9142965674]], [[0.8409639597]], [[1.7251753807]], [[0.9407642484]], [[0.3790206909]], [[0.8525667191]], [[0.9202812314]], [[1.5657070875]], [[1.7019331455]], [[0.8058447838]], [[1.2599406242]], [[1.0811512470]], [[0.2884830236]], [[0.8904762864]], [[1.5979690552]], [[0.2773270607]], [[1.1953361034]], [[1.0576372147]], [[0.9459879398]], [[1.0093901157]], [[1.0895709991]], [[0.6732312441]], [[1.8147554398]], [[1.5178475380]], [[1.7566626072]], [[0.8072297573]], [[0.5243775845]], [[0.8326777220]], [[0.5956603289]], [[1.4438424110]], [[1.0184992552]], [[1.5396673679]], [[0.8962315321]], [[0.8289303780]], [[0.4120954275]], [[1.9315304756]], [[0.9600917697]], [[0.0578875542]], [[1.2344982624]], [[1.6590018272]], [[1.3460221291]], [[1.6152875423]], [[0.3640214205]], [[1.3402307034]], [[1.1487448215]], [[1.0526651144]], [[1.5204478502]], [[1.0734834671]], [[1.2665985823]], [[0.8304731250]], [[0.5000120401]], [[0.9450234771]], [[1.5022659302]], [[0.5552158952]], [[1.3547190428]], [[1.0915019512]], [[0.8338022232]], [[1.1628080606]], [[0.4205737710]], [[1.1702573299]], [[0.5874050260]], [[0.6610674262]], [[0.6462526321]], [[1.8536940813]], [[1.2344515324]], [[0.6597561836]], [[0.9900156260]], [[1.1069537401]], [[1.0626744032]], [[0.7757476568]], [[0.6020132899]], [[0.3877503276]], [[0.5153367519]], [[0.6789799929]], [[0.8397269249]], [[0.3578631878]], [[0.6087327003]], [[0.8021908998]], [[0.7505891919]], [[0.4473019242]], [[0.5650752187]], [[0.9881923199]], [[0.9567764997]], [[0.3280965090]], [[1.0560011864]], [[0.3441302776]], [[1.4203759432]], [[0.2027670145]], [[0.1740507483]], [[1.6892080307]], [[0.2988101840]], [[0.7169445753]], [[1.2419877052]], [[0.9881546497]], [[1.3436825275]], [[1.1840724945]], [[0.7823909521]], [[0.4669194818]], [[0.7073495984]], [[1.2108719349]], [[0.7535682321]], [[0.6408327818]], [[1.6926358938]], [[0.6132938266]], [[0.6561486125]], [[1.6334128380]], [[0.7722112536]], [[1.5425380468]], [[0.9021289349]], [[1.0968539715]], [[0.2407435775]], [[1.2702008486]], [[0.9405024648]]]])
test_add.test_module[Add1-[((1, 1024, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.9612236023]], [[1.2331888676]], [[0.5534444451]], ..., [[0.6525920630]], [[0.6748203039]], [[1.1859661341]]]]), compiled_model=tensor([[[[0.9612236023]], [[1.1095082760]], [[0.5631945133]], ..., [[2.1295104027]], [[1.9586073160]], [[2.1236968040]]]])
test_advindex.test_module[Advindex1-[((1, 1, 1024, 72), torch.float32)]]: forge/test/models_ops/test_advindex.py#L34
IndexError: index 758 is out of bounds for dimension 0 with size 1
test_advindex.test_module[Advindex2-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L47
IndexError: index 792 is out of bounds for dimension 0 with size 169
test_advindex.test_module[Advindex4-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L73
IndexError: index 989 is out of bounds for dimension 0 with size 169
test_argmax.test_module[Argmax0-[((1, 32), torch.int32)]]: forge/test/models_ops/test_argmax.py#L84
RuntimeError: Generated MLIR module failed verification.
test_avgpool2d.test_module[Avgpool2D9-[((1, 384, 35, 35), torch.float32)]]: forge/test/models_ops/test_avgpool2d.py#L1472
RuntimeError: Tensor 1 - stride mismatch: expected [1225, 1], got [0, 0]
test_avgpool2d.test_module[Avgpool2D9-[((1, 1024, 17, 17), torch.float32)]]: forge/test/models_ops/test_avgpool2d.py#L1472
RuntimeError: Tensor 1 - stride mismatch: expected [289, 1], got [0, 0]
test_broadcast.test_module[Broadcast1-[((1, 1, 1, 384), torch.bool)]]: forge/test/models_ops/test_broadcast.py#L172
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((2, 1, 1, 13), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427., 503., 497., 683., 101., 866., 756.]]], [[[399., 878., 376., 56., 868., 794., 33., 126., 119., 391., 254., 824., 841.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760., 0., 963., 0., 379., 0., 427.]]], [[[ 0., 503., 0., 497., 0., 683., 0., 101., 0., 866., 0., 756., 0.]]]])
test_cast.test_module[Cast0-[((2, 1, 7, 7), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427.], [503., 497., 683., 101., 866., 756., 399.], [878., 376., 56., 868., 794., 33., 126.], [119., 391., 254., 824., 841., 269., 969.], [549., 480., 481., 12., 363., 360., 995.], [ 85., 822., 999., 711., 288., 778., 43.], [896., 489., 771., 757., 283., 395., 782.]]], [[[671., 840., 969., 673., 441., 111., 280.], [503., 706., 476., 327., 699., 326., 363.], [474., 775., 700., 218., 632., 568., 12.], [277., 645., 110., 80., 348., 521., 119.], [916., 881., 490., 682., 519., 944., 633.], [169., 863., 409., 233., 219., 78., 735.], [ 83., 222., 858., 235., 416., 446., 835.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760.], [ 0., 963., 0., 379., 0., 427., 0.], [503., 0., 497., 0., 683., 0., 101.], [ 0., 866., 0., 756., 0., 399., 0.], [878., 0., 376., 0., 56., 0., 868.], [ 0., 794., 0., 33., 0., 126., 0.], [119., 0., 391., 0., 254., 0., 824.]]], [[[ 0., 841., 0., 269., 0., 969., 0.], [549., 0., 480., 0., 481., 0., 12.], [ 0., 363., 0., 360., 0., 995., 0.], [ 85., 0., 822., 0., 999., 0., 711.], [ 0., 288., 0., 778., 0., 43., 0.], [896., 0., 489., 0., 771., 0., 757.], [ 0., 283., 0., 395., 0., 782., 0.]]]])
test_cast.test_module[Cast0-[((1, 1, 256, 256), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., ..., 374., 973., 703.], [757., 943., 848., ..., 762., 629., 216.], [606., 763., 616., ..., 786., 391., 916.], ..., [889., 108., 668., ..., 29., 931., 519.], [326., 328., 926., ..., 422., 582., 607.], [867., 283., 592., ..., 712., 264., 295.]]]]), compiled_model=tensor([[[[ 44., 0., 239., ..., 0., 389., 0.], [793., 0., 611., ..., 0., 703., 0.], [757., 0., 943., ..., 0., 370., 0.], ..., [178., 0., 972., ..., 0., 393., 0.], [274., 0., 790., ..., 0., 874., 0.], [869., 0., 258., ..., 0., 375., 0.]]]])
test_cast.test_module[Cast1-[((1, 1, 256, 256), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 128), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast2-[((1, 128), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 384), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast2-[((1, 384), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 12, 384, 384), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 1, 256, 256), torch.int32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast1-[((1, 1, 7, 7), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 1, 1, 7), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760.]]]])
test_cast.test_module[Cast0-[((1, 1, 32, 32), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., ..., 480., 481., 12.], [363., 360., 995., ..., 326., 363., 474.], [775., 700., 218., ..., 858., 235., 416.], ..., [326., 721., 442., ..., 346., 346., 564.], [386., 447., 788., ..., 593., 762., 312.], [714., 724., 526., ..., 122., 978., 438.]]]]), compiled_model=tensor([[[[ 44., 0., 239., ..., 0., 376., 0.], [ 56., 0., 868., ..., 0., 12., 0.], [363., 0., 360., ..., 0., 395., 0.], ..., [108., 0., 590., ..., 0., 827., 0.], [556., 0., 895., ..., 0., 746., 0.], [403., 0., 958., ..., 0., 216., 0.]]]])
test_cast.test_module[Cast1-[((1, 16, 320, 1024), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_conv2d.test_module[Conv2D1-[((1, 1024, 3000, 1), torch.float32), ((1024, 1024, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[497.3341369629], [752.2816772461], [745.8151855469], ..., [748.7922973633], [751.2034301758], [758.1015014648]], [[516.6516723633], [763.2521972656], [751.9768066406], ..., [776.1329956055], [769.6298217773], [756.2926635742]], [[508.2720336914], [745.5457153320], [746.0203247070], ..., [750.8545532227], [748.2382202148], [741.9406738281]], ..., [[510.7730407715], [747.9530029297], [756.9498901367], ..., [774.0913085938], [777.7090454102], [763.4789428711]], [[517.1864013672], [770.1347045898], [773.8180541992], ..., [780.8870849609], [781.6174926758], [774.4715576172]], [[512.4339599609], [756.8847656250], [759.8025512695], ..., [763.7913818359], [769.0996704102], [768.2415771484]]]]), compiled_model=tensor([[[[498.], [760.], [740.], ..., [752.], [764.], [752.]], [[520.], [776.], [744.], ..., [784.], [780.], [768.]], [[510.], [764.], [744.], ..., [760.], [756.], [740.]], ..., [[502.], [760.], [760.], ..., [764.], [784.], [756.]], [[516.], [780.], [784.], ..., [776.], [788.], [780.]], [[504.], [768.], [756.], ..., [764.], [776.], [768.]]]])
test_conv2d.test_module[Conv2D1-[((1, 384, 3000, 1), torch.float32), ((384, 384, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7f823925b288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7f82278e97a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7f82278dcfc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7f8227943829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7f8229abafda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7f8229abb3e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7f8229aba72b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f8229aba3e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f8227942b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f82278da6bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f82278c9f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30]
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((512, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7f823925b288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7f82278e97a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7f82278dcfc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7f8227943829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7f8229abafda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7f8229abb3e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7f8229aba72b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f8229aba3e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f8227942b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f82278da6bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f82278c9f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30]
test_conv2d.test_module[Conv2D1-[((1, 768, 3000, 1), torch.float32), ((768, 768, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7f823925b288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7f82278e97a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7f82278dcfc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7f8227943829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7f8229abafda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7f8229abb3e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7f8229aba72b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f8229aba3e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f8227942b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f82278da6bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f82278c9f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30]
test_conv2d.test_module[Conv2D2-[((1, 3072, 1, 128), torch.float32), ((768, 768, 1, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[199.4852752686, 195.3149871826, 186.9613647461, ..., 181.2562255859, 199.4642333984, 188.8912200928]], [[200.4397735596, 203.4648284912, 191.3027954102, ..., 190.6445617676, 194.6260070801, 197.4597930908]], [[198.7064208984, 199.6289825439, 191.5614471436, ..., 185.8805847168, 193.2582244873, 191.8967285156]], ..., [[185.1807556152, 179.1775054932, 192.7327270508, ..., 187.5320434570, 195.1790008545, 180.5238952637]], [[198.3394775391, 193.2522277832, 207.2843322754, ..., 204.6146240234, 208.7871246338, 195.6759033203]], [[194.1520996094, 185.4377593994, 198.5037689209, ..., 191.9805755615, 198.8300933838, 187.4386749268]]]]), compiled_model=tensor([[[[196., 192., 187., ..., 179., 193., 189.]], [[200., 197., 189., ..., 187., 193., 194.]], [[194., 196., 187., ..., 183., 193., 191.]], ..., [[183., 174., 191., ..., 187., 194., 179.]], [[195., 191., 201., ..., 200., 207., 196.]], [[192., 182., 197., ..., 188., 195., 184.]]]])
test_conv2d.test_module[Conv2D4-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/program/program.cpp:905: tt::exception info: Statically allocated circular buffers in program 171680 clash with L1 buffers on core range [(x=0,y=0) - (x=7,y=5)]. L1 buffer allocated at 947328 and static circular buffer region ends at 1180896 backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x1712d5) [0x7f82386b52d5] --- tt::tt_metal::detail::Program_::validate_circular_buffer_region(tt::tt_metal::v0::IDevice const*) --- tt::tt_metal::v0::EnqueueProgram(tt::tt_metal::CommandQueue&, tt::tt_metal::v0::Program&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x336) [0x7f8229aba7b6] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f8229aba3e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f8227942b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f82278da6bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f82278c9f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702]
test_conv2d.test_module[Conv2D9-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/program/program.cpp:896: tt::exception info: Statically allocated circular buffers on core range [(x=0,y=0) - (x=7,y=5)] grow to 1676512 B which is beyond max L1 size of 1499136 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x17129b) [0x7f82386b529b] --- tt::tt_metal::detail::Program_::validate_circular_buffer_region(tt::tt_metal::v0::IDevice const*) --- tt::tt_metal::v0::EnqueueProgram(tt::tt_metal::CommandQueue&, tt::tt_metal::v0::Program&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x336) [0x7f8229aba7b6] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7f8229aba3e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7f8227942b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7f82278da6bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7f82278c9f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702]
test_conv2d.test_module[Conv2D15-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7f823869e7e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7f8238669e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7f82284f9833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7f82284f9699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7f8229aba39a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7f82284f869a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7f8227930f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7f82278c9d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464]
test_conv2d.test_module[Conv2D16-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7f823869e7e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7f8238669e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7f82284f9833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7f82284f9699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7f8229aba39a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7f82284f869a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7f8227930f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7f82278c9d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464]
test_conv2d.test_module[Conv2D521-[((1, 192, 17, 17), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[416.8596496582, 432.2359924316, 431.2136535645, ..., 435.4002685547, 429.5439758301, 438.3637695312], [438.0344238281, 434.8787841797, 447.7115173340, ..., 427.4629516602, 435.3717651367, 431.3978271484], [440.0999755859, 435.0092468262, 443.9841003418, ..., 433.8666992188, 437.3807983398, 427.4100646973], ..., [429.1873474121, 435.2698364258, 429.9173889160, ..., 433.3015441895, 438.7906494141, 429.1213684082], [433.7449951172, 437.3067626953, 437.7668151855, ..., 424.1896667480, 425.4040832520, 439.9102478027], [430.1427001953, 445.3991088867, 437.3016052246, ..., 439.9053344727, 424.5428161621, 431.0160827637]], [[417.6659851074, 429.5426940918, 426.4377136230, ..., 428.0760498047, 422.9101867676, 436.4093627930], [430.3937683105, 432.2353210449, 436.0132141113, ..., 430.4488525391, 430.4654235840, 430.7756347656], [438.3454895020, 435.6012268066, 432.5616455078, ..., 428.9048767090, 431.8200683594, 427.5857238770], ..., [427.3694152832, 428.4201049805, 426.7932739258, ..., 431.1933898926, 432.4385986328, 427.4636230469], [431.4783325195, 432.2781372070, 430.9346618652, ..., 420.9787292480, 426.7428283691, 431.8652954102], [425.0969238281, 442.2406616211, 427.8924560547, ..., 429.7329101562, 421.6875610352, 432.9624633789]], [[412.8145141602, 425.4304809570, 421.1839294434, ..., 424.2558288574, 419.7536315918, 426.0528869629], [423.4242858887, 430.0761413574, 423.8686828613, ..., 420.4244995117, 426.9145202637, 429.1798706055], [432.4686584473, 429.3939819336, 437.7819213867, ..., 425.9971008301, 423.0393981934, 412.3337402344], ..., [416.0044250488, 428.1051635742, 423.3621520996, ..., 427.3386535645, 428.5605468750, 424.5195617676], [431.3450927734, 430.0262145996, 422.5646667480, ..., 421.5997619629, 419.3545227051, 431.8386535645], [422.9151916504, 433.2700195312, 425.0297546387, ..., 424.5514526367, 416.2740478516, 423.0010375977]], ..., [[419.7507019043, 429.4707641602, 425.2191467285, ..., 432.6334533691, 431.5886230469, 436.8674011230], [425.1297912598, 435.9379882812, 443.7702331543, ..., 429.2365417480, 442.1405639648, 433.2154235840], [441.3648376465, 440.7537231445, 444.0763549805, ..., 434.5533142090, 435.2050170898, 431.8185729980], ..., [435.7179260254, 426.3558959961, 435.9078979492, ..., 439.4034423828, 434.7845458984, 436.1127929688], [434.9248046875, 440.4300842285, 435.1066894531, ..., 424.9306335449, 437.7442016602, 441.1544189453], [424.5058898926, 439.4247131348, 434.3027343750, ..., 434.0754394531, 426.0892639160, 437.7467956543]], [[412.5895385742, 433.6454772949, 422.6614074707, ..., 433.8635559082, 435.7726440430, 436.7060852051], [428.8263244629, 436.7127685547, 439.5891113281, ..., 428.5089111328, 443.4781494141, 427.7095336914], [446.0644226074, 438.5823059082, 438.7479553223, ..., 431.8514404297, 433.5840454102, 421.0302734375], ..., [431.9471740723, 429.9702758789, 429.3433532715, ..., 436.1724243164, 437.5598449707, 425.5983276367], [428.5778198242, 442.1039733887, 435.1011352539, ..., 426.1305541992, 427.0152282715, 436.2427368164], [425.3134155273, 432.7832946777, 432.6957397461, ..., 436.3270568848, 421.2727661133, 438.8211364746]], [[419.0739135742, 427.7665405273, 423.8292846680, ..., 438.3381347656, 429.6674499512, 444.5022583008], [434.7643432617, 434.9652099609, 441.7892150879, ..., 425.9905700684, 440.1138000488, 435.2943420410], [441.3544616699, 436.4108581543, 442.4325561523, ..., 441.1099853516, 440.0770568848, 427.3345642090], ..., [432.7418518066, 434.9036254883, 428.7971496582, ..., 432.5161437988, 432.6544494629, 436.4313354492], [441.9117431641, 444.2507019043, 438.4055786133, ..., 423.4554138184, 432.0736083984, 439.2597656250], [434.0770568848, 446.8493652344, 432.4900817871, ..., 435.5930175781, 422.1889038086, 433.6976623535]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[418., 436., 424., ..., 428., 430., 442.], [430., 438., 446., ..., 426., 444., 428.], [438., 440., 442., ..., 442., 436., 424.], ..., [430., 434., 426., ..., 430., 438., 426.], [430., 436., 442., ..., 422., 422., 440.], [434., 442., 444., ..., 440., 420., 424.]], [[412., 424., 420., ..., 430., 420., 432.], [434., 432., 430., ..., 430., 428., 426.], [434., 434., 422., ..., 428., 420., 426.], ..., [424., 420., 424., ..., 430., 428., 422.], [430., 428., 430., ..., 420., 420., 432.], [424., 442., 430., ..., 434., 426., 430.]], [[418., 426., 418., ..., 424., 420., 422.], [426., 438., 428., ..., 418., 432., 432.], [430., 426., 444., ..., 422., 424., 418.], ..., [410., 422., 418., ..., 428., 432., 414.], [436., 428., 422., ..., 420., 422., 428.], [426., 438., 422., ..., 422., 408., 426.]], ..., [[426., 428., 424., ..., 424., 430., 436.], [426., 438., 444., ..., 422., 436., 438.], [442., 440., 444., ..., 428., 434., 430.], ..., [438., 422., 430., ..., 430., 432., 440.], [430., 438., 428., ..., 428., 436., 438.], [426., 440., 434., ..., 432., 418., 436.]], [[412., 436., 422., ..., 424., 432., 428.], [430., 434., 440., ..., 424., 448., 428.], [446., 438., 440., ..., 428., 424., 422.], ..., [426., 422., 430., ..., 428., 432., 426.], [428., 442., 434., ..., 432., 428., 432.], [422., 438., 436., ..., 438., 422., 440.]], [[412., 422., 432., ..., 432., 428., 448.], [434., 438., 438., ..., 416., 442., 436.], [442., 434., 444., ..., 444., 442., 422.], ..., [434., 430., 424., ..., 428., 434., 438.], [446., 438., 432., ..., 418., 430., 436.], [430., 440., 430., ..., 434., 414., 434.]]]])
test_conv2d.test_module[Conv2D545-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7f823869e7e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7f8238669e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7f82284f9833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7f82284f9699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7f8229aba39a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7f82284f869a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7f8227930f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7f82278c9d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464]
test_conv2d.test_module[Conv2D547-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 6422528 B L1 buffer across 2 banks, where each bank needs to store 3211264 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7f823869e7e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7f8238669e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations13data_movement35InterleavedToShardedDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS8_EE5valueEE4typeERKS8_RKSt6vectorINS0_6TensorESaISE_EERKSD_ISt8optionalISE_ESaISK_EE+0x1cf) [0x7f82392f229f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x409139) [0x7f8227903139] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7f8229aba39a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x40052f) [0x7f82278fa52f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff92f) [0x7f82278f992f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff708) [0x7f82278f9708] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff66c) [0x7f82278f966c] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff44d) [0x7f82278f944d] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3fee51) [0x7f82278f8e51] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3fe8d9) [0x7f82278f88d9] --- std::tuple<tt::tt_metal::Tensor, ttnn::operations::sliding_window::ParallelConfig, ttnn::operations::sliding_window::ParallelConfig> ttnn::operations::conv::shard_or_reshard_tensor_if_required<tt::tt_metal::v0::IDevice>(tt::tt_metal::v0::IDevice*, tt::tt_metal::Tensor const&, ttnn::operations::conv::conv2d::Conv2dConfig const&, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, bool, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0x697) [0x7f82278c9747] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702]
test_conv2d.test_module[Conv2D66-[((1, 3, 320, 1024), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 88612864 B L1 buffer across 64 banks, where each bank needs to store 1384576 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7f823869e7e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7f8238669e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7f82284f9833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7f82284f9699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7f8229aba39a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7f8229aba2d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7f8229ab949f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7f82284f869a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7f8229ac0fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7f8229ac17cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7f8227930f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7f82278c9d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7f8239256365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7f82396c59f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7f82396c593e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7f8239490b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5567e1e75b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x5567e1e65a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x5567e1e6b574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5567e1e84172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x5567e1e6c39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x5567e1e6699e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x5567e1e6097f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5567e1e8410b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x5567e1e60790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x5567e1e62c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5567e1e619ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5567e1e834d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x5567e1e65702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5567e1e7638c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x5567e1e6b61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5567e1e8062c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5567e1f88464]
test_conv2d.test_module[Conv2D655-[((1, 256, 22, 66), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[570.9465332031, 574.1500854492, 568.3541259766, ..., 578.8297729492, 586.1707153320, 592.3475341797], [584.7786254883, 581.8095703125, 570.3929443359, ..., 574.6200561523, 580.8138427734, 586.6017456055], [598.0877075195, 598.2860717773, 585.3776855469, ..., 586.0259399414, 585.0142822266, 599.1822509766], ..., [572.1325073242, 575.2939453125, 572.7546386719, ..., 577.7442016602, 581.7818603516, 581.0496215820], [583.0501098633, 578.4487304688, 576.2739257812, ..., 578.1917114258, 580.6842041016, 574.9422607422], [585.0590820312, 575.4932250977, 572.7432861328, ..., 573.8483886719, 587.0266113281, 589.3324584961]], [[573.4533081055, 579.2991333008, 566.8145751953, ..., 575.4982299805, 580.0259399414, 587.6243286133], [587.0394897461, 580.7179565430, 574.9628906250, ..., 589.3240356445, 587.7585449219, 591.7296142578], [600.8475341797, 595.1917724609, 583.5447998047, ..., 588.7973022461, 590.6857299805, 587.9582519531], ..., [586.3887939453, 565.6766967773, 578.3594360352, ..., 578.7778930664, 577.6987304688, 570.6153564453], [578.1883544922, 567.6721801758, 577.8622436523, ..., 576.7630615234, 580.8159790039, 590.1273803711], [579.9798583984, 570.3214111328, 571.2749633789, ..., 587.2895507812, 586.7373657227, 592.7112426758]], [[579.5585937500, 595.9021606445, 587.2987670898, ..., 594.6026611328, 592.4993896484, 604.9803466797], [598.6318969727, 600.0046386719, 593.1022338867, ..., 590.9971313477, 594.6500854492, 609.2246093750], [605.3466796875, 608.8524169922, 602.3565063477, ..., 594.9855957031, 605.6119384766, 610.5812377930], ..., [585.6695556641, 580.2279663086, 582.5025634766, ..., 594.8723144531, 600.2037353516, 588.7517089844], [593.6161499023, 595.9530029297, 586.0278320312, ..., 592.7636718750, 590.8412475586, 593.6564331055], [590.0739135742, 588.4582519531, 588.1378784180, ..., 594.7924194336, 595.9433593750, 602.3705444336]], ..., [[571.3972778320, 575.1876831055, 567.7849731445, ..., 582.2424316406, 575.7076416016, 577.0570068359], [579.5119018555, 576.3999023438, 576.3774414062, ..., 582.0012207031, 585.7007446289, 586.3353881836], [590.6553955078, 597.5267944336, 591.2234497070, ..., 584.8522338867, 587.7366943359, 584.3203125000], ..., [573.8484497070, 562.1511230469, 575.5940551758, ..., 575.2089843750, 582.0255737305, 580.2094726562], [580.4846801758, 572.6994628906, 575.3505249023, ..., 573.3739013672, 580.1397094727, 577.8396606445], [574.1153564453, 571.3304443359, 562.8356323242, ..., 586.3873291016, 589.8883666992, 577.4711914062]], [[578.3197631836, 581.3527221680, 570.2843627930, ..., 589.2136840820, 588.0834350586, 585.2647705078], [584.3297119141, 588.3203735352, 583.2098388672, ..., 586.6118164062, 584.4938964844, 587.3795776367], [591.4717407227, 598.4740600586, 588.3948364258, ..., 585.6416015625, 588.7778320312, 598.9887695312], ..., [575.6362304688, 572.6663818359, 575.7432250977, ..., 586.5990600586, 582.0677490234, 581.7916259766], [576.8594360352, 574.7938232422, 579.9046630859, ..., 584.2249755859, 575.5092773438, 578.5739135742], [579.0380249023, 574.7623291016, 571.6876220703, ..., 586.5312500000, 587.0781250000, 580.0599975586]], [[583.3511352539, 591.6868286133, 579.8598022461, ..., 584.7136840820, 592.1408691406, 595.9363403320], [598.3667602539, 593.0548095703, 583.0578613281, ..., 594.3792114258, 589.5164184570, 597.5725097656], [607.4893188477, 608.0999145508, 584.9462280273, ..., 595.7608032227, 599.2401123047, 601.2327880859], ..., [581.5227050781, 580.3803710938, 584.1896362305, ..., 588.2476806641, 587.6591796875, 583.0761108398], [589.0547485352, 584.6388549805, 580.5606079102, ..., 595.1897583008, 596.6239013672, 589.3002929688], [581.4307250977, 582.4873046875, 587.3427124023, ..., 584.6014404297, 599.2371826172, 599.5585327148]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[576., 576., 572., ..., 572., 584., 588.], [592., 576., 564., ..., 564., 576., 580.], [588., 596., 584., ..., 580., 580., 596.], ..., [580., 572., 572., ..., 564., 584., 584.], [584., 568., 572., ..., 584., 572., 580.], [584., 572., 580., ..., 580., 580., 584.]], [[568., 580., 572., ..., 576., 584., 592.], [592., 592., 568., ..., 584., 588., 592.], [596., 584., 584., ..., 588., 600., 584.], ..., [592., 564., 592., ..., 580., 572., 560.], [584., 560., 576., ..., 580., 584., 588.], [572., 568., 580., ..., 576., 576., 596.]], [[576., 600., 588., ..., 596., 588., 588.], [592., 600., 580., ..., 588., 588., 600.], [600., 608., 600., ..., 584., 604., 604.], ..., [584., 576., 576., ..., 604., 588., 584.], [584., 588., 576., ..., 584., 588., 592.], [592., 580., 580., ..., 604., 600., 616.]], ..., [[576., 572., 580., ..., 580., 584., 580.], [584., 568., 576., ..., 592., 588., 584.], [580., 608., 588., ..., 584., 596., 588.], ..., [572., 560., 580., ..., 572., 584., 580.], [576., 576., 564., ..., 568., 580., 584.], [576., 572., 568., ..., 592., 588., 576.]], [[576., 584., 572., ..., 592., 580., 584.], [584., 588., 592., ..., 580., 588., 568.], [592., 596., 584., ..., 584., 600., 588.], ..., [584., 564., 576., ..., 580., 584., 596.], [576., 564., 580., ..., 580., 572., 580.], [580., 584., 576., ..., 588., 592., 576.]], [[580., 592., 588., ..., 584., 592., 588.], [600., 592., 576., ..., 592., 592., 596.], [604., 612., 584., ..., 596., 596., 600.], ..., [572., 584., 588., ..., 580., 588., 592.], [576., 588., 572., ..., 596., 584., 588.], [584., 588., 584., ..., 580., 596., 600.]]]])
test_conv2d.test_module[Conv2D655-[((1, 256, 42, 130), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[598.8853759766, 580.4720458984, 586.9128417969, ..., 582.2377929688, 587.7333984375, 593.7083740234], [594.6994628906, 576.0783081055, 587.8958129883, ..., 587.2708740234, 585.5787963867, 580.0180664062], [588.3737792969, 579.6992797852, 581.9427490234, ..., 596.4152221680, 579.8858032227, 585.7383422852], ..., [583.6843261719, 577.9436645508, 575.7821044922, ..., 596.8139038086, 586.6145019531, 587.0209960938], [583.6085205078, 582.8132934570, 574.9748535156, ..., 577.1127929688, 581.1231689453, 589.5464477539], [587.6045532227, 585.6358642578, 586.0929565430, ..., 584.8721313477, 588.2424926758, 592.5081176758]], [[588.0150756836, 587.3713378906, 586.6544799805, ..., 591.2977905273, 591.7604980469, 591.2839965820], [594.6317749023, 584.9010009766, 588.8774414062, ..., 588.9233398438, 584.2275390625, 582.8882446289], [587.5820922852, 575.6299438477, 577.2281494141, ..., 588.5680541992, 579.9023437500, 589.1492919922], ..., [589.9266357422, 583.1177368164, 577.7224731445, ..., 584.5738525391, 587.5339355469, 595.0134277344], [585.2255249023, 584.9296264648, 584.9175415039, ..., 583.7152709961, 588.4314575195, 590.5807495117], [592.4172973633, 583.1607666016, 577.8162231445, ..., 589.5422973633, 593.0539550781, 590.6262817383]], [[579.5445556641, 565.4553833008, 574.6454467773, ..., 578.5651245117, 579.3829345703, 576.0475463867], [575.9824829102, 574.5457153320, 576.0711669922, ..., 579.6744384766, 574.0231323242, 570.4569702148], [578.5188598633, 574.5841064453, 566.8877563477, ..., 581.1574096680, 565.3106079102, 572.2462158203], ..., [572.0764770508, 567.0270385742, 567.3983154297, ..., 574.4309082031, 570.9078369141, 586.8668823242], [581.1430664062, 582.8333129883, 578.4231567383, ..., 569.0416259766, 579.5055541992, 582.0878906250], [581.5795898438, 577.5976562500, 573.3123168945, ..., 569.9739379883, 585.3902587891, 583.6161499023]], ..., [[568.3110351562, 565.9423217773, 556.5574340820, ..., 561.5817260742, 561.0871582031, 570.3499755859], [568.9535522461, 558.2981567383, 568.3852539062, ..., 559.3556518555, 562.6853637695, 561.5355224609], [564.9357299805, 561.7120971680, 557.1622314453, ..., 572.7874755859, 567.1071777344, 556.8593139648], ..., [565.0920410156, 562.3305664062, 554.2069091797, ..., 565.8663330078, 562.0339965820, 561.8309936523], [564.6188354492, 564.3046264648, 557.1024169922, ..., 556.6748046875, 559.8143310547, 576.8017578125], [569.9698486328, 558.7568969727, 559.2803344727, ..., 567.6201171875, 562.3820800781, 565.1788940430]], [[575.2861328125, 551.0554809570, 558.0361938477, ..., 560.0009765625, 564.0620117188, 570.3765869141], [571.2079467773, 556.0663452148, 566.1015625000, ..., 565.0631103516, 558.5059204102, 562.0018310547], [559.9179077148, 553.8132324219, 557.9412231445, ..., 567.5860595703, 558.0103759766, 561.2146606445], ..., [561.8624877930, 549.5789794922, 551.8266601562, ..., 556.2504882812, 564.6066284180, 570.1931762695], [567.2437133789, 564.4642944336, 558.5789184570, ..., 555.2012939453, 570.6615600586, 560.5758056641], [574.1638793945, 553.7282104492, 549.0383300781, ..., 567.4617309570, 566.7985839844, 561.6907348633]], [[594.0852050781, 569.1358642578, 578.8757324219, ..., 575.6434936523, 585.1112060547, 578.2572021484], [581.8900756836, 573.3086547852, 584.8018798828, ..., 576.3016967773, 569.0521240234, 582.4326171875], [581.8967895508, 577.0387573242, 567.2719116211, ..., 585.1683349609, 570.5974731445, 573.2645874023], ..., [577.6184082031, 578.1207885742, 566.8510742188, ..., 578.5949707031, 580.1333007812, 589.6390991211], [581.1672973633, 583.5149536133, 577.5324707031, ..., 584.8950195312, 573.8441162109, 588.4443969727], [585.4483642578, 568.6907958984, 571.3514404297, ..., 582.8408203125, 583.0100097656, 583.6928710938]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[592., 572., 576., ..., 580., 584., 604.], [584., 576., 572., ..., 584., 576., 580.], [588., 572., 576., ..., 588., 580., 584.], ..., [584., 588., 572., ..., 588., 580., 588.], [576., 580., 572., ..., 584., 580., 588.], [584., 584., 584., ..., 584., 576., 600.]], [[588., 584., 584., ..., 588., 600., 592.], [596., 580., 588., ..., 592., 588., 580.], [584., 572., 584., ..., 580., 588., 580.], ..., [588., 588., 584., ..., 584., 580., 588.], [584., 592., 592., ..., 584., 604., 588.], [584., 588., 576., ..., 588., 600., 592.]], [[580., 564., 576., ..., 580., 592., 580.], [576., 572., 568., ..., 572., 564., 568.], [580., 572., 576., ..., 576., 572., 576.], ..., [576., 564., 568., ..., 564., 576., 584.], [576., 580., 584., ..., 572., 568., 580.], [588., 576., 568., ..., 564., 588., 576.]], ..., [[572., 564., 556., ..., 560., 568., 568.], [572., 552., 568., ..., 560., 556., 560.], [572., 556., 556., ..., 572., 560., 564.], ..., [568., 560., 548., ..., 568., 560., 568.], [568., 556., 560., ..., 564., 552., 576.], [572., 556., 548., ..., 580., 564., 568.]], [[568., 540., 564., ..., 560., 568., 576.], [572., 540., 560., ..., 564., 560., 568.], [556., 560., 560., ..., 576., 564., 544.], ..., [564., 552., 568., ..., 556., 568., 568.], [580., 576., 568., ..., 540., 572., 564.], [572., 556., 556., ..., 568., 580., 556.]], [[596., 564., 576., ..., 572., 592., 568.], [572., 572., 580., ..., 572., 572., 576.], [576., 572., 568., ..., 576., 560., 576.], ..., [572., 584., 572., ..., 576., 568., 576.], [584., 572., 588., ..., 580., 576., 600.], [576., 564., 568., ..., 580., 580., 580.]]]])
test_conv2d.test_module[Conv2D656-[((1, 128, 42, 130), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[293.4818725586, 280.2920532227, 281.2923889160, ..., 287.2152099609, 290.8133239746, 293.0928955078], [294.2870483398, 285.0880737305, 287.2414855957, ..., 291.0156860352, 282.2409973145, 281.7530517578], [288.7466125488, 274.4450988770, 275.1029052734, ..., 298.0076293945, 288.3718566895, 290.8059997559], ..., [283.0640258789, 285.7520446777, 278.3713989258, ..., 297.4022216797, 288.9410400391, 290.2017211914], [288.8997802734, 280.7722167969, 285.1314086914, ..., 291.9512329102, 293.1516418457, 296.3615112305], [290.2527770996, 284.0832824707, 285.9339904785, ..., 295.8726806641, 298.1438293457, 294.3631286621]], [[295.0975036621, 279.0133056641, 284.1961364746, ..., 289.0271606445, 291.2322692871, 284.7347412109], [288.9143676758, 279.4721374512, 283.4042053223, ..., 291.6141052246, 287.7746887207, 288.2086486816], [282.3192749023, 280.0502929688, 276.9532470703, ..., 293.9095458984, 281.8836059570, 285.6516723633], ..., [285.7076416016, 288.8883972168, 284.4853515625, ..., 296.0803222656, 287.1823730469, 290.8903198242], [291.4388427734, 286.7259521484, 285.9679260254, ..., 291.3361816406, 285.7776794434, 298.9487915039], [292.0816040039, 285.1328430176, 286.0491638184, ..., 297.1561279297, 286.3880310059, 297.2870483398]], [[290.9516296387, 273.9775085449, 274.8743896484, ..., 284.8003845215, 284.2392272949, 285.2824707031], [291.1639709473, 277.7492675781, 276.1764831543, ..., 287.6536254883, 279.3783569336, 283.8845825195], [276.8178405762, 266.9535827637, 274.3260803223, ..., 293.0250549316, 273.8762207031, 276.1171264648], ..., [284.3625183105, 286.5006713867, 279.8199768066, ..., 287.5968017578, 283.8709716797, 284.2675476074], [280.8894958496, 278.7579956055, 279.4757690430, ..., 283.8580932617, 282.3452148438, 285.9756164551], [288.6196594238, 277.9557189941, 278.9501953125, ..., 290.4832763672, 283.0315856934, 286.0246582031]], ..., [[298.0978698730, 282.4517211914, 285.6117248535, ..., 289.4939575195, 288.6869201660, 293.0394897461], [290.5906372070, 281.5510559082, 281.7156372070, ..., 293.0162963867, 282.7958679199, 282.9885864258], [281.8029479980, 273.7663879395, 274.6151733398, ..., 295.6578063965, 290.8417968750, 285.8348388672], ..., [287.5820007324, 285.3754882812, 285.2482910156, ..., 289.1954345703, 288.2568359375, 293.9799804688], [290.5526733398, 282.1519470215, 282.1951599121, ..., 291.9362792969, 285.4445800781, 291.7232360840], [288.1904296875, 285.6292724609, 282.3803710938, ..., 292.8878173828, 287.4517822266, 290.2451782227]], [[304.4683837891, 292.0606079102, 297.0228576660, ..., 299.6366882324, 309.2948303223, 299.7484130859], [297.1445617676, 284.4284057617, 286.9755554199, ..., 302.8532714844, 293.1519775391, 294.4190368652], [296.7998352051, 289.9638366699, 284.7037048340, ..., 303.1863098145, 290.4290161133, 296.1446533203], ..., [298.5397949219, 293.4275512695, 293.3388061523, ..., 299.6168823242, 297.8313293457, 306.2517395020], [295.3922119141, 294.9130249023, 293.8356933594, ..., 296.4296875000, 293.0944213867, 302.4342346191], [299.0019226074, 294.1735229492, 297.4301757812, ..., 309.2293701172, 304.8518981934, 304.8682556152]], [[300.5449523926, 287.6016235352, 294.4804992676, ..., 301.6820678711, 300.0545349121, 297.8362731934], [297.5196838379, 290.4733276367, 289.0950317383, ..., 300.1343383789, 296.3655395508, 294.8783264160], [294.9892883301, 279.7920837402, 276.3822326660, ..., 303.3762817383, 292.9403991699, 293.1867065430], ..., [298.6578979492, 292.5007934570, 295.5617065430, ..., 296.8863830566, 298.1687927246, 303.9887390137], [297.6807556152, 291.5948486328, 293.4550170898, ..., 299.1921386719, 302.0131835938, 301.4419555664], [298.3737487793, 299.4902648926, 293.6893310547, ..., 305.1280822754, 293.1234436035, 304.3649902344]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[290., 280., 280., ..., 286., 282., 290.], [292., 284., 282., ..., 292., 280., 282.], [282., 274., 274., ..., 292., 284., 292.], ..., [282., 282., 272., ..., 294., 284., 286.], [288., 280., 282., ..., 288., 288., 290.], [286., 278., 280., ..., 290., 296., 292.]], [[292., 274., 280., ..., 284., 292., 282.], [284., 278., 282., ..., 288., 286., 292.], [278., 276., 276., ..., 292., 278., 282.], ..., [280., 288., 282., ..., 296., 282., 286.], [288., 282., 288., ..., 288., 282., 294.], [288., 284., 284., ..., 298., 288., 292.]], [[284., 270., 270., ..., 284., 282., 280.], [288., 274., 272., ..., 284., 272., 278.], [276., 262., 272., ..., 290., 268., 270.], ..., [278., 284., 274., ..., 282., 278., 284.], [278., 272., 276., ..., 280., 276., 286.], [286., 274., 278., ..., 286., 276., 280.]], ..., [[300., 276., 280., ..., 288., 286., 288.], [288., 280., 282., ..., 288., 280., 280.], [276., 272., 270., ..., 290., 294., 286.], ..., [286., 280., 278., ..., 284., 292., 292.], [288., 280., 280., ..., 292., 282., 294.], [284., 282., 272., ..., 288., 286., 284.]], [[306., 294., 292., ..., 298., 310., 296.], [290., 280., 282., ..., 300., 290., 290.], [296., 284., 282., ..., 302., 290., 294.], ..., [298., 290., 296., ..., 294., 294., 302.], [294., 292., 288., ..., 288., 290., 300.], [298., 292., 296., ..., 308., 300., 306.]], [[298., 284., 288., ..., 298., 294., 296.], [298., 290., 284., ..., 298., 294., 294.], [288., 274., 272., ..., 300., 290., 290.], ..., [296., 288., 292., ..., 294., 294., 300.], [294., 286., 292., ..., 294., 298., 298.], [302., 296., 290., ..., 302., 292., 302.]]]])
test / run-tests (n300, 1)
Process completed with exit code 1.
test_add.test_module[Add46-[((1, 32), torch.int64)]]: forge/test/models_ops/test_add.py#L18561
TypeError: Dtype mismatch: framework_model.dtype=torch.int64, compiled_model.dtype=torch.int32
test_add.test_module[Add46-[((1, 128), torch.int64)]]: forge/test/models_ops/test_add.py#L18561
TypeError: Dtype mismatch: framework_model.dtype=torch.int64, compiled_model.dtype=torch.int32
test_add.test_module[Add1-[((1, 72, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1927615404]], [[1.4647266865]], [[0.7849823833]], [[0.8285354376]], [[1.0039277077]], [[1.3305835724]], [[1.1865983009]], [[1.5929496288]], [[1.1521329880]], [[1.3288111687]], [[1.0453984737]], [[1.0982222557]], [[0.7188307047]], [[0.8653638959]], [[0.9903934002]], [[1.2150267363]], [[1.3941725492]], [[1.4965163469]], [[0.8575344086]], [[0.9787735343]], [[1.3781135082]], [[1.6116988659]], [[1.0936048031]], [[1.5706608295]], [[1.1159132719]], [[1.2494120598]], [[1.6492431164]], [[0.7326697707]], [[0.8817359805]], [[1.0699223280]], [[1.0016050339]], [[1.6285053492]], [[0.8724151254]], [[0.9663385153]], [[0.8471847177]], [[0.7282244563]], [[0.9046347141]], [[1.6263039112]], [[1.4196140766]], [[1.4388412237]], [[1.2228007317]], [[0.9401631951]], [[1.2810972929]], [[0.7296575904]], [[0.8352218270]], [[0.9387399554]], [[1.5119738579]], [[1.4896655083]], [[0.9747574329]], [[1.1784636974]], [[1.5162853003]], [[1.6935715675]], [[1.3949460983]], [[1.2640514374]], [[1.5317480564]], [[0.9021037817]], [[1.2896769047]], [[0.8088521957]], [[0.8499618769]], [[0.9382131696]], [[1.4227414131]], [[1.3975851536]], [[0.9003286958]], [[1.3475584984]], [[1.4709908962]], [[1.1333962679]], [[1.2155957222]], [[1.3123573065]], [[1.5066932440]], [[1.6766020060]], [[0.8111931682]], [[1.0132701397]]]]), compiled_model=tensor([[[[1.1927615404]], [[1.6824965477]], [[1.0235810280]], [[1.0732088089]], [[0.9069300890]], [[0.6992873549]], [[1.0360896587]], [[1.0836420059]], [[0.4896509051]], [[1.5765525103]], [[1.2290732861]], [[0.4029533267]], [[0.6159117818]], [[0.5846289396]], [[0.7116078734]], [[0.7896433473]], [[1.3899457455]], [[1.0038596392]], [[0.8443251252]], [[1.0351226330]], [[1.5395443439]], [[1.6021494865]], [[0.4022322893]], [[1.0498074293]], [[1.1690658331]], [[1.1575577259]], [[1.0626960993]], [[0.2482550740]], [[1.1556056738]], [[1.2103263140]], [[0.5870874524]], [[1.3061580658]], [[0.1996111274]], [[0.7608464956]], [[0.2741503119]], [[0.1460411549]], [[0.6805799603]], [[1.5048716068]], [[1.0183440447]], [[1.5390250683]], [[0.7220262289]], [[1.1973433495]], [[1.4272422791]], [[0.1115111709]], [[0.5142747164]], [[0.7647963166]], [[1.3884195089]], [[1.4117476940]], [[0.9744666219]], [[1.0119088888]], [[1.0758159161]], [[1.7336610556]], [[0.7188166380]], [[0.7711930871]], [[1.2100782394]], [[0.4620421529]], [[0.9182553291]], [[0.2025364041]], [[0.5470993519]], [[0.8485864401]], [[0.9005036354]], [[1.1754205227]], [[1.0617492199]], [[1.0996534824]], [[1.2883820534]], [[0.8937568665]], [[1.1202814579]], [[1.4337720871]], [[1.7838113308]], [[1.7976250648]], [[1.0893950462]], [[0.7806043029]]]])
test_add.test_module[Add1-[((1, 480, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6432905197]], [[0.9152557254]], [[0.2355113626]], [[0.2790644169]], [[0.4544567466]], [[0.7811126113]], [[0.6371273398]], [[1.0434787273]], [[0.6026619077]], [[0.7793402076]], [[0.4959273934]], [[0.5487512350]], [[0.1693596840]], [[0.3158928752]], [[0.4409223795]], [[0.6655557156]], [[0.8447015285]], [[0.9470453262]], [[0.3080633879]], [[0.4293025136]], [[0.8286424875]], [[1.0622279644]], [[0.5441338420]], [[1.0211898088]], [[0.5664422512]], [[0.6999409795]], [[1.0997719765]], [[0.1831987500]], [[0.3322649598]], [[0.5204513073]], [[0.4521339536]], [[1.0790343285]], [[0.3229441047]], [[0.4168674946]], [[0.2977136970]], [[0.1787534356]], [[0.3551636934]], [[1.0768330097]], [[0.8701431155]], [[0.8893702030]], [[0.6733297110]], [[0.3906921744]], [[0.7316262722]], [[0.1801865697]], [[0.2857508063]], [[0.3892689347]], [[0.9625028968]], [[0.9401945472]], [[0.4252864122]], [[0.6289927363]], [[0.9668142796]], [[1.1441004276]], [[0.8454750180]], [[0.7145803571]], [[0.9822770953]], [[0.3526327610]], [[0.7402059436]], [[0.2593811750]], [[0.3004908562]], [[0.3887421489]], [[0.8732704520]], [[0.8481141329]], [[0.3508576751]], [[0.7980874777]], [[0.9215199351]], [[0.5839252472]], [[0.6661247015]], [[0.7628862858]], [[0.9572222233]], [[1.1271309853]], [[0.2617221475]], [[0.4637990594]], [[0.8435388803]], [[1.0613086224]], [[1.0821375847]], [[1.0882122517]], [[0.7465412021]], [[0.2122426033]], [[0.6930301785]], [[0.3342312574]], [[0.1810568571]], [[1.0912802219]], [[1.0272138119]], [[0.1482699513]], [[0.7406199574]], [[0.5628039241]], [[0.5647533536]], [[0.4181554914]], [[0.8393120170]], [[0.3508821726]], [[0.8303295970]], [[0.8998879790]], [[1.0049697161]], [[0.8339895010]], [[0.1521663070]], [[0.3226854801]], [[0.8966914415]], [[0.7516846061]], [[0.2569919229]], [[0.3591241837]], [[1.1174085140]], [[0.9839428663]], [[0.4290213585]], [[0.5211915374]], [[0.1707348824]], [[0.6380468607]], [[0.2705044746]], [[0.2613555789]], [[0.6194841266]], [[0.7221064568]], [[0.4422687888]], [[0.9437227249]], [[0.3427643776]], [[1.1007189751]], [[0.9896838665]], [[0.2253924608]], [[0.5225917697]], [[0.6695952415]], [[0.7199844718]], [[0.7656210661]], [[0.8432480693]], [[0.6769840121]], [[0.4030695558]], [[0.8836284280]], [[0.1674094796]], [[0.3506805897]], [[0.5218690038]], [[0.4034772515]], [[0.4721172452]], [[0.2372230887]], [[0.5406763554]], [[0.7539121509]], [[0.3213010430]], [[0.6213742495]], [[1.0049593449]], [[0.5956338048]], [[0.6609300375]], [[0.6038994789]], [[0.7482246161]], [[0.9649536610]], [[1.1206569672]], [[0.9645618796]], [[1.1217406988]], [[0.6108731031]], [[0.1978731751]], [[0.4099953175]], [[0.9874865413]], [[0.6437926888]], [[0.3985107541]], [[0.2638780475]], [[0.1791079044]], [[0.2250298262]], [[0.5456155539]], [[0.9212369323]], [[0.9173544645]], [[0.1648179889]], [[0.9589249492]], [[0.2557792068]], [[0.5413287878]], [[0.4442976117]], [[0.5507262945]], [[0.5488625765]], [[0.1983589530]], [[0.2153149843]], [[0.5687941909]], [[0.6535000205]], [[0.4198964834]], [[0.8353835344]], [[0.1970047355]], [[0.6132903099]], [[1.0867435932]], [[0.4430879354]], [[1.0985355377]], [[0.8281108141]], [[0.1958034635]], [[0.9633826017]], [[0.5893366933]], [[0.4238305092]], [[1.0468605757]], [[0.2429863214]], [[0.7006863952]], [[0.5423495770]], [[1.0040903091]], [[0.7866061926]], [[0.8872866035]], [[0.8236134052]], [[0.5267965794]], [[0.5418812037]], [[0.2349932194]], [[0.9179559946]], [[1.0440230370]], [[0.9891463518]], [[0.2943447828]], [[0.6693338752]], [[0.2945668697]], [[0.3717918396]], [[0.3556811810]], [[0.8179064393]], [[0.3490773439]], [[0.6361253262]], [[0.6680679917]], [[0.9693454504]], [[0.2690739036]], [[0.3037778139]], [[0.3567031622]], [[0.9970006347]], [[0.4673014283]], [[1.0687782764]], [[0.8278377056]], [[0.7103469372]], [[0.6433119178]], [[0.5481931567]], [[0.7097671032]], [[0.5328615904]], [[0.6435207129]], [[0.7108304501]], [[0.2559313774]], [[0.3849682212]], [[1.0507802963]], [[0.2412605882]], [[0.6111308336]], [[1.1416532993]], [[0.8276524544]], [[0.6611904502]], [[0.2137289643]], [[0.8947228789]], [[0.2908935547]], [[0.5051018000]], [[0.4792757630]], [[0.5729902387]], [[0.6525030732]], [[1.0594377518]], [[0.7094533443]], [[1.0948803425]], [[0.9528901577]], [[0.3309269547]], [[0.8712859750]], [[0.2935858965]], [[0.4351213574]], [[0.7940952778]], [[0.8121299744]], [[1.0221478939]], [[0.4860759974]], [[0.6478343606]], [[0.9044457078]], [[0.1634878516]], [[1.0085241795]], [[0.2335727215]], [[0.6539251804]], [[0.5620255470]], [[0.3836974502]], [[0.7131194472]], [[1.0604932308]], [[0.5008741617]], [[0.3501868844]], [[0.4621145129]], [[0.1514597535]], [[0.8727309108]], [[0.4069020748]], [[0.3133637905]], [[0.3589832187]], [[0.9345119596]], [[0.9118207693]], [[1.0307948589]], [[0.8283954859]], [[0.4800540805]], [[0.5073098540]], [[0.7947489023]], [[1.0580513477]], [[0.7829800248]], [[0.4104612470]], [[0.4119952321]], [[0.1742994189]], [[0.7550579309]], [[0.3664426804]], [[0.2012460232]], [[1.0854721069]], [[0.3223283291]], [[0.5901531577]], [[0.7902806997]], [[0.6629630923]], [[0.3105930686]], [[0.2428730726]], [[1.0455751419]], [[0.7284532785]], [[1.0618462563]], [[0.4794136286]], [[0.7943116426]], [[0.5326958299]], [[0.6247983575]], [[0.3425118327]], [[0.8161343932]], [[0.8051168919]], [[0.6367196441]], [[0.5345828533]], [[0.3388190269]], [[0.9928063154]], [[0.2748218775]], [[0.8518666625]], [[0.4789071679]], [[0.4057999253]], [[0.7368564606]], [[0.3873054385]], [[0.7622363567]], [[0.7452243567]], [[0.2757892013]], [[0.7302832007]], [[0.8599985242]], [[0.8449494243]], [[0.5840945840]], [[0.2371401191]], [[0.5699568987]], [[0.8206877708]], [[0.4645953774]], [[0.8368244767]], [[0.9800152183]], [[0.3859581351]], [[0.6519648433]], [[0.8537834287]], [[0.6862228513]], [[0.6887996793]], [[0.7094666362]], [[0.2539600730]], [[0.6863280535]], [[0.9932688475]], [[1.0975908041]], [[0.9409093857]], [[0.7140354514]], [[0.8805410266]], [[0.4037941098]], [[0.2326897383]], [[0.2170733809]], [[1.1458387375]], [[0.9644286036]], [[0.3014170527]], [[0.8426552415]], [[1.0246176720]], [[1.1468412876]], [[1.0842313766]], [[1.0344108343]], [[0.5324043036]], [[0.4715631008]], [[1.0575582981]], [[0.9272237420]], [[0.3461444378]], [[1.0965425968]], [[0.8886132240]], [[0.9196016788]], [[0.3336457610]], [[0.7904838324]], [[0.4717527032]], [[1.0377122164]], [[0.5570636392]], [[0.8416901827]], [[0.7358448505]], [[0.8597680330]], [[0.4771201611]], [[0.8908196092]], [[0.2977984548]], [[0.7599614263]], [[0.3087379932]], [[0.1537651420]], [[0.2455060482]], [[1.0417690277]], [[0.9175517559]], [[1.1161417961]], [[1.0475945473]], [[0.2005112171]], [[0.3058183789]], [[0.5662426353]], [[0.3223177791]], [[0.9942321777]], [[0.2690300345]], [[0.4030832052]], [[0.1639882326]], [[0.3631611466]], [[1.0582726002]], [[1.0564138889]], [[1.0049527884]], [[1.0330975056]], [[1.0916271210]], [[0.5190024376]], [[0.8670349717]], [[1.0924991369]], [[0.8124433160]], [[1.1468720436]], [[0.9063653946]], [[0.9578580856]], [[0.4720378518]], [[0.8869476914]], [[0.7044872046]], [[0.5276252627]], [[0.3651790023]], [[0.3664756417]], [[0.2622935176]], [[0.9826990962]], [[1.0024995804]], [[0.5901275873]], [[0.3576909304]], [[1.0334866047]], [[0.9667783976]], [[0.6842009425]], [[0.4109662771]], [[1.1065578461]], [[0.8515042663]], [[0.2674612403]], [[1.1255753040]], [[1.0267207623]], [[0.4647884965]], [[0.9281111956]], [[0.3629394174]], [[0.5686767697]], [[1.0715844631]], [[0.6676934958]], [[0.2934248447]], [[0.4799174666]], [[0.5113113523]], [[0.5505695343]], [[0.6948842406]], [[1.1094486713]], [[0.6738041043]], [[0.3383139372]], [[0.6726610661]], [[0.8867775202]], [[0.8950540423]], [[0.1900646687]], [[0.5575627089]], [[0.2754616737]], [[0.4336911440]], [[0.8271806240]], [[0.2919688225]], [[0.8328973651]], [[1.0714230537]], [[0.6798281074]], [[0.3137900829]], [[0.4678914547]], [[0.7562159896]], [[0.2658749223]], [[0.8954390883]], [[0.1930991411]], [[0.1663872600]], [[0.1612036228]], [[0.5456022620]], [[0.9832503796]], [[0.1737944484]], [[1.0626275539]], [[0.4470224977]], [[0.7934759855]], [[0.6698353887]], [[0.1961743832]], [[1.0616984367]], [[0.9162556529]], [[1.1440317631]], [[0.8996400237]], [[0.3169994354]], [[1.0643258095]], [[0.6739061475]], [[0.8841421604]], [[0.2461193800]], [[0.5032206774]], [[0.1560951471]], [[0.4522876740]], [[0.7548995018]], [[0.2544530630]], [[0.8064160347]], [[0.9154373407]], [[0.7166885734]], [[0.3124922514]], [[0.2593740821]], [[0.4927756190]], [[0.8665130138]], [[1.1402320862]], [[0.9345484376]], [[0.5907292962]], [[0.8223421574]], [[0.1565024853]], [[0.2199832797]], [[0.8803378940]], [[0.3638263345]], [[0.8875817657]]]]), compiled_model=tensor([[[[0.6432905197]], [[1.5860929489]], [[1.4839851856]], [[1.1095695496]], [[1.6394541264]], [[2.0266568661]], [[1.5623590946]], [[2.5243744850]], [[1.3247685432]], [[1.3422672749]], [[1.8293621540]], [[1.9349204302]], [[0.5706655979]], [[1.1835074425]], [[1.4325603247]], [[1.0546545982]], [[2.4515738487]], [[2.0158317089]], [[1.7694278955]], [[1.8955497742]], [[1.6903976202]], [[1.5831627846]], [[1.8931937218]], [[1.8487652540]], [[1.5473380089]], [[1.5743913651]], [[1.1608679295]], [[1.1797194481]], [[0.9860122800]], [[1.5345501900]], [[1.6283421516]], [[2.5970394611]], [[1.1148750782]], [[1.0623140335]], [[1.7483360767]], [[1.0405085087]], [[1.8282469511]], [[2.5342912674]], [[1.5722303391]], [[1.6632347107]], [[1.8768817186]], [[1.5229551792]], [[2.0142798424]], [[1.0946760178]], [[1.0420372486]], [[1.2041490078]], [[2.2432034016]], [[1.9425747395]], [[1.0248345137]], [[1.1430916786]], [[1.6923389435]], [[1.5908164978]], [[1.4416050911]], [[1.8068041801]], [[2.0217666626]], [[1.5093097687]], [[1.7660236359]], [[1.3428159952]], [[1.0201561451]], [[1.1997160912]], [[1.4562170506]], [[2.2313537598]], [[1.3786284924]], [[1.4650183916]], [[1.5005602837]], [[0.9205338955]], [[1.6909657717]], [[1.5894851685]], [[1.4298171997]], [[1.6881048679]], [[1.4642975330]], [[1.1849291325]], [[1.4845908880]], [[1.3957200050]], [[1.4590294361]], [[2.0612955093]], [[1.2894487381]], [[1.0915758610]], [[0.9363771081]], [[1.0460840464]], [[1.0789448023]], [[2.7420978546]], [[2.7112345695]], [[1.1945953369]], [[1.5281562805]], [[1.7653794289]], [[1.5544381142]], [[1.4244418144]], [[1.6190359592]], [[1.3161529303]], [[0.8469919562]], [[1.1085669994]], [[1.3286389112]], [[1.5624438524]], [[0.0902764201]], [[0.2314071655]], [[1.8668450117]], [[2.3097288609]], [[1.1695282459]], [[1.9689261913]], [[1.7125620842]], [[1.3063913584]], [[1.3269093037]], [[1.3009154797]], [[1.8186228275]], [[1.8591769934]], [[1.1381189823]], [[1.0493803024]], [[1.4690322876]], [[1.8055412769]], [[2.0784380436]], [[2.1179778576]], [[0.9960234165]], [[2.0122787952]], [[1.5023179054]], [[0.6677139997]], [[1.1236047745]], [[2.2159206867]], [[1.7663099766]], [[1.5214191675]], [[1.3563704491]], [[1.3365907669]], [[1.8058402538]], [[1.4934303761]], [[1.1522114277]], [[1.3686857224]], [[1.0935850143]], [[0.6575663686]], [[1.2035012245]], [[1.2425329685]], [[0.9912986755]], [[1.3100032806]], [[1.1249506474]], [[1.5807856321]], [[1.5166168213]], [[1.0965490341]], [[2.1506147385]], [[1.6580374241]], [[1.7662297487]], [[1.9897947311]], [[2.1240136623]], [[2.0421373844]], [[1.6841794252]], [[1.1410852671]], [[0.7584071755]], [[0.8722981811]], [[1.7760847807]], [[1.1541067362]], [[0.9490612149]], [[0.8512895107]], [[0.6369504333]], [[0.9708097577]], [[0.8400620222]], [[0.8447772861]], [[1.2869737148]], [[0.7349209785]], [[1.4250295162]], [[0.1661503315]], [[0.5071737766]], [[0.5157698989]], [[0.7135948539]], [[0.7856910229]], [[1.0506021976]], [[0.7729890943]], [[0.7200428843]], [[1.0869300365]], [[0.6944676638]], [[1.4320430756]], [[0.4332394600]], [[0.8272414207]], [[1.7381489277]], [[0.3761420250]], [[1.4599676132]], [[1.0507605076]], [[1.0372548103]], [[1.7043678761]], [[0.6955987215]], [[0.6430185437]], [[1.2860949039]], [[0.1072397828]], [[1.4495565891]], [[1.0913509130]], [[1.5293928385]], [[1.3190903664]], [[0.8330075741]], [[1.0656106472]], [[1.3584403992]], [[0.5599542856]], [[0.2324775457]], [[1.2183134556]], [[1.7585444450]], [[1.1703993082]], [[0.9701071978]], [[0.9641240835]], [[0.6735355258]], [[0.8158783913]], [[0.2118984461]], [[1.0762689114]], [[0.2441940308]], [[0.8807785511]], [[0.6234783530]], [[1.5102820396]], [[0.6983748674]], [[1.1073065996]], [[0.9458004832]], [[1.2852017879]], [[1.0783727169]], [[1.7398881912]], [[1.2536394596]], [[1.2976171970]], [[1.3143935204]], [[1.2756410837]], [[1.3734918833]], [[1.0153359175]], [[1.1239473820]], [[0.6585488915]], [[0.6740365624]], [[0.3822367191]], [[1.3755034208]], [[0.6563866138]], [[1.1509908438]], [[1.7988681793]], [[1.5780091286]], [[1.3942213058]], [[0.9887727499]], [[1.0772136450]], [[0.9736913443]], [[1.0532585382]], [[0.5128701925]], [[0.6240890026]], [[0.7025446296]], [[1.4692983627]], [[1.0231510401]], [[1.3390438557]], [[1.1348956823]], [[0.3971082568]], [[1.6884371042]], [[0.1620073318]], [[0.8208938241]], [[1.5072610378]], [[0.7119740844]], [[0.8861027360]], [[0.9012952447]], [[1.0488648415]], [[1.6656954288]], [[0.7355698943]], [[1.0088002682]], [[0.3571452498]], [[0.5910126567]], [[0.4874385595]], [[0.4701657295]], [[1.5092194080]], [[1.3240481615]], [[0.4311599731]], [[0.8999097943]], [[1.1467518806]], [[0.1392654777]], [[1.5595889091]], [[0.5665434599]], [[0.7062265873]], [[0.4960951805]], [[0.8419075012]], [[0.8744081855]], [[1.4949700832]], [[1.0758250952]], [[0.6459352374]], [[0.8322960734]], [[0.7796265483]], [[1.4818115234]], [[1.3749754429]], [[0.8315638900]], [[1.2457473278]], [[0.5702579021]], [[1.0161709785]], [[0.3893322349]], [[0.1322010756]], [[1.3040784597]], [[0.3744184971]], [[1.4107502699]], [[0.7911782861]], [[1.1093621254]], [[1.1368401051]], [[0.1245256662]], [[1.5452265739]], [[1.4389731884]], [[1.3646998405]], [[0.7592597008]], [[1.0266125202]], [[1.3701941967]], [[0.6172183752]], [[1.0986692905]], [[1.6448457241]], [[0.8362820745]], [[1.1224486828]], [[0.8640732169]], [[0.3410556316]], [[1.1856603622]], [[0.7435573936]], [[0.9468892813]], [[1.0356345177]], [[0.5026663542]], [[0.6182712913]], [[1.2342550755]], [[1.1052479744]], [[0.6321507096]], [[0.1892203689]], [[1.3278805017]], [[1.3074564934]], [[0.7250729203]], [[1.0390214920]], [[0.4370844364]], [[1.0160447359]], [[1.1123795509]], [[0.4167149067]], [[0.8946704865]], [[1.5376076698]], [[0.3898609877]], [[0.9615180492]], [[0.8347385526]], [[0.5713572502]], [[1.0407650471]], [[0.6021842360]], [[0.5894266963]], [[1.1019103527]], [[1.3491635323]], [[1.7385807037]], [[1.3502504826]], [[1.2279213667]], [[1.7156231403]], [[0.5907594562]], [[0.9260461926]], [[0.4085202813]], [[1.8786287308]], [[1.4627962112]], [[1.1028025150]], [[1.1330316067]], [[1.0316700935]], [[1.6049160957]], [[1.0271592140]], [[1.7390085459]], [[1.2954764366]], [[0.3823590279]], [[1.6627001762]], [[1.3369359970]], [[1.1441019773]], [[1.6484005451]], [[0.8784171939]], [[1.5073474646]], [[0.3282203674]], [[1.1474816799]], [[0.6562858224]], [[1.1535165310]], [[0.8223814964]], [[0.9577039480]], [[0.8755698204]], [[1.4556984901]], [[0.6794393659]], [[1.5139436722]], [[0.8331122994]], [[1.4824255705]], [[0.3680773973]], [[0.1129165292]], [[0.4119288921]], [[1.8584219217]], [[1.4071812630]], [[1.2876267433]], [[1.7812303305]], [[0.3061612844]], [[1.0781075954]], [[0.9523036480]], [[1.1408522129]], [[1.4197332859]], [[0.9201971292]], [[0.5385010242]], [[0.1713696122]], [[0.3693567514]], [[1.2868671417]], [[1.4342957735]], [[1.2082762718]], [[1.7020332813]], [[1.5586743355]], [[1.2357254028]], [[1.6423799992]], [[1.5353609324]], [[1.3682303429]], [[1.1724286079]], [[1.6445872784]], [[1.6160199642]], [[0.3670911789]], [[1.2709438801]], [[1.3650162220]], [[0.5141481757]], [[0.6773358583]], [[0.9235632420]], [[0.7739562988]], [[0.9538368583]], [[1.0993044376]], [[1.0128610134]], [[1.1830574274]], [[1.1936874390]], [[1.7439882755]], [[1.4445157051]], [[0.2704749703]], [[1.6646010876]], [[1.5720013380]], [[1.0796248913]], [[1.5574651957]], [[1.6437653303]], [[0.3694860339]], [[1.3546001911]], [[0.6652881503]], [[0.8953785896]], [[1.4195116758]], [[1.1981751919]], [[0.1678333282]], [[1.2554483414]], [[0.6564683318]], [[0.4239755273]], [[0.6409259439]], [[1.5412329435]], [[0.8628174067]], [[0.7406799793]], [[0.6612382531]], [[0.9435313344]], [[0.7591368556]], [[0.2681972980]], [[1.1831872463]], [[0.1627495885]], [[0.7053192258]], [[1.5131113529]], [[0.9360464215]], [[0.9490385056]], [[1.4581763744]], [[1.1453273296]], [[0.8286214471]], [[0.5965632796]], [[1.3307435513]], [[0.7596735954]], [[1.4636592865]], [[0.2137073874]], [[0.2024887800]], [[0.1788253784]], [[1.2611703873]], [[1.3293912411]], [[0.2315820456]], [[1.5807569027]], [[0.4851566553]], [[1.3578138351]], [[1.1398043633]], [[0.3204129934]], [[1.6336282492]], [[1.3205339909]], [[1.7525398731]], [[0.8867415190]], [[0.7172477841]], [[1.7041357756]], [[1.0881819725]], [[1.3323907852]], [[0.9500837922]], [[1.2986679077]], [[0.6984318495]], [[1.2528644800]], [[0.8309126496]], [[0.1922032833]], [[0.8135689497]], [[1.3641623259]], [[1.5342931747]], [[0.7031964660]], [[1.0833247900]], [[0.7237421870]], [[1.4870373011]], [[1.1216611862]], [[0.9390876293]], [[0.9042983651]], [[1.5470068455]], [[0.0222479701]], [[0.3076067567]], [[1.2334625721]], [[0.8989537954]], [[0.7425754666]]]])
test_add.test_module[Add1-[((1, 672, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.9084333777]], [[1.1803985834]], [[0.5006542206]], [[0.5442072749]], [[0.7195996046]], [[1.0462554693]], [[0.9022701979]], [[1.3086215258]], [[0.8678047657]], [[1.0444830656]], [[0.7610702515]], [[0.8138940930]], [[0.4345025420]], [[0.5810357332]], [[0.7060652375]], [[0.9306985736]], [[1.1098444462]], [[1.2121882439]], [[0.5732062459]], [[0.6944453716]], [[1.0937852859]], [[1.3273707628]], [[0.8092767000]], [[1.2863326073]], [[0.8315851092]], [[0.9650838375]], [[1.3649148941]], [[0.4483416080]], [[0.5974078178]], [[0.7855941653]], [[0.7172768116]], [[1.3441772461]], [[0.5880869627]], [[0.6820103526]], [[0.5628565550]], [[0.4438962936]], [[0.6203065515]], [[1.3419758081]], [[1.1352859735]], [[1.1545131207]], [[0.9384725690]], [[0.6558350325]], [[0.9967691302]], [[0.4453294277]], [[0.5508936644]], [[0.6544117928]], [[1.2276457548]], [[1.2053374052]], [[0.6904292703]], [[0.8941355944]], [[1.2319571972]], [[1.4092433453]], [[1.1106178761]], [[0.9797232151]], [[1.2474199533]], [[0.6177756190]], [[1.0053488016]], [[0.5245240331]], [[0.5656337142]], [[0.6538850069]], [[1.1384133101]], [[1.1132569313]], [[0.6160005331]], [[1.0632302761]], [[1.1866627932]], [[0.8490681052]], [[0.9312675595]], [[1.0280292034]], [[1.2223651409]], [[1.3922739029]], [[0.5268650055]], [[0.7289419174]], [[1.1086816788]], [[1.3264515400]], [[1.3472805023]], [[1.3533551693]], [[1.0116840601]], [[0.4773854613]], [[0.9581730366]], [[0.5993741155]], [[0.4461997151]], [[1.3564230204]], [[1.2923567295]], [[0.4134128094]], [[1.0057628155]], [[0.8279467821]], [[0.8298962116]], [[0.6832983494]], [[1.1044548750]], [[0.6160250306]], [[1.0954724550]], [[1.1650308371]], [[1.2701125145]], [[1.0991322994]], [[0.4173091650]], [[0.5878283381]], [[1.1618342400]], [[1.0168274641]], [[0.5221347809]], [[0.6242670417]], [[1.3825514317]], [[1.2490856647]], [[0.6941642165]], [[0.7863343954]], [[0.4358777404]], [[0.9031897187]], [[0.5356473327]], [[0.5264984369]], [[0.8846269846]], [[0.9872493148]], [[0.7074116468]], [[1.2088656425]], [[0.6079072356]], [[1.3658618927]], [[1.2548267841]], [[0.4905353189]], [[0.7877346277]], [[0.9347380996]], [[0.9851273298]], [[1.0307638645]], [[1.1083909273]], [[0.9421268702]], [[0.6682124138]], [[1.1487712860]], [[0.4325523376]], [[0.6158234477]], [[0.7870118618]], [[0.6686201096]], [[0.7372601032]], [[0.5023659468]], [[0.8058192134]], [[1.0190550089]], [[0.5864439011]], [[0.8865171075]], [[1.2701022625]], [[0.8607766628]], [[0.9260728955]], [[0.8690423369]], [[1.0133674145]], [[1.2300965786]], [[1.3857998848]], [[1.2297047377]], [[1.3868834972]], [[0.8760159612]], [[0.4630160332]], [[0.6751381755]], [[1.2526293993]], [[0.9089355469]], [[0.6636536121]], [[0.5290209055]], [[0.4442507625]], [[0.4901726842]], [[0.8107584119]], [[1.1863797903]], [[1.1824972630]], [[0.4299608469]], [[1.2240678072]], [[0.5209220648]], [[0.8064716458]], [[0.7094404697]], [[0.8158691525]], [[0.8140054345]], [[0.4635018110]], [[0.4804578424]], [[0.8339370489]], [[0.9186428785]], [[0.6850393414]], [[1.1005263329]], [[0.4621475935]], [[0.8784331679]], [[1.3518865108]], [[0.7082307935]], [[1.3636783361]], [[1.0932536125]], [[0.4609463215]], [[1.2285254002]], [[0.8544795513]], [[0.6889733672]], [[1.3120033741]], [[0.5081291795]], [[0.9658292532]], [[0.8074924350]], [[1.2692331076]], [[1.0517489910]], [[1.1524294615]], [[1.0887563229]], [[0.7919394374]], [[0.8070240617]], [[0.5001360774]], [[1.1830987930]], [[1.3091658354]], [[1.2542891502]], [[0.5594876409]], [[0.9344767332]], [[0.5597097278]], [[0.6369346976]], [[0.6208240390]], [[1.0830492973]], [[0.6142202020]], [[0.9012681842]], [[0.9332108498]], [[1.2344882488]], [[0.5342167616]], [[0.5689206719]], [[0.6218460202]], [[1.2621434927]], [[0.7324442863]], [[1.3339211941]], [[1.0929806232]], [[0.9754897952]], [[0.9084547758]], [[0.8133360147]], [[0.9749099612]], [[0.7980044484]], [[0.9086635709]], [[0.9759733081]], [[0.5210742354]], [[0.6501110792]], [[1.3159232140]], [[0.5064034462]], [[0.8762736917]], [[1.4067962170]], [[1.0927953720]], [[0.9263333082]], [[0.4788718224]], [[1.1598657370]], [[0.5560364127]], [[0.7702446580]], [[0.7444186211]], [[0.8381330967]], [[0.9176459312]], [[1.3245805502]], [[0.9745962024]], [[1.3600232601]], [[1.2180330753]], [[0.5960698128]], [[1.1364288330]], [[0.5587287545]], [[0.7002642155]], [[1.0592381954]], [[1.0772728920]], [[1.2872908115]], [[0.7512188554]], [[0.9129772186]], [[1.1695885658]], [[0.4286307096]], [[1.2736670971]], [[0.4987155795]], [[0.9190680385]], [[0.8271684051]], [[0.6488403082]], [[0.9782623053]], [[1.3256361485]], [[0.7660170197]], [[0.6153297424]], [[0.7272573709]], [[0.4166026115]], [[1.1378737688]], [[0.6720449328]], [[0.5785066485]], [[0.6241260767]], [[1.1996548176]], [[1.1769635677]], [[1.2959377766]], [[1.0935382843]], [[0.7451969385]], [[0.7724527121]], [[1.0598917007]], [[1.3231941462]], [[1.0481228828]], [[0.6756041050]], [[0.6771380901]], [[0.4394422770]], [[1.0202007294]], [[0.6315855384]], [[0.4663888812]], [[1.3506150246]], [[0.5874711871]], [[0.8552960157]], [[1.0554234982]], [[0.9281059504]], [[0.5757359266]], [[0.5080159307]], [[1.3107180595]], [[0.9935961366]], [[1.3269890547]], [[0.7445564866]], [[1.0594544411]], [[0.7978386879]], [[0.8899412155]], [[0.6076546907]], [[1.0812772512]], [[1.0702598095]], [[0.9018625021]], [[0.7997257113]], [[0.6039618850]], [[1.2579491138]], [[0.5399647355]], [[1.1170095205]], [[0.7440500259]], [[0.6709427834]], [[1.0019993782]], [[0.6524482965]], [[1.0273792744]], [[1.0103671551]], [[0.5409320593]], [[0.9954260588]], [[1.1251413822]], [[1.1100922823]], [[0.8492374420]], [[0.5022829771]], [[0.8350997567]], [[1.0858306885]], [[0.7297382355]], [[1.1019673347]], [[1.2451580763]], [[0.6511009932]], [[0.9171077013]], [[1.1189262867]], [[0.9513657093]], [[0.9539425373]], [[0.9746094942]], [[0.5191029310]], [[0.9514709115]], [[1.2584116459]], [[1.3627336025]], [[1.2060523033]], [[0.9791783094]], [[1.1456838846]], [[0.6689369678]], [[0.4978325963]], [[0.4822162390]], [[1.4109816551]], [[1.2295714617]], [[0.5665599108]], [[1.1077980995]], [[1.2897605896]], [[1.4119842052]], [[1.3493742943]], [[1.2995536327]], [[0.7975471616]], [[0.7367059588]], [[1.3227012157]], [[1.1923666000]], [[0.6112872958]], [[1.3616853952]], [[1.1537561417]], [[1.1847445965]], [[0.5987886190]], [[1.0556266308]], [[0.7368955612]], [[1.3028550148]], [[0.8222064972]], [[1.1068329811]], [[1.0009877682]], [[1.1249108315]], [[0.7422630191]], [[1.1559624672]], [[0.5629413128]], [[1.0251042843]], [[0.5738808513]], [[0.4189080000]], [[0.5106489062]], [[1.3069118261]], [[1.1826946735]], [[1.3812847137]], [[1.3127374649]], [[0.4656540751]], [[0.5709612370]], [[0.8313854933]], [[0.5874606371]], [[1.2593750954]], [[0.5341728926]], [[0.6682260633]], [[0.4291310906]], [[0.6283040047]], [[1.3234155178]], [[1.3215568066]], [[1.2700955868]], [[1.2982404232]], [[1.3567700386]], [[0.7841452956]], [[1.1321778297]], [[1.3576419353]], [[1.0775861740]], [[1.4120149612]], [[1.1715083122]], [[1.2230010033]], [[0.7371807098]], [[1.1520905495]], [[0.9696300626]], [[0.7927681208]], [[0.6303218603]], [[0.6316184998]], [[0.5274363756]], [[1.2478419542]], [[1.2676423788]], [[0.8552704453]], [[0.6228337884]], [[1.2986295223]], [[1.2319211960]], [[0.9493438005]], [[0.6761091352]], [[1.3717007637]], [[1.1166471243]], [[0.5326040983]], [[1.3907182217]], [[1.2918636799]], [[0.7299313545]], [[1.1932539940]], [[0.6280822754]], [[0.8338196278]], [[1.3367273808]], [[0.9328363538]], [[0.5585677028]], [[0.7450603247]], [[0.7764542103]], [[0.8157123923]], [[0.9600270987]], [[1.3745915890]], [[0.9389469624]], [[0.6034567952]], [[0.9378039241]], [[1.1519203186]], [[1.1601969004]], [[0.4552075267]], [[0.8227055669]], [[0.5406045318]], [[0.6988340020]], [[1.0923235416]], [[0.5571116805]], [[1.0980402231]], [[1.3365658522]], [[0.9449709654]], [[0.5789329410]], [[0.7330343127]], [[1.0213588476]], [[0.5310177803]], [[1.1605819464]], [[0.4582419991]], [[0.4315301180]], [[0.4263464808]], [[0.8107451200]], [[1.2483932972]], [[0.4389373064]], [[1.3277703524]], [[0.7121653557]], [[1.0586187840]], [[0.9349782467]], [[0.4613172412]], [[1.3268413544]], [[1.1813985109]], [[1.4091746807]], [[1.1647828817]], [[0.5821422935]], [[1.3294687271]], [[0.9390490055]], [[1.1492850780]], [[0.5112622380]], [[0.7683635354]], [[0.4212380052]], [[0.7174305320]], [[1.0200424194]], [[0.5195959210]], [[1.0715589523]], [[1.1805801392]], [[0.9818314314]], [[0.5776351094]], [[0.5245169401]], [[0.7579184771]], [[1.1316559315]], [[1.4053750038]], [[1.1996912956]], [[0.8558721542]], [[1.0874850750]], [[0.4216453433]], [[0.4851261377]], [[1.1454807520]], [[0.6289691925]], [[1.1527246237]], [[0.5592107177]], [[0.6645213366]], [[0.5003322959]], [[1.1730971336]], [[0.8612292409]], [[1.2969772816]], [[1.2216129303]], [[1.1888480186]], [[0.9282548428]], [[0.7575877905]], [[0.8034609556]], [[0.9786270857]], [[1.1600315571]], [[0.5618818402]], [[1.3318157196]], [[0.8578117490]], [[0.4932039976]], [[0.6416478753]], [[1.3545856476]], [[1.3694403172]], [[0.4490369558]], [[1.2648259401]], [[1.1627564430]], [[1.2081360817]], [[1.3354413509]], [[0.6427016258]], [[1.0700646639]], [[1.1167933941]], [[0.7644301057]], [[1.0795032978]], [[0.7683200836]], [[1.2213075161]], [[0.7734494805]], [[0.7257789373]], [[1.0380513668]], [[1.0895235538]], [[0.6678909659]], [[0.9563759565]], [[1.2019445896]], [[0.8624284267]], [[1.0643436909]], [[0.7915832996]], [[1.0874266624]], [[0.5499796867]], [[0.6181623936]], [[0.6583815217]], [[1.3716821671]], [[0.7776347399]], [[0.9108116031]], [[0.6699290276]], [[1.4113218784]], [[1.4005118608]], [[0.5350834727]], [[0.5068418384]], [[0.5331802964]], [[0.9097654223]], [[0.7847248316]], [[0.5849039555]], [[0.7328407168]], [[1.0066422224]], [[0.6509314179]], [[1.0229680538]], [[0.7975188494]], [[0.6698951125]], [[0.9808695316]], [[1.3233058453]], [[0.5741371512]], [[0.9353939891]], [[0.7277929187]], [[1.4028298855]], [[0.4377951622]], [[0.4328461885]], [[1.4048693180]], [[0.5958318114]], [[1.0080385208]], [[0.8690237403]], [[0.8068482876]], [[0.8005028963]], [[1.2298970222]], [[0.9360752702]], [[0.4253695011]], [[0.6169967055]], [[0.7417266369]], [[1.1637849808]], [[0.5886099339]], [[1.3836365938]], [[0.8008121252]], [[0.8223958015]], [[1.3039557934]], [[1.1634806395]], [[1.3362369537]], [[1.2013989687]], [[0.7604998350]], [[0.5804405808]], [[0.8749752641]], [[1.3259955645]], [[0.7443668842]], [[0.4484918118]], [[1.1171410084]], [[1.3989124298]], [[0.7698351741]], [[0.4981650114]], [[0.4586469531]], [[1.0374765396]], [[0.8743169308]], [[0.6596831679]], [[1.0132427216]], [[1.1020562649]], [[1.3098362684]], [[1.3003678322]], [[0.8373349309]], [[0.4712932110]], [[0.4603652358]], [[1.3790216446]], [[1.1332044601]], [[1.1301305294]], [[0.4795662761]], [[1.3751846552]], [[1.3858482838]], [[1.3636069298]], [[0.4903842807]], [[0.7235150933]], [[0.5682948828]], [[1.3856595755]], [[0.6973367929]], [[0.6838995218]], [[1.1741333008]], [[0.6808806062]], [[0.6659157276]], [[0.8684410453]], [[0.8641170263]], [[0.5226875544]], [[1.3290108442]], [[0.6916159391]], [[1.0895342827]], [[1.3470685482]], [[1.1643350124]], [[0.9829667211]], [[1.3376052380]], [[0.9793899059]], [[0.6808484793]], [[1.3851765394]], [[1.0305171013]], [[0.4243363738]], [[0.7698310018]], [[0.5715930462]], [[1.3505532742]], [[0.8296038508]], [[0.4564143419]], [[0.8807451129]], [[1.2261831760]], [[1.0420770645]], [[1.0702790022]], [[0.9586133957]], [[1.0985860825]], [[0.7903507948]], [[0.7132766843]], [[0.4448225498]], [[0.5355083346]], [[1.1288781166]], [[0.6161201000]], [[0.9839100242]], [[1.0717725754]], [[0.9475787282]], [[0.5880019069]], [[1.3903410435]], [[0.6214270592]], [[1.3234270811]], [[0.5144226551]], [[0.7919025421]], [[1.1841611862]], [[0.7078864574]], [[1.3321982622]], [[0.5680985451]], [[0.4922910333]], [[0.6867172122]], [[0.9930019975]], [[1.3725547791]], [[0.6734763384]], [[1.0909909010]], [[0.7868232727]], [[0.8037456274]], [[1.2798285484]], [[0.5246838331]], [[0.9652834535]], [[1.3823814392]], [[0.8434706926]], [[1.3003958464]], [[0.7581983805]], [[1.3146579266]], [[0.4285116196]], [[0.8401149511]]]]), compiled_model=tensor([[[[0.9084333777]], [[1.2100459337]], [[0.6144800186]], [[0.7231509686]], [[0.3106740117]], [[1.0394752026]], [[0.5322440267]], [[1.2881319523]], [[0.5580722690]], [[1.3202767372]], [[0.9252283573]], [[1.3522800207]], [[0.7584570050]], [[0.6040940285]], [[1.0519936085]], [[1.3366656303]], [[1.2705032825]], [[1.5343155861]], [[0.9791449904]], [[1.1567504406]], [[1.4923672676]], [[1.5447022915]], [[1.0245604515]], [[0.9689082503]], [[0.9845474362]], [[0.6972094774]], [[1.4244952202]], [[0.5983247757]], [[0.8721249700]], [[1.1776661873]], [[1.2024906874]], [[1.8120651245]], [[1.0979878902]], [[0.5993582606]], [[0.9805114865]], [[0.7269101739]], [[0.3887581229]], [[1.1279317141]], [[0.9201846719]], [[1.2992308140]], [[0.9870274067]], [[0.6348556876]], [[0.9136317968]], [[0.2463678718]], [[1.1029019356]], [[0.2576903701]], [[1.3482754230]], [[1.6533603668]], [[0.3251305223]], [[0.4929475188]], [[1.3820335865]], [[1.5451309681]], [[1.6067247391]], [[1.2866623402]], [[0.9825531244]], [[0.4762052894]], [[0.6772934198]], [[0.1847941875]], [[0.3869591355]], [[1.1848421097]], [[1.1368253231]], [[0.7783999443]], [[0.9005805850]], [[1.4827249050]], [[0.9093256593]], [[1.2707833052]], [[0.8257660866]], [[1.1557490826]], [[1.0943341255]], [[1.0345265865]], [[0.2243095636]], [[0.9279742837]], [[1.0909684896]], [[1.2271897793]], [[1.4071238041]], [[1.0730899572]], [[1.1703014374]], [[0.8042379618]], [[1.1141328812]], [[1.1679832935]], [[0.5770153403]], [[1.3523931503]], [[1.0501034260]], [[0.0792250037]], [[0.9592262506]], [[0.6148940921]], [[1.3853504658]], [[0.4190530777]], [[1.2857110500]], [[1.1771291494]], [[0.7119821906]], [[1.3995393515]], [[1.7154896259]], [[1.1368432045]], [[0.4320123792]], [[0.5549864173]], [[1.7341898680]], [[0.7441046238]], [[1.0131494999]], [[1.1878354549]], [[1.1485737562]], [[1.4696718454]], [[0.7585117221]], [[0.5234281421]], [[0.3635888696]], [[1.1067824364]], [[0.3655270934]], [[0.8180829883]], [[0.7163505554]], [[0.6035212874]], [[1.2892184258]], [[1.2867343426]], [[0.2296907306]], [[1.0141501427]], [[1.5872812271]], [[0.6728504300]], [[0.4027152658]], [[1.1245222092]], [[0.9199287891]], [[1.2117089033]], [[1.1349399090]], [[0.6291035414]], [[0.4609155655]], [[1.4412208796]], [[0.1713123322]], [[0.6602337956]], [[0.5028241277]], [[0.2886116505]], [[0.8240826130]], [[0.1299406886]], [[0.8761429787]], [[1.1694943905]], [[0.6771956682]], [[1.2623641491]], [[1.4143004417]], [[1.1095197201]], [[1.4960122108]], [[0.7908648252]], [[1.4415810108]], [[1.1564005613]], [[1.8534469604]], [[1.4629294872]], [[1.9231262207]], [[0.9012494683]], [[0.2049254775]], [[0.8680700660]], [[0.9304143786]], [[1.3483903408]], [[1.1615829468]], [[0.1746739745]], [[0.7842497826]], [[0.6347420216]], [[1.3435730934]], [[1.4730949402]], [[0.9071584344]], [[0.7525637746]], [[0.9534995556]], [[0.6127769947]], [[0.7258619070]], [[0.5601019859]], [[0.8160441518]], [[0.6648763418]], [[0.3380839229]], [[0.8112453818]], [[0.7711133957]], [[1.2766239643]], [[0.9552103281]], [[1.5578477383]], [[0.2563441396]], [[0.5724416971]], [[1.2531664371]], [[1.2597408295]], [[1.5881649256]], [[0.9995957613]], [[0.9294392467]], [[1.0690326691]], [[1.3616259098]], [[0.8098915219]], [[1.8653950691]], [[0.6684874296]], [[1.3518534899]], [[0.6777673960]], [[1.0114716291]], [[0.7928017974]], [[1.1158812046]], [[1.2014951706]], [[0.7301201224]], [[1.2108169794]], [[0.7020403147]], [[1.6346790791]], [[1.8193680048]], [[1.4320081472]], [[0.8501318693]], [[0.6948903203]], [[1.0327887535]], [[1.0299537182]], [[0.2507345080]], [[1.2019026279]], [[1.0096063614]], [[0.6226482391]], [[0.9802248478]], [[1.5264329910]], [[0.7807366848]], [[0.2749155760]], [[0.4535081387]], [[1.4197341204]], [[1.2926678658]], [[1.2289791107]], [[1.6050475836]], [[1.4706616402]], [[0.5028206110]], [[1.1062363386]], [[1.4302642345]], [[1.3450253010]], [[1.0754106045]], [[1.3278748989]], [[0.1606289148]], [[0.8114572763]], [[1.3531290293]], [[0.5679624081]], [[0.9590579867]], [[1.6721351147]], [[0.7020609379]], [[1.4367213249]], [[0.3588859439]], [[0.7681288719]], [[0.2369352579]], [[0.9368860126]], [[0.6682890654]], [[0.9753562808]], [[0.6410802603]], [[1.1161915064]], [[0.5735361576]], [[1.1730129719]], [[1.5785148144]], [[0.2182148695]], [[1.1429140568]], [[0.9795166254]], [[1.0791989565]], [[0.9102364182]], [[1.1988832951]], [[1.4876472950]], [[1.0009074211]], [[0.7765061855]], [[1.4789731503]], [[0.6572865248]], [[1.5767443180]], [[0.2541809678]], [[0.6900267005]], [[0.5796473026]], [[1.0992655754]], [[1.0592603683]], [[1.1182808876]], [[1.0190036297]], [[0.3883210421]], [[1.0264523029]], [[0.6214287281]], [[0.9969695210]], [[0.9788318872]], [[0.7176420689]], [[0.9674912691]], [[0.9216134548]], [[1.3120691776]], [[1.6706048250]], [[1.2426713705]], [[0.9283027053]], [[1.2112742662]], [[1.5901961327]], [[1.6003880501]], [[1.5835568905]], [[0.4864743948]], [[0.3497454524]], [[0.1814523339]], [[1.2037829161]], [[1.1840472221]], [[0.5919502378]], [[1.9094228745]], [[0.5532948971]], [[1.2106773853]], [[0.7717097402]], [[0.6675022840]], [[0.6241621375]], [[0.9675378203]], [[0.9113206267]], [[0.8160767555]], [[1.4149708748]], [[1.0145411491]], [[0.6493053436]], [[0.5020034909]], [[1.0083611012]], [[0.6005051732]], [[1.3698458672]], [[1.0936900377]], [[1.2760031223]], [[0.4453179836]], [[0.5152152181]], [[1.4365240335]], [[0.4378890395]], [[0.7885467410]], [[0.9955444932]], [[0.9020565152]], [[2.1759154797]], [[1.7242567539]], [[1.7247719765]], [[2.2376446724]], [[2.1554121971]], [[2.1456084251]], [[3.2373390198]], [[2.0226840973]], [[1.7793278694]], [[1.9194684029]], [[2.3578433990]], [[1.2443194389]], [[1.5010688305]], [[2.1223509312]], [[1.8876359463]], [[2.6904981136]], [[2.5207626820]], [[2.4761774540]], [[2.4347386360]], [[2.2321634293]], [[2.1455955505]], [[2.0001199245]], [[2.3880593777]], [[2.3935728073]], [[2.5249481201]], [[1.9547433853]], [[1.7467210293]], [[1.7195193768]], [[1.7913103104]], [[1.7139979601]], [[2.6670789719]], [[2.1136798859]], [[1.8797087669]], [[1.9027192593]], [[1.7361297607]], [[2.7058308125]], [[3.5340986252]], [[2.5094277859]], [[2.5506114960]], [[2.2622520924]], [[1.8474843502]], [[2.9248042107]], [[1.8748657703]], [[1.2411477566]], [[2.1536576748]], [[2.9847826958]], [[2.7151424885]], [[1.2114462852]], [[1.7865415812]], [[2.0170576572]], [[2.4814949036]], [[1.8516347408]], [[2.5014605522]], [[2.6105775833]], [[2.2220439911]], [[2.0961098671]], [[2.0866017342]], [[1.1709206104]], [[1.8126435280]], [[1.6179211140]], [[2.2380850315]], [[1.4771006107]], [[2.3597533703]], [[2.2710781097]], [[1.8896417618]], [[2.5915265083]], [[1.6429624557]], [[1.5886015892]], [[2.1073136330]], [[1.6395814419]], [[2.0321273804]], [[1.6065869331]], [[1.6517692804]], [[1.4759837389]], [[2.2774226665]], [[2.2006874084]], [[2.0009558201]], [[1.7942960262]], [[1.9321477413]], [[2.0235381126]], [[3.1140663624]], [[3.4312355518]], [[2.1400604248]], [[2.1935656071]], [[2.7652175426]], [[2.3137695789]], [[2.2352659702]], [[1.9440398216]], [[2.0560667515]], [[1.4044451714]], [[1.4891583920]], [[1.5467839241]], [[1.7818856239]], [[0.2055360079]], [[1.0670723915]], [[2.7223105431]], [[2.7528223991]], [[1.3801852465]], [[2.8553788662]], [[2.5323066711]], [[1.8435583115]], [[1.5908416510]], [[2.2604393959]], [[2.5230932236]], [[1.9796042442]], [[2.1166603565]], [[1.9290671349]], [[1.7867867947]], [[2.5866184235]], [[2.2943434715]], [[2.5396206379]], [[1.9205739498]], [[2.5329384804]], [[1.6487088203]], [[1.0005974770]], [[1.4878821373]], [[2.6194562912]], [[2.3141603470]], [[2.4838340282]], [[1.8831405640]], [[1.5278707743]], [[2.3314673901]], [[2.2331738472]], [[1.9002315998]], [[1.4117164612]], [[1.5041137934]], [[0.7859941125]], [[1.4901584387]], [[1.9226796627]], [[1.1362335682]], [[1.9958667755]], [[2.0493397713]], [[2.1135797501]], [[1.6833729744]], [[1.4174065590]], [[2.7597968578]], [[1.7768783569]], [[2.5146348476]], [[2.0358600616]], [[2.1433670521]], [[2.0563070774]], [[2.0827476978]], [[1.9773017168]], [[0.7851676941]], [[1.7878917456]], [[2.0760734081]], [[1.8005487919]], [[1.4718626738]], [[0.9004299641]], [[1.5516149998]], [[1.7400314808]], [[1.8370598555]], [[1.5973833799]], [[1.4569392204]], [[1.6522128582]], [[1.9519016743]], [[0.9032585621]], [[0.6062592268]], [[0.8719566464]], [[0.7226560712]], [[1.0909447670]], [[1.6584677696]], [[0.8804082274]], [[1.3794250488]], [[1.8553334475]], [[1.2641222477]], [[1.5975013971]], [[0.5455796123]], [[1.1729831696]], [[2.4576280117]], [[1.3693401814]], [[2.2474820614]], [[1.4944558144]], [[1.7125630379]], [[1.7138364315]], [[0.7685480714]], [[1.3763225079]], [[1.5028872490]], [[0.8477876186]], [[1.5965905190]], [[1.3436954021]], [[1.6175483465]], [[2.0800106525]], [[1.2820600271]], [[1.9504112005]], [[2.1678764820]], [[1.3366255760]], [[0.7485556006]], [[1.5637245178]], [[2.1498286724]], [[1.7368495464]], [[1.7179620266]], [[1.1138291359]], [[1.5931744576]], [[1.2615133524]], [[0.2929256558]], [[1.3057399988]], [[1.1866028309]], [[1.8380421400]], [[0.6603385210]], [[2.3629312515]], [[1.4489544630]], [[1.9032658339]], [[1.8690650463]], [[1.5157265663]], [[1.7362606525]], [[2.4445047379]], [[1.6058927774]], [[1.9649437666]], [[1.6705367565]], [[2.0847716331]], [[1.7347645760]], [[1.3289380074]], [[1.7498219013]], [[1.3358957767]], [[0.9297507405]], [[0.9264358878]], [[2.1652712822]], [[1.1066381931]], [[1.8031578064]], [[2.1782746315]], [[2.2532589436]], [[1.5320241451]], [[1.1947584152]], [[1.3234183788]], [[1.9331967831]], [[1.4187164307]], [[1.0115050077]], [[0.8818412423]], [[1.7016897202]], [[2.4576334953]], [[1.1460577250]], [[1.4337089062]], [[1.2558991909]], [[0.8946968913]], [[2.0609850883]], [[0.3347344995]], [[1.1415576935]], [[2.1017265320]], [[0.9507287145]], [[1.4968940020]], [[1.2866373062]], [[1.3065831661]], [[2.2343881130]], [[1.6466989517]], [[1.1707606316]], [[0.8803624511]], [[0.9066287875]], [[1.4780917168]], [[0.4957841039]], [[1.5298888683]], [[2.3167405128]], [[0.6148149967]], [[1.4957714081]], [[1.6035988331]], [[0.5339369774]], [[1.9479150772]], [[1.3842637539]], [[1.2301250696]], [[0.5092878938]], [[1.0467274189]], [[1.2039580345]], [[2.2465782166]], [[1.2522583008]], [[1.6173950434]], [[1.2209314108]], [[1.1898455620]], [[2.3735904694]], [[2.1262793541]], [[1.7556240559]], [[2.0349695683]], [[0.9185809493]], [[1.1844347715]], [[0.8521307111]], [[1.0460197926]], [[1.6362686157]], [[0.4107335210]], [[2.1157145500]], [[1.7779140472]], [[1.4670205116]], [[1.2228283882]], [[0.1709958315]], [[2.1705262661]], [[1.9011132717]], [[1.6122062206]], [[1.3603255749]], [[1.7164919376]], [[2.2678537369]], [[1.5054094791]], [[1.5238274336]], [[1.7039620876]], [[0.8844705224]], [[2.0892934799]], [[1.5851008892]], [[1.0590093136]], [[1.2530498505]], [[1.7065652609]], [[1.9205608368]], [[1.9870646000]], [[0.5808738470]], [[0.9296095967]], [[1.3903732300]], [[2.0787308216]], [[0.9173107147]], [[0.4609431028]], [[2.0898370743]], [[1.5761603117]], [[0.9788118601]], [[1.4952857494]], [[0.8890246749]], [[1.1265554428]], [[2.0292136669]], [[0.6961540580]], [[1.5720279217]], [[2.4724993706]], [[1.1420192719]], [[1.5323079824]], [[1.7601671219]], [[1.1385703087]], [[1.3094367981]], [[1.5751839876]], [[1.2077670097]], [[1.1140699387]], [[1.7068177462]], [[1.8979969025]], [[2.2886269093]], [[1.6453484297]], [[1.7598607540]], [[1.0593278408]], [[1.7400527000]], [[1.0384204388]], [[2.5367310047]], [[2.0092327595]], [[1.7892118692]], [[1.5112056732]], [[1.3327699900]], [[1.6375617981]], [[1.1504907608]], [[2.4557099342]], [[1.4994196892]], [[0.9540922642]], [[2.3222959042]], [[1.8723379374]], [[1.3199270964]], [[2.6265647411]], [[1.0876674652]], [[2.4185976982]], [[0.4304662347]], [[1.5272073746]], [[1.4282702208]], [[1.4492261410]], [[1.7424030304]], [[1.1136257648]], [[0.9556840658]], [[1.7302389145]], [[1.2602646351]], [[2.4743216038]], [[1.0944118500]], [[2.1612396240]], [[0.7427238822]], [[0.5044853687]], [[1.2795805931]], [[1.9709289074]], [[1.9602879286]], [[2.2578313351]], [[2.2125241756]], [[1.1943802834]], [[1.4241292477]], [[1.8547847271]], [[1.1571869850]], [[1.8476715088]]]])
test_add.test_module[Add1-[((1, 960, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6389201880]], [[0.9108853936]], [[0.2311410308]], [[0.2746940851]], [[0.4500864148]], [[0.7767422795]], [[0.6327570081]], [[1.0391082764]], [[0.5982915759]], [[0.7749698758]], [[0.4915570617]], [[0.5443809032]], [[0.1649893522]], [[0.3115225434]], [[0.4365520477]], [[0.6611853838]], [[0.8403311968]], [[0.9426749945]], [[0.3036930561]], [[0.4249321818]], [[0.8242721558]], [[1.0578575134]], [[0.5397635102]], [[1.0168194771]], [[0.5620719194]], [[0.6955706477]], [[1.0954017639]], [[0.1788284183]], [[0.3278946280]], [[0.5160809755]], [[0.4477636218]], [[1.0746639967]], [[0.3185737729]], [[0.4124971628]], [[0.2933433652]], [[0.1743831038]], [[0.3507933617]], [[1.0724625587]], [[0.8657727838]], [[0.8849998713]], [[0.6689593792]], [[0.3863218427]], [[0.7272559404]], [[0.1758162379]], [[0.2813804746]], [[0.3848986030]], [[0.9581325650]], [[0.9358242154]], [[0.4209160805]], [[0.6246224046]], [[0.9624439478]], [[1.1397302151]], [[0.8411046863]], [[0.7102100253]], [[0.9779067636]], [[0.3482624292]], [[0.7358356118]], [[0.2550108433]], [[0.2961205244]], [[0.3843718171]], [[0.8689001203]], [[0.8437438011]], [[0.3464873433]], [[0.7937171459]], [[0.9171496034]], [[0.5795549154]], [[0.6617543697]], [[0.7585159540]], [[0.9528518915]], [[1.1227606535]], [[0.2573518157]], [[0.4594287276]], [[0.8391685486]], [[1.0569382906]], [[1.0777672529]], [[1.0838420391]], [[0.7421708703]], [[0.2078722715]], [[0.6886598468]], [[0.3298609257]], [[0.1766865253]], [[1.0869097710]], [[1.0228434801]], [[0.1438996196]], [[0.7362496257]], [[0.5584335923]], [[0.5603830218]], [[0.4137851596]], [[0.8349416852]], [[0.3465118408]], [[0.8259592652]], [[0.8955176473]], [[1.0005993843]], [[0.8296191692]], [[0.1477959752]], [[0.3183151484]], [[0.8923211098]], [[0.7473142743]], [[0.2526215911]], [[0.3547538519]], [[1.1130383015]], [[0.9795725346]], [[0.4246510267]], [[0.5168212056]], [[0.1663645506]], [[0.6336765289]], [[0.2661341429]], [[0.2569852471]], [[0.6151137948]], [[0.7177361250]], [[0.4378984571]], [[0.9393523932]], [[0.3383940458]], [[1.0963486433]], [[0.9853135347]], [[0.2210221291]], [[0.5182214379]], [[0.6652249098]], [[0.7156141400]], [[0.7612507343]], [[0.8388777375]], [[0.6726136804]], [[0.3986992240]], [[0.8792580962]], [[0.1630391479]], [[0.3463102579]], [[0.5174986720]], [[0.3991069198]], [[0.4677469134]], [[0.2328527570]], [[0.5363060236]], [[0.7495418191]], [[0.3169307113]], [[0.6170039177]], [[1.0005890131]], [[0.5912634730]], [[0.6565597057]], [[0.5995291471]], [[0.7438542843]], [[0.9605833292]], [[1.1162867546]], [[0.9601915479]], [[1.1173703671]], [[0.6065027714]], [[0.1935028434]], [[0.4056249857]], [[0.9831162095]], [[0.6394223571]], [[0.3941404223]], [[0.2595077157]], [[0.1747375727]], [[0.2206594944]], [[0.5412452221]], [[0.9168666005]], [[0.9129841328]], [[0.1604476571]], [[0.9545546174]], [[0.2514088750]], [[0.5369584560]], [[0.4399272799]], [[0.5463559628]], [[0.5444922447]], [[0.1939886212]], [[0.2109446526]], [[0.5644238591]], [[0.6491296887]], [[0.4155261517]], [[0.8310132027]], [[0.1926344037]], [[0.6089199781]], [[1.0823733807]], [[0.4387176037]], [[1.0941650867]], [[0.8237404823]], [[0.1914331317]], [[0.9590122700]], [[0.5849663615]], [[0.4194601774]], [[1.0424902439]], [[0.2386159897]], [[0.6963160634]], [[0.5379792452]], [[0.9997199178]], [[0.7822358608]], [[0.8829162717]], [[0.8192430735]], [[0.5224262476]], [[0.5375108719]], [[0.2306228876]], [[0.9135856628]], [[1.0396525860]], [[0.9847760201]], [[0.2899744511]], [[0.6649635434]], [[0.2901965380]], [[0.3674215078]], [[0.3513108492]], [[0.8135361075]], [[0.3447070122]], [[0.6317549944]], [[0.6636976600]], [[0.9649751186]], [[0.2647035718]], [[0.2994074821]], [[0.3523328304]], [[0.9926303029]], [[0.4629310966]], [[1.0644079447]], [[0.8234673738]], [[0.7059766054]], [[0.6389415860]], [[0.5438228250]], [[0.7053967714]], [[0.5284912586]], [[0.6391503811]], [[0.7064601183]], [[0.2515610456]], [[0.3805978894]], [[1.0464099646]], [[0.2368902564]], [[0.6067605019]], [[1.1372829676]], [[0.8232821226]], [[0.6568201184]], [[0.2093586326]], [[0.8903525472]], [[0.2865232229]], [[0.5007314682]], [[0.4749054313]], [[0.5686199069]], [[0.6481327415]], [[1.0550673008]], [[0.7050830126]], [[1.0905100107]], [[0.9485198259]], [[0.3265566230]], [[0.8669156432]], [[0.2892155647]], [[0.4307510257]], [[0.7897249460]], [[0.8077596426]], [[1.0177776814]], [[0.4817056656]], [[0.6434640288]], [[0.9000753760]], [[0.1591175199]], [[1.0041539669]], [[0.2292023897]], [[0.6495548487]], [[0.5576552153]], [[0.3793271184]], [[0.7087491155]], [[1.0561230183]], [[0.4965038300]], [[0.3458165526]], [[0.4577441812]], [[0.1470894217]], [[0.8683605790]], [[0.4025317430]], [[0.3089934587]], [[0.3546128869]], [[0.9301416278]], [[0.9074504375]], [[1.0264245272]], [[0.8240251541]], [[0.4756837487]], [[0.5029395223]], [[0.7903785706]], [[1.0536808968]], [[0.7786096931]], [[0.4060909152]], [[0.4076249003]], [[0.1699290872]], [[0.7506875992]], [[0.3620723486]], [[0.1968756914]], [[1.0811018944]], [[0.3179579973]], [[0.5857828259]], [[0.7859103680]], [[0.6585927606]], [[0.3062227368]], [[0.2385027409]], [[1.0412048101]], [[0.7240829468]], [[1.0574758053]], [[0.4750432968]], [[0.7899413109]], [[0.5283254981]], [[0.6204280257]], [[0.3381415009]], [[0.8117640615]], [[0.8007465601]], [[0.6323493123]], [[0.5302125216]], [[0.3344486952]], [[0.9884359837]], [[0.2704515457]], [[0.8474963307]], [[0.4745368361]], [[0.4014295936]], [[0.7324861288]], [[0.3829351068]], [[0.7578660250]], [[0.7408540249]], [[0.2714188695]], [[0.7259128690]], [[0.8556281924]], [[0.8405790925]], [[0.5797242522]], [[0.2327697873]], [[0.5655865669]], [[0.8163174391]], [[0.4602250457]], [[0.8324541450]], [[0.9756448865]], [[0.3815878034]], [[0.6475945115]], [[0.8494130969]], [[0.6818525195]], [[0.6844293475]], [[0.7050963044]], [[0.2495897412]], [[0.6819577217]], [[0.9888985157]], [[1.0932204723]], [[0.9365390539]], [[0.7096651196]], [[0.8761706948]], [[0.3994237781]], [[0.2283194065]], [[0.2127030492]], [[1.1414685249]], [[0.9600582719]], [[0.2970467210]], [[0.8382849097]], [[1.0202474594]], [[1.1424710751]], [[1.0798610449]], [[1.0300405025]], [[0.5280339718]], [[0.4671927691]], [[1.0531880856]], [[0.9228534102]], [[0.3417741060]], [[1.0921721458]], [[0.8842428923]], [[0.9152313471]], [[0.3292754292]], [[0.7861135006]], [[0.4673823714]], [[1.0333418846]], [[0.5526933074]], [[0.8373198509]], [[0.7314745188]], [[0.8553977013]], [[0.4727498293]], [[0.8864492774]], [[0.2934281230]], [[0.7555910945]], [[0.3043676615]], [[0.1493948102]], [[0.2411357164]], [[1.0373985767]], [[0.9131814241]], [[1.1117714643]], [[1.0432242155]], [[0.1961408854]], [[0.3014480472]], [[0.5618723035]], [[0.3179474473]], [[0.9898618460]], [[0.2646597028]], [[0.3987128735]], [[0.1596179008]], [[0.3587908149]], [[1.0539023876]], [[1.0520436764]], [[1.0005824566]], [[1.0287272930]], [[1.0872569084]], [[0.5146321058]], [[0.8626646399]], [[1.0881288052]], [[0.8080729842]], [[1.1425018311]], [[0.9019950628]], [[0.9534877539]], [[0.4676675200]], [[0.8825773597]], [[0.7001168728]], [[0.5232549310]], [[0.3608086705]], [[0.3621053100]], [[0.2579231858]], [[0.9783287644]], [[0.9981291890]], [[0.5857572556]], [[0.3533205986]], [[1.0291163921]], [[0.9624080658]], [[0.6798306108]], [[0.4065959454]], [[1.1021875143]], [[0.8471339345]], [[0.2630909085]], [[1.1212050915]], [[1.0223505497]], [[0.4604181647]], [[0.9237408638]], [[0.3585690856]], [[0.5643064380]], [[1.0672142506]], [[0.6633231640]], [[0.2890545130]], [[0.4755471349]], [[0.5069410205]], [[0.5461992025]], [[0.6905139089]], [[1.1050784588]], [[0.6694337726]], [[0.3339436054]], [[0.6682907343]], [[0.8824071884]], [[0.8906837106]], [[0.1856943369]], [[0.5531923771]], [[0.2710913420]], [[0.4293208122]], [[0.8228102922]], [[0.2875984907]], [[0.8285270333]], [[1.0670526028]], [[0.6754577756]], [[0.3094197512]], [[0.4635211229]], [[0.7518456578]], [[0.2615045905]], [[0.8910687566]], [[0.1887288094]], [[0.1620169282]], [[0.1568332911]], [[0.5412319303]], [[0.9788800478]], [[0.1694241166]], [[1.0582571030]], [[0.4426521659]], [[0.7891056538]], [[0.6654650569]], [[0.1918040514]], [[1.0573281050]], [[0.9118853211]], [[1.1396614313]], [[0.8952696919]], [[0.3126291037]], [[1.0599554777]], [[0.6695358157]], [[0.8797718287]], [[0.2417490482]], [[0.4988503456]], [[0.1517248154]], [[0.4479173422]], [[0.7505291700]], [[0.2500827312]], [[0.8020457029]], [[0.9110670090]], [[0.7123182416]], [[0.3081219196]], [[0.2550037503]], [[0.4884052873]], [[0.8621426821]], [[1.1358618736]], [[0.9301781058]], [[0.5863589644]], [[0.8179718256]], [[0.1521321535]], [[0.2156129479]], [[0.8759675622]], [[0.3594560027]], [[0.8832114339]], [[0.2896975279]], [[0.3950081468]], [[0.2308191061]], [[0.9035839438]], [[0.5917160511]], [[1.0274641514]], [[0.9520996809]], [[0.9193348885]], [[0.6587416530]], [[0.4880746007]], [[0.5339477658]], [[0.7091138959]], [[0.8905183673]], [[0.2923686504]], [[1.0623025894]], [[0.5882985592]], [[0.2236908078]], [[0.3721346855]], [[1.0850725174]], [[1.0999271870]], [[0.1795237660]], [[0.9953127503]], [[0.8932431936]], [[0.9386228323]], [[1.0659282207]], [[0.3731884360]], [[0.8005514741]], [[0.8472802043]], [[0.4949169159]], [[0.8099901676]], [[0.4988068938]], [[0.9517942667]], [[0.5039362907]], [[0.4562657475]], [[0.7685381174]], [[0.8200104237]], [[0.3983777761]], [[0.6868627667]], [[0.9324314594]], [[0.5929152369]], [[0.7948305607]], [[0.5220701098]], [[0.8179134130]], [[0.2804664969]], [[0.3486492038]], [[0.3888683319]], [[1.1021690369]], [[0.5081215501]], [[0.6412984133]], [[0.4004158378]], [[1.1418086290]], [[1.1309986115]], [[0.2655702829]], [[0.2373286486]], [[0.2636671066]], [[0.6402522326]], [[0.5152116418]], [[0.3153907657]], [[0.4633275270]], [[0.7371290326]], [[0.3814182281]], [[0.7534548640]], [[0.5280056596]], [[0.4003819227]], [[0.7113563418]], [[1.0537927151]], [[0.3046239614]], [[0.6658807993]], [[0.4582797289]], [[1.1333167553]], [[0.1682819724]], [[0.1633329988]], [[1.1353560686]], [[0.3263186216]], [[0.7385252714]], [[0.5995105505]], [[0.5373350978]], [[0.5309897065]], [[0.9603838921]], [[0.6665620804]], [[0.1558563113]], [[0.3474835157]], [[0.4722134471]], [[0.8942717910]], [[0.3190967441]], [[1.1141233444]], [[0.5312989354]], [[0.5528826118]], [[1.0344426632]], [[0.8939674497]], [[1.0667238235]], [[0.9318857789]], [[0.4909866452]], [[0.3109273911]], [[0.6054620743]], [[1.0564823151]], [[0.4748536944]], [[0.1789786220]], [[0.8476278186]], [[1.1293992996]], [[0.5003219843]], [[0.2286518216]], [[0.1891337633]], [[0.7679632902]], [[0.6048037410]], [[0.3901699781]], [[0.7437295318]], [[0.8325430155]], [[1.0403230190]], [[1.0308547020]], [[0.5678217411]], [[0.2017800212]], [[0.1908520460]], [[1.1095083952]], [[0.8636912107]], [[0.8606172800]], [[0.2100530863]], [[1.1056714058]], [[1.1163351536]], [[1.0940937996]], [[0.2208710909]], [[0.4540019035]], [[0.2987816930]], [[1.1161463261]], [[0.4278236032]], [[0.4143863320]], [[0.9046200514]], [[0.4113674164]], [[0.3964025378]], [[0.5989278555]], [[0.5946038365]], [[0.2531743646]], [[1.0594975948]], [[0.4221027493]], [[0.8200210333]], [[1.0775554180]], [[0.8948218226]], [[0.7134535313]], [[1.0680921078]], [[0.7098767161]], [[0.4113352895]], [[1.1156632900]], [[0.7610038519]], [[0.1548231840]], [[0.5003178120]], [[0.3020798564]], [[1.0810401440]], [[0.5600906610]], [[0.1869011521]], [[0.6112319231]], [[0.9566700459]], [[0.7725638151]], [[0.8007658124]], [[0.6891002059]], [[0.8290729523]], [[0.5208376050]], [[0.4437634945]], [[0.1753093600]], [[0.2659951448]], [[0.8593648672]], [[0.3466069102]], [[0.7143968344]], [[0.8022593260]], [[0.6780655384]], [[0.3184887171]], [[1.1208277941]], [[0.3519138694]], [[1.0539138317]], [[0.2449094653]], [[0.5223893523]], [[0.9146479964]], [[0.4383732677]], [[1.0626850128]], [[0.2985853553]], [[0.2227778435]], [[0.4172040224]], [[0.7234888077]], [[1.1030415297]], [[0.4039631486]], [[0.8214777112]], [[0.5173100829]], [[0.5342324376]], [[1.0103152990]], [[0.2551706433]], [[0.6957702637]], [[1.1128681898]], [[0.5739575028]], [[1.0308825970]], [[0.4886851907]], [[1.0451446772]], [[0.1589984298]], [[0.5706017613]], [[0.5548403859]], [[0.8047060966]], [[0.8388273120]], [[1.0265719891]], [[0.5681706071]], [[0.6228360534]], [[0.9850692153]], [[0.5073770285]], [[1.0809804201]], [[0.3097575307]], [[0.5885586739]], [[0.6158081293]], [[0.8657685518]], [[0.9845016599]], [[0.5634117126]], [[0.2284232974]], [[0.8904097080]], [[0.7922294736]], [[0.8435132504]], [[0.3342297673]], [[0.9644613862]], [[1.1162410975]], [[0.6860492826]], [[0.1756393909]], [[0.9936585426]], [[0.2719379663]], [[0.7575996518]], [[0.7153224945]], [[0.4086453319]], [[0.8167613745]], [[0.1954472661]], [[0.7565433383]], [[0.3256881237]], [[0.5885967612]], [[0.7069780231]], [[1.0686358213]], [[0.4041034579]], [[0.9629816413]], [[0.5791466832]], [[0.4051997066]], [[0.2072637677]], [[0.1839044094]], [[1.1309689283]], [[0.5179638267]], [[0.6676380038]], [[0.7782197595]], [[0.9825296998]], [[1.0694006681]], [[1.0481722355]], [[0.2722220421]], [[0.5625551939]], [[0.3467947841]], [[0.3569877148]], [[0.7612671256]], [[1.1119875908]], [[0.2421116233]], [[0.9452658296]], [[0.3834302425]], [[0.5452764630]], [[1.0395491123]], [[0.5295784473]], [[0.6881914735]], [[0.2931473255]], [[1.0682840347]], [[0.5780286789]], [[0.2769683599]], [[0.7890687585]], [[0.2871549129]], [[0.2459099293]], [[0.6731231809]], [[1.0390853882]], [[0.5011715889]], [[0.8779970407]], [[1.0723242760]], [[0.9742940664]], [[0.3803830743]], [[0.5878536701]], [[0.4853211641]], [[0.2406021357]], [[0.6428701878]], [[1.0188810825]], [[1.0639214516]], [[0.6893139482]], [[0.7562251091]], [[0.4262027740]], [[1.0200848579]], [[0.4346093535]], [[0.2953372002]], [[0.7196603417]], [[0.9423562884]], [[0.1918678284]], [[1.0946474075]], [[0.8225291371]], [[0.2923572063]], [[0.5349598527]], [[1.0764535666]], [[0.2590462565]], [[0.4965249896]], [[0.8066537380]], [[0.2046210766]], [[0.9167627096]], [[0.9029374123]], [[0.9527663589]], [[0.3238900304]], [[1.1406641006]], [[0.3462828398]], [[1.1418337822]], [[0.1628183722]], [[0.1971786022]], [[0.9497617483]], [[0.6949200630]], [[0.6715082526]], [[0.3657844067]], [[0.4329258800]], [[0.4964698553]], [[0.1555858850]], [[0.6686479449]], [[0.7310930490]], [[0.6422402859]], [[0.8041287661]], [[1.1170890331]], [[0.7756056190]], [[0.4596273899]], [[0.4368902445]], [[0.3227608204]], [[0.2960541844]], [[0.5621386766]], [[0.5542308092]], [[0.8650948405]], [[0.4289451838]], [[1.0412654877]], [[0.2918235064]], [[0.6440846920]], [[1.0921235085]], [[1.1398561001]], [[0.3530324697]], [[0.7317280769]], [[0.7017249465]], [[0.4082377553]], [[0.4699161649]], [[0.7780905962]], [[0.2949810028]], [[0.7251614332]], [[0.8590259552]], [[0.4456285238]], [[1.0579841137]], [[0.6097583771]], [[0.8695199490]], [[1.1378167868]], [[0.4898294210]], [[0.9143676758]], [[0.4996489882]], [[0.5696271062]], [[0.5579280257]], [[0.6395573020]], [[0.4537793994]], [[0.7598570585]], [[0.6615037918]], [[0.9596084952]], [[0.5414609313]], [[0.6928036809]], [[0.4566684365]], [[0.2239367366]], [[0.8449972272]], [[0.7066425681]], [[0.4424217343]], [[0.4736111164]], [[0.7734242678]], [[0.5522551537]], [[0.9694015384]], [[0.6709851623]], [[0.8315319419]], [[0.8605117202]], [[0.5199804306]], [[0.8589960337]], [[1.0116302967]], [[0.6656727791]], [[0.7405100465]], [[0.6608313918]], [[0.9888473153]], [[0.4315659404]], [[0.3768786788]], [[0.8605403900]], [[0.2075065374]], [[0.6506798267]], [[0.4127180576]], [[0.9726803899]], [[0.1771440506]], [[0.9458298683]], [[1.1373419762]], [[0.7438336611]], [[0.6093502641]], [[1.1348385811]], [[0.4289874434]], [[0.6025968194]], [[0.4237111211]], [[0.5736429691]], [[0.7513330579]], [[0.7094442844]], [[0.5482261181]], [[0.1427432895]], [[0.6682656407]], [[0.6278526187]], [[0.7178819776]], [[1.0257887840]], [[1.1286683083]], [[0.3460223675]], [[0.6114863753]], [[0.4423267841]], [[0.1818407774]], [[0.2788340449]], [[1.1119441986]], [[0.9341941476]], [[0.9112391472]], [[0.9205352068]], [[0.2454240918]], [[0.6951732039]], [[1.1080209017]], [[0.3636206388]], [[1.0874607563]], [[0.7740828395]], [[0.9944353700]], [[0.4284697771]], [[0.8731724620]], [[0.1994197369]], [[0.6109158397]], [[0.8094391823]], [[0.7926108241]], [[1.0610675812]], [[1.1339921951]], [[1.0975420475]], [[0.9783596992]], [[0.9941946268]], [[0.5781487226]], [[0.2546401620]], [[0.4570500255]], [[0.5865313411]], [[0.3724703193]], [[0.8982490301]], [[0.8181772232]], [[0.7233395576]], [[0.7634439468]], [[1.0818897486]], [[0.8247792721]], [[0.2041938901]], [[0.2797232270]], [[0.8630384207]], [[0.7108032107]], [[0.8865094185]], [[0.1433036327]], [[0.1812289357]], [[0.9492768049]], [[0.9624864459]], [[0.1903080940]], [[0.8324562311]], [[0.2522414923]], [[1.0213174820]], [[0.7995588183]], [[1.1370570660]], [[0.1496676207]], [[0.1613255143]], [[0.9807146192]], [[1.0865751505]], [[0.9490035772]], [[0.9750785232]], [[0.2215279341]], [[1.0045704842]], [[0.1705900431]], [[0.7543644309]], [[0.3151462078]], [[0.4440658689]], [[0.7311980128]], [[0.4460735917]], [[0.5635521412]], [[0.1687514186]], [[0.1712269783]], [[0.8363451362]], [[0.4310243726]], [[0.3384687901]], [[0.4453724623]], [[0.2338793278]], [[0.2429634929]], [[0.9418497682]], [[0.8967846632]], [[0.7092653513]], [[0.3695337772]], [[0.8090140820]], [[0.6651262045]]]]), compiled_model=tensor([[[[0.6389201880]], [[1.2988183498]], [[0.4935047030]], [[0.8327759504]], [[0.7430298924]], [[1.4203960896]], [[0.5478624701]], [[1.2198748589]], [[1.0463795662]], [[0.9424073696]], [[0.4326074719]], [[1.0653885603]], [[0.6656162739]], [[1.7549519539]], [[1.7778736353]], [[1.6280913353]], [[2.3371217251]], [[2.8266682625]], [[1.7233885527]], [[2.8066430092]], [[2.0063772202]], [[2.2574613094]], [[2.2264621258]], [[2.8090763092]], [[0.9900739193]], [[1.7364144325]], [[2.3852984905]], [[1.0908193588]], [[2.6368048191]], [[2.3892490864]], [[2.0745279789]], [[2.8275501728]], [[1.8663077354]], [[1.8529963493]], [[2.0438735485]], [[1.8804848194]], [[1.7554677725]], [[2.5041904449]], [[1.8839771748]], [[1.9220557213]], [[1.5123081207]], [[1.7782084942]], [[2.2129344940]], [[2.6301920414]], [[1.2535920143]], [[1.3045489788]], [[2.5638051033]], [[1.8336691856]], [[2.1064994335]], [[3.0162501335]], [[2.3920106888]], [[2.6603012085]], [[2.5753228664]], [[2.0905015469]], [[2.8495230675]], [[1.3002748489]], [[1.6352093220]], [[1.3164962530]], [[2.3966603279]], [[2.1842830181]], [[1.7510709763]], [[1.8441718817]], [[1.8961627483]], [[2.2418699265]], [[2.2160911560]], [[2.2436954975]], [[2.5408573151]], [[2.1251621246]], [[2.5762119293]], [[2.3229131699]], [[1.1348443031]], [[1.5164811611]], [[2.1527218819]], [[3.1456284523]], [[2.3137321472]], [[2.4061968327]], [[2.1000676155]], [[0.9857425690]], [[2.2369620800]], [[1.7766824961]], [[1.4638401270]], [[2.6323511600]], [[2.3444774151]], [[1.1861650944]], [[2.0781769753]], [[1.8114900589]], [[1.8767488003]], [[2.3324170113]], [[1.9817268848]], [[1.2954241037]], [[1.6196727753]], [[1.7989380360]], [[1.9368805885]], [[3.4290533066]], [[2.7163670063]], [[1.3702468872]], [[2.2778139114]], [[2.3700301647]], [[1.6643960476]], [[1.6365320683]], [[2.5894105434]], [[2.1530618668]], [[1.1289794445]], [[1.4827246666]], [[1.3523398638]], [[2.0534567833]], [[0.2137469649]], [[0.3457288146]], [[2.3392951488]], [[2.8848013878]], [[1.4647631645]], [[2.7656149864]], [[1.9082925320]], [[2.2600765228]], [[2.1695592403]], [[1.3792740107]], [[2.1941807270]], [[2.3817381859]], [[1.7110695839]], [[1.6679674387]], [[2.1652464867]], [[2.3354914188]], [[2.3344736099]], [[2.8545722961]], [[1.0163989067]], [[2.2159254551]], [[1.8771529198]], [[0.9241573215]], [[1.4486880302]], [[2.3061099052]], [[2.1599524021]], [[2.1282973289]], [[1.5306375027]], [[1.8109310865]], [[2.6637656689]], [[1.9420301914]], [[1.6661075354]], [[1.8255512714]], [[1.6947757006]], [[1.4754860401]], [[2.1771242619]], [[2.0600609779]], [[1.9660054445]], [[1.7738424540]], [[1.1757898331]], [[1.8437470198]], [[2.3570694923]], [[1.5933077335]], [[2.4020915031]], [[1.7748816013]], [[1.7983037233]], [[2.0677905083]], [[2.5225954056]], [[2.8163404465]], [[2.4544999599]], [[1.1588692665]], [[1.5702981949]], [[0.9810434580]], [[2.1703796387]], [[1.4513704777]], [[1.3527536392]], [[1.2531181574]], [[0.6882754564]], [[1.0390908718]], [[1.2618222237]], [[1.3512433767]], [[1.5598362684]], [[1.4232705832]], [[1.4750003815]], [[0.6324067116]], [[1.4468834400]], [[0.8118239045]], [[1.6650964022]], [[1.4667679071]], [[1.0993716717]], [[1.5893378258]], [[1.1623456478]], [[1.3637266159]], [[1.5942943096]], [[1.5279954672]], [[0.9868919253]], [[1.2225570679]], [[2.5952053070]], [[1.0157142878]], [[2.2002203465]], [[1.7273399830]], [[1.4170174599]], [[2.0992150307]], [[0.7835580111]], [[1.4139406681]], [[2.1830840111]], [[0.9493522048]], [[1.5968674421]], [[1.6136507988]], [[1.6769257784]], [[1.5438482761]], [[1.0416548252]], [[1.7364830971]], [[1.5604838133]], [[1.0490456820]], [[0.7535116076]], [[2.0406250954]], [[1.8805844784]], [[1.3271431923]], [[1.1797764301]], [[1.8140907288]], [[0.9938030243]], [[1.7376227379]], [[0.8927022219]], [[1.6395819187]], [[0.7404720187]], [[1.2819378376]], [[1.1862115860]], [[1.8961097002]], [[1.1948616505]], [[1.6711031199]], [[1.0546979904]], [[1.5231361389]], [[1.9821190834]], [[1.8341147900]], [[1.7177363634]], [[2.2922365665]], [[1.9950120449]], [[1.7897975445]], [[1.4401869774]], [[1.7630248070]], [[1.2678070068]], [[1.0166168213]], [[1.0062783957]], [[0.8081930280]], [[1.8809726238]], [[1.5687904358]], [[1.7134102583]], [[2.7467145920]], [[2.3838653564]], [[1.5781142712]], [[1.7130248547]], [[1.2237656116]], [[1.2617788315]], [[1.7003198862]], [[1.1779662371]], [[1.4992029667]], [[1.0415866375]], [[1.9700987339]], [[1.7805628777]], [[1.3554978371]], [[1.9963860512]], [[0.4836470485]], [[2.1953282356]], [[0.5769989491]], [[1.0575573444]], [[2.0733466148]], [[1.6254334450]], [[1.2399430275]], [[1.1044481993]], [[1.3639454842]], [[1.6701211929]], [[1.4612668753]], [[1.2686684132]], [[0.5234751105]], [[0.8029619455]], [[1.2749166489]], [[1.2349525690]], [[2.3929803371]], [[2.0054097176]], [[0.7641801238]], [[1.2601857185]], [[1.7944668531]], [[1.0502828360]], [[2.1955349445]], [[0.8299707770]], [[0.9711878896]], [[0.5233606696]], [[1.4499315023]], [[1.0938169956]], [[1.5491821766]], [[2.0142633915]], [[0.8212296367]], [[1.2754153013]], [[1.4228732586]], [[1.9977407455]], [[1.5385346413]], [[0.9274030328]], [[2.1442885399]], [[1.1516772509]], [[1.9309833050]], [[0.7217119336]], [[0.7794787884]], [[1.6897404194]], [[0.8521829247]], [[1.6062281132]], [[1.4602787495]], [[1.7674450874]], [[1.6265258789]], [[0.5120745897]], [[1.7370116711]], [[2.2847456932]], [[1.4924877882]], [[1.4640924931]], [[1.3584856987]], [[1.6289601326]], [[1.2070409060]], [[1.3389408588]], [[2.2600481510]], [[1.4344725609]], [[1.2512040138]], [[1.4473224878]], [[1.0540201664]], [[1.8835759163]], [[1.1806180477]], [[1.0369954109]], [[1.4585574865]], [[1.1763201952]], [[0.9358327389]], [[1.9240455627]], [[1.9382293224]], [[0.8710749149]], [[0.6941512823]], [[2.0346300602]], [[1.8466453552]], [[1.2668386698]], [[1.6014542580]], [[0.5440105796]], [[1.5553388596]], [[1.9586144686]], [[1.3672717810]], [[1.6885459423]], [[2.1046092510]], [[1.1233680248]], [[1.2182781696]], [[0.9203943610]], [[0.6413967013]], [[2.0395698547]], [[1.4195789099]], [[0.7438098192]], [[1.7975316048]], [[2.2267472744]], [[2.7383880615]], [[2.2874479294]], [[2.1152982712]], [[2.1009936333]], [[0.9152886271]], [[1.8365706205]], [[1.1887100935]], [[2.0777392387]], [[2.4123048782]], [[1.8443818092]], [[1.9055993557]], [[1.2182819843]], [[2.2483658791]], [[1.3518779278]], [[2.6296868324]], [[1.7055060863]], [[1.0770152807]], [[2.2515110970]], [[2.0496702194]], [[1.4741882086]], [[2.3921861649]], [[1.0291817188]], [[2.1202750206]], [[0.4899244308]], [[1.1542129517]], [[0.7547579408]], [[2.0482516289]], [[1.5928993225]], [[1.9268118143]], [[1.7761304379]], [[1.5091757774]], [[0.8382238150]], [[1.9331524372]], [[1.0083961487]], [[2.3296236992]], [[0.4900735021]], [[0.3689658046]], [[0.4288831949]], [[2.0745491982]], [[2.3184199333]], [[2.1970067024]], [[2.6391491890]], [[1.1922249794]], [[2.0227007866]], [[1.3242721558]], [[1.8608531952]], [[2.3651986122]], [[1.5856065750]], [[1.5383391380]], [[0.9307010770]], [[1.1801809072]], [[1.6118710041]], [[2.1742095947]], [[1.7657295465]], [[2.0826246738]], [[1.7768194675]], [[1.4551670551]], [[1.7576396465]], [[2.3710260391]], [[1.7638989687]], [[1.6531395912]], [[0.8251370192]], [[1.6096036434]], [[1.1304185390]], [[1.5766422749]], [[0.7961763740]], [[2.2476558685]], [[1.2625426054]], [[1.4407041073]], [[1.9037697315]], [[2.2319669724]], [[1.0762115717]], [[1.3851712942]], [[1.2678990364]], [[1.7583084106]], [[2.1950538158]], [[2.0549750328]], [[1.1255359650]], [[1.4896340370]], [[1.8566446304]], [[1.9482378960]], [[1.5724108219]], [[1.9313230515]], [[1.5113176107]], [[0.8884894848]], [[1.9501223564]], [[1.3380683661]], [[1.6201450825]], [[1.2206969261]], [[1.6130194664]], [[1.9404928684]], [[1.3846451044]], [[1.2795050144]], [[1.1254463196]], [[1.4127736092]], [[1.3131471872]], [[1.6607258320]], [[1.0869407654]], [[1.6200883389]], [[1.5962095261]], [[0.7536966801]], [[1.6620368958]], [[0.2924330831]], [[1.1222553253]], [[0.2718600631]], [[1.7468438148]], [[2.4895768166]], [[0.3518910408]], [[1.4085410833]], [[1.6820220947]], [[2.1915731430]], [[2.1295261383]], [[1.3358027935]], [[1.8972176313]], [[1.2454270124]], [[1.6742912531]], [[0.9374002814]], [[0.5569246411]], [[2.1021339893]], [[1.6636974812]], [[1.5155081749]], [[0.9996660352]], [[1.8389116526]], [[0.9183868766]], [[1.5760370493]], [[1.4336316586]], [[1.2631682158]], [[1.7537162304]], [[1.8029299974]], [[0.7939642072]], [[1.0934326649]], [[1.2033085823]], [[1.5729315281]], [[2.1266028881]], [[2.0662882328]], [[1.9578158855]], [[1.2479333878]], [[1.7894411087]], [[1.1774518490]], [[0.6499646902]], [[2.0856971741]], [[1.2668957710]], [[0.8197728395]], [[1.1062601805]], [[0.8672386408]], [[1.4735059738]], [[1.1799733639]], [[1.7347635031]], [[2.0619297028]], [[1.5214183331]], [[2.1762106419]], [[2.2315676212]], [[1.4822542667]], [[0.8232965469]], [[1.1214367151]], [[2.4820446968]], [[0.8938096762]], [[1.9327883720]], [[1.6334704161]], [[1.2296009064]], [[1.6991429329]], [[1.7009205818]], [[1.4806916714]], [[0.4004490376]], [[1.9594316483]], [[1.1161067486]], [[1.6140422821]], [[1.6396151781]], [[0.8340461254]], [[1.9471063614]], [[1.9913508892]], [[0.5819440484]], [[1.6814767122]], [[1.9434244633]], [[1.4819810390]], [[0.7639879584]], [[1.4381244183]], [[1.5458033085]], [[1.8890557289]], [[1.3906540871]], [[1.1733026505]], [[1.2506834269]], [[1.8914725780]], [[0.8234792948]], [[1.0396403074]], [[1.1780738831]], [[0.4264145494]], [[1.0300681591]], [[0.3761454225]], [[1.8356484175]], [[1.5349524021]], [[1.1758304834]], [[1.5201163292]], [[2.4134454727]], [[2.0978548527]], [[1.6189188957]], [[0.8855298758]], [[1.5625845194]], [[1.6539891958]], [[2.2259950638]], [[1.6356565952]], [[2.2437901497]], [[1.4957149029]], [[0.4436801076]], [[1.4788613319]], [[1.3157564402]], [[1.6061086655]], [[1.7302756310]], [[1.0858030319]], [[0.9462101460]], [[1.1579592228]], [[1.6591892242]], [[2.4637479782]], [[0.9327768087]], [[0.7732331753]], [[1.9461920261]], [[0.7964320183]], [[1.3217235804]], [[1.0169489384]], [[1.2107156515]], [[1.0532023907]], [[1.1558041573]], [[1.3351438046]], [[0.7843061090]], [[1.4814438820]], [[1.2847602367]], [[2.3094558716]], [[0.4327772856]], [[1.5439014435]], [[1.6418018341]], [[1.6699597836]], [[2.4799439907]], [[1.7508995533]], [[1.8534994125]], [[1.8582549095]], [[1.7099490166]], [[0.9781553149]], [[2.3281936646]], [[1.5823061466]], [[1.6840436459]], [[0.7140824199]], [[1.7164359093]], [[1.7795374393]], [[1.4735395908]], [[1.2874834538]], [[0.7765902877]], [[1.8361166716]], [[1.1641805172]], [[1.8821854591]], [[2.4204339981]], [[2.1218876839]], [[1.7477912903]], [[1.5830814838]], [[1.4579468966]], [[1.0890700817]], [[0.2989229560]], [[2.1687474251]], [[1.7306339741]], [[1.3406019211]], [[1.0476143360]], [[2.4894409180]], [[1.7544082403]], [[1.2263457775]], [[0.5317156315]], [[1.7310724258]], [[1.4487860203]], [[2.2024619579]], [[1.8902075291]], [[1.7423844337]], [[1.2647770643]], [[1.3749401569]], [[1.6840031147]], [[1.8012895584]], [[1.5273509026]], [[1.4383857250]], [[1.0774629116]], [[1.0908963680]], [[2.0304865837]], [[1.5028542280]], [[1.7112162113]], [[2.2429251671]], [[1.6274894476]], [[2.0039343834]], [[0.6275576353]], [[1.7411286831]], [[0.8552755117]], [[0.9490455985]], [[1.0259432793]], [[1.1347725391]], [[1.5794568062]], [[1.5336185694]], [[0.6177737117]], [[1.6415812969]], [[2.3925213814]], [[0.8481150866]], [[1.8010163307]], [[1.5259532928]], [[1.7656083107]], [[1.2884104252]], [[1.4999831915]], [[1.5202929974]], [[1.1242389679]], [[1.4932074547]], [[1.6829164028]], [[1.2290197611]], [[2.2363400459]], [[0.7895829082]], [[0.8658518195]], [[1.5578114986]], [[1.3085157871]], [[1.9705107212]], [[1.2205266953]], [[1.3987293243]], [[1.1603055000]], [[1.3221619129]], [[1.5414502621]], [[1.1528912783]], [[1.0589461327]], [[0.9921824932]], [[1.5483164787]], [[1.8819913864]], [[1.5733687878]], [[2.3494188786]], [[1.6173179150]], [[1.3198715448]], [[2.0789260864]], [[1.7027032375]], [[2.1534948349]], [[2.5537614822]], [[0.9177682996]], [[1.2379645109]], [[0.5274739265]], [[2.1062641144]], [[1.2003819942]], [[1.0198884010]], [[2.3215997219]], [[1.2153373957]], [[1.9068410397]], [[1.6556181908]], [[1.0930092335]], [[1.1043345928]], [[1.8099434376]], [[1.2760341167]], [[1.7543935776]], [[1.5820648670]], [[1.4604362249]], [[1.1224498749]], [[1.2251083851]], [[1.8501992226]], [[1.0212533474]], [[1.4556055069]], [[1.8414361477]], [[1.9255690575]], [[1.1461676359]], [[0.7067813873]], [[2.2583217621]], [[1.4114665985]], [[1.3319324255]], [[1.0285203457]], [[1.7530515194]], [[2.3051898479]], [[2.3391928673]], [[2.2974309921]], [[2.5036263466]], [[2.8295099735]], [[2.1983921528]], [[3.8512187004]], [[2.2057085037]], [[2.2252609730]], [[2.4837827682]], [[3.2838156223]], [[1.5057592392]], [[2.3213868141]], [[2.5588340759]], [[2.1501719952]], [[2.7550983429]], [[2.5620036125]], [[3.4644827843]], [[2.8100388050]], [[2.7571377754]], [[2.7811517715]], [[2.8399860859]], [[3.3147964478]], [[3.2990813255]], [[2.6545066833]], [[2.3746349812]], [[1.9508521557]], [[1.9338434935]], [[2.4099137783]], [[2.6833219528]], [[2.7665269375]], [[2.9162821770]], [[2.1204752922]], [[2.3053321838]], [[2.6330151558]], [[3.0927457809]], [[4.0796265602]], [[2.6599116325]], [[3.4762320518]], [[2.6976170540]], [[1.9817891121]], [[3.5712094307]], [[2.0193572044]], [[1.3443940878]], [[2.6841173172]], [[3.8812046051]], [[3.0736503601]], [[1.9467797279]], [[2.7162022591]], [[2.8486881256]], [[2.7192144394]], [[2.2968249321]], [[2.8441181183]], [[2.7085161209]], [[2.7222504616]], [[2.9723274708]], [[3.0078594685]], [[1.7175710201]], [[2.4262051582]], [[1.9014602900]], [[3.1155061722]], [[1.7690463066]], [[2.5124268532]], [[2.8480749130]], [[2.6893343925]], [[2.6407308578]], [[2.5949461460]], [[2.2684671879]], [[2.2570071220]], [[2.0318777561]], [[2.9659173489]], [[1.7229695320]], [[2.0056307316]], [[2.1399738789]], [[2.3393802643]], [[2.9747865200]], [[2.7612295151]], [[2.6043987274]], [[2.1133742332]], [[3.0215387344]], [[3.3176856041]], [[4.4304056168]], [[2.1602151394]], [[2.2480807304]], [[3.5723156929]], [[2.8660259247]], [[2.7641105652]], [[2.1671605110]], [[2.3463289738]], [[1.7582514286]], [[1.5020806789]], [[2.0727682114]], [[2.3703150749]], [[0.7051126957]], [[1.7285375595]], [[3.6967360973]], [[3.3857643604]], [[1.6971490383]], [[3.1496055126]], [[2.7124037743]], [[1.9969489574]], [[2.0103168488]], [[2.6720066071]], [[3.2455244064]], [[2.2658858299]], [[3.0152621269]], [[2.0782270432]], [[2.2882080078]], [[3.5360784531]], [[3.2915360928]], [[2.7499895096]], [[2.5096383095]], [[3.0919997692]], [[1.9142830372]], [[1.3278501034]], [[2.1233091354]], [[2.7717738152]], [[2.8966581821]], [[3.2001962662]], [[2.1861054897]], [[2.4431912899]], [[2.7985620499]], [[2.9600300789]], [[2.8953847885]], [[1.7588822842]], [[2.2758178711]], [[1.1429795027]], [[1.9171218872]], [[2.3379440308]], [[1.6331272125]], [[2.3069825172]], [[2.6665332317]], [[2.6324200630]], [[2.5003178120]], [[1.8162038326]], [[3.3099370003]], [[2.0908832550]], [[2.5959079266]], [[2.7381937504]], [[2.7073459625]], [[2.3560652733]], [[2.4136953354]], [[2.6080622673]], [[1.1947592497]], [[2.6146297455]], [[2.6043949127]], [[2.4894170761]], [[2.1897108555]], [[1.2777467966]], [[2.2679474354]], [[2.6089980602]], [[2.3600690365]], [[2.1952297688]], [[1.9751069546]], [[2.4983966351]], [[2.2408039570]], [[1.1374735832]], [[1.3241360188]], [[0.9367995858]], [[1.2306723595]], [[1.3609992266]], [[2.4884846210]], [[0.9148886800]], [[2.1825914383]], [[2.8500118256]], [[1.8652923107]], [[2.0641880035]], [[1.5377545357]], [[1.4593069553]], [[2.9175612926]], [[1.6503877640]], [[2.6784615517]], [[2.1031253338]], [[2.2793436050]], [[2.1193990707]], [[0.7686277628]], [[1.9019246101]], [[1.9880762100]], [[1.4230060577]], [[2.4797155857]], [[2.3296999931]], [[1.8209071159]], [[2.5488333702]], [[1.5817232132]], [[1.9895883799]], [[2.3040468693]], [[2.3059060574]], [[1.5400861502]], [[2.3323001862]], [[2.9277002811]], [[1.8396100998]], [[2.2704715729]], [[2.0791864395]], [[1.8141314983]], [[2.2063105106]], [[0.9243448973]], [[2.1575117111]], [[1.4724090099]], [[2.5685510635]], [[0.7170946598]], [[2.8311834335]], [[2.1157300472]], [[2.5532131195]], [[2.7874691486]], [[2.5070550442]], [[2.6911392212]], [[3.2802009583]], [[2.4574236870]], [[2.4004287720]], [[1.7825133801]], [[2.3991580009]], [[2.1786322594]], [[1.5587446690]], [[2.5054073334]], [[2.0114092827]], [[1.5104267597]], [[1.5472161770]], [[3.1044974327]], [[1.7887538671]], [[1.8646881580]], [[2.3153343201]], [[2.9736337662]], [[2.1001636982]], [[1.9386042356]], [[1.3240584135]], [[1.9717621803]], [[2.2253296375]], [[1.8313279152]], [[0.9294857383]], [[2.3914823532]], [[2.5672113895]], [[2.0247116089]], [[2.0906040668]], [[2.2502927780]], [[0.9017009139]], [[2.0796470642]], [[1.1727855206]], [[2.0854692459]], [[2.9080665112]], [[1.7831436396]], [[1.5757583380]], [[2.1485440731]], [[1.3345096111]], [[2.8460888863]], [[1.8191815615]], [[1.4721629620]], [[1.4688968658]], [[1.2100387812]], [[1.8989802599]], [[0.5218719244]], [[1.5584522486]], [[3.0104219913]], [[0.9031757712]], [[1.6915766001]], [[1.9063076973]], [[0.6251527071]], [[2.0482149124]], [[2.1834499836]], [[1.9842461348]], [[1.0758895874]], [[1.2735975981]], [[1.8703085184]], [[2.7690408230]]]])
test_add.test_module[Add1-[((1, 96, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.2459141016]], [[1.5178792477]], [[0.8381349444]], [[0.8816879988]], [[1.0570802689]], [[1.3837361336]], [[1.2397508621]], [[1.6461021900]], [[1.2052855492]], [[1.3819637299]], [[1.0985510349]], [[1.1513748169]], [[0.7719832659]], [[0.9185164571]], [[1.0435459614]], [[1.2681792974]], [[1.4473251104]], [[1.5496689081]], [[0.9106869698]], [[1.0319261551]], [[1.4312660694]], [[1.6648514271]], [[1.1467573643]], [[1.6238133907]], [[1.1690658331]], [[1.3025646210]], [[1.7023956776]], [[0.7858223319]], [[0.9348885417]], [[1.1230748892]], [[1.0547575951]], [[1.6816579103]], [[0.9255676866]], [[1.0194910765]], [[0.9003372788]], [[0.7813770175]], [[0.9577872753]], [[1.6794564724]], [[1.4727666378]], [[1.4919937849]], [[1.2759532928]], [[0.9933157563]], [[1.3342498541]], [[0.7828101516]], [[0.8883743882]], [[0.9918925166]], [[1.5651264191]], [[1.5428180695]], [[1.0279099941]], [[1.2316162586]], [[1.5694378614]], [[1.7467241287]], [[1.4480986595]], [[1.3172039986]], [[1.5849006176]], [[0.9552563429]], [[1.3428294659]], [[0.8620047569]], [[0.9031144381]], [[0.9913657308]], [[1.4758939743]], [[1.4507377148]], [[0.9534812570]], [[1.4007110596]], [[1.5241434574]], [[1.1865488291]], [[1.2687482834]], [[1.3655098677]], [[1.5598458052]], [[1.7297545671]], [[0.8643457294]], [[1.0664227009]], [[1.4461624622]], [[1.6639322042]], [[1.6847611666]], [[1.6908359528]], [[1.3491647243]], [[0.8148661852]], [[1.2956538200]], [[0.9368548393]], [[0.7836804390]], [[1.6939036846]], [[1.6298373938]], [[0.7508935332]], [[1.3432435989]], [[1.1654274464]], [[1.1673769951]], [[1.0207791328]], [[1.4419355392]], [[0.9535057545]], [[1.4329531193]], [[1.5025115013]], [[1.6075932980]], [[1.4366130829]], [[0.7547898889]], [[0.9253090620]]]]), compiled_model=tensor([[[[1.2459141016]], [[1.0340647697]], [[0.5653267503]], [[0.3302959204]], [[0.3118000627]], [[1.1341459751]], [[0.9378650188]], [[1.4618458748]], [[1.2964828014]], [[1.5308226347]], [[0.8844418526]], [[1.0286352634]], [[0.2210917473]], [[1.1680202484]], [[0.9377326965]], [[0.8014040589]], [[1.2389695644]], [[1.0983805656]], [[1.1475552320]], [[1.0712819099]], [[0.7867957950]], [[1.1084175110]], [[1.2318291664]], [[1.2277557850]], [[1.0086843967]], [[0.8349524736]], [[1.2771945000]], [[0.9569998980]], [[0.5309362411]], [[1.0555912256]], [[0.3512288332]], [[1.4466662407]], [[0.2271723151]], [[1.0743741989]], [[0.9881686568]], [[0.2309532166]], [[0.3315879703]], [[1.4679858685]], [[1.0489867926]], [[1.0845279694]], [[1.3283915520]], [[1.2306580544]], [[0.6233332753]], [[0.2639709115]], [[0.5764793754]], [[1.0849223137]], [[0.8675936460]], [[0.8159333467]], [[0.4865965247]], [[1.2791895866]], [[0.9710702300]], [[1.8456575871]], [[1.0519739389]], [[0.6486096978]], [[1.4917651415]], [[0.7440613508]], [[1.5495531559]], [[0.6414812803]], [[0.3291386366]], [[0.3380817771]], [[0.7774683237]], [[0.9861490130]], [[1.0417611599]], [[1.0269501209]], [[1.4290406704]], [[1.3491560221]], [[1.0330395699]], [[1.0594881773]], [[0.8385707736]], [[1.9541437626]], [[0.5883243084]], [[0.4998229146]], [[0.7064986825]], [[1.0706689358]], [[1.8342959881]], [[1.0476934910]], [[1.5373592377]], [[1.0360078812]], [[1.0707864761]], [[0.3940286636]], [[0.6628341675]], [[1.8462545872]], [[1.2278873920]], [[0.4544646740]], [[1.1460801363]], [[1.1235973835]], [[0.6782103777]], [[0.5204688907]], [[1.4165614843]], [[0.4917340279]], [[0.9354569316]], [[1.0117805004]], [[1.4885549545]], [[1.3919384480]], [[0.1944850683]], [[0.3329601288]]]])
test_add.test_module[Add1-[((1, 576, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.8284466863]], [[1.1004118919]], [[0.4206675291]], [[0.4642205834]], [[0.6396129131]], [[0.9662687778]], [[0.8222835064]], [[1.2286348343]], [[0.7878180742]], [[0.9644963741]], [[0.6810835600]], [[0.7339074016]], [[0.3545158505]], [[0.5010490417]], [[0.6260785460]], [[0.8507118821]], [[1.0298576355]], [[1.1322014332]], [[0.4932195544]], [[0.6144586802]], [[1.0137987137]], [[1.2473840714]], [[0.7292900085]], [[1.2063460350]], [[0.7515984178]], [[0.8850971460]], [[1.2849282026]], [[0.3683549166]], [[0.5174211264]], [[0.7056074739]], [[0.6372901201]], [[1.2641904354]], [[0.5081002712]], [[0.6020236611]], [[0.4828698635]], [[0.3639096022]], [[0.5403198600]], [[1.2619891167]], [[1.0552992821]], [[1.0745263100]], [[0.8584858775]], [[0.5758483410]], [[0.9167824388]], [[0.3653427362]], [[0.4709069729]], [[0.5744251013]], [[1.1476590633]], [[1.1253507137]], [[0.6104425788]], [[0.8141489029]], [[1.1519703865]], [[1.3292566538]], [[1.0306311846]], [[0.8997365236]], [[1.1674332619]], [[0.5377889276]], [[0.9253621101]], [[0.4445373416]], [[0.4856470227]], [[0.5738983154]], [[1.0584266186]], [[1.0332703590]], [[0.5360138416]], [[0.9832436442]], [[1.1066761017]], [[0.7690814137]], [[0.8512808681]], [[0.9480424523]], [[1.1423783302]], [[1.3122870922]], [[0.4468783140]], [[0.6489552259]], [[1.0286951065]], [[1.2464647293]], [[1.2672936916]], [[1.2733684778]], [[0.9316973686]], [[0.3973987699]], [[0.8781863451]], [[0.5193874240]], [[0.3662130237]], [[1.2764363289]], [[1.2123699188]], [[0.3334261179]], [[0.9257761240]], [[0.7479600906]], [[0.7499095201]], [[0.6033116579]], [[1.0244681835]], [[0.5360383391]], [[1.0154857635]], [[1.0850441456]], [[1.1901259422]], [[1.0191457272]], [[0.3373224735]], [[0.5078416467]], [[1.0818476677]], [[0.9368407726]], [[0.4421480894]], [[0.5442803502]], [[1.3025647402]], [[1.1690990925]], [[0.6141775250]], [[0.7063477039]], [[0.3558910489]], [[0.8232030272]], [[0.4556606412]], [[0.4465117455]], [[0.8046402931]], [[0.9072626233]], [[0.6274249554]], [[1.1288788319]], [[0.5279205441]], [[1.2858750820]], [[1.1748399734]], [[0.4105486274]], [[0.7077479362]], [[0.8547514081]], [[0.9051406384]], [[0.9507772326]], [[1.0284042358]], [[0.8621401787]], [[0.5882257223]], [[1.0687845945]], [[0.3525656462]], [[0.5358367562]], [[0.7070251703]], [[0.5886334181]], [[0.6572734118]], [[0.4223792553]], [[0.7258325219]], [[0.9390683174]], [[0.5064572096]], [[0.8065304160]], [[1.1901154518]], [[0.7807899714]], [[0.8460862041]], [[0.7890556455]], [[0.9333807826]], [[1.1501097679]], [[1.3058131933]], [[1.1497180462]], [[1.3068969250]], [[0.7960292697]], [[0.3830293417]], [[0.5951514840]], [[1.1726427078]], [[0.8289488554]], [[0.5836669207]], [[0.4490342140]], [[0.3642640710]], [[0.4101859927]], [[0.7307717204]], [[1.1063930988]], [[1.1025106907]], [[0.3499741554]], [[1.1440811157]], [[0.4409353733]], [[0.7264849544]], [[0.6294537783]], [[0.7358824611]], [[0.7340187430]], [[0.3835151196]], [[0.4004711509]], [[0.7539503574]], [[0.8386561871]], [[0.6050526500]], [[1.0205397606]], [[0.3821609020]], [[0.7984464765]], [[1.2718998194]], [[0.6282441020]], [[1.2836916447]], [[1.0132670403]], [[0.3809596300]], [[1.1485388279]], [[0.7744928598]], [[0.6089866757]], [[1.2320168018]], [[0.4281424880]], [[0.8858425617]], [[0.7275057435]], [[1.1892464161]], [[0.9717623591]], [[1.0724427700]], [[1.0087695122]], [[0.7119527459]], [[0.7270373702]], [[0.4201493859]], [[1.1031122208]], [[1.2291791439]], [[1.1743025780]], [[0.4795009494]], [[0.8544900417]], [[0.4797230363]], [[0.5569480062]], [[0.5408373475]], [[1.0030626059]], [[0.5342335105]], [[0.8212814927]], [[0.8532241583]], [[1.1545016766]], [[0.4542300701]], [[0.4889339805]], [[0.5418593287]], [[1.1821568012]], [[0.6524575949]], [[1.2539343834]], [[1.0129938126]], [[0.8955031037]], [[0.8284680843]], [[0.7333493233]], [[0.8949232697]], [[0.7180177569]], [[0.8286768794]], [[0.8959866166]], [[0.4410875440]], [[0.5701243877]], [[1.2359364033]], [[0.4264167547]], [[0.7962870002]], [[1.3268094063]], [[1.0128085613]], [[0.8463466167]], [[0.3988851309]], [[1.0798790455]], [[0.4760497212]], [[0.6902579665]], [[0.6644319296]], [[0.7581464052]], [[0.8376592398]], [[1.2445938587]], [[0.8946095109]], [[1.2800364494]], [[1.1380462646]], [[0.5160831213]], [[1.0564421415]], [[0.4787420630]], [[0.6202775240]], [[0.9792514443]], [[0.9972861409]], [[1.2073041201]], [[0.6712321639]], [[0.8329905272]], [[1.0896018744]], [[0.3486440182]], [[1.1936804056]], [[0.4187288880]], [[0.8390813470]], [[0.7471817136]], [[0.5688536167]], [[0.8982756138]], [[1.2456494570]], [[0.6860303283]], [[0.5353430510]], [[0.6472706795]], [[0.3366159201]], [[1.0578870773]], [[0.5920582414]], [[0.4985199571]], [[0.5441393852]], [[1.1196681261]], [[1.0969769955]], [[1.2159509659]], [[1.0135517120]], [[0.6652102470]], [[0.6924660206]], [[0.9799050689]], [[1.2432074547]], [[0.9681361914]], [[0.5956174135]], [[0.5971513987]], [[0.3594555855]], [[0.9402140975]], [[0.5515988469]], [[0.3864021897]], [[1.2706283331]], [[0.5074844956]], [[0.7753093243]], [[0.9754368663]], [[0.8481192589]], [[0.4957492352]], [[0.4280292392]], [[1.2307312489]], [[0.9136094451]], [[1.2470023632]], [[0.6645697951]], [[0.9794678092]], [[0.7178519964]], [[0.8099545240]], [[0.5276679993]], [[1.0012905598]], [[0.9902730584]], [[0.8218758106]], [[0.7197390199]], [[0.5239751935]], [[1.1779625416]], [[0.4599780440]], [[1.0370228291]], [[0.6640633345]], [[0.5909560919]], [[0.9220126271]], [[0.5724616051]], [[0.9473925233]], [[0.9303805232]], [[0.4609453678]], [[0.9154393673]], [[1.0451546907]], [[1.0301055908]], [[0.7692507505]], [[0.4222962856]], [[0.7551130652]], [[1.0058438778]], [[0.6497515440]], [[1.0219806433]], [[1.1651713848]], [[0.5711143017]], [[0.8371210098]], [[1.0389395952]], [[0.8713790178]], [[0.8739558458]], [[0.8946228027]], [[0.4391162395]], [[0.8714842200]], [[1.1784250736]], [[1.2827470303]], [[1.1260654926]], [[0.8991916180]], [[1.0656971931]], [[0.5889502764]], [[0.4178459048]], [[0.4022295475]], [[1.3309949636]], [[1.1495847702]], [[0.4865732193]], [[1.0278114080]], [[1.2097738981]], [[1.3319975138]], [[1.2693874836]], [[1.2195670605]], [[0.7175604701]], [[0.6567192674]], [[1.2427145243]], [[1.1123799086]], [[0.5313006043]], [[1.2816987038]], [[1.0737693310]], [[1.1047577858]], [[0.5188019276]], [[0.9756399989]], [[0.6569088697]], [[1.2228684425]], [[0.7422198057]], [[1.0268464088]], [[0.9210010171]], [[1.0449242592]], [[0.6622763276]], [[1.0759757757]], [[0.4829546213]], [[0.9451175928]], [[0.4938941598]], [[0.3389213085]], [[0.4306622148]], [[1.2269251347]], [[1.1027078629]], [[1.3012979031]], [[1.2327506542]], [[0.3856673837]], [[0.4909745455]], [[0.7513988018]], [[0.5074739456]], [[1.1793882847]], [[0.4541862011]], [[0.5882393718]], [[0.3491443992]], [[0.5483173132]], [[1.2434288263]], [[1.2415701151]], [[1.1901090145]], [[1.2182537317]], [[1.2767833471]], [[0.7041586041]], [[1.0521911383]], [[1.2776553631]], [[0.9975994825]], [[1.3320282698]], [[1.0915215015]], [[1.1430141926]], [[0.6571940184]], [[1.0721038580]], [[0.8896433711]], [[0.7127814293]], [[0.5503351688]], [[0.5516318083]], [[0.4474496841]], [[1.1678552628]], [[1.1876556873]], [[0.7752837539]], [[0.5428470969]], [[1.2186428308]], [[1.1519346237]], [[0.8693571091]], [[0.5961224437]], [[1.2917139530]], [[1.0366604328]], [[0.4526174068]], [[1.3107315302]], [[1.2118769884]], [[0.6499446630]], [[1.1132674217]], [[0.5480955839]], [[0.7538329363]], [[1.2567406893]], [[0.8528496623]], [[0.4785810113]], [[0.6650736332]], [[0.6964675188]], [[0.7357257009]], [[0.8800404072]], [[1.2946048975]], [[0.8589602709]], [[0.5234701037]], [[0.8578172326]], [[1.0719337463]], [[1.0802102089]], [[0.3752208352]], [[0.7427188754]], [[0.4606178403]], [[0.6188473105]], [[1.0123367310]], [[0.4771249890]], [[1.0180535316]], [[1.2565791607]], [[0.8649842739]], [[0.4989462495]], [[0.6530476213]], [[0.9413721561]], [[0.4510310888]], [[1.0805952549]], [[0.3782553077]], [[0.3515434265]], [[0.3463597894]], [[0.7307584286]], [[1.1684064865]], [[0.3589506149]], [[1.2477836609]], [[0.6321786642]], [[0.9786321521]], [[0.8549915552]], [[0.3813305497]], [[1.2468545437]], [[1.1014118195]], [[1.3291878700]], [[1.0847961903]], [[0.5021556020]], [[1.2494819164]], [[0.8590623140]], [[1.0692982674]], [[0.4312755466]], [[0.6883768439]], [[0.3412513137]], [[0.6374438405]], [[0.9400556684]], [[0.4396092296]], [[0.9915722013]], [[1.1005935669]], [[0.9018447399]], [[0.4976484179]], [[0.4445302486]], [[0.6779317856]], [[1.0516691208]], [[1.3253883123]], [[1.1197046041]], [[0.7758854628]], [[1.0074982643]], [[0.3416586518]], [[0.4051394463]], [[1.0654940605]], [[0.5489825010]], [[1.0727379322]], [[0.4792240262]], [[0.5845346451]], [[0.4203456044]], [[1.0931104422]], [[0.7812425494]], [[1.2169907093]], [[1.1416261196]], [[1.1088614464]], [[0.8482681513]], [[0.6776010990]], [[0.7234742641]], [[0.8986403942]], [[1.0800448656]], [[0.4818951488]], [[1.2518290281]], [[0.7778250575]], [[0.4132173061]], [[0.5616611838]], [[1.2745989561]], [[1.2894537449]], [[0.3690502644]], [[1.1848392487]], [[1.0827696323]], [[1.1281492710]], [[1.2554547787]], [[0.5627149343]], [[0.9900779724]], [[1.0368067026]], [[0.6844434142]], [[0.9995166659]], [[0.6883333921]], [[1.1413207054]], [[0.6934627891]], [[0.6457922459]], [[0.9580646157]], [[1.0095369816]], [[0.5879042745]], [[0.8763892651]], [[1.1219580173]], [[0.7824417353]], [[0.9843570590]], [[0.7115966082]], [[1.0074398518]], [[0.4699929953]], [[0.5381757021]], [[0.5783948302]], [[1.2916955948]], [[0.6976480484]], [[0.8308249116]], [[0.5899423361]], [[1.3313350677]], [[1.3205251694]], [[0.4550967813]], [[0.4268551469]], [[0.4531936049]], [[0.8297787309]], [[0.7047381401]], [[0.5049172640]], [[0.6528540254]], [[0.9266555309]], [[0.5709447265]], [[0.9429813623]], [[0.7175321579]], [[0.5899084210]], [[0.9008828402]], [[1.2433192730]], [[0.4941504598]], [[0.8554072976]], [[0.6478062272]], [[1.3228433132]], [[0.3578084707]], [[0.3528594971]], [[1.3248825073]], [[0.5158451200]], [[0.9280517697]], [[0.7890370488]], [[0.7268615961]], [[0.7205162048]], [[1.1499104500]], [[0.8560885787]], [[0.3453828096]], [[0.5370100141]], [[0.6617399454]], [[1.0837982893]], [[0.5086232424]], [[1.3036499023]], [[0.7208254337]], [[0.7424091101]], [[1.2239691019]], [[1.0834939480]], [[1.2562502623]], [[1.1214122772]], [[0.6805131435]], [[0.5004538894]], [[0.7949885726]], [[1.2460088730]]]]), compiled_model=tensor([[[[0.8284466863]], [[0.8045368195]], [[0.7934416533]], [[1.1187661886]], [[0.6650812030]], [[0.7200669050]], [[0.5365635753]], [[1.5217444897]], [[0.9177681208]], [[0.8798126578]], [[0.9499593973]], [[1.0915967226]], [[0.9199852347]], [[1.0570499897]], [[0.7190465927]], [[0.5776382089]], [[0.7458560467]], [[1.7668561935]], [[0.8820570707]], [[1.0002222061]], [[0.7489980459]], [[1.8782018423]], [[1.3707714081]], [[1.8255860806]], [[0.4976158142]], [[0.8642453551]], [[1.1088562012]], [[1.0096476078]], [[0.4703910351]], [[0.6451401114]], [[1.0670564175]], [[1.2007042170]], [[0.4296491146]], [[0.7260978222]], [[0.6026200056]], [[0.1422302723]], [[1.1249637604]], [[1.2092381716]], [[1.4004666805]], [[1.6772280931]], [[1.2784540653]], [[0.8144481778]], [[1.5100208521]], [[0.6003657579]], [[0.4073885679]], [[1.2152347565]], [[1.4338092804]], [[0.8053202033]], [[0.6359066963]], [[0.6413750648]], [[1.7581567764]], [[1.4144935608]], [[0.7426786423]], [[1.0361146927]], [[1.6492495537]], [[0.8354990482]], [[1.2512742281]], [[0.6587838531]], [[0.8398662806]], [[0.6198822260]], [[1.0273363590]], [[0.7337259650]], [[0.3271552920]], [[1.3677548170]], [[0.9784293175]], [[1.0086245537]], [[1.1786864996]], [[1.1512542963]], [[0.9860134125]], [[1.9582612514]], [[0.3239384890]], [[1.2280154228]], [[0.7987508178]], [[1.2940003872]], [[1.7070879936]], [[1.2368880510]], [[1.5195287466]], [[0.2211304307]], [[0.6261104941]], [[0.4617377520]], [[0.6148481369]], [[1.9046242237]], [[1.1414794922]], [[0.6800501347]], [[0.9682325125]], [[0.8073388338]], [[1.2853710651]], [[0.3836286068]], [[1.2453846931]], [[1.1740528345]], [[1.1145895720]], [[1.6410731077]], [[1.2039573193]], [[1.5894366503]], [[0.0214672089]], [[0.6035897136]], [[1.1618342400]], [[1.2666931152]], [[0.8061217070]], [[1.0959987640]], [[1.3958816528]], [[1.3170814514]], [[1.1243929863]], [[0.7388710380]], [[0.9620177746]], [[0.6581068635]], [[0.5693656206]], [[0.5874661803]], [[1.1955552101]], [[1.4169106483]], [[0.7159829736]], [[0.8824484944]], [[0.9434765577]], [[1.6032509804]], [[1.5434995890]], [[0.2699247003]], [[1.1973556280]], [[1.4961388111]], [[1.1163362265]], [[0.6515629292]], [[1.5472090244]], [[0.6592244506]], [[0.8709716797]], [[1.3092534542]], [[0.2863572836]], [[0.8777444363]], [[0.4276187420]], [[0.8703230619]], [[0.5081078410]], [[0.5361223221]], [[0.9579568505]], [[1.5328505039]], [[0.4357069731]], [[1.2946584225]], [[1.2944085598]], [[0.7111359835]], [[0.5784962773]], [[0.4981063604]], [[1.5894960165]], [[1.1932199001]], [[1.4985975027]], [[1.4530841112]], [[1.8145728111]], [[1.3905762434]], [[0.9563478231]], [[0.3925198317]], [[1.2603442669]], [[0.7008899450]], [[0.4658009410]], [[0.7354476452]], [[1.0013979673]], [[0.1774439216]], [[1.2011837959]], [[1.0149695873]], [[1.1729333401]], [[0.9146695137]], [[1.1988058090]], [[0.6542731524]], [[0.5447785854]], [[1.2228841782]], [[0.8390574455]], [[0.5361334085]], [[0.6977301836]], [[0.2127723694]], [[0.5250065923]], [[1.0369256735]], [[1.1692843437]], [[1.0468575954]], [[0.7853042483]], [[1.3959169388]], [[1.7713401318]], [[0.5337734818]], [[1.3966915607]], [[1.0237344503]], [[0.1467080712]], [[1.3165552616]], [[1.3185203075]], [[1.1980543137]], [[1.4464769363]], [[0.7095139027]], [[0.8371916413]], [[1.2727367878]], [[1.1490020752]], [[0.7922458649]], [[1.3172494173]], [[1.4762721062]], [[0.4289668798]], [[1.3468310833]], [[0.7678248286]], [[0.9206156731]], [[1.2892853022]], [[1.7759023905]], [[0.2636935115]], [[0.8761613369]], [[0.8115230799]], [[0.2867153883]], [[0.9827463627]], [[1.4311463833]], [[1.0121462345]], [[0.6703178287]], [[1.5190346241]], [[1.0259307623]], [[1.1212102175]], [[0.1768986583]], [[0.2641842365]], [[1.6570649147]], [[0.8725239635]], [[1.4505889416]], [[0.9039245844]], [[0.8535752892]], [[0.8500842452]], [[0.4140815139]], [[1.0887174606]], [[0.9742571115]], [[0.9960634708]], [[1.2252616882]], [[1.0833230019]], [[0.8708763123]], [[1.2207101583]], [[0.3884533048]], [[0.6441941261]], [[1.1480100155]], [[1.1000936031]], [[0.9257237315]], [[0.7891262770]], [[1.0339705944]], [[1.0424613953]], [[0.5072277784]], [[0.8336629272]], [[1.3754162788]], [[1.5026617050]], [[1.1227726936]], [[1.1514838934]], [[1.5069077015]], [[1.0714304447]], [[0.5111455917]], [[1.3596789837]], [[0.2988693714]], [[0.8705852628]], [[1.3634237051]], [[0.9680609703]], [[1.7904345989]], [[0.8061368465]], [[1.2276568413]], [[1.7525649071]], [[0.3636197448]], [[1.6331944466]], [[0.4435241818]], [[0.9338547587]], [[0.8302560449]], [[0.7335572243]], [[0.8772013187]], [[1.5306527615]], [[0.8726804256]], [[1.0200978518]], [[0.7138779163]], [[0.5545659065]], [[1.0397018194]], [[0.3411412835]], [[0.8686634898]], [[0.7759282589]], [[1.0872361660]], [[1.0957343578]], [[1.5145215988]], [[1.0909531116]], [[1.1597580910]], [[0.8885974884]], [[1.3365833759]], [[1.6288654804]], [[1.0132629871]], [[0.9797597528]], [[1.1339279413]], [[0.5502746701]], [[1.2058703899]], [[0.7375765443]], [[0.9003958106]], [[1.2273405790]], [[0.4095094800]], [[1.1609959602]], [[0.7080897093]], [[1.0239453316]], [[0.4336135983]], [[0.9258559346]], [[0.9330216646]], [[1.3845856190]], [[1.9094905853]], [[0.9335497618]], [[1.1139643192]], [[1.3778368235]], [[0.7640882730]], [[0.6554111242]], [[0.9501479864]], [[1.0890623331]], [[1.0983551741]], [[0.9543296099]], [[0.5973476171]], [[0.8458520770]], [[0.6533899903]], [[1.1900217533]], [[0.9070916176]], [[1.1418911219]], [[1.5758272409]], [[0.4436302781]], [[1.0840251446]], [[0.8978536129]], [[0.1679324508]], [[0.7194197178]], [[1.6822451353]], [[1.4894460440]], [[1.2056362629]], [[0.8679777980]], [[0.5256834626]], [[1.2261633873]], [[1.2829186916]], [[0.9107475877]], [[1.7777783871]], [[0.8703434467]], [[1.3567026854]], [[0.9925556779]], [[1.2696977854]], [[0.5985218883]], [[1.0306849480]], [[0.7737017274]], [[1.1892414093]], [[1.7646389008]], [[1.9418854713]], [[1.7487540245]], [[1.4026975632]], [[1.5850381851]], [[0.6922453046]], [[0.1976323724]], [[0.3844258785]], [[1.4426726103]], [[1.0472013950]], [[0.9099685550]], [[1.3711349964]], [[1.4582598209]], [[1.6205878258]], [[1.8764235973]], [[1.5694925785]], [[0.4469006658]], [[0.4615888000]], [[1.6308991909]], [[1.3483294249]], [[0.9429563284]], [[0.9501486421]], [[0.7801446319]], [[1.5791809559]], [[1.0064346790]], [[0.6910943985]], [[1.0145113468]], [[1.0002561808]], [[1.2886836529]], [[1.3515515327]], [[1.5832043886]], [[0.7197381258]], [[0.3487481475]], [[1.5818367004]], [[1.0946760178]], [[1.4192674160]], [[0.9941189885]], [[0.0855955482]], [[0.9603789449]], [[0.9226614833]], [[1.3822185993]], [[1.1415904760]], [[1.2019629478]], [[0.6420117021]], [[0.4621944427]], [[0.8400972486]], [[0.2013716698]], [[0.8757616282]], [[0.8156776428]], [[0.5444100499]], [[0.2127594948]], [[0.5188360810]], [[1.0024545193]], [[1.0096799135]], [[1.6571049690]], [[1.6401846409]], [[1.5111949444]], [[0.5988386869]], [[1.3863515854]], [[1.4679278135]], [[0.8080729842]], [[1.6074330807]], [[1.7146034241]], [[0.8190209866]], [[0.3278078437]], [[1.3039762974]], [[1.4339644909]], [[0.5763464570]], [[0.8282136321]], [[1.0709342957]], [[0.7694177628]], [[1.4438705444]], [[1.8543286324]], [[1.2688300610]], [[0.4294434786]], [[1.4296710491]], [[1.5751806498]], [[1.4781217575]], [[0.2664659023]], [[1.7302765846]], [[1.0593044758]], [[0.1308166981]], [[1.5895284414]], [[1.1074800491]], [[0.8886106610]], [[0.9564538002]], [[0.3396409750]], [[0.8792173266]], [[1.2565877438]], [[1.3130919933]], [[0.6894999743]], [[1.2231628895]], [[0.9579789042]], [[0.7427549362]], [[1.3865170479]], [[1.4897372723]], [[1.1206159592]], [[0.5304687023]], [[1.0274214745]], [[1.1436070204]], [[1.2055114508]], [[0.1530910730]], [[0.9074538350]], [[0.2134056091]], [[0.6790918112]], [[1.4699198008]], [[0.2972597480]], [[0.8488375545]], [[1.3023008108]], [[0.8957271576]], [[0.7513998747]], [[0.9038653374]], [[1.3627302647]], [[0.2744007111]], [[1.2131195068]], [[0.8257108331]], [[0.9518390894]], [[0.4776969552]], [[0.6092582345]], [[1.4799478054]], [[0.8114991784]], [[1.1032185555]], [[0.5098418593]], [[1.3674411774]], [[0.9877684712]], [[0.0768911242]], [[1.1263719797]], [[1.4719882011]], [[1.3274099827]], [[1.5977206230]], [[1.0652353764]], [[1.4900268316]], [[0.9873348475]], [[1.0783946514]], [[0.5738025308]], [[0.9471971989]], [[0.1271356940]], [[0.6857559085]], [[0.6919651031]], [[0.9142965674]], [[0.8409639597]], [[1.7251753807]], [[0.9407642484]], [[0.3790206909]], [[0.8525667191]], [[0.9202812314]], [[1.5657070875]], [[1.7019331455]], [[0.8058447838]], [[1.2599406242]], [[1.0811512470]], [[0.2884830236]], [[0.8904762864]], [[1.5979690552]], [[0.2773270607]], [[1.1953361034]], [[1.0576372147]], [[0.9459879398]], [[1.0093901157]], [[1.0895709991]], [[0.6732312441]], [[1.8147554398]], [[1.5178475380]], [[1.7566626072]], [[0.8072297573]], [[0.5243775845]], [[0.8326777220]], [[0.5956603289]], [[1.4438424110]], [[1.0184992552]], [[1.5396673679]], [[0.8962315321]], [[0.8289303780]], [[0.4120954275]], [[1.9315304756]], [[0.9600917697]], [[0.0578875542]], [[1.2344982624]], [[1.6590018272]], [[1.3460221291]], [[1.6152875423]], [[0.3640214205]], [[1.3402307034]], [[1.1487448215]], [[1.0526651144]], [[1.5204478502]], [[1.0734834671]], [[1.2665985823]], [[0.8304731250]], [[0.5000120401]], [[0.9450234771]], [[1.5022659302]], [[0.5552158952]], [[1.3547190428]], [[1.0915019512]], [[0.8338022232]], [[1.1628080606]], [[0.4205737710]], [[1.1702573299]], [[0.5874050260]], [[0.6610674262]], [[0.6462526321]], [[1.8536940813]], [[1.2344515324]], [[0.6597561836]], [[0.9900156260]], [[1.1069537401]], [[1.0626744032]], [[0.7757476568]], [[0.6020132899]], [[0.3877503276]], [[0.5153367519]], [[0.6789799929]], [[0.8397269249]], [[0.3578631878]], [[0.6087327003]], [[0.8021908998]], [[0.7505891919]], [[0.4473019242]], [[0.5650752187]], [[0.9881923199]], [[0.9567764997]], [[0.3280965090]], [[1.0560011864]], [[0.3441302776]], [[1.4203759432]], [[0.2027670145]], [[0.1740507483]], [[1.6892080307]], [[0.2988101840]], [[0.7169445753]], [[1.2419877052]], [[0.9881546497]], [[1.3436825275]], [[1.1840724945]], [[0.7823909521]], [[0.4669194818]], [[0.7073495984]], [[1.2108719349]], [[0.7535682321]], [[0.6408327818]], [[1.6926358938]], [[0.6132938266]], [[0.6561486125]], [[1.6334128380]], [[0.7722112536]], [[1.5425380468]], [[0.9021289349]], [[1.0968539715]], [[0.2407435775]], [[1.2702008486]], [[0.9405024648]]]])
test_add.test_module[Add1-[((1, 1024, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.9612236023]], [[1.2331888676]], [[0.5534444451]], ..., [[0.6525920630]], [[0.6748203039]], [[1.1859661341]]]]), compiled_model=tensor([[[[0.9612236023]], [[1.1095082760]], [[0.5631945133]], ..., [[2.1295104027]], [[1.9586073160]], [[2.1236968040]]]])
test_advindex.test_module[Advindex1-[((1, 1, 1024, 72), torch.float32)]]: forge/test/models_ops/test_advindex.py#L34
IndexError: index 758 is out of bounds for dimension 0 with size 1
test_advindex.test_module[Advindex2-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L47
IndexError: index 417 is out of bounds for dimension 0 with size 169
test_advindex.test_module[Advindex4-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L73
IndexError: index 758 is out of bounds for dimension 0 with size 169
test_argmax.test_module[Argmax0-[((1, 32), torch.int32)]]: forge/test/models_ops/test_argmax.py#L84
RuntimeError: Generated MLIR module failed verification.
test_avgpool2d.test_module[Avgpool2D9-[((1, 384, 35, 35), torch.float32)]]: forge/test/models_ops/test_avgpool2d.py#L1472
RuntimeError: Tensor 1 - stride mismatch: expected [1225, 1], got [0, 0]
test_avgpool2d.test_module[Avgpool2D9-[((1, 1024, 17, 17), torch.float32)]]: forge/test/models_ops/test_avgpool2d.py#L1472
RuntimeError: Tensor 1 - stride mismatch: expected [289, 1], got [0, 0]
test_broadcast.test_module[Broadcast1-[((1, 1, 1, 384), torch.bool)]]: forge/test/models_ops/test_broadcast.py#L172
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((2, 1, 1, 13), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427., 503., 497., 683., 101., 866., 756.]]], [[[399., 878., 376., 56., 868., 794., 33., 126., 119., 391., 254., 824., 841.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760., 0., 963., 0., 379., 0., 427.]]], [[[ 0., 503., 0., 497., 0., 683., 0., 101., 0., 866., 0., 756., 0.]]]])
test_cast.test_module[Cast0-[((2, 1, 7, 7), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427.], [503., 497., 683., 101., 866., 756., 399.], [878., 376., 56., 868., 794., 33., 126.], [119., 391., 254., 824., 841., 269., 969.], [549., 480., 481., 12., 363., 360., 995.], [ 85., 822., 999., 711., 288., 778., 43.], [896., 489., 771., 757., 283., 395., 782.]]], [[[671., 840., 969., 673., 441., 111., 280.], [503., 706., 476., 327., 699., 326., 363.], [474., 775., 700., 218., 632., 568., 12.], [277., 645., 110., 80., 348., 521., 119.], [916., 881., 490., 682., 519., 944., 633.], [169., 863., 409., 233., 219., 78., 735.], [ 83., 222., 858., 235., 416., 446., 835.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760.], [ 0., 963., 0., 379., 0., 427., 0.], [503., 0., 497., 0., 683., 0., 101.], [ 0., 866., 0., 756., 0., 399., 0.], [878., 0., 376., 0., 56., 0., 868.], [ 0., 794., 0., 33., 0., 126., 0.], [119., 0., 391., 0., 254., 0., 824.]]], [[[ 0., 841., 0., 269., 0., 969., 0.], [549., 0., 480., 0., 481., 0., 12.], [ 0., 363., 0., 360., 0., 995., 0.], [ 85., 0., 822., 0., 999., 0., 711.], [ 0., 288., 0., 778., 0., 43., 0.], [896., 0., 489., 0., 771., 0., 757.], [ 0., 283., 0., 395., 0., 782., 0.]]]])
test_cast.test_module[Cast0-[((1, 1, 256, 256), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., ..., 374., 973., 703.], [757., 943., 848., ..., 762., 629., 216.], [606., 763., 616., ..., 786., 391., 916.], ..., [889., 108., 668., ..., 29., 931., 519.], [326., 328., 926., ..., 422., 582., 607.], [867., 283., 592., ..., 712., 264., 295.]]]]), compiled_model=tensor([[[[ 44., 0., 239., ..., 0., 389., 0.], [793., 0., 611., ..., 0., 703., 0.], [757., 0., 943., ..., 0., 370., 0.], ..., [178., 0., 972., ..., 0., 393., 0.], [274., 0., 790., ..., 0., 874., 0.], [869., 0., 258., ..., 0., 375., 0.]]]])
test_cast.test_module[Cast1-[((1, 1, 256, 256), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 128), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast2-[((1, 128), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 384), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast2-[((1, 384), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 12, 384, 384), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 1, 256, 256), torch.int32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast1-[((1, 1, 7, 7), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 1, 1, 7), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760.]]]])
test_cast.test_module[Cast0-[((1, 1, 32, 32), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., ..., 480., 481., 12.], [363., 360., 995., ..., 326., 363., 474.], [775., 700., 218., ..., 858., 235., 416.], ..., [326., 721., 442., ..., 346., 346., 564.], [386., 447., 788., ..., 593., 762., 312.], [714., 724., 526., ..., 122., 978., 438.]]]]), compiled_model=tensor([[[[ 44., 0., 239., ..., 0., 376., 0.], [ 56., 0., 868., ..., 0., 12., 0.], [363., 0., 360., ..., 0., 395., 0.], ..., [108., 0., 590., ..., 0., 827., 0.], [556., 0., 895., ..., 0., 746., 0.], [403., 0., 958., ..., 0., 216., 0.]]]])
test_cast.test_module[Cast1-[((1, 16, 320, 1024), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_conv2d.test_module[Conv2D1-[((1, 1024, 3000, 1), torch.float32), ((1024, 1024, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[497.3335876465], [752.2802124023], [745.8143310547], ..., [748.7915039062], [751.2043457031], [758.1017456055]], [[516.6517333984], [763.2512207031], [751.9757690430], ..., [776.1340942383], [769.6301269531], [756.2929077148]], [[508.2722778320], [745.5457763672], [746.0208740234], ..., [750.8541870117], [748.2391967773], [741.9400024414]], ..., [[510.7725524902], [747.9537353516], [756.9500122070], ..., [774.0905761719], [777.7086791992], [763.4777832031]], [[517.1862182617], [770.1345825195], [773.8189697266], ..., [780.8861083984], [781.6167602539], [774.4717407227]], [[512.4338378906], [756.8844604492], [759.8029174805], ..., [763.7905273438], [769.1001586914], [768.2404174805]]]]), compiled_model=tensor([[[[498.], [760.], [740.], ..., [752.], [764.], [752.]], [[520.], [776.], [744.], ..., [784.], [780.], [768.]], [[510.], [764.], [744.], ..., [760.], [756.], [740.]], ..., [[502.], [760.], [760.], ..., [764.], [784.], [756.]], [[516.], [780.], [784.], ..., [776.], [788.], [780.]], [[504.], [768.], [756.], ..., [764.], [776.], [768.]]]])
test_conv2d.test_module[Conv2D1-[((1, 384, 3000, 1), torch.float32), ((384, 384, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7fadc5e6f288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7fadc1fa07a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7fadc1f93fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7fadc1ffa829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7fadc4171fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7fadc41723e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7fadc417172b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fadc41713e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fadc1ff9b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fadc1f916bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fadc1f80f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30]
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((512, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7fadc5e6f288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7fadc1fa07a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7fadc1f93fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7fadc1ffa829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7fadc4171fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7fadc41723e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7fadc417172b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fadc41713e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fadc1ff9b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fadc1f916bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fadc1f80f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30]
test_conv2d.test_module[Conv2D1-[((1, 768, 3000, 1), torch.float32), ((768, 768, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7fadc5e6f288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7fadc1fa07a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7fadc1f93fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7fadc1ffa829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7fadc4171fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7fadc41723e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7fadc417172b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fadc41713e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fadc1ff9b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fadc1f916bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fadc1f80f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30]
test_conv2d.test_module[Conv2D2-[((1, 3072, 1, 128), torch.float32), ((768, 768, 1, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[199.4854583740, 195.3149261475, 186.9613647461, ..., 181.2562866211, 199.4641876221, 188.8912506104]], [[200.4394531250, 203.4648284912, 191.3027648926, ..., 190.6446533203, 194.6259613037, 197.4597473145]], [[198.7062530518, 199.6290435791, 191.5615234375, ..., 185.8805236816, 193.2581634521, 191.8968505859]], ..., [[185.1808166504, 179.1774597168, 192.7327880859, ..., 187.5319976807, 195.1789245605, 180.5239105225]], [[198.3394165039, 193.2522430420, 207.2843170166, ..., 204.6146087646, 208.7871704102, 195.6758270264]], [[194.1520996094, 185.4378204346, 198.5037689209, ..., 191.9806671143, 198.8300628662, 187.4386596680]]]]), compiled_model=tensor([[[[196., 192., 187., ..., 179., 193., 189.]], [[200., 197., 189., ..., 187., 193., 194.]], [[194., 196., 187., ..., 183., 193., 191.]], ..., [[183., 174., 191., ..., 187., 194., 179.]], [[195., 191., 201., ..., 200., 207., 196.]], [[192., 182., 197., ..., 188., 195., 184.]]]])
test_conv2d.test_module[Conv2D4-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/program/program.cpp:905: tt::exception info: Statically allocated circular buffers in program 171677 clash with L1 buffers on core range [(x=0,y=0) - (x=7,y=5)]. L1 buffer allocated at 947328 and static circular buffer region ends at 1180896 backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x1712d5) [0x7fadc52c92d5] --- tt::tt_metal::detail::Program_::validate_circular_buffer_region(tt::tt_metal::v0::IDevice const*) --- tt::tt_metal::v0::EnqueueProgram(tt::tt_metal::CommandQueue&, tt::tt_metal::v0::Program&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x336) [0x7fadc41717b6] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fadc41713e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fadc1ff9b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fadc1f916bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fadc1f80f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702]
test_conv2d.test_module[Conv2D9-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/program/program.cpp:896: tt::exception info: Statically allocated circular buffers on core range [(x=0,y=0) - (x=7,y=5)] grow to 1676512 B which is beyond max L1 size of 1499136 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x17129b) [0x7fadc52c929b] --- tt::tt_metal::detail::Program_::validate_circular_buffer_region(tt::tt_metal::v0::IDevice const*) --- tt::tt_metal::v0::EnqueueProgram(tt::tt_metal::CommandQueue&, tt::tt_metal::v0::Program&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x336) [0x7fadc41717b6] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fadc41713e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fadc1ff9b70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fadc1f916bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fadc1f80f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702]
test_conv2d.test_module[Conv2D15-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7fadc52b27e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7fadc527de9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7fadc2bb0833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7fadc2bb0699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7fadc417139a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7fadc2baf69a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7fadc1fe7f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7fadc1f80d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464]
test_conv2d.test_module[Conv2D16-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7fadc52b27e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7fadc527de9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7fadc2bb0833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7fadc2bb0699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7fadc417139a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7fadc2baf69a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7fadc1fe7f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7fadc1f80d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464]
test_conv2d.test_module[Conv2D521-[((1, 192, 17, 17), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[416.8599243164, 432.2357788086, 431.2140502930, ..., 435.4002380371, 429.5440063477, 438.3638000488], [438.0341186523, 434.8786621094, 447.7113952637, ..., 427.4633483887, 435.3717956543, 431.3974609375], [440.1000976562, 435.0090942383, 443.9839782715, ..., 433.8667297363, 437.3811340332, 427.4101257324], ..., [429.1877746582, 435.2696228027, 429.9178771973, ..., 433.3016357422, 438.7910461426, 429.1210937500], [433.7450256348, 437.3068847656, 437.7666015625, ..., 424.1896057129, 425.4042968750, 439.9100952148], [430.1431274414, 445.3993835449, 437.3017883301, ..., 439.9054260254, 424.5427246094, 431.0164794922]], [[417.6658325195, 429.5430297852, 426.4378662109, ..., 428.0760803223, 422.9098205566, 436.4092407227], [430.3939514160, 432.2352294922, 436.0129699707, ..., 430.4487609863, 430.4653320312, 430.7752990723], [438.3452148438, 435.6008300781, 432.5618286133, ..., 428.9048461914, 431.8200378418, 427.5862426758], ..., [427.3694763184, 428.4196777344, 426.7937927246, ..., 431.1934814453, 432.4386291504, 427.4638366699], [431.4784240723, 432.2783203125, 430.9342651367, ..., 420.9784851074, 426.7427368164, 431.8651733398], [425.0971984863, 442.2405700684, 427.8922119141, ..., 429.7329406738, 421.6875305176, 432.9625854492]], [[412.8150939941, 425.4307861328, 421.1838378906, ..., 424.2559814453, 419.7538146973, 426.0534362793], [423.4242553711, 430.0765991211, 423.8687744141, ..., 420.4248046875, 426.9146728516, 429.1799621582], [432.4685363770, 429.3944091797, 437.7822265625, ..., 425.9971923828, 423.0391845703, 412.3336486816], ..., [416.0045776367, 428.1044006348, 423.3621520996, ..., 427.3383178711, 428.5604248047, 424.5193481445], [431.3448181152, 430.0260620117, 422.5646362305, ..., 421.5996093750, 419.3544311523, 431.8387451172], [422.9151916504, 433.2698974609, 425.0296325684, ..., 424.5509643555, 416.2740783691, 423.0010375977]], ..., [[419.7501525879, 429.4707641602, 425.2193908691, ..., 432.6336669922, 431.5887145996, 436.8670654297], [425.1300659180, 435.9381713867, 443.7704467773, ..., 429.2366333008, 442.1405029297, 433.2154846191], [441.3649291992, 440.7537841797, 444.0762634277, ..., 434.5529785156, 435.2048339844, 431.8186950684], ..., [435.7177734375, 426.3558959961, 435.9081420898, ..., 439.4028015137, 434.7848510742, 436.1129150391], [434.9247436523, 440.4299316406, 435.1066894531, ..., 424.9306640625, 437.7439880371, 441.1544494629], [424.5056152344, 439.4252929688, 434.3029174805, ..., 434.0750122070, 426.0892028809, 437.7471313477]], [[412.5895385742, 433.6462402344, 422.6610412598, ..., 433.8632507324, 435.7726135254, 436.7062988281], [428.8259887695, 436.7130126953, 439.5889587402, ..., 428.5087890625, 443.4784240723, 427.7095947266], [446.0645751953, 438.5822143555, 438.7476806641, ..., 431.8515930176, 433.5840148926, 421.0300903320], ..., [431.9470520020, 429.9702148438, 429.3434143066, ..., 436.1727905273, 437.5594177246, 425.5984497070], [428.5779724121, 442.1037902832, 435.1010131836, ..., 426.1306152344, 427.0154418945, 436.2428588867], [425.3132019043, 432.7833251953, 432.6956787109, ..., 436.3273010254, 421.2727966309, 438.8209838867]], [[419.0737304688, 427.7669677734, 423.8291320801, ..., 438.3382873535, 429.6676025391, 444.5024414062], [434.7646484375, 434.9653625488, 441.7894897461, ..., 425.9906616211, 440.1138610840, 435.2942810059], [441.3545837402, 436.4105834961, 442.4326782227, ..., 441.1101989746, 440.0775451660, 427.3348388672], ..., [432.7417297363, 434.9034729004, 428.7973632812, ..., 432.5160827637, 432.6542968750, 436.4313659668], [441.9112854004, 444.2503967285, 438.4054565430, ..., 423.4556274414, 432.0733642578, 439.2596740723], [434.0776672363, 446.8495178223, 432.4898376465, ..., 435.5931701660, 422.1891784668, 433.6974792480]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[418., 436., 424., ..., 428., 430., 442.], [430., 438., 446., ..., 426., 444., 428.], [438., 440., 442., ..., 442., 436., 424.], ..., [430., 434., 426., ..., 430., 438., 426.], [430., 436., 442., ..., 422., 422., 440.], [434., 442., 444., ..., 440., 420., 424.]], [[412., 424., 420., ..., 430., 420., 432.], [434., 432., 430., ..., 430., 428., 426.], [434., 434., 422., ..., 428., 420., 426.], ..., [424., 420., 424., ..., 430., 428., 422.], [430., 428., 430., ..., 420., 420., 432.], [424., 442., 430., ..., 434., 426., 430.]], [[418., 426., 418., ..., 424., 420., 422.], [426., 438., 428., ..., 418., 432., 432.], [430., 426., 444., ..., 422., 424., 418.], ..., [410., 422., 418., ..., 428., 432., 414.], [436., 428., 422., ..., 420., 422., 428.], [426., 438., 422., ..., 422., 408., 426.]], ..., [[426., 428., 424., ..., 424., 430., 436.], [426., 438., 444., ..., 422., 436., 438.], [442., 440., 444., ..., 428., 434., 430.], ..., [438., 422., 430., ..., 430., 432., 440.], [430., 438., 428., ..., 428., 436., 438.], [426., 440., 434., ..., 432., 418., 436.]], [[412., 436., 422., ..., 424., 432., 428.], [430., 434., 440., ..., 424., 448., 428.], [446., 438., 440., ..., 428., 424., 422.], ..., [426., 422., 430., ..., 428., 432., 426.], [428., 442., 434., ..., 432., 428., 432.], [422., 438., 436., ..., 438., 422., 440.]], [[412., 422., 432., ..., 432., 428., 448.], [434., 438., 438., ..., 416., 442., 436.], [442., 434., 444., ..., 444., 442., 422.], ..., [434., 430., 424., ..., 428., 434., 438.], [446., 438., 432., ..., 418., 430., 436.], [430., 440., 430., ..., 434., 414., 434.]]]])
test_conv2d.test_module[Conv2D545-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7fadc52b27e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7fadc527de9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7fadc2bb0833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7fadc2bb0699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7fadc417139a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7fadc2baf69a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7fadc1fe7f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7fadc1f80d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464]
test_conv2d.test_module[Conv2D547-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 6422528 B L1 buffer across 2 banks, where each bank needs to store 3211264 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7fadc52b27e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7fadc527de9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations13data_movement35InterleavedToShardedDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS8_EE5valueEE4typeERKS8_RKSt6vectorINS0_6TensorESaISE_EERKSD_ISt8optionalISE_ESaISK_EE+0x1cf) [0x7fadc5f0629f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x409139) [0x7fadc1fba139] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7fadc417139a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x40052f) [0x7fadc1fb152f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff92f) [0x7fadc1fb092f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff708) [0x7fadc1fb0708] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff66c) [0x7fadc1fb066c] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3ff44d) [0x7fadc1fb044d] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3fee51) [0x7fadc1fafe51] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x3fe8d9) [0x7fadc1faf8d9] --- std::tuple<tt::tt_metal::Tensor, ttnn::operations::sliding_window::ParallelConfig, ttnn::operations::sliding_window::ParallelConfig> ttnn::operations::conv::shard_or_reshard_tensor_if_required<tt::tt_metal::v0::IDevice>(tt::tt_metal::v0::IDevice*, tt::tt_metal::Tensor const&, ttnn::operations::conv::conv2d::Conv2dConfig const&, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, bool, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0x697) [0x7fadc1f80747] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702]
test_conv2d.test_module[Conv2D66-[((1, 3, 320, 1024), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 88612864 B L1 buffer across 64 banks, where each bank needs to store 1384576 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7fadc52b27e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7fadc527de9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7fadc2bb0833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7fadc2bb0699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7fadc417139a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fadc41712d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fadc417049f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7fadc2baf69a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fadc4177fe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fadc41787cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7fadc1fe7f8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7fadc1f80d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fadc5e6a365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fadcf8389f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fadcf83893e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fadcf603b75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x559add527b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x559add517a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x559add51d574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x559add536172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x559add51e39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x559add51899e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x559add51297f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x559add53610b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x559add512790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x559add514c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x559add5139ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x559add5354d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x559add517702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x559add52838c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x559add51d61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x559add53262c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x559add63a464]
test_conv2d.test_module[Conv2D655-[((1, 256, 22, 66), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[570.9462280273, 574.1496582031, 568.3541259766, ..., 578.8296508789, 586.1709594727, 592.3473510742], [584.7791748047, 581.8095703125, 570.3928222656, ..., 574.6198730469, 580.8138427734, 586.6016235352], [598.0871582031, 598.2870483398, 585.3781127930, ..., 586.0263671875, 585.0138549805, 599.1818237305], ..., [572.1325683594, 575.2940063477, 572.7550048828, ..., 577.7442626953, 581.7819213867, 581.0496826172], [583.0502929688, 578.4487915039, 576.2745361328, ..., 578.1917724609, 580.6840820312, 574.9425659180], [585.0589599609, 575.4931640625, 572.7432250977, ..., 573.8483276367, 587.0273437500, 589.3324584961]], [[573.4531250000, 579.2989501953, 566.8147583008, ..., 575.4981079102, 580.0260009766, 587.6251220703], [587.0395507812, 580.7183837891, 574.9629516602, ..., 589.3232421875, 587.7586669922, 591.7297363281], [600.8474121094, 595.1917724609, 583.5451049805, ..., 588.7978515625, 590.6854248047, 587.9579467773], ..., [586.3883666992, 565.6767578125, 578.3594970703, ..., 578.7777709961, 577.6995239258, 570.6154174805], [578.1880493164, 567.6723632812, 577.8620605469, ..., 576.7629394531, 580.8159179688, 590.1276245117], [579.9801025391, 570.3218994141, 571.2749023438, ..., 587.2904052734, 586.7377929688, 592.7113037109]], [[579.5585937500, 595.9021606445, 587.2991943359, ..., 594.6030883789, 592.4997558594, 604.9805908203], [598.6312255859, 600.0043334961, 593.1022949219, ..., 590.9971313477, 594.6500854492, 609.2249755859], [605.3463745117, 608.8527221680, 602.3562622070, ..., 594.9850463867, 605.6123657227, 610.5809936523], ..., [585.6690063477, 580.2275390625, 582.5021972656, ..., 594.8728637695, 600.2041015625, 588.7520141602], [593.6163940430, 595.9530639648, 586.0277709961, ..., 592.7637939453, 590.8413696289, 593.6563720703], [590.0734252930, 588.4578857422, 588.1378784180, ..., 594.7918701172, 595.9435424805, 602.3709716797]], ..., [[571.3966064453, 575.1876831055, 567.7845458984, ..., 582.2432250977, 575.7072143555, 577.0569458008], [579.5115356445, 576.3997802734, 576.3776245117, ..., 582.0000610352, 585.7001953125, 586.3352661133], [590.6561889648, 597.5269165039, 591.2229003906, ..., 584.8515625000, 587.7363281250, 584.3200073242], ..., [573.8482666016, 562.1514282227, 575.5936279297, ..., 575.2089843750, 582.0252685547, 580.2095947266], [580.4851074219, 572.6992187500, 575.3505859375, ..., 573.3737182617, 580.1395874023, 577.8393554688], [574.1156005859, 571.3304443359, 562.8349609375, ..., 586.3864746094, 589.8875122070, 577.4716796875]], [[578.3203735352, 581.3521118164, 570.2847900391, ..., 589.2144165039, 588.0828857422, 585.2642822266], [584.3300170898, 588.3199462891, 583.2088623047, ..., 586.6121215820, 584.4935302734, 587.3798217773], [591.4719238281, 598.4736938477, 588.3948364258, ..., 585.6420898438, 588.7777709961, 598.9888305664], ..., [575.6354980469, 572.6661987305, 575.7431640625, ..., 586.5986328125, 582.0685424805, 581.7919921875], [576.8596801758, 574.7929687500, 579.9046630859, ..., 584.2254638672, 575.5091552734, 578.5747070312], [579.0372314453, 574.7625122070, 571.6882934570, ..., 586.5312500000, 587.0773925781, 580.0599975586]], [[583.3513183594, 591.6869506836, 579.8595581055, ..., 584.7144165039, 592.1401367188, 595.9358520508], [598.3665771484, 593.0552978516, 583.0579833984, ..., 594.3779907227, 589.5156250000, 597.5732421875], [607.4887084961, 608.1002197266, 584.9455566406, ..., 595.7606811523, 599.2399291992, 601.2327880859], ..., [581.5216674805, 580.3801269531, 584.1895141602, ..., 588.2474975586, 587.6593627930, 583.0759887695], [589.0550537109, 584.6390380859, 580.5605468750, ..., 595.1901245117, 596.6243286133, 589.3004150391], [581.4307861328, 582.4866333008, 587.3435058594, ..., 584.6020507812, 599.2372436523, 599.5579833984]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[576., 576., 572., ..., 572., 584., 588.], [592., 576., 564., ..., 564., 576., 580.], [588., 596., 584., ..., 580., 580., 596.], ..., [580., 572., 572., ..., 564., 584., 584.], [584., 568., 572., ..., 584., 572., 580.], [584., 572., 580., ..., 580., 580., 584.]], [[568., 580., 572., ..., 576., 584., 592.], [592., 592., 568., ..., 584., 588., 592.], [596., 584., 584., ..., 588., 600., 584.], ..., [592., 564., 592., ..., 580., 572., 560.], [584., 560., 576., ..., 580., 584., 588.], [572., 568., 580., ..., 576., 576., 596.]], [[576., 600., 588., ..., 596., 588., 588.], [592., 600., 580., ..., 588., 588., 600.], [600., 608., 600., ..., 584., 604., 604.], ..., [584., 576., 576., ..., 604., 588., 584.], [584., 588., 576., ..., 584., 588., 592.], [592., 580., 580., ..., 604., 600., 616.]], ..., [[576., 572., 580., ..., 580., 584., 580.], [584., 568., 576., ..., 592., 588., 584.], [580., 608., 588., ..., 584., 596., 588.], ..., [572., 560., 580., ..., 572., 584., 580.], [576., 576., 564., ..., 568., 580., 584.], [576., 572., 568., ..., 592., 588., 576.]], [[576., 584., 572., ..., 592., 580., 584.], [584., 588., 592., ..., 580., 588., 568.], [592., 596., 584., ..., 584., 600., 588.], ..., [584., 564., 576., ..., 580., 584., 596.], [576., 564., 580., ..., 580., 572., 580.], [580., 584., 576., ..., 588., 592., 576.]], [[580., 592., 588., ..., 584., 592., 588.], [600., 592., 576., ..., 592., 592., 596.], [604., 612., 584., ..., 596., 596., 600.], ..., [572., 584., 588., ..., 580., 588., 592.], [576., 588., 572., ..., 596., 584., 588.], [584., 588., 584., ..., 580., 596., 600.]]]])
test_conv2d.test_module[Conv2D655-[((1, 256, 42, 130), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[598.8849487305, 580.4715576172, 586.9127197266, ..., 582.2382812500, 587.7333984375, 593.7080688477], [594.6992797852, 576.0781860352, 587.8957519531, ..., 587.2709350586, 585.5784301758, 580.0172119141], [588.3734130859, 579.6995849609, 581.9422607422, ..., 596.4155273438, 579.8858642578, 585.7386474609], ..., [583.6843261719, 577.9443969727, 575.7817382812, ..., 596.8135986328, 586.6149902344, 587.0212402344], [583.6083984375, 582.8132324219, 574.9744873047, ..., 577.1130371094, 581.1229858398, 589.5457763672], [587.6046142578, 585.6354980469, 586.0928344727, ..., 584.8721313477, 588.2430419922, 592.5085449219]], [[588.0153808594, 587.3704833984, 586.6550903320, ..., 591.2977294922, 591.7604980469, 591.2844238281], [594.6320190430, 584.9013061523, 588.8773193359, ..., 588.9233398438, 584.2270507812, 582.8884277344], [587.5815429688, 575.6303710938, 577.2279663086, ..., 588.5678710938, 579.9028930664, 589.1498413086], ..., [589.9268188477, 583.1175537109, 577.7232055664, ..., 584.5743408203, 587.5334472656, 595.0125732422], [585.2252197266, 584.9291992188, 584.9176635742, ..., 583.7155151367, 588.4307861328, 590.5812377930], [592.4174804688, 583.1611328125, 577.8160400391, ..., 589.5422973633, 593.0537109375, 590.6257324219]], [[579.5441894531, 565.4550781250, 574.6450805664, ..., 578.5653686523, 579.3826293945, 576.0480346680], [575.9826660156, 574.5454711914, 576.0714111328, ..., 579.6741943359, 574.0231933594, 570.4577026367], [578.5194091797, 574.5836791992, 566.8870849609, ..., 581.1571044922, 565.3106079102, 572.2456054688], ..., [572.0755615234, 567.0267944336, 567.3985595703, ..., 574.4299926758, 570.9069824219, 586.8672485352], [581.1436767578, 582.8332519531, 578.4226684570, ..., 569.0413818359, 579.5062866211, 582.0877685547], [581.5795288086, 577.5985107422, 573.3118896484, ..., 569.9739990234, 585.3901367188, 583.6173706055]], ..., [[568.3113403320, 565.9417114258, 556.5572509766, ..., 561.5818481445, 561.0867919922, 570.3497314453], [568.9544067383, 558.2978515625, 568.3854980469, ..., 559.3563232422, 562.6853637695, 561.5353393555], [564.9359741211, 561.7117919922, 557.1620483398, ..., 572.7876586914, 567.1069946289, 556.8594360352], ..., [565.0916748047, 562.3317871094, 554.2063598633, ..., 565.8666992188, 562.0341796875, 561.8312377930], [564.6190795898, 564.3048095703, 557.1021118164, ..., 556.6746215820, 559.8140869141, 576.8021850586], [569.9702758789, 558.7573242188, 559.2811279297, ..., 567.6199340820, 562.3828125000, 565.1786499023]], [[575.2864379883, 551.0559692383, 558.0357055664, ..., 560.0012207031, 564.0629272461, 570.3760986328], [571.2073364258, 556.0667724609, 566.1008300781, ..., 565.0621337891, 558.5066528320, 562.0023193359], [559.9176635742, 553.8130493164, 557.9414062500, ..., 567.5859985352, 558.0100097656, 561.2138061523], ..., [561.8636474609, 549.5798339844, 551.8265991211, ..., 556.2512207031, 564.6067504883, 570.1931152344], [567.2440185547, 564.4647827148, 558.5792236328, ..., 555.2014770508, 570.6610717773, 560.5759277344], [574.1644897461, 553.7271728516, 549.0382080078, ..., 567.4614868164, 566.7980346680, 561.6908569336]], [[594.0851440430, 569.1356811523, 578.8761596680, ..., 575.6440429688, 585.1105957031, 578.2573242188], [581.8900756836, 573.3090209961, 584.8013305664, ..., 576.3011474609, 569.0520629883, 582.4334106445], [581.8972167969, 577.0387573242, 567.2723388672, ..., 585.1683959961, 570.5975952148, 573.2642211914], ..., [577.6181030273, 578.1201171875, 566.8511962891, ..., 578.5949096680, 580.1334228516, 589.6394653320], [581.1671752930, 583.5147094727, 577.5323486328, ..., 584.8950805664, 573.8445434570, 588.4440917969], [585.4477539062, 568.6908569336, 571.3515014648, ..., 582.8416137695, 583.0099487305, 583.6927490234]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[592., 572., 576., ..., 580., 584., 604.], [584., 576., 572., ..., 584., 576., 580.], [588., 572., 576., ..., 588., 580., 584.], ..., [584., 588., 572., ..., 588., 580., 588.], [576., 580., 572., ..., 584., 580., 588.], [584., 584., 584., ..., 584., 576., 600.]], [[588., 584., 584., ..., 588., 600., 592.], [596., 580., 588., ..., 592., 588., 580.], [584., 572., 584., ..., 580., 588., 580.], ..., [588., 588., 584., ..., 584., 580., 588.], [584., 592., 592., ..., 584., 604., 588.], [584., 588., 576., ..., 588., 600., 592.]], [[580., 564., 576., ..., 580., 592., 580.], [576., 572., 568., ..., 572., 564., 568.], [580., 572., 576., ..., 576., 572., 576.], ..., [576., 564., 568., ..., 564., 576., 584.], [576., 580., 584., ..., 572., 568., 580.], [588., 576., 568., ..., 564., 588., 576.]], ..., [[572., 564., 556., ..., 560., 568., 568.], [572., 552., 568., ..., 560., 556., 560.], [572., 556., 556., ..., 572., 560., 564.], ..., [568., 560., 548., ..., 568., 560., 568.], [568., 556., 560., ..., 564., 552., 576.], [572., 556., 548., ..., 580., 564., 568.]], [[568., 540., 564., ..., 560., 568., 576.], [572., 540., 560., ..., 564., 560., 568.], [556., 560., 560., ..., 576., 564., 544.], ..., [564., 552., 568., ..., 556., 568., 568.], [580., 576., 568., ..., 540., 572., 564.], [572., 556., 556., ..., 568., 580., 556.]], [[596., 564., 576., ..., 572., 592., 568.], [572., 572., 580., ..., 572., 572., 576.], [576., 572., 568., ..., 576., 560., 576.], ..., [572., 584., 572., ..., 576., 568., 576.], [584., 572., 588., ..., 580., 576., 600.], [576., 564., 568., ..., 580., 580., 580.]]]])
test_conv2d.test_module[Conv2D656-[((1, 128, 42, 130), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[293.4818115234, 280.2921447754, 281.2926025391, ..., 287.2152099609, 290.8132324219, 293.0929870605], [294.2868652344, 285.0881347656, 287.2413635254, ..., 291.0156860352, 282.2413635254, 281.7529907227], [288.7467956543, 274.4447326660, 275.1027832031, ..., 298.0074157715, 288.3718261719, 290.8058471680], ..., [283.0641174316, 285.7518310547, 278.3716430664, ..., 297.4023742676, 288.9408569336, 290.2016906738], [288.8995361328, 280.7724609375, 285.1314697266, ..., 291.9512023926, 293.1519165039, 296.3617553711], [290.2529296875, 284.0832519531, 285.9339599609, ..., 295.8728637695, 298.1440124512, 294.3629760742]], [[295.0973815918, 279.0131530762, 284.1960144043, ..., 289.0270385742, 291.2324523926, 284.7346191406], [288.9144287109, 279.4722900391, 283.4040832520, ..., 291.6141052246, 287.7746582031, 288.2087707520], [282.3191528320, 280.0502014160, 276.9533691406, ..., 293.9096374512, 281.8837585449, 285.6517028809], ..., [285.7076416016, 288.8884277344, 284.4855346680, ..., 296.0806884766, 287.1823730469, 290.8904113770], [291.4389038086, 286.7257995605, 285.9681396484, ..., 291.3363342285, 285.7774353027, 298.9489746094], [292.0814819336, 285.1325988770, 286.0491943359, ..., 297.1560974121, 286.3880004883, 297.2867431641]], [[290.9515686035, 273.9773559570, 274.8745727539, ..., 284.8002014160, 284.2394714355, 285.2827148438], [291.1639709473, 277.7492675781, 276.1764831543, ..., 287.6536560059, 279.3781738281, 283.8843383789], [276.8177795410, 266.9536437988, 274.3259887695, ..., 293.0251464844, 273.8762817383, 276.1171875000], ..., [284.3624572754, 286.5005187988, 279.8199768066, ..., 287.5964660645, 283.8707275391, 284.2675170898], [280.8894958496, 278.7581176758, 279.4758911133, ..., 283.8580627441, 282.3450622559, 285.9754943848], [288.6195678711, 277.9557495117, 278.9500427246, ..., 290.4832763672, 283.0315551758, 286.0247192383]], ..., [[298.0980834961, 282.4517822266, 285.6116943359, ..., 289.4940490723, 288.6870422363, 293.0394897461], [290.5907287598, 281.5507507324, 281.7158813477, ..., 293.0163879395, 282.7959594727, 282.9884643555], [281.8031005859, 273.7664184570, 274.6153869629, ..., 295.6575317383, 290.8420410156, 285.8348083496], ..., [287.5823364258, 285.3755798340, 285.2485046387, ..., 289.1954345703, 288.2567443848, 293.9799499512], [290.5525512695, 282.1520385742, 282.1953430176, ..., 291.9361572266, 285.4444580078, 291.7230529785], [288.1904602051, 285.6293640137, 282.3803710938, ..., 292.8878784180, 287.4517211914, 290.2450256348]], [[304.4684448242, 292.0605773926, 297.0230102539, ..., 299.6366577148, 309.2947692871, 299.7487182617], [297.1444702148, 284.4284973145, 286.9756164551, ..., 302.8532714844, 293.1518249512, 294.4190063477], [296.8000488281, 289.9638061523, 284.7035827637, ..., 303.1865539551, 290.4288940430, 296.1447753906], ..., [298.5395812988, 293.4274597168, 293.3386840820, ..., 299.6168823242, 297.8313598633, 306.2516479492], [295.3921813965, 294.9132080078, 293.8356323242, ..., 296.4295349121, 293.0945129395, 302.4345703125], [299.0017700195, 294.1734924316, 297.4299926758, ..., 309.2293701172, 304.8518371582, 304.8681030273]], [[300.5447998047, 287.6018981934, 294.4805297852, ..., 301.6821289062, 300.0543823242, 297.8362121582], [297.5195617676, 290.4735107422, 289.0948486328, ..., 300.1345825195, 296.3657226562, 294.8784179688], [294.9891357422, 279.7922363281, 276.3819274902, ..., 303.3764648438, 292.9402465820, 293.1866455078], ..., [298.6577148438, 292.5009765625, 295.5618896484, ..., 296.8864440918, 298.1687011719, 303.9886169434], [297.6807250977, 291.5947570801, 293.4549865723, ..., 299.1923217773, 302.0130920410, 301.4416503906], [298.3737792969, 299.4903564453, 293.6893310547, ..., 305.1280212402, 293.1235961914, 304.3650207520]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[290., 280., 280., ..., 286., 282., 290.], [292., 284., 282., ..., 292., 280., 282.], [282., 274., 274., ..., 292., 284., 292.], ..., [282., 282., 272., ..., 294., 284., 286.], [288., 280., 282., ..., 288., 288., 290.], [286., 278., 280., ..., 290., 296., 292.]], [[292., 274., 280., ..., 284., 292., 282.], [284., 278., 282., ..., 288., 286., 292.], [278., 276., 276., ..., 292., 278., 282.], ..., [280., 288., 282., ..., 296., 282., 286.], [288., 282., 288., ..., 288., 282., 294.], [288., 284., 284., ..., 298., 288., 292.]], [[284., 270., 270., ..., 284., 282., 280.], [288., 274., 272., ..., 284., 272., 278.], [276., 262., 272., ..., 290., 268., 270.], ..., [278., 284., 274., ..., 282., 278., 284.], [278., 272., 276., ..., 280., 276., 286.], [286., 274., 278., ..., 286., 276., 280.]], ..., [[300., 276., 280., ..., 288., 286., 288.], [288., 280., 282., ..., 288., 280., 280.], [276., 272., 270., ..., 290., 294., 286.], ..., [286., 280., 278., ..., 284., 292., 292.], [288., 280., 280., ..., 292., 282., 294.], [284., 282., 272., ..., 288., 286., 284.]], [[306., 294., 292., ..., 298., 310., 296.], [290., 280., 282., ..., 300., 290., 290.], [296., 284., 282., ..., 302., 290., 294.], ..., [298., 290., 296., ..., 294., 294., 302.], [294., 292., 288., ..., 288., 290., 300.], [298., 292., 296., ..., 308., 300., 306.]], [[298., 284., 288., ..., 298., 294., 296.], [298., 290., 284., ..., 298., 294., 294.], [288., 274., 272., ..., 300., 290., 290.], ..., [296., 288., 292., ..., 294., 294., 300.], [294., 286., 292., ..., 294., 298., 298.], [302., 296., 290., ..., 302., 292., 302.]]]])
test / run-tests (n150, 1)
Process completed with exit code 1.
test_add.test_module[Add0-[((2, 1, 2048), torch.float32), ((1, 2048), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[1.3599805832, 1.2153040171, 0.3786585331, ..., 0.6706525087, 1.7745194435, 0.3440857530]], [[1.1024224758, 0.8026893139, 0.4665691257, ..., 0.8005325794, 1.5674996376, 0.1167586446]]]), compiled_model=tensor([[[1.3599805832, 1.2153040171, 0.3786585331, ..., 0.6706525087, 1.7745194435, 0.3440857530]], [[0.9646650553, 0.4311635494, 0.8605672121, ..., 1.2258419991, 0.9294204712, 0.9939993024]]])
test_add.test_module[Add0-[((2, 1, 1536), torch.float32), ((1, 1536), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[1.4630522728, 1.6477220058, 0.1509722471, ..., 1.4470398426, 0.5820842981, 1.1419926882]], [[1.3005423546, 1.5406150818, 0.7836635709, ..., 1.5978899002, 0.8315522671, 1.7088755369]]]), compiled_model=tensor([[[1.4630522728, 1.6477220058, 0.1509722471, ..., 1.4470398426, 0.5820842981, 1.1419926882]], [[0.5512740016, 1.4046423435, 1.5053542852, ..., 0.6760584116, 1.2907935381, 1.5603219271]]])
test_add.test_module[Add0-[((2, 1, 1024), torch.float32), ((1, 1024), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[0.7349550724, 1.1238288879, 0.2648654580, ..., 0.8409469128, 0.9050856233, 1.4962310791]], [[0.7036654949, 0.3833577633, 0.3880954981, ..., 1.2408131361, 1.5215494633, 1.0661480427]]]), compiled_model=tensor([[[0.7349550724, 1.1238288879, 0.2648654580, ..., 0.8409469128, 0.9050856233, 1.4962310791]], [[0.5946006775, 0.1729068160, 0.9467984438, ..., 0.5874912739, 0.8263171315, 0.2909160852]]])
test_add.test_module[Add46-[((1, 256), torch.int64)]]: forge/test/models_ops/test_add.py#L18561
TypeError: Dtype mismatch: framework_model.dtype=torch.int64, compiled_model.dtype=torch.int32
test_add.test_module[Add1-[((1, 120, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1924707890]], [[1.4644359350]], [[0.7846915722]], [[0.8282446265]], [[1.0036369562]], [[1.3302928209]], [[1.1863075495]], [[1.5926588774]], [[1.1518421173]], [[1.3285204172]], [[1.0451076031]], [[1.0979313850]], [[0.7185398936]], [[0.8650730848]], [[0.9901025891]], [[1.2147359848]], [[1.3938817978]], [[1.4962255955]], [[0.8572435975]], [[0.9784827232]], [[1.3778226376]], [[1.6114081144]], [[1.0933140516]], [[1.5703699589]], [[1.1156225204]], [[1.2491211891]], [[1.6489522457]], [[0.7323789597]], [[0.8814451694]], [[1.0696315765]], [[1.0013141632]], [[1.6282145977]], [[0.8721243143]], [[0.9660477042]], [[0.8468939066]], [[0.7279336452]], [[0.9043439031]], [[1.6260131598]], [[1.4193233252]], [[1.4385504723]], [[1.2225098610]], [[0.9398723841]], [[1.2808065414]], [[0.7293667793]], [[0.8349310160]], [[0.9384491444]], [[1.5116831064]], [[1.4893747568]], [[0.9744666219]], [[1.1781729460]], [[1.5159945488]], [[1.6932806969]], [[1.3946552277]], [[1.2637605667]], [[1.5314573050]], [[0.9018129706]], [[1.2893861532]], [[0.8085613847]], [[0.8496710658]], [[0.9379223585]], [[1.4224506617]], [[1.3972942829]], [[0.9000378847]], [[1.3472676277]], [[1.4707001448]], [[1.1331055164]], [[1.2153048515]], [[1.3120665550]], [[1.5064024925]], [[1.6763112545]], [[0.8109023571]], [[1.0129792690]], [[1.3927190304]], [[1.6104888916]], [[1.6313178539]], [[1.6373925209]], [[1.2957214117]], [[0.7614228129]], [[1.2422103882]], [[0.8834114671]], [[0.7302370667]], [[1.6404603720]], [[1.5763940811]], [[0.6974501610]], [[1.2898001671]], [[1.1119841337]], [[1.1139335632]], [[0.9673357010]], [[1.3884922266]], [[0.9000623822]], [[1.3795098066]], [[1.4490681887]], [[1.5541498661]], [[1.3831696510]], [[0.7013465166]], [[0.8718656898]], [[1.4458715916]], [[1.3008648157]], [[0.8061721325]], [[0.9083043933]], [[1.6665887833]], [[1.5331230164]], [[0.9782015681]], [[1.0703717470]], [[0.7199150920]], [[1.1872270107]], [[0.8196846843]], [[0.8105357885]], [[1.1686643362]], [[1.2712867260]], [[0.9914489985]], [[1.4929029942]], [[0.8919445872]], [[1.6498992443]], [[1.5388641357]], [[0.7745726705]], [[1.0717719793]], [[1.2187755108]], [[1.2691646814]], [[1.3148012161]]]]), compiled_model=tensor([[[[1.1924707890]], [[0.8148299456]], [[0.6633052826]], [[0.2286634445]], [[1.3055853844]], [[0.9863031507]], [[1.1027418375]], [[1.2536740303]], [[0.7628709674]], [[1.1382660866]], [[1.3192000389]], [[0.9223065376]], [[0.1159237027]], [[0.7854200006]], [[0.8128086329]], [[0.6726787686]], [[0.8068621159]], [[1.3291288614]], [[0.8704766035]], [[0.8339579105]], [[0.6927861571]], [[1.1125488281]], [[1.0079274178]], [[1.1160495281]], [[0.6277523637]], [[1.3501378298]], [[1.2956428528]], [[0.9237640500]], [[0.8132946491]], [[0.8395851851]], [[0.5404931903]], [[1.1268116236]], [[0.6457512975]], [[1.0180232525]], [[0.4824424386]], [[0.0413870811]], [[0.7002156377]], [[1.5226066113]], [[1.6835284233]], [[0.8462309241]], [[1.1808503866]], [[1.1559228897]], [[0.9755937457]], [[0.5071379542]], [[0.9486122727]], [[0.4561477900]], [[1.2769508362]], [[0.8887930512]], [[1.7509087324]], [[1.6343026161]], [[2.6938037872]], [[1.9780235291]], [[2.1134800911]], [[1.4381518364]], [[2.2766494751]], [[1.3393878937]], [[1.5243244171]], [[0.4717222452]], [[1.1534569263]], [[0.5437101722]], [[2.3805334568]], [[2.1952209473]], [[1.8717925549]], [[1.7291785479]], [[2.4024157524]], [[1.7025163174]], [[1.2690907717]], [[1.2935867310]], [[1.7511062622]], [[1.7696478367]], [[0.8046296239]], [[1.3133471012]], [[1.4113487005]], [[1.5270676613]], [[2.2407677174]], [[2.2585611343]], [[1.4315385818]], [[1.0232164860]], [[0.8506837487]], [[0.5651270151]], [[1.6746479273]], [[2.3368244171]], [[2.5706095695]], [[1.0940093994]], [[1.7166328430]], [[1.2639145851]], [[1.2058053017]], [[1.0001254082]], [[2.1825125217]], [[1.5895904303]], [[1.9450144768]], [[2.1591041088]], [[2.6421155930]], [[1.7767993212]], [[1.0637261868]], [[1.9813156128]], [[2.1842279434]], [[1.6359007359]], [[0.6158173680]], [[1.3820121288]], [[1.9801402092]], [[1.4038034678]], [[0.7981007099]], [[1.5333373547]], [[0.7468454838]], [[1.2454074621]], [[1.1195642948]], [[1.0777006149]], [[2.2947158813]], [[1.4905998707]], [[0.5606157184]], [[1.1611907482]], [[1.3861601353]], [[1.9556381702]], [[1.8524155617]], [[1.3283585310]], [[0.8848351836]], [[1.4883816242]], [[1.7438490391]], [[1.1415363550]]]])
test_add.test_module[Add1-[((1, 16, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1939241886]], [[1.4658894539]], [[0.7861450315]], [[0.8296980858]], [[1.0050904751]], [[1.3317463398]], [[1.1877610683]], [[1.5941123962]], [[1.1532955170]], [[1.3299739361]], [[1.0465610027]], [[1.0993849039]], [[0.7199933529]], [[0.8665265441]], [[0.9915560484]], [[1.2161893845]]]]), compiled_model=tensor([[[[1.1939241886]], [[1.3211288452]], [[1.0412155390]], [[0.1681953073]], [[0.4926538467]], [[1.0074961185]], [[0.7951934338]], [[1.8284451962]], [[0.6315381527]], [[0.9021398425]], [[0.4995732307]], [[0.4334368110]], [[0.2304555178]], [[1.0986579657]], [[1.0169975758]], [[1.2608580589]]]])
test_add.test_module[Add1-[((1, 240, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[1.1613526344]], [[1.4333178997]], [[0.7535734773]], [[0.7971265316]], [[0.9725188613]], [[1.2991747856]], [[1.1551895142]], [[1.5615408421]], [[1.1207239628]], [[1.2974023819]], [[1.0139894485]], [[1.0668133497]], [[0.6874217987]], [[0.8339549899]], [[0.9589844942]], [[1.1836178303]], [[1.3627636433]], [[1.4651074409]], [[0.8261255026]], [[0.9473646283]], [[1.3467046022]], [[1.5802900791]], [[1.0621960163]], [[1.5392519236]], [[1.0845043659]], [[1.2180030346]], [[1.6178340912]], [[0.7012608647]], [[0.8503270745]], [[1.0385134220]], [[0.9701960683]], [[1.5970964432]], [[0.8410062194]], [[0.9349296093]], [[0.8157758117]], [[0.6968155503]], [[0.8732258081]], [[1.5948951244]], [[1.3882052898]], [[1.4074323177]], [[1.1913918257]], [[0.9087542892]], [[1.2496883869]], [[0.6982486844]], [[0.8038129210]], [[0.9073310494]], [[1.4805650711]], [[1.4582567215]], [[0.9433485270]], [[1.1470549107]], [[1.4848763943]], [[1.6621625423]], [[1.3635370731]], [[1.2326424122]], [[1.5003392696]], [[0.8706948757]], [[1.2582681179]], [[0.7774432898]], [[0.8185529709]], [[0.9068042636]], [[1.3913326263]], [[1.3661762476]], [[0.8689197898]], [[1.3161495924]], [[1.4395821095]], [[1.1019873619]], [[1.1841868162]], [[1.2809484005]], [[1.4752843380]], [[1.6451931000]], [[0.7797842622]], [[0.9818611741]], [[1.3616009951]], [[1.5793707371]], [[1.6001996994]], [[1.6062743664]], [[1.2646033764]], [[0.7303047180]], [[1.2110922337]], [[0.8522933722]], [[0.6991189718]], [[1.6093423367]], [[1.5452759266]], [[0.6663320661]], [[1.2586820126]], [[1.0808660984]], [[1.0828154087]], [[0.9362176061]], [[1.3573741913]], [[0.8689442873]], [[1.3483917713]], [[1.4179501534]], [[1.5230318308]], [[1.3520516157]], [[0.6702284217]], [[0.8407475948]], [[1.4147535563]], [[1.2697467804]], [[0.7750540376]], [[0.8771862984]], [[1.6354706287]], [[1.5020049810]], [[0.9470834732]], [[1.0392537117]], [[0.6887969971]], [[1.1561089754]], [[0.7885665894]], [[0.7794176936]], [[1.1375463009]], [[1.2401685715]], [[0.9603309035]], [[1.4617848396]], [[0.8608264923]], [[1.6187810898]], [[1.5077459812]], [[0.7434545755]], [[1.0406539440]], [[1.1876573563]], [[1.2380466461]], [[1.2836831808]], [[1.3613102436]], [[1.1950461864]], [[0.9211316705]], [[1.4016904831]], [[0.6854715943]], [[0.8687427044]], [[1.0399310589]], [[0.9215393662]], [[0.9901793599]], [[0.7552852035]], [[1.0587384701]], [[1.2719743252]], [[0.8393631577]], [[1.1394363642]], [[1.5230214596]], [[1.1136958599]], [[1.1789921522]], [[1.1219615936]], [[1.2662867308]], [[1.4830157757]], [[1.6387190819]], [[1.4826240540]], [[1.6398028135]], [[1.1289352179]], [[0.7159352899]], [[0.9280574322]], [[1.5055487156]], [[1.1618547440]], [[0.9165728688]], [[0.7819401622]], [[0.6971700191]], [[0.7430919409]], [[1.0636776686]], [[1.4392991066]], [[1.4354165792]], [[0.6828801036]], [[1.4769871235]], [[0.7738413215]], [[1.0593909025]], [[0.9623597264]], [[1.0687884092]], [[1.0669246912]], [[0.7164210677]], [[0.7333770990]], [[1.0868563652]], [[1.1715621948]], [[0.9379585981]], [[1.3534456491]], [[0.7150668502]], [[1.1313524246]], [[1.6048057079]], [[0.9611500502]], [[1.6165976524]], [[1.3461729288]], [[0.7138655782]], [[1.4814447165]], [[1.1073987484]], [[0.9418926239]], [[1.5649226904]], [[0.7610484362]], [[1.2187485695]], [[1.0604116917]], [[1.5221524239]], [[1.3046683073]], [[1.4053487778]], [[1.3416755199]], [[1.0448586941]], [[1.0599433184]], [[0.7530553341]], [[1.4360181093]], [[1.5620851517]], [[1.5072084665]], [[0.8124068975]], [[1.1873960495]], [[0.8126289845]], [[0.8898539543]], [[0.8737432957]], [[1.3359684944]], [[0.8671394587]], [[1.1541874409]], [[1.1861300468]], [[1.4874075651]], [[0.7871360183]], [[0.8218399286]], [[0.8747652769]], [[1.5150628090]], [[0.9853635430]], [[1.5868403912]], [[1.3458998203]], [[1.2284090519]], [[1.1613740921]], [[1.0662553310]], [[1.2278292179]], [[1.0509237051]], [[1.1615828276]], [[1.2288925648]], [[0.7739934921]], [[0.9030303359]], [[1.5688424110]], [[0.7593227029]], [[1.1291929483]], [[1.6597154140]], [[1.3457145691]], [[1.1792526245]], [[0.7317910790]], [[1.4127850533]], [[0.8089556694]], [[1.0231639147]], [[0.9973378778]], [[1.0910522938]], [[1.1705651283]], [[1.5774998665]], [[1.2275154591]], [[1.6129424572]], [[1.4709522724]], [[0.8489890695]], [[1.3893480301]], [[0.8116480112]], [[0.9531834722]], [[1.3121573925]]]]), compiled_model=tensor([[[[1.1613526344]], [[1.5249770880]], [[1.7245111465]], [[0.9210493565]], [[1.3505897522]], [[1.6944515705]], [[0.8003258109]], [[2.0040967464]], [[1.3811956644]], [[1.5067905188]], [[0.6095312238]], [[0.6455270648]], [[1.2008302212]], [[1.9355669022]], [[1.2989851236]], [[1.6350157261]], [[1.2476643324]], [[1.5346825123]], [[0.8690923452]], [[0.4297428727]], [[1.2927756310]], [[1.7325015068]], [[1.5078036785]], [[2.4640054703]], [[0.8933912516]], [[1.9885509014]], [[2.6151683331]], [[1.1115899086]], [[1.2592298985]], [[1.4635250568]], [[1.7132937908]], [[1.7561863661]], [[1.4652962685]], [[0.9121308923]], [[0.5601723194]], [[1.0100221634]], [[0.9547418356]], [[1.8345258236]], [[1.3017680645]], [[1.6498332024]], [[1.5868426561]], [[1.5327208042]], [[1.3922095299]], [[0.7406721711]], [[0.9189903140]], [[0.9814928174]], [[1.7915432453]], [[1.4640591145]], [[0.9672173262]], [[1.7065682411]], [[2.0951709747]], [[1.9135704041]], [[1.5807652473]], [[1.3043627739]], [[2.0041885376]], [[0.9785480499]], [[1.1542072296]], [[0.9824644327]], [[0.8565819263]], [[0.8017668128]], [[1.3843195438]], [[1.1907658577]], [[0.5913726687]], [[0.8428386450]], [[1.6202583313]], [[0.5646792650]], [[1.2239234447]], [[0.9477255940]], [[1.0689542294]], [[1.5699195862]], [[0.3549597263]], [[0.9319675565]], [[1.2946953773]], [[1.0430300236]], [[1.5183529854]], [[1.6541429758]], [[1.2974227667]], [[0.5022693276]], [[0.6361024380]], [[0.6101202965]], [[0.7076767683]], [[1.2618076801]], [[1.5699703693]], [[0.8342173100]], [[0.8325102329]], [[0.9207009077]], [[1.1244689226]], [[0.8103104830]], [[1.2340438366]], [[0.7662809491]], [[0.7902218103]], [[1.2921481133]], [[1.7041707039]], [[1.6375124454]], [[0.7990078330]], [[0.7426530719]], [[1.4831645489]], [[0.8614108562]], [[0.1956138015]], [[0.2821297050]], [[1.9691795111]], [[1.6543035507]], [[0.4363705516]], [[1.0697789192]], [[0.9012847543]], [[1.4908204079]], [[1.0606679916]], [[1.0016984940]], [[0.8578205705]], [[0.8996016979]], [[1.2057592869]], [[1.5768785477]], [[0.3948409557]], [[1.9031937122]], [[1.5842292309]], [[0.8509262800]], [[0.5621696711]], [[1.1660112143]], [[0.8976693153]], [[1.5092654228]], [[1.1062438488]], [[1.2246062756]], [[0.8448465466]], [[1.4493286610]], [[0.3504617810]], [[0.9474323392]], [[0.5255995989]], [[0.8693708181]], [[0.4867873788]], [[0.0969203711]], [[0.4921145439]], [[1.5016132593]], [[0.9447849393]], [[1.4434481859]], [[1.7584860325]], [[0.5020771623]], [[0.6726805568]], [[0.8760742545]], [[0.7764745355]], [[1.6651179790]], [[1.0956192017]], [[1.0735771656]], [[0.9916610718]], [[0.6799663901]], [[0.9620779753]], [[1.1723413467]], [[1.6983714104]], [[1.3828223944]], [[1.1960700750]], [[0.4888126254]], [[0.7520750165]], [[1.0234611034]], [[1.0639910698]], [[1.7740411758]], [[1.5296519995]], [[0.8286082149]], [[1.1368949413]], [[0.8486590385]], [[0.9517481327]], [[0.6778550148]], [[0.6218374372]], [[0.6212703586]], [[0.1665846109]], [[0.9039462209]], [[1.2772258520]], [[0.9495597482]], [[0.4835195541]], [[1.5748023987]], [[0.8697152734]], [[1.0034234524]], [[1.2036421299]], [[1.2555779219]], [[1.6559718847]], [[0.8015041947]], [[1.0273109674]], [[1.6960356236]], [[0.7600573301]], [[1.0578738451]], [[1.1157321930]], [[0.5175952315]], [[1.4782030582]], [[0.9159752131]], [[1.0034472942]], [[0.9724557996]], [[1.1045300961]], [[1.0801150799]], [[0.9276129603]], [[1.3572621346]], [[0.6147294641]], [[0.9622020721]], [[1.4226162434]], [[1.5818560123]], [[0.8953309655]], [[0.5653306842]], [[0.5580617189]], [[0.3531856537]], [[0.4953044653]], [[1.3510191441]], [[0.3469783068]], [[1.1749548912]], [[1.4454231262]], [[1.3551056385]], [[0.2887961268]], [[0.4776014090]], [[0.8188512921]], [[0.9688076973]], [[1.0686726570]], [[0.9678095579]], [[0.7001571059]], [[0.5774827003]], [[0.8948463202]], [[1.2373757362]], [[0.5894936919]], [[1.3014211655]], [[0.7964753509]], [[1.2102385759]], [[0.6316989064]], [[0.2870747447]], [[1.8184108734]], [[0.8634483814]], [[1.4610947371]], [[1.7472255230]], [[0.8505840302]], [[1.4314484596]], [[0.5935672522]], [[1.4847972393]], [[0.2429450750]], [[0.7142546177]], [[0.3413030505]], [[0.7312100530]], [[1.1133346558]], [[1.0198228359]], [[1.2218015194]], [[1.7162498236]], [[1.3755109310]], [[0.3493513465]], [[0.8365921974]], [[0.4922936559]], [[1.0075664520]], [[1.6402595043]]]])
test_add.test_module[Add1-[((1, 144, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.5470958352]], [[0.8190610409]], [[0.1393166780]], [[0.1828697324]], [[0.3582620621]], [[0.6849179268]], [[0.5409326553]], [[0.9472839832]], [[0.5064672232]], [[0.6831455231]], [[0.3997327089]], [[0.4525565505]], [[0.0731649995]], [[0.2196981907]], [[0.3447276950]], [[0.5693610311]], [[0.7485068440]], [[0.8508506417]], [[0.2118687034]], [[0.3331078291]], [[0.7324478030]], [[0.9660332203]], [[0.4479391575]], [[0.9249951243]], [[0.4702475667]], [[0.6037462950]], [[1.0035773516]], [[0.0870040655]], [[0.2360702753]], [[0.4242566228]], [[0.3559392691]], [[0.9828396440]], [[0.2267494202]], [[0.3206728101]], [[0.2015190125]], [[0.0825587511]], [[0.2589690089]], [[0.9806382656]], [[0.7739484310]], [[0.7931755185]], [[0.5771350265]], [[0.2944974899]], [[0.6354315877]], [[0.0839918852]], [[0.1895561218]], [[0.2930742502]], [[0.8663082123]], [[0.8439998627]], [[0.3290917277]], [[0.5327980518]], [[0.8706195951]], [[1.0479058027]], [[0.7492803335]], [[0.6183856726]], [[0.8860824108]], [[0.2564380765]], [[0.6440112591]], [[0.1631864905]], [[0.2042961717]], [[0.2925474644]], [[0.7770757675]], [[0.7519194484]], [[0.2546629906]], [[0.7018927932]], [[0.8253252506]], [[0.4877305627]], [[0.5699300170]], [[0.6666916013]], [[0.8610275388]], [[1.0309362411]], [[0.1655274630]], [[0.3676043749]], [[0.7473441958]], [[0.9651139379]], [[0.9859429002]], [[0.9920176268]], [[0.6503465176]], [[0.1160479188]], [[0.5968354940]], [[0.2380365729]], [[0.0848621726]], [[0.9950854778]], [[0.9310191274]], [[0.0520752668]], [[0.6444252729]], [[0.4666092396]], [[0.4685586691]], [[0.3219608068]], [[0.7431173325]], [[0.2546874881]], [[0.7341349125]], [[0.8036932945]], [[0.9087750316]], [[0.7377948165]], [[0.0559716225]], [[0.2264907956]], [[0.8004967570]], [[0.6554899216]], [[0.1607972383]], [[0.2629294991]], [[1.0212138891]], [[0.8877481818]], [[0.3328266740]], [[0.4249968529]], [[0.0745401978]], [[0.5418521762]], [[0.1743097901]], [[0.1651608944]], [[0.5232894421]], [[0.6259117723]], [[0.3460741043]], [[0.8475280404]], [[0.2465696931]], [[1.0045242310]], [[0.8934891820]], [[0.1291977763]], [[0.4263970852]], [[0.5734005570]], [[0.6237897873]], [[0.6694263816]], [[0.7470533848]], [[0.5807893276]], [[0.3068748713]], [[0.7874337435]], [[0.0712147951]], [[0.2544859052]], [[0.4256743193]], [[0.3072825670]], [[0.3759225607]], [[0.1410284042]], [[0.4444816709]], [[0.6577174664]], [[0.2251063585]], [[0.5251795650]], [[0.9087646604]], [[0.4994391203]], [[0.5647353530]], [[0.5077047944]], [[0.6520299315]], [[0.8687589765]], [[1.0244623423]], [[0.8683671951]], [[1.0255460739]], [[0.5146784186]]]]), compiled_model=tensor([[[[0.5470958352]], [[1.0311832428]], [[0.9289300442]], [[0.6287892461]], [[0.5588996410]], [[0.7509227991]], [[0.5221673846]], [[0.9744406343]], [[0.8542096019]], [[1.4065092802]], [[1.1192140579]], [[0.4195013642]], [[0.8342167735]], [[0.2776042223]], [[0.6881833076]], [[0.8157854676]], [[1.1013599634]], [[1.2018400431]], [[0.2123544812]], [[0.3505496383]], [[1.1033687592]], [[1.4216600657]], [[0.6699624658]], [[1.5625054836]], [[0.4693791270]], [[1.0191633701]], [[1.8924478292]], [[0.3322188258]], [[1.1367325783]], [[1.0544942617]], [[0.3538695574]], [[1.7483490705]], [[0.6182129383]], [[0.5466301441]], [[1.0505063534]], [[0.1276718974]], [[0.7617822289]], [[1.3251147270]], [[1.5801655054]], [[1.3819085360]], [[1.2665483952]], [[0.9202377200]], [[0.9643549919]], [[0.4279999137]], [[0.2266761661]], [[1.0131571293]], [[1.7124580145]], [[1.6352729797]], [[0.4255633354]], [[1.0042587519]], [[0.9673132896]], [[1.2218244076]], [[0.9070883393]], [[1.2384189367]], [[1.0372865200]], [[0.6946902275]], [[1.1142060757]], [[0.9346587658]], [[0.2754969001]], [[0.3984521031]], [[0.9359057546]], [[1.5510468483]], [[0.5240912437]], [[1.5727978945]], [[1.4552898407]], [[1.0002043247]], [[1.0153687000]], [[1.0170116425]], [[1.3729214668]], [[1.3659247160]], [[0.6111750007]], [[0.8805616498]], [[0.8054023981]], [[1.1522090435]], [[1.8388500214]], [[1.0354050398]], [[1.0636041164]], [[1.0598280430]], [[1.2266147137]], [[0.7013538480]], [[0.1007179618]], [[1.6919351816]], [[1.0240395069]], [[0.3593038917]], [[0.9258278608]], [[0.8417263031]], [[0.9231885672]], [[1.1835253239]], [[1.2546975613]], [[1.1516946554]], [[1.4891519547]], [[0.9367470741]], [[1.5821878910]], [[0.8335075378]], [[0.2932198048]], [[0.8227128983]], [[1.4147535563]], [[1.3614059687]], [[1.7459917068]], [[1.0011091232]], [[2.0135414600]], [[1.8972818851]], [[0.5922198296]], [[1.4818096161]], [[0.9492686391]], [[1.3654971123]], [[0.3841083050]], [[0.3581314087]], [[1.6509547234]], [[2.3417804241]], [[1.3003315926]], [[1.9131827354]], [[0.7457271814]], [[1.6883561611]], [[1.5507128239]], [[0.2258328199]], [[0.9867249131]], [[1.3398687840]], [[1.6836543083]], [[2.2084364891]], [[1.1701970100]], [[1.9655940533]], [[1.9184658527]], [[1.8120195866]], [[1.0943744183]], [[1.2937543392]], [[1.7830288410]], [[1.0806293488]], [[1.6144695282]], [[0.7324864864]], [[0.8031349778]], [[1.5851809978]], [[0.9208791852]], [[1.3790671825]], [[1.4365842342]], [[1.3560967445]], [[1.5744429827]], [[1.7459280491]], [[1.4088078737]], [[1.5254392624]], [[1.7538964748]], [[1.5567858219]], [[1.9507809877]], [[1.1347376108]]]])
test_add.test_module[Add1-[((1, 288, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.8819184899]], [[1.1538836956]], [[0.4741393328]], [[0.5176923871]], [[0.6930847168]], [[1.0197405815]], [[0.8757553101]], [[1.2821066380]], [[0.8412898779]], [[1.0179681778]], [[0.7345553637]], [[0.7873792052]], [[0.4079876542]], [[0.5545208454]], [[0.6795503497]], [[0.9041836858]], [[1.0833294392]], [[1.1856732368]], [[0.5466913581]], [[0.6679304838]], [[1.0672705173]], [[1.3008558750]], [[0.7827618122]], [[1.2598178387]], [[0.8050702214]], [[0.9385689497]], [[1.3384000063]], [[0.4218267202]], [[0.5708929300]], [[0.7590792775]], [[0.6907619238]], [[1.3176622391]], [[0.5615720749]], [[0.6554954648]], [[0.5363416672]], [[0.4173814058]], [[0.5937916636]], [[1.3154609203]], [[1.1087710857]], [[1.1279981136]], [[0.9119576812]], [[0.6293201447]], [[0.9702542424]], [[0.4188145399]], [[0.5243787766]], [[0.6278969049]], [[1.2011308670]], [[1.1788225174]], [[0.6639143825]], [[0.8676207066]], [[1.2054421902]], [[1.3827284575]], [[1.0841029882]], [[0.9532083273]], [[1.2209050655]], [[0.5912607312]], [[0.9788339138]], [[0.4980091453]], [[0.5391188264]], [[0.6273701191]], [[1.1118984222]], [[1.0867421627]], [[0.5894856453]], [[1.0367155075]], [[1.1601479053]], [[0.8225532174]], [[0.9047526717]], [[1.0015141964]], [[1.1958501339]], [[1.3657588959]], [[0.5003501177]], [[0.7024270296]], [[1.0821669102]], [[1.2999365330]], [[1.3207654953]], [[1.3268402815]], [[0.9851691723]], [[0.4508705735]], [[0.9316581488]], [[0.5728592277]], [[0.4196848273]], [[1.3299081326]], [[1.2658417225]], [[0.3868979216]], [[0.9792479277]], [[0.8014318943]], [[0.8033813238]], [[0.6567834616]], [[1.0779399872]], [[0.5895101428]], [[1.0689575672]], [[1.1385159492]], [[1.2435977459]], [[1.0726175308]], [[0.3907942772]], [[0.5613134503]], [[1.1353194714]], [[0.9903125763]], [[0.4956198931]], [[0.5977521539]], [[1.3560365438]], [[1.2225708961]], [[0.6676493287]], [[0.7598195076]], [[0.4093628526]], [[0.8766748309]], [[0.5091324449]], [[0.4999835491]], [[0.8581120968]], [[0.9607344270]], [[0.6808967590]], [[1.1823506355]], [[0.5813923478]], [[1.3393468857]], [[1.2283117771]], [[0.4640204310]], [[0.7612197399]], [[0.9082232118]], [[0.9586124420]], [[1.0042490959]], [[1.0818760395]], [[0.9156119823]], [[0.6416975260]], [[1.1222563982]], [[0.4060374498]], [[0.5893085599]], [[0.7604969740]], [[0.6421052217]], [[0.7107452154]], [[0.4758510590]], [[0.7793043256]], [[0.9925401211]], [[0.5599290133]], [[0.8600022197]], [[1.2435872555]], [[0.8342617750]], [[0.8995580077]], [[0.8425274491]], [[0.9868525863]], [[1.2035815716]], [[1.3592849970]], [[1.2031898499]], [[1.3603687286]], [[0.8495010734]], [[0.4365011454]], [[0.6486232877]], [[1.2261145115]], [[0.8824206591]], [[0.6371387243]], [[0.5025060177]], [[0.4177358747]], [[0.4636577964]], [[0.7842435241]], [[1.1598649025]], [[1.1559824944]], [[0.4034459591]], [[1.1975529194]], [[0.4944071770]], [[0.7799567580]], [[0.6829255819]], [[0.7893542647]], [[0.7874905467]], [[0.4369869232]], [[0.4539429545]], [[0.8074221611]], [[0.8921279907]], [[0.6585244536]], [[1.0740115643]], [[0.4356327057]], [[0.8519182801]], [[1.3253716230]], [[0.6817159057]], [[1.3371634483]], [[1.0667388439]], [[0.4344314337]], [[1.2020106316]], [[0.8279646635]], [[0.6624584794]], [[1.2854886055]], [[0.4816142917]], [[0.9393143654]], [[0.7809775472]], [[1.2427182198]], [[1.0252342224]], [[1.1259145737]], [[1.0622413158]], [[0.7654245496]], [[0.7805091739]], [[0.4736211896]], [[1.1565840244]], [[1.2826509476]], [[1.2277743816]], [[0.5329727530]], [[0.9079618454]], [[0.5331948400]], [[0.6104198098]], [[0.5943091512]], [[1.0565344095]], [[0.5877053142]], [[0.8747532964]], [[0.9066959620]], [[1.2079734802]], [[0.5077018738]], [[0.5424057841]], [[0.5953311324]], [[1.2356286049]], [[0.7059293985]], [[1.3074061871]], [[1.0664656162]], [[0.9489749074]], [[0.8819398880]], [[0.7868211269]], [[0.9483950734]], [[0.7714895606]], [[0.8821486831]], [[0.9494584203]], [[0.4945593476]], [[0.6235961914]], [[1.2894082069]], [[0.4798885584]], [[0.8497588038]], [[1.3802812099]], [[1.0662803650]], [[0.8998184204]], [[0.4523569345]], [[1.1333508492]], [[0.5295215249]], [[0.7437297702]], [[0.7179037333]], [[0.8116182089]], [[0.8911310434]], [[1.2980656624]], [[0.9480813146]], [[1.3335082531]], [[1.1915180683]], [[0.5695549250]], [[1.1099139452]], [[0.5322138667]], [[0.6737493277]], [[1.0327231884]], [[1.0507578850]], [[1.2607759237]], [[0.7247039676]], [[0.8864623308]], [[1.1430736780]], [[0.4021158218]], [[1.2471522093]], [[0.4722006917]], [[0.8925531507]], [[0.8006535172]], [[0.6223254204]], [[0.9517474174]], [[1.2991212606]], [[0.7395021319]], [[0.5888148546]], [[0.7007424831]], [[0.3900877237]], [[1.1113588810]], [[0.6455300450]], [[0.5519917607]], [[0.5976111889]], [[1.1731399298]], [[1.1504487991]], [[1.2694227695]], [[1.0670235157]], [[0.7186820507]], [[0.7459378242]], [[1.0333769321]], [[1.2966792583]], [[1.0216079950]], [[0.6490892172]], [[0.6506232023]], [[0.4129273891]], [[0.9936859012]], [[0.6050706506]], [[0.4398739934]], [[1.3241001368]], [[0.5609562993]], [[0.8287811279]], [[1.0289087296]], [[0.9015910625]], [[0.5492210388]], [[0.4815010428]], [[1.2842030525]], [[0.9670812488]], [[1.3004741669]], [[0.7180415988]], [[1.0329396725]]]]), compiled_model=tensor([[[[0.8819184899]], [[1.2905809879]], [[0.6117566228]], [[0.2748561502]], [[0.6144023538]], [[0.9872891903]], [[0.7578272820]], [[1.2063671350]], [[1.0140738487]], [[1.6125154495]], [[0.6422309875]], [[0.4687388539]], [[0.6594837308]], [[1.1170353889]], [[0.3021837473]], [[1.4577327967]], [[1.5587748289]], [[1.4526448250]], [[0.9559433460]], [[0.4654363394]], [[0.7262198329]], [[1.8695368767]], [[1.1443428993]], [[1.0461037159]], [[0.5686058402]], [[1.1867637634]], [[1.7802474499]], [[0.9739457369]], [[0.7481628656]], [[1.1660766602]], [[1.0003373623]], [[1.6088039875]], [[0.4255697131]], [[0.4076765776]], [[1.1362328529]], [[0.6355115771]], [[0.2586230636]], [[1.2449125051]], [[1.2727414370]], [[1.3262169361]], [[0.6524124146]], [[1.1804633141]], [[0.6234925985]], [[0.7781375647]], [[0.4177418351]], [[0.3882884979]], [[1.5366132259]], [[1.1733272076]], [[0.5794606209]], [[0.9295134544]], [[1.1411900520]], [[1.9625688791]], [[0.7275142074]], [[0.8961406946]], [[1.2745496035]], [[0.5665950775]], [[0.7864114046]], [[0.6786054373]], [[0.6635442972]], [[1.0593845844]], [[1.3904476166]], [[1.5434199572]], [[0.3506729007]], [[1.5681738853]], [[1.0652906895]], [[0.9305160046]], [[1.2031782866]], [[1.0125243664]], [[1.1775515079]], [[1.7073320150]], [[0.9749337435]], [[0.4915892482]], [[1.4845354557]], [[1.7454097271]], [[1.6429271698]], [[1.6019043922]], [[1.3679342270]], [[1.0388302803]], [[1.3856425285]], [[0.7031388283]], [[0.1926941872]], [[1.4320185184]], [[1.0928537846]], [[0.9421565533]], [[0.9451145530]], [[0.9518136382]], [[1.1292307377]], [[0.9450285435]], [[0.7822364569]], [[0.3810953498]], [[1.2687476873]], [[0.8856007457]], [[1.5525890589]], [[1.3194885254]], [[0.3104778528]], [[0.2801598907]], [[1.3262844086]], [[0.8254917860]], [[1.0898035765]], [[1.0349967480]], [[1.7903680801]], [[1.5708017349]], [[0.9769369960]], [[0.4988374710]], [[0.9997662902]], [[0.8888364434]], [[0.9131596684]], [[1.0734157562]], [[1.0686025620]], [[0.7127540112]], [[0.7715033293]], [[1.0055313110]], [[0.2886002660]], [[1.8109288216]], [[1.1538258791]], [[0.9800461531]], [[0.5848827958]], [[1.2297117710]], [[1.0392913818]], [[1.0629239082]], [[1.4850080013]], [[0.5772201419]], [[1.1739001274]], [[1.1340763569]], [[0.7461866140]], [[0.4069098234]], [[0.6140337586]], [[0.7448495030]], [[0.7054163814]], [[0.3869484663]], [[1.1642048359]], [[1.0522904396]], [[0.4690603018]], [[0.9172146320]], [[1.8058063984]], [[0.5386640429]], [[0.9306843281]], [[0.8786532879]], [[1.3446345329]], [[1.7300952673]], [[1.1596635580]], [[1.2248980999]], [[1.0093631744]], [[0.5940698981]], [[0.7383006215]], [[0.7085132003]], [[1.7106468678]], [[0.9491928220]], [[0.5231848359]], [[0.4519074559]], [[0.5850267410]], [[0.8982954025]], [[0.9395377040]], [[1.3882429600]], [[1.0947352648]], [[0.7077455521]], [[0.9744313955]], [[0.9193599820]], [[0.8727592826]], [[1.2172904015]], [[0.6491720080]], [[1.3919873238]], [[0.8689889908]], [[0.1644607186]], [[0.7733674645]], [[1.4431345463]], [[0.9426790476]], [[1.6042072773]], [[0.4451043010]], [[0.7527263165]], [[1.7713172436]], [[0.6460267901]], [[1.9049454927]], [[1.4730522633]], [[1.0465383530]], [[1.6934378147]], [[0.7938317657]], [[1.1985013485]], [[1.0119162798]], [[1.0852077007]], [[1.0523424149]], [[0.6674425602]], [[1.5030601025]], [[1.4079259634]], [[1.2637164593]], [[1.5319538116]], [[0.8442266583]], [[0.4034960866]], [[0.9104084969]], [[1.2754373550]], [[1.6775839329]], [[1.0951545238]], [[0.9739524126]], [[1.2100857496]], [[0.3570188284]], [[0.2599551082]], [[0.6919922233]], [[1.2640715837]], [[0.4004518986]], [[0.8563141823]], [[0.9132691622]], [[0.8861645460]], [[0.4125419259]], [[0.3938953280]], [[0.7332614660]], [[1.8460478783]], [[1.1237337589]], [[1.6750257015]], [[1.6144652367]], [[1.1031165123]], [[1.0992052555]], [[0.6234764457]], [[1.3412384987]], [[1.1457130909]], [[1.1194753647]], [[0.7569130063]], [[0.7360689640]], [[0.7995284200]], [[1.7021248341]], [[0.5470088720]], [[1.4125430584]], [[1.4078139067]], [[1.4750069380]], [[1.1136422157]], [[0.1784448624]], [[1.4374086857]], [[0.1957352161]], [[0.4323809147]], [[1.2142665386]], [[0.6173107624]], [[0.6784688234]], [[1.2818841934]], [[1.4743580818]], [[1.9413571358]], [[1.1064112186]], [[0.4062357545]], [[0.9223269820]], [[0.4107528925]], [[0.3109846115]], [[0.9190632105]], [[1.2791144848]], [[1.2253818512]], [[0.3572148085]], [[0.9373351336]], [[1.7143471241]], [[0.6262775660]], [[1.5294530392]], [[0.2023904920]], [[0.6212516427]], [[0.5845983624]], [[0.5906627774]], [[1.0082302094]], [[1.1991169453]], [[1.0248451233]], [[1.0223071575]], [[0.6893806458]], [[0.0441462398]], [[1.0224981308]], [[1.1920220852]], [[0.2591621280]], [[0.5325894356]], [[1.5007548332]], [[1.1491987705]], [[1.8549207449]], [[1.0292843580]], [[1.1905257702]], [[1.1300239563]], [[0.6929511428]], [[1.8924626112]], [[0.8223097920]], [[1.0685437918]], [[0.4495807886]], [[0.3425320387]], [[1.3117964268]], [[0.6296908855]], [[0.2178926468]], [[1.7841215134]], [[0.8222935200]], [[0.9522280693]], [[1.5562748909]], [[1.2103344202]], [[1.1194603443]], [[0.5959815979]], [[0.9722241163]], [[1.3725852966]], [[1.0164916515]], [[1.0131268501]], [[1.1399058104]]]])
test_add.test_module[Add1-[((1, 1280, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6227548122]], [[0.8947200179]], [[0.2149756551]], ..., [[0.8623308539]], [[0.9484047294]], [[0.9666029215]]]]), compiled_model=tensor([[[[0.6227548122]], [[1.7608976364]], [[0.6885750890]], ..., [[1.4462819099]], [[1.8873362541]], [[1.5632492304]]]])
test_add.test_module[Add1-[((1, 256, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.5006824136]], [[0.7726476192]], [[0.0929032564]], [[0.1364563107]], [[0.3118486404]], [[0.6385045052]], [[0.4945192337]], [[0.9008705616]], [[0.4600538015]], [[0.6367321014]], [[0.3533192873]], [[0.4061431289]], [[0.0267515779]], [[0.1732847691]], [[0.2983142734]], [[0.5229476094]], [[0.7020934224]], [[0.8044372201]], [[0.1654552817]], [[0.2866944075]], [[0.6860343814]], [[0.9196197987]], [[0.4015257359]], [[0.8785817027]], [[0.4238341451]], [[0.5573328733]], [[0.9571639299]], [[0.0405906439]], [[0.1896568537]], [[0.3778432012]], [[0.3095258474]], [[0.9364262223]], [[0.1803359985]], [[0.2742593884]], [[0.1551055908]], [[0.0361453295]], [[0.2125555873]], [[0.9342248440]], [[0.7275350094]], [[0.7467620969]], [[0.5307216048]], [[0.2480840683]], [[0.5890181661]], [[0.0375784636]], [[0.1431427002]], [[0.2466608286]], [[0.8198947906]], [[0.7975864410]], [[0.2826783061]], [[0.4863846302]], [[0.8242061734]], [[1.0014923811]], [[0.7028669119]], [[0.5719722509]], [[0.8396689892]], [[0.2100246549]], [[0.5975978374]], [[0.1167730689]], [[0.1578827500]], [[0.2461340427]], [[0.7306623459]], [[0.7055060267]], [[0.2082495689]], [[0.6554793715]], [[0.7789118290]], [[0.4413171411]], [[0.5235165954]], [[0.6202781796]], [[0.8146141171]], [[0.9845228791]], [[0.1191140413]], [[0.3211909533]], [[0.7009307742]], [[0.9187005162]], [[0.9395294785]], [[0.9456042051]], [[0.6039330959]], [[0.0696344972]], [[0.5504220724]], [[0.1916231513]], [[0.0384487510]], [[0.9486720562]], [[0.8846057057]], [[0.0056618452]], [[0.5980118513]], [[0.4201958179]], [[0.4221452475]], [[0.2755473852]], [[0.6967039108]], [[0.2082740664]], [[0.6877214909]], [[0.7572798729]], [[0.8623616099]], [[0.6913813949]], [[0.0095582008]], [[0.1800773740]], [[0.7540833354]], [[0.6090764999]], [[0.1143838167]], [[0.2165160775]], [[0.9748004675]], [[0.8413347602]], [[0.2864132524]], [[0.3785834312]], [[0.0281267762]], [[0.4954387546]], [[0.1278963685]], [[0.1187474728]], [[0.4768760204]], [[0.5794983506]], [[0.2996606827]], [[0.8011146188]], [[0.2001562715]], [[0.9581108689]], [[0.8470757604]], [[0.0827843547]], [[0.3799836636]], [[0.5269871354]], [[0.5773763657]], [[0.6230129600]], [[0.7006399632]], [[0.5343759060]], [[0.2604614496]], [[0.7410203218]], [[0.0248013735]], [[0.2080724835]], [[0.3792608976]], [[0.2608691454]], [[0.3295091391]], [[0.0946149826]], [[0.3980682492]], [[0.6113040447]], [[0.1786929369]], [[0.4787661433]], [[0.8623512387]], [[0.4530256987]], [[0.5183219314]], [[0.4612913728]], [[0.6056165099]], [[0.8223455548]], [[0.9780489206]], [[0.8219537735]], [[0.9791325927]], [[0.4682649970]], [[0.0552650690]], [[0.2673872113]], [[0.8448784351]], [[0.5011845827]], [[0.2559026480]], [[0.1212699413]], [[0.0364997983]], [[0.0824217200]], [[0.4030074477]], [[0.7786288261]], [[0.7747463584]], [[0.0222098827]], [[0.8163168430]], [[0.1131711006]], [[0.3987206817]], [[0.3016895056]], [[0.4081181884]], [[0.4062544703]], [[0.0557508469]], [[0.0727068782]], [[0.4261860847]], [[0.5108919144]], [[0.2772883773]], [[0.6927754283]], [[0.0543966293]], [[0.4706822038]], [[0.9441355467]], [[0.3004798293]], [[0.9559273720]], [[0.6855027080]], [[0.0531953573]], [[0.8207744956]], [[0.4467285872]], [[0.2812224030]], [[0.9042524695]], [[0.1003782153]], [[0.5580782890]], [[0.3997414708]], [[0.8614821434]], [[0.6439980865]], [[0.7446784973]], [[0.6810052991]], [[0.3841884732]], [[0.3992730975]], [[0.0923851132]], [[0.7753478885]], [[0.9014148712]], [[0.8465382457]], [[0.1517366767]], [[0.5267257690]], [[0.1519587636]], [[0.2291837335]], [[0.2130730748]], [[0.6752983332]], [[0.2064692378]], [[0.4935172200]], [[0.5254598856]], [[0.8267373443]], [[0.1264657974]], [[0.1611697078]], [[0.2140950561]], [[0.8543925285]], [[0.3246933222]], [[0.9261701703]], [[0.6852295995]], [[0.5677388310]], [[0.5007038116]], [[0.4055850506]], [[0.5671589971]], [[0.3902534842]], [[0.5009126067]], [[0.5682223439]], [[0.1133232713]], [[0.2423601151]], [[0.9081721902]], [[0.0986524820]], [[0.4685227275]], [[0.9990451932]], [[0.6850443482]], [[0.5185823441]], [[0.0711208582]], [[0.7521147728]], [[0.1482854486]], [[0.3624936938]], [[0.3366676569]], [[0.4303821325]], [[0.5098949671]], [[0.9168295860]], [[0.5668452382]], [[0.9522722363]], [[0.8102820516]], [[0.1883188486]], [[0.7286778688]], [[0.1509777904]], [[0.2925132513]], [[0.6514871716]], [[0.6695218682]], [[0.8795398474]], [[0.3434678912]], [[0.5052262545]], [[0.7618376017]], [[0.0208797455]], [[0.8659161329]], [[0.0909646153]], [[0.5113170743]], [[0.4194174409]], [[0.2410893440]], [[0.5705113411]], [[0.9178851843]], [[0.3582660556]], [[0.2075787783]], [[0.3195064068]]]]), compiled_model=tensor([[[[0.5006824136]], [[1.0833477974]], [[0.1110562086]], [[0.7675865293]], [[0.5814625621]], [[1.3749606609]], [[0.9302139878]], [[1.7932457924]], [[1.3674836159]], [[1.3608345985]], [[0.7057628632]], [[0.7968124151]], [[0.0291234851]], [[0.6526416540]], [[1.0567274094]], [[0.7176864147]], [[1.5162909031]], [[0.8479704857]], [[1.0810496807]], [[0.2880194783]], [[0.8246139288]], [[1.6699842215]], [[0.4614775181]], [[1.3097944260]], [[0.8109716773]], [[1.1752138138]], [[1.9228601456]], [[1.0305089951]], [[0.9995988011]], [[1.1786675453]], [[1.1050875187]], [[1.5775334835]], [[0.4781830907]], [[1.2506749630]], [[1.0430227518]], [[0.3705975413]], [[0.2833987474]], [[1.5364664793]], [[1.4344650507]], [[1.0363306999]], [[0.5581429601]], [[1.1543004513]], [[0.6859285831]], [[0.6291643977]], [[1.1190173626]], [[0.7156679630]], [[0.9704498649]], [[1.6686431170]], [[0.3301787972]], [[1.4490891695]], [[1.2032957077]], [[1.7087013721]], [[1.6378870010]], [[0.8311963677]], [[0.9579418898]], [[1.0897927284]], [[0.7372111678]], [[0.5308936238]], [[0.8487515450]], [[1.1007540226]], [[1.1074311733]], [[1.3518707752]], [[0.6930467486]], [[0.7697823048]], [[1.5999943018]], [[0.6748240590]], [[1.4036210775]], [[0.7495485544]], [[0.8659349084]], [[1.5589847565]], [[0.7656476498]], [[0.5788133144]], [[0.7978674769]], [[1.6396186352]], [[1.9307172298]], [[1.5490343571]], [[0.6193802953]], [[0.2392237782]], [[1.4746016264]], [[0.3025166392]], [[0.5460868478]], [[1.5064201355]], [[1.0947132111]], [[0.2485121489]], [[0.8000474572]], [[0.8700510859]], [[1.1739122868]], [[0.3325448632]], [[1.2653946877]], [[1.1192004681]], [[1.0589790344]], [[1.4133529663]], [[1.4217550755]], [[1.6082479954]], [[0.4277974963]], [[0.7034077644]], [[1.5208597183]], [[1.0721201897]], [[0.1941823959]], [[0.8429183364]], [[1.7675752640]], [[1.3874099255]], [[1.1294555664]], [[0.8937734962]], [[0.3886032104]], [[1.0207626820]], [[0.1447409987]], [[0.1624627113]], [[1.4292457104]], [[1.1861082315]], [[1.1628693342]], [[1.4072325230]], [[1.1781387329]], [[1.2112262249]], [[1.2314789295]], [[0.7690786719]], [[1.1927492619]], [[1.2691438198]], [[1.3597199917]], [[0.6425309777]], [[1.3034188747]], [[0.6378029585]], [[0.9813074470]], [[1.1829757690]], [[0.9426329136]], [[0.3475562334]], [[1.2770906687]], [[0.5963469148]], [[0.7398305535]], [[0.1748156548]], [[0.7787978649]], [[0.7381939292]], [[1.0013251305]], [[0.7252888680]], [[1.1447434425]], [[1.3771451712]], [[1.4588483572]], [[0.5633358955]], [[1.3754683733]], [[1.0138785839]], [[1.3044414520]], [[1.7729154825]], [[1.5133036375]], [[0.5622811317]], [[0.4008420706]], [[1.2544045448]], [[1.1566540003]], [[1.2808033228]], [[0.7197513580]], [[0.3123416305]], [[0.1334270239]], [[0.1668441892]], [[1.3709633350]], [[1.5535278320]], [[1.1276317835]], [[0.6145148873]], [[0.8178412318]], [[0.4642982483]], [[0.8554183245]], [[1.0430237055]], [[0.4856245518]], [[1.2424443960]], [[0.9182355404]], [[0.2088889480]], [[0.6493129730]], [[0.7188361287]], [[0.4653552771]], [[1.0898897648]], [[0.5885711908]], [[1.0847048759]], [[1.4840241671]], [[1.1220130920]], [[1.5738663673]], [[1.4431281090]], [[0.9975739717]], [[1.1009795666]], [[0.6033211946]], [[0.3333778977]], [[0.9069392681]], [[0.2684272528]], [[1.2295963764]], [[0.7039475441]], [[1.5562336445]], [[0.8000905514]], [[1.2128707170]], [[0.6899572611]], [[0.7915813923]], [[0.9015500546]], [[0.9757099748]], [[1.7020306587]], [[1.1031522751]], [[1.4291446209]], [[0.8792157173]], [[0.9529598951]], [[0.6342623234]], [[1.0525233746]], [[0.3331921101]], [[0.7084751725]], [[0.9348928332]], [[1.1062912941]], [[0.6541148424]], [[1.4603705406]], [[1.0427803993]], [[1.0933626890]], [[0.6376793385]], [[1.1437503099]], [[1.1743478775]], [[1.0291879177]], [[1.1689777374]], [[1.3791936636]], [[1.1073300838]], [[0.8540123105]], [[0.9149041176]], [[0.6280267239]], [[1.3783056736]], [[0.5695406795]], [[0.7668276429]], [[0.7210475802]], [[1.0442273617]], [[1.0575194359]], [[1.0337744951]], [[1.4431829453]], [[1.0379821062]], [[1.3695058823]], [[0.5026146173]], [[1.0429258347]], [[0.5018640161]], [[1.0990166664]], [[0.6048610806]], [[0.5165776014]], [[0.9275364280]], [[1.1373569965]], [[1.1138606071]], [[1.7617065907]], [[1.3167059422]], [[0.7532671690]], [[1.0603981018]], [[0.7477260828]], [[0.9575616717]], [[1.1988310814]], [[1.6289732456]], [[1.7912716866]], [[1.3225145340]], [[1.0912194252]], [[1.2269043922]], [[0.5664322376]], [[1.7082040310]], [[0.3873854280]], [[1.3478140831]], [[0.5135604143]], [[0.9858227372]], [[0.9751204848]], [[1.5082988739]], [[0.7858108878]], [[0.2723414302]], [[0.9851081371]]]])
test_add.test_module[Add1-[((1, 512, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.8575292826]], [[1.1294944286]], [[0.4497501254]], [[0.4933031797]], [[0.6686955094]], [[0.9953513741]], [[0.8513661027]], [[1.2577173710]], [[0.8169006705]], [[0.9935789704]], [[0.7101661563]], [[0.7629899979]], [[0.3835984468]], [[0.5301316381]], [[0.6551611423]], [[0.8797944784]], [[1.0589402914]], [[1.1612840891]], [[0.5223021507]], [[0.6435412765]], [[1.0428812504]], [[1.2764666080]], [[0.7583726048]], [[1.2354285717]], [[0.7806810141]], [[0.9141797423]], [[1.3140108585]], [[0.3974375129]], [[0.5465037227]], [[0.7346900702]], [[0.6663727164]], [[1.2932730913]], [[0.5371828675]], [[0.6311062574]], [[0.5119524598]], [[0.3929921985]], [[0.5694024563]], [[1.2910716534]], [[1.0843818188]], [[1.1036089659]], [[0.8875684738]], [[0.6049309373]], [[0.9458650351]], [[0.3944253325]], [[0.4999895692]], [[0.6035076976]], [[1.1767416000]], [[1.1544332504]], [[0.6395251751]], [[0.8432314992]], [[1.1810530424]], [[1.3583393097]], [[1.0597138405]], [[0.9288191199]], [[1.1965157986]], [[0.5668715239]], [[0.9544447064]], [[0.4736199379]], [[0.5147296190]], [[0.6029809117]], [[1.0875091553]], [[1.0623528957]], [[0.5650964379]], [[1.0123262405]], [[1.1357586384]], [[0.7981640100]], [[0.8803634644]], [[0.9771250486]], [[1.1714609861]], [[1.3413697481]], [[0.4759609103]], [[0.6780378222]], [[1.0577776432]], [[1.2755473852]], [[1.2963763475]], [[1.3024511337]], [[0.9607799649]], [[0.4264813662]], [[0.9072689414]], [[0.5484700203]], [[0.3952956200]], [[1.3055188656]], [[1.2414525747]], [[0.3625087142]], [[0.9548587203]], [[0.7770426869]], [[0.7789921165]], [[0.6323942542]], [[1.0535507202]], [[0.5651209354]], [[1.0445683002]], [[1.1141266823]], [[1.2192084789]], [[1.0482282639]], [[0.3664050698]], [[0.5369242430]], [[1.1109302044]], [[0.9659233689]], [[0.4712306857]], [[0.5733629465]], [[1.3316473961]], [[1.1981816292]], [[0.6432601213]], [[0.7354303002]], [[0.3849736452]], [[0.8522856236]], [[0.4847432375]], [[0.4755943418]], [[0.8337228894]], [[0.9363452196]], [[0.6565075517]], [[1.1579614878]], [[0.5570031404]], [[1.3149577379]], [[1.2039226294]], [[0.4396312237]], [[0.7368305326]], [[0.8838340044]], [[0.9342232347]], [[0.9798598289]], [[1.0574867725]], [[0.8912227750]], [[0.6173083186]], [[1.0978672504]], [[0.3816482425]], [[0.5649193525]], [[0.7361077666]], [[0.6177160144]], [[0.6863560081]], [[0.4514618516]], [[0.7549151182]], [[0.9681509137]], [[0.5355398059]], [[0.8356130123]], [[1.2191981077]], [[0.8098725677]], [[0.8751688004]], [[0.8181382418]], [[0.9624633789]], [[1.1791924238]], [[1.3348958492]], [[1.1788005829]], [[1.3359794617]], [[0.8251118660]], [[0.4121119380]], [[0.6242340803]], [[1.2017252445]], [[0.8580314517]], [[0.6127495170]], [[0.4781168103]], [[0.3933466673]], [[0.4392685890]], [[0.7598543167]], [[1.1354756355]], [[1.1315932274]], [[0.3790567517]], [[1.1731636524]], [[0.4700179696]], [[0.7555675507]], [[0.6585363746]], [[0.7649650574]], [[0.7631013393]], [[0.4125977159]], [[0.4295537472]], [[0.7830329537]], [[0.8677387834]], [[0.6341352463]], [[1.0496222973]], [[0.4112434983]], [[0.8275290728]], [[1.3009824753]], [[0.6573266983]], [[1.3127741814]], [[1.0423495770]], [[0.4100422263]], [[1.1776213646]], [[0.8035754561]], [[0.6380692720]], [[1.2610993385]], [[0.4572250843]], [[0.9149251580]], [[0.7565883398]], [[1.2183289528]], [[1.0008449554]], [[1.1015253067]], [[1.0378521681]], [[0.7410353422]], [[0.7561199665]], [[0.4492319822]], [[1.1321947575]], [[1.2582616806]], [[1.2033851147]], [[0.5085835457]], [[0.8835726380]], [[0.5088056326]], [[0.5860306025]], [[0.5699199438]], [[1.0321452618]], [[0.5633161068]], [[0.8503640890]], [[0.8823067546]], [[1.1835842133]], [[0.4833126664]], [[0.5180165768]], [[0.5709419250]], [[1.2112393379]], [[0.6815401912]], [[1.2830170393]], [[1.0420764685]], [[0.9245857000]], [[0.8575506806]], [[0.7624319196]], [[0.9240058661]], [[0.7471003532]], [[0.8577594757]], [[0.9250692129]], [[0.4701701403]], [[0.5992069840]], [[1.2650190592]], [[0.4554993510]], [[0.8253695965]], [[1.3558920622]], [[1.0418912172]], [[0.8754292130]], [[0.4279677272]], [[1.1089615822]], [[0.5051323175]], [[0.7193405628]], [[0.6935145259]], [[0.7872290015]], [[0.8667418361]], [[1.2736763954]], [[0.9236921072]], [[1.3091191053]], [[1.1671289206]], [[0.5451657176]], [[1.0855247974]], [[0.5078246593]], [[0.6493601203]], [[1.0083340406]], [[1.0263687372]], [[1.2363867760]], [[0.7003147602]], [[0.8620731235]], [[1.1186845303]], [[0.3777266145]], [[1.2227630615]], [[0.4478114843]], [[0.8681639433]], [[0.7762643099]], [[0.5979362130]], [[0.9273582101]], [[1.2747321129]], [[0.7151129246]], [[0.5644256473]], [[0.6763532758]], [[0.3656985164]], [[1.0869696140]], [[0.6211408377]], [[0.5276025534]], [[0.5732219815]], [[1.1487507820]], [[1.1260595322]], [[1.2450336218]], [[1.0426342487]], [[0.6942928433]], [[0.7215486169]], [[1.0089876652]], [[1.2722899914]], [[0.9972187877]], [[0.6247000098]], [[0.6262339950]], [[0.3885381818]], [[0.9692966938]], [[0.5806814432]], [[0.4154847860]], [[1.2997109890]], [[0.5365670919]], [[0.8043919206]], [[1.0045194626]], [[0.8772018552]], [[0.5248318315]], [[0.4571118355]], [[1.2598139048]], [[0.9426920414]], [[1.2760848999]], [[0.6936523914]], [[1.0085504055]], [[0.7469345927]], [[0.8390371203]], [[0.5567505956]], [[1.0303730965]], [[1.0193556547]], [[0.8509584069]], [[0.7488216162]], [[0.5530577898]], [[1.2070450783]], [[0.4890606403]], [[1.0661053658]], [[0.6931459308]], [[0.6200386882]], [[0.9510952234]], [[0.6015442014]], [[0.9764751196]], [[0.9594631195]], [[0.4900279641]], [[0.9445219636]], [[1.0742373466]], [[1.0591881275]], [[0.7983333468]], [[0.4513788819]], [[0.7841956615]], [[1.0349265337]], [[0.6788341403]], [[1.0510632992]], [[1.1942539215]], [[0.6001968980]], [[0.8662036061]], [[1.0680222511]], [[0.9004616141]], [[0.9030384421]], [[0.9237053990]], [[0.4681988358]], [[0.9005668163]], [[1.2075076103]], [[1.3118295670]], [[1.1551481485]], [[0.9282742143]], [[1.0947797298]], [[0.6180328727]], [[0.4469285011]], [[0.4313121438]], [[1.3600776196]], [[1.1786673069]], [[0.5156558156]], [[1.0568940639]], [[1.2388565540]], [[1.3610801697]], [[1.2984701395]], [[1.2486495972]], [[0.7466430664]], [[0.6858018637]], [[1.2717971802]], [[1.1414625645]], [[0.5603832006]], [[1.3107812405]], [[1.1028519869]], [[1.1338404417]], [[0.5478845239]], [[1.0047225952]], [[0.6859914660]], [[1.2519509792]], [[0.7713024020]], [[1.0559289455]], [[0.9500836134]], [[1.0740067959]], [[0.6913589239]], [[1.1050584316]], [[0.5120372176]], [[0.9742001891]], [[0.5229767561]], [[0.3680039048]], [[0.4597448111]], [[1.2560076714]], [[1.1317905188]], [[1.3303805590]], [[1.2618333101]], [[0.4147499800]], [[0.5200571418]], [[0.7804813981]], [[0.5365565419]], [[1.2084709406]], [[0.4832687974]], [[0.6173219681]], [[0.3782269955]], [[0.5773999095]], [[1.2725114822]], [[1.2706527710]], [[1.2191915512]], [[1.2473363876]], [[1.3058660030]], [[0.7332412004]], [[1.0812737942]], [[1.3067378998]], [[1.0266821384]], [[1.3611109257]], [[1.1206041574]], [[1.1720968485]], [[0.6862766147]], [[1.1011865139]], [[0.9187259674]], [[0.7418640256]], [[0.5794177651]], [[0.5807144046]], [[0.4765322804]], [[1.1969377995]], [[1.2167382240]], [[0.8043663502]], [[0.5719296932]], [[1.2477254868]], [[1.1810171604]], [[0.8984397054]], [[0.6252050400]], [[1.3207966089]], [[1.0657429695]], [[0.4817000031]], [[1.3398141861]], [[1.2409596443]], [[0.6790272593]], [[1.1423499584]], [[0.5771781802]], [[0.7829155326]], [[1.2858233452]], [[0.8819322586]], [[0.5076636076]], [[0.6941562295]], [[0.7255501151]], [[0.7648082972]], [[0.9091230035]], [[1.3236875534]], [[0.8880428672]], [[0.5525527000]], [[0.8868998289]], [[1.1010162830]], [[1.1092927456]], [[0.4043034315]], [[0.7718014717]], [[0.4897004366]], [[0.6479299068]], [[1.0414193869]], [[0.5062075853]], [[1.0471360683]], [[1.2856616974]], [[0.8940668702]], [[0.5280288458]], [[0.6821302176]], [[0.9704547524]], [[0.4801136851]], [[1.1096777916]], [[0.4073379040]], [[0.3806260228]], [[0.3754423857]], [[0.7598410249]], [[1.1974891424]], [[0.3880332112]], [[1.2768661976]], [[0.6612612605]], [[1.0077147484]], [[0.8840741515]], [[0.4104131460]], [[1.2759371996]], [[1.1304943562]], [[1.3582705259]], [[1.1138787270]], [[0.5312381983]], [[1.2785645723]], [[0.8881449103]], [[1.0983809233]], [[0.4603581429]], [[0.7174594402]], [[0.3703339100]], [[0.6665264368]], [[0.9691382647]], [[0.4686918259]], [[1.0206547976]], [[1.1296761036]], [[0.9309273362]], [[0.5267310143]], [[0.4736128449]], [[0.7070143819]], [[1.0807517767]], [[1.3544709682]], [[1.1487872601]], [[0.8049680591]], [[1.0365809202]], [[0.3707412481]], [[0.4342220426]], [[1.0945765972]], [[0.5780650973]], [[1.1018204689]], [[0.5083066225]], [[0.6136172414]], [[0.4494282007]], [[1.1221930981]], [[0.8103251457]], [[1.2460732460]], [[1.1707087755]], [[1.1379439831]], [[0.8773507476]], [[0.7066836953]], [[0.7525568604]], [[0.9277229905]], [[1.1091275215]], [[0.5109777451]], [[1.2809116840]], [[0.8069076538]], [[0.4422999024]], [[0.5907437801]], [[1.3036816120]], [[1.3185362816]], [[0.3981328607]], [[1.2139217854]], [[1.1118522882]], [[1.1572319269]], [[1.2845373154]], [[0.5917975307]], [[1.0191605091]], [[1.0658893585]], [[0.7135260105]], [[1.0285992622]], [[0.7174159884]], [[1.1704033613]]]]), compiled_model=tensor([[[[0.8575292826]], [[1.2001924515]], [[0.1576659083]], [[0.8020580411]], [[0.4637765884]], [[0.9230792522]], [[0.9659639597]], [[1.1811567545]], [[1.0998113155]], [[1.6225578785]], [[1.0790004730]], [[0.7035893798]], [[0.1202192903]], [[0.4158164263]], [[0.6009347439]], [[1.1998798847]], [[1.4442123175]], [[1.4048823118]], [[0.6396138668]], [[0.7370626926]], [[0.9748380184]], [[1.3475704193]], [[0.6025542617]], [[1.6757814884]], [[0.9200907350]], [[1.6362547874]], [[1.0637943745]], [[0.8037513494]], [[0.7666935921]], [[1.7483780384]], [[1.2353140116]], [[2.7252461910]], [[1.5433938503]], [[1.6306681633]], [[0.8564426303]], [[0.8285319209]], [[0.2372532487]], [[1.5824406147]], [[1.7798366547]], [[1.4600226879]], [[2.0425868034]], [[1.0916287899]], [[1.6656420231]], [[0.3211721182]], [[0.9633308053]], [[1.9122192860]], [[1.2769465446]], [[2.1029551029]], [[1.0892241001]], [[1.6571726799]], [[2.7426404953]], [[2.0275754929]], [[1.6980398893]], [[1.7462139130]], [[1.9403307438]], [[1.7831323147]], [[1.0713551044]], [[1.3630222082]], [[1.1964796782]], [[0.6123057604]], [[1.0096352100]], [[2.2375466824]], [[1.6382887363]], [[1.6873842478]], [[1.3326289654]], [[1.5911917686]], [[1.2050193548]], [[1.2450168133]], [[1.9292056561]], [[1.6957650185]], [[1.0851380825]], [[1.9854083061]], [[1.0266838074]], [[2.3633637428]], [[2.1383993626]], [[2.6498796940]], [[2.2373943329]], [[0.8964050412]], [[1.5039381981]], [[1.2769900560]], [[0.7712340951]], [[1.4751398563]], [[1.7289314270]], [[1.1019899845]], [[1.7010171413]], [[1.7676408291]], [[1.1107661724]], [[1.0409038067]], [[2.2922723293]], [[0.8786723018]], [[2.0869166851]], [[1.5024025440]], [[1.7238707542]], [[2.2459402084]], [[0.7707800269]], [[0.7544648647]], [[1.5475249290]], [[2.2442693710]], [[2.0406751633]], [[1.7611246109]], [[1.5897549391]], [[1.0761327744]], [[1.7565890551]], [[0.6766742468]], [[0.5697878003]], [[1.9974330664]], [[1.2181837559]], [[0.3628337979]], [[1.2724976540]], [[1.4451236725]], [[1.4691472054]], [[1.1292335987]], [[1.4611251354]], [[2.0728855133]], [[1.9016289711]], [[1.4917114973]], [[1.7973129749]], [[2.1308093071]], [[1.0007480383]], [[1.3219949007]], [[2.2170739174]], [[1.6020703316]], [[0.4502180219]], [[1.5795128345]], [[1.7879507542]], [[1.5910565853]], [[1.5042905807]], [[1.1502168179]], [[0.7136865258]], [[1.1109519005]], [[0.5383834243]], [[0.7693409324]], [[1.6035127640]], [[1.6604485512]], [[2.0207948685]], [[1.8558323383]], [[1.6920348406]], [[1.6680917740]], [[1.8326696157]], [[1.5869984627]], [[2.1663722992]], [[2.0866718292]], [[2.3344268799]], [[1.1063702106]], [[1.3542580605]], [[0.9007643461]], [[1.8217600584]], [[1.6797344685]], [[1.1941096783]], [[0.4644003510]], [[1.3091646433]], [[0.6743428111]], [[1.1384122372]], [[0.9490186572]], [[1.5491183996]], [[0.7559779882]], [[1.8132162094]], [[0.8340341449]], [[1.5390383005]], [[1.6744089127]], [[1.8625407219]], [[0.9651645422]], [[1.4267933369]], [[1.0821596384]], [[1.7262017727]], [[2.2793815136]], [[1.7861661911]], [[1.2506307364]], [[0.4508128762]], [[1.7206609249]], [[2.0963637829]], [[1.5768573284]], [[1.6712529659]], [[0.9934185147]], [[0.1821965575]], [[0.9831928611]], [[1.8132660389]], [[1.8303244114]], [[2.0274584293]], [[0.7104672790]], [[1.3714936972]], [[0.8596138954]], [[1.7124745846]], [[1.6825959682]], [[1.2258772850]], [[1.9190238714]], [[1.2979981899]], [[0.6037362218]], [[0.7372722626]], [[1.4897582531]], [[1.3623442650]], [[1.9320021868]], [[0.7358820438]], [[1.6070048809]], [[1.6315571070]], [[1.3467710018]], [[1.7825136185]], [[2.1140005589]], [[1.1996173859]], [[1.5900709629]], [[1.1243553162]], [[1.1556894779]], [[1.0289793015]], [[0.4251711369]], [[1.4392656088]], [[1.5539143085]], [[1.8765010834]], [[1.7218348980]], [[1.8936744928]], [[1.2532702684]], [[1.2878594398]], [[1.3027093410]], [[1.5384430885]], [[2.0878582001]], [[1.5996390581]], [[1.9929411411]], [[0.9881131649]], [[1.1908941269]], [[1.5380086899]], [[1.1467499733]], [[0.7972890139]], [[1.7030944824]], [[1.6155114174]], [[1.6204478741]], [[0.7208098769]], [[2.2080595493]], [[1.1866400242]], [[1.4514305592]], [[0.9699211717]], [[1.5697066784]], [[1.6798169613]], [[1.9415917397]], [[1.7313971519]], [[2.3270401955]], [[1.9131863117]], [[1.0379053354]], [[1.6391561031]], [[0.7745786905]], [[1.6663930416]], [[1.2166020870]], [[1.4319236279]], [[1.5961616039]], [[1.3832694292]], [[1.5583198071]], [[1.7911863327]], [[1.4596369267]], [[1.8994724751]], [[1.4560446739]], [[1.0095058680]], [[1.4579174519]], [[0.7385275364]], [[1.6651022434]], [[1.5183204412]], [[0.8704178333]], [[1.1306893826]], [[1.4524376392]], [[1.1182863712]], [[2.4874036312]], [[1.5765740871]], [[0.9195970297]], [[1.2723474503]], [[1.5352041721]], [[1.7223484516]], [[2.0825920105]], [[2.3103346825]], [[2.1242918968]], [[1.6827905178]], [[1.7389343977]], [[2.1379218102]], [[1.2023782730]], [[1.9716312885]], [[0.6523467302]], [[1.3750796318]], [[1.1215844154]], [[1.2052314281]], [[1.0293326378]], [[2.4467370510]], [[0.9611052871]], [[0.7154606581]], [[1.6283549070]], [[1.2166228294]], [[0.7671896815]], [[0.7799763083]], [[1.7803325653]], [[0.6429398656]], [[1.2454398870]], [[1.1296789646]], [[1.3791359663]], [[0.7692775130]], [[0.6952179670]], [[0.2589014173]], [[1.1585400105]], [[1.3116261959]], [[0.6901603341]], [[1.0277174711]], [[0.2491824627]], [[1.7297222614]], [[1.0723693371]], [[1.2788963318]], [[0.5271777511]], [[1.0689792633]], [[1.3448563814]], [[0.7402169704]], [[1.3474802971]], [[0.6096336246]], [[0.6339100003]], [[1.2281837463]], [[0.7491844893]], [[1.3977982998]], [[0.4481288791]], [[0.1411890984]], [[1.3362350464]], [[1.0955452919]], [[0.4765564203]], [[0.8053911328]], [[1.2084470987]], [[1.1421155930]], [[1.4806761742]], [[1.3951719999]], [[1.5139091015]], [[1.5270276070]], [[1.1961309910]], [[0.8209366798]], [[1.0057734251]], [[1.7108345032]], [[1.1902736425]], [[1.3432796001]], [[0.5701828599]], [[1.5378180742]], [[0.4359030724]], [[0.6303409934]], [[0.6148982048]], [[1.8985724449]], [[1.3364098072]], [[0.7588420510]], [[1.6761090755]], [[1.1060397625]], [[1.1356997490]], [[0.9787989855]], [[1.7734458447]], [[0.8209749460]], [[0.9709702134]], [[1.3724482059]], [[1.2376850843]], [[0.7510981560]], [[1.5959185362]], [[1.3023754358]], [[1.5547666550]], [[0.6445395947]], [[0.8528273702]], [[0.6511801481]], [[1.1111459732]], [[0.7650070190]], [[1.3583387136]], [[1.4053772688]], [[1.6183717251]], [[0.4462670088]], [[1.3500680923]], [[0.5422837138]], [[0.9391174912]], [[0.3913900852]], [[0.2065189481]], [[0.1858202815]], [[1.4509575367]], [[1.1555013657]], [[1.3981623650]], [[1.0965645313]], [[1.0151765347]], [[0.9288395047]], [[0.8125447631]], [[0.7315537333]], [[1.3855805397]], [[0.2485027313]], [[0.6885975003]], [[0.2137898207]], [[1.0915374756]], [[1.2278151512]], [[1.5854835510]], [[1.3820955753]], [[1.5365397930]], [[1.7579220533]], [[0.6860464215]], [[1.4526795149]], [[1.9742578268]], [[0.6925668120]], [[1.6017990112]], [[1.6256244183]], [[1.6716916561]], [[0.6166850328]], [[0.9098251462]], [[0.8156998754]], [[0.7174608111]], [[0.6117798090]], [[0.6547924280]], [[0.5114901662]], [[1.2242264748]], [[1.1405919790]], [[1.1411576271]], [[0.9965093136]], [[1.5876427889]], [[1.6618618965]], [[1.2432291508]], [[0.3718621135]], [[1.0525003672]], [[1.6241483688]], [[0.2873747349]], [[1.3563001156]], [[0.9926975369]], [[1.1459527016]], [[1.1103409529]], [[0.6716848612]], [[0.7467820644]], [[1.8900060654]], [[1.1136982441]], [[1.0430732965]], [[0.7741590142]], [[0.5460400581]], [[1.2806985378]], [[0.8603605628]], [[1.4355990887]], [[0.9726317525]], [[1.1066961288]], [[0.8686225414]], [[0.9767237306]], [[1.5341241360]], [[0.0504578948]], [[0.9922752976]], [[0.4055736065]], [[0.7894771099]], [[1.3152654171]], [[0.6909883618]], [[0.7475339770]], [[1.5039846897]], [[0.5601560473]], [[0.9777237773]], [[0.9258729219]], [[1.2704555988]], [[0.8207780719]], [[1.2260283232]], [[0.3255383372]], [[0.9669243097]], [[0.6988929510]], [[1.0978255272]], [[1.2834125757]], [[0.5981218219]], [[1.2173681259]], [[0.8165432215]], [[0.6857398152]], [[0.8697797060]], [[0.6422622204]], [[1.7603673935]], [[1.6309589148]], [[1.6785702705]], [[1.1189146042]], [[0.7442741990]], [[1.7579883337]], [[0.8312751651]], [[0.9020650387]], [[0.8011379242]], [[0.5256346464]], [[0.7265797257]], [[0.5863713622]], [[0.9959354997]], [[0.8732218742]], [[0.7662373185]], [[1.7563755512]], [[1.1258082390]], [[0.8768648505]], [[0.3532485366]], [[1.1574939489]], [[1.0183255672]], [[1.2149668932]], [[1.4583182335]], [[0.8874040246]], [[1.6400185823]], [[0.8228977919]], [[0.2676283121]], [[1.5155203342]], [[0.3331905603]], [[1.0227701664]], [[0.2706133127]], [[0.7630570531]], [[0.9096294641]], [[1.7061514854]], [[0.5169041157]], [[1.3513360023]], [[1.6300863028]], [[1.3423221111]], [[0.5803250074]], [[1.2377030849]], [[1.0897190571]], [[1.2744610310]], [[1.6742122173]], [[0.4750971794]], [[1.4729528427]], [[1.2368690968]], [[1.0011106730]], [[1.0953425169]], [[1.5687527657]], [[1.0690221786]], [[0.9843560457]], [[1.2124843597]], [[0.7603266835]], [[1.5375943184]], [[1.4566502571]], [[0.9513447285]], [[1.0859515667]], [[1.1685860157]], [[1.2674055099]], [[1.1347281933]], [[1.3529055119]], [[1.2791579962]]]])
test_add.test_module[Add1-[((1, 768, 1, 1), torch.float32)]]: forge/test/models_ops/test_add.py#L18561
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[0.6126392484]], [[0.8846044540]], [[0.2048600912]], [[0.2484131455]], [[0.4238054752]], [[0.7504613400]], [[0.6064760685]], [[1.0128273964]], [[0.5720106363]], [[0.7486889362]], [[0.4652761221]], [[0.5180999637]], [[0.1387084126]], [[0.2852416039]], [[0.4102711082]], [[0.6349044442]], [[0.8140502572]], [[0.9163940549]], [[0.2774121165]], [[0.3986512423]], [[0.7979912162]], [[1.0315766335]], [[0.5134825706]], [[0.9905385375]], [[0.5357909799]], [[0.6692897081]], [[1.0691207647]], [[0.1525474787]], [[0.3016136885]], [[0.4898000360]], [[0.4214826822]], [[1.0483829975]], [[0.2922928333]], [[0.3862162232]], [[0.2670624256]], [[0.1481021643]], [[0.3245124221]], [[1.0461816788]], [[0.8394918442]], [[0.8587189317]], [[0.6426784396]], [[0.3600409031]], [[0.7009750009]], [[0.1495352983]], [[0.2550995350]], [[0.3586176634]], [[0.9318516254]], [[0.9095432758]], [[0.3946351409]], [[0.5983414650]], [[0.9361630082]], [[1.1134492159]], [[0.8148237467]], [[0.6839290857]], [[0.9516258240]], [[0.3219814897]], [[0.7095546722]], [[0.2287299037]], [[0.2698395848]], [[0.3580908775]], [[0.8426191807]], [[0.8174628615]], [[0.3202064037]], [[0.7674362063]], [[0.8908686638]], [[0.5532739758]], [[0.6354734302]], [[0.7322350144]], [[0.9265709519]], [[1.0964796543]], [[0.2310708761]], [[0.4331477880]], [[0.8128876090]], [[1.0306572914]], [[1.0514862537]], [[1.0575610399]], [[0.7158899307]], [[0.1815913320]], [[0.6623789072]], [[0.3035799861]], [[0.1504055858]], [[1.0606288910]], [[0.9965625405]], [[0.1176186800]], [[0.7099686861]], [[0.5321526527]], [[0.5341020823]], [[0.3875042200]], [[0.8086607456]], [[0.3202309012]], [[0.7996783257]], [[0.8692367077]], [[0.9743184447]], [[0.8033382297]], [[0.1215150356]], [[0.2920342088]], [[0.8660401702]], [[0.7210333347]], [[0.2263406515]], [[0.3284729123]], [[1.0867573023]], [[0.9532915950]], [[0.3983700871]], [[0.4905402660]], [[0.1400836110]], [[0.6073955894]], [[0.2398532033]], [[0.2307043076]], [[0.5888328552]], [[0.6914551854]], [[0.4116175175]], [[0.9130714536]], [[0.3121131063]], [[1.0700676441]], [[0.9590325952]], [[0.1947411895]], [[0.4919404984]], [[0.6389439702]], [[0.6893332005]], [[0.7349697948]], [[0.8125967979]], [[0.6463327408]], [[0.3724182844]], [[0.8529771566]], [[0.1367582083]], [[0.3200293183]], [[0.4912177324]], [[0.3728259802]], [[0.4414659739]], [[0.2065718174]], [[0.5100250840]], [[0.7232608795]], [[0.2906497717]], [[0.5907229781]], [[0.9743080735]], [[0.5649825335]], [[0.6302787662]], [[0.5732482076]], [[0.7175733447]], [[0.9343023896]], [[1.0900057554]], [[0.9339106083]], [[1.0910894871]], [[0.5802218318]], [[0.1672219038]], [[0.3793440461]], [[0.9568352699]], [[0.6131414175]], [[0.3678594828]], [[0.2332267761]], [[0.1484566331]], [[0.1943785548]], [[0.5149642825]], [[0.8905856609]], [[0.8867031932]], [[0.1341667175]], [[0.9282736778]], [[0.2251279354]], [[0.5106775165]], [[0.4136463404]], [[0.5200750232]], [[0.5182113051]], [[0.1677076817]], [[0.1846637130]], [[0.5381429195]], [[0.6228487492]], [[0.3892452121]], [[0.8047322631]], [[0.1663534641]], [[0.5826390386]], [[1.0560923815]], [[0.4124366641]], [[1.0678842068]], [[0.7974595428]], [[0.1651521921]], [[0.9327313304]], [[0.5586854219]], [[0.3931792378]], [[1.0162093639]], [[0.2123350501]], [[0.6700351238]], [[0.5116983056]], [[0.9734389782]], [[0.7559549212]], [[0.8566353321]], [[0.7929621339]], [[0.4961453080]], [[0.5112299323]], [[0.2043419480]], [[0.8873047233]], [[1.0133717060]], [[0.9584950805]], [[0.2636935115]], [[0.6386826038]], [[0.2639155984]], [[0.3411405683]], [[0.3250299096]], [[0.7872551680]], [[0.3184260726]], [[0.6054740548]], [[0.6374167204]], [[0.9386941791]], [[0.2384226322]], [[0.2731265426]], [[0.3260518909]], [[0.9663493633]], [[0.4366501570]], [[1.0381269455]], [[0.7971864343]], [[0.6796956658]], [[0.6126606464]], [[0.5175418854]], [[0.6791158319]], [[0.5022103190]], [[0.6128694415]], [[0.6801791787]], [[0.2252801061]], [[0.3543169498]], [[1.0201289654]], [[0.2106093168]], [[0.5804795623]], [[1.1110019684]], [[0.7970011830]], [[0.6305391788]], [[0.1830776930]], [[0.8640716076]], [[0.2602422833]], [[0.4744505286]], [[0.4486244917]], [[0.5423389673]], [[0.6218518019]], [[1.0287864208]], [[0.6788020730]], [[1.0642290115]], [[0.9222388864]], [[0.3002756834]], [[0.8406347036]], [[0.2629346251]], [[0.4044700861]], [[0.7634440064]], [[0.7814787030]], [[0.9914966822]], [[0.4554247260]], [[0.6171830893]], [[0.8737944365]], [[0.1328365803]], [[0.9778729677]], [[0.2029214501]], [[0.6232739091]], [[0.5313742757]], [[0.3530461788]], [[0.6824681759]], [[1.0298420191]], [[0.4702228904]], [[0.3195356131]], [[0.4314632416]], [[0.1208084822]], [[0.8420796394]], [[0.3762508035]], [[0.2827125192]], [[0.3283319473]], [[0.9038606882]], [[0.8811694980]], [[1.0001435280]], [[0.7977442145]], [[0.4494028091]], [[0.4766585827]], [[0.7640976310]], [[1.0274000168]], [[0.7523287535]], [[0.3798099756]], [[0.3813439608]], [[0.1436481476]], [[0.7244066596]], [[0.3357914090]], [[0.1705947518]], [[1.0548208952]], [[0.2916770577]], [[0.5595018864]], [[0.7596294284]], [[0.6323118210]], [[0.2799417973]], [[0.2122218013]], [[1.0149238110]], [[0.6978020072]], [[1.0311949253]], [[0.4487623572]], [[0.7636603713]], [[0.5020445585]], [[0.5941470861]], [[0.3118605614]], [[0.7854831219]], [[0.7744656205]], [[0.6060683727]], [[0.5039315820]], [[0.3081677556]], [[0.9621550441]], [[0.2441706061]], [[0.8212153912]], [[0.4482558966]], [[0.3751486540]], [[0.7062051892]], [[0.3566541672]], [[0.7315850854]], [[0.7145730853]], [[0.2451379299]], [[0.6996319294]], [[0.8293472528]], [[0.8142981529]], [[0.5534433126]], [[0.2064888477]], [[0.5393056273]], [[0.7900364995]], [[0.4339441061]], [[0.8061732054]], [[0.9493639469]], [[0.3553068638]], [[0.6213135719]], [[0.8231321573]], [[0.6555715799]], [[0.6581484079]], [[0.6788153648]], [[0.2233088017]], [[0.6556767821]], [[0.9626175761]], [[1.0669395924]], [[0.9102581143]], [[0.6833841801]], [[0.8498897552]], [[0.3731428385]], [[0.2020384669]], [[0.1864221096]], [[1.1151875257]], [[0.9337773323]], [[0.2707657814]], [[0.8120039701]], [[0.9939664602]], [[1.1161900759]], [[1.0535800457]], [[1.0037596226]], [[0.5017530322]], [[0.4409118295]], [[1.0269070864]], [[0.8965724707]], [[0.3154931664]], [[1.0658912659]], [[0.8579619527]], [[0.8889504075]], [[0.3029944897]], [[0.7598325610]], [[0.4411014318]], [[1.0070610046]], [[0.5264123678]], [[0.8110389113]], [[0.7051935792]], [[0.8291167617]], [[0.4464688897]], [[0.8601683378]], [[0.2671471834]], [[0.7293101549]], [[0.2780867219]], [[0.1231138706]], [[0.2148547769]], [[1.0111176968]], [[0.8869004846]], [[1.0854904652]], [[1.0169432163]], [[0.1698599458]], [[0.2751671076]], [[0.5355913639]], [[0.2916665077]], [[0.9635809064]], [[0.2383787632]], [[0.3724319339]], [[0.1333369613]], [[0.3325098753]], [[1.0276213884]], [[1.0257626772]], [[0.9743015170]], [[1.0024462938]], [[1.0609759092]], [[0.4883511662]], [[0.8363837004]], [[1.0618479252]], [[0.7817920446]], [[1.1162208319]], [[0.8757141232]], [[0.9272068143]], [[0.4413865805]], [[0.8562964201]], [[0.6738359332]], [[0.4969739914]], [[0.3345277309]], [[0.3358243704]], [[0.2316422462]], [[0.9520478249]], [[0.9718482494]], [[0.5594763160]], [[0.3270396590]], [[1.0028353930]], [[0.9361271262]], [[0.6535496712]], [[0.3803150058]], [[1.0759065151]], [[0.8208529949]], [[0.2368099689]], [[1.0949240923]], [[0.9960695505]], [[0.4341372252]], [[0.8974599242]], [[0.3322881460]], [[0.5380254984]], [[1.0409332514]], [[0.6370422244]], [[0.2627735734]], [[0.4492661953]], [[0.4806600809]], [[0.5199182630]], [[0.6642329693]], [[1.0787974596]], [[0.6431528330]], [[0.3076626658]], [[0.6420097947]], [[0.8561262488]], [[0.8644027710]], [[0.1594133973]], [[0.5269114375]], [[0.2448104024]], [[0.4030398726]], [[0.7965293527]], [[0.2613175511]], [[0.8022460938]], [[1.0407717228]], [[0.6491768360]], [[0.2831388116]], [[0.4372401834]], [[0.7255647182]], [[0.2352236509]], [[0.8647878170]], [[0.1624478698]], [[0.1357359886]], [[0.1305523515]], [[0.5149509907]], [[0.9525991082]], [[0.1431431770]], [[1.0319762230]], [[0.4163712263]], [[0.7628247142]], [[0.6391841173]], [[0.1655231118]], [[1.0310471058]], [[0.8856043816]], [[1.1133804321]], [[0.8689887524]], [[0.2863481641]], [[1.0336744785]], [[0.6432548761]], [[0.8534908891]], [[0.2154681087]], [[0.4725694060]], [[0.1254438758]], [[0.4216364026]], [[0.7242482305]], [[0.2238017917]], [[0.7757647634]], [[0.8847860694]], [[0.6860373020]], [[0.2818409801]], [[0.2287228107]], [[0.4621243477]], [[0.8358617425]], [[1.1095808744]], [[0.9038971663]], [[0.5600780249]], [[0.7916908860]], [[0.1258512139]], [[0.1893320084]], [[0.8496866226]], [[0.3331750631]], [[0.8569304943]], [[0.2634165883]], [[0.3687272072]], [[0.2045381665]], [[0.8773030043]], [[0.5654351115]], [[1.0011832714]], [[0.9258187413]], [[0.8930539489]], [[0.6324607134]], [[0.4617936611]], [[0.5076668262]], [[0.6828329563]], [[0.8642374277]], [[0.2660877109]], [[1.0360215902]], [[0.5620176196]], [[0.1974098682]], [[0.3458537459]], [[1.0587915182]], [[1.0736463070]], [[0.1532428265]], [[0.9690318108]], [[0.8669622540]], [[0.9123418927]], [[1.0396473408]], [[0.3469074965]], [[0.7742705345]], [[0.8209992647]], [[0.4686359763]], [[0.7837092280]], [[0.4725259542]], [[0.9255133271]], [[0.4776553512]], [[0.4299848080]], [[0.7422571778]], [[0.7937294841]], [[0.3720968366]], [[0.6605818272]], [[0.9061505198]], [[0.5666342974]], [[0.7685496211]], [[0.4957891703]], [[0.7916324735]], [[0.2541855574]], [[0.3223682642]], [[0.3625873923]], [[1.0758881569]], [[0.4818406105]], [[0.6150174737]], [[0.3741348982]], [[1.1155276299]], [[1.1047177315]], [[0.2392893434]], [[0.2110477090]], [[0.2373861670]], [[0.6139712930]], [[0.4889307022]], [[0.2891098261]], [[0.4370465875]], [[0.7108480930]], [[0.3551372886]], [[0.7271739244]], [[0.5017247200]], [[0.3741009831]], [[0.6850754023]], [[1.0275118351]], [[0.2783430219]], [[0.6395998597]], [[0.4319987893]], [[1.1070358753]], [[0.1420010328]], [[0.1370520592]], [[1.1090750694]], [[0.3000376821]], [[0.7122443318]], [[0.5732296109]], [[0.5110541582]], [[0.5047087669]], [[0.9341029525]], [[0.6402811408]], [[0.1295753717]], [[0.3212025762]], [[0.4459325075]], [[0.8679908514]], [[0.2928158045]], [[1.0878424644]], [[0.5050179958]], [[0.5266016722]], [[1.0081616640]], [[0.8676865101]], [[1.0404428244]], [[0.9056048393]], [[0.4647057056]], [[0.2846464515]], [[0.5791811347]], [[1.0302014351]], [[0.4485727549]], [[0.1526976824]], [[0.8213468790]], [[1.1031184196]], [[0.4740410447]], [[0.2023708820]], [[0.1628528237]], [[0.7416823506]], [[0.5785228014]], [[0.3638890386]], [[0.7174485922]], [[0.8062620759]], [[1.0140421391]], [[1.0045738220]], [[0.5415408015]], [[0.1754990816]], [[0.1645711064]], [[1.0832273960]], [[0.8374102712]], [[0.8343363404]], [[0.1837721467]], [[1.0793905258]], [[1.0900542736]], [[1.0678128004]], [[0.1945901513]], [[0.4277209640]], [[0.2725007534]], [[1.0898654461]], [[0.4015426636]], [[0.3881053925]], [[0.8783391118]], [[0.3850864768]], [[0.3701215982]], [[0.5726469159]], [[0.5683228970]], [[0.2268934250]], [[1.0332167149]], [[0.3958218098]], [[0.7937400937]], [[1.0512745380]], [[0.8685408831]], [[0.6871725917]], [[1.0418112278]], [[0.6835957766]], [[0.3850543499]], [[1.0893824100]], [[0.7347229123]], [[0.1285422444]], [[0.4740368724]], [[0.2757989168]], [[1.0547591448]], [[0.5338097215]], [[0.1606202126]], [[0.5849509835]], [[0.9303891063]], [[0.7462828755]], [[0.7744848728]], [[0.6628192663]], [[0.8027920127]], [[0.4945566654]], [[0.4174825549]], [[0.1490284204]], [[0.2397142053]], [[0.8330839276]], [[0.3203259706]], [[0.6881158948]], [[0.7759783864]], [[0.6517845988]], [[0.2922077775]], [[1.0945467949]], [[0.3256329298]], [[1.0276329517]], [[0.2186285257]], [[0.4961084127]], [[0.8883670568]], [[0.4120923281]], [[1.0364041328]], [[0.2723044157]], [[0.1964969039]], [[0.3909230828]], [[0.6972078681]], [[1.0767605305]], [[0.3776822090]], [[0.7951967716]], [[0.4910291433]], [[0.5079514980]], [[0.9840343595]], [[0.2288897038]], [[0.6694893241]], [[1.0865871906]], [[0.5476765633]], [[1.0046017170]], [[0.4624042511]], [[1.0188636780]], [[0.1327174902]], [[0.5443208218]], [[0.5285594463]], [[0.7784251571]], [[0.8125463724]], [[1.0002911091]], [[0.5418896675]], [[0.5965551138]], [[0.9587882757]], [[0.4810960889]], [[1.0546994209]], [[0.2834765911]], [[0.5622777343]], [[0.5895271897]], [[0.8394876122]], [[0.9582207203]], [[0.5371307731]], [[0.2021423578]], [[0.8641287684]], [[0.7659485340]], [[0.8172323108]], [[0.3079488277]], [[0.9381804466]], [[1.0899600983]], [[0.6597683430]], [[0.1493584514]], [[0.9673776031]], [[0.2456570268]], [[0.7313187122]], [[0.6890415549]], [[0.3823643923]], [[0.7904804349]], [[0.1691663265]], [[0.7302623987]], [[0.2994071841]], [[0.5623158216]], [[0.6806970835]], [[1.0423548222]], [[0.3778225183]], [[0.9367007017]], [[0.5528657436]], [[0.3789187670]], [[0.1809828281]], [[0.1576234698]], [[1.1046879292]], [[0.4916828871]], [[0.6413570642]], [[0.7519388199]], [[0.9562487602]], [[1.0431196690]], [[1.0218912363]], [[0.2459411025]], [[0.5362742543]], [[0.3205138445]], [[0.3307067752]], [[0.7349861860]], [[1.0857067108]], [[0.2158306837]], [[0.9189848900]], [[0.3571493030]], [[0.5189955235]], [[1.0132681131]], [[0.5032975078]], [[0.6619105339]], [[0.2668663859]], [[1.0420031548]], [[0.5517477393]], [[0.2506874204]], [[0.7627878189]], [[0.2608739734]], [[0.2196289897]], [[0.6468422413]], [[1.0128045082]], [[0.4748906493]], [[0.8517161012]], [[1.0460432768]], [[0.9480131269]], [[0.3541021347]], [[0.5615727305]], [[0.4590402246]], [[0.2143211961]], [[0.6165892482]], [[0.9926002026]], [[1.0376404524]], [[0.6630330086]], [[0.7299441695]], [[0.3999218345]], [[0.9938038588]], [[0.4083284140]], [[0.2690562606]], [[0.6933794022]], [[0.9160753489]], [[0.1655868888]], [[1.0683664083]], [[0.7962481976]], [[0.2660762668]], [[0.5086789131]], [[1.0501725674]]]]), compiled_model=tensor([[[[0.6126392484]], [[1.2847993374]], [[1.0160138607]], [[1.2693874836]], [[1.4212834835]], [[2.3957853317]], [[1.8067994118]], [[1.6497118473]], [[1.5160260201]], [[1.3800323009]], [[1.3064551353]], [[1.6005483866]], [[1.6512989998]], [[1.9601306915]], [[1.6164029837]], [[1.6097412109]], [[1.9245719910]], [[1.3664436340]], [[1.8692334890]], [[0.6696540117]], [[2.0294227600]], [[1.4287543297]], [[1.3829226494]], [[1.8492763042]], [[1.9277071953]], [[1.3387179375]], [[1.2250795364]], [[1.0212728977]], [[0.8859246373]], [[0.9770479202]], [[0.9892371893]], [[1.8137917519]], [[0.2374306917]], [[0.6004611850]], [[0.9479789734]], [[0.7635777593]], [[0.5917453766]], [[1.1472525597]], [[0.7865327001]], [[1.2317757607]], [[1.1798388958]], [[0.4441328645]], [[1.2247608900]], [[0.0905500054]], [[1.0226666927]], [[1.1868164539]], [[1.3895325661]], [[0.9884651303]], [[1.0884656906]], [[1.2369925976]], [[1.3197257519]], [[1.7293443680]], [[0.7098842859]], [[1.0727012157]], [[1.4801776409]], [[0.2418187261]], [[1.2930548191]], [[0.1234154701]], [[0.2045398355]], [[1.1550202370]], [[1.1481279135]], [[0.8600751758]], [[0.3194243312]], [[1.0265192986]], [[1.6776773930]], [[1.4126365185]], [[1.2075133324]], [[1.5905725956]], [[1.7954502106]], [[1.6137953997]], [[0.8286987543]], [[0.7832444310]], [[1.5611045361]], [[1.1539914608]], [[1.4845077991]], [[0.9443597198]], [[1.4038182497]], [[0.2443515658]], [[1.0906814337]], [[0.7320560813]], [[0.9337904453]], [[1.4632613659]], [[1.4846388102]], [[0.9817238450]], [[0.8220419288]], [[0.5516623259]], [[0.4593209624]], [[1.1571905613]], [[1.1278827190]], [[0.8502892852]], [[1.1452194452]], [[1.2103493214]], [[1.4099234343]], [[1.3333654404]], [[0.5659285188]], [[0.9578503966]], [[1.2075853348]], [[0.8140281439]], [[0.4364193678]], [[0.4325580001]], [[1.3253519535]], [[1.5005913973]], [[1.0985537767]], [[1.2797951698]], [[0.1398817301]], [[1.0972954035]], [[0.5149897337]], [[0.4405116439]], [[0.7021362185]], [[0.7748602629]], [[0.3825830221]], [[1.3529112339]], [[0.5807139874]], [[1.3827395439]], [[1.0386538506]], [[1.0400578976]], [[1.1456129551]], [[0.9158973694]], [[1.1292204857]], [[1.1569693089]], [[0.8227207661]], [[0.9624983072]], [[0.4528711438]], [[1.6120047569]], [[0.3369519711]], [[0.8797501922]], [[0.8990117311]], [[0.9069194794]], [[1.1384121180]], [[0.4042670727]], [[1.1263208389]], [[1.6356709003]], [[0.2014245391]], [[1.0763012171]], [[1.7242183685]], [[1.3094674349]], [[0.8055772185]], [[0.6267769337]], [[0.8594372869]], [[1.1547892094]], [[1.3672578335]], [[1.2528786659]], [[1.3709373474]], [[0.8524004221]], [[0.3359656334]], [[0.9610252976]], [[1.6263048649]], [[1.1979488134]], [[1.0935943127]], [[0.8229061961]], [[0.1400037408]], [[0.1709723473]], [[1.3182597160]], [[0.9411504269]], [[1.1480791569]], [[0.1307947040]], [[1.6400890350]], [[0.4380089641]], [[0.8500742316]], [[0.6224029064]], [[1.3691478968]], [[0.9948672652]], [[0.9480074048]], [[0.5095565319]], [[0.6035228968]], [[1.3836289644]], [[0.5853728056]], [[1.1615338326]], [[0.4958323836]], [[1.3816725016]], [[1.2827050686]], [[0.5330341458]], [[1.7376055717]], [[0.6885040402]], [[0.6305160522]], [[1.0934945345]], [[0.9451226592]], [[0.9119152427]], [[1.4458801746]], [[0.1576229334]], [[1.1332480907]], [[0.4226775169]], [[1.6680239439]], [[1.2445876598]], [[1.4015262127]], [[1.3785165548]], [[0.8573858142]], [[0.6743203998]], [[1.0355303288]], [[1.4556453228]], [[1.5962462425]], [[1.2893085480]], [[0.7186721563]], [[0.8240745664]], [[0.6640875936]], [[0.2640556693]], [[0.5556254983]], [[1.2639942169]], [[1.0477461815]], [[1.3508286476]], [[1.2026064396]], [[1.1886200905]], [[0.6963486671]], [[0.9974403381]], [[0.5140721798]], [[1.0149235725]], [[1.0223200321]], [[1.0911922455]], [[1.3983223438]], [[0.8444306254]], [[0.8843479156]], [[1.1669619083]], [[0.6695883870]], [[1.3737998009]], [[1.0526404381]], [[1.2752029896]], [[0.3498058319]], [[1.0496865511]], [[1.2025928497]], [[0.3159953356]], [[1.1349006891]], [[1.4383280277]], [[1.6453288794]], [[1.3275856972]], [[0.2613739967]], [[1.5299053192]], [[0.2602577806]], [[0.6402902603]], [[0.4558212161]], [[0.9366688132]], [[1.3269431591]], [[1.8576347828]], [[0.6302710772]], [[1.4143818617]], [[1.6265064478]], [[0.7495439053]], [[0.7884989977]], [[1.0388441086]], [[0.9865222573]], [[1.3550720215]], [[1.5914535522]], [[1.2005062103]], [[0.8923559785]], [[1.2920346260]], [[1.6774952412]], [[0.8823253512]], [[1.4878342152]], [[0.1982974410]], [[1.4543871880]], [[0.7748268247]], [[0.2464106083]], [[1.3077206612]], [[1.4468449354]], [[1.0746600628]], [[0.6312167048]], [[0.7790499926]], [[0.9195779562]], [[1.1930985451]], [[1.2566304207]], [[0.6363571882]], [[0.8210837245]], [[1.3929480314]], [[1.4141800404]], [[1.6696994305]], [[1.5049270391]], [[1.1231745481]], [[1.0634955168]], [[1.4801609516]], [[1.2590243816]], [[0.7796180844]], [[0.4874807596]], [[0.8763029575]], [[0.4816260338]], [[1.3006384373]], [[0.7318294048]], [[0.8392173648]], [[1.7959675789]], [[1.3754868507]], [[0.6007851362]], [[1.4453048706]], [[0.9797057509]], [[1.0866384506]], [[1.0618031025]], [[2.0796980858]], [[1.6812306643]], [[2.5373702049]], [[1.4113801718]], [[1.3508670330]], [[0.5058811903]], [[0.8935808539]], [[0.7964126468]], [[1.8689804077]], [[2.1022953987]], [[1.8945679665]], [[1.0271627903]], [[0.9288477898]], [[1.8206104040]], [[1.4753583670]], [[1.3073869944]], [[2.0076546669]], [[1.1788567305]], [[2.2260773182]], [[1.3040659428]], [[1.4189537764]], [[1.3648840189]], [[1.8771333694]], [[1.8185632229]], [[3.4382107258]], [[2.2413094044]], [[2.0677287579]], [[0.9465488195]], [[1.2514548302]], [[0.9109070897]], [[1.9000020027]], [[2.4696271420]], [[2.2930040359]], [[2.2815110683]], [[1.5965597630]], [[2.3723914623]], [[0.8603610396]], [[1.5050965548]], [[2.4746520519]], [[1.3838727474]], [[2.6422491074]], [[1.9354590178]], [[2.6077294350]], [[3.5365159512]], [[2.5945770741]], [[2.4315469265]], [[2.0029740334]], [[2.0259866714]], [[1.8531718254]], [[2.0701599121]], [[2.1804168224]], [[1.3508627415]], [[1.3079271317]], [[1.8872189522]], [[3.2373540401]], [[2.5754861832]], [[2.5747611523]], [[1.7179993391]], [[1.9157209396]], [[2.1155438423]], [[2.0252065659]], [[2.1283161640]], [[2.6452736855]], [[1.8267173767]], [[2.7579760551]], [[1.2132956982]], [[3.0068135262]], [[2.4631180763]], [[3.5405578613]], [[2.6474239826]], [[1.5910613537]], [[2.0927491188]], [[1.9897241592]], [[1.1013202667]], [[2.2189254761]], [[1.8796958923]], [[1.7149174213]], [[1.8627212048]], [[1.7743721008]], [[1.2092382908]], [[1.9356389046]], [[3.0627901554]], [[1.8477802277]], [[2.9874773026]], [[1.5558798313]], [[1.8826551437]], [[2.6651489735]], [[0.9460638762]], [[1.6016631126]], [[1.6695210934]], [[2.5003185272]], [[2.0576295853]], [[1.9772517681]], [[2.5009937286]], [[1.9855127335]], [[2.6145079136]], [[1.5627379417]], [[1.5143810511]], [[2.3694014549]], [[1.9381847382]], [[1.3082990646]], [[1.9379069805]], [[2.4449617863]], [[2.2284786701]], [[1.9400577545]], [[1.7861289978]], [[2.8127992153]], [[2.4590821266]], [[1.8723027706]], [[2.0154581070]], [[2.3502509594]], [[1.1160075665]], [[2.1576600075]], [[3.0725395679]], [[2.0451641083]], [[0.6608750224]], [[2.4659655094]], [[2.6076951027]], [[2.1282236576]], [[1.7682229280]], [[2.1097407341]], [[1.4181568623]], [[1.2313792706]], [[1.5169248581]], [[1.6490278244]], [[1.9212672710]], [[2.4415259361]], [[2.2367002964]], [[2.2774751186]], [[2.6165854931]], [[2.1887512207]], [[1.9790605307]], [[1.9198820591]], [[2.5306496620]], [[2.4902074337]], [[2.8822772503]], [[2.0687849522]], [[1.8810281754]], [[1.0920443535]], [[2.3473873138]], [[2.4194779396]], [[1.9421298504]], [[0.5074310899]], [[1.7196934223]], [[0.8027705550]], [[1.4250694513]], [[1.6291654110]], [[1.6940532923]], [[1.4418413639]], [[2.7376053333]], [[1.3668283224]], [[1.7057944536]], [[1.9952664375]], [[2.4717228413]], [[1.0840055943]], [[2.1751985550]], [[1.1282248497]], [[1.7455551624]], [[2.2935512066]], [[2.1847345829]], [[2.0868473053]], [[0.4775733948]], [[2.6362545490]], [[2.3963522911]], [[2.2232995033]], [[2.1940543652]], [[1.0425589085]], [[1.0968611240]], [[1.7524145842]], [[2.8102638721]], [[2.5829305649]], [[2.1974239349]], [[1.6277592182]], [[1.8983659744]], [[1.5967221260]], [[1.8115600348]], [[2.0387825966]], [[1.2349385023]], [[2.2242774963]], [[1.9058637619]], [[0.7111553550]], [[1.3966543674]], [[2.2581615448]], [[1.9319989681]], [[2.0974605083]], [[0.8482221961]], [[1.9527466297]], [[2.3510360718]], [[2.3399691582]], [[2.5700280666]], [[2.5576958656]], [[1.8749256134]], [[1.5995395184]], [[1.1973047256]], [[1.8889935017]], [[1.2457716465]], [[1.1657190323]], [[1.5862995386]], [[1.8062589169]], [[1.9646565914]], [[2.4827551842]], [[2.3427269459]], [[2.1380708218]], [[2.0972955227]], [[2.0793805122]], [[2.0545210838]], [[2.4332692623]], [[1.9909231663]], [[2.5593914986]], [[1.7359678745]], [[1.3405991793]], [[2.4576475620]], [[1.5923849344]], [[0.8783162236]], [[1.9325655699]], [[2.5579202175]], [[2.5777115822]], [[0.7576700449]], [[3.0607087612]], [[1.9372196198]], [[2.2473897934]], [[1.8931858540]], [[1.8002314568]], [[2.3377048969]], [[2.6462082863]], [[2.0836505890]], [[2.9943666458]], [[2.2693295479]], [[1.8470360041]], [[2.0004286766]], [[1.0881807804]], [[2.2922675610]], [[1.8939489126]], [[1.6876378059]], [[2.1403608322]], [[2.1730372906]], [[2.0085713863]], [[2.4433531761]], [[1.8390433788]], [[2.5747222900]], [[1.5938475132]], [[1.2154915333]], [[1.7041221857]], [[1.6980329752]], [[2.0305602551]], [[2.0169553757]], [[1.1281700134]], [[2.1298344135]], [[2.4407727718]], [[1.2411930561]], [[2.5820686817]], [[1.6975775957]], [[1.4171856642]], [[1.6448955536]], [[1.7079312801]], [[2.0430123806]], [[2.6770575047]], [[2.5490894318]], [[2.7350831032]], [[2.0681326389]], [[1.9966527224]], [[2.7066144943]], [[2.1135072708]], [[2.1335916519]], [[1.1755639315]], [[1.6906957626]], [[2.1122374535]], [[1.2308497429]], [[1.0500020981]], [[3.4394295216]], [[1.1447603703]], [[1.3113223314]], [[2.0852017403]], [[1.6112942696]], [[1.1555157900]], [[1.5976965427]], [[2.3042311668]], [[0.6561325788]], [[1.4502598047]], [[1.4592287540]], [[2.1307442188]], [[0.9457106590]], [[1.6666777134]], [[0.6475367546]], [[1.5687589645]], [[2.2034051418]], [[1.4414641857]], [[1.9517776966]], [[1.0384047031]], [[2.0780453682]], [[1.2406331301]], [[1.7416948080]], [[1.4409965277]], [[1.4011693001]], [[1.3811714649]], [[1.4451811314]], [[2.3342161179]], [[0.9672920108]], [[0.7198982239]], [[1.2746539116]], [[1.3744841814]], [[1.8599383831]], [[0.6956352592]], [[0.7422550321]], [[2.0261144638]], [[1.9932048321]], [[1.3647475243]], [[1.2305493355]], [[1.2675635815]], [[1.1903040409]], [[2.4475209713]], [[2.1161994934]], [[2.2318627834]], [[1.5944170952]], [[2.1591389179]], [[1.7946082354]], [[1.9572036266]], [[1.7890419960]], [[1.5016119480]], [[1.4993977547]], [[1.5436656475]], [[1.8229780197]], [[0.7076258063]], [[1.3922975063]], [[0.8836020231]], [[2.1523113251]], [[1.7926740646]], [[1.2107822895]], [[1.7866199017]], [[2.0228738785]], [[1.4151389599]], [[1.6561564207]], [[2.7083377838]], [[1.5731332302]], [[1.5417602062]], [[2.2978768349]], [[1.8048982620]], [[1.0197699070]], [[2.5689182281]], [[1.9207156897]], [[1.5669262409]], [[1.0021938086]], [[1.0122436285]], [[1.5895566940]], [[1.5285730362]], [[0.8092445731]], [[1.8269070387]], [[2.2193837166]], [[2.2482719421]], [[1.1043691635]], [[1.8965046406]], [[1.2286930084]], [[1.3172914982]], [[0.6924899817]], [[0.2391647100]], [[0.3091518283]], [[2.1676588058]], [[1.3594446182]], [[1.9698956013]], [[1.7561602592]], [[1.5505784750]], [[1.1046645641]], [[1.7907090187]], [[0.9408040047]], [[2.2968308926]], [[0.3507485986]], [[1.0683232546]], [[0.9857742190]], [[1.3872470856]], [[2.1478366852]], [[1.7414052486]], [[1.4622098207]], [[1.8110802174]], [[2.3387472630]], [[1.6464242935]], [[1.7139790058]], [[2.6530718803]], [[1.0672132969]], [[1.9933679104]], [[2.4932761192]], [[1.7841987610]], [[1.1697916985]], [[1.8800296783]], [[1.2469937801]], [[1.6056798697]], [[0.9578014016]], [[1.5572735071]], [[0.5278249979]], [[1.6521646976]], [[1.5527687073]], [[1.8032001257]], [[1.6926729679]], [[2.4715511799]], [[2.0873689651]], [[1.7234015465]], [[1.2142677307]], [[1.4172137976]], [[2.5624651909]], [[0.4544686675]], [[1.8021951914]], [[1.4658420086]], [[1.8690576553]], [[1.9521789551]], [[1.0924329758]], [[0.8325417638]], [[2.6377520561]], [[1.7632641792]], [[1.7439229488]], [[0.9657251835]], [[1.3678379059]], [[2.2542760372]], [[1.4037462473]], [[1.4685748816]], [[1.8236267567]], [[1.2359704971]], [[1.4835586548]], [[1.5493826866]], [[1.8001058102]], [[0.7245556712]], [[1.0450589657]], [[1.0194532871]], [[0.9725016356]], [[1.7611985207]], [[1.2553027868]], [[1.6735062599]], [[1.7654244900]], [[1.3804740906]], [[1.4142068624]], [[1.1884090900]], [[1.3350558281]], [[0.8620188832]], [[2.2143335342]], [[0.7008385658]], [[1.4918987751]], [[1.3344490528]], [[1.9376916885]], [[2.2101497650]], [[1.5036303997]], [[1.3469265699]], [[1.2364348173]], [[0.8898710012]], [[1.0841038227]], [[1.2608656883]], [[2.7296915054]], [[1.7304069996]], [[2.4811725616]], [[1.3596812487]], [[1.1468870640]], [[2.6548738480]], [[1.2181899548]], [[1.4475929737]], [[0.9516216516]], [[1.4512550831]], [[1.1619448662]], [[0.7206761241]], [[1.6423406601]], [[1.0177131891]], [[0.8694836497]], [[2.2868351936]], [[2.0222301483]], [[1.2353727818]], [[1.0885820389]], [[2.0871546268]], [[1.8499560356]], [[1.4526863098]], [[1.9035083055]], [[1.2300615311]], [[1.7379571199]], [[1.3231043816]], [[1.1438457966]], [[2.4367780685]], [[0.8798409104]], [[1.6363316774]], [[0.5541524887]], [[1.6404782534]], [[1.2015752792]], [[1.8588250875]], [[1.0939009190]], [[2.1510286331]], [[1.6792905331]], [[2.2943058014]], [[1.2601904869]], [[1.3873966932]], [[1.4820153713]], [[2.2082509995]]]])
test_advindex.test_module[Advindex1-[((1, 2), torch.float32)]]: forge/test/models_ops/test_advindex.py#L34
IndexError: index 933 is out of bounds for dimension 0 with size 1
test_advindex.test_module[Advindex0-[((32, 2), torch.float32), ((1,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L24
IndexError: index 775 is out of bounds for dimension 0 with size 32
test_advindex.test_module[Advindex3-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L60
IndexError: index 992 is out of bounds for dimension 0 with size 169
test_advindex.test_module[Advindex5-[((2401,), torch.int64)]]: forge/test/models_ops/test_advindex.py#L86
IndexError: index 979 is out of bounds for dimension 0 with size 169
test_avgpool2d.test_module[Avgpool2D9-[((1, 1536, 8, 8), torch.float32)]]: forge/test/models_ops/test_avgpool2d.py#L1472
RuntimeError: Tensor 1 - stride mismatch: expected [64, 1], got [0, 0]
test_broadcast.test_module[Broadcast1-[((1, 1, 1, 128), torch.bool)]]: forge/test/models_ops/test_broadcast.py#L172
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((2, 13, 1), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[ 44.], [239.], [933.], [760.], [963.], [379.], [427.], [503.], [497.], [683.], [101.], [866.], [756.]], [[399.], [878.], [376.], [ 56.], [868.], [794.], [ 33.], [126.], [119.], [391.], [254.], [824.], [841.]]]), compiled_model=tensor([[[ 44.], [ 0.], [239.], [ 0.], [933.], [ 0.], [760.], [ 0.], [963.], [ 0.], [379.], [ 0.], [427.]], [[ 0.], [503.], [ 0.], [497.], [ 0.], [683.], [ 0.], [101.], [ 0.], [866.], [ 0.], [756.], [ 0.]]])
test_cast.test_module[Cast0-[((2, 1, 1, 13), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((2, 1, 7, 7), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 1, 1, 128), torch.int64)]]: forge/test/models_ops/test_cast.py#L502
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[ 44., 239., 933., 760., 963., 379., 427., 503., 497., 683., 101., 866., 756., 399., 878., 376., 56., 868., 794., 33., 126., 119., 391., 254., 824., 841., 269., 969., 549., 480., 481., 12., 363., 360., 995., 85., 822., 999., 711., 288., 778., 43., 896., 489., 771., 757., 283., 395., 782., 671., 840., 969., 673., 441., 111., 280., 503., 706., 476., 327., 699., 326., 363., 474., 775., 700., 218., 632., 568., 12., 277., 645., 110., 80., 348., 521., 119., 916., 881., 490., 682., 519., 944., 633., 169., 863., 409., 233., 219., 78., 735., 83., 222., 858., 235., 416., 446., 835., 877., 212., 505., 946., 156., 315., 284., 750., 108., 71., 847., 108., 75., 84., 884., 148., 144., 534., 519., 260., 907., 514., 175., 663., 613., 157., 133., 120., 249., 389.]]]]), compiled_model=tensor([[[[ 44., 0., 239., 0., 933., 0., 760., 0., 963., 0., 379., 0., 427., 0., 503., 0., 497., 0., 683., 0., 101., 0., 866., 0., 756., 0., 399., 0., 878., 0., 376., 0., 56., 0., 868., 0., 794., 0., 33., 0., 126., 0., 119., 0., 391., 0., 254., 0., 824., 0., 841., 0., 269., 0., 969., 0., 549., 0., 480., 0., 481., 0., 12., 0., 363., 0., 360., 0., 995., 0., 85., 0., 822., 0., 999., 0., 711., 0., 288., 0., 778., 0., 43., 0., 896., 0., 489., 0., 771., 0., 757., 0., 283., 0., 395., 0., 782., 0., 671., 0., 840., 0., 969., 0., 673., 0., 441., 0., 111., 0., 280., 0., 503., 0., 706., 0., 476., 0., 327., 0., 699., 0., 326., 0., 363., 0., 474., 0.]]]])
test_cast.test_module[Cast0-[((1, 1, 256, 256), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast2-[((1, 1, 256, 256), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 128), torch.int32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast0-[((1, 12, 128, 128), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 384), torch.int32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast1-[((1, 1, 32, 32), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast0-[((1, 1, 32, 32), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 1, 256, 256), torch.float32)]]: forge/test/models_ops/test_cast.py#L502
TypeError: Dtype mismatch: framework_model.dtype=torch.int8, compiled_model.dtype=torch.uint8
test_cast.test_module[Cast2-[((1, 32), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_cast.test_module[Cast1-[((1, 16, 192, 640), torch.bool)]]: forge/test/models_ops/test_cast.py#L502
RuntimeError: Tensor 0 - data type mismatch: expected UInt8, got Float32
test_concatenate.test_module[Concatenate52-[((256, 1536, 1, 1), torch.float32), ((384, 1536, 1, 1), torch.float32), ((384, 1536, 1, 1), torch.float32)]]: forge/test/models_ops/test_concatenate.py#L12958
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 6442450944 B DRAM buffer across 12 banks, where each bank needs to store 536870912 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7fbf6a1e77e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7fbf6a1b2e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x1ebfb43) [0x7fbf689a5b43] --- tt::tt_metal::allocate_tensor_on_devices(tt::tt_metal::TensorSpec const&, std::vector<tt::tt_metal::v0::IDevice*, std::allocator<tt::tt_metal::v0::IDevice*> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xdbf0e) [0x7fbf784f8f0e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xdba65) [0x7fbf784f8a65] --- tt::runtime::ttnn::operations::creation::run(tt::target::ttnn::EmptyOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fbf789449f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fbf7894493e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fbf7870fb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5584161a6b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x558416196a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x55841619c574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5584161b5172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x55841619197f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d]
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((1024, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7fbf784da288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7fbf66ed57a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7fbf66ec8fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7fbf66f2f829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7fbf690a6fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7fbf690a73e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7fbf690a672b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fbf690a63e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fbf690a62d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fbf690a549f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fbf66f2eb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fbf690acfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fbf690ad7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fbf66ec66bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fbf66eb5f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fbf784d5365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fbf789449f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fbf7894493e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fbf7870fb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5584161a6b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x558416196a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x55841619c574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5584161b5172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x55841619197f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30]
test_conv2d.test_module[Conv2D1-[((1, 1280, 3000, 1), torch.float32), ((1280, 1280, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[633.6936035156], [945.1713867188], [946.9920654297], ..., [953.5192871094], [948.5264892578], [940.5952758789]], [[635.0741577148], [950.9416503906], [948.4129638672], ..., [941.4180297852], [953.1661376953], [947.6674804688]], [[652.5841064453], [962.0004272461], [961.6913452148], ..., [961.0721435547], [963.7883911133], [960.6213378906]], ..., [[647.2454223633], [947.0665283203], [955.3347778320], ..., [965.5654907227], [965.4136962891], [966.5708007812]], [[639.2039184570], [952.5627441406], [947.1251831055], ..., [951.1443481445], [948.8281250000], [956.2294921875]], [[642.4251098633], [964.2901611328], [956.3991699219], ..., [979.0765991211], [967.5089111328], [959.4161376953]]]]), compiled_model=tensor([[[[636.], [952.], [956.], ..., [944.], [952.], [936.]], [[632.], [976.], [944.], ..., [956.], [956.], [964.]], [[644.], [976.], [992.], ..., [972.], [984.], [984.]], ..., [[644.], [952.], [964.], ..., [972.], [956.], [984.]], [[636.], [976.], [956.], ..., [960.], [948.], [964.]], [[648.], [984.], [964.], ..., [980.], [972.], [964.]]]])
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((384, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7fbf784da288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7fbf66ed57a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7fbf66ec8fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7fbf66f2f829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7fbf690a6fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7fbf690a73e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7fbf690a672b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fbf690a63e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fbf690a62d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fbf690a549f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fbf66f2eb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fbf690acfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fbf690ad7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fbf66ec66bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fbf66eb5f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fbf784d5365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fbf789449f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fbf7894493e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fbf7870fb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5584161a6b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x558416196a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x55841619c574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5584161b5172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x55841619197f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30]
test_conv2d.test_module[Conv2D1-[((1, 512, 3000, 1), torch.float32), ((512, 512, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7fbf784da288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7fbf66ed57a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7fbf66ec8fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7fbf66f2f829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7fbf690a6fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7fbf690a73e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7fbf690a672b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fbf690a63e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fbf690a62d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fbf690a549f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fbf66f2eb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fbf690acfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fbf690ad7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fbf66ec66bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fbf66eb5f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fbf784d5365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fbf789449f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fbf7894493e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fbf7870fb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5584161a6b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x558416196a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x55841619c574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5584161b5172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x55841619197f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30]
test_conv2d.test_module[Conv2D0-[((1, 80, 3000, 1), torch.float32), ((768, 80, 3, 1), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_FATAL @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:1180: act_block_w_datums == round_up(conv_act_size_c * filter_w, TILE_WIDTH) info: Error backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xbd288) [0x7fbf784da288] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal2v07ProgramERKNS4_6TensorESA_RKNS4_5ShapeESt8optionalIS9_ERKSF_RKNS0_14sliding_window19SlidingWindowConfigEjjbbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbbRS8_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x90c4) [0x7fbf66ed57a4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EEbS8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x492) [0x7fbf66ec8fc2] --- ttnn::operations::conv::conv2d::OptimizedConvNew::create_program(std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&) const --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x449829) [0x7fbf66f2f829] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c0fda) [0x7fbf690a6fda] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c13e4) [0x7fbf690a73e4] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_worker_threadIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEENS3_3stl10StrongTypeIhNS_10QueueIdTagEEElNS5_15DeviceOperationISA_EENSB_13tensor_args_tESA_PNS4_2v07IDeviceEEEvT0_T1_RKT2_RKT3_RT4_RT5_+0x2ab) [0x7fbf690a672b] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x83) [0x7fbf690a63e3] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fbf690a62d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fbf690a549f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run_without_autoformat<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x448b70) [0x7fbf66f2eb70] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fbf690acfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fbf690ad7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d18optimized_conv_newERKN2tt8tt_metal6TensorES7_St8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbbRKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigERKNS4_12MemoryConfigENS4_8DataTypeESt5arrayIjLm4EEbRKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x6bd) [0x7fbf66ec66bd] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xec6) [0x7fbf66eb5f76] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fbf784d5365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fbf789449f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fbf7894493e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fbf7870fb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5584161a6b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x558416196a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x55841619c574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5584161b5172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x55841619197f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30]
test_conv2d.test_module[Conv2D17-[((1, 3, 224, 224), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
RuntimeError: TT_THROW @ /__w/tt-forge-fe/tt-forge-fe/third_party/tt-mlir/third_party/tt-metal/src/tt-metal/tt_metal/impl/allocator/bank_manager.cpp:132: tt::exception info: Out of Memory: Not enough space to allocate 12500992 B L1 buffer across 7 banks, where each bank needs to store 1785856 B backtrace: --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x15a7e5) [0x7fbf6a1e77e5] --- tt::tt_metal::BankManager::allocate_buffer(unsigned long, unsigned long, bool, CoreRangeSet const&, std::optional<unsigned int>) --- tt::tt_metal::Allocator::allocate_buffer(tt::tt_metal::v0::Buffer*) --- tt::tt_metal::v0::Buffer::allocate_impl() --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libtt_metal.so(+0x125e9f) [0x7fbf6a1b2e9f] --- tt::tt_metal::v0::Buffer::create(tt::tt_metal::v0::IDevice*, unsigned long, unsigned long, tt::tt_metal::BufferType, tt::tt_metal::TensorMemoryLayout, std::optional<tt::tt_metal::ShardSpecBuffer> const&, std::optional<bool>, std::optional<tt::tt_metal::SubDeviceId>) --- tt::tt_metal::tensor_impl::allocate_buffer_on_device(tt::tt_metal::v0::IDevice*, tt::tt_metal::TensorSpec const&) --- tt::tt_metal::create_device_tensor(tt::tt_metal::TensorSpec const&, tt::tt_metal::v0::IDevice*) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN2tt8tt_metal9operation29default_create_output_tensorsIN4ttnn10operations14sliding_window4halo19HaloDeviceOperationEEENS1_21program_output_helperIT_Xsr18has_create_programIS9_EE5valueEE4typeERKS9_RKSt6vectorINS0_6TensorESaISF_EERKSE_ISt8optionalISF_ESaISL_EE+0x193) [0x7fbf67ae5833] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xfff699) [0x7fbf67ae5699] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn16device_operation6detail23launch_on_single_deviceIN2tt8tt_metal9operation23OldInfraDeviceOperationISt6vectorINS4_6TensorESaIS8_EEEEEENT_21tensor_return_value_tENS3_3stl10StrongTypeIhNS_10QueueIdTagEEERKNSC_22operation_attributes_tERKNSC_13tensor_args_tE+0x3a) [0x7fbf690a639a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c02d7) [0x7fbf690a62d7] --- tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_return_value_t ttnn::device_operation::detail::invoke<tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > > >(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::operation_attributes_t const&, tt::tt_metal::operation::OldInfraDeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >::tensor_args_t const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25bf49f) [0x7fbf690a549f] --- std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > tt::tt_metal::operation::run<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(tt::tt_metal::operation::DeviceOperation<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >&&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0xffe69a) [0x7fbf67ae469a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c6fe2) [0x7fbf690acfe2] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x25c77cc) [0x7fbf690ad7cc] --- void tt::tt_metal::operation::launch_op_func<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > >(std::function<std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > (std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> > const&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > > const&, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > > const&)> const&, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >, std::vector<tt::tt_metal::Tensor, std::allocator<tt::tt_metal::Tensor> >&, std::vector<std::optional<tt::tt_metal::Tensor const>, std::allocator<std::optional<tt::tt_metal::Tensor const> > >, std::vector<std::optional<tt::tt_metal::Tensor>, std::allocator<std::optional<tt::tt_metal::Tensor> > >, bool) --- ttnn::operations::sliding_window::halo::halo_op(tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- ttnn::operations::sliding_window::halo::HaloOperation::invoke(tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::Tensor const&, ttnn::operations::sliding_window::SlidingWindowConfig const&, unsigned int, bool, bool, unsigned int, tt::tt_metal::MemoryConfig const&, bool) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(+0x436f8f) [0x7fbf66f1cf8f] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/_ttnn.so(_ZN4ttnn10operations4conv6conv2d6conv2dIN2tt8tt_metal2v07IDeviceEEESt5tupleIJNS5_6TensorEjjS9_St8optionalIS9_EEERKS9_SE_PT_jjjjjSt5arrayIjLm2EESI_SI_SI_jSA_ISD_ERKSA_IKNS2_12Conv2dConfigEERKSA_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERKSA_IKNS5_12MemoryConfigEE+0xcaa) [0x7fbf66eb5d5a] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/lib/libTTMLIRRuntime.so(+0xb8365) [0x7fbf784d5365] --- tt::runtime::ttnn::operations::conv::run(tt::target::ttnn::Conv2dOp const*, tt::runtime::ttnn::ProgramContext&) --- tt::runtime::ttnn::runProgram(tt::tt_metal::distributed::MeshDevice&, tt::runtime::Binary, unsigned int, std::vector<tt::tt_metal::Tensor*, std::allocator<tt::tt_metal::Tensor*> > const&) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::run_binary(tt::runtime::Binary&, int, std::vector<at::Tensor, std::allocator<at::Tensor> > const&) --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x3159f0) [0x7fbf789449f0] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0x31593e) [0x7fbf7894493e] --- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/forge/_C.so(+0xe0b75) [0x7fbf7870fb75] --- /opt/ttforge-toolchain/venv/bin/python(+0x18ab32) [0x5584161a6b32] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x59c7) [0x558416196a97] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0xc4) [0x55841619c574] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0x122) [0x5584161b5172] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_MakeTpCall+0x25b) [0x55841619d39b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x68ce) [0x55841619799e] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x8af) [0x55841619197f] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464] --- /opt/ttforge-toolchain/venv/bin/python(PyObject_Call+0xbb) [0x5584161b510b] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x6c0) [0x558416191790] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x2b60) [0x558416193c30] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x18db) [0x5584161929ab] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(+0x1984d1) [0x5584161b44d1] --- /opt/ttforge-toolchain/venv/bin/python(_PyEval_EvalFrameDefault+0x5632) [0x558416196702] --- /opt/ttforge-toolchain/venv/bin/python(_PyFunction_Vectorcall+0x7c) [0x5584161a738c] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_FastCallDictTstate+0x16d) [0x55841619c61d] --- /opt/ttforge-toolchain/venv/bin/python(_PyObject_Call_Prepend+0x5c) [0x5584161b162c] --- /opt/ttforge-toolchain/venv/bin/python(+0x29d464) [0x5584162b9464]
test_conv2d.test_module[Conv2D516-[((1, 64, 147, 147), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[144.5902709961, 146.8499145508, 148.8105163574, ..., 149.3406219482, 144.7736663818, 150.6027069092], [154.3025207520, 151.8691101074, 145.3336791992, ..., 149.5804138184, 149.9938049316, 151.3875122070], [148.8136138916, 149.7991638184, 149.2318725586, ..., 153.0614013672, 147.5971527100, 157.3221740723], ..., [148.7757263184, 154.6059265137, 147.5536651611, ..., 143.4194641113, 144.4597167969, 142.1168670654], [151.3589477539, 147.3621215820, 148.7238616943, ..., 147.4233093262, 147.5432128906, 149.1204376221], [147.1672363281, 147.2913208008, 146.9279785156, ..., 145.6753234863, 149.8157806396, 147.3130798340]], [[140.7183990479, 144.2240447998, 138.4780426025, ..., 143.4066162109, 139.7475585938, 148.9764404297], [148.4001464844, 145.3047637939, 140.8634185791, ..., 144.0821533203, 144.9893646240, 145.4881896973], [144.0110778809, 147.9011230469, 145.4527893066, ..., 146.0332031250, 149.6197204590, 147.2288208008], ..., [141.2115783691, 145.7017059326, 145.4075775146, ..., 139.7186889648, 137.5713958740, 140.2689514160], [149.5879364014, 147.2991943359, 147.5192565918, ..., 142.9397888184, 141.5360717773, 139.4499053955], [141.3296508789, 139.4872741699, 142.7448577881, ..., 143.9296722412, 138.2051086426, 142.2441406250]], [[140.0239257812, 144.8148040771, 146.0733795166, ..., 146.1656341553, 143.6527709961, 151.0458221436], [150.7276306152, 146.1761627197, 138.6366577148, ..., 148.8565063477, 147.3908691406, 151.4685974121], [143.5761260986, 147.9291381836, 146.8161926270, ..., 149.2856750488, 146.2793731689, 144.9579467773], ..., [144.2278137207, 147.8683166504, 151.5810852051, ..., 140.3446350098, 140.5933227539, 141.3679962158], [151.0900573730, 148.8614196777, 141.8109436035, ..., 147.4037475586, 144.2275543213, 143.9185791016], [145.4204711914, 142.1257324219, 145.7842102051, ..., 143.2630615234, 145.9328918457, 146.4102172852]], ..., [[134.7500610352, 143.8243103027, 139.1241302490, ..., 139.3365325928, 139.1329193115, 149.3362884521], [148.0198974609, 143.2199554443, 134.1446685791, ..., 144.3829650879, 138.9014129639, 142.9081726074], [140.7189025879, 141.4165649414, 142.2360382080, ..., 144.0924835205, 143.3411254883, 144.8575439453], ..., [140.1158447266, 142.4668579102, 142.7185058594, ..., 132.8171997070, 134.7519836426, 137.1859436035], [144.0683746338, 140.9325866699, 142.5516052246, ..., 142.4145050049, 140.0835571289, 143.4492187500], [138.6912231445, 137.5113983154, 136.3057098389, ..., 140.7984466553, 138.6293487549, 147.8158874512]], [[140.1265258789, 146.6797485352, 147.7294311523, ..., 145.9745025635, 146.7216033936, 151.2752075195], [152.9240264893, 149.2511291504, 143.1197509766, ..., 143.7355346680, 145.5089416504, 151.1295318604], [146.5888977051, 147.2341918945, 150.8454437256, ..., 147.5010681152, 149.2423095703, 147.6086578369], ..., [144.1636962891, 146.8798675537, 144.5799865723, ..., 136.1078643799, 137.8281097412, 142.2316741943], [149.4997100830, 147.0645294189, 147.9990997314, ..., 146.5351562500, 142.9446105957, 140.9439392090], [140.8222961426, 141.2861633301, 140.4582214355, ..., 144.4272918701, 143.2029266357, 145.1761322021]], [[144.1779174805, 145.3422851562, 143.0584411621, ..., 147.6744995117, 146.9033508301, 154.4301147461], [157.9370269775, 150.3078155518, 148.8755187988, ..., 149.0837707520, 148.6465454102, 154.1902465820], [146.4096984863, 152.5246887207, 153.4474792480, ..., 153.1374053955, 152.7330932617, 148.0358581543], ..., [147.6069030762, 154.3356018066, 151.9167633057, ..., 141.9215240479, 142.0694580078, 143.5207214355], [152.3010559082, 153.6093902588, 152.7453460693, ..., 153.0643920898, 146.0541839600, 148.6673278809], [150.5097656250, 140.0808258057, 147.7681427002, ..., 150.7264556885, 145.5346374512, 152.1295471191]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[142., 146., 147., ..., 149., 142., 147.], [153., 150., 143., ..., 148., 148., 150.], [147., 146., 146., ..., 149., 146., 156.], ..., [145., 151., 145., ..., 142., 142., 141.], [150., 146., 147., ..., 145., 146., 147.], [145., 144., 144., ..., 142., 146., 145.]], [[139., 143., 137., ..., 140., 136., 147.], [147., 144., 139., ..., 143., 143., 141.], [141., 146., 142., ..., 143., 148., 144.], ..., [140., 144., 142., ..., 138., 136., 139.], [148., 144., 145., ..., 142., 139., 137.], [140., 137., 141., ..., 142., 137., 140.]], [[139., 143., 144., ..., 145., 141., 148.], [149., 145., 137., ..., 146., 144., 149.], [141., 145., 144., ..., 148., 143., 142.], ..., [141., 146., 148., ..., 139., 139., 140.], [149., 145., 140., ..., 145., 142., 142.], [143., 140., 143., ..., 140., 144., 144.]], ..., [[133., 141., 138., ..., 138., 139., 149.], [146., 141., 132., ..., 142., 137., 142.], [140., 140., 139., ..., 143., 141., 142.], ..., [139., 140., 140., ..., 131., 132., 134.], [143., 140., 140., ..., 139., 139., 142.], [135., 136., 135., ..., 138., 138., 147.]], [[138., 144., 146., ..., 146., 144., 150.], [149., 147., 141., ..., 142., 143., 148.], [146., 144., 149., ..., 144., 147., 146.], ..., [143., 145., 143., ..., 134., 137., 141.], [148., 146., 147., ..., 145., 141., 139.], [139., 140., 140., ..., 143., 141., 143.]], [[141., 143., 141., ..., 146., 145., 153.], [155., 149., 147., ..., 146., 145., 151.], [145., 150., 152., ..., 150., 149., 145.], ..., [146., 152., 150., ..., 140., 138., 143.], [149., 151., 150., ..., 153., 144., 147.], [149., 138., 146., ..., 150., 144., 150.]]]])
test_conv2d.test_module[Conv2D518-[((1, 64, 73, 73), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[152.2259521484, 145.7772216797, 136.8718109131, ..., 140.4228820801, 139.6538391113, 139.1413879395], [151.6833038330, 143.0866699219, 140.6795654297, ..., 143.0262451172, 144.8131408691, 139.7745208740], [145.7750396729, 146.3048858643, 143.8915252686, ..., 141.5709686279, 148.6717681885, 145.0984191895], ..., [140.6621551514, 145.6004638672, 144.2947235107, ..., 143.6639862061, 140.7307739258, 142.5148315430], [145.8384704590, 149.2471313477, 145.4357299805, ..., 141.1892547607, 141.5985565186, 142.9531402588], [145.2830200195, 151.6161956787, 146.1196289062, ..., 139.2727050781, 140.4277954102, 139.2373046875]], [[146.7124328613, 151.6223449707, 136.7942047119, ..., 137.8146514893, 138.8585815430, 139.5962219238], [149.2643737793, 144.1325531006, 143.4167480469, ..., 142.7655487061, 142.1508178711, 141.0497589111], [145.1377716064, 145.3029022217, 143.3576354980, ..., 139.8086547852, 145.8719024658, 142.0795593262], ..., [145.3661804199, 143.0798950195, 137.8924407959, ..., 139.2702026367, 138.6483917236, 139.6412658691], [145.1908874512, 145.6476593018, 142.2647399902, ..., 138.0929718018, 138.2590789795, 144.4056701660], [143.4348297119, 143.0312500000, 142.1493835449, ..., 140.3581237793, 146.5321350098, 137.0543212891]], [[150.0823059082, 143.9369964600, 138.4826049805, ..., 136.4377441406, 139.2085266113, 137.6575622559], [150.1298828125, 145.9816284180, 142.4093170166, ..., 142.6005249023, 146.3442382812, 137.2539978027], [140.4223480225, 143.3133087158, 141.5793762207, ..., 137.3434906006, 145.7197570801, 142.0828399658], ..., [144.6627807617, 146.5065765381, 140.1580352783, ..., 141.2983856201, 141.3062591553, 143.0518188477], [141.2523040771, 147.6970520020, 142.5403289795, ..., 140.4929199219, 140.3352813721, 138.5058288574], [138.2370147705, 145.4068450928, 146.2414245605, ..., 141.8226470947, 137.8505554199, 140.3447723389]], ..., [[147.7367858887, 145.0466461182, 140.2164001465, ..., 140.2339477539, 139.2319488525, 136.4789733887], [145.3736114502, 145.5704498291, 143.2474975586, ..., 139.9947052002, 142.2647705078, 138.7484893799], [144.3772277832, 145.1488494873, 145.6206512451, ..., 137.1409759521, 145.8037719727, 145.4990539551], ..., [139.1537628174, 142.3866882324, 141.2739562988, ..., 141.3141174316, 137.8016662598, 139.2997436523], [144.3430633545, 141.6740722656, 144.3353576660, ..., 138.9774932861, 137.9252014160, 137.9963836670], [142.9683685303, 141.9563598633, 144.3373107910, ..., 140.6262207031, 138.6582183838, 135.2939147949]], [[154.3753051758, 149.1354370117, 141.2576293945, ..., 142.9515838623, 141.2592773438, 136.3095397949], [152.1967926025, 153.2928771973, 146.0510864258, ..., 145.8646545410, 150.6550292969, 143.8346862793], [152.5205688477, 144.7463073730, 146.8518524170, ..., 143.7776489258, 151.7974853516, 145.5718383789], ..., [145.7089538574, 148.1030883789, 141.7923431396, ..., 146.3827819824, 145.0093231201, 144.0107727051], [142.0967102051, 152.5759429932, 148.8595275879, ..., 143.5588836670, 140.8418426514, 144.6685791016], [143.5753326416, 148.1425628662, 145.0681304932, ..., 146.1748352051, 140.6717224121, 141.0068664551]], [[157.2637176514, 155.8987426758, 145.4243927002, ..., 143.9485015869, 150.4982299805, 149.9706420898], [156.8666076660, 156.2886199951, 147.1239624023, ..., 148.1587219238, 149.9689178467, 149.1417541504], [154.3150024414, 151.2000885010, 151.9583129883, ..., 150.6881713867, 150.9894714355, 152.1381683350], ..., [151.2052307129, 148.8263397217, 149.0452270508, ..., 153.9623718262, 146.6357574463, 146.3478393555], [152.0362243652, 157.2518310547, 149.7592773438, ..., 148.9167022705, 146.1963958740, 145.5577239990], [150.8255310059, 153.9378051758, 151.1777191162, ..., 143.6190948486, 146.9277343750, 143.7196044922]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[152., 145., 135., ..., 138., 138., 138.], [149., 141., 138., ..., 141., 143., 138.], [144., 144., 142., ..., 138., 147., 142.], ..., [138., 144., 143., ..., 143., 140., 141.], [143., 148., 144., ..., 140., 138., 139.], [144., 148., 143., ..., 138., 138., 137.]], [[146., 149., 135., ..., 137., 137., 138.], [148., 143., 142., ..., 140., 141., 139.], [143., 143., 141., ..., 139., 142., 139.], ..., [144., 141., 136., ..., 137., 136., 140.], [144., 143., 141., ..., 135., 135., 144.], [141., 141., 141., ..., 138., 144., 136.]], [[148., 142., 136., ..., 134., 136., 136.], [147., 145., 141., ..., 139., 145., 135.], [139., 142., 142., ..., 136., 143., 139.], ..., [142., 144., 136., ..., 139., 140., 141.], [139., 146., 140., ..., 139., 137., 136.], [137., 143., 145., ..., 141., 135., 139.]], ..., [[145., 141., 138., ..., 137., 136., 133.], [141., 143., 141., ..., 138., 140., 138.], [142., 143., 142., ..., 135., 143., 144.], ..., [137., 140., 138., ..., 139., 137., 136.], [141., 138., 142., ..., 137., 137., 136.], [141., 139., 141., ..., 137., 136., 132.]], [[154., 146., 140., ..., 142., 140., 134.], [152., 152., 145., ..., 144., 149., 142.], [151., 144., 145., ..., 141., 148., 144.], ..., [144., 146., 140., ..., 143., 143., 142.], [139., 150., 147., ..., 141., 138., 143.], [142., 146., 142., ..., 144., 139., 139.]], [[156., 154., 145., ..., 140., 150., 149.], [154., 155., 146., ..., 146., 148., 146.], [152., 150., 150., ..., 149., 149., 150.], ..., [150., 145., 148., ..., 152., 145., 143.], [149., 154., 147., ..., 147., 142., 143.], [148., 151., 147., ..., 141., 145., 141.]]]])
test_conv2d.test_module[Conv2D521-[((1, 192, 71, 71), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[446.1500244141, 439.1384277344, 434.7012329102, ..., 444.4458007812, 445.5368652344, 421.7438354492], [440.6470947266, 445.6110229492, 437.2182617188, ..., 439.5468444824, 429.5156860352, 431.2485351562], [432.5223388672, 441.4844665527, 427.5327758789, ..., 426.8598327637, 426.2634277344, 426.5023498535], ..., [436.9984130859, 428.2519531250, 432.4371948242, ..., 431.2190856934, 422.9771728516, 431.0124206543], [431.6505126953, 432.1033630371, 422.4633483887, ..., 422.2445068359, 426.3516845703, 434.2154846191], [440.1676940918, 429.5302734375, 428.5906982422, ..., 422.3389892578, 422.3145751953, 431.9897155762]], [[450.5714721680, 431.4641113281, 435.4294433594, ..., 444.5223999023, 449.1513061523, 424.6431274414], [444.6395568848, 437.6935424805, 431.0649719238, ..., 450.2823181152, 435.8484802246, 424.6562194824], [434.7048950195, 438.0634155273, 428.8528747559, ..., 431.3271484375, 425.4387512207, 428.4915771484], ..., [430.4140930176, 425.7988586426, 430.9104919434, ..., 440.3789978027, 426.6936035156, 431.3436279297], [429.0862731934, 434.0320129395, 417.7902526855, ..., 422.2330017090, 418.9494628906, 438.0584106445], [436.3234558105, 428.4629821777, 426.1377258301, ..., 424.9827880859, 419.9928894043, 433.1639099121]], [[453.0538940430, 448.7878417969, 439.6736145020, ..., 447.6927490234, 453.1119384766, 426.3741760254], [451.5824890137, 443.2856750488, 439.6746520996, ..., 448.9715270996, 438.2503662109, 431.6139526367], [434.7268676758, 440.1856079102, 424.9761962891, ..., 438.2861938477, 429.3561401367, 427.5081787109], ..., [441.4554748535, 430.2236328125, 434.4478149414, ..., 439.5084228516, 424.2331237793, 430.3131713867], [437.1467895508, 438.3942871094, 423.3197937012, ..., 425.7505798340, 423.9236755371, 439.2760009766], [443.4287414551, 433.0767211914, 432.6028442383, ..., 426.0621948242, 426.8213500977, 429.5514831543]], ..., [[453.8679199219, 446.4808349609, 434.6714782715, ..., 455.5363159180, 444.9446411133, 433.0401611328], [452.1379089355, 444.3341674805, 445.6447448730, ..., 449.3180541992, 438.0433044434, 430.9374084473], [435.1299133301, 450.6659545898, 439.9477539062, ..., 435.2498168945, 433.1172180176, 436.8755187988], ..., [443.2763061523, 434.5806579590, 436.6311340332, ..., 439.7202758789, 427.7365417480, 435.8312683105], [439.5126342773, 443.5434570312, 422.8626098633, ..., 427.2471008301, 430.1891479492, 432.3753967285], [442.5857849121, 437.5104064941, 439.4240722656, ..., 428.2677307129, 429.5154724121, 439.9343566895]], [[451.0897827148, 439.7805786133, 436.0215759277, ..., 445.0819091797, 446.1299133301, 428.2847290039], [437.6216430664, 436.9979858398, 433.5463256836, ..., 445.3563537598, 434.3789672852, 423.7323303223], [429.3495788574, 437.1765747070, 427.9952392578, ..., 432.7473754883, 430.1035156250, 425.7379760742], ..., [430.3339538574, 418.1119384766, 430.6286315918, ..., 433.4542236328, 423.3659667969, 425.9112548828], [433.1029663086, 438.1243286133, 417.4966125488, ..., 421.3308105469, 416.8927001953, 435.6392517090], [441.5573730469, 428.5757141113, 430.2178649902, ..., 425.6438598633, 417.5133972168, 433.5839843750]], [[449.6306457520, 430.5096130371, 429.0747375488, ..., 443.6842041016, 440.6119384766, 419.7158813477], [439.8434753418, 432.2814331055, 431.9724121094, ..., 438.8265075684, 429.8711853027, 428.9610900879], [432.6672363281, 438.9446105957, 422.2741088867, ..., 427.5865173340, 420.1713256836, 429.8716735840], ..., [430.6725158691, 426.7984924316, 431.3894653320, ..., 432.4364929199, 415.1751708984, 429.2265930176], [428.8142089844, 432.2534790039, 409.3339538574, ..., 419.8291320801, 418.7741088867, 433.3815917969], [437.2994079590, 429.8685302734, 426.2242431641, ..., 427.5977783203, 418.4594116211, 431.1556701660]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[454., 438., 430., ..., 440., 448., 416.], [444., 442., 442., ..., 434., 426., 426.], [426., 436., 430., ..., 428., 426., 424.], ..., [428., 422., 436., ..., 438., 424., 428.], [434., 428., 420., ..., 418., 424., 438.], [440., 430., 436., ..., 410., 418., 430.]], [[448., 430., 434., ..., 442., 454., 422.], [442., 434., 428., ..., 448., 430., 416.], [432., 440., 424., ..., 426., 424., 424.], ..., [426., 426., 430., ..., 436., 422., 434.], [422., 428., 416., ..., 418., 412., 432.], [436., 422., 426., ..., 420., 422., 434.]], [[450., 438., 430., ..., 446., 456., 420.], [456., 444., 430., ..., 440., 428., 432.], [430., 436., 430., ..., 436., 424., 422.], ..., [444., 432., 432., ..., 438., 418., 432.], [438., 440., 424., ..., 426., 428., 438.], [448., 430., 430., ..., 428., 434., 422.]], ..., [[454., 448., 434., ..., 456., 446., 436.], [452., 444., 442., ..., 444., 438., 430.], [432., 454., 438., ..., 428., 428., 438.], ..., [446., 436., 436., ..., 440., 430., 430.], [438., 446., 426., ..., 428., 432., 432.], [438., 440., 428., ..., 428., 432., 444.]], [[448., 436., 438., ..., 452., 442., 432.], [444., 438., 436., ..., 448., 426., 416.], [434., 432., 424., ..., 432., 430., 424.], ..., [430., 414., 436., ..., 436., 418., 428.], [424., 442., 426., ..., 412., 414., 438.], [442., 438., 424., ..., 418., 420., 430.]], [[450., 428., 432., ..., 438., 440., 410.], [440., 426., 430., ..., 434., 428., 432.], [434., 440., 414., ..., 426., 414., 428.], ..., [432., 432., 426., ..., 432., 412., 424.], [432., 430., 404., ..., 416., 420., 436.], [438., 436., 424., ..., 420., 414., 430.]]]])
test_conv2d.test_module[Conv2D524-[((1, 384, 35, 35), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[866.4607543945, 863.1016845703, 861.1956176758, ..., 855.0999755859, 840.2875366211, 847.4694213867], [875.8908691406, 877.8522338867, 859.6835327148, ..., 848.9241333008, 850.2929687500, 858.7661132812], [868.1459960938, 869.0175170898, 869.6133422852, ..., 854.4506835938, 861.1555175781, 877.7508544922], ..., [864.5763549805, 878.4547729492, 872.5306396484, ..., 842.3853759766, 860.4354858398, 849.8452758789], [868.0845947266, 862.4204101562, 862.9453125000, ..., 864.3536376953, 877.8131713867, 868.6020507812], [871.2568969727, 879.8212890625, 878.2059326172, ..., 872.2755737305, 869.0942993164, 879.1583862305]], [[868.2850341797, 870.1876831055, 851.5877075195, ..., 854.5398559570, 840.5253906250, 852.4250488281], [869.6328735352, 867.2083740234, 859.9771728516, ..., 847.4662475586, 836.6459960938, 863.3942871094], [871.5944213867, 870.6664428711, 870.5274658203, ..., 853.5540771484, 862.3122558594, 875.3737182617], ..., [878.3991699219, 876.8424072266, 869.4636230469, ..., 844.9429931641, 851.4466552734, 842.9074707031], [860.2490234375, 873.5955810547, 859.1199951172, ..., 858.1122436523, 868.3363037109, 867.8045043945], [865.4127197266, 872.1903076172, 874.1329345703, ..., 859.1617431641, 874.2640991211, 881.2451171875]], [[888.0039672852, 881.4798583984, 883.1144409180, ..., 870.9652099609, 858.5319824219, 874.5659790039], [892.3176269531, 886.7056884766, 869.8299560547, ..., 855.6347045898, 854.0805664062, 871.8778076172], [878.8856811523, 890.1937255859, 873.0161132812, ..., 853.2692871094, 878.5581054688, 881.8140869141], ..., [887.5664672852, 881.7894897461, 885.1113281250, ..., 852.0238647461, 868.1290893555, 864.4165039062], [882.2729492188, 898.7322387695, 882.9472656250, ..., 876.7220458984, 889.6724243164, 877.4081420898], [882.8354492188, 900.7934570312, 893.5081787109, ..., 871.3879394531, 871.6428222656, 907.1149902344]], ..., [[866.0534057617, 873.6550903320, 859.7093505859, ..., 854.6380615234, 847.8020629883, 853.6875610352], [877.1881713867, 877.3087768555, 873.7102050781, ..., 843.3227539062, 838.6889648438, 858.4749145508], [866.7683715820, 881.3969726562, 876.8781127930, ..., 847.8245849609, 862.9209594727, 883.8549804688], ..., [864.8294067383, 882.9351806641, 869.4398803711, ..., 848.6583251953, 855.0922851562, 840.2235717773], [865.5087890625, 870.6390991211, 856.5816040039, ..., 873.1359252930, 876.6809692383, 865.3837890625], [875.5211181641, 882.4803466797, 875.6945800781, ..., 871.4721069336, 874.3912353516, 883.8248901367]], [[890.4672851562, 871.8563232422, 873.5863647461, ..., 872.9142456055, 864.4791870117, 872.6688232422], [895.8986816406, 878.6241455078, 876.0815429688, ..., 870.8352661133, 862.5122070312, 864.1354370117], [879.8025512695, 895.7088012695, 887.4453125000, ..., 866.4882812500, 884.2217407227, 888.7052001953], ..., [880.4241943359, 882.7203369141, 883.0754394531, ..., 860.8107299805, 871.7085571289, 868.9783935547], [879.2627563477, 887.0874023438, 882.5967407227, ..., 876.7019042969, 891.1706542969, 875.5416870117], [878.4302368164, 889.8891601562, 895.8449096680, ..., 883.0540161133, 889.5223999023, 898.1984252930]], [[870.4879760742, 862.5693969727, 857.4490356445, ..., 855.5866699219, 853.5473022461, 862.2806396484], [876.2677612305, 883.4030151367, 872.8103027344, ..., 854.5828857422, 849.0673828125, 859.8843994141], [868.9296875000, 883.5654907227, 872.2229614258, ..., 858.1086425781, 869.1034545898, 886.0482177734], ..., [875.6426391602, 888.0510253906, 872.6620483398, ..., 839.7253417969, 869.1248779297, 855.5281982422], [869.0089111328, 876.9663085938, 868.3325805664, ..., 863.1636352539, 874.4519042969, 865.7647094727], [866.9956665039, 881.1237182617, 882.6040039062, ..., 870.8355712891, 871.3969726562, 887.7743530273]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[868., 856., 872., ..., 860., 864., 872.], [884., 872., 868., ..., 864., 852., 864.], [884., 872., 880., ..., 868., 856., 872.], ..., [876., 880., 872., ..., 860., 868., 860.], [880., 860., 872., ..., 864., 892., 872.], [884., 884., 864., ..., 884., 868., 892.]], [[872., 864., 848., ..., 864., 868., 860.], [872., 860., 872., ..., 860., 856., 868.], [872., 860., 880., ..., 868., 868., 876.], ..., [880., 876., 876., ..., 864., 864., 856.], [860., 872., 880., ..., 856., 868., 876.], [852., 876., 876., ..., 872., 868., 892.]], [[880., 876., 868., ..., 864., 852., 876.], [904., 888., 884., ..., 868., 868., 860.], [872., 892., 860., ..., 868., 872., 876.], ..., [892., 884., 876., ..., 860., 864., 864.], [880., 876., 880., ..., 872., 884., 864.], [880., 888., 888., ..., 872., 872., 900.]], ..., [[856., 860., 864., ..., 864., 860., 848.], [872., 860., 864., ..., 864., 844., 852.], [872., 880., 876., ..., 844., 860., 880.], ..., [884., 880., 880., ..., 872., 864., 856.], [876., 864., 856., ..., 868., 888., 872.], [872., 872., 864., ..., 876., 868., 876.]], [[876., 872., 868., ..., 876., 860., 872.], [896., 872., 872., ..., 872., 868., 864.], [884., 892., 876., ..., 872., 884., 884.], ..., [884., 892., 880., ..., 872., 884., 876.], [888., 884., 888., ..., 872., 888., 876.], [872., 900., 892., ..., 880., 888., 896.]], [[864., 852., 860., ..., 848., 856., 864.], [884., 876., 872., ..., 856., 852., 856.], [860., 868., 856., ..., 860., 880., 880.], ..., [880., 876., 876., ..., 848., 876., 856.], [868., 876., 876., ..., 860., 868., 864.], [872., 884., 880., ..., 864., 868., 884.]]]])
test_conv2d.test_module[Conv2D526-[((1, 224, 35, 35), torch.float32)]]: forge/test/models_ops/test_conv2d.py#L56786
ValueError: Data mismatch -> AutomaticValueChecker (compare_with_golden): framework_model=tensor([[[[500.3654174805, 489.4805603027, 488.6660461426, ..., 498.1959838867, 485.7237548828, 486.2214355469], [503.1026000977, 495.0940551758, 496.3488159180, ..., 490.6144104004, 489.8197021484, 489.8639526367], [498.7122192383, 494.7256774902, 487.2427978516, ..., 487.9171142578, 495.7424011230, 503.2261352539], ..., [498.4995422363, 518.2473144531, 501.5747070312, ..., 495.7087707520, 495.1421203613, 481.3145751953], [495.4401245117, 508.5326538086, 506.0977783203, ..., 495.9432983398, 509.0796813965, 495.8168029785], [490.7166442871, 501.0508728027, 505.4311218262, ..., 497.9173889160, 504.0026855469, 511.4383850098]], [[523.4201660156, 508.6379394531, 507.7438964844, ..., 516.4188842773, 496.1733398438, 508.5603637695], [518.5769042969, 515.1815185547, 513.1820068359, ..., 503.7814941406, 504.7009277344, 501.2026367188], [516.2856445312, 514.2278442383, 512.8794555664, ..., 509.4266967773, 516.1943359375, 511.4126586914], ..., [519.9569702148, 523.6143798828, 522.6581420898, ..., 512.8380126953, 522.3416137695, 503.4331359863], [507.0757751465, 521.7188110352, 518.8157348633, ..., 512.7855224609, 525.6205444336, 506.8906555176], [513.5720214844, 518.0313720703, 518.3280029297, ..., 520.2257690430, 526.4735717773, 524.2275390625]], [[517.4383544922, 492.0398559570, 502.1506958008, ..., 497.5055541992, 492.6846313477, 497.6663513184], [510.4726562500, 498.3111267090, 509.2691650391, ..., 498.5928039551, 493.6994628906, 495.6733398438], [510.4765014648, 503.6241455078, 497.2258911133, ..., 500.2603454590, 504.0606079102, 509.7190551758], ..., [507.8261108398, 517.9183349609, 515.7430419922, ..., 502.1657409668, 510.3342590332, 494.9484252930], [507.4068298340, 513.5440063477, 500.5324707031, ..., 501.5933837891, 517.8280639648, 502.3204650879], [507.5400085449, 518.0346679688, 508.5068054199, ..., 515.4729003906, 511.8546752930, 514.9834594727]], ..., [[513.7336425781, 496.5385437012, 499.4909362793, ..., 498.3542480469, 491.6601562500, 497.3191223145], [514.0242309570, 502.1476440430, 507.0021362305, ..., 496.7256469727, 498.2814941406, 507.7484741211], [507.5863647461, 512.1271972656, 508.1876220703, ..., 497.1118469238, 509.3463745117, 511.1384887695], ..., [513.8402709961, 523.5373535156, 516.0033569336, ..., 502.9669799805, 510.3091735840, 494.7480468750], [498.4073181152, 507.9765014648, 509.7408447266, ..., 500.9190979004, 523.5088500977, 503.1334228516], [510.1004333496, 515.4464111328, 505.5420837402, ..., 514.4184570312, 521.7769775391, 521.4561767578]], [[515.0070800781, 509.7843017578, 509.4789428711, ..., 513.8732299805, 494.9671020508, 509.1306457520], [531.3634643555, 509.8542175293, 510.9259643555, ..., 507.6829223633, 516.3023071289, 502.5185546875], [521.5579833984, 516.9341430664, 512.7404174805, ..., 517.3096313477, 520.7438964844, 511.6298522949], ..., [520.1965942383, 532.0807495117, 530.5952148438, ..., 514.3909912109, 517.5526123047, 500.3167114258], [509.4294128418, 520.7053833008, 517.2093505859, ..., 513.5693969727, 534.2973022461, 505.8469848633], [508.5120239258, 521.9262695312, 518.3895874023, ..., 518.3444213867, 525.2760009766, 526.2166748047]], [[508.9140625000, 502.6702880859, 506.0244140625, ..., 500.0993347168, 491.6978759766, 498.5715026855], [509.7070312500, 504.4248352051, 506.1205444336, ..., 498.9411621094, 500.4955139160, 498.3578796387], [510.6956787109, 505.6959228516, 501.4794921875, ..., 496.7246704102, 507.0609130859, 508.7588806152], ..., [509.0236511230, 523.0482177734, 517.4687500000, ..., 505.1234436035, 516.2899169922, 499.3425292969], [501.1952819824, 515.9888916016, 505.2563171387, ..., 502.7287597656, 519.1704101562, 506.7722778320], [503.6124572754, 515.0338134766, 503.2249755859, ..., 514.8375244141, 521.2164306641, 521.6481933594]]]], grad_fn=<ConvolutionBackward0>), compiled_model=tensor([[[[498., 494., 486., ..., 504., 480., 486.], [502., 498., 492., ..., 490., 484., 490.], [496., 496., 488., ..., 480., 498., 502.], ..., [502., 516., 498., ..., 490., 498., 480.], [492., 502., 502., ..., 496., 510., 504.], [492., 506., 502., ..., 498., 500., 510.]], [[524., 508., 504., ..., 512., 494., 496.], [520., 512., 512., ..., 492., 508., 496.], [516., 508., 508., ..., 516., 516., 512.], ..., [512., 524., 512., ..., 498., 524., 498.], [504., 524., 524., ..., 508., 520., 506.], [512., 512., 520., ..., 516., 528., 524.]], [[512., 486., 506., ..., 492., 492., 498.], [506., 498., 508., ..., 498., 492., 490.], [508., 494., 488., ..., 504., 512., 508.], ..., [500., 508., 516., ..., 502., 500., 494.], [512., 512., 498., ..., 500., 512., 508.], [510., 524., 498., ..., 502., 508., 520.]], ..., [[516., 492., 498., ..., 496., 496., 506.], [508., 502., 502., ..., 494., 494., 502.], [506., 504., 506., ..., 498., 502., 512.], ..., [512., 528., 512., ..., 494., 506., 486.], [492., 510., 508., ..., 498., 524., 502.], [504., 510., 516., ..., 512., 524., 524.]], [[516., 508., 520., ..., 508., 492., 512.], [520., 508., 508., ..., 504., 516., 508.], [520., 512., 510., ..., 520., 532., 512.], ..., [520., 536., 532., ..., 512., 512., 496.], [502., 520., 532., ..., 508., 528., 504.], [504., 516., 520., ..., 524., 520., 528.]], [[516., 500., 502., ..., 510., 488., 500.], [508., 510., 504., ..., 494., 500., 502.], [516., 504., 502., ..., 502., 506., 512.], ..., [506., 528., 516., ..., 504., 512., 498.], [508., 520., 502., ..., 500., 510., 508.], [498., 512., 506., ..., 512., 512., 524.]]]])
test / run-tests (n150, 2)
Process completed with exit code 1.
check-all-green
Process completed with exit code 1.
pre-commit / pre-commit
The `python-version` input is not set. The version of Python currently in `PATH` will be used.

Artifacts

Produced during runtime
Name Size
forge-wheel
253 MB
test-log-n150-1
490 KB
test-log-n150-2
538 KB
test-log-n300-1
490 KB
test-log-n300-2
538 KB
test-reports-n150-1
627 KB
test-reports-n150-2
659 KB
test-reports-n300-1
626 KB
test-reports-n300-2
653 KB