start 2026-10-04T22:04:02Z
cmd: python -m model_zoo.train deepsets --dataset massspecgym --candidate-map official_candidates_by_mass --split smoke_formula_seed1_valonly --config smoke --workers 3 --no-logger --wandb-run-name smoke_deepsets --result-path /Users/timrichardson/Documents/projects/personal/blog/rewire.it/workbench/outputs/blog-post-msms-model-selection-20261003-230223/runs/S03-smoke-deepsets-20261004T220402Z/result.json
Seed set to 42
GPU available: True (mps), used: True
TPU available: False, using: 0 TPU cores
💡 Tip: For seamless cloud logging and experiment tracking, try installing [litlogger](https://pypi.org/project/litlogger/) to enable LitLogger, which logs metrics and artifacts automatically to the Lightning Experiments platform.

  | Name     | Type            | Params | Mode  | FLOPs
-------------------------------------------------------------
0 | backbone | DeepSetBackbone | 1.9 M  | train | 0    
-------------------------------------------------------------
1.9 M     Trainable params
0         Non-trainable params
1.9 M     Total params
7.470     Total estimated model params size (MB)
27        Modules in train mode
0         Modules in eval mode
0         Total Flops

Sanity Checking: |          | 0/? [00:00<?, ?it/s]
Sanity Checking: |          | 0/? [00:00<?, ?it/s]
Sanity Checking DataLoader 0:   0%|          | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0:  50%|█████     | 1/2 [00:00<00:00,  1.45it/s]
Sanity Checking DataLoader 0: 100%|██████████| 2/2 [00:00<00:00,  2.81it/s]
                                                                           

Training: |          | 0/? [00:00<?, ?it/s]
Training: |          | 0/? [00:00<?, ?it/s]
Epoch 0:   0%|          | 0/188 [00:00<?, ?it/s]
Epoch 0:   1%|          | 1/188 [00:00<02:33,  1.21it/s]
Epoch 0:   1%|          | 1/188 [00:00<02:33,  1.21it/s]
Epoch 0:   1%|          | 2/188 [00:00<01:17,  2.40it/s]
Epoch 0:   1%|          | 2/188 [00:00<01:17,  2.40it/s]
Epoch 0:   2%|▏         | 3/188 [00:00<00:51,  3.56it/s]
Epoch 0:   2%|▏         | 3/188 [00:00<00:51,  3.56it/s]
Epoch 0:   2%|▏         | 4/188 [00:00<00:39,  4.69it/s]
Epoch 0:   2%|▏         | 4/188 [00:00<00:39,  4.69it/s]
Epoch 0:   3%|▎         | 5/188 [00:00<00:31,  5.78it/s]
Epoch 0:   3%|▎         | 5/188 [00:00<00:31,  5.78it/s]
Epoch 0:   3%|▎         | 6/188 [00:00<00:26,  6.85it/s]
Epoch 0:   3%|▎         | 6/188 [00:00<00:26,  6.85it/s]
Epoch 0:   4%|▎         | 7/188 [00:00<00:22,  7.90it/s]
Epoch 0:   4%|▎         | 7/188 [00:00<00:22,  7.90it/s]
Epoch 0:   4%|▍         | 8/188 [00:00<00:20,  8.80it/s]
Epoch 0:   4%|▍         | 8/188 [00:00<00:20,  8.80it/s]
Epoch 0:   5%|▍         | 9/188 [00:00<00:18,  9.80it/s]
Epoch 0:   5%|▍         | 9/188 [00:00<00:18,  9.80it/s]
Epoch 0:   5%|▌         | 10/188 [00:00<00:16, 10.74it/s]
Epoch 0:   5%|▌         | 10/188 [00:00<00:16, 10.74it/s]
Epoch 0:   6%|▌         | 11/188 [00:00<00:15, 11.20it/s]
Epoch 0:   6%|▌         | 11/188 [00:00<00:15, 11.20it/s]
Epoch 0:   6%|▋         | 12/188 [00:00<00:14, 12.11it/s]
Epoch 0:   6%|▋         | 12/188 [00:00<00:14, 12.11it/s]
Epoch 0:   7%|▋         | 13/188 [00:01<00:13, 12.95it/s]
Epoch 0:   7%|▋         | 13/188 [00:01<00:13, 12.95it/s]
Epoch 0:   7%|▋         | 14/188 [00:01<00:13, 13.24it/s]
Epoch 0:   7%|▋         | 14/188 [00:01<00:13, 13.24it/s]
Epoch 0:   8%|▊         | 15/188 [00:01<00:12, 14.07it/s]
Epoch 0:   8%|▊         | 15/188 [00:01<00:12, 14.07it/s]
Epoch 0:   9%|▊         | 16/188 [00:01<00:11, 14.84it/s]
Epoch 0:   9%|▊         | 16/188 [00:01<00:11, 14.84it/s]
Epoch 0:   9%|▉         | 17/188 [00:01<00:11, 15.03it/s]
Epoch 0:   9%|▉         | 17/188 [00:01<00:11, 15.03it/s]
Epoch 0:  10%|▉         | 18/188 [00:01<00:10, 15.78it/s]
Epoch 0:  10%|▉         | 18/188 [00:01<00:10, 15.78it/s]
Epoch 0:  10%|█         | 19/188 [00:01<00:10, 16.52it/s]
Epoch 0:  10%|█         | 19/188 [00:01<00:10, 16.51it/s]
Epoch 0:  11%|█         | 20/188 [00:01<00:10, 16.61it/s]
Epoch 0:  11%|█         | 20/188 [00:01<00:10, 16.61it/s]
Epoch 0:  11%|█         | 21/188 [00:01<00:09, 17.30it/s]
Epoch 0:  11%|█         | 21/188 [00:01<00:09, 17.30it/s]
Epoch 0:  12%|█▏        | 22/188 [00:01<00:09, 17.98it/s]
Epoch 0:  12%|█▏        | 22/188 [00:01<00:09, 17.98it/s]
Epoch 0:  12%|█▏        | 23/188 [00:01<00:09, 18.01it/s]
Epoch 0:  12%|█▏        | 23/188 [00:01<00:09, 18.01it/s]
Epoch 0:  13%|█▎        | 24/188 [00:01<00:08, 18.65it/s]
Epoch 0:  13%|█▎        | 24/188 [00:01<00:08, 18.64it/s]
Epoch 0:  13%|█▎        | 25/188 [00:01<00:08, 19.27it/s]
Epoch 0:  13%|█▎        | 25/188 [00:01<00:08, 19.27it/s]
Epoch 0:  14%|█▍        | 26/188 [00:01<00:08, 19.28it/s]
Epoch 0:  14%|█▍        | 26/188 [00:01<00:08, 19.27it/s]
Epoch 0:  14%|█▍        | 27/188 [00:01<00:08, 19.86it/s]
Epoch 0:  14%|█▍        | 27/188 [00:01<00:08, 19.86it/s]
Epoch 0:  15%|█▍        | 28/188 [00:01<00:07, 20.43it/s]
Epoch 0:  15%|█▍        | 28/188 [00:01<00:07, 20.43it/s]
Epoch 0:  15%|█▌        | 29/188 [00:01<00:07, 20.41it/s]
Epoch 0:  15%|█▌        | 29/188 [00:01<00:07, 20.41it/s]
Epoch 0:  16%|█▌        | 30/188 [00:01<00:07, 20.91it/s]
Epoch 0:  16%|█▌        | 30/188 [00:01<00:07, 20.91it/s]
Epoch 0:  16%|█▋        | 31/188 [00:01<00:07, 21.45it/s]
Epoch 0:  16%|█▋        | 31/188 [00:01<00:07, 21.45it/s]
Epoch 0:  17%|█▋        | 32/188 [00:01<00:07, 21.40it/s]
Epoch 0:  17%|█▋        | 32/188 [00:01<00:07, 21.40it/s]
Epoch 0:  18%|█▊        | 33/188 [00:01<00:07, 21.88it/s]
Epoch 0:  18%|█▊        | 33/188 [00:01<00:07, 21.88it/s]
Epoch 0:  18%|█▊        | 34/188 [00:01<00:06, 22.39it/s]
Epoch 0:  18%|█▊        | 34/188 [00:01<00:06, 22.39it/s]
Epoch 0:  19%|█▊        | 35/188 [00:01<00:06, 22.30it/s]
Epoch 0:  19%|█▊        | 35/188 [00:01<00:06, 22.30it/s]
Epoch 0:  19%|█▉        | 36/188 [00:01<00:06, 22.77it/s]
Epoch 0:  19%|█▉        | 36/188 [00:01<00:06, 22.77it/s]
Epoch 0:  20%|█▉        | 37/188 [00:01<00:06, 23.26it/s]
Epoch 0:  20%|█▉        | 37/188 [00:01<00:06, 23.25it/s]
Epoch 0:  20%|██        | 38/188 [00:01<00:06, 23.13it/s]
Epoch 0:  20%|██        | 38/188 [00:01<00:06, 23.12it/s]
Epoch 0:  21%|██        | 39/188 [00:01<00:06, 23.58it/s]
Epoch 0:  21%|██        | 39/188 [00:01<00:06, 23.58it/s]
Epoch 0:  21%|██▏       | 40/188 [00:01<00:06, 24.05it/s]
Epoch 0:  21%|██▏       | 40/188 [00:01<00:06, 24.05it/s]
Epoch 0:  22%|██▏       | 41/188 [00:01<00:06, 23.89it/s]
Epoch 0:  22%|██▏       | 41/188 [00:01<00:06, 23.89it/s]
Epoch 0:  22%|██▏       | 42/188 [00:01<00:06, 24.31it/s]
Epoch 0:  22%|██▏       | 42/188 [00:01<00:06, 24.31it/s]
Epoch 0:  23%|██▎       | 43/188 [00:01<00:05, 24.77it/s]
Epoch 0:  23%|██▎       | 43/188 [00:01<00:05, 24.76it/s]
Epoch 0:  23%|██▎       | 44/188 [00:01<00:05, 24.60it/s]
Epoch 0:  23%|██▎       | 44/188 [00:01<00:05, 24.59it/s]
Epoch 0:  24%|██▍       | 45/188 [00:01<00:05, 24.99it/s]
Epoch 0:  24%|██▍       | 45/188 [00:01<00:05, 24.99it/s]
Epoch 0:  24%|██▍       | 46/188 [00:01<00:05, 25.41it/s]
Epoch 0:  24%|██▍       | 46/188 [00:01<00:05, 25.41it/s]
Epoch 0:  25%|██▌       | 47/188 [00:01<00:05, 25.25it/s]
Epoch 0:  25%|██▌       | 47/188 [00:01<00:05, 25.25it/s]
Epoch 0:  26%|██▌       | 48/188 [00:01<00:05, 25.65it/s]
Epoch 0:  26%|██▌       | 48/188 [00:01<00:05, 25.64it/s]
Epoch 0:  26%|██▌       | 49/188 [00:01<00:05, 26.04it/s]
Epoch 0:  26%|██▌       | 49/188 [00:01<00:05, 26.04it/s]
Epoch 0:  27%|██▋       | 50/188 [00:01<00:05, 25.87it/s]
Epoch 0:  27%|██▋       | 50/188 [00:01<00:05, 25.86it/s]
Epoch 0:  27%|██▋       | 51/188 [00:01<00:05, 26.22it/s]
Epoch 0:  27%|██▋       | 51/188 [00:01<00:05, 26.22it/s]
Epoch 0:  28%|██▊       | 52/188 [00:01<00:05, 26.59it/s]
Epoch 0:  28%|██▊       | 52/188 [00:01<00:05, 26.59it/s]
Epoch 0:  28%|██▊       | 53/188 [00:02<00:05, 26.38it/s]
Epoch 0:  28%|██▊       | 53/188 [00:02<00:05, 26.38it/s]
Epoch 0:  29%|██▊       | 54/188 [00:02<00:05, 26.74it/s]
Epoch 0:  29%|██▊       | 54/188 [00:02<00:05, 26.74it/s]
Epoch 0:  29%|██▉       | 55/188 [00:02<00:04, 27.11it/s]
Epoch 0:  29%|██▉       | 55/188 [00:02<00:04, 27.11it/s]
Epoch 0:  30%|██▉       | 56/188 [00:02<00:04, 26.89it/s]
Epoch 0:  30%|██▉       | 56/188 [00:02<00:04, 26.89it/s]
Epoch 0:  30%|███       | 57/188 [00:02<00:04, 27.25it/s]
Epoch 0:  30%|███       | 57/188 [00:02<00:04, 27.25it/s]
Epoch 0:  31%|███       | 58/188 [00:02<00:04, 27.61it/s]
Epoch 0:  31%|███       | 58/188 [00:02<00:04, 27.61it/s]
Epoch 0:  31%|███▏      | 59/188 [00:02<00:04, 27.36it/s]
Epoch 0:  31%|███▏      | 59/188 [00:02<00:04, 27.35it/s]
Epoch 0:  32%|███▏      | 60/188 [00:02<00:04, 27.70it/s]
Epoch 0:  32%|███▏      | 60/188 [00:02<00:04, 27.70it/s]
Epoch 0:  32%|███▏      | 61/188 [00:02<00:04, 28.05it/s]
Epoch 0:  32%|███▏      | 61/188 [00:02<00:04, 28.05it/s]
Epoch 0:  33%|███▎      | 62/188 [00:02<00:04, 27.79it/s]
Epoch 0:  33%|███▎      | 62/188 [00:02<00:04, 27.78it/s]
Epoch 0:  34%|███▎      | 63/188 [00:02<00:04, 28.12it/s]
Epoch 0:  34%|███▎      | 63/188 [00:02<00:04, 28.12it/s]
Epoch 0:  34%|███▍      | 64/188 [00:02<00:04, 28.45it/s]
Epoch 0:  34%|███▍      | 64/188 [00:02<00:04, 28.45it/s]
Epoch 0:  35%|███▍      | 65/188 [00:02<00:04, 28.21it/s]
Epoch 0:  35%|███▍      | 65/188 [00:02<00:04, 28.21it/s]
Epoch 0:  35%|███▌      | 66/188 [00:02<00:04, 28.54it/s]
Epoch 0:  35%|███▌      | 66/188 [00:02<00:04, 28.53it/s]
Epoch 0:  36%|███▌      | 67/188 [00:02<00:04, 28.85it/s]
Epoch 0:  36%|███▌      | 67/188 [00:02<00:04, 28.85it/s]
Epoch 0:  36%|███▌      | 68/188 [00:02<00:04, 28.61it/s]
Epoch 0:  36%|███▌      | 68/188 [00:02<00:04, 28.61it/s]
Epoch 0:  37%|███▋      | 69/188 [00:02<00:04, 28.91it/s]
Epoch 0:  37%|███▋      | 69/188 [00:02<00:04, 28.91it/s]
Epoch 0:  37%|███▋      | 70/188 [00:02<00:04, 29.21it/s]
Epoch 0:  37%|███▋      | 70/188 [00:02<00:04, 29.20it/s]
Epoch 0:  38%|███▊      | 71/188 [00:02<00:04, 28.98it/s]
Epoch 0:  38%|███▊      | 71/188 [00:02<00:04, 28.98it/s]
Epoch 0:  38%|███▊      | 72/188 [00:02<00:03, 29.26it/s]
Epoch 0:  38%|███▊      | 72/188 [00:02<00:03, 29.26it/s]
Epoch 0:  39%|███▉      | 73/188 [00:02<00:03, 29.55it/s]
Epoch 0:  39%|███▉      | 73/188 [00:02<00:03, 29.55it/s]
Epoch 0:  39%|███▉      | 74/188 [00:02<00:03, 29.34it/s]
Epoch 0:  39%|███▉      | 74/188 [00:02<00:03, 29.34it/s]
Epoch 0:  40%|███▉      | 75/188 [00:02<00:03, 29.61it/s]
Epoch 0:  40%|███▉      | 75/188 [00:02<00:03, 29.61it/s]
Epoch 0:  40%|████      | 76/188 [00:02<00:03, 29.89it/s]
Epoch 0:  40%|████      | 76/188 [00:02<00:03, 29.89it/s]
Epoch 0:  41%|████      | 77/188 [00:02<00:03, 29.68it/s]
Epoch 0:  41%|████      | 77/188 [00:02<00:03, 29.68it/s]
Epoch 0:  41%|████▏     | 78/188 [00:02<00:03, 29.95it/s]
Epoch 0:  41%|████▏     | 78/188 [00:02<00:03, 29.95it/s]
Epoch 0:  42%|████▏     | 79/188 [00:02<00:03, 30.21it/s]
Epoch 0:  42%|████▏     | 79/188 [00:02<00:03, 30.21it/s]
Epoch 0:  43%|████▎     | 80/188 [00:02<00:03, 30.00it/s]
Epoch 0:  43%|████▎     | 80/188 [00:02<00:03, 30.00it/s]
Epoch 0:  43%|████▎     | 81/188 [00:02<00:03, 30.25it/s]
Epoch 0:  43%|████▎     | 81/188 [00:02<00:03, 30.25it/s]
Epoch 0:  44%|████▎     | 82/188 [00:02<00:03, 30.50it/s]
Epoch 0:  44%|████▎     | 82/188 [00:02<00:03, 30.50it/s]
Epoch 0:  44%|████▍     | 83/188 [00:02<00:03, 30.32it/s]
Epoch 0:  44%|████▍     | 83/188 [00:02<00:03, 30.32it/s]
Epoch 0:  45%|████▍     | 84/188 [00:02<00:03, 30.53it/s]
Epoch 0:  45%|████▍     | 84/188 [00:02<00:03, 30.53it/s]
Epoch 0:  45%|████▌     | 85/188 [00:02<00:03, 30.80it/s]
Epoch 0:  45%|████▌     | 85/188 [00:02<00:03, 30.79it/s]
Epoch 0:  46%|████▌     | 86/188 [00:02<00:03, 30.59it/s]
Epoch 0:  46%|████▌     | 86/188 [00:02<00:03, 30.58it/s]
Epoch 0:  46%|████▋     | 87/188 [00:02<00:03, 30.84it/s]
Epoch 0:  46%|████▋     | 87/188 [00:02<00:03, 30.84it/s]
Epoch 0:  47%|████▋     | 88/188 [00:02<00:03, 31.09it/s]
Epoch 0:  47%|████▋     | 88/188 [00:02<00:03, 31.09it/s]
Epoch 0:  47%|████▋     | 89/188 [00:02<00:03, 30.85it/s]
Epoch 0:  47%|████▋     | 89/188 [00:02<00:03, 30.85it/s]
Epoch 0:  48%|████▊     | 90/188 [00:02<00:03, 31.07it/s]
Epoch 0:  48%|████▊     | 90/188 [00:02<00:03, 31.06it/s]
Epoch 0:  48%|████▊     | 91/188 [00:02<00:03, 31.31it/s]
Epoch 0:  48%|████▊     | 91/188 [00:02<00:03, 31.31it/s]
Epoch 0:  49%|████▉     | 92/188 [00:02<00:03, 31.07it/s]
Epoch 0:  49%|████▉     | 92/188 [00:02<00:03, 31.07it/s]
Epoch 0:  49%|████▉     | 93/188 [00:02<00:03, 31.30it/s]
Epoch 0:  49%|████▉     | 93/188 [00:02<00:03, 31.30it/s]
Epoch 0:  50%|█████     | 94/188 [00:02<00:02, 31.54it/s]
Epoch 0:  50%|█████     | 94/188 [00:02<00:02, 31.54it/s]
Epoch 0:  51%|█████     | 95/188 [00:03<00:02, 31.31it/s]
Epoch 0:  51%|█████     | 95/188 [00:03<00:02, 31.31it/s]
Epoch 0:  51%|█████     | 96/188 [00:03<00:02, 31.53it/s]
Epoch 0:  51%|█████     | 96/188 [00:03<00:02, 31.53it/s]
Epoch 0:  52%|█████▏    | 97/188 [00:03<00:02, 31.76it/s]
Epoch 0:  52%|█████▏    | 97/188 [00:03<00:02, 31.76it/s]
Epoch 0:  52%|█████▏    | 98/188 [00:03<00:02, 31.56it/s]
Epoch 0:  52%|█████▏    | 98/188 [00:03<00:02, 31.56it/s]
Epoch 0:  53%|█████▎    | 99/188 [00:03<00:02, 31.77it/s]
Epoch 0:  53%|█████▎    | 99/188 [00:03<00:02, 31.77it/s]
Epoch 0:  53%|█████▎    | 100/188 [00:03<00:02, 31.99it/s]
Epoch 0:  53%|█████▎    | 100/188 [00:03<00:02, 31.99it/s]
Epoch 0:  54%|█████▎    | 101/188 [00:03<00:02, 31.78it/s]
Epoch 0:  54%|█████▎    | 101/188 [00:03<00:02, 31.78it/s]
Epoch 0:  54%|█████▍    | 102/188 [00:03<00:02, 32.00it/s]
Epoch 0:  54%|█████▍    | 102/188 [00:03<00:02, 32.00it/s]
Epoch 0:  55%|█████▍    | 103/188 [00:03<00:02, 32.20it/s]
Epoch 0:  55%|█████▍    | 103/188 [00:03<00:02, 32.20it/s]
Epoch 0:  55%|█████▌    | 104/188 [00:03<00:02, 32.01it/s]
Epoch 0:  55%|█████▌    | 104/188 [00:03<00:02, 32.01it/s]
Epoch 0:  56%|█████▌    | 105/188 [00:03<00:02, 32.22it/s]
Epoch 0:  56%|█████▌    | 105/188 [00:03<00:02, 32.22it/s]
Epoch 0:  56%|█████▋    | 106/188 [00:03<00:02, 32.42it/s]
Epoch 0:  56%|█████▋    | 106/188 [00:03<00:02, 32.42it/s]
Epoch 0:  57%|█████▋    | 107/188 [00:03<00:02, 32.21it/s]
Epoch 0:  57%|█████▋    | 107/188 [00:03<00:02, 32.21it/s]
Epoch 0:  57%|█████▋    | 108/188 [00:03<00:02, 32.43it/s]
Epoch 0:  57%|█████▋    | 108/188 [00:03<00:02, 32.42it/s]
Epoch 0:  58%|█████▊    | 109/188 [00:03<00:02, 32.64it/s]
Epoch 0:  58%|█████▊    | 109/188 [00:03<00:02, 32.64it/s]
Epoch 0:  59%|█████▊    | 110/188 [00:03<00:02, 32.41it/s]
Epoch 0:  59%|█████▊    | 110/188 [00:03<00:02, 32.40it/s]
Epoch 0:  59%|█████▉    | 111/188 [00:03<00:02, 32.62it/s]
Epoch 0:  59%|█████▉    | 111/188 [00:03<00:02, 32.61it/s]
Epoch 0:  60%|█████▉    | 112/188 [00:03<00:02, 32.83it/s]
Epoch 0:  60%|█████▉    | 112/188 [00:03<00:02, 32.82it/s]
Epoch 0:  60%|██████    | 113/188 [00:03<00:02, 32.61it/s]
Epoch 0:  60%|██████    | 113/188 [00:03<00:02, 32.61it/s]
Epoch 0:  61%|██████    | 114/188 [00:03<00:02, 32.81it/s]
Epoch 0:  61%|██████    | 114/188 [00:03<00:02, 32.81it/s]
Epoch 0:  61%|██████    | 115/188 [00:03<00:02, 33.00it/s]
Epoch 0:  61%|██████    | 115/188 [00:03<00:02, 33.00it/s]
Epoch 0:  62%|██████▏   | 116/188 [00:03<00:02, 32.79it/s]
Epoch 0:  62%|██████▏   | 116/188 [00:03<00:02, 32.79it/s]
Epoch 0:  62%|██████▏   | 117/188 [00:03<00:02, 33.00it/s]
Epoch 0:  62%|██████▏   | 117/188 [00:03<00:02, 33.00it/s]
Epoch 0:  63%|██████▎   | 118/188 [00:03<00:02, 33.19it/s]
Epoch 0:  63%|██████▎   | 118/188 [00:03<00:02, 33.19it/s]
Epoch 0:  63%|██████▎   | 119/188 [00:03<00:02, 32.97it/s]
Epoch 0:  63%|██████▎   | 119/188 [00:03<00:02, 32.96it/s]
Epoch 0:  64%|██████▍   | 120/188 [00:03<00:02, 33.15it/s]
Epoch 0:  64%|██████▍   | 120/188 [00:03<00:02, 33.15it/s]
Epoch 0:  64%|██████▍   | 121/188 [00:03<00:02, 33.34it/s]
Epoch 0:  64%|██████▍   | 121/188 [00:03<00:02, 33.33it/s]
Epoch 0:  65%|██████▍   | 122/188 [00:03<00:01, 33.13it/s]
Epoch 0:  65%|██████▍   | 122/188 [00:03<00:01, 33.13it/s]
Epoch 0:  65%|██████▌   | 123/188 [00:03<00:01, 33.31it/s]
Epoch 0:  65%|██████▌   | 123/188 [00:03<00:01, 33.31it/s]
Epoch 0:  66%|██████▌   | 124/188 [00:03<00:01, 33.49it/s]
Epoch 0:  66%|██████▌   | 124/188 [00:03<00:01, 33.49it/s]
Epoch 0:  66%|██████▋   | 125/188 [00:03<00:01, 33.32it/s]
Epoch 0:  66%|██████▋   | 125/188 [00:03<00:01, 33.32it/s]
Epoch 0:  67%|██████▋   | 126/188 [00:03<00:01, 33.47it/s]
Epoch 0:  67%|██████▋   | 126/188 [00:03<00:01, 33.47it/s]
Epoch 0:  68%|██████▊   | 127/188 [00:03<00:01, 33.65it/s]
Epoch 0:  68%|██████▊   | 127/188 [00:03<00:01, 33.65it/s]
Epoch 0:  68%|██████▊   | 128/188 [00:03<00:01, 33.47it/s]
Epoch 0:  68%|██████▊   | 128/188 [00:03<00:01, 33.47it/s]
Epoch 0:  69%|██████▊   | 129/188 [00:03<00:01, 33.62it/s]
Epoch 0:  69%|██████▊   | 129/188 [00:03<00:01, 33.62it/s]
Epoch 0:  69%|██████▉   | 130/188 [00:03<00:01, 33.80it/s]
Epoch 0:  69%|██████▉   | 130/188 [00:03<00:01, 33.80it/s]
Epoch 0:  70%|██████▉   | 131/188 [00:03<00:01, 33.62it/s]
Epoch 0:  70%|██████▉   | 131/188 [00:03<00:01, 33.62it/s]
Epoch 0:  70%|███████   | 132/188 [00:03<00:01, 33.74it/s]
Epoch 0:  70%|███████   | 132/188 [00:03<00:01, 33.74it/s]
Epoch 0:  71%|███████   | 133/188 [00:03<00:01, 33.92it/s]
Epoch 0:  71%|███████   | 133/188 [00:03<00:01, 33.92it/s]
Epoch 0:  71%|███████▏  | 134/188 [00:03<00:01, 33.76it/s]
Epoch 0:  71%|███████▏  | 134/188 [00:03<00:01, 33.76it/s]
Epoch 0:  72%|███████▏  | 135/188 [00:03<00:01, 33.88it/s]
Epoch 0:  72%|███████▏  | 135/188 [00:03<00:01, 33.88it/s]
Epoch 0:  72%|███████▏  | 136/188 [00:03<00:01, 34.06it/s]
Epoch 0:  72%|███████▏  | 136/188 [00:03<00:01, 34.05it/s]
Epoch 0:  73%|███████▎  | 137/188 [00:04<00:01, 33.91it/s]
Epoch 0:  73%|███████▎  | 137/188 [00:04<00:01, 33.91it/s]
Epoch 0:  73%|███████▎  | 138/188 [00:04<00:01, 34.04it/s]
Epoch 0:  73%|███████▎  | 138/188 [00:04<00:01, 34.04it/s]
Epoch 0:  74%|███████▍  | 139/188 [00:04<00:01, 34.20it/s]
Epoch 0:  74%|███████▍  | 139/188 [00:04<00:01, 34.20it/s]
Epoch 0:  74%|███████▍  | 140/188 [00:04<00:01, 34.04it/s]
Epoch 0:  74%|███████▍  | 140/188 [00:04<00:01, 34.04it/s]
Epoch 0:  75%|███████▌  | 141/188 [00:04<00:01, 34.18it/s]
Epoch 0:  75%|███████▌  | 141/188 [00:04<00:01, 34.17it/s]
Epoch 0:  76%|███████▌  | 142/188 [00:04<00:01, 34.33it/s]
Epoch 0:  76%|███████▌  | 142/188 [00:04<00:01, 34.33it/s]
Epoch 0:  76%|███████▌  | 143/188 [00:04<00:01, 34.17it/s]
Epoch 0:  76%|███████▌  | 143/188 [00:04<00:01, 34.17it/s]
Epoch 0:  77%|███████▋  | 144/188 [00:04<00:01, 34.30it/s]
Epoch 0:  77%|███████▋  | 144/188 [00:04<00:01, 34.30it/s]
Epoch 0:  77%|███████▋  | 145/188 [00:04<00:01, 34.46it/s]
Epoch 0:  77%|███████▋  | 145/188 [00:04<00:01, 34.46it/s]
Epoch 0:  78%|███████▊  | 146/188 [00:04<00:01, 34.30it/s]
Epoch 0:  78%|███████▊  | 146/188 [00:04<00:01, 34.30it/s]
Epoch 0:  78%|███████▊  | 147/188 [00:04<00:01, 34.42it/s]
Epoch 0:  78%|███████▊  | 147/188 [00:04<00:01, 34.42it/s]
Epoch 0:  79%|███████▊  | 148/188 [00:04<00:01, 34.59it/s]
Epoch 0:  79%|███████▊  | 148/188 [00:04<00:01, 34.59it/s]
Epoch 0:  79%|███████▉  | 149/188 [00:04<00:01, 34.42it/s]
Epoch 0:  79%|███████▉  | 149/188 [00:04<00:01, 34.42it/s]
Epoch 0:  80%|███████▉  | 150/188 [00:04<00:01, 34.54it/s]
Epoch 0:  80%|███████▉  | 150/188 [00:04<00:01, 34.54it/s]
Epoch 0:  80%|████████  | 151/188 [00:04<00:01, 34.70it/s]
Epoch 0:  80%|████████  | 151/188 [00:04<00:01, 34.70it/s]
Epoch 0:  81%|████████  | 152/188 [00:04<00:01, 34.54it/s]
Epoch 0:  81%|████████  | 152/188 [00:04<00:01, 34.53it/s]
Epoch 0:  81%|████████▏ | 153/188 [00:04<00:01, 34.65it/s]
Epoch 0:  81%|████████▏ | 153/188 [00:04<00:01, 34.65it/s]
Epoch 0:  82%|████████▏ | 154/188 [00:04<00:00, 34.81it/s]
Epoch 0:  82%|████████▏ | 154/188 [00:04<00:00, 34.81it/s]
Epoch 0:  82%|████████▏ | 155/188 [00:04<00:00, 34.62it/s]
Epoch 0:  82%|████████▏ | 155/188 [00:04<00:00, 34.62it/s]
Epoch 0:  83%|████████▎ | 156/188 [00:04<00:00, 34.77it/s]
Epoch 0:  83%|████████▎ | 156/188 [00:04<00:00, 34.77it/s]
Epoch 0:  84%|████████▎ | 157/188 [00:04<00:00, 34.92it/s]
Epoch 0:  84%|████████▎ | 157/188 [00:04<00:00, 34.92it/s]
Epoch 0:  84%|████████▍ | 158/188 [00:04<00:00, 34.73it/s]
Epoch 0:  84%|████████▍ | 158/188 [00:04<00:00, 34.72it/s]
Epoch 0:  85%|████████▍ | 159/188 [00:04<00:00, 34.87it/s]
Epoch 0:  85%|████████▍ | 159/188 [00:04<00:00, 34.87it/s]
Epoch 0:  85%|████████▌ | 160/188 [00:04<00:00, 35.02it/s]
Epoch 0:  85%|████████▌ | 160/188 [00:04<00:00, 35.02it/s]
Epoch 0:  86%|████████▌ | 161/188 [00:04<00:00, 34.83it/s]
Epoch 0:  86%|████████▌ | 161/188 [00:04<00:00, 34.83it/s]
Epoch 0:  86%|████████▌ | 162/188 [00:04<00:00, 34.97it/s]
Epoch 0:  86%|████████▌ | 162/188 [00:04<00:00, 34.97it/s]
Epoch 0:  87%|████████▋ | 163/188 [00:04<00:00, 35.12it/s]
Epoch 0:  87%|████████▋ | 163/188 [00:04<00:00, 35.11it/s]
Epoch 0:  87%|████████▋ | 164/188 [00:04<00:00, 34.93it/s]
Epoch 0:  87%|████████▋ | 164/188 [00:04<00:00, 34.93it/s]
Epoch 0:  88%|████████▊ | 165/188 [00:04<00:00, 35.06it/s]
Epoch 0:  88%|████████▊ | 165/188 [00:04<00:00, 35.06it/s]
Epoch 0:  88%|████████▊ | 166/188 [00:04<00:00, 35.20it/s]
Epoch 0:  88%|████████▊ | 166/188 [00:04<00:00, 35.20it/s]
Epoch 0:  89%|████████▉ | 167/188 [00:04<00:00, 35.02it/s]
Epoch 0:  89%|████████▉ | 167/188 [00:04<00:00, 35.02it/s]
Epoch 0:  89%|████████▉ | 168/188 [00:04<00:00, 35.15it/s]
Epoch 0:  89%|████████▉ | 168/188 [00:04<00:00, 35.15it/s]
Epoch 0:  90%|████████▉ | 169/188 [00:04<00:00, 35.28it/s]
Epoch 0:  90%|████████▉ | 169/188 [00:04<00:00, 35.28it/s]
Epoch 0:  90%|█████████ | 170/188 [00:04<00:00, 35.12it/s]
Epoch 0:  90%|█████████ | 170/188 [00:04<00:00, 35.12it/s]
Epoch 0:  91%|█████████ | 171/188 [00:04<00:00, 35.26it/s]
Epoch 0:  91%|█████████ | 171/188 [00:04<00:00, 35.25it/s]
Epoch 0:  91%|█████████▏| 172/188 [00:04<00:00, 35.38it/s]
Epoch 0:  91%|█████████▏| 172/188 [00:04<00:00, 35.38it/s]
Epoch 0:  92%|█████████▏| 173/188 [00:04<00:00, 35.21it/s]
Epoch 0:  92%|█████████▏| 173/188 [00:04<00:00, 35.21it/s]
Epoch 0:  93%|█████████▎| 174/188 [00:04<00:00, 35.34it/s]
Epoch 0:  93%|█████████▎| 174/188 [00:04<00:00, 35.34it/s]
Epoch 0:  93%|█████████▎| 175/188 [00:04<00:00, 35.47it/s]
Epoch 0:  93%|█████████▎| 175/188 [00:04<00:00, 35.47it/s]
Epoch 0:  94%|█████████▎| 176/188 [00:04<00:00, 35.30it/s]
Epoch 0:  94%|█████████▎| 176/188 [00:04<00:00, 35.30it/s]
Epoch 0:  94%|█████████▍| 177/188 [00:04<00:00, 35.43it/s]
Epoch 0:  94%|█████████▍| 177/188 [00:04<00:00, 35.43it/s]
Epoch 0:  95%|█████████▍| 178/188 [00:05<00:00, 35.57it/s]
Epoch 0:  95%|█████████▍| 178/188 [00:05<00:00, 35.57it/s]
Epoch 0:  95%|█████████▌| 179/188 [00:05<00:00, 35.38it/s]
Epoch 0:  95%|█████████▌| 179/188 [00:05<00:00, 35.38it/s]
Epoch 0:  96%|█████████▌| 180/188 [00:05<00:00, 35.52it/s]
Epoch 0:  96%|█████████▌| 180/188 [00:05<00:00, 35.52it/s]
Epoch 0:  96%|█████████▋| 181/188 [00:05<00:00, 35.65it/s]
Epoch 0:  96%|█████████▋| 181/188 [00:05<00:00, 35.65it/s]
Epoch 0:  97%|█████████▋| 182/188 [00:05<00:00, 35.47it/s]
Epoch 0:  97%|█████████▋| 182/188 [00:05<00:00, 35.47it/s]
Epoch 0:  97%|█████████▋| 183/188 [00:05<00:00, 35.61it/s]
Epoch 0:  97%|█████████▋| 183/188 [00:05<00:00, 35.60it/s]
Epoch 0:  98%|█████████▊| 184/188 [00:05<00:00, 35.74it/s]
Epoch 0:  98%|█████████▊| 184/188 [00:05<00:00, 35.74it/s]
Epoch 0:  98%|█████████▊| 185/188 [00:05<00:00, 35.57it/s]
Epoch 0:  98%|█████████▊| 185/188 [00:05<00:00, 35.57it/s]
Epoch 0:  99%|█████████▉| 186/188 [00:05<00:00, 35.70it/s]
Epoch 0:  99%|█████████▉| 186/188 [00:05<00:00, 35.70it/s]
Epoch 0:  99%|█████████▉| 187/188 [00:05<00:00, 35.83it/s]
Epoch 0:  99%|█████████▉| 187/188 [00:05<00:00, 35.83it/s]
Epoch 0: 100%|██████████| 188/188 [00:05<00:00, 32.66it/s]
Epoch 0: 100%|██████████| 188/188 [00:05<00:00, 32.66it/s]

Validation: |          | 0/? [00:00<?, ?it/s][A

Validation: |          | 0/? [00:00<?, ?it/s][A

Validation DataLoader 0:   0%|          | 0/19 [00:00<?, ?it/s][A

Validation DataLoader 0:   5%|▌         | 1/19 [00:00<00:00, 47.22it/s][A

Validation DataLoader 0:  11%|█         | 2/19 [00:00<00:00, 46.87it/s][A

Validation DataLoader 0:  16%|█▌        | 3/19 [00:00<00:00, 46.77it/s][A

Validation DataLoader 0:  21%|██        | 4/19 [00:00<00:00, 23.12it/s][A

Validation DataLoader 0:  26%|██▋       | 5/19 [00:00<00:00, 25.70it/s][A

Validation DataLoader 0:  32%|███▏      | 6/19 [00:00<00:00, 27.63it/s][A

Validation DataLoader 0:  37%|███▋      | 7/19 [00:00<00:00, 21.67it/s][A

Validation DataLoader 0:  42%|████▏     | 8/19 [00:00<00:00, 23.15it/s][A

Validation DataLoader 0:  47%|████▋     | 9/19 [00:00<00:00, 24.42it/s][A

Validation DataLoader 0:  53%|█████▎    | 10/19 [00:00<00:00, 20.81it/s][A

Validation DataLoader 0:  58%|█████▊    | 11/19 [00:00<00:00, 21.81it/s][A

Validation DataLoader 0:  63%|██████▎   | 12/19 [00:00<00:00, 22.77it/s][A

Validation DataLoader 0:  68%|██████▊   | 13/19 [00:00<00:00, 20.35it/s][A

Validation DataLoader 0:  74%|███████▎  | 14/19 [00:00<00:00, 21.15it/s][A

Validation DataLoader 0:  79%|███████▉  | 15/19 [00:00<00:00, 21.88it/s][A

Validation DataLoader 0:  84%|████████▍ | 16/19 [00:00<00:00, 20.58it/s][A

Validation DataLoader 0:  89%|████████▉ | 17/19 [00:00<00:00, 21.24it/s][A

Validation DataLoader 0:  95%|█████████▍| 18/19 [00:00<00:00, 21.88it/s][A

Validation DataLoader 0: 100%|██████████| 19/19 [00:01<00:00, 16.79it/s][A

                                                                        [A
Epoch 0: 100%|██████████| 188/188 [00:07<00:00, 26.62it/s, R@1 (val)=0.020, R@5 (val)=0.0567, R@20 (val)=0.200, loss (val)=0.599]
Epoch 0: 100%|██████████| 188/188 [00:07<00:00, 26.61it/s, R@1 (val)=0.020, R@5 (val)=0.0567, R@20 (val)=0.200, loss (val)=0.599, loss (train)=0.709]`Trainer.fit` stopped: `max_epochs=1` reached.

Epoch 0: 100%|██████████| 188/188 [00:07<00:00, 26.49it/s, R@1 (val)=0.020, R@5 (val)=0.0567, R@20 (val)=0.200, loss (val)=0.599, loss (train)=0.709]Restoring states from the checkpoint path at /Users/timrichardson/Documents/projects/personal/blog/rewire.it/workbench/outputs/blog-post-msms-model-selection-20261003-230223/runs/external/msalign-src/checkpoints/paper_competitors/smoke_deepsets/best-epoch=00-loss (val)=0.5992.ckpt
Loaded model weights from the checkpoint at /Users/timrichardson/Documents/projects/personal/blog/rewire.it/workbench/outputs/blog-post-msms-model-selection-20261003-230223/runs/external/msalign-src/checkpoints/paper_competitors/smoke_deepsets/best-epoch=00-loss (val)=0.5992.ckpt


Validation: |          | 0/? [00:00<?, ?it/s]
Validation: |          | 0/? [00:00<?, ?it/s]
Validation DataLoader 0:   0%|          | 0/19 [00:00<?, ?it/s]
Validation DataLoader 0:   5%|▌         | 1/19 [00:00<00:00, 36.37it/s]
Validation DataLoader 0:  11%|█         | 2/19 [00:00<00:00, 38.44it/s]
Validation DataLoader 0:  16%|█▌        | 3/19 [00:00<00:00, 38.60it/s]
Validation DataLoader 0:  21%|██        | 4/19 [00:00<00:00, 22.82it/s]
Validation DataLoader 0:  26%|██▋       | 5/19 [00:00<00:00, 25.41it/s]
Validation DataLoader 0:  32%|███▏      | 6/19 [00:00<00:00, 27.55it/s]
Validation DataLoader 0:  37%|███▋      | 7/19 [00:00<00:00, 21.79it/s]
Validation DataLoader 0:  42%|████▏     | 8/19 [00:00<00:00, 23.26it/s]
Validation DataLoader 0:  47%|████▋     | 9/19 [00:00<00:00, 24.62it/s]
Validation DataLoader 0:  53%|█████▎    | 10/19 [00:00<00:00, 20.73it/s]
Validation DataLoader 0:  58%|█████▊    | 11/19 [00:00<00:00, 21.70it/s]
Validation DataLoader 0:  63%|██████▎   | 12/19 [00:00<00:00, 22.68it/s]
Validation DataLoader 0:  68%|██████▊   | 13/19 [00:00<00:00, 20.46it/s]
Validation DataLoader 0:  74%|███████▎  | 14/19 [00:00<00:00, 21.25it/s]
Validation DataLoader 0:  79%|███████▉  | 15/19 [00:00<00:00, 21.95it/s]
Validation DataLoader 0:  84%|████████▍ | 16/19 [00:00<00:00, 20.65it/s]
Validation DataLoader 0:  89%|████████▉ | 17/19 [00:00<00:00, 21.33it/s]
Validation DataLoader 0:  95%|█████████▍| 18/19 [00:00<00:00, 21.97it/s]
Validation DataLoader 0: 100%|██████████| 19/19 [00:00<00:00, 21.87it/s]
Validation DataLoader 0: 100%|██████████| 19/19 [00:00<00:00, 21.77it/s]Restoring states from the checkpoint path at /Users/timrichardson/Documents/projects/personal/blog/rewire.it/workbench/outputs/blog-post-msms-model-selection-20261003-230223/runs/external/msalign-src/checkpoints/paper_competitors/smoke_deepsets/best-epoch=00-loss (val)=0.5992.ckpt
Loaded model weights from the checkpoint at /Users/timrichardson/Documents/projects/personal/blog/rewire.it/workbench/outputs/blog-post-msms-model-selection-20261003-230223/runs/external/msalign-src/checkpoints/paper_competitors/smoke_deepsets/best-epoch=00-loss (val)=0.5992.ckpt

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     Validate metric           DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
        R@1 (val)          0.019999999552965164
       R@20 (val)           0.20000000298023224
        R@5 (val)           0.05666666850447655
       loss (val)           0.5992007255554199
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Testing: |          | 0/? [00:00<?, ?it/s]
Testing: |          | 0/? [00:00<?, ?it/s]
Testing DataLoader 0:   0%|          | 0/19 [00:00<?, ?it/s]
Testing DataLoader 0:   5%|▌         | 1/19 [00:00<00:00, 37.20it/s]
Testing DataLoader 0:  11%|█         | 2/19 [00:00<00:00, 41.41it/s]
Testing DataLoader 0:  16%|█▌        | 3/19 [00:00<00:00, 42.37it/s]
Testing DataLoader 0:  21%|██        | 4/19 [00:00<00:00, 23.46it/s]
Testing DataLoader 0:  26%|██▋       | 5/19 [00:00<00:00, 25.96it/s]
Testing DataLoader 0:  32%|███▏      | 6/19 [00:00<00:00, 28.06it/s]
Testing DataLoader 0:  37%|███▋      | 7/19 [00:00<00:00, 21.75it/s]
Testing DataLoader 0:  42%|████▏     | 8/19 [00:00<00:00, 23.22it/s]
Testing DataLoader 0:  47%|████▋     | 9/19 [00:00<00:00, 24.54it/s]
Testing DataLoader 0:  53%|█████▎    | 10/19 [00:00<00:00, 20.77it/s]
Testing DataLoader 0:  58%|█████▊    | 11/19 [00:00<00:00, 21.78it/s]
Testing DataLoader 0:  63%|██████▎   | 12/19 [00:00<00:00, 22.76it/s]
Testing DataLoader 0:  68%|██████▊   | 13/19 [00:00<00:00, 20.38it/s]
Testing DataLoader 0:  74%|███████▎  | 14/19 [00:00<00:00, 21.21it/s]
Testing DataLoader 0:  79%|███████▉  | 15/19 [00:00<00:00, 21.95it/s]
Testing DataLoader 0:  84%|████████▍ | 16/19 [00:00<00:00, 20.54it/s]
Testing DataLoader 0:  89%|████████▉ | 17/19 [00:00<00:00, 21.19it/s]
Testing DataLoader 0:  95%|█████████▍| 18/19 [00:00<00:00, 21.86it/s]
Testing DataLoader 0: 100%|██████████| 19/19 [00:00<00:00, 21.82it/s]
Testing DataLoader 0: 100%|██████████| 19/19 [00:00<00:00, 21.74it/s]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       Test metric             DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       R@1 (test)          0.019999999552965164
       R@20 (test)          0.20666666328907013
       R@5 (test)          0.046666666865348816
       loss (test)           0.605099081993103
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       82.86 real        48.93 user         4.56 sys
          1496678400  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
             1460259  page reclaims
               14898  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                2544  messages sent
                1272  messages received
                   6  signals received
               72384  voluntary context switches
               45889  involuntary context switches
         81743976799  instructions retired
         25846694859  cycles elapsed
          2093041704  peak memory footprint
exit 0 end 2026-10-04T22:05:25Z
