
  0%|          | 0/1 [00:00<?, ?it/s]

  0%|          | 0/150 [elapsed: 00:00 remaining: ?][A

SUBMIT:   0%|          | 0/150 [elapsed: 00:00 remaining: ?][A

COMPLETE:   0%|          | 0/150 [elapsed: 00:00 remaining: ?][A

COMPLETE: 100%|██████████| 150/150 [elapsed: 00:00 remaining: 00:00][A
COMPLETE: 100%|██████████| 150/150 [elapsed: 00:02 remaining: 00:00]

100%|██████████| 1/1 [00:02<00:00,  2.41s/it]
100%|██████████| 1/1 [00:02<00:00,  2.41s/it]
Using bfloat16 Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py:76: Starting from v1.9.0, `tensorboardX` has been removed as a dependency of the `pytorch_lightning` package, due to potential conflicts with other packages in the ML ecosystem. For this reason, `logger=True` will use `CSVLogger` as the default logger, unless the `tensorboard` or `tensorboardX` packages are found. Please `pip install lightning[extra]` or one of them to enable TensorBoard support by default
Wed Oct 21 14:29:34 2026: Loading Boltz structure prediction weights
/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/utilities/migration/utils.py:56: The loaded checkpoint was produced with Lightning v2.5.0.post0, which is newer than your current Lightning version: v2.5.0
Wed Oct 21 14:29:49 2026: Finished loading Boltz structure prediction weights
Wed Oct 21 14:29:49 2026: Starting structure inference
/home/bmcuser/boltz22/lib/python3.11/site-packages/torch/cuda/__init__.py:287: UserWarning: 
NVIDIA GeForce RTX 5070 Ti with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA GeForce RTX 5070 Ti GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(
You are using a CUDA device ('NVIDIA GeForce RTX 5070 Ti') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Traceback (most recent call last):
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/call.py", line 47, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 898, in _predict_impl
    results = self._run(model, ckpt_path=ckpt_path)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 982, in _run
    results = self._run_stage()
              ^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 1021, in _run_stage
    return self.predict_loop.run()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/loops/utilities.py", line 179, in _decorator
    return loop_run(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/loops/prediction_loop.py", line 105, in run
    self.setup_data()
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/loops/prediction_loop.py", line 162, in setup_data
    length = len(dl) if has_len_all_ranks(dl, trainer.strategy, allow_zero_length) else float("inf")
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/utilities/data.py", line 105, in has_len_all_ranks
    if total_length == 0:
       ^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bmcuser/boltz22/bin/boltz", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/boltz/main.py", line 1355, in predict
    trainer.predict(
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 859, in predict
    return call._call_and_handle_interrupt(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/call.py", line 68, in _call_and_handle_interrupt
    trainer._teardown()
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 1005, in _teardown
    self.strategy.teardown()
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/pytorch_lightning/strategies/strategy.py", line 536, in teardown
    self.lightning_module.cpu()
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/lightning_fabric/utilities/device_dtype_mixin.py", line 82, in cpu
    return super().cpu()
           ^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1133, in cpu
    return self._apply(lambda t: t.cpu())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/torch/nn/modules/module.py", line 915, in _apply
    module._apply(fn)
  File "/home/bmcuser/boltz22/lib/python3.11/site-packages/torchmetrics/metric.py", line 907, in _apply
    _dummy_tensor = fn(torch.zeros(1, device=self.device))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

