Hi Alexis,

  Thanks for the videos!  I am amazed at how well Claude handled ChimeraX and my example queries.  A key aspect I didn't understand until you made these videos is that when Claude / MCP sends a command to ChimeraX it gets back the JSON response which includes what was written to the Log.  So it sees when the command generated an error, and then it tries to fix it.  Wow, and does it ever try!  I was dazzled by the number of different approaches it took to my final question about how many hydrogen bonds fentanyl makes with the opioid receptor.

  Zach on the UCSF ChimeraX team setup Claude with MCP and we ran the opioid example here to get a better look.  I was amazed in your video that Claude figured out which chain was the anti-body by asking ChimeraX for the sequences of the chains and then identifying which sequence looked like an antibody.  The PDB data does not say "antibody".  Likewise the PDB entry 8ef5 identifies fentanyl as chemical component code :7v7 and iupac name N-phenyl-N-[1-(2-phenylethyl)piperidin-4-yl]propanamide output to the Log that Claude got as json and figured out that was fentanyl.  Very nice how Claude heavily uses ChimeraX command output.  Claude was often trying the "info" ChimeraX command to get more knowledge about the atomic structures.

  Of course, it was also stunning how simple things eluded it that no human would trip over.  The "hbond" command reported "0 hydrogen bonds found" and yet Claude tried heroically to count the hydrogen bonds with many commands.  And Claude hid the entire opioid structure so nothing was shown and of course was oblivious because it cannot see the image of the graphics is black.  Or maybe MCP is able to capture an image of the application.

  My impression is that ChimeraX is remarkably well suited for AI agent control since everything can be done by commands, the command documentation is extremely detailed thanks to Elaine in our lab, and ChimeraX gives JSON feedback on the results of every command.  One way ChimeraX is not ideal for agents is that Claude often tried old Chimera command syntax and has a hard time distinguishing online information about the two different programs Chimera and ChimeraX.  Humans have the same problem.

  I agree that if we study some of the failure modes we can probably make Claude / MCP work better with ChimeraX.  Claude queried ChimeraX with the info command many times.  I think if we exposed more of the PDB human readable text annotations through the info command it would help Claude connect what we request in our typed prompts to the atomic structures.

Tom



On Oct 16, 2025, at 1:09 PM, Alexis Rohou via ChimeraX-users <chimerax-users@cgl.ucsf.edu> wrote:

Hi Tom,

Thanks for suggesting test scenarios. See the video captures below. Basically, it can't quite succeed without some help. The basic thing is that I should probably give it a "recipe" for getting the chain annotations at the command line (what would you suggest?). In the case of the opioid receptor it also confused itself and hid all the chains so I had to tell it about that. This is why I think having visual feedback would be very valuable (also, to click on buttons when setting up ISOLDE, which doesn't have every function available at the command line).

Ntca (no help; it selects the wrong chain)
Ntca (with help - I pointed it to the correct chains, asked it to double-check but I'm not sure it really double-checked or just blindly followed by hint):

 251016_chimerax_mcp_demo_ntca_tf_only_with_help.mov

Opioid receptor (it needed my help there because early on it hid all the chains):

 251016_chimerax_mcp_demo_fenta_OR_with_help.mov


Note that all I've told the MCP so far is how to start, stop chimerax, check if it's still running, list sessions that are running, and how to submit commands. I have given it no additional context to help it use the commands properly (the obvious thing to do would be to tell it about the "help" and "usage" commands, and to point it to the docs). So any success at all is from the LLM's baked-in training.

I think by studying a few failure modes and giving it tools to help it work things out, it should work pretty well, eventually.

Cheers,
Alexis

On Thu, Oct 16, 2025 at 10:16 AM Tom Goddard <goddard@sonic.net> wrote:
Hi Alexis,

  Beautiful!  You and AI are in charge of making our end-of-year ChimeraX holiday card!

  I'm impressed Claude got the right ChimeraX commands.  A user on the ChimeraX mailing list several days ago asked about how to color a structure red and said he tried many proposals from ChatGPT and none worked.  (They were all using old Chimera syntax "color red #1" instead of ChimeraX "color #1 red").

  My first thought is when does this fail.  What if you try a more realistic use case?  For example,

  Show me trascription factor Ntca in pdb 8h40
  I'd like to see just the trascription factor bound to the DNA without the polymerase
  Could you show those in ribbon style without atoms

Or how about

  I'd like to look at how fentanyl interacts with the opioid receptor, PDB 8ef5
  Could you hide the G proteins?  Also hide the antibody.
  Show the receptor as ribbon
  Show just one copy of the receptor
  Can you show the protein residues near fentanyl in stick style
  Can you show hydrogen bonds between fentanyl and the protein?  How many are there?

If the Claude MCP agent could do this I would be blown away.

  Regarding having the agent control the ChimeraX GUI (push buttons, use menus) when there are no equivalent commands, I think that is not too necessary with ChimeraX because almost every GUI actions has an equivalent command and those commands are echoed to the ChimeraX Log panel, so it is even pretty easy to learn the equivalent commands.

Tom




On Oct 15, 2025, at 10:25 PM, Alexis Rohou via ChimeraX-users <chimerax-users@cgl.ucsf.edu> wrote:

Hi Tom,

I mocked something up quickly. See demos below, where I'm using Cursor as the MCP client and agent with Claude 4.5 Sonnet as the LLM:

Demo 0 (start ChimeraX, open a pdb, color it)
Demo 1 (change colors)
Demo 2 (change colors again, the JA version)

Curious to hear if anyone else is working on something similar. I think getting commands to work nicely should be easy (famous last words?). Next level will be to have the agent interact directly with the GUI, check what is being displayed, and possibly click on buttons when there are no commands to perform specific tasks. There are existing MCPs that I hope will make this feasible.

Cheers,
Alexis

On Wed, Oct 15, 2025 at 7:12 PM Tom Goddard <goddard@sonic.net> wrote:
Hi Alexis,

  The UCSF ChimeraX team has not looked at using ChimeraX as an AI agent or at Model Context Protocol (MCP).  Your links to github repositories that have tried to hook up AI agents to ChimeraX with MCP were a surprise to me.  It would be cool to see a video using an AI agent with ChimeraX.

Tom



On Oct 15, 2025, at 6:29 PM, Alexis Rohou via ChimeraX-users <chimerax-users@cgl.ucsf.edu> wrote:

Hi all,

I would like to use ChimeraX via an agent. Quick googling suggests I'm not the first person to think of this (here, here), but before I dive in I thought I'd check - anyone else actively working on this? Or maybe you have worked on it and hit a roadblock?

Cheers,
Alexis
_______________________________________________
ChimeraX-users mailing list -- chimerax-users@cgl.ucsf.edu
To unsubscribe send an email to chimerax-users-leave@cgl.ucsf.edu
Archives: https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/

_______________________________________________
ChimeraX-users mailing list -- chimerax-users@cgl.ucsf.edu
To unsubscribe send an email to chimerax-users-leave@cgl.ucsf.edu
Archives: https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/

_______________________________________________
ChimeraX-users mailing list -- chimerax-users@cgl.ucsf.edu
To unsubscribe send an email to chimerax-users-leave@cgl.ucsf.edu
Archives: https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/