You may have seen a recent pre-print [1] from Jain et al. with strongly worded claims against the experimental results in our DiffDock paper [2]. We initially declined to respond as we saw that this preprint contained falsehoods, misleading comparisons, seemingly deliberate omissions, ...1/n - ThreadSky

gcorso.bsky.social • 80 days ago

You may have seen a recent pre-print [1] from Jain et al. with strongly worded claims against the experimental results in our DiffDock paper [2]. We initially declined to respond as we saw that this preprint contained falsehoods, misleading comparisons, seemingly deliberate omissions, ...1/n

Comments

gcorso.bsky.social•80 days ago

and was written in a tone not intended as a serious research paper. Unfortunately, the preprint has now received undue attention on social media and beyond. After receiving inquiries for us to respond, we decided to do so. ... 2/n

gcorso.bsky.social•80 days ago

First, we wish to emphasize that, similar to all of our papers, we published all the code and data. Doing so ensures that the work is always reproducible and any aspect of it can be openly assessed and critiqued by anyone. ...3/n

gcorso.bsky.social•80 days ago

Further, we have openly acknowledged limitations in the original manuscript, and have addressed these limitations in follow-up papers [3,4,30]. We cherish all the constructive feedback we have received on our work and the collaborations that arose to improve the method. ... 4/n

gcorso.bsky.social•80 days ago

The main false claim in Jain et al. is that the DiffDock paper poorly selected train/test splits and incorrectly ran baselines. As Jain et al. well know, neither the dataset split nor the contended baselines were introduced in the DiffDock paper ... 5/n

gcorso.bsky.social•80 days ago

we used baselines and benchmarks established by prior peer-reviewed work [6,7], and these have been used in many other concurrent [8,9] and subsequent works [10,11,12, …] as well. We certainly agree that the specific benchmark did not sufficiently evaluate docking generalization, ... 6/n

gcorso.bsky.social•80 days ago

which is why we also reported performance on unseen receptors in the original paper [2, Table 7 and Figure 7] and we proposed better evaluations in later papers [3,4]. This conclusion was shared by many other groups in the field and has helped build better alternatives [13,4]. ... 7/n

fdreyer.bsky.social•79 days ago

DiffDock was a really cool paper that kick-started a whole wave of creative new approaches. There has been a lot of exciting work since then and even if AI-based docking is not fully mature yet, recent progress has been impressive. Looking forward to reading about the next iterations of Boltz-1!

Comments

Posting Rules

Reply