Copyright©2007 Antony Williams
As a web subscriber to JCIM (and frequent author to said journal) I was reading the recent articles related to PubChem. If bandwidth allows I’ll post some additional comments shortly but Rich Apodaca has already highlighted them.
In his posting Rich commented “The public is free to download and re-use the entire database of molecules and associated data. <...> witness both eMolecules and ChemSpider, two services that unashamedly exploit the PubChem resource. Expect to see more of this in the months ahead.”
Did we do this? Absolutely. We downloaded the data and used it to test our system since we couldn’t easily source 10 million chemical structures anywhere else. We were very honest in our first press release on March 24th where we said “… the launch of their ChemSpider Service (www.chemspider.com), an online resource for chemists to search, aggregate and data mine publicly available chemical data. At time of release over 10 million compounds are indexed in the ChemSpider database including the PubChem collection and data provided by a number of other collaborators. ….”
There is no Open Source software that we are aware of that can handle 10 million compounds. So, having constructed our system we needed to performance test it and PubChem was a great resource so we used it. The quality of the database (and I have talked about Zen and Quality already) was irrelevant at that point. We needed to test the system.
The ChemSpider dataset is now over 16 million compounds and the PubChem dataset contributes 10.3 Million of those (10297250 to be precise), just over 60%. We continue to add databases (1,2,3) and have put significant effort into curating the data ourselves and with the assistance of our users (4,5,6). We’ve committed to return curated records back to Pubchem and glad to do it.
We presently have more data to add to the ChemSpider database, unique structures that are not contained in PubChem. Where allowable we will deposit these to PubChem in the future. For right now we have more important things to do, specifically address those Five Things We Don’t Like About ChemSpider.
The work on Quality checking continues and there will be an abundance of data to return to PubChem. See the recent discussions about Taxol and Thimerasol as examples. There’s a lot more where those came from.
Regarding the comment “… eMolecules and ChemSpider, two services that unashamedly exploit the PubChem resource.” I’ll confess that the PubChem database is on ChemSpider. However, eMolecules is a database of 7 million chemicals now and they have received data from many chemical vendors so it appears that they have only utilized a subset of PubChem as appropriate to their business model.
PubChem is a great resource..for those of you interested in how the data are being used/examined I do recommend the papers Rich has pointed out. Also, add Depth-First, Rich’s site, to your BlogRoll…it’s one of the best reads in the blogosphere in my opinion.Stumble it!