Comments on: Who Would Like to Have the Entire ChemSpider Database? http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html Building Community for Chemists Tue, 16 Mar 2010 21:05:32 +0000 http://wordpress.org/?v=2.8.4 hourly 1 By: ChemSpider Blog » Blog Archive » Another Response to Constructive Feedback from Peter Murray-Rust… http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html/comment-page-1#comment-3582 ChemSpider Blog » Blog Archive » Another Response to Constructive Feedback from Peter Murray-Rust… Mon, 15 Oct 2007 06:44:23 +0000 http://www.chemspider.com/blog/?p=146#comment-3582 [...] ChemSpider contribute to the community…and support PubChem [...] [...] ChemSpider contribute to the community…and support PubChem [...]

]]> By: Antony Williams http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html/comment-page-1#comment-3554 Antony Williams Sun, 14 Oct 2007 18:50:54 +0000 http://www.chemspider.com/blog/?p=146#comment-3554 Yes, you are right...we have no experience of what it will do to our servers to have everyone downloading data in one big file. But, once it's deposited at PubChem people can take it from there anyways and they will likely be able to support that better than us. One thing to clarify is that the ChemSpider database we are donating to PubChem will NOT contain any of the predicted properties etc as we do not have permission to give them away. It is irrelevant anyway...PubChem will remove them from the submission and replace with their own predicted properties for homogeneity purposes. Yes, you are right…we have no experience of what it will do to our servers to have everyone downloading data in one big file. But, once it’s deposited at PubChem people can take it from there anyways and they will likely be able to support that better than us.

One thing to clarify is that the ChemSpider database we are donating to PubChem will NOT contain any of the predicted properties etc as we do not have permission to give them away. It is irrelevant anyway…PubChem will remove them from the submission and replace with their own predicted properties for homogeneity purposes.

]]> By: Egon Willighagen http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html/comment-page-1#comment-3531 Egon Willighagen Sun, 14 Oct 2007 07:45:56 +0000 http://www.chemspider.com/blog/?p=146#comment-3531 Antony, with "You have limited rights in this regard", I assume you really mean: "You have limited options in this regard: while you are entitled (have the rights) to download everything, because of practical reasons we do not provide an easy to download one-contains-all zip file." Is that a correct analysis? Antony, with “You have limited rights in this regard”, I assume you really mean: “You have limited options in this regard: while you are entitled (have the rights) to download everything, because of practical reasons we do not provide an easy to download one-contains-all zip file.”

Is that a correct analysis?

]]> By: Antony Williams http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html/comment-page-1#comment-2798 Antony Williams Mon, 24 Sep 2007 23:41:12 +0000 http://www.chemspider.com/blog/?p=146#comment-2798 Rajarshi...as yet we have not needed to delete any entries. It does not mean that it might never happen but reasons for this are not foreseen at present. In terms of incremental updates we are more likely to do it on an as-needed basis (when there is a significant jump in the number of structures). At least once a month is foreseen...maybe more often. It is all about where we apply our limited resources at present. Rajarshi…as yet we have not needed to delete any entries. It does not mean that it might never happen but reasons for this are not foreseen at present.

In terms of incremental updates we are more likely to do it on an as-needed basis (when there is a significant jump in the number of structures). At least once a month is foreseen…maybe more often. It is all about where we apply our limited resources at present.

]]> By: Rajarshi Guha http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html/comment-page-1#comment-2786 Rajarshi Guha Mon, 24 Sep 2007 13:32:01 +0000 http://www.chemspider.com/blog/?p=146#comment-2786 Egons solution is a good idea for bandwidth managment, but that is not really a policy problem. What gets placed in the update (if there is one) is more important for downstream infrastructure. We face a similar problem with Pubchem Bioassay data - they have no update mechamism and basically dump new assays as they arrive. I could impose a monthly schedule on my side, but there are no monthly incrementals Egons solution is a good idea for bandwidth managment, but that is not really a policy problem.

What gets placed in the update (if there is one) is more important for downstream infrastructure. We face a similar problem with Pubchem Bioassay data – they have no update mechamism and basically dump new assays as they arrive. I could impose a monthly schedule on my side, but there are no monthly incrementals

]]> By: Egon Willighagen http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html/comment-page-1#comment-2779 Egon Willighagen Mon, 24 Sep 2007 08:02:40 +0000 http://www.chemspider.com/blog/?p=146#comment-2779 This is an interesting situation... if all data is (or would be) 'open data', then I could imagine a distributed FTP server setup; Linux distributions face the same problem. Or maybe a P2P network would even be better, with 100MB SDF (/CML.gz/whatever) files or so. The latter seems actually the most interesting setup. Really, I was waiting for this to happen; this allows you to make a difference; take things forward. ChemSpider, have a look at "debtorrent". It's a the Debian/Ubuntu package distribution system on top of torrent, one of the P2P systems. It was developed this summer within the Google Summer of Code, but it needs to be worked out how the .deb files are replaced by .cml.gz files (or sdf.gz). This is an interesting situation… if all data is (or would be) ‘open data’, then I could imagine a distributed FTP server setup; Linux distributions face the same problem. Or maybe a P2P network would even be better, with 100MB SDF (/CML.gz/whatever) files or so. The latter seems actually the most interesting setup.

Really, I was waiting for this to happen; this allows you to make a difference; take things forward. ChemSpider, have a look at “debtorrent”. It’s a the Debian/Ubuntu package distribution system on top of torrent, one of the P2P systems. It was developed this summer within the Google Summer of Code, but it needs to be worked out how the .deb files are replaced by .cml.gz files (or sdf.gz).

]]> By: Rajarshi Guha http://www.chemspider.com/blog/who-would-like-to-have-the-entire-chemspider-database.html/comment-page-1#comment-2771 Rajarshi Guha Mon, 24 Sep 2007 03:36:26 +0000 http://www.chemspider.com/blog/?p=146#comment-2771 What would your update policy be? Would incremental updates be provided (such as the monthlies that Pubchem does)? Do you ever delete any entries? What would your update policy be? Would incremental updates be provided (such as the monthlies that Pubchem does)? Do you ever delete any entries?

]]>