cclib-devel Mailing List for cclib

Jeff,

 I've installed 5.1 already. This is a good occasion to stir things up with 
cclib again, since the project has been dormant for some time now :)

 I think the idea behind the cclib test suite is to have tests as similiar as 
possible for all the parsers. So the first thing would be to create and run 
jobs in NWChem that correspond to the tests that already exist for the other 
parsers. For example, yesterday I added single-point Hartree-Fock and DFT 
inputs/outputs (for divinylbenzene like for all our supported programs).

 There is also a regression system in cclib. That is the place to add QA jobs 
shipped with NWChem - at least those that are not parsed satisfactorily when 
the parser is up and running.

 I guess Noel or Adam might have something to add to that, and perhaps some 
general comments about the future of cclib?

Cheers,
Karol

On Monday 31 March 2008 16:10, Jeff Hammond wrote:
> Karol,
>
> That sounds good.  It might work well to add the test jobs that come
> with NWChem in the QA (quality assurance) directory.  Those tests are
> up-to-date - unlike those in the examples directory - and are already
> parsed by an included Perl script, hence one need only translate Perl
> to Python if that is the easiest route.
>
> The biggest changes from 5.0 to 5.1 other than bug fixes and tuning
> are the inclusion of property codes (primarily polarizabilities).
> Since this was my contribution to 5.1, I'm going to start there with
> respect to adding parsing features to CCLib.  Thus, it is not
> necessary to get 5.1 if you don't otherwise need it.
>
> Best,
>
> Jeff
>
> On Fri, Mar 21, 2008 at 8:21 AM, Karol M. Langner
>
> <kar...@gm...> wrote:
> >   I'd help out with NWChem, since I sometimes use it also. I've just
> > created a working branch for it in our SVN repository. When I get the
> > newest version (NWChem5.1 is it?) and the time I'll add some test jobs.
> >
> >  Cheers,
> >  Karol
> >
> >  --
> >  written by Karol Langner
> >  Fri Mar 21 14:20:59 CET 2008

-- 
written by Karol Langner
Mon Mar 31 18:16:51 CEST 2008

Karol,

That sounds good.  It might work well to add the test jobs that come
with NWChem in the QA (quality assurance) directory.  Those tests are
up-to-date - unlike those in the examples directory - and are already
parsed by an included Perl script, hence one need only translate Perl
to Python if that is the easiest route.

The biggest changes from 5.0 to 5.1 other than bug fixes and tuning
are the inclusion of property codes (primarily polarizabilities).
Since this was my contribution to 5.1, I'm going to start there with
respect to adding parsing features to CCLib.  Thus, it is not
necessary to get 5.1 if you don't otherwise need it.

Best,

Jeff

On Fri, Mar 21, 2008 at 8:21 AM, Karol M. Langner
<kar...@gm...> wrote:
>   I'd help out with NWChem, since I sometimes use it also. I've just created a
>  working branch for it in our SVN repository. When I get the newest version
>  (NWChem5.1 is it?) and the time I'll add some test jobs.
>
>  Cheers,
>  Karol
>
>  --
>  written by Karol Langner
>  Fri Mar 21 14:20:59 CET 2008
>



-- 
Jeff Hammond
The University of Chicago

JCompChem have sent me a complimentary copy of the cclib issue (as I
call it :-). It makes me realise how long it's been since I actually
looked through a paper copy of a journal. Probably not since my PhD...

Noel

On Monday 10 March 2008 15:20, Noel O'Boyle wrote:
> On 10/03/2008, Jeff Hammond <jef...@gm...> wrote:
> > Dear cclib developers,
> >
> >  I read your JCC article and felt very lame because I'm still using
> >  'grep' to parse output files, despite knowing far better tools for the
> >  purpose.  cclib would seem to mitigate my laziness, except that it
> >  doesn't currently support NWChem, Aces or Dalton, three of the
> >  programs I use most often.  Although I use a fourth code, PSI,
> >  infrequently, but as it is the only GPLed quantum chemistry code that
> >  compares to the commercial codes, perhaps you are interested in
> >  supporting it as well.  It's also the only fully C/C++ QC code out
> >  there and will likely overtake GAMESS as the de facto free code for
> >  non-DOE (ie supercomputer) users.  While PSI and Aces are more for the
> >  high-accuracy community, NWChem and Dalton are popular in the DFT
> >  crowd, particularly for their extensibility (parallel and properties)
> >  which is unrivaled by the codes you currently support.
> >
> >  I wonder if I can help you support these programs.  I don't have time
> >  to learn all that is necessary to write the parser code for the MO
> >  vectors, but I should be able to write a parser for all the scalers
> >  and other simple stuff.  If someone wants to deal with the MO vectors
> >  and related quantities, I would be happy to provide all manner of
> >  output files and technical support to that end.  All these codes all
> >  store MO vectors as binary data somewhere and should use a standard
> >  format.  Dalton and Aces may be the same since they use the integral
> >  code.  I don't know the internal format of NWChem but I'm a developer
> >  so it would somewhat pathetic if I was not able to figure it out from
> >  the source.
> >
> >  If you have any suggestions for how to proceed, please let me know.  I
> >  can't say that I'm going to make progress in the next few weeks due to
> >  ACS preparations, but I should have time for it after that.
>
> This sounds fantastic Jeff. Of course we're interested, and we'd
> appreciate any help you can give. I'm up to my eyeballs until the ACS
> also. Why don't we meet up there and have a chat? (cclib has blanket
> coverage of the ACS this year with me at New Orleans, and Adam at
> Philadelphia)

Jeff & Noel,

 I'd help out with NWChem, since I sometimes use it also. I've just created a 
working branch for it in our SVN repository. When I get the newest version 
(NWChem5.1 is it?) and the time I'll add some test jobs.

Cheers,
Karol

-- 
written by Karol Langner
Fri Mar 21 14:20:59 CET 2008

On 20 Mar 2008, at 16:17, Noel O'Boyle wrote:
> As you are possibly aware (?), Peter Knowles of Molpro was at the
> meeting in Runcorn and spoke about the XML developments, which I was
> very much in favour of. We do currently have some support for Molpro.
> For sure the scripting capabilities mean that the general problem is
> intractable, but if we can extract the relevant data from the most
> common types of jobs, we will be happy. We welcome any help we can
> get, so feel free to start asking naive questions at the first
> opportunity.

I knew Peter was at the meeting, I'm glad you managed to get a word in  
with him as he obviously knows a lot more about the xml than I do.  
I'll certainly _try_ to see if I can help with parsing some of it soon.

> Also, as I said, along with OpenBabel, Avogadro, GAMESS-UK and others,
> we are going to start pushing for QM codes to release a specification
> of their machine-readable formats (which I assume that Molpro also
> has). This should be a more tractable problem.

Molpro can dump several different machine readable files, although one  
is probably much more interesting than the others. I don't personally  
know the format for them but there are other devs down the corridor  
who can probably tell me. I'll have a chat with them when I get back  
from vacation in a weeks time.


Grant

On 18/03/2008, Grant Hill <Hi...@ca...> wrote:
> Firstly, congrats on the paper guys.
>
>
>
>  On Fri, 2008-03-14 at 19:58 +0000, Noel O'Boyle wrote:
>
>  > Someone pointed me to a UK calculation farm which might give me access
>  > to a variety of codes. I haven't chased this up - it may/may not be
>  > possible.
>
>
> Is that NSCCS? I've had pretty good experiences with them so far,
>  although it's been for more "traditional" uses. Grant proposals could be
>  reasonably short and we heard back within a couple of weeks. Nice piece
>  of iron too.
It is. My requirements are quite different from the usual user, e.g.
typical jobs should run in less than 5 minutes. As I said, I have
still to look into this. I hope they will look kindly on a project
which attempts to serve the broader community.

>  > At the meeting, there was also a consensus that we should contact all
>  > QM codes and ask for a formal description of their machine readable
>  > formats. If any QM codes respond, both ourselves and OpenBabel will
>  > then add support.
>
>
> The new version of Molpro (shouldn't be too far off) looks like it'll be
>  dumping xml'ised output by default, or certainly with very little
>  effort. While this should make the output easy to parse there will still
>  be the Molpro scripting capabilities stumbling block. Assuming you guys
>  can put up with some very naive questions I may be able to take a look
>  at adding something to cclib to parse the xml output in at least simple
>  cases. That, of course, depends on my free time and learning enough
>  Python to make it happen. Despite my best intentions I'm only half way
>  through the O'Reilly Learning Python.

As you are possibly aware (?), Peter Knowles of Molpro was at the
meeting in Runcorn and spoke about the XML developments, which I was
very much in favour of. We do currently have some support for Molpro.
For sure the scripting capabilities mean that the general problem is
intractable, but if we can extract the relevant data from the most
common types of jobs, we will be happy. We welcome any help we can
get, so feel free to start asking naive questions at the first
opportunity.

Also, as I said, along with OpenBabel, Avogadro, GAMESS-UK and others,
we are going to start pushing for QM codes to release a specification
of their machine-readable formats (which I assume that Molpro also
has). This should be a more tractable problem.

Noel

>  Regards,
>
>  Grant
>
>
>
>  -------------------------------------------------------------------------
>  This SF.net email is sponsored by: Microsoft
>  Defy all challenges. Microsoft(R) Visual Studio 2008.
>  http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>  _______________________________________________
>  cclib-devel mailing list
>  ccl...@li...
>  https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Firstly, congrats on the paper guys.


On Fri, 2008-03-14 at 19:58 +0000, Noel O'Boyle wrote:

> Someone pointed me to a UK calculation farm which might give me access
> to a variety of codes. I haven't chased this up - it may/may not be
> possible.

Is that NSCCS? I've had pretty good experiences with them so far,
although it's been for more "traditional" uses. Grant proposals could be
reasonably short and we heard back within a couple of weeks. Nice piece
of iron too.


> At the meeting, there was also a consensus that we should contact all
> QM codes and ask for a formal description of their machine readable
> formats. If any QM codes respond, both ourselves and OpenBabel will
> then add support.

The new version of Molpro (shouldn't be too far off) looks like it'll be
dumping xml'ised output by default, or certainly with very little
effort. While this should make the output easy to parse there will still
be the Molpro scripting capabilities stumbling block. Assuming you guys
can put up with some very naive questions I may be able to take a look
at adding something to cclib to parse the xml output in at least simple
cases. That, of course, depends on my free time and learning enough
Python to make it happen. Despite my best intentions I'm only half way
through the O'Reilly Learning Python.

Regards,

Grant

Hello all,

I will write in more detail later, but things went very well at the
tools meeting. It was really great to meet all these people, mainly
developers, who seemed to be interested in the same stuff.

GAMESS-UK developer Paul Sherwood and CCP1GUI developer Jens Thomas
are interested in using cclib as a front end to their code. CCP1GUI
actually does some pretty cool visualisations and is GPL to boot. It's
a well kept secret, it seems. Basically, the only thing holding them
back is how we handle gbasis. We're going to work with them to sort it
out, and maybe we can get them to help with other codes too (but one
step at a time). Basically, the problem is handling of Cartesian as
well as spherical basis functions, and also, it seems that the order
of the basis functions needs to be standardised (I'm a bit hazy on
this, but they seemed to be quite clear on the necessity for this).

Someone pointed me to a UK calculation farm which might give me access
to a variety of codes. I haven't chased this up - it may/may not be
possible.

At the meeting, there was also a consensus that we should contact all
QM codes and ask for a formal description of their machine readable
formats. If any QM codes respond, both ourselves and OpenBabel will
then add support.

Plugins are good. I'm thinking very strongly we should go down the
plugin route. It would allow upgrades to the latest parser simply by
dropping a new parser into a particular directory. This is especially
useful for programs that use cclib. It could just be a trivial change.

This is basically a brain dump, above. I will probably think of more,
and maybe write a blog post or two too. Any actual work on my part
I'll have to put off till after the ACS, though.

Regards,
   Noel

On 10/03/2008, Jeff Hammond <jef...@gm...> wrote:
> Dear cclib developers,
>
>  I read your JCC article and felt very lame because I'm still using
>  'grep' to parse output files, despite knowing far better tools for the
>  purpose.  cclib would seem to mitigate my laziness, except that it
>  doesn't currently support NWChem, Aces or Dalton, three of the
>  programs I use most often.  Although I use a fourth code, PSI,
>  infrequently, but as it is the only GPLed quantum chemistry code that
>  compares to the commercial codes, perhaps you are interested in
>  supporting it as well.  It's also the only fully C/C++ QC code out
>  there and will likely overtake GAMESS as the de facto free code for
>  non-DOE (ie supercomputer) users.  While PSI and Aces are more for the
>  high-accuracy community, NWChem and Dalton are popular in the DFT
>  crowd, particularly for their extensibility (parallel and properties)
>  which is unrivaled by the codes you currently support.
>
>  I wonder if I can help you support these programs.  I don't have time
>  to learn all that is necessary to write the parser code for the MO
>  vectors, but I should be able to write a parser for all the scalers
>  and other simple stuff.  If someone wants to deal with the MO vectors
>  and related quantities, I would be happy to provide all manner of
>  output files and technical support to that end.  All these codes all
>  store MO vectors as binary data somewhere and should use a standard
>  format.  Dalton and Aces may be the same since they use the integral
>  code.  I don't know the internal format of NWChem but I'm a developer
>  so it would somewhat pathetic if I was not able to figure it out from
>  the source.
>
>  If you have any suggestions for how to proceed, please let me know.  I
>  can't say that I'm going to make progress in the next few weeks due to
>  ACS preparations, but I should have time for it after that.

This sounds fantastic Jeff. Of course we're interested, and we'd
appreciate any help you can give. I'm up to my eyeballs until the ACS
also. Why don't we meet up there and have a chat? (cclib has blanket
coverage of the ACS this year with me at New Orleans, and Adam at
Philadelphia)

Noel


>  Best,
>
>  Jeff
>
>  --
>  Jeff Hammond
>  The University of Chicago
>
>  -------------------------------------------------------------------------
>  This SF.net email is sponsored by: Microsoft
>  Defy all challenges. Microsoft(R) Visual Studio 2008.
>  http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>  _______________________________________________
>  cclib-devel mailing list
>  ccl...@li...
>  https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Dear cclib developers,

I read your JCC article and felt very lame because I'm still using
'grep' to parse output files, despite knowing far better tools for the
purpose.  cclib would seem to mitigate my laziness, except that it
doesn't currently support NWChem, Aces or Dalton, three of the
programs I use most often.  Although I use a fourth code, PSI,
infrequently, but as it is the only GPLed quantum chemistry code that
compares to the commercial codes, perhaps you are interested in
supporting it as well.  It's also the only fully C/C++ QC code out
there and will likely overtake GAMESS as the de facto free code for
non-DOE (ie supercomputer) users.  While PSI and Aces are more for the
high-accuracy community, NWChem and Dalton are popular in the DFT
crowd, particularly for their extensibility (parallel and properties)
which is unrivaled by the codes you currently support.

I wonder if I can help you support these programs.  I don't have time
to learn all that is necessary to write the parser code for the MO
vectors, but I should be able to write a parser for all the scalers
and other simple stuff.  If someone wants to deal with the MO vectors
and related quantities, I would be happy to provide all manner of
output files and technical support to that end.  All these codes all
store MO vectors as binary data somewhere and should use a standard
format.  Dalton and Aces may be the same since they use the integral
code.  I don't know the internal format of NWChem but I'm a developer
so it would somewhat pathetic if I was not able to figure it out from
the source.

If you have any suggestions for how to proceed, please let me know.  I
can't say that I'm going to make progress in the next few weeks due to
ACS preparations, but I should have time for it after that.

Best,

Jeff

-- 
Jeff Hammond
The University of Chicago