cclib-devel Mailing List for cclib

1) I've been working on the ADF parser, however I ran into a problem with 
reading "aonames". Basically, if there is any symmetry (and nosymmetry isn't 
specified for the ADF calc), it uses linear combinations of orbitals on 
different atoms for the atomic basis. As an example, look for the SFOs 
section on line 1371 of dvb_sp.adfout. SFO 1Ag is constant (C1_1S + C4_1S), 
where constant is 1/sqrt(2). I can think of two solutions. First, parse it as 
it is so aonames become something like 1/2 C1_1S + C4_1S. My problem with 
this is handling more complicated cases with 3+ atoms might lead to more 
difficult coefficients. Second, ignore it and make sure cclib users know that 
if they want to do anything with aonames, they need to specify nosymmetry in 
the ADF calc.

2) In looking at the Gaussian parser, I wasn't able to find a variable for 
storing each structure found for a gaussian calculation. What do you think 
about adding a class for Atoms (name or number, x, y, z in either angs or 
bohr), a class for Molecules (either a list of Atoms, or more complicated 
with dictionaries?), and geoms[ ] and geomcoords[ ] in the parser classes?

3) What if certain calculations don't implement the same parameters for 
geotargets/geovalues and scftargets/scfvalues? Look at the differences 
between the ADF and Gaussian calculations. Is there going to be a variable to 
keep track of the names for each parameter? Or maybe a set of pre-defined 
dictionaries (e.g. MaxForce:[ ], RMS Force:[ ], Energy:[ ], dEnergy:[ ], 
etc.) should be used instead to keep some consistency across each type of 
calculation?

Adam

P.S. Noel, I hope you enjoyed your travels...

-- 
Want a web browser that is fully customizable? 

Go to http://www.mozilla.org/products/firefox/

to download the newest version of Firefox and 
its available extentions.

> Got it. Works fine, although doesn't play well with the logger, but you
> can turn the logger off or get it to write to a file.

Yeah. This is done with calling G03 with a third argument of logging.ERROR or 
such.

> I don't know where the slow step is. I've added a call to random so that
> it only calls it every so many steps, but there isn't a massive
> improvement.

I also added optional arguments to parse so that you can specify how "fine" or 
"course" you want the update to be. The first option is the fine argument, 
which is used for checks against random() inside "slow" parts like parsing 
overlap and coefficients. The second option is the course argument, which is 
used for checks that aren't "slow".

> Not really. I think this is the price you pay. A bit like Heisenberg's
> uncertainty principle - by observing the progress you change it. :-) If
> we could identify more clearly the slow step - for example, is it the
> call to tell()? - then we could think about how to improve it. I tried
> various things to speed up the textprogress, but they only made a minor
> difference.

I agree that it is probably the price we pay. I've tried numerous things to 
speed it up, but nothing seems to be too significant. I think I'd rather know 
the progress and add on (at most) a few seconds, and of course, it's 
ultimately up to the developer using cclib.

> BTW, you may be interested in:
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/299207
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/168639

I didn't exactly take their work, but I did figure out how to make the text 
progress work like is expected. I haven't finished every part of it, but it 
should still be useful. Of course, the text progress is more an example for 
people to follow. I just wanted to establish the API used for any progress 
classes. Basically two functions need to be implemented:

1) initialize, which expects nsteps and text
2) update, which expects step and text

I've also added a few test files for ADF, although I haven't started working 
on the parser. Such a daunting task!

Adam

On Thu, 2006-03-09 at 12:03 -0800, Adam Tenderholt wrote:
> So I added an example progress class called TextProgress which basically 
> prints out bars saying how much progress has been made, and made necessary 
> changes to G03 parser to allow use of this class.
> 
> If you want to use it, first create an instance, and pass that instance to the 
> G03 class along with the filename:
> 
> progress=TextProgress()
> p=G03(filename,progress)
> p.parse()

Got it. Works fine, although doesn't play well with the logger, but you
can turn the logger off or get it to write to a file.

> I changed the __init__ function of G03 to accept any amount of arguments, and 
> pass that onto the baseclass. See the SVN diffs for more details. This allows 
> the user of cclib to not specify a parser and continue as before.
Yes, and seems to be just as fast if you don't specify a progress.

> Unfortunately, not all of this message is good news. Calling G03 with a 
> TextParser adds a 20-25% overhead: 4 seconds becomes 5ish, 21 seconds becomes 
> more like 26, etc. I think the problem is that it calls the 
> progress.update(step) far too frequently.
I don't know where the slow step is. I've added a call to random so that
it only calls it every so many steps, but there isn't a massive
improvement.

>  It may make more sense to identify 
> the slower parts of the parsing class, and only add progress.update there, 
> but I don't know for sure.
> 
> Any ideas on how to improve this?
Not really. I think this is the price you pay. A bit like Heisenberg's
uncertainty principle - by observing the progress you change it. :-) If
we could identify more clearly the slow step - for example, is it the
call to tell()? - then we could think about how to improve it. I tried
various things to speed up the textprogress, but they only made a minor
difference.

BTW, you may be interested in:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/299207
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/168639

Regards,
 Noel

So I added an example progress class called TextProgress which basically 
prints out bars saying how much progress has been made, and made necessary 
changes to G03 parser to allow use of this class.

If you want to use it, first create an instance, and pass that instance to the 
G03 class along with the filename:

progress=TextProgress()
p=G03(filename,progress)
p.parse()

I changed the __init__ function of G03 to accept any amount of arguments, and 
pass that onto the baseclass. See the SVN diffs for more details. This allows 
the user of cclib to not specify a parser and continue as before.

Unfortunately, not all of this message is good news. Calling G03 with a 
TextParser adds a 20-25% overhead: 4 seconds becomes 5ish, 21 seconds becomes 
more like 26, etc. I think the problem is that it calls the 
progress.update(step) far too frequently. It may make more sense to identify 
the slower parts of the parsing class, and only add progress.update there, 
but I don't know for sure.

Any ideas on how to improve this?

Adam



-- 
Want a web browser that is fully customizable? 

Go to http://www.mozilla.org/products/firefox/

to download the newest version of Firefox and 
its available extentions.

> cclib is now self-parsing!

Excellent. Did you upload utils.py, or is that not required?

> OK, not quite, but it does parse your test file now. The necessary magic
> is:
> $ python setup.py install (as root, although there are alternatives)
>
> >>> from cclib.parser import G03
> >>> p = G03("Mo4OSibdt2-sp.log")
> >>> p.parse()
>
> Things to do:
> (1) Change variable names
> (2) Change variable types
> (3) Move logging code into Logfile superclass
> (4) Code for progress bars/indicators
> (5) Write parsers for other programs.
>
> I will do 1-->3 for the Gaussian parser. I will put the names we agreed
> on into the docstring for the Logfile class, and we can take that as our
> guide.

I will work on 4, and learn how to parse ADF and Jaguar files (some of 5). 
Also, is it possible to make it so each parser has its own file? I feel like 
that would make the code more maintanable, but I don't know how easily it can 
be done to keep syntax like we have.

> Let's start sharing test files, and putting a few onto sourceforge too.
> It might be good to run the same few calculations on the same molecule
> using lots of different software. It would be good to submit these test
> files to the OpenBabel repository too. I'm reasonably familiar with the
> Python unit test framework, and so will cobble together some tests once
> we have a few files.

Is there an upper limit for filesize for test files? I think most of my single 
point Gaussian files are 50-100 MB, but I could probably find a few smaller 
ones if necessary. 

> For our first release, 0.1, 0.5 or whatever, let's aim towards getting
> functional parsers.
>
> Oh yeah, BTW, I think I've set up the SVN to send the commit comments to
> the cclib-svn mailing list. You might want to subscribe to this to keep
> uptodate. I'm not yet sure if I've set it up properly though.
>
> What do you think?

I think this is a good start. I'll play with it when I get the chance. :o)

Adam

cclib is now self-parsing!

OK, not quite, but it does parse your test file now. The necessary magic
is:
$ python setup.py install (as root, although there are alternatives)

>>> from cclib.parser import G03
>>> p = G03("Mo4OSibdt2-sp.log")
>>> p.parse()

Things to do:
(1) Change variable names
(2) Change variable types
(3) Move logging code into Logfile superclass
(4) Code for progress bars/indicators
(5) Write parsers for other programs.

I will do 1-->3 for the Gaussian parser. I will put the names we agreed
on into the docstring for the Logfile class, and we can take that as our
guide.

Let's start sharing test files, and putting a few onto sourceforge too.
It might be good to run the same few calculations on the same molecule
using lots of different software. It would be good to submit these test
files to the OpenBabel repository too. I'm reasonably familiar with the
Python unit test framework, and so will cobble together some tests once
we have a few files.

For our first release, 0.1, 0.5 or whatever, let's aim towards getting
functional parsers.

Oh yeah, BTW, I think I've set up the SVN to send the commit comments to
the cclib-svn mailing list. You might want to subscribe to this to keep
uptodate. I'm not yet sure if I've set it up properly though.

What do you think?

Regards,
 Noel

Hello everyone,

This is a test message to make sure that everything is set up correctly.

Regards,
 Noel