docutils-develop Mailing List for Docutils: Documentation Utilities (Page 407)

On Wed, 06 Nov 2002 21:26:26 -0500 David Goodger <go...@py...> wrote:

[...]
> So should directive options be translated too?

IMHO, yes. If we translate everything else, directive options should also be
translated. For consistency, if nothing else...

---
Adam Chodorowski <ad...@ch...>

File cabinet:
   A four drawer, manually activated trash compactor.

On Wed, 6 Nov 2002, David Goodger wrote:

> gr...@us... wrote:
> > and maybe this should be more visible to translators not only
> > a unittest, and could give warnings (verbose mode).
>
> What do you mean?

anyone doing a translation should start all_test, but at least
test_language.

> BTW, de.py needs translations of the "raw", "include", and "replace"
> directives.  Could you please?

i put the in as comments, dont know a proper word yet, would be a sentence.
but saying "include" in a document is not document content, but a processin=
g
command, the document would say "here the contents of xxx would be inserted=
"
or "see xxxx for details" but not "include". for me these are two
different audiences.

should we support both 'raw' and 'unbearbeitet'.

>
> There's a comment::

i like to comment, so i donot have to fetch TFM to know what i
thought last time.

>
>       'topic': 'topic',  # Inhalt, Thema or =DCberbegriff
>
> There can be many keys with the same value.  Two of them anyway;
> "inhalt" is already used for "contents".  Are they homonyms?

actually it is =FCberbegriff (a summarizing (herding) concept ?)
but this does not mean that this is the exact usage in document
processing. e.g. contents is something different for a book writer
an e-business monkey or a grocery worker.

inside the writer many things come as a topic and the writer has to
decide when it is a title, that is why topic_class is there.
so what is a topic: something summarizing (the title sums up the
document as does a chapter title ...)

but for me topic is also usable in german conversation.

i have to see in i18n to get the =FC right.

>
> Another::
>
>       'bild': 'image',
>       'figure': 'figure', # also Bild ?
>
> You can't use "bild" twice, but surely there's an appropriate word in
> German?  Something like "titled-image" (translated) may suffice.  The
> English "image" just means "picture", whereas "figure" has a larger
> meaning, like "diagram/illustration/explanation".

changed to "abbildung"

>
> I know no German, so I have to trust you as a native speaker to do
> your best.  Just don't translate "contents" as "my hovercraft is full
> of eels".  (Monty Python reference)

i am more in programming so thinking half english, as i understand
these are domain specific translations, so xp says get a domain expert.

> >> In programming languages, most people seem to be happy with
> >> English-based syntax.  AppleScript had language dialects, but I
> >> don't think it caught on (code in Japanese looked *very* strange to
> >> my eyes, but I'm biased).  Directives are similar to programming
> >> language keywords/statements/commands, but not the same.  What do
> >> people think?
> >
> > reST is no programming language,
>
> I agree.
>
> > i thought the text should be a valid document (as far as possible)
> > otherwise we might need a text-writer.
>
> Don't know what you mean by "we might need a text-writer".

if rest would be a programming language, like latex, or postscript
we might need a docutils/writers/text.py module.

>
> > and the test of course must know if it is required or optional.
>
> I think a test should fail if any aspect of a language isn't there.

yes, so your intention is "directives translation is optional"
so it is not an error if none is present, but if they are there
we test them.

>
> >> Another question, from the to-do list: Should we use the language
> >> module for directive option names?  In other words, should
> >> directive option names be translated?
>
> How about this one?  Right now in English we would write::
>
>     .. contents:: :local:
>
> In German, "contents" is translated, but "local" is not::
>
>     .. inhalt:: :local:
>
> So should directive options be translated too?

i had no need for directives up to know do not ask me about
directives options :-)

--=20
 BINGO: Lassen Sie sich was einfallen!
 --- Engelbert Gruber -------+
 SSG Fintl,Gruber,Lassnig   /
 A6410 Telfs Untermarkt 9  /
 Tel. ++43-5262-64727 ----+

On Wed, 6 Nov 2002, David Goodger wrote:

> Thanks for posting new PDFs.  For the benefit of others, here are the
> URLs:
>
>     http://docutils.sf.net/sandbox/grubert/tools.pdf
>     http://docutils.sf.net/sandbox/grubert/test.pdf
>
> These documents look *very* good!  You've made a lot of progress.

julien did the table stuff and i think we both learned a lot about
latexs table and other commands by this.

> Some observations for tools.pdf:
>
> * Since TeX uses curly quotes (which it expresses ``thusly''), perhaps
>   the LaTeX writer could convert regular double-quotes to TeX-style
>   quotes?  Or can TeX/LaTeX do that automatically?  Currently all
>   double-quotes look like (just blur your eyes a bit ;)::
>
>       99    99
>         this
>
>   whereas they should look like this::
>
>       66    99
>         this
>
>   If it's not feasible to get proper curly quotes, then the Writer
>   should use straight quotes.  Better to be ugly (straight quotes)
>   than wrong (bad curlies).

is in the todo (latex2e-README.txt (someday i wipe my sand/dustbox))


>   If the LaTeX Writer does this conversion, it should be careful *not*
>   to do it for literal blocks and inline literals.  No curly quotes in
>   Python code ;)
>
> * Field lists should be rendered as two-column tables, like the
>   docinfo fields.  See
>   http://docutils.sf.net/spec/doctree.html#field-list ("Details"
>   subsection, "Processing" item).

i want to stay away of tables as long as possible. in latex you
have tabular, tabularx, supertabular, longtable, ltxtable and
table. some are duplicates or extensions. but there is more than
one table environment. e.g. some tables may not be broken so
i use tabular, but then it might be moved around.

i would go for decriptions for docinfo fields too, this is like:

<b>field:</b> the description breaking here
  and continuing intended.

<b>another:</b> some other ...

the reason why i donot use it is the space between the entries.
but these are much easier than tables.

>
> * In tools.pdf, the source text ``<directory>`` is being rendered as
>   ``!directory?``, but with the inverse-! (&iexcl;) and inverse-?
>   (&iquest;) characters.
>
> * Shouldn't literal blocks be indented, or have some kind of
>   decoration (solid left margin, etc.)?  I think relying on the
>   typeface is too subtle; indentation helps distinguish literal blocks
>   from the surrounding text.  Inline literals look fine the way they
>   are.

donot talk about intendation (see test.txt line_block) latex
is sovereign over space vertical and horicontal.

> * (I'm sure you already are aware of this.)  The table in tools.pdf,
>   which is longer than one page, is not breaking across page
>   boundaries.  Also, the paragraphs inside table cells are being
>   merged together (ISTR this coming up before).

i did not have a look at it, packed tools.pdf just because
i wanted a real document, not only the test. but the breaking
environment is already in todo.

> * Table column widths can be taken from the "colwidth" attributes of
>   "colspec" elements in the doctree.  Attribute values are relative
>   numbers, originating from the number of character columns in the
>   source text.  For tools.txt, the table's colwidth values are "20"
>   and "50", so the first column should be 2/7 and the second column
>   5/7 wide.
>
> And test.pdf:
>
> * Bibliographic field lists (docinfo) should have ":" appended to each
>   field name.
>
> * I see you got the address line-breaking right.  Great!
>
> * The "dedication" and "abstract" bibliographic fields get transformed
>   to "topic" elements, specifically <topic class="dedication"> and
>   <topic class="abstract"> in the internal doctree.  Topic should be
>   rendered like sections (i.e., title above the body), only indented
>   or otherwise set off from the rest of the document.  See
>   http://docutils.sf.net/spec/doctree.html#topic .

i know: latex has an abstract environment and thanks (a footnote to
the document title), but as described earlier, i am not sure about
the direction. we use pagenumbering from latex and section numbering
from docutils (docutils has sectionnumbering because html does not have
it, but docutils uses <ol> numbering from html).

> * It would be useful if "problematic" elements acted as internal
>   references to their system messages.  Back-links are also useful,
>   for system messages as well as for footnotes and citations.

i tried to get the picture, linking except the bookmarks, are by
julien. so more to learn for me.

>
> * From doctree.txt, definition list terms and classifiers "should be
>   separated visually, typically by spaces plus a colon or dash."  I'd
>   recommend a colon, so the input looks like the output:
>
>       term : classifier
>           definition...
>
> * I'd recommend that option list tables *not* have visible borders and
>   row/column separators.

i too ?

> * From test.pdf, section 2.10, I guess that it's hard to render
>   complex tables properly in LaTeX? ;)

because it is hard.
latex does it great (for me). up to know i seldom used multicolumns
not speaking of multirows.

what we might learn from latex is longtables have
  header for first page : e.g. column headings
  footer                : "continued on next page" (could be generated by
                          docutils).
  header on following pages : "continued from previous page" and the
                          column headings
  footer on last page   : plain line.

> * The "image" directive in section 2.14.2 is producing an *inline*
>   image, whereas it should be a block element (flush left, separated
>   by a blank line from the text above & below).
>
> * Why does the "figure" (section 2.14.2) appear at the very end of the
>   document?  I could understand it floating to the top or bottom of
>   the page it appears on, but it moves too far.

latex figures are moveable (as are tables, what should it do if it
does not fit on this page) therefore a reference to a figure says
"see fig.132" and there is a "list of figures".

<SNIP>
> Please feel free to take this list and incorporat it in your To-Do

feel free to do it yourself :-) my sandbox has no fence.

> I bet you're learning a lot about LaTeX!
>
> Do you consider the Writer ready to migrate into the docutils package?
> It doesn't have to be perfect, just `reasonably complete`_ as I've
> defined before.  I didn't notice anything missing from test.pdf, so it
> looks like it is reasonably complete.  I'd like to review the code
> first though; please let me know if you think its ready.

for me it is ready, except for one or two things
BUG all enumerations start at one.
BUG enumerations can not nest ad lib.

then i would have real documents: i go for the docutils documentation.
tools/test.txt is then used to make short snippets of problemtatic
constructs.

review and criticize

-- 
 BINGO:
 --- Engelbert Gruber -------+
 SSG Fintl,Gruber,Lassnig   /
 A6410 Telfs Untermarkt 9  /
 Tel. ++43-5262-64727 ----+

On Wed, 6 Nov 2002, David Goodger wrote:

<SNIP>
>
> [Vasko]
> > no, absolutely not. directives names and options should be in
> > english, for sure if it'll be translated, it will be total mess
>
> [Adam]
> > I think it makes sense to translate directive names, and therefore
> > also the options. It simply flows naturally with the text. That's
> > the way it works right now (except the option names), and I like
> > it. :)
>
> [Engelbert]
> > reST is no programming language,

i was wrong. it is a programming language :-)
BUT: see low end.

> Here we see both sides of the argument.  I can understand both points
> of view.  For people working in multiple languages, translated
> directives and bibliographic fields could be a burden.  But for people
> working exclusively in one (non-English) language, it would be
> unreasonable to force them to use English.
>
> [Adam]
> > It is also the only way people that do not know english will be able
> > to use docutils (like for example on a Zope site).
>
> Agreed.  Docutils is meant to be a tool for everyone, not just
> programmers who understand the concept of "little language".  It may
> now be limited to hackers & techies, but that's just because it hasn't
> matured enough.

.. note:: matureing means we end where latex is.

.. note:: my first usage of a directive :-)

> However, all the documentation is in English, so how would people know
> about translated directive names etc.?

this is a bug in the documentation not in the concept.
a translation table could be generated from the source modules.
and if directives translation are code e.g.::

  'wichtig': 'important',  # Den Eintrag als wichtig markieren.

we could also include the short description. looks silly and
is not always necessary as directives should be descriptive.

better yet::

  'wichtig': 'important',  # _directives_important

which would link to the language dependend directive documentation.

> I think all aspects of the document should be in the language of the
> document, including directives (both names & options) and
> bibliographic field names.  To allow for "under development" use,
> English can be used as a fallback for directive names, but not
> bibliographic field lists.  For those who would prefer to use English
> directive names, a command-line option could be added.

can we have the fallback AND the document_language, means
in german i want to use *raw* or *unbearbeitet*.

Translation of directives
=========================

:author: gr...@us...
:date: 2002-11-07

Should directives be translated or not ?

*Yes* because it should be useable for authors in other languages.
Not only for people understanding english.

But ...
-------

as a reST document should be consumable for the reader not only the writer
and the processor we have to consider both audiences: *author* and *reader*

Considering this, directoves must be translated.

But again ...
-------------

We have a third audience, although a silent one, the processing system.

``include`` is not real a document content, but a processing directive,
if this is for the *reader* it should read ``see file``.

Arguments
---------

vasko

  no, absolutely not. directives names and options should be in english, for
sure
  if it'll be translated, it will be total mess.

right when considering processing commands: include in english is not a word
it is a verb and a noun. Know should i translate *include*-theverb or
*include*-thenoun.

Solution
--------

I am for quick starts. The fastest solution is *make it a policy*.

  There are directives to the reader, usually nouns e.g. *danger*, *hint*, ...
  and directives to the processing system, commands.

Directives to the reader must be language dependend.

* Directives must be descriptive to all audience.



-- 
 BINGO: Liegt am Server
 --- Engelbert Gruber -------+
 SSG Fintl,Gruber,Lassnig   /
 A6410 Telfs Untermarkt 9  /
 Tel. ++43-5262-64727 ----+

Richard Jones wrote:
> I've been sent a patch to support unicode in ZReST - I'm unlikely to be able
> to apply it today, so if someone else wants to, I've attached it :)

After a quick look at the patch, I'd be reluctant to apply it.  It's
duplicating effort that Docutils already takes care of: just set
settings.input_encoding and settings.output_encoding properly.  The patch is
limited to UTF-8 only.  Also, there appear to be some indentation issues,
probably hard tabs.

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

On Thu, 7 Nov 2002 1:31 pm, David Goodger wrote:
> Andreas Jung wrote:
> > How unicode-aware is Docutils? How does it treat either
> > unicode strings or e.g. UTF-8 encoded strings?
>
> Docutils is very Unicode-aware.  It uses Unicode strings internally,
> and provides for encoded files and strings on input and output.  If
> you're calling Docutils from code, see the
> docutils.core.publish_string convenience function's docstring for
> details.  From the command-line, use the "--input-encoding" and
> "--output-encoding" options.  Details in
> http://docutils.sf.net/docs/tools.html .

I've been sent a patch to support unicode in ZReST - I'm unlikely to be able 
to apply it today, so if someone else wants to, I've attached it :)


    Richard

Andreas Jung wrote:
> How unicode-aware is Docutils? How does it treat either
> unicode strings or e.g. UTF-8 encoded strings?

Docutils is very Unicode-aware.  It uses Unicode strings internally,
and provides for encoded files and strings on input and output.  If
you're calling Docutils from code, see the
docutils.core.publish_string convenience function's docstring for
details.  From the command-line, use the "--input-encoding" and
"--output-encoding" options.  Details in
http://docutils.sf.net/docs/tools.html .

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

[David]
>> I notice that the "bibliographic_fields" dict is not translated.  I
>> can do this easily though, since all the terms are already in
>> "labels".

[Vasko]
> I do not want to translate these fields, because I think, they have
> to be in english for convenience and consistency between multiple
> languages

I do not agree with this at all.  The "bibliographic fields" feature,
if used (it's not required), is language-dependent.  For a French
document's author to have to write ":Author:" instead of ":Auteur:"
would be awkward, obtrusive, and probably quite galling.  Presumably
the rendered output of the bibliographic field list would say
"Auteur:", so why shouldn't the input also?

> I think, that all these "system" fields have to be in english,
> *maybe* with translation, but not only translated to local language

I don't understand.  Explain please?

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Engelbert wrote:
>>> p.s. still no interest complaining about rst to latex to pdf
>>> conversions ?

Thanks for posting new PDFs.  For the benefit of others, here are the
URLs:

    http://docutils.sf.net/sandbox/grubert/tools.pdf
    http://docutils.sf.net/sandbox/grubert/test.pdf

These documents look *very* good!  You've made a lot of progress.

Some observations for tools.pdf:

* Since TeX uses curly quotes (which it expresses ``thusly''), perhaps
  the LaTeX writer could convert regular double-quotes to TeX-style
  quotes?  Or can TeX/LaTeX do that automatically?  Currently all
  double-quotes look like (just blur your eyes a bit ;)::

      99    99
        this

  whereas they should look like this::

      66    99
        this

  If it's not feasible to get proper curly quotes, then the Writer
  should use straight quotes.  Better to be ugly (straight quotes)
  than wrong (bad curlies).

  If the LaTeX Writer does this conversion, it should be careful *not*
  to do it for literal blocks and inline literals.  No curly quotes in
  Python code ;)

* Field lists should be rendered as two-column tables, like the
  docinfo fields.  See
  http://docutils.sf.net/spec/doctree.html#field-list ("Details"
  subsection, "Processing" item).

* In tools.pdf, the source text ``<directory>`` is being rendered as
  ``!directory?``, but with the inverse-! (&iexcl;) and inverse-?
  (&iquest;) characters.

* Shouldn't literal blocks be indented, or have some kind of
  decoration (solid left margin, etc.)?  I think relying on the
  typeface is too subtle; indentation helps distinguish literal blocks
  from the surrounding text.  Inline literals look fine the way they
  are.

* (I'm sure you already are aware of this.)  The table in tools.pdf,
  which is longer than one page, is not breaking across page
  boundaries.  Also, the paragraphs inside table cells are being
  merged together (ISTR this coming up before).

* Table column widths can be taken from the "colwidth" attributes of
  "colspec" elements in the doctree.  Attribute values are relative
  numbers, originating from the number of character columns in the
  source text.  For tools.txt, the table's colwidth values are "20"
  and "50", so the first column should be 2/7 and the second column
  5/7 wide.

And test.pdf:

* Bibliographic field lists (docinfo) should have ":" appended to each
  field name.

* I see you got the address line-breaking right.  Great!

* The "dedication" and "abstract" bibliographic fields get transformed
  to "topic" elements, specifically <topic class="dedication"> and
  <topic class="abstract"> in the internal doctree.  Topic should be
  rendered like sections (i.e., title above the body), only indented
  or otherwise set off from the rest of the document.  See
  http://docutils.sf.net/spec/doctree.html#topic .

* It would be useful if "problematic" elements acted as internal
  references to their system messages.  Back-links are also useful,
  for system messages as well as for footnotes and citations.

* From doctree.txt, definition list terms and classifiers "should be
  separated visually, typically by spaces plus a colon or dash."  I'd
  recommend a colon, so the input looks like the output:

      term : classifier
          definition...

* I'd recommend that option list tables *not* have visible borders and
  row/column separators.

* From test.pdf, section 2.10, I guess that it's hard to render
  complex tables properly in LaTeX? ;)

* The "image" directive in section 2.14.2 is producing an *inline*
  image, whereas it should be a block element (flush left, separated
  by a blank line from the text above & below).

* Why does the "figure" (section 2.14.2) appear at the very end of the
  document?  I could understand it floating to the top or bottom of
  the page it appears on, but it moves too far.

* Why are footnotes (a) mostly at the end of the document, and (b)
  spaced so far apart?

* In line blocks, all whitespace is significant.  Look at the stanza
  beginning "But can a bee be said to be" in section 2.14.5; each line
  should be indented more than the last.

  The same thing applies to inline literals, although to a lesser
  extent.  In other words, it would be nice to have significant
  whitespace there, but not crucial.  Using a recent browser, look at
  the whitespace in the inline literals in the second paragraph of
  http://docutils.sf.net/tools/test.html#inline-markup .

* The admonitions ought to be more distinctive.  They should jump out
  at the reader.  There are degrees of alerts.  The HTML Writer shows
  a little bit of what I mean, but it should be more distinctive too.

Please feel free to take this list and incorporat it in your To-Do
list.

I bet you're learning a lot about LaTeX!

Do you consider the Writer ready to migrate into the docutils package?
It doesn't have to be perfect, just `reasonably complete`_ as I've
defined before.  I didn't notice anything missing from test.pdf, so it
looks like it is reasonably complete.  I'd like to review the code
first though; please let me know if you think its ready.

.. _reasonably complete:
   http://docutils.sf.net/spec/notes.html#reasonably-complete

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

gr...@us... wrote:
> and maybe this should be more visible to translators not only
> a unittest, and could give warnings (verbose mode).

What do you mean?

>>> This is how i understood the datastructure, might be totally
>>> wrong.
>>
>> The "labels" dict can be xor-compared (it's a mapping of English
>> terms to terms in the given language), but the
>> "bibliographic_fields" dict cannot (it's a mapping of terms in the
>> given language to node classes).  You'll notice that all the
>> bibliographic_fields items failed for de.py, even though they're
>> correct.
>
> they failed because of first letter uppercase (i fixed this or
> did i break something ? another test required!)

I'm sorry, I mistakenly thought that "xor" was being applied to
"bibliographic_fields" in docutils/languages/de.py.

You're correct to have made the keys lowercase in
docutils/parsers/rst/languages/de.py.

BTW, de.py needs translations of the "raw", "include", and "replace"
directives.  Could you please?

There's a comment::

      'topic': 'topic',  # Inhalt, Thema or =DCberbegriff

There can be many keys with the same value.  Two of them anyway;
"inhalt" is already used for "contents".  Are they homonyms?

Another::

      'bild': 'image',
      'figure': 'figure', # also Bild ?

You can't use "bild" twice, but surely there's an appropriate word in
German?  Something like "titled-image" (translated) may suffice.  The
English "image" just means "picture", whereas "figure" has a larger
meaning, like "diagram/illustration/explanation".

I know no German, so I have to trust you as a native speaker to do
your best.  Just don't translate "contents" as "my hovercraft is full
of eels".  (Monty Python reference)

>> The length of "bibliographic_fields" also cannot be compared, since
>
> This was a try to test what might be possible, if not this
> will be removed.

Probably cannot be tested.

>> languages behave differently.  In Swedish there's no difference
>> between "Author" and "Authors", so there's one less entry.
>
> learned this from adam (maybe one could put comments in the file for
> the next peer)

Good idea.

>> If no parsers.rst.languages module exists for the
>> language, English is currently used as a fallback.
>
> tools/html.py -l de README.txt donotcare.html
...
>   File ".../docutils/parsers/rst/languages/__init__.py", line
> 19, in get_language
>     module =3D __import__(language_code, globals(), locals())
> ImportError: No module named de

Looks like a bug!  I should qualify: the *intention* is for English to
be used as a fallback for directive names.  Added to to-do/bugs.

>> In programming languages, most people seem to be happy with
>> English-based syntax.  AppleScript had language dialects, but I
>> don't think it caught on (code in Japanese looked *very* strange to
>> my eyes, but I'm biased).  Directives are similar to programming
>> language keywords/statements/commands, but not the same.  What do
>> people think?
>
> reST is no programming language,

I agree.

> i thought the text should be a valid document (as far as possible)
> otherwise we might need a text-writer.

Don't know what you mean by "we might need a text-writer".

> and the test of course must know if it is required or optional.

I think a test should fail if any aspect of a language isn't there.

>> Another question, from the to-do list: Should we use the language
>> module for directive option names?  In other words, should
>> directive option names be translated?

How about this one?  Right now in English we would write::

    .. contents:: :local:

In German, "contents" is translated, but "local" is not::

    .. inhalt:: :local:

So should directive options be translated too?

--=20
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

[David]
>> Question for you (and anyone else using Docutils with other
>> languages): what about reStructuredText directive names?  Should
>> they be translated into Slovak (etc.)?  Do you want to use::
>>
>>     .. contents::
>>
>> or::
>>
>>     .. obsah::
>>
>> ?  And should directive options be translated as well?

[Vasko]
> no, absolutely not. directives names and options should be in
> english, for sure if it'll be translated, it will be total mess

[Adam]
> I think it makes sense to translate directive names, and therefore
> also the options. It simply flows naturally with the text. That's
> the way it works right now (except the option names), and I like
> it. :)

[Engelbert]
> reST is no programming language,

Here we see both sides of the argument.  I can understand both points
of view.  For people working in multiple languages, translated
directives and bibliographic fields could be a burden.  But for people
working exclusively in one (non-English) language, it would be
unreasonable to force them to use English.

[Adam]
> It is also the only way people that do not know english will be able
> to use docutils (like for example on a Zope site).

Agreed.  Docutils is meant to be a tool for everyone, not just
programmers who understand the concept of "little language".  It may
now be limited to hackers & techies, but that's just because it hasn't
matured enough.

However, all the documentation is in English, so how would people know
about translated directive names etc.?

I think all aspects of the document should be in the language of the
document, including directives (both names & options) and
bibliographic field names.  To allow for "under development" use,
English can be used as a fallback for directive names, but not
bibliographic field lists.  For those who would prefer to use English
directive names, a command-line option could be added.

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

gr...@us... wrote:
> On Tue, 5 Nov 2002, David Goodger wrote:
>> I noticed that Engelbert Gruber has already added sk.py to CVS.
>> Unfortunately,
>>
>>     --- NEW FILE: sk.py ---
>>     (This appears to be a binary file; contents omitted.)
>>
>> That's exactly what I'm talking about.
>
> sorry but since a week or so cvs hangs after the last file
> so this did not get through to me.

I saw it on the docutils-checkins list.  Subscribe at
http://lists.sf.net/lists/listinfo/docutils-checkins to keep up to
date on changes to CVS.

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

[David]
>> Does that code live in Zope or in Docutils?

Perhaps I should have asked: Where *should* that code live?

[Andreas]
> The code is in the ajung-restructuredtext-integration-branch
> of Zope

I see the docutils/writers/html4zope.py module.  Any other additions
or changes?

Would these files be maintained there?  Or is that just a snapshot of
the Docutils package, to be updated from time to time?  I hope the
latter.

Docutils is still very much in the experimentation/exploration stage of
development.  As I've discovered flaws in the design of Docutils, I've make
some sweeping changes to its internals.  Although never without good reason,
APIs *are* subject to change, sometimes breaking client code.

If the Zope integration code were part of Docutils itself, there's a
much better chance (>0 vs. 0) that API changes will be applied to that
code as well.  Code in the main package must continue to work; any
bugs are fixed ASAP.  I try to update code in the sandbox where I can.
Code in another CVS repository won't be seen by me.

So I think that html4zope.py ought to live in Docutils.  However, if we
implement the changes it needs in html4css1.py (HTML header tags starting
at <h3>), it could be that there is no need for it.

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

How unicode-aware is Docutils? How does it treat either
unicode strings or e.g. UTF-8 encoded strings?

-aj

    ---------------------------------------------------------------------
   -    Andreas Jung                     http://www.andreas-jung.com   -
  -   EMail: andreas at andreas-jung.com                              -
   -            "Life is too short to (re)write parsers"               -
    ---------------------------------------------------------------------

hi,

one more thing

> I notice that the "bibliographic_fields" dict is not translated.  I
> can do this easily though, since all the terms are already in
> "labels".

I do not want to translate these fields, because I think, they have to =
be
in english for convenience and consistency between multiple languages

I think, that all these "system" fields have to be in english, *maybe*
with translation, but not only translated to local language

thank you,
miro

On Tue, 5 Nov 2002, David Goodger wrote:

> eng...@ss... wrote:
> > attached a quick script that checks for language.labels
> > having all entries that en has.
> >
> > and if language directives are found.
> >
> > Could be put into a test if correct (david ?).
>
> Certainly, with pleasure.  It needs a bit of work to make it more
> general-purpose:
>
> * The language codes should not be hard-coded, but determined
>   dynamically from directory listings.
>
> * The module should be converted to be compatible with unittest.py.
>   See the other test modules, like docutils/test/test_utils.py.
>
> Could you make these changes?  I'd also prefer to call it
> "test_languages.py" (verbose is better).

yes, but what can we check .. see below

and maybe this should be more visible to translators not only
a unittest, and could give warnings (verbose mode).

> > This is how i understood the datastructure, might be totally wrong.
>
> The "labels" dict can be xor-compared (it's a mapping of English terms
> to terms in the given language), but the "bibliographic_fields" dict
> cannot (it's a mapping of terms in the given language to node
> classes).  You'll notice that all the bibliographic_fields items
> failed for de.py, even though they're correct.

they failed because of first letter uppercase (i fixed this or
did i break something ? another test required!)

> The length of "bibliographic_fields" also cannot be compared, since

This was a try to test what might be possible, if not this
will be removed.

> languages behave differently.  In Swedish there's no difference
> between "Author" and "Authors", so there's one less entry.  Also, in

learned this from adam (maybe one could put comments in the
file for the next peer)

> accented languages it may be useful to have both accented and
> unaccented entries for versatility.
>
> > Is it an error if rst.language has no language module but
> > languages has ? e.g. sk.
>
> That's a good question, and one that I've had myself.  I don't know
> the answer.  If no parsers.rst.languages module exists for the
> language, English is currently used as a fallback.

tools/html.py -l de README.txt donotcare.html
says
  File "/home/bert/lib/python/docutils/parsers/rst/states.py", line 148, in run
    self.language = languages.get_language(
  File "/home/bert/lib/python/docutils/parsers/rst/languages/__init__.py", line
19, in get_language
    module = __import__(language_code, globals(), locals())
ImportError: No module named de

> In programming languages, most people seem to be happy with
> English-based syntax.  AppleScript had language dialects, but I don't
> think it caught on (code in Japanese looked *very* strange to my eyes,
> but I'm biased).  Directives are similar to programming language
> keywords/statements/commands, but not the same.  What do people think?

reST is no programming language, i thought the text should be a valid
document (as far as possible) otherwise we might need a text-writer.

and the test of course must know if it is required or optional.

> Another question, from the to-do list: Should we use the language
> module for directive option names?  In other words, should directive
> option names be translated?
>
> > p.s. still no interest complaining about rst to latex to pdf
> > conversions ?
>
> Sorry, I don't have a latex toolchain set up.  What are the
> requirements?

te_latex on my linux install this seams to include what is needed.
but if someone tries we will see clearer.

pdflatex is better as it can handle the pngs directly, so there
is no problem for needing eps-with bounding boxes (quality might
be nonoptimal though)

but within limits i offered to do conversions, not because test.txt
is too sick, but because we will learn more from different inputs.
e.g. i had to remove juliens pretty paladino font because it blows
up pdfs.
currently the latex output is a mixture of using latex's title
but not author or date. the table of contents is numbered by docutils
(because i have no interest in spending hours to find out how to
get the test.txt example of with the second table of contents into
latex)
i donot like the colored links ... asf.

> Could you put an example PDF file in your sandbox?  (Directly on the
> web site please, not under CVS.)

yessir, you mean the test.txt ? by consulting the latex2e-README.txt
one could see what i am pondering at too.

tools.pdf and test.pdf

-- 
 BINGO: innovative solutions
 --- Engelbert Gruber -------+
 SSG Fintl,Gruber,Lassnig   /
 A6410 Telfs Untermarkt 9  /
 Tel. ++43-5262-64727 ----+

On Tue, 05 Nov 2002 21:41:26 -0500 David Goodger
<go...@us...> wrote:

[...]
> Question for you (and anyone else using Docutils with other
> languages): what about reStructuredText directive names?  Should they
> be translated into Slovak (etc.)?  Do you want to use::
> 
>     .. contents::
> 
> or::
> 
>     .. obsah::
> 
> ?  And should directive options be translated as well?

I think it makes sense to translate directive names, and therefore also the
options. It simply flows naturally with the text. That's the way it works
right now (except the option names), and I like it. :)

It is also the only way people that do not know english will be able to use
docutils (like for example on a Zope site).

---
Adam Chodorowski <ad...@ch...>

If studies were interesting, the world would come to a halt and nothing 
would ever get done :) 
    -- James Kehl

hi,

> Unicode escapes for the accented characters here.  See the sv.py
> Swedish-language module for an example, and
> http://docutils.sf.net/spec/howto/i18n.html (newly revised) for a
> rationale.

ok, I'll look at the doc today, thank you

> Question for you (and anyone else using Docutils with other
> languages): what about reStructuredText directive names?  Should they
> be translated into Slovak (etc.)?  Do you want to use::
>     .. contents::
> or::
>     .. obsah::
> ?  And should directive options be translated as well?

no, absolutely not. directives names and options should be in english, =
for sure
if it'll be translated, it will be total mess

thanks,
miro

On Tue, 5 Nov 2002, David Goodger wrote:

> Vasko Miroslav wrote:
>
> I noticed that Engelbert Gruber has already added sk.py to CVS.
> Unfortunately,
>
>     --- NEW FILE: sk.py ---
>     (This appears to be a binary file; contents omitted.)
>
> That's exactly what I'm talking about.

sorry but since a week or so cvs hangs after the last file
so this did not get through to me.

-- 
 BINGO: This left unindentionally unblank
 --- Engelbert Gruber -------+
 SSG Fintl,Gruber,Lassnig   /
 A6410 Telfs Untermarkt 9  /
 Tel. ++43-5262-64727 ----+

The code is in the ajung-restructuredtext-integration-branch
of Zope (available from cvs.zope.org). docutils
is checked in into the CVS and I made some extensions
to the DTML code. There will be further changes to include
support reST into CMF. Also Richards ZreST product lives
in the branch.

-aj

--On Dienstag, 5. November 2002 21:38 -0500 David Goodger 
<go...@us...> wrote:

> Andreas Jung wrote:
>> Meanwhile I resolved the problems with a derived Writer class (from
>> html4css1).  Zope btw. now supports reSTX both inside DTML and with
>> Richards reSTX product.
>
> Great!  Does that code live in Zope or in Docutils?  Could you share
> it with us?  (BTW we don't say "reSTX".  Please use "reST" instead.)
>
> --
> David Goodger  <go...@us...>  Open-source projects:
>   - Python Docutils: http://docutils.sourceforge.net/
>     (includes reStructuredText: http://docutils.sf.net/rst.html)
>   - The Go Tools Project: http://gotools.sourceforge.net/
>
>




    ---------------------------------------------------------------------
   -    Andreas Jung                     http://www.andreas-jung.com   -
  -   EMail: andreas at andreas-jung.com                              -
   -            "Life is too short to (re)write parsers"               -
    ---------------------------------------------------------------------

eng...@ss... wrote:
> attached a quick script that checks for language.labels
> having all entries that en has.
> 
> and if language directives are found.
> 
> Could be put into a test if correct (david ?).

Certainly, with pleasure.  It needs a bit of work to make it more
general-purpose:

* The language codes should not be hard-coded, but determined
  dynamically from directory listings.

* The module should be converted to be compatible with unittest.py.
  See the other test modules, like docutils/test/test_utils.py.

Could you make these changes?  I'd also prefer to call it
"test_languages.py" (verbose is better).

> This is how i understood the datastructure, might be totally wrong.

The "labels" dict can be xor-compared (it's a mapping of English terms
to terms in the given language), but the "bibliographic_fields" dict
cannot (it's a mapping of terms in the given language to node
classes).  You'll notice that all the bibliographic_fields items
failed for de.py, even though they're correct.

The length of "bibliographic_fields" also cannot be compared, since
languages behave differently.  In Swedish there's no difference
between "Author" and "Authors", so there's one less entry.  Also, in
accented languages it may be useful to have both accented and
unaccented entries for versatility.

> Is it an error if rst.language has no language module but
> languages has ? e.g. sk.

That's a good question, and one that I've had myself.  I don't know
the answer.  If no parsers.rst.languages module exists for the
language, English is currently used as a fallback.

In programming languages, most people seem to be happy with
English-based syntax.  AppleScript had language dialects, but I don't
think it caught on (code in Japanese looked *very* strange to my eyes,
but I'm biased).  Directives are similar to programming language
keywords/statements/commands, but not the same.  What do people think?

Another question, from the to-do list: Should we use the language
module for directive option names?  In other words, should directive
option names be translated?

> p.s. still no interest complaining about rst to latex to pdf
> conversions ?

Sorry, I don't have a latex toolchain set up.  What are the
requirements?

Could you put an example PDF file in your sandbox?  (Directly on the
web site please, not under CVS.)

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Vasko Miroslav wrote:
> I'm not on the list, I hope it does not matter

You're welcome to join!

> this is diff against docutils 0.2.4, it adds slovak
> support for docutils.

Thank you.  You sent this to me by email a week or so ago; sorry I
didn't get back to you yet.

I notice that the "bibliographic_fields" dict is not translated.  I
can do this easily though, since all the terms are already in
"labels".

> it's gzipped and attached, because
> it contains iso-8859-2 charset and i'm afraid of
> mail-client mangling of that characters

Yes, that's a valid concern.  I was concerned that mangling had
happened on my end as well.  For that reason, I think there should
only be ASCII characters in Python source files; we need to use
Unicode escapes for the accented characters here.  See the sv.py
Swedish-language module for an example, and
http://docutils.sf.net/spec/howto/i18n.html (newly revised) for a
rationale.

I noticed that Engelbert Gruber has already added sk.py to CVS.
Unfortunately,

    --- NEW FILE: sk.py ---
    (This appears to be a binary file; contents omitted.)

That's exactly what I'm talking about.

> please, let me know, if the patch was applied or not,
> i can watch it on the list archives

I will be sure to add the file.

Question for you (and anyone else using Docutils with other
languages): what about reStructuredText directive names?  Should they
be translated into Slovak (etc.)?  Do you want to use::

    .. contents::

or::

    .. obsah::

?  And should directive options be translated as well?

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Andreas Jung wrote:
> Meanwhile I resolved the problems with a derived Writer class (from
> html4css1).  Zope btw. now supports reSTX both inside DTML and with
> Richards reSTX product.

Great!  Does that code live in Zope or in Docutils?  Could you share
it with us?  (BTW we don't say "reSTX".  Please use "reST" instead.)

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

[Andreas Jung]
>>> I am currently working on the integration of reStructuredText into
>>> Zope.
>>>
>>> I have currently two unresolved issues:
>>>
>>>  - Zope includes (re)STX documents into an existing document with
>>>    a starting level of 3 (means it starts with the <H3> header)

[David]
>> Sorry, I don't know exactly what you mean or what you want here.  I
>> can guess, but I'd rather not.  Please give examples and ask
>> questions.

[Engelbert]
> i used stx from zope prior to reST. AFAIR when converting stx to
> html one can set the level for top level sections.
> 
> in stx::
> 
>   My Chapter
> 
>     my top level.
> 
>     sub section to chapter
> 
>       some text.
> 
> means for rst file::
> 
>   My Chapter
>   ==========
> 
>   my top level.
> 
>   sub section to chapter
>   ----------------------
> 
>   some text.
> 
> the Chapter should become <h3>Chapter</h3> section would become
> <h4>section</h4>
> 
> so that one can produce sites from several documents.

This is what I thought might be the case.  The functionality isn't
implemented now, but it easily could be.  A runtime setting &
command-line option could be introduced to modify the current
behavior.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

--On Dienstag, 5. November 2002 00:20 -0500 David Goodger 
<go...@us...> wrote:


>
> Yes.  Look at the docutils/writers/html4css1.py module's Writer class.
> In the "translate" method, it copies several attributes from its
> "visitor" (HTMLTranslator object), from "head_prefix" to
> "body_suffix".  These are pieces of HTML that may or may not be broken
> up the way you want.  See Ollie Rutherfurd's Writer for ht2html for an
> example: http://docutils.sf.net/sandbox/oliverr/ht/hthtml.py .
>

Meanwhile I resolved the problems with a derived Writer class (from 
html4css1).
Zope btw. now supports reSTX both inside DTML and with Richards reSTX 
product.

Andreas

    ---------------------------------------------------------------------
   -    Andreas Jung                     http://www.andreas-jung.com   -
  -   EMail: andreas at andreas-jung.com                              -
   -            "Life is too short to (re)write parsers"               -
    ---------------------------------------------------------------------