4:55 p.m.
Hello,
To solve an issue with the man module, I've implemented a way to specify
some (multi-lines) verbatim blocks.
I'm wondering if this functionality has an interest for po4a users (it
may be error prone), and if it can be useful for other modules (in this
case it should be implemented in Transtractor).
The major cause (50%) of failure of the man module (po4a exits with an
error message) concerns blocks in roff language (using .de, .if, .ie or
.el requests).
They are usually used to define new macros or different ways to do the
same thing depending on the parser (e.g. nroff or troff).
Copying these block of code from the original to the translation is
particularly useful for the man module because the roff language is quite
complicated, which pushed authors to just cut & past definitions found in
other pages (a lot of the defined macros are not even used).
With my first try, I could correctly process 60 additional files (without
even defining additional macros). 25 different blocks were used (each one
defined in a file).
Everything is not so neat:
- if the block appear in the middle of a paragraph, the block will be
copied verbatim at a wrong place: at the beginning of the paragraph
(this could be fixed by adding a function to "flush" the parser).
Most of the time, it is not a concern because the macros are defined
in the header and it is up to the user to specify or not a verbatim
block, but it should probably be fixed.
- it makes po4a slower (I don't think it's an issue, except for the
testsuite;): each lines of the input document has to be compared with
a line of each specified files (and lines have to be shifted/unshifted
many times).
- the user can do whatever he wants and it is, as the addendums, a quite
complicated feature which can be error prone.
- ...
Do you think it may be worth having such a mechanism in po4a?
Is there an interest for the other modules?
Note:
For the man module, I'm also planing to add a verbatim and translate
option (like in the sgml module), which should allow to specify the
behaviour of the parser for these additional macros.
Another Note:
I've not tried this, but it may solve #263298 (Please let -gettextize know
about addendums and remove them automatically).
Kind Regards,
--
Nekral
6:43 a.m.
On Mon, Nov 15, 2004 at 11:55:31PM +0100, Nicolas François wrote:
Hello,
To solve an issue with the man module, I've implemented a way to specify
some (multi-lines) verbatim blocks.
I'm not sure I understand what you're talking about. Could you please give
an example of use? Why can't you simply pushline all the lines until you
encounter the end block boundary?
I need more details to tell you what I think of this...
Another Note:
I've not tried this, but it may solve #263298 (Please let -gettextize know
about addendums and remove them automatically).
It's not explicitely tagged, but I tend to think that this bug is a
"wontfix" one. The asked feature could incitate upstream maintainer to do
stupidities with the translation files.
But you already implemented the font stack feature, even if I used to think
that it was too hard to worth the work, with some supurious justifications
about finding the badly formated pages :)
Thanks for your time,
Mt.
4:40 p.m.
On Tue, Nov 16, 2004 at 01:43:35PM +0100, martin.quinson(a)imag.fr wrote:
On Mon, Nov 15, 2004 at 11:55:31PM +0100, Nicolas François wrote:
> Hello,
>
> To solve an issue with the man module, I've implemented a way to specify
> some (multi-lines) verbatim blocks.
I'm not sure I understand what you're talking about. Could you please give
an example of use? Why can't you simply pushline all the lines until you
encounter the end block boundary?
Here is what most tetex-bin manpages contain:
(it is not always exactly the same header, I'm using amstex as an example)
.\"=====================================================================
.if t .ds TX
\fRT\\h'-0.1667m'\\v'0.20v'E\\v'-0.20v'\\h'-0.125m'X\fP
.if n .ds TX TeX
.ie t .ds OX \fIT\v'+0.25m'E\v'-0.25m'X\fP\" for troff
.el .ds OX TeX\" for nroff
.\" the same but obliqued
.\" BX definition must follow TX so BX can use TX
.if t .ds BX \fRB\s-2IB\s0\fP\*(TX
.if n .ds BX BibTeX
.\" LX definition must follow TX so LX can use TX
.if t .ds LX
\fRL\\h'-0.36m'\\v'-0.15v'\s-2A\s0\\h'-0.15m'\\v'0.15v'\fP\*(TX
.if n .ds LX LaTeX
.if t .ds AX
\fRA\\h'-0.1667m'\\v'0.20v'M\\v'-0.20v'\\h'-0.125m'S\fP\*(TX
.if n .ds AX AmSTeX
.if t .ds AY
\fRA\\h'-0.1667m'\\v'0.20v'M\\v'-0.20v'\\h'-0.125m'S\fP\*(LX
.if n .ds AY AmSLaTeX
.\"=====================================================================
This permits to define nice formating for TeX, LaTeX, etc. if the document
is processed in troff mode (it only works with some devices, like ps) and
fall back to a simple representation in nroff mode.
These strings are later used as "\*(TX", "\*(NX", etc.
I wanted to indicate to po4a that when this sequence of lines is
encountered, they only have to be copied in the translation as is, without
trying to parse them.
(Note that finally I decided to split this block in smaller blocks for a
better reuse, because the header vary from one page to the other)
Here is another example. Some pages of the groff packages define a OP
macro, which simplify writing options:
.de OP
.ie \\n(.$-1 .RI "[\ \fB\\$1\fP" "\\$2" "\ ]"
.el .RB "[\ " "\\$1" "\ ]"
..
(I'm not sure, but it seems to specify that depending on the number of
arguments, it uses the first or the second line; this permits to always
have the options between brackets in the roman font, the option in bold
and the argument, if any, in italic).
Here is another example with the cron-apt.sgml file (old #278365)
The user would have specified the following sequence of lines:
<!-- Process this file with docbook-to-man to generate an nroff manual
page: `docbook-to-man manpage.sgml > manpage.1'. You may view
the manual page with: `docbook-to-man manpage.sgml | nroff -man |
less'. A typical entry in a Makefile or Makefile.am is:
manpage.8: manpage.sgml
docbook-to-man $< > $@
-->
Your fix is much better in this case. I just want to show that it could
give users a solution for some points we can't (or are not inclined to)
deal with.
I have to admit that I didn't really think about using begin and end
boundaries.
Boundaries are simpler to implement. They may be a little bit more risky
(if the end boundary is not found, all the document will be untranslated),
but it will up to the user to specify the right ones and verify the result.
> Another Note:
> I've not tried this, but it may solve #263298 (Please let -gettextize know
> about addendums and remove them automatically).
It's not explicitely tagged, but I tend to think that this bug is a
"wontfix" one. The asked feature could incitate upstream maintainer to do
stupidities with the translation files.
The last example using the po4a-gettextize manpage, imagine it belongs to
manpages-fr, and this page was not translated with po4a), then somebody
(lets call him Denis) would create a file containing:
.SH "TRADUCTION"
.IX Header "TRADUCTION"
.Vb 1
\& Martin Quinson <martin.quinson(a)tuxfamily.org>
.Ve
(it could latter be used as an addendum by adding a header). Then these
strings would not be given to the parser, they wouldn't appear in the
french po during gettextization, and the English and French POs should then
match (I still did not try it).
If needed, the addendum format could be used/supported.
I hope what I'm proposing is more clear now.
I'm still not convinced it is really useful or easy to use.
Kind Regards,
--
Nekral
9:22 a.m.
On Thu, Nov 18, 2004 at 11:40:38PM +0100, Nicolas François wrote:
On Tue, Nov 16, 2004 at 01:43:35PM +0100, martin.quinson(a)imag.fr
wrote:
> On Mon, Nov 15, 2004 at 11:55:31PM +0100, Nicolas François wrote:
> > Hello,
> >
> > To solve an issue with the man module, I've implemented a way to specify
> > some (multi-lines) verbatim blocks.
>
> I'm not sure I understand what you're talking about. Could you please give
> an example of use? Why can't you simply pushline all the lines until you
> encounter the end block boundary?
Here is what most tetex-bin manpages contain:
(it is not always exactly the same header, I'm using amstex as an example)
.\"=====================================================================
.if t .ds TX
\fRT\\h'-0.1667m'\\v'0.20v'E\\v'-0.20v'\\h'-0.125m'X\fP
.if n .ds TX TeX
.ie t .ds OX \fIT\v'+0.25m'E\v'-0.25m'X\fP\" for troff
.el .ds OX TeX\" for nroff
.\" the same but obliqued
.\" BX definition must follow TX so BX can use TX
.if t .ds BX \fRB\s-2IB\s0\fP\*(TX
.if n .ds BX BibTeX
.\" LX definition must follow TX so LX can use TX
.if t .ds LX
\fRL\\h'-0.36m'\\v'-0.15v'\s-2A\s0\\h'-0.15m'\\v'0.15v'\fP\*(TX
.if n .ds LX LaTeX
.if t .ds AX
\fRA\\h'-0.1667m'\\v'0.20v'M\\v'-0.20v'\\h'-0.125m'S\fP\*(TX
.if n .ds AX AmSTeX
.if t .ds AY
\fRA\\h'-0.1667m'\\v'0.20v'M\\v'-0.20v'\\h'-0.125m'S\fP\*(LX
.if n .ds AY AmSLaTeX
.\"=====================================================================
This permits to define nice formating for TeX, LaTeX, etc. if the document
is processed in troff mode (it only works with some devices, like ps) and
fall back to a simple representation in nroff mode.
These strings are later used as "\*(TX", "\*(NX", etc.
I wanted to indicate to po4a that when this sequence of lines is
encountered, they only have to be copied in the translation as is, without
trying to parse them.
(Note that finally I decided to split this block in smaller blocks for a
better reuse, because the header vary from one page to the other)
Ok. Thanks for the example. This is close to an idea I have in mind since a
while. I never discussed it here because of lack of time (and idea's lack of
maturity).
There is a bug against po4a in Debian asking for a text module. It could
split classical itemize with '*' and indentation. Assume it's written for a
while.
I would like to write a debian changelog module by reusing this text module.
A clean way to do so is to make a "meta parser" separating each entries of
the changelog, and passing each of them to the text module above.
As you can see, this is just like your example of parts being not handled
the regular way. But I'm the more crazy over there. I want to use this kind
of feature to design meta parsers. ;)
This is also exactly the kind of feature we need for a wml module. It should
switch between html, mp4h and perl contexts.
If we add an "echo" module not changing anything to the input text (2
lines?), my idea would encompas yours.
I'd prefer us to think about the more general concept here. I'm not sure how
to do so, not even whether it's doable (ie, if the complexity added worth
the gain)...
Here is another example with the cron-apt.sgml file (old #278365)
[...]
Your fix is much better in this case. I just want to show that it
could
give users a solution for some points we can't (or are not inclined to)
deal with.
Yeah your solution would have been really hackish. He could have edited the
original document, to :)
I have to admit that I didn't really think about using begin and
end
boundaries.
Boundaries are simpler to implement. They may be a little bit more risky
(if the end boundary is not found, all the document will be untranslated),
but it will up to the user to specify the right ones and verify the result.
Exactly. And they are more expressive. For example, I'd use <perl> as a
boundary in the wml meta-parser.
(it could latter be used as an addendum by adding a header). Then
these
strings would not be given to the parser, they wouldn't appear in the
french po during gettextization, and the English and French POs should then
match (I still did not try it).
If needed, the addendum format could be used/supported.
Addendum were already difficult to use. I fear that after this
generalization, nobody will ever be able to use them. Including us :)
That being said, I feel that we are near from a clean design, but I cannot
narrow it yet. I've too much things to think about...
Thanks for your time,
Mt.
6:17 p.m.
On Mon, Nov 15, 2004 at 11:55:31PM +0100, Nicolas François wrote:
Hello,
To solve an issue with the man module, I've implemented a way to specify
some (multi-lines) verbatim blocks.
I did not understand your message at first reading because I thought
that 'verbatim blocks' were unformatted, maybe you should talk about
'untranslated blocks' instead.
I'm wondering if this functionality has an interest for po4a
users (it
may be error prone), and if it can be useful for other modules (in this
case it should be implemented in Transtractor).
SGML prologs seem to be quite similar, and almost all formats can define
header comments. So if I understand you right, it seems to be quite
generic.
The major cause (50%) of failure of the man module (po4a exits with
an
error message) concerns blocks in roff language (using .de, .if, .ie or
.el requests).
They are usually used to define new macros or different ways to do the
same thing depending on the parser (e.g. nroff or troff).
Copying these block of code from the original to the translation is
particularly useful for the man module because the roff language is quite
complicated, which pushed authors to just cut & past definitions found in
other pages (a lot of the defined macros are not even used).
Agreed.
With my first try, I could correctly process 60 additional files
(without
even defining additional macros). 25 different blocks were used (each one
defined in a file).
Everything is not so neat:
- if the block appear in the middle of a paragraph, the block will be
copied verbatim at a wrong place: at the beginning of the paragraph
(this could be fixed by adding a function to "flush" the parser).
Most of the time, it is not a concern because the macros are defined
in the header and it is up to the user to specify or not a verbatim
block, but it should probably be fixed.
IMO such blocks should split paragraphs into 3 parts: before, block
itself, after.
- it makes po4a slower (I don't think it's an issue, except
for the
testsuite;): each lines of the input document has to be compared with
a line of each specified files (and lines have to be shifted/unshifted
many times).
Do not worry, I have some ideas to optimize regexes. Do you have some
large file which could be useful for benchmarking?
- the user can do whatever he wants and it is, as the addendums, a
quite
complicated feature which can be error prone.
I do not follow you here, my understanding was that blocks were copied
from original file, so translators have almost no control here.
Do you think it may be worth having such a mechanism in po4a?
Is there an interest for the other modules?
Sounds like a very good idea, but maybe you should first explain in more
details how you want to proceed.
Note:
For the man module, I'm also planing to add a verbatim and translate
option (like in the sgml module), which should allow to specify the
behaviour of the parser for these additional macros.
Another Note:
I've not tried this, but it may solve #263298 (Please let -gettextize know
about addendums and remove them automatically).
This would be really cool, I am converting man pages of manpages-fr by hand,
and it is quite boring ;)
Denis
3:03 p.m.
On Wed, Nov 17, 2004 at 01:17:12AM +0100, Denis Barbier wrote:
On Mon, Nov 15, 2004 at 11:55:31PM +0100, Nicolas François wrote:
> Hello,
>
> To solve an issue with the man module, I've implemented a way to specify
> some (multi-lines) verbatim blocks.
I did not understand your message at first reading because I thought
that 'verbatim blocks' were unformatted, maybe you should talk about
'untranslated blocks' instead.
I wanted to insist on the fact that these blocks are not only
untranslated, but also not touched by the parser (without rewrapping, or
any kind of reformating).
> With my first try, I could correctly process 60 additional files
(without
> even defining additional macros). 25 different blocks were used (each one
> defined in a file).
>
>
> Everything is not so neat:
> - if the block appear in the middle of a paragraph, the block will be
> copied verbatim at a wrong place: at the beginning of the paragraph
> (this could be fixed by adding a function to "flush" the parser).
> Most of the time, it is not a concern because the macros are defined
> in the header and it is up to the user to specify or not a verbatim
> block, but it should probably be fixed.
IMO such blocks should split paragraphs into 3 parts: before, block
itself, after.
I'm not really worried about notifying the parser. Do you have some ideas
for the user interface?
- The user has to be notified in the po (it will probably be module
specific), but something like a U<There is an untranslated block
here> could be OK for the man module.
- The user has to specify if the block ends a paragraph or not
> - it makes po4a slower (I don't think it's an issue,
except for the
> testsuite;): each lines of the input document has to be compared with
> a line of each specified files (and lines have to be shifted/unshifted
> many times).
Do not worry, I have some ideas to optimize regexes. Do you have some
large file which could be useful for benchmarking?
My first idea is to sort man pages on their size. Next time I will run the
check script, I will take the cpu time used by po4a for each pages.
I will try to build an interesting man page.
> - the user can do whatever he wants and it is, as the
addendums, a quite
> complicated feature which can be error prone.
I do not follow you here, my understanding was that blocks were copied
from original file, so translators have almost no control here.
Forget about it:
I was only thinking of a scenario where the user is bored and add
something that should be translated to a "multi-lines untranslated block".
> Do you think it may be worth having such a mechanism in po4a?
> Is there an interest for the other modules?
Sounds like a very good idea, but maybe you should first explain in more
details how you want to proceed.
Please find attached my current implementation (don't worry, I don't
intend to commit it as is).
It is not optimized at all (I'm reading all the little files for each
lines of the man page). It was done to test if the man pages were really
using always the same blocks.
I've put all "untranslated blocks" in different files in the same directory
(it's path is currently hardcoded).
BTW, is there a way to declare a function in Perl (I moved initialize at
the end because I needed \&translate_joined and \&untranslated).
I'm also adding some of the files I used with the man module.
> Note:
> For the man module, I'm also planing to add a verbatim and translate
> option (like in the sgml module), which should allow to specify the
> behaviour of the parser for these additional macros.
>
> Another Note:
> I've not tried this, but it may solve #263298 (Please let -gettextize know
> about addendums and remove them automatically).
This would be really cool, I am converting man pages of manpages-fr by hand,
and it is quite boring ;)
If you need the verbatim and translate options, I can push them quickly in
CVS.
--
Nekral
7338
days inactive
7342
days old
5 comments
4 participants
participants (4)
-
Denis Barbier -
Martin Quinson -
martin.quinson＠imag.fr
-
Nicolas François