5:18 p.m.
On Mon, 12 May 2008, Russ Allbery wrote:
Hi,
> Package: perl-doc
> Version: 5.10.0-10
>
> UTF8 gets turned into "X" by pod2man, besides other disturbing issues:
pod2man doesn't support non-ISO characters because doing so completely
breaks (and by breaks, I mean up to and including segfaults) nroff on some
platforms and the intention is that pod2man output be portable nroff.
Previous version (5.8.8) does support it. Anyway, this bug breaks
po4a package. After rebuilding po4a with new pod2man almast all its
translated man pages are unreadable...
Regards,
robert
8:12 p.m.
New subject: [Po4a-devel] Bug#480997: pod2man turns UTF8 into "X"s
Robert Luberda <robert(a)debian.org> writes:
On Mon, 12 May 2008, Russ Allbery wrote:
> pod2man doesn't support non-ISO characters because doing so
completely
> breaks (and by breaks, I mean up to and including segfaults) nroff on
> some platforms and the intention is that pod2man output be portable
> nroff.
Previous version (5.8.8) does support it.
As far as I can tell, no version of pod2man has ever supported non-ASCII
characters except for those in the ISO 8859 subset for which it has troff
substitutions (and mangles the character in nroff). Certainly no version
I've maintained has, and if it worked with Tom's version, that was by
accident. Its failure mode may well have changed with Pod::Simple,
however.
It sounds like po4a was relying on unsupported behavior, which is
unfortunate; I've tried to be clear from the start that pod2man doesn't
support and won't support non-ASCII characters until it can output Unicode
directly. Pod::Simple, which is the new parsing layer in 5.10, is much
more formal about how it handles character set parsing.
Basically, the end result is that this is not a bug that I can fix without
doing work that I'm not sure I have time to do. I would certainly welcome
patches to teach it to (optionally) output UTF-8 directly and just assume
that the resulting device can cope; it should be a command-line option to
start with. This may not be too bad, and if I find time, I can try to see
what I can do for the next release, but no promises there.
--
Russ Allbery (rra(a)debian.org) <http://www.eyrie.org/~eagle/>
10:28 a.m.
New subject: [Po4a-devel] Bug#480997: pod2man turns UTF8 into "X"s
tags 480997 patch
thanks
On Mon, May 19, 2008 at 06:12:57PM -0700, rra(a)debian.org wrote:
Basically, the end result is that this is not a bug that I can fix without
doing work that I'm not sure I have time to do. I would certainly welcome
patches to teach it to (optionally) output UTF-8 directly and just assume
that the resulting device can cope; it should be a command-line option to
start with. This may not be too bad, and if I find time, I can try to see
what I can do for the next release, but no promises there.
In fact it is not that far from being implemented.
What is missing is an option to modify the "convert" formatting option of
Pod::Man from pod2man.
Here is a patch to do that.
It adds a "convert" option to Pod::Man (set by default), and a --noascii
option to pod2man, which permits to disable the Pod::Man's convert option.
Best Regards,
--
Nekral
2:40 p.m.
New subject: [Po4a-devel] Bug#480997: pod2man turns UTF8 into "X"s
Nicolas François <nicolas.francois(a)centraliens.net> writes:
In fact it is not that far from being implemented.
What is missing is an option to modify the "convert" formatting option of
Pod::Man from pod2man.
Here is a patch to do that.
It adds a "convert" option to Pod::Man (set by default), and a --noascii
option to pod2man, which permits to disable the Pod::Man's convert option.
Thank you for the patch! It has a few misunderstandings (Pod::Simple
always converts the input character set to UTF-8 and handles all E<>
escapes internally, for example), but I think it looks like the right
approach, and it's much simpler than I expected.
I'm going to try to get a release out today that incorporates a version of
this patch.
--
Russ Allbery (rra(a)debian.org) <http://www.eyrie.org/~eagle/>
4:20 p.m.
New subject: [Po4a-devel] Bug#480997: pod2man turns UTF8 into "X"s
Hello Russ,
Thanks for the new version of podlator.
I had a look at podlators-2.1.1.
I think an additional change will be needed for the handling of the S<>
blocks.
Pod::Simple translate them to 0xA0. This value should depend on the output
encoding.
When I manually fix Pod::Simple, I had good results.
Best Regards,
--
Nekral
4:23 p.m.
New subject: [Po4a-devel] Bug#480997: pod2man turns UTF8 into "X"s
Nicolas François <nicolas.francois(a)centraliens.net> writes:
Thanks for the new version of podlator.
I had a look at podlators-2.1.1.
I think an additional change will be needed for the handling of the S<>
blocks.
Pod::Simple translate them to 0xA0. This value should depend on the output
encoding.
When I manually fix Pod::Simple, I had good results.
Could you give me an example so that I can understand the problem?
--
Russ Allbery (rra(a)debian.org) <http://www.eyrie.org/~eagle/>
4:33 p.m.
New subject: [Po4a-devel] Bug#480997: pod2man turns UTF8 into "X"s
On Sun, Jul 20, 2008 at 02:23:06PM -0700, rra(a)debian.org wrote:
Could you give me an example so that I can understand the problem?
Here's the POD version of the French translation of the po4a.1 manpage.
You can look for "suivant.:" in this file and in the generated file.
When "S<suivant :>" generates "suivant\xA0:", this lead an
invalid UTF-8
file, which will be considered as a latin-1 file by programs like vim or man.
Best Regards,
--
Nekral
6:23 p.m.
New subject: [Po4a-devel] Bug#480997: pod2man turns UTF8 into "X"s
Nicolas François <nicolas.francois(a)centraliens.net> writes:
Here's the POD version of the French translation of the po4a.1
manpage.
You can look for "suivant.:" in this file and in the generated file.
When "S<suivant :>" generates "suivant\xA0:", this lead an
invalid UTF-8
file, which will be considered as a latin-1 file by programs like vim or
man.
Wow, that's remarkably broken (and the bug is in Pod::Simple). Thanks.
I'll try to find a good way to work around this bug while still preserving
the intended behavior, and failing that, will just not convert
non-breaking spaces.
--
Russ Allbery (rra(a)debian.org) <http://www.eyrie.org/~eagle/>
5971
days inactive
6033
days old
7 comments
3 participants
participants (3)
-
Nicolas François -
Robert Luberda -
Russ Allbery