[tpm] Solved! Was: "Wide character in print" error

[Madison Kelly]

Madison Kelly wrote:
> Hi all,
> 
>    I've got another question, if I may... *shock!* :)
> 
>    I've got a program that prints to a browser. Simple enough so far. 
> Some text comes from an XML file (via XML::Simple), and other text comes 
> from a postgres database (set to UTF-8).
> 
>    So then; If I print to the browser normally, the unicode text coming 
> from the pgsql db prints fine, but when unicode text from the XML file 
> prints, I get the "Wide character in print" error, but it still prints 
> okay. I don't want to "just live with it" though because this floods the 
>   logs...
> 
>    If I switch binmode on STDOUT to ':utf8' (binmode STDOUT, ":utf8";), 
> then the text coming from the DB is double-encoded and looks garballed, 
> but the data from the XML file looks fine and *doesn't* generate the 
> "Wide character..." error.
> 
>    I've written a very stripped down script to test this:
> 
> -=] test.pl [=-
> #!/usr/bin/perl
> 
> use strict;
> use warnings;
> use DBI qw(:sql_types);
> use XML::Simple;
> 
> # Tell the program where to find the 'words' file.
> my $file="./test.xml";
> 
> # Read in the words file.
> my $word=XMLin($file);
> 
> my $dbh=DBI->connect("DBI:Pg:dbname=dbname", "user", "secret",
> 	{
> 		RaiseError => 1,
> 		AutoCommit => 1
> 	}
> ) || die "DBI connect error: $DBI::errstr\n";
> 
> binmode STDOUT, ":utf8";
> 
> # Print unicode data from the DB
> my $query="SELECT usr_note FROM users WHERE usr_id=1";
> my $DBreq=$dbh->prepare($query) || die "Error with query: [$query;], 
> error: $DBI::errstr\n";
> $DBreq->execute()  || die "Error with query: [$query;], error: 
> $DBI::errstr\n";
> my ($note)=$DBreq->fetchrow_array();
> print "Note: [$note]\n";
> 
> # Print unicode data from the XML file
> print "English : [$$word{lang}{en_CA}{key}{long_name}{content}]\n";
> print "Japanese: [$$word{lang}{jp}{key}{long_name}{content}]\n";
> 
> exit(0);
> -=] test.pl [=-
> 
>    Here is the XML file:
> 
> -=] test.xml [=-
> <?xml version="1.0" encoding="UTF-8" ?>
> 
> <words>
> 	<lang name="en_CA">
> 		<key name="long_name">Canadian English</key>
> 		<key name="bar">FOO.</key>
> 	</lang>
> 	<lang name="jp">
> 		<key name="long_name">日本語</key>
> 		<key name="bar">FOO.</key>
> 	</lang>
> </words>
> -=] test.xml [=-
> 
>    And this is a copy of the DB encoding string and stripped down schema 
> and data:
> 
> -=] PgSQL stuff [=-
> SET client_encoding = 'UTF8';
> SET check_function_bodies = false;
> SET client_min_messages = warning;
> 
> CREATE SEQUENCE usr_seq
>      INCREMENT BY 1
>      NO MAXVALUE
>      MINVALUE 0
>      CACHE 1;
> 
> CREATE TABLE users (
>      usr_id integer DEFAULT nextval('usr_seq'::regclass) NOT NULL,
>      usr_note text,
> );
> 
> COPY users (usr_id, usr_note) FROM stdin;
> 1	Just some text but now with kanji; 私は名前ケッリです。 How is this 
> handled?
> \.
> 
> -=] PgSQL stuff [=-
> 
>    When I run it with (line 21): # binmode STDOUT, ":utf8";
> 
> I get:
> 
> -=] With 'binmode' commented out [=-
> -Note: [Just some text but now with kanji; 私は名前ケッリです。 How is 
> this handled? And still again...]
> English : [Canadian English]
> Wide character in print at ./test.pl line 32.
> Japanese: [日本語]
> =] With 'binmode' commented out [=-
> 
>    And when I run it with (line 21): binmode STDOUT, ":utf8";
> 
> -=] With 'binmode' enabled [=-
> Note: [Just some text but now with kanji; 私は名前ケッリです。 
> How is this handled? And still again...]
> English : [Canadian English]
> Japanese: [日本語]
> -=] With 'binmode' enabled [=-
> 
>    Any tips/help?
> 
>    Thanks!
> 
> A desperate Madi :)

   Drew from TLUG (off the list) helped me figure this one out. I had to 
enable binmode and change my DB connect string to:

my $dbh=DBI->connect("DBI:Pg:dbname=dbname", "user", "secret",
	{
		RaiseError => 1,
		AutoCommit => 1,
		pg_enable_utf8 => 1
	}
) || die "DBI connect error: $DBI::errstr\n";
binmode STDOUT, ":utf8";

   And there ya go. :)

Madi