<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=us-ascii" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.18783"></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=907282416-16062009><FONT color=#0000ff
size=2 face=Arial>Phil,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=907282416-16062009><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=907282416-16062009> <FONT
color=#0000ff size=2 face=Arial>This is a shot in the dark, but is the encoding
on the page that interacts with the web service via AJAX by chance ISO-8859-1
(either set explicitly or by default on the server)? You didn't
mention whether "utf8" or "UTF-8" is the character encoding you are now
specifying when calling Encode::decode on the JSON text stream, so I'm not sure
how that might have affected the resulting text stream as it enters the wire
between the two processes.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=907282416-16062009><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=907282416-16062009> <FONT
color=#0000ff size=2 face=Arial>The world of character encoding is fraught with
all kinds of interesting situations. Once upon a time a "byte" and
"character" were interchangeable, and all was well with the world ... 8^))
... In the world of processes operating under different character encoding
schemes, however, sometimes the waters get muddled. If the sender and
receiver are not in absolute agreement as to the character encoding of a stream
of bytes (_and_ capable of translating between character encodings when
necessary), the interpretation of what characters are meant by a particular
byte stream can be different, which leads to the type of problem you
describe.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=907282416-16062009><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=907282416-16062009><FONT color=#0000ff
size=2 face=Arial> --Mark</FONT></SPAN></DIV>
<DIV><FONT color=#0000ff size=2 face=Arial></FONT> </DIV><BR>
<DIV dir=ltr lang=en-us class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>From:</B>
spug-list-bounces+mark.johnston=pnl.gov@pm.org
[mailto:spug-list-bounces+mark.johnston=pnl.gov@pm.org] <B>On Behalf Of </B>Phil
Mitchell<BR><B>Sent:</B> Wednesday, June 10, 2009 12:41 PM<BR><B>To:</B>
spug-list@mail.pm.org<BR><B>Subject:</B> SPUG: ajax character encoding issue
solved, but WHY?<BR></FONT><BR></DIV>
<DIV></DIV>I solved the character encoding issue that I wrote about earlier, but
I don't understand why the fix works. I'm hoping someone can explain this to
me!<BR><BR>The issue was that non-ascii chars were appearing as junk BUT only
when retrieved via ajax calls. Otherwise, they displayed fine. The junk display
was due to them being interpreted as ISO-8859-1, but I could not figure out why
the browser was interpreting that way. All my data is handled as
UTF-8.<BR><BR>The problem was fixed by calling Encode::decode on the data prior
to sending back via ajax. BUT WHY?<BR><BR>I am using the JSON view to render
ajax responses, and it sets the charset header correctly to UTF-8. Of course,
even when you decode, perl still represents as "internal" utf8. But why should
this be necessary?<BR clear=all><BR>-- <BR>==========================<BR><A
href="http://www.bikewise.org">http://www.bikewise.org</A><BR><BR>2People
citizen's network for climate action: <A
href="http://www.2people.org">http://www.2people.org</A><BR><BR>Greater Seattle
Climate Dialogues: <A
href="http://www.climatedialogues.org">http://www.climatedialogues.org</A><BR>==========================<BR></BODY></HTML>