[PyQt] Logging UTF-8 when pyqtdeployed [CLOSED]

[Barry]

> On 11 Feb 2017, at 00:45, Dave Gradwell <davegradwell at yahoo.co.uk> wrote:
> 
> Hi Barry,
> 
> Thanks for your message and advice.
> Your mention of py2app led me to find that environment variables can also be set in .app/Contents/Info.plist.
> So this is yet another way of dodging the problem - and possibly in a better way than my previous workaround.
> 
> <key>LSEnvironment</key>
>     <dict>
>     <key>LC_CTYPE</key>
>         <string>en_US.UTF-8</string>
>     </dict>
> 
> Thanks, Dave.


Nice to know about that.

Beware that having a fixed locale means you cannot internationalise your app
with translations etc.

Barry

> 
> 
> 
> 
>>> On 10 Feb 2017, at 07:18, Barry Scott <barry at barrys-emacs.org> wrote:
>>> 
>>> 
>>> On 9 Feb 2017, at 00:24, Dave Gradwell <davegradwell at yahoo.co.uk> wrote:
>>> 
>>> Just for the archives…
>>> To workaround this, I changed my Python Logging configuration:
>>> from	:	logging.basicConfig(filename=APP_LOG_PATH, level=logging.DEBU…..
>>> to	:	logging.basicConfig(handlers=[logging.FileHandler(filename=APP_LOG_PATH, encoding='utf-8')], level=logging.DEBU…...
>>> With the explicit encoding parameter, UTF-8 characters are now logged without complaint.
>>> 
>>> During investigation, I dumped os.environ in the different launching contexts and noticed:
>>> - When launched from PyCharm, PYTHONIOENCODING and LC_CTYPE are set to “UTF-8” and “en_US.UTF-8” respectively.
>>> - When invoked by launching .app/Contents/MacOS/<binary> from the Terminal.app, LANG is set to "en_GB.UTF-8”.
>>> - When launched by pyqtdeploy, LANG is set to "en_GB.UTF-8” (perhaps unsurprisingly, as pyqtdeploy was itself launched from Terminal.app and presumably confers its environment onto the binary it makes and launches).
>> 
>> 
>>> - When the app is just double-clicked, however, no environment variables are set relating to any of PYTHONIOENCODING, LC_CTYPE or LANG.   I see that __CF_USER_TEXT_ENCODING is set but I don’t think Python is particularly wowed by this.
>> 
>> True python does not look a __CF_USER_TEXT_ENCODING.
>> 
>>> 
>>> I checked pyqtdeploy_start.cpp.  
>>> locale_codec_name is "UTF-8” (which is good).  
>>> I also hacked-in  Py_SetStandardStreamEncoding("UTF-8", "strict”);   just before   Py_Initialize();   but this had no discernible effect.
>>> 
>>> I wonder if the lack of 'LANGUAGE', 'LC_ALL', 'LC_CTYPE' or ‘LANG’ is causing Python to fall back on a more primitive encoding…
>>> 
>>> locale.getdefaultlocale()  consistently returns None, None.
>> 
>> My double clicked app has LC_CTYPE=en_US.UTF-8, which is not my locale, I’m guessing that it is put there by py2app that I use
>> to create my apps, to get around these issues maybe.
>> 
>> You could add code that looks at __CF_USER_TEXT_ENCODING and do a local.setlocale early on in main.
>> I used to do this to make wxPython apps work well on Mac.
>> 
>> As yet I have not coded the conversion code for PyQt apps. But the shape of the solution is to decode the __CF_USER_TEXT_ENCODING
>> Into a pair of ints. Then use them in a look up table to get the country code to use in setlocale. Always using UTF-8 as the encoding.
>> 
>> Note that if your locale is setup then you would not have a problem with the log file. The docs for open state that the default encoding
>> is locale.getpreferredencoding(False) which should be utf-8 on macOS if the locale is setup.
>> 
>> If you do handle __CF_USER_TEXT_ENCODING be aware that some versions of macOS hex for the values and others decimal.
>> 0x1f or 31 for example.
>> 
>> Of course the simplest fix is to add setlocale( ‘en_US.utf-8’ ) at the start of main() until you care about the locale in detail.
>> 
>>> 
>>> Possibly of interest:
>>> - https://docs.python.org/2/library/locale.html#locale.getdefaultlocale
>>> - http://stackoverflow.com/questions/25661295/why-does-qcoreapplication-call-setlocalelc-all-by-default-on-unix-linux
>>> 
>>> Anyway I’m just sending this into the ether so maybe it can sit in the archives and possibly help someone else one day…
>>> — Dave
>> 
>> Barry
>> 
>> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20170211/938bf121/attachment.html>