<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 9 Feb 2017, at 00:24, Dave Gradwell <<a href="mailto:davegradwell@yahoo.co.uk" class="">davegradwell@yahoo.co.uk</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Just for the archives…</div><div class="">To workaround this, I changed my Python Logging configuration:</div><div class="">from<span class="Apple-tab-span" style="white-space:pre"> </span>:<span class="Apple-tab-span" style="white-space:pre"> </span>logging.basicConfig(filename=APP_LOG_PATH, level=logging.DEBU…..</div><div class="">to<span class="Apple-tab-span" style="white-space:pre"> </span>:<span class="Apple-tab-span" style="white-space:pre"> </span>logging.basicConfig(handlers=[logging.FileHandler(filename=APP_LOG_PATH, encoding='utf-8')], level=logging.DEBU…...</div><div class="">With the explicit encoding parameter, UTF-8 characters are now logged without complaint.</div><div class=""><br class=""></div><div class="">During investigation, I dumped os.environ in the different launching contexts and noticed:</div><div class="">- When launched from PyCharm, PYTHONIOENCODING and LC_CTYPE are set to “UTF-8” and “en_US.UTF-8” respectively.</div><div class="">- When invoked by launching .app/Contents/MacOS/<binary> from the Terminal.app, LANG is set to "en_GB.UTF-8”.</div><div class="">- When launched by pyqtdeploy, LANG is set to "en_GB.UTF-8” (perhaps unsurprisingly, as pyqtdeploy was itself launched from Terminal.app and presumably confers its environment onto the binary it makes and launches).</div></div></div></blockquote><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">- When the app is just double-clicked, however, no environment variables are set relating to any of PYTHONIOENCODING, LC_CTYPE or LANG. I see that __CF_USER_TEXT_ENCODING is set but I don’t think Python is particularly wowed by this.</div></div></div></blockquote><div><br class=""></div><div>True python does not look a __CF_USER_TEXT_ENCODING.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">I checked pyqtdeploy_start.cpp. </div><div class="">locale_codec_name is "UTF-8” (which is good). </div><div class="">I also hacked-in Py_SetStandardStreamEncoding("UTF-8", "strict”); just before <span style="color: rgb(192, 192, 192);" class=""> </span>Py_Initialize(); but this had no discernible effect.</div><div class=""><br class=""></div><div class="">I wonder if the lack of 'LANGUAGE', 'LC_ALL', 'LC_CTYPE' or ‘LANG’ is causing Python to fall back on a more primitive encoding…</div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">locale.getdefaultlocale() consistently returns None, None.</div></div></div></blockquote><div><br class=""></div><div>My double clicked app has LC_CTYPE=en_US.UTF-8, which is not my locale, I’m guessing that it is put there by py2app that I use</div><div>to create my apps, to get around these issues maybe.</div><div><br class=""></div><div>You could add code that looks at __CF_USER_TEXT_ENCODING and do a local.setlocale early on in main.</div><div>I used to do this to make wxPython apps work well on Mac.</div><div><br class=""></div><div>As yet I have not coded the conversion code for PyQt apps. But the shape of the solution is to decode the __CF_USER_TEXT_ENCODING</div><div>Into a pair of ints. Then use them in a look up table to get the country code to use in setlocale. Always using UTF-8 as the encoding.</div><div><br class=""></div><div>Note that if your locale is setup then you would not have a problem with the log file. The docs for open state that the default encoding</div><div>is locale.getpreferredencoding(False) which should be utf-8 on macOS if the locale is setup.</div><div><br class=""></div><div>If you do handle __CF_USER_TEXT_ENCODING be aware that some versions of macOS hex for the values and others decimal.</div><div>0x1f or 31 for example.</div><div><br class=""></div><div>Of course the simplest fix is to add setlocale( ‘en_US.utf-8’ ) at the start of main() until you care about the locale in detail.</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">Possibly of interest:</div><div class="">- <a href="https://docs.python.org/2/library/locale.html#locale.getdefaultlocale" class="">https://docs.python.org/2/library/locale.html#locale.getdefaultlocale</a></div><div class="">- <a href="http://stackoverflow.com/questions/25661295/why-does-qcoreapplication-call-setlocalelc-all-by-default-on-unix-linux" class="">http://stackoverflow.com/questions/25661295/why-does-qcoreapplication-call-setlocalelc-all-by-default-on-unix-linux</a></div><div class=""><br class=""></div><div class="">Anyway I’m just sending this into the ether so maybe it can sit in the archives and possibly help someone else one day…</div><div class="">— Dave</div></div></div></blockquote><br class=""></div><div>Barry</div><div><br class=""></div><br class=""></body></html>