I'm guessing you're on a UNIX-like system, and your environment set LANG [or LC_ALL or whatever] to C.
Try editing your default shell's startup file to set LANG to something like en_US.utf-8 [or whatever locale makes sense for you]? For example, in bash, edit ~/.bash_profile [or ~/.profile if you're using that instead for sh compatibility] and add:
export LANG="en_US.utf-8"
For [t]csh, edit ~/.cshrc [or ~/.tcshrc if that's what you're using] to add:
setenv LANG "en_US.utf-8"
Making
the changes "live" doesn't work, because your shell is likely hosted in a terminal that has configured itself solely for ASCII display, based on the LANG=C in effect when it was launched [and many terminals do session coalescence, so even if you changed LANG and then launched a new terminal, it would coalesce with the shared terminal process with the out-of-date LANG]. So after you change ~/.bash_profile, log out and then log back in so your root shell will set LANG correctly for every
other process [since they all ultimately fork from the root shell].
Describe the bug
Running a program with a rich progressbar inside a docker container, I got the following exception:
File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 150, in track
sequence, total=total, description=description, update_period=update_period
File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 667, in track
with self:
File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 627, in __enter__
self.start[]
File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 596, in start
self.refresh[]
File "/usr/local/lib/python3.6/dist-packages/rich/progress.py", line 812, in refresh
self.console.print[Control[""]]
File "/usr/local/lib/python3.6/dist-packages/rich/console.py", line 448, in __exit__
self._exit_buffer[]
File "/usr/local/lib/python3.6/dist-packages/rich/console.py", line 426, in _exit_buffer
self._check_buffer[]
File "/usr/local/lib/python3.6/dist-packages/rich/console.py", line 993, in _check_buffer
self.file.write[text]
UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-55: ordinal not in range[128]
The problem can be easily solved in my case by setting the encoding properly:
export PYTHONIOENCODING=utf8
After that, the script runs perfectly fine with the beautiful rich progressbar.
However, setting this env var in some environments might not be possible or have other undesired consequences. It would be great if there was some kind of failback mode, or a WARNING with a suggestion to set this var, instead of failing without further information.
To Reproduce
Use the README progressbar example, let's call this file progress.py:
from rich.progress import track for step in track[range[100]]: pass
Run it with ascii encoding vs. utf8 and check the difference:
PYTHONIOENCODING=ascii python progress.py # Exception thrown PYTHONIOENCODING=utf8 python progress.py # Works
Platform
rich==5.0.0- Probably works on any python3 installation. Reproduced on Python 3.6.10 on OSX Catalina, and Python 3.6.9 on Ubuntu Linux native and the same version on Linux for tegra [ARM] docker image.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| from: | |
| http://chase-seibert.github.io/blog/2014/01/12/python-unicode-console-output.html | |
| Print to the console in Python without UnicodeEncodeErrors 12 Jan 2014 | |
| I can't believe I just found out about this! If you use Python with unicode data, such as Django database records, you may have seen cases where you print a value to the console, and if you hit a record with an extended [non-ascii] character, your program crashes with the following: | |
| Traceback [most recent call last]: | |
| File "foobar.py", line 792, in | |
| print value | |
| UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 20: ordinal not in range[128] | |
| This often comes up when you have a Django model instance with a user-entered field in the __unicode__ return value. In the past, I have solved this by changing the print statement to print something that cannot be unicode, such as a database ID. Alternatively, you could do something like value.encode['ascii', 'ignore']. But just this week, I realized that there is a much better solution. | |
| Your termianl can typically handle UTF8 characters just fine. The issue is actually that Python is just getting confused about what encoding the terminal accepts. However, you can explicitly set this value. | |
| The PYTHONIOENCODING environment variable controls what encoding stdout/stderr uses. If you do an export PYTHONIOENCODING=UTF-8, it will solve the problem. You can also prefix any given single python command-line invocation with this value, such as PYTHONIOENCODING=UTF-8 ./manage.py runserver. | |
| Alternatively, you can set this value in your actual code, say in your settings.py file: | |
| import sys | |
| import codecs | |
| sys.stdout = codecs.getwriter['utf8'][sys.stdout] | |
| sys.stderr = codecs.getwriter['utf8'][sys.stderr] | |
| I'm currently working at NerdWallet, a startup in San Francisco trying to bring clarity to all of life's financial decisions. We're hiring like crazy. Hit me up on Twitter, I would love to talk. | |
| Follow @chase_seibert on Twitter |