0
193
ViewsDjango dumpdata fails on special characters
I'm trying to dump my entire DB to a json. When I run python manage.py dumpdata > data.json I get an error:
(env) PS C:\dev\watch_something> python manage.py dumpdata > data.json
CommandError: Unable to serialize database: 'charmap' codec can't encode character '\u0130' in position 1: character maps to <undefined>
Exception ignored in: <generator object cursor_iter at 0x0460C140>
Traceback (most recent call last):
File "C:\dev\watch_something\env\lib\site-packages\django\db\models\sql\compiler.py", line 1602, in cursor_iter
cursor.close()
sqlite3.ProgrammingError: Cannot operate on a closed database.
It's because one of the characters in my DB is a sepcial character. How can I dump the DB correctly?
FYI, all other DB functionalities work fine
3 answers
Answer question0
One solution is to use ./manage.py dumpdata -o data.json instead of ./manage.py dumpdata > data.json.
Another solution is to use Python's UTF-8 mode, run:
python -Xutf8 ./manage.py dumpdata > data.json
0
To save json data in django the TextIOWrapper is used:
The default encoding is now
locale.getpreferredencoding(False)(...)
In documentation of locale.getpreferredencoding fuction we can read:
Return the encoding used for text data, according to user preferences. User preferences are expressed differently on different systems, and might not be available programmatically on some systems, so this function only returns a guess.
Here I found "hacky" but working method to overwrite these settings:
In file settings.py of your django project add these lines:
import _locale
_locale._getdefaultlocale = (lambda *args: ['en_US', 'utf8'])
0
Here is the solution from djangoproject.com
You go to Settings there's a "Use Unicode UTF-8 for worldwide language support", box in "Language" - "Administrative Language Settings" - "Change system locale" - "Region Settings".
If we apply that, and reboot, then we get a sensible, modern, default encoding from Python.
djangoproject.com
the settings box looks like this. Enable the check and restart your system
