I assumably managed to work it by setting some .leftpanel width style as !important. So the left panel width is very fixed.
But as a side effect it might have affected Term Store layout: Term Store content doesn't occupy left panel space anymore as it does OOB.