Querying for fontsize in User Settings window yields only settings for:
This has been a feature request for some time. The short answer is right now there's no VSCode setting that allows you to customize the sidebar font size. There is a hacky workaround solution proposed by @lindesvard in the same issue thread that involves using the Custom CSS and JS Loader extension to load in a custom CSS file that modifies the sidebar styling if you're willing to experiment with it.
To get this working follow the Getting Started section in the extension details, then add URLs to your custom files to the "vscode_custom_css.imports": [""] array. (Note that on Windows mine looks like the following: "vscode_custom_css.imports": ["file:///C:\\dev\\vscode-styling.css"])
Below are before and after adding the CSS mentioned by @PSVapour in a later comment:
Note that this also affects the command palette font.