When trying to apply a custom installation mask with radio buttons (I used code provided here) I see I am unable to use higher fonts as a little spacing should be needed b
Checkboxes and Radio buttons created on runtime in Inno Setup do not scale their height automatically with a DPI/font size.

So you have to scale them programmatically.
...
RadioButton.Left := WizardForm.TypesCombo.Left;
RadioButton.Height := ScaleY(RadioButton.Height);
RadioButton.Top := WizardForm.TypesCombo.Top + I * RadioButton.Height;
...
The ScaleY(RadioButton.Height) takes the default combobox/radiobutton height, which is designed for the default font and no display scaling (100%) and scales that to the custom font and actual display scaling.

Though note that using a non-default font-size for your application/setup is not a good idea. The user should choose a font size he/she is comfortable with in Windows preferences. You should not override his/her choice.
When changing the font size, do not modify the shared default.isl, use the [LangOptions] section of your project file instead:
[LangOptions]
DialogFontSize=20