I would like to use a system fact for a host times a number/percentage as a base for a variable. What I am trying to do specifically is use the ansible_memtotal_mb
You're really close! I use calculations to set some default java memory sizes, which is similar to what you are doing. Here's an example:
{{ (ansible_memtotal_mb*0.8-700)|int|abs }}
That shows a couple of things- first, it's using jinja math, so do the calculations inside the {{ jinja }}. Second, int and abs do what you'd expect- ensure the result is an unsigned integer.
In your case, the correct code would be:
vars:
ramsize: "{{ ansible_memtotal_mb * 0.8 }}"