The algorithm is well known, you do 8 left shifts and check the units, tens or hundreds bits (4 each) after each shift. If they are above 4 you add 3 to the group and so on.
At least two issues appear:
Adding is done after shift, and not before as described in the Double dabble algorithm
The bcd shift goes bcd(11 downto 1), but should be bcd(10 downto 0)
So try with the code:
process ( hex_in )
variable hex_src : std_logic_vector (7 downto 0) ;
variable bcd : std_logic_vector (11 downto 0) ;
begin
hex_src := hex_in ;
bcd := (others => '0') ;
for i in 0 to 7 loop
if bcd(3 downto 0) > "0100" then
bcd(3 downto 0) := bcd(3 downto 0) + "0011" ;
end if ;
if bcd(7 downto 4) > "0100" then
bcd(7 downto 4) := bcd(7 downto 4) + "0011" ;
end if ;
if bcd(11 downto 8) > "0100" then
bcd(11 downto 8) := bcd(11 downto 8) + "0011" ;
end if ;
bcd := bcd(10 downto 0) & hex_src(7) ; -- shift bcd + 1 new entry
hex_src := hex_src(6 downto 0) & '0' ; -- shift src + pad with 0
end loop ;
bcd_hun <= bcd(11 downto 8) ;
bcd_ten <= bcd(7 downto 4) ;
bcd_uni <= bcd(3 downto 0) ;
end process ;
However, the implementation may require a slow clock...
Based on Davids observations in the comments, the code be optimized to:
process ( hex_in )
variable hex_src : std_logic_vector (4 downto 0) ;
variable bcd : std_logic_vector (11 downto 0) ;
begin
bcd := (others => '0') ;
bcd(2 downto 0) := hex_in(7 downto 5) ;
hex_src := hex_in(4 downto 0) ;
for i in hex_src'range loop
if bcd(3 downto 0) > "0100" then
bcd(3 downto 0) := bcd(3 downto 0) + "0011" ;
end if ;
if bcd(7 downto 4) > "0100" then
bcd(7 downto 4) := bcd(7 downto 4) + "0011" ;
end if ;
-- No roll over for hundred digit, since in 0 .. 2
bcd := bcd(10 downto 0) & hex_src(hex_src'left) ; -- shift bcd + 1 new entry
hex_src := hex_src(hex_src'left - 1 downto hex_src'right) & '0' ; -- shift src + pad with 0
end loop ;
bcd_hun <= bcd(11 downto 8) ;
bcd_ten <= bcd(7 downto 4) ;
bcd_uni <= bcd(3 downto 0) ;
end process ;