Amazon recently rolled out a new feature on CloudFront that supports custom SSL certificates at no charge using SNI (Server Name Indication).
I got my distribution
I had a similar issue to this and, as @Michael-sqlbot suggested, switched from custom origin to S3. That did not, by itself, resolve the issue.
In addition to switching the origin, Andrew from AWS support said that aliases work better than CNAMEs. I had been using CNAMEs. When I switched to aliases (one for IPv4 and one for IPv6) it worked. Here is the Route 53 documentation for CloudFront that shows how to setup aliases for CloudFront.