Improving performance for converting numbers to lists, and base10 to base2

Question

Many Project Euler problems require manipulating integers and their digits, both in base10 and base2. While I have no problem with converting integers in lists of digits, or

Answer 1

Making your existing code more efficient

Have you considered getting the list of bits using any of Racket's bitwise operations? E.g.,

(define (bits n)
  (let loop ((n n) (acc '()))
    (if (= 0 n) 
        acc
        (loop (arithmetic-shift n -1) (cons (bitwise-and n 1) acc)))))

> (map bits '(1 3 4 5 7 9 10))
'((1) (1 1) (1 0 0) (1 0 1) (1 1 1) (1 0 0 1) (1 0 1 0))

It'd be interesting to see whether that speeds anything up. I expect it would help a bit, since your 10->bin procedure currently makes a call to expt, quotient, and remainder, whereas bit shifting, depending on the representations used by the compiler, would probably be more efficient.

Your integer->lon is also using a lot more memory than it needs to, since the append is copying most of the result at each step. This is kind of interesting, because you were already using the more memory efficient approach in bin->10. Something like this is more efficient:

(define (digits n)
  (let loop ((n n) (acc '()))
    (if (zero? n)
        acc
        (loop (quotient n 10) (cons (remainder n 10) acc)))))

> (map digits '(1238 2391 3729))
'((1 2 3 8) (2 3 9 1) (3 7 2 9))

More efficient approaches

All that said, perhaps you should consider the approach that you're using. It appears that right now, you're iterating through the numbers 1…MAX, checking whether each one is a palindrome, and if it is, adding it to the sum. That means you're doing something with MAX numbers, all in all. Rather than checking for palindromic numbers, why not just generate them directly in one base and then check whether they're a palindrome in the other. I.e., instead of of checking 1…MAX, check:

• 1
• 11
• 101, and 111
• 1001, and 1111
• 10001, 10101, 11011, and 11111,
• and so on, up until the numbers are too big.

This list is all the binary palindromes, and only some of those will be decimal palindromes. If you can generate the binary palindromes using bit-twiddling techniques (so you're actually working with the integers), it's easy to write those to a string, and checking whether a string is a palindrome is probably much faster than checking whether a list is a palindrome.

Answer 2

Are you running these timings in DrRacket by any chance? The IDE slows down things quite a bit, especially if you happen to have debugging and/or profiling turned on, so I'd recommend doing these tests from the command line.

Also, you can usually improve the brute-force approach. For example, you can say here that we only have to consider odd numbers, because even numbers are never a palindrome when expressed as binaries (a trailing 0, but the way you represent them there's never a heading 0). This divides the execution time by 2 regardless of the algorithm.

Your code runs on my laptop in 2.4 seconds. I wrote an alternative version using strings and build-in functions that runs in 0.53 seconds (including Racket startup; execution time in Racket is 0.23 seconds):

#!/usr/bin/racket
#lang racket

(define (is-palindrome? lon)
  (let ((lst (string->list lon)))
    (equal? lst (reverse lst))))

(define (sum-them max)
  (for/sum ((i (in-range 1 max 2))
             #:when (and (is-palindrome? (number->string i))
                         (is-palindrome? (number->string i 2))))
    i))

(time (sum-them 1000000))

yields

pu@pumbair: ~/Projects/L-Racket  time ./speed3.rkt
cpu time: 233 real time: 233 gc time: 32
872187

real    0m0.533s
user    0m0.472s
sys     0m0.060s

and I'm pretty sure that people with more experience in Racket profiling will come up with faster solutions.

So I could give you the following tips:

• Think about how you may improve the brute force approach
• Get to know your language better. Some constructs are faster than others for no apparent reason
  • see http://docs.racket-lang.org/guide/performance.html and http://jeapostrophe.github.io/2013-08-19-reverse-post.html
• use parallelism when applicable
• Get used to the Racket profiler

N.B. Your 10->bin function returns #f for the value 0, I guess it should return '(0).