How to efficiently create a multi-row photo collage from an array of images in Swift

Question

Problem

I am building a collage of photos from an array of images that I am placing onto a tableview. I want to make the images wrap when the number

Answer 1

Since you're using Swift 2.0: UIStackView does exactly what you're trying to do manually, and is significantly easier to use than UICollectionView. Assuming you're using Storyboards, creating a prototype TableViewCell with multiple nested UIStackViews should do exactly what you want. You'll just need to make sure that the UIImages you're inserting are all the same aspect ratio if that's what you want.

Your algorithm is highly inefficient because it has to re-draw, with multiple Core Animation transforms, every image any time you add or remove an image from your array. UIStackView supports dynamically adding and removing objects.

If you still, for some reason, need to snapshot the resulting collage as a UIImage, you can still do this on the UIStackView.

Answer 2

Building on alternative 2 provided by Tommie C above I created a function that

• Always fills the total rectangle, without spaces in the collage
• determines the number of rows and columns automatically (maximum 1 more nrOfColumns than nrOfRows)
• To prevent the spaces mentioned above all individual pics are drawn with "Aspect Fill" (so for some pics this means that parts will be cropped)

Here's the function:

func collageImage(rect: CGRect, images: [UIImage]) -> UIImage {
    if images.count == 1 {
        return images[0]
    }

    UIGraphicsBeginImageContextWithOptions(rect.size, false,  UIScreen.mainScreen().scale)

    let nrofColumns: Int = max(2, Int(sqrt(Double(images.count-1)))+1)
    let nrOfRows: Int = (images.count)/nrofColumns
    let remaindingPics: Int = (images.count) % nrofColumns
    print("columns: \(nrofColumns) rows: \(nrOfRows) first \(remaindingPics) columns will have 1 pic extra")

    let w: CGFloat = rect.width/CGFloat(nrofColumns)
    var hForColumn = [CGFloat]()
    for var c=1;c<=nrofColumns;++c {
        if remaindingPics >= c {
            hForColumn.append(rect.height/CGFloat(nrOfRows+1))
        }
        else {
            hForColumn.append(rect.height/CGFloat(nrOfRows))
        }
    }
    var colNr = 0
    var rowNr = 0
    for var i=1; i<images.count; ++i {
        images[i].drawInRectAspectFill(CGRectMake(CGFloat(colNr)*w,CGFloat(rowNr)*hForColumn[colNr],w,hForColumn[colNr]))
        if i == nrofColumns || ((i % nrofColumns) == 0 && i > nrofColumns) {
            ++rowNr
            colNr = 0
        }
        else {
            ++colNr
        }
    }

    let outputImage = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();
    return outputImage
}

This uses the UIImage extension drawInRectAspectFill:

extension UIImage {
    func drawInRectAspectFill(rect: CGRect, opacity: CGFloat = 1.0) {
        let targetSize = rect.size
        let scaledImage: UIImage
        if targetSize == CGSizeZero {
            scaledImage = self
        } else {
            let scalingFactor = targetSize.width / self.size.width > targetSize.height / self.size.height ? targetSize.width / self.size.width : targetSize.height / self.size.height
            let newSize = CGSize(width: self.size.width * scalingFactor, height: self.size.height * scalingFactor)
            UIGraphicsBeginImageContext(targetSize)
            self.drawInRect(CGRect(origin: CGPoint(x: (targetSize.width - newSize.width) / 2, y: (targetSize.height - newSize.height) / 2), size: newSize), blendMode: CGBlendMode.Normal, alpha: opacity)
            scaledImage = UIGraphicsGetImageFromCurrentImageContext()
            UIGraphicsEndImageContext()
        }
        scaledImage.drawInRect(rect)
    }
}

Answer 3

To build the collage in memory, and to be as efficient as possible, I'd suggest looking into Core Image. You can combine multiple CIFilters to create your output image.

You could apply CIAffineTransform filters to each of your images to line them up (cropping them to size with CICrop if necessary), then combine them using CISourceOverCompositing filters. Core Image doesn't process anything until you ask for the output; and because it's all happening in the GPU, it's fast and efficient.

Here's a bit of code. I tried to keep it as close to your example as possible for the sake of understanding. It's not necessarily how I'd structure the code were I to use core image from scratch.

class func collageImage (rect: CGRect, images: [UIImage]) -> UIImage {

    let maxImagesPerRow = 3
    var maxSide : CGFloat = 0.0

    if images.count >= maxImagesPerRow {
        maxSide = max(rect.width / CGFloat(maxImagesPerRow), rect.height / CGFloat(maxImagesPerRow))
    } else {
        maxSide = max(rect.width / CGFloat(images.count), rect.height / CGFloat(images.count))
    }

    var index = 0
    var currentRow = 1
    var xtransform:CGFloat = 0.0
    var ytransform:CGFloat = 0.0
    var smallRect:CGRect = CGRectZero

    var composite: CIImage? // used to hold the composite of the images

    for img in images {

        let x = ++index % maxImagesPerRow //row should change when modulus is 0

        //row changes when modulus of counter returns zero @ maxImagesPerRow
        if x == 0 {

            //last column of current row
            smallRect = CGRectMake(xtransform, ytransform, maxSide, maxSide)

            //reset for new row
            ++currentRow
            xtransform = 0.0
            ytransform = (maxSide * CGFloat(currentRow - 1))

        } else {

            //not a new row
            smallRect = CGRectMake(xtransform, ytransform, maxSide, maxSide)
            xtransform += CGFloat(maxSide)
        }

        // Note, this section could be done with a single transform and perhaps increase the
        // efficiency a bit, but I wanted it to be explicit.
        //
        // It will also use the CI coordinate system which is bottom up, so you can translate
        // if the order of your collage matters.
        //
        // Also, note that this happens on the GPU, and these translation steps don't happen
        // as they are called... they happen at once when the image is rendered. CIImage can 
        // be thought of as a recipe for the final image.
        //
        // Finally, you an use core image filters for this and perhaps make it more efficient.
        // This version relies on the convenience methods for applying transforms, etc, but 
        // under the hood they use CIFilters
        var ci = CIImage(image: img)!

        ci = ci.imageByApplyingTransform(CGAffineTransformMakeScale(maxSide / img.size.width, maxSide / img.size.height))
        ci = ci.imageByApplyingTransform(CGAffineTransformMakeTranslation(smallRect.origin.x, smallRect.origin.y))!

        if composite == nil {

            composite = ci

        } else {

            composite = ci.imageByCompositingOverImage(composite!)
        }
    }

    let cgIntermediate = CIContext(options: nil).createCGImage(composite!, fromRect: composite!.extent())
    let finalRenderedComposite = UIImage(CGImage: cgIntermediate)!

    return finalRenderedComposite
}

You may find that your CIImage is rotated incorrectly. You can correct it with code like the following:

var transform = CGAffineTransformIdentity

switch ci.imageOrientation {

case UIImageOrientation.Up:
    fallthrough
case UIImageOrientation.UpMirrored:
    println("no translation necessary. I am ignoring the mirrored cases because in my code that is ok.")
case UIImageOrientation.Down:
    fallthrough
case UIImageOrientation.DownMirrored:
    transform = CGAffineTransformTranslate(transform, ci.size.width, ci.size.height)
    transform = CGAffineTransformRotate(transform, CGFloat(M_PI))
case UIImageOrientation.Left:
    fallthrough
case UIImageOrientation.LeftMirrored:
    transform = CGAffineTransformTranslate(transform, ci.size.width, 0)
    transform = CGAffineTransformRotate(transform, CGFloat(M_PI_2))
case UIImageOrientation.Right:
    fallthrough
case UIImageOrientation.RightMirrored:
    transform = CGAffineTransformTranslate(transform, 0, ci.size.height)
    transform = CGAffineTransformRotate(transform, CGFloat(-M_PI_2))
}

ci = ci.imageByApplyingTransform(transform)

Note that this code ignores fixing several mirrored cases. I'll leave that as an exercise up to you, but the gist of it is here.

If you've optimized your Core Image processing, then at this point any slowdown you see is probably due to transforming your CIImage into a UIImage; that's because your image has to make the transition from the GPU to the CPU. If you want to skip this step in order to display the results to the user, you can. Simply render your results to a GLKView directly. You can always transition to a UIImage or CGImage at the point the user wants to save the collage.

// this would happen during setup
let eaglContext = EAGLContext(API: .OpenGLES2)
glView.context = eaglContext

let ciContext = CIContext(EAGLContext: glView.context)

// this would happen whenever you want to put your CIImage on screen
if glView.context != EAGLContext.currentContext() {
    EAGLContext.setCurrentContext(glView.context)
}

let result = ViewController.collageImage(glView.bounds, images: images)

glView.bindDrawable()
ciContext.drawImage(result, inRect: glView.bounds, fromRect: result.extent())
glView.display()

Answer 4

If someone is looking for Objective C code, this repository might be useful.